feat(labels): recover C++ symbols and normalize language metadata

[r0ny123]

Summary

This completes the next label-recovery tier for C++ binaries and makes serialized language metadata consistent: every defined ELF dynamic export is recovered (including data objects such as vtables and RTTI), Itanium and MSVC C++ names are demangled in process, language identification is driven by validated symbol evidence instead of prefix guesses, and metadata.language becomes a score-only map. Released as 4.4.0 with Python 3.11+ support.

It builds on #209 (tier-1). Until that merges, this PR's commit range includes those four commits; the new work is 2fcf8bb..cf95489.

Motivation

ELF exports were treated as function-only metadata. On a representative ELF sample that exposed 3 function exports even though the dynamic symbol table holds 26 defined exports: 3 functions and 23 objects, the objects being useful C++ metadata (vtables, typeinfo, typeinfo names).

Three related problems sat on the same path:

1. host-dependent C++ demangling (c++filt / llvm-cxxfilt subprocesses) did not consistently cover ELF relocation imports and Mach-O symbols, so results varied per machine;
2. broad _ZN-prefix checks classified ordinary Itanium C++ symbols as Rust;
3. metadata.language mixed public scores with internal counters and a private guess, and used backend-specific string values.

Changed behavior

ELF export and import recovery

• xmetadata.exported_symbols is new: every defined ELF dynamic export, including data symbols, keyed by virtual address. Thread-local symbols are excluded because their value is an offset inside the TLS block rather than an address, and SHN_COMMON / SHN_ABS entries are excluded because they are not laid out in the image. Aliases sharing one address (for example the C1/C2 constructor pair) keep the first name the dynamic symbol table spells, so repeated runs agree.
• xmetadata.exported_functions and the legacy xmetadata.symbols map stay function-only, so data objects never become function-entry candidates.
• xmetadata.imported_functions is demangled. API references (SmdaFunction.apirefs) deliberately keep the undecorated import name, so they stay comparable with PE and Mach-O imports and with previously produced reports.
• Undefined dynamic symbols that LIEF reports with a PLT address are no longer recovered as local functions. The same predicate is applied to Rust ELF symbol recovery, which had the identical exposure.
• A malformed symbol name skips only that entry instead of aborting or truncating collection. This covers ELF exports, dynamic and static symbols, relocation imports, Mach-O exports and symbols, Rust ELF/Mach-O scans, and Visual Basic import inspection.

On the representative ELF sample: 26 / 26 defined exports recovered, split 3 functions and 23 objects; 241 relocation-backed import slots recovered; no Itanium-mangled labels left in the report label maps.

C++ and Rust evidence

• pycxxfilt's vendored LLVM demangler replaces the host-tool subprocesses, so demangling is deterministic across machines and available on Windows.
• An Itanium name must demangle successfully before it counts as C++ evidence. MSVC decorated names count too, both functions and decorated data such as class statics.
• Thiscall-shaped instruction sequences are no longer scored as C++. They are an ABI-level pattern, not language identity, and the old score divided by the function count at a point in analysis where no function existed yet, so a single matching byte pattern inflated it to 1.0.
• The symbol scan stops as soon as the score saturates, so a large symbol table is not walked in full for a result that cannot change.
• Fat Mach-O binaries select the active architecture/bitness slice before C++ symbols are scored.
• Rust evidence is centralized and requires a parseable v0 or hashed-legacy symbol, so ordinary Itanium C++ names no longer read as Rust.

Language report contract

• metadata.language contains only language -> float score entries. Internal counters and the private guess are gone.
• To pick one language from the map: highest score, except that go and rust win outright above 0.5, since a build ID, pclntab header, or demangled Rust symbol is conclusive while the other scores are graded evidence. The README documents this.
• CIL reports use {".net": 1.0} and Dalvik reports use {"dalvik": 1.0} instead of backend-specific strings.
• SmdaReport.fromDict normalizes legacy string values and removes private legacy keys, so older reports load into the new contract.
• Go, Delphi, Visual Basic, C++, and Rust scores require stronger format/symbol evidence before activating language-specific recovery. Go pclntab headers are validated structurally (version magic, padding, PC quantum, pointer size) and accepted for big-endian targets as well.

Deliberately unchanged

The Delphi VMT scan stays gated on the TObject marker rather than on a valid PE header, and the Visual Basic runtime lookup is likewise ungated. Memory dumps that start below the MZ header are a primary input for SMDA, and a header gate would silently disable recovery for exactly those samples. The scan result is memoized and shared with Delphi candidate recovery, so a real Delphi image pays for it once either way.

Compatibility

This is a breaking report and runtime change:

• Python 3.11+ is required.
• pycxxfilt>=0.1.0 is a new runtime dependency and replaces host-tool demangling.
• Consumers must treat metadata.language as a score map and must not read _guess, _count_thiscalls, _count_delphi_objects, or delphi_kb_file.
• ELF C++ import labels are now demangled. API reference names are unchanged.
• xmetadata.exported_symbols is additive; the existing function-oriented maps keep their meaning.

Validation

• make lint — clean
• make test — 701 passed, 1 skipped
• python -m pip check — clean
• source distribution and wheel build — successful
• representative ELF analysis — 26 exports, 241 import slots, zero remaining mangled export/import labels

The large AArch64 Mach-O corpus test previously used a 20-second per-sample wall-clock guard that tripped twice on contended runners. Profiling showed the C++ symbol scorer costs roughly 1 ms per pass on the largest corpus sample, so it was not the cause and a sampled scorer was rejected as inaccurate. The guard is now a 45-second default, raisable through SMDA_TEST_CORPUS_TIMEOUT on a slow runner. Production timeout behavior is unchanged.

[danielplohmann]

Superseded by #211, which rebases the #210 work (its four commits cherry-picked onto master after #209 landed) so it merges cleanly without the duplicate fork commits.