Materialize singleton classes

[soutaro]

Summary

This PR materializes singleton class declarations during resolution.

• Materialize the singleton class of every class and module after the convergence loop and remaining definitions have settled.
• Materialize deeper singleton classes only for descendants of explicit/source-backed nested singleton classes, so nested class << self hierarchies stay complete for subclasses without trying to materialize every possible singleton class.
• Keep singleton classes out of declaration search/workspace-symbol results because they are internal declarations, while graph-level declaration enumeration and stats now include them.
• Update Rust, MCP/CLI, and Ruby expectations for the additional singleton class declarations.

Why

Rubydex already creates singleton classes on demand for self methods, extend, class << self, and class-level instance variables. That lazy behavior is correct for Ruby, but it means singleton classes exist in the graph only when source happens to force them into existence.

This lazy graph shape leaves gaps in queries that need to walk through singleton-class ancestors and descendants:

• When a module is extended into a class/module object, consumers may need the descendants of that extended module through singleton-class ancestors.
• When starting from a leaf singleton class, consumers may want to call find_member and rely on the singleton ancestor chain being present.
• When subclasses inherit from a class that has explicit nested singleton classes, the corresponding singleton classes for those subclasses need to exist for ancestor/descendant queries to be complete.

For example:

class Foo
  class << self        # Foo::<Foo>, the singleton class of Foo
    class << self      # Foo::<Foo>::<<Foo>>, the singleton class of Foo::<Foo>
    end
  end
end

class Bar < Foo
end

The first stage makes sure both Foo::<Foo> and Bar::<Bar> exist, so Foo::<Foo> can be recorded as an ancestor of Bar::<Bar>. Because source explicitly opens the singleton class of Foo::<Foo>, the second stage also creates Bar::<Bar>::<<Bar>>, the corresponding singleton class of Bar::<Bar>.

This PR makes singleton class materialization eager after the main resolution work has settled. The additional declarations are internal and are filtered out of declaration search, so they do not become user-facing workspace symbols. In practice, the runtime and memory impact is negligible, so eager materialization removes these graph completeness gaps without a meaningful cost.

This PR is stacked on #892: singleton classes are materialized before descendants are recomputed, so the descendant relation is derived from the completed ancestor graph.

Notes

The materialization pass is intentionally bounded.

The first stage creates the singleton class of every class and module. For class Bar < Foo, this creates Foo::<Foo> and Bar::<Bar> even if neither class had a singleton class in source.

The second stage creates deeper singleton classes only when source has already created a nested singleton class at that depth. In the example above, Foo::<Foo>::<<Foo>> is source-backed because it came from the nested class << self, so Foo::<Foo> seeds creation of the matching singleton class for descendants like Bar::<Bar>.

Pure singleton classes created only while linearizing ancestors are not used as seeds. For example, linearizing Foo::<Foo>::<<Foo>> may require Object::<Object>::<<Object>>, but that artifact should not cause Rubydex to create every possible singleton class under Object.

Declaration search filters out singleton classes so internal names like Foo::<Foo> do not surface as user-facing search results.

Validation

• cargo test -p rubydex
• Ruby test expectations were updated for materialized singleton classes

Performance and memory comparison against main

Target: large Ruby codebase

Commits:

• main: f50e21de
• This PR: 848d5e5b

Commands:

utils/mem-use rust/target/release/rubydex_cli [path] --stats
cargo bench --bench graph_memory -- [path]

Metric	main	This PR	Delta
Execution time	25.47s	22.79s	-2.68s (-10.5%)
--stats total time	25.033s	22.725s	-2.308s (-9.2%)
Resolution time	18.214s	15.975s	-2.239s (-12.3%)
Peak memory footprint	5236.47 MB	4994.45 MB	-242.02 MB (-4.6%)
Total graph memory	4225.43 MB	4251.15 MB	+25.72 MB (+0.6%)
Declarations	1,102,491	1,120,349	+17,858 (+1.6%)

The process-level peak memory number is from /usr/bin/time -l and can be noisy. The graph-memory benchmark is more directly tied to Rubydex's graph allocation; it shows only a +25.72 MB (+0.6%) increase while materializing 17,858 additional singleton class declarations.

Because this PR is stacked, comparing the PR head against its immediate base branch can show different numbers than comparing the listed main commit against the PR head.

[amomchilov]

This lazy behaviour seems desirable. What's the purpose of eagerly materializing all of them?

[soutaro]

To make find_member() work for all leaf singleton classes and keep the descendants include them.
The execution time is slightly improved compared to main. The graph memory footprint is <1% increased (because we have them), but I think it's acceptable.

[soutaro]

Another option is to make ancestors/descendants calculation entirely lazy. The graph will only keep direct ancestors/descendants edges, and we provide a utility that traverses them when the transitive ancestors or descendants are needed. (But we need to handle un-materialized singleton classes even in that case.)

[st0012]

A few feedback while I'm still understanding the algorithm:

• I'd appreciate if the rank-N concept can be explained with examples. Like:

class Foo
  class << self # Foo::<Foo> rank-1 singleton
    class << self # Foo::<Foo>::<<Foo>> rank-2 singleton 
    end
  end
end

• The Notes section can also be better explained with examples.
• I ran this PR and its based main commit with --stats a few times, and my results show that the PR runs slower, not faster. I got the number with 3 runs on each commit and average them.

Metric	Base avg	PR #893 avg	Delta
SingletonClass declarations	147,362	165,199	+17,837 (+12.10%)
Total time	21.365s	23.226s	+1.860s slower (+8.71%)
Resolution time	15.146s	16.563s	+1.417s slower (+9.35%)

But I can confirm memory usage (both RSS and graph memory) is roughly the same.

[st0012]

I don't think the ranking terminology is well-established in the repo yet. So this is a bit confusing to new readers.

Also, it's not entirely accurate to call Foo::<Foo> a rank-1 singleton class IMO as it implies a class can have multiple singleton classes, which is not true. Foo only has 1 singleton class. Foo::<Foo>::<<Foo>> should be called Foo::<Foo>'s singleton class. Not Foo's rank-2 singleton.

I'd change the comment to something like:

Suggested change

	// A rank-1 singleton is materialized for every class...
	// Every declared class has its singleton class automatically populated

[st0012]

Can we explain this in more plain English? Like:

Materialize singleton classes for every class/module declaration.
This runs after the main singleton class creation paths (the main resolution loop and handle_remaining_definitions) so
it only creates singleton classes to fill the existing singleton classes' ancestors/descendants chain

[soutaro]

@st0012 I think you compared the base commit of this PR and the HEAD. If we compare main branch and the HEAD, it's a bit faster. I run the benchmark a few times, and the result was consistent -- HEAD is ~5% faster than main.