Shape the generator infrastructure for extraction - #188
Conversation
Roughly half of Semantics.SourceGenerators has nothing to do with physical quantities: the metadata-driven IIncrementalGenerator base, metadata loading, deterministic output handling, and the diagnostics plumbing would work unchanged for any metadata-driven generator, but were only usable from this repository. Those parts now sit in CodeGen/ with the physics-specific code depending on them and nothing depending back, so moving them into a shared package is a mechanical lift once that package has a home. The base supports N metadata files. Two generators - not one - had worked around its single-file limit with their own copy of the same scaffolding: an Initialize override, a private JSON loader, a combining type, and a shim to satisfy an abstract contract they never used. Both are gone; QuantitiesGenerator and UnitsGenerator now just declare the files they read. The file header is a parameter rather than a hard-coded literal, with this repository's copyright bound once in SemanticsGeneratorSettings. Diagnostics are allocated from one catalogue, which fixes the prefix and category. That closes three gaps: - The base reported parse failures as CONV001 in category "SourceGenerator", a leftover from when it served only ConversionsGenerator, while everything derived from it used SEM00x. It is now SEM007. - The second-metadata-file loader caught JsonException and returned null, so a malformed units.json produced no diagnostic at all and the generator silently emitted identity conversions. Parse failures are now always reported. - A declared metadata file that is missing produced no output and no explanation. That is now SEM006. Metadata discovery matches the whole last path segment rather than using EndsWith, which also matched any file whose name merely ended with the wanted one. SEM004 is reported at the position in dimensions.json where the offending unit name is written, so a warning about a name in a large file says where the name is; SEM001 and SEM003 still report without a location. Emit.cs is split: the C# vocabulary moves to CodeGen.CSharpKeywords, and what stays is the part that only means something here - the parameter names this generator uses and a suppression that is about physics. The CSharpGeneratorDriver harness moves out of SourceGeneratorTests into a standalone GeneratorHarness that supplies every metadata file the way MSBuild's AdditionalFiles item group does, and can drive a generator twice to check it reuses its output. New tests cover what was untested: each SEM00x firing on the input it is meant to catch, every descriptor being tracked in the analyzer release files, and every generator behaving incrementally. Generated output under Semantics.Quantities/Generated/ is byte-identical. Test suite: 1019 passed, 92 failed - the same 92 Windows-path tests that already failed on Linux before this change, and 13 more tests passing. Refs #182, #183, #185, #186 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sagQjzFv3cnFNM3G271TJ
IDE0042 on a tuple pattern in the incrementality check, and two using directives left behind by code that moved into GeneratorHarness. Both are enforced as build errors by EnforceCodeStyleInBuild, which the local build had to turn off: the container's SDK ships Roslyn 5.0 and this repository pins Microsoft.CodeAnalysis 5.9, so the IDE analyzers disagree with CI about an unrelated pre-existing file. Verified this time by patching that one file locally and building with style enforcement on, which now reports nothing. Refs #182, #186 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sagQjzFv3cnFNM3G271TJ
Three loops flagged by github-code-quality for filtering their sequence implicitly. The metadata dictionary is now built by grouping, which states "first duplicate wins" rather than leaving it to a ContainsKey guard - and avoids the shape the bot suggested, where a Where predicate would read a dictionary the loop body is still mutating. The missing-file loop filters with Where. The incrementality check collapses to a single Any over the flattened output steps, replacing two nested loops and an early return. No behaviour change; generated output still byte-identical. Refs #182, #186 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015sagQjzFv3cnFNM3G271TJ
|
main @ 3b40272 (run) |
this PR @ ff7ee56 (run) |
|
|---|---|---|
| Test on ubuntu-latest | ❌ 92 failed, 1006 passed | ❌ 92 failed, 1019 passed |
| Test on windows-latest | ✅ | ✅ |
Same 92 failures, same platform, and 13 more passing tests here — the new diagnostic, analyzer-release-tracking and incrementality tests. This PR adds no failure.
Why now: 8b9df02 ("ci: adopt the unified dotnet workflow") added Linux to the test matrix two days ago — its own commit message calls this out as "new coverage for libraries that ship netstandard and claim to run anywhere but were only ever tested on one operating system." The coverage immediately found what it was looking for. Both main runs since then are red.
What is failing: all 92 are in Semantics.Paths tests that assume Windows path semantics — drive letters, \ separators, and the 259-character limit. A sample: SemanticPath_RootPath, PathOperators_CrossPlatformSeparators_HandleCorrectly, RelativeFilePath_AsAbsoluteWithBase_ResolvesCorrectly, IsChildOf_WithMixedSeparators_WorksCorrectly.
Not fixing it here. No fix exists to port, and writing one means reworking the path-test suite's platform assumptions across roughly a dozen files — entirely unrelated to extracting the code generator, and it would bury this diff. It wants its own issue and its own PR.
The shape of that fix, for whoever picks it up: the failures are in the tests, not in Semantics.Paths itself, so the work is to express each expectation in terms of the running platform — build fixtures with Path.Combine and Path.DirectorySeparatorChar instead of literal C:\... strings, take the root from Path.GetPathRoot(Environment.CurrentDirectory), and gate the 259-character limit behind [TestCategory("OS-Specific")], which CLAUDE.md already prescribes for exactly this. Worth checking as you go whether any failure is a genuine Semantics.Paths bug on Linux rather than a test assumption — that is what the new matrix is for.
I have not re-run the job: a re-run reproduces this identically, and it is already reproduced on main.
Generated by Claude Code
Layer 2 of #181 — isolating the parts of
Semantics.SourceGeneratorsthat have nothing to do with physical quantities, so moving them into a shared package becomes a mechanical lift.Closes #182, closes #183, closes #185, closes #186. #184 and #187 are blocked — see below.
What moved
CodeGen/now holds the reusable layer, with the physics-specific code depending on it and nothing depending back:CodeGen/GeneratorBase.csGenerators/GeneratorBase.csCodeGen/MetadataFile.csCodeGen/DiagnosticCatalog.csCodeGen/GeneratedSource.csGeneratedSource.csCodeGen/CSharpKeywords.csEmit.csMulti-file metadata (#182)
Two generators had worked around the base's single-file limit, not one.
UnitsGeneratorcarried its own copy of exactly whatQuantitiesGeneratorhad — anInitializeoverride, a privateLoadJson, aCombinedMetadatatype, and a deadGenerateshim to satisfy an abstract contract it never used. Both are gone; each now just declares what it reads:Also: the file header is a parameter rather than a hard-coded literal (bound once in
SemanticsGeneratorSettings), and metadata discovery matches the whole last path segment instead ofEndsWith, which also matched any file whose name merely ended with the wanted one.Diagnostics (#185)
One
DiagnosticCatalogfixes the prefix and the category, which closed three real gaps:CONV001is gone. The base reported parse failures under idCONV001, categorySourceGenerator— a leftover from when it served onlyConversionsGenerator— while everything derived from it usedSEM00x. Now SEM007.JsonExceptionand returnednull, so a malformedunits.jsonproduced no diagnostic at all and the generator silently emitted identity conversions — factories with no scale factor.SEM004 is now reported at the position in
dimensions.jsonwhere the offending unit name is written, so a warning about a name in a 5,000-line file says where the name is. SEM001 and SEM003 still report without a location — they are raised from static collectors that would need the metadata file threaded through six call levels; worth a follow-up, not worth widening this PR.Test harness (#186)
The
CSharpGeneratorDriversetup moves out ofSourceGeneratorTestsinto a standaloneGeneratorHarnessthat supplies every metadata file, the way MSBuild'sAdditionalFiles Include="Metadata/*.json"does. Handing a generator only the one file a test cares about is not how it runs for real.New coverage for things that were untested:
GeneratorDiagnosticTests— each of SEM001–SEM007 fires on the input it is meant to catch. Previously the suite only asserted the real metadata produces no diagnostics, which means all five warnings could have stopped firing and every test would still have passed.AnalyzerReleaseTrackingTests— every descriptor is tracked in the analyzer release files, identifiers are unique and consecutive, and the category is consistent. RS2008 no longer surfaces only after a push.IIncrementalGenerators and nothing checked they behave like one; a generator that recomputes everything on every keystroke passes every output assertion, it just makes the IDE slow.Verification
Semantics.Quantities/Generated/is byte-identical — the whole point for the mechanical half of this change.1019 passed, 92 failed. The 92 are the same Windows-path tests that already fail on Linux onmain(this container); 13 more tests pass than before.Blocked, not done
GeneratedSource) needsNewLinesfrom Make output line endings deterministic and configurable CodeBlocker#81.Both ship in ktsu-dev/CodeBlocker#87, but this repo pins
ktsu.CodeBlocker1.2.15 from NuGet, so neither can land until that PR merges and publishes.CodeGen/'sWriteFileHeader/WriteSourceFileareprivate protectedpurely becauseSourceFileTemplateis still local; they becomeprotectedwhen the templates come from the package, where they are already public.Notes for the reviewer
ktsu.Sdk.Analyzers(the container's SDK is 10.0.111 / Roslyn 5.0, the analyzer wants 5.9) and temporarily pinningMicrosoft.CodeAnalysis.CSharpto 5.0.0 so the generator could load at all.Directory.Packages.propsis restored and unchanged in the diff — please confirm CI agrees, since it is the first place a Roslyn-version assumption would show up.CLAUDE.mdanddocs/physics-generator.mdgain the SEM006/SEM007 entries and a diagnostics table.Generated by Claude Code