diff --git a/CLAUDE.md b/CLAUDE.md index 48dcdc77..25fb6734 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -41,7 +41,7 @@ The opt-in lives in `.sonarlint/sonar-local.props` (analyzer package) and `.sona | `Semantics.Music` | Immutable musical value types (`Pitch`, `Interval`, `Scale`, `Chord`, `Key`, `Duration`, `TimeSignature`) plus an analysis aggregate layer (`Progression`, `Section`, `Arrangement`, `Form`) computing roman numerals, cadences, key inference, chromatic identification, and named forms. Targets `net8.0`–`net10.0` + `netstandard2.0`/`netstandard2.1`. | | `Semantics.Color` | Physically-grounded color types. Canonical linear-RGB `Color` hub plus color-space satellites (`Srgb`, `Hsl`, `Hsv`, `Oklab`, `Oklch`); every type converts to and from every other, routed through the nearest shared hub (`Srgb` within the sRGB family, `Oklab` within the perceptual family, linear `Color` across families) so no conversion takes a redundant gamma round-trip. Also WCAG accessibility tooling, HSL/perceptual adjustment operations (lighten/saturate/hue/invert), and `NamedColors`. Targets `net8.0`–`net10.0` + `netstandard2.0`/`netstandard2.1`. | | `Semantics.Quantities` | Hand-written runtime types (`PhysicalQuantity`, `IVector0`..`IVector4`, `UnitSystem`) plus generator output under `Generated/`. | -| `Semantics.SourceGenerators` | Roslyn incremental generators that emit quantity types, units, conversions, magnitudes, physical constants, and storage-type helpers from metadata. | +| `Semantics.SourceGenerators` | Roslyn incremental generators that emit quantity types, units, conversions, magnitudes, physical constants, and storage-type helpers from metadata. `CodeGen/` holds the parts that are not specific to physics — the metadata-driven generator base, metadata loading, the diagnostic catalogue — and is being extracted into a shared toolkit (#181). | | `Semantics.Quantities.{Double,Float,Decimal}` | Props-only satellite packages. Each ships a `buildTransitive` props file (generated by `scripts/Generate-AliasProps.ps1`) that injects global-using aliases binding every quantity to one storage type, so consumers write `Mass` instead of `Mass`. | | `Semantics.Test` | MSTest project covering all of the above. | @@ -168,6 +168,7 @@ var converted = sourceString.As(); ### Working with the source generator +- A generator declares the metadata files it reads via `MetadataFileNames` and derives from `SemanticsGenerator` (one file) or `SemanticsMultiFileGenerator` (several). Neither needs to override `Initialize`. - Edit `Semantics.SourceGenerators/Metadata/dimensions.json` to add a dimension, vector form, semantic overload, or relationship. - Rebuild `Semantics.SourceGenerators` and the consuming `Semantics.Quantities` project; emitted files appear in `Semantics.Quantities/Generated/Semantics.SourceGenerators//`. - Treat generator output as committed source. Diff it before commit so accidental regressions are visible. @@ -179,6 +180,9 @@ var converted = sourceString.As(); - **SEM003** — a relationship's explicit `forms` list references a vector form not declared on a participating dimension. Use `forms` to constrain a relationship to specific vector forms (e.g. `crossProducts: [{ "other": "Length", "result": "Torque", "forms": [3] }]`); when omitted, the legacy "emit at every common form" behaviour is preserved. - **SEM004** — a dimension's `availableUnits` array references a unit name that isn't declared anywhere in `units.json`. Without the diagnostic the generator silently emits an identity-conversion `From{Unit}` factory, which is wrong for any non-base unit; SEM004 catches the typo at build time. - **SEM005** — schema-level validation issue in `logarithmic.json` (missing or duplicate scale names, a conversion with no linear type). + - **SEM006** — a metadata file a generator declared in `MetadataFileNames` was not supplied as an `AdditionalFile`. Previously this produced no output and no explanation, which is indistinguishable from a generator that simply had nothing to emit. + - **SEM007** — a metadata file could not be parsed. Replaces the base generator's `CONV001` in category `SourceGenerator`, and covers the path that used to swallow the exception, where a malformed `units.json` silently produced factories with no scale factor. + - Descriptors are allocated from `SemanticsDiagnostics`, which is the one place to add a new one. `AnalyzerReleaseTrackingTests` fails if the identifier is missing from `AnalyzerReleases.Unshipped.md`, so RS2008 no longer surfaces only after a push. - See `docs/physics-generator.md` for the full schema and an end-to-end "add a dimension" walk-through. This file is the entry point. For deeper material: diff --git a/Semantics.SourceGenerators/AnalyzerReleases.Unshipped.md b/Semantics.SourceGenerators/AnalyzerReleases.Unshipped.md index 63910aa1..92319b68 100644 --- a/Semantics.SourceGenerators/AnalyzerReleases.Unshipped.md +++ b/Semantics.SourceGenerators/AnalyzerReleases.Unshipped.md @@ -10,3 +10,5 @@ SEM002 | Semantics.SourceGenerators | Warning | Reports schema-level validation SEM003 | Semantics.SourceGenerators | Warning | Reports a relationship whose explicit `forms` list references a vector form not declared on a participating dimension. SEM004 | Semantics.SourceGenerators | Warning | Reports a `dimensions.json` `availableUnits` entry that doesn't match any unit declared in `units.json`. SEM005 | Semantics.SourceGenerators | Warning | Reports schema-level validation issues in logarithmic.json (missing or duplicate scale names, conversions with no linear type). +SEM006 | Semantics.SourceGenerators | Warning | Reports a metadata file a generator declared that was not supplied as an AdditionalFile. +SEM007 | Semantics.SourceGenerators | Error | Reports a metadata file that could not be parsed. Replaces the base generator's CONV001. diff --git a/Semantics.SourceGenerators/CodeGen/CSharpKeywords.cs b/Semantics.SourceGenerators/CodeGen/CSharpKeywords.cs new file mode 100644 index 00000000..d8662c3b --- /dev/null +++ b/Semantics.SourceGenerators/CodeGen/CSharpKeywords.cs @@ -0,0 +1,21 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace Semantics.SourceGenerators.CodeGen; + +/// +/// C# vocabulary a generator emits, named so a typo in a keyword is a compile error rather than +/// malformed generated source. +/// +/// +/// Nothing here is specific to any one generator; it is part of the reusable layer. The XML +/// documentation delimiters that used to live alongside these are gone: documentation is written +/// through the template model's DocComment, which owns the tags and escapes their content. +/// +internal static class CSharpKeywords +{ + /// The public modifier. + internal const string Public = "public"; + + /// The static modifier. + internal const string Static = "static"; +} diff --git a/Semantics.SourceGenerators/CodeGen/DiagnosticCatalog.cs b/Semantics.SourceGenerators/CodeGen/DiagnosticCatalog.cs new file mode 100644 index 00000000..e91c6651 --- /dev/null +++ b/Semantics.SourceGenerators/CodeGen/DiagnosticCatalog.cs @@ -0,0 +1,103 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace Semantics.SourceGenerators.CodeGen; + +using System.Collections.Generic; +using System.Globalization; +using Microsoft.CodeAnalysis; + +/// +/// Declares and reports a generator's diagnostics under one identifier prefix and category. +/// +/// +/// Every generator that reads metadata needs the same handful of diagnostics, and hand-rolling a +/// plus a private reporting helper per generator is how the +/// identifiers drift: the base generator used to report parse failures as CONV001 in category +/// SourceGenerator while everything derived from it used SEM00x in its own category. +/// Allocating them from one catalogue keeps the scheme consistent and gives +/// AnalyzerReleaseTrackingTests something to enumerate, so a descriptor that is missing from +/// AnalyzerReleases.Unshipped.md fails a test rather than RS2008 at build time. +/// +/// Nothing in this type is specific to any one generator; it is part of the reusable layer. +/// +/// +/// The identifier prefix, for example SEM. +/// The category reported on every descriptor. +public sealed class DiagnosticCatalog(string idPrefix, string category) +{ + private readonly List descriptors = []; + + /// Gets the category every descriptor in this catalogue is reported under. + public string Category { get; } = category; + + /// + /// Gets every descriptor allocated from this catalogue, in allocation order. + /// + public IReadOnlyList Descriptors => descriptors; + + /// + /// Allocates a warning descriptor. + /// + /// The numeric part of the identifier, formatted to three digits. + /// The diagnostic title. + /// The message format string. + /// The descriptor, also recorded in . + public DiagnosticDescriptor Warning(int number, string title, string messageFormat) => + Add(number, title, messageFormat, DiagnosticSeverity.Warning); + + /// + /// Allocates an error descriptor. + /// + /// The numeric part of the identifier, formatted to three digits. + /// The diagnostic title. + /// The message format string. + /// The descriptor, also recorded in . + public DiagnosticDescriptor Error(int number, string title, string messageFormat) => + Add(number, title, messageFormat, DiagnosticSeverity.Error); + + private DiagnosticDescriptor Add(int number, string title, string messageFormat, DiagnosticSeverity severity) + { + DiagnosticDescriptor descriptor = new( + id: idPrefix + number.ToString("D3", CultureInfo.InvariantCulture), + title: title, + messageFormat: messageFormat, + category: Category, + defaultSeverity: severity, + isEnabledByDefault: true); + + descriptors.Add(descriptor); + return descriptor; + } +} + +/// +/// Reporting helpers that keep a diagnostic to one call at the site that found the problem. +/// +public static class DiagnosticReporting +{ + /// + /// Reports a diagnostic with no source location. + /// + /// The source production context to report to. + /// The descriptor to report. + /// Arguments for the descriptor's message format. + public static void Report( + this SourceProductionContext context, + DiagnosticDescriptor descriptor, + params object?[] messageArgs) => + context.ReportDiagnostic(Diagnostic.Create(descriptor, Location.None, messageArgs)); + + /// + /// Reports a diagnostic pointing at a position in a metadata file. + /// + /// The source production context to report to. + /// The descriptor to report. + /// Where in the metadata the problem is, or . + /// Arguments for the descriptor's message format. + public static void ReportAt( + this SourceProductionContext context, + DiagnosticDescriptor descriptor, + Location? location, + params object?[] messageArgs) => + context.ReportDiagnostic(Diagnostic.Create(descriptor, location ?? Location.None, messageArgs)); +} diff --git a/Semantics.SourceGenerators/GeneratedSource.cs b/Semantics.SourceGenerators/CodeGen/GeneratedSource.cs similarity index 97% rename from Semantics.SourceGenerators/GeneratedSource.cs rename to Semantics.SourceGenerators/CodeGen/GeneratedSource.cs index 85187edd..e09bef63 100644 --- a/Semantics.SourceGenerators/GeneratedSource.cs +++ b/Semantics.SourceGenerators/CodeGen/GeneratedSource.cs @@ -1,6 +1,6 @@ // Copyright (c) 2023-2026 ktsu-dev contributors -namespace Semantics.SourceGenerators; +namespace Semantics.SourceGenerators.CodeGen; using System; using Microsoft.CodeAnalysis; diff --git a/Semantics.SourceGenerators/CodeGen/GeneratorBase.cs b/Semantics.SourceGenerators/CodeGen/GeneratorBase.cs new file mode 100644 index 00000000..265585d5 --- /dev/null +++ b/Semantics.SourceGenerators/CodeGen/GeneratorBase.cs @@ -0,0 +1,196 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace Semantics.SourceGenerators.CodeGen; + +using System; +using System.Collections.Generic; +using System.Collections.Immutable; +using System.Linq; +using ktsu.CodeBlocker; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.Text; +using Semantics.SourceGenerators.Templates; + +/// +/// Base class for a generator driven by one or more JSON metadata files supplied as +/// AdditionalFiles. +/// +/// +/// Nothing here is specific to any one generator: it finds the declared metadata files, hands them +/// to , and reports a diagnostic for anything missing or malformed. Declaring +/// more than one file is the normal case rather than a reason to reimplement +/// . +/// +public abstract class GeneratorBase : IIncrementalGenerator +{ + /// + /// Gets the names of the metadata files this generator reads, without their directories. + /// + protected abstract IReadOnlyList MetadataFileNames { get; } + + /// + /// Gets the catalogue this generator's diagnostics are allocated from. + /// + protected abstract DiagnosticCatalog Diagnostics { get; } + + /// + /// Gets the descriptor reported when a declared metadata file is not in the compilation. + /// + protected abstract DiagnosticDescriptor MetadataFileMissing { get; } + + /// + /// Gets the descriptor reported when a metadata file cannot be parsed. Its message format takes + /// the file name and the reason. + /// + protected abstract DiagnosticDescriptor MetadataParseFailed { get; } + + /// + public void Initialize(IncrementalGeneratorInitializationContext context) + { + IReadOnlyList wanted = MetadataFileNames; + + IncrementalValueProvider> metadataFiles = context.AdditionalTextsProvider + .Where(file => wanted.Any(name => IsNamed(file.Path, name))) + .Select((file, cancellationToken) => + { + SourceText? sourceText = file.GetText(cancellationToken); + return new MetadataFile(NameOf(file.Path), sourceText?.ToString() ?? string.Empty, sourceText, file.Path); + }) + .Where(file => file.Text.Length > 0) + .Collect(); + + context.RegisterSourceOutput(metadataFiles, (productionContext, files) => + { + // A duplicate name means the same metadata reached the compilation twice; the first + // wins, which is what the old EndsWith-plus-FirstOrDefault matching did implicitly. + Dictionary byName = files + .GroupBy(file => file.FileName, StringComparer.Ordinal) + .ToDictionary(group => group.Key, group => group.First(), StringComparer.Ordinal); + + // Previously a missing file produced no output and no explanation. + foreach (string name in wanted.Where(name => !byName.ContainsKey(name))) + { + productionContext.Report(MetadataFileMissing, name); + } + + Generate(productionContext, new MetadataSet(byName)); + }); + } + + /// + /// Emits this generator's sources. + /// + /// The source production context to add sources to. + /// The metadata files this generator declared. + protected abstract void Generate(SourceProductionContext context, MetadataSet metadata); + + /// + /// Creates a configured the way generated sources are written. + /// + /// A new . + protected static CodeBlocker CreateCodeBlocker() => CodeBlocker.Create(); + + /// + /// Writes the header every generated file starts with. + /// + /// The to write to. + /// + /// The copyright line written above the generated-file marker, without its // prefix. + /// or empty writes only the marker. + /// + /// + /// A parameter rather than the literal this used to hard-code, so a consuming repository can + /// keep the header in step with its own file header template from one place. + /// + /// Assembly-private only because SourceFileTemplate still lives in this project. Both + /// this and become protected once the template model comes + /// from ktsu.CodeBlocker, where it is already public. + /// + /// + /// is . + private protected static void WriteFileHeader(CodeBlocker codeBlocker, string? copyright) + { + if (codeBlocker is null) + { + throw new ArgumentNullException(nameof(codeBlocker)); + } + + if (!string.IsNullOrEmpty(copyright)) + { + codeBlocker.WriteLine($"// {copyright}"); + } + + codeBlocker.WriteLine("// "); + codeBlocker.NewLine(); + } + + /// + /// Writes a whole source file, header included. + /// + /// The to write to. + /// The file to write. + /// The copyright line, as for . + private protected static void WriteSourceFile(CodeBlocker codeBlocker, SourceFileTemplate sourceFileTemplate, string? copyright) + { + WriteFileHeader(codeBlocker, copyright); + codeBlocker.AddSourceFile(sourceFileTemplate); + } + + /// + /// Whether a path names the given file. + /// + /// The additional file's path. + /// The file name to match. + /// True when the path's last segment is exactly . + /// + /// Matched on the whole last segment rather than with EndsWith, which also matched + /// anything whose name merely ended with the wanted one. + /// + private static bool IsNamed(string path, string fileName) => + string.Equals(NameOf(path), fileName, StringComparison.Ordinal); + + /// + /// The last segment of a path, handling either directory separator so the generator behaves the + /// same wherever it runs. + /// + /// The path to take the name of. + /// The path's last segment. + private static string NameOf(string path) + { + int separator = path.LastIndexOfAny(['/', '\\']); + return separator < 0 ? path : path.Substring(separator + 1); + } +} + +/// +/// Base class for a generator driven by exactly one JSON metadata file. +/// +/// The shape the metadata file deserializes into. +/// The metadata file's name, without its directory. +public abstract class GeneratorBase(string metadataFileName) : GeneratorBase + where T : class +{ + /// + protected sealed override IReadOnlyList MetadataFileNames => [metadataFileName]; + + /// + protected sealed override void Generate(SourceProductionContext context, MetadataSet metadata) + { + T? deserialized = metadata[metadataFileName]?.Deserialize(context, MetadataParseFailed); + if (deserialized is null) + { + return; + } + + using CodeBlocker codeBlocker = CreateCodeBlocker(); + Generate(context, deserialized, codeBlocker); + } + + /// + /// Emits this generator's sources from its metadata. + /// + /// The source production context to add sources to. + /// The deserialized metadata. + /// A to build the output in. + protected abstract void Generate(SourceProductionContext context, T metadata, CodeBlocker codeBlocker); +} diff --git a/Semantics.SourceGenerators/CodeGen/MetadataFile.cs b/Semantics.SourceGenerators/CodeGen/MetadataFile.cs new file mode 100644 index 00000000..a7824a69 --- /dev/null +++ b/Semantics.SourceGenerators/CodeGen/MetadataFile.cs @@ -0,0 +1,105 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace Semantics.SourceGenerators.CodeGen; + +using System; +using System.Collections.Generic; +using System.Text.Json; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.Text; + +/// +/// One metadata file a generator was asked for, with the text it was found to contain. +/// +/// The file's name, without its directory. +/// The file's contents. +/// The underlying , used to build locations. +/// The file's full path, used to build locations. +public sealed class MetadataFile(string fileName, string text, SourceText? sourceText, string path) +{ + /// Gets the file's name, without its directory. + public string FileName { get; } = fileName; + + /// Gets the file's contents. + public string Text { get; } = text; + + /// + /// Finds the first occurrence of in the file and returns a location + /// covering it. + /// + /// The text to find — typically the offending name from the metadata. + /// + /// A location in the metadata file, or when the text is not found. + /// + /// + /// A diagnostic reported at tells the reader which name is wrong but + /// not where it is written, which for a file the size of dimensions.json is most of the + /// work. Matching on the name itself is approximate — the first occurrence wins, and a name that + /// appears in several entries points at the first — but it is navigable, which nothing was + /// before. + /// + public Location FindLocation(string needle) + { + if (sourceText is null || string.IsNullOrEmpty(needle)) + { + return Location.None; + } + + int index = Text.IndexOf(needle, StringComparison.Ordinal); + return index < 0 + ? Location.None + : Location.Create(path, new TextSpan(index, needle.Length), sourceText.Lines.GetLinePositionSpan(new TextSpan(index, needle.Length))); + } + + /// + /// Deserializes the file into . + /// + /// The metadata shape to deserialize into. + /// The source production context, used to report a parse failure. + /// The descriptor reported when the file cannot be parsed. + /// The deserialized metadata, or when parsing failed. + /// + /// A parse failure is always reported. It used to be swallowed on the path that loaded a second + /// metadata file, so a malformed units.json produced no diagnostic at all and the + /// generator silently emitted identity conversions. + /// + public T? Deserialize(SourceProductionContext context, DiagnosticDescriptor parseFailed) + where T : class + { + try + { + JsonSerializerOptions options = new() + { + PropertyNameCaseInsensitive = true + }; + + T? metadata = JsonSerializer.Deserialize(Text, options); + if (metadata is not null) + { + return metadata; + } + + context.Report(parseFailed, FileName, "the document deserialized to null"); + return null; + } + catch (JsonException ex) + { + context.Report(parseFailed, FileName, ex.Message); + return null; + } + } +} + +/// +/// The metadata files a generator asked for, keyed by file name. +/// +public sealed class MetadataSet(IReadOnlyDictionary files) +{ + /// + /// Gets the named metadata file. + /// + /// The file name the generator declared. + /// The file, or when it was not supplied to the compilation. + public MetadataFile? this[string fileName] => + files.TryGetValue(fileName, out MetadataFile? file) ? file : null; +} diff --git a/Semantics.SourceGenerators/Emit.cs b/Semantics.SourceGenerators/Emit.cs index 3b1be231..27e712f5 100644 --- a/Semantics.SourceGenerators/Emit.cs +++ b/Semantics.SourceGenerators/Emit.cs @@ -3,17 +3,21 @@ namespace Semantics.SourceGenerators; /// -/// Literal fragments the generators emit into the generated C#. Naming them keeps the emission -/// sites readable and means a typo in a keyword or a documentation delimiter is a compile error -/// rather than malformed generated source. +/// Literal fragments specific to this repository's generated output. /// +/// +/// The general C# vocabulary that used to sit alongside these — the public and +/// static modifiers, the XML documentation delimiters — has moved to the reusable layer as +/// CodeGen.CSharpKeywords. What is left is the part that only means something here: the +/// names this generator gives its parameters, and a suppression that is about physics. +/// internal static class Emit { /// The public modifier. - internal const string Public = "public"; + internal const string Public = CodeGen.CSharpKeywords.Public; /// The static modifier. - internal const string Static = "static"; + internal const string Static = CodeGen.CSharpKeywords.Static; /// Opening delimiter of an XML documentation summary. internal const string SummaryOpen = "/// "; @@ -27,9 +31,6 @@ internal static class Emit /// Conventional name of the right-hand operand on generated binary operators. internal const string RightParameter = "right"; - /// Category reported on generator diagnostics. - internal const string DiagnosticCategory = "Semantics.SourceGenerators"; - /// /// Suppression emitted onto generated physics operators. CA2225 wants named alternates such as /// Add or Multiply, but those names do not carry the dimensional meaning the diff --git a/Semantics.SourceGenerators/Generators/ConversionsGenerator.cs b/Semantics.SourceGenerators/Generators/ConversionsGenerator.cs index a73ed7c8..0150b73d 100644 --- a/Semantics.SourceGenerators/Generators/ConversionsGenerator.cs +++ b/Semantics.SourceGenerators/Generators/ConversionsGenerator.cs @@ -5,13 +5,14 @@ namespace Semantics.SourceGenerators; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; using Semantics.SourceGenerators.Models; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Templates; /// /// Source generator that creates the ConversionConstants.cs file from JSON metadata. /// [Generator] -public class ConversionsGenerator : GeneratorBase +public class ConversionsGenerator : SemanticsGenerator { public ConversionsGenerator() : base("conversions.json") { } diff --git a/Semantics.SourceGenerators/Generators/DimensionsGenerator.cs b/Semantics.SourceGenerators/Generators/DimensionsGenerator.cs index 87badaf1..58b883c9 100644 --- a/Semantics.SourceGenerators/Generators/DimensionsGenerator.cs +++ b/Semantics.SourceGenerators/Generators/DimensionsGenerator.cs @@ -7,13 +7,14 @@ namespace Semantics.SourceGenerators; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; using Semantics.SourceGenerators.Models; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Templates; /// /// Source generator that creates the PhysicalDimensions.cs file from JSON metadata. /// [Generator] -public class DimensionsGenerator : GeneratorBase +public class DimensionsGenerator : SemanticsGenerator { public DimensionsGenerator() : base("dimensions.json") { } diff --git a/Semantics.SourceGenerators/Generators/GeneratorBase.cs b/Semantics.SourceGenerators/Generators/GeneratorBase.cs deleted file mode 100644 index d58c4de5..00000000 --- a/Semantics.SourceGenerators/Generators/GeneratorBase.cs +++ /dev/null @@ -1,71 +0,0 @@ -// Copyright (c) 2023-2026 ktsu-dev contributors - -namespace Semantics.SourceGenerators; - -using System.Linq; -using System.Text.Json; -using ktsu.CodeBlocker; -using Microsoft.CodeAnalysis; -using Semantics.SourceGenerators.Templates; - -public abstract class GeneratorBase(string metadataFilename) : IIncrementalGenerator -{ - public virtual void Initialize(IncrementalGeneratorInitializationContext context) - { - // Find the conversions metadata JSON file - IncrementalValuesProvider metadataFiles = context.AdditionalTextsProvider - .Where(file => file.Path.EndsWith(metadataFilename, System.StringComparison.InvariantCulture)) - .Select((file, cancellationToken) => file.GetText(cancellationToken)?.ToString() ?? "") - .Where(content => !string.IsNullOrEmpty(content)); - - // Generate code from metadata - context.RegisterSourceOutput(metadataFiles, (ctx, jsonContent) => - { - if (string.IsNullOrEmpty(jsonContent)) - { - return; - } - - try - { - JsonSerializerOptions options = new() - { - PropertyNameCaseInsensitive = true - }; - - T metadata = JsonSerializer.Deserialize(jsonContent, options) ?? - throw new JsonException("Failed to deserialize metadata"); - - using CodeBlocker codeBlocker = CodeBlocker.Create(); - Generate(ctx, metadata, codeBlocker); - } - catch (JsonException ex) - { - // Report JSON parsing error - DiagnosticDescriptor descriptor = new( - "CONV001", - "JSON parsing error", - "Failed to parse metadata JSON: {0}", - "SourceGenerator", - DiagnosticSeverity.Error, - isEnabledByDefault: true); - - ctx.ReportDiagnostic(Diagnostic.Create(descriptor, Location.None, ex.Message)); - } - }); - } - - protected abstract void Generate(SourceProductionContext context, T metadata, CodeBlocker codeBlocker); - protected static void WriteHeaderTo(CodeBlocker codeBlocker) - { - codeBlocker.WriteLine("// Copyright (c) 2023-2026 ktsu-dev contributors"); - codeBlocker.WriteLine("// "); - codeBlocker.NewLine(); - } - - internal static void WriteSourceFileTo(CodeBlocker codeBlocker, SourceFileTemplate sourceFileTemplate) - { - WriteHeaderTo(codeBlocker); - codeBlocker.AddSourceFile(sourceFileTemplate); - } -} diff --git a/Semantics.SourceGenerators/Generators/LogarithmicScalesGenerator.cs b/Semantics.SourceGenerators/Generators/LogarithmicScalesGenerator.cs index c44f3328..888138e8 100644 --- a/Semantics.SourceGenerators/Generators/LogarithmicScalesGenerator.cs +++ b/Semantics.SourceGenerators/Generators/LogarithmicScalesGenerator.cs @@ -6,6 +6,7 @@ namespace Semantics.SourceGenerators; using System.Globalization; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Models; /// @@ -17,16 +18,8 @@ namespace Semantics.SourceGenerators; /// (named constants, cross-scale conversions) live in hand-written partials. /// [Generator] -public class LogarithmicScalesGenerator : GeneratorBase +public class LogarithmicScalesGenerator : SemanticsGenerator { - private static readonly DiagnosticDescriptor InvalidScaleDefinition = new( - id: "SEM005", - title: "logarithmic.json scale definition is invalid", - messageFormat: "logarithmic.json validation issue: {0}", - category: Emit.DiagnosticCategory, - defaultSeverity: DiagnosticSeverity.Warning, - isEnabledByDefault: true); - public LogarithmicScalesGenerator() : base("logarithmic.json") { } /// @@ -57,7 +50,7 @@ protected override void Generate(SourceProductionContext context, LogarithmicMet } private static void Report(SourceProductionContext context, string message) => - context.ReportDiagnostic(Diagnostic.Create(InvalidScaleDefinition, Location.None, message)); + context.Report(SemanticsDiagnostics.InvalidScaleDefinition, message); private static void EmitScale(SourceProductionContext context, LogarithmicScaleDefinition scale) { diff --git a/Semantics.SourceGenerators/Generators/MagnitudesGenerator.cs b/Semantics.SourceGenerators/Generators/MagnitudesGenerator.cs index a671f990..103c84d7 100644 --- a/Semantics.SourceGenerators/Generators/MagnitudesGenerator.cs +++ b/Semantics.SourceGenerators/Generators/MagnitudesGenerator.cs @@ -5,13 +5,14 @@ namespace Semantics.SourceGenerators; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; using Semantics.SourceGenerators.Models; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Templates; /// /// Source generator that creates the MetricMagnitudes.cs file from JSON metadata. /// [Generator] -public class MagnitudesGenerator : GeneratorBase +public class MagnitudesGenerator : SemanticsGenerator { public MagnitudesGenerator() : base("magnitudes.json") { } diff --git a/Semantics.SourceGenerators/Generators/PhysicalConstantsGenerator.cs b/Semantics.SourceGenerators/Generators/PhysicalConstantsGenerator.cs index 6209f059..9ae254de 100644 --- a/Semantics.SourceGenerators/Generators/PhysicalConstantsGenerator.cs +++ b/Semantics.SourceGenerators/Generators/PhysicalConstantsGenerator.cs @@ -7,6 +7,7 @@ namespace Semantics.SourceGenerators; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; using Semantics.SourceGenerators.Models; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Templates; /// @@ -20,7 +21,7 @@ namespace Semantics.SourceGenerators; /// significand/exponent representation rounds twice and loses the tail of the long CODATA literals. /// [Generator] -public class PhysicalConstantsGenerator : GeneratorBase +public class PhysicalConstantsGenerator : SemanticsGenerator { /// /// Name of the private nested holder that caches the parsed value of each constant per closed diff --git a/Semantics.SourceGenerators/Generators/PrecisionGenerator.cs b/Semantics.SourceGenerators/Generators/PrecisionGenerator.cs index c91c35e7..f07afaaa 100644 --- a/Semantics.SourceGenerators/Generators/PrecisionGenerator.cs +++ b/Semantics.SourceGenerators/Generators/PrecisionGenerator.cs @@ -6,13 +6,14 @@ namespace Semantics.SourceGenerators; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; using Semantics.SourceGenerators.Models; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Templates; /// /// Source generator that creates the StorageTypes.cs file from JSON metadata. /// [Generator] -public class PrecisionGenerator : GeneratorBase +public class PrecisionGenerator : SemanticsGenerator { public PrecisionGenerator() : base("precision.json") { } diff --git a/Semantics.SourceGenerators/Generators/QuantitiesGenerator.cs b/Semantics.SourceGenerators/Generators/QuantitiesGenerator.cs index d210c996..0aa156d9 100644 --- a/Semantics.SourceGenerators/Generators/QuantitiesGenerator.cs +++ b/Semantics.SourceGenerators/Generators/QuantitiesGenerator.cs @@ -8,6 +8,7 @@ namespace Semantics.SourceGenerators; using System.Text.Json; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Models; using Semantics.SourceGenerators.Templates; @@ -17,115 +18,40 @@ namespace Semantics.SourceGenerators; /// then generates each type with its assigned operators. /// [Generator] -public class QuantitiesGenerator : GeneratorBase +public class QuantitiesGenerator : SemanticsMultiFileGenerator { - private static readonly DiagnosticDescriptor UnknownDimensionReference = new( - id: "SEM001", - title: "Unknown dimension reference in physics relationship", - messageFormat: "Dimension '{0}' references unknown dimension '{1}' in {2}; the operator will not be generated. Check spelling and that the referenced dimension exists in dimensions.json.", - category: Emit.DiagnosticCategory, - defaultSeverity: DiagnosticSeverity.Warning, - isEnabledByDefault: true); - - private static readonly DiagnosticDescriptor MetadataValidationFailed = new( - id: "SEM002", - title: "dimensions.json metadata validation failed", - messageFormat: "dimensions.json validation issue: {0}", - category: Emit.DiagnosticCategory, - defaultSeverity: DiagnosticSeverity.Warning, - isEnabledByDefault: true); - - private static readonly DiagnosticDescriptor RelationshipFormMissing = new( - id: "SEM003", - title: "Relationship requires a vector form not declared on a participating dimension", - messageFormat: "Relationship in dimension '{0}' ({1}) explicitly requests form V{2}, but '{3}' does not declare that form. The operator will not be generated.", - category: Emit.DiagnosticCategory, - defaultSeverity: DiagnosticSeverity.Warning, - isEnabledByDefault: true); - - private static readonly DiagnosticDescriptor UnknownUnitReference = new( - id: "SEM004", - title: "dimensions.json references a unit not declared in units.json", - messageFormat: "Unit '{0}' (referenced by dimension '{1}'.availableUnits) is not declared in units.json; the generated From{0} factory will use an identity conversion. Add the unit to units.json or fix the spelling.", - category: Emit.DiagnosticCategory, - defaultSeverity: DiagnosticSeverity.Warning, - isEnabledByDefault: true); - - public QuantitiesGenerator() : base("dimensions.json") { } - /// - /// Holds the metadata that drives quantity emission. Combined from dimensions.json and - /// units.json so factory methods can apply per-unit conversion factors. Plain class - /// (not a positional record) because the netstandard2.0 source-generator target lacks - /// System.Runtime.CompilerServices.IsExternalInit. + /// Both metadata files, because per-unit conversion factors are needed to emit + /// From{Unit} factories for units that are not the SI base unit. /// - private sealed class CombinedMetadata + /// + /// This used to require overriding outright, along with a + /// private JSON loader, a combining type, and a dead shim to satisfy the single-file base's + /// abstract contract. Multi-file metadata is the normal case, so the base handles it. + /// + protected override IReadOnlyList MetadataFileNames => ["dimensions.json", "units.json"]; + + /// + protected override void Generate(SourceProductionContext context, MetadataSet metadata) { - public DimensionsMetadata Dimensions { get; } - public UnitsMetadata Units { get; } - - public CombinedMetadata(DimensionsMetadata dimensions, UnitsMetadata units) + MetadataFile? dimensionsFile = metadata["dimensions.json"]; + DimensionsMetadata? dimensions = dimensionsFile?.Deserialize(context, MetadataParseFailed); + if (dimensions is null) { - Dimensions = dimensions; - Units = units; + return; } - } - - /// - /// Override to load both dimensions.json and units.json. The base class only loads a single - /// metadata file; we need both because per-unit conversion factors are required to emit - /// From{Unit} factories that aren't the SI base unit. - /// - public override void Initialize(IncrementalGeneratorInitializationContext context) - { - IncrementalValueProvider dimensionsProvider = LoadJson(context, "dimensions.json"); - IncrementalValueProvider unitsProvider = LoadJson(context, "units.json"); - IncrementalValueProvider combined = dimensionsProvider.Combine(unitsProvider).Select(static (pair, _) => - pair.Left == null ? null : new CombinedMetadata(pair.Left, pair.Right ?? new UnitsMetadata())); - - context.RegisterSourceOutput(combined, (ctx, metadata) => - { - if (metadata == null) - { - return; - } - GenerateInner(ctx, metadata.Dimensions, metadata.Units); - }); - } + // A missing units.json is already reported as SEM006; carrying on with an empty set keeps + // the base-unit factories generatable. A malformed one is now reported as SEM007 rather + // than swallowed, which is what used to leave the generator silently emitting identity + // conversions. + UnitsMetadata units = + metadata["units.json"]?.Deserialize(context, MetadataParseFailed) ?? new UnitsMetadata(); - private static IncrementalValueProvider LoadJson(IncrementalGeneratorInitializationContext context, string filename) - where TMeta : class - { - return context.AdditionalTextsProvider - .Where(file => file.Path.EndsWith(filename, StringComparison.InvariantCulture)) - .Select((file, ct) => file.GetText(ct)?.ToString() ?? "") - .Where(content => !string.IsNullOrEmpty(content)) - .Select((content, _) => - { - try - { - return JsonSerializer.Deserialize(content, new JsonSerializerOptions { PropertyNameCaseInsensitive = true }); - } - catch (JsonException) - { - return null; - } - }) - .Where(m => m != null) - .Collect() - .Select((arr, _) => arr.FirstOrDefault()); + GenerateInner(context, dimensions, units, dimensionsFile); } - /// - /// The legacy abstract entry point is unused: the - /// overridden handles registration and calls - /// directly. This shim exists to satisfy the abstract contract. - /// - protected override void Generate(SourceProductionContext context, DimensionsMetadata metadata, CodeBlocker codeBlocker) - => GenerateInner(context, metadata, new UnitsMetadata()); - - private void GenerateInner(SourceProductionContext context, DimensionsMetadata metadata, UnitsMetadata units) + private void GenerateInner(SourceProductionContext context, DimensionsMetadata metadata, UnitsMetadata units, MetadataFile? dimensionsFile) { if (metadata.PhysicalDimensions == null || metadata.PhysicalDimensions.Count == 0) { @@ -137,10 +63,7 @@ private void GenerateInner(SourceProductionContext context, DimensionsMetadata m List validationIssues = metadata.Validate(); foreach (string issue in validationIssues) { - context.ReportDiagnostic(Diagnostic.Create( - MetadataValidationFailed, - Location.None, - issue)); + context.Report(SemanticsDiagnostics.MetadataValidationFailed, issue); } Dictionary unitMap = BuildUnitMap(units); @@ -150,7 +73,7 @@ private void GenerateInner(SourceProductionContext context, DimensionsMetadata m // back to identity conversion in that case, which is wrong for any non-base unit // — a typo (e.g. "Kilometres" vs "Kilometers") would silently produce a factory // with no scale factor. SEM004 catches that at build time. - ReportUnknownUnitReferences(context, metadata, unitMap); + ReportUnknownUnitReferences(context, metadata, unitMap, dimensionsFile); // Phase A: Build maps and collect operators Dictionary dimensionMap = BuildDimensionMap(metadata); @@ -586,7 +509,7 @@ private static void AddOp(List list, HashSet seen, string private static void ReportUnknownReference(SourceProductionContext context, string owningDimension, string unknownReference, string fieldPath) { context.ReportDiagnostic(Diagnostic.Create( - UnknownDimensionReference, + SemanticsDiagnostics.UnknownDimensionReference, Location.None, owningDimension, unknownReference, @@ -650,7 +573,7 @@ private static int[] ResolveForms( private static void ReportFormMissing(SourceProductionContext context, string owningDimension, string fieldPath, int form, string offendingDimension) { context.ReportDiagnostic(Diagnostic.Create( - RelationshipFormMissing, + SemanticsDiagnostics.RelationshipFormMissing, Location.None, owningDimension, fieldPath, @@ -686,12 +609,12 @@ private static Dictionary BuildUnitMap(UnitsMetadata uni private static void ReportUnknownUnitReferences( SourceProductionContext context, DimensionsMetadata metadata, - Dictionary unitMap) + Dictionary unitMap, + MetadataFile? dimensionsFile) { // If units.json wasn't loaded the map is empty; treating every unit as "unknown" - // would flood the build log. The CombinedMetadata loader already supplies a - // non-null UnitsMetadata even when units.json is missing — check for that case - // and bail rather than report a useless wall of warnings. + // would flood the build log. Its absence is already reported as SEM006, so bail + // rather than add a wall of warnings on top of it. if (unitMap.Count == 0) { return; @@ -713,11 +636,13 @@ private static void ReportUnknownUnitReferences( continue; } - context.ReportDiagnostic(Diagnostic.Create( - UnknownUnitReference, - Location.None, + // Pointed at where the name is actually written, so the warning is navigable + // rather than just naming a string to go and search a large file for. + context.ReportAt( + SemanticsDiagnostics.UnknownUnitReference, + dimensionsFile?.FindLocation(unitName), unitName, - dim.Name)); + dim.Name); } } } diff --git a/Semantics.SourceGenerators/Generators/UnitsGenerator.cs b/Semantics.SourceGenerators/Generators/UnitsGenerator.cs index 436e832b..ffda80db 100644 --- a/Semantics.SourceGenerators/Generators/UnitsGenerator.cs +++ b/Semantics.SourceGenerators/Generators/UnitsGenerator.cs @@ -8,6 +8,7 @@ namespace Semantics.SourceGenerators; using System.Text.Json; using ktsu.CodeBlocker; using Microsoft.CodeAnalysis; +using Semantics.SourceGenerators.CodeGen; using Semantics.SourceGenerators.Models; using Semantics.SourceGenerators.Templates; @@ -22,67 +23,38 @@ namespace Semantics.SourceGenerators; /// generated quantities can accept dimensionally-correct units only at compile time. /// [Generator] -public class UnitsGenerator : GeneratorBase +public class UnitsGenerator : SemanticsMultiFileGenerator { - public UnitsGenerator() : base("units.json") { } - - private sealed class CombinedMetadata + /// + /// Both metadata files, because each unit's declaration records which dimensions use it, and + /// that mapping only exists in dimensions.json. + /// + /// + /// This generator carried its own copy of the same workaround QuantitiesGenerator did — + /// an Initialize override, a private JSON loader, a combining type, and a dead shim for + /// the single-file base's abstract contract. Two generators reimplementing the same thing is + /// what made multi-file support belong in the base. + /// + protected override IReadOnlyList MetadataFileNames => ["units.json", "dimensions.json"]; + + /// + protected override void Generate(SourceProductionContext context, MetadataSet metadata) { - public UnitsMetadata Units { get; } - public DimensionsMetadata Dimensions { get; } - - public CombinedMetadata(UnitsMetadata units, DimensionsMetadata dimensions) + UnitsMetadata? units = metadata["units.json"]?.Deserialize(context, MetadataParseFailed); + if (units is null) { - Units = units; - Dimensions = dimensions; + return; } - } - public override void Initialize(IncrementalGeneratorInitializationContext context) - { - IncrementalValueProvider unitsProvider = LoadJson(context, "units.json"); - IncrementalValueProvider dimensionsProvider = LoadJson(context, "dimensions.json"); - IncrementalValueProvider combined = unitsProvider.Combine(dimensionsProvider).Select(static (pair, _) => - pair.Left == null ? null : new CombinedMetadata(pair.Left, pair.Right ?? new DimensionsMetadata())); - - context.RegisterSourceOutput(combined, (ctx, metadata) => - { - if (metadata == null) - { - return; - } + // A missing dimensions.json is reported as SEM006; an empty set still lets the unit + // declarations themselves be emitted, just without their dimension cross-references. + DimensionsMetadata dimensions = + metadata["dimensions.json"]?.Deserialize(context, MetadataParseFailed) ?? new DimensionsMetadata(); - using CodeBlocker codeBlocker = CodeBlocker.Create(); - GenerateInner(ctx, metadata.Units, metadata.Dimensions, codeBlocker); - }); + using CodeBlocker codeBlocker = CreateCodeBlocker(); + GenerateInner(context, units, dimensions, codeBlocker); } - private static IncrementalValueProvider LoadJson(IncrementalGeneratorInitializationContext context, string filename) - where TMeta : class - { - return context.AdditionalTextsProvider - .Where(file => file.Path.EndsWith(filename, StringComparison.InvariantCulture)) - .Select((file, ct) => file.GetText(ct)?.ToString() ?? "") - .Where(content => !string.IsNullOrEmpty(content)) - .Select((content, _) => - { - try - { - return JsonSerializer.Deserialize(content, new JsonSerializerOptions { PropertyNameCaseInsensitive = true }); - } - catch (JsonException) - { - return null; - } - }) - .Where(m => m != null) - .Collect() - .Select((arr, _) => arr.FirstOrDefault()); - } - - protected override void Generate(SourceProductionContext context, UnitsMetadata metadata, CodeBlocker codeBlocker) - => GenerateInner(context, metadata, new DimensionsMetadata(), codeBlocker); - private static void GenerateInner(SourceProductionContext context, UnitsMetadata units, DimensionsMetadata dimensions, CodeBlocker codeBlocker) { Dictionary> unitToDimensions = BuildUnitToDimensionsMap(dimensions); diff --git a/Semantics.SourceGenerators/SemanticsDiagnostics.cs b/Semantics.SourceGenerators/SemanticsDiagnostics.cs new file mode 100644 index 00000000..2261ef85 --- /dev/null +++ b/Semantics.SourceGenerators/SemanticsDiagnostics.cs @@ -0,0 +1,81 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace Semantics.SourceGenerators; + +using System.Collections.Generic; +using Microsoft.CodeAnalysis; +using Semantics.SourceGenerators.CodeGen; + +/// +/// Every diagnostic this repository's generators report, allocated from one catalogue so the +/// identifiers stay consecutive and the category stays consistent. +/// +/// +/// Documented in CLAUDE.md and docs/physics-generator.md; keep those in step when +/// adding one, and add the new identifier to AnalyzerReleases.Unshipped.md — +/// AnalyzerReleaseTrackingTests checks that it is there. +/// +public static class SemanticsDiagnostics +{ + /// The catalogue every descriptor below is allocated from. + public static DiagnosticCatalog Catalog { get; } = new("SEM", "Semantics.SourceGenerators"); + + /// Gets every descriptor this repository's generators can report. + public static IReadOnlyList All => Catalog.Descriptors; + + /// SEM001: a relationship names a dimension that does not exist. + public static DiagnosticDescriptor UnknownDimensionReference { get; } = Catalog.Warning( + 1, + "Unknown dimension reference in physics relationship", + "Dimension '{0}' references unknown dimension '{1}' in {2}; the operator will not be generated. Check spelling and that the referenced dimension exists in dimensions.json."); + + /// SEM002: dimensions.json failed schema-level validation. + public static DiagnosticDescriptor MetadataValidationFailed { get; } = Catalog.Warning( + 2, + "dimensions.json metadata validation failed", + "dimensions.json validation issue: {0}"); + + /// SEM003: a relationship requires a vector form a participant does not declare. + public static DiagnosticDescriptor RelationshipFormMissing { get; } = Catalog.Warning( + 3, + "Relationship requires a vector form not declared on a participating dimension", + "Relationship in dimension '{0}' ({1}) explicitly requests form V{2}, but '{3}' does not declare that form. The operator will not be generated."); + + /// SEM004: dimensions.json names a unit that units.json does not declare. + public static DiagnosticDescriptor UnknownUnitReference { get; } = Catalog.Warning( + 4, + "dimensions.json references a unit not declared in units.json", + "Unit '{0}' (referenced by dimension '{1}'.availableUnits) is not declared in units.json; the generated From{0} factory will use an identity conversion. Add the unit to units.json or fix the spelling."); + + /// SEM005: logarithmic.json failed schema-level validation. + public static DiagnosticDescriptor InvalidScaleDefinition { get; } = Catalog.Warning( + 5, + "logarithmic.json scale definition is invalid", + "logarithmic.json validation issue: {0}"); + + /// + /// SEM006: a metadata file a generator declared is not in the compilation. + /// + /// + /// Previously the generator produced no output and no explanation, which is indistinguishable + /// from a generator that simply had nothing to emit. + /// + public static DiagnosticDescriptor MetadataFileMissing { get; } = Catalog.Warning( + 6, + "A metadata file is missing from the compilation", + "Metadata file '{0}' was not supplied as an AdditionalFile; the generator produced nothing. Check the AdditionalFiles item group in the consuming project."); + + /// + /// SEM007: a metadata file could not be parsed. + /// + /// + /// Replaces the base generator's CONV001 in category SourceGenerator, a leftover + /// from when the base served only ConversionsGenerator. It also covers the path that used + /// to swallow the exception and return null, where a malformed units.json produced no + /// diagnostic and the generator silently emitted identity conversions. + /// + public static DiagnosticDescriptor MetadataParseFailed { get; } = Catalog.Error( + 7, + "A metadata file could not be parsed", + "Metadata file '{0}' could not be parsed: {1}"); +} diff --git a/Semantics.SourceGenerators/SemanticsGenerator.cs b/Semantics.SourceGenerators/SemanticsGenerator.cs new file mode 100644 index 00000000..cd8b80b7 --- /dev/null +++ b/Semantics.SourceGenerators/SemanticsGenerator.cs @@ -0,0 +1,90 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace Semantics.SourceGenerators; + +using ktsu.CodeBlocker; +using Microsoft.CodeAnalysis; +using Semantics.SourceGenerators.CodeGen; +using Semantics.SourceGenerators.Templates; + +/// +/// The settings every generator in this repository shares, in one place. +/// +internal static class SemanticsGeneratorSettings +{ + /// + /// The copyright line written above the generated-file marker. + /// + /// + /// Must match file_header_template in .editorconfig, which ktsu.Sdk syncs from + /// COPYRIGHT.md on every build. Generated output is committed, so a drift shows up as a + /// diff rather than a build error — SourceGeneratorTests asserts the emitted header to + /// turn that into a failing test instead. + /// + internal const string Copyright = "Copyright (c) 2023-2026 ktsu-dev contributors"; +} + +/// +/// Base class for a generator in this repository driven by one metadata file. +/// +/// The shape the metadata file deserializes into. +/// The metadata file's name. +public abstract class SemanticsGenerator(string metadataFileName) : GeneratorBase(metadataFileName) + where T : class +{ + /// + protected sealed override DiagnosticCatalog Diagnostics => SemanticsDiagnostics.Catalog; + + /// + protected sealed override DiagnosticDescriptor MetadataFileMissing => SemanticsDiagnostics.MetadataFileMissing; + + /// + protected sealed override DiagnosticDescriptor MetadataParseFailed => SemanticsDiagnostics.MetadataParseFailed; + + + /// + /// Writes the header every generated file in this repository starts with. + /// + /// The to write to. + private protected static void WriteHeaderTo(CodeBlocker codeBlocker) => + WriteFileHeader(codeBlocker, SemanticsGeneratorSettings.Copyright); + + /// + /// Writes a whole source file, header included. + /// + /// The to write to. + /// The file to write. + private protected static void WriteSourceFileTo(CodeBlocker codeBlocker, SourceFileTemplate sourceFileTemplate) => + WriteSourceFile(codeBlocker, sourceFileTemplate, SemanticsGeneratorSettings.Copyright); +} + +/// +/// Base class for a generator in this repository driven by more than one metadata file. +/// +public abstract class SemanticsMultiFileGenerator : GeneratorBase +{ + /// + protected sealed override DiagnosticCatalog Diagnostics => SemanticsDiagnostics.Catalog; + + /// + protected sealed override DiagnosticDescriptor MetadataFileMissing => SemanticsDiagnostics.MetadataFileMissing; + + /// + protected sealed override DiagnosticDescriptor MetadataParseFailed => SemanticsDiagnostics.MetadataParseFailed; + + + /// + /// Writes the header every generated file in this repository starts with. + /// + /// The to write to. + private protected static void WriteHeaderTo(CodeBlocker codeBlocker) => + WriteFileHeader(codeBlocker, SemanticsGeneratorSettings.Copyright); + + /// + /// Writes a whole source file, header included. + /// + /// The to write to. + /// The file to write. + private protected static void WriteSourceFileTo(CodeBlocker codeBlocker, SourceFileTemplate sourceFileTemplate) => + WriteSourceFile(codeBlocker, sourceFileTemplate, SemanticsGeneratorSettings.Copyright); +} diff --git a/Semantics.Test/Quantities/AnalyzerReleaseTrackingTests.cs b/Semantics.Test/Quantities/AnalyzerReleaseTrackingTests.cs new file mode 100644 index 00000000..11e87a0f --- /dev/null +++ b/Semantics.Test/Quantities/AnalyzerReleaseTrackingTests.cs @@ -0,0 +1,79 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace ktsu.Semantics.Test.Quantities; + +using System.Collections.Generic; +using System.Linq; +using Microsoft.CodeAnalysis; +using Microsoft.VisualStudio.TestTools.UnitTesting; +using global::Semantics.SourceGenerators; + +/// +/// Checks that every diagnostic the generators can report is tracked in the analyzer release files. +/// +/// +/// EnforceExtendedAnalyzerRules is on, so an untracked descriptor fails the generator build +/// with RS2008 — after a push, and with an error that reads like a tooling problem rather than a +/// missing line in a markdown table. Enumerating the catalogue turns it into a test failure that +/// names the identifier. +/// +[TestClass] +public class AnalyzerReleaseTrackingTests +{ + [TestMethod] + public void EveryDescriptorIsTrackedInAnAnalyzerReleaseFile() + { + string tracked = ReadReleaseFiles(); + + List untracked = + [ + .. SemanticsDiagnostics.All + .Select(descriptor => descriptor.Id) + .Where(id => !tracked.Contains(id, StringComparison.Ordinal)) + ]; + + Assert.IsEmpty( + untracked, + $"Add to AnalyzerReleases.Unshipped.md: {string.Join(", ", untracked)}"); + } + + [TestMethod] + public void DescriptorIdentifiersAreUniqueAndConsecutive() + { + List ids = [.. SemanticsDiagnostics.All.Select(descriptor => descriptor.Id)]; + + Assert.HasCount(ids.Count, ids.Distinct().ToList(), "Two diagnostics share an identifier."); + + for (int index = 0; index < ids.Count; index++) + { + Assert.AreEqual($"SEM{index + 1:D3}", ids[index]); + } + } + + [TestMethod] + public void EveryDescriptorSharesTheOneCategory() + { + foreach (DiagnosticDescriptor descriptor in SemanticsDiagnostics.All) + { + // The base generator used to report parse failures under "SourceGenerator" while + // everything derived from it used this one. + Assert.AreEqual("Semantics.SourceGenerators", descriptor.Category, descriptor.Id); + } + } + + private static string ReadReleaseFiles() + { + DirectoryInfo? directory = new(AppContext.BaseDirectory); + while (directory is not null && !Directory.Exists(Path.Combine(directory.FullName, "Semantics.SourceGenerators"))) + { + directory = directory.Parent; + } + + Assert.IsNotNull(directory, "Could not locate the repository root from the test output directory."); + + string generatorDirectory = Path.Combine(directory!.FullName, "Semantics.SourceGenerators"); + return string.Concat( + File.ReadAllText(Path.Combine(generatorDirectory, "AnalyzerReleases.Shipped.md")), + File.ReadAllText(Path.Combine(generatorDirectory, "AnalyzerReleases.Unshipped.md"))); + } +} diff --git a/Semantics.Test/Quantities/GeneratorDiagnosticTests.cs b/Semantics.Test/Quantities/GeneratorDiagnosticTests.cs new file mode 100644 index 00000000..8dccde9d --- /dev/null +++ b/Semantics.Test/Quantities/GeneratorDiagnosticTests.cs @@ -0,0 +1,178 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace ktsu.Semantics.Test.Quantities; + +using System.Collections.Generic; +using System.Linq; +using Microsoft.CodeAnalysis; +using Microsoft.VisualStudio.TestTools.UnitTesting; +using global::Semantics.SourceGenerators; + +/// +/// Proves each generator diagnostic fires on the input it is meant to catch. +/// +/// +/// The existing generator tests only asserted that the real metadata produces no +/// diagnostics. That leaves the diagnostics themselves untested: SEM001 through SEM005 could stop +/// firing entirely and every test would still pass, which for a set of warnings whose whole job is +/// to catch typos in a large JSON file is the wrong way round. +/// +[TestClass] +public class GeneratorDiagnosticTests +{ + private static string MetadataDirectory => Path.Combine(AppContext.BaseDirectory, "GeneratorMetadata"); + + private static GeneratorHarness Harness => new(MetadataDirectory); + + /// + /// A dimensions document small enough to reason about, with one hook for the test to break. + /// + /// Relationship JSON to splice into the Length dimension. + /// The units Length declares. + /// The document. + private static string DimensionsDocument(string relationships = "", string availableUnits = "\"Meter\"") => + $$""" + { + "physicalDimensions": [ + { + "name": "Length", + "symbol": "L", + "dimensionalFormula": { "length": 1 }, + "availableUnits": [ {{availableUnits}} ], + "quantities": { "vector0": { "base": "Length" }, "vector3": { "base": "Displacement3D" } }{{relationships}} + } + ] + } + """; + + private static IReadOnlyList Run(string generatorMetadata, IIncrementalGenerator generator, string fileName) => + [.. Harness.Run(generator, new Dictionary { [fileName] = generatorMetadata }).Diagnostics]; + + private static void AssertReports(IReadOnlyList diagnostics, string id) + { + Assert.IsTrue( + diagnostics.Any(diagnostic => diagnostic.Id == id), + $"Expected {id}. Got: {(diagnostics.Count == 0 ? "no diagnostics" : string.Join("; ", diagnostics.Select(d => $"{d.Id}: {d.GetMessage()}")))}"); + } + + [TestMethod] + public void Sem001_IsReportedForARelationshipNamingAnUnknownDimension() + { + string metadata = DimensionsDocument( + relationships: ",\n \"integrals\": [ { \"other\": \"Tiem\", \"result\": \"Length\" } ]"); + + AssertReports(Run(metadata, new QuantitiesGenerator(), "dimensions.json"), "SEM001"); + } + + [TestMethod] + public void Sem002_IsReportedForADimensionMissingItsSymbol() + { + string metadata = + """ + { + "physicalDimensions": [ + { + "name": "Length", + "availableUnits": [ "Meter" ], + "quantities": { "vector0": { "base": "Length" } } + } + ] + } + """; + + AssertReports(Run(metadata, new QuantitiesGenerator(), "dimensions.json"), "SEM002"); + } + + [TestMethod] + public void Sem003_IsReportedWhenARelationshipRequestsAnUndeclaredForm() + { + // Length declares vector0 and vector3; asking for the cross product at V2 cannot be honoured. + string metadata = DimensionsDocument( + relationships: ",\n \"crossProducts\": [ { \"other\": \"Length\", \"result\": \"Length\", \"forms\": [ 2 ] } ]"); + + AssertReports(Run(metadata, new QuantitiesGenerator(), "dimensions.json"), "SEM003"); + } + + [TestMethod] + public void Sem004_IsReportedForAUnitThatUnitsJsonDoesNotDeclare() + { + string metadata = DimensionsDocument(availableUnits: "\"Meter\", \"Kilometres\""); + + AssertReports(Run(metadata, new QuantitiesGenerator(), "dimensions.json"), "SEM004"); + } + + [TestMethod] + public void Sem004_PointsAtWhereTheUnitIsWrittenRatherThanAtNothing() + { + string metadata = DimensionsDocument(availableUnits: "\"Meter\", \"Kilometres\""); + + Diagnostic diagnostic = Run(metadata, new QuantitiesGenerator(), "dimensions.json") + .First(candidate => candidate.Id == "SEM004"); + + Assert.AreNotEqual( + Location.None, + diagnostic.Location, + "A warning about a name in a large JSON file is only actionable if it says where the name is."); + Assert.EndsWith("dimensions.json", diagnostic.Location.GetLineSpan().Path); + } + + [TestMethod] + public void Sem005_IsReportedForADuplicateLogarithmicScale() + { + string metadata = + """ + { + "logarithmicScales": [ + { "name": "Decibels", "description": "A.", "base": 10, "multiplier": 20, "reference": 1 }, + { "name": "Decibels", "description": "B.", "base": 10, "multiplier": 20, "reference": 1 } + ] + } + """; + + AssertReports(Run(metadata, new LogarithmicScalesGenerator(), "logarithmic.json"), "SEM005"); + } + + [TestMethod] + public void Sem006_IsReportedWhenAGeneratorsSecondMetadataFileIsMissing() + { + // QuantitiesGenerator needs units.json for its non-base-unit conversion factors. Without + // SEM006 its absence produced no output and no explanation. + GeneratorRunResult result = Harness.RunWithOnly(new QuantitiesGenerator(), "dimensions.json"); + + AssertReports([.. result.Diagnostics], "SEM006"); + } + + [TestMethod] + public void Sem007_IsReportedForASecondMetadataFileThatIsMalformed() + { + // This path used to swallow the JsonException and carry on with an empty unit set, so a + // malformed units.json silently produced factories with no scale factor. + IReadOnlyList diagnostics = Run("{ not valid json", new QuantitiesGenerator(), "units.json"); + + AssertReports(diagnostics, "SEM007"); + } + + [TestMethod] + public void TheRealMetadataReportsNothing() + { + List generators = + [ + new ConversionsGenerator(), + new DimensionsGenerator(), + new LogarithmicScalesGenerator(), + new MagnitudesGenerator(), + new PhysicalConstantsGenerator(), + new PrecisionGenerator(), + new QuantitiesGenerator(), + new UnitsGenerator(), + ]; + + foreach (IIncrementalGenerator generator in generators) + { + GeneratorRunResult result = Harness.Run(generator); + Assert.IsEmpty( + result.Diagnostics, + $"{generator.GetType().Name}: {string.Join("; ", result.Diagnostics.Select(d => $"{d.Id}: {d.GetMessage()}"))}"); + } + } +} diff --git a/Semantics.Test/Quantities/GeneratorHarness.cs b/Semantics.Test/Quantities/GeneratorHarness.cs new file mode 100644 index 00000000..b4f64ba7 --- /dev/null +++ b/Semantics.Test/Quantities/GeneratorHarness.cs @@ -0,0 +1,159 @@ +// Copyright (c) 2023-2026 ktsu-dev contributors + +namespace ktsu.Semantics.Test.Quantities; + +using System.Collections.Generic; +using System.Threading; +using Microsoft.CodeAnalysis; +using Microsoft.CodeAnalysis.CSharp; +using Microsoft.CodeAnalysis.Text; + +/// +/// Runs an incremental generator over a set of metadata files and hands back what it produced. +/// +/// +/// Standing a generator up under is fiddly — reference +/// resolution, an shim, and the packaging escape hatch this repository +/// needed so the test project can reference the generator as a plain library +/// (BundleAnalyzerDependencies=false) — and none of it is specific to these generators. +/// It is written as a standalone harness so it can move to the shared generator toolkit alongside +/// GeneratorBase, rather than being rediscovered by the next project that writes one. +/// +/// The directory the real metadata files were copied to. +internal sealed class GeneratorHarness(string metadataDirectory) +{ + /// + /// Runs a generator against the real metadata. + /// + /// The generator to run. + /// + /// Metadata to substitute or add, keyed by file name. An entry replaces the real file's contents; + /// a name that is not a real file is added. + /// + /// The generator's run result. + /// + /// Every metadata file in the directory is supplied, the way MSBuild's + /// AdditionalFiles Include="Metadata/*.json" item group supplies them. Handing a + /// generator only the one file the test is interested in is not how it runs for real, and a + /// generator that reads two files would report one of them missing. + /// + internal GeneratorRunResult Run( + IIncrementalGenerator generator, + IReadOnlyDictionary? overrides = null) => + RunAll([generator], overrides).Results[0]; + + /// + /// Runs a generator against a metadata set that contains only the named files. + /// + /// The generator to run. + /// The metadata file names to supply. + /// The generator's run result. + internal GeneratorRunResult RunWithOnly(IIncrementalGenerator generator, params string[] fileNames) + { + List texts = []; + foreach (string fileName in fileNames) + { + texts.Add(new InMemoryAdditionalText( + Path.Combine(metadataDirectory, fileName), + File.ReadAllText(Path.Combine(metadataDirectory, fileName)))); + } + + return Drive([generator], texts).Results[0]; + } + + /// + /// Runs generators against the real metadata and returns the whole driver result, so a caller + /// can inspect tracked steps as well as output. + /// + /// The generators to run. + /// Metadata to substitute or add, keyed by file name. + /// The driver's run result. + internal GeneratorDriverRunResult RunAll( + IReadOnlyList generators, + IReadOnlyDictionary? overrides = null) => + Drive(generators, BuildTexts(overrides)); + + /// + /// Runs a generator twice over identical metadata and reports whether the second run reused the + /// first run's cached outputs. + /// + /// The generator to run. + /// True when no tracked output step had to be recomputed on the second run. + /// + /// These are s, and nothing checked that they behave like + /// one: a generator that recomputes everything on every keystroke still passes every output + /// assertion, it just makes the IDE slow. + /// + internal bool ReusesCachedOutputOnRerun(IIncrementalGenerator generator) + { + List texts = BuildTexts(null); + CSharpCompilation compilation = CreateCompilation(); + + GeneratorDriver driver = CSharpGeneratorDriver.Create( + generators: [generator.AsSourceGenerator()], + additionalTexts: texts, + parseOptions: null, + optionsProvider: null, + driverOptions: new GeneratorDriverOptions(IncrementalGeneratorOutputKind.None, trackIncrementalGeneratorSteps: true)); + + driver = driver.RunGenerators(compilation); + GeneratorDriverRunResult second = driver.RunGenerators(compilation).GetRunResult(); + + return !second.Results[0].TrackedOutputSteps + .SelectMany(pair => pair.Value) + .SelectMany(step => step.Outputs) + .Any(output => output.Reason is not (IncrementalStepRunReason.Cached or IncrementalStepRunReason.Unchanged)); + } + + private List BuildTexts(IReadOnlyDictionary? overrides) + { + Dictionary byName = []; + foreach (string path in Directory.GetFiles(metadataDirectory, "*.json")) + { + byName[Path.GetFileName(path)] = File.ReadAllText(path); + } + + if (overrides is not null) + { + foreach (KeyValuePair entry in overrides) + { + byName[entry.Key] = entry.Value; + } + } + + List texts = []; + foreach (KeyValuePair entry in byName) + { + texts.Add(new InMemoryAdditionalText(Path.Combine(metadataDirectory, entry.Key), entry.Value)); + } + + return texts; + } + + private static GeneratorDriverRunResult Drive(IReadOnlyList generators, List texts) + { + GeneratorDriver driver = CSharpGeneratorDriver.Create( + generators: [.. generators.Select(generator => generator.AsSourceGenerator())], + additionalTexts: texts); + + return driver.RunGenerators(CreateCompilation()).GetRunResult(); + } + + private static CSharpCompilation CreateCompilation() => + CSharpCompilation.Create( + assemblyName: "GeneratorHost", + syntaxTrees: [], + references: [MetadataReference.CreateFromFile(typeof(object).Assembly.Location)], + options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + + /// + /// Supplies metadata JSON to the generators the same way MSBuild's AdditionalFiles would. + /// + private sealed class InMemoryAdditionalText(string path, string text) : AdditionalText + { + public override string Path { get; } = path; + + public override SourceText GetText(CancellationToken cancellationToken = default) => + SourceText.From(text); + } +} diff --git a/Semantics.Test/Quantities/SourceGeneratorTests.cs b/Semantics.Test/Quantities/SourceGeneratorTests.cs index d920d0ae..41bc0acb 100644 --- a/Semantics.Test/Quantities/SourceGeneratorTests.cs +++ b/Semantics.Test/Quantities/SourceGeneratorTests.cs @@ -2,10 +2,8 @@ namespace ktsu.Semantics.Test.Quantities; -using System.Threading; using Microsoft.CodeAnalysis; using Microsoft.CodeAnalysis.CSharp; -using Microsoft.CodeAnalysis.Text; using Microsoft.VisualStudio.TestTools.UnitTesting; // Fully qualified: inside namespace ktsu.Semantics.Test, a bare "Semantics.SourceGenerators" // would bind to ktsu.Semantics.SourceGenerators, which does not exist. @@ -30,6 +28,8 @@ public class SourceGeneratorTests private static string MetadataDirectory => Path.Combine(AppContext.BaseDirectory, "GeneratorMetadata"); + private static GeneratorHarness Harness => new(MetadataDirectory); + /// /// Every generator paired with the metadata file it consumes. /// @@ -57,7 +57,7 @@ public void EveryGenerator_EmitsSourcesWithTheCanonicalHeader() foreach ((IIncrementalGenerator generator, string metadataFileName) in Generators) { string generatorName = generator.GetType().Name; - GeneratorRunResult result = RunGenerator(generator, metadataFileName); + GeneratorRunResult result = Harness.Run(generator); Assert.IsEmpty( result.Diagnostics, @@ -86,7 +86,7 @@ public void EveryGenerator_EmitsSourcesWithTheCanonicalHeader() [TestMethod] public void QuantitiesGenerator_EmitsTheExpectedBreadthOfTypes() { - GeneratorRunResult result = RunGenerator(new QuantitiesGenerator(), "dimensions.json"); + GeneratorRunResult result = Harness.Run(new QuantitiesGenerator()); // Sanity bound: the dimensions metadata drives a large catalogue, so a collapse to a handful // of sources means the metadata or the filter regressed rather than that the shape changed. @@ -96,42 +96,26 @@ public void QuantitiesGenerator_EmitsTheExpectedBreadthOfTypes() [TestMethod] public void Generator_WithMalformedMetadata_ReportsDiagnosticInsteadOfThrowing() { - GeneratorRunResult result = RunGenerator(new PrecisionGenerator(), "precision.json", "{ not valid json"); + GeneratorRunResult result = Harness.Run( + new PrecisionGenerator(), + new Dictionary { ["precision.json"] = "{ not valid json" }); Assert.IsEmpty(result.GeneratedSources); Assert.IsNotEmpty(result.Diagnostics, "Malformed metadata should surface as a diagnostic."); - Assert.AreEqual("CONV001", result.Diagnostics[0].Id); - } - - private static GeneratorRunResult RunGenerator(IIncrementalGenerator generator, string metadataFileName, string? metadataOverride = null) - { - string metadataPath = Path.Combine(MetadataDirectory, metadataFileName); - string metadata = metadataOverride ?? File.ReadAllText(metadataPath); - - CSharpCompilation compilation = CSharpCompilation.Create( - assemblyName: "GeneratorHost", - syntaxTrees: [], - references: [MetadataReference.CreateFromFile(typeof(object).Assembly.Location)], - options: new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); - - GeneratorDriver driver = CSharpGeneratorDriver.Create( - generators: [generator.AsSourceGenerator()], - additionalTexts: [new InMemoryAdditionalText(metadataPath, metadata)]); - - GeneratorDriverRunResult runResult = driver.RunGenerators(compilation).GetRunResult(); - - Assert.HasCount(1, runResult.Results); - return runResult.Results[0]; + Assert.AreEqual("SEM007", result.Diagnostics[0].Id); } - /// - /// Supplies metadata JSON to the generators the same way MSBuild's AdditionalFiles would. - /// - private sealed class InMemoryAdditionalText(string path, string text) : AdditionalText + [TestMethod] + public void EveryGenerator_ReusesItsOutputWhenTheMetadataHasNotChanged() { - public override string Path { get; } = path; - - public override SourceText GetText(CancellationToken cancellationToken = default) => - SourceText.From(text); + // These are IIncrementalGenerators, and nothing checked that they behave like one: a + // generator that recomputes everything on every keystroke still passes every output + // assertion, it just makes the IDE slow. + foreach ((IIncrementalGenerator generator, string _) in Generators) + { + Assert.IsTrue( + Harness.ReusesCachedOutputOnRerun(generator), + $"{generator.GetType().Name} recomputed its output for unchanged metadata."); + } } } diff --git a/docs/physics-generator.md b/docs/physics-generator.md index 1ba7e42a..e5e0103e 100644 --- a/docs/physics-generator.md +++ b/docs/physics-generator.md @@ -161,7 +161,20 @@ type. ## Validation, diagnostics, and gotchas -- Unknown dimension references in `integrals` / `derivatives` / `dotProducts` / `crossProducts` are currently dropped silently; this is tracked as a generator diagnostic improvement (issue #56). Until it lands, **diff the output** when editing metadata to catch typos. +- Unknown dimension references in `integrals` / `derivatives` / `dotProducts` / `crossProducts` report **SEM001** and the operator is dropped. +- Every diagnostic is declared in `SemanticsDiagnostics` and allocated from one `DiagnosticCatalog`, which fixes the identifier prefix and the category: + + | ID | Reports | + |----|---------| + | SEM001 | A relationship naming a dimension that does not exist. | + | SEM002 | A schema-level problem in `dimensions.json`. | + | SEM003 | A relationship whose explicit `forms` list names a form a participant does not declare. | + | SEM004 | An `availableUnits` entry naming a unit `units.json` does not declare. Reported at the position in `dimensions.json` where the name is written. | + | SEM005 | A schema-level problem in `logarithmic.json`. | + | SEM006 | A metadata file a generator declared that was not supplied as an `AdditionalFile`. | + | SEM007 | A metadata file that could not be parsed. | + + Adding one means adding it to `SemanticsDiagnostics` and to `AnalyzerReleases.Unshipped.md`; `AnalyzerReleaseTrackingTests` fails if the second step is forgotten. `GeneratorDiagnosticTests` proves each one still fires on the input it is meant to catch. - `availableUnits` order matters: the first entry is treated as the SI base unit by `UnitsGenerator`. - `relationships` expressions are emitted verbatim into method bodies. Use `Value` for the current quantity and `T.CreateChecked(...)` (not literal numerics) for constants so all storage types stay correct. - Generator output is committed. CI must catch metadata/code drift; `git status` should be clean after a build.