diff --git a/docs/user/reference/config/overlays.md b/docs/user/reference/config/overlays.md index 8db2ea77..04863f38 100644 --- a/docs/user/reference/config/overlays.md +++ b/docs/user/reference/config/overlays.md @@ -163,7 +163,9 @@ When a single logical change (a CVE backport, a feature disablement, a Fedora ch Set `overlay-files` on the component to one or more globs (relative to the component config) and drop one overlay file per logical change into a directory of your choosing. The conventional layout uses a sibling `overlays/` directory and a `*.overlay.toml` filename suffix, but neither is required — `overlay-files` is just a glob, so any layout you can describe with `**`/`*` patterns works. -`overlay-files` can also be inherited from `default-component-config` at the project, distro, or component-group level. Inherited relative patterns are still resolved for each concrete component: from its component config file when it has one, or from the matched spec file's directory when it is discovered by a component group's `specs` pattern. This makes defaults useful for component-local discovery patterns such as `overlay-files = ["overlays/*.overlay.toml"]`. If a component sets `overlay-files`, that value replaces the inherited list; use `overlay-files = []` to disable inherited overlay files for a component, or include both patterns explicitly when you want to keep default discovery and add component-specific locations. +`overlay-files` on `[components.X]` is a plain glob resolved relative to the component's config file. `overlay-files` can also be set on the project-level `default-component-config`, where **every entry must contain the `{component}` placeholder** (see [Discovering components by pattern](#discovering-components-by-pattern-component-in-overlay-files) below). A component that doesn't declare its own `overlay-files` inherits the project pattern with `{component}` substituted for its own name; use `overlay-files = []` at component scope to disable the inherited list, or set your own list to replace it wholesale. + +`overlay-files` is only settable at project scope (the top-level `default-component-config`) or at component scope (`[components.X]`). Setting it inside a distro-version or component-group `default-component-config` is rejected at load time — even an empty list (`overlay-files = []`). ``` base/comps/mypackage/ @@ -210,6 +212,60 @@ section = "%changelog" lines = ["- Fix CVE-2024-1234"] ``` +### Discovering components by pattern (`{component}` in `overlay-files`) + +For projects that keep each component's overlay files in a predictable per-component directory (e.g. `base/comps//overlays/*.overlay.toml`), you can skip declaring a one-line `[components.]` table for every component. Set `overlay-files` on the **project-level** `default-component-config` to a pattern containing `{component}`, and every match becomes an implicit component. + +Project-scope `overlay-files` entries **must** contain `{component}` exactly once, as a whole path segment, and **must** have at least one more path segment after it (the placeholder names a component directory, not a file). Additional constraints enforced at load time: + +* Use `/` as the path separator (backslashes are rejected, even on Windows). +* No glob metacharacters (`*`, `?`, `[`) before `{component}` — the prefix is a literal used to capture the component name. +* The rest of the entry (after `{component}/`) must be a valid `doublestar` glob (validated with `doublestar.ValidatePattern`). +* The captured segment must be a safe component name (no spaces, no path traversal, no absolute paths); matches with unsafe names are skipped with a `slog.Debug` entry. + +For each entry, `{component}` is expanded to `*` for globbing, and the captured path segment becomes the discovered component's name. The matched files are attached to that component (as if it had declared `overlay-files = []` explicitly). Explicit components with no `overlay-files` of their own inherit the same pattern and re-expand it with their own name substituted for `{component}`. + +```toml +# components.toml (project-level) +[default-component-config] +overlay-files = ["base/comps/{component}/overlays/*.overlay.toml"] +``` + +With the layout below, no per-component `[components.]` table is needed — `openssl` and `curl` are discovered automatically, each carrying its overlay files. + +``` +base/comps/ +├── openssl/ +│ └── overlays/ +│ ├── 0001-cve-2024-1234.overlay.toml +│ └── 0002-disable-fips-tests.overlay.toml +└── curl/ + └── overlays/ + └── 0001-cve-2024-5678.overlay.toml +``` + +**Where `{component}` is allowed.** Only in **project-level** `default-component-config`. At every project-level `overlay-files` entry the placeholder is *required* — plain globs at that scope are rejected because they would apply the same files to every component in the project, which is almost never what you want. At component scope, `overlay-files` entries are plain globs relative to the declaring config file; `{component}` is rejected there. `overlay-files` cannot be set at all in distro-version or component-group `default-component-config` — those broad-scope defaults would silently displace project-level discovery. + +**Inheritance with `{component}`.** A project-level `overlay-files` entry containing `{component}` serves two purposes at once: + +1. **Discovery.** Every captured directory becomes an implicit component unless there is already an explicit `[components.]` table with the same name. +2. **Inheritance.** An explicitly-declared component that does not override `overlay-files` inherits the project entry and expands `{component}` with its own name at glob time. + +A component that sets its own `overlay-files = [...]` replaces the inherited value; the placeholder pattern no longer applies to that component. Setting `overlay-files = []` disables inheritance entirely for that component. + +**Same-scope collisions are a hard error.** If two project-scope `overlay-files` entries both produce the same component name, the resolver fails with a diagnostic listing both entries. Restrict one of the entries or rename one of the directories. + +**Shadowing.** An explicit `[components.]` declaration supersedes pattern discovery for the same name. If that declaration also sets a non-empty `overlay-files` list (which replaces inheritance with a different set of files), `azldev` emits an `slog.Warn` naming the shadowed pattern so you can spot the genuine displacement. An explicit declaration that leaves `overlay-files` unset still inherits the project pattern, so the discovered files apply and no warning is emitted. An explicit empty list (`overlay-files = []`) is the documented off-switch for a component and is not treated as shadowing. + +**Limitation: explicit component-group members need a `[components.]` table.** A component group can enumerate its members two ways: via a `specs` glob (spec files on disk) or via an explicit `components = ["", ...]` list. Only the *explicit list* interacts with pattern discovery: every name in that list must have a matching `[components.]` table in the project (validated at load time, error `ErrUndefinedComponent`). Even a one-liner is enough: + +```toml +# base/comps/openssl/openssl.comp.toml +[components.openssl] +``` + +Once the table exists, the component inherits the project-level `{component}` overlay pattern normally. Group members discovered via `specs` (not the explicit list) do **not** need a `.comp.toml` — the resolver synthesizes an empty config for them and inheritance still applies. + ## Examples ### Adding a Build Dependency diff --git a/internal/app/azldev/core/components/pattern_discovery.go b/internal/app/azldev/core/components/pattern_discovery.go new file mode 100644 index 00000000..b94b0bde --- /dev/null +++ b/internal/app/azldev/core/components/pattern_discovery.go @@ -0,0 +1,217 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +package components + +import ( + "errors" + "fmt" + "log/slog" + "path" + "path/filepath" + "slices" + "strings" + + "github.com/bmatcuk/doublestar/v4" + "github.com/microsoft/azure-linux-dev-tools/internal/app/azldev" + "github.com/microsoft/azure-linux-dev-tools/internal/projectconfig" + "github.com/microsoft/azure-linux-dev-tools/internal/utils/fileutils" +) + +// patternDiscoveredComponent captures the result of expanding a single +// project-level [projectconfig.ComponentConfig.OverlayFiles] entry that +// contains the `{component}` placeholder. +type patternDiscoveredComponent struct { + // Name of the discovered component (the captured {component} segment). + Name string + // ReferenceDir is the absolute directory the pattern captured for this + // component (i.e. the prefix directory + captured name). Used as the + // component's reference directory for downstream operations. + ReferenceDir string + // OverlayFiles is the deduplicated, sorted list of absolute overlay file + // paths matched by the winning pattern for this component. + OverlayFiles []string + // SourcePattern is the exact entry string that produced this match, used + // in shadow-lint diagnostics. + SourcePattern string +} + +// discoverComponentsFromPatterns walks the project-level +// default-component-config, expands every 'overlay-files' entry that contains +// the `{component}` placeholder, and returns the resulting pattern-discovered +// components. Two entries at project scope producing the same component name +// is a hard error. +// +// The caller is responsible for further precedence against explicitly-declared +// components (that override happens at the resolver layer; see +// [Resolver.patternDiscoveredComponents]). +func discoverComponentsFromPatterns(env *azldev.Env) (map[string]*patternDiscoveredComponent, error) { + result := make(map[string]*patternDiscoveredComponent) + + config := env.Config() + if config == nil { + return result, nil + } + + var patterns []string + + for _, entry := range config.DefaultComponentConfig.OverlayFiles { + if strings.Contains(entry, projectconfig.OverlayFilesComponentPlaceholder) { + patterns = append(patterns, entry) + } + } + + if len(patterns) == 0 { + return result, nil + } + + for _, pattern := range patterns { + byName, err := expandSinglePattern(env, pattern) + if err != nil { + return nil, fmt.Errorf("project 'overlay-files': %w", err) + } + + for name, match := range byName { + if existing, ok := result[name]; ok { + return nil, fmt.Errorf( + "%w: component %#q is discovered by both %#q and %#q; "+ + "restrict one of the patterns or rename one of the directories", + ErrPatternDiscoveryCollision, name, existing.SourcePattern, match.SourcePattern, + ) + } + + result[name] = match + } + } + + return result, nil +} + +// expandSinglePattern expands a single validated pattern and groups matched +// files by captured component name. +func expandSinglePattern(env *azldev.Env, pattern string) (map[string]*patternDiscoveredComponent, error) { + prefix, suffix := projectconfig.SplitOverlayFilesPlaceholder(pattern) + + // Substitute a single-segment wildcard for {component} to build the actual + // glob. Because {component} is validated as a whole path segment (see + // ConfigFile.Validate), replacing it with "*" produces a well-formed + // doublestar pattern that captures exactly one path segment. + globPattern := prefix + "*" + suffix + + // doublestar.WithFailOnIOErrors() is intentionally not set, mirroring + // findComponentGroupSpecPaths: we may legitimately hit unreadable subtrees + // under the project root. Non-IO errors (bad pattern, etc.) are still + // returned. + matches, err := fileutils.Glob(env.FS(), globPattern, doublestar.WithFilesOnly()) + if err != nil { + return nil, fmt.Errorf("failed to expand pattern %#q:\n%w", pattern, err) + } + + byName := make(map[string]*patternDiscoveredComponent) + + for _, match := range matches { + name, referenceDir, err := extractCapturedName(pattern, prefix, match) + if err != nil { + // Skip matches we can't parse (should not happen with a validated + // pattern, but guard anyway). + slog.Debug("pattern-discovery: skipping unparseable match", + "pattern", pattern, "match", match, "err", err) + + continue + } + + // Reject matches whose captured segment is not a safe component name + // (spaces, path traversal, absolute paths, etc). Downstream code uses + // the name as a filesystem path fragment (see fileutils.ValidateFilename) + // so we fail fast rather than let it explode later. + if err := fileutils.ValidateFilename(name); err != nil { + slog.Debug("pattern-discovery: skipping match with unsafe captured name", + "pattern", pattern, "match", match, "name", name, "err", err) + + continue + } + + bucket, ok := byName[name] + if !ok { + bucket = &patternDiscoveredComponent{ + Name: name, + ReferenceDir: referenceDir, + SourcePattern: pattern, + } + byName[name] = bucket + } + + bucket.OverlayFiles = append(bucket.OverlayFiles, match) + } + + // Sort the overlay-files list per component (filename first, then full path). + for _, bucket := range byName { + slices.SortFunc(bucket.OverlayFiles, func(left, right string) int { + if result := strings.Compare(filepath.Base(left), filepath.Base(right)); result != 0 { + return result + } + + return strings.Compare(left, right) + }) + } + + return byName, nil +} + +// extractCapturedName strips prefix from match and returns the first path +// segment (the captured component name) together with the concrete directory +// that captured segment lives in. The returned ReferenceDir is the natural +// per-component anchor for downstream operations. +func extractCapturedName(pattern, prefix, match string) (name, referenceDir string, err error) { + rel := match + if prefix != "" { + if !strings.HasPrefix(match, prefix) { + return "", "", fmt.Errorf( + "match %#q does not start with expected prefix %#q for pattern %#q", + match, prefix, pattern, + ) + } + + rel = match[len(prefix):] + } + + // The captured segment is everything up to (but not including) the first + // path separator in the residual, or the entire residual if there is no + // suffix. Patterns are validated to use '/' only. + sep := strings.IndexByte(rel, '/') + if sep < 0 { + name = rel + } else { + name = rel[:sep] + } + + if name == "" { + return "", "", fmt.Errorf( + "empty captured component name for match %#q against pattern %#q", + match, pattern, + ) + } + + // The reference dir is the prefix + captured segment (no trailing separator). + // Use path.Clean (POSIX '/') rather than filepath.Clean so ReferenceDir stays + // consistent with the placeholder contract, which validates and operates on + // POSIX-style separators only (filepath.Clean would emit '\' on Windows). + referenceDir = path.Clean(prefix + name) + + return name, referenceDir, nil +} + +// logShadowedByDeclaration emits a warning when an explicit component +// declaration overrides pattern-discovered overlay files for the same name. +func logShadowedByDeclaration(shadowed *patternDiscoveredComponent) { + slog.Warn( + "project 'overlay-files' pattern match shadowed by explicit component declaration; overlay files will not be applied", + "component", shadowed.Name, + "shadowed_pattern", shadowed.SourcePattern, + "shadowed_files", shadowed.OverlayFiles, + ) +} + +// ErrPatternDiscoveryCollision wraps project-scope pattern collision errors so +// tests can assert on them. +var ErrPatternDiscoveryCollision = errors.New("project 'overlay-files' pattern collision") diff --git a/internal/app/azldev/core/components/pattern_discovery_test.go b/internal/app/azldev/core/components/pattern_discovery_test.go new file mode 100644 index 00000000..5b18d37e --- /dev/null +++ b/internal/app/azldev/core/components/pattern_discovery_test.go @@ -0,0 +1,191 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +package components_test + +import ( + "testing" + + "github.com/microsoft/azure-linux-dev-tools/internal/app/azldev/core/components" + "github.com/microsoft/azure-linux-dev-tools/internal/app/azldev/core/testutils" + "github.com/microsoft/azure-linux-dev-tools/internal/projectconfig" + "github.com/microsoft/azure-linux-dev-tools/internal/utils/fileperms" + "github.com/microsoft/azure-linux-dev-tools/internal/utils/fileutils" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +const patternTestOverlayContent = ` +[metadata] +category = "azl-branding-policy" + +[[overlays]] +description = "Bump release" +type = "spec-search-replace" +section = "%install" +regex = "unused" +` + +// writeOverlayFile writes a minimal but valid overlay TOML at path. +func writeOverlayFile(t *testing.T, env *testutils.TestEnv, path string) { + t.Helper() + + require.NoError(t, fileutils.MkdirAll(env.TestFS, dirOf(path))) + require.NoError(t, fileutils.WriteFile( + env.TestFS, path, []byte(patternTestOverlayContent), fileperms.PrivateFile, + )) +} + +func dirOf(path string) string { + for i := len(path) - 1; i >= 0; i-- { + if path[i] == '/' { + return path[:i] + } + } + + return "." +} + +func TestFindAllComponents_PatternDiscoveryAtProjectScope(t *testing.T) { + env := testutils.NewTestEnv(t) + + env.Config.DefaultComponentConfig.OverlayFiles = []string{ + "/project/comps/{component}/overlays/*.overlay.toml", + } + + writeOverlayFile(t, env, "/project/comps/foo/overlays/0001.overlay.toml") + writeOverlayFile(t, env, "/project/comps/foo/overlays/0002.overlay.toml") + writeOverlayFile(t, env, "/project/comps/bar/overlays/0001.overlay.toml") + + all, err := components.NewResolver(env.Env).FindAllComponents() + require.NoError(t, err) + + assert.ElementsMatch(t, []string{"foo", "bar"}, all.Names()) + + foo, ok := all.TryGet("foo") + require.True(t, ok) + assert.Len(t, foo.GetConfig().Overlays, 2, "both foo overlay files should attach") + + bar, ok := all.TryGet("bar") + require.True(t, ok) + assert.Len(t, bar.GetConfig().Overlays, 1) +} + +// Declared components with no 'overlay-files' inherit the project pattern; the +// placeholder is substituted with the component's own name at glob time. +func TestFindAllComponents_DeclaredComponentInheritsProjectPattern(t *testing.T) { + env := testutils.NewTestEnv(t) + + env.Config.DefaultComponentConfig.OverlayFiles = []string{ + "/project/comps/{component}/overlays/*.overlay.toml", + } + + writeOverlayFile(t, env, "/project/comps/foo/overlays/0001.overlay.toml") + writeOverlayFile(t, env, "/project/comps/foo/overlays/0002.overlay.toml") + + env.Config.Components["foo"] = projectconfig.ComponentConfig{Name: "foo"} + + all, err := components.NewResolver(env.Env).FindAllComponents() + require.NoError(t, err) + + foo, ok := all.TryGet("foo") + require.True(t, ok) + assert.Len(t, foo.GetConfig().Overlays, 2, + "foo should inherit the project pattern with {component}=foo") +} + +// A component that sets its own 'overlay-files' replaces the inherited project +// pattern wholesale; the placeholder pattern no longer applies. +func TestFindAllComponents_ComponentOverlayFilesReplacesInherited(t *testing.T) { + env := testutils.NewTestEnv(t) + + env.Config.DefaultComponentConfig.OverlayFiles = []string{ + "/project/comps/{component}/overlays/*.overlay.toml", + } + + // This file would match the inherited pattern (component=bar). + writeOverlayFile(t, env, "/project/comps/bar/overlays/0001.overlay.toml") + + // But this file lives elsewhere and is the only one bar's config points to. + writeOverlayFile(t, env, "/project/custom/bar/0001.overlay.toml") + + env.Config.Components["bar"] = projectconfig.ComponentConfig{ + Name: "bar", + OverlayFiles: []string{"/project/custom/bar/*.overlay.toml"}, + } + + all, err := components.NewResolver(env.Env).FindAllComponents() + require.NoError(t, err) + + bar, ok := all.TryGet("bar") + require.True(t, ok) + assert.Len(t, bar.GetConfig().Overlays, 1, + "bar overrides overlay-files wholesale; inherited pattern must not apply") +} + +// Empty per-component 'overlay-files' disables the inherited project pattern. +func TestFindAllComponents_ComponentOverlayFilesEmptyDisablesInheritance(t *testing.T) { + env := testutils.NewTestEnv(t) + + env.Config.DefaultComponentConfig.OverlayFiles = []string{ + "/project/comps/{component}/overlays/*.overlay.toml", + } + + writeOverlayFile(t, env, "/project/comps/baz/overlays/0001.overlay.toml") + + env.Config.Components["baz"] = projectconfig.ComponentConfig{ + Name: "baz", + OverlayFiles: []string{}, + } + + all, err := components.NewResolver(env.Env).FindAllComponents() + require.NoError(t, err) + + baz, ok := all.TryGet("baz") + require.True(t, ok) + assert.Empty(t, baz.GetConfig().Overlays, + "empty overlay-files must disable inherited pattern") +} + +func TestFindAllComponents_PatternDiscoverySameScopeCollisionErrors(t *testing.T) { + env := testutils.NewTestEnv(t) + + env.Config.DefaultComponentConfig.OverlayFiles = []string{ + "/project/a/{component}/*.overlay.toml", + "/project/b/{component}/*.overlay.toml", + } + + writeOverlayFile(t, env, "/project/a/dup/0001.overlay.toml") + writeOverlayFile(t, env, "/project/b/dup/0002.overlay.toml") + + _, err := components.NewResolver(env.Env).FindAllComponents() + require.Error(t, err) + require.ErrorIs(t, err, components.ErrPatternDiscoveryCollision, + "expected ErrPatternDiscoveryCollision, got %v", err) + assert.Contains(t, err.Error(), "dup") +} + +// A pattern match whose captured segment isn't a safe component name +// (spaces, dot-directory, etc.) is skipped without failing overall discovery. +func TestFindAllComponents_PatternDiscoverySkipsUnsafeCapturedNames(t *testing.T) { + env := testutils.NewTestEnv(t) + + env.Config.DefaultComponentConfig.OverlayFiles = []string{ + "/project/comps/{component}/overlays/*.overlay.toml", + } + + // Well-formed match — should be discovered. + writeOverlayFile(t, env, "/project/comps/openssl/overlays/0001.overlay.toml") + + // Unsafe captured segments — should be skipped without erroring the whole + // discovery. These simulate directories a user might create that don't + // correspond to valid component names. + writeOverlayFile(t, env, "/project/comps/has space/overlays/0001.overlay.toml") + writeOverlayFile(t, env, "/project/comps/./overlays/0001.overlay.toml") + + all, err := components.NewResolver(env.Env).FindAllComponents() + require.NoError(t, err) + + assert.ElementsMatch(t, []string{"openssl"}, all.Names(), + "only the safely-named directory should be discovered") +} diff --git a/internal/app/azldev/core/components/resolver.go b/internal/app/azldev/core/components/resolver.go index fad536a4..0a891a83 100644 --- a/internal/app/azldev/core/components/resolver.go +++ b/internal/app/azldev/core/components/resolver.go @@ -7,10 +7,12 @@ import ( "errors" "fmt" "log/slog" + "maps" "path" "path/filepath" "slices" "strings" + "sync" "github.com/bmatcuk/doublestar/v4" "github.com/microsoft/azure-linux-dev-tools/internal/app/azldev" @@ -42,13 +44,22 @@ type Resolver struct { // Only 'component update' enables this — it uses the freshness status to // decide which components need re-resolution and which can be skipped. CheckFreshness bool + + // patternDiscovery lazily holds the result of pattern-based component + // discovery so repeated resolver calls don't re-glob the filesystem. + patternDiscovery func() (map[string]*patternDiscoveredComponent, error) } // NewResolver constructs a new [Resolver] for the given environment. func NewResolver(env *azldev.Env) *Resolver { - return &Resolver{ + resolver := &Resolver{ env: env, } + resolver.patternDiscovery = sync.OnceValues(func() (map[string]*patternDiscoveredComponent, error) { + return discoverComponentsFromPatterns(env) + }) + + return resolver } // Given a component filter, finds all components defined in the environment that match the filter. @@ -109,7 +120,14 @@ func (r *Resolver) FindComponents(filter *ComponentFilter) (components *Componen return components, r.validateLockFiles(components, false, skipValidation) } -// Finds *all* components defined in the environment. +// Finds *all* components defined in the environment. This unions three sources: +// - components reached via component-group spec-path patterns, +// - loose components declared in the project config, and +// - components discovered by a project-level 'overlay-files' entry containing +// the '{component}' placeholder. +// +// Explicit declarations shadow pattern-discovered matches of the same name; a +// warning is emitted for the shadowed entry. func (r *Resolver) FindAllComponents() (components *ComponentSet, err error) { components = NewComponentSet() @@ -167,6 +185,49 @@ func (r *Resolver) FindAllComponents() (components *ComponentSet, err error) { components.Add(comp) } + // Finally, add pattern-discovered components (from project-level 'overlay-files' + // entries containing '{component}') that were not already covered by an explicit + // component declaration. When an explicit declaration exists but doesn't set its + // own overlay-files, it inherits the project pattern and still receives the + // discovered files (no warning). A warning is only emitted when the explicit + // declaration replaces overlay-files with a non-empty list, genuinely displacing + // the discovered files. An explicit empty list ('overlay-files = []') is the + // documented off-switch and is not treated as shadowing. + discovered, err := r.patternDiscoveredComponents() + if err != nil { + return components, err + } + + for _, name := range slices.Sorted(maps.Keys(discovered)) { + entry := discovered[name] + + if components.Contains(name) { + // Only warn when the explicit declaration provides a non-empty + // 'overlay-files' list, which replaces inheritance with a different + // set of files (see [projectconfig.ComponentConfig.MergeUpdatesFrom]). + // A declaration with no 'overlay-files' inherits the pattern and the + // discovered files still apply. An explicit empty list is the + // documented way to opt out of overlays for a component and is not + // treated as shadowing. + // + // 'overlay-files' can only be set at project or component scope + // (validated by [projectconfig.ConfigFile.validateOverlayFilesByScope]), + // so a shadowing declaration always lives in Config().Components. + if declared, ok := r.env.Config().Components[name]; ok && len(declared.OverlayFiles) > 0 { + logShadowedByDeclaration(entry) + } + + continue + } + + comp, addErr := r.componentFromPatternDiscovered(entry, nil) + if addErr != nil { + return components, addErr + } + + components.Add(comp) + } + return components, nil } @@ -274,6 +335,10 @@ func (r *Resolver) GetComponentGroupByName(componentGroupName string) (component componentGroup.Components = append(componentGroup.Components, groupEntry) } + // Pattern-discovered components (from a project-level 'overlay-files' entry + // containing '{component}') do not belong to any specific component group; + // they are added to the project-wide component set in FindAllComponents. + return componentGroup, nil } @@ -892,6 +957,35 @@ func componentGroupNames(env *azldev.Env, componentName string, extraGroupNames return groupNames } +// patternDiscoveredComponents returns the (lazily-computed) map of components +// discovered by a project-level 'overlay-files' entry containing the +// '{component}' placeholder. Returns an empty (non-nil) map and nil error when +// the project default-component-config declares no such entry. +func (r *Resolver) patternDiscoveredComponents() (map[string]*patternDiscoveredComponent, error) { + return r.patternDiscovery() +} + +// componentFromPatternDiscovered synthesizes a [Component] for a pattern +// discovered entry, applying inherited defaults with the entry's captured +// directory as the overlay-files reference dir. +func (r *Resolver) componentFromPatternDiscovered( + entry *patternDiscoveredComponent, groupNames []string, +) (Component, error) { + synth := projectconfig.ComponentConfig{ + Name: entry.Name, + OverlayFiles: slices.Clone(entry.OverlayFiles), + } + + updated, err := applyInheritedDefaultsToComponent(r.env, synth, entry.ReferenceDir, groupNames) + if err != nil { + return nil, fmt.Errorf( + "applying defaults to pattern-discovered component %#q:\n%w", entry.Name, err, + ) + } + + return r.createComponentFromConfig(updated) +} + // validateLockFiles checks lock file consistency against the resolved component // set. Skipped when skipValidation is true (set per-filter or via the global // '--skip-lock-validation' flag). diff --git a/internal/projectconfig/component.go b/internal/projectconfig/component.go index 88d18e47..79ba564e 100644 --- a/internal/projectconfig/component.go +++ b/internal/projectconfig/component.go @@ -312,20 +312,42 @@ type ComponentConfig struct { // Overlays to apply to sources after they've been acquired. May mutate the spec as well as sources. Overlays []ComponentOverlay `toml:"overlays,omitempty" json:"overlays,omitempty" table:"-" jsonschema:"title=Overlays,description=Overlays to apply to this component's spec and/or sources"` - // OverlayFiles, if set, lists path or glob patterns (relative to this component config file) - // matched against the filesystem after component config resolution to locate per-file overlay documents. - // Each matched file is parsed as an [OverlayFile]: one logical change consisting of a - // file-level `[metadata]` block plus an ordered list of `[[overlays]]`. The per-file - // metadata is stamped onto every overlay in the file. Matches are concatenated in the - // order patterns are declared; within a single pattern, matches are applied in - // filename (lexicographic) order, using the full path as a tie-breaker when - // filenames match. Duplicate matches are de-duplicated, preserving first - // occurrence. The resulting overlays are appended to [ComponentConfig.Overlays] - // after any inline overlays. A value set in a higher-priority config layer replaces - // lower-priority overlay-files values; an explicit empty list disables inherited - // overlay files. Excluded from the fingerprint because the value affects only where - // overlays are sourced from, not their content. - OverlayFiles []string `toml:"overlay-files,omitempty" json:"overlayFiles,omitempty" table:"-" validate:"dive,required" jsonschema:"title=Overlay files,description=Path or glob patterns (relative to the component config file or matched spec directory) matched against the filesystem to locate per-file overlay documents after component config resolution. Use an empty list to disable inherited overlay-file patterns" fingerprint:"-"` + // OverlayFiles lists path or glob patterns matched against the filesystem after + // component config resolution to locate per-file overlay documents. Each matched + // file is parsed as an [OverlayFile]: one logical change consisting of a file-level + // `[metadata]` block plus an ordered list of `[[overlays]]`. The per-file metadata + // is stamped onto every overlay in the file. Matches are concatenated in the order + // entries are declared; within a single entry, matches are applied in filename + // (lexicographic) order, using the full path as a tie-breaker when filenames match. + // Duplicate matches are de-duplicated, preserving first occurrence. The resulting + // overlays are appended to [ComponentConfig.Overlays] after any inline overlays. + // + // A value set in a higher-priority config layer replaces lower-priority values + // wholesale; an explicit empty list disables inherited overlay files. + // + // `overlay-files` may only be set at project scope (the top-level + // `default-component-config`) or at component scope (`[components.X]`). + // Distro-version and component-group `default-component-config` tables must + // not set this field: broad-scope defaults for overlay files silently + // displace project-level discovery and per-component overrides, so we + // require callers to place the list at project or component scope where the + // intent is unambiguous. + // + // At project scope, every entry MUST contain the `{component}` placeholder exactly + // once as a whole path segment: those entries drive component discovery (each + // captured segment becomes an implicit component) and, when inherited by an explicit + // component that does not override, `{component}` is substituted with that + // component's name at glob time. Two project-scope entries that produce the same + // component name is a hard error. + // + // At component scope, entries are plain globs interpreted relative to the + // declaring config file (or the matched spec directory); `{component}` is + // not allowed. Setting an empty list on a component (`overlay-files = []`) + // disables inheritance from the project default. + // + // Excluded from the fingerprint because the value affects only where overlays are + // sourced from, not their content. + OverlayFiles []string `toml:"overlay-files,omitempty" json:"overlayFiles,omitempty" table:"-" validate:"dive,required" jsonschema:"title=Overlay files,description=Path or glob patterns matched against the filesystem to locate per-file overlay documents. Only settable at project scope (top-level 'default-component-config') or at component scope ('[components.X]'). At project scope every entry must contain the {component} placeholder as a whole path segment (drives discovery and per-component substitution). At component scope {component} is not allowed and entries are plain globs relative to the config file. Distro-version and component-group 'default-component-config' tables must not set this field. Use an empty list at component scope to disable inheritance." fingerprint:"-"` // Configuration for building the component. Build ComponentBuildConfig `toml:"build,omitempty" json:"build,omitempty" table:"-" jsonschema:"title=Build configuration,description=Configuration for building the component"` @@ -443,10 +465,15 @@ func (c *ComponentConfig) WithAbsolutePaths(referenceDir string) *ComponentConfi SourceFiles: deep.MustCopy(c.SourceFiles), Packages: deep.MustCopy(c.Packages), Publish: deep.MustCopy(c.Publish), - // OverlayFiles is consumed after component config resolution; preserve it verbatim - // here so inherited patterns can be interpreted relative to the concrete component - // config file. - OverlayFiles: slices.Clone(c.OverlayFiles), + // OverlayFiles: entries with the {component} placeholder are project-scope + // discovery patterns and are absolutized now so downstream code sees stable + // absolute paths regardless of where the config was inherited from. Non-placeholder + // entries are left verbatim so component-scope patterns are still interpreted + // relative to the concrete component config file at glob time. (Project-scope + // entries must all contain the placeholder; distro-version and component-group + // scopes may not set overlay-files at all — see + // [ConfigFile.validateOverlayFilesByScope].) + OverlayFiles: absolutizeOverlayFilesPlaceholderEntries(referenceDir, c.OverlayFiles), } // Fix up paths. diff --git a/internal/projectconfig/configfile.go b/internal/projectconfig/configfile.go index b1975b3b..36918129 100644 --- a/internal/projectconfig/configfile.go +++ b/internal/projectconfig/configfile.go @@ -97,6 +97,16 @@ func (f ConfigFile) Validate() error { } } + // Validate 'overlay-files' scope and placeholder usage. 'overlay-files' is + // only settable at project scope (the top-level 'default-component-config', + // where every entry must contain the '{component}' placeholder as a whole + // path segment) and at component scope ('[components.X]', where entries are + // plain globs and '{component}' is forbidden). Distro and component-group + // 'default-component-config' entries may not set 'overlay-files' at all. + if err := f.validateOverlayFilesByScope(); err != nil { + return err + } + // Per-component snapshot timestamps are not allowed. Components inherit // the snapshot from the distro/group default-component-config or the // project's default-distro. Per-component snapshots would create @@ -105,6 +115,12 @@ func (f ConfigFile) Validate() error { // Validate overlay configurations for each component. for componentName, component := range f.Components { + if err := validateOverlayFilesEntries( + component.OverlayFiles, validateComponentOverlayFilesEntry, + ); err != nil { + return fmt.Errorf("invalid 'overlay-files' for component %#q:\n%w", componentName, err) + } + for i, overlay := range component.Overlays { err := overlay.Validate() if err != nil { @@ -150,6 +166,96 @@ func (f ConfigFile) Validate() error { return nil } +// validateOverlayFilesByScope enforces where 'overlay-files' may appear. +// +// - Project scope (top-level 'default-component-config'): each entry must +// contain the '{component}' placeholder as a whole path segment; the +// placeholder drives pattern-based component discovery. +// - Component scope ('[components.X]'): plain globs; the '{component}' +// placeholder is forbidden (the name is already fixed). +// - Distro-version and component-group 'default-component-config': +// 'overlay-files' is not allowed at all. Broad-scope defaults for this +// field silently displace project-level discovery and per-component +// overrides, so we require callers to place the list at project or +// component scope where the intent is unambiguous. +func (f ConfigFile) validateOverlayFilesByScope() error { + if f.DefaultComponentConfig != nil { + if err := validateOverlayFilesEntries( + f.DefaultComponentConfig.OverlayFiles, validateProjectOverlayFilesEntry, + ); err != nil { + return fmt.Errorf("invalid project 'default-component-config':\n%w", err) + } + } + + for distroName, distro := range f.Distros { + for versionName, version := range distro.Versions { + if version.DefaultComponentConfig.OverlayFiles != nil { + return fmt.Errorf( + "invalid 'default-component-config' for distro %#q version %#q:\n"+ + "%w: 'overlay-files' is only allowed on the project-level "+ + "'default-component-config' (with '%s' patterns) or on individual "+ + "'[components.X]' entries", + distroName, versionName, + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, + ) + } + } + } + + for groupName, group := range f.ComponentGroups { + if group.DefaultComponentConfig.OverlayFiles != nil { + return fmt.Errorf( + "invalid 'default-component-config' for component group %#q:\n"+ + "%w: 'overlay-files' is only allowed on the project-level "+ + "'default-component-config' (with '%s' patterns) or on individual "+ + "'[components.X]' entries", + groupName, + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, + ) + } + } + + return nil +} + +// overlayFilesValidator validates a single 'overlay-files' entry against +// the placeholder rules of a particular config scope. +type overlayFilesValidator func(entry string) error + +// validateProjectOverlayFilesEntry validates entries in the project-level +// default-component-config. Every entry MUST contain the '{component}' +// placeholder exactly once as a whole path segment — it's the discovery +// mechanism. +func validateProjectOverlayFilesEntry(entry string) error { + return validateOverlayFilesPlaceholder(entry) +} + +// validateComponentOverlayFilesEntry validates entries in a '[components.X]' +// table. Entries are plain globs; '{component}' is forbidden because the +// component name is already fixed by the table key. +func validateComponentOverlayFilesEntry(entry string) error { + if !hasOverlayFilesPlaceholder(entry) { + return nil + } + + return fmt.Errorf( + "%w: %q is only allowed in project-level 'default-component-config' entries; entry %#q", + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, entry, + ) +} + +// validateOverlayFilesEntries runs validate against each entry in overlayFiles, +// annotating the returned error with the offending index. +func validateOverlayFilesEntries(overlayFiles []string, validate overlayFilesValidator) error { + for idx, entry := range overlayFiles { + if err := validate(entry); err != nil { + return fmt.Errorf("'overlay-files'[%d]: %w", idx, err) + } + } + + return nil +} + // validateSourceFiles checks 'source-files' configuration for a component: // - All filenames must be unique. // - Hash type must be a supported algorithm when specified. diff --git a/internal/projectconfig/configfile_test.go b/internal/projectconfig/configfile_test.go index b6d8f56f..17753244 100644 --- a/internal/projectconfig/configfile_test.go +++ b/internal/projectconfig/configfile_test.go @@ -584,3 +584,123 @@ func TestValidateCustomSourceRef_InvalidScriptFilename(t *testing.T) { require.Error(t, err) assert.Contains(t, err.Error(), "script") } + +func TestProjectConfigFileValidation_OverlayFilesPlaceholderOnProjectDefault(t *testing.T) { + file := projectconfig.ConfigFile{ + DefaultComponentConfig: &projectconfig.ComponentConfig{ + OverlayFiles: []string{"comps/{component}/overlays/*.overlay.toml"}, + }, + } + assert.NoError(t, file.Validate()) +} + +func TestProjectConfigFileValidation_OverlayFilesOnDistroDefaultRejected(t *testing.T) { + file := projectconfig.ConfigFile{ + Distros: map[string]projectconfig.DistroDefinition{ + "my-distro": { + Versions: map[string]projectconfig.DistroVersionDefinition{ + "1.0": { + ReleaseVer: "1.0", + DefaultComponentConfig: projectconfig.ComponentConfig{ + OverlayFiles: []string{"distro/*.overlay.toml"}, + }, + }, + }, + }, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "my-distro") + assert.Contains(t, err.Error(), "overlay-files") +} + +func TestProjectConfigFileValidation_OverlayFilesOnGroupDefaultRejected(t *testing.T) { + file := projectconfig.ConfigFile{ + ComponentGroups: map[string]projectconfig.ComponentGroupConfig{ + "core": { + DefaultComponentConfig: projectconfig.ComponentConfig{ + OverlayFiles: []string{"core/*.overlay.toml"}, + }, + }, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "core") + assert.Contains(t, err.Error(), "overlay-files") +} + +func TestProjectConfigFileValidation_OverlayFilesPlaceholderOnComponentRejected(t *testing.T) { + file := projectconfig.ConfigFile{ + Components: map[string]projectconfig.ComponentConfig{ + "my-comp": { + OverlayFiles: []string{"comps/{component}/overlays/*.overlay.toml"}, + }, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "my-comp") + assert.Contains(t, err.Error(), "{component}") +} + +func TestProjectConfigFileValidation_PlainGlobOnProjectDefaultRejected(t *testing.T) { + file := projectconfig.ConfigFile{ + DefaultComponentConfig: &projectconfig.ComponentConfig{ + OverlayFiles: []string{"comps/no-placeholder/*.overlay.toml"}, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "invalid project 'default-component-config'") + assert.Contains(t, err.Error(), "{component}") +} + +func TestProjectConfigFileValidation_MalformedPlaceholderOnProjectDefault(t *testing.T) { + file := projectconfig.ConfigFile{ + DefaultComponentConfig: &projectconfig.ComponentConfig{ + OverlayFiles: []string{"prefix-{component}/*.overlay.toml"}, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "whole path segment") +} + +func TestProjectConfigFileValidation_EmptyOverlayFilesOnDistroDefaultRejected(t *testing.T) { + file := projectconfig.ConfigFile{ + Distros: map[string]projectconfig.DistroDefinition{ + "my-distro": { + Versions: map[string]projectconfig.DistroVersionDefinition{ + "1.0": { + ReleaseVer: "1.0", + DefaultComponentConfig: projectconfig.ComponentConfig{ + OverlayFiles: []string{}, + }, + }, + }, + }, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "my-distro") + assert.Contains(t, err.Error(), "overlay-files") +} + +func TestProjectConfigFileValidation_EmptyOverlayFilesOnGroupDefaultRejected(t *testing.T) { + file := projectconfig.ConfigFile{ + ComponentGroups: map[string]projectconfig.ComponentGroupConfig{ + "core": { + DefaultComponentConfig: projectconfig.ComponentConfig{ + OverlayFiles: []string{}, + }, + }, + }, + } + err := file.Validate() + require.Error(t, err) + assert.Contains(t, err.Error(), "core") + assert.Contains(t, err.Error(), "overlay-files") +} diff --git a/internal/projectconfig/overlay_file.go b/internal/projectconfig/overlay_file.go index eb04ff25..82ebccdb 100644 --- a/internal/projectconfig/overlay_file.go +++ b/internal/projectconfig/overlay_file.go @@ -56,6 +56,11 @@ type OverlayFile struct { // [OverlayFile], stamps the per-file metadata onto each overlay, appends the // resulting overlays after inline overlays, and clears [ComponentConfig.OverlayFiles] // so the returned config is not expanded twice. +// +// Entries containing [OverlayFilesComponentPlaceholder] have the placeholder +// substituted with the component's own name before globbing; those entries are +// inherited from a project-level default and become concrete per-component +// globs here. func ExpandResolvedOverlayFiles( fs opctx.FS, component ComponentConfig, referenceDir string, permissiveConfigParsing bool, ) (ComponentConfig, error) { @@ -63,14 +68,19 @@ func ExpandResolvedOverlayFiles( return component, nil } - if referenceDir == "" && overlayFilesHasRelativePattern(component.OverlayFiles) { + expanded := make([]string, len(component.OverlayFiles)) + for i, entry := range component.OverlayFiles { + expanded[i] = substituteOverlayFilesPlaceholder(entry, component.Name) + } + + if referenceDir == "" && overlayFilesHasRelativePattern(expanded) { return ComponentConfig{}, fmt.Errorf( "component %#q has relative 'overlay-files' entries but no reference directory", component.Name, ) } - loaded, err := loadOverlayFiles(fs, referenceDir, component.OverlayFiles, permissiveConfigParsing) + loaded, err := loadOverlayFiles(fs, referenceDir, expanded, permissiveConfigParsing) if err != nil { return ComponentConfig{}, fmt.Errorf("component %#q overlay-files:\n%w", component.Name, err) } diff --git a/internal/projectconfig/overlay_files_placeholder.go b/internal/projectconfig/overlay_files_placeholder.go new file mode 100644 index 00000000..23578869 --- /dev/null +++ b/internal/projectconfig/overlay_files_placeholder.go @@ -0,0 +1,167 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +package projectconfig + +import ( + "errors" + "fmt" + "strings" + + "github.com/bmatcuk/doublestar/v4" +) + +// OverlayFilesComponentPlaceholder is the single supported placeholder in an +// [ComponentConfig.OverlayFiles] entry. It is required in every project-scope +// entry (discovery) and forbidden in distro/component-group/component-scope +// entries; when present it must appear exactly once and must be a whole path +// segment. +const OverlayFilesComponentPlaceholder = "{component}" + +// ErrInvalidOverlayFilesEntry is returned when an entry in +// [ComponentConfig.OverlayFiles] does not conform to the placeholder rules +// documented on that field. +var ErrInvalidOverlayFilesEntry = errors.New("invalid overlay-files entry") + +// hasOverlayFilesPlaceholder reports whether entry contains at least one +// occurrence of [OverlayFilesComponentPlaceholder]. +func hasOverlayFilesPlaceholder(entry string) bool { + return strings.Contains(entry, OverlayFilesComponentPlaceholder) +} + +// validateOverlayFilesPlaceholder verifies that entry contains exactly one +// [OverlayFilesComponentPlaceholder] and that the placeholder is a whole path +// segment. Path separators must be '/' (POSIX-style), and no glob +// metacharacters ('*', '?', '[') may appear before the placeholder because +// pattern-discovery extracts the captured component name by literal-prefix +// matching against the concrete glob result. Metacharacters after the +// placeholder are allowed. Returns nil when the entry is well-formed. +func validateOverlayFilesPlaceholder(entry string) error { + if entry == "" { + return fmt.Errorf("%w: entry is empty", ErrInvalidOverlayFilesEntry) + } + + count := strings.Count(entry, OverlayFilesComponentPlaceholder) + if count == 0 { + return fmt.Errorf( + "%w: entry %#q must contain %q exactly once", + ErrInvalidOverlayFilesEntry, entry, OverlayFilesComponentPlaceholder, + ) + } + + if count > 1 { + return fmt.Errorf( + "%w: entry %#q contains %q %d times; it must appear exactly once", + ErrInvalidOverlayFilesEntry, entry, OverlayFilesComponentPlaceholder, count, + ) + } + + if strings.ContainsRune(entry, '\\') { + return fmt.Errorf( + "%w: entry %#q must use %q as the path separator (backslash is not accepted)", + ErrInvalidOverlayFilesEntry, entry, "/", + ) + } + + idx := strings.Index(entry, OverlayFilesComponentPlaceholder) + if idx > 0 && entry[idx-1] != '/' { + return fmt.Errorf( + "%w: %q in entry %#q must be a whole path segment (preceded by %q or start of entry)", + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, entry, "/", + ) + } + + tail := idx + len(OverlayFilesComponentPlaceholder) + if tail == len(entry) { + return fmt.Errorf( + "%w: %q in entry %#q must be followed by %q; it names a directory, not a file", + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, entry, "/", + ) + } + + if entry[tail] != '/' { + return fmt.Errorf( + "%w: %q in entry %#q must be a whole path segment (followed by %q)", + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, entry, "/", + ) + } + + if tail+1 == len(entry) { + return fmt.Errorf( + "%w: %q in entry %#q must be followed by at least one more path segment; "+ + "it names a directory, not a file", + ErrInvalidOverlayFilesEntry, OverlayFilesComponentPlaceholder, entry, + ) + } + + prefix := entry[:idx] + if globMetaIdx := strings.IndexAny(prefix, "*?["); globMetaIdx >= 0 { + return fmt.Errorf( + "%w: entry %#q must not contain glob metacharacters (%q) before %q; "+ + "the discovery capture uses a literal prefix", + ErrInvalidOverlayFilesEntry, entry, string(prefix[globMetaIdx]), OverlayFilesComponentPlaceholder, + ) + } + + // Substitute '*' for {component} to build the concrete glob that + // pattern-discovery will hand to doublestar. + globPattern := prefix + "*" + entry[tail:] + if !doublestar.ValidatePattern(globPattern) { + return fmt.Errorf( + "%w: entry %#q expands to invalid glob %#q", + ErrInvalidOverlayFilesEntry, entry, globPattern, + ) + } + + return nil +} + +// SplitOverlayFilesPlaceholder splits an entry at its +// [OverlayFilesComponentPlaceholder] and returns the prefix and suffix. The +// caller can then substitute a concrete component name (or a glob wildcard) +// for the placeholder. Passing an entry without a placeholder is a programming +// error and panics with a diagnostic message. +func SplitOverlayFilesPlaceholder(entry string) (prefix, suffix string) { + idx := strings.Index(entry, OverlayFilesComponentPlaceholder) + if idx < 0 { + panic(fmt.Sprintf( + "SplitOverlayFilesPlaceholder: entry %q does not contain %q", + entry, OverlayFilesComponentPlaceholder, + )) + } + + return entry[:idx], entry[idx+len(OverlayFilesComponentPlaceholder):] +} + +// substituteOverlayFilesPlaceholder returns entry with the single +// [OverlayFilesComponentPlaceholder] replaced by componentName. Entries with +// no placeholder are returned unchanged. +func substituteOverlayFilesPlaceholder(entry, componentName string) string { + if !hasOverlayFilesPlaceholder(entry) { + return entry + } + + return strings.Replace(entry, OverlayFilesComponentPlaceholder, componentName, 1) +} + +// absolutizeOverlayFilesPlaceholderEntries returns a copy of entries where every +// entry containing [OverlayFilesComponentPlaceholder] has been absolutized +// against referenceDir. Non-placeholder entries are cloned verbatim so they can +// still be resolved relative to a per-component reference directory later. +func absolutizeOverlayFilesPlaceholderEntries(referenceDir string, entries []string) []string { + if entries == nil { + return nil + } + + result := make([]string, len(entries)) + + for i, entry := range entries { + if hasOverlayFilesPlaceholder(entry) { + result[i] = makeAbsolute(referenceDir, entry) + } else { + result[i] = entry + } + } + + return result +} diff --git a/internal/projectconfig/overlay_files_placeholder_test.go b/internal/projectconfig/overlay_files_placeholder_test.go new file mode 100644 index 00000000..95b00707 --- /dev/null +++ b/internal/projectconfig/overlay_files_placeholder_test.go @@ -0,0 +1,115 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//nolint:testpackage // Allow to test unexported helpers. +package projectconfig + +import ( + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestValidateOverlayFilesPlaceholder(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + entry string + wantErr bool + }{ + {name: "empty", entry: "", wantErr: true}, + {name: "missing placeholder", entry: "comps/*/overlays/*.overlay.toml", wantErr: true}, + {name: "multiple placeholders", entry: "comps/{component}/{component}/x.toml", wantErr: true}, + {name: "placeholder is prefix substring", entry: "prefix-{component}/x.toml", wantErr: true}, + {name: "placeholder is suffix substring", entry: "{component}-suffix/x.toml", wantErr: true}, + {name: "backslash separator rejected", entry: "comps\\{component}\\x.toml", wantErr: true}, + {name: "glob metachar before placeholder", entry: "**/{component}/*.overlay.toml", wantErr: true}, + {name: "star before placeholder", entry: "comps*/{component}/*.overlay.toml", wantErr: true}, + {name: "whole segment at start", entry: "{component}/overlays/*.overlay.toml", wantErr: false}, + {name: "whole segment in middle", entry: "comps/{component}/overlays/*.overlay.toml", wantErr: false}, + {name: "placeholder at end rejected", entry: "comps/overlays/{component}", wantErr: true}, + {name: "placeholder followed by only slash rejected", entry: "comps/{component}/", wantErr: true}, + {name: "doublestar in suffix allowed", entry: "comps/{component}/**/*.overlay.toml", wantErr: false}, + {name: "absolute path", entry: "/project/base/comps/{component}/overlays/*.overlay.toml", wantErr: false}, + } + + for _, testCase := range tests { + t.Run(testCase.name, func(t *testing.T) { + t.Parallel() + + err := validateOverlayFilesPlaceholder(testCase.entry) + if testCase.wantErr { + require.Error(t, err) + require.ErrorIs(t, err, ErrInvalidOverlayFilesEntry, + "expected ErrInvalidOverlayFilesEntry, got %v", err) + } else { + assert.NoError(t, err) + } + }) + } +} + +func TestSplitOverlayFilesPlaceholder(t *testing.T) { + t.Parallel() + + tests := []struct { + name string + entry string + wantPrefix string + wantSuffix string + }{ + { + name: "middle", + entry: "comps/{component}/overlays/*.overlay.toml", + wantPrefix: "comps/", + wantSuffix: "/overlays/*.overlay.toml", + }, + { + name: "start", + entry: "{component}/overlays/*.overlay.toml", + wantPrefix: "", + wantSuffix: "/overlays/*.overlay.toml", + }, + { + name: "end", + entry: "comps/overlays/{component}", + wantPrefix: "comps/overlays/", + wantSuffix: "", + }, + } + + for _, testCase := range tests { + t.Run(testCase.name, func(t *testing.T) { + t.Parallel() + + prefix, suffix := SplitOverlayFilesPlaceholder(testCase.entry) + assert.Equal(t, testCase.wantPrefix, prefix) + assert.Equal(t, testCase.wantSuffix, suffix) + }) + } +} + +func TestSplitOverlayFilesPlaceholder_PanicsWithoutPlaceholder(t *testing.T) { + t.Parallel() + + assert.PanicsWithValue(t, + `SplitOverlayFilesPlaceholder: entry "comps/foo/overlays/x.toml" does not contain "{component}"`, + func() { + SplitOverlayFilesPlaceholder("comps/foo/overlays/x.toml") + }) +} + +func TestSubstituteOverlayFilesPlaceholder(t *testing.T) { + t.Parallel() + + assert.Equal(t, + "comps/foo/overlays/*.toml", + substituteOverlayFilesPlaceholder("comps/{component}/overlays/*.toml", "foo")) + + // No placeholder → unchanged. + assert.Equal(t, + "overlays/*.toml", + substituteOverlayFilesPlaceholder("overlays/*.toml", "foo")) +} diff --git a/scenario/__snapshots__/TestSnapshotsContainer_config_generate-schema_stdout_1.snap b/scenario/__snapshots__/TestSnapshotsContainer_config_generate-schema_stdout_1.snap index 71992096..53b281e7 100755 --- a/scenario/__snapshots__/TestSnapshotsContainer_config_generate-schema_stdout_1.snap +++ b/scenario/__snapshots__/TestSnapshotsContainer_config_generate-schema_stdout_1.snap @@ -141,7 +141,7 @@ }, "type": "array", "title": "Overlay files", - "description": "Path or glob patterns (relative to the component config file or matched spec directory) matched against the filesystem to locate per-file overlay documents after component config resolution. Use an empty list to disable inherited overlay-file patterns" + "description": "Path or glob patterns matched against the filesystem to locate per-file overlay documents. Only settable at project scope (top-level 'default-component-config') or at component scope ('[components.X]'). At project scope every entry must contain the {component} placeholder as a whole path segment (drives discovery and per-component substitution). At component scope {component} is not allowed and entries are plain globs relative to the config file. Distro-version and component-group 'default-component-config' tables must not set this field. Use an empty list at component scope to disable inheritance." }, "build": { "$ref": "#/$defs/ComponentBuildConfig", diff --git a/scenario/__snapshots__/TestSnapshots_config_generate-schema_stdout_1.snap b/scenario/__snapshots__/TestSnapshots_config_generate-schema_stdout_1.snap index 71992096..53b281e7 100755 --- a/scenario/__snapshots__/TestSnapshots_config_generate-schema_stdout_1.snap +++ b/scenario/__snapshots__/TestSnapshots_config_generate-schema_stdout_1.snap @@ -141,7 +141,7 @@ }, "type": "array", "title": "Overlay files", - "description": "Path or glob patterns (relative to the component config file or matched spec directory) matched against the filesystem to locate per-file overlay documents after component config resolution. Use an empty list to disable inherited overlay-file patterns" + "description": "Path or glob patterns matched against the filesystem to locate per-file overlay documents. Only settable at project scope (top-level 'default-component-config') or at component scope ('[components.X]'). At project scope every entry must contain the {component} placeholder as a whole path segment (drives discovery and per-component substitution). At component scope {component} is not allowed and entries are plain globs relative to the config file. Distro-version and component-group 'default-component-config' tables must not set this field. Use an empty list at component scope to disable inheritance." }, "build": { "$ref": "#/$defs/ComponentBuildConfig", diff --git a/schemas/azldev.schema.json b/schemas/azldev.schema.json index 71992096..53b281e7 100644 --- a/schemas/azldev.schema.json +++ b/schemas/azldev.schema.json @@ -141,7 +141,7 @@ }, "type": "array", "title": "Overlay files", - "description": "Path or glob patterns (relative to the component config file or matched spec directory) matched against the filesystem to locate per-file overlay documents after component config resolution. Use an empty list to disable inherited overlay-file patterns" + "description": "Path or glob patterns matched against the filesystem to locate per-file overlay documents. Only settable at project scope (top-level 'default-component-config') or at component scope ('[components.X]'). At project scope every entry must contain the {component} placeholder as a whole path segment (drives discovery and per-component substitution). At component scope {component} is not allowed and entries are plain globs relative to the config file. Distro-version and component-group 'default-component-config' tables must not set this field. Use an empty list at component scope to disable inheritance." }, "build": { "$ref": "#/$defs/ComponentBuildConfig",