feat(source-files): generative source dependencies#271
Conversation
There was a problem hiding this comment.
Pull request overview
Adds support for declaring “already-fetched” source-file inputs for origin.type = "custom" generation scripts, and adjusts source preparation ordering so upstream sources are available before custom generation runs.
Changes:
- Introduces
origin.inputsforcomponents.*.source-filesand validates it in config parsing (schema + docs + tests updated). - Updates source preparation flow to fetch component sources before running custom generation, and prevents lookaside extraction from overwriting configured
source-filesnames. - Enhances custom generation execution to optionally stream script output with
--verboseand include stdout/stderr in failure diagnostics.
Reviewed changes
Copilot reviewed 14 out of 14 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| schemas/azldev.schema.json | Adds JSON schema for origin.inputs. |
| scenario/snapshots/TestSnapshotsContainer_config_generate-schema_stdout_1.snap | Updates schema snapshot to include origin.inputs. |
| scenario/snapshots/TestSnapshots_config_generate-schema_stdout_1.snap | Updates schema snapshot to include origin.inputs. |
| internal/providers/sourceproviders/sourcemanager.go | Passes verbosity into custom provider; prevents lookaside extraction from overwriting configured source-files. |
| internal/providers/sourceproviders/customsourceprovider.go | Stages declared inputs next to script; refactors runner setup; improves script output handling/diagnostics. |
| internal/providers/sourceproviders/customsourceprovider_internal_test.go | Adds unit tests for input staging and output formatting. |
| internal/projectconfig/configfile.go | Validates origin.inputs (only allowed for custom, and entries must be valid filenames). |
| internal/projectconfig/configfile_test.go | Adds tests covering origin.inputs validation. |
| internal/projectconfig/component.go | Adds Origin.Inputs and fingerprints it when non-empty. |
| internal/app/azldev/core/sources/sourceprep.go | Reorders source fetching so component sources come before source-files fetching. |
| internal/app/azldev/core/sources/sourceprep_test.go | Updates expectations for the new fetch ordering. |
| internal/app/azldev/cmds/component/history_internal_test.go | Extends fingerprintable-field coverage to include Origin.Inputs. |
| internal/app/azldev/cmds/component/history_customizations.go | Surfaces source-files.inputs in customization history. |
| docs/user/reference/config/components.md | Documents origin.inputs and clarifies upstream tarballs are already available as inputs. |
4330c9f to
96dcb51
Compare
| component.GetName(), err) | ||
| } | ||
|
|
||
| // Fetch custom and downloaded source files. |
There was a problem hiding this comment.
I think there is an ordering issue here, FetchComponent can grab a dist-git file first, and if it happens to have the same name as something in source-files, we skip further checks. We should probably just not allow things like combining replace-upstream with this.
There was a problem hiding this comment.
In the latest iteration I make sure source preparation never downloads any file already defined in source-files. Of course we continue to support replace-upstream to ensure the dev understands they are replacing a source already present in the upstream
b5068fa to
4377f80
Compare
Adds support for custom source generation inputs.
Custom source-files entries can now declare
origin.inputs, a list of already-fetched source filenames that azldev stages next to the generation script inside the mock chroot. This lets scripts consume upstream tarballs or earlier source files directly by relative filename, without knowing azldev’s internal staging paths.Also updates source preparation ordering so component sources are fetched before custom source generation runs, making upstream source tarballs available as inputs. Custom script stdout/stderr remain quiet by default, but are streamed with
--verboseand included in failure diagnostics.Inside gen-telegraf-vendor-stripped.sh, the input is available beside the script: