bundle: warn when a task parameter is an empty string under the terraform engine#5951
Draft
ankit481 wants to merge 2 commits into
Draft
bundle: warn when a task parameter is an empty string under the terraform engine#5951ankit481 wants to merge 2 commits into
ankit481 wants to merge 2 commits into
Conversation
…form engine An empty string element in a task parameters list passes bundle validate but fails bundle deploy under the terraform engine: the provider reads the element back as nil and job creation panics with "parameters[<nil>] is not a string". The generated bundle.tf.json is well formed (the element renders as ""), and the direct engine deploys the same configuration correctly, so the defect sits in the provider layer. See databricks#5950. This adds a validate-time warning that names the config location of the empty parameter and suggests removing it or using the direct engine. The warning is only emitted when the effective engine is not direct. Acceptance tests cover both engines: the direct variant deploys successfully and jobs/create receives the empty string; the terraform variant records the warning and the current provider panic. A second validate-only test pins that every empty-variable shape (target overrides, BUNDLE_VAR, --var, var-in-var, complex field) resolves to an empty string rather than null.
Contributor
|
An authorized user can trigger integration tests manually by following the instructions below: Trigger: Inputs:
Checks will be approved automatically on success. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Adds a validate-time warning when a task's
parameterslist contains an empty string element and the bundle deploys with the terraform engine. Such deploys currently passbundle validateand then fail while creating the job with a provider panic:The warning names the config location of the empty element and suggests removing it or deploying with the direct engine, which handles the same configuration correctly. It is emitted only when the effective engine is not direct, following the same engine resolution as
validate.TFOnlyReferences.This is a draft on purpose. The underlying defect looks like it sits in the terraform provider's schema read layer (the generated
bundle.tf.jsonis well formed and renders the element as""), so the real fix may belong in terraform-provider-databricks. This PR covers the CLI side: catching it at validate time instead of letting terraform apply crash. Direction welcome on warning versus error, the exact wording, and whether you would rather see only the regression tests merged here.Related: #5950. The warning is a guard on the CLI side; the provider panic itself remains open.
Why
An empty parameter usually comes from a bundle variable that is deliberately blank in some targets, for example a per-developer prefix that is empty in shared environments. That configuration is legal, validate passes, and the crash appears only at deploy time in whichever target leaves the variable empty. We hit this four times over two months before spotting the pattern. A warning at validate time with the file and line turns an opaque provider panic into something the user can act on before deploying.
Tests
spark_python_task,python_wheel_task, and afor_each_tasknestedspark_jar_taskunder the terraform engine, and silence under the direct engine or when no parameter is empty.bundle/variables/empty-in-task-paramsruns validate and deploy under both engines. The direct variant deploys successfully andjobs/createreceives the empty string. The terraform variant records the new warning and the current provider panic, and also records the parameters as rendered intobundle.tf.jsonto show the generated configuration is well formed.bundle/variables/empty-in-task-params-variantspins that every empty-variable shape (short, long, and null target overrides,BUNDLE_VAR_*,--var, a variable referencing another variable, and a complex variable field) resolves to an empty string rather than null.