Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion acceptance/localenv/constraints-only/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

>>> [CLI] environments setup-local --serverless v4 --constraints-only --check --output json
>>> [CLI] environments setup-local --serverless-version v4 --constraints-only --dry-run --output json
{
"schemaVersion": 1,
"command": "environments setup-local",
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/constraints-only/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
trace $CLI environments setup-local --serverless v4 --constraints-only --check --output json
trace $CLI environments setup-local --serverless-version v4 --constraints-only --dry-run --output json
2 changes: 1 addition & 1 deletion acceptance/localenv/env-unsupported/output.txt
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
preflight ok check
resolve ok source=cluster envKey=dbr/15.4.x-scala2.12
fetch error no published environment for "dbr/15.4.x-scala2.12". If this is a new runtime, try the latest LTS target (e.g. --serverless v4 or a supported --cluster DBR): GET [DATABRICKS_URL]/dbr/15.4.x-scala2.12/pyproject.toml: environment key not found
fetch error no published environment for "dbr/15.4.x-scala2.12". If this is a new runtime, try the latest LTS target (e.g. --serverless-version v4 or a supported --cluster-id DBR): GET [DATABRICKS_URL]/dbr/15.4.x-scala2.12/pyproject.toml: environment key not found
merge pending
provision pending
validate pending
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/env-unsupported/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
musterr $CLI environments setup-local --cluster test-cluster-id --check
musterr $CLI environments setup-local --cluster-id test-cluster-id --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/flag-conflict/output.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
Error: if any flags in the group [cluster serverless job] are set none of the others can be; [cluster serverless] were all set
Error: if any flags in the group [cluster-id serverless-version job-id] are set none of the others can be; [cluster-id serverless-version] were all set
2 changes: 1 addition & 1 deletion acceptance/localenv/flag-conflict/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
musterr $CLI environments setup-local --cluster abc --serverless v4
musterr $CLI environments setup-local --cluster-id abc --serverless-version v4
12 changes: 6 additions & 6 deletions acceptance/localenv/help/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -10,12 +10,12 @@ Usage:
databricks environments setup-local [flags]

Flags:
--check compute the plan without writing files or provisioning
--cluster string cluster ID to use as the compute target
--constraints-only apply the Python version and constraints without adding the databricks-connect dependency
-h, --help help for setup-local
--job string job ID to use as the compute target
--serverless string serverless version to use as the compute target (e.g. v4)
--cluster-id string cluster ID to use as the compute target
--constraints-only apply the Python version and constraints without adding the databricks-connect dependency
--dry-run compute the plan without writing files or provisioning
-h, --help help for setup-local
--job-id string job ID to use as the compute target
--serverless-version string serverless version to use as the compute target (e.g. v4)

Global Flags:
--debug enable debug logging
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/job-ambiguous-compute/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
preflight ok check
resolve error resolving job 12345: job 12345 has both serverless environments and job clusters; pass --cluster or --serverless explicitly to disambiguate
resolve error resolving job 12345: job 12345 has both serverless environments and job clusters; pass --cluster-id or --serverless-version explicitly to disambiguate
fetch pending
merge pending
provision pending
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/job-ambiguous-compute/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
musterr $CLI environments setup-local --job 12345 --check
musterr $CLI environments setup-local --job-id 12345 --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/job-classic-check/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

>>> [CLI] environments setup-local --job 12345 --check
>>> [CLI] environments setup-local --job-id 12345 --dry-run
preflight ok check
resolve ok source=job envKey=dbr/15.4.x-scala2.12
fetch ok source=[DATABRICKS_URL]/dbr/15.4.x-scala2.12/pyproject.toml fromCache=false
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/job-classic-check/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
trace $CLI environments setup-local --job 12345 --check
trace $CLI environments setup-local --job-id 12345 --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/job-multicluster-mismatch/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
preflight ok check
resolve error resolving job 12345: job 12345 has job clusters with differing spark_version; pass --cluster or --serverless explicitly to disambiguate
resolve error resolving job 12345: job 12345 has job clusters with differing spark_version; pass --cluster-id or --serverless-version explicitly to disambiguate
fetch pending
merge pending
provision pending
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/job-multicluster-mismatch/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
musterr $CLI environments setup-local --job 12345 --check
musterr $CLI environments setup-local --job-id 12345 --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/job-serverless-check/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

>>> [CLI] environments setup-local --job 12345 --check
>>> [CLI] environments setup-local --job-id 12345 --dry-run
preflight ok check
resolve ok source=job envKey=serverless/serverless-v3
fetch ok source=[DATABRICKS_URL]/serverless/serverless-v3/pyproject.toml fromCache=false
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/job-serverless-check/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
trace $CLI environments setup-local --job 12345 --check
trace $CLI environments setup-local --job-id 12345 --dry-run
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
preflight ok check
resolve error resolving job 12345: job 12345 has serverless environments with differing versions; pass --serverless explicitly to disambiguate
resolve error resolving job 12345: job 12345 has serverless environments with differing versions; pass --serverless-version explicitly to disambiguate
fetch pending
merge pending
provision pending
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/job-serverless-version-mismatch/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
musterr $CLI environments setup-local --job 12345 --check
musterr $CLI environments setup-local --job-id 12345 --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/json-error/output.txt
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@
"error": {
"code": "E_NO_TARGET",
"failurePhase": "resolve",
"message": "No compute target is selected. Select a cluster or serverless target, or pass --cluster / --serverless / --job",
"message": "No compute target is selected. Select a cluster or serverless target, or pass --cluster-id / --serverless-version / --job-id",
"diskMutated": false
},
"durationMs": 0
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/manager-unsupported/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
musterr $CLI environments setup-local --serverless v4 --check
musterr $CLI environments setup-local --serverless-version v4 --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/no-target/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
preflight ok uv [UV_VERSION]
resolve error No compute target is selected. Select a cluster or serverless target, or pass --cluster / --serverless / --job
resolve error No compute target is selected. Select a cluster or serverless target, or pass --cluster-id / --serverless-version / --job-id
fetch pending
merge pending
provision pending
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/serverless-check/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

>>> [CLI] environments setup-local --serverless v4 --check
>>> [CLI] environments setup-local --serverless-version v4 --dry-run
preflight ok check
resolve ok source=serverless envKey=serverless/serverless-v4
fetch ok source=[DATABRICKS_URL]/serverless/serverless-v4/pyproject.toml fromCache=false
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/serverless-check/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
trace $CLI environments setup-local --serverless v4 --check
trace $CLI environments setup-local --serverless-version v4 --dry-run
2 changes: 1 addition & 1 deletion acceptance/localenv/serverless-json/output.txt
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@

>>> [CLI] environments setup-local --serverless v4 --check --output json
>>> [CLI] environments setup-local --serverless-version v4 --dry-run --output json
{
"schemaVersion": 1,
"command": "environments setup-local",
Expand Down
2 changes: 1 addition & 1 deletion acceptance/localenv/serverless-json/script
Original file line number Diff line number Diff line change
@@ -1 +1 @@
trace $CLI environments setup-local --serverless v4 --check --output json
trace $CLI environments setup-local --serverless-version v4 --dry-run --output json
10 changes: 5 additions & 5 deletions cmd/environments/compute.go
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ func (c sdkCompute) GetClusterSparkVersion(ctx context.Context, clusterID string
// job-level job_clusters) is not resolved here: it may vary per task and an
// existing_cluster_id would need a second lookup, which is out of scope for the
// initial job support. Such a job returns an actionable error rather than a wrong
// guess; use --cluster or --serverless explicitly instead.
// guess; use --cluster-id or --serverless-version explicitly instead.
func (c sdkCompute) GetJobSparkVersion(ctx context.Context, jobID string) (sparkVersion string, isServerless bool, version string, err error) {
id, err := strconv.ParseInt(jobID, 10, 64)
if err != nil {
Expand All @@ -53,7 +53,7 @@ func (c sdkCompute) GetJobSparkVersion(ctx context.Context, jobID string) (spark
// ambiguous: its tasks can run on different compute, so there is no single
// correct local environment to provision. Refuse rather than guess serverless.
if len(job.Settings.Environments) > 0 && len(job.Settings.JobClusters) > 0 {
return "", false, "", fmt.Errorf("job %d has both serverless environments and job clusters; pass --cluster or --serverless explicitly to disambiguate", id)
return "", false, "", fmt.Errorf("job %d has both serverless environments and job clusters; pass --cluster-id or --serverless-version explicitly to disambiguate", id)
}

// Serverless jobs have Environments populated; classic compute uses JobClusters.
Expand All @@ -69,7 +69,7 @@ func (c sdkCompute) GetJobSparkVersion(ctx context.Context, jobID string) (spark
// first. A pinned-vs-unpinned mix is also ambiguous, so compare raw values.
for _, e := range job.Settings.Environments[1:] {
if environmentVersion(e) != version {
return "", false, "", fmt.Errorf("job %d has serverless environments with differing versions; pass --serverless explicitly to disambiguate", id)
return "", false, "", fmt.Errorf("job %d has serverless environments with differing versions; pass --serverless-version explicitly to disambiguate", id)
}
}
return "", true, version, nil
Expand All @@ -85,13 +85,13 @@ func (c sdkCompute) GetJobSparkVersion(ctx context.Context, jobID string) (spark
// Refuse rather than silently provisioning for the first cluster.
for _, jc := range job.Settings.JobClusters[1:] {
if jc.NewCluster.SparkVersion != sv {
return "", false, "", fmt.Errorf("job %d has job clusters with differing spark_version; pass --cluster or --serverless explicitly to disambiguate", id)
return "", false, "", fmt.Errorf("job %d has job clusters with differing spark_version; pass --cluster-id or --serverless-version explicitly to disambiguate", id)
}
}
return sv, false, sv, nil
}

return "", false, "", fmt.Errorf("could not determine compute for job %d from its environments or job clusters (task-level compute is not supported); pass --cluster or --serverless explicitly", id)
return "", false, "", fmt.Errorf("could not determine compute for job %d from its environments or job clusters (task-level compute is not supported); pass --cluster-id or --serverless-version explicitly", id)
}

// environmentVersion returns the serverless environment version recorded on a
Expand Down
30 changes: 15 additions & 15 deletions cmd/environments/sync.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,27 +47,27 @@ env-owned sections are refreshed, user-owned content is preserved).`,

// addTargetFlags adds the shared target and mode flags to a command.
func addTargetFlags(cmd *cobra.Command) {
cmd.Flags().String("cluster", "", "cluster ID to use as the compute target")
cmd.Flags().String("serverless", "", "serverless version to use as the compute target (e.g. v4)")
cmd.Flags().String("job", "", "job ID to use as the compute target")
cmd.Flags().String("cluster-id", "", "cluster ID to use as the compute target")
cmd.Flags().String("serverless-version", "", "serverless version to use as the compute target (e.g. v4)")
cmd.Flags().String("job-id", "", "job ID to use as the compute target")
cmd.Flags().Bool("constraints-only", false, "apply the Python version and constraints without adding the databricks-connect dependency")
cmd.Flags().Bool("check", false, "compute the plan without writing files or provisioning")
cmd.Flags().String("constraint-source", "", "URL for the constraint source (overrides "+envConstraintSource+")")
// Hide constraint-source from casual --help output; it is a power-user escape hatch.
_ = cmd.Flags().MarkHidden("constraint-source")
cmd.MarkFlagsMutuallyExclusive("cluster", "serverless", "job")
cmd.Flags().Bool("dry-run", false, "compute the plan without writing files or provisioning")
cmd.Flags().String("constraint-source-url", "", "URL for the constraint source (overrides "+envConstraintSource+")")
// Hide constraint-source-url from casual --help output; it is a power-user escape hatch.
_ = cmd.Flags().MarkHidden("constraint-source-url")
cmd.MarkFlagsMutuallyExclusive("cluster-id", "serverless-version", "job-id")
}

// runPipeline builds and runs the setup-local Pipeline.
func runPipeline(cmd *cobra.Command) error {
ctx := cmd.Context()

cluster, _ := cmd.Flags().GetString("cluster")
serverless, _ := cmd.Flags().GetString("serverless")
job, _ := cmd.Flags().GetString("job")
cluster, _ := cmd.Flags().GetString("cluster-id")
serverless, _ := cmd.Flags().GetString("serverless-version")
job, _ := cmd.Flags().GetString("job-id")
constraintsOnly, _ := cmd.Flags().GetBool("constraints-only")
check, _ := cmd.Flags().GetBool("check")
constraintSource, _ := cmd.Flags().GetString("constraint-source")
check, _ := cmd.Flags().GetBool("dry-run")
constraintSource, _ := cmd.Flags().GetString("constraint-source-url")

targetFlags := libslocalenv.TargetFlags{
Cluster: cluster,
Expand Down Expand Up @@ -100,7 +100,7 @@ func runPipeline(cmd *cobra.Command) error {
cacheDir = filepath.Join(cacheDir, "databricks", "localenv")

// The bundle is only a fallback: ResolveTarget consults it solely when no
// explicit --cluster/--serverless/--job flag is set. Skip the bundle load
// explicit --cluster-id/--serverless-version/--job-id flag is set. Skip the bundle load
// entirely when a flag is present — it would otherwise re-run TryConfigureBundle
// (a second full load) and re-print any bundle load-time diagnostics for nothing.
var bt libslocalenv.BundleTarget
Expand All @@ -126,7 +126,7 @@ func runPipeline(cmd *cobra.Command) error {
}

// resolveConstraintBaseURL returns the constraint base URL using ordered precedence:
// an explicit --constraint-source flag, then a full-URL override from
// an explicit --constraint-source-url flag, then a full-URL override from
// DATABRICKS_LOCALENV_CONSTRAINT_SOURCE, then the URL derived from the hosting repo
// (libslocalenv.RepoConstraintBaseURL). All three may be unset, in which case it
// returns "" and the pipeline reports the missing source at the fetch phase.
Expand Down
6 changes: 3 additions & 3 deletions libs/localenv/constraints.go
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ func writeCacheAtomic(path string, data []byte) error {
// reported below as a fetch-phase error.
//
// writeCache controls whether a successful live fetch populates the on-disk
// cache. Callers pass false for a dry run (--check), which must not mutate
// cache. Callers pass false for a dry run (--dry-run), which must not mutate
// disk; an existing cache is still read for offline fallback, since reading is
// not a mutation.
func FetchConstraints(ctx context.Context, baseURL, envKey, cacheDir string, writeCache bool) (*Constraints, error) {
Expand All @@ -164,7 +164,7 @@ func FetchConstraints(ctx context.Context, baseURL, envKey, cacheDir string, wri
}
// Write the cache copy (creating cacheDir if needed, atomically); non-fatal
// so a read-only cacheDir doesn't break the command. Skipped under a dry
// run so --check performs no disk writes at all.
// run so --dry-run performs no disk writes at all.
if writeCache {
if err := writeCacheAtomic(cachePath, data); err != nil {
log.Debugf(ctx, "failed to write constraint cache %s: %v", filepath.ToSlash(cachePath), err)
Expand All @@ -184,7 +184,7 @@ func FetchConstraints(ctx context.Context, baseURL, envKey, cacheDir string, wri
// fallback: the target resolved to an environment that isn't published.
if errors.Is(fetchErr, errEnvKeyNotFound) {
return nil, NewError(ErrEnvUnsupported, fetchErr,
"no published environment for %q. If this is a new runtime, try the latest LTS target (e.g. --serverless v4 or a supported --cluster DBR)", envKey)
"no published environment for %q. If this is a new runtime, try the latest LTS target (e.g. --serverless-version v4 or a supported --cluster-id DBR)", envKey)
}

// Network or HTTP failure: attempt to serve from cache.
Expand Down
10 changes: 5 additions & 5 deletions libs/localenv/pipeline.go
Original file line number Diff line number Diff line change
Expand Up @@ -95,12 +95,12 @@ func (p *Pipeline) run(ctx context.Context) error {
if m := detectManager(p.ProjectDir); m != managerUv {
return p.fail(PhasePreflight, false, NewError(ErrManagerUnsupported, nil, "%s", managerGuidance(m)))
}
// Under --check the pipeline only reads and reports a plan, so it must not
// Under --dry-run the pipeline only reads and reports a plan, so it must not
// mutate anything at preflight. Two preflight steps can write:
// - ensureWritable creates and removes a temp file (and would fail a
// read-only project the user only wants to inspect);
// - PackageManager.EnsureAvailable may install the manager (uv) if missing.
// Both exist to fail fast before real writes, which --check never performs, so
// Both exist to fail fast before real writes, which --dry-run never performs, so
// they are skipped in a dry run. Neither result is needed to compute the plan.
if p.Check {
p.markOK(PhasePreflight, "check")
Expand Down Expand Up @@ -191,7 +191,7 @@ func (p *Pipeline) resolve(ctx context.Context) (*TargetInfo, error) {
}

// fetch fetches constraints for the resolved target and records the fetch phase.
// Under --check the cache is not populated, so a dry run performs no disk writes
// Under --dry-run the cache is not populated, so a dry run performs no disk writes
// (an existing cache is still read for offline fallback).
func (p *Pipeline) fetch(ctx context.Context, target *TargetInfo) (*Constraints, error) {
c, err := FetchConstraints(ctx, p.ConstraintBaseURL, target.EnvKey, p.CacheDir, !p.Check)
Expand All @@ -218,7 +218,7 @@ func (p *Pipeline) backupPath() string {

// mergePlan computes the merged pyproject.toml bytes (without writing to disk),
// decides greenfield vs. existing, and builds the Plan (populated only under
// --check). dbcPin is the databricks-connect pin to inject, or "" in
// --dry-run). dbcPin is the databricks-connect pin to inject, or "" in
// constraints-only mode.
func (p *Pipeline) mergePlan(_ context.Context, pyMinor string, c *Constraints, dbcPin string) (merged []byte, greenfield bool, err error) {
pyproject := p.pyprojectPath()
Expand Down Expand Up @@ -267,7 +267,7 @@ func (p *Pipeline) mergePlan(_ context.Context, pyMinor string, c *Constraints,
}
}

// Under --check, build the plan (with a diff) for reporting. A real run does
// Under --dry-run, build the plan (with a diff) for reporting. A real run does
// not need the diff.
if p.Check {
oldStr := ""
Expand Down
2 changes: 1 addition & 1 deletion libs/localenv/result.go
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ type ResolvedInfo struct {
ArtifactSource string `json:"artifactSource"`
}

// Plan describes the changes a --check run would apply (spec §6.3).
// Plan describes the changes a --dry-run run would apply (spec §6.3).
// ChangedRegions is retained for text output only and is not serialized.
type Plan struct {
WouldWrite string `json:"wouldWrite"`
Expand Down
10 changes: 5 additions & 5 deletions libs/localenv/target.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,20 +32,20 @@ type BundleTarget struct {

// noTargetMessage is the actionable message shown when no target is selected,
// matching spec §2.3.
const noTargetMessage = "No compute target is selected. Select a cluster or serverless target, or pass --cluster / --serverless / --job"
const noTargetMessage = "No compute target is selected. Select a cluster or serverless target, or pass --cluster-id / --serverless-version / --job-id"

// ValidateTargetFlags returns an error if more than one of the three flags is set.
// Cobra marks them mutually exclusive too; this guards the library path.
func ValidateTargetFlags(f TargetFlags) error {
var set []string
if f.Cluster != "" {
set = append(set, "--cluster")
set = append(set, "--cluster-id")
}
if f.Serverless != "" {
set = append(set, "--serverless")
set = append(set, "--serverless-version")
}
if f.Job != "" {
set = append(set, "--job")
set = append(set, "--job-id")
}
if len(set) > 1 {
return fmt.Errorf("flags %s are mutually exclusive; specify at most one", strings.Join(set, " and "))
Expand All @@ -54,7 +54,7 @@ func ValidateTargetFlags(f TargetFlags) error {
}

// ResolveTarget resolves the compute target using ordered precedence:
// --cluster flag → --serverless flag → --job flag → bundle target.
// --cluster-id flag → --serverless-version flag → --job-id flag → bundle target.
// PythonVersion is left empty; it is filled later from constraint data.
//
// Incompatible flags are rejected up front: without this a library caller that
Expand Down
Loading