[Feature]: Add a safe, non-attached lifecycle for named dev environments

[wolegechu]

Problem

A long-lived development environment is usually an explicitly named dev-environment run, often with a persistent volume. Its daily lifecycle currently requires users or automation to compose several lower-level operations:

1. submit dstack apply -d;
2. poll the run and selected job;
3. keep dstack attach running to obtain SSH or IDE access;
4. stop the run by name when it is no longer needed.

This leaves three gaps.

First, dstack apply is not a fail-closed "ensure this exact environment is running" operation. Depending on the plan, it may update, stop, or recreate an active run. Automation also cannot distinguish a cancelled apply, a safe no-op, and a newly submitted run through a structured result, or fence subsequent polling to the exact run/deployment that was planned.

Second, attaching should not be required merely to open an SSH or IDE session. The run API already exposes proxied SSH and IDE connection information, but there is no public Python/CLI consumer that validates it and opens a direct SSHProxy connection without starting a resident attach process.

Third, repositories cannot express a small shared lifecycle contract around a standard named dev-environment configuration, such as an expected SSHProxy endpoint or minimum persistent-volume requirements.

Solution

I propose an additive implementation delivered in four reviewable stages.

1. Fail-closed apply for named runs

Add dstack apply --no-recreate.

The option would:

• require an explicitly named run configuration;
• be mutually exclusive with --force;
• use normal apply behavior when the run is absent or terminal;
• return a no-op only when the active run belongs to the requesting user, has no effective configuration drift, requires no in-place change or recreation, and is in an explicitly safe active state;
• fail before mutation for owner mismatch, drift, pending/terminating state, in-place update, or recreation;
• never silently stop or replace an active environment.

Detached planning would expose a structured internal result such as:

ApplyPlanOutcome(
    result = cancelled | noop | submitted,
    fence = {run_id, deployment_num, user} | None,
)

A submitted fence must come from the Run returned by the actual apply operation; a no-op fence comes from the current resource inspected by the plan. Existing callers of apply_configuration() would retain their current behavior and return contract.

2. Direct SSHProxy and IDE connection

Add a public run connection resolver and a dstack connect command:

dstack connect my-dev
dstack connect my-dev --ide
dstack connect my-dev --zed

The resolver would refresh the current run, require owner access to a running dev environment and selected job, validate the proxy endpoint/upstream identifier/SSH argv/IDE URL/working directory as one record, optionally pin the expected SSHProxy endpoint, and execute commands as argv without a shell. Invalid or missing proxy metadata would fail closed without legacy host-SSH fallback. Local SSH state would be written only after the complete server-provided record passes pure validation.

This is not a replacement for dstack attach: attach remains useful for log streaming, port forwarding, and an attached foreground lifecycle.

3. Repository-local dstack dev lifecycle

Add:

dstack dev status
dstack dev up
dstack dev connect
dstack dev open
dstack dev down

The initial command would operate on a standard explicitly named dev-environment configuration. up would use fail-closed detached apply and exact run/deployment/user fences while waiting. open would check the local IDE before allocation, then safely start/reuse the run and open the direct IDE connection. down would validate owner/type/identity, confirm, re-read identity immediately before the existing name-based stop, and never delete persistent volumes. status --json would emit exactly one JSON document and treat an absent environment as a successful lifecycle state.

4. Optional repository descriptor and read-only doctor

If maintainers agree that a small repository-level descriptor is appropriate, a follow-up could support nearest-upward discovery of .dstack/dev.yml:

schema: dstack.dev/v1
project: my-project
configuration: dev.dstack.yml
apply:
  no_repo: false
checks:
  sshproxy_endpoint: ssh.example.com:22
  volumes:
    - name: workspace
      mount_path: /workspace
      min_size: 512GB
      access_modes: [ReadWriteOnce]

The descriptor would not define another server configuration type. It would only select a normal named dev-environment configuration and add local guardrails. Expected rules include strict schema validation, workspace path containment, explicit option precedence, and read-only checks for volume requirements, SSHProxy, IDE, run type, and ownership.

doctor must remain genuinely read-only. Since planning configurations with repos or files may stage or upload data, doctor must not claim those plans are safe: for an active environment it should fail closed when it cannot certify the effective configuration; for absent/terminal runs it may skip that planning check and report the boundary explicitly.

Security and compatibility boundaries:

• no provider, fleet, proxy, storage, or IDE defaults;
• no new server resource type or initial database migration;
• no automatic volume creation/deletion;
• no legacy host-SSH fallback or shell invocation with server values;
• existing apply, attach, and stop behavior remains unchanged unless a new option/command is used;
• owner, type, state, run ID, deployment number, and user identity are checked at the relevant lifecycle boundaries;
• machine-readable commands emit one JSON document without banners.

The current stop endpoint accepts a run name rather than an expected run ID/deployment number. A CLI re-read narrows the race but cannot make down atomic. Maintainer guidance would be helpful on whether this best-effort first version is acceptable or server-side compare-and-stop should be a prerequisite.

Planned tests cover the full no-recreate owner/status/diff/action matrix; cancelled/no-op/submitted fences; same-name replacement; malicious connection metadata; no-write-on-validation-failure; replica/job and IDE selection; descriptor discovery and containment; volume/proxy guardrails; JSON output; finite waits; IDE preflight; down re-read ordering; and volume preservation. Each PR would include scoped pytest, Ruff, Pyright, the full Python suite where applicable, and relevant documentation checks.

Related work: #3681, #3711, and #3947 established SSHProxy/IDE/Zed foundations. #3768 was UI-oriented. #4045/#4053 concern capacity reuse rather than fail-closed same-name run lifecycle.

Workaround

Today users can compose dstack apply -d, dstack run get --json, dstack attach, and dstack stop -y. Scripts can inspect and poll, but there is no single fail-closed apply outcome/fence, and every direct-access script must independently validate proxy metadata and build SSH/IDE state. Keeping attach resident works, but is unnecessary when the user only needs a direct SSH or IDE session.

Would you like to help us implement this feature by sending a PR?

Yes

[wolegechu]

The first two independently reviewable foundations are now available as draft PRs:

• Add fail-closed apply for named runs #4124 adds fail-closed dstack apply --no-recreate and the internal apply outcome/fence.
• Add direct dev environment connections #4125 adds validated direct SSHProxy/IDE resolution and dstack connect.

I am keeping the repository-local dstack dev lifecycle and optional descriptor/doctor stages behind review of these foundations. That keeps each current PR based directly on master and avoids duplicating foundational diffs across stacked cross-fork PRs.

[wolegechu]

The first two implementation slices are ready and up to date with master:

• Add fail-closed apply for named runs #4124: fail-closed apply --no-recreate for named runs
• Add direct dev environment connections #4125: direct SSH/Zed/IDE connection through SSHProxy without a resident attach process

Both PRs now include local test/lint evidence and a CLI transcript. Their fork workflow runs are currently action_required with zero jobs, so a maintainer needs to approve the runs before CI can start.

Could @r4victor and @un-def review #4124, and @un-def and @peterschmidt85 review #4125? I will address CI or review feedback promptly. The remaining dstack dev orchestration and lifecycle formatting slices will stay separate as proposed here.

[r4victor]

@wolegechu, the dstack apply -y contract is that it reconciles the resource's state with the configuration. It can do in-place update to re-create the resource if the update is not possible. This is the same way terraform apply -auto-approve works. I agree that auto-destroying the resource may not be desirable so we can introduce a way to prevent re-create either via a CLI flag or a configuration parameter like prevent_destroy in terraform.

On the other hand, it's not obvious why you'd run dstack apply -y and want "fail before mutation for owner mismatch, drift, pending/terminating state, in-place update". What are the use cases when you change the run configuration and run dstack apply -y but don't want dstack to reconcile the resource state? Please expand