Support system database sharing (application names) - #471
Open
devhawk wants to merge 3 commits into
Open
Conversation
Ports python#809 / ts#1326 / go#444+#445: every DBOS object -- workflows, steps, queues, schedules, application versions -- records the application that owns it, so several applications can share one system database in isolation, or interoperate deliberately by naming each other's objects. NULL means unclaimed: the columns are added nullable with no default, so every pre-existing row, and every row an SDK that does not know the column writes, belongs to all of them. Reads match `application_name = ? OR application_name IS NULL` throughout. Schema: migrations 100-107, the start of a cross-SDK shared history where every SDK defines the same migration at the same index. This language's own history is padded out to 99; the runner and the generated script skip the padding without a round trip each and record it in one write. Scoped: listings (workflows, queues, schedules, versions, aggregates, metrics), the queue dequeue and its flow-control counts, the latest-version lookup, the delayed-workflow and garbage-collection sweeps, and the debounce holder lookup. Recovery follows, since it lists by executor ID -- which defaults to the literal `local` in all four SDKs, so two unnamed applications on one database would otherwise recover each other's work. Unscoped, deliberately: everything addressed by workflow ID. A fork is the forking application's own work, steps included. Registering a queue, schedule, or application version under a name another application holds raises DBOSApplicationNameConflictException: names are shared address space, so a collision is not ours to resolve. The application name is hashed into the computed application version, so two applications built from one jar do not collide on a single version row. Existing deployments therefore compute a new version on upgrade. DBOSClient takes an optional application name; without one it owns nothing and sees every application's rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The other four read types this change touches -- WorkflowStatus, Queue, WorkflowSchedule, VersionInfo -- expose the application that owns the row. StepInfo was the odd one out, which also cost export/import fidelity: it is what carries steps through the export, so importing collapsed every step onto its workflow's owner. The two normally agree, but they need not: a workflow one application owns can be resumed or restarted by another addressing it by ID -- deliberately unscoped -- and the steps that run then belong to the application that ran them. An import now keeps the owner each step was exported with, falling back to the workflow's for an export predating the column. Python and Go do not expose this on their step type, so this is Java reading one column further than they do, not a divergence in what is stored. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fallback to the workflow's owner invented ownership the payload did not
carry. Python (`output.get("application_name")`) and TypeScript
(`output.application_name ?? null`) both take the exported value verbatim, so
an export predating the column imports its steps unclaimed -- which is the
representation's own rule: NULL means unclaimed, and no writer should guess an
owner for a row it did not stamp.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #468. Port of python#809, ts#1326, and go #444/#445.
Every DBOS object — workflows, steps, queues, schedules, application versions — records the application that owns it, so several applications can share one system database in isolation, or interoperate deliberately by naming each other's objects.
NULLmeans unclaimed. The columns are added nullable with no default, so every pre-existing row, and every row written by an SDK that does not know the column, belongs to all of them. Reads matchapplication_name = ? OR application_name IS NULLthroughout. That is what makes the migration safe, and also what makes the isolation one-directional: naming your applications hides them from each other, but not from an executor that ignores the column.Schema
Migrations 100–107, the start of a cross-SDK shared history: from index 100 on, every SDK defines the same migration at the same index. This language's own history (1–47) is padded out to 99, and both the runner and
dbos migrate --printskip the padding without a round trip each, recording it in a single write.application_nameonworkflow_status,queues,workflow_schedules,application_versions,operation_outputsenqueue_workflowgains a trailingapplication_name; every parameter is defaulted, so callers that omit it still resolve to itversion_name's global uniqueness (which is not dropped — not until every SDK reaching a shared database is past 107)What is scoped
Listings (workflows, queues, schedules, versions, aggregates, metrics), the queue dequeue and its rate-limit and concurrency counts, the latest-version lookup, the delayed-workflow and garbage-collection sweeps, and the debounce holder lookup. Recovery follows from the workflow listing — which matters, because
executor_iddefaults to the literallocalin all four SDKs, so two unnamed applications sharing a database would otherwise each treat the other's PENDING workflows as their own to recover.Deliberately not scoped: anything addressed by workflow ID. A fork is the forking application's own work, steps included.
Registering a queue, schedule, or application version under a name another application holds raises
DBOSApplicationNameConflictException— names are shared address space, so a collision is not ours to resolve.Behaviour changes worth knowing
DBOSClienttakes an optional application name; without one it owns nothing and sees every application's rows.application_nameon workflow, queue, schedule, and version rows, and accepts it as a filter on the list and aggregate requests. A Conductor that predates the field sends nothing, which leaves every listing scoped to the application.Not included
rename-application(re-owning rows after a rename, or adopting unclaimed ones) is a follow-up, matching Go. Until it lands, the collision error points at the Python or TypeScript CLI, which operates on the shared system database whatever wrote the rows.Testing
New
ApplicationNameTestruns two applications against one system database: stamping across all five tables, listing isolation and the explicit cross-application filter, unclaimed rows visible to both, dequeue isolation, fork ownership, the three collision errors, version scoping, and a nameless client.MigrationManagerTestcovers upgrading a database recorded at 47 through the padding to 107. Full suite green.🤖 Generated with Claude Code