refactor(backend): the two contract schemas the tracks meet through by lassebenni · Pull Request #5 · HackYourFuture/final-project-template

lassebenni · 2026-08-11T09:58:23Z

What I built

V2__analytics_schema.sql: the two schemas the backend and the data pipeline meet through, the roles that cross them, and the grants that keep each side out of the other's internals.

Why this approach

Two contract schemas, each owned by the side that writes it.

Schema	Written by	Read by	Contains
`public`	this application	this application only	our internal tables
`app`	this application	`app_reader`	views we choose to expose
`analytics`	`analytics_writer`	`${app_role}`	the data team's published marts

The earlier version let the pipeline select straight from public. That coupled their nightly run to our internal schema: renaming a column here would break their 6am job, and nobody would notice until the numbers went stale. A view is a promise, and changing one is a visible act.

It also moves the hashing of personal data to where it belongs. Previously the pipeline read raw user ids and discarded them afterwards. Now the view hashes, so the raw value never leaves this database at all. That is a better answer to who is responsible for it.

analytics_reader is now app_reader, because it never read analytics. It reads this application's data.

No passwords in the migration. A password in a migration is a password in git, and passing one as a placeholder only moves where it is written down: it lands inside a single-quoted SQL literal, so a rotated password containing a quote breaks the migration in a way that is hard to diagnose. The roles are created able to log in but unable to authenticate until a password is set out of band, which is what the provisioning script does when it writes them to Key Vault. One placeholder remains, ${app_role}.

Contract impact

This is the contract. Nothing existing changes: public is untouched and no application table is altered. The pipeline side is in #9.

How to run

Needs CREATEROLE, so it runs as the Entra administrator rather than hyfadmin. See the header.

psql -d <db> -v ON_ERROR_STOP=1 -f V2__analytics_schema.sql   # with ${app_role} substituted

Self-check

I ran this and it works
Tests pass locally
No secrets, tokens, or connection strings in the diff
This pull request does one thing

Verified, not assumed

Against the live PostgreSQL 16 server, on a scratch database, dropped afterwards:

Behaviour	Result
Migration runs on an empty database	exit 0
`analytics_writer` publishes a mart	1 row written
`${app_role}` reads it, having granted itself nothing	1 row
`${app_role}` tries to delete it	`permission denied for table fct_postings`
`app_reader` reads `app.saved_jobs`	`user_hash=a1d0c6e8…`, raw id never exposed
`app_reader` tries `public.saved_jobs`	`permission denied for table saved_jobs`

What the review found

Two things the tests only found by failing:

The app schema needed two ALTER DEFAULT PRIVILEGES, not one. A view created by a later migration belongs to the migration runner, while a view created by the running application belongs to ${app_role}. Default privileges cover one creator each, so naming only ${app_role} left migration-created views invisible to the sync, as a permission error on a view plainly sitting there.
hyfadmin has neither CREATEROLE nor CREATEDB on this server. ERROR: permission denied to create role. The Entra administrator is the identity that has them, and the header now says so.

Copilot's five comments are addressed and resolved. Stas' four are still open and unanswered, including the one about whether this migration should create app_role itself, which is a decision about the application's own login rather than about this boundary.

🤖 Generated with Claude Code

The data team publishes marts into an analytics schema next to the application tables, and reads application rows back through a separate read-only role. Both roles and the schema belong in the backend repo as a migration, so the setup is reproducible rather than a console command someone ran once. ALTER DEFAULT PRIVILEGES names the writer role. Without FOR ROLE the grant only covers tables created by whoever ran the migration, and the sync creates a new table on every publish, so the application would be locked out of its own marts. Reproduced against PG16 and verified fixed.

stasel · 2026-08-11T14:16:34Z

+--   flyway.placeholders.app_role=                  the role the backend logs in as
+--   flyway.placeholders.analytics_writer_password=
+--   flyway.placeholders.analytics_reader_password=


Can you also add those value to the config file? Together with an env variables

stasel · 2026-08-11T14:17:48Z

+--   flyway.placeholders.analytics_writer_password=
+--   flyway.placeholders.analytics_reader_password=
+
+CREATE SCHEMA IF NOT EXISTS analytics;


That's an interesting schema name analytics - doesn't sound very "data engineering" related to me. Is this a standard name?

stasel · 2026-08-11T14:19:07Z

+        CREATE ROLE analytics_writer LOGIN PASSWORD '${analytics_writer_password}';
+    END IF;
+    IF NOT EXISTS (SELECT FROM pg_roles WHERE rolname = 'analytics_reader') THEN
+        CREATE ROLE analytics_reader LOGIN PASSWORD '${analytics_reader_password}';
+    END IF;
+END
+$$;


Do we need the analytics reader role? Because the app_role already has a read only access to the analytics

Copilot

Pull request overview

Adds a Flyway migration to establish a consistent database security boundary between the backend application and the analytics/data pipeline by creating an analytics schema, dedicated roles, and grants/default-privileges.

Changes:

Introduces analytics schema plus analytics_writer/analytics_reader roles.
Grants read-only access for the app role to analytics marts, and sets default privileges so newly published marts remain readable.
Grants read access for the analytics reader role to current and future application tables in public.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

stasel · 2026-08-11T14:24:30Z

+--
+-- Three placeholders. Set them in your Flyway configuration and take the
+-- values from Key Vault. Never put a password in this file: it is committed.
+--   flyway.placeholders.app_role=                  the role the backend logs in as


Maybe it's a good idea to create the app_role in this script as well. and replace this with app_role_password.

Otherwise, we either set admin as the app_role or manually create the role

…ation Reworks the boundary after review. Three changes, each verified against the live PostgreSQL 16 server rather than reasoned about. The pipeline no longer reads public. It reads an app schema of views this application chooses to expose, mirroring the analytics schema it publishes into. Each side owns the schema it writes and neither reads the other's internals, so renaming a column here cannot break their 6am run. It also moves the hashing of personal data into our view, where it belongs: the raw user id never leaves this database rather than being discarded by someone else afterwards. analytics_reader is renamed app_reader, because it never read analytics. It reads the application's data, and the old name invited exactly the question it got in review. The password placeholders are gone. Passing a password through a placeholder only moves the problem: it lands inside a single-quoted SQL literal, so a rotated password containing a quote breaks the migration in a way that is hard to diagnose. Roles are created able to log in but unable to authenticate until a password is set out of band, which is what the provisioning script does when it writes them to Key Vault. One placeholder remains, app_role. Schemas now declare an owner. Without AUTHORIZATION the schema belongs to whoever ran the migration, and if that is the application's own login then the application can drop the marts it is only meant to read. The app schema needed two ALTER DEFAULT PRIVILEGES rather than one, which the test found by failing: a view created by a migration belongs to the migration runner, not to app_role, so naming only app_role left it invisible to the sync as a permission error on a view plainly sitting there. Verified end to end on a scratch database: writer publishes, the app reads the mart without ever granting itself anything, the app cannot delete it, the pipeline reads the view and sees only a hash, and the pipeline is refused on public. Scratch database and roles dropped afterwards. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Without it the migration cannot run at all. Flyway refuses an unresolved placeholder, so the application context fails to start and every test fails with 'No value provided for placeholder: ${app_role}'. The previous version left this to whoever deployed it, which meant the build was red for everyone. The default is CURRENT_USER, the role running the migration. That is what makes it work with no configuration in tests and in local development, where the application and the migration are the same login. Verified that PostgreSQL accepts CURRENT_USER as a role specification in all four places the migration uses it: GRANT, ALTER DEFAULT PRIVILEGES FOR ROLE, and CREATE SCHEMA AUTHORIZATION. Set APP_ROLE wherever the two differ, so the grants land on the role that serves requests rather than the one that happened to run the migration. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…s-schema # Conflicts: # backend/src/main/resources/application.yaml

stasel · 2026-08-12T10:36:42Z

Closing the PR in favor of a DB setup script outside of flyway

stasel reviewed Aug 11, 2026

View reviewed changes

stasel requested a lite review from Copilot August 11, 2026 14:25

Copilot started reviewing on behalf of stasel August 11, 2026 14:25 View session

Copilot AI reviewed Aug 11, 2026

View reviewed changes

stasel reviewed Aug 11, 2026

View reviewed changes

lassebenni mentioned this pull request Aug 11, 2026

feat(data): one data starter for the monorepo, on the stack the teams will run #9

Merged

4 tasks

lassebenni changed the title ~~feat(backend): ship the analytics boundary as a Flyway migration~~ refactor(backend): the two contract schemas the tracks meet through Aug 11, 2026

Lasse Benninga and others added 2 commits August 12, 2026 01:16

Merge remote-tracking branch 'origin/main' into feat/backend-analytic…

b78bb3e

…s-schema # Conflicts: # backend/src/main/resources/application.yaml

stasel closed this Aug 12, 2026

stasel deleted the feat/backend-analytics-schema branch August 13, 2026 10:35

Navigation Menu

Uh oh!

refactor(backend): the two contract schemas the tracks meet through - #5

refactor(backend): the two contract schemas the tracks meet through#5
lassebenni wants to merge 4 commits into
mainfrom
feat/backend-analytics-schema

lassebenni commented Aug 11, 2026 •

edited

Loading

Uh oh!

stasel Aug 11, 2026

Uh oh!

stasel Aug 11, 2026

Uh oh!

stasel Aug 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stasel Aug 11, 2026

Uh oh!

stasel commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

lassebenni commented Aug 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What I built

Why this approach

Contract impact

How to run

Self-check

Verified, not assumed

What the review found

Uh oh!

stasel Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

stasel Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

stasel Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stasel Aug 11, 2026

Choose a reason for hiding this comment

Uh oh!

stasel commented Aug 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lassebenni commented Aug 11, 2026 •

edited

Loading