feat(personal-context): generate & persist user personal context

[capJavert]

Write-only integration with the personal-context-engine (PCE):

• CDC on user-updated (portfolio change) triggers the website source
• better-auth account.create.after hook triggers the github source, resolving the login via the GitHub API from the linked ba_account
• shared requestPersonalContext helper upserts a pending row and publishes api.v1.generate-personal-context in one transaction
• result worker persists profileText + boost/mute tags, guarded by a per-(userId,source) correlationId stale check
• gated behind the opt-in personalContextEnabled remote config flag
• still not doing anything with it, just saving
• social links handling coming later once I confirm this works

[pulumi]

🍹 The Update (preview) for dailydotdev/api/prod (at a6b8c60) was successful.

Resource Changes

    Name                                                       Type                                  Operation
~   vpc-native-materialize-monthly-best-post-archives-cron     kubernetes:batch/v1:CronJob           update
~   vpc-native-worker-job-deployment                           kubernetes:apps/v1:Deployment         update
~   vpc-native-clean-expired-better-auth-sessions-cron         kubernetes:batch/v1:CronJob           update
~   vpc-native-hourly-notification-cron                        kubernetes:batch/v1:CronJob           update
~   vpc-native-post-analytics-history-day-clickhouse-cron      kubernetes:batch/v1:CronJob           update
~   vpc-native-user-profile-updated-sync-cron                  kubernetes:batch/v1:CronJob           update
~   vpc-native-temporal-deployment                             kubernetes:apps/v1:Deployment         update
~   vpc-native-clean-channel-highlights-cron                   kubernetes:batch/v1:CronJob           update
~   vpc-native-private-deployment                              kubernetes:apps/v1:Deployment         update
~   vpc-native-calculate-top-readers-cron                      kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-zombie-images-cron                        kubernetes:batch/v1:CronJob           update
~   vpc-native-update-tag-materialized-views-cron              kubernetes:batch/v1:CronJob           update
~   vpc-native-update-views-cron                               kubernetes:batch/v1:CronJob           update
~   vpc-native-update-trending-cron                            kubernetes:batch/v1:CronJob           update
~   vpc-native-sync-subscription-with-cio-cron                 kubernetes:batch/v1:CronJob           update
~   vpc-native-update-current-streak-cron                      kubernetes:batch/v1:CronJob           update
~   vpc-native-post-analytics-clickhouse-cron                  kubernetes:batch/v1:CronJob           update
~   vpc-native-personalized-digest-cron                        kubernetes:batch/v1:CronJob           update
~   vpc-native-channel-digests-cron                            kubernetes:batch/v1:CronJob           update
-   vpc-native-api-db-migration-5ff3281e                       kubernetes:batch/v1:Job               delete
+   vpc-native-api-clickhouse-migration-7ac75367               kubernetes:batch/v1:Job               create
~   vpc-native-daily-digest-cron                               kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-zombie-user-companies-cron                kubernetes:batch/v1:CronJob           update
~   vpc-native-rotate-weekly-quests-cron                       kubernetes:batch/v1:CronJob           update
~   vpc-native-channel-highlights-cron                         kubernetes:batch/v1:CronJob           update
~   vpc-native-update-highlighted-views-cron                   kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-stale-user-transactions-cron              kubernetes:batch/v1:CronJob           update
~   vpc-native-materialize-yearly-best-post-archives-cron      kubernetes:batch/v1:CronJob           update
~   vpc-native-personalized-digest-deployment                  kubernetes:apps/v1:Deployment         update
~   vpc-native-clean-zombie-opportunities-cron                 kubernetes:batch/v1:CronJob           update
+   api-sub-api.personal-context-generated                     gcp:pubsub/subscription:Subscription  create
~   vpc-native-bg-deployment                                   kubernetes:apps/v1:Deployment         update
~   vpc-native-clean-zombie-users-cron                         kubernetes:batch/v1:CronJob           update
~   vpc-native-subscription-anniversary-achievements-cron      kubernetes:batch/v1:CronJob           update
~   vpc-native-user-profile-analytics-history-clickhouse-cron  kubernetes:batch/v1:CronJob           update
~   vpc-native-clean-gifted-plus-cron                          kubernetes:batch/v1:CronJob           update
~   vpc-native-generic-referral-reminder-cron                  kubernetes:batch/v1:CronJob           update
~   vpc-native-rotate-daily-quests-cron                        kubernetes:batch/v1:CronJob           update
~   vpc-native-check-analytics-report-cron                     kubernetes:batch/v1:CronJob           update
~   vpc-native-squad-posts-analytics-refresh-cron              kubernetes:batch/v1:CronJob           update
+   api-sub-api.github-account-linked-personal-context         gcp:pubsub/subscription:Subscription  create
... and 19 other changes

[capJavert]

@claude review

[claude]

Claude finished @capJavert's task in 2m 36s —— View job

---

PR Review: Personal Context generation & persistence

Nicely scoped write-only integration — the correlationId stale guard, feature-flag gating, transactional upsert+publish, and the test coverage across all three workers are all solid. A few things worth addressing before this grows further.

Potential issues

1. Entity vs migration column-type mismatch (userId) — src/entity/user/UserPersonalContext.ts:26 declares @PrimaryColumn({ type: 'text' }), but the migration creates "userId" character varying (src/migration/1784811495083-UserPersonalContext.ts:8), and the referenced user.id is varchar(36) (src/entity/user/User.ts:116). The entity/migration drift means the next db:migrate:make will try to ALTER userId from varchar → text. Align the entity with the migration (e.g. @PrimaryColumn({ type: 'varchar', length: 36 }) to match user.id).
Fix this →

2. Event published inside the DB transaction, before commit — in src/common/personalContext/requestPersonalContext.ts:39, triggerTypedEvent runs as the last statement inside con.transaction(...). Two concerns:

• Ordering hazard: the PCE receives generate-personal-context before our row is committed. If it ever responds faster than our commit, personalContextGeneratedWorker's findOneBy({ userId, correlationId }) finds nothing and silently drops the result (src/workers/personalContext/personalContextGenerated.ts:20). Low risk today (PCE does LLM work), but fragile.
• It also holds the transaction open across network I/O.

Consider committing first, then publishing (outsource the publish to after the transaction resolves). The PR description frames the atomicity as intentional, so if the current ordering is deliberate, a comment explaining why would help.

3. personalContextGeneratedWorker isn't gated on personalContextEnabled — the other two workers early-return when the flag is off, but the result worker (src/workers/personalContext/personalContextGenerated.ts) always processes. In-flight results after a flag flip will still be persisted. Likely harmless (and arguably correct so you don't lose in-flight work), but worth a deliberate decision.

Suggestions

4. Duplicated GitHub-token lookup — the raw SELECT "accessToken" FROM ba_account ... in src/workers/personalContext/githubAccountLinked.ts:19 is now the third copy of this exact query (see src/schema/users.ts:4560, src/workers/userDeletionCleanup.ts:368 deletes it). Raw SQL is acceptable here since ba_account is better-auth-managed with no entity, but per CLAUDE.md ("when logic is needed in two places, export from the original location") this is worth extracting into a shared getGithubAccessToken(con, userId) helper.

5. Redundant index — IDX_user_personal_context_user_id on (userId) is redundant: the composite PK (userId, source) already provides a btree usable for userId-leading lookups. You can drop the separate index unless you have a specific reason.

6. Tag data duplicated — boostTags/muteTags are persisted as dedicated array columns and inside context.ranking_signals in the jsonb blob (personalContextGenerated.ts:34-37). Fine if the columns are for query/index performance, but keep in mind they can drift from context.

7. Stale fields on re-request — the upsert in requestPersonalContext.ts:29 flips status back to pending but doesn't clear the previous profileText/boostTags/muteTags/generatedAt. So a row can read pending while still exposing stale generated data. Probably intended (keep last-good until refresh completes), but flag it for whoever wires up the read path later.

Nits

• githubAccountLinked will retry indefinitely on GitHub API errors (expired/revoked token) until DLQ — acceptable given the retry model, but a revoked-token case may be worth handling explicitly later.

Overall this is clean and well-tested. Items 1 and 2 are the ones I'd resolve before merge.
· feat/personal-context

[capJavert]

@claude review again 3. and 4. and 7. are fine

[claude]

Claude finished @capJavert's task in 1m 8s —— View job

---

Re-review — personal context

Re-reviewed the current branch. Items 3, 4, 7 acknowledged as fine per your call — skipped.

The remaining items from the first pass are all resolved:

• Item 1 — userId column drift: fixed. src/entity/user/UserPersonalContext.ts:25 now uses @PrimaryColumn({ type: 'varchar', length: 36 }), matching the migration's character varying(36) and user.id. No drift.
• Item 2 — publish inside transaction: fixed. requestPersonalContext.ts no longer wraps the upsert + publish in con.transaction(...); the upsert runs first, then triggerTypedEvent after it resolves. Callers pass the worker's con (autocommit), so the row is committed before the event goes out — the ordering hazard is gone.
• Item 5 — redundant index: fixed. The migration (1784811495083) now creates only the composite PK (userId, source) and the FK; the separate IDX_user_personal_context_user_id is gone.
• Item 6 — tag column/jsonb duplication: fixed. The dedicated boostTags/muteTags columns were dropped from the entity; tags now live only inside the context jsonb (personalContextGenerated.ts:36). No duplication left to drift.

Nothing new surfaced in the delta — the account.create.after GitHub hook (betterAuth.ts:780), the schema, and the workers all read cleanly. LGTM. 👍
· feat/personal-context

[rebelchris]

Looks solid