Description
Part 1, Track 2 of #36763 — teach the analytics ingest pipeline to read experiment context off the event payload and write it to the columns added by the schema issue.
Implemented in dotCMS/dot-ca-event-manager. Tracked here for sprint and team ownership.
No new endpoint, no contract change
Worth stating up front because the epic's framing suggests otherwise: nothing is created here. POST /v1/event/ingest already exists and is already the target of core's EventAnalyticsProxyResource. The URL, the auth, the batch shape, and the response contract all stay identical. What widens is only the payload model — one new optional block inside Context, and three more columns in the INSERT.
Payload shape
The experiment block sits in Context, not in event.data, so it is shared across every event in a batch. A visitor is in one experiment at a time, so attaching it once per batch is simpler than repeating it per event.
{
"context": {
"session_id": "abc-123",
"site_id": "site-xyz",
"user_id": "user-456",
"device": { "screen_resolution": "1920x1080", "language": "en" },
"experiment": {
"id": "exp-789",
"running_id": "run-001",
"variant": "B"
}
},
"events": [ ... ]
}
When no experiment is running the block is omitted entirely and the event is byte-identical to today's.
Files
Both verified present on main:
| File |
Change |
src/main/java/com/dotcms/analytics/ingestion/web/UserEventPayload.java |
Experiment nested record + experiment field on Context |
src/main/java/com/dotcms/analytics/ingestion/repository/SaveEventRepository.java |
3 columns and placeholders in INSERT_SQL; writes in insertChunk |
Follow the existing Device record for the nested-record pattern, and the existing str() null-to-empty-string helper for the writes so DEFAULT '' semantics hold even when a sub-field is missing.
Validation
The block is optional, but not partially optional. null when no experiment is running is fine; a block that is present with any of id, running_id, or variant missing or blank is a client bug and must be rejected rather than silently written as '' — a half-attributed event is worse than an unattributed one, because it lands in the experiment's result set with a missing dimension. Enforce with @NotBlank on all three fields so Bean Validation rejects before any DB write.
Acceptance Criteria
Payload model
Persistence
Tests — new ExperimentIngestionIT
A dedicated class extending AbstractIntegrationTest, following UserAgentIngestionIT. Auth and generic bean validation are already covered by EventIngestionControllerIT; this class covers only experiment-block behaviour.
Priority
High
Additional Context
Blocked by the schema issue — the three columns must exist before the INSERT can reference them.
This does not make experiment data flow end to end. Nothing sets context.experiment until the SDK work in Part 3, and that developer is unavailable this sprint. Until then the only producers are the synthetic fixture (tracked separately) and ExperimentIngestionIT itself. That is expected for this sprint, but it means "ingest works" is demonstrated by tests, not by real traffic.
Today's experiment events do not travel this path at all. They go browser → /api/v1/event → EventLogWebInterceptor → ExperimentEventsPayload → EventLogSubmitter → the Jitsu collector at analyticsApp.getAnalyticsProperties().analyticsWriteUrl(). Retiring that path is Part 3, not this issue.
Parent epic: #36763
Description
Part 1, Track 2 of #36763 — teach the analytics ingest pipeline to read experiment context off the event payload and write it to the columns added by the schema issue.
No new endpoint, no contract change
Worth stating up front because the epic's framing suggests otherwise: nothing is created here.
POST /v1/event/ingestalready exists and is already the target of core'sEventAnalyticsProxyResource. The URL, the auth, the batch shape, and the response contract all stay identical. What widens is only the payload model — one new optional block insideContext, and three more columns in the INSERT.Payload shape
The
experimentblock sits inContext, not inevent.data, so it is shared across every event in a batch. A visitor is in one experiment at a time, so attaching it once per batch is simpler than repeating it per event.{ "context": { "session_id": "abc-123", "site_id": "site-xyz", "user_id": "user-456", "device": { "screen_resolution": "1920x1080", "language": "en" }, "experiment": { "id": "exp-789", "running_id": "run-001", "variant": "B" } }, "events": [ ... ] }When no experiment is running the block is omitted entirely and the event is byte-identical to today's.
Files
Both verified present on
main:src/main/java/com/dotcms/analytics/ingestion/web/UserEventPayload.javaExperimentnested record +experimentfield onContextsrc/main/java/com/dotcms/analytics/ingestion/repository/SaveEventRepository.javaINSERT_SQL; writes ininsertChunkFollow the existing
Devicerecord for the nested-record pattern, and the existingstr()null-to-empty-string helper for the writes soDEFAULT ''semantics hold even when a sub-field is missing.Validation
The block is optional, but not partially optional.
nullwhen no experiment is running is fine; a block that is present with any ofid,running_id, orvariantmissing or blank is a client bug and must be rejected rather than silently written as''— a half-attributed event is worse than an unattributed one, because it lands in the experiment's result set with a missing dimension. Enforce with@NotBlankon all three fields so Bean Validation rejects before any DB write.Acceptance Criteria
Payload model
Experimentnested record added toUserEventPayloadwith@NotBlank @JsonPropertyonid,running_id,variant@JsonProperty("experiment") Experiment experimentadded to theContextrecordexperimentis optional at block level — a payload with no block is accepted unchangedPersistence
experiment_id,running_id,variantadded toINSERT_SQLwith matching?placeholdersinsertChunkwrites all three afterconversion_name, usingstr()so a missing sub-field becomes''rather than nullsrc/main/resources/analytics/validators/— they coverevent.datasub-maps only, and theexperimentblock lives inContext. Confirm this rather than assume it.Tests — new
ExperimentIngestionITA dedicated class extending
AbstractIntegrationTest, followingUserAgentIngestionIT. Auth and generic bean validation are already covered byEventIngestionControllerIT; this class covers only experiment-block behaviour.experimentblock → all three columns land correctly inanalytics.events;awaitCount+assertReplicatedCountconfirm replication to ch-02experimentblock → all three columns are'', no null, no erroridpresent,variantblank or absent) → 400, and nothing is written to the databaseContextPriority
High
Additional Context
Blocked by the schema issue — the three columns must exist before the INSERT can reference them.
This does not make experiment data flow end to end. Nothing sets
context.experimentuntil the SDK work in Part 3, and that developer is unavailable this sprint. Until then the only producers are the synthetic fixture (tracked separately) andExperimentIngestionITitself. That is expected for this sprint, but it means "ingest works" is demonstrated by tests, not by real traffic.Today's experiment events do not travel this path at all. They go
browser → /api/v1/event→EventLogWebInterceptor→ExperimentEventsPayload→EventLogSubmitter→ the Jitsu collector atanalyticsApp.getAnalyticsProperties().analyticsWriteUrl(). Retiring that path is Part 3, not this issue.Parent epic: #36763