fix(g2): B1 live-generator kitchen/RUB + M2/M3/m4/m5/m8 partial de-brand#153
Merged
Merged
Conversation
BLOCKER B1 from G2_AUDIT_REPORT.md. The live producer + edge emitter (activated on the F2 3-Space deploy) drove a generic USD electronics catalog into the serving витрина, overwriting the kitchen-legend SKUs, breaking bimodality and the out-of-stock-bestseller story. - event_producer.PRODUCT_CATALOG -> the 10 pinned kitchen SKUs (§9), RUB retail prices, each carrying its seeded stock. - Currency enum gains RUB (now the default for Order/Payment/Product events); USD/EUR/GBP kept valid for historical fixtures / cross-border. - generate_payment amount moved to the ₽ retail band (790-7990), deliberately below the 10k-25k bimodality dead-zone. - generate_product emits the SKU's seeded stock instead of randomising, so PROD-001 (kettle) stays out of stock under the live feed. - Serving-sink currency defaults 'USD'->'RUB' (local_pipeline DDL+upsert, clickhouse_sink, iceberg_sink x3). Consequence: the auto-generated stable order contracts drift, so config/contracts/order.v1+v2.yaml regenerated — currency enum +RUB and total_amount.unit USD->RUB (stale legacy annotation; orders were always RUB in the kitchen legend, so no version bump, breaking_changes stays []). Verified: runtime smoke (1200 events RUB-only; витрina mirrors §9 row-for- row; PROD-001 stays in_stock=False/qty=0 after 573 live orders). Unit suite 1665 passed, ruff+mypy clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MAJOR M5 + MINOR m1 from G2_AUDIT_REPORT.md. The bv_order_canonical smoke
seed carried GROSS header totals (total_amount = subtotal + tax), while the
production satellite_seed.sql and generator-spec.md §1 ("all money net of
VAT") use NET (total_amount mirrors the pricing subtotal, tax separate).
On ClickHouse this collapsed branch_pnl.gross_revenue == net_revenue.
- Every header total_amount now equals its order's pricing subtotal (net):
O1 2400->2000, O2 2600->2166.67 (winner) / 2100 / 2050, O3 3500->3000,
O4 60000->50000 (both versions), O5 48000->40000, O6 36000->30000,
O7 42000->40000. O8 (no pricing row) stays 30000 as a standalone net B2B.
- Dissolves m1: O3's header (3500) no longer disagrees with subtotal+tax
(3600) because total_amount IS the subtotal under the net convention.
- Re-pinned verify_bv_order.sh (Σ 224500.00 -> 197166.67; O2 winner
shipped|2600.00 -> shipped|2166.67) and README table + convention note.
Verified: DuckDB replication of the view's is_deleted=0 + argMax(load_ts)
collapse yields Σ=197166.67, O2=shipped|2166.67, O4=confirmed (tombstone
excluded). Postgres parse-gate tests/unit/test_dv2_postgres_ddl.py 28/28.
Live psql run stays the Mac-tail.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MAJOR M4 from G2_AUDIT_REPORT.md. postgres_oltp/seed.sql and fanout/02_seed.sql
carried channels (web/mobile/retail/call-center), statuses (new/paid/shipped/
refunded) and amounts (500-24,999 ₽ — mass inside the forbidden 10k-25k
bimodality dead-zone) that predate the legend, and dxb looked retail rather
than b2b re-export. These promote straight into bv_order_canonical.
postgres_oltp/seed.sql (feeds promote_to_raw_vault -> vault):
- Channels -> §2 vocabulary {marketplace, d2c, b2b}; msk is the marketplace-
dominant mix (186 mp / 7 d2c / 7 b2b), dxb is 'b2b' only.
- Statuses -> §2 ladder pending/confirmed/shipped/delivered/cancelled at the
production 8/10/12/62/8 weights.
- Amounts -> §1 bimodal bands (mp 1.5-3k, d2c 2-5k, b2b msk 30-80k, dxb
60-130k ₽ net), mirroring satellite_seed*.sql — none in the 10k-25k dead-zone.
fanout/02_seed.sql (per-branch topology proof): §2 statuses only; msk RUB
retail/d2c + a few b2b, dxb AED at export-pallet scale (2.5-5.1k AED); no
amount in the RUB or AED dead-zone.
Verified: arithmetic check confirms 0 dead-zone hits across all generated and
literal amounts, bands correct, every status in the §2 ladder; both files
parse under sqlglot's postgres dialect. No test pins the old seed values.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…g-code split
Autonomous-safe doc↔code accuracy batch from G2_AUDIT_REPORT.md.
- M6: domain.md §6 cited `/v1/analytics/*` for the e-com and data-engineer
personas, but the router is admin-gated — real path `/v1/admin/analytics/*`
(admin.py prefix `/admin` + require_admin_key; openapi confirms). Corrected
both cells and flagged the admin-key requirement.
- m11: §4 called the three ops surfaces "planned"; Order 360 timeline,
stuck-orders worklist and exception inbox are live (D2/D3/D4), already listed
as live endpoints in §6. Dropped "planned", named the live routes.
- n1: demo_evidence.md called all 12,160 marking codes "per-unit"; it's 160
SKU-level GTIN templates + 12,000 per-unit codes (§11 / synthetic_seed.sql).
Skipped m2 ("32/32"→29): not reproducible — the primary evidence doc
(vault-pii-governance-verify-2026-07-02.md) and every surviving citation
(README, clickhouse-cutover-plan) consistently say 32/32; the audit's "29"
and its RELEASE_STATUS.md/architecture.md line refs don't check out.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…urned' MINOR m3 from G2_AUDIT_REPORT.md. branch_pnl, returns_velocity and customer_360 counted `order_status = 'returned'`, but no order ever carries that status — the §2 vocabulary is pending/confirmed/shipped/delivered/ cancelled (OrderStatus enum, contracts, seeds). Every returns metric (returned_orders, returned_value, return_rate, returned_tax_unrecovered) was therefore a constant 0. The legend has no dedicated return status: its terminal-negative bucket 'cancelled' folds cancellations and marketplace returns together (§2 "cancel/return allowance"; satellite_seed.sql — "marketplace cancels dominate the last bucket"). Remapped the filters to 'cancelled' so the metrics measure that ~8% bucket. Column names kept (load-test queries + demo materials reference them); a comment documents the semantics in each model. Verified: no 'returned' filter remains; 'cancelled' is emitted by the seed status ladder (number%100>=92); test_dv2_governance_ddl (customer_360 PII-free contract) 9/9. Live dbt run stays the Mac-tail. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…tial) First slice of the X5 -> synthetic track (G2_AUDIT_REPORT.md). These sites describe the *vault data*, which is synthetic (synthetic_seed.sql), not the X5 load-test path — so the X5 branding was both off-legend and factually wrong, and removing it needs no loader decision: - reference/__init__.py (M3): "grocery reference ... transactional X5 feed" -> "kitchen-appliance reference ... transactional order feed". - bv_order_canonical.sql + postgres/03_business_vault.sql (m8): the "1C / X5 Retail Hero order headers, so real X5 volume flows through" comment claimed X5 volume in a view fed by synthetic order headers -> "1C order headers ... full order volume". - customer_360.sql: "At X5 scale" perf rationale -> "At benchmark scale". Deferred (loader-coupled, needs the replacement-shape decision): the x5_retail_hero loader itself, its by-name references in vault_mapping.py / load_postgres.py / pg_vault_writer.py, schema_dv2.md Sources table, dbt/README scale note, and the load-test infra. CHANGELOG history left as-is. Verified: postgres parse-gate 28/28; diffs minimal, no EOL flips. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MAJOR M2 from G2_AUDIT_REPORT.md. The public DV2 diagram titled the lane "Data Vault 2.0 на реальных данных X5" and drew the source node as "X5 Retail Hero CSV / 45.8M строк / Kaggle" — presenting a third-party retailer's dataset as the project's real data, which contradicts the finalized canon (desc_for_julia_fin.md: external source is a deliberate non-goal; the demo runs on synthetic data). Retitled to "на синтетических данных" and replaced the source node with "Синтетический генератор / кухонная легенда · ₽ / clients·products·purchases, детерминир. seed, масштабируемый объём". No fabricated row count — the scale-generator is a separate (Mac-tail) build. Remaining X5 sites are loader-coupled (schema_dv2 Sources table, vault_mapping/load_postgres/pg_vault_writer, the x5_retail_hero loader + its X5-grocery-shaped vault schemas, load-test infra) and belong to the loader replacement sub-project. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
DORA Metrics
MTTR note: No failed mainline CI runs in the selected window. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes ~55% of G2_AUDIT_REPORT.md findings (batch 1 of the new_plen_05_07_26.md plan, step S0):
dataflow.htmldiagram + loader-independent X5 references to a synthetic sourcecancelled, not the phantomreturnedstatetotal_amountcorrected to net-of-VAT, aligned with the prod seedM1-remainder (entity contracts still saying "in USD", notebooks, EUR/GBP fixtures) and the X5 loader removal are follow-up steps (S1/S2) — out of scope here.
Test plan
ruff check/ruff format --check— cleanmypy src/ --ignore-missing-imports— cleanpytest tests/unit/ tests/property/— 1677 passedpytest tests/integration/— 263 passed, 52 skipped (no live Kafka/ClickHouse/Postgres on this Windows box), 4 errors from Docker being unavailable locally (testcontainers — expected on this host, not a regression)scripts/export_openapi.py --check— no driftscripts/check_schema_evolution.py— passed