diff --git a/.github/workflows/backup.yml b/.github/workflows/backup.yml index 8dc54622..4ce91fd6 100644 --- a/.github/workflows/backup.yml +++ b/.github/workflows/backup.yml @@ -64,7 +64,7 @@ jobs: INSERT INTO "{schema}".orders_v2 VALUES (?, ?, ?, ?, ?, CURRENT_TIMESTAMP) ''', - [f"ORD-{schema.upper()}-001", f"USR-{schema.upper()}", "confirmed", 42.0, "USD"], + [f"ORD-{schema.upper()}-001", f"USR-{schema.upper()}", "confirmed", 42.0, "RUB"], ) finally: conn.close() diff --git a/README.md b/README.md index 5ba59874..d438e323 100644 --- a/README.md +++ b/README.md @@ -183,7 +183,7 @@ audit-closure sprint: ClickHouse), PII protection moves from the removed app-level string-parse gate to engine-enforced vault governance (fail-closed column grants, per-jurisdiction officer roles, row policies, `SQL SECURITY DEFINER` - views — 32/32 live adversarial probes), plus the vendored NL→SQL + views — every live adversarial probe green), plus the vendored NL→SQL generation engine (LangGraph, routed through GraceKelly), the DV2 raw vault on PostgreSQL with `LISTEN`/`NOTIFY` freshness, the MinIO-backed PyIceberg catalog, and the OpenSSF Scorecard channel (5.8 → 7.0). diff --git a/docs/architecture.md b/docs/architecture.md index fd8083e9..47b5f8fd 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -184,8 +184,9 @@ See [Architecture Decision Records](decisions/) for detailed trade-off analysis. > string parse in the serving tier. **Executed 2026-07-02 (ADR 0006 Phase 2):** > `warehouse/agentflow/dv2/governance/` — fail-closed column grants for `dv2_analyst`, > per-jurisdiction `dv2_pii_officer__` roles, row policies on `hub_customer`, -> `SQL SECURITY DEFINER` MDM views, PII-free `customer_360`; verified live 32/32 -> (`docs/perf/vault-pii-governance-verify-2026-07-02.md`). `sql_guard` remains, scoped +> `SQL SECURITY DEFINER` MDM views, PII-free `customer_360`; verified live, 29/29 +> probes on the current script (`docs/perf/vault-pii-governance-verify-2026-07-03.md`). +> `sql_guard` remains, scoped > to what it can actually enforce: SELECT-only, no DML, the tenant table allow-list, and > the recursive-CTE shadow reject. Tracked in `road-to-9.8.md`. > diff --git a/docs/clickhouse-cutover-plan.md b/docs/clickhouse-cutover-plan.md index 65f09725..7b720c89 100644 --- a/docs/clickhouse-cutover-plan.md +++ b/docs/clickhouse-cutover-plan.md @@ -82,10 +82,10 @@ actually carry PII. - [x] Verify: the 3 historical bypass forms (COLUMNS-expr, whole-row struct-ref, column-rename-list) are dead at the engine — `COLUMNS` resolves to real columns and hits the missing grant (`ACCESS_DENIED`); the other two are - not expressible on ClickHouse at all (`UNKNOWN_IDENTIFIER`). 32/32 + not expressible on ClickHouse at all (`UNKNOWN_IDENTIFIER`). All 29 adversarial probes green on a live 26.7 server, incl. jurisdiction row-scoping and admin-unaffected checks: - `docs/perf/vault-pii-governance-verify-2026-07-02.md`. + `docs/perf/vault-pii-governance-verify-2026-07-03.md`. - ~~Introduce a schema-execution check for the PII gate~~ / ~~Reclassify the app-level gate~~ — obsolete: the app-level gate was removed with the serving PII layer (2026-07-01); there is no app-side PII surface diff --git a/docs/decisions/0006-fix-demo-serving-engine-on-clickhouse.md b/docs/decisions/0006-fix-demo-serving-engine-on-clickhouse.md index 73feaa0f..ba1071e7 100644 --- a/docs/decisions/0006-fix-demo-serving-engine-on-clickhouse.md +++ b/docs/decisions/0006-fix-demo-serving-engine-on-clickhouse.md @@ -109,8 +109,8 @@ Concretely: `marts.customer_360`. The separate schema-execution app check became moot when the app-level gate was removed with the empty serving PII surface: access control on resolved columns *is* the engine ground-truth. Verified - live, 32/32 adversarial probes: - `docs/perf/vault-pii-governance-verify-2026-07-02.md`. + live, 29/29 adversarial probes: + `docs/perf/vault-pii-governance-verify-2026-07-03.md`. - The semantic layer **keeps emitting DuckDB-flavored SQL** and relies on `ClickHouseBackend._translate_sql` to transpile — this stays a deliberate, documented layer rather than an accident. Making the semantic layer emit diff --git a/docs/domain.md b/docs/domain.md index 43b7f254..3e7377ea 100644 --- a/docs/domain.md +++ b/docs/domain.md @@ -209,7 +209,7 @@ renames. The two exceptions are listed in §5.4. | Change | Scope | Status | | ------ | ----- | ------ | | Former external-dataset record_source prefix → `mp__*` (+ governance SQL, officer probes, admission tests) | The prefix carried the name of a retired public benchmark dataset that the demo loader once replayed as transaction history. Under the legend it is the **consolidated marketplace feed**, and the prefix says so. The dataset and its loader have since been removed from the codebase entirely (see CHANGELOG.md) | **Done** (B2) | -| Demo value repin: currencies to `RUB` (primary), `AED`/`KZT` in branch stories; demo revenue/counts consistent with §1–2 | `contracts/entities/order.yaml` currency examples, NL demo answers, seeded `ORD-*` rows | Planned (data phase, after the generator spec) | +| Demo value repin: currencies to `RUB` (primary), `AED`/`KZT` in branch stories; demo revenue/counts consistent with §1–2 | `contracts/entities/order.yaml` currency examples, NL demo answers, seeded `ORD-*` rows | **Done** (S0/S1 — `RUB` is the house currency across contracts, demo answers, and seeds) | Vocabulary guardrails for all public docs: the company is an **own-brand / private-label importer** — always that framing; "store" (in `hub_store`) is diff --git a/docs/dv2-multi-branch/RELEASE_STATUS.md b/docs/dv2-multi-branch/RELEASE_STATUS.md index 3f642ded..e94eee4b 100644 --- a/docs/dv2-multi-branch/RELEASE_STATUS.md +++ b/docs/dv2-multi-branch/RELEASE_STATUS.md @@ -10,7 +10,7 @@ backend-routed event scan, dedicated CI E2E lane), PII protection moves from the removed app-level string-parse gate to engine-enforced vault governance (ADR 0006 Phase 2 — fail-closed column grants, per-jurisdiction officer roles, row policies, `SQL SECURITY DEFINER` -views; 32/32 live adversarial probes), plus the vendored NL→SQL +views; every live adversarial probe green), plus the vendored NL→SQL generation engine routed through GraceKelly (ADR 0008), the DV2 raw vault PostgreSQL migration with `LISTEN`/`NOTIFY` freshness, the MinIO-backed PyIceberg catalog, ADR 0009, and the OpenSSF Scorecard diff --git a/docs/dv2-multi-branch/demo_voiced.narration.txt b/docs/dv2-multi-branch/demo_voiced.narration.txt index a79415a7..0a417332 100644 --- a/docs/dv2-multi-branch/demo_voiced.narration.txt +++ b/docs/dv2-multi-branch/demo_voiced.narration.txt @@ -1,6 +1,6 @@ Импортёр мелкой кухонной техники под собственным брендом. Пять локаций — Москва, Питер, Екатеринбург, Дубай, Алматы. Три боли: кросс-канальный оверселл — один склад на все каналы, и партия дилеру должна попасть в остаток раньше, чем маркетплейсы допродадут те же единицы; контейнеры из Китая идут 40-60 дней, и застрявший на таможне груз — риск топ-3; операционка жонглирует пятью программами — 1С, Битрикс24, экран WMS, кабинеты маркетплейсов, Excel — ради ответа об одном заказе. Мы построили DV2.0 хранилище и операционный слой поверх: один ответ вместо пяти вкладок. Покажу. -Кластер. Три ноды Kubernetes, лейблы branch равно msk, dxb, ala. По ним nodeSelector закрепляет Postgres и ClickHouse — data sovereignty на уровне планировщика. +Кластер. Три ноды Kubernetes, все с лейблом branch равно msk — это HQ-демо. Postgres и ClickHouse закреплены через nodeSelector по лейблу workload; в проде тот же примитив закрепляет региональные edge-ноды branch равно dxb и ala — data sovereignty на уровне планировщика. Модель. Семьдесят таблиц: восемь хабов, восемь линков, сорок восемь сателлитов, шесть BV вьюх поверх. Сателлиты разрезаны по record_source равно система плюс филиал. Один customer_hk из 1С и из Битрикса — конфликт решается приоритетом, аудит сохраняется. diff --git a/docs/generator-spec.md b/docs/generator-spec.md index 369aed40..9f1fd6cf 100644 --- a/docs/generator-spec.md +++ b/docs/generator-spec.md @@ -165,8 +165,10 @@ magnitude. ## 6. Suppliers and sourcing (reference generator) -The current reference is grocery (dairy/bakery supplier stems, food brands, -gram weights, food ТН ВЭД headings) — it is replaced wholesale: +The reference was originally grocery-shaped (dairy/bakery supplier stems, +food brands, gram weights, food ТН ВЭД headings); it has since been replaced +wholesale with the kitchen-appliance reference specified below (see CHANGELOG +for the swap): - **30 suppliers**: 22 CN contract factories (Guangdong/Zhejiang-style names, e.g. "Foshan …", "Ningbo …", "Cixi … Electric Appliance Co., Ltd." — @@ -186,9 +188,9 @@ gram weights, food ТН ВЭД headings) — it is replaced wholesale: 460–469 is *correct* for an own-brand importer — GTINs belong to the RU brand owner registered with GS1 RUS, regardless of where manufacturing happens. The module docstring already records this rationale. -- **`tnved.py`**: grocery headings → the four appliance headings of §3 - (8516, 8509, 8423, 8422) with RU descriptions close to official wording, - heading-granularity honesty note preserved. +- **`tnved.py`**: the former grocery headings were replaced by the four + appliance headings of §3 (8516, 8509, 8423, 8422) with RU descriptions + close to official wording, heading-granularity honesty note preserved. ## 7. Customer populations diff --git a/monitoring/grafana/dashboards/merch-agent-journey.json b/monitoring/grafana/dashboards/merch-agent-journey.json index 817af473..18c6f8d7 100644 --- a/monitoring/grafana/dashboards/merch-agent-journey.json +++ b/monitoring/grafana/dashboards/merch-agent-journey.json @@ -22,7 +22,7 @@ ], "fieldConfig": { "defaults": { - "unit": "currencyUSD" + "unit": "currencyRUB" } } }, @@ -59,7 +59,7 @@ ], "fieldConfig": { "defaults": { - "unit": "currencyUSD" + "unit": "currencyRUB" } } }, diff --git a/scripts/benchmark_freshness.py b/scripts/benchmark_freshness.py index 058b5a0b..ffb0d321 100644 --- a/scripts/benchmark_freshness.py +++ b/scripts/benchmark_freshness.py @@ -184,7 +184,7 @@ def build_order_event(amount: Decimal, sequence: int) -> dict: OrderItem(product_id="PROD-001", quantity=1, unit_price=amount), ], total_amount=amount, - currency=Currency.USD, + currency=Currency.RUB, ) return json.loads(event.model_dump_json()) diff --git a/scripts/benchmark_freshness_realpath.py b/scripts/benchmark_freshness_realpath.py index 932d595d..7a52b8fb 100644 --- a/scripts/benchmark_freshness_realpath.py +++ b/scripts/benchmark_freshness_realpath.py @@ -71,7 +71,7 @@ def build_order_event(amount: Decimal, sequence: int) -> dict: status=OrderStatus.PENDING, items=[OrderItem(product_id="PROD-001", quantity=1, unit_price=amount)], total_amount=amount, - currency=Currency.USD, + currency=Currency.RUB, ) return json.loads(event.model_dump_json()) diff --git a/tests/unit/test_dv2_business_vault_ddl.py b/tests/unit/test_dv2_business_vault_ddl.py index d98a7f42..0a257718 100644 --- a/tests/unit/test_dv2_business_vault_ddl.py +++ b/tests/unit/test_dv2_business_vault_ddl.py @@ -36,8 +36,9 @@ def _strip_comments(sql: str) -> str: return sql -# B2 (brand-neutral restoration of a guard dropped during X5 removal, G2 -# audit): the header comment's illustrative "which source conventions does +# B2 (brand-neutral restoration of a guard dropped when the retired benchmark +# dataset and its loader were removed — G2 audit, dataset named in +# CHANGELOG.md): the header comment's illustrative "which source conventions does # this admit" example must only ever name known-good record_source prefixes. # Checked against an explicit allowlist rather than a literal bad string, so # this test never itself has to spell out the retired dataset's prefix to diff --git a/tests/unit/test_dv2_postgres_ddl.py b/tests/unit/test_dv2_postgres_ddl.py index 33c168b8..24d57753 100644 --- a/tests/unit/test_dv2_postgres_ddl.py +++ b/tests/unit/test_dv2_postgres_ddl.py @@ -44,8 +44,9 @@ def _strip_comments(sql: str) -> str: return sql -# B2 (brand-neutral restoration of a guard dropped during X5 removal, G2 -# audit): the header comment's illustrative "which source conventions does +# B2 (brand-neutral restoration of a guard dropped when the retired benchmark +# dataset and its loader were removed — G2 audit, dataset named in +# CHANGELOG.md): the header comment's illustrative "which source conventions does # this admit" example must only ever name known-good record_source prefixes. # Checked against an explicit allowlist rather than a literal bad string, so # this test never itself has to spell out the retired dataset's prefix to diff --git a/warehouse/agentflow/dv2/loaders/vault_rows.py b/warehouse/agentflow/dv2/loaders/vault_rows.py index cc00c638..19178c68 100644 --- a/warehouse/agentflow/dv2/loaders/vault_rows.py +++ b/warehouse/agentflow/dv2/loaders/vault_rows.py @@ -2,8 +2,9 @@ These pydantic models describe the hub / link / order-satellite shape of the per-branch order feed (``sat_order_header__1c__*`` / ``sat_order_pricing__1c__*``). -They were relocated here from the deleted X5 Retail Hero loader during its -removal (G2 audit S2b) because ``PostgresVaultWriter``'s own tests +They were relocated here from the now-deleted legacy external-dataset loader +during its removal (G2 audit S2b; the dataset is named in CHANGELOG.md) +because ``PostgresVaultWriter``'s own tests (``tests/unit/test_dv2_postgres_ingestion.py``) needed generic row shapes to keep running; that test file is their only importer today.