Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,36 @@ All notable changes to AgentFlow are documented in this file.

## [Unreleased]

### Changed — spec/seed number consistency: daily rate, GTIN check digits, band centering, FX honesty (G2 S3, 2026-07-06)

- **Seed daily rate now matches §1 (audit m5).** `satellite_seed*.sql` order
dates spread over a ~122-hour (≈ 5.1-day) flat window instead of 21 days:
10,000 orders ≈ 1,965/day — generator-spec §1's baseline rate. §11 now
documents that §4's monthly seasonality is deliberately not encoded in a
5-baseline-day seed (a 5-day snapshot cannot express a 12-month curve).
- **Vault-seed GTINs are valid GTIN-13 (audit m6).** `synthetic_seed.sql`'s
`gs1_gtin` values now append the genuine GS1 mod-10 check digit via a
pinned 160-digit string, asserted against `reference/gs1.py`'s
`gtin13_check_digit` by a new invariant test — §12 #7 now holds for the
vault seed too, not only the reference catalog.
- **Amount bands re-centered on §1's average checks (audit m7).** The old
equal-width bands ran ~5% (dxb: ~12%) above target; new bands are centered
and multipliers re-chosen so small branch slices equidistribute:
marketplace 1.5k–2.8k (mean ≈ 2,150), D2C 2k–4.6k (≈ 3,300), B2B RU
30k–74k (≈ 52k), dxb 60k–120k (≈ 90k), ala 25k–65k (≈ 45k).
`postgres_oltp/seed.sql` mirrors the same formulas.
- **§12 #4 no longer contradicts §1; invariant tests tightened (audit m4).**
§12 #4 now claims the order-weighted aggregate B2B avg check (≈ 54.9k ∈
[30k, 80k]) and names dxb's 90k export-pallet check as the by-design
outlier. Tests now assert the aggregate averages (not only per-branch
proxies) and pin the spec-fixed defaults: 160 SKUs, 30 suppliers
(22 CN + 5 RU + 2 AE + 1 KZ), sourcing coverage for every SKU.
- **§10 FX constants declared documentation-only (audit n2).** §1/§10 now
state that every branch is seeded in ₽ and no generator or seed performs an
FX conversion at runtime; the pinned AED/KZT/CNY constants remain in
`reference/legend.py` solely as the fixed basis for doc/evidence-level
conversions.

### Removed — X5 Retail Hero loader deleted; at-scale benchmark retired as historical (G2 S2b, 2026-07-05)

- **Deleted `warehouse/agentflow/dv2/loaders/x5_retail_hero/`** in full
Expand Down
41 changes: 31 additions & 10 deletions docs/generator-spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,10 @@ Consumers, in execution order:

## 1. Master matrix — baseline day

All money **net of VAT, in ₽** (branch-local currency stored per §10; demo FX
constants convert). "Baseline day" = seasonal multiplier 1.0; the seasonal
calendar (§4) modulates it and averages to exactly 1.0 over the year.
All money **net of VAT, in ₽** — every branch is seeded in ₽; the pinned demo
FX constants of §10 are documentation-only. "Baseline day" = seasonal
multiplier 1.0; the seasonal calendar (§4) modulates it and averages to
exactly 1.0 over the year.

| Channel | Branch | Orders/day | Avg check, ₽ | Revenue, ₽/day |
| ------- | ------ | ---------: | -----------: | -------------: |
Expand Down Expand Up @@ -257,11 +258,16 @@ legend. Targets:

## 10. Currencies and determinism

- Vault-side branch currencies: RU = RUB, dxb = AED, ala = KZT. **Pinned demo
- **All seeded amounts are ₽, in every branch.** In the legend narrative dxb
invoices in AED and ala in KZT, but the v1 seeds and the vault store only
the ₽ figures of §1 — no generator or seed performs an FX conversion at
runtime, and cross-branch aggregates work directly in ₽. The **pinned demo
FX constants** (not live rates; internally consistent with a 90 ₽/USD
world): `AED = 24.50 ₽`, `KZT = 0.175 ₽`, `CNY = 12.40 ₽`. All cross-branch
aggregates in docs/evidence use these constants; write them wherever a
conversion happens.
world): `AED = 24.50 ₽`, `KZT = 0.175 ₽`, `CNY = 12.40 ₽` — kept in
`reference/legend.py` solely as the fixed conversion basis for any
doc/evidence sentence that quotes a non-₽ figure (e.g. FOB in CNY). If a
future revision stores branch-local currencies, these are the constants it
must use.
- Generator seed constant stays `20260626`; everything derives
deterministically from it. Timestamps keep today's mechanics (relative
`NOW()` in serving demo, `load_ts = now64()` in vault seeds).
Expand All @@ -281,6 +287,13 @@ Target row counts for the rebuilt `synthetic_seed.sql` + satellites
| `hub_marking_code` | 160 SKU GTINs + ~12,000 per-unit code sample (≈ one container), statuses issued 25 / in_circulation 60 / withdrawn 15 | per-product only |
| `hub_supplier` | 30 | 40 |

Order dates spread uniformly over a ~122-hour (≈ 5.1-day) window ending at
load time — 10,000 orders / 5.1 days ≈ **1,965 orders/day**, exactly §1's
baseline rate. Baseline days carry seasonal multiplier 1.0 by definition, so
§4's monthly curves are deliberately **not** encoded in this seed: a 5-day
snapshot cannot express a 12-month shape; the seasonality belongs to the
long-horizon generator narrative, not the vault seed.

Customer→branch and order→channel assignments follow §1/§7 proportions; the
`multiIf(number % 100 < …)` slicing technique stays, only the cut points
move. Order `record_source` reflects the channel: `mp__msk` (marketplace
Expand All @@ -295,11 +308,19 @@ this list is the definition of "цифры взаимно согласованы
1. Annual revenue (Σ channels × 365 × seasonal avg 1.0) ∈ **[3.5, 5.0] B ₽**.
2. Order-count mix: marketplaces 88–90%, B2B 7–9%, D2C 2–4%.
3. Revenue mix: B2B 65–72% of ₽; marketplaces 27–33%.
4. Avg B2B check ∈ [30k, 80k] ₽; avg marketplace check ∈ [1.5k, 3.0k] ₽ —
the AOV distribution is bimodal with no mass between 10k and 25k.
4. Order-weighted avg B2B check (all B2B branches together) ∈ [30k, 80k] ₽ —
§1 puts it at ≈ 54.9k. Per-branch B2B avg checks span 45k (ala) to 90k
(dxb): the RU + EAEU wholesale channels each sit inside [30k, 80k], while
dxb's 90k export-pallet check sits above that band **by design** (§1) and
is not a violation. Avg marketplace check ∈ [1.5k, 3.0k] ₽. The AOV
distribution is bimodal with no channel average between 10k and 25k.
5. Per SKU: FOB < landed < wholesale < marketplace-net < RRC (§5 ladder).
6. Each seasonal curve's 12 multipliers average exactly 1.0.
7. Every GTIN passes `is_valid_gtin13`, prefix ∈ 460–469.
7. Every GTIN passes `is_valid_gtin13`, prefix ∈ 460–469 — both the
reference-catalog GTINs (minted via `gs1.make_gtin13`) and the vault
seed's `gs1_gtin` literals in `synthetic_seed.sql`, whose check digits are
precomputed with the same GS1 mod-10 algorithm and pinned by the invariant
tests.
8. Every `tnved_code` is one of the §3 headings, 10-digit zero-padded form.
9. Dealer accounts × ordering frequency ⇒ 150–200 B2B orders/day.
10. Branch revenue shares sum to 100%; msk ∈ [55%, 65%].
Expand Down
14 changes: 14 additions & 0 deletions tests/unit/test_dv2_supplier_reference.py
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,20 @@ def test_tnved_headings_are_real_format():
# --- generator ---------------------------------------------------------------


def test_build_reference_default_counts_match_spec():
# generator-spec.md pins the defaults: §3 — 160 SKUs; §6 — 30 suppliers
# split 22 CN + 5 RU + 2 AE + 1 KZ. build_reference() must reproduce them
# exactly (a silent tax-id collision drop in _make_suppliers would shrink
# the supplier list — this pin catches that too).
tables = build_reference()
assert len(tables.products) == 160
assert len(tables.suppliers) == 30
by_country: dict[str, int] = {}
for supplier in tables.suppliers:
by_country[supplier.tax_country_code] = by_country.get(supplier.tax_country_code, 0) + 1
assert by_country == {"CN": 22, "RU": 5, "AE": 2, "KZ": 1}


def test_build_reference_is_deterministic():
a = build_reference(n_suppliers=25, n_products=120, seed=7)
b = build_reference(n_suppliers=25, n_products=120, seed=7)
Expand Down
62 changes: 52 additions & 10 deletions tests/unit/test_generator_spec_invariants.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@

from warehouse.agentflow.dv2.reference import legend
from warehouse.agentflow.dv2.reference.generator import build_reference
from warehouse.agentflow.dv2.reference.gs1 import is_valid_gtin13
from warehouse.agentflow.dv2.reference.gs1 import gtin13_check_digit, is_valid_gtin13
from warehouse.agentflow.dv2.reference.tnved import TNVED_HEADINGS

DV2_ROOT = Path(__file__).resolve().parents[2] / "warehouse" / "agentflow" / "dv2"
Expand Down Expand Up @@ -92,21 +92,34 @@ def test_invariant_3_revenue_mix():


def test_invariant_4_bimodal_avg_checks_no_mass_in_gap():
# §1's master matrix pins dxb (re-export, "export pallets", thinner
# margin per §5) at a 90k avg check — outside the general [30k, 80k] B2B
# band the same table implies for the domestic + EAEU wholesale channels.
# Read narrowly: the [30k, 80k] band covers RU + ala wholesale; dxb is a
# documented, table-explicit outlier, not a spec violation.
# §12 #4's primary claim is the *order-weighted aggregate*: avg B2B check
# across all B2B branches together ∈ [30k, 80k] ₽ (§1 puts it at ≈54.9k).
# Per-branch checks span 45k (ala) to 90k (dxb export pallets) — the RU +
# EAEU wholesale channels each sit inside the band, dxb's 90k sits above
# it by design (§1) and is not a violation.
b2b_rows = [
(orders, check)
for channel, _, orders, check in legend.MASTER_MATRIX
if channel in _B2B_CHANNELS
]
b2b_avg_check = sum(o * c for o, c in b2b_rows) / sum(o for o, _ in b2b_rows)
assert 30_000 <= b2b_avg_check <= 80_000

mp_rows = [
(orders, check)
for channel, _, orders, check in legend.MASTER_MATRIX
if channel in _MARKETPLACE_CHANNELS
]
mp_avg_check = sum(o * c for o, c in mp_rows) / sum(o for o, _ in mp_rows)
assert 1_500 <= mp_avg_check <= 3_000

# per-branch letter of §12 #4: RU + EAEU wholesale inside [30k, 80k].
domestic_b2b_checks = [
check
for channel, branch, _, check in legend.MASTER_MATRIX
if channel in _B2B_CHANNELS and branch != "dxb"
]
marketplace_checks = [
check for channel, _, _, check in legend.MASTER_MATRIX if channel in _MARKETPLACE_CHANNELS
]
assert all(30_000 <= c <= 80_000 for c in domestic_b2b_checks)
assert all(1_500 <= c <= 3_000 for c in marketplace_checks)
# no channel's avg check falls in the 10k-25k dead zone (holds for all
# channels, including dxb)
assert all(not (10_000 < check < 25_000) for _, _, _, check in legend.MASTER_MATRIX)
Expand All @@ -124,6 +137,12 @@ def test_invariant_5_pricing_ladder_bands_are_disjoint_and_ordered():

def test_invariant_5_pricing_ladder_holds_per_sku():
tables = build_reference()
# Pin the default catalog shape so the per-SKU loop cannot pass vacuously:
# §3 fixes 160 SKUs, §6 fixes 30 suppliers and 1-2 sources per SKU.
assert len(tables.products) == 160
assert len(tables.suppliers) == 30
sourced_skus = {s.product_bk for s in tables.sourcing}
assert sourced_skus == {p.product_bk for p in tables.products}
rrc_by_sku = {p.product_bk: p.rrc_price for p in tables.products}
for sourcing in tables.sourcing:
rrc = rrc_by_sku[sourcing.product_bk]
Expand All @@ -146,11 +165,34 @@ def test_invariant_6_seasonal_curves_average_to_one():

def test_invariant_7_every_gtin_valid_and_in_eaeu_range():
tables = build_reference()
assert len(tables.products) == 160 # §3: fixed catalog size — loop is not vacuous
for product in tables.products:
assert is_valid_gtin13(product.gtin)
assert 460 <= int(product.gtin[:3]) <= 469


def test_invariant_7_seed_sql_gtin_check_digit_string_is_genuine():
"""synthetic_seed.sql mints its 160 vault-seed GTIN stems as
``concat(460 + k % 10, lpad(200000 + k * 617, 9, '0'))`` and appends the
k-th character of a pinned 160-digit string as the check digit. Recompute
that string with the genuine GS1 mod-10 algorithm and assert the SQL
carries exactly it — so the seed's GTINs satisfy §12 #7 and cannot drift
from ``gs1.gtin13_check_digit`` silently. If the stem formula changes,
the structural asserts below fail first and point here.
"""
seed_sql = (DV2_ROOT / "synthetic_seed.sql").read_text(encoding="utf-8")
# stem formula guards (both the 160-template and the per-unit block)
assert "lpad(toString(200000 + number * 617), 9, '0')" in seed_sql
assert "lpad(toString(200000 + (number % 160) * 617), 9, '0')" in seed_sql
stems = [f"{460 + k % 10}{200000 + k * 617:09d}" for k in range(160)]
check_digits = "".join(str(gtin13_check_digit(stem)) for stem in stems)
assert check_digits in seed_sql
for stem, digit in zip(stems, check_digits, strict=True):
gtin = stem + digit
assert is_valid_gtin13(gtin)
assert 460 <= int(gtin[:3]) <= 469


# --- #8 tnved headings ----------------------------------------------------------


Expand Down
20 changes: 10 additions & 10 deletions warehouse/agentflow/dv2/postgres_oltp/seed.sql
Original file line number Diff line number Diff line change
Expand Up @@ -53,10 +53,10 @@ CREATE TABLE IF NOT EXISTS ops_dxb.orders (

-- ============ Seed: 50 msk customers + 200 msk orders ============
-- Channels / statuses / amounts mirror satellite_seed.sql (generator-spec.md
-- §1/§2): msk carries the marketplace-dominant mix (marketplace 1.5k-3k, d2c
-- 2k-5k, b2b 30k-80k ₽ net-of-VAT), status ladder pending/confirmed/shipped/
-- delivered/cancelled at 8/10/12/62/8. Amounts stay clear of the 10k-25k
-- bimodality dead-zone (§12 #4).
-- §1/§2): msk carries the marketplace-dominant mix (marketplace 1.5k-2.8k,
-- d2c 2k-4.6k, b2b 30k-74k ₽ net-of-VAT — bands centered on §1's avg checks),
-- status ladder pending/confirmed/shipped/delivered/cancelled at 8/10/12/62/8.
-- Amounts stay clear of the 10k-25k bimodality dead-zone (§12 #4).
INSERT INTO ops_msk.customers (customer_id, first_name, last_name, email, phone)
SELECT
'CUST-MSK-' || lpad(n::text, 4, '0'),
Expand Down Expand Up @@ -85,18 +85,18 @@ SELECT
ELSE 'cancelled'
END,
CASE
WHEN n <= 186 THEN (1500 + (n * 17) % 1501)::numeric(18, 2) -- marketplace 1.5k-3.0k
WHEN n <= 193 THEN (2000 + (n * 23) % 3001)::numeric(18, 2) -- d2c 2.0k-5.0k
ELSE (30000 + (n * 137) % 50001)::numeric(18, 2) -- b2b msk 30k-80k
WHEN n <= 186 THEN (1500 + (n * 17) % 1301)::numeric(18, 2) -- marketplace 1.5k-2.8k
WHEN n <= 193 THEN (2000 + (n * 37) % 2601)::numeric(18, 2) -- d2c 2.0k-4.6k
ELSE (30000 + (n * 329) % 44001)::numeric(18, 2) -- b2b msk 30k-74k
END
FROM generate_series(1, 200) AS n
ON CONFLICT (order_id) DO NOTHING;

-- ============ Seed: 20 dxb customers + 80 dxb orders ============
-- dxb is the b2b re-export branch (generator-spec.md §1: no marketplace/D2C
-- volume). All orders are 'b2b'; amounts follow the export-pallet band
-- (60k-130k ₽ net, mirrors satellite_seed_all_branches.sql), well above the
-- 10k-25k dead-zone.
-- (60k-120k ₽ net, centered on §1's 90k avg check, mirrors
-- satellite_seed_all_branches.sql), well above the 10k-25k dead-zone.
INSERT INTO ops_dxb.customers (customer_id, first_name, last_name, email, phone)
SELECT
'CUST-DXB-' || lpad(n::text, 4, '0'),
Expand All @@ -120,6 +120,6 @@ SELECT
WHEN n % 100 < 92 THEN 'delivered'
ELSE 'cancelled'
END,
(60000 + (n * 191) % 70001)::numeric(18, 2) -- b2b dxb export pallets 60k-130k
(60000 + (n * 355) % 60001)::numeric(18, 2) -- b2b dxb export pallets 60k-120k
FROM generate_series(1, 80) AS n
ON CONFLICT (order_id) DO NOTHING;
14 changes: 7 additions & 7 deletions warehouse/agentflow/dv2/satellite_seed.sql
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ SELECT
now64(3) AS load_ts,
MD5(concat(order_bk, '|hdr|v1')) AS hash_diff,
'bitrix__msk' AS record_source,
now64(3) - toIntervalHour((number * 7) % (24 * 21)) AS order_date,
now64(3) - toIntervalHour((number * 7) % 122) AS order_date, -- ≈5.1-day flat window: 10,000 orders / 122 h ≈ 1,965/day (§1/§11)
channel,
multiIf(
number % 100 < 8, 'pending',
Expand All @@ -113,9 +113,9 @@ FROM (
) AS order_bk,
multiIf(number < 8900, 'marketplace', number < 9180, 'd2c', 'b2b') AS channel,
multiIf(
number < 8900, toDecimal64(1500 + (number * 17) % 1501, 2), -- marketplace: 1.5k-3.0k
number < 9180, toDecimal64(2000 + (number * 23) % 3001, 2), -- D2C: 2.0k-5.0k
toDecimal64(30000 + (number * 137) % 50001, 2) -- B2B msk: 30k-80k
number < 8900, toDecimal64(1500 + (number * 17) % 1301, 2), -- marketplace: 1.5k-2.8k, mean ≈2,150 (§1)
number < 9180, toDecimal64(2000 + (number * 37) % 2601, 2), -- D2C: 2.0k-4.6k, mean ≈3,300 (§1)
toDecimal64(30000 + (number * 329) % 44001, 2) -- B2B msk: 30k-74k, mean ≈52k (§1)
) AS total_amount
FROM numbers(9540)
);
Expand Down Expand Up @@ -143,9 +143,9 @@ FROM (
'__', lpad(toString(number), 7, '0')
) AS order_bk,
multiIf(
number < 8900, toDecimal64(1500 + (number * 17) % 1501, 2),
number < 9180, toDecimal64(2000 + (number * 23) % 3001, 2),
toDecimal64(30000 + (number * 137) % 50001, 2)
number < 8900, toDecimal64(1500 + (number * 17) % 1301, 2),
number < 9180, toDecimal64(2000 + (number * 37) % 2601, 2),
toDecimal64(30000 + (number * 329) % 44001, 2)
) AS subtotal_amount
FROM numbers(9540)
);
Loading