From 1d33e7be3f271d1300d91c10ca556018b9e72807 Mon Sep 17 00:00:00 2001 From: Milind Srivastava Date: Sun, 28 Jun 2026 15:48:09 -0400 Subject: [PATCH] docs: added some design docs on optimization problem formulation for asap-planner --- .design_docs/optimizer-mip-formulation.md | 190 +++++++++++ .../optimizer-v1-implementation-plan.md | 306 ++++++++++++++++++ 2 files changed, 496 insertions(+) create mode 100644 .design_docs/optimizer-mip-formulation.md create mode 100644 .design_docs/optimizer-v1-implementation-plan.md diff --git a/.design_docs/optimizer-mip-formulation.md b/.design_docs/optimizer-mip-formulation.md new file mode 100644 index 0000000..c654c1c --- /dev/null +++ b/.design_docs/optimizer-mip-formulation.md @@ -0,0 +1,190 @@ +# Sketch Config Selection: MIP Formulation + +## Sets + +| Symbol | Definition | +|--------|-----------| +| $A = \{a_1, \ldots, a_m\}$ | All AQEs (Atomic Query Expressions), deduplicated across all RQEs | +| $R = \{r_1, \ldots, r_n\}$ | All RQEs (Repeating Query Expressions); each $r = (\text{QE}_r, T_r)$ | +| $R_a \subseteq R$ | RQEs that reference AQE $a$ | +| $G$ | Candidate streaming configs, exhaustively enumerated over all combinations of $(s, p, W, S)$ from parameter ranges derived from the AQE workload. Each $g = (s, p, \tau, W, S, n_g, d_g, \text{metric}, L, \phi)$ where $s$ = sketch family, $p$ = params, $\tau \in \{\text{tumbling}, \text{sliding}\}$, $W$ = window size, $S$ = slide interval ($S = W$ for tumbling), $n_g$ = number of windows used at query time, $d_g$ = physical storage depth (number of completed windows retained), $L$ = label set, $\phi$ = spatial filter (a filter on label values / metadata restricting which time series are ingested into this config). For every $a \in A$, $\text{EXACT}_a \in G$. | + +**Relationship between $n_g$ and $d_g$:** +- Tumbling: $d_g = n_g$ — every retained window is used at query time. +- Sliding: $d_g \geq (n_g - 1)(W_g/S_g) + 1$ — to retrieve $n_g$ non-overlapping windows spaced $W_g$ apart, the system must retain $(n_g-1)(W_g/S_g)+1$ completed windows in storage; only $n_g$ of those are loaded at query time. The cost formulation charges for the $n_g$ loaded windows only; the remaining $d_g - n_g$ retained-but-unused windows are storage overhead not modeled in $\text{Mem\_query}$. For sliding, $n_g$ is a config-generation parameter used to size $d_g$; the optimizer operates on $d_g$ directly via (RETs) and does not reference $n_g$. + +## Inputs + +| Symbol | Source | Definition | +|--------|--------|-----------| +| $T_r$ | RQE | Repeat interval for RQE $r$ (ms) | +| $\varepsilon_a$ | User-specified | Accuracy tolerance for AQE $a$ | +| $\text{SLA}_a$ | User-specified | Latency budget for AQE $a$ (default $\infty$) | +| $\text{range}_a$ | AQE | Lookback duration baked into $a$'s range vector | +| $\theta_a = (N_a, D_a)$ | Profiled | $N_a$ = cardinality; $D_a$ = data distribution, family-specific: Zipf (with parameter $\alpha$), Gaussian (with parameters $\mu, \sigma$), or empirical trace | +| $\rho_g$ | Measured from Prometheus | Item arrival rate (items/sec) for $g$'s metric and label filter | +| $N_g$ | Measured from Prometheus | Number of distinct label-group combinations for $g$'s metric and label filter; used when $\lnot\,\text{subpop\_aware}(s)$ to determine how many sketch instances config $g$ must maintain | +| $\text{AtomicCosts}(s, p)$ | sketch-bench | $\text{mem}(s,p),\ \text{ins}(s,p),\ \text{mrg}(s,p),\ \text{sub}(s,p),\ \text{qry}(s,p)$ | +| $w = (w_1, w_2, w_3, w_4)$ | User-specified | Weights for ingest CPU, ingest memory, query CPU, query memory | + +## Sketch Properties (per family $s$) + +| Property | Meaning | +|----------|---------| +| $\text{mergeable}(s)$ | Two instances can be merged into one representing their union | +| $\text{subtractable}(s)$ | Incremental update is defined: the current query result can be derived from the previous result by adding new data and subtracting evicted data, requiring O(1) sketch operations regardless of $n$ | +| $\text{subpop\_aware}(s)$ | One instance handles all label-group keys internally | + +## Derived Quantities + +**Query frequency:** +$$f_a = \sum_{r \in R_a} \frac{1}{T_r} \quad \text{(queries/sec)}$$ + +*(Assumes no cross-RQE result reuse: simultaneous RQEs referencing the same AQE each count as independent query work.)* + +**Label-group multiplier:** +$$N(s, g) = \begin{cases} 1 & \text{if } \text{subpop\_aware}(s) \\ N_g & \text{otherwise} \end{cases}$$ + +where $N_g$ is measured from Prometheus for $g$'s metric and label filter (see Inputs). *(Requires a profiling step before the optimizer runs — see Open Items.)* + +**Windows needed to cover $a$'s range from config $g$:** +$$n(a, g) = \left\lceil \frac{\text{range}_a}{W_g} \right\rceil$$ + +**Query method** — fully determined by $(\tau_g, n(a,g), s)$; not a decision variable. For sliding configs, only those with $S_g \mid W_g$ (slide divides window size) are included in $G$, ensuring the $n$ non-overlapping windows used in a merge/subtract land on exact slide boundaries. + +| Condition | Method | +|-----------|--------| +| $\text{EXACT}_a$ | Exact | +| $n(a,g) = 1$ | Direct | +| $n(a,g) > 1$, $\text{subtractable}(s)$ | Subtract | +| $n(a,g) > 1$, $\text{mergeable}(s)$, not subtractable | Merge | +| $n(a,g) > 1$, neither property | Infeasible | + +When both Subtract and Merge apply ($n(a,g) > 1$, $\text{subtractable}(s)$, $\text{mergeable}(s)$), Subtract takes priority: it requires $O(1)$ sketch operations regardless of $n(a,g)$ vs. $O(n(a,g))$ merges for Merge. The CPU advantage compounds as $n(a,g)$ grows; Mem\_query is the same ($2 \cdot N(s,g) \cdot \text{mem}$) in both cases. + +The Subtract method uses the incremental update formula: let $[t_0, t_1]$ be the current query window ($t_1 - t_0 = \text{range}_a$). Given the previous query result $T[t_0 - T_r,\, t_1 - T_r]$ already in the $+1$ $\text{Mem\_active}$ slot, the newly completed window $T[t_1 - T_r,\, t_1]$ (loaded from storage), and the evicted window $T[t_0 - T_r,\, t_0]$ (loaded from storage), compute $T[t_0, t_1] = T[t_0-T_r, t_1-T_r] + T[t_1-T_r, t_1] - T[t_0-T_r, t_0]$. This applies to both tumbling and sliding ingest types. + +## Feasibility Constraints + +Rather than folding all conditions into an opaque $\text{Feasible}(a,g)$ predicate, each condition is expressed as an explicit pre-computed parameter $c_{a,g} \in \{0,1\}$ that bounds $x_{a,g}$. $\text{Feasible}(a,g) = 1$ iff all of the following hold: + +**Capability matching:** +$$x_{a,g} \leq \mathbf{1}[\text{CapMatch}(a, g)] \tag{CAP}$$ + +$\text{CapMatch}(a,g) = 1$ iff $g$'s label set, metric, and spatial filter cover $a$'s requirements. Directional: a finer-grained $g$ can serve a coarser $a$ via merge at query time; the reverse is infeasible — once $g$ aggregates away a label, that information is gone. + +**Retention** (tumbling): +$$x_{a,g} \leq \mathbf{1}[n_g \geq n(a,g)] \quad \tau_g = \text{tumbling} \tag{RETt}$$ + +For tumbling, $n_g = d_g$ — the query working set and the storage depth are the same thing. This is equivalent to $d_g \geq n(a,g)$, which is the tumbling special case of (RETs) with $S_g = W_g$. + +**Retention** (sliding): +$$x_{a,g} \leq \mathbf{1}\!\left[d_g \geq (n(a,g)-1)\cdot\frac{W_g}{S_g} + 1\right] \quad \tau_g = \text{sliding} \tag{RETs}$$ + +For sliding, $n_g$ (query working set) and $d_g$ (storage depth) differ. To retrieve $n(a,g)$ non-overlapping windows spaced $W_g$ apart, the config must have retained at least $(n(a,g)-1)(W_g/S_g)+1$ completed windows. Note that $n_g \cdot W_g \geq \text{range}_a$ is trivially satisfied for sliding by the ceiling construction and gives no useful information — the binding constraint is on $d_g$. + +**Window size:** +$$x_{a,g} \leq \mathbf{1}[W_g \leq \text{range}_a] \tag{WIN}$$ + +A sketch must not over-cover the query range; data absorbed before $\text{range}_a$ ago cannot be extracted after the fact. + +*Implication for Direct queries:* $n(a,g) = 1$ requires $\lceil \text{range}_a / W_g \rceil = 1$, i.e. $W_g \geq \text{range}_a$. Combined with (WIN) this means the Direct method is only possible when $W_g = \text{range}_a$ exactly — Direct configs are uniquely sized per AQE. + +**Freshness** (ingest-type specific): +$$x_{a,g} \leq \mathbf{1}\!\left[W_g \leq \min_{r \in R_a} T_r\right] \quad \tau_g = \text{tumbling} \tag{FRESHt}$$ +$$x_{a,g} \leq \mathbf{1}\!\left[S_g \leq \min_{r \in R_a} T_r\right] \quad \tau_g = \text{sliding} \tag{FRESHs}$$ + +For tumbling, a completed window must exist for every query cycle, so $W \leq T_r$. For sliding, a new completed window appears every $S$ seconds, so the binding constraint is $S \leq T_r$ — $W$ can exceed $T_r$ for sliding. + +**Accuracy:** +$$x_{a,g} \leq \mathbf{1}[\text{Error}(a, g, \theta_a) \leq \varepsilon_a] \tag{ACC}$$ + +$\text{Error}(a, g, \theta_a)$ is an empirical lookup from sketch-bench, indexed by $(s, p, \theta_a)$. $\text{Error}(\text{EXACT}_a) = 0$ always. + +**Latency:** +$$x_{a,g} \leq \mathbf{1}[\text{Latency}(a, g) \leq \text{SLA}_a] \tag{LAT}$$ + +$\text{Latency}(a,g) \approx \text{CPU\_query}(a,g)$ *(to be confirmed against end-to-end sketch-bench measurements)*. + +**Valid query method:** +$$x_{a,g} \leq \mathbf{1}[\text{QueryMethodValid}(a, g)] \tag{QM}$$ + +$\text{QueryMethodValid}(a,g) = 1$ iff the query method derived from $(\tau_g, n(a,g), s)$ is not Infeasible in the table above. + +$\text{EXACT}_a$ satisfies all conditions for any $a$: $\text{Error} = 0$, $\text{Latency} = \text{exact query cost}$, no window constraints apply. + +## Cost Functions + +### Ingestion memory (steady-state active windows only) + +Retained completed windows are charged at query time via $\text{Mem\_query}$ below, not as a continuous allocation. + +$$\text{Mem\_active}(g) = \begin{cases} (1 + \mathbf{1}[\text{subtractable}(s)]) \cdot N(s,g) \cdot \text{mem}(s,p) & \tau_g = \text{tumbling} \\ (\lceil W_g / S_g \rceil + \mathbf{1}[\text{subtractable}(s)]) \cdot N(s,g) \cdot \text{mem}(s,p) & \tau_g = \text{sliding} \end{cases}$$ + +The $\mathbf{1}[\text{subtractable}(s)]$ term accounts for the one extra instance needed to store the previous query result for the Subtract incremental update. + +### Ingestion CPU (rate) + +$$\text{CPU\_ingest}(g) = \begin{cases} \rho_g \cdot \text{ins}(s,p) & \tau_g = \text{tumbling} \\ \rho_g \cdot \lceil W_g / S_g \rceil \cdot \text{ins}(s,p) & \tau_g = \text{sliding} \end{cases}$$ + +### Ingestion cost (rate — independent of AQE assignment) + +$$\text{IngestCost}(g) = w_1 \cdot \text{CPU\_ingest}(g) + w_2 \cdot \text{Mem\_active}(g)$$ + +### Query cost (per invocation — depends on query method) + +$\text{Mem\_query}$ is the transient peak working memory per query invocation (retained windows loaded from storage), not steady-state. + +| Method | $\text{CPU\_query}(a,g)$ | $\text{Mem\_query}(a,g)$ | +|--------|--------------------------|--------------------------| +| Direct | $N(s,g) \cdot \text{qry}(s,p)$ | $N(s,g) \cdot \text{mem}(s,p)$ | +| Merge | $N(s,g) \cdot \bigl((n(a,g)-1) \cdot \text{mrg}(s,p) + \text{qry}(s,p)\bigr)$ | $n(a,g) \cdot N(s,g) \cdot \text{mem}(s,p)$ | +| Subtract | $N(s,g) \cdot \bigl(\text{mrg}(s,p) + \text{sub}(s,p) + \text{qry}(s,p)\bigr)$ | $2 \cdot N(s,g) \cdot \text{mem}(s,p)$ | +| Exact | $c_{\text{exact}}$ (system constant) | $0$ | + +$$\text{QueryCost}(a,g) = w_3 \cdot \text{CPU\_query}(a,g) + w_4 \cdot \text{Mem\_query}(a,g)$$ + +## Decision Variables + +$$y_g \in \{0,1\} \quad \forall g \in G \qquad \text{(is config } g \text{ deployed)}$$ + +$$x_{a,g} \in \{0,1\} \quad \forall a \in A,\ g \in G \text{ s.t. } \text{Feasible}(a,g) = 1 \qquad \text{(is AQE } a \text{ served by config } g\text{)}$$ + +## Objective + +$$\min \sum_{g \in G} y_g \cdot \text{IngestCost}(g) \ + \ \sum_{\substack{a \in A,\ g \in G \\ \text{Feasible}(a,g)=1}} x_{a,g} \cdot f_a \cdot \text{QueryCost}(a,g)$$ + +Both terms are cost rates (cost/sec). $f_a$ converts the per-query $\text{QueryCost}$ into a rate commensurate with the continuously-accruing $\text{IngestCost}$. + +## MIP Constraints + +$$\sum_{\substack{g \in G \\ \text{Feasible}(a,g)=1}} x_{a,g} = 1 \qquad \forall a \in A \tag{1}$$ + +$$x_{a,g} \leq y_g \qquad \forall a \in A,\ g \in G \tag{2}$$ + +$$x_{a,g},\ y_g \in \{0,1\} \tag{3}$$ + +**(1)** Every AQE is assigned to exactly one config. Always satisfiable since $\text{EXACT}_a$ satisfies all feasibility conditions for any $a$. +**(2)** An AQE cannot be served by a config that is not deployed. +**(3)** Integrality. Feasibility is enforced by restricting $x_{a,g}$ to the domain where $\text{Feasible}(a,g) = 1$ — expanding into (CAP), (RET), (WIN), (FRESHt)/(FRESHs), (ACC), (LAT), (QM); all pre-computed, adding no solver complexity. + +## Problem Class + +Uncapacitated facility-location MIP: $y_g$ = "open facility $g$", $x_{a,g}$ = "assign demand $a$ to facility $g$", $\text{IngestCost}(g)$ = fixed facility cost, $f_a \cdot \text{QueryCost}(a,g)$ = assignment cost. Standard structure — solvable with off-the-shelf MIP solvers (CBC, OR-Tools, Gurobi). + +## Open Items + +1. **$\text{Latency}(a,g) \approx \text{CPU\_query}(a,g)$**: to be confirmed against end-to-end sketch-bench measurements. If other latency components (storage reads, network) dominate, this approximation needs revision. +2. **$N_g$ (distinct label-group count)**: for non-subpop-aware sketch families, $N(s,g) = N_g$ requires a profiling step (e.g. `count(metric{filters...})` from Prometheus) before the optimizer runs. +3. **$\theta_a$ profiling**: cardinality $N_a$ and distribution $D_a$ must be estimated before $\text{Error}(a,g,\theta_a)$ can be looked up from sketch-bench. Profiling strategy not yet designed. +4. **sketch-bench data gap**: empirical error and cost lookup requires cardinality and distribution shape to be swept parameters in sketch-bench benchmarks. This data does not yet exist and must be collected. +5. **Sliding storage overhead**: $d_g$ is the physical storage depth for sliding configs; only $n_g \leq d_g$ windows are loaded at query time. The $d_g - n_g$ retained-but-unused windows represent storage cost not captured in $\text{Mem\_query}$. +6. **Mem\_active subtractable scaling**: The $+\mathbf{1}[\text{subtractable}(s)]$ term in $\text{Mem\_active}(g)$ allocates one previous-result slot per config, but each AQE assigned to $g$ using the Subtract method needs its own slot (different range $\Rightarrow$ different previous result). With $k$ such AQEs, the correct term is $+k$, not $+1$. Long-term fix: move the per-previous-result cost into $\text{QueryCost}$ so it is charged per assignment. +7. **Subtract formula assumes $T_r = W_g$**: The incremental formula treats $T[t_1 - T_r, t_1]$ as a single newly completed window. When $T_r > W_g$ (allowed by FRESHt), this span covers $\lfloor T_r / W_g \rfloor$ completed windows; the Subtract cost (merging in new data and subtracting evicted data) then scales with $\lfloor T_r / W_g \rfloor$, not O(1). The current cost table silently assumes $T_r = W_g$. +8. **IngestCost(EXACT_a) is undefined**: $\text{EXACT}_a \in G$ participates in $\sum_g y_g \cdot \text{IngestCost}(g)$, but IngestCost is defined only for sketch configs (via $s, p, \rho_g, W_g, S_g$) — none of which exist for an EXACT config. Presumably $\text{IngestCost}(\text{EXACT}_a) = 0$ since Prometheus handles raw ingest, but this must be stated explicitly. + +## TODO + +1. **Label-dimension merge cost**: (CAP) allows a finer-grained $g$ to serve a coarser $a$ by aggregating away label dimensions at query time, but the query cost table has no entry for this. When $g$'s label set is a proper superset of $a$'s, querying $a$ from $g$ requires a group-by and reduce over the extra label values — a cost proportional to $N_g / N_a$ operations — currently silent in the model. Either add a label-merge cost term to $\text{CPU\_query}$, or restrict (CAP) to require exact label-set match. + +2. **FRESHt per-RQE vs. per-AQE**: (FRESHt) uses $\min_{r \in R_a} T_r$, banning any tumbling window $W_g > \min T_r$ for AQE $a$ even though slower RQEs could tolerate it. If $a$ is queried by a 1s RQE and a 60s RQE, all configs with $W_g > 1\text{s}$ are ruled infeasible, forcing small windows even when most query load runs at 60s cadence. Consider whether (FRESHt) should be relaxed per-$(a, r)$ pair — at the cost of introducing per-$(a, r, g)$ assignment variables. diff --git a/.design_docs/optimizer-v1-implementation-plan.md b/.design_docs/optimizer-v1-implementation-plan.md new file mode 100644 index 0000000..71fd625 --- /dev/null +++ b/.design_docs/optimizer-v1-implementation-plan.md @@ -0,0 +1,306 @@ +# ASAP Optimizer: V1 Implementation Plan + +**Branch**: `405-feat-optimization-based-sketch-config-selection` +**GitHub Issue**: #405 +**Design doc** (read first): `.design_docs/sketch-config-optimization-formulation.md` +**Module**: `asap-planner-rs/src/optimizer/` + +--- + +## Background + +`asap-planner-rs` currently hardcodes sketch/config decisions. This module replaces that +with an optimization that selects, for each atomic query expression (AQE), the best streaming +config to deploy (or falls back to exact/raw computation if no sketch is cost-effective). + +The optimizer output type is `OptimizerSolution`, which is then translated into the existing +`(StreamingConfig, InferenceConfig)` deployment artifacts. + +--- + +## Key Vocabulary + +- **RQE** (`Rqe`): a repeating query — one PromQL string + its repetition interval T_r. +- **AQE** (`Aqe`): a deduplicated atomic leaf aggregation extracted from RQEs. Has: + - `requirements: QueryRequirements` — metric, statistic, range, labels, filter + - `query_frequency_hz: f64` — Σ 1/T_r (MIP objective weight) + - `min_t_repeat_secs: u64` — min(T_r), bounds W ≤ min_t (freshness) + - `t_repeat_gcd_secs: u64` — GCD(T_r), natural slide interval S +- **g** (`CandidateConfig`): a candidate streaming config — agg type, params, window W, slide S, retention depth. +- **EXACT_a fallback**: always feasible; means raw Prometheus query at query time; zero ingest cost. + +### Three algebraic sketch properties (per `AggregationType`) + +| Property | Meaning | +|---|---| +| `mergeable` | Two instances can be combined → supports Merge query method | +| `subtractable` | Element-wise difference defined → supports Subtract (tumbling only) | +| `subpopulation_aware` | One instance handles multiple label-group keys → N(s,g)=1 | + +### Query method (not a free variable — derived from ingest_type × W vs range_a × algebra) + +| Ingest | W vs range_a | Algebra | Method | +|---|---|---|---| +| Any | W = range_a | any | Neither | +| Tumbling | W < range_a | mergeable | Merge{n=range_a/W} | +| Tumbling | W < range_a | subtractable | Subtract | +| Tumbling | W < range_a | neither | infeasible | +| Sliding | W < range_a | any | infeasible | + +### Window constraints + +- `W ≤ range_a` (no over-coverage) +- `W ≤ min_t_repeat_secs` (freshness — fastest dashboard is binding) +- Sliding forces W = range_a exactly + +### Cost model (objective) + +``` +minimize Σ_g y_g · IngestCost(g) + Σ_{a,g} x_{a,g} · f_a · QueryCost(a,g) +s.t. Σ_g x_{a,g} = 1 for all a (every AQE assigned) + x_{a,g} ≤ y_g for all a, g (only deploy what is used) + x_{a,g}, y_g ∈ {0,1} +``` + +`IngestCost(g)` must be independent of which AQEs use g (facility-location property). +`f_a = Σ 1/T_r` (sum, not GCD) — each dashboard independently queries the sketch. + +--- + +## File Map + +``` +asap-planner-rs/src/optimizer/ +├── mod.rs re-exports +├── solution.rs Aqe, QueryMethod, AqeAssignment, OptimizerSolution +├── translator.rs translate(&OptimizerSolution) → (StreamingConfig, InferenceConfig) +├── aqe_extractor.rs Rqe, extract_aqes() +├── pipeline.rs run_all_exact_pipeline(), run_greedy_pipeline() +├── sketch_properties.rs algebraic properties per AggregationType [Phase 2a, done] +├── candidate_gen.rs enumerate candidate configs per AQE [Phase 2b, done] +├── cost_model.rs ingest/query cost formulas [Phase 2c, done] +├── greedy.rs per-AQE greedy assignment [Phase 2e, done] +│ +│ ── TO BE ADDED ── +├── feasibility.rs Feasible(a,g) predicate — only meaningful once configs +│ are shared across AQEs; skipped in Phase 2 since every +│ candidate is already self-consistent by construction [Phase 3] +└── mip_solver.rs facility-location MIP (good_lp + coin_cbc) [Phase 3] +``` + +Existing infrastructure to reuse (do not re-implement): +- `asap_types::capability_matching::{compatible_agg_types, window_compatible, spatial_filter_compatible, topk_weighting_compatible, labels_compatible}` (PR #259) +- `asap_types::streaming_config::StreamingConfig`, `::inference_config::InferenceConfig`, `::aggregation_config::AggregationConfig` +- `asap_planner::planner::patterns::build_patterns()` — 5 PromQL pattern types +- `promql_utilities::query_logics::parsing::{get_metric_and_spatial_filter, get_statistics_to_compute, get_spatial_aggregation_output_labels}` + +--- + +## Phase Status + +### ✅ Phase 1 — Scaffolding (3 commits, all tests pass) + +**Commits**: `89d8bd7`, `4965ebf`, `9269ce7` + +What exists: +- `solution.rs`: `Aqe`, `QueryMethod`, `AqeAssignment`, `OptimizerSolution` (with `all_exact()` constructor) +- `translator.rs`: `translate()` — Phase 1 stub, emits empty `InferenceConfig` (TODO Phase 2) +- `aqe_extractor.rs`: `Rqe`, `extract_aqes()` with GCD/min/sum frequency tracking; `decompose_to_leaves()` splits binary arithmetic; `AqeKey` for deduplication +- `pipeline.rs`: `run_all_exact_pipeline(config, schema) → (StreamingConfig, InferenceConfig)` + - Flow: `ControllerConfig → config_to_rqes() → extract_aqes() → OptimizerSolution::all_exact() → translate()` + +**Result**: 98 existing + 7 new optimizer tests pass. All AQEs fall back to EXACT; no configs deployed. + +--- + +### ✅ Phase 2 — Per-AQE Greedy Sketch Selection (4 commits, all tests pass) + +Each AQE gets its own best config independently (no cross-AQE sharing). MIP sharing is Phase 3. + +**Commits**: `6899787` (2a+2b), `0d59c18` (2c), `cb2b9dc` (2e + pipeline/translator wiring). +2d (`feasibility.rs`) was deliberately skipped — see note below. + +#### 2a — `sketch_properties.rs` (done) + +```rust +pub struct SketchProperties { pub mergeable: bool, pub subtractable: bool, pub subpopulation_aware: bool } +pub fn sketch_properties(t: AggregationType) -> SketchProperties +``` + +Implemented values — `CountMinSketchWithHeap` ended up `mergeable=false` (not `true`/unclear as +originally guessed): the heap top-k list doesn't compose across merged/subtracted windows even +though the underlying CMS cells would. Everything else matches the original guess: +- `CountMinSketch`: mergeable=T, subtractable=T, subpopulation_aware=T +- `CountMinSketchWithHeap`: mergeable=F, subtractable=F, subpopulation_aware=T +- `DatasketchesKLL`, `HydraKLL`: mergeable=T, subtractable=F, subpopulation_aware=F (true for HydraKLL too) +- `Sum`/`MultipleSum`: mergeable=T, subtractable=T, subpopulation_aware=F/T respectively +- `HLL`, `SetAggregator`, `DeltaSetAggregator`: mergeable=T, subtractable=F, subpopulation_aware=F +- `MinMax`/`MultipleMinMax`, `Increase`/`MultipleIncrease`: mergeable=T, subtractable=F, subpopulation_aware=F/T +- `SingleSubpopulation`/`MultipleSubpopulation` (legacy wrapper types): all false (unknown, treated conservatively) + +When a type is both `mergeable` and `subtractable` (e.g. `Sum`, `CountMinSketch`), +`candidate_gen.rs` prefers `Subtract` — it's O(1) regardless of `n`, strictly cheaper than `Merge`'s O(n). + +#### 2b — `candidate_gen.rs` (done) + +```rust +pub struct CandidateConfig { + pub config: Option, // None = EXACT fallback + pub query_method: QueryMethod, + pub n_windows: u64, +} +pub fn enumerate_candidates(aqe: &Aqe, scrape_interval_secs: u64) -> Vec +``` + +Enumeration: `compatible_agg_types(stat)` × param grid (hardcoded small grids per sketch type, +see `CMS_DEPTHS`/`CMS_WIDTHS`/etc. constants — replace with sketch-bench sweep results in Phase 3) +× window candidates × ingest type: +- **Tumbling**: all `W` that divide `range_a`, are multiples of `scrape_interval_secs`, and + `≤ min(range_a, min_t_repeat_secs)`. Slide interval = `W` (tumbles by its own width). +- **Sliding**: `W = range_a` exactly is forced (overlapping sliding windows can't merge/subtract + to cover a larger range — only mutually-exclusive tumbling windows compose). But the slide + interval `S` is still a free choice ≤ `W`: smaller `S` means more concurrent overlapping + sub-windows maintained internally (`⌈W/S⌉`), costing more ingest CPU/mem for fresher results. + `S` doubles from `scrape_interval_secs` up to `W` to cover that tradeoff cheaply. +- Multi-statistic AQEs (e.g. `avg` = `[Sum, Count]`) return only the EXACT candidate — a single + sketch family can't serve two incompatible statistics in v1. +- Always appends an EXACT candidate (no config, `query_method = Exact`) — always feasible. + +Label granularity: only proposes configs at the label granularity the AQE itself needs +(reactive, no superset matching — that's Phase 3b). + +#### 2c — `cost_model.rs` (done) + +```rust +pub struct AtomicCosts { mem_bytes_per_instance, insert_cpu_secs, merge_cpu_secs, subtract_cpu_secs, + query_cpu_secs, exact_query_cpu_secs: f64 } +pub struct CostWeights { ingest_mem, ingest_cpu, query_mem, query_cpu: f64 } +pub fn ingest_cost(candidate: &CandidateConfig, rho_g: f64, costs: &AtomicCosts, weights: &CostWeights) -> f64 +pub fn query_cost(a: &Aqe, candidate: &CandidateConfig, costs: &AtomicCosts, weights: &CostWeights) -> f64 +pub fn total_cost_rate(a: &Aqe, candidate: &CandidateConfig, rho_g: f64, costs: &AtomicCosts, weights: &CostWeights) -> f64 +``` + +Implements the design doc's formulas with `N(s,g) = 1` hardcoded everywhere (real `N_g` — +distinct label-group count — needs Prometheus series-count profiling, not wired up; Phase 3). +`AtomicCosts` added `exact_query_cpu_secs` (not in the original plan) — without a non-zero cost +for the EXACT fallback's query, `IngestCost=0, QueryCost=0` would make EXACT always win trivially. + +**Important finding**: `CostWeights::default()` originally weighted memory (bytes) and CPU +(seconds) equally (`1.0` each) — but those aren't comparable units, and combining them at parity +made EXACT win regardless of workload (a sketch's flat memory-deployment cost always dwarfed any +per-query savings). Fixed by scaling memory weights down to `1e-9` relative to CPU weights, loosely +reflecting that RAM-held-over-time costs ~1e6x less per unit than CPU-time in real cloud pricing +(~$5/GB-month vs ~$0.04/vCPU-hour). This is still a stub, not real calibration — but it's at least +self-consistent enough that sketches can plausibly win when the workload calls for it. + +#### 2d — `feasibility.rs` — **skipped, deliberately** + +Originally planned as a `Feasible(a,g)` predicate wrapping `capability_matching.rs`. Not built: +every candidate `candidate_gen.rs` produces is already constructed *from* that exact AQE's own +labels/spatial-filter/metric/window constraints, so a feasibility check would always return `true` +at this phase — unexercised scaffolding. It becomes meaningful in **Phase 3**, once configs are +shared across AQEs (a config built for AQE X needs checking against AQE Y's requirements). Build +it then, reusing `window_compatible`, `spatial_filter_compatible`, `topk_weighting_compatible`, +`labels_compatible` from `asap_types::capability_matching` as originally planned. + +#### 2e — `greedy.rs` + `pipeline.rs` + `translator.rs` (done) + +```rust +pub fn greedy_assign(aqes: Vec, scrape_interval_secs: u64, rho_g: f64, + costs: &AtomicCosts, weights: &CostWeights) -> OptimizerSolution +``` + +For each AQE independently: `argmin` over `enumerate_candidates(aqe, ...)` by `total_cost_rate`. +No feasibility filter needed (see 2d note) — every candidate from `enumerate_candidates` is valid +for that AQE by construction. Assigns sequential `aggregation_id`s to deployed configs. + +`run_greedy_pipeline(config, schema, scrape_interval_secs, rho_g) -> (StreamingConfig, InferenceConfig)` +added to `pipeline.rs` alongside `run_all_exact_pipeline()`. `rho_g` is currently a single +placeholder value applied uniformly to every candidate — see open TODO below. + +`translator.rs::build_inference_config()` now populates `query_configs`: for each assignment with +a real `aggregation_id`, emits one `QueryConfig` per original query string, with +`num_aggregates_to_retain` from `retention_count_for_assignment()`. + +--- + +### 🔲 Phase 3 — Full MIP with Cross-AQE Sharing + +#### 3a — `mip_solver.rs` + +Add `good_lp` + `coin_cbc` to `Cargo.toml`. Implement facility-location MIP above. +Add `run_mip_pipeline()` to `pipeline.rs`. + +#### 3b — Label superset matching + +Relax `labels_compatible()` in `asap_types::capability_matching` at line 86 (TODO comment already there). +Allow a config with labels ⊇ query labels to serve that AQE. This is what enables cross-AQE sharing. + +#### 3c — sketch-bench cardinality sweep + +- Add cardinality to sweep grids in `sketch-bench` +- Add `CountMinSketchWithHeap` wrapper in `sketch-cli/src/wrappers/` +- Plug real `AtomicCosts` values into cost model + +#### 3d — Accuracy constraint + +Implement `Error(a,g,θ_a) ≤ ε_a` in `is_feasible()`: +- CMS: use analytic ε-δ guarantee +- Others: empirical lookup from sketch-bench +- Add θ_a profiling step (Prometheus series count API) before optimizer runs + +--- + +## Offline Testing — Not Wired Into the Real Planner + +**The optimizer is not used by `Controller::generate()`.** That method calls +`generator::generate_plan()` (the existing hardcoded path in `promql/generator.rs`) +unconditionally — confirmed by grepping for callers of `run_greedy_pipeline`/ +`run_all_exact_pipeline` outside the `optimizer` module: there are none. Running +`asap-planner` (the real CLI) today is completely unaffected by anything in this module. + +To test the optimizer against real workload YAMLs before wiring it in, use the +standalone `asap-optimizer-cli` binary (`asap-planner-rs/src/bin/optimizer_cli.rs`): + +``` +cargo run -p asap_planner --bin asap-optimizer-cli -- \ + --input_config \ + --prometheus_scrape_interval 60 \ + [--rho 1.0] +``` + +Takes the same `ControllerConfig` YAML format as `asap-planner --input_config` +(with a `metrics:` hints block for label schema — no live Prometheus needed). +Prints deployed streaming configs and query configs to stdout. `--rho` is the +placeholder arrival rate (see TODOs below — not real yet). + +Wire-in decision (deferred): once Phase 3 (MIP + feasibility + label superset +matching) lands, swap `Controller::generate()` to call `run_mip_pipeline()` +instead of `generator::generate_plan()`, likely behind an opt-in flag first. + +--- + +## Open TODOs in Code + +| File | Location | TODO | +|---|---|---| +| `aqe_extractor.rs` | `extract_requirements()` | Duplicates `build_query_requirements_promql` in `asap-query-engine/src/engines/simple_engine/promql.rs:614` — extract to shared free fn in `asap_types::query_requirements` | +| `capability_matching.rs` | `labels_compatible()`, line 86 | Relax to superset matching — do in Phase 3b | +| `cost_model.rs` | `ingest_cost()`, `query_cost()` | `N(s,g)` hardcoded to `1` everywhere; real `N_g` (distinct label-group count) needs Prometheus series-count profiling | +| `cost_model.rs` | `CostWeights::default()` | Self-consistent stub ratio (mem:cpu ≈ 1e-9:1), not real $/byte-sec vs $/cpu-sec calibration | +| `greedy.rs`, `pipeline.rs` | `rho_g` parameter | Single placeholder value applied uniformly; real per-config rates need Prometheus scrape-rate × active-series-count, not wired up | +| `translator.rs` | `retention_count_for_assignment(Subtract)` | Returns hardcoded `1`; should be the actual checkpoint count needed to cover the full lookback | +| `promql/generator.rs` | `generate_plan()` doc comment | Flags that `Controller::generate()` still uses the hardcoded path, not the optimizer — see "Offline Testing" section above | +| — | Accuracy constraint | No `Error(a,g) ≤ ε_a` check exists anywhere; nothing stops picking an under-provisioned sketch (Phase 3d) | +| — | sketch-bench | No cardinality sweep, no `CountMinSketchWithHeap` wrapper; `AtomicCosts` are still stub numbers (Phase 3c) | + +--- + +## Out of Scope for V1 + +- Hard cluster budget constraints (Σ Mem ≤ M_total) +- Workload drift / re-planning (static one-shot only) +- Per-AQE weight vectors (global w₁..w₄ only) +- Binary-op evaluation cost +- OneTemporalOneSpatial pipeline decomposition +- `ρ_g` as a new profiling mechanism (assume available or use placeholder)