Design Review: MIP Formulation for Sketch Config Selection
https://github.com/ProjectASAP/ASAPQuery/blob/main/.design_docs/optimizer-mip-formulation.md
1. Missing context / framing
Problem definition is underspecified upfront.
- What is a config? The term is used immediately but never explicitly defined before the formulation begins.
- What setting is this targeting? Be explicit — streaming/continuous vs. offline/oneshot? Predictable query cadence vs. ad hoc? Or is this intentionally generic?
- Design doc should be self-contained. Add a figure (or diagram) showing: inputs → system → outputs, along with the metric of success and the status quo baseline.
2. Terminology
Too many acronyms introduced at once (QE, AQE, RQE). Are there standard terms from the DB or streaming literature that readers would already know? If not, a brief glossary or a running example introduced early would help ground all three before the formulation.
Also missing: a high-level intuition for the solution approach before diving into the MIP. A 2–3 sentence "here's the shape of the solution" before the formal inputs would go a long way.
3. Problem abstraction / OR framing
Facility location may not be the right framing. The doc maps this to an uncapacitated facility-location MIP, but it's worth justifying why — is this the tightest known reduction, or just a convenient one? If the problem structure is actually closer to a set-cover or coverage problem, that matters for solver choice and approximation guarantees.
4. Cross-query reuse not modeled
This seems like a large omission. The doc explicitly assumes no cross-RQE result reuse (f_a treats every RQE invocation independently). But:
- Multiple stats over the same dataset, or multiple subexpressions sharing an AQE, are common in dashboarding and alerting workloads.
- The current model overestimates query cost in exactly these cases.
- A natural starting point: assume synchronous queries with the same period (the dashboarding/alerting model), solve that simpler case first, then generalize. This would likely produce the biggest wins with the least complexity.
Why was the simpler synchronous model not used as a baseline before going to the general formulation?
Related: if modeled as a coverage problem, cross-query reuse may fall out naturally (see item 4).
5. Config abstraction is unclear
What does a config actually do? After reading the full formulation it's still not clear whether a config:
- Takes raw data → produces a sketch summary per window, or
- Takes raw data → produces a query result directly
A simple abstraction layer would help: incoming data → [tumbling/sliding window] → sketch summary per window → query result. Which of these steps is "the config"? Is the config responsible for the summary, the result, or both?
6. Solution detail too early
The Derived Quantities / Query Method section dives into implementation before the problem is fully stated. The merge/subtract logic (Direct, Merge, Subtract, Exact) is part of the solution design space, not the problem formulation. Consider:
- State the problem (what we're optimizing, constraints, feasibility) at a high level first.
- Defer merge/subtract mechanics to a "Solution Details" section.
Also: is the subtraction/merging logic something that needs to be built (i.e., new code), or is it assumed to already exist? That distinction should be explicit.
Design Review: MIP Formulation for Sketch Config Selection
https://github.com/ProjectASAP/ASAPQuery/blob/main/.design_docs/optimizer-mip-formulation.md
1. Missing context / framing
Problem definition is underspecified upfront.
2. Terminology
Too many acronyms introduced at once (QE, AQE, RQE). Are there standard terms from the DB or streaming literature that readers would already know? If not, a brief glossary or a running example introduced early would help ground all three before the formulation.
Also missing: a high-level intuition for the solution approach before diving into the MIP. A 2–3 sentence "here's the shape of the solution" before the formal inputs would go a long way.
3. Problem abstraction / OR framing
Facility location may not be the right framing. The doc maps this to an uncapacitated facility-location MIP, but it's worth justifying why — is this the tightest known reduction, or just a convenient one? If the problem structure is actually closer to a set-cover or coverage problem, that matters for solver choice and approximation guarantees.
4. Cross-query reuse not modeled
This seems like a large omission. The doc explicitly assumes no cross-RQE result reuse (
f_atreats every RQE invocation independently). But:Why was the simpler synchronous model not used as a baseline before going to the general formulation?
Related: if modeled as a coverage problem, cross-query reuse may fall out naturally (see item 4).
5. Config abstraction is unclear
What does a config actually do? After reading the full formulation it's still not clear whether a config:
A simple abstraction layer would help:
incoming data → [tumbling/sliding window] → sketch summary per window → query result. Which of these steps is "the config"? Is the config responsible for the summary, the result, or both?6. Solution detail too early
The Derived Quantities / Query Method section dives into implementation before the problem is fully stated. The merge/subtract logic (Direct, Merge, Subtract, Exact) is part of the solution design space, not the problem formulation. Consider:
Also: is the subtraction/merging logic something that needs to be built (i.e., new code), or is it assumed to already exist? That distinction should be explicit.