Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -16,24 +16,16 @@ Let it run for a representative period (hours to days). Each line is a JSON entr
{"params":{"query":"rate(http_requests_total[5m])","start":"2025-12-02T18:00:00Z","end":"2025-12-02T18:00:00Z","step":0},"ts":"2025-12-02T18:00:00.001Z"}
```

### 2. Create a metrics config

List the metrics you want ASAP to sketch:
```yaml
# metrics.yaml
metrics:
- metric: http_requests_total
labels: [instance, job, method, status]
- metric: node_cpu_seconds_total
labels: [instance, mode]
```
### 2. Run the planner

### 3. Run the planner
`--query-log` mode has no separate metrics-hint file — label sets are fetched directly from
your live Prometheus instance for whichever metrics show up in the log, so `--prometheus-url`
is required:

```bash
asap-planner \
--query-log /var/log/prometheus/query.log \
--metrics-config metrics.yaml \
--prometheus-url http://localhost:9090 \
--output_dir ./configs \
--prometheus_scrape_interval 15 \
--streaming_engine arroyo
Expand All @@ -44,6 +36,8 @@ This writes `streaming_config.yaml` and `inference_config.yaml` to `./configs/`.
## Notes

- Queries appearing only once are skipped (need frequency to infer repeat interval)
- Queries referencing metrics not in `metrics.yaml` are skipped with a warning
- Unsupported PromQL patterns (e.g. `absent`, complex multi-level aggregations) are skipped
- Repeat interval is inferred from median inter-arrival time, rounded to the nearest scrape interval
- If you'd rather hand-author the workload (no query log needed) or supply label sets without a
live Prometheus, use `--input_config` instead — see
[Try asap-planner on a PromQL Workload](try-asap-planner-promql.md)
93 changes: 93 additions & 0 deletions docs/03-how-to-guides/operations/try-asap-planner-promql.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Try asap-planner on a PromQL Workload

Check whether your PromQL workload is amenable to ASAPQuery acceleration by running just the
planner.

## 1. Build

```bash
cargo build --release -p asap_planner
```

This only compiles `asap-planner-rs` and its small internal deps — not the full query engine.

## 2. Write a workload config

```yaml
# promql_workload.yaml
metrics:
- metric: http_requests_total
labels: [instance, job, method, status]
query_groups:
- id: 1
repetition_delay: 300 # seconds between repeats of this query
controller_options:
accuracy_sla: 0.95
latency_sla: 100.0
queries:
- "sum by (job) (rate(http_requests_total[5m]))"
aggregate_cleanup:
policy: read_based
```

List each recurring query your application actually runs (dashboard panels, alerting rules,
recording rules), with its real repeat interval.

### `metrics` field

`metrics` is a hint: for each metric your queries reference, list every label it carries. The
planner uses the full label set to figure out which dimensions to roll up vs. keep grouped — it
isn't limited to the labels in this particular query's `by (...)` clause. If you have a live
Prometheus instance, you can skip this and pass `--prometheus-url` instead (see below) to
auto-infer label sets per metric.

### `controller_options`

`accuracy_sla` and `latency_sla` are required by the config schema but not currently used by
the planner's decision logic — any numeric values are fine (e.g. the placeholders above).

### Choosing `repetition_delay` and `--prometheus_scrape_interval`

- `--prometheus_scrape_interval` is your actual Prometheus scrape interval — a fact about your
existing setup, not something to tune.
- `repetition_delay` is how often this specific query actually re-runs — e.g. `300` for a
dashboard panel refreshing every 5 minutes, or an alert rule's `evaluation_interval`.
- Unlike SQL mode, there's no hard error if `repetition_delay` isn't a multiple of the scrape
interval, but `rate`/`increase`/`quantile_over_time` queries need at least 60 scraped data
points per repeat window to be considered worth accelerating when `--enable-punting` is set —
i.e. `repetition_delay >= 60 * prometheus_scrape_interval`.

## 3. Run the planner

```bash
asap-planner \
--input_config promql_workload.yaml \
--output_dir ./out \
--prometheus_scrape_interval 15 \
--streaming_engine precompute \
-v
```

- `--streaming_engine` just needs a valid value (`precompute`, `arroyo`, or `flink`) — none are
actually started.
- `-v` logs which queries were skipped and why.
- Optional: add `--prometheus-url http://localhost:9090` to auto-infer label sets from a live
Prometheus instead of hand-listing them under `metrics`.

## 4. Read the result

The planner writes `streaming_config.yaml` and `inference_config.yaml` to `./out`.

- Queries that show up there as aggregations are ones ASAPQuery can accelerate.
- Queries silently skipped (visible with `-v`) are not currently supported — e.g. unsupported
PromQL patterns (`absent`, complex multi-level aggregations) or queries seen too rarely to
infer a repeat interval.

If most of your workload appears in `streaming_config.yaml`, ASAPQuery is likely a good fit.

## Alternative: bootstrap from a real query log

Instead of hand-authoring `query_groups`, you can feed the planner a Prometheus query log
(`--query.log-file`) directly with `--query-log` + `--prometheus-url`, and it will infer queries
and repeat intervals from real traffic. See
[Bootstrap Config from Query Log](bootstrap-config-from-query-log.md) for that workflow.
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Task-oriented guides for common operations:
- [Manual Stack Run for Prometheus](03-how-to-guides/operations/manual-stack-run-prometheus.md) - Run ASAP components manually to accelerate Prometheus
- [Bootstrap Config from Query Log](03-how-to-guides/operations/bootstrap-config-from-query-log.md) - Auto-generate sketch configs from Prometheus query traffic
- [Try asap-planner on a SQL Workload](03-how-to-guides/operations/try-asap-planner-sql.md) - Check SQL workload amenability without running the full stack
- [Try asap-planner on a PromQL Workload](03-how-to-guides/operations/try-asap-planner-promql.md) - Check PromQL workload amenability without running the full stack
- [Manual Stack Run for Clickhouse](03-how-to-guides/operations/manual-stack-run-clickhouse.md) - Run ASAP components manually to accelerate Clickhouse
- [Deploy to CloudLab](03-how-to-guides/operations/deploy-cloudlab.md) - Deployment guide
- [Troubleshooting](03-how-to-guides/operations/troubleshooting.md) - Common issues & solutions
Expand Down
Loading