The /metrics endpoint p95 is 2.4s in Sky presumably due to run metrics generated via a ~2s select that scans and joins the entire run table:
|
res = await session.execute( |
|
select(ProjectModel.name, UserModel.name, RunModel.status, func.count(RunModel.id)) |
|
.join_from(RunModel, ProjectModel) |
|
.join_from(RunModel, UserModel, RunModel.user_id == UserModel.id) |
|
.group_by(ProjectModel.name, UserModel.name, RunModel.status) |
|
.order_by(ProjectModel.name, UserModel.name, RunModel.status) |
|
) |
The /metrics endpoint p95 is 2.4s in Sky presumably due to run metrics generated via a ~2s select that scans and joins the entire run table:
dstack/src/dstack/_internal/server/services/prometheus/custom_metrics.py
Lines 91 to 97 in 26247fd