feat(api): add an optional stateless odbselect query node group - #15
Draft
tdakkota wants to merge 2 commits into
Draft
feat(api): add an optional stateless odbselect query node group#15tdakkota wants to merge 2 commits into
tdakkota wants to merge 2 commits into
Conversation
spec.query is the read half of spec.ingest: an optional *QuerySpec that deploys a Deployment of stateless odbselect pods reading through the ring. Absent means no pool and byte-identical rendering to today; the storage StatefulSet's selector and the client Service are untouched. Unlike odbingest's single shared listener, odbselect serves four query APIs on four listeners plus health on 13133. One <name>-query Service publishes them all — one pod set, one selector, a port each — and the probes hit the health listener rather than a data port. spec.signals decides which APIs are served: a disabled signal renders odbselect's bind: "-", which is required because odbselect defaults any API block it does not find. The ring comes from the shared ringConfig() helper with the cluster block reserved from extraConfig, and the listener binds are reserved too: odbselect keeps serving its other listeners if one moves, so a rebind leaves a published Service port pointing at nothing while the pod stays Ready. TestRoleRingsMatch now pins that all three roles render the same ring key for key. odbselect is not in any released image yet; its packaging ships with oteldb/oteldb#1266.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #14 — base is
claude/role-node-groups, review that first. Closes the query half of #11.Adds
spec.query, the read mirror of #14'sspec.ingest: an optional*QuerySpecdeploying a poolof stateless
odbselectquery nodes so query capacity scales independently of storage.Grounded in
cmd/odbselect/config.go,app.goandapi.goon oteldb'sclaude/1263-odbselect(oteldb/oteldb#1266), where the config is composed from the shared
internal/configblocks.The structural difference from ingest
odbingestserves OTLP/HTTP, remote write and health on one listener.odbselectdoes not — itserves four query APIs on four listeners plus health on a fifth:
prometheus:9090prom-httploki:3100loki-httptempo:3200tempo-httppyroscope:4040pyroscopehealth:13133Three decisions follow.
One Service, several ports
<name>-querypublishes every enabled API plus8090self-metrics. The four APIs are one pod setbehind one selector and each already has its own port, so
<name>-query:3100is exactly asaddressable as a hypothetical
<name>-query-loki:3100would be. A Service per API would add fourDNS names, four objects to prune, and — at
type: LoadBalancer— four load balancers, for nothing.The health listener is deliberately not published: it is what the probes hit, not a client API.
No per-API binds in the CRD;
spec.signalsis the off switchQuerySpecexposes no bind fields. The four APIs map 1:1 onto the four signalsspec.signalsalready toggles, and serving an API for a signal the cluster does not store can only answer empty —
so a second, independent toggle would only let a user express a pool that disagrees with its
cluster. A disabled signal renders
bind: "-"(odbselect's own disable convention) and drops theport from the container and the Service.
validateSignalsalready rejects all-disabled, which linesup with odbselect's own "every query API is disabled: odbselect would serve nothing".
The
"-"is required, not cosmetic:Config.setDefaultsruns over every API blockunconditionally, so an omitted block gets its default bind and is served anyway. Omitting is not
disabling. Pinned by
TestRenderQueryConfigDisabledSignal.The binds are reserved from
extraConfig— the one place this goes further than #14spec.ingest.extraConfigreserves onlycluster.spec.query.extraConfigreservesclusterandprometheus.bind,loki.bind,tempo.bind,pyroscope.bind,health.bind.That is a consequence of the four-listener shape, not a change of mind. For odbingest, a moved bind
breaks the one listener the probes use, so the pod never goes Ready and the mistake is loud. For
odbselect, moving one bind leaves the other four listeners serving, the health listener answering,
and the pod Ready — while the Service port the operator published points at nothing. That is the
same silent-misconfiguration class the ring argument is built on, so it gets the same treatment.
Everything else in a block stays open, which is the point:
prometheus.max_samples,loki.max_sample_rows, per-listenerauth,shutdown_timeoutare all still reachable.Carried over from #14
object — a user's own HPA/KEDA can target
apps/v1 Deployment <name>-queryalready.ringConfig()withclusterreserved.TestIngestRingMatchesStorageRingis generalized toTestRoleRingsMatch, asserting all threeroles agree key for key on
etcd/rf/shards_per_tenant/root. A mismatch does not fail, itresolves a different owner set.
repointing a live endpoint is a data-path change).
TestQuerySelectorIsDisjointFromOtherRolespins that no role's selector matches another role's pods.
spec.queryprunes its Deployment, Service and ConfigMap.status.queryReplicas/queryReadyReplicas. The phase still tracks the storage nodes.Small shared extractions along the way:
roleProbe(path, portName),statelessPodEnv(odbingest'sand odbselect's pod env were identical),
bindAll(port), and constants for the repeated env-var andannotation names.
Known caveat, not solved here
odbselectis not in any released image. Its release-packaging commit lives onclaude/1263-odbselectand ships when oteldb/oteldb#1266 merges — the same situationodbingestisin on #14. The operator side is correct regardless (the container runs
/usr/local/bin/odbselectout of the cluster image,
spec.query.imageoverrides it), but the pool cannot start until oteldbships the binary. Noted in the README, the
QuerySpecgodoc and the sample CR.Verification
make lint(0 issues),make test(envtest, controller package at 83.1%) andgo build ./...allpass. Generated manifests and deepcopy are in a separate commit.
🤖 Generated with Claude Code