Skip to content

Cache parsed credential bundles in credbundle loaders#537

Open
botengyao wants to merge 1 commit into
agent-substrate:mainfrom
botengyao:credbundle-cache
Open

Cache parsed credential bundles in credbundle loaders#537
botengyao wants to merge 1 commit into
agent-substrate:mainfrom
botengyao:credbundle-cache

Conversation

@botengyao

Copy link
Copy Markdown

Resolves the TODO: Introduce caching. on credbundle.Loader / credbundle.ClientLoader. The TODO dates back to the package's introduction; since #237 established mTLS between all ate system components, every inter-component handshake is on this path.

Problem

Loader/ClientLoader re-read and re-parse the credential bundle from disk on every TLS handshake so that pod-certificate rotations get picked up. That costs ~125µs and ~120 allocations per handshake (RSA bundle) for a file that almost never changes.

Approach: stat-validated cache

Each loader keeps the last successful parse together with the os.FileInfo taken just before that parse. A handshake stats the file (~1µs) and serves the cached certificate while file identity (os.SameFile, i.e. device+inode on Unix), mtime, and size all match; any mismatch re-reads and re-parses.

Invalidation and eviction:

  • The kubelet rotates projected-volume contents (including pod certificates) by writing a fresh timestamped directory and atomically swapping a symlink, so a rotation always surfaces as a change of file identity and is picked up on the next handshake — no staleness window. mtime and size additionally cover in-place rewrites.
  • No TTL: freshness is derived from the file, never from time, so an unchanged bundle is never re-parsed and a changed one is never served. (A refresh-interval floor to throttle stats was considered and rejected: it would introduce a staleness window to save a ~1µs syscall that only runs at connection-establishment rate. fsnotify was also considered and rejected: watching through symlink swaps is error-prone and adds watcher lifecycle for no latency benefit over the stat.)
  • Memory: exactly one parsed certificate per loader (one loader per tls.Config), replaced on rotation — nothing unbounded to evict.
  • Failure semantics unchanged: stat/read/parse errors are returned to the caller (never a stale certificate), the previous entry stays in place, and every later handshake retries until a parse succeeds — exactly the uncached behavior.
  • The file can change between the stat and the read; the parse is then stored against the older stat, so the next handshake detects the mismatch and re-reads. The cache never lags a rotation by more than one handshake.

Benchmarks

Apple M5 Pro, RSA-2048 bundle:

ns/op B/op allocs/op
BenchmarkLoaderCached 1,001 304 2
BenchmarkParseUncached 124,843 19,544 120

Parse stays exported and uncached; Loader/ClientLoader signatures are unchanged, so no call sites move.

  • Tests pass (go test ./internal/credbundle/ -race: cache-hit proof via unchanged-stat corruption, atomic-rename rotation, in-place rewrite, missing-file error, error recovery, concurrent handshakes under the race detector)
  • Appropriate changes to documentation are included in the PR (doc comments on Loader/ClientLoader/certCache describe the caching, rotation, and failure semantics)

Loader and ClientLoader re-read and re-parsed the credential bundle
file on every TLS handshake so that pod-certificate rotations are
picked up. The caching TODO dates back to the package's introduction,
and since mTLS was established between all ate system components
(agent-substrate#237), every inter-component handshake pays that cost: ~125us and
~120 allocations per handshake for an RSA bundle.

Resolve the TODO by caching the parsed bundle behind a stat check:

- Each handshake stats the bundle and serves the cached parse while
  file identity (os.SameFile: device and inode), mtime, and size all
  match the stat recorded at parse time. Any mismatch triggers a
  re-read and re-parse.
- The kubelet rotates projected volume contents, including pod
  certificates, by writing a fresh timestamped directory and
  atomically swapping a symlink, so a rotation always changes file
  identity and is picked up on the next handshake. mtime and size
  additionally cover in-place rewrites. There is no TTL: freshness is
  derived from the file, never from time, and an unchanged bundle is
  never re-parsed.
- Errors leave the previous entry in place and are returned to the
  caller, so failure behavior is unchanged from the uncached code,
  and later handshakes keep retrying until a parse succeeds.

The cached path costs ~1us and 2 allocations per handshake, ~125x
less than the uncached parse it replaces:

    BenchmarkLoaderCached-18     2421314     1001 ns/op     304 B/op     2 allocs/op
    BenchmarkParseUncached-18      19284   124843 ns/op   19544 B/op   120 allocs/op

Parse remains exported and uncached, and the Loader/ClientLoader
signatures are unchanged, so no call sites move.
@zoez7 Zoe Zhao (zoez7) self-assigned this Jul 25, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants