security(dci): reject non-ASCII bearer tokens with 401 (avoid auth 500s)

[gonzalesedwin1123]

Problem

spp_dci_server/middleware/signature.py::verify_bearer_token passes the raw
Authorization header token straight to hmac.compare_digest(token, candidate)
when static tokens are configured (dci.api_tokens non-empty).

HTTP headers are decoded as latin-1, so a request whose Bearer token contains
non-ASCII bytes reaches this code as a non-ASCII str. hmac.compare_digest
accepts str operands only when they are ASCII — a non-ASCII operand raises
TypeError: comparing strings with non-ASCII characters is not supported.

verify_bearer_token has no try/except around the compare, and TypeError is
not a DCIHTTPException. As a FastAPI dependency it runs before the route
body, so the routers' own try/except → 500 wrappers don't cover it — the error
escapes as a generic 500 and logs a stack trace on every bearer-authenticated
DCI endpoint (search, bulk_upload, async, receipt, callbacks,
registry_aliases, ping). An unauthenticated caller can trigger it at will.

Regression from the switch to hmac.compare_digest for constant-time token
comparison; the prior token in accepted_tokens membership test handled such
input as an ordinary non-match (clean 401). Severity: low/medium — an
unauthenticated error/DoS-flavoured path plus log noise; no data exposure or
auth bypass.

Fix

Reject non-ASCII credentials with a 401 before any comparison, placed right
after the empty-token check so the single guard covers the constant-time loop,
the OAuth2-JWT path, and the empty-list opt-out return:

if not token.isascii():
    raise DCIHTTPException(status_code=401, error_code="err.auth.invalid_token", ...)

No legitimate bearer credential is ever non-ASCII (OAuth2 JWTs are base64url;
configured static tokens are ASCII), so this changes no valid-caller behaviour
and preserves the timing-safe comparison for ASCII tokens. Reuses the existing
err.auth.invalid_token code (a non-ASCII token is, semantically, just invalid).

Tests

Written test-first (red → green). Added to tests/test_bearer_middleware.py:

• test_non_ascii_bearer_token_rejected_with_401 — with dci.api_tokens set, a
  non-ASCII token must 401. Errored with TypeError before the fix (the exact
  reproduction); passes after.
• test_non_ascii_bearer_token_rejected_even_with_empty_list — with the empty-list
  opt-out (dci.api_tokens_required=false), a non-ASCII token must still 401 rather
  than be accepted. Failed before the fix (it was returned as valid); passes after.

Existing test_configured_tokens_* and test_token_comparison_is_constant_time
remain green (ASCII happy path and per-candidate compare count unchanged).
./spp t spp_dci_server → 0 failed, 0 error(s) of 334 tests. Ruff/ruff-format clean.

Scope

• Single sink: compare_digest appears exactly once in the module. The sibling
  verify_dci_signature already wraps its body in try/except → 401, so it is
  not affected by this class of crash.
• Version bump 19.0.2.0.3 → 19.0.2.0.4 + HISTORY fragment. README.rst /
  static/description/index.html to be regenerated from CI's pinned generator.

Merge order (vs #303 DCI envelope search)

Per the cross-PR interaction analysis (2026-07-24, internal/plans/security-prs-interaction-analysis.md): #303 and this PR both bump spp_dci_server to the identical 19.0.2.0.4 — the manifest line auto-merges with no conflict marker. Either merge order works (code is fully disjoint: middleware vs router, and the middleware runs before any router), but whichever merges second must deliberately re-bump to 19.0.2.0.5 and split the HISTORY heading; the metadata files (HISTORY/README/index.html) will conflict and remind, the manifest will not.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 75.20%. Comparing base (1caf794) to head (7ccff30).
⚠️ Report is 3 commits behind head on 19.0.

Additional details and impacted files

@@            Coverage Diff             @@
##             19.0     #325      +/-   ##
==========================================
+ Coverage   74.28%   75.20%   +0.92%     
==========================================
  Files         372      173     -199     
  Lines       25385    12334   -13051     
==========================================
- Hits        18857     9276    -9581     
+ Misses       6528     3058    -3470     

Flag	Coverage Δ
spp_analytics	?
spp_api_v2_cycles	?
spp_api_v2_entitlements	?
spp_api_v2_gis	?
spp_api_v2_programs	?
spp_api_v2_simulation	?
spp_audit_programs	?
spp_base_common	91.07% <ø> (+0.80%)	⬆️
spp_case_demo	?
spp_case_entitlements	?
spp_case_programs	?
spp_cr_type_assign_program	?
spp_dci_client_dr	85.77% <ø> (?)
spp_dci_client_ibr	92.47% <ø> (?)
spp_dci_compliance	93.01% <ø> (+0.24%)	⬆️
spp_dci_demo	?
spp_dci_indicators	96.23% <ø> (?)
spp_dci_server	90.38% <100.00%> (?)
spp_dci_server_social	89.57% <ø> (+0.19%)	⬆️
spp_demo	?
spp_demo_phl_luzon	?
spp_drims	?
spp_drims_sl	?
spp_drims_sl_demo	?
spp_farmer_registry_demo	?
spp_gis_report	?
spp_grm_demo	?
spp_indicator	?
spp_indicator_studio	?
spp_metric_service	?
spp_mis_demo_v2	?
spp_programs	65.27% <ø> (+<0.01%)	⬆️
spp_registry	86.94% <ø> (+0.10%)	⬆️
spp_security	69.56% <ø> (+2.89%)	⬆️
spp_simulation	?
spp_starter_sp_mis	86.66% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
spp_dci_server/middleware/signature.py	91.72% <100.00%> (ø)

... and 318 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.