Fix PPCB failback with missing or stale addresses

[Annie Liang (xinlian12)]

Issue

In some cases, the Per-Partition Circuit Breaker (PPCB) fails a partition over to another region but never fails it back to the preferred region, even after the preferred region becomes available again.

Root cause

This occurs when the partition address cache is not populated before PPCB failover. For example, the initial address refresh can fail with a 503 or connection timeout, leaving the partition with no cached addresses. The background recovery task previously assumed that a cache entry existed and attempted to open connections from a null entry, causing a NullPointerException and preventing recovery from completing.

The failure workflow is:

1. The VM starts and sends a request to partition 1 in the preferred region.
2. Address resolution fails with a 503, timeout, or similar network error, so the address cache remains empty for that partition.
3. Repeated request failures reach the PPCB threshold, and the partition fails over to another region.
4. After the 60-second recovery interval, PPCB probes the preferred region.
5. The recovery probe encounters the missing cache entry and fails with a NullPointerException.
6. Subsequent recovery cannot complete, so requests never fail back to the preferred region.

A related case can occur when the cache is populated but contains stale replica addresses: recovery probes those stale addresses and cannot reconnect after the backend addresses change.

Solution

The PPCB recovery path now:

• Resolves partition addresses when the cache entry is missing.
• Probes all cached replica addresses.
• If any probe fails with a network-related connection error, force-refreshes the addresses and retries the refreshed replicas once.
• Propagates failures from the refreshed retry so PPCB does not incorrectly mark an unreachable partition as recovered.

Tests

Added coverage to the existing Cosmos test suites for:

• Missing address-cache entries, stale addresses, forced refresh, and failure after the refreshed retry in GatewayAddressCacheTest.
• Scheduled PPCB recovery with missing and stale address-cache states in GlobalPartitionEndpointManagerForPPCBUnitTests.
• The customer workflow in PerPartitionCircuitBreakerE2ETests: fault-injected address refresh failures leave the cache empty, PPCB fails the partition over, and background recovery later resolves the addresses and fails it back.

[azure-pipelines]

Azure Pipelines:
Successfully started running 2 pipeline(s).
33 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

[Copilot]

Pull request overview

Fixes a Per-Partition Circuit Breaker (PPCB) recovery edge case in the Cosmos direct connectivity stack where failback could get stuck if failover happens before the partition address cache is populated. The change ensures the recovery probe can resolve addresses via the gateway on a cache miss, and adds a regression unit test plus a changelog entry.

Changes:

• Update GatewayAddressCache.submitOpenConnectionTasks to resolve partition addresses via gateway when the address-cache entry is missing.
• Add a hermetic unit test covering the missing-cache path and verifying the resolved address is cached for subsequent probes.
• Document the fix in the unreleased Cosmos CHANGELOG.md.

Show a summary per file

File	Description
sdk/cosmos/azure-cosmos/src/main/java/com/azure/cosmos/implementation/directconnectivity/GatewayAddressCache.java	Adds cache-miss gateway address resolution before scheduling proactive open-connection tasks.
sdk/cosmos/azure-cosmos-tests/src/test/java/com/azure/cosmos/implementation/directconnectivity/GatewayAddressCacheUnitTest.java	Adds a unit regression test validating cache-miss resolution and subsequent cache hit behavior.
sdk/cosmos/azure-cosmos/CHANGELOG.md	Notes the PPCB failback/cache-miss recovery fix in the unreleased bugs-fixed section.

Review details

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

• Files reviewed: 3/3 changed files
• Comments generated: 1
• Review effort level: Lite

[Annie Liang (xinlian12)]

@sdkReviewAgent

[Annie Liang (xinlian12)]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Annie Liang (xinlian12)]

✅ Review complete (52:39)

Posted 8 inline comment(s).

_{Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage}

[Annie Liang (xinlian12)]

@sdkReviewAgent

[Annie Liang (xinlian12)]

✅ Review complete (28:51)

Posted 3 inline comment(s).

_{Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage}

[Annie Liang (xinlian12)]

@sdkReviewAgent

[Annie Liang (xinlian12)]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Annie Liang (xinlian12)]

✅ Review complete (02:39)

Posted 3 inline comment(s).

_{Steps: ✓ context, correctness, cross-sdk, design, history, past-prs, synthesis, test-coverage}

[Annie Liang (xinlian12)]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Annie Liang (xinlian12)]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[Annie Liang (xinlian12)]

/azp run java - cosmos - tests

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).