Skip to content

feat: Add environment ID support for hooks. - #484

Merged
kinyoklion merged 6 commits into
mainfrom
devin/1786489071-hook-environment-id
Aug 13, 2026
Merged

feat: Add environment ID support for hooks.#484
kinyoklion merged 6 commits into
mainfrom
devin/1786489071-hook-environment-id

Conversation

@kinyoklion

@kinyoklion kinyoklion commented Aug 11, 2026

Copy link
Copy Markdown
Member

Requirements

  • I have added test coverage for new or changed functionality
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions

Related issues

Implements the environmentId field of EvaluationSeriesContext from the hooks spec, which the OTel tracing hook uses for feature_flag.set.id. Mirrors launchdarkly/dotnet-core#81.

Describe the solution you've provided

EvaluationSeriesContext gains an optional environment_id, populated from the X-LD-EnvID response header sent by LaunchDarkly. Both data systems are supported, and each exposes it through a new DataSystem.environment_id property that the client reads when building the series context. Per the spec, it is only recorded from successful responses, so hooks never see an environment ID scraped off an error page.

  • FDv2 already parsed the header into Basis.environment_id / Update.environment_id but discarded it; FDv2 now latches the last value seen on a success path (a _Success basis, or an update whose state is VALID).
  • FDv1 had no access to the header. Streaming reads it from the Start action of the SSE client, polling reads it from the response headers, and both record it on DataSourceUpdateSinkImpl.

Sketch of the FDv1 path:

# streaming
if isinstance(action, Start):
    record_environment_id(self._data_source_update_sink, action.headers)

# polling
(all_data, headers) = self._get_all_data_with_headers()
record_environment_id(self._data_source_update_sink, headers)

Describe alternatives you've considered

Following the .NET implementation more literally, where the environment ID is stored as init metadata on the data store, would require optional extension interfaces on the public FeatureStore/DataSourceUpdateSink types. Instead, header handling stays inside the data sources and is surfaced by the data system, so externally implemented stores, sinks, feature requesters, and update processors continue to work unchanged (they simply report no environment ID).

Additional context

  • No change is needed in launchdarkly-eventsource: it already exposes response headers on Start and Fault (unlike the .NET event source, which needed feat: Report event source headers on open. dotnet-eventsource#104).
  • The async (experimental) client does not run hooks yet, so the async data source path is untouched.
  • The environmentId support in the OTel tracing hook lives in python-server-sdk-otel and is a follow-up.

Link to Devin session: https://app.devin.ai/sessions/bfe54128e2804a96bb100e6120e9a3ef
Requested by: @kinyoklion


Note

Overview
Adds optional environment_id on EvaluationSeriesContext so evaluation hooks (e.g. OTel feature_flag.set.id) can see which LaunchDarkly environment the SDK is connected to, matching the hooks spec.

The ID comes from the X-LD-EnvID header on successful data-source responses only—not from error responses. FDv1 records it via streaming Start headers and polling response headers into DataSourceUpdateSinkImpl; FDv2 latches values already present on successful Basis / VALID Update objects. DataSystem.environment_id exposes the value and LDClient passes it when building hook series context.

Custom FeatureRequester implementations without get_all_data_with_headers still work (no env ID). Contract tests add the hook-environment-id capability.

Reviewed by Cursor Bugbot for commit 7ce7b6a. Bugbot is set up for automated code reviews on this repo. Configure here.

@kinyoklion kinyoklion self-assigned this Aug 11, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

@kinyoklion
kinyoklion marked this pull request as ready for review August 12, 2026 17:18
@kinyoklion
kinyoklion requested a review from a team as a code owner August 12, 2026 17:18

@kinyoklion kinyoklion left a comment

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

These comments were generated by Claude (multi-agent review: general / security / adversarial agents), run and posted by @rlamb.

No crash or flag-data-integrity issues found — thread safety, None-safety, backward compat, and the eventsource header contract all checked out under adversarial testing (full report available on request). Two findings worth addressing, posted inline:

  1. Environment ID is recorded from error responses and exposed to hooks before initialization — diverges from the HOOK spec and the .NET implementation this PR mirrors (proven with a runnable repro).
  2. The two new polling tests use unbounded ready.wait() — a regression in the fallback path hangs CI instead of failing (proven by mutation).

Comment thread ldclient/impl/datasource/streaming.py Outdated
log.info("StreamingUpdateProcessor initialized ok.")
self._ready.set()
elif isinstance(action, Fault):
record_environment_id(self._data_source_update_sink, action.headers)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Claude] Medium: env ID is latched from error responses and exposed to hooks pre-initialization

This Fault-path record means the env ID is captured from failed responses: per ld_eventsource, Fault.headers is populated exactly when the response was non-2xx or wrong content type. The same happens in FDv2 from failed-initializer headers (fdv2.py:226) and from every synchronizer update including INTERRUPTED/OFF (fdv2.py:425), and the value is never cleared once set.

Proven with a repro: a 401 (invalid SDK key) response carrying X-LD-EnvID: env-from-401 leaves the SDK permanently uninitialized (variation() returns defaults), yet every hook receives environment_id == 'env-from-401' for the process lifetime.

This diverges from both references:

  • The hooks spec says the environment ID is "provided by LaunchDarkly and only available once initialization has completed."
  • The .NET implementation this PR mirrors (feat: Add environment ID support for hooks. dotnet-core#81) records it only as store init metadata written together with successfully applied data; its error handlers read only x-ld-fd-fallback. Under the same 401, .NET hooks get null.

Practical impact: any middlebox that synthesizes an error page on the SDK↔LD path (TLS-intercepting proxy, LB in front of a Relay Proxy) can permanently poison the value that downstream OTel hooks emit as feature_flag.set.id — even after the real connection recovers.

Suggestion: record only from success paths — Start here, the _Success basis in _run_initializers (drop the _Fail-headers record at fdv2.py:226), and gate fdv2.py:425 on VALID updates — then update/remove test_records_environment_id_from_error_response_headers and test_environment_id_from_initializer_error_headers, which currently codify the error-latching. If latching from errors is intentional, it deserves an explicit callout since it's a Python-only behavioral divergence.

Rider: FDv2._record_environment_id (fdv2.py:511) only skips None, so a present-but-empty X-LD-EnvID overwrites a previously correct value; the FDv1 helper filters '' (datasource_common.py). Worth applying the same isinstance(str) and != '' guard there for consistency.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed — fixed in b9fcb4c. Env ID is now only recorded from success paths: Start in FDv1 streaming (dropped the Fault record), the _Success basis in _run_initializers (dropped the _Fail-headers record), and synchronizer updates gated on DataSourceState.VALID. FDv2._record_environment_id now applies the same isinstance(str) and != '' guard as the FDv1 helper. The two tests that codified error-latching now assert the opposite (test_does_not_record_environment_id_from_error_response_headers, test_environment_id_is_not_recorded_from_initializer_error_headers), plus a new test_environment_id_is_not_recorded_from_non_valid_updates.

sink = DataSourceUpdateSinkImpl(store, Listeners(), Listeners())
config._data_source_update_sink = sink
setup_processor(config)
ready.wait()

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Claude] Medium: unbounded ready.wait() turns a regression into a CI hang

This wait (and the one at line 180) has no timeout. If the external-requester fallback in PollingUpdateProcessor._get_all_data_with_headers regresses, the resulting exception is swallowed by _poll's generic handler, ready is never set, and the test hangs forever — the repo configures no pytest-timeout, so CI hangs until the job limit rather than reporting a failure.

Verified by mutation: replacing the fallback with an unconditional self._requester.get_all_data_with_headers() call makes test_environment_id_is_none_when_requester_provides_no_headers hang rather than fail.

Suggestion: assert ready.wait(2) in both new tests, matching the bounded waits already used elsewhere in this file (e.g. ready.wait(0.5) at lines 115/137).

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in b9fcb4c — both new polling tests now use assert ready.wait(2).

@jsonbailey

Copy link
Copy Markdown
Contributor

@devin rebase this PR on main and resolve the conflicts

devin-ai-integration Bot and others added 4 commits August 12, 2026 18:10
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot force-pushed the devin/1786489071-hook-environment-id branch from b51e14c to 585459b Compare August 12, 2026 18:11
@devin-ai-integration

Copy link
Copy Markdown
Contributor

Rebased on main and pushed (585459b). The only conflict was the typing import in ldclient/impl/datasystem/__init__.py, now TYPE_CHECKING, Optional, Protocol, runtime_checkable. make lint and the full suite pass (1339 passed).

kinyoklion added a commit to launchdarkly/sdk-test-harness that referenced this pull request Aug 12, 2026
**Requirements**

- [x] I have added test coverage for new or changed functionality
- [x] I have followed the repository's [pull request submission
guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests)
- [x] I have validated my changes against all supported platform
versions

**Related issues**

Follows the hooks spec requirement that `EvaluationSeriesContext`
carries the environment ID reported by LaunchDarkly. Companion SDK work:
launchdarkly/python-server-sdk#484.

**Describe the solution you've provided**

`servicedef.EvaluationSeriesContext`/`TrackSeriesContext` already had an
`environmentId` field, but nothing exercised it and there was no
capability for SDKs to declare support. This adds:

- the `hook-environment-id` capability, documented in
`docs/service_spec.md`
- `DataSourceOptionEnvironmentID(...)`, which wraps the mock
streaming/polling handler so responses carry an `X-LD-EnvID` header
- an evaluation-series test that configures that header and asserts
`evaluationSeriesContext.environmentId` matches on `beforeEvaluation`

**Describe alternatives you've considered**

Setting the header inside `mockld.StreamingService`/`PollingService`
themselves rather than via a handler wrapper; the wrapper avoids
touching both services and works for either.

**Additional context**

Track hooks (`trackSeriesContext.environmentId`) are described in the
docs but not yet asserted by a test; that can follow once an SDK
supports both capabilities.


Link to Devin session:
https://app.devin.ai/sessions/bfe54128e2804a96bb100e6120e9a3ef
Requested by: @kinyoklion

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Adds the **`hook-environment-id`** capability so SDKs can declare that
they propagate LaunchDarkly’s environment ID from streaming/polling into
hook series contexts, with matching documentation in
`docs/service_spec.md`.
> 
> The mock data source can now emit **`X-LD-EnvID`** via
`DataSourceOptionEnvironmentID`, using a small handler wrapper so both
streaming and polling paths behave like production. Hook integration
tests gain **`provides the environment ID`**, which checks
`evaluationSeriesContext.environmentId` on **`afterEvaluation`** for
default and polling data sources when the capability is present. Hook
test setup is extended so data source options can be passed through
`createClientForHooksWithDataSourceOptions`.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
ccd1596. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Devin AI <devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
kinyoklion added a commit to launchdarkly/sdk-test-harness that referenced this pull request Aug 12, 2026
**Requirements**

- [x] I have added test coverage for new or changed functionality
- [x] I have followed the repository's [pull request submission
guidelines](../blob/master/CONTRIBUTING.md#submitting-pull-requests)
- [x] I have validated my changes against all supported platform
versions

**Related issues**

Ports #410 (v2) to v3. Companion SDK work:
launchdarkly/python-server-sdk#484, launchdarkly/cpp-sdks#594.

**Describe the solution you've provided**

Same capability and test as #410, adapted to the v3 data system:

- the `hook-environment-id` capability, documented in
`docs/service_spec.md`
- `DataSystemOptionEnvironmentID(...)`, a top-level
`SDKDataSystemOption` that makes every mock service of the data system
(initializers and synchronizers, in all connection modes) send an
`X-LD-EnvID` response header
- an evaluation-series test (`hooks/evaluation/provides the environment
ID`) asserting `evaluationSeriesContext.environmentId` on
`afterEvaluation`, run against the SDK's default data source plus an
explicit polling subtest for client-side/`server-side-polling` SDKs

Unlike v2, where the option wrapped the single `SDKDataSource` handler,
the header is applied in `createEndpoints`, so both FDv2 initializers
and synchronizers report it — matching production, where the header is
on every flag-delivery response.

**Describe alternatives you've considered**

Making the option per-connection-mode. Environment ID is a property of
the environment, not of a connection mode, so a single top-level option
keeps call sites simple and covers every mode.

**Additional context**

Verified against the FDv2 contract services of python-server-sdk (#484)
and cpp-sdks (#594): `hooks/evaluation/provides the environment ID`
passes for both, in default (streaming) and polling modes.

Track hooks (`trackSeriesContext.environmentId`) are documented but not
yet asserted, same as in v2.


Link to Devin session:
https://app.devin.ai/sessions/bfe54128e2804a96bb100e6120e9a3ef
Requested by: @kinyoklion

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Adds the **`hook-environment-id`** capability so SDKs that read
LaunchDarkly’s `X-LD-EnvID` from streaming/polling responses must
surface it on hook series contexts
(`evaluationSeriesContext.environmentId`, and `trackSeriesContext` when
track hooks are supported). The harness documents this in
`service_spec.md` and gates a new evaluation hook test on the
capability.
> 
> **Mock flag delivery** now supports
`DataSystemOptionEnvironmentID(...)`, a top-level data-system option
that wraps every initializer and synchronizer endpoint (all connection
modes) to emit `X-LD-EnvID`, matching production flag-delivery responses
in the v3 data system.
> 
> The test **“provides the environment ID”** evaluates a flag and
asserts `environmentId` on the `afterEvaluation` hook payload, under the
default data source and an explicit polling subtest for client-side /
server-side-polling SDKs. Track-hook `environmentId` is documented but
not asserted yet (same as v2).
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
8331c29. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

---------

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
kinyoklion added a commit to launchdarkly/sdk-test-harness that referenced this pull request Aug 12, 2026
**Requirements**

- [ ] I have added test coverage for new or changed functionality
- [x] I have followed the repository's pull request submission
guidelines
- [x] I have validated my changes against all supported platform
versions

**Related issues**

Intermittent `Download failed` failures in SDK contract-test jobs, e.g.
launchdarkly/python-server-sdk#484 where 4 of 5
matrix jobs died at the same second while the rest passed on identical
code.

**Describe the solution you've provided**

`downloader/run.sh` resolves the release version through the GitHub API
(authenticated when `GITHUB_TOKEN` is set), but then downloads the
release asset with a single un-retried `curl --fail -s`, and `-s` hides
the reason for a failure:

```diff
-curl --fail -s -L -o "${TEMP_DIR}/archive.${EXTENSION}" "${DOWNLOAD_URL}" || (echo "Download failed" >&2; exit 1)
+curl --fail -sS -L --retry 5 --retry-delay 2 \
+  -o "${TEMP_DIR}/archive.${EXTENSION}" "${DOWNLOAD_URL}" \
+  || { echo "Download failed" >&2; exit 1; }
```

`--retry` covers transient 5xx/408/429 responses (consistent with
several matrix jobs failing at once while others succeed), and `-sS`
means the next failure reports the actual curl error instead of a bare
`Download failed`.

**Describe alternatives you've considered**

- `--retry-all-errors`: rejected, it requires curl 7.71+ and this script
also runs on older macOS runners.
- Sending the token on the asset download: the asset URL redirects to an
unauthenticated object store, so an `Authorization` header there is at
best useless and can break the redirect.

**Additional context**

The `contract-tests` GitHub action reads this script from the `v2`
branch by default, so this also covers repos running the v3 harness
through that action. `main` has the same line and should get the same
change.


Link to Devin session:
https://app.devin.ai/sessions/bfe54128e2804a96bb100e6120e9a3ef
Requested by: @kinyoklion

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Overview**
> Contract-test jobs intermittently failed with a bare **Download
failed** when fetching the `sdk-test-harness` release asset from GitHub.
> 
> The download `curl` in `downloader/run.sh` now uses **`--retry 5
--retry-delay 2`** for transient HTTP failures (aligned with matrix jobs
failing together while others pass) and **`-sS`** instead of `-s` so
failures include the actual curl error message.
> 
> <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit
2c23020. Bugbot is set up for automated
code reviews on this repo. Configure
[here](https://www.cursor.com/dashboard/bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: rlamb@launchdarkly.com <4955475+kinyoklion@users.noreply.github.com>
@devin-ai-integration devin-ai-integration Bot changed the title feat: Propagate environment ID to evaluation hooks feat: Add environment ID support for hooks. Aug 12, 2026
@kinyoklion
kinyoklion merged commit 49e809f into main Aug 13, 2026
16 checks passed
@kinyoklion
kinyoklion deleted the devin/1786489071-hook-environment-id branch August 13, 2026 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants