[SDK] Implement the ProbabilitySampler

[ayush-that]

Fixes #4127

ref: https://opentelemetry.io/docs/specs/otel/trace/sdk/#probabilitysampler

Changes

adds the ProbabilitySampler from the spec. it turns the configured ratio into a 56 bit threshold, takes the randomness value from the rv key in the ot tracestate if there is one, otherwise from the last 7 bytes of the trace id, and samples the span when the randomness is greater than or equal to the threshold. sampled spans get the threshold written into the tracestate as the th key. the file configuration for the composable probability sampler now uses this sampler instead of TraceIdRatioBasedSampler.

the threshold keeps full 56 bit precision (same as opentelemetry-java-contrib). composable samplers are a separate effort (#4028), and the deprecated TraceIdRatioBased sampler is not changed.

• CHANGELOG.md updated for non-trivial changes
• Unit tests have been added
• Changes in public API reviewed

[ayush-that]

here is a demo against this branch to show the behaviour end to end. it shows the threshold boundary at R >= T, the th encoding, rv taking precedence over the trace id, th replacement, and a foreign tracestate entry surviving untouched.

output:

[----------] Global test environment tear-down
[==========] 14 tests from 1 test suite ran. (922 ms total)
[  PASSED  ] 14 tests.
ratio 1.0    -> SAMPLE, tracestate: ot=th:0
ratio 0.5, R=0x80000000000000 -> SAMPLE, tracestate: ot=th:8
ratio 0.5, R=0x7fffffffffffff -> DROP
ratio 0.5, 100000 random trace ids -> sampled 49958 (0.500)
ratio 0.5, rv=ff.. trace id low bits 0 -> SAMPLE, tracestate: ot=rv:ffffffffffffff;th:8
ratio 0.25, parent 'ot=th:8,congo=..' -> SAMPLE, tracestate: ot=th:c,congo=t61rcWkgMzE

demo.cc

#include <cstdint>
#include <cstdio>
#include <map>
#include <string>
#include <utility>
#include <vector>

#include "opentelemetry/common/key_value_iterable_view.h"
#include "opentelemetry/sdk/trace/samplers/probability.h"
#include "opentelemetry/trace/span_context.h"
#include "opentelemetry/trace/span_context_kv_iterable_view.h"
#include "opentelemetry/trace/trace_state.h"
#include "src/common/random.h"

namespace trace_api = opentelemetry::trace;
using opentelemetry::sdk::common::Random;
using opentelemetry::sdk::trace::Decision;
using opentelemetry::sdk::trace::ProbabilitySampler;

namespace {
trace_api::TraceId TraceIdWithLow56(uint64_t v) {
  uint8_t buf[16] = {0};
  for (int i = 0; i < 7; ++i)
    buf[15 - i] = static_cast<uint8_t>(v >> (8 * i));
  return trace_api::TraceId(buf);
}

opentelemetry::sdk::trace::SamplingResult
Sample(ProbabilitySampler &s, const trace_api::SpanContext &ctx,
       trace_api::TraceId tid) {
  std::map<std::string, int> attrs;
  std::vector<
      std::pair<trace_api::SpanContext, std::map<std::string, std::string>>>
      links;
  opentelemetry::common::KeyValueIterableView<decltype(attrs)> attrs_view{
      attrs};
  trace_api::SpanContextKeyValueIterableView<decltype(links)> links_view{links};
  return s.ShouldSample(ctx, tid, "demo", trace_api::SpanKind::kInternal,
                        attrs_view, links_view);
}

const char *Header(const opentelemetry::sdk::trace::SamplingResult &r) {
  static std::string h;
  h = r.trace_state ? r.trace_state->ToHeader() : "(parent tracestate kept)";
  return h.c_str();
}
} // namespace

int main() {
  // ratio 1.0: always sampled, threshold 0 emitted as ot=th:0
  ProbabilitySampler all(1.0);
  auto r =
      Sample(all, trace_api::SpanContext::GetInvalid(), TraceIdWithLow56(0));
  printf("ratio 1.0    -> %s, tracestate: %s\n",
         r.decision == Decision::RECORD_AND_SAMPLE ? "SAMPLE" : "DROP",
         Header(r));

  // ratio 0.5: threshold is 2^55. R = 2^55 samples, R = 2^55-1 drops.
  ProbabilitySampler half(0.5);
  r = Sample(half, trace_api::SpanContext::GetInvalid(),
             TraceIdWithLow56(0x80000000000000));
  printf("ratio 0.5, R=0x80000000000000 -> %s, tracestate: %s\n",
         r.decision == Decision::RECORD_AND_SAMPLE ? "SAMPLE" : "DROP",
         Header(r));
  r = Sample(half, trace_api::SpanContext::GetInvalid(),
             TraceIdWithLow56(0x7fffffffffffff));
  printf("ratio 0.5, R=0x7fffffffffffff -> %s\n",
         r.decision == Decision::RECORD_AND_SAMPLE ? "SAMPLE" : "DROP");

  // 100000 random trace ids at ratio 0.5
  int sampled = 0;
  for (int i = 0; i < 100000; ++i) {
    uint8_t buf[16];
    Random::GenerateRandomBuffer(buf);
    if (Sample(half, trace_api::SpanContext::GetInvalid(),
               trace_api::TraceId(buf))
            .decision == Decision::RECORD_AND_SAMPLE)
      ++sampled;
  }
  printf("ratio 0.5, 100000 random trace ids -> sampled %d (%.3f)\n", sampled,
         sampled / 100000.0);

  // explicit rv in the parent tracestate wins over trace id randomness
  uint8_t id[16] = {1};
  trace_api::SpanContext rv_high(
      trace_api::TraceId(id), trace_api::SpanId(), trace_api::TraceFlags{0},
      false, trace_api::TraceState::FromHeader("ot=rv:ffffffffffffff"));
  r = Sample(half, rv_high, TraceIdWithLow56(0));
  printf("ratio 0.5, rv=ff.. trace id low bits 0 -> %s, tracestate: %s\n",
         r.decision == Decision::RECORD_AND_SAMPLE ? "SAMPLE" : "DROP",
         Header(r));

  // existing th is replaced, other entries survive
  ProbabilitySampler quarter(0.25);
  trace_api::SpanContext with_th(
      trace_api::TraceId(id), trace_api::SpanId(), trace_api::TraceFlags{0},
      false, trace_api::TraceState::FromHeader("ot=th:8,congo=t61rcWkgMzE"));
  r = Sample(quarter, with_th, TraceIdWithLow56(0xffffffffffffff));
  printf("ratio 0.25, parent 'ot=th:8,congo=..' -> %s, tracestate: %s\n",
         r.decision == Decision::RECORD_AND_SAMPLE ? "SAMPLE" : "DROP",
         Header(r));

  return 0;
}

[codecov]

Codecov Report

❌ Patch coverage is 98.09524% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.92%. Comparing base (d6035a8) to head (6abbf4b).
⚠️ Report is 2 commits behind head on main.

Files with missing lines	Patch %	Lines
sdk/src/trace/samplers/probability.cc	98.10%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #4135      +/-   ##
==========================================
+ Coverage   82.82%   82.92%   +0.10%     
==========================================
  Files         406      407       +1     
  Lines       16913    17018     +105     
==========================================
+ Hits        14007    14110     +103     
- Misses       2906     2908       +2     

Files with missing lines	Coverage Δ
sdk/src/trace/samplers/probability.cc	98.10% <98.10%> (ø)

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[ayush-that]

about last force push-

1. fixed the iwyu failures.
2. rebased on latest main.

[lalitb]

Can we reuse CalculateThreshold, most of the logic seems same?

[ayush-that]

Okay. I'm reusing CalculateThreshold and deleting the duplicate.

[lalitb]

Same as here.

[ayush-that]

Okay. Reusing GetRandomnessFromTraceId.

[lalitb]

If this sampler drops the span, it returns no updated tracestate, so the child span falls back to the parent tracestate and can keep an old ot=th value. Since this is a new independent probability decision, that threshold is no longer valid. Can you check the logic here.

[ayush-that]

Fixed it to strip th on the drop path and return the recomputed tracestate, with rv and the other sub keys preserved. Added a test.