Skip to content

ci: Configure SDK compliance capture suite#711

Closed
marandaneto wants to merge 1 commit into
mainfrom
fix/sdk-harness-options-20260630
Closed

ci: Configure SDK compliance capture suite#711
marandaneto wants to merge 1 commit into
mainfrom
fix/sdk-harness-options-20260630

Conversation

@marandaneto

Copy link
Copy Markdown
Member

Problem

SDK compliance workflows need explicit harness suite/sdk-type selection and configurable blocking behavior so CI only runs the intended contract checks.

Changes

  • Point SDK compliance to the harness branch with new workflow inputs.
  • Run capture v0 as a server SDK.
  • Keep compliance failures non-blocking for now.
  • Align local compose with the same suite/sdk type.

Testing

  • Parsed workflow YAML.
  • Ran docker compose config for the adapter compose file.
  • Ran git diff --check.

Release / changeset

No SDK package changeset: CI/local compliance configuration only.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with Pi using dedicated git worktrees. The change was requested to align SDK compliance harness setup across SDK repositories while keeping non-ready SDKs non-blocking.

@marandaneto marandaneto self-assigned this Jun 30, 2026
@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Reviews (1): Last reviewed commit: "ci: Configure SDK compliance capture sui..." | Re-trigger Greptile

compliance:
name: PostHog SDK compliance tests
uses: PostHog/posthog-sdk-test-harness/.github/workflows/test-sdk-action.yml@be8b8d5a3f94a249659844e94832e874f049c1e4
uses: PostHog/posthog-sdk-test-harness/.github/workflows/test-sdk-action.yml@fix/sdk-harness-options-20260630

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Mutable branch reference in reusable workflow call

The workflow now references fix/sdk-harness-options-20260630 (a branch) instead of the previously pinned SHA. Branch refs are mutable — any commit pushed to that branch will be picked up on the next run without any code review on this side. The previous SHA pin (be8b8d5a3f94a249659844e94832e874f049c1e4) gave a guaranteed-immutable reference. Once the harness branch is merged and a stable commit is available, this should be pinned back to a SHA to avoid unexpected behavior or supply-chain risk.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

adapter-context: "."
test-harness-version: "0.8.0"
suite: "capture"
sdk-type: "server"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 continue-on-error as a with input may not make failures non-blocking

continue-on-error: true passed via with: is just a named input to the reusable workflow — it only suppresses failures if the callee explicitly declares it as an input and sets it on its own jobs. It does not automatically affect the calling job. To guarantee that a compliance failure doesn't block the PR regardless of how the harness workflow is implemented, add continue-on-error: true as a top-level job property here as well. If the harness branch does handle this input internally, the two settings are complementary and the job-level property is still the safe approach given that it's a temporary non-blocking window.

@github-actions

Copy link
Copy Markdown
Contributor

posthog-python Compliance Report

Date: 2026-06-30 10:02:24 UTC
Duration: 159559ms

✅ All Tests Passed!

29/29 tests passed


Capture Tests

29/29 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields 517ms
Format Validation.Event Has Uuid 1506ms
Format Validation.Event Has Lib Properties 1505ms
Format Validation.Distinct Id Is String 1507ms
Format Validation.Token Is Present 1507ms
Format Validation.Custom Properties Preserved 1507ms
Format Validation.Event Has Timestamp 1506ms
Retry Behavior.Retries On 503 9516ms
Retry Behavior.Does Not Retry On 400 3506ms
Retry Behavior.Does Not Retry On 401 3506ms
Retry Behavior.Respects Retry After Header 9513ms
Retry Behavior.Implements Backoff 23516ms
Retry Behavior.Retries On 500 7513ms
Retry Behavior.Retries On 502 7508ms
Retry Behavior.Retries On 504 7511ms
Retry Behavior.Max Retries Respected 23530ms
Deduplication.Generates Unique Uuids 1495ms
Deduplication.Preserves Uuid On Retry 7510ms
Deduplication.Preserves Uuid And Timestamp On Retry 14524ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry 7506ms
Deduplication.No Duplicate Events In Batch 1502ms
Deduplication.Different Events Have Different Uuids 1506ms
Compression.Sends Gzip When Enabled 1506ms
Batch Format.Uses Proper Batch Structure 1506ms
Batch Format.Flush With No Events Sends Nothing 1004ms
Batch Format.Multiple Events Batched Together 1504ms
Error Handling.Does Not Retry On 403 3507ms
Error Handling.Does Not Retry On 413 3507ms
Error Handling.Retries On 408 7512ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant