-
Notifications
You must be signed in to change notification settings - Fork 66
feat: SDK Compliance #424
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
feat: SDK Compliance #424
Conversation
posthog-ios Compliance ReportDate: 2026-01-28 13:11:43 UTC
|
| Test | Status | Duration |
|---|---|---|
| Format Validation.Event Has Required Fields | ✅ | 2174ms |
| Format Validation.Event Has Uuid | ✅ | 2135ms |
| Format Validation.Event Has Lib Properties | ✅ | 2204ms |
| Format Validation.Distinct Id Is String | ✅ | 2226ms |
| Format Validation.Token Is Present | ✅ | 2189ms |
| Format Validation.Custom Properties Preserved | ✅ | 2324ms |
| Format Validation.Event Has Timestamp | ✅ | 2152ms |
| Retry Behavior.Retries On 503 | ❌ | 7274ms |
| Retry Behavior.Does Not Retry On 400 | ✅ | 4299ms |
| Retry Behavior.Does Not Retry On 401 | ✅ | 4232ms |
| Retry Behavior.Respects Retry After Header | ❌ | 7240ms |
| Retry Behavior.Implements Backoff | ❌ | 17242ms |
| Retry Behavior.Retries On 500 | ❌ | 7204ms |
| Retry Behavior.Retries On 502 | ❌ | 7201ms |
| Retry Behavior.Retries On 504 | ❌ | 7208ms |
| Retry Behavior.Max Retries Respected | ❌ | 17276ms |
| Deduplication.Generates Unique Uuids | ✅ | 2331ms |
| Deduplication.Preserves Uuid On Retry | ❌ | 7223ms |
| Deduplication.Preserves Uuid And Timestamp On Retry | ❌ | 12228ms |
| Deduplication.Preserves Uuid And Timestamp On Batch Retry | ❌ | 7372ms |
| Deduplication.No Duplicate Events In Batch | ✅ | 2415ms |
| Deduplication.Different Events Have Different Uuids | ✅ | 2284ms |
| Compression.Sends Gzip When Enabled | ✅ | 2303ms |
| Batch Format.Uses Proper Batch Structure | ✅ | 2184ms |
| Batch Format.Flush With No Events Sends Nothing | ✅ | 2166ms |
| Batch Format.Multiple Events Batched Together | ❌ | 2349ms |
| Error Handling.Does Not Retry On 403 | ✅ | 4132ms |
| Error Handling.Does Not Retry On 413 | ✅ | 4229ms |
| Error Handling.Retries On 408 | ❌ | 7206ms |
Failures
retry_behavior.retries_on_503
Expected at least 3 requests, got 1
retry_behavior.respects_retry_after_header
Expected at least 2 requests, got 1
retry_behavior.implements_backoff
Expected at least 3 requests, got 1
retry_behavior.retries_on_500
Expected at least 2 requests, got 1
retry_behavior.retries_on_502
Expected at least 2 requests, got 1
retry_behavior.retries_on_504
Expected at least 2 requests, got 1
retry_behavior.max_retries_respected
Expected 4 requests, got 1
deduplication.preserves_uuid_on_retry
Need at least 2 requests to check retry
deduplication.preserves_uuid_and_timestamp_on_retry
Expected at least 3 requests, got 1
deduplication.preserves_uuid_and_timestamp_on_batch_retry
UUIDs changed on retry: ['019C04BA-A776-7065-B102-C9E08B43C20B'] != ['019C04BA-A7D9-7E42-8B07-5E1972DF5189', '019C04BA-A807-7594-89B6-6DE51431B533']
batch_format.multiple_events_batched_together
Expected 1 requests, got 3
error_handling.retries_on_408
Expected at least 2 requests, got 1
💡 Motivation and Context
We need a language-agnostic way to verify that all PostHog SDKs implement the capture API correctly. The SDK Test Harness provides a shared contract and test runner. This PR integrates it into this SDK.
This adds:
The test harness runs a mock PostHog server, exercises the SDK through the adapter, and verifies behavior matches the contract defined in CONTRACT.yaml (event format, retries, compression, deduplication, etc.).
#skip-changelog
💚 How did you test it?
The compliance tests run as part of this PR's CI. The adapter was also tested locally via
docker-compose.📝 Checklist