test(openfeature): add null targeting key test coverage by leoromanovsky · Pull Request #5523 · DataDog/dd-trace-rb

leoromanovsky · 2026-03-27T18:54:36Z

What does this PR do?

Adds test coverage for OpenFeature spec OF.2 (optional targeting key) to prove libdatadog handles nil targeting key correctly via the Ruby FFI layer.

Motivation:

The OpenFeature spec (Requirement 3.1.1, OF.2) defines targeting key as optional. The correct behavior (matching libdatadog's reference implementation) is:

	Static (no rules, no shards)	Split (shards)	Targeted (rules)
`""` empty string	Returns value	Hashes `""`, returns value	Evaluates rules with `id=""`, returns value
`null`/missing	Returns value	`TARGETING_KEY_MISSING` error, returns default	Evaluates rules (no `id` fallback), returns value or falls through

Ruby's evaluator delegates to libdatadog which already handles this correctly — the C extension passes NULL when targeting_key is absent from the context hash. This PR adds explicit test coverage proving the behavior.

Change log entry

No customer-visible change. Test-only — adds coverage for optional targeting key behavior that already works correctly.

Additional Notes:

Tests placed in spec/datadog/open_feature/ so @DataDog/feature-flagging-and-experimentation-sdk is auto-requested via CODEOWNERS
Tests call Configuration#get_assignment directly to exercise the Rust FFI layer without mocks
Used partial shard range (0-4999 of 10000) instead of full range to avoid libdatadog's insignificant shard optimization
No logic changes to Ruby code

Question for reviewers: The existing evaluation_engine_spec.rb mocks NativeEvaluator, so there's no integration spec testing the full OpenFeature provider → native evaluator → libdatadog path with real flag configs. Should we add one?

How to test the change?

bundle exec rspec spec/datadog/open_feature/null_targeting_key_spec.rb

3 scenarios:

Static flag + nil targeting key → returns value with STATIC reason
Sharded flag + nil targeting key → returns TARGETING_KEY_MISSING error
Rule-match flag + nil targeting key → returns value with TARGETING_MATCH reason

Next Steps

Add test-case-null-targeting-key.json to ffe-system-test-data shared fixtures (separate fixture centralization project)
Wire the fixture loop spec to cover null targeting key via JSON fixtures
System test coverage for null targeting key across all tracers

…ags_spec - Add 3 new test cases: static flag (returns value), sharded flag (returns TARGETING_KEY_MISSING), rule-match flag (returns value) - All 3 scenarios pass empty context hash to simulate nil targeting key - Confirms Ruby FFI correctly delegates null targeting key handling to libdatadog

github-actions · 2026-03-27T18:54:47Z

Thank you for updating Change log entry section 👏

^{Visited at: 2026-03-27 21:29:50 UTC}

datadog-datadog-prod-us1 · 2026-03-27T19:12:24Z

⚠️ Tests

✨ Fix all issues with BitsAI or with Cursor

⚠️ Warnings

❄️ 3 New flaky tests detected

OpenFeature OF.2: Optional Targeting Key with nil targeting key evaluates rule-match flag when rule matches on non-id attribute

from rspec

(Fix with Cursor)

wrong number of arguments (given 1, expected 0)

Failure/Error: let(:config) { Datadog::Core::FeatureFlags::Configuration.new(flags_json) }

ArgumentError:
  wrong number of arguments (given 1, expected 0)
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`initialize'
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`new'
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`block (2 levels) in <top (required)>'
./spec/datadog/open_feature/null_targeting_key_spec.rb:66:in \`block (3 levels) in <top (required)>'
...

OpenFeature OF.2: Optional Targeting Key with nil targeting key evaluates static flag successfully from rspec

(Fix with Cursor)

wrong number of arguments (given 1, expected 0)

Failure/Error: let(:config) { Datadog::Core::FeatureFlags::Configuration.new(flags_json) }

ArgumentError:
  wrong number of arguments (given 1, expected 0)
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`initialize'
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`new'
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`block (2 levels) in <top (required)>'
./spec/datadog/open_feature/null_targeting_key_spec.rb:53:in \`block (3 levels) in <top (required)>'
...

OpenFeature OF.2: Optional Targeting Key with nil targeting key returns TARGETING_KEY_MISSING for sharded flag from rspec

(Fix with Cursor)

wrong number of arguments (given 1, expected 0)

Failure/Error: let(:config) { Datadog::Core::FeatureFlags::Configuration.new(flags_json) }

ArgumentError:
  wrong number of arguments (given 1, expected 0)
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`initialize'
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`new'
./spec/datadog/open_feature/null_targeting_key_spec.rb:49:in \`block (2 levels) in <top (required)>'
./spec/datadog/open_feature/null_targeting_key_spec.rb:60:in \`block (3 levels) in <top (required)>'
...

ℹ️ Info

No other issues found (see more)

🧪 All tests passed

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 95.35% (+0.01%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 460509d | Docs | Datadog PR Page | Was this helpful? React with 👍/👎 or give us feedback!}

… for FFE team CODEOWNERS visibility

…lable Configuration.new(json) is defined by the C extension as a singleton method. Without the extension, Ruby's default Class#new calls initialize with 0 args, causing ArgumentError.

pr-commenter · 2026-03-31T03:43:12Z

Benchmarks

Benchmark execution time: 2026-03-31 03:43:09

Comparing candidate commit 460509d in PR branch leo.romanovsky/fix-optional-targeting-key with baseline commit eac7550 in branch master.

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

dd-oleksii · 2026-03-31T11:27:25Z

spec/datadog/open_feature/null_targeting_key_spec.rb

+# Tests for OpenFeature spec OF.2: optional targeting key
+# See: https://openfeature.dev/specification/sections/evaluation-context#requirement-311
+#
+# When targeting key is missing (nil), SDKs must still attempt evaluation:
+# - Static flags (no shards): return value normally
+# - Sharded flags: return TARGETING_KEY_MISSING error
+# - Rule-match flags (no shards): return value if rule matches on non-id attribute
+
+RSpec.describe 'OpenFeature OF.2: Optional Targeting Key' do


minor: OF.2 is our internal SDK spec identifier, not openfeature

github-actions bot added the dev/testing Involves testing processes (e.g. RSpec) label Mar 27, 2026

leoromanovsky added 2 commits March 27, 2026 17:28

refactor: move null targeting key tests to spec/datadog/open_feature/…

7b03379

… for FFE team CODEOWNERS visibility

fix(test): skip null targeting key tests when native extension unavai…

dc46453

…lable Configuration.new(json) is defined by the C extension as a singleton method. Without the extension, Ruby's default Class#new calls initialize with 0 args, causing ArgumentError.

leoromanovsky marked this pull request as ready for review March 30, 2026 18:34

leoromanovsky requested a review from a team as a code owner March 30, 2026 18:34

leoromanovsky requested review from dd-oleksii and typotter March 30, 2026 18:34

Merge branch 'master' into leo.romanovsky/fix-optional-targeting-key

460509d

dd-oleksii approved these changes Mar 31, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(openfeature): add null targeting key test coverage#5523

test(openfeature): add null targeting key test coverage#5523
leoromanovsky wants to merge 4 commits intomasterfrom
leo.romanovsky/fix-optional-targeting-key

leoromanovsky commented Mar 27, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 27, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1 bot commented Mar 27, 2026 •

edited

Loading

Uh oh!

pr-commenter bot commented Mar 31, 2026

Explanation

More details about the CI and significant changes

Uh oh!

dd-oleksii Mar 31, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

leoromanovsky commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Next Steps

Uh oh!

github-actions bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

datadog-datadog-prod-us1 bot commented Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

pr-commenter bot commented Mar 31, 2026

Benchmarks

Explanation

More details about the CI and significant changes

Uh oh!

dd-oleksii Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

leoromanovsky commented Mar 27, 2026 •

edited

Loading

github-actions bot commented Mar 27, 2026 •

edited

Loading

datadog-datadog-prod-us1 bot commented Mar 27, 2026 •

edited

Loading