Skip to content

feat: collect all subtype errors in compatibility check + release 0.10.27#725

Merged
lwshang merged 9 commits intomasterfrom
feat/collect-all-subtype-errors
Apr 8, 2026
Merged

feat: collect all subtype errors in compatibility check + release 0.10.27#725
lwshang merged 9 commits intomasterfrom
feat/collect-all-subtype-errors

Conversation

@sasa-tomic
Copy link
Copy Markdown
Member

@sasa-tomic sasa-tomic commented Apr 8, 2026

Release

  • candid 0.10.27
  • candid_parser 0.3.1
  • didc 0.6.1

Summary

  • The compatibility check previously stopped at the first incompatibility, forcing users into a fix-and-retry loop. Now it collects all breaking changes and renders them as a grouped, hierarchical report. The number of reported errors is bounded only by the interface size (no artificial cap); recursion depth is bounded by the existing stack/depth guard (~512 levels).
  • Error messages are clearer: "missing in new interface" instead of "is only in expected type"; "function annotation changed from query to update" instead of "Function mode mismatch".
  • New public API: subtype_check_all(), Incompatibility, format_report() in candid; service_compatibility_report() in candid_parser.

Before (stops at first error)

Method budget_check_v1: func (BudgetCheckRequest) -> (BudgetCheckResult) query
  is not a subtype of func (BudgetCheckRequest/1) -> (BudgetCheckResult/1) query

After (all errors, grouped by method)

Error: 7 incompatible changes found:

  - method "config" is expected by the old interface but missing in the new one
  method "audit":
    - function annotation changed from query to update
    return type:
      record field log: nat is not a subtype of text
      record field count: text is not a subtype of nat
  method "balance":
    return type: text is not a subtype of nat
  method "transfer":
    input type:
      record field amount: nat is not a subtype of text
    return type:
      record field ok: text is not a subtype of bool
      record field balance: text is not a subtype of nat

Changes

File What
rust/candid/src/types/subtype.rs Incompatibility struct, subtype_check_all(), format_report(), internal subtype_collect_()
rust/candid_parser/src/utils.rs service_compatibility_report()
tools/didc/src/main.rs didc check uses new report
tools/ui/src/didjs/lib.rs Web UI uses new report
rust/candid_parser/tests/compatibility.rs 29 new tests

Test plan

  • 8 tests: backward-compatible changes (add opt field, add method, widen nat→int in input, etc.) must NOT be flagged
  • 6 tests: backward-incompatible changes (remove method, change type, add required field, etc.) must be caught
  • 6 tests: multiple errors are ALL collected (multiple methods, multiple fields, both input+return)
  • 3 tests: error message quality (mentions method names, types, clear wording)
  • 3 tests: hierarchical report formatting (grouping, inlining, empty for compatible)
  • 3 tests: variant/edge cases
  • All 175 existing tests continue to pass

The compatibility check previously stopped at the first incompatibility,
forcing users to fix-and-retry in a loop. Now it collects every breaking
change and renders them as a grouped, hierarchical report:

  method "transfer":
    input type:
      record field amount: nat is not a subtype of text
    return type:
      record field ok: text is not a subtype of bool
      record field balance: text is not a subtype of nat
  method "get_user":
    - missing in new interface

Key changes:
- Add `subtype_check_all` / `Incompatibility` struct in subtype.rs
- Add `format_report` for hierarchical grouped display
- Add `service_compatibility_report` in candid_parser utils
- Update `didc check` and didjs UI to show full report
- 29 new tests covering compatible passes, incompatible catches,
  multi-error collection, error message quality, and formatting
- 29 tests → 18: dropped redundant tests, folded table-driven cases
- Fixed buggy test (compatible_add_variant_case_in_return tested identity)
- DRY: table-driven helpers for compatible/incompatible pass/fail checks
- Added missing coverage: vec element type, multi-arg functions, mixed
  compatible+incompatible methods, service_compatible vs report agreement,
  pathless errors in format_report, remove field from input record
- Every test now asserts something the others don't
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 8, 2026

Name Max Mem (Kb) Encode Decode
blob 4_224 4_207_756 ($\textcolor{red}{0.01\%}$) 2_122_083 ($\textcolor{red}{0.00\%}$)
btreemap 75_456 536_171_057 ($\textcolor{red}{0.00\%}$) 10_185_892_755 ($\textcolor{red}{0.88\%}$)
double_option 128 1_321_763 ($\textcolor{red}{0.37\%}$) 28_922_393 ($\textcolor{red}{0.12\%}$)
large_variant 320 1_045_484 ($\textcolor{red}{0.20\%}$) 21_225_794 ($\textcolor{red}{0.51\%}$)
multi_arg 64 551_056 ($\textcolor{red}{0.26\%}$) 6_594_641 ($\textcolor{red}{0.52\%}$)
nns 192 2_027_160 ($\textcolor{red}{0.29\%}$) 5_656_329 ($\textcolor{red}{0.43\%}$)
nns_list_neurons 1_152 6_650_608 ($\textcolor{red}{0.17\%}$) 220_248_227 ($\textcolor{red}{0.01\%}$)
nns_list_proposal 1_216 7_046_366 ($\textcolor{red}{0.40\%}$) 60_862_842 ($\textcolor{green}{-0.03\%}$)
option_list 128 717_275 ($\textcolor{red}{0.08\%}$) 16_789_150 ($\textcolor{green}{-0.52\%}$)
result_variant 192 1_325_080 ($\textcolor{red}{0.17\%}$) 17_993_824 ($\textcolor{red}{0.46\%}$)
subtype_decode 512 2_654_127 ($\textcolor{red}{0.08\%}$) 50_815_825 ($\textcolor{red}{0.23\%}$)
text 6_336 4_204_597 ($\textcolor{red}{0.01\%}$) 7_877_544 ($\textcolor{red}{0.00\%}$)
variant_list 128 711_413 ($\textcolor{red}{0.04\%}$) 15_981_399 ($\textcolor{red}{0.49\%}$)
vec_int16 12_480 8_405_171 ($\textcolor{red}{0.01\%}$) 249_586_296 ($\textcolor{red}{0.00\%}$)
vec_nat 11_008 67_096_148 ($\textcolor{red}{0.00\%}$) 277_077_517 ($\textcolor{red}{0.00\%}$)
vec_nat32 24_768 16_793_779 ($\textcolor{red}{0.00\%}$) 243_295_129 ($\textcolor{red}{0.00\%}$)
vec_nat64 49_344 33_570_977 ($\textcolor{red}{0.00\%}$) 251_684_001 ($\textcolor{red}{0.00\%}$)
vec_service 64 781_319 ($\textcolor{red}{0.02\%}$) 96_311_260 ($\textcolor{green}{-0.00\%}$)
wide_record 1_152 3_271_804 ($\textcolor{red}{0.07\%}$) 48_410_050 ($\textcolor{green}{-0.00\%}$)
  • Parser cost: 16_174_059 ($\textcolor{green}{-0.00\%}$)
  • Extra args: 2_859_731 ($\textcolor{green}{-0.00\%}$)
Click to see raw report
---------------------------------------------------

Benchmark: blob
  total:
    instructions: 6.33 M (0.00%) (change within noise threshold)
    heap_increase: 66 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 4.21 M (0.01%) (change within noise threshold)
    heap_increase: 66 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 2.12 M (0.00%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: btreemap
  total:
    instructions: 10.72 B (0.83%) (change within noise threshold)
    heap_increase: 1179 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 536.17 M (0.00%) (change within noise threshold)
    heap_increase: 159 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 10.19 B (0.88%) (change within noise threshold)
    heap_increase: 1020 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: double_option
  total:
    instructions: 30.25 M (0.13%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 1.32 M (0.37%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 28.92 M (0.12%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: extra_args
  total:
    instructions: 2.86 M (-0.00%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: large_variant
  total:
    instructions: 22.27 M (0.49%) (change within noise threshold)
    heap_increase: 5 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 1.05 M (0.20%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 21.23 M (0.51%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: multi_arg
  total:
    instructions: 7.15 M (0.50%) (change within noise threshold)
    heap_increase: 1 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 551.06 K (0.26%) (change within noise threshold)
    heap_increase: 1 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 6.59 M (0.52%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: nns
  total:
    instructions: 24.70 M (0.13%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

  0. Parsing (scope):
    calls: 1 (no change)
    instructions: 16.17 M (-0.00%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 2.03 M (0.29%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 5.66 M (0.43%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: nns_list_neurons
  total:
    instructions: 226.90 M (0.02%) (change within noise threshold)
    heap_increase: 18 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 6.65 M (0.17%) (change within noise threshold)
    heap_increase: 18 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 220.25 M (0.01%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: nns_list_proposal
  total:
    instructions: 67.91 M (0.02%) (change within noise threshold)
    heap_increase: 19 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 7.05 M (0.40%) (change within noise threshold)
    heap_increase: 5 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 60.86 M (-0.03%) (change within noise threshold)
    heap_increase: 14 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: option_list
  total:
    instructions: 17.51 M (-0.50%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 717.27 K (0.08%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 16.79 M (-0.52%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: result_variant
  total:
    instructions: 19.32 M (0.44%) (change within noise threshold)
    heap_increase: 3 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 1.33 M (0.17%) (change within noise threshold)
    heap_increase: 1 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 17.99 M (0.46%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: subtype_decode
  total:
    instructions: 53.47 M (0.22%) (change within noise threshold)
    heap_increase: 8 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 2.65 M (0.08%) (change within noise threshold)
    heap_increase: 8 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 50.82 M (0.23%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: text
  total:
    instructions: 12.08 M (0.00%) (change within noise threshold)
    heap_increase: 99 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 4.20 M (0.01%) (change within noise threshold)
    heap_increase: 66 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 7.88 M (0.00%) (change within noise threshold)
    heap_increase: 33 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: variant_list
  total:
    instructions: 16.70 M (0.47%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 711.41 K (0.04%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 15.98 M (0.49%) (change within noise threshold)
    heap_increase: 2 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: vec_int16
  total:
    instructions: 257.99 M (0.00%) (change within noise threshold)
    heap_increase: 195 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 8.41 M (0.01%) (change within noise threshold)
    heap_increase: 130 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 249.59 M (0.00%) (change within noise threshold)
    heap_increase: 65 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: vec_nat
  total:
    instructions: 344.18 M (0.00%) (change within noise threshold)
    heap_increase: 172 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 67.10 M (0.00%) (change within noise threshold)
    heap_increase: 33 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 277.08 M (0.00%) (change within noise threshold)
    heap_increase: 139 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: vec_nat32
  total:
    instructions: 260.09 M (0.00%) (change within noise threshold)
    heap_increase: 387 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 16.79 M (0.00%) (change within noise threshold)
    heap_increase: 258 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 243.30 M (0.00%) (change within noise threshold)
    heap_increase: 129 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: vec_nat64
  total:
    instructions: 285.26 M (0.00%) (change within noise threshold)
    heap_increase: 771 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 33.57 M (0.00%) (change within noise threshold)
    heap_increase: 514 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 251.68 M (0.00%) (change within noise threshold)
    heap_increase: 257 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: vec_service
  total:
    instructions: 97.09 M (0.00%) (change within noise threshold)
    heap_increase: 1 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 781.32 K (0.02%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 96.31 M (-0.00%) (change within noise threshold)
    heap_increase: 1 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Benchmark: wide_record
  total:
    instructions: 51.68 M (0.00%) (change within noise threshold)
    heap_increase: 18 pages (no change)
    stable_memory_increase: 0 pages (no change)

  1. Encoding (scope):
    calls: 1 (no change)
    instructions: 3.27 M (0.07%) (change within noise threshold)
    heap_increase: 18 pages (no change)
    stable_memory_increase: 0 pages (no change)

  2. Decoding (scope):
    calls: 1 (no change)
    instructions: 48.41 M (-0.00%) (change within noise threshold)
    heap_increase: 0 pages (no change)
    stable_memory_increase: 0 pages (no change)

---------------------------------------------------

Summary:
  instructions:
    status:   No significant changes 👍
    counts:   [total 20 | regressed 0 | improved 0 | new 0 | unchanged 20]
    change:   [max +88.68M | p75 +48.67K | median +7.13K | p25 +453 | min -87.50K]
    change %: [max +0.83% | p75 +0.28% | median +0.01% | p25 0.00% | min -0.50%]

  heap_increase:
    status:   No significant changes 👍
    counts:   [total 20 | regressed 0 | improved 0 | new 0 | unchanged 20]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

  stable_memory_increase:
    status:   No significant changes 👍
    counts:   [total 20 | regressed 0 | improved 0 | new 0 | unchanged 20]
    change:   [max 0 | p75 0 | median 0 | p25 0 | min 0]
    change %: [max 0.00% | p75 0.00% | median 0.00% | p25 0.00% | min 0.00%]

---------------------------------------------------
Successfully persisted results to canbench_results.yml

@sasa-tomic sasa-tomic marked this pull request as ready for review April 8, 2026 09:33
@sasa-tomic sasa-tomic requested a review from a team as a code owner April 8, 2026 09:33
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR enhances the Candid service compatibility/subtyping workflow so it reports all incompatibilities in one pass (with a hierarchical, grouped report), and wires that reporting into CLI/UI consumers while adding parser-level helpers and test coverage.

Changes:

  • Add Incompatibility, subtype_check_all(), and format_report() to collect and render all subtype incompatibilities.
  • Add service_compatibility_report() in candid_parser and update didc check + web UI to display the new report.
  • Add extensive compatibility/report-formatting tests.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tools/ui/src/didjs/lib.rs Switch UI subtype check to collect all incompatibilities and display formatted report.
tools/didc/src/main.rs Update didc check to print grouped incompatibility report and fail with breaking-change summary.
rust/candid/src/types/subtype.rs Implement incompatibility collection + hierarchical report formatting and expose new public APIs.
rust/candid/src/types/internal.rs Adjust internal TypeId ID computation casting.
rust/candid_parser/src/utils.rs Add service_compatibility_report() helper returning all incompatibilities.
rust/candid_parser/tests/compatibility.rs Add new tests validating compatibility detection, message/path quality, and report formatting.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread rust/candid/src/types/subtype.rs Outdated
Comment thread rust/candid/src/types/subtype.rs Outdated
Comment thread rust/candid/src/types/subtype.rs Outdated
lwshang and others added 3 commits April 8, 2026 16:01
Three tests covering the contravariant (input type) path for record field
missing, variant case removed, and arg-count mismatch error messages.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Thread `is_input: bool` through `subtype_collect_` so record and variant
error messages correctly identify which side is missing a field/case in
the contravariant (function input) checking path. Also fix the swapped
old/new arg-count values in the `check_func_params` input branch.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@lwshang lwshang changed the title feat: collect all subtype errors in compatibility check feat: collect all subtype errors in compatibility check + release 0.10.27 Apr 8, 2026
@lwshang lwshang merged commit 36a8034 into master Apr 8, 2026
11 checks passed
@lwshang lwshang deleted the feat/collect-all-subtype-errors branch April 8, 2026 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants