Skip to content

Run OZ tests by file or filter, and choose port - #286

Open
arner wants to merge 2 commits into
hyperledger:mainfrom
arner:split-tests
Open

Run OZ tests by file or filter, and choose port#286
arner wants to merge 2 commits into
hyperledger:mainfrom
arner:split-tests

Conversation

@arner

@arner arner commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

This PR improves the developer experience of working with the OpenZeppelin hardhat tests.

Signed-off-by: Arne Rutjes <arne123@gmail.com>
@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 24.80%. Comparing base (aaad3b9) to head (f9a37b8).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #286   +/-   ##
=======================================
  Coverage   24.80%   24.80%           
=======================================
  Files          74       74           
  Lines       10736    10736           
=======================================
  Hits         2663     2663           
  Misses       7842     7842           
  Partials      231      231           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Jul 28, 2026

Copy link
Copy Markdown

✅ OZ Hardhat Compatibility — 84.1% passing (4755/5652)

No regressions, no stale entries.

Full breakdown — 4755 passed, 896 failed, 1 skipped (5652 total, 84.1% passing)

By suite

  • access: 354/578 passing (61%)
  • crosschain: 10/15 passing (67%)
  • finance: 0/2 passing (0%)
  • governance: 77/185 passing (42%)
  • metatx: 0/12 passing (0%)
  • proxy: 194/252 passing (77%)
  • token: 1637/1852 passing (88%)
  • utils: 2483/2755 passing (90%)

Expected failures by cause (896)

  • max-code-size: 212
  • eth_signTypedData_v4: 165
  • execution reverted: 152
  • insufficient-funds: 94
  • fixed-timestamp: 68
  • gas-stub: 36
  • hardhat_impersonateAccount: 29
  • Expected transaction to be reverted with custom error 'AccessManagerNotScheduled', but it reverted with custom error 'AccessManagerUnauthorizedAccount': 27
  • fixed-block-number: 17
  • Expected transaction to be reverted without a reason, but it didn't revert: 9
  • eth_getProof: 6
  • evm_setAutomine: 6
  • personal_sign: 5
  • Expected transaction to be reverted with custom error 'FailedCall', but it reverted with custom error 'TimelockUnexpectedOperationState': 4
  • expected 0 to equal 864000.: 4
  • expected 1000180 to equal 1785317293.: 4
  • expected false to be true: 3
  • hardhat_setStorageAt: 3
  • Error in "RoleGranted" event: Error in the 4th argument assertion: expected 1000000 to equal 1785317122.: 2
  • Error in "RoleGranted" event: Error in the 4th argument assertion: expected 1000000 to equal 1786526721.: 2
  • Error in "RoleGranted" event: Error in the 4th argument assertion: expected 1345600 to equal 1785662722.: 2
  • Expected transaction to be reverted with custom error 'AccessManagerNotScheduled', but it reverted with a different custom error: 2
  • Expected transaction to be reverted with custom error 'FailedCall', but it didn't revert: 2
  • expected 0 to equal 10.: 2
  • expected 1000000 to equal 1785317113.: 2
  • expected 1345600 to equal 1785662722.: 2
  • expected 1786181114 to equal 1786181113.: 2
  • expected 518400 to equal 172800.: 2
  • expected Result(2) [ 10000n, 0n ] to deeply equal [ 0n, 10000n ]: 2
  • expected Result(2) [ 17n, 9983n ] to deeply equal [ 15n, 9985n ]: 2
  • expected Result(2) [ 42n, 9958n ] to deeply equal [ 40n, 9960n ]: 2
  • expected Result(2) [ 5000n, 5000n ] to deeply equal [ 0n, 10000n ]: 2
  • Error in "RoleGranted" event: Error in the 4th argument assertion: expected 1000000 to equal 1785317121.: 1
  • Expected transaction NOT to be reverted: 1
  • Expected transaction to be reverted with custom error 'AccessManagerUnauthorizedCall', but it didn't revert: 1
  • Expected transaction to be reverted with custom error 'TimelockUnexecutedPredecessor', but it reverted with custom error 'TimelockUnexpectedOperationState': 1
  • Expected transaction to be reverted with panic code 0x01 (Assertion error), but it reverted with a custom error: 1
  • expected '0x825f7f53b82abaa9f2aaed46dd265460f3a…' to equal '0xd95ed73cb4091f22c77fc6b8c487f5ff168…': 1
  • expected 0 to equal 36000.: 1
  • expected 0 to equal 950400.: 1
  • expected 1086400 to equal 1785403535.: 1
  • expected 1086400 to equal 1785403536.: 1
  • expected 1259200 to equal 1785576325.: 1
  • expected 1786181114 to equal 0.: 1
  • expected 36000 to equal 4919.: 1
  • expected 4919 to equal 0.: 1
  • expected 9941 to equal 9958.: 1
  • expected Result(2) [ 10000n, 0n ] to deeply equal [ 9953n, 47n ]: 1
  • expected Result(2) [ 10000n, 10000n ] to deeply equal [ 9962n, 10038n ]: 1
  • expected Result(2) [ 10n, 9990n ] to deeply equal [ 0n, 10000n ]: 1
  • expected Result(2) [ 42n, 9958n ] to deeply equal [ 0n, 10000n ]: 1
  • expected [ false, '0' ] to deeply equal [ true, '0' ]: 1
  • failed to revert Store to block 9584: snapshot for block 9584 not found: 1
  • invalid argument 1: json: cannot unmarshal hex string of odd length into Go value of type common.Hash: 1
  • out of gas: 1
  • transaction already pending: 1

Adds --port (PORT= via make) so a second suite run can go alongside one already in progress, instead
of the two fighting over 8545. start_testnode now refuses a port that is already listening rather than
killing whatever holds it.

Signed-off-by: Arne Rutjes <arne123@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant