Skip to content

test: skip tests that cause timeouts on IBM i #59014

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

abmusse
Copy link
Contributor

@abmusse abmusse commented Jul 10, 2025

I missed marking a couple of test cases flaky in my previous PR:
#58583

tests missed were:

  • parallel.test-inspector-network-content-type
  • benchmark.test-benchmark-websocket

After marking the above flaky I re-ran a test build and noticed
the build still failed.

Unfortuantely the build still fails because the build detects that
node process are left behind after the tests, presumably because tests
that spawn child processes are being timed out and only the parent
process is killed off, leaving the child processes behind.

see: #59014 (comment)

We've decided to skip these tests until the underlying issue is
resolved.

The tests being skipped are:

  • parallel.test-http-proxy-fetch
  • parallel.test-https-proxy-fetch
  • benchmark.test-benchmark-websocket
  • parallel.test-inspector-network-fetch
  • parallel.test-inspector-network-content-type
  • es-module.test-wasm-web-api
  • parallel.test-fetch
  • parallel.test-without-async-context-frame
  • report.test-report-exclude-network

@nodejs-github-bot nodejs-github-bot added needs-ci PRs that need a full CI run. test Issues and PRs related to the tests. labels Jul 10, 2025
Copy link

codecov bot commented Jul 10, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.08%. Comparing base (09b4c57) to head (db5c570).
Report is 6 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff            @@
##             main   #59014    +/-   ##
========================================
  Coverage   90.07%   90.08%            
========================================
  Files         641      645     +4     
  Lines      188998   189128   +130     
  Branches    37069    37094    +25     
========================================
+ Hits       170246   170370   +124     
- Misses      11462    11464     +2     
- Partials     7290     7294     +4     

see 44 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@richardlau
Copy link
Member

Test build: https://ci.nodejs.org/job/node-test-commit-ibmi/nodes=ibmi74-ppc64/2003/console

So unfortunately the build still fails because the build detects that node process are left behind after the tests, presumably because tests that spawn child processes are being timed out and only the parent process is killed off, leaving the child processes behind:

05:57:54 All tests passed.
05:57:54 "/home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/node" ./test/embedding/test-embedding.js
05:58:14 ps awwx | grep Release/node | grep -v grep | cat
05:58:14  5257763      - A    28:20 /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/out/Release/node /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/test/fixtures/fetch-and-log.mjs 
05:58:14  5258203      - A    27:59 /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/out/Release/node /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/test/fixtures/fetch-and-log.mjs 
05:58:14  5265655      - A     0:00 /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/out/Release/node --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/benchmark/run.js test websocket 
05:58:14  5265656      - A     0:00 /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/out/Release/node --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/benchmark/websocket/simple.js --test 
05:58:14  5265660      - A    13:38 /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/out/Release/node --test-reporter=./test/common/test-error-reporter.js --test-reporter-destination=stdout /home/IOJS/build/workspace/node-test-commit-ibmi/nodes/ibmi74-ppc64/benchmark/websocket/simple.js roundtrips=1 useBinary=true size=1 
05:58:14 gmake[1]: *** [Makefile:609: test-ci] Error 1

@abmusse
Copy link
Contributor Author

abmusse commented Jul 10, 2025

So unfortunately the build still fails because the build detects that node process are left behind after the tests, presumably because tests that spawn child processes are being timed out and only the parent process is killed off, leaving the child processes behind

Thanks for pointing that out. Since that behavior is happening do you think we should skip the tests (until we resolve the underlying issue) so that they don't spawn the child processes?

@richardlau
Copy link
Member

So unfortunately the build still fails because the build detects that node process are left behind after the tests, presumably because tests that spawn child processes are being timed out and only the parent process is killed off, leaving the child processes behind

Thanks for pointing that out. Since that behavior is happening do you think we should skip the tests (until we resolve the underlying issue) so that they don't spawn the child processes?

Yes, let's go with skipping until we figure out why they're timing out.

@richardlau richardlau added the ibm i Issues and PRs related to the IBM i platform. label Jul 10, 2025
abmusse added a commit to abmusse/node that referenced this pull request Jul 10, 2025
I missed adding a couple of test cases in my previous PR:
nodejs#58583

We have to skip these instead of marking as flaky
because the build still fails because the build detects that node process
are left behind after the tests, presumably because tests that spawn
child processes are being timed out and only the parent process
is killed off, leaving the child processes behind.

see: nodejs#59014 (comment)

The tests being skipped are:

- parallel.test-http-proxy-fetch
- parallel.test-https-proxy-fetch
- benchmark.test-benchmark-websocket
- parallel.test-inspector-network-fetch
- parallel.test-inspector-network-content-type
- es-module.test-wasm-web-api
- parallel.test-fetch
- parallel.test-without-async-context-frame
- report.test-report-exclude-network
abmusse added a commit to abmusse/node that referenced this pull request Jul 10, 2025
I missed marking a couple of test cases flaky in my previous PR:
nodejs#58583

tests missed were:

- parallel.test-inspector-network-content-type
- benchmark.test-benchmark-websocket

After marking the above flaky I re-ran a test build and noticed
the build still failed.

Unfortuantely the build still fails because the build detects that node process
are left behind after the tests, presumably because tests that spawn
child processes are being timed out and only the parent process
is killed off, leaving the child processes behind.

see: nodejs#59014 (comment)

We've decided to skip these tests until the underlying issue is
resolved.

The tests being skipped are:
- parallel.test-http-proxy-fetch
- parallel.test-https-proxy-fetch
- benchmark.test-benchmark-websocket
- parallel.test-inspector-network-fetch
- parallel.test-inspector-network-content-type
- es-module.test-wasm-web-api
- parallel.test-fetch
- parallel.test-without-async-context-frame
- report.test-report-exclude-network
@abmusse abmusse force-pushed the ibmi-mark-timeouts-flaky-again branch from b2cf9dd to 8902afc Compare July 10, 2025 22:49
@abmusse abmusse changed the title test: mark timeouts as flaky on IBM i again test: skip tests that cause timeouts on IBM i Jul 10, 2025
I missed marking a couple of test cases flaky in my previous PR:
nodejs#58583

tests missed were:

- parallel.test-inspector-network-content-type
- benchmark.test-benchmark-websocket

After marking the above flaky I re-ran a test build and noticed
the build still failed.

Unfortuantely the build still fails because the build detects that
node process are left behind after the tests, presumably because tests
that spawn child processes are being timed out and only the parent
process is killed off, leaving the child processes behind.

see: nodejs#59014 (comment)

We've decided to skip these tests until the underlying issue is
resolved.

The tests being skipped are:
- parallel.test-http-proxy-fetch
- parallel.test-https-proxy-fetch
- benchmark.test-benchmark-websocket
- parallel.test-inspector-network-fetch
- parallel.test-inspector-network-content-type
- es-module.test-wasm-web-api
- parallel.test-fetch
- parallel.test-without-async-context-frame
- report.test-report-exclude-network
@abmusse abmusse force-pushed the ibmi-mark-timeouts-flaky-again branch from 8902afc to db5c570 Compare July 10, 2025 22:55
@abmusse
Copy link
Contributor Author

abmusse commented Jul 10, 2025

@richardlau richardlau added the request-ci Add this label to start a Jenkins CI on a PR. label Jul 11, 2025
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jul 11, 2025
@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@mhdawson mhdawson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ibm i Issues and PRs related to the IBM i platform. needs-ci PRs that need a full CI run. test Issues and PRs related to the tests.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants