Skip to content

JUnit reporter treats requests with code 500 as testScriptError #3354

Description

@gabrielbergoc

newman version: 6.2.2

JUnit reporter seems to be treating requests with code 500 as testScriptErrors instead of test failures.

My test script:

pm.test(`Response status code is 200`, function () {
    pm.expect(pm.response.code).to.equal(200);
});

const responseData = pm.response.json();
if (responseData && typeof responseData === 'object' && 'message' in responseData) {
    // if there's an error, the API returns an error code with a body like this:
    // { "message": "description of the error" }
    // so I expect to see a test failure with the error message from the API
    pm.expect.fail(`error: ${responseData.message}`);
}

pm.test(\"Response has the correct shape\", function () { // other assertions }

The CLI reporter prints it like this:

   #  failure              detail

 01.  AssertionError       Response status code is 200
      iteration: 1         expected 500 to equal 200
                           at assertion:0 in test-script
                           inside "test-suite-name"

 02.  AssertionError       error: description of the error # note that this is the pm.expect.fail message I wrote
      iteration: 1         at test-script:11:15
                           inside "/price"

The JUnit reporter, instead, writes this result like this:

<?xml version="1.0" encoding="UTF-8"?>
<testsuites name="collection-name" tests="20" time="49.785">
  <testsuite name="test-suite-name" id="d2b44653-e615-40a3-ad6b-51a446f848c2" timestamp="2026-04-01T18:26:50.667Z" tests="2" failures="11" errors="11" time="2.489">
    <system-err><![CDATA[Iteration: 0

---
testScriptError: Error: error: description of the error
    at Object.eval (eval at exec (evalmachine.<anonymous>:66:1819788), <anonymous>:13:24)
    at u.exec (evalmachine.<anonymous>:66:1819823)
    at t.exports (evalmachine.<anonymous>:66:5848)
    at Object.<anonymous> (evalmachine.<anonymous>:66:8247)
    at evalmachine.<anonymous>:15:26
    at Array.forEach (<anonymous>)
    at Object.emit (evalmachine.<anonymous>:14:59)
    at evalmachine.<anonymous>:59:24
    at evalmachine.<anonymous>:5:21
    at evalmachine.<anonymous>:6:18
---
Iteration: 1
...
<!-- there are 20 iterations in total, they all failed with the same errors -->
</system-err>
    <testcase name="Response status code is 200" time="4.845" classname="test-suite-name"/>
    <testcase name="Response has the correct shape" time="4.845" classname="test-suite-name"/>
  </testsuite>
</testsuites>

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions