JUnit reporter's testsuite tests attribute is wrong (last iteration's count, not total unique assertions) on multi-iteration runs

[Adityaj0]

Description

The JUnit reporter groups every execution (i.e. every iteration) of the same collection item into a single <testsuite> element, and emits one <testcase> per unique assertion name seen across all of those iterations:

https://github.com/postmanlabs/newman/blob/develop/lib/reporters/junit/index.js#L47-L157

The tests attribute on that <testsuite>, however, is set inside the per-execution loop:

if (execution.assertions) {
    suite.att('tests', execution.assertions.length);
}
else {
    suite.att('tests', 0);
}

Since this runs once per execution (i.e. once per iteration) and simply overwrites the attribute each time, the final value left on the suite after the loop is whatever the last iteration's assertion count happened to be — not the number of <testcase> elements actually generated for that suite.

Impact

Whenever a collection is run with more than one iteration (--iteration-count / -d with multiple data rows) and the number of assertions differs between iterations (e.g. a conditional test script, or a test that only runs on the first iteration), the reported tests count is wrong and internally inconsistent with the emitted XML — CI systems that parse JUnit XML (Jenkins, GitLab, etc.) will see a <testsuite tests="N"> that doesn't match its actual <testcase> child count, which can under/over-count executed tests in dashboards.

Repro

const EventEmitter = require('events');
const sdk = require('postman-collection');
const JunitReporter = require('./lib/reporters/junit');

const collection = new sdk.Collection({ item: [{ id: 'i1', name: 'Req1', request: 'http://localhost/1' }] });
const item = collection.items.one('i1');

const emitter = new EventEmitter();
emitter.exports = [];
emitter.summary = {
  collection,
  run: {
    executions: [
      { item, id: item.id, cursor: { iteration: 0 }, response: { responseTime: 10 }, assertions: [
          { assertion: 'status is 200', error: null },
          { assertion: 'body has id', error: null },
          { assertion: 'header is set', error: null }
      ] },
      { item, id: item.id, cursor: { iteration: 1 }, response: { responseTime: 10 }, assertions: [
          { assertion: 'status is 200', error: null }
      ] }
    ],
    timings: { started: Date.now() },
    stats: { tests: { total: 4 } }
  }
};

JunitReporter(emitter, {});
emitter.emit('beforeDone');
console.log(emitter.exports[0].content);

Output:

<testsuite name="Req1" id="i1" ... tests="1" failures="0" errors="0" ...>
  <testcase name="status is 200" .../>
  <testcase name="body has id" .../>
  <testcase name="header is set" .../>
</testsuite>

tests="1" but there are 3 <testcase> children.

Fix

Move the tests attribute assignment out of the per-execution loop and set it once from the number of unique assertions actually collected:

suite.att('tests', _.size(tests));

PR with fix + a unit test reproducing this (fails on current develop, passes with the fix): #3372

[Adityaj0]

PR with fix + regression test: #3372