UTF-8 does not reset state when returning error

[ChALkeR]

What is the issue with the Encoding Standard?

Same as #358 but for Unicode BOM
If the proposal of #358 is to reset state for errors, then what should happen to BOM seen?

I don't argue that it should be reset, but there is definitely some sort of issue and inconsistency there

---

Platform status is highly inconsistent:

const r = (d, ...a) => {
  try {
    return d.decode(...a).length
  } catch {}
  return 'e'
}

const a = new TextDecoder('utf8', { fatal: true })
console.log('A',
  r(a, Uint8Array.of(0xef, 0xbb, 0xbf, 0xff), { stream: true }),
  r(a, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }), // error does not stick in Chrome/Safari
)

const b = new TextDecoder('utf8', { fatal: true })
console.log('B',
  r(b, Uint8Array.of(0xef, 0xbb, 0xbf, 0xef), { stream: true }),
  r(b, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }),
  r(b, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }), // error sticks in Chrome / Safari
  r(b, Uint8Array.of(0xbb, 0xbf), { stream: true }),
  r(b, Uint8Array.of(), { stream: true }),
)

const c = new TextDecoder('utf8', { fatal: true })
console.log('C',
  r(c, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }),
  r(c, Uint8Array.of(0xff), { stream: true }),
  r(c, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }),
)

// Bonus: if BOM is not reset, is it processed on errors?
const d = new TextDecoder('utf8', { fatal: true })
const e = new TextDecoder('utf8', { fatal: true })
console.log('D',
  r(d, Uint8Array.of(0x20, 0xff), { stream: true }),
  r(d, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }),
  r(d, Uint8Array.of(0xff), { stream: true }),
  r(d, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }),
  r(e, Uint8Array.of(0xff), { stream: true }),
  r(e, Uint8Array.of(0xef, 0xbb, 0xbf), { stream: true }),
)

Chrome: first error did not get stuck, second error got stuck

A e 0
B 0 e e e e
C 0 e 1
D e 0 e 1 e 0

WebKit: first error did not get stuck, second error got stuck

A e 1
B 0 e e e e
C 0 e 1
D e 1 e 1 e 0

Firefox, Servo, Deno, Static Hermes: errors do not stick, bom does not get reset, bom seen is set on errors

A e 1
B 0 e 1 e 0
C 0 e 1
D e 1 e 1 e 1

Node.js: errors do not stick

A e 0
B 0 e 1 e 0
C 0 e 1
D e 0 e 1 e 0

Bun: just broken

A e 0
B 0 0 0 1 0
C 0 e 0
D e 0 e 0 e 0

[ChALkeR]

Also it looks like there is no browser consistent with the spec

The per-spec output in A is A e 0 and in D is D e 0 e 1 e 0

"BOM seen" is set on the current decode call only if it did not error, in "serialize I/O queue" after decoding, it should not be set if an error is occured anywhere

Only engines that don't have that wrong are Chrome and Node.js, but Chrome has also errors sticking

[ChALkeR]

See tests added in ExodusOSS/bytes@7b2ca41 and ExodusOSS/bytes@8f308ba and comments about platforms there

Both commits are for cases of continuing a fatal stream after error

---

In general, I think that continuing a fatal stream after error is deeply cursed, both here and in #358.
Continuing does not work in Chrome on UTF-8 - errors get stuck in some cases
No setups should be using that and expect to work

I wonder if the spec can just prohibit that instead of trying to figure out what to do with ISO-2022-JP and Unicode BOM states