RYW transactions don't handle append_if_fits correctly

[atn34]

$ cat test.py && echo "===" && python3 test.py 
import fdb

fdb.api_version(730)
db = fdb.open()


class Cancel(Exception):
    pass


@fdb.transactional
def do_setup(tr):
    tr[b""] = b"a" * 70000


do_setup(db)


@fdb.transactional
def do_test(tr, write_first):
    if write_first:
        do_setup(tr)
    tr.append_if_fits(b"", b"b" * 20000)
    tr.append_if_fits(b"", b"b" * 20000)
    print(len(tr[b""]))
    raise Cancel()


try:
    do_test(db, False)
except Cancel:
    pass

try:
    do_test(db, True)
except Cancel:
    pass
===
70000
90000

[Ronitsabhaya75]

@atn34 The issue is in WriteMap::coalesce() it eagerly merges two AppendIfFits operands via doAppendIfFits, but append_if_fits isn't associative when truncation kicks in since the size check depends on the intermediate accumulated value. So the coalesced 40KB operand hits the size limit against the 70KB base all at once, instead of each 20KB append being checked sequentially. The fix would be to skip coalescing AppendIfFits + AppendIfFits in coalesceOver() and just push onto the stack, so they get applied one at a time in coalesceUnder() when the base value is known.

[Divyansh151005]

I'd like to take this.

Agreed with the analysis above: AppendIfFits is not associative under truncation, so WriteMap::coalesceOver must not merge two AppendIfFits operands before the base value is known. I'll push them onto the stack instead and apply them sequentially in coalesceUnder, plus add a WriteMap unit test covering the 70KB + 20KB + 20KB case from the report.