Repository navigation
Reduce peek memory use. #125
Description
Recent analysis shows that during Simplicity execution, peek memory use could be as high as 720 bytes per DAG node. We should rearrange computations and allocations to try to reduce that as much as possible. Ideally I'd like to see less than 128 bytes per DAG node if possible.
- Postpone the CMR computation until after type inference, and move the CMR out of the
dagstructure. Only the Root CMR and one CMR perdisconnectnode is needed, so perhaps there is a way to keep only these placing the CMR in a union in thedagstructure only accessible todisconnect. - Move the TMR out of the
typestructure. Compute it as part of the IMR computation and free it afterwards. (The IMR calculation seems to require up to 4*32 bytes per DAG node to just by itself, so keeping peek memory use under 128 bytes per DAG node seems impossible). - Replace all
size_tvalues withuint32_tvalues. Ensure that the maximum index of arrays is less than 2^32. - Replace all
unification_var*with auint32_tvalue and rearrange theunification_varallocations so that they all are in one single array. - make an
isChildfield inunification_varinstead of using aNULLpointer. Make the parent pointer part of a union.
Activity
apoelstra commentedon Sep 23, 2026
Thinking about this a bit more, since we've gotten multiple security reports over the last few months that ultimately boil down to this issue.
In addition to reducing the peak memory usage for the type checker, we could also move several checks in front of typechecking. In particular, we should be able to:
DAG_LEN_MAX. (AlsoDAG_LEN_MAXshould be configurable so we can reduce it in policy, if need be). Only if this passes will we allocate any memory at all. This eliminates some particularly egregious memory attacks where you can goad a node into allocating a bunch of memory just in response to a big length prefix.disconnectnodes; we won't need the root CMR or any intermediate CMRs again. Doing this early means that an attacker will need an actual UTXO that commits to a well-formed program before they can trigger type-inference memory allocations.disconnects" knob in there as well which can be tuned by policy which will limit the amount of space used for these CMRs.Certainly step (1) is worth doing, because it's super easy and prevents somebody from allocating a bunch of memory without even creating a super big transaction.
The rest though are essentially just minor roadblocks. Ultimately it's always possible for somebody to trigger the maximum amount of memory allowed for type inference (and have their transaction fail afterward), so in the end we just need to reduce that maximum.