BIP Draft: Optimal Batch Proofs for Utreexo

[EntropyMaBoyy]

This draft proposes a deterministic, canonical, and bandwidth-optimal batch proof format for Utreexo accumulator forests.

The proposal defines a canonical ordering of proof elements and a bitmap-guided verification stream that eliminates redundant hashes while preserving full verifiability and non-malleability.

The specification includes normative pseudocode, explicit failure modes, and reference test vectors. A Python reference implementation was used to validate all vectors.

This proposal does not modify consensus rules and is intended for use by wallets, light clients, and Utreexo-based systems.

Feedback is welcome, particularly regarding serialization choices and deployment considerations.

[murchandamus]

Hello Lucas,

I don’t think I have seen this idea discussed on the Bitcoin mailing list. Could you please provide the link to the thread where your idea was previously discussed?
The document has various formatting issues, and at first glance, the text doesn’t seem to align well with the Utreexo proposals that are being discussed in #1923. Could you please tell us a little more about your process and how your proposal fits into the bigger picture regarding Utreexo?

[EntropyMaBoyy]

Hi Murch, Thanks for the quick feedback. You're absolutely right to flag the disconnect with PR #1923. I've reviewed Calvin Kim's proposal and can now articulate the technical friction clearly: Core Difference: Implicit vs. Explicit Path Encoding - #1923 (Utreexo Standard): Uses implicit path derivation. The verifier calculates traversal order from leaf indices and `num_leaves`, saving bandwidth by deducing structure mathematically. This is optimal for full Utreexo nodes that maintain the accumulator state (`Stump`). - My Proposal (BIP-XXXX): Uses an explicit bitmap. The verifier follows a simple FIFO stack algorithm with no index calculations. This costs 1 bit per internal node but eliminates the need for `num_leaves` or tree topology awareness. The "Stateless Verifier" Use Case My target is ultra-light clients that want to verify a single proof without syncing the accumulator. For example: - Hardware wallets receiving a one-off payment proof. - Cross-protocol bridges where the verifier doesn't track Bitcoin's UTXO set state. - Environments where even small indexing logic is a security/complexity risk (e.g., firmware). In these cases, transmitting a small bitmap is a deliberate trade-off: bandwidth for verifier simplicity and zero state. Process Question I now realize I may have jumped the gun. I haven't posted to the Bitcoin mailing list yet—should I initiate discussion there first, even if this is positioned as a complementary (not competing) format? Or would it be more productive to propose this as an optional "stateless mode" extension within #1923's framework? Cleaned Pseudocode for Reference Below is the final, tested implementation that aligns with the BIP's security model (SHA256, exact consumption, no malleability): ```python import hashlib from collections import deque from typing import List, Deque # ----------------------------------------------------------------------------- # Constants & Hash Function # ----------------------------------------------------------------------------- BIT_LEFT = 0 # Bitmap 0: H(a || b) BIT_RIGHT = 1 # Bitmap 1: H(b || a) def sha256(data: bytes) -> bytes: """Cryptographic hash function as specified by BIP.""" return hashlib.sha256(data).digest() # ----------------------------------------------------------------------------- # Single Tree Proof Verification # ----------------------------------------------------------------------------- def verify_single_tree_proof( leaves: List[bytes], hashes: List[bytes], bitmap: List[int], expected_root: bytes ) -> bool: """ Verifies a deterministic batch proof for one Merkle tree. Args: leaves: Leaf hashes to prove (target nodes). hashes: Canonical sibling hashes (deduplicated, height-sorted). bitmap: Explicit L/R composition instructions (0=Left, 1=Right). expected_root: Known tree root for validation. Returns: True iff proof is valid and consumes all elements exactly. """ if not leaves: return False # Initialize FIFO queue (not a stack) for sequential composition queue: Deque[bytes] = deque(leaves) hash_idx = 0 bit_idx = 0 # Reconstruct root until one element remains while len(queue) > 1: # Exhaustion check: must have exactly enough elements if hash_idx >= len(hashes) or bit_idx >= len(bitmap): return False a = queue.popleft() # Current accumulated hash b = hashes[hash_idx] # Next sibling hash # Compose based on explicit bitmap direction if bitmap[bit_idx] == BIT_LEFT: c = sha256(a + b) else: # BIT_RIGHT c = sha256(b + a) queue.append(c) hash_idx += 1 bit_idx += 1 # Exact consumption is mandatory for non-malleability if hash_idx != len(hashes) or bit_idx != len(bitmap): return False # Final root equality check return queue[0] == expected_root # ----------------------------------------------------------------------------- # Forest Batch Proof Verification # ----------------------------------------------------------------------------- class TreeProof: """Container for per-tree proof elements.""" def __init__(self, leaves, hashes, bitmap, expected_root): self.leaves = leaves self.hashes = hashes self.bitmap = bitmap self.expected_root = expected_root def verify_forest_batch_proof(forest_proofs: List[TreeProof]) -> bool: """ Verifies independence of proofs across a Utreexo forest. Each tree is validated in isolation; no cross-tree replay allowed. """ for tree_proof in forest_proofs: if not verify_single_tree_proof( tree_proof.leaves, tree_proof.hashes, tree_proof.bitmap, tree_proof.expected_root ): return False return True ``` El vie, 9 de ene de 2026, 5:54 p. m., murchandamus ***@***.***> escribió:

…

***@***.**** commented on this pull request. Hello Lucas, I don’t think I have seen this idea discussed on the Bitcoin mailing list. Could you please provide the link to the thread where your idea was previously discussed? The document has various formatting issues, and at first glance, the text doesn’t seem to align well with the Utreexo proposals that are being discussed in #1923 <#1923>. Could you please tell us a little more about your process and how your proposal fits into the bigger picture regarding Utreexo? — Reply to this email directly, view it on GitHub <#2079 (review)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/BUSRMZC2XA3O3C3XR7ZE2HT4GAIPLAVCNFSM6AAAAACRHFG7NOVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTMNBVGU2DEOBVGY> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[murchandamus]

Yes, BIP ideas should be discussed on the mailing list first.

Also, whatever process may be used to compile a document, it is upon the author to ensure that their proposal is clear and comprehensive, an obvious net improvement, and of high quality. This document has severe quality issues, and leaves me with the impression that it was LLM generated. Please don’t waste our time.