feat: per-contract explorer parser modules#2044
Draft
Maxnflaxl wants to merge 2 commits into
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat: per-contract explorer parser modules
Summary
Replaces the single monolithic
bvm/Shaders/Explorer/Parser.cpp(~3.5k LOC, one giantswitchover every contract SID) with a folder of small per-contract parser wasm modules that the explorer node loads at startup and dispatches by SID.Each contract now ships its own
parser.cppnext tocontract.h/app.cpp, exporting a 4-method ABI (Method_0/1/2mirror the old monolith entrypoints;Method_3is new — it returns the SIDs the module claims). The explorer compiles every*.wasmin the configured folder, asks each module which SIDs it handles, builds a SID → bytecode map, and routes parse calls per-SID. Modules unknown to the explorer are simply skipped (parse-only feature, no consensus impact).Motivation
s_pSID[]from one module and recoveriVerinternally — no more central registry.Background: why the regular node ever had a parser at all
In
master,--contract_rich_parser <file>was registered on both the mainbeam-nodebinary (beam/cli.cpp) andexplorer-node(explorer/explorer_node.cpp), with identical wiring on each side: read the path,bvm2::Processor::Compile(..., Kind::Manager), stash the bytes innode.m_Cfg.m_ProcessorParams.m_RichParser, and setRichInfo::UpdShader. That last bit triggered persistence insideNodeProcessor::Initialize:So the compiled blob was written into the node DB itself, under
ParamID::RichContractParser, and any subsequent rich-info parsing (ProcessorInfoParser::Initinnode/processor.cpp) read it straight back out:That meant operators only had to pass
--contract_rich_parseronce (or whenever they wanted to swap the shader); bothbeam-nodeandexplorer-nodecould install or replace it, and either binary would dutifully run it on every contract invocation during block interpretation.…but the regular node never consumed the output
A close reading of
mastershows that the parser machinery in the node was always producer-only:BlockInterpretCtx::BvmProcessor::ParseExtraInforuns the parser on every contract invocation and writes the resulting string intoContractInvokeExtraInfo::m_sParser.NodeProcessor::get_ContractDescrruns it on demand for a given sid/cid.But every consumer of
m_sParserandget_ContractDescrlives in explorer / JSON-RPC plumbing (the contract-info RPC,extract_contract_invoke_extra_info, etc.). Nothing innode/,core/,pow/, the wallet, or the P2P/consensus paths reads either field. Abeam-nodestarted with--contract_rich_parserbut no explorer-style client on top would happily compile the shader, persist it inNodeDB, and run it on every block — and then throw the result away.The reason it lived on the node side was purely mechanical: rich info is generated during block interpretation, which is a node responsibility, and persisting the parser in
NodeDBsaved operators from re-passing the shader on every restart. Semantically it was always an explorer concern.That history is what makes this branch's split clean: lifting the parser map into the explorer process and dropping the DB row removes a feature path that no node-only deployment ever benefited from.
Changes
Host (explorer + node)
explorer/explorer_node.cpp--contract_rich_parser <file>with--contract_rich_parser_folder <dir>.--contract_rich_parser_folder_dryrun: scans the folder, prints the SID → module map, exits.load_parser_modules()reads + compiles every*.wasmat CLI-parse time (deterministic order);apply_parser_modules()queriesMethod_3afternode.Initialize()to discover SIDs and refuses to start on duplicate SID claims across modules.node/processor.{h,cpp}NodeProcessor::m_RichParserModules(map<ShaderID, ByteBuffer>), populated by the explorer beforeInitialize().ProcessorInfoParser::Init()now takes a SID and looks up the module in the map; returns false (skip) if no module is registered for that SID.m_Cfg.m_ProcessorParams.m_RichParser,StartParams::RichInfo::UpdShader, and theParamSet/ParamGetofNodeDB::ParamID::RichContractParserare all gone.ProcessorInfoParserno longer carriesm_bufParsereither — bytecode is borrowed from the in-memory map.ParserModule_GetSupportedSids()runs the two-callMethod_3protocol against a compiled module and returns its SID list.beam/cli.cpp,utility/cli/options.{h,cpp}— drop theCONTRACT_RICH_PARSERparsing block frombeam-nodeentirely. The string constantcli::CONTRACT_RICH_PARSERis retained because the explorer's option registration still uses it; no other binary references it.DB migration (implicit)
NodeDB::ParamID::RichContractParseris no longer read or written anywhere. The enum value is kept (no renumbering), andNodeDB::Openalready runsParamDelSafe(ParamID::RichContractParser)from a prior schema migration, so any leftover row from oldbeam-noderuns is silently dropped on next open.Shaders
bvm/Shaders/Explorer/parser_module_abi.h— defines the 4-method module ABI, the two-callMethod_3SID-discovery protocol, and convenience macros (PARSER_MODULE_EXPORT_SIDS,PARSER_MODULE_EXPORT_KIND_ONLY).bvm/Shaders/Explorer/parser_common.h— sharedDoc*helpers extracted from the monolith.parser.cppadded under each contract dir (amm, bans, blackhole, dao-accumulator, dao-core, dao-core2, dao-vault, dao-vote, faucet, faucet2, gallery, minter, nephrite, oracle2, pbft, sidechain_pos, vault, vault_anon).*.parser.wasmartifacts checked in underbvm/Shaders/Explorer/modules/so explorers can run without a local toolchain.bvm/Shaders/Explorer/Parser.cpp/Parser.wasmdeleted.Build tooling
make_all.sh/make_shader.sh— adds an--export [out_dir]mode that builds everything and collects*/parser.wasmintoExplorer/modules/(default), ready to be pointed at by--contract_rich_parser_folder.Operator notes / breaking changes
--contract_rich_parser <file>must switch to--contract_rich_parser_folder <dir>. There is no auto-fallback to the old single-file form.beam-nodeno longer accepts--contract_rich_parserat all. As described above, parsing was already an explorer-only concern in practice; the flag is removed from the main node binary and rich parsing now requires runningexplorer-node.NodeDB::ParamID::RichContractParserrow is ignored on read; nothing reads or writes it anymore. Operators do not need to wipe or migrate node DBs.--contract_rich_parser_folderwere unset.Method_3,apply_parser_modules()refuses to start the explorer with a clear error — easier to diagnose than the previous monolith's silent-precedence behavior.Test plan
bvm/Shaders/make_all.sh --exportproduces the expected set of modules inExplorer/modules/.--contract_rich_parser_folder bvm/Shaders/Explorer/modulesand serves rich info for at least one tx of each shipped contract (amm, nephrite, dao-vote, gallery, faucet/faucet2, vault/vault_anon, bans, blackhole, sidechain_pos, pbft, oracle2, minter, dao-* family).--contract_rich_parser_folder_dryrunprints a SID → module map and exits 0.beam-nodestill builds and runs; no surprises from removingm_RichParserfromStartParams.ParamID::RichContractParserwritten by an olderbeam-nodeopens cleanly under the new build (the row should be dropped silently on first open).node_testpasses (it's been heavily reformatted in this branch — verify it actually still tests what it did before).