Completes support for PathMap pattern matching and integrates support for MeTTaTron#426
Completes support for PathMap pattern matching and integrates support for MeTTaTron#426
Conversation
…lementation method for run stub on PathMap data types
… PathMap data type
…y with parallelized version of run_state (eval) in MeTTaTron; refactors metta_compile to be compatible with !? operator and drops metta_compile_sync
…of the MeTTa environment since they are not serialized as byte arrays to avoid corrupting them upon deserialization
Problem: - MeTTa-Compiler PR #4 refactored backend::types to backend::models - f1r3node was still using mettatron::backend::types::MettaState Solution: - Add MettaState to the mettatron import list - Use imported MettaState directly instead of fully qualified path - Follows the new module structure from MeTTa-Compiler Changes: - rholang/src/rust/interpreter/reduce.rs: - Added MettaState to mettatron imports - Changed mettatron::backend::types::MettaState to MettaState This aligns with the models/ refactoring merged in MeTTa-Compiler. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The metta_error_to_par function expects a &str, but was being passed a &SyntaxError. Convert the error to string with .to_string() before passing it. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
- Add METTA_LARGE_EXPRS_MAGIC constant and is_metta_large_exprs_bytes() detection function for large expression byte arrays - Update is_metta_environment() to accept 2 or 3 element tuples to support environments with large expressions - Update format_metta_environment() to display large expressions count when present in the environment - Update PathMap dependency to use local path with jemalloc and arena_compact features 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
The Match target field is Option<T>, not T directly. Updated the pretty printer to use as_ref().expect() to properly dereference the optional target field. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Switch rholang-parser dependency from dylon/named-comment-nodes branch to f1r3node_dependecies branch in node, rholang, and rspace++ crates.
…integration for updated function signatures and semantics
Make the MeTTaTron compiler an opt-in dependency (disabled by default) so the project builds without MeTTaTron and its transitive dependencies (Cranelift JIT, tree-sitter-metta, etc.) unless explicitly requested with --features mettatron.
…alization The MeTTaTron environment serialization previously used a separate MTTM byte array for multiplicities, which had an ordering bug: set_multiplicities() correctly restored counts, but the subsequent create_space() + btm.insert() overwrote every count to 1, and update_pathmap() then clobbered the correct values. This caused all multiplicities to be lost during round-trip. Fix by encoding multiplicities inline with each path in MTTS/MTTL byte arrays (8-byte big-endian u64 after each path), reducing the environment ETuple from 3 fields to 2 (space, large_exprs). Remove the now-dead MTTM serialization/deserialization code, decode_multiplicities_bytes_to_pairs, set_multiplicities, and the decodeMultiplicities Rholang method. Update the pretty printer to display the new 2-tuple format with inline multiplicity counts. Add PathMap EPathMap support including sexpr_to_par conversion, spatial matcher integration, element extraction, pattern matching, and proptest-based property tests.
Introduces two new standalone test files for PathMap: - pathmap_method_dispatch_spec.rs: Tests method dispatch pipeline (size, length, toList, toSet, paths, first, last) through the interpreter - pathmap_e2e_type_decomposition_spec.rs: 45 end-to-end tests covering compile → embed → decompose pipeline across all Rholang type variants (scalars, collections, processes, nested structures, remainder decomposition, order independence, mismatches, and round-trips)
Incorporates 148 commits from rust/dev including abort system process, deploy data channel, Ollama AI services, metrics/monitoring, Docker CI, REV→Vault renaming, overflow fixes, and transaction API tests. Preserves all dylon/pln-support features: MeTTaTron integration (feature-gated behind "mettatron"), PathMap tests, and bug fixes. Conflict resolutions: - .cargo/config.toml: adopted rust/dev's minimal [build] section - Cargo.lock: adopted rust/dev's version (validated by cargo check) - casper in_memory_key_value_store_spec: took rust/dev's overflow-safe avg formula - rholang/src/lib.rs: kept both mettatron and external services imports - system_processes.rs: adopted rust/dev's BodyRefs numbering + METTA_COMPILE:200, VaultAddress import + mettatron imports, both metta_compile and abort handlers
Move -D warnings from RUSTFLAGS to [workspace.lints.rust] so warnings are denied only for workspace members, not external path dependencies like MORK (which has 76 warnings that broke the rholang --features mettatron build). Fix optional-tests.yml artifact names to match build-test-and-deploy.yml (rchain-worktree → f1r3fly-worktree, artifacts-docker → artifacts-docker-amd64, rnode-docker → rust-node-docker) and add run-id/github-token for cross-workflow artifact downloads.
Integrates 759 upstream commits (performance optimizations, robustness improvements, correctness fixes) while preserving all PLN/MeTTaTron support features. Resolved 3 conflicts: - models/src/rust/mod.rs: kept all module declarations from both branches - models/src/rust/pathmap_crate_type_mapper.rs: kept Par import for explicit type annotation - rholang/src/rust/interpreter/reduce.rs: kept full MeTTaTron RunMethod implementation Also fixed pathmap_method_dispatch_spec.rs test to account for upstream's DebruijnInterpreter::new() now returning Arc<Self>.
…display Spatial matcher: when a GByteArray with MTTS/MTTL magic bytes is matched against a PathMap pattern, decode the bytes on demand rather than requiring explicit deserialization. This enables 3-layer PathMap decomposition of MeTTa state (state → environment → atom space) in Rholang match expressions. Pretty printer: decode MeTTa environment atoms to readable s-expressions instead of displaying raw hex-encoded MORK paths. Reduce: return structured error expressions from .run() instead of propagating InterpreterErrors, enabling graceful error handling in Rholang.
…failure xx-cargo's clang wrappers (e.g. x86_64-unknown-linux-gnu-clang) have an incomplete sysroot that breaks tikv-jemalloc-sys's autoconf strerror_r detection on both amd64 and arm64 CI runners. Since both runners are native (not cross-compiling), xx-cargo is unnecessary overhead. The build step now detects native vs cross-compilation by comparing uname -m against TARGETARCH: native builds use cargo directly (system cc/pkg-config work fine, jemalloc compiles normally), while cross builds fall back to xx-cargo with its sysroot/pkg-config setup.
… ETuple MeTTaTron state serialization changed from ETuple to EList to enable ...rest decomposition in Rholang patterns. This updates is_metta_environment() to match the new EListBody structure at all nesting levels.
# Conflicts: # .gitmodules
MeTTa s-expressions now use EList, so environment display format changes
from (("space", ...)) to [["space", ...]] for consistency.
AndriiS-DevBrother
left a comment
There was a problem hiding this comment.
Suggested Improvements
1. Restore -D warnings in CI
It looks like -D warnings was dropped from RUSTFLAGS — would it be possible to restore it? If MeTTaTron emits warnings that block the build, perhaps targeted #[allow(...)] or -A specific_lint could work instead of removing it entirely?
2. External dependencies on personal forks
CI and [patch] sections currently point to github.com/dylon/PathMap and github.com/dylon/MORK. Are there plans to migrate these under the F1R3FLY-io org or publish to crates.io? That would help avoid any risk if the personal repos change.
3. Deduplicate spatial matcher logic
The three EPathmapBody / GByteArray(MTTS) / GByteArray(MTTL) match arms in spatial_matcher.rs share ~40 lines of identical sort/dedup/remainder/merger code. Would it make sense to extract a shared helper to reduce duplication?
4. block_in_place runtime assumption
The run method uses block_in_place + Handle::current().block_on(), which requires a multi-threaded Tokio runtime. The CLI switch to new_multi_thread() covers that path — could we also verify FFI callers are safe, or add a guard with a friendlier error message?
5. Error-as-value pattern in run
Deserialization/evaluation failures return Ok(metta_run_error_expr(...)) instead of Err(...). Would it be helpful to add a doc comment explaining this contract so callers know they need to pattern-match for errors?
6. Pin Docker dependency clones
The Dockerfile uses git clone --branch <branch> — would pinning to specific commit SHAs be feasible for more reproducible builds?
7. Document ../PathMap build prerequisite
The [patch.crates-io] with path = "../PathMap" means the workspace won't build without a sibling clone. It might help new contributors to mention this in the README or a CONTRIBUTING guide.
8. Incorrect Rholang syntax in metta_compile doc comment
The doc examples use @"rho:metta:compile" which is not valid Rholang syntax. System URN channels must be bound via new, e.g. new mettaCompile(\rho:metta:compile`) in { ... }, consistent with how stdout(`rho:io:stdout`)` works.
9. Missing Rholang example for MeTTa/PathMap
Other system processes like stdout, gpt4, ollama, etc. have corresponding example .rho files in rholang/examples/. Would it be worth adding a rholang/examples/metta_compile.rho (and maybe a pathmap.rho) to demonstrate the new functionality?
10. Undocumented binary format parsing in pretty_printer.rs
~230 lines of new code in pretty_printer.rs manually parse custom binary formats (MTTS, MTTL) using raw byte offsets and magic numbers. The functions is_metta_space_bytes and is_metta_large_exprs_bytes are marked #[allow(dead_code)] (appear unused in this file). The binary wire format (magic bytes, field sizes, offsets) has no specification reference. Could we get documentation on these formats and clarify whether the dead-code functions are needed?
11. Unnecessary build.sbt changes?
This PR targets rust/dev where Scala no longer invokes Rust at runtime. The build.sbt changes add run / fork, JNA library paths, and LD_PRELOAD for .so files — are these still needed? If the Scala node doesn't load Rust libraries, these changes may be dead configuration.
| shell: bash -ex {0} | ||
| run: | | ||
| git clone --depth 1 --branch feature/mettatron-fixes \ | ||
| https://github.com/dylon/PathMap.git \ |
There was a problem hiding this comment.
It looks like -D warnings was removed here — was this intentional to work around MeTTaTron warnings? If so, would it be possible to use targeted #[allow(...)] on specific items or -A specific_lint in RUSTFLAGS to keep the warnings-as-errors policy for the rest of the codebase?
| [profile.dev] | ||
| debug = true | ||
|
|
||
| # Unify all pathmap sources to the single local clone. |
There was a problem hiding this comment.
[patch.crates-io] pathmap = { path = "../PathMap" } means the workspace won't build without a sibling PathMap clone. New contributors might hit a confusing build failure — would it be worth adding a note about this in the README or a setup guide?
| let rem = &p_pathmap.remainder; | ||
|
|
||
| let is_wildcard = matches!( | ||
| rem, |
There was a problem hiding this comment.
The EPathmapBody ↔ EPathmapBody, GByteArray(MTTS) ↔ EPathmapBody, and GByteArray(MTTL) ↔ EPathmapBody arms share ~40 lines of identical logic (sort/dedup/remainder extraction/merger/list_match_single_). Would it make sense to extract this into a shared helper? Something like:
fn pathmap_match_inner(
&mut self,
mut t_elements: Vec<Par>,
p_pathmap: &EPathMap,
) -> Option<()> {
t_elements.sort(); t_elements.dedup();
// ... shared remainder + merger logic
}| use mettatron::{ | ||
| decode_large_exprs_bytes_to_pars, | ||
| decode_space_bytes_to_pars, metta_run_error_expr, | ||
| metta_state_to_pathmap_par, pathmap_par_to_metta_state, |
There was a problem hiding this comment.
block_in_place + Handle::current().block_on() will panic on single-threaded Tokio runtimes. The rholang_cli.rs switch to new_multi_thread() covers the CLI path — could we also verify that FFI callers (e.g. create_runtime_with_test_framework) use a multi-threaded runtime? Adding a guard with a friendly error message instead of a raw panic might be worth considering.
Also: deserialization/evaluation failures in the run method return Ok(metta_run_error_expr(...)) — errors as data values rather than Err(...). This is a valid design choice, but callers need to know they must pattern-match the result Par to detect failures. Would it be helpful to add a doc comment explaining this error-as-value contract and what the error Par structure looks like?
| return Err(illegal_argument_error("metta_compile")); | ||
| }; | ||
|
|
||
| let mut src = self.pretty_printer.build_string_from_message(source); |
There was a problem hiding this comment.
Extracting source code via self.pretty_printer.build_string_from_message(source) and then stripping quotes could be fragile if source isn't a simple GString. Would it be safer to extract the string directly from the Par?
let src = match source.exprs.first().and_then(|e| e.expr_instance.as_ref()) {
Some(ExprInstance::GString(s)) => s.clone(),
_ => return Err(illegal_argument_error("metta_compile: expected string argument")),
};| use rust::interpreter::system_processes::metta_contracts; | ||
| use rust::interpreter::external_services::ExternalServices; | ||
| use rust::interpreter::ollama_service::OllamaConfig; | ||
| use rust::interpreter::openai_service::OpenAIConfig; |
There was a problem hiding this comment.
Other system processes like stdout, gpt4, ollama, etc. have corresponding example .rho files in rholang/examples/. Would it be worth adding a rholang/examples/metta_compile.rho (and maybe a pathmap.rho) to demonstrate the new MeTTa and PathMap functionality? It would help users discover and test the new features.
| https://github.com/dylon/PathMap.git /PathMap && \ | ||
| git clone --depth 1 --branch feature/arbitrary-space-value-types \ | ||
| https://github.com/dylon/MORK.git /MORK && \ | ||
| git clone --depth 1 --branch feature/pln-support \ |
There was a problem hiding this comment.
git clone --branch <branch> could be affected by force-pushes — would it be feasible to pin to specific commit SHAs for more reproducible Docker builds?
RUN git clone https://github.com/dylon/PathMap.git /PathMap && \
cd /PathMap && git checkout <sha>| }) | ||
| } | ||
|
|
||
| /// Decode a tagged S-expression list: (tag child1 child2 ...) |
There was a problem hiding this comment.
The catch-all here treats any unrecognized symbol as GString, which could silently mask bugs if new Par types are added to the encoder but missed in the decoder. Would it make sense to log a warning for unexpected symbols, or maintain an explicit allowlist of known patterns?
| @@ -428,6 +428,38 @@ pub fn new_eset_expr( | |||
| } | |||
| } | |||
|
|
|||
There was a problem hiding this comment.
Minor nit: parameters named _ps, _locally_free, etc. conventionally signal unused parameters in Rust. Since these are all used, dropping the underscore prefixes might be clearer.
| let inner = &s[1..s.len() - 1]; | ||
| return Ok(Par { | ||
| exprs: vec![Expr { | ||
| expr_instance: Some(ExprInstance::GString(inner.to_string())), |
There was a problem hiding this comment.
Ground strings should support escaped characters since https://github.com/F1R3FLY-io/f1r3node/pull/228/changes
| } | ||
|
|
||
| // Bound variable: _N (where N is an integer) | ||
| if s.starts_with('_') && s.len() > 1 { |
There was a problem hiding this comment.
If you don't get back the original variable names, please add a comment to that effect to the description of the function.
The main changes include:
The original PathMap integration was added as part of the initial MeTTaTron demo and its development was deprioritized shortly before pattern matching support was completed. This completes it as part of the MeTTaTron integration.
The sequence of evaluating a MeTTaTron application in parts follows:
new mettaCompile(`rho:metta:compile`) { ... }newblock, compile a snippet of MeTTa text via:for (@compiledState <- mettaCompile!?(code)) { ... }. This will return an instance of a MeTTaTron state whose inputs have been populated with the parsed atoms from the code snippet (s-expressions).forblock, begin evaluation of the MeTTaTron app against an empty PathMap context, e.g.{||}.run(compiledState)(where{||}denotes the empty PathMap). This will evaluate the atoms incompiledStatewithin the empty context (no pre-existing atoms in its atom space). Most likely, the MeTTa snippet being evaluated will insert atoms into the atom space. The return value will be a new MeTTaTron state representing the updated context after evaluation, which is to say, it contains the side-effects to the atom space of the evaluated expressions and all the outputs from evaluation.persistentState!({||}.run(compiledState)).persistentStatevia:for (@state <- persistentState) { ... }and usestatein place of the empty PathMap{||}from step3. Repeat the steps as needed.For a complete example how this works, take a look at the
robot_planning.rhoexample from MeTTaTron.The communication mechanism will be greatly simplified and made much more efficient once we have a Reified RSpace for MeTTaTron, but that has not been designed yet.
The MeTTaTron integration is feature-gated behind
"mettatron"in case others wish to build Rholang without it.