Support source-module wrapped analysis replay

[FBartos]

Summary

• let runWrappedAnalysis() resolve analyses from explicit source module paths and QML files, not only installed module packages
• persist and retrieve standalone analysis state through the native callback-file contract instead of reaching into jaspTools internals
• expose .readFullDatasetToEnd to standalone bridge callbacks and make result write/seal/send operations harmless outside Desktop
• fix factor input handling in the common variance checks used during local module replay
• decode R-facing result copies returned by $toRObject(), including table names/footnotes and stored ggplot labels, without changing the encoded backend names used while analyses run
• expose decodeJaspResultState() as the public jaspBase API for decoding stored figure objects in result state payloads

Why

This is the runtime support layer for the jaspTools to jaspSyntax bridge. jaspTools now delegates QML/runtime option preparation to jaspSyntax and then calls jaspBase::runWrappedAnalysis() with explicit source-module provenance. That requires jaspBase to accept source paths cleanly and to use the same state callback-file contract that the native bridge exposes.

The direct jaspSyntax replay path also exposed a display-boundary gap: modules correctly receive encoded dataset names while fitting models, but R-facing result objects and stored figure state could still expose internal names such as JaspColumn_2_Encoded. The fix keeps encoded names in the backend/runtime path and decodes only R-facing copies returned to callers.

jaspTools previously had to walk state$figures itself and call jaspBase:::decodeplot(). This PR makes that an explicit jaspBase::decodeJaspResultState() responsibility, so jaspTools can stay a developer orchestration wrapper instead of owning result-state plot semantics or reaching into jaspBase internals.

Related PRs

• jaspSyntax: Add native bridge APIs for JASP replay jaspSyntax#7 consumes this runtime contract from the bridge helpers.
• jaspSyntax: Decode embedded column names in analysis results jaspSyntax#8 decodes embedded encoded names in JSON/result text returned through the jaspTools path.
• jasp-desktop: Expose SyntaxInterface bridge APIs for jaspSyntax jasp-desktop#6221 updates Engine/jaspBase to this runtime support line and exposes the matching SyntaxInterface APIs.
• jasp-desktop: Update jaspBase for decoded result objects jasp-desktop#6235 updates Engine/jaspBase to the result-object/state decoding commits.
• jaspTools: Migrate local analysis replay to jaspSyntax jaspTools#79 uses runWrappedAnalysis() with explicit source module/QML context and delegates result-state decoding to jaspBase.
• jaspDescriptives: Fix Pareto count variable visibility binding jaspDescriptives#500 removes the QML blocker that lets real Descriptives replay stay in the bridge test set.

Verification

• Rscript -e "pkgload::load_all('C:/JASP-Packages/jasp-desktop/Engine/jaspBase', quiet = TRUE); testthat::test_dir('tests/testthat', reporter = 'summary')"
• Direct R 4.6 replay of the lme4::cake/ABC MixedModelsLMM example through jaspSyntax::loadDataSet() and res$toRObject(); ANOVA footnote decoded to recipe, plot labels decoded to ABC and temperature, and the returned object had zero scalar jaspColumn/JaspColumn_ hits.
• git diff --check

[vandenman]

Was this already the jaspBase version, or is it the version from the jasp module?

[FBartos]

Good catch. The existing version field is the generated module wrapper version, not the jaspBase version. I kept that field unchanged for compatibility and added jaspBaseVersion as the separate runtime-contract version. I also added an inline comment in f51c8d7 so this distinction is explicit at the call site.

[vandenman]

I'm wondering if we shouldn't use rlang for many of these predicates and fs for file system operations. We import those packages no matter what and they might clean up a lot of these verbose helpers and file lookup functions.

[FBartos]

Agreed. I incorporated this in f51c8d7: fs and rlang are now declared imports, the string/list predicates use rlang, and the QML/state path handling uses fs for path construction, existence checks, normalization, and directory creation. The calls are namespace-qualified so the dependency use is explicit.

[vandenman]

This feels very verbose. Perhaps add a comment with why there are so many checks? Are other possible values allowed? If yes, shouldn't this be streamlined elsewhere?

[FBartos]

Agreed. I tightened this in f51c8d7. The helper now validates the callback shape and errors clearly for unsupported values. The intended Desktop/native contract is list(root, relativePath), while a standalone callback may provide only relativePath, which is interpreted relative to the current working directory. I also added a focused test for unsupported callback shapes so this contract is covered.

[vandenman]

Two issues.

1. If a user loads a dataset, then does the analysis, it works. If they load another dataset afterward, and show then print/ save the plot from the previous analysis, this no longer works, no? We need some hook to know which encoding/ decoding object should be used for which plot object.
2. If we do save or saveRDS on the output from jasp, then restart the r session and then try to replay the plot, all the information about encoding/ decoding is no longer available because this lives on the C++ side of jaspSyntax. We need a way around this. This could be done by immediately storing the decodedplot and not doing it on demand.

[FBartos]

Implemented the eager-decoding follow-up from the review/spec discussion.

What changed:

• Added an internal per-analysis decode context in jaspBase that captures the column mapping and factor-level mapping while the native/jaspSyntax state is still live.
• Materialize decoded state before saving: finishJaspResults() now decodes figures and other state, and .saveState() defensively enforces the same contract.
• Kept plot objects editable: ggplot/jaspGraphs/qgraph objects are stored as decoded editable objects, not rendered gTrees; base/function plots still materialize through recorded/grob behavior as before.
• Updated the C++ plot handoff so the decoded object returned by writeImageJaspResults() is written back into plot state for all plot types, not only function plots.
• Routed edit options and plotly conversion through the decoded object, so PNG/edit metadata/interactive JSON have the same labels.
• decodeJaspResultState() is no longer exported; the state repair/materialization helper is internal now.
• Extended R-facing object decoding to names, attributes, data frames, character vectors, factors, and numeric factor tokens using the captured context.

Companion jaspTools follow-up:

• Pushed Fix plot fallback snapshots jaspTools#82 so the fast test plot writer also asks jaspBase for the decoded plot object instead of storing the raw encoded object.
• Fixed the viewer helper to create the html output directory before copying assets; this came up when replaying the MixedModels example with view = TRUE.

Verification:

• R CMD INSTALL . for jaspBase passes.
• R CMD check --no-manual --no-vignettes . completes and runs testthat. It still reports the existing source-tree/package-metadata warnings/notes, including the already-known undeclared optional jaspSyntax usage, but no test failures.
• Focused jaspBase tests pass: test-result-object-decoding.R, test-runWrappedAnalysis.R.
• Focused jaspTools tests pass: test-runAnalysis-fast-test-plots.R, test-view.R.
• Replayed MixedModelsGLMM from Larks and Owls.jasp; it returns results successfully and the returned state has no JaspColumn_.../jaspColumn... tokens.

[FBartos]

Follow-up from the final desiderata audit: I found one remaining provenance gap in the live R6 wrapper path. oRObject() called the C++ oRObject() materializer first, and that C++ code can consult the currently active global decoder before the R-side eager decoder sees the object. That meant a live wrapper could still be decoded against the wrong dataset after an analysis/dataset switch, even though saved state was already eager-decoded.

Fixed in 17d2533:

unJaspResults() stores the per-analysis decode context on the result wrapper and propagates it to child wrappers.

• R-facing oRObject() materialization now temporarily installs a context-backed internal decoder while C++ materializes the object, then applies the recursive R decoder with the same context. This keeps C++ behavior intact but prevents late active-dataset provenance leaks.
• saveImage(), editImage(), and
  ewriteImages() now treat loaded plot state as stored result state rather than consulting a live analysis decoder. If encoded legacy state is encountered without context, it warns and leaves it unchanged instead of silently decoding against the wrong dataset.
• Added a regression test where the active decoder intentionally maps the same token to a wrong dataset name; the R6 wrapper still returns the original analysis names and factor labels.

Verification:

• est-result-object-decoding.R passes against the installed temp-library jaspBase.
• est-runWrappedAnalysis.R passes.
• jaspTools focused fast-plot/view tests pass with the updated jaspBase source.
• MixedModels MixedModelsGLMM replay from Larks and Owls.jasp passes through jaspTools::runAnalysis() with no encoded state tokens.
• R CMD check --no-manual --no-vignettes exits 0; it still reports the existing source-tree warnings/notes unrelated to this change.

[FBartos]

@vandenman this is ready for your review. The final desiderata pass is pushed, including the decode-context provenance follow-up for live result materialization and stored plot replay/export.

[FBartos]

Follow-up after the direct jaspMixedModels::MixedModelsLMM() cake repro: this did point to a structural boundary issue in the decoder, not just an lme4 edge case.

I pushed 0a64ba4 with three related changes:

• Decode only JASP-owned / display-facing result surfaces. .decodeJaspRObject() now preserves opaque S3/S4/call/name objects instead of recursively rewriting arbitrary package-owned internals. This avoids mutating lme4 internals such as lmerResp.
• Treat JASP mixed table cells by structure, not by the bare class name. The repro also exposed that afex::mixed() returns model objects with class mixed, so using inherits(x, mixed) as provenance was wrong. The decoder now recognizes only JASP mixed vectors/cells and leaves foreign mixed model objects intact.
• Keep state$other opaque. That field contains jaspState payloads restored into later runs, so it is analysis-owned state. Plot replay/export/edit state under state$figures is still eagerly decoded.

I also moved the per-analysis decode-context capture until after dataset preload, because that is when jaspSyntax exposes the requested encoded column names. Without that, toRObject() had a context object but not the JaspColumn_* -> original name mapping.

Verification after the change:

• pkgload::load_all('.', quiet = TRUE); testthat::test_file('tests/testthat/test-result-object-decoding.R') passes.
• Installed jaspBase into C:/JASP-Packages/_verify-lib-4.6 and reran the cake MixedModelsLMM repro. It returns ANOVA Summary, Plot, and scanning the R-facing toRObject() output finds no visible JaspColumn_* tokens.
• R.exe CMD check --no-manual --no-vignettes . completes with the existing source-tree warnings/notes; tests pass.