fix(mappers): Support simpleeval 1.0.5+ by subclassing simpleeval.EvalWithCompoundTypes

[edgarrmondragon]

Related

• chore: Support simpleeval 1.0.5+ by implementing ModuleWrapper for modules exposed to stream maps expressions #3561
• fix(mappers): Support simpleeval 1.0.5+ by subclassing simpleeval.EvalWithCompoundTypes #3595
• Re-implements upstream perf: check disallowed items at name-resolution boundary, not every AST node danthedeckie/simpleeval#181

Summary by Sourcery

Update stream mapper expression evaluation to be compatible with simpleeval 1.0.5+ while preserving the mapper’s existing safety guarantees.

New Features:

• Introduce a dedicated mapper expression evaluator subclass to control how AST nodes, names, and disallowed items are handled during evaluation.

Bug Fixes:

• Fix incompatibility with simpleeval 1.0.5+ by bypassing its new redundant container safety scan in a controlled way.

Enhancements:

• Wrap datetime and json modules with simpleeval.ModuleWrapper when exposing them as mapper functions to align with simpleeval’s safety model.

Build:

• Pin simpleeval to version 1.0.7 in project dependencies.

[sourcery-ai]

Reviewer's Guide

Adds a custom _MapperEval subclass of simpleeval.EvalWithCompoundTypes to restore pre-1.0.5 evaluation behavior while remaining compatible with simpleeval>=1.0.5 and tightening safety by wrapping module functions, and wires this evaluator into mapper usage while pinning simpleeval to 1.0.7.

Sequence diagram for expression evaluation with _MapperEval

sequenceDiagram
    participant Caller
    participant CustomStreamMap
    participant _MapperEval
    participant simpleeval_ModuleWrapper

    Caller->>CustomStreamMap: transform(record)
    CustomStreamMap->>CustomStreamMap: functions() builds dict
    CustomStreamMap->>simpleeval_ModuleWrapper: wrap(datetime)
    CustomStreamMap->>simpleeval_ModuleWrapper: wrap(json)
    CustomStreamMap->>_MapperEval: init(functions=functions)
    CustomStreamMap->>_MapperEval: _eval(expr, record, context)
    activate _MapperEval
    _MapperEval->>_MapperEval: _eval(ast_node)
    _MapperEval->>_MapperEval: _eval_name(ast_name)
    _MapperEval->>_MapperEval: _check_disallowed_items(item)
    _MapperEval-->>CustomStreamMap: result
    deactivate _MapperEval
    CustomStreamMap-->>Caller: transformed_record

Loading

Class diagram for _MapperEval and CustomStreamMap evaluator integration

classDiagram
    class simpleeval_EvalWithCompoundTypes {
        <<external>>
        +eval(expr)
        +_eval(node)
        +_eval_name(node)
        +_check_disallowed_items(item)
    }

    class _MapperEval {
        +_check_disallowed_items(item)
        +_eval(node)
        +_eval_name(node)
    }

    class CustomStreamMap {
        -functions: dict~str, Callable~
        -expr_evaluator: simpleeval_EvalWithCompoundTypes
        +functions() dict~str, Callable~
        +transform(record) dict | None
        +_eval(expr, record, context) Any
    }

    simpleeval_EvalWithCompoundTypes <|-- _MapperEval
    CustomStreamMap ..> _MapperEval : uses as expr_evaluator

    class simpleeval_ModuleWrapper {
        <<external>>
    }

    class datetime_module {
        <<module>>
    }

    class json_module {
        <<module>>
    }

    CustomStreamMap ..> simpleeval_ModuleWrapper : wraps datetime, json
    CustomStreamMap ..> datetime_module
    CustomStreamMap ..> json_module

Loading

File-Level Changes

Change	Details	Files
Introduce a custom evaluator subclass to control simpleeval behavior for mapper expressions and name resolution.	Add version-gated import for typing.override (typing vs typing_extensions). Define _MapperEval subclass of simpleeval.EvalWithCompoundTypes that no-ops _check_disallowed_items to bypass the new container scan introduced in simpleeval 1.0.5 while documenting safety assumptions. Override _eval to provide a local node-dispatch implementation that raises FeatureNotAvailable for unsupported AST node types. Override _eval_name to mirror upstream logic while explicitly running _check_disallowed_items on values from names/functions, preserving simpleeval’s disallowed item checks where intentional.	singer_sdk/mapper.py
Wire the new evaluator into existing mapper paths and harden function definitions to be safe under simpleeval 1.0.5+ security changes.	Replace direct uses of simpleeval.EvalWithCompoundTypes in CustomStreamMap.init and _eval_stream with the new _MapperEval class while keeping type hints compatible. Wrap datetime and json in simpleeval.ModuleWrapper before exposing them in the evaluator functions dict to satisfy simpleeval’s module safety expectations.	singer_sdk/mapper.py
Update dependency constraints to a known-good simpleeval release that includes the upstream fix this PR reimplements locally.	Change simpleeval dependency specifier from a version range excluding 1.0.5 to an exact pin at simpleeval==1.0.7. Refresh uv.lock to reflect the new simpleeval version and its dependency resolution.	pyproject.toml uv.lock

Possibly linked issues

• #unknown: They address the same breakage: supporting simpleeval 1.0.5+ for stream maps by wrapping modules and adjusting evaluation.
• chore: Support simpleeval 1.0.5+ by implementing ModuleWrapper for modules exposed to stream maps expressions #3561: PR updates mapper evaluation and wraps modules to restore stream_map support with simpleeval 1.0.5+ as described.

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've reviewed your changes and they look great!

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 94.13%. Comparing base (bae8c09) to head (29780d5).
⚠️ Report is 1 commits behind head on v0.53.

Additional details and impacted files

@@           Coverage Diff           @@
##            v0.53    #3601   +/-   ##
=======================================
  Coverage   94.13%   94.13%           
=======================================
  Files          69       69           
  Lines        5783     5783           
  Branches      716      716           
=======================================
  Hits         5444     5444           
  Misses        236      236           
  Partials      103      103           

Flag	Coverage Δ
core	81.98% <ø> (ø)
end-to-end	76.37% <ø> (ø)
optional-components	43.48% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[codspeed-hq]

Merging this PR will not alter performance

✅ 8 untouched benchmarks

---

_{Comparing simpleeval-vuln (29780d5) with v0.53 (bae8c09)}

[read-the-docs-community]

Documentation build overview

📚 Meltano SDK | 🛠️ Build #32280714 | 📁 Comparing 29780d5 against latest (7208b6f)

  🔍 Preview build  

48 files changed · ± 40 modified · - 8 deleted

± Modified

• CONTRIBUTING.html
• batch.html
• cli_commands.html
• code_samples.html
• deprecation.html
• dev_guide.html
• genindex.html
• incremental_replication.html
• parent_streams.html
• reference.html
• and 30 more...

- Deleted

• classes/singer_sdk.pagination.OffsetPaginator.html
• classes/singer_sdk.pagination.PageNumberPaginator.html
• guides/consolidate-sql-imports.html
• guides/decouple-authenticators.html
• classes/typing/singer_sdk.typing.NumberType.html
• implementation/errors/design.html
• implementation/errors/hierarchy.html
• implementation/errors/index.html