feat: Pytest markers for tap built-in tests

[edgarrmondragon]

Summary by Sourcery

Introduce pytest markers for Singer SDK built-in tests, enabling granular test selection and updating documentation, project templates, and CI workflows to support the new markers.

New Features:

• Add pytest markers singer_stream and singer_stream_attribute to auto-generated tests

Enhancements:

• Apply markers automatically in pytest_generate_tests for BaseTestClass subclasses
• Annotate sync_all methods with override decorator in testing runners
• Refine BaseTestClass.params type annotation to list of parameter mappings

Build:

• Include packages/** path in test and CodeQL workflows
• Add pytest.ini marker definitions and addopts to cookiecutter and root pyproject.toml

CI:

• Update GitHub test workflow to include packages/** in paths

Documentation:

• Add Pytest Markers section with usage examples and benefits in testing documentation

[sourcery-ai]

Reviewer's Guide

This PR introduces pytest markers for auto-generated Singer SDK tests by enhancing the pytest plugin logic, updating documentation and project templates, refining test factory typing, annotating runner overrides, and adjusting CI/workflow configurations.

Class diagram for updated BaseTestClass and pytest_generate_tests

classDiagram
    class BaseTestClass {
        params: dict[str, list[dict[str, Any]]]
        param_ids: dict[str, list[str]]
    }
    class pytest_generate_tests {
        +pytest_generate_tests(metafunc: pytest.Metafunc) -> None
    }
    pytest_generate_tests ..> BaseTestClass : uses

Loading

Flow diagram for pytest marker assignment in auto-generated tests

flowchart TD
    A[pytest_generate_tests called] --> B{Is metafunc.cls subclass of BaseTestClass?}
    B -- No --> Z[Do nothing]
    B -- Yes --> C[Get test function name]
    C --> D{Name contains 'tap_stream_'?}
    D -- Yes and not 'attribute' --> E[Assign singer_stream marker]
    D -- No --> F{Name contains 'tap_stream_attribute_'?}
    F -- Yes --> G[Assign singer_stream_attribute marker]
    F -- No --> H[No marker assigned]
    E & G & H --> I[pytest.param created with appropriate marks]
    I --> J[metafunc.parametrize called]

Loading

File-Level Changes

Change	Details	Files
Implemented pytest marker assignment in the test generation plugin	Limit pytest_generate_tests to BaseTestClass subclasses Identify stream vs attribute tests based on function name patterns Attach corresponding pytest marks to each parameterized test	singer_sdk/testing/pytest_plugin.py
Added detailed pytest markers documentation	Introduce “Pytest Markers” section in testing docs List available markers, usage examples, and benefits	docs/testing.md
Updated project and template configs with pytest markers and options	Add addopts durations and marker definitions to cookiecutter pyproject.toml for target, mapper, and tap templates Append singer_stream and singer_stream_attribute markers to root pyproject.toml	cookiecutter/target-template/{{cookiecutter.target_id}}/pyproject.toml cookiecutter/mapper-template/{{cookiecutter.mapper_id}}/pyproject.toml cookiecutter/tap-template/{{cookiecutter.tap_id}}/pyproject.toml pyproject.toml
Corrected test factory parameter typing	Change BaseTestClass.params annotation to dict[str, list[dict[str, Any]]]	singer_sdk/testing/factory.py
Annotated runner methods with @OverRide	Conditionally import override from typing_extensions or typing Add @OverRide decorator to sync_all methods	singer_sdk/testing/runners.py
Expanded CI/workflow and CodeQL paths	Include packages/** in GitHub Actions test triggers Add packages path to CodeQL configuration	.github/workflows/test.yml .github/codeql-config.yml

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[read-the-docs-community]

Documentation build overview

📚 Meltano SDK | 🛠️ build #28933006 (237dbf0) | 🔍 preview

Files changed

Comparing with latest (4cceea9...237dbf0)

Show files (1) | 1 modified | 0 added | 0 deleted

File	Status
testing.html	📝 modified

[codecov]

Codecov Report

❌ Patch coverage is 94.11765% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 92.46%. Comparing base (c39924f) to head (237dbf0).
⚠️ Report is 466 commits behind head on main.

Files with missing lines	Patch %	Lines
singer_sdk/testing/pytest_plugin.py	87.50%	0 Missing and 1 partial ⚠️

❌ Your patch check has failed because the patch coverage (94.11%) is below the target coverage (100.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3176      +/-   ##
==========================================
- Coverage   92.46%   92.46%   -0.01%     
==========================================
  Files          63       63              
  Lines        5363     5375      +12     
  Branches      674      677       +3     
==========================================
+ Hits         4959     4970      +11     
  Misses        285      285              
- Partials      119      120       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[codspeed-hq]

CodSpeed Performance Report

Merging #3176 will not alter performance

_{Comparing edgarrmondragon/feat/builtin-tests-markers (237dbf0) with main (c39924f)}

Summary

✅ 8 untouched benchmarks