Skip to content

Conversation

@radoering
Copy link
Member

@radoering radoering commented May 11, 2025

Pull Request Check List

Resolves: #10299

  • Added tests for changed code.
  • Updated documentation for changed code.

Example:

[tool.poetry.build-constraints]
some-package = { setuptools = "<78" }

At some point, I wondered if build constraints should be part of the pyproject.toml or part of the configuration. I stuck with the pyproject.toml because it is easier to implement.

A small shortcoming is that the constraints are only applied when installing dependencies - not when building a package to extract metadata. I think we can live with that for the time being.

Another shortcoming is that the constraints are not really constraints but implemented as dependencies of an optional group, which is not installed. In most cases, this will not make a difference but in rare cases it could result in unnecessary solver errors. Implementing real constraints will cost more effort and could be done later if it proves to be necessary.

Summary by Sourcery

Introduce support for specifying build constraints for dependencies in pyproject.toml, allowing users to control build-time dependency versions for specific packages.

New Features:

  • Add support for [tool.poetry.build-constraints] section in pyproject.toml to specify build-time dependency constraints for individual packages.

Enhancements:

  • Propagate build constraints through the installation and build process, including installer, executor, and isolated build environments.

Documentation:

  • Document the new build-constraints feature and its usage in the user guide.

Tests:

  • Add tests to verify correct parsing, propagation, and enforcement of build constraints during installation and isolated builds.

@sourcery-ai
Copy link

sourcery-ai bot commented May 11, 2025

Reviewer's Guide

This pull request introduces support for specifying build constraints for dependencies via a new [tool.poetry.build-constraints] section in pyproject.toml, propagating these constraints through the installation and build process, updating the relevant core logic, and adding comprehensive tests and documentation.

File-Level Changes

Change Details Files
Add support for specifying build constraints for dependencies in pyproject.toml and propagate them through the build and installation process.
  • Introduce [tool.poetry.build-constraints] section parsing in Factory and store constraints in Poetry object.
  • Update Installer, Executor, and Chef to accept and pass build constraints to isolated build environments.
  • Modify IsolatedEnv and isolated_builder to install dependencies with constraints during build environment setup.
  • Ensure build constraints are applied when preparing and installing packages from source or VCS.
src/poetry/factory.py
src/poetry/poetry.py
src/poetry/installation/installer.py
src/poetry/installation/executor.py
src/poetry/installation/chef.py
src/poetry/utils/isolated_build.py
Update documentation and schema to describe and validate the new build constraints feature.
  • Add documentation for build-constraints in pyproject.md with usage examples and explanation.
  • Update JSON schema to support and validate build-constraints section.
docs/pyproject.md
src/poetry/json/schemas/poetry.json
Add and update tests to cover build constraints parsing, propagation, and enforcement.
  • Add fixture pyproject.toml files with build constraints for testing.
  • Add and update tests for Factory, IsolatedEnv, and installation logic to verify correct handling of build constraints.
  • Test error cases and correct application of constraints during installation.
tests/test_factory.py
tests/installation/test_executor.py
tests/utils/test_isolated_build.py
tests/json/test_schema.py
tests/fixtures/build_constraints/pyproject.toml
tests/json/fixtures/build_constraints.toml
Wire build constraints through CLI application setup to ensure they are respected in all install flows.
  • Update console application to pass build constraints to the installer during command configuration.
src/poetry/console/application.py

Possibly linked issues


Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @radoering - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 2 issues found
  • 🟢 Security: all looks good
  • 🟡 Testing: 2 issues found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@radoering radoering force-pushed the build-constraints branch from f440bc7 to 92eeb2d Compare May 11, 2025 16:04
@radoering radoering added the impact/docs Contains or requires documentation changes label May 11, 2025
@github-actions
Copy link

github-actions bot commented May 11, 2025

Deploy preview for website ready!

✅ Preview
https://website-r1vlhty3t-python-poetry.vercel.app

Built with commit 780c7cc.
This pull request is being automatically deployed with vercel-action

@vfazio
Copy link
Contributor

vfazio commented Sep 17, 2025

I've tested this out with an internal scenario (see linked issue) and it seemed to work as expected!

Testing with a constrained setuptools
[project]
name = "tmp-6g7i6voyom"
version = "0.1.0"
description = ""
authors = [
    {name = "Your Name",email = "[email protected]"}
]
readme = "README.md"
requires-python = ">=3.11, <4"
dependencies = [
]

[tool.poetry]
packages = [{include = "tmp"}]

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.build-constraints]
smbus-cffi = { setuptools = "<80.7.0" }

[[tool.poetry.source]]
name = "upstream-pypi"
url = "https://nexus.xes-mad.com/repository/upstream-pypi/simple"

[[tool.poetry.source]]
name = "xes-pypi"
url = "https://nexus.xes-mad.com/repository/xes-pypi/simple"
root@rpi-533736:/var/tmp/tmp.7vVXr3Q77v# /root/.local/bin/poetry@pr10388 add xes-bscan==0.9
Creating virtualenv tmp-6g7i6voyom in /var/tmp/tmp.7vVXr3Q77v/.venv

Updating dependencies
Resolving dependencies... (9.0s)

Package operations: 9 installs, 0 updates, 0 removals

  - Installing typing-extensions (4.15.0)
  - Installing cffi (2.0.0)
  - Installing python-utils (3.9.1)
  - Installing cython (3.1.4)
  - Installing pkgconfig (1.5.5)
  - Installing progressbar2 (4.5.0)
  - Installing smbus-cffi (0.5.1): Failed

PEP517 build of a dependency failed

Backend subprocess exited when trying to invoke build_wheel

    | Command '['/var/tmp/tmpfyk05_7o/.venv/bin/python', '/root/.local/share/pipx/venvs/poetry@pr10388/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py', 'build_wheel', '/var/tmp/tmpx2ml5vof']' returned non-zero exit status 1.
    | 
    | Traceback (most recent call last):
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/cparser.py", line 5, in <module>
    |     from . import _pycparser as pycparser
    | ImportError: cannot import name '_pycparser' from 'cffi' (/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/__init__.py)
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/root/.local/share/pipx/venvs/poetry@pr10388/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
    |     main()
    |   File "/root/.local/share/pipx/venvs/poetry@pr10388/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 373, in main
    |     json_out["return_val"] = hook(**hook_input["kwargs"])
    |                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/root/.local/share/pipx/venvs/poetry@pr10388/lib/python3.11/site-packages/pyproject_hooks/_in_process/_in_process.py", line 280, in build_wheel
    |     return _build_backend().build_wheel(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 432, in build_wheel
    |     return _build(['bdist_wheel'])
    |            ^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 423, in _build
    |     return self._build_with_temp_dir(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 404, in _build_with_temp_dir
    |     self.run_setup()
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 512, in run_setup
    |     super().run_setup(setup_script=setup_script)
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/build_meta.py", line 317, in run_setup
    |     exec(code, locals())
    |   File "<string>", line 17, in <module>
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/__init__.py", line 117, in setup
    |     return distutils.core.setup(**attrs)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 148, in setup
    |     _setup_distribution = dist = klass(attrs)
    |                                  ^^^^^^^^^^^^
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/dist.py", line 323, in __init__
    |     _Distribution.__init__(self, dist_attrs)
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 309, in __init__
    |     self.finalize_options()
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/dist.py", line 788, in finalize_options
    |     ep(self)
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools/dist.py", line 808, in _finalize_setup_keywords
    |     ep.load()(self, ep.name, value)
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/setuptools_ext.py", line 229, in cffi_modules
    |     add_cffi_module(dist, cffi_module)
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/setuptools_ext.py", line 50, in add_cffi_module
    |     execfile(build_file_name, mod_vars)
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/setuptools_ext.py", line 26, in execfile
    |     exec(code, glob, glob)
    |   File "smbus_cffi_build.py", line 6, in <module>
    |     ffi = FFI()
    |           ^^^^^
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/api.py", line 65, in __init__
    |     from . import cparser
    |   File "/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/cffi/cparser.py", line 7, in <module>
    |     import pycparser
    | ModuleNotFoundError: No module named 'pycparser'

Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with smbus-cffi (0.5.1)

  - not supporting PEP 517 builds
  - not specifying PEP 517 build requirements correctly
  - the build requirements are incompatible with your operating system or Python version
  - the build requirements are missing system dependencies (eg: compilers, libraries, headers).

You can verify this by running pip wheel --no-cache-dir --use-pep517 "smbus-cffi (==0.5.1)".

  - Installing tatsu (5.6.1)
root@rpi-533736:/var/tmp/tmp.7vVXr3Q77v# find /var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/ -name "setuptools-*"
/var/tmp/tmpfyk05_7o/.venv/lib/python3.11/site-packages/setuptools-80.6.0.dist-info
Testing with a constrained cffi
[project]
name = "tmp-6g7i6voyom"
version = "0.1.0"
description = ""
authors = [
    {name = "Your Name",email = "[email protected]"}
]
readme = "README.md"
requires-python = ">=3.11, <4"
dependencies = [
    "xes-bscan (==0.9)"
]

[tool.poetry]
packages = [{include = "tmp"}]

[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.build-constraints]
smbus-cffi = { cffi = "<2.0.0" }

[[tool.poetry.source]]
name = "upstream-pypi"
url = "https://nexus.xes-mad.com/repository/upstream-pypi/simple"

[[tool.poetry.source]]
name = "xes-pypi"
url = "https://nexus.xes-mad.com/repository/xes-pypi/simple"
root@rpi-533736:/var/tmp/tmp.7vVXr3Q77v# /root/.local/bin/poetry@pr10388 add xes-bscan==0.9
Creating virtualenv tmp-6g7i6voyom in /var/tmp/tmp.7vVXr3Q77v/.venv

Updating dependencies
Resolving dependencies... (0.2s)

Package operations: 9 installs, 0 updates, 0 removals

  - Installing typing-extensions (4.15.0)
  - Installing cffi (2.0.0)
  - Installing python-utils (3.9.1)
  - Installing cython (3.1.4)
  - Installing pkgconfig (1.5.5)
  - Installing progressbar2 (4.5.0)
  - Installing smbus-cffi (0.5.1)
  - Installing tatsu (5.6.1)
  - Installing xes-bscan (0.9): Installing...
  - Installing xes-bscan (0.9)


root@rpi-533736:/var/tmp/tmp.7vVXr3Q77v# ls /var/tmp/tmp7vgrkovx/.venv/lib/python3.11/site-packages/{setuptools-*,cffi-*}
/var/tmp/tmp7vgrkovx/.venv/lib/python3.11/site-packages/cffi-1.17.1.dist-info:
entry_points.txt  INSTALLER  LICENSE  METADATA	RECORD	top_level.txt  WHEEL

/var/tmp/tmp7vgrkovx/.venv/lib/python3.11/site-packages/setuptools-80.9.0.dist-info:
entry_points.txt  INSTALLER  licenses  METADATA  RECORD  top_level.txt	WHEEL

@vfazio
Copy link
Contributor

vfazio commented Sep 17, 2025

I tried getting hacky to add pip a la:

[tool.poetry.build-constraints]
smbus-cffi = { setuptools = "<78", pip = ">23" }

so the build would work with an older setuptools, but pip was not inserted into the environment. So existing dependencies can be constrained but new dependencies cannot be inserted (or at least not pip).

@radoering
Copy link
Member Author

So existing dependencies can be constrained but new dependencies cannot be inserted (or at least not pip).

This is expected because:

implemented as dependencies of an optional group, which is not installed

build-constraints works in analogy to a constraints.txt file.

If we decided to allow to add dependencies, build-constraints would no longer be a suitable name.

@vfazio
Copy link
Contributor

vfazio commented Sep 17, 2025

So existing dependencies can be constrained but new dependencies cannot be inserted (or at least not pip).

This is expected because:

implemented as dependencies of an optional group, which is not installed

build-constraints works in analogy to a constraints.txt file.

If we decided to allow to add dependencies, build-constraints would no longer be a suitable name.

I think this is absolutely fine, i just wanted to mention it.

This PR functions as I expected. I don't know if you want input from others i know its been open for a while.

@vfazio
Copy link
Contributor

vfazio commented Sep 19, 2025

lmk if there's anything else I can help with.

@radoering
Copy link
Member Author

lmk if there's anything else I can help with.

Thanks for trying it out. I have it on my agenda for the next minor release. Since this is not imminent, I will probably keep this PR open a bit longer and merge it as soon as the next release is near (assuming there is no other feedback in the meantime).

@vfazio
Copy link
Contributor

vfazio commented Nov 6, 2025

Out of curiosity, is there a milestone or target for 2.3.x where this might be included? I'm trying to decide if I can hold out for that release or if I should stay pinned on the current version of poetry we have for an upcoming release.

Thanks!

@radoering
Copy link
Member Author

Things may change but I would not expect it in the next few weeks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

impact/docs Contains or requires documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow for additional constraints on dependencies used in build environments

2 participants