Improved Guessing of boost_python version by kjmeagher · Pull Request #16027 · mesonbuild/meson

kjmeagher · 2026-07-20T20:14:47Z

The most important thing about linking with boost python is to make sure
that you match the version of python you are compiling with to the
boost_python library. It is only nescessary to match to minor version
not the patch version. Originally boost_python did not contain the
version in the filename and it was difficult to determine which python
version a given boost_python module was supposed to be. At some point,
boost renamed the library to boost_pythonXYY where X is the python major
version and YY is the minor version. This made things significatnly
easier as you could search for a filename with the target version of
python that you wanted.

Meson added support to autodetect the python version in #5596 and #6855.
The way the auto detection works is very ill-advised because there is no
gaurentee that the boost_python version you find will match the version
of python you are compiling for. This PR rectifies this by removing the
autodetection feature and adding documentation informing the user the
correct way to detect boost_python. If the user attempts to use
auto-detection an error message is printed directing them to the
documentation.

Rejected Alternatives:

Find a way to pass the python instillation object to dependency. This
might look a little bit nicer but doesn't really add much over
appending the string to the end of the moudle name. I don't think
anyone wants to add another parameter to dependency() just for boost
python
Add boost_python support to the python module and fail if you cant
find a boost_python which matches the python version. This would be
more inline with the way dependency detection works in meson, but it
is more complicated than just appending strings, and in my opinion not
really worth the effort
A dedicated boost_python meson module. This would search both the
python space and the boost space looking for a version match. This
would be the most likely to find a viable boost python, but ususally
people know which version of python they want to compile so this
method isn't very helpful.

kjmeagher · 2026-07-20T21:12:47Z

This is an alternative to #15982

dcbaker

Even if the default guessing was ill advised, it works fine in configurations that only have one system python and one boost. Removing the feature will make builds that used to work fine stop working, and that is a heavy hammer.

@eli-schwartz @thesamesam are probably interested in looking at this as well.

dcbaker · 2026-07-20T21:15:37Z

+        if mod_name in self.boost_python_libs:
+            mlog.error(
+                "Support for ",  mlog.bold(mod_name), ' without specifying the version was dropped in Meson 1.12. ',
+                "For the correct solution for these libraries, please see ",
+                "https://mesonbuild.com/Dependencies.html#boost-python-and-boost-numpy"
+                )
+        return False


This would be a hard compatibility break. We generally do not do that unless the feature being removed wouldn't have worked anyway ever (say because it would raise a python exception).

In this case I think we need to keep this, with a FeatureDeprecated for plain python and plain numpy. I would also not object to a mlog.warning() here that is non_fatal so that users of new Meson get the notice even if their minimum version is less than 1.12.0

The simple way would be to check if the dependency name has any version listed and if not, print a warning and fall back to the old code.

An even simpler version: if the version number is not specified, print the error and append the Python version of the current interpreter. It should be "the right thing" to do most of the time.

I changed it to print a warning if any version guessing is taking place. Then if more than one possible version exists, it selects the current interpreter, if that isn't present it takes the highest version

eli-schwartz · 2026-07-20T21:50:00Z

I agree with the PR that we should make people switch to specifying the module name correctly, and I agree with @dcbaker that for backwards compatibility reasons we should continue to accept the old form with a warning.

I think we can still raise a hard error if there are multiple libboost_python* installed. The error message in that case would say "ambiguous module, we cannot possibly know what you actually want, but hey good news it's deprecated anyway so you should go fix that".

dcbaker · 2026-08-04T16:15:53Z

I'm putting this back on 1.12 since It's the last thing we're waiting for.

@kjmeagher do you have time to work on this? If not I'll make the fixups I suggested so we can merge this, as it's the last thing we're waiting on for the 1.12 release.

kjmeagher · 2026-08-04T17:50:35Z

I tried to come up with a solution that did the correct thing and was backwards compatible with the old behavior, but this was more challenging than I anticipated. The way filter_libraries() removes libraries before the check makes it difficult. I think you should go forward with what ever you think is best for the release

dcbaker · 2026-08-06T16:20:56Z

Okay, I'll attempt to update this tomorrow (I have other things that are higher priority for today).

The most important thing about linking with boost python is to make sure that you match the version of python you are compiling with to the boost_python library. It is only nescessary to match to minor version not the patch version. Originally boost_python did not contain the version in the filename and it was difficult to determine which python version a given boost_python module was supposed to be. At some point, boost renamed the library to boost_pythonXYY where X is the python major version and YY is the minor version. This made things significatnly easier as you could search for a filename with the target version of python that you wanted. Meson added support to autodetect the python version in mesonbuild#5596 and mesonbuild#6855. The way the auto detection works is very ill-advised because there is no gaurentee that the boost_python version you find will match the version of python you are compiling for. This PR rectifies this by removing the autodetection feature and adding documentation informing the user the correct way to detect boost_python. If the user attempts to use auto-detection an error message is printed directing them to the documentation. Rejected Alternatives: * Find a way to pass the python instillation object to dependency. This might look a little bit nicer but doesn't really add much over appending the string to the end of the moudle name. I don't think anyone wants to add another parameter to `dependency()` just for boost python * Add boost_python support to the python module and fail if you cant find a boost_python which matches the python version. This would be more inline with the way dependency detection works in meson, but it is more complicated than just appending strings, and in my opinion not really worth the effort * A dedicated boost_python meson module. This would search both the python space and the boost space looking for a version match. This would be the most likely to find a viable boost python, but ususally people know which version of python they want to compile so this method isn't very helpful.

kjmeagher · 2026-08-20T21:26:08Z

I had to rewrite the whole way it tries to deal with python modules. Now it tags the version of python modules in the constructor. If you specify a python module without a version it will look at all the python modules and if there is exactly one it will accept that one, then if there is one matches the current interpreter it will take that one, if that fails it will take the newest version. It will warn on any attempt to use an unversioned module

windows can understand to `42 boost python`

kjmeagher · 2026-08-21T00:52:25Z

I disabled boost_python2 testing because gentoo still has python2 but the boost emerge doesn't provide boost_python2. It doesn't worth the effort to investigate

dnicolodi · 2026-08-21T06:31:18Z

+
+        # if one of the libs matches the current interperter got with that one
+        for lib in pylibs:
+            if lib.python_version[0] == version_info[0] and lib.python_version[1] == version_info[1]:


The Python interpreter executing Meson is not necessarily the one that the compilation is targeting, thus sys.version_info is not necessarily the same as the one returned by the Meson's python module.

Anyhow, this would be better spelled:

Suggested change

if lib.python_version[0] == version_info[0] and lib.python_version[1] == version_info[1]:

if lib.python_version == version_info[:2]:

The main point in this comment has not been addressed!

This was based on a suggestion from @jpakkane above, ill let him weigh in if he thinks it is still a good idea

dnicolodi · 2026-08-21T06:39:12Z

+        # Python libraries are special because of the included
+        # minor version in the module name.
+        self.python_version: tuple[int, int] | None = None
+        for bpl in BoostDependency.boost_python_libs:
+            if self.mod_name.startswith(bpl):
+                python_version_str = self.mod_name[len(bpl):]
+                if not python_version_str.isdigit():
+                    continue
+                self.python_version = (int(python_version_str[:1]), int(python_version_str[1:]))


This looses the handling of versioned names spelled in different ways that are currently handled in fix_python_name below. If this is intentional, it need to be documented, and it needs to be checked that the "alternative" naming schemes handled in fix_python_name are not in use on any supported platform.

The more complicated behavior was due to Gentoo and OpenSUSE renameing the libraries circa 2020-2019 because they did not understand how boost_python was supposed to work. I do not believe there is any utility in meson continuing to support this behavior. I added a short note to the code

I don't think a comment in the code is useful for this. Anyone interested in the history of this could simply look at the commit log. The change should be reported in the commit message. Also, "circa 2020-2019" is not very informative. Which distribution releases are affected? Are any of these still supported?

You can see in the reports in #4788 and #6886 when and which distribution were renaming library filenames, they all stopped doing it around 2020 and these platforms are already EOL. I really don't think that doing anymore detective work is worth it for a library that hardly anyone uses.

Co-authored-by: Daniele Nicolodi <daniele@grinta.net>

kjmeagher requested a review from jpakkane as a code owner July 20, 2026 20:14

kjmeagher force-pushed the kjm/stop_guessing_boost_python branch 2 times, most recently from 71bd861 to 4450b5a Compare July 20, 2026 20:49

dcbaker requested changes Jul 20, 2026

View reviewed changes

dcbaker added this to the 1.12 milestone Jul 24, 2026

bonzini modified the milestones: 1.12, 1.13 Jul 29, 2026

dcbaker modified the milestones: 1.13, 1.12 Aug 4, 2026

bonzini modified the milestones: 1.12, 1.13 Aug 18, 2026

kjmeagher added 2 commits August 20, 2026 16:03

handle python correctly

f6e3477

kjmeagher force-pushed the kjm/stop_guessing_boost_python branch from 4450b5a to f6e3477 Compare August 20, 2026 21:06

pylint and flake8

7898f18

kjmeagher added 2 commits August 20, 2026 19:21

remove python2 tests in 1 boost and add fake boost libraries that

b62a8bb

windows can understand to `42 boost python`

use set and only test in certian cases

1793a99

kjmeagher changed the title ~~Stop Guessing which version of boost_python to select~~ Improved Guessing of boost_python version Aug 21, 2026

rename windows files

58cace0

dnicolodi suggested changes Aug 21, 2026

View reviewed changes

kjmeagher and others added 3 commits August 21, 2026 11:09

Update mesonbuild/dependencies/boost.py

f3d3cfd

Co-authored-by: Daniele Nicolodi <daniele@grinta.net>

incorperate suggestions from dnicolodi

553c465

install boost_numpy on suse and ubuntu and disable windows tests

de729d3

	if lib.python_version[0] == version_info[0] and lib.python_version[1] == version_info[1]:
	if lib.python_version == version_info[:2]:

Uh oh!

Uh oh!

Conversation

kjmeagher commented Jul 20, 2026

Uh oh!

kjmeagher commented Jul 20, 2026

Uh oh!

dcbaker left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eli-schwartz commented Jul 20, 2026

Uh oh!

dcbaker commented Aug 4, 2026

Uh oh!

kjmeagher commented Aug 4, 2026

Uh oh!

dcbaker commented Aug 6, 2026

Uh oh!

kjmeagher commented Aug 20, 2026

Uh oh!

kjmeagher commented Aug 21, 2026

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants