Cannot use C++20 features in conda-forge cppyy 3.1.2

[how4rd]

The latest osx-arm64 version of C++ on conda-forge (3.1.2) appears not to support C++20 by default, although the docs suggest that versions above 3.0.1 should default to C++20. Is there a way to enable C++20 in the latest conda-forge version, or if not, what version do you recommend installing to support C++20? (Also, if this question fits better elsewhere, please let me know.)

Minimal example

I created a minimal test script, test_script.py, with C++20 syntax:

import cppyy

cppyy.cppdef("""
template<class T>
concept Integral = std::is_integral<T>::value;
""")

(I copied the C++ from cppreference.)

Then I created a minimal conda environment that uses the latest cppyy from conda-forge:

conda create --name cppyy-cpp20-test
conda activate cppyy-cpp20-test
conda install -c conda-forge cppyy

When I ran test_script.py in the environment, cppyy raised an error suggesting it was not using C++20:

python test_script.py
(Re-)building pre-compiled headers (options: -O2); this may take a minute ...
warning: unknown warning option '-Wno-enum-constexpr-conversion' [-Wunknown-warning-option]
warning: unknown warning option '-Wno-enum-constexpr-conversion' [-Wunknown-warning-option]
Traceback (most recent call last):
  File "/Users/howard/Desktop/test_script.py", line 3, in <module>
    cppyy.cppdef("""
    ~~~~~~~~~~~~^^^^
    template<class T>
    ^^^^^^^^^^^^^^^^^
    concept Integral = std::is_integral<T>::value;
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    """)
    ^^^^
  File "/opt/homebrew/Caskroom/miniconda/base/envs/cppyy-cpp20-test/lib/python3.13/site-packages/cppyy/__init__.py", line 234, in cppdef
    raise SyntaxError('Failed to parse the given C++ code%s' % err.err)
SyntaxError: Failed to parse the given C++ code
input_line_18:3:1: error: unknown type name 'concept'
concept Integral = std::is_integral<T>::value;
^

Dependencies

I ran this example on macOS 15.6.1 (24G90) on an M1 Mac. Here is the output of conda list for the minimal environment above:

conda list
# packages in environment at /opt/homebrew/Caskroom/miniconda/base/envs/cppyy-cpp20-test:
#
# Name                     Version          Build                Channel
bzip2                      1.0.8            hd037594_8           conda-forge
c-compiler                 1.10.0           hdf49b6b_0           conda-forge
ca-certificates            2025.10.5        hbd8a1cb_0           conda-forge
cctools                    1021.4           hb4fb6a3_0           conda-forge
cctools_osx-arm64          1021.4           h12580ec_0           conda-forge
clang                      18.1.8           default_h675cc0c_3   conda-forge
clang-18                   18.1.8           default_h5c12605_3   conda-forge
clang_impl_osx-arm64       18.1.8           h2ae9ea5_25          conda-forge
clang_osx-arm64            18.1.8           h07b0088_25          conda-forge
clangxx                    18.1.8           default_h675cc0c_3   conda-forge
clangxx_impl_osx-arm64     18.1.8           h555f467_25          conda-forge
clangxx_osx-arm64          18.1.8           h07b0088_25          conda-forge
compiler-rt                18.1.8           h855ad52_2           conda-forge
compiler-rt_osx-arm64      18.1.8           he32a8d3_2           conda-forge
cppyy                      3.1.2            py313h4749f97_3      conda-forge
cppyy-backend              1.15.2           py313h0ebd0e5_2      conda-forge
cppyy-cling                6.30.0           py313h9db1142_8      conda-forge
cpycppyy                   1.12.16          py313ha61f8ec_2      conda-forge
cxx-compiler               1.10.0           hba80287_0           conda-forge
icu                        75.1             hfee45f7_0           conda-forge
ld64                       954.16           h4c6efb1_0           conda-forge
ld64_osx-arm64             954.16           h9d5fcb0_0           conda-forge
libclang-cpp18.1           18.1.8           default_h5c12605_3   conda-forge
libcxx                     21.1.5           hf598326_0           conda-forge
libcxx-devel               17.0.6           h86353a2_6           conda-forge
libexpat                   2.7.1            hec049ff_0           conda-forge
libffi                     3.5.2            he5f378a_0           conda-forge
libiconv                   1.18             h23cfdf5_2           conda-forge
libllvm18                  18.1.8           default_h3f38c9c_10  conda-forge
liblzma                    5.8.1            h39f12f2_2           conda-forge
libmpdec                   4.0.0            h5505292_0           conda-forge
libsqlite                  3.51.0           h8adb53f_0           conda-forge
libxml2                    2.15.1           h9329255_0           conda-forge
libxml2-16                 2.15.1           h0ff4647_0           conda-forge
libzlib                    1.3.1            h8359307_2           conda-forge
llvm-openmp                21.1.5           h4a912ad_0           conda-forge
llvm-tools                 18.1.8           default_h3f38c9c_10  conda-forge
llvm-tools-18              18.1.8           default_h3f38c9c_10  conda-forge
ncurses                    6.5              h5e97a16_3           conda-forge
openssl                    3.5.4            h5503f6c_0           conda-forge
pip                        25.2             pyh145f28c_0         conda-forge
python                     3.13.9           hfc2f54d_101_cp313   conda-forge
python_abi                 3.13             8_cp313              conda-forge
readline                   8.2              h1d1bf99_2           conda-forge
sigtool                    0.1.3            h44b9a77_0           conda-forge
tapi                       1300.6.5         h03f4b80_0           conda-forge
tk                         8.6.13           h892fb3f_2           conda-forge
tzdata                     2025b            h78e105d_0           conda-forge
zstd                       1.5.7            h6491c7d_2           conda-forge

[wlav]

The version on conda is quite old at this point, with an older LLVM/Clang, hence these support failures. Code works fine with master, which is on Clang16 since 3.5.0 (available on pip). Concepts only seem to have been fully supported since 16, see here: https://clang.llvm.org/cxx_status.html .

Note that most cppyy development effort is currently going here https://github.com/compiler-research/ with them close to having a full release. They're on Clang20.

[saraedum]

The version on conda is quite old at this point

Yes, sorry for that. Packaging cppyy-cling is quite difficult and my most recent attempts at it have failed.

I seem to have some time over the next ten days to try again.

Should I try to package compiler-research's version right away? I see that both projects have diverged quite a bit by now:

This branch is 150 commits ahead of and 168 commits behind wlav/cppyy-backend:master.

[wlav]

The differences between the backends should be immaterial as that's the part that is wholesale replaced. I have not tried it myself, though: I only have their affirmation that it passes almost all tests. They also want to replace the current structure with a single mono-repo, which would allow releasing it as a single Python package. That technically drops support for PyPy, but I haven't worked on PyPy myself either since a long time (just tested that it still works from time to time). They are also thinking of renaming it, with a module called cppyy as a backwards-compatibility overlay, so then it would still be 2 packages (but the cppyy one being pure Python, as is now, so easy to package).

It's been a while since I cut a release for PyPI also. Maybe I should just cut one with master as-is call it 3.5.1 or 3.6?

[how4rd]

Thank you for such a quick response!

I got cppyy working using a combination of conda and pip installations. For others reading, I've written below how I resolved the issue.

If a more recent version of cppyy could be backaged on conda (be that a version from here or a more recent one from compiler-research), that would be excellent, thank you!

Solution

I created a conda environment containing the pip, python, and compilers packages as described in the docs. Then using pip I installed cppyy.

Here is a simple environment.yaml file:

name: example-environment
channels:
  - conda-forge
  - defaults
dependencies:
  - compilers
  - pip
  - python
  - pip:
      - cppyy

I ended up with these versions:

compilers                      1.11.0                hce30654_0               conda-forge
pip                            25.3                  pyh145f28c_0             conda-forge
python                         3.13.9                hfc2f54d_101_cp313       conda-forge
cppyy                          3.5.0                 pypi_0                   pypi

When I ran import cppyy inside Python, I got library import errors similar to #308. As discussed in that issue (and as warned about in the docs), the problem was that the pip installation of cppyy uses environment variables incompatible with the conda environment.

Once I activated the environment, I could run the fix: find the path where that conda environment stores libraries, and add that path to the environment variables where cppyy looks for libraries. For example, all my conda environments are inside the directory /opt/homebrew/Caskroom/miniconda/base/envs (although it's likely different on your machine), and the attached conda environment is called example-environment, so the folder containing that environment's libraries is

/opt/homebrew/Caskroom/miniconda/base/envs/example-environment/lib

and the two commands that I ran are

export LD_LIBRARY_PATH=/opt/homebrew/Caskroom/miniconda/base/envs/example-environment/lib:$LD_LIBRARY_PATH
export DYLD_LIBRARY_PATH=/opt/homebrew/Caskroom/miniconda/base/envs/example-environment/lib:$DYLD_LIBRARY_PATH

Unfortunately, I have to run those two export commands every time I activate the conda environment. Perhaps I could avoid rerunning them manually by placing them in a script that runs whenever the conda environment activates.