Skip to content

vtk: 9.2.6 -> 9.3.1#354855

Closed
MatthewCroughan wants to merge 2 commits intoNixOS:masterfrom
MatthewCroughan:mc/update-vtk
Closed

vtk: 9.2.6 -> 9.3.1#354855
MatthewCroughan wants to merge 2 commits intoNixOS:masterfrom
MatthewCroughan:mc/update-vtk

Conversation

@MatthewCroughan
Copy link
Contributor

Things done

Routine update.

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@MatthewCroughan MatthewCroughan mentioned this pull request Nov 9, 2024
13 tasks
@ofborg ofborg bot requested a review from tfmoraes November 10, 2024 05:47
@ofborg ofborg bot added 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Nov 10, 2024
@MatthewCroughan
Copy link
Contributor Author

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review


x86_64-linux

❌ 6 packages failed to build:
  • cloudcompare
  • elmerfem
  • pcl
  • python312Packages.dicom2nifti
  • python312Packages.dicom2nifti.dist
  • rtabmap
✅ 91 packages built:
  • ants
  • appcsxcad
  • c3d
  • csxcad
  • elastix
  • exhibit
  • ezminc
  • f3d
  • f3d.man
  • f3d_egl
  • f3d_egl.man
  • freecad
  • freecad-wayland
  • gdcm
  • intensity-normalization (python312Packages.intensity-normalization)
  • intensity-normalization.dist (python312Packages.intensity-normalization.dist)
  • itk
  • itk_5_2
  • libsForQt5.qcsxcad
  • libsForQt5.qcsxcad.dev
  • mirtk
  • mrtrix
  • octavePackages.dicom
  • openems
  • python311Packages.dicom2nifti
  • python311Packages.dicom2nifti.dist
  • python311Packages.f3d
  • python311Packages.f3d.man
  • python311Packages.f3d_egl
  • python311Packages.f3d_egl.man
  • python311Packages.gdcm
  • python311Packages.intensity-normalization
  • python311Packages.intensity-normalization.dist
  • python311Packages.itk
  • python311Packages.mayavi
  • python311Packages.mayavi.dist
  • python311Packages.medpy
  • python311Packages.medpy.dist
  • python311Packages.pydicom-seg
  • python311Packages.pydicom-seg.dist
  • python311Packages.pymedio
  • python311Packages.pymedio.dist
  • python311Packages.pyradiomics
  • python311Packages.pyradiomics.dist
  • python311Packages.python-csxcad
  • python311Packages.python-csxcad.dist
  • python311Packages.python-openems
  • python311Packages.python-openems.dist
  • python311Packages.pyvista
  • python311Packages.pyvista.dist
  • python311Packages.sfepy
  • python311Packages.sfepy.dist
  • python311Packages.simpleitk
  • python311Packages.simpleitk.dist
  • python311Packages.torchio
  • python311Packages.torchio.dist
  • python311Packages.vtk
  • python312Packages.f3d
  • python312Packages.f3d.man
  • python312Packages.f3d_egl
  • python312Packages.f3d_egl.man
  • python312Packages.gdcm
  • python312Packages.itk
  • python312Packages.mayavi
  • python312Packages.mayavi.dist
  • python312Packages.medpy
  • python312Packages.medpy.dist
  • python312Packages.pydicom-seg
  • python312Packages.pydicom-seg.dist
  • python312Packages.pymedio
  • python312Packages.pymedio.dist
  • python312Packages.pyradiomics
  • python312Packages.pyradiomics.dist
  • python312Packages.python-csxcad
  • python312Packages.python-csxcad.dist
  • python312Packages.python-openems
  • python312Packages.python-openems.dist
  • python312Packages.pyvista
  • python312Packages.pyvista.dist
  • python312Packages.sfepy
  • python312Packages.sfepy.dist
  • python312Packages.simpleitk
  • python312Packages.simpleitk.dist
  • python312Packages.torchio
  • python312Packages.torchio.dist
  • python312Packages.vtk
  • simpleitk
  • therion
  • vtk
  • vtkWithQt5
  • vtk_9_egl

@MatthewCroughan
Copy link
Contributor Author

@nh2 @Wulfsta @broke @bcdarwin I've been recommended to ping you all to see if we can get VTK updated. It seems like your packages are incompatible with VTK 9.3.1, which might just be because the packages need to be updated.

Otherwise, let me know if there's a deeper issue with updating VTK, and if we need to have a vtk_9_2_6 package to support apps that haven't upgraded and won't upgrade for a while!

@bcdarwin
Copy link
Member

bcdarwin commented Nov 28, 2024

I haven't looked into the python312Packages.dicom2nifti issue yet, but this PR still has the problem that caused #277465 not to be merged, i.e., the Darwin build timing out in a pretty ridiculous way, apparently only finishing ~30% of the build. I don't have any Darwin machines to investigate but I really don't think 9.3.x should be taking 3x longer to build than 9.2.6.

@emilazy curious if you have any thoughts on this given you changed some Darwin-related compiler warnings settings for this package recently?

@bcdarwin
Copy link
Member

@MatthewCroughan 9.4.0 is now out 😑 - might be worth updating this PR?

@MatthewCroughan
Copy link
Contributor Author

@bcdarwin 9.4.0 breaks a lot more, so maybe not.

@jys1670
Copy link
Contributor

jys1670 commented Dec 24, 2024

Thanks for working on this one.
Since some patching is included with the version bump, could you also fix this line?
"-DOPENGL_INCLUDE_DIR=${libGL}/include" in generic.nix
The issue is that libGL does not contain an include directory; it should be libGL.dev.
For some reason, it doesn't affect compilation with Qt5, but when Qt6 is used, the incorrect OpenGL include directory causes a bunch of strange CMake errors, and the compilation fails.

@bcdarwin
Copy link
Member

Thanks for working on this one. Since some patching is included with the version bump, could you also fix this line? "-DOPENGL_INCLUDE_DIR=${libGL}/include" in generic.nix The issue is that libGL does not contain an include directory; it should be libGL.dev. For some reason, it doesn't affect compilation with Qt5, but when Qt6 is used, the incorrect OpenGL include directory causes a bunch of strange CMake errors, and the compilation fails.

Given that the version bump seems fairly stalled due to OS X build timeouts, seems like this change should be made in a separate PR (assuming the current VTK 9.2.6 is compatible with Qt6?).

@emilazy
Copy link
Member

emilazy commented Dec 27, 2024

I don’t think a timeout on x86_64-darwin ofborg is worth holding this up over. ofborg is separate infrastructure to Hydra and it doesn’t necessarily mean there’ll be issues on the latter. Probably the builder was just overloaded.

BTW, ${lib.getInclude libGL}/include is the most correct thing here.

@emilazy
Copy link
Member

emilazy commented Dec 27, 2024

FWIW:

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 354855


aarch64-darwin

⏩ 30 packages marked as broken and skipped:
  • c3d
  • f3d_egl
  • intensity-normalization
  • intensity-normalization.dist
  • python311Packages.f3d_egl
  • python311Packages.intensity-normalization
  • python311Packages.intensity-normalization.dist
  • python311Packages.medpy
  • python311Packages.medpy.dist
  • python311Packages.pydicom-seg
  • python311Packages.pydicom-seg.dist
  • python311Packages.pymedio
  • python311Packages.pymedio.dist
  • python311Packages.pyradiomics
  • python311Packages.pyradiomics.dist
  • python311Packages.torchio
  • python311Packages.torchio.dist
  • python312Packages.f3d_egl
  • python312Packages.intensity-normalization
  • python312Packages.intensity-normalization.dist
  • python312Packages.medpy
  • python312Packages.medpy.dist
  • python312Packages.pydicom-seg
  • python312Packages.pydicom-seg.dist
  • python312Packages.pymedio
  • python312Packages.pymedio.dist
  • python312Packages.pyradiomics
  • python312Packages.pyradiomics.dist
  • python312Packages.torchio
  • python312Packages.torchio.dist
❌ 33 packages failed to build:
  • ants
  • elmerfem
  • f3d
  • gdcm
  • itk (itk_5)
  • itk_5_2
  • octavePackages.dicom
  • pcl
  • python311Packages.dicom2nifti
  • python311Packages.dicom2nifti.dist
  • python311Packages.f3d
  • python311Packages.gdcm
  • python311Packages.mayavi
  • python311Packages.mayavi.dist
  • python311Packages.pyvista
  • python311Packages.pyvista.dist
  • python311Packages.sfepy
  • python311Packages.sfepy.dist
  • python311Packages.vtk
  • python312Packages.dicom2nifti
  • python312Packages.dicom2nifti.dist
  • python312Packages.f3d
  • python312Packages.gdcm
  • python312Packages.mayavi
  • python312Packages.mayavi.dist
  • python312Packages.pyvista
  • python312Packages.pyvista.dist
  • python312Packages.sfepy
  • python312Packages.sfepy.dist
  • python312Packages.vtk
  • therion
  • vtk (vtk_9)
  • vtkWithQt5 (vtk_9_withQt5)

@bcdarwin bcdarwin mentioned this pull request Dec 27, 2024
13 tasks
@wegank
Copy link
Member

wegank commented Jan 3, 2025

Since fdopen was removed in Kitware/VTK@5a568a6, if we don't patch zlib as in 94d2817, we'll have to bump to 9.4.0+.

@emilazy
Copy link
Member

emilazy commented Jan 3, 2025

I am in favour of bumping further given the huge number of vendored libraries. However, we may also just be able to devendor zlib; cf. #171934 (comment). Sadly it seems dependent on the module whether it can be devendored or not…

@github-actions github-actions bot added 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. and removed 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. labels Jan 3, 2025
@wegank
Copy link
Member

wegank commented Jan 4, 2025

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 354855


aarch64-darwin

⏩ 5 packages marked as broken and skipped:
  • c3d
  • python312Packages.pylibjpeg-openjpeg
  • python312Packages.pylibjpeg-openjpeg.dist
  • python313Packages.pylibjpeg-openjpeg
  • python313Packages.pylibjpeg-openjpeg.dist
❌ 99 packages failed to build:
  • elmerfem
  • pcl
  • python312Packages.dcmstack
  • python312Packages.dcmstack.dist
  • python312Packages.deid
  • python312Packages.deid.dist
  • python312Packages.dicom-numpy
  • python312Packages.dicom-numpy.dist
  • python312Packages.dicom2nifti
  • python312Packages.dicom2nifti.dist
  • python312Packages.dicomweb-client
  • python312Packages.dicomweb-client.dist
  • python312Packages.dipy
  • python312Packages.dipy.dist
  • python312Packages.fslpy
  • python312Packages.fslpy.dist
  • python312Packages.heudiconv
  • python312Packages.heudiconv.dist
  • python312Packages.highdicom
  • python312Packages.highdicom.dist
  • python312Packages.mayavi
  • python312Packages.mayavi.dist
  • python312Packages.nibabel
  • python312Packages.nibabel.dist
  • python312Packages.nilearn
  • python312Packages.nilearn.dist
  • python312Packages.nipy
  • python312Packages.nipy.dist
  • python312Packages.nipype
  • python312Packages.nipype.dist
  • python312Packages.nitime
  • python312Packages.nitime.dist
  • python312Packages.nitransforms
  • python312Packages.nitransforms.dist
  • python312Packages.niworkflows
  • python312Packages.niworkflows.dist
  • python312Packages.pybids
  • python312Packages.pybids.dist
  • python312Packages.pydicom
  • python312Packages.pydicom.dist
  • python312Packages.pylibjpeg
  • python312Packages.pylibjpeg.dist
  • python312Packages.pynetdicom
  • python312Packages.pynetdicom.dist
  • python312Packages.pyorthanc
  • python312Packages.pyorthanc.dist
  • python312Packages.sfepy
  • python312Packages.sfepy.dist
  • python312Packages.templateflow
  • python312Packages.templateflow.dist
  • python312Packages.trx-python
  • python312Packages.trx-python.dist
  • python312Packages.xnatpy
  • python312Packages.xnatpy.dist
  • python313Packages.dcmstack
  • python313Packages.dcmstack.dist
  • python313Packages.deid
  • python313Packages.deid.dist
  • python313Packages.dicom-numpy
  • python313Packages.dicom-numpy.dist
  • python313Packages.dicom2nifti
  • python313Packages.dicom2nifti.dist
  • python313Packages.dicomweb-client
  • python313Packages.dicomweb-client.dist
  • python313Packages.dipy
  • python313Packages.dipy.dist
  • python313Packages.fslpy
  • python313Packages.fslpy.dist
  • python313Packages.highdicom
  • python313Packages.highdicom.dist
  • python313Packages.nibabel
  • python313Packages.nibabel.dist
  • python313Packages.nilearn
  • python313Packages.nilearn.dist
  • python313Packages.nipy
  • python313Packages.nipy.dist
  • python313Packages.nitime
  • python313Packages.nitime.dist
  • python313Packages.nitransforms
  • python313Packages.nitransforms.dist
  • python313Packages.pybids
  • python313Packages.pybids.dist
  • python313Packages.pydicom
  • python313Packages.pydicom.dist
  • python313Packages.pylibjpeg
  • python313Packages.pylibjpeg.dist
  • python313Packages.pynetdicom
  • python313Packages.pynetdicom.dist
  • python313Packages.pyorthanc
  • python313Packages.pyorthanc.dist
  • python313Packages.pyvista
  • python313Packages.pyvista.dist
  • python313Packages.templateflow
  • python313Packages.templateflow.dist
  • python313Packages.trx-python
  • python313Packages.trx-python.dist
  • python313Packages.xnatpy
  • python313Packages.xnatpy.dist
  • therion
✅ 16 packages built:
  • ants
  • f3d
  • gdcm
  • itk (itk_5)
  • itk_5_2
  • octavePackages.dicom
  • python312Packages.f3d
  • python312Packages.gdcm
  • python312Packages.pyvista
  • python312Packages.pyvista.dist
  • python312Packages.vtk
  • python313Packages.f3d
  • python313Packages.gdcm
  • python313Packages.vtk
  • vtk (vtk_9)
  • vtkWithQt5 (vtk_9_withQt5)

@github-actions github-actions bot added the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 4, 2025
@nix-owners nix-owners bot requested a review from bcdarwin January 4, 2025 17:55
@bcdarwin
Copy link
Member

bcdarwin commented Jan 4, 2025

Probably outside the scope of this PR, but I would be happy to work on some unvendoring following #171934 or #187016, with the caveat that I don't have a Darwin machine to build on so not sure how I'd get that working.

(Likewise, I don't love disabling the pydicom tests as I've been trying to enable more tests in the medical imaging ecosystem, but again don't actually use these arches.)

@bcdarwin
Copy link
Member

bcdarwin commented Jan 4, 2025

(Again maybe out of scope, but) given that 9.4.0 is already out and that some science packages (elmerfem/pcl) seemingly will be broken by this upgrade, would it make sense to introduce a vtk_9_2 for now and then in a future PR try the 9.3.x -> 9.4.x upgrade?

@emilazy
Copy link
Member

emilazy commented Jan 4, 2025

You can apply for an account on the Darwin community builder to test builds. Or I’m happy to run builds for you if prompted :) I think a lot of the Darwin issues right now are just LLVM 19, though (like when I ran nixpkgs-review and everything was just broken).

I think we should just bump as high as we can. There’s (a bit under) six months until the release and all the packages in #354855 (comment) are currently failing to build on Hydra anyway. There’s time for upstreams to adapt to the new versions, or someone to patch them, or for us to re‐introduce the old version for ZHF if necessary.

There’s also #370734.

@wegank
Copy link
Member

wegank commented Jan 4, 2025

nixpkgs-review result

Generated using nixpkgs-review.

Command: nixpkgs-review pr 354855


aarch64-darwin

⏩ 5 packages marked as broken and skipped:
  • c3d
  • python312Packages.pylibjpeg-openjpeg
  • python312Packages.pylibjpeg-openjpeg.dist
  • python313Packages.pylibjpeg-openjpeg
  • python313Packages.pylibjpeg-openjpeg.dist
❌ 43 packages failed to build:
  • elmerfem
  • pcl
  • python312Packages.deid
  • python312Packages.deid.dist
  • python312Packages.dicom2nifti
  • python312Packages.dicom2nifti.dist
  • python312Packages.dicomweb-client
  • python312Packages.dicomweb-client.dist
  • python312Packages.fslpy
  • python312Packages.fslpy.dist
  • python312Packages.heudiconv
  • python312Packages.heudiconv.dist
  • python312Packages.highdicom
  • python312Packages.highdicom.dist
  • python312Packages.mayavi
  • python312Packages.mayavi.dist
  • python312Packages.nipy
  • python312Packages.nipy.dist
  • python312Packages.niworkflows
  • python312Packages.niworkflows.dist
  • python312Packages.pyorthanc
  • python312Packages.pyorthanc.dist
  • python312Packages.sfepy
  • python312Packages.sfepy.dist
  • python313Packages.deid
  • python313Packages.deid.dist
  • python313Packages.dicom2nifti
  • python313Packages.dicom2nifti.dist
  • python313Packages.dicomweb-client
  • python313Packages.dicomweb-client.dist
  • python313Packages.fslpy
  • python313Packages.fslpy.dist
  • python313Packages.highdicom
  • python313Packages.highdicom.dist
  • python313Packages.nipy
  • python313Packages.nipy.dist
  • python313Packages.pynetdicom
  • python313Packages.pynetdicom.dist
  • python313Packages.pyorthanc
  • python313Packages.pyorthanc.dist
  • python313Packages.pyvista
  • python313Packages.pyvista.dist
  • therion
✅ 72 packages built:
  • ants
  • f3d
  • gdcm
  • itk (itk_5)
  • itk_5_2
  • octavePackages.dicom
  • python312Packages.dcmstack
  • python312Packages.dcmstack.dist
  • python312Packages.dicom-numpy
  • python312Packages.dicom-numpy.dist
  • python312Packages.dipy
  • python312Packages.dipy.dist
  • python312Packages.f3d
  • python312Packages.gdcm
  • python312Packages.nibabel
  • python312Packages.nibabel.dist
  • python312Packages.nilearn
  • python312Packages.nilearn.dist
  • python312Packages.nipype
  • python312Packages.nipype.dist
  • python312Packages.nitime
  • python312Packages.nitime.dist
  • python312Packages.nitransforms
  • python312Packages.nitransforms.dist
  • python312Packages.pybids
  • python312Packages.pybids.dist
  • python312Packages.pydicom
  • python312Packages.pydicom.dist
  • python312Packages.pylibjpeg
  • python312Packages.pylibjpeg.dist
  • python312Packages.pynetdicom
  • python312Packages.pynetdicom.dist
  • python312Packages.pyvista
  • python312Packages.pyvista.dist
  • python312Packages.templateflow
  • python312Packages.templateflow.dist
  • python312Packages.trx-python
  • python312Packages.trx-python.dist
  • python312Packages.vtk
  • python312Packages.xnatpy
  • python312Packages.xnatpy.dist
  • python313Packages.dcmstack
  • python313Packages.dcmstack.dist
  • python313Packages.dicom-numpy
  • python313Packages.dicom-numpy.dist
  • python313Packages.dipy
  • python313Packages.dipy.dist
  • python313Packages.f3d
  • python313Packages.gdcm
  • python313Packages.nibabel
  • python313Packages.nibabel.dist
  • python313Packages.nilearn
  • python313Packages.nilearn.dist
  • python313Packages.nitime
  • python313Packages.nitime.dist
  • python313Packages.nitransforms
  • python313Packages.nitransforms.dist
  • python313Packages.pybids
  • python313Packages.pybids.dist
  • python313Packages.pydicom
  • python313Packages.pydicom.dist
  • python313Packages.pylibjpeg
  • python313Packages.pylibjpeg.dist
  • python313Packages.templateflow
  • python313Packages.templateflow.dist
  • python313Packages.trx-python
  • python313Packages.trx-python.dist
  • python313Packages.vtk
  • python313Packages.xnatpy
  • python313Packages.xnatpy.dist
  • vtk (vtk_9)
  • vtkWithQt5 (vtk_9_withQt5)

@wegank
Copy link
Member

wegank commented Jan 4, 2025

My main motivation is to fix VTK on darwin, which doesn't build on master: https://hydra.nixos.org/job/nixpkgs/trunk/vtk.aarch64-darwin
(which effectively made the pydicom update untestable on darwin...)

I'll check if the workaround also applies for VTK 9.2.6, and if so, move the last two commits to a separate PR.

@wegank wegank mentioned this pull request Jan 4, 2025
13 tasks
@github-actions github-actions bot removed the 6.topic: python Python is a high-level, general-purpose programming language. label Jan 4, 2025
@bcdarwin bcdarwin mentioned this pull request Feb 25, 2025
13 tasks
@wegank wegank added the 2.status: merge conflict This PR has merge conflicts with the target branch label Apr 2, 2025
@bcdarwin
Copy link
Member

Closing in favour of #385080 since that PR at least builds on x64 Linux and updates to a newer VTK.

@bcdarwin bcdarwin closed this Apr 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

2.status: merge conflict This PR has merge conflicts with the target branch 10.rebuild-darwin: 11-100 This PR causes between 11 and 100 packages to rebuild on Darwin. 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants