Skip to content

Conversation

@makortel
Copy link
Contributor

@makortel makortel commented Aug 29, 2025

PR description:

Testing ROOT PR on an option to disable header parsing during TClass::GetClass() call (root-project/root#18402) it was noticed the type alias gets listed in the rootmap file only if the alias is requested before the real type (see root-project/root#19705 for more details).

In the mean time, having the type alias in the rootmap file is necessary to avoid header parsing for the execution of the read rules that use the type alias names, e.g.

rule.fSource = type + "::Layout layout_;";

, and therefore it seemed worthwhile to change the dictionaries with PortableHost{Collection,Object}.

In the present state this PR demonstrates what the impact would be for the PortableTestObjects. If deemed viable, the next steps would be to update DataFormats/Portable README and scripts, and then update all the other classes_def.xml files that declare these portable data products.

Resolves cms-sw/framework-team#1542

PR validation:

The testHeterogeneousCoreAlpakaTestWriteReadSerialSync unit test succeeds with the build of cms-sw/root#222 (comment). The resulting rootmap contains the ::Product, ::Layout, and ::Implementation type aliases that the read rules use.

@cmsbuild
Copy link
Contributor

cmsbuild commented Aug 29, 2025

cms-bot internal usage

@cmsbuild
Copy link
Contributor

@cmsbuild
Copy link
Contributor

A new Pull Request was created by @makortel for master.

It involves the following packages:

  • DataFormats/PortableTestObjects (heterogeneous)

@cmsbuild, @fwyzard, @makortel can you please review it and eventually sign? Thanks.
@missirol, @mmusich, @rovere this is something you requested to watch as well.
@ftenchini, @mandrenguyen, @sextonkennedy you are the release manager for this.

cms-bot commands are listed here

@makortel
Copy link
Contributor Author

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

+1

Size: This PR adds an extra 20KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-556bb6/47912/summary.html
COMMIT: cb4f965
CMSSW: CMSSW_15_1_X_2025-08-29-1100/el8_amd64_gcc12
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week0/cms-sw/cmssw/48824/47912/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • No significant changes to the logs found
  • Reco comparison results: 0 differences found in the comparisons
  • DQMHistoTests: Total files compared: 50
  • DQMHistoTests: Total histograms compared: 4115479
  • DQMHistoTests: Total failures: 29
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4115430
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 49 files compared)
  • Checked 215 log files, 184 edm output root files, 50 DQM output files
  • TriggerResults: no differences found

@cmsbuild
Copy link
Contributor

Milestone for this pull request has been moved to CMSSW_16_0_X. Please open a backport if it should also go in to CMSSW_15_1_X.

@makortel
Copy link
Contributor Author

@fwyzard Given

regardless of #49458 we should probably revive this PR and go systematically through all dictionary definitions of Portable* types (and update documentation).

I have a vague recollection though there was some detail I was not happy with, but I can't remember what it was.

@fwyzard
Copy link
Contributor

fwyzard commented Nov 25, 2025

I must admit I actually didn't remember about this PR :-(

@makortel makortel force-pushed the portableCollectionDictionary branch from cb4f965 to d001897 Compare December 30, 2025 20:23
@makortel makortel changed the title [RFC] Improve PortableHost{Collection,Object} dictionary declarations Improve PortableHost{Collection,Object} dictionary declarations Dec 30, 2025
@makortel
Copy link
Contributor Author

I rebased the branch to 16_0_0_pre4, updated the documentation for the necessity to include the ::Product, ::Layout, and ::Implementation nested type aliases in the classes_def.xml, and updated the classes_def.xml for all remaining PortableHost{Product,Collection,MultiCollection} dictionaries.

Comment on lines +10 to 17
<!-- TODO: we should find a better way than replicating the class versions and checksums -->
<class name="BeamSpotHost::Product" ClassVersion="3">
<version ClassVersion="3" checksum="280341519"/>
</class>

<class name="BeamSpotPOD" ClassVersion="3">
<version ClassVersion="3" checksum="280341519"/>
</class>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pcanal I think we discussed about the apparent need to define the class versions and checksums twice (the BeamSpotHost::Product is an alias for BeamSpotPOD). Or, I didn't manage to come up with any other way that would result in both type names to end up in the .rootmap file. I wouldn't hold this PR for an improvement, but we should follow up in #49458.

Comment on lines -4 to -6
<class name="reco::CaloRecHitHostCollection" ClassVersion="3">
<version ClassVersion="3" checksum="1876594952"/>
</class>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also removed the explicit versions and checksums for the PortableHostCollections that still had it. Their versioning should be done in a different way because PortableHostCollection<T> is a class template.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that we could extend the mechanism documented at core/dictgen/res/DictSelectionReader.h which allows the selection to be done via C++ code/template instantiation rather than xml or linkdef.


<class name="lst::LSTInputHostCollection"/>
<class name="edm::Wrapper<lst::LSTInputHostCollection>" splitLevel="0"/>
<class name="edm::Wrapper<lst::LSTInputHostCollection>" splitLevel="0" persistent="false"/>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also marked the LST collections as transient (by policy non-DataFormat packages should not define persistable data products).

@makortel
Copy link
Contributor Author

enable gpu

@makortel
Copy link
Contributor Author

makortel commented Jan 5, 2026

code-checks

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 5, 2026

+code-checks

Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-48824/47300

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 5, 2026

Pull request #48824 was updated. @Moanwar, @civanch, @cmsbuild, @fwyzard, @jfernan2, @kpedro88, @makortel, @mandrenguyen, @mdhildreth, @srimanob can you please check and sign again.

@makortel
Copy link
Contributor Author

makortel commented Jan 5, 2026

@cmsbuild, please test

@cmsbuild
Copy link
Contributor

cmsbuild commented Jan 6, 2026

+1

Size: This PR adds an extra 16KB to repository
Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-556bb6/50438/summary.html
COMMIT: d001897
CMSSW: CMSSW_16_1_X_2026-01-05-1100/el8_amd64_gcc13
Additional Tests: GPU,AMD_MI300X,AMD_W7900,NVIDIA_H100,NVIDIA_L40S,NVIDIA_T4
User test area: For local testing, you can use /cvmfs/cms-ci.cern.ch/week1/cms-sw/cmssw/48824/50438/install.sh to create a dev area with all the needed externals and cmssw changes.

Comparison Summary

Summary:

  • You potentially added 2 lines to the logs
  • ROOTFileChecks: Some differences in event products or their sizes found
  • Reco comparison results: 0 differences found in the comparisons
  • Reco comparison had 4 failed jobs
  • DQMHistoTests: Total files compared: 55
  • DQMHistoTests: Total histograms compared: 4513958
  • DQMHistoTests: Total failures: 61
  • DQMHistoTests: Total nulls: 0
  • DQMHistoTests: Total successes: 4513877
  • DQMHistoTests: Total skipped: 20
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 54 files compared)
  • Checked 235 log files, 208 edm output root files, 55 DQM output files
  • TriggerResults: no differences found

AMD_W7900 Comparison Summary

Summary:

  • You potentially added 24 lines to the logs
  • Reco comparison results: 251 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 31271
  • DQMHistoTests: Total nulls: 9
  • DQMHistoTests: Total successes: 118091
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

NVIDIA_H100 Comparison Summary

Summary:

  • You potentially removed 15 lines from the logs
  • Reco comparison results: 258 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 28936
  • DQMHistoTests: Total nulls: 7
  • DQMHistoTests: Total successes: 120428
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

NVIDIA_L40S Comparison Summary

Summary:

  • You potentially removed 4 lines from the logs
  • Reco comparison results: 253 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 26087
  • DQMHistoTests: Total nulls: 9
  • DQMHistoTests: Total successes: 123275
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

NVIDIA_T4 Comparison Summary

Summary:

  • You potentially removed 7 lines from the logs
  • Reco comparison results: 256 differences found in the comparisons
  • Reco comparison had 6 failed jobs
  • DQMHistoTests: Total files compared: 11
  • DQMHistoTests: Total histograms compared: 149371
  • DQMHistoTests: Total failures: 27452
  • DQMHistoTests: Total nulls: 11
  • DQMHistoTests: Total successes: 121908
  • DQMHistoTests: Total skipped: 0
  • DQMHistoTests: Total Missing objects: 0
  • DQMHistoSizes: Histogram memory added: 0.0 KiB( 10 files compared)
  • Checked 42 log files, 45 edm output root files, 11 DQM output files
  • TriggerResults: no differences found

@makortel
Copy link
Contributor Author

makortel commented Jan 7, 2026

How should we proceed with this PR? Given that the omission of the ::Layout etc nested type aliases from the classes_def.xml files caused real problems (crashes) in the recent past, that also took some effort to diagnose, I think it would be safest to merge it soon and backport to 16_0_X.

I see #49734 was opened to remove the PortableHostMultiCollection, that conflicts with this PR (easy to resolve though). Is #49734 planned to be backported to 16_0_X?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

As a workaround add PortableHost* type aliases to their classes_def.xml files.

4 participants