Skip to content

Conversation

@cgcgcg
Copy link
Contributor

@cgcgcg cgcgcg commented Nov 10, 2025

@trilinos/tpetra

Motivation

This PR moves the fence from MultiVector::copyAndPermute to the end of DistObject::endTransfer.

Questions:

  • There was no fence for CM == ADD_ASSIGN in MultiVector::copyAndPermute. I think that we need the same fence, regardless of the combine mode?
  • I looked for fences in copyAndPermute for other objects derived from DistObject, but didn't find them.

@cgcgcg cgcgcg self-assigned this Nov 10, 2025
@github-actions
Copy link

CDash for AT1 results [Only accessible from Sandia networks]
CDash for AT2 results [Currently only accessible from Sandia networks]

@cgcgcg cgcgcg force-pushed the tpetraTranferFences branch 2 times, most recently from 6f4ba34 to a335ad1 Compare November 10, 2025 17:08
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

  • Build Num: 2641
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA a335ad1
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Build Information

Test Name: PR_CompSim

  • Build Num: 952
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
BLOCKING_BUILD false
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_aue-gcc-openmpi_release_static_no-kokkos-arch_no-asan_complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_compsim
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request (Non-blocking)
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA a335ad1
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Using Repos:

Repo: TRILINOS (cgcgcg/Trilinos)
  • Branch: tpetraTranferFences
  • SHA: a335ad1
  • Mode: TEST_REPO

Pull Request Author: cgcgcg

Copy link
Member

@jhux2 jhux2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One change, one question.

} // if (CM != ZERO)

// Guarantee that copyAndPermute and unpackAndCombine are done.
execution_space().fence("Tpetra::DistObject::endTransfer");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the comment above on line 935-937 apply here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so. We don't really know where the data will get accessed next. So we cannot rely on fences in sync. (And ideally we run everything on device anyway, so there is no sync.)

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

  • Build Num: 2641
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA a335ad1
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Build Information

Test Name: PR_CompSim

  • Build Num: 952
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
BLOCKING_BUILD false
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_aue-gcc-openmpi_release_static_no-kokkos-arch_no-asan_complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_compsim
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request (Non-blocking)
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA a335ad1
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82


CDash Test Results for PR# 14677.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
THE LAST COMMIT TO THIS PULL REQUEST HAS BEEN REVIEWED, BUT NOT ACCEPTED OR REQUIRES CHANGES!

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@cgcgcg cgcgcg force-pushed the tpetraTranferFences branch from a335ad1 to b7b3844 Compare November 10, 2025 20:38
Copy link
Member

@jhux2 jhux2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

  • Build Num: 2643
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA b7b3844
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Build Information

Test Name: PR_CompSim

  • Build Num: 954
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
BLOCKING_BUILD false
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_aue-gcc-openmpi_release_static_no-kokkos-arch_no-asan_complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_compsim
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request (Non-blocking)
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA b7b3844
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Using Repos:

Repo: TRILINOS (cgcgcg/Trilinos)
  • Branch: tpetraTranferFences
  • SHA: b7b3844
  • Mode: TEST_REPO

Pull Request Author: cgcgcg

@trilinos-autotester
Copy link
Contributor

NOTICE: The AutoTester has encountered an internal error (usually a Communications Timeout), testing will be restarted, previous tests may still be running but will be ignored by the AutoTester...

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

  • Build Num: 2645
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA b7b3844
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Build Information

Test Name: PR_CompSim

  • Build Num: 956
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
BLOCKING_BUILD false
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_aue-gcc-openmpi_release_static_no-kokkos-arch_no-asan_complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_compsim
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request (Non-blocking)
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA b7b3844
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Using Repos:

Repo: TRILINOS (cgcgcg/Trilinos)
  • Branch: tpetraTranferFences
  • SHA: b7b3844
  • Mode: TEST_REPO

Pull Request Author: cgcgcg

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: 1 or more Jobs FAILED

Note: Testing will normally be attempted again in approx. 2 Hrs 30 Mins. If a change to the PR source branch occurs, the testing will be attempted again on next available autotester run.

Pull Request Auto Testing has FAILED (click to expand)

Build Information

Test Name: PR_gcc-openmpi-openmp

  • Build Num: 2645
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_sems-gnu-8.5.0-openmpi-4.1.6-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA b7b3844
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82

Build Information

Test Name: PR_CompSim

  • Build Num: 956
  • Status: FAILED

Jenkins Parameters

Parameter Name Value
BLOCKING_BUILD false
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel8_aue-gcc-openmpi_release_static_no-kokkos-arch_no-asan_complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_compsim
PR_LABELS pkg: Tpetra
PULLREQUESTNUM 14677
PULLREQUEST_CDASH_TRACK Pull Request (Non-blocking)
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL rhel8
TRILINOS_SOURCE_REPO https://github.com/cgcgcg/Trilinos
TRILINOS_SOURCE_SHA b7b3844
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA 7afec82


CDash Test Results for PR# 14677.


Wiki: How to Reproduce PR Testing Builds and Errors.

@cgcgcg cgcgcg added the AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) label Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) pkg: Tpetra

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants