Update API to leverage cuVS Spectral clustering #5326

jnke2016 · 2025-10-26T02:31:19Z

This PR integrates cuVS spectral clustering from upstream into cuGraph while maintaining full backward compatibility. The integration deprecates the legacy balanced cut clustering C++ and Python APIs in favor of spectral_modularity_maximization. In fact, cuVS consolidated the spectral clustering API to assign vertices to clusters that by default maximizes the modularity score

Static Linking with Conda Fallback:

cuGraph is configured to statically link libcuvs for optimal performance
The build system first attempts to use conda-installed libcuvs if available
If libcuvs is not present in the conda environment or static linking is required, the build system automatically clones the main cuVS repository (rapidsai/cuvs) and builds it from source
Uses the standard RAPIDS release branch (${rapids-cmake-checkout-tag}) ensuring version compatibility

Note: The header files raft/spectral/partition.cuh and raft/spectral/modularity_maximization.cuh are still necessary in spectraly_clustering.cu because APIs (like raft::spectral::analyzePartition, raft::spectral::analyzeModularity) are still being called in cuGraph and are not supported in cuVS. This is not an issue because those APIs are not deprecated in raft in contrary to kmeans and solvers

copy-pr-bot · 2025-10-26T02:31:23Z

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

aamijar

Hi @jnke2016, thanks for the PR! Looks good overall, just left some comments for you to check.

aamijar · 2025-10-29T04:22:17Z

cpp/src/community/legacy/spectral_clustering.cu

-  unsigned long long seed2 = seed1 + 1;
+  // Use cuVS for float precision until double precision is supported
+  if constexpr (std::is_same_v<weight_t, float>) {
+    // Convert CSR to COO using thrust


You might be able to use raft::sparse::convert::csr_to_coo here if its more convenient.

Thanks. I leveraged this raft utils function to convert from csr to coo

aamijar · 2025-10-29T04:24:01Z

cpp/src/community/legacy/spectral_clustering.cu

+      coo_matrix,
+      raft::make_device_vector_view<vertex_t, vertex_t>(clustering, graph.number_of_vertices));
+  } else {
+    CUGRAPH_FAIL("Double precision spectral clustering not yet supported with cuVS");


I added double support for the cuvs PR so we should be able to try with that.

Thanks. I updated the PR to support double and after merging your latest changes it worked

… support

aamijar

Thanks for incorporating the suggestions!

…-spectral-clustering

ChuckHastings

Looks good, will do a final review after it passes CI

ChuckHastings

Looks good once we solve the build problem.

jnke2016 · 2025-11-15T03:40:03Z

/ok to test fed3f33

jnke2016 · 2025-11-15T14:56:11Z

@robertmaynard @bdice . I am still getting an overlinking error during the conda build. Can the error be caused by the fact that rapids_cpm_find only discovers shared library (libcuvs.so) which forces CMake to fallback to cuvs::cuvs (dynamic linking? If it is the case, would building libcuvs statically with BUILD_STATIC_LIBS ON in rapids_cpm_find enable libcuvs to be statically linked to libcugraph through cuvs::cuvs_static?

jnke2016 · 2025-11-15T16:47:04Z

I am still trying to figure out the source of the overlinking issue, and I think it arose when we introduced the COMPONENTS element. Unless I am wrong, there is no cuvs_static component defined, and both cuvs::cuvs (shared) and cuvs::cuvs_static (static) targets are always loaded together in the main/default package (no component specification needed). cuvs_c is the only component, which provides the C API set(cuvs_comp_names c_api). I am not sure what the expected behavior is in this case, but I can speculate. Since the cuvs_static component doesn't exist, it falls back to shared → overlinking because libcuvs becomes a runtime dependency but is not listed in the conda package metadata (which expects static linking).

jnke2016 · 2025-11-15T16:57:13Z

/ok to test d8f2df6

jnke2016 · 2025-11-15T18:25:23Z

/ok to test b4117ba

jnke2016 · 2025-11-16T00:30:50Z

/ok to test 2e677c8

robertmaynard · 2025-11-17T15:12:25Z

I am still trying to figure out the source of the overlinking issue, and I think it arose when we introduced the COMPONENTS element. Unless I am wrong, there is no cuvs_static component defined, and both cuvs::cuvs (shared) and cuvs::cuvs_static (static) targets are always loaded together in the main/default package (no component specification needed). cuvs_c is the only component, which provides the C API set(cuvs_comp_names c_api). I am not sure what the expected behavior is in this case, but I can speculate. Since the cuvs_static component doesn't exist, it falls back to shared → overlinking because libcuvs becomes a runtime dependency but is not listed in the conda package metadata (which expects static linking).

Yeah I was wrong about the components. The cuvs::cuvs_static target will be brought in with find_package(cuvs) whenever it is available

bdice · 2025-11-17T16:07:58Z

ci/build_wheel.sh

 sccache --show-adv-stats

 EXCLUDE_ARGS=(
+  --exclude "libcuvs.so"


This shouldn't be needed. Wheels should always be statically linking, so shared libraries shouldn't be present in the first place. See cuML's wheel building script, which doesn't need this exclusion. cuGraph should be the same. https://github.com/rapidsai/cuml/blob/main/ci/build_wheel_libcuml.sh

I am confused. Aren't wheels using prebuilt packages and link dynamically (hence we have libcuvs.so in the build artifact). Am I missing something?

Wheels should always be statically linking, so shared libraries shouldn't be present in the first place

In that case, I should update the wheels to compile and build from source instead of using prebuilt packages?

Got it now after reviewing cuML. Thanks

bdice · 2025-11-17T16:10:30Z

python/cugraph/cugraph/community/spectral_clustering.py

    >>> from cugraph.datasets import karate
    >>> G = karate.get_graph(download=True)
-    >>> df = cugraph.spectralBalancedCutClustering(G, 5)
+    >>> df = cugraph.spectralModularityMaximizationClustering(G, 5)


This is an example for the docs of spectralBalancedCutClustering. We shouldn't change the example code here. We could add a deprecation notice in the docs, though.

Suggested change

>>> df = cugraph.spectralModularityMaximizationClustering(G, 5)

>>> df = cugraph.spectralBalancedCutClustering(G, 5)

We shouldn't change the example code here. We could add a deprecation notice in the docs, though.

Right but this is what I did initially and the pytest doctests would fail unless that warning failure get skipped. @rlratzel any thoughts?

You need something like this: https://github.com/rapidsai/cudf/blob/cff81d86667c1a0fbcbf7887c28e640a29bc7dfd/python/cudf/cudf/tests/test_doctests.py#L16

We should probably just remove the example entirely. I think an example is useful for someone trying to learn how to add this function to their code, which we/they don't want since it's going away. Make sure the docstring is updated to indicate it's deprecated and direct them to the new API they should call.

You should still add the ignores I mentioned above because this will apply to all deprecated APIs in the future.

You should still add the ignores I mentioned above because this will apply to all deprecated APIs in the future.

I'm thinking that ignoring errors from deprecated APIs in doctests is not good though, and perhaps we should just adopt the policy we're doing here (remove example docstring, ensure new API is referenced) for all deprecated APIs in the future.

I'm concerned the ignores will mask other API deprecations we need to know about (eg. an example docstring uses a deprecated cuDF API, or a cugraph docstring for function A uses deprecated cugraph API B as part of the example and should be updated).

I'm concerned the ignores will mask other API deprecations we need to know about

If it helps, ignore::FutureWarning:cugraph will ignore just the deprecations in the cugraph module, or you can use ignore:MESSAGE_REGEX:FutureWarning:cugraph with some regex string to ignore specific (known) deprecations from cugraph.

bdice · 2025-11-17T16:12:32Z

dependencies.yaml

    includes:
      - common_build
      - depends_on_libcugraph
+      - depends_on_libcuvs


Wheel builds shouldn't depend on libcuvs libraries, it should be fetched and built statically by CMake.

Suggested change

- depends_on_libcuvs

Got it. I understand what you mean here based on this

bdice · 2025-11-17T16:12:50Z

dependencies.yaml

    includes:
      - common_build
      - depends_on_librmm
+      - depends_on_libcuvs


Wheel builds shouldn't depend on libcuvs libraries, it should be fetched and built statically by CMake.

Suggested change

- depends_on_libcuvs

bdice · 2025-11-17T16:13:23Z

dependencies.yaml

    extras:
      table: project
    includes:
+      - depends_on_libcuvs


We do not want a runtime dependency on libcuvs, that's the point of statically linking.

Suggested change

- depends_on_libcuvs

bdice · 2025-11-17T16:13:39Z

dependencies.yaml

    includes:
      - common_build
      - depends_on_libcugraph
+      - depends_on_libcuvs


Wheel builds shouldn't depend on libcuvs libraries, it should be fetched and built statically by CMake.

Suggested change

- depends_on_libcuvs

bdice · 2025-11-17T16:17:23Z

dependencies.yaml

+    common:
+      - output_types: conda
+        packages:
+          - &libcuvs_unsuffixed libcuvs==25.12.*,>=0.0.0a0


Here's what I think you want:

conda developer environments: depend on shared library

conda devcontainers: depend on shared library

conda recipes: fetch with CMake, build and link libcuvs statically

No dependency in host or run

Note: this differs from cuML, which depends on a shared library here. This can go either way in my opinion.

wheel builds: fetch with CMake, build and link libcuvs statically

wheel at runtime: no dependency (it's statically linked)

pip devcontainers: ???

This one we might want to discuss. Probably fetch with CMake, build and link libcuvs statically? I haven't checked what this PR does right now.

robertmaynard · 2025-11-17T19:38:09Z

@jnke2016 Are you still planning on getting this into 25.12? If so please don't merge in main anymore and re-target release/25.12 as soon as possible.

jnke2016 · 2025-11-17T19:40:05Z

Are you still planning on getting this into 25.12? If so please don't merge in main anymore and re-target release/25.12 as soon as possible.

Got it @robertmaynard . @ChuckHastings can you retarget it please?

ChuckHastings · 2025-11-17T19:44:04Z

Are you still planning on getting this into 25.12? If so please don't merge in main anymore and re-target release/25.12 as soon as possible.

Got it @robertmaynard . @ChuckHastings can you retarget it please?

Done

jnke2016 · 2025-11-17T19:56:06Z

pip devcontainers: ???
This one we might want to discuss. Probably fetch with CMake, build and link libcuvs statically? I haven't checked what this PR does right now.

@bdice Good catch thanks. I didn't have a devcontainers section. I just added it and it installs pre-built libcuvs and link dynamically

…ependency

jnke2016 · 2025-11-17T21:07:24Z

/ok to test 0adde0d

jnke2016 · 2025-11-17T23:12:50Z

/ok to test db3dbea

bdice · 2025-11-18T19:13:58Z

dependencies.yaml

      - test_python_common
      - test_python_cugraph
      - test_python_pylibcugraph
+  devcontainers:


This devcontainers key isn't doing anything right now. To make it take effect, you'd need to update the devcontainers repo like this: https://github.com/rapidsai/devcontainers/blob/f9a516d1e4cd9b669845527d4ae13a4fdc720933/features/src/rapids-build-utils/opt/rapids-build-utils/manifest.yaml#L186-L187

However, why do we need this? I think there should be a solution that doesn't require a new dependency file key.

Look at how cuML is solving this: https://github.com/rapidsai/cuml/blob/6b42da514957a909a6d9d8921bad9d705161d8d8/dependencies.yaml#L684-L688

cuML only defines libcuvs for conda output types and not pyproject or requirements. I think that's probably what you want here too. Following the other discussion where we broke down the expected behavior by build type, you probably never want libcuvs wheels because you'll always build and statically link libcuvs. The only cases where you want libcuvs packages are for conda developer environments and devcontainers (but not conda recipes!).

I propose:

Delete the devcontainers key here

Modify depends_on_libcuvs below to only have conda outputs, matching cuML

So if I understand correctly, you are suggesting to modify depends_on_libcuvs to only output for conda (not pyproject or requirements) because:

Conda developer environments & devcontainers automatically get libcuvs from the all: section

Wheel builds & pip devcontainers don't get libcuvs because pyproject outputs won't include it (statically link)

Also:

Conda recipes don't get libcuvs as a runtime dependency (they also statically link cuVS)

… output types only

…12_update-spectral-clustering

bdice

Perfect. Approved!

jnke2016 · 2025-11-19T17:34:01Z

/ok to test 20dfd1d

ChuckHastings · 2025-11-19T18:57:36Z

/merge

update API to leverage cuvs Spectral clustering

0d0d6f5

jnke2016 requested a review from ChuckHastings October 26, 2025 02:35

jnke2016 mentioned this pull request Oct 26, 2025

Spectral Clustering rapidsai/cuvs#1425

Merged

aamijar reviewed Oct 29, 2025

View reviewed changes

use raft helper function to convert from csr to coo and enable double…

543488a

… support

aamijar approved these changes Nov 3, 2025

View reviewed changes

jnke2016 added 3 commits November 3, 2025 08:05

update conda env

ce452d6

fix style

47ccca1

fix style

32d01b0

jnke2016 requested a review from rlratzel November 3, 2025 16:29

fix style

e6c7d4b

jnke2016 marked this pull request as ready for review November 3, 2025 16:30

jnke2016 requested review from a team as code owners November 3, 2025 16:30

jnke2016 added 2 commits November 3, 2025 08:31

Merge remote-tracking branch 'upstream/main' into branch-25.12_update…

7654c7e

…-spectral-clustering

fix typo

7b4cd2a

ChuckHastings reviewed Nov 3, 2025

View reviewed changes

jnke2016 requested a review from a team as a code owner November 3, 2025 18:20

jnke2016 requested a review from KyleFromNVIDIA November 3, 2025 18:20

jnke2016 added 4 commits November 3, 2025 10:21

add cuvs to our dependency list

ee89faa

Add libcuvs as a dependency for spectral clustering support

38e9220

fix style

6c6fc9b

Add libcuvs to CI scripts

a89de7b

jnke2016 requested review from a team as code owners November 4, 2025 04:36

jnke2016 added 3 commits November 4, 2025 09:04

Integrate libcuvs for spectral clustering with static linking

638a2f6

Disable cuVS multi-GPU algorithms to fix NCCL overlinking error

a056532

Add retry strategy to modularity clustering tests

583c7d2

ChuckHastings approved these changes Nov 6, 2025

View reviewed changes

Simplify CUVS_LIB logic to fix conda overlinking error

fed3f33

jnke2016 added 2 commits November 15, 2025 08:57

Fix overlinking by removing COMPONENTS logic for cuVS

d8f2df6

Pass EXTRA_CMAKE_ARGS to libcugraph CMake configuration

b4117ba

Update spectral clustering doctests to use non-deprecated API

2e677c8

bdice reviewed Nov 17, 2025

View reviewed changes

ChuckHastings changed the base branch from main to release/25.12 November 17, 2025 19:43

jnke2016 added 2 commits November 17, 2025 12:39

Configure cuGraph wheels to statically link cuVS and remove libcuvs d…

c27afb2

…ependency

update branch

0adde0d

fix style

db3dbea

robertmaynard approved these changes Nov 18, 2025

View reviewed changes

bdice requested changes Nov 18, 2025

View reviewed changes

jnke2016 added 2 commits November 19, 2025 09:22

Remove devcontainers section and restrict depends_on_libcuvs to conda…

fbe6dce

… output types only

Merge remote-tracking branch 'upstream/release/25.12' into branch-25.…

20dfd1d

…12_update-spectral-clustering

bdice approved these changes Nov 19, 2025

View reviewed changes

rapids-bot bot merged commit 0f1afef into rapidsai:release/25.12 Nov 19, 2025
100 of 101 checks passed

	>>> df = cugraph.spectralModularityMaximizationClustering(G, 5)
	>>> df = cugraph.spectralBalancedCutClustering(G, 5)

Update API to leverage cuVS Spectral clustering #5326

Update API to leverage cuVS Spectral clustering #5326

Uh oh!

Conversation

jnke2016 commented Oct 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

copy-pr-bot bot commented Oct 26, 2025

Uh oh!

aamijar left a comment

Choose a reason for hiding this comment

Uh oh!

aamijar Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aamijar left a comment

Choose a reason for hiding this comment

Uh oh!

ChuckHastings left a comment

Choose a reason for hiding this comment

Uh oh!

ChuckHastings left a comment

Choose a reason for hiding this comment

Uh oh!

jnke2016 commented Nov 15, 2025

Uh oh!

jnke2016 commented Nov 15, 2025

Uh oh!

jnke2016 commented Nov 15, 2025

Uh oh!

jnke2016 commented Nov 15, 2025

Uh oh!

jnke2016 commented Nov 15, 2025

Uh oh!

jnke2016 commented Nov 16, 2025

Uh oh!

robertmaynard commented Nov 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bdice Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

jnke2016 commented Oct 26, 2025 •

edited

Loading

aamijar Oct 29, 2025 •

edited

Loading

bdice Nov 19, 2025 •

edited

Loading

jnke2016 commented Nov 17, 2025 •

edited

Loading

jnke2016 Nov 19, 2025 •

edited

Loading