[cuda.core] Add support for Multicast Objects

[rparolin]

Feature Request

Add Python bindings and a cuda.core abstraction for CUDA Multicast Objects
(the cuMulticastCreate / cuMulticastAddDevice / cuMulticastBindMem /
cuMulticastBindAddr / cuMulticastUnbind family of driver APIs).

Motivation

Multicast Objects enable a single virtual address to multicast memory accesses
across multiple devices, which is foundational for efficient multi-GPU
collectives and SHARP-style reductions on NVLink-connected systems (Hopper+
and beyond). They are already exposed via the low-level cuda.bindings driver
API, but there is no high-level cuda.core object wrapping them.

This is one of the last major CUDA driver features without a cuda.core
counterpart. Internal teams building multi-GPU libraries (RAPIDS, Triton,
cuBLASMp, etc.) currently have to drop to raw driver calls.

Proposed Scope

• A MulticastObject (or similarly named) class with:
  • Constructor / factory taking granularity + participating-device list
  • add_device(device) / bind_memory(buffer) / bind_address(...) /
    unbind(...) methods
  • Context-manager lifecycle (auto-release on exit)
  • Integration with existing Device, Buffer, and VirtualMemoryResource
• Query helpers for multicast granularity (cuMulticastGetGranularity)
• Example(s) under cuda_core/examples/ demonstrating a simple 2-GPU
  multicast allreduce-style pattern
• API reference entry in cuda_core/docs/source/api.rst

Related

• Tracking gaps identified in cuda.core feature audit (Nov 2025) — other
  untracked gaps include host_launch and Library (cuLibrary) APIs; those
  can be filed separately.
• Relevant driver APIs: cuMulticastCreate, cuMulticastAddDevice,
  cuMulticastBindMem, cuMulticastBindAddr, cuMulticastUnbind,
  cuMulticastGetGranularity.

[leofang]

I have mixed feelings in this. On the one hand it's good to have a centralized, pythonic multicast object, on the other hand we simply don't have any hw that could test this feature (which requires at least Hopper+ GPUs, NVLink, and NVSwitch all together in one system, such as NVL72).

Also, from the CUDA perspective this is considered as a central part of VMM, for example cuMulticastBindMem takes a VMM handle CUmemGenericAllocationHandle so as to map VA across devices. Furthermore, the CUDA doc suggests

Application developers generally should use the higher-level MPI, NCCL, or NVSHMEM interfaces instead of this API.

which is inline with our user/developer guidance: Always start with a high-level library first. So I would wait until someone comes and asks us for coverage.

cc @benhg for thoughts from nvSHMEM & VMM perspectives

[benhg]

I also have mixed feeling son this. In NVSHMEM, we bind multicast address ranges to VMM memprops (moral equivalent of VirtualMemoryResource-allocated objects) and actually can register VirtualMemoryResource buffers against the heap and use for Multimem access.

We are doing all this in the CPP library today, with Python bindings, rather than re-exposing in pure Python so far.

I do know of many users who build this functionality themselves (e.g. TorchSymmetric's CUDA backend)

[rparolin]

Did we want to close this as "won't implement"?
cc: @leofang @benhg