Performance and package size

cudaverse is designed for numerical R workflows that are large enough to benefit from CUDA but do not need a complete deep-learning framework. It uses the NVIDIA CUDA installation already on your computer, keeping the R package small.

Choose precision for the workload

Native float32 tensors use standard cuBLAS single-precision multiplication by default. They are not automatically promoted to float64. Float64 tensors remain available for analyses requiring greater accuracy.

On supported NVIDIA GPUs (compute capability 8.0 or newer), an explicit TF32 request permits cuBLAS to use reduced-precision TensorFloat-32 multiplication.

The precision = "tf32" option requires cudaverse 0.4.2 or newer. Check your installed version with packageVersion("cudaverse") before running this example.

a <- cuda_tensor(matrix(rnorm(1024 * 1024), 1024), "cuda", "float32")
b <- cuda_tensor(matrix(rnorm(1024 * 1024), 1024), "cuda", "float32")
standard <- tensor_matmul(a, b)
fast <- tensor_matmul(a, b, precision = "tf32")
cuda_provenance(fast)

TF32 keeps float32 storage and output while allowing reduced input precision. Its kernel choice and speed depend on matrix dimensions, cuBLAS and the GPU. It applies only to the requested call; later standard calls and %*% retain their existing behavior. Unsupported requests raise an error, including on CPU or torch tensors, float64 inputs, older GPUs, or when NVIDIA_TF32_OVERRIDE=0. The provenance reason matmul_tf32_permitted records the permitted compute policy; it does not prove a particular hardware kernel was selected. See the NVIDIA cuBLAS guide.

Assess numerical error against the analysis requirements. Cancellation can make a small output particularly sensitive to floating-point rounding. TF32 permits larger differences than standard float32, and a fixed relative error target near zero is not a universal float32 guarantee. The development benchmark checks standard float32 dot products against a componentwise rounding-error model and also reports the original strict comparison.

For good throughput, upload data once, keep intermediate tensors on the GPU, reuse them across calls and download results when the host needs them. Profile the complete workflow: the native PCA and distance paths currently use float64, so this tensor-matmul option does not change their precision. The historical results below use the earlier standard path and do not measure the new TF32 option.

What was compared?

The retained 0.4 benchmark runs the same public cudaverse workload with three backends:

base R, the numerical reference on the host;
native CUDA, the lightweight cudaverse backend; and
R torch, the optional CUDA compatibility backend backed by LibTorch.

Each result below is the median of ten runs after five warm-up runs. The complete R function call is timed, and results are checked for numerical agreement before the timing is accepted.

Selected full-benchmark results

These measurements come from one NVIDIA GPU system. Lower time is better, and your result will vary with the GPU, driver, data, and other running programs.

Workload	Base R	Native CUDA	R torch	Native vs base	Native vs torch
256 x 256 float32 matrix multiplication	0.0061 s	0.0055 s	0.0096 s	1.1x	1.8x
1024 x 1024 float32 matrix multiplication	0.3324 s	0.0243 s	0.0389 s	13.7x	1.6x
4096 x 4096 float32 matrix multiplication	36.1725 s	0.5803 s	0.8134 s	62.3x	1.4x
Dense PCA + exact kNN, 1,000 x 50	0.0899 s	0.0267 s	0.1026 s	3.4x	3.8x
Dense PCA + exact kNN, 10,000 x 100	11.9146 s	0.2747 s	6.0944 s	43.4x	22.2x
Dense PCA + exact kNN, 50,000 x 128	668.4177 s	2.8068 s	126.5523 s	238.1x	45.1x
Sparse PCA + exact kNN, 10,000 x 100	10.2651 s	0.2029 s	5.1979 s	50.6x	25.6x
Sparse PCA + exact kNN, 50,000 x 128	217.6324 s	2.7995 s	126.7389 s	77.7x	45.3x

In this test, cudaverse was most useful for the medium and large PCA followed by exact-kNN workflows. These numbers are examples, not a promise for every computer or dataset.

Small workloads can be faster without CUDA

Starting GPU work and transferring data have fixed costs. In the retained sparse 1,000 x 50 pipeline, base R took 0.0780 s and native CUDA took 0.0930 s. Use cudaverse for workloads with enough arithmetic or a long enough device-resident continuation to repay that overhead.

The 256 x 256 matrix multiplication result shows the same principle: all three backends completed in milliseconds, so the absolute difference was small even though native CUDA had the lowest median.

Why can the native pipeline be faster?

The performance gain is not only a faster matrix multiplication call. Supported native workflows can:

upload the input once;
keep PCA scores on the device;
compute distance blocks on CUDA;
select stable top-k neighbours on CUDA; and
materialize only the final neighbour index and distance matrices in R.

This avoids repeated host/device boundaries between separate high-level tasks. See Keep a workflow on the GPU for practical code.

Installed footprint

The same retained environment reported:

Component	Installed size
cudaverse	1,447,216 bytes (about 1.45 MB)
optional R torch installation	7,367,799,444 bytes (about 7.37 GB)
CUDA runtime bundled by cudaverse	0 bytes

The native package is small because it uses compatible NVIDIA runtime libraries installed by the user. This is a deployment tradeoff: cudaverse does not download or redistribute those libraries, so the Windows or Linux system must be prepared first using the CUDA setup guide.

When cudaverse is a good fit

Good candidates include:

repeated dense matrix operations on an already uploaded tensor;
PCA followed by distance or exact kNN;
sparse normalization followed by PCA and exact kNN;
medium or large float32 matrix multiplication; and
analyses that can delay to_cpu() until the final output.

It is less useful for a single tiny matrix operation, a workflow that downloads after every step, or a task listed as only partly accelerated in What runs on CUDA?.