NVIDIA GPUs are programmed from Julia with two packages:

Most code only needs CUDA.jl’s arrays. Custom kernels are for the parts that array operations do not cover or do not run fast enough, and cuTile.jl is worth considering when such a kernel is a dense computation over blocks of data, like a matrix multiplication or an attention layer.

Installation #

Install CUDA.jl with Julia’s package manager:

pkg> add CUDA

All you need on the system is a recent NVIDIA driver. There is no need to install the CUDA toolkit: CUDA.jl downloads a version that is compatible with your driver when it is first used. CUDA.jl supports the latest Julia LTS release and newer, on Linux (x86-64 and ARM64, including NVIDIA Jetson boards) and Windows (x86-64), with GPUs from the Maxwell architecture onwards as long as the selected toolkit still supports them.

CUDA.versioninfo() shows what was installed and which GPUs were detected, which makes it the first thing to run, and to include in bug reports:

julia> using CUDA

julia> CUDA.versioninfo()
CUDA toolchain: 
- runtime 13.4.0, artifact installation
- driver 615.78.8 for CUDA 13.5, forward-compatible
- compiler 13.5.37, artifact installation

CUDA libraries: 
- cuBLAS: 13.8.0
- cuSPARSE: 12.8.6
- cuSOLVER: 12.3.5
- cuFFT: 12.4.0
- cuRAND: 10.4.4
- CUPTI: 2026.3.1 (API 13.4.1)
- NVML: 13.0.0+615.78.8
⋮

1 device:
  0: NVIDIA GeForce RTX 5080 (sm_120, 15.522 GiB / 15.921 GiB available)
     compiles to sm_120a / PTX 9.4 (LLVM: sm_120a / PTX 9.2)

Array programming #

A CuArray is a Julia array that lives in GPU memory. Operations on it run on the GPU, and most generic Julia code that works with arrays works with a CuArray too:

julia> a = CuArray(Float32[1 2; 3 4])
2×2 CuArray{Float32, 2, CUDACore.DeviceMemory}:
 1.0  2.0
 3.0  4.0

julia> a * a'
2×2 CuArray{Float32, 2, CUDACore.DeviceMemory}:
  5.0  11.0
 11.0  25.0

julia> sqrt.(a .+ 1)
2×2 CuArray{Float32, 2, CUDACore.DeviceMemory}:
 1.41421  1.73205
 2.0      2.23607

julia> sum(a)
10.0f0

julia> Array(a)
2×2 Matrix{Float32}:
 1.0  2.0
 3.0  4.0

CUDA.jl wraps NVIDIA’s libraries, and most of their functionality is reached through Julia’s standard interfaces: the matrix multiplication above calls cuBLAS, cholesky and \ call cuSOLVER, fft from AbstractFFTs.jl calls cuFFT, and sparse matrices like CuSparseMatrixCSR are backed by cuSPARSE. Libraries such as cuDNN and cuTENSOR are available as separate packages.

Kernel programming #

When array operations are not enough, you can write a kernel as a regular Julia function and launch it with @cuda. Kernels follow CUDA’s SIMT model: the function runs once per thread, and uses the thread and block indices to decide which elements to work on:

julia> function vadd!(c, a, b)
           i = (blockIdx().x - 1) * blockDim().x + threadIdx().x
           if i <= length(c)
               @inbounds c[i] = a[i] + b[i]
           end
           return
       end
vadd! (generic function with 1 method)

julia> a = CUDA.rand(Float32, 10_000);

julia> b = CUDA.rand(Float32, 10_000);

julia> c = similar(a);

julia> @cuda threads=256 blocks=cld(length(c), 256) vadd!(c, a, b);

julia> c == a .+ b
true

If you want kernels that also run on GPUs from other vendors, write them with KernelAbstractions.jl instead, which CUDA.jl supports as a back-end.

Tile kernels with cuTile.jl #

In a cuTile kernel there is no threadIdx. Each block loads tiles from its input arrays, computes on them with ordinary Julia operations, and stores the result. Here is a matrix multiplication where every block computes one 64×64 tile of C:

julia> using CUDA, cuTile

julia> import cuTile as ct

julia> function matmul!(C, A, B)
           a_tiles = ct.eachtile(A, (64, 32))
           b_tiles = ct.eachtile(B, (32, 64))
           c_tiles = ct.eachtile(C, (64, 64))
           m, n = ct.bid(1), ct.bid(2)
           acc = zeros(Float32, (64, 64))
           for k in Int32(1):Int32(size(a_tiles, 2))
               acc = muladd(a_tiles[m, k], b_tiles[k, n], acc)
           end
           c_tiles[m, n] = acc
           return
       end
matmul! (generic function with 1 method)

julia> A = CUDA.rand(Float16, 1024, 512);

julia> B = CUDA.rand(Float16, 512, 1024);

julia> C = CUDA.zeros(Float32, 1024, 1024);

julia> @cuda backend=cuTile blocks=(16, 16) matmul!(C, A, B);

julia> C ≈ Float32.(A) * Float32.(B)
true

The kernel does not say how the work is divided among threads or how to use tensor cores: the Tile IR compiler makes those decisions. cuTile.jl kernels take CuArrays, are launched with @cuda, and run on CUDA.jl’s current stream, so they mix freely with array operations, library calls and SIMT kernels.

Choose cuTile.jl for kernels that decompose into regular blocks of work, especially when they should use tensor cores: matrix multiplications, attention, normalization layers, softmax. Choose CUDA.jl kernels when you need control over individual threads, for example for irregular data structures or data-dependent control flow, or when you need to support GPUs older than Ampere. cuTile.jl requires Julia 1.11 or newer, an NVIDIA driver that supports CUDA 13 (version 580 or later), and a GPU with compute capability 8.0 or newer (Ampere or later).

Learn more #