# GPU loops

> Run a counted loop on the graphics card with gpu for, keep arrays on the device with gpu regions, reduce into device values, and know which loops are admitted.

Canonical: https://alice-lang.org/documentation/guides/gpu/

[Documentation index](https://alice-lang.org/documentation/llms.txt) · [Complete book](https://alice-lang.org/llms-full.txt)

A `gpu for` loop runs on the graphics card. The compiler turns the loop body into an
[ILGPU](https://ilgpu.net) kernel and runs it on the preferred device (CUDA, OpenCL, or ILGPU's CPU accelerator when no
GPU is present). The program stays ordinary Alice: the kernel is not written by hand, and the arrays are the same
`Array` values the rest of the program uses. The design is [ADR 0021](https://github.com/alicelanguage/alice/blob/main/archicture/adr/0021-gpu-loops.md).

## A first kernel

```alice
def saxpy(a : Float32, xs : Array(Float32), ys : Array(Float32))
  gpu for i in 0...xs.size
    ys[i] = a * xs[i] + ys[i]
  end
end
```

Add ILGPU to the project with `alice add ILGPU`. Each iteration of the loop runs as one GPU thread and writes its own
element. The body may use:

- arrays of numbers and number values from the enclosing scope;
- arithmetic, indexing, `Math.*` and number conversions.

The loop is admitted only when iterations are provably independent, the same proof `par for` requires, plus the GPU
rules. A loop that calls a method or allocates is a compile error, and the error names the cause.

`ALICE_GPU=cpu` runs the same loops on ILGPU's CPU accelerator. Results match the GPU run; use it on machines without a
GPU, or to compare the two.

## Reductions

`t += e` into an outer `Int32`, `Int64`, `Float32` or `Float64` is an atomic add on the device:

```alice
def total(xs : Array(Int64)) : Int64
  t = 0_i64
  gpu for i in 0...xs.size
    t += xs[i] * 2_i64
  end
  t
end
```

The device result of an integer reduction is exact; it matches the CPU accelerator's result.

## Keep arrays on the device

Each `gpu for` loop uploads its arrays and copies written arrays back. A `gpu ... end` region keeps the arrays of the
loops inside it on the device: each is uploaded once, and the written ones are copied back at `end`.

```alice
def smooth(xs : Array(Float32), ys : Array(Float32), steps : Int32)
  gpu
    k = 0
    while k < steps
      gpu for i in 0...xs.size
        ys[i] = ys[i] + xs[i]
      end
      k += 1
    end
    gpu for i in 0...ys.size
      ys[i] = ys[i] * 0.5_f32
    end
  end
end
```

Inside a region the device holds the current values, so reading `xs` or `ys` on the CPU is a compile error until the
region ends. In a run of 51 launches over 1M floats, the region took 29 ms, against 132 ms when every launch transferred
its arrays.

## Explicit kernels

When you need control over the launch, write the kernel as a module function and load it with ILGPU's API. The
repository's `examples/gpu/ilgpu` does this with `->Kernels.saxpy` and runs the same computation both ways.

## Limits

- One preferred device per process. A region synchronizes with the device at its end. Multiple devices and
  asynchronous overlap of transfers and kernels are not offered; use explicit ILGPU code for those.
- Kernel arithmetic is not overflow-checked, and Alice does not contract multiply-add pairs in kernel bodies. The
  device compiler decides that.
- Only loops that the proof admits run on the GPU. Loops that are not admitted are compiled as they were before:
  see [fork/join](https://alice-lang.org/documentation/concurrency/fork-join.md) for how CPU loops run in parallel.
- On a continuous-integration machine without a GPU, `ALICE_GPU=cpu` runs the same loops on the CPU accelerator.

See [production readiness](https://alice-lang.org/documentation/reference/production.md) for the tests behind this feature.

## Continue reading

- Previous: [Build and distribute programs](https://alice-lang.org/documentation/guides/publishing.md)
- Next: [WebAssembly](https://alice-lang.org/documentation/guides/webassembly.md)