Document device allocation limits and failure behavior

Explain the per-work-item heap's size, alignment, lifetime, and cumulative
usage in loops. Make clear that exhaustion exits the work-item without a
host exception and can leave kernel output incomplete. Avoid promising that
the device compiler eliminates the storage or its performance cost.

Keep the existing scalar-indexing and Diagonal error overrides: their printed
diagnostics remain useful even with an allocator, since ordinary device
exceptions still do not report their reason to the host.

Author: maleadt

docs/src/kernels.md

@@ -62,3 +62,23 @@ and correspond to the standard OpenCL built-in functions. Note that the indices
 are 1-based, so they can be used to index Julia arrays directly. See
 [Device Intrinsics](device.md) for the full list.
 
+
+## Dynamic Memory Allocation
+
+Kernels can allocate Julia objects, such as a `Ref` passed to a `@noinline` function or a
+boxed value in an `Any` field. Allocations that survive optimization use a 1 KiB heap
+private to each work-item. Each allocation is rounded up to 16 bytes, and memory is only
+reclaimed when the work-item exits. Allocated objects must not be shared with other
+work-items or retained across kernel launches.
+
+When the heap is exhausted, the work-item prints an error and exits without completing
+its work. This does not raise a host-side exception, and kernel output may be incomplete:
+
+```
+ERROR: Out of dynamic GPU memory (trying to allocate 4 bytes)
+```
+
+Kernels without remaining allocations do not reserve an arena. The device compiler may
+optimize away some heap storage, but allocations can increase private-memory use and
+reduce performance. Avoid repeated allocations in loops: even short-lived objects consume
+heap space until the work-item exits.

src/device/quirks.jl

@@ -38,7 +38,7 @@ end
     @print_and_throw "sincos(x) is only defined for finite x."
 
 # diagonal.jl
-# XXX: remove when we have malloc
+# Base's version throws an ArgumentError; this one prints the reason
 import LinearAlgebra
 @device_override function Base.setindex!(D::LinearAlgebra.Diagonal, v, i::Int, j::Int)
     @boundscheck checkbounds(D, i, j)
@@ -51,7 +51,7 @@ import LinearAlgebra
 end
 
 # number.jl
-# XXX: remove when we have malloc
+# Base's version throws a BoundsError; this one prints the reason
 @device_override @inline function Base.getindex(x::Number, I::Integer...)
     @boundscheck all(isone, I) ||
         @print_and_throw "Out-of-bounds access of scalar value"