Document dynamic allocation in kernels

Describe the per-work-item heap, its size, its failure mode and its cost in
the kernel programming docs, and stop announcing the removal of the two
quirks that were waiting for `malloc`: they stay for the reason they print.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Fy9hCZkRFuf9fbRQNagCxf

Author: maleadt

docs/src/kernels.md

@@ -62,3 +62,22 @@ and correspond to the standard OpenCL built-in functions. Note that the indices
 are 1-based, so they can be used to index Julia arrays directly. See
 [Device Intrinsics](device.md) for the full list.
 
+
+## Dynamic Memory Allocation
+
+Kernels can allocate Julia objects: a `Ref` handed to a `@noinline` function, a struct with
+an `Any` field, or the exception object of a `throw` that survives optimization. Such
+allocations are served by a small heap private to each work-item: `oneAPI.HEAP_SIZE` bytes
+(1 KiB) of private memory, bump-allocated and released as a whole when the work-item
+exits. Nothing is freed before that, so a work-item that allocates more than the heap
+holds runs out of memory, reports it on the standard output of the process, and exits
+without finishing its work:
+
+```
+ERROR: Out of dynamic GPU memory (trying to allocate 4 bytes)
+```
+
+Only kernels that allocate reserve a heap, and Intel's compiler usually optimizes it away
+entirely when the allocated objects do not outlive the code it inlines. Allocations on a hot
+path are nonetheless a sign that the kernel is not fully specialized: restructure such code
+rather than rely on the heap.

src/device/quirks.jl

@@ -38,7 +38,7 @@ end
     @print_and_throw "sincos(x) is only defined for finite x."
 
 # diagonal.jl
-# XXX: remove when we have malloc
+# Base's version throws an ArgumentError; this one prints the reason
 import LinearAlgebra
 @device_override function Base.setindex!(D::LinearAlgebra.Diagonal, v, i::Int, j::Int)
     @boundscheck checkbounds(D, i, j)
@@ -51,7 +51,7 @@ import LinearAlgebra
 end
 
 # number.jl
-# XXX: remove when we have malloc
+# Base's version throws a BoundsError; this one prints the reason
 @device_override @inline function Base.getindex(x::Number, I::Integer...)
     @boundscheck all(isone, I) ||
         @print_and_throw "Out-of-bounds access of scalar value"