Skip to content

Commit c544dc7

Browse files
SamuelBelangerclaudenathanielsimard
authored
Adaptive autotune scheduler with early elimination (#1449)
* Implement probabilistic autotuning scheduler with early elimination and sample convergence tracking * Abstract autotuning logic into a TuneJob struct * add adaptive env variable * fix schedule * format * fix bench error handling * add limit and bounds to tune cache * set key value public * fix schedule dry run * tune: fold the new bench helpers into TuneFn and reuse env_bool `warmup_once` and `sample_once` took the operation as their first argument, so they are methods on it; warmup is then sampling with the profile dropped, which removes the duplicated match. `CUBECL_AUTOTUNE_BENCH_ADAPTIVE` re-implemented `env_bool` verbatim. `any_under` replaces a bare `1` at the call site, and the sampler module is gated with the schedule it exists for so wasm has no dead code. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * tune: move the private items of sampler.rs and schedule.rs to the bottom Pure move: `MIN_SURVIVORS` and `Candidate` sit below `Schedule`, and the sampler constants below `SampleSet`, so a reader meets the public surface first. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * tune: pass the cache value as a struct rather than four positional slots `persistent_cache_insert` reassembled its arguments into the `PersistentCacheValue` the caller could already name, so the two `Option`s this branch added went in as bare positional slots. Building the value at the call site names them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * tune: read every bench knob through a clamping accessor `short_circuit_samples.max(1)` appeared at two call sites and `speed_factor.max(1.0)` at a third, so three places had to remember a repair that belongs to the config. They join `samples()` as accessors on `BenchConfig`. Each field now also says which strategy reads it: only `max_samples` and `adaptive` mean anything to the fixed-count pass, and `adaptive` is native-only, so on wasm the elimination knobs never apply at all. A knob that silently does nothing on the strategy actually running is worse than no knob. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP * tune: correct the schedule's claim about what a batch can cost The doc said a batch "can never cost more than a fixed-sample-count pass over the same candidates". The sampling cannot, but the batch can: the short circuit exits on the first candidate confirmed under the limit, and eliminating cheap candidates early can push that exit past a kernel the fixed pass would have accepted and stopped at. Measured at +13.6% total tuning cost on one card against -20% on another, both dominated by where the short circuit fires rather than by the sample budget. The comment now says that instead of the opposite. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP * tune: keep cache entries written before bounds and limit readable `bounds` and `limit` were added to `PersistentCacheValue` without `#[serde(default)]`, so every entry written before them failed to decode. Nothing crashes -- `decode` degrades to `None` and the row is repaired on the next write -- but it means every cached key on every existing installation silently re-tunes from scratch on upgrade. The storage is CBOR, which is self-describing, so a default is all it takes. Also records what the two fields are for: nothing reads them back, and hydration only ever consults `fastest_index`. They exist so an entry can be inspected after the fact -- why a kernel won, against which measurements, under which bounds -- which is the question a finished tuning run can no longer answer. That is also why the types are `pub`. Undocumented, the pair reads as an oversight. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP * tune: apply the configured sample ceiling to the fixed-count pass `num_samples` was hardcoded to 10, so `[autotune.bench] max_samples` moved nothing unless the adaptive scheduler was running -- and on wasm, which has no adaptive scheduler, it moved nothing at all. It now reads the same budget the scheduler does, taking the ceiling: with no elimination there is nothing a smaller budget buys, and a candidate stopping early here would just be measured on less evidence than its rivals. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP * tune: let the scheduler declare its own winner The tuner picked the fastest kernel by sorting every result on `BenchmarkComputations::score`, which is only a fair comparison when the results carry the same amount of evidence. Under elimination they do not: a dropped candidate froze at `min_samples` while a survivor kept sampling, and `score` inflates with observed spread, which grows with the sample count. A candidate could be handed an advantage for having been eliminated early -- the same hazard `sampler.rs` already documents for the elimination decision, left in place for the decision that picks the kernel. `Schedule::outcome` now names the winner among the candidates that survived, and `process_request` takes it when it is offered. The sort stays: it still orders what gets logged and persisted, and the fixed-count pass -- where every candidate carries the same sample count -- still decides by it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP * tune: resolve pending benchmarks concurrently and time each one The fixed-count pass queues every candidate's profiles before polling any of them, then `process_request` awaited them one benchmark at a time inside the timing loop. The first benchmark therefore absorbed the drain of the entire device queue and the rest read as nearly free, which makes the per-candidate step durations in the log meaningless. Resolved with `join_all` and timed per benchmark, which is what the adaptive scheduler's round loop already does, and for the reason its comment already gives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP * tune: cover the round robin end to end The scheduler's unit tests reach `SampleSet`, `eliminate` and `short_circuit_hit`, all pure. `Schedule::drive` -- the round robin itself, the per-candidate ceiling, the convergence break -- had no test at all, and the one integration test on the branch passes identically under both strategies, so nothing distinguished them. `addition_set_with_slow_candidate` puts one sleeping kernel against two fast ones and counts each candidate's calls. Three candidates because the survivor floor keeps two alive regardless, so a two-candidate set can never eliminate anything -- which is itself worth having encoded somewhere. The test asserts the slow candidate collected its evidence, stopped short of both the ceiling and the survivors, and lost. Verified by mutation: raising MIN_SURVIVORS to 3 disables elimination for this set and the test fails with 6 calls against 6. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Pbbmu4yjxdphy5zGTvfmaP --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> Co-authored-by: nathaniel <nathaniel.simard.42@gmail.com>
1 parent 05cac67 commit c544dc7

11 files changed

Lines changed: 1454 additions & 93 deletions

File tree

crates/cubecl-runtime/src/config/autotune.rs

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,85 @@ pub struct AutotuneConfig {
2929
/// Whether to disable the short circuit logic during autotuning.
3030
#[serde(default)]
3131
pub disable_short_circuit: bool,
32+
33+
/// Sampling budget and elimination thresholds used while benchmarking candidates.
34+
#[serde(default)]
35+
pub bench: BenchConfig,
36+
}
37+
38+
/// Controls how many samples autotune collects per candidate and when candidates are dropped.
39+
///
40+
/// Only [`max_samples`](Self::max_samples) and [`adaptive`](Self::adaptive) mean anything to the
41+
/// fixed-count pass; the rest describe elimination, which only the adaptive scheduler performs.
42+
/// Each field says so, because a knob that silently does nothing on the strategy actually running
43+
/// is worse than no knob at all — and `adaptive` is native-only, so on wasm that is every run.
44+
#[derive(Clone, Debug, serde::Serialize, serde::Deserialize)]
45+
#[serde(default)]
46+
pub struct BenchConfig {
47+
/// Samples every surviving candidate gets before any elimination happens.
48+
///
49+
/// Adaptive only: the fixed-count pass eliminates nothing, so every candidate gets
50+
/// [`max_samples`](Self::max_samples) regardless.
51+
pub min_samples: usize,
52+
53+
/// Upper bound on samples collected for a single candidate.
54+
///
55+
/// Read by both strategies: the ceiling for the adaptive scheduler, and the flat count for
56+
/// the fixed pass, which has no elimination to spend a smaller budget on.
57+
pub max_samples: usize,
58+
59+
/// Samples that must independently land under the time limit to short circuit.
60+
///
61+
/// A short circuit decision is written to the persistent cache and reused on later runs, so
62+
/// it is confirmed rather than taken from a single possibly-lucky sample.
63+
///
64+
/// Adaptive only: the fixed pass has the whole sample set in hand before it tests the limit,
65+
/// so it has nothing to confirm.
66+
pub short_circuit_samples: usize,
67+
68+
/// How many times slower than the current best a candidate may be before elimination.
69+
///
70+
/// Adaptive only. Values below `1.0` are read as `1.0`, which eliminates every candidate
71+
/// slower than the leader that the survivor floor allows.
72+
pub speed_factor: f64,
73+
74+
/// Whether to use the adaptive round robin benchmark instead of a fixed sample count.
75+
///
76+
/// Ignored on wasm, which cannot resolve samples between rounds and so always takes the
77+
/// fixed-count pass.
78+
pub adaptive: bool,
79+
}
80+
81+
impl Default for BenchConfig {
82+
fn default() -> Self {
83+
Self {
84+
min_samples: 3,
85+
max_samples: 10,
86+
short_circuit_samples: 2,
87+
speed_factor: 1.5,
88+
adaptive: true,
89+
}
90+
}
91+
}
92+
93+
/// Every knob is read through an accessor that clamps it into its usable range, so a config file
94+
/// can hold a nonsensical value without any single call site having to remember the repair.
95+
impl BenchConfig {
96+
/// The sample budget, clamped so the range is always usable.
97+
pub fn samples(&self) -> (usize, usize) {
98+
let min = self.min_samples.max(1);
99+
(min, self.max_samples.max(min))
100+
}
101+
102+
/// How many samples must land under the limit, clamped so a short circuit always needs one.
103+
pub fn short_circuit_samples(&self) -> usize {
104+
self.short_circuit_samples.max(1)
105+
}
106+
107+
/// The elimination threshold, clamped so it can never sit below the leader's own time.
108+
pub fn speed_factor(&self) -> f64 {
109+
self.speed_factor.max(1.0)
110+
}
32111
}
33112

34113
/// Log levels for autotune logging in `CubeCL`.

crates/cubecl-runtime/src/config/base.rs

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,10 @@ impl RuntimeConfig for CubeClRuntimeConfig {
170170
self.autotune.disable_short_circuit = !enabled;
171171
}
172172

173+
if let Some(enabled) = env_bool("CUBECL_AUTOTUNE_BENCH_ADAPTIVE") {
174+
self.autotune.bench.adaptive = enabled;
175+
}
176+
173177
self
174178
}
175179
}

crates/cubecl-runtime/src/tune/bounds_generator.rs

Lines changed: 18 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,15 +6,18 @@ use crate::throughput::{ThroughputKey, ThroughputValue};
66
use crate::tune::TuneInputs;
77

88
/// A set of [`AutotuneBound`]s for a given key and reference inputs, with a launch overhead.
9-
#[derive(Debug, Clone)]
10-
#[cfg_attr(std_io, derive(serde::Serialize, serde::Deserialize))]
9+
#[derive(Debug, Clone, PartialEq)]
10+
#[cfg_attr(autotune_persistence, derive(serde::Serialize, serde::Deserialize))]
1111
pub struct Bounds {
1212
/// The bounds for autotuning.
1313
pub bounds: Vec<AutotuneBound>,
1414
/// The launch overhead for autotuning.
1515
pub launch_overhead: Duration,
1616
}
1717

18+
// Sound because [`AutotuneBound`] compares its floats bitwise, so equality stays reflexive.
19+
impl Eq for Bounds {}
20+
1821
/// Produces a set of [`AutotuneBound`]s for a given key and reference inputs.
1922
#[diagnostic::on_unimplemented(
2023
message = "`{Self}` is not a valid bounds generator",
@@ -47,7 +50,7 @@ pub trait TimeBound {
4750

4851
/// A bound for autotuning a throughput kernel, specifying the key, threshold, and number of operations.
4952
#[derive(Debug, Clone)]
50-
#[cfg_attr(std_io, derive(serde::Serialize, serde::Deserialize))]
53+
#[cfg_attr(autotune_persistence, derive(serde::Serialize, serde::Deserialize))]
5154
pub struct AutotuneBound {
5255
/// Peak throughput of the reference kernel, in ops (or bytes) per second.
5356
pub throughput: f64,
@@ -57,6 +60,18 @@ pub struct AutotuneBound {
5760
pub ops_count: usize,
5861
}
5962

63+
/// Bitwise comparison of the measured throughputs, so that equality is reflexive even if a
64+
/// degenerate measurement ever produces a `NaN`, which is what makes the [`Eq`] below sound.
65+
impl PartialEq for AutotuneBound {
66+
fn eq(&self, other: &Self) -> bool {
67+
self.throughput.to_bits() == other.throughput.to_bits()
68+
&& self.threshold.to_bits() == other.threshold.to_bits()
69+
&& self.ops_count == other.ops_count
70+
}
71+
}
72+
73+
impl Eq for AutotuneBound {}
74+
6075
/// Standardizes the creation of compute and memory [`AutotuneBound`]s.
6176
pub fn calculate_bounds(
6277
compute_throughput: &ThroughputValue,

crates/cubecl-runtime/src/tune/mod.rs

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,11 @@ mod key_generator;
3333
mod local;
3434
mod log;
3535
mod operation;
36+
// Both are the adaptive strategy, which only the native driver can run.
37+
#[cfg(not(target_family = "wasm"))]
38+
mod sampler;
39+
#[cfg(not(target_family = "wasm"))]
40+
mod schedule;
3641
mod tune_benchmark;
3742
mod tune_cache;
3843
mod tune_inputs;
Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
use alloc::vec::Vec;
2+
use core::time::Duration;
3+
use cubecl_common::benchmark::{BenchmarkComputations, BenchmarkDurations};
4+
use cubecl_common::profile::TimingMethod;
5+
6+
/// The timings collected for one candidate, plus the small amount of state needed to decide
7+
/// whether it is still worth sampling.
8+
///
9+
/// `BenchmarkComputations::score` is not usable while tuning is in flight: at a single sample the
10+
/// variance is zero and reads as perfect stability. This tracks the sample count explicitly so
11+
/// elimination can require evidence instead of inferring it from a degenerate variance.
12+
#[derive(Debug, Default)]
13+
pub(crate) struct SampleSet {
14+
durations: Vec<Duration>,
15+
stalled: u8,
16+
}
17+
18+
impl SampleSet {
19+
pub(crate) fn push(&mut self, duration: Duration) {
20+
let before = self.best();
21+
self.durations.push(duration);
22+
23+
// This push ages the biased first sample out, so the two bests are taken over different
24+
// samples and their difference says nothing about progress.
25+
if self.durations.len() == DISCARD_THRESHOLD {
26+
self.stalled = 0;
27+
return;
28+
}
29+
30+
let improved = match (before, self.best()) {
31+
(Some(before), Some(after)) => after < before.mul_f64(1.0 - CONVERGENCE_EPSILON),
32+
_ => true,
33+
};
34+
35+
self.stalled = if improved {
36+
0
37+
} else {
38+
self.stalled.saturating_add(1)
39+
};
40+
}
41+
42+
pub(crate) fn len(&self) -> usize {
43+
self.durations.len()
44+
}
45+
46+
pub(crate) fn is_empty(&self) -> bool {
47+
self.durations.is_empty()
48+
}
49+
50+
/// The first sample follows a single warmup, so it still carries allocation and clock ramp
51+
/// costs. It is dropped once enough samples remain without it.
52+
fn reliable(&self) -> &[Duration] {
53+
if self.durations.len() >= DISCARD_THRESHOLD {
54+
&self.durations[1..]
55+
} else {
56+
&self.durations
57+
}
58+
}
59+
60+
pub(crate) fn best(&self) -> Option<Duration> {
61+
self.reliable().iter().min().copied()
62+
}
63+
64+
pub(crate) fn converged(&self) -> bool {
65+
self.stalled >= CONVERGENCE_ROUNDS
66+
}
67+
68+
/// Whether enough samples independently landed under `limit` to trust a short circuit.
69+
///
70+
/// Unlike [`Self::best`] this counts the warmup-biased first sample, deliberately: the bias
71+
/// is toward being slower, so clearing the limit despite it is the conservative direction.
72+
/// Excluding it would also make a short circuit impossible during the first pass, where it
73+
/// is the only sample there is.
74+
pub(crate) fn confirmed_under(&self, limit: Duration, required: usize) -> bool {
75+
self.durations.iter().filter(|d| **d <= limit).count() >= required
76+
}
77+
78+
/// Whether any sample at all landed under `limit` — the cheap gate before a candidate is
79+
/// worth spending confirmation samples on.
80+
pub(crate) fn any_under(&self, limit: Duration) -> bool {
81+
self.durations.iter().any(|d| *d <= limit)
82+
}
83+
84+
pub(crate) fn computation(&self, method: TimingMethod) -> BenchmarkComputations {
85+
BenchmarkComputations::new(&BenchmarkDurations::from_durations(
86+
method,
87+
self.reliable().to_vec(),
88+
))
89+
}
90+
}
91+
92+
/// Total samples required before the warmup-biased first sample is discarded.
93+
const DISCARD_THRESHOLD: usize = 3;
94+
/// Relative improvement below which a new sample counts as no progress.
95+
const CONVERGENCE_EPSILON: f64 = 0.02;
96+
/// Consecutive non-improving samples before a candidate is considered converged.
97+
const CONVERGENCE_ROUNDS: u8 = 2;
98+
99+
#[cfg(test)]
100+
mod tests {
101+
use super::*;
102+
103+
fn set(durations: impl IntoIterator<Item = u64>) -> SampleSet {
104+
let mut set = SampleSet::default();
105+
for millis in durations {
106+
set.push(Duration::from_millis(millis));
107+
}
108+
set
109+
}
110+
111+
#[test]
112+
fn keeps_every_sample_below_the_discard_threshold() {
113+
assert_eq!(
114+
set([10, 20]).reliable(),
115+
&[Duration::from_millis(10), Duration::from_millis(20)]
116+
);
117+
}
118+
119+
#[test]
120+
fn drops_the_warmup_biased_first_sample_once_enough_remain() {
121+
// The first sample is the fastest here, so dropping it has to move `best` upward:
122+
// that is the bias being removed, not data being lost.
123+
let set = set([5, 20, 21]);
124+
assert_eq!(set.reliable().len(), 2);
125+
assert_eq!(set.best(), Some(Duration::from_millis(20)));
126+
}
127+
128+
#[test]
129+
fn converges_after_consecutive_non_improving_samples() {
130+
// Gains under the 2% epsilon do not count as progress. The third push only shifts the
131+
// window, so two further flat samples are what trips convergence.
132+
let mut set = set([100, 20, 20]);
133+
assert!(!set.converged());
134+
set.push(Duration::from_millis(20));
135+
assert!(!set.converged());
136+
set.push(Duration::from_millis(20));
137+
assert!(set.converged());
138+
}
139+
140+
#[test]
141+
fn aging_out_the_biased_sample_does_not_count_as_a_stall() {
142+
// The first sample is the fastest, so dropping it raises `best`. That is the window
143+
// moving, not the candidate failing to improve, and must not count toward convergence.
144+
let mut set = set([5, 20, 21]);
145+
assert!(!set.converged());
146+
set.push(Duration::from_millis(20));
147+
assert!(!set.converged());
148+
set.push(Duration::from_millis(20));
149+
assert!(set.converged());
150+
}
151+
152+
#[test]
153+
fn a_real_improvement_resets_convergence() {
154+
let mut set = set([20, 20, 20, 20, 20]);
155+
assert!(set.converged());
156+
set.push(Duration::from_millis(5));
157+
assert!(!set.converged());
158+
}
159+
160+
#[test]
161+
fn short_circuit_needs_independent_confirmations() {
162+
let limit = Duration::from_millis(10);
163+
// One fast sample among slow ones is exactly the lucky measurement that must not
164+
// commit a decision to the persistent cache on its own.
165+
assert!(!set([5, 50]).confirmed_under(limit, 2));
166+
assert!(set([5, 50, 6]).confirmed_under(limit, 2));
167+
}
168+
169+
#[test]
170+
fn computation_is_built_from_the_reliable_samples_only() {
171+
let set = set([1, 20, 30]);
172+
let computation = set.computation(TimingMethod::System);
173+
assert_eq!(computation.min, Duration::from_millis(20));
174+
assert_eq!(computation.max, Duration::from_millis(30));
175+
}
176+
177+
#[test]
178+
fn empty_set_has_no_best() {
179+
let set = SampleSet::default();
180+
assert!(set.is_empty());
181+
assert_eq!(set.best(), None);
182+
assert_eq!(set.len(), 0);
183+
}
184+
}

0 commit comments

Comments
 (0)