|
| 1 | +# SOF Scheduling Architecture |
| 2 | + |
| 3 | +This directory (`src/schedule`) contains the Sound Open Firmware (SOF) scheduling infrastructure, deeply integrated with the underlying Zephyr RTOS. SOF utilizes a multi-tiered scheduling approach to cater to different real-time constraints, ranging from hard real-time, low-latency requirements to more relaxed, compute-intensive data processing tasks. |
| 4 | + |
| 5 | +## Overview of Schedulers |
| 6 | + |
| 7 | +SOF categorizes tasks and assigns them to specialized schedulers: |
| 8 | + |
| 9 | +1. **LL (Low Latency) Scheduler**: For tasks that require strict, predictable, and lowest possible latency bound to hardware events (Timers, DMA interrupts). |
| 10 | +2. **DP (Data Processing) Scheduler**: For compute-intensive components that process large chunks of data and operate on deadlines rather than strict cycles. |
| 11 | +3. **TWB (Thread With Budget) Scheduler**: For tasks that are allotted a specific execution "budget" per scheduler tick, expressed in Zephyr time-slice ticks (e.g. derived from `ZEPHYR_TWB_BUDGET_MAX` in OS ticks), which the runtime then uses to limit and account for CPU cycles to prevent starvation. |
| 12 | + |
| 13 | +Below is a high-level component interaction architecture of the SOF scheduling domains on top of Zephyr. |
| 14 | + |
| 15 | +```mermaid |
| 16 | +graph TD |
| 17 | + subgraph Zephyr RTOS |
| 18 | + Timer[Hardware Timer] |
| 19 | + DMA[DMA Controller] |
| 20 | + Threads[Zephyr Threads] |
| 21 | + end |
| 22 | +
|
| 23 | + subgraph Generic Scheduler API |
| 24 | + API[schedule.c API] |
| 25 | + end |
| 26 | +
|
| 27 | + subgraph LL Scheduler Domain |
| 28 | + LL[zephyr_ll.c] |
| 29 | + LLDomain[zephyr_domain.c] |
| 30 | + DMADomain[zephyr_dma_domain.c] |
| 31 | + end |
| 32 | +
|
| 33 | + subgraph DP Scheduler Domain |
| 34 | + DP[zephyr_dp_schedule.c] |
| 35 | + DPThread[zephyr_dp_schedule_thread.c] |
| 36 | + end |
| 37 | +
|
| 38 | + subgraph TWB Scheduler Domain |
| 39 | + TWB[zephyr_twb_schedule.c] |
| 40 | + end |
| 41 | +
|
| 42 | + API --> LL |
| 43 | + API --> DP |
| 44 | + API --> TWB |
| 45 | +
|
| 46 | + Timer -.->|Interrupt| LLDomain |
| 47 | + DMA -.->|Interrupt| DMADomain |
| 48 | +
|
| 49 | + LLDomain --> |Wakeup| LL |
| 50 | + DMADomain --> |Wakeup| LL |
| 51 | + LL -->|Runs tasks| Threads |
| 52 | +
|
| 53 | + LL -->|NOTIFIER_ID_LL_POST_RUN| DP |
| 54 | + DP -->|Recalculate Deadlines| DPThread |
| 55 | + DPThread -->|Update Thread Deadlines| Threads |
| 56 | +
|
| 57 | + LL -->|LL Tick Source| TWB |
| 58 | + TWB -->|Update Time Slices| Threads |
| 59 | +``` |
| 60 | + |
| 61 | +--- |
| 62 | + |
| 63 | +## 1. LL (Low Latency) Scheduler |
| 64 | + |
| 65 | +The LL scheduler (`zephyr_ll.c`) is designed for extreme low-latency processing. It bypasses complex generic Zephyr scheduling for its internal tasks to minimize overhead, executing a list of registered SOF tasks in a strict priority order. |
| 66 | + |
| 67 | +### Architecture |
| 68 | + |
| 69 | +- **Domain Threads**: The LL scheduler runs within a dedicated high-priority Zephyr thread (`ll_thread0`, etc.) pinned to each core (`zephyr_domain.c`). |
| 70 | +- **Triggers**: It is woken up by a hardware timer (e.g., a 1ms tick) or directly by hardware DMA interrupts (`zephyr_dma_domain.c`). |
| 71 | +- **Execution**: Once woken up, it locks the domain, iterates through all scheduled tasks in priority order, moves them to a temporary list, and calls their `.run()` functions. |
| 72 | +- **Post-Run**: After all tasks execute, it triggers a `NOTIFIER_ID_LL_POST_RUN` event. This event cascades to wake up other dependent schedulers like DP and TWB. Event not run on LL userspace configuration. |
| 73 | + |
| 74 | +### Task State Diagram |
| 75 | + |
| 76 | +```mermaid |
| 77 | +stateDiagram-v2 |
| 78 | + [*] --> INIT: task_init |
| 79 | + INIT --> QUEUED: schedule_task |
| 80 | + QUEUED --> RUNNING: zephyr_ll_run (Timer/DMA Tick) |
| 81 | + RUNNING --> RUNNING: return RESCHEDULE |
| 82 | + RUNNING --> FREE: return COMPLETED |
| 83 | + RUNNING --> CANCEL: task_cancel |
| 84 | +
|
| 85 | + QUEUED --> CANCEL: task_cancel |
| 86 | + CANCEL --> FREE: task_free |
| 87 | +
|
| 88 | + FREE --> [*] |
| 89 | +``` |
| 90 | + |
| 91 | +*(Note: State transitions handle Zephyr SMP locking to ensure a task is safely dequeued before state shifts)* |
| 92 | + |
| 93 | +--- |
| 94 | + |
| 95 | +## 2. DP (Data Processing) Scheduler |
| 96 | + |
| 97 | +The DP scheduler (`zephyr_dp_schedule.c`) manages asynchronous, compute-heavy tasks that process data when enough input is available and sufficient output space is free. It effectively relies on Zephyr's EDF (Earliest Deadline First) or standard preemptive scheduling capabilities. |
| 98 | + |
| 99 | +### Architecture |
| 100 | + |
| 101 | +- **Separate Threads**: Unlike LL which multiplexes tasks inside a single thread, **each DP task is assigned its own Zephyr thread**. |
| 102 | +- **Wakeup Mechanism**: DP scheduling is evaluated at the end of each LL tick (`scheduler_dp_recalculate()`). |
| 103 | +- **Readiness**: It checks if a component has sufficient data across its sinks and sources. If so, it transitions to `RUNNING` and signals the individual DP thread via a Zephyr Event object. |
| 104 | +- **Deadlines**: Once ready, the DP thread computes its deadline absolute timestamp (`module_get_deadline()`) and calls `k_thread_absolute_deadline_set()`, submitting to the Zephyr kernel's EDF scheduler. |
| 105 | + |
| 106 | +### Task State Diagram |
| 107 | + |
| 108 | +```mermaid |
| 109 | +stateDiagram-v2 |
| 110 | + [*] --> INIT: task_init |
| 111 | + INIT --> QUEUED: schedule_task |
| 112 | +
|
| 113 | + note right of QUEUED |
| 114 | + Wait for LL POST RUN event |
| 115 | + to evaluate Readiness. |
| 116 | + end note |
| 117 | +
|
| 118 | + QUEUED --> RUNNING: resources ready (set priority/deadline) |
| 119 | + RUNNING --> QUEUED: return RESCHEDULE (processed chunk) |
| 120 | + RUNNING --> COMPLETED: return COMPLETED |
| 121 | + RUNNING --> CANCEL: task_cancel |
| 122 | +
|
| 123 | + QUEUED --> CANCEL: task_cancel |
| 124 | + COMPLETED --> FREE: task_free |
| 125 | + CANCEL --> FREE: task_free |
| 126 | +
|
| 127 | + FREE --> [*] |
| 128 | +``` |
| 129 | + |
| 130 | +--- |
| 131 | + |
| 132 | +## 3. TWB (Thread With Budget) Scheduler |
| 133 | + |
| 134 | +The TWB scheduler (`zephyr_twb_schedule.c`) provides execution budget limits for specific tasks to prevent them from starving the CPU. This is useful for intensive workloads that shouldn't disrupt the overall systemic low-latency chain. |
| 135 | + |
| 136 | +### Architecture |
| 137 | + |
| 138 | +- **Separate Threads**: Similar to DP, each TWB task executes in its own Zephyr thread. |
| 139 | +- **Time Slicing**: When scheduled, the thread's execution budget is configured in OS ticks via `k_thread_time_slice_set()`. This tick-based budget is internally converted to hardware cycles for accounting against the CPU cycles actually consumed. |
| 140 | +- **Budget Exhaustion**: If the thread consumes its budget (as measured in hardware cycles derived from the tick budget) before completing its work for the tick, a callback (`scheduler_twb_task_cb()`) is invoked by the Zephyr kernel. This callback immediately drops the thread's priority to a background level (`CONFIG_TWB_THREAD_LOW_PRIORITY`), preventing starvation of other threads. |
| 141 | +- **Replenishment**: On the next LL tick (`scheduler_twb_ll_tick()`), the consumed hardware cycles are reset, and the thread's original priority and time slice are restored, granting it a fresh tick-based budget. |
| 142 | + |
| 143 | +### Task State Diagram |
| 144 | + |
| 145 | +```mermaid |
| 146 | +stateDiagram-v2 |
| 147 | + [*] --> INIT: task_init |
| 148 | + INIT --> RUNNING: schedule_task (Thread Created) |
| 149 | +
|
| 150 | + state RUNNING { |
| 151 | + [*] --> HighPriority: Budget replenished |
| 152 | + HighPriority --> LowPriority: Budget Exhausted (Callback) |
| 153 | + LowPriority --> HighPriority: Next LL Tick |
| 154 | + } |
| 155 | +
|
| 156 | + RUNNING --> QUEUED: return RESCHEDULE |
| 157 | + QUEUED --> RUNNING: Next LL Tick (Restore Priority) |
| 158 | +
|
| 159 | + RUNNING --> CANCEL: task_cancel |
| 160 | + QUEUED --> CANCEL: task_cancel |
| 161 | +
|
| 162 | + RUNNING --> COMPLETED: return COMPLETED |
| 163 | +
|
| 164 | + CANCEL --> FREE: task_free |
| 165 | + COMPLETED --> FREE: task_free |
| 166 | +
|
| 167 | + FREE --> [*] |
| 168 | +``` |
0 commit comments