You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/source/en/modular_diffusers/modular_diffusers_states.md
+41-16Lines changed: 41 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,25 +10,25 @@ an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express o
10
10
specific language governing permissions and limitations under the License.
11
11
-->
12
12
13
-
# PipelineState and BlockState
13
+
# Block states
14
14
15
-
<Tipwarning={true}>
15
+
Blocks rely on the [`PipelineState`] and [`BlockState`] data structures for communicating and sharing data.
16
16
17
-
🧪 **Experimental Feature**: Modular Diffusers is an experimental feature we are actively developing. The API may be subject to breaking changes.
17
+
| State | Description |
18
+
|-------|-------------|
19
+
|`PipelineState`| Maintains the overall data required for a pipeline's execution and allows blocks to read and update its data. |
20
+
|`BlockState`| Allows each block to perform its computation with the necessary data from `inputs` and `intermediate_inputs`|
18
21
19
-
</Tip>
22
+
This guide explains how states work and how they connect blocks.
20
23
21
-
In Modular Diffusers, `PipelineState` and `BlockState` are the core data structures that enable blocks to communicate and share data. The concept is fundamental to understand how blocks interact with each other and the pipeline system.
24
+
## PipelineState
22
25
23
-
In the modular diffusers system, `PipelineState` acts as the global state container that all pipeline blocks operate on. It maintains the complete runtime state of the pipeline and provides a structured way for blocks to read from and write to shared data.
26
+
The [`PipelineState`] is a global state container for all pipeline blocks. It maintains the complete runtime state of the pipeline and provides a structured way for blocks to read from and write to shared data.
24
27
25
-
A `PipelineState` consists of two distinct states:
28
+
There are two dict's in [`PipelineState`] for structuring data.
26
29
27
-
-**The immutable state** (i.e. the `inputs` dict) contains a copy of values provided by users. Once a value is added to the immutable state, it cannot be changed. Blocks can read from the immutable state but cannot write to it.
28
-
29
-
-**The mutable state** (i.e. the `intermediates` dict) contains variables that are passed between blocks and can be modified by them.
30
-
31
-
Here's an example of what a `PipelineState` looks like:
30
+
- The `inputs` dict is an **immutable** state containing a copy of user provided values. A value added to `inputs` cannot be changed. Blocks can read from `inputs` but cannot write to it.
31
+
- The `intermediates` dict is a **mutable** state containing variables that are passed between blocks and can be modified by them.
32
32
33
33
```py
34
34
PipelineState(
@@ -44,16 +44,41 @@ PipelineState(
44
44
)
45
45
```
46
46
47
-
Each pipeline blocks define what parts of that state they can read from and write to through their `inputs`, `intermediate_inputs`, and `intermediate_outputs` properties. At run time, they gets a local view (`BlockState`) of the relevant variables it needs from `PipelineState`, performs its operations, and then updates `PipelineState` with any changes.
47
+
## BlockState
48
+
49
+
The [`BlockState`] is a local view of the relevant variables, `inputs` and `intermediate_inputs`, that an individual pipeline block needs from [`PipelineState`] for performing it's computations.
48
50
49
-
For example, if a block defines an input `image`, inside the block's `__call__` method, the `BlockState` would contain:
51
+
You can access these variables directly as attributes like `block_state.image`.
50
52
51
53
```py
52
54
BlockState(
53
55
image: <PIL.Image.Image image mode=RGBsize=512x512 at 0x7F3ECC494640>
54
56
)
55
57
```
56
58
57
-
You can access the variables directly as attributes: `block_state.image`.
59
+
When a block's `__call__` method is executed, it retrieves the [`BlockState`] with `self.get_block_state(state)`, performs it's operations, and updates [`PipelineState`] with `self.set_block_state(state, block_state)`.
60
+
61
+
```py
62
+
def__call__(self, components, state):
63
+
# retrieve BlockState
64
+
block_state =self.get_block_state(state)
65
+
66
+
# computation logic on inputs and intermediate_inputs
67
+
68
+
# update PipelineState
69
+
self.set_block_state(state, block_state)
70
+
return components, state
71
+
```
72
+
73
+
## State interaction
74
+
75
+
[`PipelineState`] and [`BlockState`] interaction is defined by a block's `inputs`, `intermediate_inputs`, and `intermediate_outputs`.
76
+
77
+
-`inputs`, a block can modify an input - like `block_state.image` - but the change is local to the [`BlockState`] and won't affect the original image in [`PipelineState`].
78
+
-`intermediate_inputs`, is often values created from a previous block. When a block modifies `intermediate_inputs` - like `batch_size` - this change is reflected in both the [`BlockState`] and [`PipelineState`]. Any subsequent blocks are also affected.
79
+
80
+
If a previous block doesn't provide an `intermediate_inputs`, then the pipeline makes it available as a user input. However, the value is still a mutable intermediate state.
81
+
82
+
-`intermediate_outputs`, is a new variable that a block creates from `intermediate_inputs`. It is added to the [`PipelineState`]'s `intermediates` dict and available as an `intermediate_inputs` for subsequent blocks or accessed by users as a final output from the pipeline.
58
83
59
-
We will explore more on how blocks interact with pipeline state through their `inputs`, `intermediate_inputs`, and `intermediate_outputs` properties, see the [PipelineBlock guide](./pipeline_block.md).
84
+
If a variable is modified in `block_state` but not declared as an `intermediate_outputs`, it won't be added to [`PipelineState`].
0 commit comments