Skip to content

Commit 714d88c

Browse files
Add first part of changelog
1 parent 0b38378 commit 714d88c

File tree

1 file changed

+86
-0
lines changed

1 file changed

+86
-0
lines changed

docs/workflows/execution_engine_changelog.md

Lines changed: 86 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,92 @@
22

33
Below you can find the changelog for Execution Engine.
44

5+
## Execution Engine `v1.6.0` | inference `v0.53.0`
6+
7+
!!! Note "Change may require attention"
8+
9+
This release introduces upgrades and new features with **no changes required** to existing workflows.
10+
Some blocks may need to be upgraded to take advantage of the latest Execution Engine capabilities.
11+
12+
Prior versions of the Execution Engine had significant limitations when interacting with certain types of
13+
blocks - specifically those operating in Single Instruction, Multiple Data (SIMD) mode. These blocks are designed to
14+
process batches of inputs at once, apply the same operation to each element, and return results for the entire batch.
15+
16+
For example, the `run(...)` method of such a block might look like:
17+
18+
```python
19+
def run(self, image: Batch[WorkflowImageData], confidence: float):
20+
pass
21+
```
22+
23+
In the manifest, the `image` field is declared as accepting batches.
24+
25+
The issue arose when the input image came from a block that did not operate on batches. In such cases, the
26+
Execution Engine was unable to construct a batch from individual images, which often resulted in frustrating
27+
compilation errors such as:
28+
29+
```
30+
Detected invalid reference plugged into property `images` of step `$steps.model` - the step property
31+
strictly requires batch-oriented inputs, yet the input selector holds non-batch oriented input - this indicates
32+
the problem with construction of your Workflow - usually the problem occurs when non-batch oriented step inputs are
33+
filled with outputs of non batch-oriented steps or non batch-oriented inputs.
34+
```
35+
36+
In Execution Engine `v1.6.0`, this limitation has been removed, introducing the following behaviour:
37+
38+
* When it is detected that a given input must be batch-oriented, a procedure called **Auto Batch Casting** is applied.
39+
This automatically converts the input into a `Batch[T]`. Since all batch-mode inputs were already explicitly denoted in
40+
manifests, most blocks (with exceptions noted below) benefit from this upgrade without requiring any internal changes.
41+
42+
* The dimensionality (level of nesting) of an auto-batch cast parameter is determined at compilation time, based on the
43+
context of the specific block in the workflow as well as its manifest. If other batch-oriented inputs are present
44+
(referred to as *lineage supports*), the Execution Engine uses them as references when constructing auto-casted
45+
batches. This ensures that the number of elements in each batch dimension matches the other data fed into the step
46+
(simulating what would have been asserted if an actual batch input had been provided). If there are no
47+
*lineage supports*, or if the block manifest requires it (e.g. input dimensionality offset is set), the missing
48+
dimensions are generated similarly to the
49+
[`torch.unsqueeze(...)` operation](https://docs.pytorch.org/docs/stable/generated/torch.unsqueeze.html).
50+
51+
* Step outputs are then evaluated against the presence of an Auto Batch Casting context. Based on the evaluation,
52+
outputs are saved either as batches or as scalars, ensuring that the effect of casting remains local, with the only
53+
exception being output dimensionality changes introduced by the block itself. As a side effect, it is now possible to:
54+
55+
* **create output batches from scalars** (when the step increases dimensionality), and
56+
57+
* **collapse batches into scalars** (when the block decreases dimensionality).
58+
59+
* The only potential friction point arises **when a block that does not accept batches** (and thus does not denote
60+
batch-accepting inputs) **decreases output dimensionality**. In previous versions, the Execution Engine handled this by
61+
applying dimensionality wrapping: all batch-oriented inputs were wrapped with an additional `Batch[T]` dimension,
62+
allowing the block’s `run(...)` method to perform reduce operations across the list dimension. With Auto Batch Casting,
63+
however, such blocks no longer provide the Execution Engine with a clear signal about whether certain inputs are
64+
scalars or batches, making casting nondeterministic. To address this, a new manifest method was introduced:
65+
`get_parameters_enforcing_auto_batch_casting(...)`. This method must return the list of parameters for which batch
66+
casting should be enforced when dimensionality is decreased. It is not expected to be used in any other context.
67+
68+
* In earlier versions, a hard constraint existed: dimensionality collapse could only occur at levels ≥ 2 (i.e. only
69+
on nested batches). This limitation is now removed. Dimensionality collapse blocks may also operate on scalars, with
70+
the output dimensionality “bouncing off” the zero ground.
71+
72+
* There is one **important limitation** uncovered by these changes. Since Auto Batch Casting allows scalars to be
73+
converted into batches (when a scalar is fed into a block that increases dimensionality), it is possible to end up with
74+
multiple batches at the first nesting level, each with a different origin (lineage). In this case, the current
75+
Execution Engine implementation cannot deterministically construct the output. Previous versions assumed that outputs
76+
were always lists of elements, with the order determined by the input batch. With dynamically generated batches,
77+
this assumption no longer holds. Fixing this design flaw would require a breaking change for all customers,
78+
so it is deferred to **Execution Engine v2.0**. For now, an assertion has been introduced in the code, raising the
79+
following error:
80+
81+
```
82+
Workflow Compiler detected that the workflow contains multiple elements which create
83+
top-level data batches - for instance inputs and blocks that create batched outputs from
84+
scalar parameters. We know it sounds convoluted, but the bottom line is that this
85+
situation is known limitation of Workflows Compiler.
86+
Contact Roboflow team through github issues (https://github.com/roboflow/inference/issues)
87+
providing full context of the problem - including workflow definition you use.
88+
```
89+
90+
591
## Execution Engine `v1.5.0` | inference `v0.38.0`
692

793
!!! Note "Change does not require any action"

0 commit comments

Comments
 (0)