|
2 | 2 |
|
3 | 3 | Below you can find the changelog for Execution Engine. |
4 | 4 |
|
| 5 | +## Execution Engine `v1.6.0` | inference `v0.53.0` |
| 6 | + |
| 7 | +!!! Note "Change may require attention" |
| 8 | + |
| 9 | + This release introduces upgrades and new features with **no changes required** to existing workflows. |
| 10 | + Some blocks may need to be upgraded to take advantage of the latest Execution Engine capabilities. |
| 11 | + |
| 12 | +Prior versions of the Execution Engine had significant limitations when interacting with certain types of |
| 13 | +blocks - specifically those operating in Single Instruction, Multiple Data (SIMD) mode. These blocks are designed to |
| 14 | +process batches of inputs at once, apply the same operation to each element, and return results for the entire batch. |
| 15 | + |
| 16 | +For example, the `run(...)` method of such a block might look like: |
| 17 | + |
| 18 | +```python |
| 19 | +def run(self, image: Batch[WorkflowImageData], confidence: float): |
| 20 | + pass |
| 21 | +``` |
| 22 | + |
| 23 | +In the manifest, the `image` field is declared as accepting batches. |
| 24 | + |
| 25 | +The issue arose when the input image came from a block that did not operate on batches. In such cases, the |
| 26 | +Execution Engine was unable to construct a batch from individual images, which often resulted in frustrating |
| 27 | +compilation errors such as: |
| 28 | + |
| 29 | +``` |
| 30 | +Detected invalid reference plugged into property `images` of step `$steps.model` - the step property |
| 31 | +strictly requires batch-oriented inputs, yet the input selector holds non-batch oriented input - this indicates |
| 32 | +the problem with construction of your Workflow - usually the problem occurs when non-batch oriented step inputs are |
| 33 | +filled with outputs of non batch-oriented steps or non batch-oriented inputs. |
| 34 | +``` |
| 35 | + |
| 36 | +In Execution Engine `v1.6.0`, this limitation has been removed, introducing the following behaviour: |
| 37 | + |
| 38 | +* When it is detected that a given input must be batch-oriented, a procedure called **Auto Batch Casting** is applied. |
| 39 | +This automatically converts the input into a `Batch[T]`. Since all batch-mode inputs were already explicitly denoted in |
| 40 | +manifests, most blocks (with exceptions noted below) benefit from this upgrade without requiring any internal changes. |
| 41 | + |
| 42 | +* The dimensionality (level of nesting) of an auto-batch cast parameter is determined at compilation time, based on the |
| 43 | +context of the specific block in the workflow as well as its manifest. If other batch-oriented inputs are present |
| 44 | +(referred to as *lineage supports*), the Execution Engine uses them as references when constructing auto-casted |
| 45 | +batches. This ensures that the number of elements in each batch dimension matches the other data fed into the step |
| 46 | +(simulating what would have been asserted if an actual batch input had been provided). If there are no |
| 47 | +*lineage supports*, or if the block manifest requires it (e.g. input dimensionality offset is set), the missing |
| 48 | +dimensions are generated similarly to the |
| 49 | +[`torch.unsqueeze(...)` operation](https://docs.pytorch.org/docs/stable/generated/torch.unsqueeze.html). |
| 50 | + |
| 51 | +* Step outputs are then evaluated against the presence of an Auto Batch Casting context. Based on the evaluation, |
| 52 | +outputs are saved either as batches or as scalars, ensuring that the effect of casting remains local, with the only |
| 53 | +exception being output dimensionality changes introduced by the block itself. As a side effect, it is now possible to: |
| 54 | + |
| 55 | + * **create output batches from scalars** (when the step increases dimensionality), and |
| 56 | + |
| 57 | + * **collapse batches into scalars** (when the block decreases dimensionality). |
| 58 | + |
| 59 | +* The only potential friction point arises **when a block that does not accept batches** (and thus does not denote |
| 60 | +batch-accepting inputs) **decreases output dimensionality**. In previous versions, the Execution Engine handled this by |
| 61 | +applying dimensionality wrapping: all batch-oriented inputs were wrapped with an additional `Batch[T]` dimension, |
| 62 | +allowing the block’s `run(...)` method to perform reduce operations across the list dimension. With Auto Batch Casting, |
| 63 | +however, such blocks no longer provide the Execution Engine with a clear signal about whether certain inputs are |
| 64 | +scalars or batches, making casting nondeterministic. To address this, a new manifest method was introduced: |
| 65 | +`get_parameters_enforcing_auto_batch_casting(...)`. This method must return the list of parameters for which batch |
| 66 | +casting should be enforced when dimensionality is decreased. It is not expected to be used in any other context. |
| 67 | + |
| 68 | +* In earlier versions, a hard constraint existed: dimensionality collapse could only occur at levels ≥ 2 (i.e. only |
| 69 | +on nested batches). This limitation is now removed. Dimensionality collapse blocks may also operate on scalars, with |
| 70 | +the output dimensionality “bouncing off” the zero ground. |
| 71 | + |
| 72 | +* There is one **important limitation** uncovered by these changes. Since Auto Batch Casting allows scalars to be |
| 73 | +converted into batches (when a scalar is fed into a block that increases dimensionality), it is possible to end up with |
| 74 | +multiple batches at the first nesting level, each with a different origin (lineage). In this case, the current |
| 75 | +Execution Engine implementation cannot deterministically construct the output. Previous versions assumed that outputs |
| 76 | +were always lists of elements, with the order determined by the input batch. With dynamically generated batches, |
| 77 | +this assumption no longer holds. Fixing this design flaw would require a breaking change for all customers, |
| 78 | +so it is deferred to **Execution Engine v2.0**. For now, an assertion has been introduced in the code, raising the |
| 79 | +following error: |
| 80 | + |
| 81 | +``` |
| 82 | +Workflow Compiler detected that the workflow contains multiple elements which create |
| 83 | +top-level data batches - for instance inputs and blocks that create batched outputs from |
| 84 | +scalar parameters. We know it sounds convoluted, but the bottom line is that this |
| 85 | +situation is known limitation of Workflows Compiler. |
| 86 | +Contact Roboflow team through github issues (https://github.com/roboflow/inference/issues) |
| 87 | +providing full context of the problem - including workflow definition you use. |
| 88 | +``` |
| 89 | + |
| 90 | + |
5 | 91 | ## Execution Engine `v1.5.0` | inference `v0.38.0` |
6 | 92 |
|
7 | 93 | !!! Note "Change does not require any action" |
|
0 commit comments