Skip to content

Commit 7a18d50

Browse files
[SDP] StreamingFlow.mustBeAppend flag (ignored)
1 parent 2330033 commit 7a18d50

File tree

2 files changed

+22
-5
lines changed

2 files changed

+22
-5
lines changed

docs/declarative-pipelines/AppendOnceFlow.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
`AppendOnceFlow` is a [ResolvedFlow](ResolvedFlow.md) with [once](Flow.md#once) enabled.
44

5-
`AppendOnceFlow` is [created](#creating-instance) for a [UnresolvedFlow](UnresolvedFlow.md) with [once](UnresolvedFlow.md#once) flag enabled.
5+
`AppendOnceFlow` is [created](#creating-instance) for an [UnresolvedFlow](UnresolvedFlow.md) with [once](UnresolvedFlow.md#once) flag enabled.
66

77
## Creating Instance
88

docs/declarative-pipelines/StreamingFlow.md

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,16 +2,33 @@
22

33
`StreamingFlow` is a [ResolvedFlow](ResolvedFlow.md) that may or may not be [append](#mustBeAppend).
44

5-
`StreamingFlow` is planned for execution as [StreamingTableWrite](StreamingTableWrite.md) by [FlowPlanner](FlowPlanner.md#plan).
5+
`StreamingFlow` is [planned for execution](FlowPlanner.md#plan) as [StreamingTableWrite](StreamingTableWrite.md) (assuming that the [Output](DataflowGraph.md#output) of [this flow](#flow)'s [destination](ResolutionCompletedFlow.md#destinationIdentifier) is a [Table](Table.md)).
66

77
## Creating Instance
88

99
`StreamingFlow` takes the following to be created:
1010

11-
* <span id="flow"> [UnresolvedFlow](UnresolvedFlow.md)
12-
* <span id="funcResult"> `FlowFunctionResult`
13-
* <span id="mustBeAppend"> `mustBeAppend` flag (default: `false`)
11+
* <span id="flow"> [UnresolvedFlow](ResolutionCompletedFlow.md#flow)
12+
* <span id="funcResult"> [FlowFunctionResult](ResolutionCompletedFlow.md#funcResult)
13+
* [mustBeAppend](#mustBeAppend) flag
1414

1515
`StreamingFlow` is created when:
1616

1717
* `FlowResolver` is requested to [convertResolvedToTypedFlow](FlowResolver.md#convertResolvedToTypedFlow) (for [UnresolvedFlow](UnresolvedFlow.md)s with their results being streaming dataframes)
18+
19+
### mustBeAppend Flag { #mustBeAppend }
20+
21+
```scala
22+
mustBeAppend: Boolean
23+
```
24+
25+
`StreamingFlow` is given `mustBeAppend` flag when [created](#creating-instance).
26+
27+
Default: `false`
28+
29+
The value of `mustBeAppend` flag is based on whether there are more than one flow to [this flow](#flow)'s [destination](UnresolvedFlow.md#destinationIdentifier) in a dataflow graph or not.
30+
31+
When enabled (`true`), [this UnresolvedFlow](#flow) is planned for execution with the `Append` output mode (as the other flows will then get their results overwritten).
32+
33+
!!! note "StreamingTableWrite ignores mustBeAppend"
34+
`StreamingFlow` is planned for execution as [StreamingTableWrite](StreamingTableWrite.md) with no execution differences based on the `mustBeAppend` flag.

0 commit comments

Comments
 (0)