feat: allow caching the results of an unfiltered UniStream (WIP)

[Christopher-Chianelli]

• Introduces ConstraintFactory.staticData(Uni), which creates an independent NodeNetwork from the provided Uni stream.

• Introduces TupleSourceRoot that represent top level tuple producers (the ForEach nodes now implement this, as well as a new StaticDataUniNode).

• The StaticDataUniNode caches the results of its internal NodeNetwork and maps the tuples to new tuples compatiable with the external NodeNetwork

• Inserts/retracts invalidate the cache and cause the output tuples to be updated.

• Updates uses the cache and do not notify the internal NodeNetwork.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
94.1% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[triceo]

Suggested change

	// Since this is called when a fact (not a variable changes),
	// Since this is called when a fact (not a variable) changes,

[triceo]

This is surprisingly simple! Some high-level comments; leaving naming out of the picture for now:

Introduces ConstraintFactory.staticData(Uni), which creates an independent NodeNetwork from the provided Uni stream.

I'm not a big fan of this. The programming model doesn't look very nice IMO.

factory.staticData(factory.forEach()...)
    .filter(...)

Something like this would be more stream-y:

factory.forEach().
    ...
    .asStatic()
    .filter(...)

This brings its own problems. But both proposals share the same issue - chaining static streams. Nothing (other than run-time fail-fasts) prevents you from doing staticData(staticData(...).filter(...)). IMO we should decide (and enforce) that a static stream can only be built once, and everything after that is dynamic. And ideally, this would happen at type level, not at runtime.

This was the main idea of filterStatic() - because we can actually guarantee, at type level, that once any "non-static" method is called, a "static" method can never be called again. (The reverse doesn't work, unless we want to duplicate the entire stream API, which we don't.) IMO we shouldn't rule out filterStatic() + joinStatic() - under the hood, it can be implemented like in this PR, but the public API IMO need not offer this absolute flexibility to turn anything into static data.

[triceo]

So this propagates to nowhere? The whole node seems to exist only to record the tuples; arguably, the code should be refactored to not require this to be a node, and just use the recorder directly.

[triceo]

I am not sure why this used to be @Nullable, but I remember clearly that I was adding it there for a reason.

I recommend adding null checks, running the entire CI incl. quickstarts, and if the null checks don't trigger, then I say it's safe for this to be non-null.

[triceo]

Doing this on every insert/retract seems excessively expensive. There will be a large burst of inserts at the beginning; they ought to be cached and processed in a batch, probably before propagation starts.

[triceo]

Arguably if these methods aren't needed, then this shouldn't implement TupleLifecycle. If it doesn't look like a duck and doesn't quack like a duck, it shouldn't be called a duck.

[triceo]

I know that we said we don't want to deal with joins yet. But considering that the same mechanism would be used to support joins as well, I think we deserve to see how this would look for a bi-stream.

(Also, considering that this suddenly looks like much less work than originally estimated, maybe we actually do take joins in scope?)

[triceo]

What happens when this stream is used both statically and non-statically?
(Think node-sharing.)

[Christopher-Chianelli]

The output stream can be node shared if the input streams are identical (not implemented currently since this is a POC). The input stream will not be node shared; its an independent network.

[triceo]

To bring an alternative API idea:

 factory.forEachXYZ(Something.class, filter)

This would produce a StaticUniStream which extends UniStream, and only adds one method - joinXYZ(); this join only allows to join other static streams and returns StaticBiStream - allowing possibly for a static bi join (and similarly for tri).

In essence, this brings a very simple, clear API:

• We support static filters and joins (maybe also ifExists, which is essentially a join).
• Static operations come first.
• Once you go dynamic, you can never go back.
• This is enforced at the type level, preventing impossible situations from being modelled.
• Node-sharing is trivial; nothing changes in the status quo.

It does have a downside - it doesn't allow arbitrary static streams, such as groupBy, map, flatten etc. Right now, nobody is asking for it, and I think it is a decent trade-off to get the benefits; but we can possibly ask other people for their opinion on this.