Reconciliation (allowing state-preserving hot-reload)

[villor]

Based on: bevyengine#20158
Depends on: #35 (commit included in this branch)

Objective

Reconciliation is the process of incrementally updating entities, components,
and relationships from a ResolvedScene by storing state from previous reconciliations.

This idea is based on previous work by @NthTensor in i-cant-believe-its-not-bsn.

Since this may be a bit controversial to merge onto the "main" BSN branch, here's how you can try it out today:

[dependencies]
bevy = { git = "https://github.com/villor/bevy.git", branch = "bsn-reconcile" }

I'll do my best to keep this up-to-date with cart/next-gen-scenes.

How does it work?

When a scene is reconciled on an entity, it will:

1. Build the templates from the scene.
2. Remove components that were previously inserted during reconciliation but should no longer be present.
   • This includes components that were present in the previous bundle but absent in the new one
   • and components that were explicit in the previous bundle but implicit (required components) in the new one.
3. Insert the new components onto the entity.
   • Note: There is no diffing of values involved here, components are re-inserted on every reconciliation.
4. Map related entities to previously reconciled entities by their ReconcileAnchors or otherwise spawn new entities
   • Entities can be explicitly "keyed" using the Name component - which makes them identifiable among their siblings. This also works with #MyEntityName syntax in bsn!.
   • Entities that are not explicitly keyed are "anonymous" and will be recycled in order, which may cause state loss.
5. Despawn any leftover orphans from outdated relationships.
6. Recursively reconcile related entities (1).
7. Store the state of the reconciliation in a ReconcileReceipt component on the entity.

Why is it useful?

Non-destructive hot reloads

The obvious use case for an algorithm like this would be hot reloading of scenes. When using reconciliation, scenes can be hot reloaded while maintaing state that isn't part of the explicit scene output.

Demo from Discord: https://discord.com/channels/691052431525675048/1264881140007702558/1396963952050835527

Reactivity

On the web, reconciliation is a common technique for updating the DOM from an intermediate representation (aka VDOM). This is sometimes referred to as "coarse grained", and does have its performance penalties compared to more fine grained surgical updates. An upside to reconciliation is that it doesn't require a lot of special casing for templates - scenes can be built using familiar rust expressions.

The algorithm included here is not reactivity by itself, but it could potentially be used for future experiments in a number of ways, for example:

• "Immediate mode reactivity" by reconciling on every frame - possibly by memoizing certain parts of the tree as an optimization.
• Coarse/medium-grained reactivity involving observers and/or change detection to reconcile parts of the tree on-demand.

The "state vs output" problem

One issue when using reconciliation (or reactivity in general) which became very clear when trying to reconcile bevy_feathers widgets, is that we are storing mutable state and scene output on the same entities. It is convenient to include the internal state components in the bsn! to instantiate them. This poses a problem when using reconciliation, as the state components would be overwritten with their defaults on every reconciliation.

One solution to this problem is to use required components for internal state. That way those components are "implicit" and won't be overwritten on each reconciliation. I have converted the internal state of all feathers controls to use this approcach, making them usable with reconciliation.

Another solution we could consider would be syntax in bsn! for marking components as implicit/required, hinting that they should only be inserted once.

Implementation caveats/TODOs

• Currently re-builds every Template on every reconciliation. This may cause issues with implementations that assume the template runs on a freshly spawned entity. When testing it with on/OnTemplate however, it looks like it doesn't create any duplicate observers
  • UPDATE: This is no longer true after 0.17, where duplicates are created on every reconcile. Worked around it by adding a OnHandle component to ensure observers of a certain type are added once, and removed when that component is removed. Seems to work well with reconciliation.
• Currently not integrated with the deferred/async scene systems, all dependencies must be loaded before reconciliation.
• Components are inserted one-by-one just like spawn_scene. Should by inserted as a single (dynamic) bundle to minimze archetype moves.
• Plenty of room for optimization (lots of allocations)
• Siblings that have equal Names are not handled and causes chaos - fall back to auto-inc and log a warning?

Example: Subsecond hot reload

This is the example from the demo on Discord, try it out with subsecond hot patching enabled!

use bevy::{
    color::palettes::tailwind,
    core_widgets::CoreWidgetsPlugins,
    feathers::{
        controls::{checkbox, CheckboxProps},
        dark_theme::create_dark_theme,
        theme::UiTheme,
        FeathersPlugin,
    },
    input_focus::{
        tab_navigation::{TabGroup, TabNavigationPlugin},
        InputDispatchPlugin,
    },
    prelude::*,
    scene2::prelude::{Scene, *},
};

fn main() {
    App::new()
        .add_plugins((
            DefaultPlugins,
            CoreWidgetsPlugins,
            InputDispatchPlugin,
            TabNavigationPlugin,
            FeathersPlugin,
        ))
        .insert_resource(UiTheme(create_dark_theme()))
        .add_systems(Startup, setup)
        .add_systems(Update, reconcile_ui)
        .run();
}

#[derive(Component, Clone, Default)]
struct UiRoot;

fn setup(mut commands: Commands) {
    commands.spawn(Camera2d);
    commands.spawn(UiRoot);
}

fn reconcile_ui(ui_root: Single<Entity, With<UiRoot>>, mut commands: Commands) {
    commands.entity(*ui_root).reconcile_scene(demo_root());
}

fn demo_root() -> impl Scene {
    bsn! {
        Node {
            width: Val::Percent(100.0),
            height: Val::Percent(100.0),
            align_items: AlignItems::Center,
            justify_content: JustifyContent::Center,
        }
        TabGroup
        BackgroundColor(tailwind::NEUTRAL_900)
        [
            Node {
                flex_direction: FlexDirection::Column,
                row_gap: Val::Px(8.0),
            } [
                #A :todo_item("Write BSN"),
                #B :todo_item("Hot reload it!"),
                #E :todo_item("Move things around"),
                #C :todo_item("Add checkboxes"),
                #D :todo_item("Try some styling"),
            ]
        ]
    }
}

fn todo_item(title: &'static str) -> impl Scene {
    bsn! {
        :checkbox(CheckboxProps::default())
        Node {
            column_gap: Val::Px(10.0),
            padding: UiRect::all(Val::Px(6.0)),
            border: UiRect::all(Val::Px(1.0)),
            align_items: AlignItems::Center,
        }
        BorderColor::all(tailwind::NEUTRAL_700.into())
        BorderRadius::all(Val::Px(5.0))
        BackgroundColor(tailwind::NEUTRAL_800)
        [
            Text(title)
            TextColor(tailwind::NEUTRAL_100) TextFont { font_size: 16.0 }
        ]
    }
}