Diagram editor

[koonpeng]

New feature implementation

Implemented feature

This still needs more testing before I would consider it stable, but I thought this is a good time to merge to main.

Even though the change is huge, most of the changes is in diagram-editor so there shouldn't be much conflicts.

GenAI Use

We follow OSRA's policy on GenAI tools

• I used a GenAI tool in this PR.
• I did not use GenAI

Generated-by: Gemini

[mxgrey]

I'm very excited for bevy impulse to have a functional visual editor! It looks like ReactFlow was a good choice, and I look forward to seeing how far we can go with it.

I don't think we have to be very strict on the first draft of the editor, but I'm seeing a few bugs in the UI that I think are likely to cause significant confusion for users in its current state. I think we should get those sorted out before merging this to main where users may feel invited to play around with it.

I've described the bugs below, and included a video for most of them. The last section is some remarks on non-critical improvements that I think we should follow up on if we don't resolve them within this PR.

Discrepancies between the displayed state and the actual state

For nodes, I see strange things happen with the config field. In the following video I have two nodes that use the mul builder. I assign one of them a config of 10 and the other a config of 100. Clicking back and forth between them, eventually they both show a config of 10, even though I never changed the one that was supposed to be 100:

mul_bug.mp4

I've seen other cases where I change the config of a node and then immediately re-click the node to find that its config did not seem to change at all.

I see similar problems with connection settings. When trying to set up a split operation, the UI doesn't respond to changes in what I select. Clicking away and re-selecting the edge allows it to update somewhat, but you can see that the sequence value never seems to update in the edit box, even though it shows correctly as 1 on the board:

split_bug.mp4

Confusing Join input

I made a test case that uses the Join operation and found that it was giving me the wrong result. It wasn't an error message; instead an incorrect number was being calculated. After inspecting the generated diagram json I noticed that only one buffer was being connected into the join even though the UI showed two connections into it. The problem is both connections were both set to Index 0, so one of them was being quietly discarded.

I understand if it's too tricky to automatically assign new connections to a unique index/key, but I would recommend these two changes to help users deal with this:

• Have the connection line display the index/key of the buffer inside the join, similar to how connections coming out of Split display their sequence number.
• Before allowing the workflow to be run or the json to be generated, check that there are no duplicate entries for Join and pop out an error if there are. We should do the same for Fork Result.

Unable to choose Key for Join connection

The UI for setting a string key for Join doesn't work. Without that we won't be able to join messages into structs; we'll only be able to join into arrays and tuples.

join_key_bug.mp4

Non-urgent quality of life improvements

There are few things that I think could improve the quality of life, although I don't consider these urgent. We can open follow-up tickets for these like #102.

• For operations like Split and Fork Clone that support multiple outputs, use a different icon for the output slot, maybe something like a small cluster of 3 or 4 circles instead of one. Fork Result could have exactly two output circles since it only supports two outputs.
• I like that buffer "outputs" and join/listen "inputs" have the same color to indicate that they can be connected to each other. I think we should do the same for ordinary inputs and outputs. I don't think it makes sense that ordinary outputs are white and ordinary inputs are orange when they can connect to each other. If we want to distinguish inputs from outputs then maybe output icons can be little downward pointing triangles instead of circles.
• The icons for Fork Result and Split should be flipped upside-down. I would also recommend using a question mark in some way for Fork Result as a reference to its similarity to the Rust question mark operator.

[mxgrey]

Let's include the instructions for setting up pnpm like what we have for rmf-web. Our target user base is the ROS community, who often have no experience using web tools like pnpm.

[koonpeng]

Discrepancies between the displayed state and the actual state

Turns out this is due to reactflow only returning the new values on the next render, fixed by avoiding use of reactflow's state.

Confusing Join input

Added labels to buffer edges.

Unable to choose Key for Join connection

fixed

The icons for Fork Result and Split should be flipped upside-down. I would also recommend using a question mark in some way for Fork Result as a reference to its similarity to the Rust question mark operator.

updated the icons.

I like that buffer "outputs" and join/listen "inputs" have the same color to indicate that they can be connected to each other. I think we should do the same for ordinary inputs and outputs. I don't think it makes sense that ordinary outputs are white and ordinary inputs are orange when they can connect to each other. If we want to distinguish inputs from outputs then maybe output icons can be little downward pointing triangles instead of circles.

Which operation did you see the inputs to be orange? Both ordinary inputs and outputs should be white-ish. Orange is for stream outputs.

[mxgrey]

Which operation did you see the inputs to be orange? Both ordinary inputs and outputs should be white-ish. Orange is for stream outputs.

In the videos you can see that mul has an orange output icon, but it doesn't produce any streams.

Would ReactFlow be able to support having streams come out of the side (maybe right side) of the box instead of the bottom?