[Implemented Feature] Drawio like functionality native in AFFiNE

[spmp]

Good day Folks,
I loved drawio, and hated having to manage two stacks to have my text and images. Over the last few weeks I have been puttering away with the help of my agent friends from Anthropic, OpenAI and Z.ai to build the minimal set of features that I need to be able to do drawio like things in AFFiNE.

The fork is here: https://github.com/spmp/AFFiNE/tree/feature/advanced-diagramming-capability
Please play

Feature summary

Here is a summary of the features introduced and work still to do:

• Grid and snapping
  • Add a grid and snapping item to the zoom menu
  • Snap to grid (shape and connectors) and snap to guides toggle
  • Adjustable grid size
  • Connector drags snap to grid
• Enhance connectors
  • Add rounded connector style
  • Make connectors draggable, preserved over move and reload
  • Add and clear connector waypoints
  • Waypoint subsumption
  • Connector jumps
  • Next connector style from the last chosen can change from properties menu
• Shape upgrades
  • Add a shape browser for additional shapes
  • Ensure shapes have edge nodes
  • Hover over shape edges exposes edge nodes
  • Clicking an edge node initiates a connector from that node
  • Add dotted line style (replaces None as None can be replaced by a transparent line)
• Simple shapes
  • General shapes
  • Arrows
  • Flowchart
• Cleanup categories in shape chooser
  • Importer for additional libraries (i.e Azure)
• Add a properties menu which allows for checking and changing details of a shape or connector
  • Colour/style
  • Type (expand)
  • Corner radius where applicable
  • Position
  • Size
  • All changeable properties
  • Shape aspect ratio lock
  • Connector endps - open circle, check other tyles for ERD's. Take from drawio,
• Shape menu and fixes
  • Shape selector scroll
  • Shape colour from last - NOTE: Conflicts with existing behaviour for rectangle, elipse, and triangle
  • Horizontal/Vertical shape flip and menu item, preserving connectors and anchors
  • Alternate colour palates from drawio on ^ arrow next to colour picker on 'edgeless toolbar'
  • Support gradients
• Advanced shapes
  • Flowchart next shape
  • Flowchart next shape inherits styling
  • Parent/child shape inheritence and visibility.
  • Container shapes and collapsibility
    • Drag in/drag out
    • Connector routing on collapse
  • Introduce mindmap heirarchically hideable shapes
• Frame size adjustments: 2, 3, 5, Custom size, full width
• Export and import
  • Import and export infra of diagrams via image metadata
    • Diagrams can be constructed from metadata
    • Diagram metadata can be embedded in PNG
    • Image autodetection of metadata with option to import diagram rather than image
  • Ensure page export to MD (and others) correctly renders frames to importable PNG
  • Add export option to Frame to save an image
• [] Tests
  • [] Check all existing tests pass
  • [] Add unit tests for functions
  • [] Add playwright tests
  • [] Merge with current master

Discussion

This code is AI generated and before submitting a massive (it is massive) PR to the devs I would love to hear from them and other users as to how this worked for you, and what I can do to make merging into mainline a possibility. All this should be backward compatible.

This is very light on testing, I haven't yet tried to get all test passing, just racing for feature parity to relieve this obsession.

I hope this is supportive for the AFFiNE community, I am really looking forward to working with y'all on the PR 8)

[spmp]

Drawio like diagramming examples

Thanks for taking a look at this one @darkskygit , @CatsJuice , @L-Sun , and others.
I put this doc together as a visual walkthrough so you can review quickly without digging through every file first. The screenshots are grouped by feature area, with before/after behavior and the key intent for each change.

Here is a colorful rendition of the changes made.

Drawings: Shapes and colors

Shape selector

A shape selector has been added in place of one of the standard shapes. Currently with General shapes, Flowchart shapes, Arrow shapes, and Advanced shapes. Further shape libraries can be imported if desired.

Color palettes, including gradient and remembering last colors used

Palette selector on up arrow from standard color picker

Gradient palette

Flowchart shapes with next shape menu

Horizontal and vertical flip - preserving connectors

Advanced shapes with hierarchical show/hide and collapsing containers (preserving connectors)

Script created to import remaining existing shape libraries should that be desired...

AWS, Azure, IBM, etc. etc.
Connector

Connectors

Added rounded connectors that can be dragged and have jumps. Shapes have more edge nodes for snapped start/end

Rounded connector style

Shape edge nodes

Shapes have multiple edge nodes - general shapes typically 5 per side, flowchart and others typically 3.

Hover to expose

Click to start connector and hover to see possible target nodes

Connector edges have dragable nodes

Connector waypoints can be added or cleared

Add waypoint

After adding select the line segment and it will be bisected

Clear waypoints

Clear waypoints to revert to auto-route

Connector waypoints auto-subsume

Drag an edge in line with another edge to subsume

Once clicked away, and clicked back there is only one horizontal edge again

Connectors can jump over lines below them

Properties menu for advanced features

Shape properties menu

Properties menu from the more⋮menu on a shape

Properties menu has all setable items - sensible list and drop downs where appropriate

Including position and size

Properties menu for connectors with extended end types

End types menu

This menu also gives access to a large

Grid and snapping menu

A grid and snapping menu is added to the zoom menu which allows selecting grid size, snap to grid, snap to guides, and snap to grids for connectors. NOTE: The background grid does not perfectly line up

Export and import

Export and import a Frame or image-to-frame

Frame export

A Frame can be exported to metadata file or PNG with embedded metadata

Import a Frame

Image import autodetects metadata and gives the option to load as Frame

/image...

Export and import to Markdown or HTML

Export to markdown or HTML

If images or frames are detected in the page, export to Markdown or HTML produces a ZIP file with PNG's. If the PNG's are from a Frame then these have the Frame metadata embedded.

Import Markdown with media recreates frames if required

Importing a Markdown with media (ZIP file) recreates Frames from images that contain metadata.

Frame view menu to expand frames horizontally and vertically

Frames have a menu to increase their vertical and horizontal size based on multiples of the original detected size:

[HeJiachen-PM]

I'm literally shocked. It would be awesome if we could have such feature, but there will be a llllllot of tests required to make it happen.

[spmp]

Thanks you for your positivity @HeJiachen-PM , and glad to hear you're heavy on testing. I started in TDD style but this did not progress well with the agents, so now is the time to bolt the test on at the end 8)
I could really use some support on the testing, what the minimum and ideal coverage is like, what you are testing for, and likely some hands on support with it. The more you give me now the more I can get done. I need the tests before merging anyway to ensure no regressions... It's testing time 😜!

[spmp]

Is there a forum/medium where I could get in a chat with some of the devs here to understand how to bring this feature into mainline AFFiNE - I am sure this could be broken up into a few PR's, and there have been some design decisions that I've made to get to MVP that could be anti-patterns for the project. Would love to discuss, and OFC, how to get all the behavior nailed down in tests! I started a thread in Discord (https://discord.com/channels/959027316334407691/1476000222210691072/1476000222210691072) as I thought that may be a better medium.
This project is such a great basis, and having these drawing features with freehand and all the other coolnesses in here has me so excited. A never ending 'whats next' stacking up behind me 8)
Nga mihi nui 🤗

[spmp]

@HeJiachen-PM , here is the list of tests 'I' am chunking through. Hopefully can get through this before running out of credit this month:
I will update as we go and push into the listed branch once tests and upstream merge complete. Then for a PR ;)

• Shape menu

  • Shape menu can be clicked outside of the square/triangle/circle swatches to open with no shape selected
    • Playwright: click empty menu area opens browser without selecting a shape

• Grid and snapping

  • Grid menu appears in zoom toolbar (horizontal and vertical layouts)
    • Unit: menu model exposes grid toggle + sizes
    • Playwright: toolbar renders in both layouts
  • Grid visibility toggle shows/hides dot grid
    • Unit: grid visibility state toggles
    • Playwright: grid DOM/canvas visibility toggles
  • Grid size presets update grid immediately
    • Unit: size state updates and stored
    • Playwright: dot spacing changes
  • Custom grid size input accepts value and updates grid
    • Unit: parsing/validation of input
    • Playwright: input updates grid spacing
  • Snap-to-guides toggle affects guide snapping
    • Unit: snap overlay respects toggle
    • Playwright: drag shows/does not show guide snapping
  • Snap-to-grid toggle works independently of snap-to-guides
    • Unit: snap-to-grid path when guides off
    • Playwright: drag snaps even when guides disabled
  • Dragging shapes snaps to grid when enabled
    • Unit: snap calculation returns grid-aligned deltas
    • Playwright: shape ends on grid intersection
  • Dragging shapes ignores grid when disabled
    • Unit: snap overlay returns no adjustment
    • Playwright: shape ends at non-grid position
  • Connector endpoint drag snaps to grid when enabled
    • Playwright: connector endpoint aligns to grid
  • Only the dragged connector endpoint snaps (other segments unaffected)
    • Playwright: non-dragged segments preserve shape
  • Snap-to-grid does not block snapping to edge nodes (edge nodes take priority)
    • Playwright: endpoint prefers edge node over grid

• Shape browser and libraries

  • Shape menu “More” opens the shape browser
    • Playwright: menu opens from toolbar
  • Shape browser resizes for mobile (min 1, max 3 columns)
    • Playwright: viewport width changes column count
  • Shape browser is scrollable in peek and edgeless modes
    • Playwright: scroll works in both modes
  • Shape browser search filters across all categories
    • Unit: search filter logic
    • Playwright: search reduces list
  • Imported libraries (e.g., Azure) appear with correct category names
    • Unit: importer maps category names
    • Playwright: categories render in UI
  • Category cleanup matches draw.io ordering
    • Unit: ordering rules
    • Playwright: ordering in UI

• Shape palettes and gradients

  • Palette picker appears for shapes, connector, and brush tools
    • Playwright: picker visible per tool
  • Palette picker remembers last palette and last color per tool
    • Unit: palette memory per tool
    • Playwright: last selection restored
  • Last stroke/fill/line style set via palette or context Color menu is applied to next new shape or connector
    • Unit: last-props applies for both palette and context menu entry points
    • Playwright: create new shape/connector after color change uses last style
  • Gradient fills render correctly on supported shapes
    • Unit: gradient model serialization
    • Playwright: gradient visible on canvas
  • Dotted line style renders and is selectable
    • Unit: stroke style enum includes dotted
    • Playwright: dotted stroke visible

• Shapes and edge nodes

  • Edge nodes appear on hover (desktop)
    • Playwright: hover shows nodes
  • Long-press exposes nodes on touch
    • Playwright: long-press shows nodes
  • Clicking an edge node starts a connector from that node
    • Playwright: connector starts at node
  • Edge node orientations match shape geometry (cloud fix)
    • Unit: node orientation computed correctly
    • Playwright: nodes align with edges
  • Extended connection points exist for supported shapes
    • Unit: anchor generation returns expected points
    • Playwright: nodes count matches expectations

• Shape transforms

  • Horizontal flip preserves connectors and anchor nodes
    • Unit: transform preserves anchors
    • Playwright: connectors remain attached
  • Vertical flip preserves connectors and anchor nodes
    • Unit: transform preserves anchors
    • Playwright: connectors remain attached
  • Shape text flips correctly with shape (no mirrored text regression)
    • Playwright: text orientation correct
  • Text stays centered when flipped
    • Playwright: text position unchanged
  • Selection outline and edge handles follow flips
    • Playwright: selected rect matches flipped bounds
  • Connector stays attached to exact edge node after flip + rotate
    • Playwright: anchor position unchanged under flip+rotate

• Connector types and defaults

  • Rounded connector appears in type menu
    • Playwright: menu shows rounded option
  • Rounded connector rendering respects corner radius
    • Unit: rounded path generation
    • Playwright: rendered corners visible
  • Rounded connectors support waypoints and segment dragging
    • Playwright: waypoint drag works in rounded mode
  • Next connector defaults to last-used connector style (toolbar selection)
    • Unit: last-props persistence
    • Playwright: new connector uses last style
  • Next connector defaults to last-used connector style (properties menu)
    • Playwright: property change updates default
  • Curve and straight connectors do not show segment handles
    • Playwright: no segment handles visible

• Connector dynamics and waypoints

  • Segment handles appear at movable segment midpoints
    • Playwright: tests/blocksuite/e2e/edgeless/connector/segment-dynamics.spec.ts
    • Playwright: tests/blocksuite/e2e/edgeless/connector/segment-dragging-spec.spec.ts
  • Tails (first/last segments) are NOT draggable
    • Playwright: tests/blocksuite/e2e/edgeless/connector/waypoint-integration.spec.ts
  • Cursor reflects orientation (row-resize for horizontal, col-resize for vertical)
    • Playwright: tests/blocksuite/e2e/edgeless/connector/segment-dynamics.spec.ts
    • Playwright: tests/blocksuite/e2e/edgeless/connector/waypoint-integration.spec.ts
  • Handles update correctly after undo/redo (no stale handles)
    • Playwright: undo/redo updates handles
  • Drag constraint clamps to axis (no diagonal segments)
    • Unit: constrainDrag
    • Playwright: drag results orthogonal
  • Dragging a segment does not detach endpoints from shapes
    • Playwright: endpoints remain attached
  • Dragging a 2-point line creates an S-shape (3 segments, 4 points)
    • Unit: splitSegmentToSShape
    • Playwright: path length expands
  • Dragging newly created segment works after split
    • Playwright: drag second segment
  • Add Waypoint action splits a segment into two equal segments
    • Unit: split logic midpoint
    • Playwright: Add Waypoint + click segment
  • Clear Waypoints resets to autorouted path
    • Unit: clear waypoints removes overrides
    • Playwright: More menu action
  • Waypoints persist after moving a connected shape
    • Playwright: move shape preserves path
  • Waypoints persist after moving both connected shapes together
    • Playwright: move both shapes preserves path
  • Waypoints persist across page reload
    • Playwright: reload keeps path
  • Waypoints preserve absolute position when connector bounds change
    • Unit: waypoint translation on bounds change
    • Playwright: no drift after move
  • Waypoint order is preserved (no sorting by X/Y)
    • Unit: generatePathThroughWaypoints order
  • Path uses tail segments consistently when routing via waypoints
    • Unit: tail generation
    • Playwright: tail orientation preserved after move
  • Subsumption removes redundant segments on pointer up
    • Unit: segment merge/subsume
    • Playwright: path simplifies after drag back
  • Negative tail drag converts tail to movable segment
    • Unit: tail conversion logic
    • Playwright: drag past tail

• Connector jumps

  • Jump style selector appears in connector toolbar
    • Playwright: selector visible
  • Jumps render when connectors intersect
    • Unit: jump calculation produces points
    • Playwright: jumps visible on canvas
  • Jumps update dynamically when dragging either connector
    • Playwright: jumps update live when dragging top connector
    • Playwright: jumps update live when dragging bottom connector
    • Playwright: smooth update check via stepped drag sampling (assert jump marker count/position changes across multiple mousemove steps)
  • Rounded connectors jump correctly
    • Playwright: rounded + jump visible
  • Line jumps do not render for Curve connectors
    • Playwright: curve connector does not show jumps when it crosses a straight/orthogonal line
    • Playwright: straight/orthogonal connector does not jump over a curve connector

• Connector labels

  • Label is centered on straight connectors
    • Playwright: label position
  • Label is centered on connectors with waypoints
    • Playwright: label position after drag
  • Label drag handles appear and allow repositioning
    • Playwright: drag label handle
  • Label drag does not alter connector path
    • Playwright: path unchanged

• Properties panel

  • Properties panel opens from More menu for shapes
    • Playwright: menu item opens panel
  • Properties panel opens from More menu for connectors
    • Playwright: menu item opens panel
  • Properties panel supports color, stroke, and line style
    • Unit: property model updates
    • Playwright: UI updates
  • Properties panel supports size and position edits
    • Unit: size/position setters
    • Playwright: UI updates element
  • Corner radius clamps to valid range
    • Unit: clamp logic
  • Shape aspect ratio lock works
    • Playwright: resize keeps ratio
  • Connector endpoint styles are selectable (including ERD-style endpoints)
    • Playwright: endpoints render correctly
  • Panel scrolls in peek mode on touch devices
    • Playwright mobile: scroll works
  • Panel is positioned correctly in peek mode (bottom of screen)
    • Playwright mobile: placement

• Containers and mindmap

  • Container shapes collapse and expand correctly
    • Playwright: collapse/expand
  • Dragging shapes into container works
    • Playwright: drag in
  • Dragging shapes out of container works
    • Playwright: drag out
  • Connector routing updates on container collapse
    • Playwright: connectors reroute
  • Mindmap nodes support hierarchical hide/show
    • Playwright: hide/show
  • Mindmap/flowchart “next item” menu resizes appropriately
    • Playwright: responsive menu

• Flowchart and arrow shapes

  • Flowchart shape palette includes all expected shapes
    • Playwright: category items exist
  • Flowchart quick-add creates next shape
    • Playwright: quick-add works
  • Next flowchart shape inherits styling
    • Playwright: style match
  • Arrow shapes appear under the Arrow category
    • Playwright: category items exist

• Frame sizing and layout

  • Frame size menu offers 2, 3, 5, Custom, Full width
    • Playwright: menu options present
  • Frame width behaves correctly on small screens
    • Playwright mobile: width correct
  • Frame size changes persist after reload
    • Playwright: reload preserves

• Export and import

  • PNG export embeds diagram metadata
    • Unit: metadata encode/decode
    • Playwright: export contains metadata
  • Importing an image with metadata offers “import as diagram”
    • Playwright: prompt shown
  • Imported diagram reconstructs shapes/connectors correctly
    • Playwright: diagram matches
  • Frame export option saves image from Frame context menu
    • Playwright: menu action exports
  • Page export to Markdown/HTML includes frames as importable PNG
    • Playwright: export contains PNG with metadata
  • Page import from Markdown Zip restores frames correctly
    • Playwright: import reconstructs frames

• Mobile/touch regressions and fixes

  • Shape menu auto-sizes on mobile
    • Playwright mobile: menu sizing
  • Shape browser closes when frame editor closes
    • Playwright: close behavior
  • Properties menu works on touch device
    • Playwright mobile: open + interact
  • Long-press on shape exposes edge nodes
    • Playwright mobile: long-press
  • Flowchart “+” click requires tap-up/tap-down (no random selection)
    • Playwright mobile: tap behavior