Explain relationship between UiGlobalTransform, entity-local coordinates, and window coordinates

[viridia]

How can Bevy's documentation be improved?

In previous versions of Bevy, there was a method that allowed you to get the logical rectangle for a UI node. This has been replaced by various properties on ComputedNode which interact in complex ways.

There are a several different variables in play here:

• The translation field on UiGlobalTransform.
• The inverse_scale_factor in ComputedNode.
• The UiScale resource.
• The coordinates in picking events.
• The distance value in picking events.

It's not always clear, in the documentation, how to combine these:

• Is the transform field on UiGlobalTransform in physical or logical coordinates?
• Are window coordinates logical or physical? By "window coordinates" I mean whatever numbers are used for absolute positioning.
• How can I compute the rectangle of a node in window coordinates?
• How can I compute the rectangle of the window in the same coordinates?
• What's the coordinate origin of a UI node? (Turns out it's the center, but the docs don't say this.)
• A picking coordinates logical or physical?
• Do I need to ever care about camera / world space when working with UI?

So far I have managed to get the right calculations, but this is mostly by accident or by using recipes supplied to me by other people on Discord.

What I would like to see is an overview or reference guide that explains all these different coordinates and how they relate to each other.

@alice-i-cecile @ickshonpe

[alice-i-cecile]

So far I have managed to get the right calculations, but this is mostly by accident or by using recipes supplied to me by other people on Discord

My preference is actually to primarily supply this via helper methods, rather than documentation. We do the same thing for projection matrices, e.g. view_to_world, with a lot of success.

[ickshonpe]

Is the transform field on UiGlobalTransform in physical or logical coordinates?

Physical

Are window coordinates logical or physical? By "window coordinates" I mean whatever numbers are used for absolute positioning.

There are three coordinate systems: logical coordinates, window coordinates and physical coordinates.
Multiply logical coordinates by UiScale to get window coordinates.
Multiply window coordinates by the window scale factor to get physical coordinates.

How can I compute the rectangle of a node in window coordinates?

Node coordinates are object centered. To calculate the coordinates in window coordinates:

let half_size = node.size() * 0.5;
let pos_top_left     = Vec2::new(-half_size.x,  -half_size.y);
let pos_top_right    = Vec2::new( half_size.x,  -half_size.y);
let pos_bottom_left  = Vec2::new(-half_size.x, half_size.y);
let pos_bottom_right = Vec2::new( half_size.x, half_size.y);

let pos_top_left_window = ui_global_transform.transform_point2(pos_top_left) * node.inverse_scale_factor() * ui_scale.0;

Are picking coordinates logical or physical?

Picking uses "Window" coordinates. That's why we had to divide them by the UiScale (to get logical coordinates) to fix the core widget bugs.

Do I need to ever care about camera / world space when working with UI?

No, not unless you are doing world UI or rendering to texture.

(Really sleepy and haven't checked any of this, so I might have got some details wrong.)

[ickshonpe]

The biggest annoyance atm is that bevy_picking doesn't know about UiScale and there doesn't seem to be any trivial way to get it to use it. Node hits are resolved by the UI's picking_backend module so they work okay but drag event offsets are reported in window coordinates and need to be divided by the UiScale to get logical UI coords.

[viridia]

That's one annoyance, but not the only one. What motivated me writing this ticket was something else: trying to decide where I needed to multiply by inverse_scale_factor and where I didn't.

[ickshonpe]

That's one annoyance, but not the only one. What motivated me writing this ticket was something else: trying to decide where I needed to multiply by inverse_scale_factor and where I didn't.

To convert a physical length value in Node to a logical length value usable in Style just multiply by inverse_scale_factor. You don't need to consider UiTransform as it's applied post layout.

To convert a physical point to window coordinates you have to use the transform_point2 method on UiGlobalTransform then divide by the window's scale factor. It's awkward to retrieve the window scale factor so it's easier to multiply by the Node's inverse_scale_factor times UiScale.

[viridia]

Another suggestion: in the documentation, when we talk about physical vs. logical pixels (such as in ComputedNode), it should include a link to https://docs.rs/bevy/latest/bevy/window/struct.WindowResolution.html which talks about what these concepts mean.

I know how to convert a physical length value to a logical length value. The question I raised (when to do it) requires knowing two things:

• Whether the source value is in physical or logical units. Some places (like ComputedNode) are clearly marked, but ComputedNode doesn't contain the node's position. For this I'm using UiGlobalTransform's translation, which doesn't say what unit it is using.
• Whether I should be doing my calculations in physical or logical units in the first place.

For popovers, I'm attempting to calculate a formula involving three rectangles: the popup area, the anchor area, and the window area. The output of this calculation gets converted directly into Val::Px units, which I'm guessing are logical pixels.

Now, it might seem obvious that logical pixels are the way to go - but the counter argument is that so many of the inputs are stored in physical pixels, and there must be a reason for this, right? So maybe we should be doing the calculation in physical pixels, and then convert them to logical after the calculation is done.

[viridia]

To put it another way, if "God wanted us to work in logical units, He would have added a .logical_size() method to ComputedNode.