Report service download progress to remote backend by pipex · Pull Request #71 · balena-io/helios

pipex · 2026-01-28T19:40:00Z

This add a create_service task that initializes the service with Installing state. This allows the reporting module to identify installing services and report as Downloading if download_progress < 100

This PR also includes some fixes to make sure helios unstable keeps working with the legacy supervisor

Use docker-cli v28 for balena-engine compatibility
Fail parsing image uri if the image or tag are <none> (a valid result from docker). Ignore parsing failures when listing images
Make sure to always pass the raw target to the legacy supervisor even on serialization errors

helios-state/src/tasks/app.rs

cywang117 · 2026-02-05T22:39:14Z

helios-state/src/tasks/image.rs

        engine_id: None,
        config: Default::default(),
-        download_progress: 0,
+        download_progress: 100,


Why initialize with 100% progress then set to 0% later?

Yes, because this is the outcome that the planner will see and simulates a successful outcome of this task.

That is a part of the interface of Mahler that is not super easy to convey unfortunately :(

cywang117 · 2026-02-05T22:44:03Z

helios-remote/src/report.rs


+#[derive(Clone, Serialize, Default, Debug)]
+#[serde(rename_all = "lowercase")]
+pub enum UpdateStatus {


What's the difference between this and the UpdateStatus in helios-state/src/seek.rs?

It's the way it's serialized mostly. The one in seek is also returned by the API /v3/status and returns {"status": "applying changes"}, while this just serializes to a string. It might be worth unifying them at some point

On the other hand, the helios-state crate should remain agnostic to reporting (and probably the local API) so types can diverge safely.

Actually, the state in seek.rs also includes any errors from an apply that might be useful to get from. the API

helios-remote/src/report.rs

cywang117 · 2026-02-05T22:55:43Z

helios-remote/src/report.rs

+        // which indicates an error condition. State changes during the request should not
+        // interrupt it - we'll report the new state after this request completes.
+        last_report = loop {
+            tokio::select! {


Claude said there may be a logic issue in this select!:

Here's what happens step by step:

First iteration: both branches are active. select! races them.

If state_rx.changed() returns Ok(()) — it matched, but the pattern Err(_) doesn't match, so the branch is completed and disabled.

If report_future hasn't finished yet, on the next loop iteration state_rx.changed() is already consumed. Now only report_future is active.

But: if state_rx.changed() returns Ok(()) and report_future completes in the same poll — both branches are disabled on the next iteration, and else fires. Then the
inner loop repeats with both branches completed → else fires again → infinite busy-loop.

Even in the non-degenerate case, once state_rx.changed() fires Ok(()), that branch is disabled for all subsequent iterations of the inner loop. This means subsequent
channel closures won't be detected until report_future completes.

I can't make heads or tails of the control flow at the moment, but commenting in case there's some validity to this

I think that's wrong. A watch channel (like state_rx) blocks on calls to changed(). Once changed() returns Ok, the value is marked as seen, meaning the next call to changed will block again until there is an error or a new value (see Receiver::changed()).

But: if state_rx.changed() returns Ok(()) and report_future completes in the same poll

I don't think this can happen either, futures in Rust don't complete until polled, so there is no such thing as both things happening at the same time.

Dockerfile

The Service status is used to identify what status is the service in its lifecycle. This commit also creates a service initialization task that runs before image downloads. The status and early initialization should allow the reporter module to report on downloading services by looking at the service status and the image download progress. Change-type: patch

Create the state patch to send to the backend from the user app list. This also fixes a bug with the reporting loop that would never send a patch Change-type: patch

This allows to identify bugs with the patch body Change-type: patch

Tokio stream has much better utilities for working with streams

This prevents multiple reports, like during a download interrupting each other which ends up in multiple patch cancellation on the logs without any reports actually being sent. The reports are limited by the min poll interval (10s) so on downloads faster than that, no download progress will show on the dashboard. Change-type: patch

This limit can be added in the future, but since the patch happens at most every 10 seconds, is better to make sure it has the most accurate progress for each image on long downloads. Change-type: patch

Also downgrade the dind version used in integration tests. Change-type: patch

Reading the list of images would fail if parsing an image URI failed, however, the list can contain `<none>:<none>` tags, in which case we want to ignore those errors. This also includes the repo_digests list, deduplicating by repo and tag. Change-type: patch

If the remote target cannot be processed during poll, only the raw_target will be available and will be passed to the legacy supervisor when available. Change-type: patch

Uses mahler exception feature. Also skip reporting with userapps disabled to prevent conflicting with the legacy supervisor.

Convert Image::download_progress to u8

d1d86d6

pipex marked this pull request as draft January 28, 2026 19:40

pipex force-pushed the state-report-and-service-download branch 7 times, most recently from 8aa4d39 to c13e431 Compare February 4, 2026 17:37

pipex force-pushed the state-report-and-service-download branch from d51a484 to c13e431 Compare February 5, 2026 13:31

pipex requested a review from cywang117 February 5, 2026 14:25

pipex marked this pull request as ready for review February 5, 2026 14:25

cywang117 reviewed Feb 5, 2026

View reviewed changes

pipex force-pushed the state-report-and-service-download branch 2 times, most recently from 6d01c0f to a12bf3c Compare February 6, 2026 15:05

pipex added 9 commits February 6, 2026 12:33

Add reporting for user apps

1a45c82

Create the state patch to send to the backend from the user app list. This also fixes a bug with the reporting loop that would never send a patch Change-type: patch

Debug client errors to remote API

39a69de

This allows to identify bugs with the patch body Change-type: patch

Replace futures-lite with tokio_stream

a75f597

Tokio stream has much better utilities for working with streams

Do not limit image progress reporting to worker

f93d5a8

This limit can be added in the future, but since the patch happens at most every 10 seconds, is better to make sure it has the most accurate progress for each image on long downloads. Change-type: patch

Use docker-cli v28 to keep compatibility with balena-engine

dbaac05

Also downgrade the dind version used in integration tests. Change-type: patch

Allow device target for local worker to be None

bc88dbf

If the remote target cannot be processed during poll, only the raw_target will be available and will be passed to the legacy supervisor when available. Change-type: patch

Ignore apps when planning if userapps feature is disabled

894b73a

Uses mahler exception feature. Also skip reporting with userapps disabled to prevent conflicting with the legacy supervisor.

pipex force-pushed the state-report-and-service-download branch from a12bf3c to 894b73a Compare February 6, 2026 15:34

flowzone-app bot enabled auto-merge February 6, 2026 15:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Report service download progress to remote backend#71

Report service download progress to remote backend#71
pipex wants to merge 11 commits intomainfrom
state-report-and-service-download

pipex commented Jan 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

cywang117 Feb 5, 2026

Uh oh!

pipex Feb 6, 2026

Uh oh!

cywang117 Feb 5, 2026

Uh oh!

pipex Feb 6, 2026

Uh oh!

pipex Feb 6, 2026

Uh oh!

pipex Feb 6, 2026

Uh oh!

Uh oh!

cywang117 Feb 5, 2026

Uh oh!

pipex Feb 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

pipex commented Jan 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

pipex commented Jan 28, 2026 •

edited

Loading