-
-
Notifications
You must be signed in to change notification settings - Fork 24
dasharo-binaries-paths-convention.md: Add #1170
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
philipanda
wants to merge
2
commits into
master
Choose a base branch
from
dasharo-paths-convention
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| # Dasharo Binaries Paths Convention | ||
|
|
||
| Dasharo firmware binaries are available to download on [dl.3mdeb.com][dl.3mdeb.com] | ||
| and [dlui.dasharo.com][dlui.dasharo.com]. These sources are being managed | ||
| - manually | ||
| - using semi-manual automation | ||
| - fully automated CI | ||
|
|
||
| The objective of this document is to describe a standard convention of the | ||
| directory tree and binary names used in Dasharo sources. It will make it easier | ||
| to navigate them and find the releases one is in interested in. | ||
|
|
||
| ## Directories Convention | ||
|
|
||
| The standardised path to a release directory is as follows: | ||
|
|
||
| `/<vendor>/<model>/<framework>[_payload]/<version>/` | ||
|
|
||
| where: | ||
| - `<vendor>` - hardware vendor, like `novacustom`, `protectli`, `hardkernel`, | ||
| `pcengines` | ||
| - `<model>` - Precise device model, like `v540tu`, `vp6670`, `odroid-h4`, | ||
| `apu4` | ||
| - `<framework>` - firmware framework, like `coreboot`, `slimbootloader`, `heads` | ||
| - `[_payload]` - optional, firmware payload, like `_uefi`, `_heads`, `_seabios` | ||
| - `<version>` - Dasharo version, like `v0.9.0`, `v1.7.2-rc1`, compliant with | ||
| the [versioning scheme](https://docs.dasharo.com/dev-proc/versioning/). | ||
|
|
||
| Examples: | ||
| - Dasharo (coreboot+Heads) Community Package for Novacustom V540TNx 14th | ||
| Gen laptops v1.0.0 | ||
| + (historically kept in one path due to compatibility): | ||
| + `/novacustom/v540tnd/coreboot_heads/1.0.0/` | ||
| + `/novacustom/v540tne/coreboot_heads/1.0.0/` | ||
| - Dasharo (coreboot+UEFI) Pro Package for Protectli V1000-series v0.9.3 | ||
| + (historically kept in two directories, a common `protectli_vault_jsl` | ||
| path, | ||
| and in more specific variants like `protectli_vault_jsl_v1210` due to | ||
| temporal compatibility): | ||
| + `/protectli/v1210/coreboot_uefi/v0.9.3/` | ||
| + `/protectli/v1410/coreboot_uefi/v0.9.3/` | ||
| + `/protectli/v1610/coreboot_uefi/v0.9.3/` | ||
| - Dasharo (coreboot+SeaBIOS) Community Package for PC Engines APU4 v24.08.00.01 | ||
| + (unique versioning scheme) | ||
| + `/pcengines/apu4/coreboot_seabios/v24.08.00.01/` | ||
|
|
||
| ### Motivation | ||
|
|
||
| #### <framework>[_payload] | ||
|
|
||
| Historically, the firmware framework and payload were generally not present in | ||
| the paths, which looked like `/<platform_family>/<version>/`. | ||
| When the support for new frameworks and payloads appeared, new directories | ||
| started to appear, like `/<platform_family>/heads/<version>/` or | ||
| `/<platform_family>/slimbootloader/<version>/` which made the paths | ||
| inconsistent with the older releases. | ||
|
|
||
| To fix this inconsistency issue the framework and (optional) payload will | ||
| always be there, even if a platform has only support for a single | ||
| framework/payload combination to reduce the ambiguity. | ||
|
|
||
| #### <vendor>/<model> | ||
|
|
||
| The idea to separate the `<platform_family>` into the vendor and specific models | ||
| has a couple reasons behind it: | ||
| 1. the number of combinations of vendor, microarchitecture and model family | ||
| resulted in too many directories on a single level | ||
| 1. knowing just a model name it could be not that obvious which family | ||
| it is supposed to belong to | ||
| 1. the firmware binaries for all the models from a family could be, or not be | ||
| compatible with the whole family, it could also change with releases and | ||
| require changing the directory structure | ||
| 1. it was not clear how to release the firmware for just one device from a | ||
| family | ||
|
|
||
| By separating the `vendor` and `model` into independent fields all the issues | ||
| above are solved. An additional bonus is that the user searching for the | ||
| binaries doesn't need to know technical details like the microarchitecture | ||
| of their CPU, just the name of the producer and the model of the device. | ||
|
|
||
| [dl.3mdeb.com]: https://dl.3mdeb.com | ||
| [dlui.dasharo.com]: https://dlui.dasharo.com | ||
|
|
||
| ## Binary Names Convention | ||
|
|
||
| To reduce the confusion and risk of mixing up binaries and allow for creating | ||
| multiple variants of a single binary without inconsistencies, the name of the | ||
| binary is also being standardised, and looks like: | ||
|
|
||
| `<vendor>_<model>_<framework>[_<payload>]_<version>[_extra].<extension>` | ||
|
|
||
| where: | ||
| - `<vendor>` - hardware vendor, like `novacustom`, `protectli`, `hardkernel`, | ||
| `pcengines` | ||
| - `<model>` - Precise device model, like `v540tu`, `vp6670`, `odroid-h4`, | ||
| `apu4` | ||
| - `<framework>` - firmware framework, like `coreboot`, `slimbootloader`, `heads` | ||
| - `[_payload]` - optional, firmware payload, like `_uefi`, `_heads`, `_seabios` | ||
| - `<version>` - Dasharo version, like `v0.9.0`, `v1.7.2-rc1` | ||
| - `[_extra]` - optional additional info, like `_dev_signed`, `_btg_provisioned`, | ||
| `_eom` | ||
| - `<extension>` - `rom` for normal binaries, `cap` for UEFI Capsules, `cab` for | ||
| FWUPD cabinets etc. | ||
pietrushnic marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| ### Motivation | ||
|
|
||
| This is especially important for the 3mdeb employees or users which flash their | ||
| devices often. Every binary should contain all the information required to | ||
| distinguish it from any other binary to reduce the risk of bricking the device | ||
| or other mishaps during flashing. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this have to be directly connected with naming convention or even be part of naming convention as part of subsection, otherwise it would be quickly out of sync
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be better if it never went out of sync by being as universal as possible. That's why I've been asking about creating more variants of Dasharo with the same framework+payload. A lot of projects (I've counted 10 repositories) depend on the directory structure and updating it will soon, and would in the future, be costly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's why such a scheme should be implemented programmatically (being derived from variables) instead of being hardcoded. Of course, wherever it makes sense.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
At the same time it should be compatile with humans too. People use dl.3mdeb.com or dlui.3mdeb.com. It should be easy to manually find what youre looking for there or just explore what's available while understanding what you're looking on
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, what would be your expectation?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. Exceptions appear from time to time. We are also not very strict about following our own naming rules when it comes to paths and files. E.g. MTL iGPU and dGPU was one of that cases where out of a sudden binaries changed their paths because nobody could distinguish dPGU from iGPU, despite it is unique (TNx is dGPU, TU is iGPU) - I was very dissatisfied with that. We often create mess ourselves when it comes to paths.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
To help with that issue I'm suggesting here to separate every unique model into its own directory. The binaries can, and should, be the same for compatible boards, but separating the models explicitly in the directory should help with similar scenarios in the future. It should be trivial to modify all the uploading scripts to upload a single binary into multiple directories if it works for more than one device model.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note that we rapidly increase the amount of artifacts. Capsules, raw binaries (those x2 because of dev_signed) and fwupd cabinets, also Boot Guard... it will mean that we need extra 100MB+ space per model for identical binaries if one binary supports multiple models. It will quickly get out of hand and deplete our storage pool. Links or redirects would be much better.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All our Dasharo releases as of today take ~ 20 GB
I'm not sure how dl.3mdeb.com and dlui.3mdeb.com are implemented, but if making symlinks would possible, then it sure would be good to save a couple gigabytes, why not.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should not optimize for size in the naming convention. If this becomes a problem, it can be resolved via other tools on higher layers.