feat: Implementation Paths V2 #2032

Dagmar-Dinjens · 2026-02-04T09:49:05Z

Description

First implementation for issue #1814 asking for a paths.json version 2 support

Implementing deserialisation for the proposed paths version 2, would love to hear some feedback where possible.

What happened

I added the field for a file being executable but did not implement any implementations with this.
I also added the offsets field for which I implemented new prefix replacement functions

What should still be done/ next steps

A next step would be the implementation of version 2 paths.json when building a package with rattler-build & a possible CEP

baszalmstra

I think this is going in the right direction.

I do think we can make this change backwards compatible. Now it isnt because the version in paths.json is bumped. This means that other installers will most likely fail with these packages. That will hamper adoption.

Instead perhaps we can keep the version the same. The offsets field is already backwards compatible. But the executable flag is not because we dont know if the absence of the field means nothing is executable or it was not written. Maybe instead we can add another field to the top-level keys to indicate that the executable field should be taken into account.

baszalmstra · 2026-02-04T10:00:42Z

crates/rattler/src/install/link.rs

+                    &target_prefix,
+                    &target_platform,
+                    *file_mode,
+                    offsets.to_vec()


Cant we keep this as a slice instead of allocating on the heap?

Yes! that seems like a way prettier solution indeed

baszalmstra · 2026-02-04T10:05:28Z

crates/rattler_cache/src/validation.rs

    MetadataMissing,

+    /// An error occurred with the version of the `path.json` file
+    #[error("")]


Could you improve this error?

I did improve it but think I am not using it anywhere, Depending on whatever leads out of the v1/v2 comment you left before I might even remove it

baszalmstra · 2026-02-04T10:06:38Z

crates/rattler_conda_types/src/package/paths.rs

+    /// When a file is executable this will be true
+    /// This entry is only present in version 2 of the paths.json file
+    #[serde(default, skip_serializing_if = "Option::is_none")]
+    pub executable: Option<bool>,


You did not implement this yet right?

I did not really understand what improvements were the goal in adding the executability to the paths, because of this I didn't yet implement them no

Makes sense. Let me explain. At the moment the code has to acquire the metadata in the cache to determine if the file has the executable permissions set. If so, then the file also needs to have it set in the environment. However, this metadata call performs a syscall which adds a bit of overhead. Using the executable flag in the paths.json we can skip this metadata check.

makes a lot more sense now! I think I now implemented this with the new commit

Not entirely. What we try to avoid is the have to query the file metadata all together. You currently implementation still requests this regardless of whether executable was set.

But what Im trying to avoid is having to bump the version field in paths.json. Because doing so will break installers that aren't compatible with this version (e.g. conda and mamba). We need to make sure that adding the executable field is a backwards compatible change. Simply adding a field is backwards compatible but then if the field is optional we dont know if the absence of the fields means either the value is false or that the value was not computed in the first place.

To fix that we could add another top-level key (next to version) that indicates that the absence of the executable field means it should be considered false. Then, if that top-level field is present we no longer have to read the metadata from disk and we can rely on the per-file executable field.

Ah correct, I hope that the current adaptation is what you meant!
I was still retrieving the metadata because it seems to be needed for the file permissions and timestamps,
should there be another solution for retrieving this metadata or does this mean we keep the current approach?

baszalmstra · 2026-02-04T10:23:15Z

crates/rattler/src/install/link.rs

+        let old_len = segment.len();
+
+        let mut out = Vec::with_capacity(old_len);
+        let mut remaining = segment;


I feel like this inner loop could be simplified. The code know exactly where the offsets are and how long they are. I dont think we need the finder or the Vec. This seems very much like the code from the previous implementation, I encourage you to rethink it.

I did indeed not push that which I had been working on, in the new commit I did.

baszalmstra · 2026-02-04T10:24:17Z

crates/rattler/src/install/link.rs

+    if offsets.is_empty() {
+        return destination.write_all(source_bytes);
+    }


This seems redundant. If the offsets is empty, the loop will have no iterations and at the end of the function the rest of the bytes will be written in a similar fashion.

You're right!

improved permission checking, improved error message, improved slice usage in link.rs, Still have to look at the v2 backwards incompatibility comment

Dagmar-Dinjens · 2026-02-04T15:15:54Z

Instead perhaps we can keep the version the same. The offsets field is already backwards compatible. But the executable flag is not because we dont know if the absence of the field means nothing is executable or it was not written. Maybe instead we can add another field to the top-level keys to indicate that the executable field should be taken into account.

The absence of the field would inherently mean it was not written otherwise it would read as true or false, I guess you assumed that it would, like the prefix_placeholder, be left empty if false.

this currently isn't the case but something like the following would be the preference?

"paths": [
                {
                    "_path": "bin/example",
                    "no_link": false,
                    "path_type": "hardlink",
                    "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
                    "size_in_bytes": 1024,
                    "executable": true,
                    "file_mode": "binary",
                    "prefix_placeholder": "/opt/conda",
                    "offsets": [100, 200, 300]
                },
                {
                    "_path": "lib/library.so",
                    "no_link": false,
                    "path_type": "hardlink",
                    "sha256": "d7a8fbb307d7809469ca9abcb0082e4f8d5651e46d3cdb762d02d0bf37c9e592",
                    "size_in_bytes": 2048
                },
                {
                    "_path": "share/doc/readme.txt",
                    "no_link": false,
                    "path_type": "hardlink",
                    "sha256": "a665a45920422f9d417e4867efdc4fb8a04a1f3fff1fa07e998e86f7f7a27ae3",
                    "size_in_bytes": 256,
                    "file_mode": "text",
                    "prefix_placeholder": "/home/builder/conda",
                    "offsets": [10, 45]
                },
                {
                    "_path": "bin/symlink-example",
                    "no_link": false,
                    "path_type": "softlink",
                    "executable": true
                }
            ],
            "has_executable": true,
            "paths_version": 1
            }"

DagmarFiep added 2 commits February 4, 2026 10:24

paths version 2 support

1a2da28

Merge branch 'main' of https://github.com/conda/rattler

b3e622a

Dagmar-Dinjens changed the title ~~Implementation Paths V2~~ FEAT: Implementation Paths V2 Feb 4, 2026

Dagmar-Dinjens changed the title ~~FEAT: Implementation Paths V2~~ fest: Implementation Paths V2 Feb 4, 2026

Dagmar-Dinjens changed the title ~~fest: Implementation Paths V2~~ feat: Implementation Paths V2 Feb 4, 2026

fixing lint error's

20fddbc

baszalmstra reviewed Feb 4, 2026

View reviewed changes

improved c-string function,

7023e68

improved permission checking, improved error message, improved slice usage in link.rs, Still have to look at the v2 backwards incompatibility comment

Dagmar-Dinjens mentioned this pull request Feb 4, 2026

Paths.json should store placeholder offset and file permissions (executable) #1814

Open

2 tasks

DagmarFiep and others added 5 commits February 5, 2026 13:55

fixed some warnings & the python lint run error's

363ac1b

Merge branch 'main' into main

03591ad

implemented keeping version 1 and the executable field

c5d2e52

Merge branch 'main' of https://github.com/Dagmar-Dinjens/rattler

bf78570

ran 'pixi run lint'

2b4e33d

feat: Implementation Paths V2 #2032

Are you sure you want to change the base?

feat: Implementation Paths V2 #2032

Uh oh!

Conversation

Dagmar-Dinjens commented Feb 4, 2026

Description

What happened

What should still be done/ next steps

Uh oh!

baszalmstra left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Dagmar-Dinjens commented Feb 4, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants