Reducing dependencies

[tamasgal]

We currently have a couple of dependencies in UnROOT.jl

AbstractTrees = "1520ce14-60c1-5f80-bbc7-55ef81b5835c"
Accessors = "7d9f7c33-5ae7-4f3b-8dc6-eff91059b697"
ArraysOfArrays = "65a8f2f4-9b39-5baf-92e2-a9cc46fdf018"
BitIntegers = "c3b6d118-76ef-56ca-8cc7-ebb389d030a1"
CodecLz4 = "5ba52731-8f18-5e0d-9241-30f10d1ec561"
CodecXz = "ba30903b-d9e8-5048-a5ec-d1f5b0d4b47b"
CodecZstd = "6b39b394-51ab-5f42-8807-6242bab2b4c2"
FHist = "68837c9b-b678-4cd5-9925-8a54edc8f695"
HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
IterTools = "c8e1da08-722c-5040-9ed9-7db0dc04731e"
LRUCache = "8ac3fa9e-de4c-5943-b1dc-09c6b5f20637"
LibDeflate = "9255714d-24a7-4b30-8ea3-d46a97f7e13b"
LorentzVectors = "3f54b04b-17fc-5cd4-9758-90c048d965e3"
Memoization = "6fafb56a-5788-4b4e-91ca-c0cea6611c73"
Mixers = "2a8e4939-dab8-5edc-8f64-72a8776f13de"
Mmap = "a63ad114-7e13-5084-954f-fe012c677804"
Parameters = "d96e819e-fc66-5662-9728-84c9c7592b0a"
PrecompileTools = "aea7be01-6a6a-4083-8856-8a6e6704d82a"
PrettyTables = "08abe8d2-0d0c-5749-adfa-8a2ac140af0d"
SentinelArrays = "91c51154-3ec4-41a3-a24f-3f23e20d615c"
StaticArrays = "90137ffa-7385-5640-81b9-e52037218182"
StructArrays = "09ab397b-f2b6-538f-b94a-2f83cf4a842a"
TOML = "fa267f1f-6049-4f14-aa54-33bafae1ed76"
Tables = "bd369af6-aec1-5ad0-b16a-f7cc5008161c"
XRootD = "164e3b87-dc46-4cbc-97a6-5b310108fce0"
XXHashNative = "e5d8e439-e7fa-4681-9c12-1c64bda517be"

and some of them are pulling a quite a lot of dependencies.
I think we could make a small inventory of packages which are not required very often and put those into extensions.

HTTP.jl, XRootD.jl, maybe even LorentzVectors are a few candidates I guess?

What do you think?

Other small packages could also be directly integrated into UnROOT.jl by copying the relevant code (since almost everything is MIT, that should be fine).

[Moelf]

are those even large or slow to load?

removing function is gonna be a breaking change

[tamasgal]

Mostly thinking about reducing dependencies, to be able to make UnROOT more "independent" and easier to maintain in future. So it's not about performance ;)

I find more and more problems with dependencies which are blocked due to 0.x updates which are not propagated to the compat entries. I will try to come up with some more specific proposals :)

removing function is gonna be a breaking change

Yes you are of course right. In the first step I'd definitely go without any breaking changes but maybe for version 1.0 we can think about a more lightweight package which uses extensions for e.g. XRootD/HTTP stuff?

[tamasgal]

I will probably start to work in this in the next weeks. Currently a bunch of users are unable to use UnROOT because CxxWrap and XRootD fail on macOS and Julia 1.12. This is really frustrating :(

Binary dependencies will always be the biggest weak points...

[oschulz]

Can we make XRootD a weak dependency?

[tamasgal]

Yes, my plan is to make it optional as an extension. Whenever XRootD is imported, the functionality will be available. I'd do the same with HTTP and maybe other stuff and that will be version 1.0 I guess. I really want to keep this library slim and maintainable for a long time. It happened way too often that this package could not be installed due to upstream dependency problems :(

[oschulz]

I agree, the more can be move into extensions, the more robust.

[tamasgal]

I wish we only had C (C99) and Julia... Who needs other languages 🤣

[Moelf]

Can we make removing XRootD by itself a release, 0.11 I guess?

I would like RNTuple writing to be part of UnROOT v1.0. And we also wanted the sink= for the reading API.

[tamasgal]

Alright, v0.11 it is with removed XRootD.jl and HTTP.jl deps in favour of extensions, see 203a3e2 and #396

[tamasgal]

Next on the list are

Memoization = "6fafb56a-5788-4b4e-91ca-c0cea6611c73"
Mixers = "2a8e4939-dab8-5edc-8f64-72a8776f13de"

😉

[tamasgal]

Current state:

Julia 1.11

v0.10.38

Installing:

97 deps
78 seconds

very first using

  2.958249 seconds (2.54 M allocations: 144.313 MiB, 3.95% gc time, 6.60% compilation time: 11% of which was recompilation)

v0.11.0

Installing:

74 deps
53 seconds

very first using

  0.762496 seconds (823.62 k allocations: 52.112 MiB, 13.86% gc time, 11.01% compilation time: 28% of which was recompilation)

[Moelf]

very nice

[tamasgal]

Mixers and Parameters are gone. Loading time is not better but the deps are decreasing 😉