Expose new libzim settings to handle memory consumption

[benoit74]

openzim/libzim#950 moved some settings from env parameters to CPP exposed settings. This has been released in libzim 9.3.0.

openzim/libzim#960, openzim/libzim#974, openzim/libzim#975 and openzim/libzim#982 pushed this further by allowing to configure the memory we want to consume. This is planned to be has been released in libzim 9.4.0.

These settings should be exposed in kiwix-tools as well.

Should we release kiwix-tools without this issue handled, it means that we will introduce a kind of regression since environment variables available in the past (and used in our infra) are not here anymore, and there is no new way to do the same thing.

This is a companion issue of kiwix/libkiwix#1199

[benoit74]

@kelson42 the fact this has been pushed back to 3.9.0 is a big issue.

We cannot configure kiwix-serve behavior with environment variables as in 3.7 because it has been removed from libzim and no alternative has been implemented in kiwix-serve to use "new" settings introduced in libzim 9.3 and 9.4. We now need to hope that default libzim settings are correct ... which cannot be true for both hotspots (very limited memory footprint expected) and library.kiwix.org (more memory available, more ZIMs diversity).

[veloman-yunkan]

There are alternative ways to provide user control over memory consumption. The main question before I enumerate possible options, is this - should it be static (fixed at application start-up time) or dynamic (possible to adjust while the application is running)?

[veloman-yunkan]

In general settings can be configured via

1. command line options (static by the method design itself)
2. environment variables (static by system spec, for known OSes)
3. configuration files (can be implemented to work in dynamic mode)

Of course, any combination of above methods can be supported with any order of precedence.

[veloman-yunkan]

From the implementation point of view, controlling those settings via command line options will require enhancing every tool/application separately. The other two methods allow for a centralized solution in libzim or libkiwix where the library is initialized automatically by reading its settings from the environment and/or the configuration file.