Research shipping GeoIP DB as asset files on mobile apps

[hellais]

This will allow us to move forward with the engine simplification work and enable us to not have to do app releases so often.

[sdsantos]

@hellais There should be nothing stopping us from shipping the GeoIP DB files as assets inside the probe-multiplatform app. The main changes would be in the engine I imagine. We just need to provide the engine with a path or input stream to the files.

But that wouldn't stop us from releasing the app less often. Maybe it would help to release updates to the engine less often.

To have fewer app releases, we would need to not only ship the GeoIP DB files, but also update them remotely from the apps.

[sdsantos]

Investigated a bit more. Android can't provide a file path from inside the APK file, while iOS can. Android can share a java.io.FileDescriptor.

Ideas:
a) Share a FileDescriptor and somehow implement that for iOS;
b) Share a common interface for how to read the file chunks of bytes (similar to an InputStream);
c) We copy the file to the app files directory and then provide a path (space wasted);
d) If the file already needs to be extracted someway by the engine, the app can do it instead, and then provide a path(s) to the extracted contents;
e) We read and hold the contents in memory, and pass that to the engine (I imagine this isn't feasible because they are large files).

[hellais]

Thanks for this research. The GeoIP files are not really that large (~8MB compressed: https://archive.org/download/ip2country-as), so in theory we could even load them all in memory and pass the bytes to the engine for it to handle the lookup. We should consider how this plays into the new engine API and if it's even needed to do so or if the geoip lookup could/should be done by the apps instead and we just pass the probe_cc, probe_asn to the engine.