Support for unix sockets

[nnuuvv]

By the looks of it erlang's httpc supports unix sockets, gleam_httpc doesn't.
adding support shouldn't be too much effort; in theory.

HTTPC docs

[nnuuvv]

So i've been fiddling with it for the past few hours.

These are the erl_options i get:

And erl_url

But i get this error

I've done some manual testing in an erlang shell to figure out whats going on.
If i specify the needed options in httpc:request i get the same old error.

But if i specify them the way the HTTPC docs do, using httpc:set_options;

i get data from it:

I think i may be running into why unix sockets in httpc are still (experimental)

[nnuuvv]

How to replicate:

Prereqs:

• Some erlang shell
• Up to date docker install

In open erlang shell:

1. start inets:

inets:start().

2. try to request data from /var/run/docker.sock/container/json (io:format added for readability)

io:format("~p~n", [httpc:request(get, {"http:/v1.49/containers/json", []}, [], [{socket_opts, [{ipfamily, local}, {unix_socket, "/var/run/docker.sock"}]}])]).

Output looks something like this:

3. set the options using set_options:

httpc:set_options([{ipfamily, local}, {unix_socket, "/var/run/docker.sock"}]).

4. Then run the same request, but without specifying the options:

io:format("~p~n", [httpc:request(get, {"http:/v1.49/containers/json", []}, [], [])]).

Yields a response like this:

(An empty list, since i don't have any docker containers running on this pc)

Footnote

• Re-running command 2. after setting the options using 3. still yields the same response.
• http:/v1.49/containers/json and http:///v1.49/containers/json work the same.
  Can be confirmed with curl:

  curl --unix-socket /var/run/docker.sock http:/v1.49/containers/json
  

  curl --unix-socket /var/run/docker.sock http:///v1.49/containers/json
  

• Im on Erlang/OTP 27

[lpil]

I've just had a quick glance at the HTTPC documentation and I believe it says that that unix_socket is a HTTPC option, but you've given it as a socket option.

Here's it being used:

https://github.com/erlang/otp/blob/ab9a8fee111ee1b16b6b620136379aaece6a29c3/lib/inets/src/http_client/httpc.erl#L1187

[nnuuvv]

so something like this?

io:format("~p~n", [httpc:request(get, {"http:/v1.49/containers/json", []}, [], [{unix_socket, "/var/run/docker.sock"}, {socket_opts, [{ipfamily, local}]}])]).

Based on the implementation, from what i can tell, it should be that; but i get this error when trying that:

And the docs

Here's that part of the docs (saw you talking about it not loading for you in discord)

[nnuuvv]

Looks to me like it just doesn't allow directly specifying unix_socket in request/4 atm

Here is the request/4 implementation: https://github.com/erlang/otp/blob/ab9a8fee111ee1b16b6b620136379aaece6a29c3/lib/inets/src/http_client/httpc.erl#L221

[lpil]

Looks like this is currently not possible to implement.

[nnuuvv]

I assume using httpc:set_options to set unix_socket, doing the request, and then setting it back to the default undefined isn't an option either?

Maybe in a dedicated(per unix_socket) Profile?

Something like this:

inets:start(httpc, [{profile, var_run_docker}]).

httpc:set_options([{ipfamily, local}, {unix_socket, "/var/run/docker.sock"}], var_run_docker).

httpc:request(get, {"http:/v1.49/containers/json", []}, [], [], var_run_docker).

inets:stop(httpc, var_run_docker).

[lpil]

I assume using httpc:set_options to set unix_socket, doing the request, and then setting it back to the default undefined isn't an option either?

No, that would only work in a language that doesn't have any concurrency.

Maybe in a dedicated(per unix_socket) Profile?

Possibly, but I'm not sure how that would work. inets will already be started by the time that the HTTP request is made. Can profiles be added at runtime? If so, how do we garbage collect them?

Will OTP28 or OTP29 support per-request configuration for unix sockets?

[nnuuvv]

Can profiles be added at runtime?

Yes. inets:start(httpc, some_name). As per the docs; Dynamically starts an Inets service after the Inets application has been started.

how do we garbage collect them?

I was thinking it could setup a profile(start an httpc process) per request i.e. start, set_options, request, stop
With a random / time based seed in the name to make sure they are unique.

From what i can tell OTP28 wont, don't know about 29.

[lpil]

Would the new inets instance be shut down after the request has completed? How would we avoid leaking these instances if the process making the request fails?

Starting and stopping an entire new application instance sounds like a large cost, and also strange code in general. I'm interested in what the intended way to use the new unix socket functionality is. It seems like we're guessing currently, and anything we do here needs to be suitable for very heavy production usage for years or even decades of Gleam programming.

[nnuuvv]

It seems like we're guessing currently

I am, very much so, throwing around ideas. I have basically no experience with erlang, barely any with OTP.

I don't know what the 'gleam' approach would be for this kinda thing.
I think ideally the usage shouldn't differ between unix sockets and inet.

But, since we do need to keep track of the httpc instance, for now;
And since inets:start/2 returns a pid, which also function as the Profile, we could return that pid to the caller.

Would look something like this:

Allowing the httpc process to be reused when necessary, and disposed off otherwise.

[lpil]

Seeing as this is a core library we have to be really conservative with the scope and the design, so I think this would be more suited to another library, at least until the Erlang API has stabilised and we understand how to use it.

[nnuuvv]

Alrighty, i guess i'll just use shellout and curl --unix-socket in my project for now