Switch from websocket-star to the rendezvous protocol

[mkg20001]

Currently peer-pad uses the websocket-star protocol.
I would like to help with switching it over to the rendezvous protocol.

This issue is currently blocked by the completion of the javascript implementation of the libp2p rendezvous protocol

[pgte]

@mkg20001 could you, in a gist, give us the advantages of the rendezvous protocol over using ws-star?
Could you also list the issues this depends on?
Thank you!

[mkg20001]

Comparison of both protocols: https://gist.github.com/mkg20001/7328e2ee1851adc2467911940983e4ec

Currently the only blocking issue is this pr but @victorbjelkholm seems to be working on a less-complicated version of this module.

[pgte]

Thank you Maciej for explaining! (when I said a gist, I meant it in the figurative sense, not necessarily a GitHub gist, but this works great!)

If I understand correctly, by PeerPad using this protocol, we can spread the load between several rendez-vous servers and this won't limit visibility of peers, as they still will be able to find each-other.

Somewhat related is the one problem we're I'm trying to solve here (and in peer-star in general) is to prevent every peer to connect to every peer. While this has allowed us to do quick peer discovery for PeerPad, it's not particularly scalable, as a connection is something expensive.

Right now, IPFS connects to every peer it discovers. Do you see an IPFS configuration that would allow us, by using this protocol, to have better scalability at the document level?

[mkg20001]

If I understand correctly, by PeerPad using this protocol, we can spread the load between several rendezvous servers and this won't limit visibility of peers, as they still will be able to find each-other.

Yes, although currently that has some limitations. If peer A and peer B announce register at rendezvous 1 only and peer C registers at rendezvous 2 only peer A/B and C will never see each other.

The planned solution for this is to use federation to exchange the registrations across multiple rendezvous servers.

Right now, IPFS connects to every peer it discovers. Do you see an IPFS configuration that would allow us, by using this protocol, to have better scalability at the document level?

Edit: This could be achieved by disabling discovery for the global namespace (which should be possible after the interface change) and then only registering to a specific namespace such as pad-123

Currently there is no example code as the libp2p interface-discovery would have to be changed to support registering/unregistering to specific namespaces.
There is some discussion going on in this issue about how the interface should be changed.

[pgte]

@mkg20001 thank you, this sounds very much like something very beneficial. Please do keep us posted on the evolution of this transport.

[mkg20001]

I created a WIP PR on peer-pad-core: ipfs-shipyard/peer-pad-core#19

[pgte]

According to our scaling requirements, we need 150 concurrent users accessing the same pad. Some back-of-the-napkin math tells me that, to endure that, each node will be connected to 15 other nodes, sending a broadcast message every 500ms.

@mkg20001 Is it possible to do some load tests to see how far off we are from scaling to that?

[mkg20001]

@pgte The rendezvous-protocol does not handle the data transfer, that is done by circuit. For the discovery part: The discovery interval could easily be changed to 500ms (from currently 10s) to enable faster discovery (also once a node is registered it does not need to re-announce itself until the TTL for the announcement expires, so there is no extra load for that).

Edit: tl;dr with a fast p2p-circuit relay server this could be done.

[pgte]

@mkg20001 this broadcast message is not for discovery, it's an estimate of the messages deriving from the operations. What do we need to scale to that?

[pgte]

@mkg20001 ping :)

[mkg20001]

@pgte If you're talking about transferring data then likely fast p2p-circuit relays and/or webrtc instead of p2p-circuit, otherwise I might not have gotten correctly what you were asking for.

[pgte]

@mkg20001 Not sure. What I'm asking is: could you please create some load tests to make sure we can endure the load when the time comes?

[pgte]

This would be a load test that connects many different peers to the same server, with them randomly connecting to each other (15 connections each), and sending messages to each other.
Then we would need to somehow, at each peer, measure message delivery and latency. At the end of the test, aggregate all the data and display it.
@mkg20001 do you think you could do something like this?