Implement the PoW in the TLS protocol directly to avoid JS

[rodarima]

From Dillo and other browsers that don't support JS, we are starting to see some websites requesting JS to pass the Anubis test. I suppose they have very strict settings, as others work fine for now. However, I predict that in the near future, bots will adapt to mimic those browsers, leaving all those sites out for us. So I think that it would be convenient to have a better solution by then.

I would like to suggest doing a dynamic proof of work in the TLS protocol itself during the handshake procedure, so it is transparent for any client and works for browsers without JS. This way, we can avoid running a full blown VM to implement JavaScript just to access a mostly text document, and tools like curl or RSS readers can also work while the server is protected against massive bot campaigns.

The Tor network introduced a PoW in the past that may be suited to address this problem:

• https://blog.torproject.org/introducing-proof-of-work-defense-for-onion-services/
• https://spec.torproject.org/hspow-spec/v1-equix.html
• https://github.com/tevador/equix/blob/master/devlog.md

I'm not particularly versed in cryptoanalysis, but it seems to solve the same problem while preserving the anonnimity of clients. The idea is to serve puzzles with variable difficulty to the clients when the server is under stress. Puzzles solved with higher effort from the client, cause that request to be placed higher in the priority queue, so it is taken care earlier. Slower clients can still access the service, but they have to wait longer. When there is no stress, no puzzle is needed, so there is no need to waste CPU energy.

The PoW is designed to be hard to implement for GPUs and to allow changes in the future to prevent any ASIC investment to be profitable.

Failing TLS, the second best choice may be the HTTP protocol itself, but at that point you already had to go though the whole TLS handshake so you have less capability to protect yourself from bots.

I'm aware of other techniques like Privacy Pass, but I have several concerns about privacy, as they assume the different parties won't collude to try to track the users. I believe a dynamic PoW is a better option for now.

It would be probably a good idea to also involve other TLS library projects, but I just wanted to comment this here first to get some preliminary feedback.

[Xe]

Anubis doesn't really have access to the raw TLS stack, and this approach is incompatible with Google chrome. Is this meant to be more than aspirational?

[guest20]

Slower clients can still access the service, but they have to wait longer.

I have a slower device, so I'm already using less resources, and this algo punishes me harder because of it? That sounds like the opposite of what you'd want when dealing with high volume abusive bots.

[rodarima]

Anubis doesn't really have access to the raw TLS stack,

Not in its current form, but I suspect this could be done by adding a module in the HTTP server that can interact with the TLS session as well. As I comment, another option is to do this in the HTTP layer via HTTP headers, but at that point you have already spent quite a lot of CPU time doing the TLS handshake, so an attacker can DoS you easily. At the TLS level is probably enough to prevent your site from being attacked from AI bots, but also possibly from L7 DDoS attacks. Either two would work without requiring JS, but doing it in HTTP is probably much easier.

Notice that it is possible to make it optional, so that if the client doesn't support the challenge at TLS or HTTP it can still fallback to the JS challenge.

and this approach is incompatible with Google chrome.

Google Chrome uses BoringSSL which is a fork from OpenSSL, so it should be doable to implement if OpenSSL manages to do it. The HTTP approach is much easier to implement.

Is this meant to be more than aspirational?

Mostly to get some feedback on the idea. I'm not sure if the investment would make economic sense.

If you are interested, I could add support for client side HTTP mechanism in Dillo (I'm maintaining it), for TLS I'm not sure. I find it surprising that there is no RFC yet (or I didn't search well).

[otonoton]

Why not just implement a challenge that forces the user to wait a certain amount of time, without performing any work? The end result looks the same to the users, but without warming the planet at the same time.

[Xe]

Please read the docs: https://anubis.techaro.lol/docs/admin/configuration/challenges/metarefresh

[Xe]

@otonoton the meta refresh method makes the user wait a certain amount of time without performing any work.

[otonoton]

Yes but that requires serving an intermediary page and then having the client make another request... why even have that at all? You could just delay the initial response itself (of the main content the user is trying to access) by the same amount of time with less complexity, as you don't even need to calculate or verify any crypto challenges then. I realize some people might not want that, but I think it should probably be an option for those that want it.

Also, does the meta refresh option actually verify the URL wasn't accessed before the time is up? Otherwise clients could just follow the URL immediately to get around the challenge.