Skip to content

v1.21.0: Minfilia Warde

Latest
Latest
Compare
Choose a tag to compare
@Xe Xe released this 17 Jul 01:25
· 1 commit to main since this release
v1.21.0
4d6b578
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Please, be at ease. You are among friends here.

In this release, Anubis becomes internationalized, gains the ability to use system load as input to issuing challenges, finally fixes the "invalid response" after "success" bug, and more! Please read these notes before upgrading as the changes are big enough that administrators should take action to ensure that the upgrade goes smoothly.

Big ticket changes

The biggest change is that the "invalid response" after "success" bug is now finally fixed for good by totally rewriting how Anubis' challenge issuance flow works. Instead of generating challenge strings from request metadata (under the assumption that the values being compared against are stable), Anubis now generates random data for each challenge. This data is stored in the active storage backend for up to 30 minutes. This also fixes #746 and other similar instances of this issue.

In order to reduce confusion, the "Success" interstitial that shows up when you pass a proof of work challenge has been removed.

Storage

Anubis now is able to store things persistently in memory, on the disk, or in Valkey (this includes other compatible software). By default Anubis uses the in-memory backend. If you have an environment with mutable storage (even if it is temporary), be sure to configure the bbolt storage backend.

Localization

Anubis now supports localized responses. Locales can be added in lib/localization/locales/. This release includes support for the following languages:

If facts or local regulations demand, you can set Anubis default language with the FORCED_LANGUAGE environment variable or the --forced-language command line argument:

FORCED_LANGUAGE=de

Load average

Anubis can dynamically take action based on the system load average, allowing you to write rules like this:

## System load based checks.
# If the system is under high load for the last minute, add weight.
- name: high-load-average
  action: WEIGH
  expression: load_1m >= 10.0 # make sure to end the load comparison in a .0
  weight:
    adjust: 20

# If it is not for the last 15 minutes, remove weight.
- name: low-load-average
  action: WEIGH
  expression: load_15m <= 4.0 # make sure to end the load comparison in a .0
  weight:
    adjust: -10

Something to keep in mind about system load average is that it is not aware of the number of cores the system has. If you have a 16 core system that has 16 processes running but none of them is hogging the CPU, then you will get a load average below 16. If you are in doubt, make your "high load" metric at least two times the number of CPU cores and your "low load" metric at least half of the number of CPU cores. For example:

Kind Core count Load threshold
high load 4 8.0
low load 4 2.0
high load 16 32.0
low load 16 8

Also keep in mind that this does not account for other kinds of latency like I/O latency. A system can have its web applications unresponsive due to high latency from a MySQL server but still have that web application server report a load near or at zero.

Other features and fixes

There are a bunch of other assorted features and fixes too:

  • Add COOKIE_SECURE option to set the cookie Secure flag
  • Sets cookie defaults to use SameSite: None
  • Determine the BIND_NETWORK/--bind-network value from the bind address (#677).
  • Implement a development container manifest to make contributions easier.
  • Fix dynamic cookie domains functionality (#731)
  • Add option for custom cookie prefix (#732)
  • Make the Open Graph subsystem and DNSBL subsystem use storage backends instead of storing everything in memory by default.
  • Allow Common Crawl by default so scrapers have less incentive to scrape
  • The bbolt storage backend now runs its cleanup every hour instead of every five minutes.
  • Don't block Anubis starting up if Thoth health checks fail.
  • A race condition involving opening two challenge pages at once in different tabs causing one of them to fail has been fixed.
  • The "Try again" button on the error page has been fixed. Previously it meant "try the solution again" instead of "try the challenge again".
  • In certain cases, a user could be stuck with a test cookie that is invalid, locking them out of the service for up to half an hour. This has been fixed with better validation of this case and clearing the cookie.
  • Start exposing JA4H fingerprints for later use in CEL expressions.
  • Add /healthz route for use in platform-based health checks.

Potentially breaking changes

We try to introduce breaking changes as much as possible, but these are the changes that may be relevant for you as an administrator:

Challenge format change

Previously Anubis did no accounting for challenges that it issued. This means that if Anubis restarted during a client, the client would be able to proceed once Anubis came back online.

During the upgrade to v1.21.0 and when v1.21.0 (or later) restarts with the in-memory storage backend, you may see a higher rate of failed challenges than normal. If this persists beyond a few minutes, open an issue.

If you are using the in-memory storage backend, please consider using a different storage backend.

Systemd service changes

The following potentially breaking change applies to native installs with systemd only:

Each instance of systemd service template now has a unique RuntimeDirectory, as opposed to each instance of the service sharing a RuntimeDirectory. This change was made to avoid the RuntimeDirectory getting nuked any time one of the Anubis instances restarts.

If you configured Anubis' unix sockets to listen on /run/anubis/foo.sock for instance anubis@foo, you will need to configure Anubis to listen on /run/anubis/foo/foo.sock and additionally configure your HTTP load balancer as appropriate.

If you need the legacy behaviour, install this systemd unit dropin:

# /etc/systemd/system/[email protected]/50-runtimedir.conf
[Service]
RuntimeDirectory=anubis

Just keep in mind that this will cause problems when Anubis restarts.

What's Changed

  • feat: implement localization system by @lolgzs in #716
  • fix: determine bind network from bind address by @littlecxm in #714
  • Add Brazilian Portuguese translation by @rffontenelle in #726
  • fix: Dynamic cookie domain not working by @Earl0fPudding in #731
  • feat(cmd): Add custom cookie prefix by @Earl0fPudding in #732
  • build(deps): bump the github-actions group with 2 updates by @dependabot[bot] in #735
  • build(deps): bump the gomod group with 2 updates by @dependabot[bot] in #736
  • feat: dev container support by @Xe in #734
  • Fix translations in pt-BR.json by @rffontenelle in #729
  • Set cookies to have the Secure flag default to true by @victorvalenca in #739
  • fix(web/main): remove the success interstitial by @Xe in #745
  • feat(localization): Add option for forcing a language by @Earl0fPudding in #742
  • fix(run/[email protected]): unique runtimedir per instance by @Xe in #750
  • feat(localization): Add German language translation by @Earl0fPudding in #741
  • docs: add BotStopper docs from the git repo by @Xe in #752
  • chore(default-config): allowlist common crawl by @Xe in #753
  • feat(localization): Add Turkish language translation by @dcelasun in #751
  • docs(known-instances): add ebird.org by @SGHFan in #755
  • feat(lib): use new challenge creation flow by @Xe in #749
  • chore(devcontainer): move playwright to its own devcontainer service by @Xe in #756
  • docs(known-instances): Add Duke University, coinhoards.org (and myself) to known instances by @lotharsm in #757
  • fix: make ogtags and dnsbl use the Store instead of memory by @Xe in #760
  • fix(lib/store/bbolt): use a multi-bucket flow instead of a single bucket flow by @Xe in #761
  • fix(lib/store/bbolt): run cleanup every hour instead of every 5 minutes by @Xe in #762
  • docs: remove proof of work branding by @Xe in #763
  • feat(localization): Update German language translation by @lotharsm in #764
  • docs(known-instances): update list of known instances by @lotharsm in #767
  • feat(localization): Add Traditional Chinese language translation by @xlionjuan in #759
  • feat(lib/policy/expressions): add system load average to bot expression inputs by @Xe in #766
  • build(deps): bump the github-actions group with 2 updates by @dependabot[bot] in #770
  • build(deps): bump github.com/shirou/gopsutil/v4 from 4.25.1 to 4.25.6 in the gomod group by @dependabot[bot] in #771
  • minor typo fix: Update apache.mdx replace nginx with Apache in place by @mihugo in #779
  • docs(known-instances): update list of known instances by @lotharsm in #776
  • feat(localization): add Simplified Chinese by @littlecxm in #774
  • docs(installation): Clarify information about private keys and multile instances by @StandingPadAnimations in #788
  • fix(localization): HTML language header and forced-language by @SlyEcho in #787
  • Update apache.mdx by @jzb in #784
  • feat(localization): add Japanese language translation by @dai in #772
  • feat(i18n): add Estonian locale by @SlyEcho in #783
  • Create is.json by @sveinki in #780
  • feat(localization): Add Italian language translation by @giomba in #778
  • feat(localization): Add Filipino language by @searinminecraft in #775
  • fix(internal/thoth): don't block Anubis starting if healthcheck fails by @Xe in #794
  • feat(blog): incident report for TI-20250709-0001 by @Xe in #795
  • chore: use nginx-micro to make the docs image 13 MB by @Xe in #796
  • docs: update CHANGELOG for language changes by @Xe in #793
  • docs(known-instances): update list of known instances by @lotharsm in #801
  • correct gitea.botPolicies extension to be yaml, not json by @evgeni in #800
  • docs(known-instances): add rpmfusion.org and wiki.freepascal.org to known instances by @lotharsm in #807
  • chore(docs): fix typo in configuration/expressions by @maximelouet in #811
  • fix(index.templ) centered-div class usage typo by @ciencia in #812
  • chore(docs): add link to status page in the footer by @Xe in #814
  • chore: release v1.21.0-pre2 by @Xe in #816
  • build(deps): bump the gomod group with 3 updates by @dependabot[bot] in #823
  • build(deps-dev): bump esbuild from 0.25.5 to 0.25.6 in the npm group by @dependabot[bot] in #825
  • fix(default-config): disable system load check by default by @Xe in #827
  • test: add smoke test for git clone by @Xe in #828
  • test: add git push smoke test by @Xe in #830
  • ci(docs): make a new docker image for the docs per commit sha by @Xe in #831
  • fix: race conditions, cookie logic, and the try again button by @Xe in #833
  • feat(cmd/anubis): capture ja4h fingerprints by @Xe in #834
  • feat: add Uptime Robot policy definition by @herrbischoff in #838
  • fix(localization): fix missing string in template by @littlecxm in #835
  • feat(anubis): add /healthz route to metrics server by @Xe in #843
  • chore: release v1.21.0 by @Xe in #844

New Contributors

Full Changelog: v1.20.0...v1.21.0

Contributors

dai, jzb, and 21 other contributors
Assets 30
Loading
0 Join discussion