You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(lib/policy/expressions): add system load average to bot expression inputs (#766)
* feat(lib/policy/expressions): add system load average to bot expression inputs
This lets Anubis dynamically react to system load in order to
increase and decrease the required level of scrutiny. High load? More
scrutiny required. Low load? Less scrutiny required.
* docs: spell system correctly
Signed-off-by: Xe Iaso <[email protected]>
* Update metadata
check-spelling run (pull_request) for Xe/load-average
Signed-off-by: check-spelling-bot <[email protected]>
on-behalf-of: @check-spelling <[email protected]>
* fix(default-config): don't enable low load average feature by default
Signed-off-by: Xe Iaso <[email protected]>
---------
Signed-off-by: Xe Iaso <[email protected]>
Signed-off-by: check-spelling-bot <[email protected]>
Signed-off-by: Xe Iaso <[email protected]>
Copy file name to clipboardExpand all lines: docs/docs/CHANGELOG.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -29,6 +29,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
29
29
- Add translation for Turkish language ([#751](https://github.com/TecharoHQ/anubis/pull/751))
30
30
- Allow [Common Crawl](https://commoncrawl.org/) by default so scrapers have less incentive to scrape
31
31
- The [bbolt storage backend](./admin/policies.mdx#bbolt) now runs its cleanup every hour instead of every five minutes.
32
+
- Added the ability for Anubis to dynamically take action [based on the system load average](./admin/configuration/expressions.mdx#using-the-system-load-average).
32
33
- Add translation for Traditional Chinese ([#759](https://github.com/TecharoHQ/anubis/pull/759))
| `headers` | `map[string, string]` | The [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) of the request being processed. | `{"User-Agent": "Mozilla/5.0 Gecko/20100101 Firefox/137.0"}` |
105
-
| `host` | `string` | The [HTTP hostname](https://web.dev/articles/url-parts#host) the request is targeted to. | `anubis.techaro.lol` |
106
-
| `method` | `string` | The [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods) in the request being processed. | `GET`, `POST`, `DELETE`, etc. |
107
-
| `path` | `string` | The [path](https://web.dev/articles/url-parts#pathname) of the request being processed. | `/`, `/api/memes/create` |
108
-
| `query` | `map[string, string]` | The [query parameters](https://web.dev/articles/url-parts#query) of the request being processed. | `?foo=bar` -> `{"foo": "bar"}` |
109
-
| `remoteAddress` | `string` | The IP address of the client. | `1.1.1.1` |
110
-
| `userAgent` | `string` | The [`User-Agent`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent) string in the request being processed. | `Mozilla/5.0 Gecko/20100101 Firefox/137.0` |
| `headers` | `map[string, string]` | The [headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers) of the request being processed. | `{"User-Agent": "Mozilla/5.0 Gecko/20100101 Firefox/137.0"}` |
105
+
| `host` | `string` | The [HTTP hostname](https://web.dev/articles/url-parts#host) the request is targeted to. | `anubis.techaro.lol` |
106
+
| `load_1m` | `double` | The current system load average over the last one minute. This is useful for making [load-based checks](#using-the-system-load-average). |
107
+
| `load_5m` | `double` | The current system load average over the last five minutes. This is useful for making [load-based checks](#using-the-system-load-average). |
108
+
| `load_15m` | `double` | The current system load average over the last fifteen minutes. This is useful for making [load-based checks](#using-the-system-load-average). |
109
+
| `method` | `string` | The [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Methods) in the request being processed. | `GET`, `POST`, `DELETE`, etc. |
110
+
| `path` | `string` | The [path](https://web.dev/articles/url-parts#pathname) of the request being processed. | `/`, `/api/memes/create` |
111
+
| `query` | `map[string, string]` | The [query parameters](https://web.dev/articles/url-parts#query) of the request being processed. | `?foo=bar` -> `{"foo": "bar"}` |
112
+
| `remoteAddress` | `string` | The IP address of the client. | `1.1.1.1` |
113
+
| `userAgent` | `string` | The [`User-Agent`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers/User-Agent) string in the request being processed. | `Mozilla/5.0 Gecko/20100101 Firefox/137.0` |
111
114
112
115
Of note: in many languages when you look up a key in a map and there is nothing there, the language will return some "falsy" value like `undefined` in JavaScript, `None` in Python, or the zero value of the type in Go. In CEL, if you try to look up a value that does not exist, execution of the expression will fail and Anubis will return an error.
113
116
@@ -141,6 +144,44 @@ X-Real-Ip: 8.8.8.8
141
144
142
145
Anubis would return a challenge because all of those conditions are true.
143
146
147
+
### Using the system load average
148
+
149
+
In Unix-like systems (such as Linux), every process on the system has to wait its turn to be able to run. This means that as more processes on the system are running, they need to wait longer to be able to execute. The [load average](<https://en.wikipedia.org/wiki/Load_(computing)>) represents the number of processes that want to be able to run but can't run yet. This metric isn't the most reliable to identify a cause, but is great at helping to identify symptoms.
150
+
151
+
Anubis lets you use the system load average as an input to expressions so that you can make dynamic rules like "when the system is under a low amount of load, dial back the protection, but when it's under a lot of load, crank it up to the mix". This lets you get all of the blocking features of Anubis in the background but only really expose Anubis to users when the system is actively being attacked.
152
+
153
+
This is best combined with the [weight](../policies.mdx#request-weight) and [threshold](./thresholds.mdx) systems so that you can have Anubis dynamically respond to attacks. Consider these rules in the default configuration file:
154
+
155
+
```yaml
156
+
## System load based checks.
157
+
# If the system is under high load for the last minute, add weight.
158
+
- name: high-load-average
159
+
action: WEIGH
160
+
expression: load_1m >= 10.0 # make sure to end the load comparison in a .0
161
+
weight:
162
+
adjust: 20
163
+
164
+
# If it is not for the last 15 minutes, remove weight.
165
+
- name: low-load-average
166
+
action: WEIGH
167
+
expression: load_15m <= 4.0 # make sure to end the load comparison in a .0
168
+
weight:
169
+
adjust: -10
170
+
```
171
+
172
+
This combination of rules makes Anubis dynamically react to the system load and only kick in when the system is under attack.
173
+
174
+
Something to keep in mind about system load average is that it is not aware of the number of cores the system has. If you have a 16 core system that has 16 processes running but none of them is hogging the CPU, then you will get a load average below 16. If you are in doubt, make your "high load" metric at least two times the number of CPU cores and your "low load" metric at least half of the number of CPU cores. For example:
175
+
176
+
| Kind | Core count | Load threshold |
177
+
| --------: | :--------- | :------------- |
178
+
| high load | 4 | `8.0` |
179
+
| low load | 4 | `2.0` |
180
+
| high load | 16 | `32.0` |
181
+
| low load | 16 | `8` |
182
+
183
+
Also keep in mind that this does not account for other kinds of latency like I/O latency. A system can have its web applications unresponsive due to high latency from a MySQL server but still have that web application server report a load near or at zero.
184
+
144
185
## Functions exposed to Anubis expressions
145
186
146
187
Anubis expressions can be augmented with the following functions:
0 commit comments