|
| 1 | +# Access and Error Logs |
| 2 | + |
| 3 | +Trickster can write per-backend HTTP access logs and error logs, with |
| 4 | +customizable formats, rotation and retention. Both logs are off by default; |
| 5 | +each is enabled by configuring its filename. |
| 6 | + |
| 7 | +## Basic Configuration |
| 8 | + |
| 9 | +```yaml |
| 10 | +backends: |
| 11 | + example1: |
| 12 | + provider: rp |
| 13 | + origin_url: http://example.com/ |
| 14 | + access_log: |
| 15 | + filename: /var/log/trickster/example1.access.log |
| 16 | + error_filename: /var/log/trickster/example1.error.log |
| 17 | +``` |
| 18 | +
|
| 19 | +- The access log receives one line per request handled by the backend and is |
| 20 | + written only when `filename` is set. |
| 21 | +- The error log receives one line per request whose response status is at or |
| 22 | + above `error_threshold` (default `400`) and is written only when |
| 23 | + `error_filename` is set. An error-logged request also appears in the access |
| 24 | + log when both are configured. |
| 25 | +- Two backends may share a filename; they will safely share the underlying |
| 26 | + file and its rotation. |
| 27 | +- When `instance_id` is set in the main config, it is inserted into log |
| 28 | + filenames just as with the application log (e.g., `example1.access.1.log`). |
| 29 | + |
| 30 | +## Log Format |
| 31 | + |
| 32 | +The `format` option accepts either a named preset or a custom format string |
| 33 | +using Apache-style `%` tokens, so the well-known conventions from Apache |
| 34 | +HTTP Server, Apache Traffic Server, Lighttpd, and similar servers apply |
| 35 | +directly. |
| 36 | + |
| 37 | +### Presets |
| 38 | + |
| 39 | +| Name | Description | |
| 40 | +| ----- | ----- | |
| 41 | +| `common` | NCSA Common Log Format: `%h %l %u %t "%r" %>s %b` | |
| 42 | +| `combined` | Apache/Nginx Combined format: `common` + Referer and User-Agent. This is the default. | |
| 43 | +| `extended` | `combined` + duration (ms), cache status and backend name | |
| 44 | +| `json` | One JSON object per line with a fixed field set (see below) | |
| 45 | + |
| 46 | +### Custom Formats |
| 47 | + |
| 48 | +```yaml |
| 49 | + access_log: |
| 50 | + filename: /var/log/trickster/example1.access.log |
| 51 | + format: '%h %u %t "%r" %>s %b %{ms}T %{cache-status}x' |
| 52 | +``` |
| 53 | + |
| 54 | +Supported tokens: |
| 55 | + |
| 56 | +| Token | Description | |
| 57 | +| ----- | ----- | |
| 58 | +| `%h`, `%a` | client IP address | |
| 59 | +| `%l` | remote logname (always `-`) | |
| 60 | +| `%u` | authenticated username (from HTTP Basic Auth), else `-` | |
| 61 | +| `%t` | request start time in CLF format: `[26/Aug/2026:10:30:00 +0000]` | |
| 62 | +| `%{sec}t`, `%{msec}t`, `%{usec}t` | request start time as a Unix epoch value | |
| 63 | +| `%{LAYOUT}t` | request start time in a custom [Go time layout](https://pkg.go.dev/time#pkg-constants) | |
| 64 | +| `%r` | first line of the request: `GET /path?query HTTP/1.1` | |
| 65 | +| `%m` | request method | |
| 66 | +| `%U` | request URL path | |
| 67 | +| `%q` | query string, prefixed with `?`, or empty when none | |
| 68 | +| `%H` | request protocol (e.g., `HTTP/1.1`) | |
| 69 | +| `%s`, `%>s` | response status code | |
| 70 | +| `%b` | response body bytes, or `-` when zero (CLF style) | |
| 71 | +| `%B` | response body bytes, numeric | |
| 72 | +| `%D` | request duration in microseconds | |
| 73 | +| `%T` | request duration in whole seconds | |
| 74 | +| `%{us}T`, `%{ms}T`, `%{s}T` | request duration in the given unit | |
| 75 | +| `%{Name}i` | request header value | |
| 76 | +| `%{Name}o` | response header value | |
| 77 | +| `%{Name}c` | request cookie value | |
| 78 | +| `%v` | requested virtual host | |
| 79 | +| `%p` | listener port that served the request | |
| 80 | +| `%A` | listener IP address that served the request | |
| 81 | +| `%%` | a literal `%` | |
| 82 | + |
| 83 | +Trickster-specific values use the `%{key}x` extension namespace: |
| 84 | + |
| 85 | +| Token | Description | |
| 86 | +| ----- | ----- | |
| 87 | +| `%{backend}x` | backend name | |
| 88 | +| `%{provider}x` | backend provider type | |
| 89 | +| `%{cache-status}x` | cache result (`hit`, `phit`, `kmiss`, ...); see [Cache Status](./caches.md#cache-status) | |
| 90 | +| `%{engine}x` | proxy engine that handled the request (e.g., `DeltaProxyCache`) | |
| 91 | +| `%{path-config}x` | the matched [path config](./paths.md) path | |
| 92 | + |
| 93 | +Missing values render as `-`. Values derived from the request (like headers |
| 94 | +and usernames) are backslash-escaped so they cannot corrupt the log line |
| 95 | +structure. Unknown tokens fail validation at startup. |
| 96 | + |
| 97 | +The `json` preset emits these fields per line: `time`, `client_ip`, `user`, |
| 98 | +`method`, `path`, `query`, `proto`, `status`, `bytes`, `duration_ms`, |
| 99 | +`host`, `referer`, `user_agent`, `backend`, `provider`, `path_config`, |
| 100 | +`cache_status`, `engine`. |
| 101 | + |
| 102 | +## Rotation and Retention |
| 103 | + |
| 104 | +Access and error logs are rotated and pruned automatically, using |
| 105 | +nginx/logrotate-style numbered archives (`example1.access.log.1.gz` is the |
| 106 | +most recent archive, `.2.gz` the next, and so on). |
| 107 | + |
| 108 | +```yaml |
| 109 | + access_log: |
| 110 | + filename: /var/log/trickster/example1.access.log |
| 111 | + rotation: |
| 112 | + size: 256MB # rotate when the live file would exceed this size (default 256MB) |
| 113 | + interval: 1d # also rotate when the live file is older than this (default off) |
| 114 | + retention: |
| 115 | + count: 3 # keep at most 3 archives (default 80) |
| 116 | + age: 7d # also prune archives older than this (default 7d) |
| 117 | + compress: true # gzip archives (default true) |
| 118 | +``` |
| 119 | + |
| 120 | +- `size` and `interval` may be combined; the log rotates when either |
| 121 | + threshold is reached. Setting both to `0` disables rotation. |
| 122 | +- Sizes accept `KB`, `MB`, `GB` and `TB` suffixes (binary multiples), or a |
| 123 | + plain byte count. |
| 124 | +- `retention.count: 0` disables count-based pruning and keeps all archives. |
| 125 | +- Writes are buffered for up to one second or 64 KiB. A process or machine |
| 126 | + crash can lose the buffered tail; an orderly shutdown flushes it. |
| 127 | +- Interval rotation keeps its epoch in a `<filename>.rotation` sidecar so a |
| 128 | + restart does not reset the interval clock. |
| 129 | + |
| 130 | +Archives created by older Trickster releases use timestamped lumberjack |
| 131 | +names and are not included in numbered-archive retention. They form a bounded |
| 132 | +legacy set and may be removed manually after upgrading. |
| 133 | + |
| 134 | +When upgrading from the original logging implementation, note these filename |
| 135 | +and retention changes: |
| 136 | + |
| 137 | +- `retention.count: 0` now keeps all archives; configure a positive count to |
| 138 | + bound archive retention. |
| 139 | +- With `main.instance_id` enabled, filenames without a `.log` suffix now also |
| 140 | + include the instance ID (`trickster.out` becomes `trickster.2.out`). Update |
| 141 | + log shippers that still follow the unsuffixed filename. |
| 142 | + |
| 143 | +The same `rotation`, `retention` and `compress` options are also available |
| 144 | +in the main `logging:` config section to control rotation of the Trickster |
| 145 | +application log, with the same defaults. |
| 146 | + |
| 147 | +## Error Log Settings |
| 148 | + |
| 149 | +Each `error_*` option inherits its value from the corresponding access log |
| 150 | +option when unset: |
| 151 | + |
| 152 | +```yaml |
| 153 | + access_log: |
| 154 | + filename: /var/log/trickster/example1.access.log |
| 155 | + format: combined |
| 156 | + error_filename: /var/log/trickster/example1.error.log |
| 157 | + error_format: '' # default: inherits format |
| 158 | + error_threshold: 400 # log responses with status >= this (default 400) |
| 159 | + error_rotation: # default: inherits rotation |
| 160 | + size: 64MB |
| 161 | + error_retention: # default: inherits retention |
| 162 | + count: 7 |
| 163 | + error_compress: true # default: inherits compress |
| 164 | +``` |
0 commit comments