|
| 1 | +<!--forhugo |
| 2 | ++++ |
| 3 | +title="Output and Error Handling" |
| 4 | ++++ |
| 5 | +forhugo--> |
| 6 | + |
| 7 | +In this project you're going to get familiar with conventions around output and exit codes, as well as error handling, and how to apply these in the [Go programming language][go]. |
| 8 | + |
| 9 | +Timebox: 3 days |
| 10 | + |
| 11 | +## Objectives: |
| 12 | + |
| 13 | +- Know when to write to standard out and standard error |
| 14 | +- Exit programs with conventional exit codes |
| 15 | +- Know when to propagate errors, wrap errors, and terminate due to errors. |
| 16 | + |
| 17 | +## Project |
| 18 | + |
| 19 | +Most programs can run into problems. Sometimes these problems are recoverable, and other times they can't be recovered from. |
| 20 | + |
| 21 | +We are going to write a program which may encounter several kinds of error, and handle them appropriately. We will also make sure we tell the user of the program information they need, in ways they can usefully consume it. |
| 22 | + |
| 23 | +### The program |
| 24 | + |
| 25 | +In this project, we have been supplied with a server - its code lives in the `server` subdirectory of this project. You can run it by `cd`ing into that directory, and running `go run .`. The server is an HTTP server, which listens on port 8080 and responds in a few different ways: |
| 26 | +* If you make an HTTP GET request to it, it will respond with the current weather. When this happens, you should display it to the user on the terminal. |
| 27 | +* Sometimes the server simulates being overloaded by too many requests, and responds with a status code 429. When this happens, the client should wait the amount of time indicated in the `Retry-After` response header, and attempt the request again. You can learn about [the `Retry-After` header on MDN](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Retry-After). Note that it has two formats, and may contain either a number of seconds or a timestamp. |
| 28 | +* Other times, it may drop a connection before responding. When this happens, you should assume the server is non-responsive (and that you making more requests to it may make things worse), and give up your request, telling the user something irrecoverable went wrong. |
| 29 | + |
| 30 | +Have a read of the server code and make sure you understand what it's doing, and what kinds of responses you may need to handle. We are not expected to change the server code as part of this project - it is intentionally buggy because we sometimes need to handle bad responses. |
| 31 | + |
| 32 | +We're going to focus in this project on how we handle errors, and how we present output to the user. |
| 33 | + |
| 34 | +### Standard out and standard error |
| 35 | + |
| 36 | +Typically, terminal programs have two places they can write output to: standard out (also known as "standard output"), and standard error. As far as your program is concerned, these are "files" you can write to, but in reality they are both by default connected to your terminal, so if you write to them, what you write will end up displayed in your terminal. |
| 37 | + |
| 38 | +You can also request that one or both of these "files" be redirected somewhere else, e.g. in your terminal you can run: |
| 39 | + |
| 40 | +```console |
| 41 | +% echo hello > /tmp/the-output-from-echo |
| 42 | +``` |
| 43 | + |
| 44 | +`echo`'s job is to write something to standard out, but if you run this command, you won't see "hello" output to the terminal, instead, it will get written to the file `/tmp/the-output-from-echo`. `echo`'s standard out was redirected. If you `cat /tmp/the-output-from-echo` you'll see `hello` was written in that file. |
| 45 | + |
| 46 | +You can redirect standard error by writing `2>` instead of `>`: |
| 47 | + |
| 48 | +```console |
| 49 | +% echo hello 2> /tmp/error-from-echo |
| 50 | +``` |
| 51 | + |
| 52 | +In this example, you'll still see `hello` on your terminal (because you didn't redirect standard out anywhere), and you'll see `/tmp/error-from-echo` was created, but is empty, because `echo` didn't write anything to standard error. |
| 53 | + |
| 54 | +You can redirect both if you want by using both redirect instructions: |
| 55 | + |
| 56 | +```console |
| 57 | +% echo hello > /tmp/file-1 2> /tmp/file-2 |
| 58 | +``` |
| 59 | + |
| 60 | +The main reason we have these two different locations is that often times we want the output of our program to be passed to some other program or file for further processing. Standard error exists as a place you can write information which a user may be interested in (e.g. information about something going wrong, or progress messages explaining what's happening), but which you don't want to pass on for that further processing. |
| 61 | + |
| 62 | +For example, imagine your program writes out a series of scores, one per line, and you were going to write those scores to a file which another program may analyse. If the output file had "Waiting for HTTP request..." or "Server error" printed in it, that would be annoying to process later on, but as a user, you my want to know why your program appears to be hanging or failing. |
| 63 | + |
| 64 | +Another example is when something goes wrong - in your score-recording program, you may want to (reasonably) assume that anything it outputs is a number. But if something goes wrong (say, your network connection was down so the scores couldn't be fetched), reporting that on standard error means you won't accidentally try to add the error string "Network was down" to some other number. (TODO: This explanation doesn't seem very clear). |
| 65 | + |
| 66 | +#### Standard out and standard error in Go |
| 67 | + |
| 68 | +In go, standard out and standard error can be accessed as `os.Stdout` and `os.Stderr`. |
| 69 | + |
| 70 | +You can write to them by writing code like `fmt.Fprint(os.Stdout, "Hello")` or `fmt.Fprint(os.Stderr, "Something went wrong")`. (The "F" before "printf" stands for "file" - we're saying "print some string to a file I'll specify as the first argument". A lot of times in Unix systems, we like to pretend anything we read or write is a file). |
| 71 | + |
| 72 | +More often, we'll write `fmt.Print("Hello")` - this is the same as writing `fmt.Fprint(os.Stdout, "Hello")` (if you look at [the Go standard library source code](https://cs.opensource.google/go/go/+/refs/tags/go1.19.5:src/fmt/print.go;l=251-253), you can see it's literally the same), but it's worth remembering we can choose to write to other locations, like standard error, if it's more appropriate. |
| 73 | + |
| 74 | +#### When to write to standard out/error |
| 75 | + |
| 76 | +As a rule, the intended output of your program should be written to standard out, and anything that isn't the intended output of your program should be written to standard error. |
| 77 | + |
| 78 | +Some things you may write to standard error: |
| 79 | +* Progress messages explaining what the program is doing. |
| 80 | +* Error information about something that went wrong. |
| 81 | + |
| 82 | +Thinking about our program we're going to write, that means we're likely to write: |
| 83 | +* The current weather to standard out - it's what our program is for. |
| 84 | +* A message saying that we've been asked to wait and retry later to standard error - it's a progress message, not the intended output of our program. |
| 85 | +* Error information if the server seems broken to standard error - it's not the intended output of our program, it's diagnostic information. |
| 86 | + |
| 87 | +### Exit codes |
| 88 | + |
| 89 | +By convention, most programs exit with an exit code of `0` when they successfully did what they expected, and any number that isn't `0` when they didn't. Often a specific program will attach specific meaning to specific non-zero exit codes (e.g. `1` may mean "You didn't specify all the flags I needed" and `2` may mean "A remote server couldn't give me information I needed"), but there are no general conventions for specific non-`0` exit codes across different programs. |
| 90 | + |
| 91 | +By default, your program will exit with exit code `0` unless you tell it to do otherwise, or it crashes. |
| 92 | + |
| 93 | +In Go, you can choose what code to exit your program with by calling [`os.Exit`](https://pkg.go.dev/os#Exit). After calling `os.Exit`, your program stops and can't do anything else. |
| 94 | + |
| 95 | +### Handling errors in your code |
| 96 | + |
| 97 | +A lot of time when writing code, we need to handle the possibility that an error has occurred. |
| 98 | + |
| 99 | +This may be an explicit error returned from a function (e.g. when you make a `GET` request, [`Get`](https://pkg.go.dev/net/http#Client.Get) returns an `error` (which may be `nil`, but will be non-`nil` if, for instance, the server couldn't be connected to) alongside the response). |
| 100 | + |
| 101 | +Alternatively, this may be something which we detect, but which other code didn't tell us was an error. For instance, if we make a `GET` request to a server which returns a 429 status code, the `error` will be `nil`, but by looking at [`Response.StatusCode`](https://pkg.go.dev/net/http#Response) we can see that something went wrong which we may need to handle. |
| 102 | + |
| 103 | +When encountering or detecting errors, there are typically four options for how to handle them: |
| 104 | +1. Propagating the error to the calling function (and possibly wrapping it with some extra contextual information). |
| 105 | +1. Working around the error to recover from it. |
| 106 | +1. Terminating the program completely. |
| 107 | +1. Ignoring the error - sometimes an error actually doesn't matter. |
| 108 | + |
| 109 | +When we should do each of these isn't always obvious, but here are some guidelines: |
| 110 | + |
| 111 | +#### Propagating the error to the calling function |
| 112 | + |
| 113 | +This is generally our default behaviour. If an error has happened, and we don't know how to handle it, we should early-return from our function, handing the error to the caller. |
| 114 | + |
| 115 | +This means that generally any time we write a function, and it calls another function which may return an error, our function will probably also possibly return an error. |
| 116 | + |
| 117 | +Often times, we want to wrap the error to provide more context. For instance, say we have the following code: |
| 118 | + |
| 119 | +```go |
| 120 | +package main |
| 121 | + |
| 122 | +import "os" |
| 123 | + |
| 124 | +func main() { |
| 125 | + password, err := readPassword() |
| 126 | + // ... |
| 127 | +} |
| 128 | + |
| 129 | +func readPassword() (string, error) { |
| 130 | + password, err := os.ReadFile(".some-file") |
| 131 | + if err != nil { |
| 132 | + return "", err |
| 133 | + } |
| 134 | + return string(password), nil |
| 135 | +} |
| 136 | +``` |
| 137 | + |
| 138 | +If the password file doesn't exist, the error message `open .some-file: no such file or directory` is less useful than an error message like `failed to read password file: open .some-file: no such file or directory`. By wrapping the error with more contextual information, you help the person seeing the error understand _what_ went wrong, _why_ it failed, and what they need to do to fix the situation. |
| 139 | + |
| 140 | +Accordingly, we may write `readPassword` instead like: |
| 141 | + |
| 142 | +```go |
| 143 | +func readPassword() (string, error) { |
| 144 | + password, err := os.ReadFile(".some-file") |
| 145 | + if err != nil { |
| 146 | + return "", fmt.Errorf("failed to read password file: %w", err) |
| 147 | + } |
| 148 | + return string(password), nil |
| 149 | +} |
| 150 | +``` |
| 151 | + |
| 152 | +You can learn more about creating and wrapping errors in Go by reading [this article](https://earthly.dev/blog/golang-errors/). |
| 153 | + |
| 154 | +#### Working around the error to recover from it |
| 155 | + |
| 156 | +Sometimes, an error may be expected, or may be recoverable. For instance, suppose we have some expensive computation we want to do, but which may have already been done and saved to a file. We may try to read the file, but if we encounter an error that it doesn't exist, we may know how to compute the answer we need instead. |
| 157 | + |
| 158 | +This kind of behaviour will depend entirely on the problem domain you're solving, and there isn't really a general rule for when it's appropriate. |
| 159 | + |
| 160 | +#### Terminating the program completely |
| 161 | + |
| 162 | +A lot of the time, when we run into errors, there's nothing we can do about them. |
| 163 | + |
| 164 | +If we're running a server, and the error happened when processing one request, normally we don't want to terminate our program - we just want to respond to that request saying an error happened, but keep trying to process other requests. |
| 165 | + |
| 166 | +Other times, for instance when first starting up a server, or when writing a program that isn't a server but just does a one-off task, it may make sense to terminate our program, and exit (with a non-`0` status code). |
| 167 | + |
| 168 | +_Where_ we do this, however, is worth thinking about. We generally don't want to call `os.Exit` from anywhere _except_ our `main` function. |
| 169 | + |
| 170 | +There are a few reasons for this: |
| 171 | +1. If we call `os.Exit`, there's no way any code can handle that or recover. Let's say we started calling `os.Exit` in some other function - it's possible we'll end up in the future calling that function from a request handler, and we'll end up terminating the whole server just because one request couldn't be handled. This will probably cause an outage, because no one will be able to talk to our server any more. |
| 172 | +1. When writing unit tests, we generally don't want our program to exit. But if you call `os.Exit` inside a unit test, it will stop running. In general, we never want to call `os.Exit` from any code which is called from a test. |
| 173 | + |
| 174 | +While you may write an `os.Exit` call in some other function, thinking it's only ever called from places it's ok to call `os.Exit`, code changes a lot over time, and you may find yourself or someone else adding calls to functions from other places without realising that your function isn't safe to be called everywhere. The easiest way to avoid this is to use the rule: only ever call `os.Exit` from your `main` function - everything else should propagate any errors they encounter. |
| 175 | + |
| 176 | +#### Ignoring the error |
| 177 | + |
| 178 | +Sometimes an error actually doesn't matter at all, and can just be ignored. This is rare, and you should be wary if you think this is the case. |
| 179 | + |
| 180 | +### Back to our program |
| 181 | + |
| 182 | +Recall the server we've been supplied with for telling the weather. |
| 183 | + |
| 184 | +Our task is to write a client, in Go, which makes HTTP requests to that server and tells the user about the weather. |
| 185 | + |
| 186 | +We should focus in this project on making sure: |
| 187 | +1. If the server replies with a retryable error, we will retry it appropriately. For a 429 response code, this means reading the `Retry-After` response header, calling `time.Sleep` until the appropriate time has passed, and trying again. |
| 188 | + * If we're going to sleep for more than 1 second, we should notify the user that things may be a bit slow because we're doing a retry. |
| 189 | + * If the server tells us we should sleep for more than 5 seconds, we should give up and tell the user we can't get them the weather. |
| 190 | + * If we can't determine how long to sleep for, consider what the best thing to do is - you should decide whether we should sleep for some amount of time (and if so what) and then retry, or give up. Make sure to write down why you decided what you dod. |
| 191 | +1. If the server terminates our connection, we will give up and tell the user that we can't get them the weather. |
| 192 | + |
| 193 | +Make sure all error messages are clear and useful to the user, that we're properly printing to standard out or standard error when appropriate, and that our program always exits with an appropriate exit code. |
| 194 | + |
| 195 | +[go]: https://go.dev/ |
0 commit comments