If your server never expects over 100kb in a single request, you can block oversized requests to protect against abuse. You could block them in a reverse proxy, e.g. NGINX, but you can also block them in application code. Here I will talk about the latter, since this can be useful if you don’t have a reverse proxy, or you need fine control over the limiting.
Blocking a large request, by counting the bytes
Let’s say you want to block requests over 100kb. This is something I looked into for a recent project. My first thought when looking into this was to replace the body with a reader that closes after it consumes 100kb and throws an error. As it happens there is reader built into the http library called MaxBytesReader that does exactly this, so you can do something like this.
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
maxBytesReader := http.MaxBytesReader(w, r.Body, maxBytes)
...
}The great thing about doing it as a reader is you don’t really impact performance that much, you do the read you’d do anyway. This is as opposed to say reading the whole body into memory, then realizing it is 100mb, wasting memory, and then rejecting the request.
One thing to note is that the user of maxBytesReader needs to determine what to do when the reader returns the error reporting the max size has been reached. It will need to determine that if it is a too large error then return the response code. Like this:
var maxBytesError *http.MaxBytesError
body, err := io.ReadAll(maxBytesReader)
if err != nil {
if errors.As(err, &maxBytesError) {
w.WriteHeader(http.StatusRequestEntityTooLarge)
fmt.Fprintf(w, "Content Too Large")
}
} Next, let’s make the typical case more efficient. Most requests will send the size of their payload in the header, so we can block earlier if we know the request will be too big. Let’s look into that.
Blocking a large request using content-length header
The HTTP/1.1 content-length header is an highly recommended (but technically optional) field to indicate how much data you intend to send in a request or response.
We can use this header to block a request that is too large before even reading the body. However we can’t trust that it will be set, or if set that is is accurate, so you will want to combine this with counting the bytes.
Combining the two methods, looks like this:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
if r.ContentLength > maxBytes {
w.WriteHeader(http.StatusRequestEntityTooLarge)
fmt.Fprintf(w, "Content Too Large")
return
}
maxBytesReader := http.MaxBytesReader(w, r.Body, maxBytes)
...
}That is it. We are all done! Now let’s now visit a couple of things related to this of interest…
Additional Notes
Sometimes the http server will check body length, but don’t rely on that.
Go’s http server (see transfer.go) will limit the reader to only read a number of bytes equal to realLength which is usually set to content-length, but only if the content-length is supplied, and there are various conditions around whether it will do it or not. This is a good feature, but not 100% reliable for the purpose of blocking large requests to protect your service.
Here is the code inside the Go http library that does this:
func readTransfer(msg any, r *bufio.Reader) (err error) {
//...
switch {
//...
case realLength > 0:
t.Body = &body{src: io.LimitReader(r, realLength), closing: t.Close}
//...LimitReader vs. MaxBytesReader
There are two similar readers for limiting request size, io.LimitReader and http.MaxBytesReader. Here is what the documentation says:
MaxBytesReader is similar to io.LimitReader but is intended for limiting the size of incoming request bodies. In contrast to io.LimitReader, MaxBytesReader’s result is a ReadCloser, returns a non-nil error of type *MaxBytesError for a Read beyond the limit, and closes the underlying reader when its Close method is called.
MaxBytesReader prevents clients from accidentally or maliciously sending a large request and wasting server resources. If possible, it tells the ResponseWriter to close the connection after the limit has been reached.
And for the complete picture, the docs for LimitReader:
LimitReader returns a Reader that reads from r but stops with EOF after n bytes. The underlying implementation is a *LimitedReader.
If we used a LimitReader, we wouldn’t know that the client had sent too much data, and instead continued processing the request, likely hitting an error later because the payload is truncated.
