Weekly AI Learnings #2

posted on June 29, 2026tagged as AI, Cache, PiAgent

Detour — Prompt Caching

This was an impromptu find. I came across https://kreidemann.com/blog/prompt-caching, it is mostly about prompt caching security and timing attacks, but it does introduce the concept of KV cache well and highlights this can be the system prompt shared by all users, or system + messages making your next turns faster and cheaper (for someone) as they hit the cache. This image is pretty good:

This rabbit hole links to https://sankalp.bearblog.dev/how-prompt-caching-works/ and https://ngrok.com/blog/prompt-caching. I read just the second one for now, and it is excellent. Fairly intense, as it explains attention from mostly from scratch, to the extent it helps explain the KV-cache. I don’t think we probably need to understand all that and the simplified version above is enough intuition to understand the cache. Basically as you run through each token from start to finish you do a calculation. Each calculation depends on the previous. Each token builds upon the last. Therefore, you can cache intermediate calculations. It is kind of like memoization it seems.

My takeaways:

Reminder — there is a cache!
It saves someone money, either it is you the consumer, or the model provider if you are paying a fixed monthly amount, or paying the same per token regardless of caching.
It works well with system prompt, as everyone gets the same one. There is a huge efficiency boost, plus no security worries.
It also works well with your sessions/prompts, but if sharing these caches across tenants, they may need to be wary of timing attacks.

Original Destination — How Does That Agent Work?

Following on from Weekly AI Learnings #1, I wanted to study how a real life agent works. The agents I am interested in are OpenCode and Pi, and I don’t fully understand how they compare/contrast yet. I have briefly used OpenCode but not Pi.

I have chosen Pi to look at first, mainly because I stumbled across agent-loop.ts, and thought this looks fairly straightforward to read. Maybe OpenCode has such a file, but didn’t find it yet.

I feel like I need to understand a few basics of this code, so git clone https://github.com/earendil-works/pi/ it is.

EventStream (packages/ai/src/utils/event-stream.ts) looks a lot like a Goroutine channel. It has a push, which will send “events” to a listener. A listener which subscribes via an iterator. If there is nothing being pulled, events get queued up. And if there are no events, pulls get queued up and wait for them. There is also a mechanism to close the stream, with a judge function that determines if an event is terminating, along with another function that produces a final result from that closing event. This seems like a useful building block object to have in an agentic system.

Knowing what EventStream is helps me understand agentLoop, which returns said stream, using type AgentEvent for the event, and AgentMessage[] for the result. Agent Events are looking like “stuff the agent does” such as messages and tool calls. This looks neatly organized so far:

export type AgentEvent =
	// Agent lifecycle
	| { type: "agent_start" }
	| { type: "agent_end"; messages: AgentMessage[] }
	// Turn lifecycle - a turn is one assistant response + any tool calls/results
	| { type: "turn_start" }
	| { type: "turn_end"; message: AgentMessage; toolResults: ToolResultMessage[] }
	// Message lifecycle - emitted for user, assistant, and toolResult messages
	| { type: "message_start"; message: AgentMessage }
	// Only emitted for assistant messages during streaming
	| { type: "message_update"; message: AgentMessage; assistantMessageEvent: AssistantMessageEvent }
	| { type: "message_end"; message: AgentMessage }
	// Tool execution lifecycle
	| { type: "tool_execution_start"; toolCallId: string; toolName: string; args: any }
	| { type: "tool_execution_update"; toolCallId: string; toolName: string; args: any; partialResult: any }
	| { type: "tool_execution_end"; toolCallId: string; toolName: string; result: any; isError: boolean };

The EventStream constructed is one that looks for agent_end as the stop signal, and when that happens the results are its messages. At this point I have no idea how this hangs together, but I guess there is some accumulation of messages going on somewhere.

Honestly I have looked ahead at more of the code, and it seems hairy, while also being compact and clever. It’ll be fun reading it all and seeing how it works. I was naively hoping to understand the whole thing in one go, but I’ll wait for next week’s post to go deeper into it.

Image by Zerro Energy from Pixabay

Making this site a little faster

posted on June 26, 2026June 28, 2026tagged as Performance

This site runs on WordPress on a cheap hosting plan, with W3TotalCache and Cloudflare in front of it to hopefully make it a bit faster. It turns out my quick setup of bundling this all together wasn’t making the site as fast as I assumed. We’ll talk about how I found out and what easy steps made it quite a bit faster, while still using the cache and Cloudflare.

Recently I found I could speed up this site in simple ways—ways that don’t involve writing code. I found this when I was experimenting with a tool called Locust which load tests services. Let’s look into that a bit more.

Locust

Locust is a performance load testing kit, written in Python. It is easy to install and run if you have Python set up (try to have 3.13 if you have a choice to run Locust). Just do this:

python -m venv .venv
source .venv/bin/activate
pip install locust
locust -V

Then create a locustfile.py like this

from locust import HttpUser, task

class HelloWorldUser(HttpUser):
    @task
    def hello_world(self):
        self.client.get("/")
        self.client.get("/images/someimage.jpg")

Then run locust

locust

And visit http://0.0.0.0:8089/

You will be asked for number of users, ramp up and host. Going with 1, 1 and https://martincapodici.com/, the system will then start polling those endpoints, and reporting stats:

We now have some stats on latency statistics e.g., median latency, 95%ile and 99%ile. If you are not familiar, the 95%ile (or p95) means the latency at which that 95% requests are lower latency (faster), and 5% are higher latency. Higher percentiles like p95 and p99 are useful to get a feel of what the more slow cases are.

The latency includes time to finish the request having downloaded the response body. Therefore, the larger image does take longer, as you would expect. This is heavily influenced by my internet connection. A quick test of which is 91.2 Mbps download / 28.1Mbps upload. Based on that a ~2.16 MB images = 17.28Mb = 189ms to download. By calculating 200-189, we get an inferred ~11ms to process request to the first byte for p50.

The above stats are the optimized versions, and it was worse before, and I’ll go into how bad and what the those optimizations are.

Before we do that notice the GET / is a lot slower, 290ms. That is because the images are behind a CDN, whereas I chose not to cache the HTML on the CDN yet. This means it crosses from somewhere in the US to where I am, adding a lot of network latency. Adding HTML to the CDN is doable, but needs a bit of configuration to avoid the cache interfering with admin/login, and for a low traffic blog this latency for the HTML part of the request is OK for now.

What is a CDN? A CDN solves the latency problem by caching copies of the file in servers in hundreds of cities. The network latency you local city can be as low as 1ms so it makes a huge difference. This is done using things like GeoDNS, Anycast IP Routing, and more.

Things that were slow

On the earlier runs of Locust, latency of the HTML home page / was much higher, I think around 2 seconds p95!

Next, there was only the PNG image and not the more highly compressed AVIF format, and you can see the difference between those two in the test results from before.

Finally, in addition to Locust I ran Google Lighthouse to look for slowness. It found a redundant JS download, and removing this saved about 800kb uncompressed / 200kb compressed. It was a math formula (TeX) renderer bundled with the theme, that I do not need.

Why the slow home page?

The reason for that 2 second latency was a simple lack of caching in WordPress itself. First I had an old cache plugin-in that broke the current one as it didn’t properly get uninstalled. Then on top of that W3TotalCache requires you to opt in to page caching.

Claude helped me diagnose this, which it did really well with just curl commands. Because of this it also noticed the header specifying a 7-day limit for caching, i.e., max-age=604800, which could lead to some people seeing old versions of the home page without the latest post. It was fixed by changing the .htaccess file to reduce the default to 0.

With the page now actually cached, the latency dropped from 2 seconds to the current 500ms p95 — with even better results if you are browsing from the US as much of that latency is the network.

The AVIF improvement?

AVIF is a recent file format which has gain widespread adoption. It is based on the AV1 video format. There are two parts to serving these, one is to have them ready to serve, and the other is to detect if the browser can support them and decide whether to server the older format or AVIF.

To do this without paying for a dedicated service required a couple of plugins, Modern Image Formats (by WordPress Team) and Regenerate Thumbnails. The former will generate the WebP and AVIF files for each image, and serve them when supported, and the latter lets you do the one-off conversion if you haven’t been generating them before.

These speed-ups are great, better for user experience, better for slower connections, better for the environment too!

Image by PublicDomainPictures from Pixabay

Weekly AI Learnings #1

posted on June 20, 2026June 28, 2026tagged as Agents, AI

This is a series where I talk about trying to learn more about AI tools, with the goal of learning practical skills to solve problems both in life and at work.

Current State

I know basically how LLMs work e.g., by doing the zero-hero course a while back, and I use AI almost exclusively for writing code at work using Rovo, and now looking at how do I effectively “harness” things to get the best results, waste less time, avoid tech debt and so on.

I am also interested in automated agents and how they can be useful, how to avoid slop from whatever source wasting human time, and when—like I am now for writing this post—not to use AI at all.

This week

This week I started with Building effective agents, which is a 2024 Anthropic article. I have a walnut set up on my personal context manager that I am using to guide me about what to learn, and it suggested this. I questioned it that it is from 2024, which is a bit old in this fast moving world, and Claude said something along the lines of:

Why it ages well: it was written at the concept level, not the tooling level. It covers workflow vs. agent distinction, five patterns and core advice.

So I gave it a read. My notes are below.

Building Effective Agents: My Notes

It starts of defining words—a very useful thing to do for common understanding. I have pasted below from the article:

Workflows are systems where LLMs and tools are orchestrated through predefined code paths.
Agents, on the other hand, are systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks.

Here are the advantages of each:

Agents	Workflows
+ Task Performance + Difficult and open-ended problems — Potential for compounding errors	+ Lower Latency + Lower Cost + Predictability and Consistency

I learned of existence of the agent SDK, which “includes built-in tools for reading files, running commands, and editing code, so your agent can start working immediately without you implementing tool execution.” It also plugs a couple of GUI tools: Rivet and Vellum.

Rivet and Vellum look very useful and polished, and that makes me wonder how they compare to the OpenClaw and Hermes hype of 2026.

In any case, Anthropic recommend developers use API directly, I guess for the obvious reasons (I won’t spell it out, but you know if you have been writing code for some time). Nonetheless, those tools look fun to play with.

It defines augmentations and the diagram mentions query/results, call/response and memory. One of the talks at this meetup mentioned the analogy here with your operating system, something like LLM is compute, memory is RAM, MCP is calling out to the internet maybe, stuff like that.

Then these workflow concepts:

Prompt Chaining — What it says on the tin I guess. Do one prompt, then do another. Perhaps with a gate between. Means the LLM can dedicate more attention to each task (my view). Tradeoff is better performance at task, for higher latency and maybe cost overall.
Routing — You can route to the right LLM. Possibly the best LLM for the job, or maybe the most cost-effective. It doesn’t mention how to route, but I imagine this could be anything from simple pattern matching/regex to using a classifier, to asking a cheap model to classify, or even asking the human. My guesses.
Parallelization — obvious use case is for speed, you have things that can be done at the same time. Also, good for sectioning I guess a bit like chaining above to focus more attention on two aspects of the same problem. There is also voting, e.g., approach the same issue from different angles then combine (their example is look for vulns). I also think this is where you could send to OpenAI and Claude and compare what they say, although this article ain’t going to say that of course!
Orchestrator — Workers — I just see this as dynamic parallelization with an LLM in charge. This is the classic thing Claude Code does all the time.
Evaluator — Optimizer — LLM evaluates result from another LLM. A loop to repeat until the evaluator is happy.

Agent concepts:

There are many fantastic diagrams in the article, but the one I will copy is this about Agent loops, saying how simple they are at the core:

Examples are coding agents and computer use.

It doesn’t go much deeper than this — it is a conceptual article. I think I could learn a lot by looking how specific agents such as OpenCode work, which would be interesting to know.

When implementing agents, we try to follow three core principles:

Maintain simplicity in your agent’s design.

Prioritize transparency by explicitly showing the agent’s planning steps.

Carefully craft your agent-computer interface (ACI) through thorough tool documentation and testing.

Appendix items

Agents in practice — they say why customer support and coding agents are good fits for using agents (I think as opposed to workflows) and I agree with that. Whether customer support agents > humans I am not sure, but I have used good customer support agents before. The Dell one is particularly good as it runs on your computer so knows your setup, can do scans then book support for you, for example.
Tools — Mentions tool use, in which LLM sends a stop block to invoke tool use. There are some suggestions on tool formats, but I wonder if those are more for the models of the time, or today if you decide to use cheaper models. However, thinking about formatting coming out of tools is good to save context and money.
Workbench — there is a workbench to test tools — workbench
Pokayoke — https://en.wikipedia.org/wiki/Poka-yoke, interesting concept: “is any mechanism in a process that helps an equipment operator avoid mistakes and defects by preventing, correcting, or drawing attention to human errors as they occur”

Experiment

My walnut session then advised to do this tutorial https://ampcode.com/notes/how-to-build-an-agent, which is pretty fun and quick to get going on.

Luckily I know Go, although if you don’t, you can have it converted to Python — it might be more educational to do that manually though.

Anyway this takes you through:

First you build a simple take-turns chatbot. To fix the terminal characters, replace \\\u with \u in the listing, and you will have to change the model as the one they specify no longer exists. Other than that it works.
Then you add tool use, they give you a gotcha for fun — they add the tool request without checking the LLM response and using the tool, so it does nothing. Then show you the code to complete this. You now have a bot that can decide to read a file if it wishes to.

What is interesting is the tool use is programmed in so you get a real feel for what is going on, but it is sort of built into Anthropic’s API, so there is some magic there, but I guess that is the way it is when the models are trained on tool use and have magic tokens or whatnot that you don’t get to see. How that works and is trained is something I would like to go deeper into at some point, but not a priority now.

Feature image generation: Nano Banana 2

The Signalflow concepts to learn, for easier charts, alerts and to spend less time in Splunk

posted on January 24, 2026January 24, 2026tagged as Blog, Splunk, SRE

It can be frustrating using Signalflow for the first time. Charts appearing with no data, or detectors going off that shouldn’t, and stuff not making sense. If you’ve ever stared at a flat line (or no lines!) on a chart wondering why your metrics aren’t showing, or been confused about why aggregation doesn’t work as expected, this post is for you.

If you are not familiar, SignalFlow is the query language used in Splunk Observability Cloud for analyzing metrics.

I’ll assume you are:

Using Splunk Observability Cloud metrics with Signalflow queries.
Want to observe how your services are performing, if there are bugs, performance issues or service degradation.

This post aims to get you to a place where you can quickly troubleshoot queries, avoid trouble in the first place, and spend less time on this part of your job. To do that, I go over some key concepts, that once you understand, will solve most puzzling things about your Signalflow queries not doing the right thing.

The concepts are listed below, and we’ll tackle them one by one.

Key Concepts:

Metrics – Basic Definitions

Let’s define a couple of terms related to metrics:

A metric is a single measurement at a specific point in time, e.g. (2026-01-04 00:00:00, 18), visualized here, with metric shown as pink diamond:

A metric data point is a metric with some metadata. It has the following values:

Timestamp — the time — usually to 1 second precision, can be lower resolution if configured.
Metric value — e.g. 15
Metric type — can be counter, cumulative counter, or gauge:
- A counter counts events since the last report
- A cumulative counter counts events since process start
- A gauge measures a value at a point in time (e.g., current CPU usage)
Metric name — e.g. cpu.idle
Dimensions — a set of key-value pairs to tag the data point. Can be anything, example is aws_region:us-east1

This is shown below. It is the same chart as before with the additional metadata, and the pink diamond is now the metric data point:

Metric Time Series, or MTS

A metric time series (MTS) is a collection of data points that have the same metric and the same set of dimensions. In other words, it is a series of Metric Data Points. For example:

Time	Value	Details
11:54:00 PM	20	Metric Name: cpu.idle Dimensions: aws_region:us-east1
11:55:00 PM	5
11:56:00 PM	17
11:57:00 PM	12
11:58:00 PM	11
11:59:00 PM	0
12:00:00 AM	18
12:01:00 AM	23

There is an MTS for every recorded combination of:

Metric Name, e.g. cpu.usage
Metric Type, e.g. gauge
Unique set of dimensions, e.g. hostname:server1,location:Tokyo

Note: it is possible to have two metrics with the same name and different types, but it is best not to do that. They discourage it in the docs here.

Dimensionality

There is an MTS for every combination of dimension values. The number of MTS grows quickly as dimensions are added, or new values added to dimensions.

For example, if you have a metric, with a server dimension with 100 values, and an operation dimension with 50 operations, and all combinations get used, you have 5000 metric time series. Having thousands isn’t necessarily a problem, but having many millions might be.

This is why we try to avoid high dimensional in metrics. For example if you use a URL as a dimension, the problem could be it may include query data, e.g. /user/1003332/post/8883 which would create a lot of time series.

Data Blocks

Now we know how the data is represented, the start of any SignalFlow query is the data block. The data block is where you define which metric(s) you want to gather data from, what filters you want to apply.

For example:

data('cpu.utilization', 
     filter=filter('host', 'hostA', 'hostB') and filter('AWSUniqueId', 'i-0403'))

This creates a data stream based on the metric cpu.utilization, and uses filters to only pick up data for specified values of dimensions. In details:

The data block defines that we want to extract data from MTS, and specifies the name of the metric.
The filter parameter defines how we want to filter all the MTS streams down the the ones we need.
The output of this is the collection of MTS streams for the metric name that have the dimensions specified by the filter.

This is combining the various MTS into a single stream. In this case, it is combining all of these:

All cpu.utilization MTS with host=hostA and AWSUniqueId=i-0403
All cpu.utilization MTS with host=hostB and AWSUniqueId=i-0403

If there are other dimensions, then there can be multiple MTS with the same host and AWSUniqueId that need to be combined.

The data block produces a stream, but to produce a chart the engine must bucket values up into time intervals, e.g. 1 minute or 1 day. What is the time interval set to? Can you control it? On to that next.

Resolution

The resolution defined the period of time MTS are bucketed in to for analysis. For example it could be 1 minute, and therefore the Metric Data Points will be aggregated into 1 minute buckets for the purpose of charting and rollups.

The use of an unexpected resolution can easily trip you up, especially when using detectors and debugging why a detector did or didn’t go off given the sent data. We might cover detectors in a future post, but if you are not familiar they are rules to trigger alerts on certain conditions.

SignalFlow usually determines the single resolution for a job by following these steps:

Determining the resolution for long transformation windows or time shifts that retrieve data, for which only a coarser resolution is available.

Analysis of the resolution of incoming metric time series.

You can control the resolution in part 1 by using the resolution parameter of the data block. Resolution coarseness caused by Part 2 is also worth watching out for, you can experiment and remove time shifts etc and see if this reduces the resolution to what you expect.

Here is an example of setting a 1 hour resolution on a datablock:

data('cpu.utilization', 
     filter=filter('host', 'hostA', 'hostB') and filter('AWSUniqueId', 'i-0403'),
     resolution='1h')

Rollup vs. Analytics

There are two types of aggregation that happen in Signalflow, and this really had me puzzled for a while, as why do I want to aggregate twice, and what happens if I choose a different aggregation for them?

The difference is Rollup aggregates over time, and analytics aggregates over dimensions. Let’s dive into that a bit more.

Rollup

A rollup decides how to aggregate the MTS into the given time resolution. For example given this MTS from earlier, let’s do an average rollup over a 2 minute resolution. The result is every 2 minutes, we take the mean average of the (in this case) 2 values. Here is the original MTS again:

Time	Value	Details
11:54:00 PM	20	Metric Name: cpu.idle Dimensions: aws_region:us-east1
11:55:00 PM	5
11:56:00 PM	17
11:57:00 PM	12
11:58:00 PM	11
11:59:00 PM	0
12:00:00 AM	18
12:01:00 AM	23

Here is the rollup:

Time	Value	Rollup
11:54:00 PM	20	12.5
11:55:00 PM	5	12.5
11:56:00 PM	17	14.5
11:57:00 PM	12	14.5
11:58:00 PM	11	5.5
11:59:00 PM	0	5.5
12:00:00 AM	18	20.5
12:01:00 AM	23	20.5

For the example we only looked at one MTS, but of course the rollup applies to all of the combined MTS in the query. The resolution as mentioned earlier is determined by factors including the original stream resolution, the specified resolution in the data block, and any other operations that may make the resolution courser.

Analytics

Analytics methods allow you to aggregate again—this time over dimensions. By default, i.e. if you don’t use analytics, you get a stream for every single combination of dimensions. You may sometimes see charts with so many colored lines it is hard to read. These probably have not been aggregated using analytics.

You can aggregate all of those lines into a single line using an aggregation function of your choice. Choices of function include count, floor, mean, delta, minimum and maximum. More details are here.

The example below shows this, on the left, without analytics, there are 2 MTS shown as separate lines. If we then apply the mean analytic function, we get a single line with the mean:

As signalflow code, these would look something like:

data('demo.cpu.utilization').publish('chart1')
data('demo.cpu.utilization').mean().publish('chart2')

Maybe you don’t want to collapse all lines into one line? Then you can also choose to group by dimensions. For example if you have MTS with dimensions for cpu number and host, you could do this:

data('demo.cpu.utilization').mean(by=['host']).publish()

Which would give you the average cpu utilization across cpus for each host.

Hopefully this helps you get started with Signalflow. Finally here are some entry points into Splunk documentation if you want to read more:

Image by PublicDomainPictures from Pixabay

Blocking large requests in Go using http library

posted on December 27, 2025December 27, 2025tagged as Go, http

If your server never expects over 100kb in a single request, you can block oversized requests to protect against abuse. You could block them in a reverse proxy, e.g. NGINX, but you can also block them in application code. Here I will talk about the latter, since this can be useful if you don’t have a reverse proxy, or you need fine control over the limiting.

Blocking a large request, by counting the bytes

Let’s say you want to block requests over 100kb. This is something I looked into for a recent project. My first thought when looking into this was to replace the body with a reader that closes after it consumes 100kb and throws an error. As it happens there is reader built into the http library called MaxBytesReader that does exactly this, so you can do something like this.

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
    maxBytesReader := http.MaxBytesReader(w, r.Body, maxBytes)
    ...
}

The great thing about doing it as a reader is you don’t really impact performance that much, you do the read you’d do anyway. This is as opposed to say reading the whole body into memory, then realizing it is 100mb, wasting memory, and then rejecting the request.

One thing to note is that the user of maxBytesReader needs to determine what to do when the reader returns the error reporting the max size has been reached. It will need to determine that if it is a too large error then return the response code. Like this:

var maxBytesError *http.MaxBytesError

body, err := io.ReadAll(maxBytesReader)
if err != nil {
	if errors.As(err, &maxBytesError) {
		w.WriteHeader(http.StatusRequestEntityTooLarge)
		fmt.Fprintf(w, "Content Too Large")
	}
}

Next, let’s make the typical case more efficient. Most requests will send the size of their payload in the header, so we can block earlier if we know the request will be too big. Let’s look into that.

Blocking a large request using `content-length` header

The HTTP/1.1 content-length header is an highly recommended (but technically optional) field to indicate how much data you intend to send in a request or response.

We can use this header to block a request that is too large before even reading the body. However we can’t trust that it will be set, or if set that is is accurate, so you will want to combine this with counting the bytes.

Combining the two methods, looks like this:

http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

	if r.ContentLength > maxBytes {
		w.WriteHeader(http.StatusRequestEntityTooLarge)
		fmt.Fprintf(w, "Content Too Large")
		return
	}

	maxBytesReader := http.MaxBytesReader(w, r.Body, maxBytes)

        ...
}

That is it. We are all done! Now let’s now visit a couple of things related to this of interest…

Additional Notes

Sometimes the `http` server will check body length, but don’t rely on that.

Go’s http server (see transfer.go) will limit the reader to only read a number of bytes equal to realLength which is usually set to content-length, but only if the content-length is supplied, and there are various conditions around whether it will do it or not. This is a good feature, but not 100% reliable for the purpose of blocking large requests to protect your service.

Here is the code inside the Go http library that does this:

func readTransfer(msg any, r *bufio.Reader) (err error) {

//...
	switch {
//...
	case realLength > 0:
		t.Body = &body{src: io.LimitReader(r, realLength), closing: t.Close}
//...

LimitReader vs. MaxBytesReader

There are two similar readers for limiting request size, io.LimitReader and http.MaxBytesReader. Here is what the documentation says:

MaxBytesReader is similar to io.LimitReader but is intended for limiting the size of incoming request bodies. In contrast to io.LimitReader, MaxBytesReader’s result is a ReadCloser, returns a non-nil error of type *MaxBytesError for a Read beyond the limit, and closes the underlying reader when its Close method is called.

MaxBytesReader prevents clients from accidentally or maliciously sending a large request and wasting server resources. If possible, it tells the ResponseWriter to close the connection after the limit has been reached.

And for the complete picture, the docs for LimitReader:

LimitReader returns a Reader that reads from r but stops with EOF after n bytes. The underlying implementation is a *LimitedReader.

If we used a LimitReader, we wouldn’t know that the client had sent too much data, and instead continued processing the request, likely hitting an error later because the payload is truncated.

Unit test mocks in Go

posted on July 24, 2025July 24, 2025tagged as Blog, Go, Testing

Recently I have been using Go for the first time seriously. Go is famously easy to learn for programmers from other languages, and I found this to be true. However, performing mocking in Go unit tests required a bit of my head scratching, so I’ve decided to write about it!

This post explores different ways you can mock in Go, and the tradeoffs involved between them.

What is test mocking?

It’s good to go over the definitions of mocks. I am going to use these definitions, taken from Martin Fowler’s article on mocks vs. stubs, which are in turn taken from Gerard Meszaros’s book xUnit Test Patterns, with my minor edits for brevity. Don’t worry about remembering all of this—I’ll recap as I use them:

Test Double is the generic term for any kind of pretend object used in place of a real object for testing purpose, can be one of:

Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists.

Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example).

Stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test.

Spies are stubs that also record some information based on how they were called. One form of this might be an email service that records how many messages it was sent.

Mocks are what we are talking about here: objects pre-programmed with expectations which form a specification of the calls they are expected to receive.

I am going to show how to do various test doubles in Go, some of which are mocks.

Test doubles help reduce the computational cost of a test, and make setup simpler (no need to create a DB, for example). They tend to help make more reliable and focused tests.

Download-a-cool-RSS-feed example program.

We will get started with testing, but we need something to test. The program we will use for our examples downloads the feed of (perhaps) your favorite programming blog. Feel free to change the URL!

It’s simple enough, but there is enough complexity here to demonstrate different types of testing. Here is the code:

package main

import (
	“io”
	“log”
	“net/http”
	"os"
)

func main() {
	err := Download("https://martincapodici.com/feed", "feed.xml")
	if err != nil {
		log.Fatalln(err)
		os.Exit(1)
	}
	log.Println("Download complete")
}

func Download(url string, fileName string) error {
	resp, err := http.Get(url)
	if err != nil {
		return err
	}

	defer resp.Body.Close()

	file, err := os.Create(fileName)
	if err != nil {
		return err
	}
	defer file.Close()

	_, err = io.Copy(file, resp.Body)
	return err
}

You can try this now, by copying this into main.go and running the command:

go run main.go

This code has a function Download(...) that performs IO, both to make an HTTP call, and to save the result to disk. I am looking to unit test this without doing any real IO, so I need to change this a bit so it can be run with or without IO. My first job is to modify the function so that mocked IO is an option.

I will create an interface called IO that will expose CreateFile() and GetUrl() methods. I can then create mock implementations of that interface.

First let’s add the interface that lets this method do the IO things it needs to do:

type IO interface {
	GetUrl(url string) (resp *http.Response, err error)
	CreateFile(fileName string) (file io.ReadWriteCloser, err error)
}

The file is passed using the io library interface ReadWriteCloser, to make it swappable for a test object.

You might notice the absence of an interface for the io.Copy call. Since io.Copy works with interfaces, it is already test-ready.

Here is the io.Copy documentation to prove it works with Reader and Writer interfaces:

// Copy copies from src to dst until either EOF is reached
// on src or an error occurs. It returns the number of bytes
// copied and the first error encountered while copying, if any.
// ...
func Copy(dst Writer, src Reader) (written int64, err error) {

Anyway, lets get this done! We now will implement the real (i.e. not test) methods for the interface. We need a “receiver” object to attach the methods to. Since there is no state we need to keep, this is an empty struct that we will call RealIO:

type RealIO struct{}


func NewRealIO() IO {
	return &RealIO{}
}

Here are its methods:



func (r *RealIO) GetUrl(url string) (resp *http.Response, err error) {
	return http.Get(url)
}

func (r *RealIO) CreateFile(fileName string) (file io.ReadWriteCloser, err error) {
	return os.Create(fileName)
}

The download function can now use the interface instead of direct library calls:

func Download(testIO IO, url string, fileName string) error {
	resp, err := testIO.GetUrl(url)
	if err != nil {
		return err
	}

	defer resp.Body.Close()

	file, err := testIO.CreateFile(fileName)
	if err != nil {
		return err
	}
	defer file.Close()

	_, err = io.Copy(file, resp.Body)
	return err
}

Finally, we can update the main program to provide this. A one line change:

err := Download(NewRealIO(), "https://martincapodici.com/feed", "feed.xml")

Testing the app, using mocks (and friends)

We will now look at a few ways we can test the code above. We’ll start with the pure Go solution, then use a library, and then touch on code generation options.

Testing with vanilla Go: Or, just use the interface

Without importing any external libraries we can create a test double that implements the IO interface, providing test data and recording calls.

However, before we get to that, lets create a fake file as well need that for constructing the test and for the interface implementation.

I have chosen to fake the file using a buffer (bytes.Buffer). A buffer is basically bytes in memory that happens to have the same Read and Write methods as a file—so swapping in a buffer for a file should be easy enough.

Buffer + Close() = What we want

A buffer however doesn’t have a Close method, so we need to add one. But how do we do this when bytes.Buffer is a standard library object we can’t modify?

To make a buffer that is a ReadWriteCloser the following code uses a technique called struct embedding. Struct embedding is syntax sugar, meaning it’s short hand for something that would normally need more code.

Struct embedding lets you to make a struct a field of another struct and have all of the functions defined on that struct be part of the outer struct.

If this is confusing, it should become clear when you see the code or try this code out for yourself and experiment.

With struct embedding we can embed the buffer in a new struct, called InMemoryFile that gets given the Read and Write methods from the buffer. We then add the additional Close method so that InMemoryFile implements the entire ReadWriteCloser interface:

type InMemoryFile struct {
	bytes.Buffer
}

func (f *InMemoryFile) Close() error { return nil }

// + Struct Embedding Magic! InMemoryFile's Read() and Write() functions are implemented behind the
// scenes by Go so we don't have to write them by hand. What do they do? They just call
// the Read() and Write() functions of the buffer.

We will now create a TestIO struct, that will be used to sort-of act like the real IO, and also help us make assertions for the test.

Lets define the struct and a function to create a new instance. We have an InMemoryFile in our struct so we can fake the file:

type TestIO struct {
	file InMemoryFile
}

func NewTestIO() *TestIO {
	file := InMemoryFile{*bytes.NewBuffer(nil)}
	return &TestIO{file}
}

The interface for IO has two methods that we need to implement. The first method is GetUrl, which is set up to return an OK response for a test URL, and otherwise an error:

func (f *TestIO) GetUrl(url string) (resp *http.Response, err error) {
	if url == "http://testurl" {
		resp = &http.Response{
			StatusCode: http.StatusOK,
			Body:       io.NopCloser(strings.NewReader("OK")),
		}
		return resp, nil
	}
	return nil, errors.New("error")
}

The CreateFile method is next, and will return the fake file, or an error:

func (f *TestIO) CreateFile(fileName string) (file io.ReadWriteCloser, err error) {
	if fileName == "testfile" {
		return &f.file, nil
	}
	return nil, errors.New("error")
}

This is a test double object that is pretty hard wired for a specific kind of test. Hint hint! (there is a quiz question later!)

So we now have enough scaffold to do a basic test that simulates a file download, and checks that the mocked file has the correct contents:

func TestDownload(t *testing.T) {
	testIO:= NewTestIO()
	err := Download(testIO, "http://testurl", "testfile")
	if err != nil {
		t.Error(err)
	}
	b := testIO.file.Buffer.String()
	if string(b) != "OK" {
		t.Errorf("expected file to contain 'OK', got %s", string(b))
	}
}

This is only tests one scenario, and doesn’t test different kinds of IO errors. To do that we’d need to extend the test double to do more, and possibly record values that are passed in to them.

Doing this by hand gets tedious, so we will look into how to make this less work with mocking libraries. There are two good options I have seen, and we can weigh up pros and cons after.

Quick quiz

What kind of test double is TestIO? (Revisit the definitions under What is test mocking if needed.)

Click for answer

In my opinion, TestIO is a Spy.

Spies are stubs that also record some information based on how they were called and stubs provide canned answers to calls made during the test, usually not responding at all to anything outside what’s programmed in for the test.

It is not a fake because this object fails outside the test scenario for the URL http://testurl. Neither is it a dummy, since it is used and interacted with. It is not a mock since it is not reprogrammable with expectations.

show less

Use the `stretchr/testify/mock` library

The stretchr/testify/mock library is straightforward to use. It’s hard to beat the example given on https://pkg.go.dev/github.com/stretchr/testify/mock#hdr-Example_Usage for a description of how this library works and how to use it. Please read that first.

In short though the idea is your mock struct embeds the mock.Mock object, which you then use in each method you want to implement in the mock, to pass in the given parameters and return mock values that were set up in the test.

Here is how to use it for our example. First install it:

go get github.com/stretchr/testify/mock

Then in a new test file, say main2_test.go, we can create the mock, along with the interface methods that delegate the work to the mock. Each method passes it’s arguments into Called to tell the mock it was called, and then the mock returns the result that can be deconstructed. Nothing specific is set up yet; we get to control the behavior from the test code.

type MockIO struct {
	mock.Mock
}

func (m *MockIO) GetUrl(url string) (resp *http.Response, err error) {
	args := m.Called(url)
	return args.Get(0).(*http.Response), args.Error(1)
}

func (m *MockIO) CreateFile(fileName string) (file io.ReadWriteCloser, err error) {
	args := m.Called(fileName)
	return args.Get(0).(io.ReadWriteCloser), args.Error(1)
}

I also like add a line that asserts that MockIO has implemented the IO interface correctly. This gives you immediate feedback via the IDE if the interfaces is not properly implemented, plus it prevents the code from compiling if such an issue exists:

var _ IO = (*MockIO)(nil)

With this in the place we can now have a succinct test that defines how the mock should behave, then tests the function with that behavior. The rather haphazard fake implementation used earlier is replaced with just two testIO.On statements:


func TestDownloadUsingMock(t *testing.T) {
	testIO := new(MockIO)

	file := &InMemoryFile{*bytes.NewBuffer(nil)}

	resp := &http.Response{
		StatusCode: http.StatusOK,
		Body:       io.NopCloser(strings.NewReader("OK")),
	}

	testIO.On("GetUrl", "http://testurl").Return(resp, nil)
	testIO.On("CreateFile", "testfile").Return(file, nil)

	err := Download(testIO, "http://testurl", "testfile")
	if err != nil {
		t.Error(err)
	}

}

Hopefully you can see that it is now much easier to test other scenarios without needing to change the mock implementation. In addition to the On/Return pattern shown here there a multitude of behaviors available in the library, as well as the ability to bake in expectations e.g. the function should be called exactly once. There is also a MatchedBy argument you can use to specify a function that determines if an argument, for example, this could be used in the test too:

	testIO.On("GetUrl", mock.MatchedBy(func(url string) bool { return strings.Contains(url, "testurl") })).Return(resp, nil)

These features make it strictly better than building your own fake implementations in my opinion, which some rare exceptions.

An example of where a fake might be preferred is you want the fake to act like the real object by maintaining internal state. For example where the real object talks to a Redis cache, and the fake object uses a crude in-memory cache, and consumers want to test their code integrated with that cache. In a test where the function you are testing is making heavy use of the cache, it might be less attractive to set up all of the possible interactions in a mock.

Mockery

I’ll give a quick shout out to mockery, a tool that generates your stretchr/testify/mock mocks. https://vektra.github.io/mockery/latest

The benefit of this is not typing out a lot of boilerplate, but the downside is perhaps having mocks for things you never need, and adding bloat to your code base.

Check out their documentation if you want to use it. I haven’t yet decided if I prefer Mockery, or hand writing mock objects as needed. If you have an opinion or know of other interesting Go mocking libraries, please let me know, e.g. leave a comment.

Image by Bianca Van Dijk from Pixabay

Production tests: a guidebook for better systems and more sleep

posted on May 13, 2025May 13, 2025tagged as Blog, Devops, SRE

Your customers expect your site to be fully working whenever they need it. This means you need to aim for near-perfect uptime not just for the site itself, but for all features customers may use.

Modern software engineering uses quality control measures such as automated test suites and observability tools (tracing, metrics, and logs) to ensure availability. Often overlooked in this landscape is production tests (also known as synthetics) that can give you immediate notification of failures in production.

Production tests can be setup up with minimal fuss—usually within one sprint—and can provide a high return on investment. In this post, I will cover how to best set up production tests and how they can help with reliability, deployments, and observability.

While I have always liked production tests, I got a real appreciation for them at Atlassian, where they are used extensively and are called “pollinators”, and I have seen first hand how they can give early warnings of problems, which can be fixed before the become incidents.

What are production tests?

A production test is any automated test that runs on the production environment. The test runs on a frequent schedule so that an on-call engineer can respond quickly. Typically, they run every minute. The test might use a headless browser to emulate user actions, or it may use an API directly to emulate the actions of browser code or backend service.

The production test should run in a reasonable time. I suggest 30 seconds or less, so that you can run the test easily once per minute. A test that takes longer than 30 seconds is probably to complex anyway for a production test. How the test deals with failure is up to your team. It could integrate with your on call paging system, send a Slack notification, or just log an error into your logging systems.

How do production tests help?

Productions tests help make your production environment more reliable by giving immediate warning of a regression. This means you can potentially fix issues before a customer discovers them.

In addition, the production test can be used as a canary before deployments, and as such acts as an integration test. The test can detect regressions that are caused by mismatches with other services, such as API shape or error-handling issues. You can run the test when deploying to development and staging environments too, to get a warning of an issue during development.

You also make it easier to debug production issues by having production tests. If you have an issue you are investigating, knowing which production tests are working and are failing gives you insight into what the issue might be. If you rely on other team’s services, and they also have production tests, you can look at their tests too to help with diagnostics.

Having production tests will reduce the time to recovery for any incident where a human has to fix it, as the engineer learns about the issue sooner, and has more information at hand to resolve it.

Important design considerations for production tests

For production-tests to be worth having, they need to be well-thought-out. If the test keeps failing it’ll probably get silenced and ignored. Even if the test is reliable it can cause other problems, such as resource usage impacting downstream systems. You may even have to change your systems to be a bit more testable. Here are some tips to consider when setting up your production tests:

Keep your production tests basic

Your production tests should cover less ground than your automated test suites. You want the tests to be reliable enough that they don’t waste your time with false alerts, which leads to frustration, lost time and possibly disabling of alerts. The goal of a production test is to be the canary that something has gone badly wrong. It gives you a head start on fixing it, hopefully before a customer gets affected.

To illustrate the required simplicity, here are some examples of what I think are good candidates for a production test:

Log in, and confirm you are on the home page, showing that user’s name.
Load the main editor of your app and type in Hello. Confirm Hello is there. Reload the browser and confirm Hello is still there.
Call 4 API endpoints to do CRUD operations for your microservice, using possibly some fake data in the API.
Ping /health and check for 200 response code within 250ms

Contrast this with these more complex and problematic production tests:

Run through a 25-step test that checks all the main functionality of the editor, asserting that elements are the correct size and are in the correct position.
Test that if you make call in quick succession to the API, they will be added to the database in the same order.
Check that the credit card page can differentiate between Visa and Mastercard based on the card number.
Ping /health and check for 200 response code within 1.5ms

The first examples are good because they are quick, reliable, and simple. They are not too functionality-specific, are so are unlikely to be broken by feature changes.

The bad examples on the other hand have problems:

The 25-step test will likely be flaky due to browser automation quirks, and feature changes. You will spend a lot of time investigating failures to the test, and concluding “It was just a one-off” or “Oh! It’s because we moved a button”.

The API calls in quick succession will sometimes fail due to network conditions changing the order in which the requests are received. It is not necessary to do something sophisticated in a production test—your goal is to know if something is horribly broken, not detect subtle timing issues.

The credit card test is probably OK in terms of not being flaky, but is a little too specific. You can almost entirely de-risk a bug in credit card UI behavior with tests that run before deployment, so do that instead.

The health check test expecting a fast response is likely to fail often enough to cause alarms that have no meaningful response. There are better ways to monitor latency in production that I will cover in a future post.

But… try go get some decent coverage

You are not aiming for anything near 100% code coverage, in fact code coverage doesn’t really matter here. That said, production tests should cover more than just “load the home page”. This is an art, but a guiding idea here is “if there is a serious problem, how likely are my production tests to detect it”. You are balancing that up with “how frequently will I get false reports of an outage”.

You also may want to think about the value of the things you are testing. Is it a minor feature used by 1% of users, or is it the page where new customers sign up? The latter is particularly important for growth companies who rely a lot on customer self-servicing to try out their software.

You don’t have to get this completely right from the beginning. You can add more tests next week. Furthermore, you can edit your existing tests, or even remove them too. Err on the side of too little coverage to begin with and consider adding more later. This way your team will get used to owning, tweaking and responding to production tests, without a cacophony of alarm bells!

As a rough guide, I would say test 3-5 simple things to begin with, with the goal to eventually move towards a reasonable amount of coverage. What is reasonable? You’ll find out after trying things out, and discussing outcomes with your team. There is no correct answer here.

Production tests are not health checks, but may overlap with them

A health check is usually a simple check that the server is running. For example, if using Node.js and Koa or Express, you might add a health route that just returns a success response on any invocation.

The purpose of this check is to assess basic server health. It allows load balancers to know which nodes to send traffic to, and deployment pipelines to know when a deployment has completed (or failed).

We expect these health checks to fail in production, possibly without affecting the customer. For example if a machine goes offline, the load balancer will detect this and stop sending traffic to that node. However, if there is no adverse effect of this on the consuming service or customer then there is no urgent problem. The system has self-healed.

Calling this health check on a node as a production test is not advisable, as it will cause false alarms. Even if it didn’t, a health check is not a good indicator of user experience.

The term “Health check” is also sometimes used to refer to checks that do more that just check the server is up. E.g., they may check dependencies such as storage systems, caches, queues and so on. A production test that calls such a health check could give early warning of an issue before it becomes noticeable by customers.

Be mindful of how your production tests affect observability

Running a test every minute, or 1440 times a day, will show up quite a lot in logs, metrics, and traces. This is often a good thing, because regions or services with very low traffic are now a bit more “observable” than they would be without such tests.

The downside is it may add costs by keeping resources spun up in those regions. Another downside is that being fake traffic, and sometimes requiring fake data (such as a fake user ID) can add noise to the logs you will sometimes need to filter out.

Fake data considerations

Consider a monolithic application — a single server that serves a website and all of its functionality. You need to write a test that logs in, enters some data in a field, saves it and checks it got saved. Such a test has a few challenges.

Firstly you need to decide how it logs in. Does it have a real account? If it does—what stops that account from expiring (e.g., from a free trial). You may need a discount code or special “fake credit card” that the system knows is for testing. If it is a “fake” account then how does that work? Is it hard coded somewhere.

The test is now generating data on each run. Will the success or failure of a previous run affect the current run of the test. Will running the test thousands of times a week use up storage space?

Think about these issues when planning the tests.

The tempting thing is to do the easiest thing to get it done. E.g., create an account using your staff email, worry about storage later, just get it working. This can come back to haunt you if the test breaks later, and you need to deal with upgrading the account, or maybe the member of staff whose email was used is on leave.

Making a production system testable may take some work and have special switches, such as user fields, or feature flags so that the behavior can be slightly different to facilitate the test.

If you use microservices, or a monolith with a separate auth service, there are ways to avoid needing a test user account. For example if your service uses a JSON Web Token (JWT) to authenticate, the authentication server could support logins for test systems, and your service can declare which tests it is authorizing to call it.

Three strikes before an alarm

If you have enough production tests, and run them on enough systems for long enough you will get false alerts. These can be caused by all sorts of things, network problems that only affect the test side, quirks in browser automation, or a genuine problem that heals itself two seconds later.

The simple way to avoid getting these kinds of false alerts is to wait for three consecutive failures before raising any alarm.

You might be giving me a side eye right now though, because should we be ignoring these false alerts? Probably not. If you have a team ritual of looking through operations data, you can incorporate non-alerting production rest failures into that ritual. The idea is that you prioritize it like any regular work, rather than making it an urgent thing that requires people to work out of hours, or stop their regular work.

Pros and cons of production tests

What is good about testing in production? I have hinted at a few of these already, however let’s get into the details:

Real world testing: Test suites—including unit tests and locally-run integration tests—that run on every build are definitely needed. However, even with all of there is nothing quite like test-driving the finished car, and that is what production tests are for. The proof of the pudding is in the eating.
Quality Control: The main reason for running tests is to know something is broken. Despite all the automated and manual testing you do on your machine and in staging environments, production is always unique. There are some things that only happen they way they do in prod. And even if your staging environment is precisely the same, generally speaking production will have more traffic. Running tests in production let you know if certain things are working in production.
Troubleshooting: If there is an incident where customers are impacted by a bug or outage, the tests are very useful. They may be failing or passing, but either way, their current state tells you something about the system, and can help narrow down the problem.
Observability for low traffic regions: If you deploy your service to multiple regions and some of those regions don’t get much traffic, the production tests will create synthetic traffic. Your observability metrics such as latency and reliability percentiles will be more meaningful with even just 1000 calls per day, than with close to zero traffic.
Safer Deployments: The same tests you use to monitor production can double up as tests you run to accept a blue/green deployment. These tests are then acting as continuous integration tests of last resort, adding to the assurance that your deployment won’t cause big issues.
Reuse in other environments: Just as you can use these tests for safer deployments, you can also use them in dev and staging environments, to get early detection of issues that your build pipeline cannot detect, such as integration issues with other services.

There are, of course some disadvantages to using production tests:

Setup and teardown challenges: Tests run on the real system, so you cannot wipe the database before each run. You need to figure out how to best set up the specific scenario you want to test for, and how to clean up after the test.
They sometimes need setup of scenarios: For example you have a test than you can upgrade your account, but you need an account set up that needs to be upgraded, and a fake payment method that will be accepted. These scenarios may mean changing production code to support them too.
They can be flaky: Any test that runs on a real system can fail from time to time for various reasons. For example network issues between the test and the service or occasional timing issues in the browser the test uses. Flakiness of tests needs to be taken into account if you intend to wake someone up when the test fails.
They cause resource usage and costs: Running tests do cost. Often tests are run across different geographic regions to ensure they are tested well, and hundreds of times a day. That compute cost adds up.
Human cost in maintaining tests: Not every test fail is an issue, but it does need to be investigated. Too many tests could leave you with a full-time position monitoring and making those tests more robust.

Production tests, vs. observability

You can also “test production” though monitoring of real traffic, and looking for problems there. This deserves a post or series of its own, but in short you can check for things like:

Latency, e.g., alert me when the top 99-percentile of latency for a particular endpoint is > 200ms, for 3 periods in a row.
Reliability, e.g., alert me when more than 0.1% of requests fail with a 5xx code.
Assertions, e.g., alert me when a code path that is considered unexpected or impossible has been triggered.
Failures, e.g., alert me when a customer couldn’t perform a task because they got an error.

The good thing about observability-based alerts is they are very simple to set up. It may need some minor code changes, and then setting up detectors in observability tools, that take look for and take an action when the condition is met.

These tests pair well with production tests. They have a different purpose though, since they detect problems after they have happened, and they piggyback on existing system use. They can be helped by production tests creating additional synthetic traffic which they can then monitor, which lets you keep checking things when natural traffic levels fall off.

There is no “versus” though! You would do well to have both. For a particular need it might be better to use a production test or better to use observability. A rule of thumb is if observability can meet the requirements of alerting you to a problem, then it will be easier to set up and tune to your needs than a production test.

Summary

Phew! We covered a lot of ground there. In a nutshell adding well-designed production tests to your systems will help in a number of ways. You’ll get earlier warnings of issues, you will be able to fix them faster, you will get observability benefits, and you can also use production tests as a deployment rollout test.

It is worth adding production tests if you are not doing so already—it is a relatively small task to fit in to your planning, and you will reap rewards. If you are already using them, keep reviewing them, and see what you need to add, remove and tweak to get the most value from them as your systems evolve.

Happy testing!

LanguageTool — Check your writing the open-source way

posted on May 10, 2025May 10, 2025tagged as Blog, Editing, Writing

If you write anything (so that’s a yes) then an in-browser spelling and grammar checker can be handy. Even if all you write is emails!

A checker that is free and good, and easy to use, is LanguageTool. As a bonus, it is open-source too. The open-source version is a bit more technical to set up, and if you don’t want to do that, they have a paid cloud version. However, it isn’t too hard to get going, as we will see below.

1. Install the browser extension

Install the LanguageTool Chrome Extension or Firefox Extension.

This will work as-is, and you can start using the tool to check your spelling on WordPress, Google Docs, Gmail and most other apps.

However, it will be limited to a certain number of words, and certain features.

You can either 1. live with that, 2. upgrade to the paid cloud version, or 3. perform the following steps to run it locally for free.

2. Run a local LanguageTool server

The instructions to do this are here: https://dev.languagetool.org/http-server

If you are lucky enough to use a Mac, there is a simple brew installation. Otherwise, I found the Docker method quite convenient, if you already have Docker installed. (If you don’t it is worth installing Docker Desktop or Rancher, as it is pretty handy for many things, including ephemeral “installations” of software like LanguageTool for example).

The docker images are maintained by different people, so there are a few options. One of these is https://github.com/meyayl/docker-languagetool, which I got working well locally. The command to do so is there, but I will repeat it here:

docker run -d \
  --name languagetool \
  --restart unless-stopped \
  --cap-drop ALL \
  --cap-add CAP_CHOWN \
  --cap-add CAP_DAC_OVERRIDE \
  --cap-add CAP_SETUID \
  --cap-add CAP_SETGID \
  --security-opt no-new-privileges \
  --publish 8081:8081 \
  --env download_ngrams_for_langs=en \
  --env MAP_UID=783 \
  --env MAG_GID=783 \
  --read-only \
  --tmpfs /tmp \
  --volume $PWD/ngrams:/ngrams \
  --volume $PWD/fasttext:/fasttext \
  meyay/languagetool:latest

3. Tell your extension to use the local server

Click the LT logo on your browser. It is either on the toolbar, or nested inside the extensions menu (jigsaw icon). It looks like this:

Click the settings (gear icon), and then scroll down to Advanced settings (only for professional users) and under LanguageTool server, choose Local server and click Save.

4. Try it out

Open your favorite digital parchment, such as Google Docs, Confluence, or Notion, or anything else. For a quick try-out, it works here too: https://www.editpad.org/. Whatever you decide, when you open the page you should see a small blue circle with a tick, somewhere in the bottom right of the editing space.

Type something with an intentional mistake, e.g., “Helo”, and you should see the circle go red, and the mistake is highlighted. You are now being watched! In a good way.

Coding Resources

posted on December 22, 2023May 3, 2025tagged as Blog

A list of resources I have found useful for programming. Mainly for my own reference later:

Icons, quite a few resources for this!
- Iconfinder – search engine, but seems to let you download the SVG version for free unlike flaticon
- Bootstap icons
- Feather icons
- Majest icons
- Unicons
- Heroicons (and v1.heroicons.com)
- Iconoir
- Iconizer
- css.gg
- Phosphor icons
- Streamlinehq icons
- Tabler Icons
- Ionicons
- Remixicon
- Eva-icons
- Mingcute icons
- Tetrisly icons
- Doodle Icons

Running local Python code on a remote GPU (using modal and lob.py)

posted on July 27, 2023July 27, 2023tagged as Blog, MachineLearning

I explored in a previous post how to run nanoGPT on modal – both the training and sampling. It was successful, but tiresome. There were a lot of changes to the downloaded code which made me unhappy. If I want to try different projects out that are on Github etc. I don’t want to be doing a lot of coding and fiddling just to get the existing code to run. I just want to run it as if I am running it locally. This is where my script lob.py comes in!

What is `lob.py`?

This is a Python script that provides a fairly easy way (all things considered!) to run your local code on a cloud GPU. It does this by running your code on Modal, and handles some of the logistics of doing so, such as uploading source code.

Let’s explore what it does, by doing what took 1.5 blog posts in 1 blog post, and train and run nanoGPT on Modal.

Using `lob.py` to train and run nanoGPT.

First, clone nanoGPT, and download lob.py from a public Gist (I may make this a full Github repo later, we will see).

git clone https://github.com/karpathy/nanoGPT
cd nanoGPT
wget -O lob.py https://gist.githubusercontent.com/mcapodici/eaa39861affe75be13badefbe9e05079/raw/bbc9e3cbb692277ffcf18406c61805685bf70d25/lob.py

Now set up a python environment you favourite way. I will use venv in this example:

python3 -m venv .
source bin/activate

Now you might want to add the following to .gitignore to avoid have lots of changes show up (this might differ if you used another Python environment tool)

bin
lib
lib64
share

Now install modal and log in, using their standard instructions:

pip install modal-client
modal token new

We will now set up the lob.py for our requirements. The version we downloaded is already set up for nanoGPT, but let’s review the contents of it’s parameters.

Setting up `lob.py` run parameters

The first one is just selecting what GPU you want to use. For nanoGPT, the cheapest one t4 is plenty enough for the task:

# Choose one of: "t4", "a10g", "inf2", "a100-20g", "a100" or None
gpu="t4"

Next we define the commands that run. These are run after copying all the local code files to the server and changing directory into that folder. We have a single command for each stage, but you can have multiple.

commands={
    'prepare': ['python data/shakespeare_char/prepare.py'],
    'train': ['python train.py config/train_shakespeare_char.py'],
    'sample': ['python sample.py --out_dir=out-shakespeare-char'],
}

Now we set verbose, which tells us what files are being uploaded, we set the name of the volume (so that we can keep our files for this project separate), a timeout of 60 minutes after which Modal will terminate the job and a list of paths to not upload:

verbose=True
volume_name_prefix="2023-07-27-10-45"
timeout_mins=60
exclude_paths_starting_with=["./.git", "./.github", "./bin", "./lib", "./share"]

Finally we define the image, which is how the container will be set up that will run the program. rsync is needed because it is used to copy up the right files (without losing generated files on the server). In addition we need to do the pip install defined in the README.md of the nanoGPT project:

image = modal.Image \
    .debian_slim() \
    .apt_install("rsync") \
    .pip_install("torch numpy transformers datasets tiktoken wandb tqdm".split(" "))

Train and run using `lob.py`

With all the set up done, running is very simple. Just run these commands one after the other. They correspond to instructions in the readme.md

modal run lob.py --command prepare
modal run lob.py --command train
modal run lob.py --command sample

Here is some output from the final phase:

ISABELLA:
This is the day of this is your land;
But I have been call'd up him been your tent?

DUKE VINCENTIO:
How far of the solemnity? who is wrong'd?
Why should we shame an arms stoop of life?
They will prove his like offence with life
And to be crave a happy model's guilty of his cheeks;
For all his foes, that are gone of me.

Here is the entire output of the 3 commands (click to expand):

Click to expand

(nanoGPT) martin@Capo:~/nanoGPT$ modal run lob.py --command prepare
💾 using volume name: 2023-07-27-10-45-2-aws
✓ Initialized. View app at https://modal.com/apps/ap-4Ed0rtqxmI5GCB0E73QVRT
 ./configurator.py
 ./LICENSE
 ./scaling_laws.ipynb
 ./README.md
 ./train.py
 ./model.py
 ./pyvenv.cfg
 ./sample.py
 ./transformer_sizing.ipynb
 ./lob.py
 ./bench.py
 ./data/shakespeare/prepare.py
 ./data/shakespeare/readme.md
 ./data/openwebtext/prepare.py
 ./data/openwebtext/readme.md
 ./data/shakespeare_char/prepare.py
 ./data/shakespeare_char/readme.md
 ./__pycache__/lob.cpython-38.pyc
 ./assets/nanogpt.jpg
 ./assets/gpt2_124M_loss.png
 ./config/train_shakespeare_char.py
 ./config/eval_gpt2.py
 ./config/eval_gpt2_large.py
 ./config/train_gpt2.py
 ./config/eval_gpt2_medium.py
 ./config/eval_gpt2_xl.py
 ./config/finetune_shakespeare.py
✓ Created objects.
├── 🔨 Created copy.
├── 🔨 Created mount .
└── 🔨 Created mount /home/martin/nanoGPT/lob.py
Command prepare was chosen.
This will run: ['python data/shakespeare_char/prepare.py']
💾 using volume name: 2023-07-27-10-45-2-aws
📁 Running rsync to copy files up to container:
sending incremental file list
./
LICENSE
          1,072 100%    0.00kB/s    0:00:00            1,072 100%    0.00kB/s    0:00:00 (xfr#1, to-chk=33/35)
README.md
         13,534 100%   12.91MB/s    0:00:00           13,534 100%   12.91MB/s    0:00:00 (xfr#2, to-chk=32/35)
bench.py
          4,815 100%    4.59MB/s    0:00:00            4,815 100%    4.59MB/s    0:00:00 (xfr#3, to-chk=31/35)
configurator.py
          1,758 100%    1.68MB/s    0:00:00            1,758 100%    1.68MB/s    0:00:00 (xfr#4, to-chk=30/35)
lob.py
          5,037 100%    4.80MB/s    0:00:00            5,037 100%    4.80MB/s    0:00:00 (xfr#5, to-chk=29/35)
model.py
         16,345 100%   15.59MB/s    0:00:00           16,345 100%   15.59MB/s    0:00:00 (xfr#6, to-chk=28/35)
pyvenv.cfg
             70 100%   68.36kB/s    0:00:00               70 100%   68.36kB/s    0:00:00 (xfr#7, to-chk=27/35)
sample.py
          3,942 100%    1.25MB/s    0:00:00            3,942 100%    1.25MB/s    0:00:00 (xfr#8, to-chk=26/35)
scaling_laws.ipynb
         32,768  12%   10.42MB/s    0:00:00          268,519 100%    1.24MB/s    0:00:00 (xfr#9, to-chk=25/35)
train.py
         14,673 100%   69.22kB/s    0:00:00           14,673 100%   69.22kB/s    0:00:00 (xfr#10, to-chk=24/35)
transformer_sizing.ipynb
         14,579 100%   68.45kB/s    0:00:00           14,579 100%   68.45kB/s    0:00:00 (xfr#11, to-chk=23/35)
__pycache__/
__pycache__/lob.cpython-38.pyc
          2,510 100%   11.73kB/s    0:00:00            2,510 100%   11.73kB/s    0:00:00 (xfr#12, to-chk=18/35)
assets/
assets/gpt2_124M_loss.png
         32,768  29%  153.11kB/s    0:00:00          110,433 100%  516.00kB/s    0:00:00 (xfr#13, to-chk=17/35)
assets/nanogpt.jpg
         32,768  27%   93.29kB/s    0:00:00          118,621 100%  337.73kB/s    0:00:00 (xfr#14, to-chk=16/35)
config/
config/eval_gpt2.py
            208 100%    0.59kB/s    0:00:00              208 100%    0.59kB/s    0:00:00 (xfr#15, to-chk=15/35)
config/eval_gpt2_large.py
            215 100%    0.61kB/s    0:00:00              215 100%    0.61kB/s    0:00:00 (xfr#16, to-chk=14/35)
config/eval_gpt2_medium.py
            216 100%    0.61kB/s    0:00:00              216 100%    0.61kB/s    0:00:00 (xfr#17, to-chk=13/35)
config/eval_gpt2_xl.py
            213 100%    0.60kB/s    0:00:00              213 100%    0.60kB/s    0:00:00 (xfr#18, to-chk=12/35)
config/finetune_shakespeare.py
            645 100%    1.82kB/s    0:00:00              645 100%    1.82kB/s    0:00:00 (xfr#19, to-chk=11/35)
config/train_gpt2.py
            681 100%    1.92kB/s    0:00:00              681 100%    1.92kB/s    0:00:00 (xfr#20, to-chk=10/35)
config/train_shakespeare_char.py
          1,132 100%    3.19kB/s    0:00:00            1,132 100%    3.19kB/s    0:00:00 (xfr#21, to-chk=9/35)
data/
data/openwebtext/
data/openwebtext/prepare.py
          3,170 100%    8.92kB/s    0:00:00            3,170 100%    8.92kB/s    0:00:00 (xfr#22, to-chk=5/35)
data/openwebtext/readme.md
            489 100%    1.38kB/s    0:00:00              489 100%    1.38kB/s    0:00:00 (xfr#23, to-chk=4/35)
data/shakespeare/
data/shakespeare/prepare.py
          1,096 100%    3.08kB/s    0:00:00            1,096 100%    3.08kB/s    0:00:00 (xfr#24, to-chk=3/35)
data/shakespeare/readme.md
            161 100%    0.45kB/s    0:00:00              161 100%    0.45kB/s    0:00:00 (xfr#25, to-chk=2/35)
data/shakespeare_char/
data/shakespeare_char/prepare.py
          2,344 100%    6.58kB/s    0:00:00            2,344 100%    6.58kB/s    0:00:00 (xfr#26, to-chk=1/35)
data/shakespeare_char/readme.md
            209 100%    0.59kB/s    0:00:00              209 100%    0.59kB/s    0:00:00 (xfr#27, to-chk=0/35)
🐍 Using remote python version:
Python 3.8.15
🏃🏽Executing command: python data/shakespeare_char/prepare.py
length of dataset in characters: 1,115,394
all the unique characters: 
 !$&',-.3:;?ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz
vocab size: 65
train has 1,003,854 tokens
val has 111,540 tokens
✓ App completed.
(nanoGPT) martin@Capo:~/nanoGPT$ modal run lob.py --command train
💾 using volume name: 2023-07-27-10-45-2-aws
✓ Initialized. View app at https://modal.com/apps/ap-qsVnQ1NihkY8FsxsG1PlYX
 ./configurator.py
 ./LICENSE
 ./scaling_laws.ipynb
 ./README.md
 ./train.py
 ./model.py
 ./pyvenv.cfg
 ./sample.py
 ./transformer_sizing.ipynb
 ./lob.py
 ./bench.py
 ./data/shakespeare/prepare.py
 ./data/shakespeare/readme.md
 ./data/openwebtext/prepare.py
 ./data/openwebtext/readme.md
 ./data/shakespeare_char/prepare.py
 ./data/shakespeare_char/readme.md
 ./__pycache__/lob.cpython-38.pyc
 ./assets/nanogpt.jpg
 ./assets/gpt2_124M_loss.png
 ./config/train_shakespeare_char.py
 ./config/eval_gpt2.py
 ./config/eval_gpt2_large.py
 ./config/train_gpt2.py
 ./config/eval_gpt2_medium.py
 ./config/eval_gpt2_xl.py
 ./config/finetune_shakespeare.py
✓ Created objects.
├── 🔨 Created copy.
├── 🔨 Created mount .
└── 🔨 Created mount /home/martin/nanoGPT/lob.py
Command train was chosen.
This will run: ['python train.py config/train_shakespeare_char.py']
💾 using volume name: 2023-07-27-10-45-2-aws
📁 Running rsync to copy files up to container:
sending incremental file list
data/shakespeare_char/
🐍 Using remote python version:
Python 3.8.15
🏃🏽Executing command: python train.py config/train_shakespeare_char.py
Overriding config with config/train_shakespeare_char.py:
# train a miniature character-level shakespeare model
# good for debugging and playing on macbooks and such

out_dir = 'out-shakespeare-char'
eval_interval = 250 # keep frequent because we'll overfit
eval_iters = 200
log_interval = 10 # don't print too too often

# we expect to overfit on this small dataset, so only save when val improves
always_save_checkpoint = False

wandb_log = False # override via command line if you like
wandb_project = 'shakespeare-char'
wandb_run_name = 'mini-gpt'

dataset = 'shakespeare_char'
gradient_accumulation_steps = 1
batch_size = 64
block_size = 256 # context of up to 256 previous characters

# baby GPT model :)
n_layer = 6
n_head = 6
n_embd = 384
dropout = 0.2

learning_rate = 1e-3 # with baby networks can afford to go a bit higher
max_iters = 5000
lr_decay_iters = 5000 # make equal to max_iters usually
min_lr = 1e-4 # learning_rate / 10 usually
beta2 = 0.99 # make a bit bigger because number of tokens per iter is small

warmup_iters = 100 # not super necessary potentially

# on macbook also add
# device = 'cpu'  # run on cpu only
# compile = False # do not torch compile the model

tokens per iteration will be: 16,384
found vocab_size = 65 (inside data/shakespeare_char/meta.pkl)
Initializing a new model from scratch
number of parameters: 10.65M
num decayed parameter tensors: 26, with 10,740,096 parameters
num non-decayed parameter tensors: 13, with 4,992 parameters
using fused AdamW: True
compiling the model... (takes a ~minute)
[2023-07-27 07:51:04,437] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:04,979] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:06,088] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:06,384] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:06,831] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:07,271] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:07,723] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:08,019] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:08,464] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:08,757] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:09,196] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
[2023-07-27 07:51:09,489] torch._inductor.utils: [WARNING] using triton random, expect difference from eager
step 0: train loss 4.2874, val loss 4.2823
iter 0: loss 4.2649, time 33065.55ms, mfu -100.00%
iter 10: loss 3.2438, time 103.08ms, mfu 3.62%
iter 20: loss 2.7899, time 103.51ms, mfu 3.61%
iter 30: loss 2.6383, time 104.21ms, mfu 3.61%
iter 40: loss 2.5763, time 103.21ms, mfu 3.61%
iter 50: loss 2.5261, time 105.02ms, mfu 3.60%
iter 60: loss 2.5136, time 104.65ms, mfu 3.60%
iter 70: loss 2.4921, time 104.66ms, mfu 3.60%
iter 80: loss 2.4932, time 103.74ms, mfu 3.60%
iter 90: loss 2.4696, time 104.93ms, mfu 3.59%
iter 100: loss 2.4526, time 104.89ms, mfu 3.59%
iter 110: loss 2.4543, time 104.67ms, mfu 3.58%
iter 120: loss 2.4223, time 102.57ms, mfu 3.59%
iter 130: loss 2.4059, time 105.67ms, mfu 3.58%
iter 140: loss 2.3925, time 104.18ms, mfu 3.58%
iter 150: loss 2.4098, time 103.93ms, mfu 3.58%
iter 160: loss 2.3675, time 104.91ms, mfu 3.58%
iter 170: loss 2.3382, time 104.84ms, mfu 3.58%
iter 180: loss 2.3011, time 105.85ms, mfu 3.57%
iter 190: loss 2.2278, time 105.65ms, mfu 3.57%
iter 200: loss 2.2004, time 105.36ms, mfu 3.56%
iter 210: loss 2.1244, time 107.28ms, mfu 3.55%
iter 220: loss 2.1338, time 107.40ms, mfu 3.55%
iter 230: loss 2.0709, time 105.63ms, mfu 3.54%
iter 240: loss 2.0742, time 104.87ms, mfu 3.55%
step 250: train loss 1.9616, val loss 2.0647
saving checkpoint to out-shakespeare-char
iter 250: loss 2.0277, time 14340.46ms, mfu 3.19%
iter 260: loss 1.9685, time 106.94ms, mfu 3.22%
iter 270: loss 1.9776, time 107.05ms, mfu 3.25%
iter 280: loss 1.9798, time 106.01ms, mfu 3.27%
iter 290: loss 1.9237, time 108.32ms, mfu 3.29%
iter 300: loss 1.8944, time 107.02ms, mfu 3.31%
iter 310: loss 1.8637, time 108.22ms, mfu 3.32%
iter 320: loss 1.8569, time 108.47ms, mfu 3.33%
iter 330: loss 1.8088, time 108.44ms, mfu 3.35%
iter 340: loss 1.7812, time 108.27ms, mfu 3.35%
iter 350: loss 1.8272, time 107.65ms, mfu 3.37%
iter 360: loss 1.7745, time 108.39ms, mfu 3.37%
iter 370: loss 1.7414, time 109.46ms, mfu 3.38%
iter 380: loss 1.7304, time 107.99ms, mfu 3.38%
iter 390: loss 1.7372, time 109.29ms, mfu 3.39%
iter 400: loss 1.7640, time 109.49ms, mfu 3.39%
iter 410: loss 1.6959, time 106.60ms, mfu 3.40%
iter 420: loss 1.7088, time 109.99ms, mfu 3.40%
iter 430: loss 1.6815, time 107.62ms, mfu 3.40%
iter 440: loss 1.6462, time 108.94ms, mfu 3.41%
iter 450: loss 1.6511, time 107.66ms, mfu 3.41%
iter 460: loss 1.6024, time 108.66ms, mfu 3.41%
iter 470: loss 1.6554, time 110.38ms, mfu 3.41%
iter 480: loss 1.6165, time 108.03ms, mfu 3.41%
iter 490: loss 1.6016, time 107.65ms, mfu 3.42%
step 500: train loss 1.5285, val loss 1.7362
saving checkpoint to out-shakespeare-char
iter 500: loss 1.6016, time 11776.34ms, mfu 3.08%
iter 510: loss 1.6162, time 110.05ms, mfu 3.11%
iter 520: loss 1.6020, time 111.90ms, mfu 3.13%
iter 530: loss 1.5657, time 110.67ms, mfu 3.16%
iter 540: loss 1.6203, time 110.58ms, mfu 3.18%
iter 550: loss 1.5671, time 111.13ms, mfu 3.19%
iter 560: loss 1.5651, time 111.00ms, mfu 3.21%
iter 570: loss 1.5745, time 110.94ms, mfu 3.23%
iter 580: loss 1.5401, time 111.85ms, mfu 3.24%
iter 590: loss 1.5016, time 111.08ms, mfu 3.25%
iter 600: loss 1.5180, time 111.45ms, mfu 3.26%
iter 610: loss 1.5552, time 110.42ms, mfu 3.27%
iter 620: loss 1.5292, time 111.15ms, mfu 3.28%
iter 630: loss 1.5179, time 111.15ms, mfu 3.29%
iter 640: loss 1.4779, time 111.21ms, mfu 3.29%
iter 650: loss 1.5043, time 110.39ms, mfu 3.30%
iter 660: loss 1.5145, time 111.28ms, mfu 3.30%
iter 670: loss 1.4491, time 111.07ms, mfu 3.31%
iter 680: loss 1.5118, time 110.52ms, mfu 3.32%
iter 690: loss 1.4611, time 111.86ms, mfu 3.32%
iter 700: loss 1.4802, time 110.82ms, mfu 3.32%
iter 710: loss 1.4568, time 111.52ms, mfu 3.32%
iter 720: loss 1.4481, time 112.24ms, mfu 3.32%
iter 730: loss 1.4214, time 120.96ms, mfu 3.30%
iter 740: loss 1.4311, time 112.04ms, mfu 3.30%
step 750: train loss 1.3611, val loss 1.5957
saving checkpoint to out-shakespeare-char
iter 750: loss 1.4226, time 12240.61ms, mfu 2.97%
iter 760: loss 1.4461, time 112.47ms, mfu 3.01%
iter 770: loss 1.4283, time 111.68ms, mfu 3.04%
iter 780: loss 1.4123, time 111.30ms, mfu 3.07%
iter 790: loss 1.4221, time 111.95ms, mfu 3.10%
iter 800: loss 1.4286, time 111.81ms, mfu 3.12%
iter 810: loss 1.4068, time 113.48ms, mfu 3.14%
iter 820: loss 1.4105, time 113.58ms, mfu 3.15%
iter 830: loss 1.3939, time 111.81ms, mfu 3.17%
iter 840: loss 1.4056, time 113.22ms, mfu 3.18%
iter 850: loss 1.3911, time 111.85ms, mfu 3.20%
iter 860: loss 1.4010, time 113.44ms, mfu 3.21%
iter 870: loss 1.4061, time 112.60ms, mfu 3.22%
iter 880: loss 1.3758, time 116.09ms, mfu 3.22%
iter 890: loss 1.3920, time 113.44ms, mfu 3.22%
iter 900: loss 1.3728, time 113.66ms, mfu 3.23%
iter 910: loss 1.3245, time 113.50ms, mfu 3.23%
iter 920: loss 1.3671, time 114.03ms, mfu 3.24%
iter 930: loss 1.3670, time 112.65ms, mfu 3.24%
iter 940: loss 1.3524, time 111.20ms, mfu 3.25%
iter 950: loss 1.3538, time 112.70ms, mfu 3.26%
iter 960: loss 1.3668, time 114.80ms, mfu 3.26%
iter 970: loss 1.3636, time 115.69ms, mfu 3.25%
iter 980: loss 1.3566, time 112.03ms, mfu 3.26%
iter 990: loss 1.3400, time 113.93ms, mfu 3.26%
step 1000: train loss 1.2762, val loss 1.5256
saving checkpoint to out-shakespeare-char
iter 1000: loss 1.3415, time 12715.10ms, mfu 2.94%
iter 1010: loss 1.3431, time 114.84ms, mfu 2.97%
iter 1020: loss 1.3133, time 114.62ms, mfu 3.00%
iter 1030: loss 1.3367, time 114.21ms, mfu 3.02%
iter 1040: loss 1.3645, time 114.66ms, mfu 3.05%
iter 1050: loss 1.2936, time 114.63ms, mfu 3.07%
iter 1060: loss 1.3426, time 115.89ms, mfu 3.08%
iter 1070: loss 1.3305, time 114.95ms, mfu 3.10%
iter 1080: loss 1.3386, time 116.72ms, mfu 3.11%
iter 1090: loss 1.3539, time 115.43ms, mfu 3.12%
iter 1100: loss 1.3168, time 117.47ms, mfu 3.12%
iter 1110: loss 1.3008, time 116.91ms, mfu 3.13%
iter 1120: loss 1.3026, time 118.77ms, mfu 3.13%
iter 1130: loss 1.3015, time 116.35ms, mfu 3.14%
iter 1140: loss 1.2992, time 112.41ms, mfu 3.16%
iter 1150: loss 1.3126, time 115.89ms, mfu 3.16%
iter 1160: loss 1.3269, time 115.05ms, mfu 3.17%
iter 1170: loss 1.3064, time 116.09ms, mfu 3.17%
iter 1180: loss 1.3226, time 115.55ms, mfu 3.18%
iter 1190: loss 1.2668, time 113.06ms, mfu 3.19%
iter 1200: loss 1.2964, time 113.89ms, mfu 3.20%
iter 1210: loss 1.2739, time 114.14ms, mfu 3.21%
iter 1220: loss 1.3009, time 114.65ms, mfu 3.21%
iter 1230: loss 1.2977, time 116.01ms, mfu 3.21%
iter 1240: loss 1.3042, time 113.79ms, mfu 3.22%
step 1250: train loss 1.2079, val loss 1.4969
saving checkpoint to out-shakespeare-char
iter 1250: loss 1.2753, time 12646.97ms, mfu 2.90%
iter 1260: loss 1.2867, time 115.14ms, mfu 2.93%
iter 1270: loss 1.2715, time 115.65ms, mfu 2.96%
iter 1280: loss 1.2605, time 117.75ms, mfu 2.98%
iter 1290: loss 1.2806, time 115.94ms, mfu 3.00%
iter 1300: loss 1.2990, time 113.14ms, mfu 3.03%
iter 1310: loss 1.2402, time 112.46ms, mfu 3.06%
iter 1320: loss 1.3072, time 113.39ms, mfu 3.08%
iter 1330: loss 1.2705, time 117.34ms, mfu 3.09%
iter 1340: loss 1.2992, time 116.71ms, mfu 3.10%
iter 1350: loss 1.2529, time 114.87ms, mfu 3.12%
iter 1360: loss 1.2679, time 114.92ms, mfu 3.13%
iter 1370: loss 1.2591, time 117.08ms, mfu 3.13%
iter 1380: loss 1.2695, time 116.04ms, mfu 3.14%
iter 1390: loss 1.2550, time 117.59ms, mfu 3.15%
iter 1400: loss 1.2619, time 116.96ms, mfu 3.15%
iter 1410: loss 1.2513, time 117.91ms, mfu 3.15%
iter 1420: loss 1.2738, time 116.81ms, mfu 3.15%
iter 1430: loss 1.2424, time 114.54ms, mfu 3.16%
iter 1440: loss 1.2576, time 114.93ms, mfu 3.17%
iter 1450: loss 1.2388, time 118.07ms, mfu 3.17%
iter 1460: loss 1.2448, time 119.56ms, mfu 3.16%
iter 1470: loss 1.2223, time 118.65ms, mfu 3.16%
iter 1480: loss 1.2091, time 116.77ms, mfu 3.17%
iter 1490: loss 1.2391, time 117.71ms, mfu 3.17%
step 1500: train loss 1.1533, val loss 1.4693
saving checkpoint to out-shakespeare-char
iter 1500: loss 1.1903, time 14834.12ms, mfu 2.85%
iter 1510: loss 1.2334, time 117.89ms, mfu 2.88%
iter 1520: loss 1.2242, time 116.96ms, mfu 2.91%
iter 1530: loss 1.2554, time 116.91ms, mfu 2.94%
iter 1540: loss 1.1952, time 116.01ms, mfu 2.97%
iter 1550: loss 1.2376, time 115.90ms, mfu 2.99%
iter 1560: loss 1.2124, time 115.27ms, mfu 3.02%
iter 1570: loss 1.2289, time 117.66ms, mfu 3.03%
iter 1580: loss 1.2126, time 116.45ms, mfu 3.05%
iter 1590: loss 1.1923, time 117.08ms, mfu 3.06%
iter 1600: loss 1.2019, time 117.95ms, mfu 3.07%
iter 1610: loss 1.2397, time 117.50ms, mfu 3.08%
iter 1620: loss 1.1842, time 119.91ms, mfu 3.08%
iter 1630: loss 1.2066, time 115.21ms, mfu 3.10%
iter 1640: loss 1.2057, time 115.86ms, mfu 3.11%
iter 1650: loss 1.1848, time 117.37ms, mfu 3.12%
iter 1660: loss 1.2201, time 115.62ms, mfu 3.13%
iter 1670: loss 1.2026, time 118.70ms, mfu 3.13%
iter 1680: loss 1.2037, time 116.72ms, mfu 3.14%
iter 1690: loss 1.2076, time 114.73ms, mfu 3.15%
iter 1700: loss 1.1913, time 115.13ms, mfu 3.16%
iter 1710: loss 1.1841, time 117.65ms, mfu 3.16%
iter 1720: loss 1.1854, time 117.87ms, mfu 3.16%
iter 1730: loss 1.1986, time 117.57ms, mfu 3.16%
iter 1740: loss 1.1763, time 117.37ms, mfu 3.16%
step 1750: train loss 1.1028, val loss 1.4603
saving checkpoint to out-shakespeare-char
iter 1750: loss 1.1805, time 12920.15ms, mfu 2.85%
iter 1760: loss 1.1936, time 117.16ms, mfu 2.88%
iter 1770: loss 1.1979, time 119.79ms, mfu 2.90%
iter 1780: loss 1.1955, time 113.89ms, mfu 2.94%
iter 1790: loss 1.1912, time 118.11ms, mfu 2.96%
iter 1800: loss 1.1752, time 117.03ms, mfu 2.98%
iter 1810: loss 1.1647, time 117.42ms, mfu 3.00%
iter 1820: loss 1.1709, time 116.32ms, mfu 3.02%
iter 1830: loss 1.1703, time 116.40ms, mfu 3.04%
iter 1840: loss 1.1592, time 117.01ms, mfu 3.06%
iter 1850: loss 1.1671, time 117.30ms, mfu 3.07%
iter 1860: loss 1.1751, time 118.06ms, mfu 3.08%
iter 1870: loss 1.1444, time 116.93ms, mfu 3.09%
iter 1880: loss 1.1820, time 116.80ms, mfu 3.10%
iter 1890: loss 1.1850, time 114.54ms, mfu 3.11%
iter 1900: loss 1.1317, time 114.22ms, mfu 3.13%
iter 1910: loss 1.1653, time 116.54ms, mfu 3.13%
iter 1920: loss 1.1680, time 115.75ms, mfu 3.14%
iter 1930: loss 1.1419, time 114.94ms, mfu 3.15%
iter 1940: loss 1.1259, time 115.84ms, mfu 3.16%
iter 1950: loss 1.1378, time 118.97ms, mfu 3.16%
iter 1960: loss 1.1573, time 116.48ms, mfu 3.16%
iter 1970: loss 1.1574, time 114.98ms, mfu 3.17%
iter 1980: loss 1.1518, time 114.50ms, mfu 3.18%
iter 1990: loss 1.1529, time 114.88ms, mfu 3.18%
step 2000: train loss 1.0560, val loss 1.4723
iter 2000: loss 1.1313, time 12257.49ms, mfu 2.87%
iter 2010: loss 1.1371, time 115.66ms, mfu 2.90%
iter 2020: loss 1.1267, time 116.53ms, mfu 2.93%
iter 2030: loss 1.1575, time 116.89ms, mfu 2.96%
iter 2040: loss 1.1369, time 116.50ms, mfu 2.98%
iter 2050: loss 1.1200, time 115.13ms, mfu 3.01%
iter 2060: loss 1.1054, time 114.00ms, mfu 3.03%
iter 2070: loss 1.1241, time 116.13ms, mfu 3.05%
iter 2080: loss 1.1240, time 117.11ms, mfu 3.06%
iter 2090: loss 1.1352, time 114.44ms, mfu 3.08%
iter 2100: loss 1.1349, time 115.23ms, mfu 3.10%
iter 2110: loss 1.1359, time 117.54ms, mfu 3.11%
iter 2120: loss 1.1305, time 117.35ms, mfu 3.11%
iter 2130: loss 1.1397, time 116.09ms, mfu 3.12%
iter 2140: loss 1.1365, time 117.49ms, mfu 3.13%
iter 2150: loss 1.1284, time 114.82ms, mfu 3.14%
iter 2160: loss 1.1414, time 116.10ms, mfu 3.15%
iter 2170: loss 1.1367, time 115.65ms, mfu 3.15%
iter 2180: loss 1.1187, time 119.46ms, mfu 3.15%
iter 2190: loss 1.1095, time 112.09ms, mfu 3.17%
iter 2200: loss 1.1194, time 115.60ms, mfu 3.17%
iter 2210: loss 1.1110, time 115.48ms, mfu 3.18%
iter 2220: loss 1.1232, time 114.33ms, mfu 3.19%
iter 2230: loss 1.1231, time 116.60ms, mfu 3.19%
iter 2240: loss 1.1356, time 115.37ms, mfu 3.19%
step 2250: train loss 1.0100, val loss 1.4719
iter 2250: loss 1.1040, time 12193.67ms, mfu 2.88%
iter 2260: loss 1.1040, time 116.32ms, mfu 2.91%
iter 2270: loss 1.1425, time 116.92ms, mfu 2.94%
iter 2280: loss 1.0969, time 115.55ms, mfu 2.97%
iter 2290: loss 1.1535, time 116.32ms, mfu 2.99%
iter 2300: loss 1.1172, time 117.35ms, mfu 3.01%
iter 2310: loss 1.0925, time 116.01ms, mfu 3.03%
iter 2320: loss 1.0992, time 116.60ms, mfu 3.04%
iter 2330: loss 1.1028, time 116.21ms, mfu 3.06%
iter 2340: loss 1.1236, time 114.90ms, mfu 3.08%
iter 2350: loss 1.1000, time 113.61ms, mfu 3.10%
iter 2360: loss 1.1014, time 113.65ms, mfu 3.12%
iter 2370: loss 1.0974, time 114.22ms, mfu 3.13%
iter 2380: loss 1.0846, time 116.35ms, mfu 3.14%
iter 2390: loss 1.0864, time 117.19ms, mfu 3.14%
iter 2400: loss 1.0765, time 113.52ms, mfu 3.16%
iter 2410: loss 1.0720, time 117.45ms, mfu 3.16%
iter 2420: loss 1.0792, time 113.19ms, mfu 3.17%
iter 2430: loss 1.0588, time 116.62ms, mfu 3.17%
iter 2440: loss 1.0537, time 113.97ms, mfu 3.18%
iter 2450: loss 1.0758, time 117.42ms, mfu 3.18%
iter 2460: loss 1.0872, time 115.39ms, mfu 3.19%
iter 2470: loss 1.0853, time 118.00ms, mfu 3.18%
iter 2480: loss 1.0859, time 115.33ms, mfu 3.19%
iter 2490: loss 1.0610, time 113.64ms, mfu 3.20%
step 2500: train loss 0.9590, val loss 1.4886
iter 2500: loss 1.0800, time 12184.79ms, mfu 2.88%
iter 2510: loss 1.0775, time 117.50ms, mfu 2.91%
iter 2520: loss 1.0480, time 115.16ms, mfu 2.94%
iter 2530: loss 1.0608, time 115.74ms, mfu 2.97%
iter 2540: loss 1.0559, time 115.97ms, mfu 2.99%
iter 2550: loss 1.0655, time 115.44ms, mfu 3.02%
iter 2560: loss 1.0615, time 116.20ms, mfu 3.04%
iter 2570: loss 1.0752, time 116.99ms, mfu 3.05%
iter 2580: loss 1.0818, time 116.49ms, mfu 3.07%
iter 2590: loss 1.0710, time 115.88ms, mfu 3.08%
iter 2600: loss 1.0741, time 112.56ms, mfu 3.10%
iter 2610: loss 1.0476, time 117.23ms, mfu 3.11%
iter 2620: loss 1.0413, time 112.69ms, mfu 3.13%
iter 2630: loss 1.0266, time 114.45ms, mfu 3.14%
iter 2640: loss 1.0461, time 116.10ms, mfu 3.15%
iter 2650: loss 1.0662, time 116.35ms, mfu 3.16%
iter 2660: loss 1.0466, time 116.22ms, mfu 3.16%
iter 2670: loss 1.0205, time 116.80ms, mfu 3.16%
iter 2680: loss 1.0463, time 117.87ms, mfu 3.16%
iter 2690: loss 1.0574, time 116.00ms, mfu 3.17%
iter 2700: loss 1.0213, time 115.56ms, mfu 3.17%
iter 2710: loss 1.0412, time 116.20ms, mfu 3.18%
iter 2720: loss 1.0447, time 120.05ms, mfu 3.17%
iter 2730: loss 1.0544, time 115.64ms, mfu 3.18%
iter 2740: loss 1.0265, time 118.42ms, mfu 3.17%
step 2750: train loss 0.9135, val loss 1.5104
iter 2750: loss 1.0351, time 12241.53ms, mfu 2.86%
iter 2760: loss 1.0271, time 116.76ms, mfu 2.89%
iter 2770: loss 1.0300, time 116.32ms, mfu 2.92%
iter 2780: loss 1.0212, time 116.84ms, mfu 2.95%
iter 2790: loss 1.0406, time 116.89ms, mfu 2.97%
iter 2800: loss 1.0119, time 116.11ms, mfu 3.00%
iter 2810: loss 1.0451, time 115.22ms, mfu 3.02%
iter 2820: loss 1.0255, time 116.07ms, mfu 3.04%
iter 2830: loss 1.0377, time 114.26ms, mfu 3.06%
iter 2840: loss 1.0019, time 114.69ms, mfu 3.08%
iter 2850: loss 1.0306, time 115.24ms, mfu 3.10%
iter 2860: loss 1.0237, time 116.07ms, mfu 3.11%
iter 2870: loss 1.0016, time 114.00ms, mfu 3.12%
iter 2880: loss 1.0270, time 117.10ms, mfu 3.13%
iter 2890: loss 1.0150, time 116.35ms, mfu 3.14%
iter 2900: loss 0.9963, time 116.32ms, mfu 3.14%
iter 2910: loss 1.0427, time 115.04ms, mfu 3.15%
iter 2920: loss 1.0190, time 115.45ms, mfu 3.16%
iter 2930: loss 0.9971, time 114.76ms, mfu 3.17%
iter 2940: loss 0.9860, time 114.46ms, mfu 3.18%
iter 2950: loss 1.0231, time 116.47ms, mfu 3.18%
iter 2960: loss 1.0000, time 116.44ms, mfu 3.18%
iter 2970: loss 0.9940, time 113.93ms, mfu 3.19%
iter 2980: loss 1.0014, time 114.07ms, mfu 3.20%
iter 2990: loss 0.9899, time 112.14ms, mfu 3.21%
step 3000: train loss 0.8673, val loss 1.5194
iter 3000: loss 0.9866, time 12198.37ms, mfu 2.89%
iter 3010: loss 0.9940, time 115.19ms, mfu 2.93%
iter 3020: loss 0.9946, time 114.00ms, mfu 2.96%
iter 3030: loss 1.0067, time 116.10ms, mfu 2.99%
iter 3040: loss 1.0279, time 112.95ms, mfu 3.02%
iter 3050: loss 0.9855, time 114.92ms, mfu 3.04%
iter 3060: loss 0.9980, time 114.28ms, mfu 3.06%
iter 3070: loss 1.0166, time 116.64ms, mfu 3.08%
iter 3080: loss 1.0064, time 116.06ms, mfu 3.09%
iter 3090: loss 0.9795, time 116.85ms, mfu 3.10%
iter 3100: loss 0.9986, time 114.91ms, mfu 3.11%
iter 3110: loss 0.9806, time 114.00ms, mfu 3.13%
iter 3120: loss 0.9921, time 115.17ms, mfu 3.14%
iter 3130: loss 0.9779, time 115.70ms, mfu 3.15%
iter 3140: loss 0.9753, time 117.19ms, mfu 3.15%
iter 3150: loss 0.9882, time 117.12ms, mfu 3.15%
iter 3160: loss 1.0155, time 117.83ms, mfu 3.15%
iter 3170: loss 0.9620, time 115.54ms, mfu 3.16%
iter 3180: loss 0.9784, time 115.59ms, mfu 3.17%
iter 3190: loss 0.9972, time 116.21ms, mfu 3.17%
iter 3200: loss 0.9690, time 115.14ms, mfu 3.18%
iter 3210: loss 0.9701, time 117.81ms, mfu 3.18%
iter 3220: loss 0.9613, time 114.30ms, mfu 3.18%
iter 3230: loss 0.9596, time 115.49ms, mfu 3.19%
iter 3240: loss 0.9608, time 116.11ms, mfu 3.19%
step 3250: train loss 0.8239, val loss 1.5611
iter 3250: loss 0.9846, time 12237.43ms, mfu 2.88%
iter 3260: loss 0.9668, time 116.44ms, mfu 2.91%
iter 3270: loss 0.9772, time 116.16ms, mfu 2.94%
iter 3280: loss 0.9445, time 116.97ms, mfu 2.96%
iter 3290: loss 0.9432, time 116.93ms, mfu 2.98%
iter 3300: loss 0.9451, time 115.18ms, mfu 3.01%
iter 3310: loss 0.9571, time 117.17ms, mfu 3.03%
iter 3320: loss 0.9727, time 116.68ms, mfu 3.04%
iter 3330: loss 0.9581, time 112.84ms, mfu 3.07%
iter 3340: loss 0.9502, time 115.18ms, mfu 3.09%
iter 3350: loss 0.9538, time 114.65ms, mfu 3.10%
iter 3360: loss 0.9362, time 115.49ms, mfu 3.11%
iter 3370: loss 0.9606, time 114.58ms, mfu 3.13%
iter 3380: loss 0.9514, time 116.54ms, mfu 3.14%
iter 3390: loss 0.9561, time 114.63ms, mfu 3.15%
iter 3400: loss 0.9596, time 116.01ms, mfu 3.15%
iter 3410: loss 0.9444, time 112.50ms, mfu 3.17%
iter 3420: loss 0.9479, time 116.45ms, mfu 3.17%
iter 3430: loss 0.9430, time 114.03ms, mfu 3.18%
iter 3440: loss 0.9693, time 116.27ms, mfu 3.18%
iter 3450: loss 0.9509, time 115.68ms, mfu 3.19%
iter 3460: loss 0.9453, time 117.12ms, mfu 3.19%
iter 3470: loss 0.9385, time 117.24ms, mfu 3.19%
iter 3480: loss 0.9504, time 115.33ms, mfu 3.19%
iter 3490: loss 0.9106, time 116.81ms, mfu 3.19%
step 3500: train loss 0.7790, val loss 1.5720
iter 3500: loss 0.9011, time 12234.12ms, mfu 2.87%
iter 3510: loss 0.9180, time 116.66ms, mfu 2.91%
iter 3520: loss 0.9251, time 118.31ms, mfu 2.93%
iter 3530: loss 0.9584, time 118.13ms, mfu 2.95%
iter 3540: loss 0.9311, time 116.41ms, mfu 2.98%
iter 3550: loss 0.9215, time 116.94ms, mfu 3.00%
iter 3560: loss 0.9519, time 116.18ms, mfu 3.02%
iter 3570: loss 0.9363, time 118.52ms, mfu 3.03%
iter 3580: loss 0.9419, time 112.09ms, mfu 3.06%
iter 3590: loss 0.9228, time 114.29ms, mfu 3.08%
iter 3600: loss 0.9237, time 113.88ms, mfu 3.10%
iter 3610: loss 0.9118, time 115.23ms, mfu 3.11%
iter 3620: loss 0.9171, time 118.56ms, mfu 3.12%
iter 3630: loss 0.9199, time 115.27ms, mfu 3.13%
iter 3640: loss 0.9228, time 115.28ms, mfu 3.14%
iter 3650: loss 0.9075, time 116.32ms, mfu 3.15%
iter 3660: loss 0.9391, time 118.61ms, mfu 3.14%
iter 3670: loss 0.9375, time 114.86ms, mfu 3.15%
iter 3680: loss 0.9084, time 116.61ms, mfu 3.16%
iter 3690: loss 0.9350, time 115.12ms, mfu 3.17%
iter 3700: loss 0.8690, time 116.40ms, mfu 3.17%
iter 3710: loss 0.8807, time 117.80ms, mfu 3.17%
iter 3720: loss 0.9150, time 115.20ms, mfu 3.18%
iter 3730: loss 0.9002, time 117.32ms, mfu 3.18%
iter 3740: loss 0.9056, time 114.51ms, mfu 3.18%
step 3750: train loss 0.7414, val loss 1.6029
iter 3750: loss 0.9066, time 12211.47ms, mfu 2.87%
iter 3760: loss 0.9350, time 120.05ms, mfu 2.89%
iter 3770: loss 0.9320, time 117.65ms, mfu 2.92%
iter 3780: loss 0.9233, time 116.81ms, mfu 2.95%
iter 3790: loss 0.9007, time 117.14ms, mfu 2.97%
iter 3800: loss 0.9027, time 116.44ms, mfu 2.99%
iter 3810: loss 0.9152, time 116.25ms, mfu 3.01%
iter 3820: loss 0.8884, time 116.74ms, mfu 3.03%
iter 3830: loss 0.9023, time 114.30ms, mfu 3.05%
iter 3840: loss 0.8860, time 115.67ms, mfu 3.07%
iter 3850: loss 0.8922, time 115.66ms, mfu 3.09%
iter 3860: loss 0.8694, time 116.67ms, mfu 3.10%
iter 3870: loss 0.8979, time 117.24ms, mfu 3.11%
iter 3880: loss 0.8883, time 116.62ms, mfu 3.11%
iter 3890: loss 0.8921, time 117.78ms, mfu 3.12%
iter 3900: loss 0.8796, time 117.16ms, mfu 3.13%
iter 3910: loss 0.8910, time 113.17ms, mfu 3.14%
iter 3920: loss 0.8786, time 116.18ms, mfu 3.15%
iter 3930: loss 0.8860, time 114.44ms, mfu 3.16%
iter 3940: loss 0.8757, time 116.20ms, mfu 3.16%
iter 3950: loss 0.8747, time 115.90ms, mfu 3.17%
iter 3960: loss 0.9120, time 115.20ms, mfu 3.18%
iter 3970: loss 0.8914, time 114.93ms, mfu 3.18%
iter 3980: loss 0.9046, time 116.67ms, mfu 3.18%
iter 3990: loss 0.8761, time 116.38ms, mfu 3.19%
step 4000: train loss 0.7089, val loss 1.6181
iter 4000: loss 0.8590, time 12261.06ms, mfu 2.87%
iter 4010: loss 0.8836, time 117.03ms, mfu 2.90%
iter 4020: loss 0.8819, time 116.28ms, mfu 2.93%
iter 4030: loss 0.8776, time 119.57ms, mfu 2.95%
iter 4040: loss 0.8846, time 116.67ms, mfu 2.97%
iter 4050: loss 0.8734, time 114.50ms, mfu 3.00%
iter 4060: loss 0.8648, time 116.99ms, mfu 3.02%
iter 4070: loss 0.8631, time 115.76ms, mfu 3.04%
iter 4080: loss 0.8867, time 115.88ms, mfu 3.06%
iter 4090: loss 0.8479, time 114.74ms, mfu 3.08%
iter 4100: loss 0.8987, time 117.46ms, mfu 3.09%
iter 4110: loss 0.8641, time 117.32ms, mfu 3.10%
iter 4120: loss 0.8797, time 116.96ms, mfu 3.10%
iter 4130: loss 0.8548, time 117.32ms, mfu 3.11%
iter 4140: loss 0.8778, time 115.72ms, mfu 3.12%
iter 4150: loss 0.8723, time 116.71ms, mfu 3.13%
iter 4160: loss 0.8512, time 116.37ms, mfu 3.14%
iter 4170: loss 0.8705, time 117.28ms, mfu 3.14%
iter 4180: loss 0.8739, time 115.51ms, mfu 3.15%
iter 4190: loss 0.8654, time 115.80ms, mfu 3.16%
iter 4200: loss 0.8579, time 117.02ms, mfu 3.16%
iter 4210: loss 0.8754, time 115.09ms, mfu 3.17%
iter 4220: loss 0.8595, time 116.92ms, mfu 3.17%
iter 4230: loss 0.8805, time 114.63ms, mfu 3.18%
iter 4240: loss 0.8654, time 115.65ms, mfu 3.18%
step 4250: train loss 0.6784, val loss 1.6442
iter 4250: loss 0.8753, time 12213.09ms, mfu 2.87%
iter 4260: loss 0.8598, time 114.89ms, mfu 2.90%
iter 4270: loss 0.8634, time 117.44ms, mfu 2.93%
iter 4280: loss 0.8539, time 1225.34ms, mfu 2.67%
iter 4290: loss 0.8318, time 112.19ms, mfu 2.73%
iter 4300: loss 0.8307, time 117.42ms, mfu 2.78%
iter 4310: loss 0.8535, time 117.93ms, mfu 2.82%
iter 4320: loss 0.8466, time 114.60ms, mfu 2.86%
iter 4330: loss 0.8621, time 116.64ms, mfu 2.89%
iter 4340: loss 0.8286, time 118.11ms, mfu 2.92%
iter 4350: loss 0.8388, time 117.73ms, mfu 2.94%
iter 4360: loss 0.8612, time 115.78ms, mfu 2.97%
iter 4370: loss 0.8579, time 112.69ms, mfu 3.00%
iter 4380: loss 0.8347, time 117.20ms, mfu 3.02%
iter 4390: loss 0.8672, time 115.48ms, mfu 3.04%
iter 4400: loss 0.8440, time 115.27ms, mfu 3.06%
iter 4410: loss 0.8630, time 118.10ms, mfu 3.07%
iter 4420: loss 0.8611, time 116.09ms, mfu 3.08%
iter 4430: loss 0.8399, time 115.48ms, mfu 3.10%
iter 4440: loss 0.8514, time 115.89ms, mfu 3.11%
iter 4450: loss 0.8549, time 117.24ms, mfu 3.12%
iter 4460: loss 0.8366, time 117.52ms, mfu 3.12%
iter 4470: loss 0.8530, time 114.76ms, mfu 3.14%
iter 4480: loss 0.8332, time 114.35ms, mfu 3.15%
iter 4490: loss 0.8419, time 116.28ms, mfu 3.15%
step 4500: train loss 0.6544, val loss 1.6613
iter 4500: loss 0.8533, time 12221.51ms, mfu 2.84%
iter 4510: loss 0.8460, time 112.49ms, mfu 2.89%
iter 4520: loss 0.8332, time 112.16ms, mfu 2.93%
iter 4530: loss 0.8519, time 115.10ms, mfu 2.96%
iter 4540: loss 0.8431, time 115.25ms, mfu 2.99%
iter 4550: loss 0.8768, time 113.19ms, mfu 3.02%
iter 4560: loss 0.8361, time 113.91ms, mfu 3.04%
iter 4570: loss 0.8384, time 112.27ms, mfu 3.07%
iter 4580: loss 0.8528, time 117.01ms, mfu 3.08%
iter 4590: loss 0.8554, time 115.77ms, mfu 3.10%
iter 4600: loss 0.8282, time 115.90ms, mfu 3.11%
iter 4610: loss 0.8602, time 115.36ms, mfu 3.12%
iter 4620: loss 0.8341, time 116.40ms, mfu 3.13%
iter 4630: loss 0.8136, time 115.64ms, mfu 3.14%
iter 4640: loss 0.8465, time 117.28ms, mfu 3.14%
iter 4650: loss 0.8606, time 116.78ms, mfu 3.15%
iter 4660: loss 0.8533, time 114.05ms, mfu 3.16%
iter 4670: loss 0.8358, time 113.38ms, mfu 3.17%
iter 4680: loss 0.8533, time 115.58ms, mfu 3.18%
iter 4690: loss 0.8411, time 114.61ms, mfu 3.18%
iter 4700: loss 0.8206, time 115.47ms, mfu 3.19%
iter 4710: loss 0.7934, time 114.49ms, mfu 3.20%
iter 4720: loss 0.8374, time 116.00ms, mfu 3.20%
iter 4730: loss 0.8223, time 114.59ms, mfu 3.20%
iter 4740: loss 0.8268, time 115.66ms, mfu 3.20%
step 4750: train loss 0.6359, val loss 1.6802
iter 4750: loss 0.8078, time 12216.16ms, mfu 2.89%
iter 4760: loss 0.8218, time 117.52ms, mfu 2.92%
iter 4770: loss 0.8055, time 115.92ms, mfu 2.95%
iter 4780: loss 0.8098, time 115.62ms, mfu 2.97%
iter 4790: loss 0.8343, time 113.37ms, mfu 3.00%
iter 4800: loss 0.8281, time 112.90ms, mfu 3.03%
iter 4810: loss 0.8343, time 116.12ms, mfu 3.05%
iter 4820: loss 0.8217, time 115.77ms, mfu 3.07%
iter 4830: loss 0.8183, time 116.31ms, mfu 3.08%
iter 4840: loss 0.8372, time 115.24ms, mfu 3.10%
iter 4850: loss 0.8202, time 112.28ms, mfu 3.12%
iter 4860: loss 0.8241, time 116.29ms, mfu 3.13%
iter 4870: loss 0.8103, time 116.90ms, mfu 3.13%
iter 4880: loss 0.8312, time 112.69ms, mfu 3.15%
iter 4890: loss 0.8079, time 115.64ms, mfu 3.16%
iter 4900: loss 0.8054, time 114.67ms, mfu 3.17%
iter 4910: loss 0.8333, time 114.81ms, mfu 3.18%
iter 4920: loss 0.8248, time 114.41ms, mfu 3.18%
iter 4930: loss 0.8068, time 116.15ms, mfu 3.19%
iter 4940: loss 0.8003, time 115.11ms, mfu 3.19%
iter 4950: loss 0.8276, time 115.75ms, mfu 3.19%
iter 4960: loss 0.8309, time 116.05ms, mfu 3.20%
iter 4970: loss 0.7909, time 112.48ms, mfu 3.21%
iter 4980: loss 0.7935, time 115.85ms, mfu 3.21%
iter 4990: loss 0.8205, time 116.26ms, mfu 3.21%
step 5000: train loss 0.6210, val loss 1.7005
iter 5000: loss 0.8196, time 12154.77ms, mfu 2.89%
✓ App completed.
(nanoGPT) martin@Capo:~/nanoGPT$ modal run lob.py --command sample
💾 using volume name: 2023-07-27-10-45-2-aws
✓ Initialized. View app at https://modal.com/apps/ap-GmvdCJZhE0rBXwoUqZzliE
 ./configurator.py
 ./LICENSE
 ./scaling_laws.ipynb
 ./README.md
 ./train.py
 ./model.py
 ./pyvenv.cfg
 ./sample.py
 ./transformer_sizing.ipynb
 ./lob.py
 ./bench.py
 ./data/shakespeare/prepare.py
 ./data/shakespeare/readme.md
 ./data/openwebtext/prepare.py
 ./data/openwebtext/readme.md
 ./data/shakespeare_char/prepare.py
 ./data/shakespeare_char/readme.md
 ./__pycache__/lob.cpython-38.pyc
 ./assets/nanogpt.jpg
 ./assets/gpt2_124M_loss.png
 ./config/train_shakespeare_char.py
 ./config/eval_gpt2.py
 ./config/eval_gpt2_large.py
 ./config/train_gpt2.py
 ./config/eval_gpt2_medium.py
 ./config/eval_gpt2_xl.py
 ./config/finetune_shakespeare.py
✓ Created objects.
├── 🔨 Created copy.
├── 🔨 Created mount .
└── 🔨 Created mount /home/martin/nanoGPT/lob.py
Command sample was chosen.
This will run: ['python sample.py --out_dir=out-shakespeare-char']
💾 using volume name: 2023-07-27-10-45-2-aws
📁 Running rsync to copy files up to container:
sending incremental file list
./
__pycache__/
🐍 Using remote python version:
Python 3.8.15
🏃🏽Executing command: python sample.py --out_dir=out-shakespeare-char
Overriding: out_dir = out-shakespeare-char
number of parameters: 10.65M
Loading meta from data/shakespeare_char/meta.pkl...


Clown:
So you will be a fellow a servant. He have not strange?

MONTAGUE:
No more, my lord: he's never court.

ANGELO:
Go on,
You shall be a common that little might show.

ANGELO:
Alack to the queen.

ANGELO:
Here's no more.

ANGELO:
But you are near like for this. Go with this good
to sweet the king Willoughby princely.

ANGELO:
Even she is the common of the king thrust for the
great of this isle of the field?

ISABELLA:
I shall to be a word: but it is so.

ISABELLA:
I tell you, my lord; I wi
---------------

MenEnius, I must thou continue me
That I have sent forth and late.

CORIOLANUS:
Change thee,
One that frowns should sold. A silly thing it
To see thee. Thou art a maid?

LARTIUS:
My very time:
Here's a Volscian, sir, come, sir, sir, thou art a very root
To an unballable: 'tis no world of this selfname
Whose lasts our voices' shore, his heir
Than he nothing Camillo is the office
To her sent a submissips; and will not so the silk
Even with the crown, to prove a watch, if
I can rest; and think it e
---------------

MARIANA:
I beseech you, I would weep.
My lord, I do dinner and I would you bring you.

ROMEO:
It is a fellow till your sons, growing as
I dream'd your gracious senIUS:
You take to the covert and in this blame
In my breath in men man is walling out of it;
And still you with that with the prisoner.

Nurse:
What does not?
Madam? O my heart with you? how I was,
Provost, 'tis quickly have been dead!'

LADY CAPULET:
Well, that I have quite to the town throughese man
I' the instrument, so much of thee

---------------

The straight will keep the sweetest princes for kove,
As every top the officers, who was straight
To score the shame, and patricians.

BENVOLIO:
I am worn. The lark of Rome, Romeo comes
Shall bear the news.

MERCUTIO:
O her love! her wife, thou with him:
Here's companion, Lucio.

MERCUTIO:
An if you be fair?

BENVOLIO:
The rest, are no like a sound fool.

MERCUTIO:
O, true, thou hast dead! most dead, an old transport
Than do thou threat parliaments!

ROMEO:
I fear, how she was ever briever and h
---------------

BUCKINGHAM:
Then, here is not some stinlight for your grace,
Not yet ever so much every are there
But in the people's parliament. Some in the man
Being moody to swimit it of you.

BUCKINGHAM:
No other more come to make his most noble great;
And, by her name well appear the nounted soldiers,
Being common your worship not and under his soundly.

GLOUCESTER:
Here comes her than the tedious course.

KING EDWARD IV:
And then moves so he that would as dead.
Go on, and will keep our grace; and then for
---------------


MENENIUS:
He's a letter for Corioli: he's
a gracious lord.

First Citizen:
Away, away!

MENENIUS:
I had a lack of the people, the
people! Take him dead, I had said to chance.

CORIOLANUS:
What's change to the people? have you not
to have an abroad with the seeming noise the proud?

Second Citizen:
Conceit it, Backingham, that walk appointed in the pride.

CORIOLANUS:
I cannot think with all the tomb, that he does
The vilgary of mine and his bed,
Her disgraced him, dear him, to do weep with the 
---------------

Shepherd:
Heaven if so, steel it not.

Clown:
I have not to many of this all, and whose
the shores: your blestings are bried intents
none world, and have you so?

Clown:
Will you not banishment, that you have deserved them, who
seen you do, sir, which I must conto any fool, a
rashning from this is little bustrains. Your
wisdom to see alone all. Proceeds to the limb; a while I woo do it alone.

Clown:
My lord, hold I tell you, and I would prophesy to
hold that which you were look not like a fool.
---------------

ISABELLA:
True, if not me, or with mine honour,
I do not banish you with her rest,
That I should not hear me with her eyes,
The world is beholding in enterial.

ANGELO:
The battle hours I do disgrace the sword
Of her tongue, and to such a brother world in earth!

ISABELLA:
Now, sir, if it breathed not, I will tell you fry them?

ANGELO:
See you, tell me this winter'd divines.

ISABELLA:
How doth here this, my lord; but it is an abuse
Within this approbation, and she was much a maid
But in the ol
---------------

GLOUCESTER:
Lords, I can not see the other sad:
And with her, which she shall seem for their swordship.

WARWICK:
Bloody maids the feast.

KING LEWIS XI:
No, my lord, Bona broke, and go to thee.

YORK:
Exeter, I mean this, by my soul rest.

KING LEWIS XI:
The king my great soul in any summer's love,
And in no hour aughty of your streets of justice;
For your blood with their grief,
And all the ground shall be so grown'd along:
Since will I shame at supper us.

GLOUCESTER:
No, no, my lord, be it w
---------------

How changed the drops of eight of his soul?

ISABELLA:
This is the day of this is your land;
But I have been call'd up him been your tent?

DUKE VINCENTIO:
How far of the solemnity? who is wrong'd?
Why should we shame an arms stoop of life?
They will prove his like offence with life
And to be crave a happy model's guilty of his cheeks;
For all his foes, that are gone of me.

DUKE VINCENTIO:
Their love and press sent to it.

CLAUDIO:
Brother is a visitor.

ANGELO:
Why, 'tis my lord, I die the Cap
---------------

Notes about how `lob.py` works

The script works by doing the following:

The is a function called copy (I should probably have called this copy_and_run) that runs on the remote machine, and copies all of the changed files from the local file system to the remote machine.
To this function we bind 2 directories that appear on the remote system:
- /source/code is a mount that is a copy of your local folder, except for the folders mentioned in exclude_paths_starting_with. This is a (I think) temporary and (for sure) read-only folder.
- /root/code which is a “network file system” which has been set up as persistent and read/write.
The copy function uses rsync to copy everything that has changed from the mount to the persistent file system. This means that future runs are quicker (they only need to copy changed files) and the running code can save data, such as model snapshots, and recover them on future runs.
Once copy has done with rsync it changes directory into the /root/code folder, then runs your commands.