The Fine Art of Cheap Infrastructure
It used to be that to even start a business in Tech, you needed expensive servers. Cloud computing solved this, and refined it as workloads can be sliced into smaller and smaller pieces, only run on demand.
We bought into the first of these, AWS Lambda, quite some time ago. At the time, only DynamoDB was available as a persistence layer, however the use of AWS API Gateway made it so that our Lambdas, to all intents and purposes, looked like a real server. Since then we’ve seen on-demand relational databases, with the only real cost being a slight amount of latency as the resources are provisioned.
For our UI, it’s even cheaper. Combining S3 and CloudFront, we have low-latency global distribution for pennies. The trick is deploying it correctly.
As such, we’ve managed to keep our costs extremely low – like, $8/month low – which is quite good given global reach. Fact is that the most expensive thing to operate right now is this Blog, as we’ve not yet put the engineering effort in to run WordPress on demand as well. This post goes into the – quite simple – technical details of how we did it.
The Backend
Aurora Serverless, AWS Lambda fronted by API Gateway V2, and a small Golang library that converts Lambda events into HTTP requests.
Because growth is assumed, but not guaranteed, we started our effort knowing that we wanted to use the smallest compute unit we could; AWS Lambda at the time. We also knew that there is a cost inflection point, where Lambda’s no longer make sense and a container solution would be necessary. The size of our original approach, using a NodeJS based lambda, worked….. however the memory overhead and startup time raised costs more than we wanted to.
Mux, and Lambda Event Conversion
Thus we switched to a golang based mux server, which is both far, far smaller as it doesn’t carry all the node_modules with it, and has a far faster startup time, and smaller memory footprint. We also decided to run the workload on ARM, as that is not only more performant, but also cheaper.
The key to running this server as a server for local development, but as a lambda in production, is a library called aws-lambda-go-api-proxy – which is selectively enabled to convert lamdba events to http.Request objects, and back. It has its glitches, however we were able to circumvent them easily enough.
Waking up the database
The next issue to address was: How to handle the cold-start time of Aurora Serverless. Since we were cost-optimizing, we didn’t set the database up to run continuously, and at any point in time an API call could arrive that needed to be satisfied.
We also noticed that we could not block on the database connection; the lambda would simply flail and error out, which led to error responses that were out of our control. Instead, started the server, and connected on-demand inside the lambda itself. That way we could still accept http requests, but we had control over the error messages that were returned in case Aurora hadn’t started yet.
And, for usability, we also implemented a simple GET /status endpoint, which pings our database and blocks until it gets a response. Our UI calls this when it first starts up, and if you’re lucky you’ll see some amusing loading messages while it waits for the database to return.
Keeping the database awake
Lambdas have a certain lifespan after they’ve been started, so they can handle multiple events. Aurora has a similar TTL, however we’ve found that the timing between each was inconsistent, and we couldn’t risk the database suddenly shutting down on us if the lambda was still expecting it to be available. Thankfully, a simple keepalive thread fixed this – as long as the lambda was still active, it would trigger a query every minute or so to ensure that Aurora restarted its keepalive timer.
The Frontend
S3, CloudFront, and careful tuning of Caching Headers
Here, our path was already paved for us. It’s trivially easy to host a SPA (in our case, written in Angular) on S3. A bit of CloudFront caching, and we have a rapid response rate for our assets.
The trick here, is to make sure that the correct HTTP headers are sent when the static resource is requested. Both because CloudFront charges by traffic, but also because we can tell the customer’s browser to permanently cache our code, substantially reducing our external traffic. The rule of thumb is: only index.html should not be cached; everything else should use a tool like webpack to version its files, and set them to be cached indefinitely. That way, the browser will always check to see if there’s a new version of the application, but if there isn’t it won’t bother pulling down the JS/CSS files again.