The Fine Art of Cheap Infrastructure
It used to be that to even start a business in Tech, you needed expensive servers. The rise of cloud computing solved this, and refined it by slicing workloads into smaller and smaller pieces that only run on demand.
The project adopted Amazon Web Services Lambda quite some time ago. At the time, only DynamoDB was available as a persistence layer, however the use of Amazon API Gateway made the Lambdas, to all intents and purposes, look like a real server. Since then the industry has seen on-demand relational databases, with the only real cost being a slight amount of latency as the resources provision.
For the UI, it’s even cheaper. Combining s3 and cloudfront provides low-latency global distribution for pennies. The trick is deploying it correctly.
As such, costs stay extremely low—like, $8/month low—which is quite good given global reach. The most expensive thing to operate right now is this blog, as the site hasn’t yet put the engineering effort in to run WordPress on demand as well. This post goes into the quite simple technical details of how the team does it.
The backend
Aurora Serverless, Amazon Web Services Lambda fronted by API Gateway V2, and a small Go library that converts Lambda events into HTTP requests.
Because the plan assumes growth, but it’s not guaranteed, the design targeted the smallest compute unit available at the time, Amazon Web Services Lambda. There is also a cost inflection point where Lambda no longer makes sense and a container solution becomes necessary. The original approach used a Node.js-based Lambda. It worked, however the memory overhead and startup time raised costs more than desired.
Mux and Lambda event conversion
Thus the project switched to a Go-based Mux server, which is far smaller because it doesn’t carry
all the node_modules with it, and it has a faster startup time and smaller memory footprint. The
workload now runs on Arm because it’s both more efficient and cheaper.
The key to running this server as a server for local development, but as a Lambda in production, is
a library called
aws-lambda-go-api-proxy, which
converts Lambda events to http.Request objects and back. It has its glitches, however the team was
able to work around them easily enough.
Waking up the database
The next issue to address was how to handle the cold-start time of Aurora Serverless. Since the system optimizes cost, the database doesn’t run continuously, and at any point in time an API call could arrive that the system still needs to satisfy.
Blocking on the database connection caused the Lambda to flail and error out, which led to error responses that were out of app control. Instead, the server starts first and connects on-demand inside the Lambda itself. That way HTTP requests still get accepted, but the app controls the error messages that the app returns in case Aurora hadn’t started yet.
And, for usability, the system implements a simple GET /status endpoint, which pings the database and blocks until it gets a response. The UI calls this when it first starts up, and if you’re lucky you’ll see some amusing loading messages while it waits for the database to return.
Keeping the database awake
Lambdas have a certain lifespan after they start, so they can handle multiple events. Aurora has a similar time to live, however the timing between each was inconsistent, and the database couldn’t shut down while the Lambda still expected it to be available. Thankfully, a simple keep-alive thread fixed this. As long as the Lambda was still active, it triggered a query every minute or so to ensure that Aurora restarted its keep-alive timer.
The frontend
s3, cloudfront, and careful tuning of Caching Headers
Here, the path was already paved. It’s trivially easy to host a single-page app on s3 with Angular. A bit of cloudfront caching provides a rapid response rate for assets.
The trick here is to make sure the server sends the correct HTTP headers when it receives a static resource request. cloudfront charges by traffic, and proper headers tell the customer’s browser to permanently cache the code, substantially reducing external traffic. The rule of thumb is to cache everything except index.html. Use a tool like Webpack to version files and cache them indefinitely. That way, the browser always checks to see if there’s a new version of the app, but if there isn’t it doesn’t pull down the JS/CSS files again.