Keeping Infrastructure Lean While Serving Real Users

Introduction

A large share of small products run on infrastructure designed for a company they are not. The architecture diagram has a queue, a cache, three services, a container orchestrator, and a plan for a second region. The traffic is a few thousand requests a day. The bill is real, the complexity is real, and the load that justified any of it never arrived.

The usual response to a high bill is to shop for a cheaper instance. That is the last lever, not the first. Price per unit of compute is the smallest number in the equation. The larger numbers are how much work you do, how often you do it, and whether you needed to do it at all.

1. The Order of Levers

The cheapest request is the one you never serve. When cost becomes a problem, work through these in order and stop when the number is acceptable.

1. Do not do the work. Remove the feature, remove the poll, remove the report nobody reads. Free.
2. Cache it. Do the work once, serve the answer many times. Close to free.
3. Do it less often. Recompute hourly rather than per request. Batch it. Nearly free.
4. Do it on cheaper hardware. Right-size, move to a cheaper tier, use a cheaper runtime.
5. Negotiate on price. Reserved capacity, committed spend, a different provider.

Teams reliably start at step five, because it looks like the finance problem it appears to be. Step five shaves a fraction off a number that steps one through three could have cut to a remainder.

A dashboard that recomputes a full aggregate on every page load is a step one and step two problem. Moving it to a cheaper instance is a step four answer to a question nobody asked.

2. Caching Is the Single Biggest Lever

Caching means storing the result of expensive work so you can hand it back without redoing it. Most applications use one layer of it when five are available.

The layers, from outermost to innermost:

• CDN and static assets at the edge. A CDN, or content delivery network, is a set of servers positioned near your users. Images, scripts, stylesheets, and fonts never change once built, so your origin should never see a request for them.
• HTTP cache headers. Instructions your response carries telling browsers and proxies how long the answer stays valid. A Cache-Control header is one line of configuration and eliminates entire classes of repeat requests.
• Full page and fragment caching. Store the rendered output of a page, or an expensive part of one. A pricing page rarely needs rebuilding per visitor.
• Application level memoisation. Holding a computed value in memory for the life of a request, so the same calculation does not run four times in one response.
• Database query caching. Storing the result of an expensive query in something like Redis, keyed on its inputs.

Browser  ->  CDN  ->  Load balancer  ->  App cache  ->  App  ->  Query cache  ->  DB
             ^                          ^                        ^
             cheapest place to stop     next cheapest            last resort

Every layer you stop at is work the layers behind it never do. Stop at the CDN and your database never learns the user exists.

Now the honest part. Cache invalidation, knowing when a stored answer has gone stale and must be thrown away, is genuinely hard. It is where caching bugs live. Two strategies keep it manageable:

• Time based expiry. The answer lives for a fixed duration and then it dies. Simple, predictable, and correct for anything that tolerates being briefly out of date.
• Key based invalidation. Include something that changes in the cache key, such as a record's last updated timestamp. Update the record and the old key is never asked for again. No deletion logic to get wrong.

I would reach for time based expiry first, and add explicit invalidation only where staleness matters to a user. Teams cache far less than they could because they assume everything must be current to the millisecond, and almost nothing is.

3. Managed Platforms Versus Self-Hosting

The self-hosting argument is that a virtual machine costs less than a managed platform for the same workload. Often true, and mostly irrelevant, because it leaves out the two largest costs.

The first is your time: patching, upgrades, TLS certificates, backups, restores you hope you never test in anger. For a small team that is the scarcest thing you have.

The second is on-call. Self-hosting means someone owns the pager, and not just the incident. The phone stays on and the weekend is provisional.

For a small team, I would reach for a managed platform first and self-host only where a specific requirement forces it. The line item is higher and the total is usually lower. Reconsider when the managed premium could fund the engineer who would replace it, which is later than most people expect.

4. You Are Not Google

Kubernetes, microservices, multi-region active-active, and a service mesh are excellent solutions to problems created by scale and organisational size. Adopted before you have either, they are a tax rather than an asset.

• Kubernetes is a container orchestrator: it runs many services across many machines with automatic placement and recovery. Before you have that, it is a second system to operate.
• Microservices split an application into independently deployable services. The real benefit is organisational: teams ship without coordinating. With one team, you have bought network calls and distributed failure in exchange for nothing.
• Multi-region active-active serves live traffic from several regions at once. Expensive, and it forces you to confront data consistency across distance. Buy it when an SLA requires it.
• A service mesh manages service to service networking. It presumes enough services for that to be a problem.

A modular monolith goes remarkably far: one deployable unit, with firm internal boundaries and no shortcuts across them. One deploy, one log stream, one place to debug, function calls instead of network hops. If you ever do need to extract a service, the seams are already there.

Complexity should be paid for by a problem you can name and measure.

5. The Database Is Usually the Bill

Databases are typically the largest and least examined line item. Three things account for most of the waste.

Connection pooling. Every database connection costs memory on the server. Applications that open connections freely exhaust a database long before they exhaust its CPU. A pooler, such as PgBouncer for Postgres, keeps a small set of connections and shares them. Close to mandatory for anything serverless, where each instance would otherwise bring its own.

The N+1 query problem. You fetch a list, then run one more query per item to get related data. Fifty posts becomes fifty-one queries. Invisible in development with ten rows, catastrophic in production, and usually the biggest source of unnecessary database load.

Add an index before adding an instance. An index is a lookup structure that lets the database find rows without scanning the table. A missing index on a filtered column turns a fast query into a full scan, and the symptom looks exactly like an undersized instance. Read the query plan before the pricing page. An index is free and permanent; a bigger instance is a monthly cost hiding the same bug.

6. Egress, Assets, and the Sneaky Bill

Egress is data leaving the provider's network, and it is the line item nobody forecasts. Compute and storage are estimated in advance; bandwidth is discovered on the invoice. It scales with traffic and response size, and is often priced well above what the same bytes cost inside the network.

The practical defences:

• Serve static assets from a CDN, where bandwidth is cheaper and most requests never reach your origin.
• Keep chatty services in the same region and network. Cross-region traffic between your own components is billable and easy to create by accident.
• Compress responses. Gzip or Brotli on text is one configuration change.

Images are usually the largest thing you ship. Formats such as WebP or AVIF cut file size substantially against JPEG and PNG at comparable quality. Serve them sized for the surface that displays them, not as the original upload, and lazy load anything below the fold.

7. Move Work Out of the Request Path

If a user's request triggers something slow that they do not need the result of, it does not belong in the request. Sending an email, generating a report, resizing an image, calling a third party API. Put it on a queue and let a worker do it.

Two things improve at once. Response times drop, because the request returns as soon as the work is recorded. And you stop sizing web servers for the peak of your slowest task: they stay cheap and numerous, while workers can be a few larger machines chewing through a backlog at their own pace. A retry on a queued job is also far safer than a retry on a live request.

8. Scale to Zero and Right-Sizing

Scale to zero means running no instances when there is no traffic, and paying nothing. It is the correct default for genuinely spiky or low traffic workloads: internal tools, staging environments, webhook receivers, anything with a pronounced business hours pattern.

The trade-off is the cold start, the delay while a stopped service starts up to serve the first request. For a background job or an internal tool, nobody notices. For a user-facing page, it is felt. The question is whether the workload can absorb one.

Right-sizing is the unglamorous companion. Most instances are substantially overprovisioned, because someone chose the size once, early, without data. That guess is now a permanent monthly cost.

Look at actual CPU and memory utilisation over a representative period, including your real peak. If a machine idles at a small fraction of its capacity through its busiest hour, it is the wrong machine. Size to peak plus headroom, not to a multiple of peak.

9. Observability Can Outgrow Compute

Logging platforms charge by volume ingested. Debug logging left on in production, a chatty library, and a request logger that prints full payloads can produce a logging bill that exceeds the servers generating the logs. It happens gradually and nobody watches the number.

Some discipline that costs nothing:

• Sample high volume events. You do not need every successful health check. Keep all the errors.
• Set retention deliberately. Most logs are useful for days, not months. Long retention should be a decision, not a default.
• Use metrics for counting. A counter is far cheaper than the log lines you would count to derive it.
• Turn debug logging off in production. It is on more often than anyone admits.

And set budgets and alerts before you need them. Configure spend alerts on day one, at a threshold that would concern you, so a runaway process announces itself in hours rather than at the end of the month. This takes minutes and is the highest return configuration change available.

Conclusion

Lean infrastructure is not about being cheap. It is about not paying for work that does not need doing, and not operating systems that solve problems you do not have.

The order matters more than any single technique. Remove the work first. Cache what remains. Do it less often. Only then think about hardware, and only last about price. Most savings worth having are found before you reach the pricing page.

Architecture should follow load. Measure first, find the actual constraint, and add complexity only when the numbers force you to.

Build for the traffic you have, and stay ready for the traffic you get.