Why Your Server Breaks at Two Instances and How to Fix It

Scaling vertically means buying a bigger server, and it works right up until you hit the ceiling of a single machine or that machine dies and takes your whole service with it. Scaling horizontally means running many copies of your application behind a load balancer, which gives you more capacity and survives the loss of any single instance. The catch is that most applications are not actually ready for it. Code that worked on one instance often breaks the moment a second appears, because it secretly relied on staying the only copy: it stored sessions in memory, kept a counter in a local variable, or wrote uploads to its own disk. The principle that makes horizontal scaling possible is statelessness. This article explains what stateless really means in practice, where hidden state hides in a typical Node app, and how to move that state somewhere all instances can share.

Vertical Versus Horizontal

There are two ways to handle more load. Vertical scaling, or scaling up, means giving one machine more CPU, memory, or faster disk. It is simple because your application does not change at all, but it has hard limits: there is a biggest machine you can buy, it costs disproportionately more at the top end, and a single machine is a single point of failure. Horizontal scaling, or scaling out, means running multiple instances of your application across machines and distributing requests among them. It scales much further, lets you add cheap commodity instances as load grows, and tolerates the failure of any single instance because the others keep serving. The trade-off is that your application must be designed to run as many independent copies that share nothing locally. Most teams use both: scale up to a sensible size, then scale out for capacity and redundancy.

What Stateless Really Means

A stateless service is one where no single instance holds information that another instance would need to handle a request correctly. Any request can go to any instance and get the same result, because all the meaningful state lives in shared external systems like a database or cache rather than in the memory or filesystem of a particular process. This does not mean your system has no state; databases obviously hold state. It means the application tier is stateless, acting as interchangeable workers that read from and write to shared stores. The benefit is that instances become disposable: you can add one, remove one, or have one crash, and nothing important is lost because nothing important lived only there. This is also what makes rolling deploys, autoscaling, and self-healing possible. Designing for statelessness is less about clever code and more about relocating any data that lingers inside a single process.

Where Hidden State Hides

In a typical Node app, state sneaks in through patterns that feel harmless on one instance. In-memory session stores are the classic offender: the default Express session store keeps sessions in process memory, so a user who logs in on instance A appears logged out when the load balancer routes their next request to instance B. In-memory caches give inconsistent results across instances and grow unbounded. Uploaded files written to local disk exist only on the instance that received them. Background timers or cron-like intervals run on every instance, so a job meant to run once runs once per instance. WebSocket connections live on whichever instance accepted them. Rate limiters that count in memory undercount because each instance keeps its own tally. The audit is straightforward: anything you store in a variable, on disk, or in a process-local structure is suspect once a second instance exists.

Externalizing Session State

Sessions are usually the first thing to break and the first thing to fix. The clean solution is to stop storing session data in process memory and put it in a shared store that every instance can reach, most commonly Redis. With a Redis-backed session store, any instance can look up a user's session by the session id in their cookie, so the load balancer is free to send requests anywhere. An alternative is to avoid server-side sessions and use stateless tokens like JWTs, where the user's identity travels in a signed token the server verifies without storing anything. Each approach has trade-offs: server-side sessions are easy to revoke but require the shared store, while tokens scale effortlessly but are harder to invalidate before they expire. Either way, the goal is identical: a request must not depend on landing back on the instance that first authenticated the user.

Avoiding Sticky Sessions

When teams first hit the session problem, a tempting shortcut is sticky sessions, where the load balancer pins each user to one instance so their in-memory session stays reachable. It works, but it undermines the very benefits you scaled out to get. If that instance dies or is replaced during a deploy, every user pinned to it loses their session and is logged out. Load also distributes unevenly, because a burst from heavy users can pile onto one instance while others sit idle, and you cannot remove an instance without disrupting its users. Stickiness turns disposable instances back into precious ones, reintroducing the fragility horizontal scaling was meant to eliminate. Treat it as a temporary crutch at best. The proper fix is to externalize the state so stickiness is unnecessary and any instance can serve any request, which makes rolling deploys and autoscaling safe.

Handling Files And Uploads

Writing uploaded files to the local filesystem is fine on one instance and broken on many, because a file saved on instance A simply does not exist on instance B, so a later request to fetch it fails depending on routing. The solution is to store files in a shared object store rather than local disk. Managed services like Amazon S3 are the common choice, and self-hosted options like MinIO give you an S3-compatible store on your own VPS. Your application uploads to and serves from that store, so all instances see the same files and instances stay disposable. To keep large file traffic off your app servers, you can have clients upload directly to the object store using presigned URLs, where your server only issues a short-lived signed permission. Local disk in a scaled app should hold only ephemeral, instance-local scratch data that nothing else depends on.

Background Jobs And Schedulers

Background work is a quiet trap when you scale out. A setInterval or an in-process cron that sends a daily report will fire on every instance, so with five instances the report goes out five times. The fix is to move background work out of the request-serving instances and into a dedicated job system. A queue like BullMQ on top of Redis lets any instance enqueue work while separate workers consume it, and the queue guarantees each job is handled once rather than once per instance. For scheduled jobs, use a single scheduler or a distributed lock so only one instance triggers the task. This separation also protects API latency, because heavy work no longer competes with request handling. The general rule is that anything time-based or asynchronous should be coordinated through a shared system rather than left to run inside every instance.

Load Balancing And Health

With a stateless application, a load balancer distributes incoming requests across your instances, and the strategy can be as simple as round robin or as smart as least connections. The load balancer needs to know which instances are healthy so it can stop sending traffic to one that is failing, which is where the health endpoint becomes essential: the balancer polls it and removes any instance that stops responding. Distinguish a liveness check, which says the process is running, from a readiness check, which says the instance is prepared to serve, so you do not send traffic to one still warming up. During deploys, drain connections gracefully by marking an instance unready, letting in-flight requests finish, then stopping it, so users never hit a half-stopped process. On a single VPS, a reverse proxy like Nginx can play the load balancer role across local container instances.