The Front Door and the Phone Book: API Gateways and Service Discovery

Split a monolith into services and two new problems appear immediately. First, clients now face a dozen endpoints instead of one, each with its own auth and address, which is miserable to consume. Second, those services no longer live at fixed addresses; they scale and restart on new hosts constantly, so nothing can rely on a hardcoded location. An API gateway solves the first by giving clients one stable front door that routes inward, and service discovery solves the second by acting as a live phone book the gateway and services consult to find each other. Coming from a single Express app where everything was one process at one address, these are the patterns that keep a distributed system usable. This article covers what a gateway does, where cross-cutting concerns belong, how discovery works, registration patterns, health and failure handling, and the traps that turn the gateway into a liability.

One Door For Many Services

An API gateway is a single entry point that sits in front of all your backend services and routes each incoming request to the right one, so clients deal with one address and one contract instead of hunting down individual services. From the outside it looks like one API; behind it, the gateway maps paths or hosts to internal services and forwards the request, often reshaping it along the way. This is more than a reverse proxy because it is aware of your services as logical units and typically owns concerns like authentication, rate limiting, and request shaping rather than just forwarding bytes. For a client developer building your React app, the win is enormous: they integrate against one base URL with one auth scheme, and the fact that ten services answer behind it is invisible. The gateway is where your distributed backend presents a single coherent face.

Cross Cutting Concerns

The strongest reason to run a gateway is that it gives you one place for the work every service would otherwise duplicate. Authentication is the prime example: the gateway validates the token or cookie once at the edge, rejects anonymous traffic before it reaches anything internal, and passes a trusted identity inward so services do not each re-implement auth. Rate limiting belongs here too, throttling abusive clients before they consume backend capacity. The same goes for request logging, CORS handling, and response compression. Centralizing these keeps your services focused on business logic and ensures a policy change happens in one spot rather than across a dozen codebases. The discipline to maintain is keeping genuine business rules out of the gateway; it should enforce broad cross-cutting policy, not encode that a particular order can only ship under specific conditions, which belongs in the service that owns that domain.

Why Discovery Exists

In a dynamic system, service instances are ephemeral. An orchestrator starts new ones to handle load, kills old ones, and reschedules them onto different hosts after failures, so the address where a service lives this minute may be gone the next. Hardcoding addresses or even maintaining a static config file falls apart immediately under this churn. Service discovery solves it by maintaining a live registry of which instances of each service are currently running and where, so that when one service needs another it asks the registry rather than assuming a fixed location. This is the phone book for your backend: services and the gateway look up healthy instances by logical name and get back current addresses. Without it, every scale event or restart would require manually rewiring who talks to whom, which is exactly the kind of brittle coupling that microservices were supposed to eliminate in the first place.

Client Side Versus Server Side

Discovery comes in two architectural flavors. In client-side discovery, the calling service queries the registry itself, gets back the list of healthy instances, and picks one using its own balancing logic, which is efficient because there is no extra hop but couples every service to the registry and its client library. In server-side discovery, the caller simply sends to a stable address, usually a load balancer or the gateway, and that intermediary consults the registry and forwards to a live instance, keeping discovery logic out of your application code at the cost of an extra network hop. Container orchestrators typically favor the server-side style, exposing a stable internal name that resolves to whatever healthy instances exist, so your code just calls the name and the platform handles the rest. Knowing which model you are in tells you where balancing happens and where to look when calls fail.

Registration Patterns

For a registry to be useful, instances have to get into it and, crucially, get removed when they die. Self-registration has each instance announce itself to the registry on startup and send periodic heartbeats to prove it is alive, with the registry evicting any instance whose heartbeats stop. It is straightforward but couples application code to the registry. The alternative, third-party registration, hands that job to a separate registrar or the orchestration platform that watches instances and updates the registry on their behalf, keeping the application ignorant of the registry entirely. Modern container platforms lean on the third-party approach, registering and deregistering containers automatically as they start and stop. The detail that bites people is deregistration on crash: an instance that dies hard cannot announce its departure, so the registry must rely on heartbeat timeouts, which means there is always a window where it still lists a dead instance.

Health Driven Routing

A registry that lists running instances is not enough; it must list healthy ones, because an instance can be alive yet unable to serve, having lost its database connection or hung on a deadlock. Discovery systems pair registration with health checks so that only instances passing their checks are returned as routable, and a failing one is held back until it recovers. This is the same principle as load balancer health checks, applied at the discovery layer, and the depth of the check matters just as much: a check that only confirms the process responds will keep routing to an instance that cannot reach its dependencies. Combine this with graceful shutdown, where an instance marks itself unhealthy and finishes in-flight requests before exiting, so deploys and scale-downs drain cleanly. Health-driven routing is what lets the system continuously steer traffic around partial failures without a human noticing or intervening.

Resilience At The Edge

Because the gateway makes downstream calls on every request, it is where you concentrate resilience patterns that stop one failing service from sinking the whole system. Timeouts prevent a slow backend from tying up gateway resources indefinitely while clients pile up waiting. Retries with backoff paper over transient blips, but only for idempotent operations, since blindly retrying a payment can charge a customer twice. The circuit breaker is the key pattern: after a backend fails repeatedly, the gateway stops calling it for a cooldown and fails fast, giving the struggling service room to recover instead of being hammered while it is down, then cautiously probes before restoring full traffic. Without these, a single slow dependency triggers a cascade where waiting requests exhaust connections everywhere upstream. The gateway is the natural chokepoint to enforce them consistently, turning isolated failures into contained, recoverable events rather than system-wide outages.

Avoiding Gateway Pitfalls

The gateway's greatest strength, being the single entry point, is also its greatest risk, and a few mistakes recur. Because all traffic flows through it, the gateway is a single point of failure and must run redundantly, or its outage takes down every service at once regardless of their health. It is also a tempting dumping ground: teams keep adding routing rules, transformations, and one-off business logic until the gateway becomes a fragile monolith of its own that everyone fears to change, quietly recreating the coupling microservices were meant to avoid. Watch its latency closely, since it adds a hop to every request and a slow gateway slows everything uniformly. Keep its responsibilities to genuine cross-cutting edge concerns, version routing changes carefully, and resist making it smart about individual domains. A lean, redundant, well-monitored gateway is an asset; a bloated one becomes the bottleneck you were trying to escape.