API Gateway vs Load Balancer vs Reverse Proxy
Back with another one in the System Design series, where I break down the concepts that tripped me up.
This one came out of a mock interview. I was designing a photo sharing backend and said "and then the load balancer routes it to the right service." The interviewer asked, "Do you mean a load balancer, a reverse proxy, or an API gateway?" I froze. In my head they were the same box with a different label. They are not. They sit in the same spot in a diagram and they all move requests around, but they solve different problems. People search for this as reverse proxy vs load balancer, load balancer vs reverse proxy, or api gateway vs load balancer, and the wrong word tells an interviewer you have only seen the diagram, not the reasoning.
"Routes traffic" is doing too much work
These three blur together because we describe all of them the same way. "It takes requests and sends them somewhere." That is also true of a CDN node, a service mesh sidecar, and your home router. The phrase is useless for telling them apart.
What separates them is the question each one answers. A reverse proxy answers "how do I put one controlled front door in front of my servers?" A load balancer answers "I have many identical servers, how do I spread work across them?" An API gateway answers "I have many different services, how do I give clients one entry point and handle auth and rate limiting in one place?" Anchor on the question, not the verb, and the overlap clears up.
Reverse proxy: one controlled front door
A reverse proxy sits in front of servers. Clients think they are talking to your application, but they are talking to the proxy, which forwards the request to whatever server should handle it.
That one front door buys you a lot. You terminate TLS in one place instead of on every server. You hide your internal topology so the outside world never learns how many servers you run. You serve cached static assets without waking the application. You compress responses and add security headers in one layer your app code never thinks about. Nginx, HAProxy, and Caddy are the common names. Point one at a single backend and it is already earning its place, and it is not balancing anything yet.
server {
listen 443 ssl;
server_name api.example.com;
ssl_certificate /etc/ssl/example.crt;
ssl_certificate_key /etc/ssl/example.key;
location / {
proxy_pass http://127.0.0.1:8080; # forward to one app server
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
}Load balancer: spread work across identical servers
Now run five copies of the same service. Every copy can handle any request because they are identical and stateless. The new problem is distribution. If every request lands on server one, the other four sit idle while server one melts.
A load balancer answers that. Given a pool of interchangeable backends, it picks one per request so load spreads evenly. The strategies have names worth knowing: round robin rotates, least connections picks the least busy server, and consistent hashing keeps a client on the same server for session affinity. I wrote a separate post on why consistent hashing shows up everywhere.
A load balancer also health-checks its backends. When a server stops responding, it pulls that server out of rotation so no traffic gets sent into a dead box. People forget this half. Distribution plus failure detection is the complete picture.
upstream app_pool {
least_conn; # send to the least busy server
server 10.0.0.11:8080 max_fails=3 fail_timeout=30s;
server 10.0.0.12:8080 max_fails=3 fail_timeout=30s;
server 10.0.0.13:8080 max_fails=3 fail_timeout=30s;
}
server {
listen 443 ssl;
location / {
proxy_pass http://app_pool; # forward to the pool, not one box
}
}Notice it is still a reverse proxy. It receives, forwards, and returns. It just points at a pool instead of one box. Every load balancer is a reverse proxy. Not every reverse proxy is a load balancer.
Load balancers come in two flavors. A Layer 4 load balancer works at the transport level, forwarding packets by IP and port without reading the request. Fast and content-blind. A Layer 7 load balancer reads the HTTP request, so it routes on path, headers, or cookies. Smarter and slower.
API gateway: one entry point for many different services
Here is where most candidates, including past me, get lost. A load balancer spreads work across copies of the same service. An API gateway sits in front of many different services and gives clients one coherent entry point to all of them.
Picture a users service, an orders service, and a payments service, each its own deployment with its own address. Without a gateway, every client has to know all three addresses, authenticate against each, and stitch responses itself. The gateway collapses that. Clients hit one host. The gateway routes by path, sending /users to the users service and /orders to the orders service, and along the way it authenticates the caller, enforces rate limits, and emits logs and metrics for the whole edge in one place. Kong, AWS API Gateway, and Apigee live here.
routes:
- path: /users
service: users-service # a different service
plugins: [jwt-auth, rate-limit]
- path: /orders
service: orders-service # a different service
plugins: [jwt-auth, rate-limit]
- path: /payments
service: payments-service # a different service
plugins: [jwt-auth, rate-limit, request-validation]The honest comparison
Remember the nesting. A reverse proxy is the base concept. A load balancer is a reverse proxy specialized for distributing load across identical backends. An API gateway is a reverse proxy specialized for routing to different services and enforcing policy. The overlap is real because one piece of software can play all three roles. Nginx can be a plain reverse proxy, a load balancer, and with enough config an API gateway.
Question it answers Tool Defining job
-------------------------- ------------- ------------------------------
One front door for servers Reverse proxy Receive, transform, forward
Spread load across copies Load balancer Distribute + health-check pool
One door to many services API gateway Route by service + auth/policy
And they stack. A request might hit a Layer 4 load balancer at the edge, reach an API gateway that authenticates and routes by path, then hit a Layer 7 load balancer spreading it across ten identical pods. Three of these concepts in one hop chain, every one "routing traffic." Now you can say which is which and why each is there.
The mistake I was making
My mistake was using "load balancer" as the generic word for the box in front of the servers. When the architecture had different services, the right word was almost always API gateway, and saying load balancer made it sound like I thought all my services were the same thing.
The fix is one question before naming the component: are the things behind it identical or different? Identical copies of one service means a load balancer. Distinct services means an API gateway, often with load balancers below it. Just need TLS and caching for one backend? A plain reverse proxy, and do not over-engineer it into a gateway you do not need. This is the same logic about not adding heavy machinery before you feel the pain that I leaned on in modular monolith vs microservices.
What to learn next
Read about Layer 4 versus Layer 7 in depth. Look at how rate limiting works at the gateway, since token bucket and leaky bucket are interview favorites. And study how a service mesh pushes routing down to a sidecar next to each service. More breakdowns live on the Levelop blog, and the full system design track is at Levelop. This one came from a problem on Levelop's system design track that I had been quietly getting wrong for months.
Frequently Asked Questions
What is the difference between an API gateway and a load balancer?
A load balancer distributes requests across identical copies of one service and pulls unhealthy ones out of rotation. An API gateway sits in front of many different services, routing each request to the right one while handling auth and rate limiting. A load balancer spreads work across copies of one thing; an API gateway routes between different things.
Is a load balancer the same as a reverse proxy?
Not quite. A reverse proxy receives client requests and forwards them to backends. A load balancer is a reverse proxy with one specialization: it distributes across a pool of identical backends and health-checks them. Every load balancer is a reverse proxy, but a reverse proxy pointed at a single backend is not a load balancer.
Do I need all three in one system?
Often yes, because they stack. A request can pass a network load balancer at the edge, reach an API gateway that routes by path, then hit a per-service load balancer across that service's pods. Each layer answers a different question, so large systems usually run all three.
When should I use a reverse proxy vs a load balancer?
Use a plain reverse proxy for a single backend that just needs TLS, caching, or topology hiding. Switch to a load balancer once you run more than one copy of a service and need to spread traffic and survive a failure. The trigger is whether there is more than one identical backend to distribute across.
What is the difference between a Layer 4 and a Layer 7 load balancer?
A Layer 4 load balancer forwards packets by IP and port without reading the request, which is fast but content-blind. A Layer 7 load balancer reads the HTTP request and routes on path, headers, or cookies, at the cost of more work per request. Choose Layer 4 for raw speed and Layer 7 when routing depends on request content.
References
- NGINX, "Using nginx as HTTP load balancer" and reverse proxy documentation, nginx.org.
- Cloudflare Learning Center, "What is a reverse proxy?" and "What is load balancing?"
- Amazon Web Services, "Amazon API Gateway" developer documentation, docs.aws.amazon.com.
- Microsoft Azure Architecture Center, "Gateway Routing pattern" and "Using API gateways in microservices."
- Kong, "What is an API Gateway?" konghq.com learning resources.