Traffic Handling

Now that we understand microservices and queues, let’s look at a few more important pieces: protecting your system from abuse, handling database load, and making things faster with caching.

Rate Limiting: Protecting Your System from Abuse

When lots of users interact with your system through the Load Balancer, some of them might send fake/abusive requests, or even try a DDoS attack (flooding your server with tons of fake requests to bring it down). To protect against this, we use Rate Limiting.

Rate Limiting simply means: limiting how many requests a user (or the system) is allowed to make in a given period of time.

For example: “I will only accept 5 requests per second from a single user. If you send more, I’ll block you temporarily.” (This is why you sometimes see errors like “Too Many Requests.”)

Rate limiting isn’t just about blocking bad users - it’s also used to protect your own servers from getting overwhelmed by controlling the flow of incoming requests in a smoother way.

There are a few common rate-limiting strategies:

• Leaky Bucket algorithm
• Token Bucket algorithm

These are technical algorithms with their own logic for how they smooth out and control request flow, but the basic goal is the same: don’t let too many requests come in at once.

Database Load: Read Replicas

When your microservices grow, they all end up reading and writing to the same database. This single database can quickly become overwhelmed, especially as more and more services depend on it.

One simple way to reduce this load is to create Read Replicas - copies of your database that are only used for reading data (not writing).

flowchart LR W[Write Requests] --> P[Primary/Master Database] P -->|Copies data over| R1[Read Replica 1] P -->|Copies data over| R2[Read Replica 2] R1 --> A[Analytics Service] R2 --> L[Logs / Reports]

• All writes (inserting/updating data) must always go to the Primary (Master) Database.
• Any real-time critical reads should also go to the Primary Database.
• But things like analytics, reports, or logs (where a small delay is okay) can read from the Read Replicas instead.

This reduces the load on the Primary Database, since it doesn’t have to handle every single read request itself.

Small downside of Read Replicas

Data in a read replica can sometimes be slightly delayed (not 100% real-time), since it takes a moment to copy data over from the primary database. This is totally fine for things like analytics, but not okay for something like checking your current account balance.

Caching: Avoid Doing the Same Work Again

If your system does any complex/expensive computation, or repeatedly fetches the same data, there’s a good chance you can cache it (store the result temporarily so you don’t have to compute/fetch it again).

For caching, people commonly use in-memory databases (extremely fast databases that store data in RAM instead of on disk). A popular tool for this is Redis.

The typical flow is:

1. Before going to the main database, check if the data already exists in the cache.
2. If it exists (cache hit) → return it directly from the cache (very fast).
3. If it doesn’t exist (cache miss) → fetch it from the main database, store it in the cache for next time, then return it.

flowchart LR A[Request comes in] --> B{Is data in Cache?} B -->|Yes - Cache Hit| C[Return from Cache - Fast] B -->|No - Cache Miss| D[Fetch from Database] D --> E[Store result in Cache] E --> C

This drastically reduces the number of calls made to your database, making your system faster and lighter on the database.

CDN: Making Content Fast for Users Everywhere in the World

Your users are spread all over the world - some in the US, some in India, some in Canada, and so on. If all of them have to reach one single server/load balancer (say, located in the US), users far away will experience slow load times (high latency).

This is solved using a CDN (Content Delivery Network).

A CDN places small “edge” servers all around the world, close to where your actual users are. Instead of routing every user directly to your main Load Balancer, users are routed to the nearest edge server.

Popular tool name: On AWS, this is called CloudFront. On other providers, it might be called something else, but the generic term is just CDN.

flowchart LR U1[User in India] --> E1[Nearest Edge Server - India] U2[User in USA] --> E2[Nearest Edge Server - USA] U3[User in Canada] --> E3[Nearest Edge Server - Canada] E1 --> LB[Main Load Balancer] E2 --> LB E3 --> LB

How Does a User Always Reach the “Nearest” Edge Server?

This works using something called Anycast - a technique where a single IP address is assigned to many different physical machines around the world, and the network automatically routes each user to the geographically closest one.

How the Edge Server Helps

Let’s say a user in India wants to view a product photo:

1. First, the request hits the nearest edge server (in India).
2. The edge server checks: “Do I already have this photo cached?”
3. First time: Not cached yet → it goes all the way to the main server, fetches the photo, and also caches it on the edge server for next time.
4. Every time after that: Any other user in India requesting the same photo gets it instantly from the nearby cached copy - without the request ever reaching your main server.

flowchart LR U[User in India requests a photo] --> E[Edge Server - India] E --> C{Is photo cached here?} C -->|Yes| R[Return instantly from cache] C -->|No, first time| M[Fetch from Main Server] M --> E E --> R

Why This Matters

Using a CDN gives you two big benefits:

1. Faster experience for users - they get content from a nearby server instead of one far away.
2. Less load on your main server - because most repeat requests are served directly from the cache at the edge, never even reaching your main servers. This also saves network bandwidth.

This is why, when you browse a product on an e-commerce site, the images/videos load quickly - especially if someone near you has already viewed the same product recently.

Putting it all together

A well-designed, scalable system usually has: DNS → CDN (edge caching) → API Gateway → Load Balancer(s) → Microservices → Database (with Read Replicas) → Caching Layer (like Redis) → Message Queues/Pub-Sub for background work. Each piece solves one specific problem, and together they make the system fast, reliable, and cost-effective.

In the final note of this series, we’ll look at how modern apps are actually deployed - covering serverless computing (like AWS Lambda), containers (like Docker), and container orchestration (like Kubernetes).