Service Layer Architecture

Big applications (like Amazon) are not built as one giant single program. Instead, they are broken into small, independent parts called microservices. But before we understand microservices, we first need to understand what came before it: monolithic architecture.

What is Monolithic Architecture?

“Monolithic” simply means one single, big block. In a monolithic architecture, your entire application - login/signup, orders, payments, everything - lives inside one single codebase, and runs as one single program/server.

flowchart TB subgraph Monolith["One Single Application (Monolith)"] A[Login/Signup Code] B[Orders Code] C[Payments Code] D[Everything else] end U[User Request] --> Monolith

When you’re just starting out (a small project, a startup, a college project), a monolith is usually the simplest and fastest way to build something. Everything is in one place, so it’s easy to develop, test, and deploy - you just build one app and run it.

Problems With Monolithic Architecture (as you grow bigger)

As your application and user base grow, a monolith starts to cause real problems:

1. Scaling is all-or-nothing. Let’s say only the “Payments” part of your app is under heavy load, but “Login” isn’t. In a monolith, you can’t scale just the Payments part - since everything is bundled together as one program, you have to scale the entire application, wasting resources on parts that don’t even need it.
2. One small bug can crash everything. Since everything runs in the same process, if one part of the code has a serious bug or memory leak (say, in the “Orders” section), it can crash or slow down the whole application - including totally unrelated features like Login.
3. Slow to deploy/update. Every time you want to fix or update even a tiny part of the code (like just the Payments logic), you have to rebuild and redeploy the entire application, which is slower and riskier as the codebase grows bigger.
4. Hard for big teams to work together. If many different teams (Auth team, Orders team, Payments team) are all working inside the same single codebase, they can easily step on each other’s toes, and the codebase becomes harder to understand and maintain over time.

This is exactly why, once an application grows large enough (like Amazon), companies move away from a monolith and split things into microservices instead - so each part can be built, scaled, deployed, and maintained independently.

Monolith vs Microservices - simple way to remember

• Monolith = one single big app doing everything together. Simple to start with, but hard to scale/maintain as it grows.
• Microservices = many small, independent apps/services, each doing one job. Harder to set up initially, but much easier to scale and maintain at a large scale.

What is Microservice Architecture?

Instead of one huge application handling everything, you break your app into small separate services, where each service does one specific job. For example, in an e-commerce app like Amazon, you might have:

• Auth Service - handles login/signup (checking who the user is)
• API Service - handles general API requests
• Orders Service - handles placing and managing orders
• Payments Service - handles payments

flowchart TB A[Auth Service] B[Orders Service] C[Payments Service] D[API Service]

Each of these services can have its own set of servers, and each can scale independently based on its own load. For example:

• The Auth service might need 4 servers (since everyone logs in).
• The Orders service might need 3 servers.
• The Payments service might only need 2 servers (since fewer people actually complete a purchase compared to how many just browse).

This way, you’re not wasting resources scaling a part of the system that doesn’t need it.

The Problem: How Do Requests Reach the Right Service?

If a user visits amazon.com/auth, that request should go to the Auth Service. If they visit amazon.com/orders, it should go to the Orders Service. How do we route (direct) requests correctly?

This is solved using something called an API Gateway.

What is an API Gateway?

An API Gateway is like a central receptionist that sits in front of all your microservices. Every request from the internet first arrives at the API Gateway, and it decides which service should handle it, based on rules (usually based on the URL path).

For example:

• /auth → goes to Auth Service’s Load Balancer
• /orders → goes to Orders Service’s Load Balancer
• /payments → goes to Payments Service’s Load Balancer

flowchart LR U[User Request] --> GW[API Gateway] GW -->|/auth| LB1[Auth Load Balancer] --> S1[Auth Servers] GW -->|/orders| LB2[Orders Load Balancer] --> S2[Order Servers] GW -->|/payments| LB3[Payments Load Balancer] --> S3[Payment Servers]

So the flow is: API Gateway routes to the correct Load Balancer → Load Balancer picks a specific server (replica) to handle the request.

Popular tool name: On AWS, this is called API Gateway (same name). It can also directly trigger serverless functions (called Lambda on AWS - we’ll cover this in the serverless note).

The API Gateway can also include an authenticator - something that checks if the user is logged in/valid before routing the request further. (A popular tool for this is Auth0, but there are many others.)

Background Jobs: When You Can’t Do Everything Instantly

Imagine a vendor uploads a big bulk list of products, or you need to send a marketing email to 1 million users. You obviously can’t run a loop that emails 1 million people one by one inside your main server - it would take forever and block everything else.

For situations like this, we use background workers - small services that run in the background and process tasks like sending emails or handling bulk uploads.

The Simple (But Flawed) Way: Synchronous Calls

One way to do this: the Payment Service directly calls the Email Worker and waits for it to finish sending the email before continuing.

flowchart LR A[Payment Service] -->|Direct Call, Waits for Response| B[Email Worker] B --> C[Sends Email via Gmail API] C --> B B --> A

Problem: This is called a synchronous approach - the Payment Service has to wait until the email is sent. If this happens for every single payment, and payments happen every second, your system becomes slow and not scalable. Waiting is wasteful.

The Better Way: Message Queues (Asynchronous)

Instead of directly calling and waiting, we can use a Message Queue. A queue works like a line at a shop: First In, First Out (FIFO).

• The Payment Service just pushes a message into the queue (like “an order was placed”) and immediately moves on - it doesn’t wait.
• The Email Worker picks up (pulls) messages from the queue whenever it’s free, processes them (sends the email), and then picks the next one.

flowchart LR A[Payment Service] -->|Pushes message, doesn't wait| Q[Message Queue] Q -->|Worker pulls messages one by one| W[Email Worker] W --> G[Sends Email via Gmail API]

Popular tool name: On AWS, this is called SQS (Simple Queue Service). Other popular tools include RabbitMQ and Kafka. The generic term is just Message Queue.

If traffic increases, you can simply add more workers to process messages faster (this increases parallelism - meaning more things get processed at the same time).

Rate Limiting on Purpose

Sometimes, the queue system is also useful for protecting yourself from external limits. For example, if Gmail only allows you to send 10 emails per second, you can configure your Email Worker to never pull more than 10 messages per second from the queue - this way, you avoid hitting Gmail’s rate limit and getting blocked, even if you have millions of orders coming in.

Pull vs Push

There are two ways a worker can get messages from a queue:

• Pull (Polling): the worker keeps asking the queue “do you have anything for me?” every few seconds.
  • Short polling: ask every second (more API calls, more cost).
  • Long polling: wait up to 10 seconds, then process everything that arrived in that window (fewer calls, cheaper).
• Push: the queue itself triggers/calls the worker whenever a new message arrives.

When One Event Needs to Trigger Multiple Actions: Pub/Sub

Sometimes one single event (like a payment) needs to trigger multiple different actions at once:

• Send a WhatsApp message
• Send an SMS
• Email the vendor
• Email the user

A basic Message Queue only sends a message to one consumer (worker) at a time - it’s a one-to-one relationship. So for this “one-to-many” case, we use a Pub/Sub (Publish-Subscribe) system, also known as an event-driven architecture or notification system.

The Payment Service publishes a notification (like “a payment happened”), and multiple different services can subscribe to and receive that same notification.

flowchart LR A[Payment Service] -->|Publishes Event| N[Notification System / Pub-Sub] N --> B[Email Worker] N --> C[WhatsApp Service] N --> D[SMS Service]

Popular tool name: On AWS, this is called SNS (Simple Notification Service). The generic term is Pub/Sub or notification system.

A downside of pure Pub/Sub

With a simple Pub/Sub system, there’s usually no acknowledgment/guarantee. If a service that was supposed to receive the message was down at that moment, or failed to process it, there’s no automatic retry - the message can simply be lost. You’d have to build your own retry logic.

Combining Both: Fan-Out Architecture

To get the best of both worlds (notify multiple services and have retry/reliability), we use a Fan-Out Architecture. This combines Pub/Sub with Message Queues:

1. The event is published to a notification system (Pub/Sub).
2. Instead of directly calling services, the notification system pushes the event into multiple separate queues (one queue per service - e.g., WhatsApp Queue, Email Queue, SMS Queue).
3. Each queue has its own worker(s) pulling and processing messages, complete with retry logic and a Dead Letter Queue (a special queue for failed messages, so they can be retried later instead of being lost).

flowchart LR A[Event: Payment Happened] --> N[Pub/Sub System] N --> Q1[WhatsApp Queue] --> W1[WhatsApp Worker] N --> Q2[Email Queue] --> W2[Email Worker] N --> Q3[SMS Queue] --> W3[SMS Worker]

This is exactly how a video platform like YouTube processes an uploaded video: one “video uploaded” event fans out into multiple tasks - like converting to different resolutions (480p, 360p), generating a thumbnail, extracting audio, etc. - each handled independently and reliably.

Why Dead Letter Queues matter

A Dead Letter Queue (DLQ) stores messages that failed to process (for example, if Gmail was down when trying to send an email). Instead of losing that message forever, it gets stored in the DLQ so it can be retried later (e.g., after 5-10 minutes).

In the next note, we’ll cover how to protect your system from abuse using rate limiting, how databases handle heavy read/write load, and how caching and CDNs make everything faster.