Testing APIs

Testing is essential if you want to make sure your API actually works, handles edge cases properly, and doesn’t break things that used to work (this is called a “regression”). Django comes with its own built-in testing tool (built on top of Python’s unittest), but a lot of developers prefer pytest instead, because it’s cleaner to write and comes with some really handy features. This guide focuses on testing behavior - in other words, what your API actually does - not the tiny details of how you built it.

In this section, we’ll cover two big topics: Automated Testing and Performance Testing. We’ll learn how to write tests that check the API’s behavior from the user’s point of view, and how to measure and improve your API’s performance.

Testing Behavior vs. Implementation

This is one of the most important ideas in this whole guide, so let’s slow down and really understand it with a simple example first.

The Microwave Analogy

Imagine you want to test if a microwave works properly.

Correct Way (Test Behavior):

• Press the start button
• Check that the screen shows a running timer for the time you asked for
• Make sure the heating stops once the timer reaches zero
• You’re testing what the user actually sees and expects

Wrong Way (Test Implementation):

• Open up the microwave’s casing
• Check the electrical signal on every single transistor
• Check the wiring on the internal circuit board
• You’re testing tiny internal details that the user never sees and doesn’t care about

The Same Idea Applies to Software

A lot of developers struggle with automated testing because they end up testing implementation details instead of behavior. Here’s why that’s a problem:

• Implementation changes a lot: You might swap a function-based view for a class-based view, split one model into two, or merge two models together
• Tests become fragile: If your tests check internal details, they’ll break every time you refactor your code, even if nothing is actually broken
• It becomes a maintenance headache: You end up spending your time fixing tests instead of fixing real bugs
• It gives you false confidence: Tests that check internal details don’t actually prove the API works the way users expect

So what’s the fix? Test the API’s behavior - what users actually see and experience - instead of testing exactly how you built it on the inside.

What to Test?

Focus on Behavior, Not Implementation

When you’re testing your API, ask yourself this: “How should this API behave, from the user’s point of view?”

Your tests should check things like:

• What HTTP status code comes back
• What data is inside the response
• How the API reacts to bad or invalid input
• Whether permission rules are actually being enforced
• Whether the feature works the way a user would expect

Your tests should NOT check things like:

• Whether you used a function-based view or a class-based view
• How data moves through your internal helper functions
• Whether you’re using Django’s ORM, raw SQL, or some outside service
• The internal structure of your database or how it’s optimized

Example: Testing an API Endpoint

Let’s say your API has a POST /collections/ endpoint that’s used to create new collections. Here’s how you’d test its behavior (not its implementation):

Scenario 1: Unauthenticated Request

• A client sends a POST request without logging in (no auth token)
• Expected behavior: The API returns 401 Unauthorized
• Why: The API should reject any request that isn’t logged in

Scenario 2: Authenticated but Unauthorized Request

• A client sends a POST request with a valid login token, but the user isn’t an admin
• Expected behavior: The API returns 403 Forbidden
• Why: Only admins should be allowed to create collections

Scenario 3: Missing Required Data

• An admin sends a POST request, but forgets to include the collection’s name
• Expected behavior: The API returns 400 Bad Request, along with an error message in the response
• Why: The API should check that required fields are actually filled in

Scenario 4: Valid Request

• An admin sends a POST request that includes the collection name
• Expected behavior: The API returns 201 Created, along with the new collection’s ID in the response
• Why: The API should be able to successfully create the resource

Notice what we did NOT test:

• Whether the view is function-based or class-based
• How the serializer checks the data behind the scenes
• The exact SQL queries that get run
• Where exactly the response gets built inside the code

We only tested how the API behaves from the outside - which is exactly what we should be testing.

Reference Code for Testing Example

# models.py
from django.db import models
class Collection(models.Model):
    name = models.CharField(max_length=255)
    created_at = models.DateTimeField(auto_now_add=True)

class Product(models.Model):
name = models.CharField(max_length=255)
collection = models.ForeignKey(Collection, related_name='products', on_delete=models.CASCADE)
created_at = models.DateTimeField(auto_now_add=True)

# serializers.py
from rest_framework import serializers
from .models import Collection, Product

class CollectionSerializer(serializers.ModelSerializer):
    class Meta:
        model = Collection
        fields = ['id', 'name']

class ProductSerializer(serializers.ModelSerializer):
    class Meta:
        model = Product
        fields = ['id', 'name', 'collection_id']

# permissions.py
from rest_framework import permissions
class IsAdminOrReadOnly(permissions.BasePermission):
    def has_permission(self, request, view):
        if request.method in permissions.SAFE_METHODS:
            return True
        return request.user and request.user.is_staff

# views.py
from rest_framework import status, viewsets
from .serializers import CollectionSerializer
from .models import Collection, Product
from .permissions import IsAdminOrReadOnly

class CollectionViewSet(viewsets.ModelViewSet):
    queryset = Collection.objects.all()
    serializer_class = CollectionSerializer
    permission_classes = [IsAdminOrReadOnly]

class ProductViewSet(viewsets.ModelViewSet):
    queryset = Product.objects.all()
    serializer_class = ProductSerializer
    permission_classes = [IsAdminOrReadOnly]

# urls.py
from django.urls import path, include
from rest_framework.routers import DefaultRouter
from .views import CollectionViewSet, ProductViewSet

router = DefaultRouter()
router.register(r'collections', CollectionViewSet, basename='collection')
router.register(r'products', ProductViewSet, basename='product')
urlpatterns = [
    path('', include(router.urls)),
]
# /collections/
# /collections/{id}/
# /products/
# /products/{id}/

Pytest

Django’s built-in testing tool is solid, but many developers prefer pytest instead, because it’s cleaner to write, gives you clearer error messages, and comes with handy features like fixtures and parameterization (we’ll cover both of these soon).

Setting Up Pytest

1. Install pytest:

   uv add --dev pytest pytest-django

2. Create a pytest.ini file in your project’s root folder, with the following content:

   [pytest]
   DJANGO_SETTINGS_MODULE = your_project_name.settings

3. Follow pytest’s naming conventions (the standard way pytest expects things to be named):

   • Directory structure: Put your tests in a tests/ folder inside each app, or in one top-level tests/ folder.
   • Test file naming: Name your test files starting with test_ (for example, test_views.py, test_models.py).
   • Test function naming: Name your test functions starting with test_ (for example, test_create_collection(), test_unauthorized_access()).
   • Test classes: You can group related tests inside classes that start with Test (for example, class TestCollectionAPI:), though this part is optional.

4. The AAA Pattern: Follow the Arrange-Act-Assert pattern in your tests, so they stay clear and easy to read:

   • Arrange: Set up your test data and the environment you need
   • Act: Actually perform the thing you’re testing (like making an API request)
   • Assert: Check that you got the result you expected (like checking the response’s status code and data)

5. Use these handy pytest features:

   • Fixtures: Use @pytest.fixture to build reusable test data and setup code (more on this below)
   • Parameterization: Use @pytest.mark.parametrize to run the exact same test multiple times, with different input values
   • Markers: Use your own custom markers (for example, @pytest.mark.slow) to group tests and control which ones actually run

6. Run your tests using pytest:

   uv run pytest
   # or for specific test file
   uv run pytest path/to/test_views.py
   # or for specific test class
   uv run pytest path/to/test_views.py::TestCollectionAPI
   # or for specific test function
   uv run pytest path/to/test_views.py::test_create_collection
   # or for specific test under a class
   uv run pytest path/to/test_views.py::TestCollectionAPI::test_create_collection
   # or a test having a text in its function name
   uv run pytest -k "anonymous"

7. Continuous Testing: Use pytest-watch to automatically rerun your tests whenever a file changes. Run it in its own separate terminal:

   uv add --dev pytest-watch
   uv run ptw

Example

from rest_framework.test import APIClient
from rest_framework import status
import pytest

@pytest.mark.django_db
class TestCollectionAPI:
    def test_if_user_is_anonymous(self):
        # Arrange
        client = APIClient()

        # Act
        response = client.post('/collections/', {'name': 'My Collection'})

        # Assert
        assert response.status_code == status.HTTP_401_UNAUTHORIZED

Skipping Tests

If you have tests that don’t make sense to run in certain environments (for example, tests that need an outside service to be available), you can skip them using the @pytest.mark.skip decorator:

@pytest.mark.skip(reason="Requires external service")
def test_external_service_integration():
    # Test code that interacts with an external service
    pass

VS Code Pytest Integration

VS Code has great built-in support for pytest. You can run and debug your tests right from inside the editor. Here’s how to set it up:

1. Install the Python extension for VS Code.
2. Open the command palette (Ctrl+Shift+P) and choose “Python: Configure Tests”.
3. Pick pytest as your testing framework.
4. Follow the prompts to set your test folder and pattern (Recommended choice: . Root Directory). Once it’s set up, you can run any test by clicking the “Run Test” or “Debug Test” link that shows up right above each test function in the editor.

Why use pytest in VS Code:

• Run a single test or a whole test class with just one click
• Debug tests with breakpoints, stepping through the code line by line
• See test results and error messages right in the built-in terminal

Writing Pytest Tests

Now let’s actually use pytest to test our API endpoints.

DB Access

Whenever you write a test that talks to the database, use the @pytest.mark.django_db decorator. This makes sure your test is allowed to access the database. It lets you create test data and check how the database behaves, without touching your real production data at all.

Best Practices

The first time you write a test, try this trick: comment out the actual code that would make the test pass, and run the test - it should fail. Then uncomment that code and run the test again - it should now pass. This way, you can be sure your test is genuinely checking your code, instead of just passing by accident.

Simple Testing

Here’s an example of how to write a test for the POST /collections/ endpoint using pytest:

from rest_framework.test import APIClient
from rest_framework import status
import pytest

@pytest.mark.django_db
class TestCollectionAPI:
    def test_if_user_is_anonymous(self):
        # Arrange
        client = APIClient()

        # Act
        response = client.post('/collections/', {'name': 'My Collection'})

        # Assert
        assert response.status_code == status.HTTP_401_UNAUTHORIZED

Authenticate User

To test an endpoint that needs the user to be logged in, create a real user, and then log that user into your test client:

The django_user_model fixture comes from pytest-django. It gives you the actual user model your project uses, so you can create real user records inside your tests. This is helpful because your test ends up working with the database and login system in the exact same way a real request would.

from rest_framework import status
import pytest
from rest_framework.test import APIClient

@pytest.mark.django_db
class TestCollectionAPI:
    def test_if_user_is_authenticated_but_not_admin(self, django_user_model):
        # Arrange
        client = APIClient()
        user = django_user_model.objects.create_user(
            username='regular-user',
            password='password123',
        )
        client.force_authenticate(user=user)

        # Act
        response = client.post('/collections/', {'name': 'My Collection'})

        # Assert
        assert response.status_code == status.HTTP_403_FORBIDDEN

Here, we create a real user, and then log the client in as that user. Since the user isn’t a staff member, the API should send back 403 Forbidden.

Multiple assertions

You can also check multiple things in a single test, to verify different parts of the response all at once:

from rest_framework import status
import pytest
from rest_framework.test import APIClient

@pytest.mark.django_db
class TestCollectionAPI:
    def test_create_collection(self, django_user_model):
        # Arrange
        client = APIClient()
        admin_user = django_user_model.objects.create_user(
            username='admin-user',
            password='password123',
            is_staff=True,
        )
        client.force_authenticate(user=admin_user)

        # Act
        response = client.post('/collections/', {'name': 'My Collection'})

        # Assert
        assert response.status_code == status.HTTP_201_CREATED
        assert 'id' in response.data
        assert response.data['id'] > 0
        assert response.data['name'] == 'My Collection'

Here, we create a real staff user and log the client in as them. This makes the example match how a real admin would actually use the API, and clearly shows why the request should succeed.

Fixtures

Pytest fixtures are a clean, simple way to share setup code between different tests. Use them whenever the same test data, client setup, or login step shows up in more than one test. They help you avoid copy-pasting the same code over and over, and make your tests easier to read.

You use a fixture by adding it as a parameter to your test function. Pytest runs the fixture first, and automatically hands the result over to your test.

Good times to use a fixture:

• When several tests all need the same API client
• When many tests all need a logged-in user
• When the setup is long, or it keeps repeating across many tests
• When you want to keep your test functions short and focused

You don’t need a fixture for every tiny test. If something is only used once, plain setup code right inside the test is often simpler to follow.

Creating a Fixture

# conftest.py
from rest_framework.test import APIClient
import pytest

@pytest.fixture
def api_client():
    return APIClient()

Using a Fixture in a Test

from rest_framework import status
import pytest

@pytest.mark.django_db
class TestCollectionAPI:
    def test_if_user_is_anonymous(self, api_client):
        # Act
        response = api_client.post('/collections/', {'name': 'My Collection'})

        # Assert
        assert response.status_code == status.HTTP_401_UNAUTHORIZED

In this example, the api_client fixture builds one single APIClient for the test to use. This keeps the test short, and lets you reuse that same setup in other tests too.

Fixture with Dependencies

You can also build a fixture on top of another fixture. This comes in handy when one setup step depends on another. For example, you might want a helper that gives you back an API client that’s already logged in:

# conftest.py
from rest_framework.test import APIClient
import pytest

@pytest.fixture
def api_client():
    return APIClient()

@pytest.fixture
def create_collection(api_client):
    def _create_collection(name='My Collection'):
        return api_client.post('/collections/', {'name': name})
    return _create_collection

@pytest.fixture
def authenticate(api_client, django_user_model):
    def _authenticate(is_staff=False):
        user = django_user_model.objects.create_user(
            username='admin-user' if is_staff else 'regular-user',
            password='password123',
            is_staff=is_staff,
        )
        api_client.force_authenticate(user=user)
        return api_client
    return _authenticate

from rest_framework import status
import pytest

@pytest.mark.django_db
class TestCollectionAPI:
    def test_if_user_is_anonymous(self, create_collection):
        # Act
        response = create_collection(name='Test Collection')

        # Assert
        assert response.status_code == status.HTTP_401_UNAUTHORIZED

    def test_create_collection_as_admin(self, authenticate, create_collection):
        # Arrange
        authenticate(is_staff=True)

        # Act
        response = create_collection(name='Test Collection')

        # Assert
        assert response.status_code == status.HTTP_201_CREATED
        assert 'id' in response.data
        assert response.data['name'] == 'Test Collection'

This pattern is handy whenever one test step needs another one to happen first. Here, authenticate depends on api_client and django_user_model, so this fixture can create a real user and log the client in, before the actual test runs.

Testing GET Endpoints with model_bakery

When you’re testing a GET endpoint, you usually need some data sitting in the database first. Instead of creating that data by hand every single time, you can use model_bakery. It quickly creates model records with sensible default values, which makes your tests shorter and easier to keep up with.

Docs: https://model-bakery.readthedocs.io/en/latest/

uv add --dev model_bakery

Example: Retrieve a Single Collection

from rest_framework import status
from model_bakery import baker
import pytest
from some_app.models import Collection

@pytest.mark.django_db
class TestCollectionAPI:
    def test_get_collection(self, api_client, authenticate):
        # Arrange
        authenticate(is_staff=True)
        collection = baker.make(Collection)

        # Act
        response = api_client.get(f'/collections/{collection.id}/')

        # Assert
        assert response.status_code == status.HTTP_200_OK
        assert response.data == {
            'id': collection.id,
            'name': collection.name,
        }

Understanding _quantity in baker.make

from rest_framework import status
from model_bakery import baker
import pytest
from some_app.models import Collection, Product

@pytest.mark.django_db
class TestProductAPI:
    def test_get_product(self, api_client, authenticate):
        # Arrange
        authenticate(is_staff=True)
        collection = baker.make(Collection)
        products = baker.make(Product, collection=collection, _quantity=10)
        product = products[0]

        # Act
        response = api_client.get(f'/products/{product.id}/')

        # Assert
        assert response.status_code == status.HTTP_200_OK
        assert response.data == {
            'id': product.id,
            'name': product.name,
            'collection_id': collection.id,
        }

If you don’t pass collection into baker.make(Product, _quantity=10), model_bakery might automatically create separate Collection records on its own, which means different products could end up linked to different collections.

If you do pass collection=collection, every product you create gets linked to that same collection. This is useful when you want to test relationships and filtering using consistent, related data.

Performance Testing with Locust

Locust is a powerful, Python-based tool used to test how your API performs by simulating lots of real users using it at the same time. Instead of writing test scripts in some special testing language, you just describe user behavior using regular Python code. This makes it especially handy if you’re a backend developer (like you, working with Django/DRF), since you already know Python.

In simple terms, Locust helps you answer questions like:

• How many users can my API handle at the same time?
• What happens if traffic suddenly spikes?
• Which endpoints are slow, or failing, when under heavy load?
• Where exactly are the bottlenecks - the database, the network, or the code itself?

How Locust Works

Think of Locust as creating a bunch of virtual (fake) users, where each one keeps repeating actions (called tasks), like this:

User
 ├── wait (thinking time)
 ├── call API endpoint (/collections/)
 ├── wait
 ├── call API endpoint (/collections/{id}/)
 ├── wait
 └── repeat...

Each virtual user acts completely on its own, and Locust runs a whole bunch of them at the same time.

Knowing What to Test

Before you write any performance test, you need to figure out what actually matters most in your system.

For a Django REST API, the usual important areas are:

1. Read-heavy endpoints

   • Listing collections
   • Viewing a single resource
   • Searching/filtering

2. Write-heavy endpoints

   • Creating collections/products
   • Updating data

3. Authentication endpoints

   • Login
   • Token refresh

4. Complex queries

   • Endpoints that involve joins, filtering, or pagination

Example Strategy

If your API has these endpoints:

• /collections/ -> list
• /collections/{id}/ -> detail
• /collections/ (POST) -> create

Then realistic real-world usage probably looks something like:

80% -> read (GET)
20% -> write (POST)

So your performance test should try to match that same kind of split.

Installing Locust

uv add --dev locust

Conventions for Locust in Django

To keep your tests clean, easy to scale up, and simple to debug, follow these habits:

• one file per scenario

  locust/
      ├── browse_collections.py
      ├── create_collection.py

• one action per task: Each @task should represent just one single user action, nothing more.

• use wait_time: This simulates the natural pause a real user would take while thinking:

  wait_time = between(1, 5)

• use on_start: This is a special function that runs once, right when a user “starts up”. Use it for setup steps like logging in or grabbing a token.

  Runs once per user when they start:

  • login
  • setup tokens

• use name for grouping: This helps keep your reports clean and readable in the UI:

  name="/collections/{id}/"

Running Locust

1. Run Locust:

   uv run locust -f locust/browse_collections.py

2. Open your browser at: http://localhost:8089

3. Fill in these fields:

   • Users: 50
   • Spawn rate: 5
   • Host: http://localhost:8000

4. Start the test

Creating Locust Tasks

Below is a clean, beginner-friendly version of the code.

# locust/browse_collections.py
from locust import HttpUser, task, between
from random import randint


class BrowseCollectionsUser(HttpUser):
    """
    This user simulates read-heavy behavior:
    - Listing collections
    - Viewing a single collection
    """

    wait_time = between(1, 5)

    @task(3)  # higher weight -> more frequent
    def browse_collections(self):
        """
        Simulates user opening collection list page
        """
        self.client.get("/collections/", name="/collections/")

    @task(1)
    def browse_collection(self):
        """
        Simulates user opening a single collection
        """
        collection_id = randint(1, 20)  # adjust to your DB
        self.client.get(
            f"/collections/{collection_id}/",
            name="/collections/{id}/"
        )

# locust/create_collection.py
from locust import HttpUser, task, between
import random
import string


class CreateCollectionUser(HttpUser):
    """
    This user simulates write-heavy behavior:
    - Logging in
    - Creating collections
    """

    wait_time = between(1, 5)

    def on_start(self):
        """
        Runs once when a user starts.
        Used for authentication.
        """

        response = self.client.post(
            "/auth/login/",
            json={
                "username": "admin",
                "password": "password123"
            }
        )

        # Basic safety check
        if response.status_code != 200:
            raise Exception("Login failed")

        data = response.json()
        access_token = data.get("access")

        if not access_token:
            raise Exception("No access token received")

        # Attach token to all future requests
        self.client.headers.update({
            "Authorization": f"Bearer {access_token}"
        })

    @task
    def create_collection(self):
        """
        Simulates user creating a collection
        """

        random_name = "".join(
            random.choices(string.ascii_letters, k=10)
        )

        self.client.post(
            "/collections/",
            json={
                "name": f"Test {random_name}"
            },
            name="/collections/"
        )

Understanding the Code

1. HttpUser

This is the base class you use to describe how a user behaves. Every instance of HttpUser represents one simulated user that will go through the tasks you’ve written against your API.

HttpUser
   ├── task()
   ├── task()
   └── wait_time

Each user keeps running its tasks over and over, in a loop.

2. @task(weight)

We use the @task decorator to mark what actions a user performs. The optional weight number controls how often that task runs, compared to the others.

@task(3) -> runs 3x more often
@task(1) -> runs normally

So in our example:

browse_collections : browse_collection
         3         :        1

3. wait_time

This simulates the natural pause a real human would take:

wait_time = between(1, 5)

Without this:

• your API gets hit way harder than it would be in real life
• your results end up misleading

4. on_start()

A lifecycle hook (a special function): it runs exactly once, right when a user starts up. Use it for setup steps like logging in or grabbing a token. It’s also sometimes used to generate any dummy data needed before testing.

User starts
   ↓
on_start() runs once
   ↓
tasks start running

Used for:

• login
• fetching tokens
• setting headers

5. client

This is a wrapper around HTTP requests that Locust gives you. It lets you call your API endpoints, just like a real user’s browser or app would.

self.client.get(...)
self.client.post(...)

It automatically:

• tracks how long each response takes
• tracks failures
• sends all this data to the UI as metrics

6. name parameter

The name parameter inside self.client.get() is used to group similar requests together in the Locust UI. This matters a lot when your API has dynamic URLs (like /collections/{id}/), which would otherwise get counted as completely separate endpoints.

self.client.get(f"/collections/{id}/", name="/collections/{id}/")

Without name, Locust treats every single ID as its own separate endpoint:

/collections/1/
/collections/2/
/collections/3/

With name, it groups them all together as one:

/collections/{id}/

7. json parameter

When making POST requests, you can use the json parameter to send JSON data inside the request body. This is a simple, convenient way to send structured data to your API.

self.client.post(
    "/collections/",
    json={
        "name": "Test Collection"
    }
)

How to Use Locust

1. Run:

uv run locust -f locust/browse_collections.py

2. Open:

http://localhost:8089

3. Fill in:

• Users: 10
• Spawn rate: 2
• Host: http://localhost:8000

4. Click Start Swarming

Understanding Metrics

Requests per second (RPS)

Higher = better throughput

Response time percentiles

This tells you how fast (or slow) different groups of users experience your API:

• 50% -> the average user
• 95% -> the slower users
• 99% -> the worst-case scenario

Example:

50% -> 120ms
95% -> 800ms   ← WARNING

Failures

Any response that isn’t in the 200 range counts as a failure:

500 -> server crash
401 -> auth issue
400 -> bad request

Testing Strategy

1. Start small:

   5 users

2. Check that everything is working correctly
3. Slowly increase the load:

   10 -> 20 -> 50 -> 100 -> 1000

4. Keep an eye on:
   • sudden jumps in response time
   • an increase in failures

Test in Production?

Testing in your real, live production environment is risky, but sometimes it’s necessary. If you have to do it, follow these guidelines:

• Use a separate test environment if you can, so you don’t affect real users
• If you must test in production, only send a small slice of real traffic (like 1%), so you don’t overwhelm the system
• Keep a close eye on things while it’s happening
• Let your team and users know testing is happening, so nobody panics or gets confused

Django Silk (Profiling Tool)

Django Silk is a profiling tool that watches every single request, and shows you why your API is slow, by looking closely at the queries, how long they take, and where the bottlenecks are.

Note

Locust vs Silk:

• Locust: Shows you which endpoints are slow
• Silk: Shows you why they’re slow (too many queries, slow queries, or an inefficient ORM setup)

Docs: Django Silk GitHub

Installation & Setup

uv add django-silk

1. Add to INSTALLED_APPS:

if DEBUG:
    INSTALLED_APPS += ["silk"]

2. Add Middleware:

if DEBUG:
    MIDDLEWARE += ["silk.middleware.SilkyMiddleware"]

Put Silk’s middleware near the end of the list, so it gets to observe the entire request from start to finish.

3. Add URL Route:

# main urls.py
from django.urls import path, include
from django.conf import settings

if settings.DEBUG:
    urlpatterns += [path("silk/", include("silk.urls", name="silk"))]

4. Run Migrations:

python manage.py migrate

Using Silk

1. Start the server: python manage.py runserver
2. Open the dashboard: http://localhost:8000/silk/
3. Generate some requests (for example, using Locust)
4. Look through the requests, queries, and timing on the dashboard

Identifying N+1 Query Problems

Here’s a real example of how to spot an N+1 problem using Silk. (An “N+1 problem” means your code runs one query, and then ends up running one extra query for every single item it loops through, instead of getting everything in one go.)

Without optimization (N+1 problem):

# Bad: Multiple queries
users = User.objects.all()  # Query 1
for user in users:
    print(user.profile.bio)  # Query N+1 (one query per user)

Silk shows:

Query count: 11 queries (1 for users + 10 for profiles)
Total time: 450ms

With optimization:

# Good: Single query with prefetch_related
users = User.objects.prefetch_related('profile').all()
for user in users:
    print(user.profile.bio)  # No additional queries

Silk shows:

Query count: 2 queries
Total time: 45ms

Tip

Use Silk’s query inspector to see:

• The exact SQL queries that ran
• How long each query took
• How many queries ran per request

This makes N+1 problems jump out immediately.

Workflow

1. Run your performance test (using Locust)
2. Find the slow endpoints in the Silk dashboard
3. Look at the queries to figure out what’s going wrong
4. Turn off Silk in your settings (it adds extra overhead of its own)
5. Fix your code (use select_related, prefetch_related, and add database indexes where needed)
6. Run your tests again, without Silk, to see the real improvement

Caution

Always turn off Silk before measuring real performance. Silk itself adds some overhead through its middleware, which can throw off your results. Clear out Silk’s saved data before each new test run, so you get clean, accurate numbers.

Optimizing ORM Queries

• Use select_related() for ForeignKey relationships (this does a single combined query, called a join)
• Use prefetch_related() for ManyToMany relationships and reverse ForeignKeys (this runs a few queries, but caches the results so they’re reused)
• Check the raw SQL with .query to see exactly what Django is generating behind the scenes
• Keep an eye on how many queries you’re running: fewer queries usually means a faster API

Note

Quick Diagnosis:

Slow API -> Run Locust -> Find slow endpoint -> Use Silk ->
Inspect queries -> Optimize ORM -> Test again

Performance Optimisation

Performance optimisation in a Django app is really about cutting out unnecessary work, at every single layer of your app:

Request -> Django ORM -> SQL Query -> Database -> Response

If even one of these layers is slow or wasteful, your whole API ends up feeling slow. The goal is to keep database load, memory use, and response time as low as possible, especially on endpoints that get hit a lot.

Optimising Django ORM Queries

Django’s ORM is powerful, but if you’re not careful, it can end up generating SQL that’s pretty inefficient.

1. Avoid N+1 Queries

This happens when Django runs one query for the main object, and then runs extra queries, one by one, for each related object.

# BAD (N+1 problem)
products = Product.objects.all()
for p in products:
    print(p.category.name)  # triggers extra query per product

Fix using select_related (for ForeignKey / OneToOne)

products = Product.objects.select_related("category").all()

• Runs one combined SQL query (a JOIN)
• Grabs the related object in that same query

---

Fix using prefetch_related (for ManyToMany / reverse FK)

products = Product.objects.prefetch_related("tags").all()

• Runs a few separate queries
• Then combines the results together in Python
• Works efficiently for many-to-many relationships

2. Load Only Required Fields

Try not to load data you don’t actually need.

Using only

products = Product.objects.only("id", "title")

• Loads only the fields you asked for
• Every other field gets skipped (deferred) automatically

Using defer

products = Product.objects.defer("description")

• Skips fields that are large or that you’re not going to use
• Basically the opposite of only

---

3. Use values() and values_list()

products = Product.objects.values("id", "title")

Returns:

[
  {"id": 1, "title": "A"},
  {"id": 2, "title": "B"}
]

products = Product.objects.values_list("id", "title")

Returns:

[(1, "A"), (2, "B")]

Why is this faster?

Model instance -> heavy (methods, state, ORM overhead)
Dict/List -> lightweight (less memory, faster)

Use these when:

• You don’t need any model methods (like save or delete)
• You only care about the raw data itself

---

4. Efficient Counting

# BAD
len(Product.objects.all())

• This loads every single record into memory first

# GOOD
Product.objects.count()

• This runs SELECT COUNT(*) directly in the database
• Much faster, and uses way less memory

5. Bulk Operations

Bulk Create

Product.objects.bulk_create([
    Product(title="A"),
    Product(title="B"),
])

Bulk Update

Product.objects.bulk_update(products, ["title"])

Why does this matter?

Loop create -> N database queries
Bulk create -> 1 database query

When ORM Is Not Enough

Sometimes Django’s ORM ends up generating SQL that’s just not efficient enough.

Rewrite Query Using Raw SQL

from django.db import connection

with connection.cursor() as cursor:
    cursor.execute("SELECT ...")
    rows = cursor.fetchall()

Use this when:

• The query is genuinely complex
• The ORM keeps generating slow SQL
• You’re comfortable writing SQL yourself

Database Optimisation

If your queries are still slow even after all this, the real problem is often in how your database is designed.

1. Add Indexes

class Product(models.Model):
    title = models.CharField(max_length=255, db_index=True)

• Speeds up filtering and searching
• Becomes essential once your tables get large

---

2. Optimise Schema

• Organize your data properly (this is called normalization)
• Avoid joins you don’t actually need
• Use the right field types for your data

---

Caching (Use Carefully)

Caching means storing results in memory, so you don’t have to keep hitting the database with the same query over and over.

First request -> slow (DB hit)
Next requests -> fast (cache hit)

Example Flow

Request -> Check cache
         ├── Hit -> return data
         └── Miss -> query DB -> store in cache -> return

Important Note

Caching is not always faster:

Cache server (Redis) -> network call
Database -> local query

If the query is simple enough, your database might actually be faster than reaching out to a cache server.

Use caching for:

• Queries that are genuinely expensive to run
• Data that gets accessed very often

Scaling the Application

If you’ve already optimized everything, but performance still drops once traffic gets heavy:

1. Vertical Scaling

Upgrade the server itself:

More CPU
More RAM
Faster disk

2. Horizontal Scaling

Add more servers instead:

Load Balancer
   ├── Server 1
   ├── Server 2
   └── Server 3

• Lets you handle more users at the same time
• Adds more complexity to your setup
• Costs more money

Practical Strategy

Don’t try to optimize absolutely everything.

Focus your time on:

High traffic endpoints
Critical user paths
Slow queries

Avoid spending time on:

Admin reports used rarely
Low-impact features

Stress Testing with Locust

Stress testing means pushing your API past its normal limits on purpose, so you can see how it behaves under extreme conditions. It helps you find the exact breaking point, spot bottlenecks, and see how well your system recovers after something goes wrong.

It’s not strictly required, but stress testing is a good habit to build into your workflow once you’ve finished your regular performance testing and optimization. It lets you confirm that your optimizations actually hold up under heavy load, and helps you spot any weak spots that are still left.

In this section, you’ll use Locust to simulate a large number of users hitting your API at the exact same time, and watch how it performs under that kind of stress.

By the end, you’ll have a clear picture of where your API’s breaking point is, and how it behaves during high-traffic situations.

Important Note

The development server isn’t built for production use, and it might not handle heavy traffic well at all. For accurate stress testing, use an environment that closely matches production (like a staging server), so your results actually reflect how your real deployment would behave.