srijan's blog

Optimizing a DRF application for I/O bound, AI workloads

Thu, 02 Oct 2025 00:00:00 +0000

DRF is my favourite toolkit for building Web API’s.

However, it has one fatal flaw – it does not support async out of the box. This shouldn’t be a problem for most use cases but if your application is I/O heavy (basically every GPT wrapper on the market right now), this can be a significant bottleneck.

What causes this bottleneck?

The GIL prevents more than one thread within a process from executing Python bytecode at any given point in time. Which means, while your worker is waiting for a response from an I/O call (API call to Gemini, Anthropic, OpenAI, etc.), it just sits there doing nothing. Any other requests assigned to the worker will have to wait until the first request concludes.

There’re a couple of ways to overcome this bottleneck.

For starters, you could try increasing the number of workers used by your application server. So when one of your workers is busy waiting for an I/O call, another worker can handle other requests in the queue. This is the easiest form of horizontal scaling. However, workers are “process-bound”.

A program, (your DRF code) is merely a collection of passive instructions. The active execution of these instructions is called a process. Processes have an independent memory space, a copy of the Python interpreter, a copy of your application’s code, and most importantly their own GIL.

This means you can only have a finite number of processes running on a CPU at any given time, considering the high resource requirement that processes bring. This finite number is not nearly large enough to handle the scale modern web applications operate at. The number of workers you can spawn on a server is usually 2n + 1, where n is the number of cores on the CPU.

Each pid corresponds to one Granian (an HTTP server similar to gunicorn) worker. Notice how each worker takes up between 400 to 600 MB of memory.

To put things into perspective, an i9-9900K, a 16-core CPU, would be able to host 33 workers at most.

But all of these limitations exist assuming we only have one thread per process. What if we had multiple threads capable of handling multiple requests?

Preemptive multitasking

This is where gunicorn’s gthread worker type comes in. While thread A waits for a response from a network call, it surrenders the GIL to thread B. This means thread B is now free to handle other requests. Once thread A’s I/O operation concludes, it will fall in line behind any other threads that are waiting to acquire the GIL. Seems like the ideal solution to our problems. WRONG.

Hopping threads is a kernel-level operation. This means the OS has to context-switch between user-mode and kernel-mode. The kernel then has to save the context of the old thread and load the entire context of the new thread before making the switch back to user-mode. This is an accurate, safe but slow operation.

So while the gthread worker type ensures that a worker is not entirely handicapped while doing I/O, it comes at the cost of having to constantly switch between threads, which is not a cheap operation and relies heavily on the kernel and the OS scheduler.

Note that the GIL allows only one thread within a process to execute Python bytecode at once, so true multithreading (parallelism) for CPU-intensive tasks isn’t possible in Python.

Cooperative multitasking

This is where the gunicorn “gevent” worker type comes in. gevent “greenlets” are lightweight Python objects that can be spawned 100s and thousands of times within a process.

Greenlets work on the principle of “cooperative multitasking”. Which means, a greenlet will continue to work on a request, without interruption, until it hits a blocking I/O call. There’s no external body like the OS scheduler or kernel making decisions about when control should be confiscated from one thread and transferred to another.

The OS has absolutely no idea that greenlets exist. A “context-switch” between greenlets is just a function call within the same process. There’s no transition to kernel mode needed to make this happen.

Then why would one bother using threads at all? While greenlets are incredibly lightweight and dramatically boost your worker’s throughput, there’re no guardrails preventing a greenlet from completely monopolizing the thread it’s running on, and pushing every other greenlet into starvation.

If a thread enters an infinite CPU-bound loop, the OS can forcefully preempt it and allow other threads to run.

There’s no mechanism that enables gevent to do the same.

It relies on the greenlet to voluntarily yield control after hitting a blocking I/O operation, leaving the other greenlets vulnerable to starvation, should the current greenlet hit an infinite CPU-bound loop.

What does Gevent do under the hood

But how does synchronous code (requests.get(), time.sleep(), etc.) suddenly start behaving cooperatively without you changing a single line? The answer is this goofy technique called monkey-patching.

The default blocking implementations of modules like socket and requests get modified by gevent to use its cooperative methods at runtime.

Emphasis on “modified” because the entire module doesn’t actually get replaced, just the blocking functions and classes.

When you do

from gevent import monkey
monkey.patch_all()

At the beginning of your wsgi.py file, patch_all() essentially imports all blocking versions of all compatible modules, and modifies them by replacing their blocking implementation with its cooperative ones. This is a one-time operation that happens before any of your other application code executes. So every time you import the requests or socket module in your source code, you’ll actually be running the version that has been patched by gevent.

TL;DR:

In your wsgi.py file, call the patch_all() method before any of the boilerplate code that you already have. Your wsgi.py file should look something like this

from gevent import monkey
monkey.patch_all()

import os
from django.core.wsgi import get_wsgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'myproject.settings')

application = get_wsgi_application()

While deploying your application (I'm assuming you’re already using a production-ready WSGI server)

gunicorn myproject.wsgi:application \
 --worker-class gevent \
 --workers 4 \
 --bind 0.0.0.0:8000

In the next part of this series, I’ll do a benchmark that compares the throughput of a single-threaded gunicorn worker vs a gevent one.

Integrating 'Sign in with Apple' into your Django project without all-auth

Thu, 19 Dec 2024 13:05:55 +0530

Apple’s documentation is pretty obscure, so here’s a comprehensive guide on integrating apple’s “Sign in with Apple” feature with your Django backend without using external libraries like Django-allauth.

Why avoid Django-allauth? While it’s a powerful library, it can be overkill for this specific use case. We’ll implement a clean, minimal solution instead.

Apple uses oauth2.0, even though its not explicitly mentioned in the docs, so if that’s something you’re already familiar with, this shouldn’t be too difficult.

Let’s start with the things you need to setup on your Apple developer console

Setting up your Apple developer account

Enroll into the Apple developer program
- This can take a while to verify even after you’hv processed the payment (upto 48 hrs in some cases)
Once you have access to the developer console, make an App ID at (list of identifiers)
- This is fairly straight forward, just make sure you select Sign in with apple under the “Capabilities” section.
- Note: Your bundle ID needs to be unique, i.e you cannot have multiple App ID’s or service ID’s with the same bundle ID.
Create a service ID, click the dropdown at (list of identifiers) and select “Service IDs”. This is what the header should look like.

After you’re done creating your service ID, go back to your list of service ID’s and select the one you want to configure with SIWA.
You’ll see the following page open,

keep the “sign in with apple” box checked and click the configure button

You should see the following pop up

In the Domains and subdomains section, enter just the domain name of your application without the https prefix. The “Return URLs” section should contain the endpoint in your application responsible for handling Apple’s callback after the User successfully completes the sign in process on the client side.

For Samantha, our “domains and subdomains” section would contain : mysamantha.ai

And our Return URL’s section should contain: https://mysamantha.ai/account/apple/callback

Notice how the return url is prefixed with https. Now, Apple being apple, does NOT allow you to use domains without a valid SSL certificate, which means you’ll only be allowed to use return URLs with HTTPS and not HTTP.

This is can be cumbersome to set for development and testing but its fairly straightforward to use once you have it setup. I used mkcert to create a locally signed SSL certificate and nginx as a reverse proxy.

Now, to make sure that your subdomains and return URLs actually register, make sure you follow the following steps:

Click Next

Click Done

After that step you’ll see this page again.

Don’t just stop here, click continue and click save in the following page.

Your configuration URL’s will not be saved otherwise, even if they seem like they’re added in the URL dropdown that looks like this:

You’ll now have to create a private key that’ll be used to Sign the JWT that’ll contain your Client ID and Team ID.
- Click configure and select the App ID that you want to associate the private key with
- Download the key. Note that this key is deleted from Apple’s servers after you download it, so make sure you keep it safe.
If you followed all of the above steps, you should now have the following:
- Team ID
- Key ID
- Service ID
- Redirect URI
- Private key (.p8 file) That concludes the setup on Apple’s side of things.

Let’s now take a look at how we should go about integrating this with our Django backend.

Setting up your Django backend

To initiate the Apple login process, you first have to redirect your user to the following link https://appleid.apple.com/auth/authorize. The following data must be sent as query parameters to Apple.

Attribute	Description
client_id	your apple client id (service id)
redirect_uri	endpoint to your application set in the console
response_type	“code” (send this exact string)
scope	“email” (send this exact string)
response_mode	“form_post”

class AppleLogin(View):
 def get(self, request):
 params = {
 "client_id": getattr(settings, "APPLE_CLIENT_ID"),
 "redirect_uri" : getattr(settings, "APPLE_REDIRECT_URI"),
 "response_type" : "code", #send this exact string, 
 "scope" : "name email",
 "response_mode": "form_post"
 }

 return HttpResponseRedirect(f"https://appleid.apple.com/auth/authorize?{urlencode(params)}")

Once the sign in process has concluded on apple’s end, on a high level, the following steps take place:
- Apple sends an authorization token to your application’s endpoint.
- You generate a JWT signed with the private key you downloaded earlier.
- This JWT is sent back to apple via a post request, along with the authorization code you received earlier. - If all goes well, apple responds with a JSON object that contains a JWT (id_token) with the user’s details.
- You can verify the signature of this JWT using apple’s public keys available at https://appleid.apple.com/auth/keys

Lets explore each of these steps in detail

Get the authorization code from Apple by retrieving the code key-value pair from the POST dictionary. This token will later be exchanged for a JWT from apple containing the user’s details.

def post(self, request):
 authorization_code = request.POST["code"]
 if not authorization_code:
 return HttpResponse("authorization code not found")

Before the token exchange can take place, Apple needs a way to verify if the request to exchange tokens is actually coming from your registered application. This is achieved by sending a JWT signed with our private key in addition to the authorization code in our post request to Apple’s token endpoint (https://appleid.apple.com/auth/token)
We’ll use the following helper function to generate the JWT that we’ll send to Apple.

def generate_client_key(self):
 headers = {
 "alg" : "ES256",
 "kid" : getattr(settings, "APPLE_PRIV_KEY_ID"),
 "typ" : "JWT"
 }

 payload = {
 "iss" : getattr(settings, "APPLE_TEAM_ID"),
 "iat" : int(timezone.now().timestamp()),
 "exp" : int((timezone.now() + timezone.timedelta(days=1)).timestamp()),
 "aud" : "https://appleid.apple.com",
 "sub" : getattr(settings, "APPLE_CLIENT_ID")
 }

 private_key = getattr(settings, "APPLE_PRIVATE_KEY")

 encoded_string = jwt.encode(
 payload=payload,
 key=private_key,
 algorithm="ES256",
 headers=headers
 )
 return encoded_string

Now that we have both the authorization token and our JWT (client_secret), the prerequisites for token exchange are fulfilled.
send a post request to https://appleid.apple.com/auth/token with the following fields in the payload.

Attribute	Description
client_id	your apple client id (service id)
client_secret	the jwt you signed with your private key
code	authorization code from apple
grant_type	“authorization_code” (send this exact string)
redirect_uri	the redirect uri you set on the developer console

Content type in the headers MUST be set to to application/x-www-form-urlencoded. application/form-data or any other MIME type for that matter, will NOT work.

token_url = "https://appleid.apple.com/auth/token"
 client_secret = self.generate_client_key()
 data = {
 "client_id": getattr(settings, "APPLE_CLIENT_ID"),
 "client_secret": client_secret,
 "code": authorization_code,
 "grant_type": "authorization_code",
 "redirect_uri": getattr(settings, "APPLE_REDIRECT_URI")
 }

 headers = {"Content-Type": "application/x-www-form-urlencoded"}

 token_exchange_response = requests.post(token_url, data = data, headers = headers)

 token_exchange_response = token_exchange_response.json()

If everything goes well, you’ll receive a response from Apple with a body that looks something like this:

{
 "access_token": "<access token here>",
 "token_type": "Bearer",
 "expires_in": 3600,
 "refresh_token": "<refresh token here>",
 "id_token": "<id token here>"
}

The id_token key holds the JWT encoded with the user’s details. Decoding this JWT should give the following result in the following format.

{
 "iss": "https://appleid.apple.com",
 "aud": "com.zemuria.test",
 "exp": 1733295098,
 "iat": 1733208698,
 "sub": "101549.1a31c4cf89484d0797s4b113fd14db7.0833",
 "at_hash": "-oF09Dzjbiiymu0wnG_ZnQ",
 "email": "example@gmail.com",
 "email_verified": true,
 "auth_time": 1733208696,
 "nonce_supported": true
}

Verifying the JWT signature

That should effectively be the end of the sign in with apple process but how would know if the JWT your endpoint received is actually sent by apple?

Best practice here would be to ensure that the JWT you got, is sent from Apple by verifying the JWT’s signature using one of Apple’s JSON Web Keys (JWK’s) at: https://appleid.apple.com/auth/keys.

Apple uses the RS256 algorithm to sign and verify their JWT’s, which means we’ll have to create an RSA public key using the n and e fields from the Apple JWK’s.

This is fairly straightforward to achieve using the cryptography library in python. Convert the resulting public key to PEM format and use it to decode the JWT. If all goes well, we should now have the JWT’s payload with the user’s details.

#verifying jwt signature
 public_keys_response = requests.get("https://appleid.apple.com/auth/keys")
 apple_pubic_keys = json.loads(public_keys_response.text)["keys"] #apple public keys
 jwt_header = jwt.get_unverified_header(token_exchange_response["id_token"]) #get header from id_token jwt
 key_id = jwt_header["kid"]

 public_key_data = next(key for key in apple_pubic_keys if key["kid"] == key_id)

 #Convert JWK to RSA Public Key
 n = int.from_bytes(base64url_decode(public_key_data["n"]), "big")
 e = int.from_bytes(base64url_decode(public_key_data["e"]), "big")
 public_key = RSAPublicNumbers(n=n, e=e).public_key(default_backend())

 public_key_pem = public_key.public_bytes(
 encoding=Encoding.PEM,
 format=PublicFormat.SubjectPublicKeyInfo,
 )

 user_data = jwt.decode(
 key = public_key_pem,
 verify = True,
 jwt = token_exchange_response["id_token"],
 algorithms = ["RS256"],
 options = {
 "verify_signature": True,
 }
 )

Tips:

Paste your private key into a .pem file instead of directly placing it in your env. Read the pem file using the os library
Make sure to follow all the way through and click save to make sure your redirect URIs register in the developer console