Comparing FastAPI behaviour and performance under various synchronous/asynchronous settings.

python3 -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

# Serve the API through Gunicorn using 1 worker (edit serve.sh to change number of workers)
sh serve.sh

# In another terminal, run a stress test against a synchronous or asynchronous endpoint:

# Make HTTP request and return
sh stress_sync_http.sh
sh stress_async_http.sh
sh stress_async_http_sync.sh

# Load JSON file from disk and return
sh stress_sync_json.sh
sh stress_async_json.sh

The service will start reporting anyio.BrokenResourceError about 30 seconds into each test. This is likely caused by the logging middleware, see this issue. Removing the middleware gets rid of the error and improves performance slightly, but the logging is needed to keep track of threads for the test results.

Following is a number of different scenario designed to shed light on how a FastAPI application behaves when doing different kinds of IO in different kinds of endpoints.

In all the scenarios, the IO being done is making a HTTP request to a simple GoLang API that waits for 1000 milliseconds before responding.

Description: Making synchronous HTTP GET requests from synchronous endpoints (using httpx.Client() for making requests)

@app.get('/http/sync')
def http_sync():
    with httpx.Client() as http:
        return http.get('https://165.227.149.214:8090?waitms=1000').content

The simulated users make GET requests to our FastAPI endpoint /sync/http. The endpoint, in turn, makes a synchronous GET request to https://165.227.149.214:8090?waitms=1000, a simple GoLang API that waits for the provided number of milliseconds on each request.

• You can also run the GoLang server locally with serve_external_http.sh.

• Num. users: 500
• Test runtime: 1 minute
• Num. requests: 2122
• Num. failures: 0
• Average requests/second: 35.6
• Average failures/second: 0
• Median response time: 1300 ms
• Average num. active threads: 41
• Num. unique threads created: 41

• Synchronous IO in synchronous endpoints is not blocking, since the max number of requests in this timespan would be 60.

Description: Making asynchronous HTTP GET requests from synchronous endpoints (using httpx.AsyncClient() for making requests)

@app.get('/async/http')
async def http_async():
    async with httpx.AsyncClient() as http:
        return (await http.get('https://165.227.149.214:8090?waitms=1000')).content

The simulated users make GET requests to our FastAPI endpoint /async/http. The endpoint, in turn, awaits an asynchronous GET request to https://165.227.149.214:8090?waitms=1000, a simple GoLang API that waits for the provided number of milliseconds on each request.

• You can also run the GoLang server locally with serve_external_http.sh.

• Num. users: 500
• Test runtime: 1 minute
• Num. requests: 3190
• Num. failures: 0
• Average requests/second: 53.2
• Average failures/second: 0
• Median response time: 8300 ms
• Average num. active threads: 1
• Num. unique threads created: 1

• More requests were handled than by /sync/http, but response times were significantly worse (9000 ms vs. 1300 ms)
• All requests were being handled by a single thread; async def endpoints are not multithreaded like their def endpoint counterparts are

Description: Making synchronous HTTP GET requests from asynchronous endpoints (using httpx.Client() for making requests)

@app.get('/http/async/sync')
async def http_sync():
    with httpx.Client() as http:
        return http.get('https://165.227.149.214:8090?waitms=1000').content

The simulated users make GET requests to our FastAPI endpoint /async/http/sync. The endpoint, in turn, makes a synchronous GET request to https://165.227.149.214:8090?waitms=1000, a simple GoLang API that waits for the provided number of milliseconds on each request.

• You can also run the GoLang server locally with serve_external_http.sh.

• Num. users: 500
• Test runtime: 1 minute
• Num. requests: 785
• Num. failures: 732 (timeouts, connection reset, remote disconnected)
• Average requests/second: 13.08
• Average failures/second: 12.2
• Median response time: 27000 ms
• Average num. active threads: 1
• Num. unique threads created: 1

• Synchronous IO in asynchronous endpoints is blocking
• Blocked requests cause other requests to time out and fail