@ -94,7 +94,7 @@ In most cases, when you create a web API, you want it to be **always running**,
### In a Remote Server
When you set up a remote server (a cloud server, a virtual machine, etc.) the simplest thing you can do is to use `fastapi run`, Uvicorn (or similar) manually, the same way you do when developing locally.
When you set up a remote server (a cloud server, a virtual machine, etc.) the simplest thing you can do is use `fastapi run` (which uses Uvicorn) or something similar, manually, the same way you do when developing locally.
And it will work and will be useful **during development**.
@ -178,7 +178,7 @@ For example, this could be handled by:
## Replication - Processes and Memory
With a FastAPI application, using a server program like Uvicorn, running it once in **one process** can serve multiple clients concurrently.
With a FastAPI application, using a server program like the `fastapi` command that runs Uvicorn, running it once in **one process** can serve multiple clients concurrently.
But in many cases, you will want to run several worker processes at the same time.
@ -232,9 +232,7 @@ The main constraint to consider is that there has to be a **single** component h
Here are some possible combinations and strategies:
* **Gunicorn** managing **Uvicorn workers**
* Gunicorn would be the **process manager** listening on the **IP** and **port**, the replication would be by having **multiple Uvicorn worker processes**.
* **Uvicorn** managing **Uvicorn workers**
* **Uvicorn** with `--workers`
* One Uvicorn **process manager** would listen on the **IP** and **port**, and it would start **multiple Uvicorn worker processes**.
* **Kubernetes** and other distributed **container systems**
* Something in the **Kubernetes** layer would listen on the **IP** and **port**. The replication would be by having **multiple containers**, each with **one Uvicorn process** running.
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
# (1)
# (1)!
COPY ./main.py /code/
# (2)
# (2)!
CMD ["fastapi", "run", "main.py", "--port", "80"]
```
@ -456,11 +456,11 @@ Without using containers, making applications run on startup and with restarts c
## Replication - Number of Processes
If you have a <abbrtitle="A group of machines that are configured to be connected and work together in some way.">cluster</abbr> of machines with **Kubernetes**, Docker Swarm Mode, Nomad, or another similar complex system to manage distributed containers on multiple machines, then you will probably want to **handle replication** at the **cluster level** instead of using a **process manager** (like Gunicorn with workers) in each container.
If you have a <abbrtitle="A group of machines that are configured to be connected and work together in some way.">cluster</abbr> of machines with **Kubernetes**, Docker Swarm Mode, Nomad, or another similar complex system to manage distributed containers on multiple machines, then you will probably want to **handle replication** at the **cluster level** instead of using a **process manager** (like Uvicorn with workers) in each container.
One of those distributed container management systems like Kubernetes normally has some integrated way of handling **replication of containers** while still supporting **load balancing** for the incoming requests. All at the **cluster level**.
In those cases, you would probably want to build a **Docker image from scratch** as [explained above](#dockerfile), installing your dependencies, and running **a single Uvicorn process** instead of running something like Gunicorn with Uvicorn workers.
In those cases, you would probably want to build a **Docker image from scratch** as [explained above](#dockerfile), installing your dependencies, and running **a single Uvicorn process** instead of using multiple Uvicorn workers.
### Load Balancer
@ -490,37 +490,44 @@ And normally this **load balancer** would be able to handle requests that go to
In this type of scenario, you probably would want to have **a single (Uvicorn) process per container**, as you would already be handling replication at the cluster level.
So, in this case, you **would not** want to have a process manager like Gunicorn with Uvicorn workers, or Uvicorn using its own Uvicorn workers. You would want to have just a **single Uvicorn process** per container (but probably multiple containers).
So, in this case, you **would not** want to have a multiple workers in the container, for example with the `--workers` command line option.You would want to have just a **single Uvicorn process** per container (but probably multiple containers).
Having another process manager inside the container (as would be with Gunicorn or Uvicorn managing Uvicorn workers) would only add **unnecessary complexity** that you are most probably already taking care of with your cluster system.
Having another process manager inside the container (as would be with multiple workers) would only add **unnecessary complexity** that you are most probably already taking care of with your cluster system.
### Containers with Multiple Processes and Special Cases
Of course, there are **special cases** where you could want to have **a container** with a **Gunicorn process manager** starting several **Uvicorn worker processes** inside.
Of course, there are **special cases** where you could want to have **a container** with several **Uvicorn worker processes** inside.
In those cases, you can use the **official Docker image** that includes **Gunicorn** as a process manager running multiple **Uvicorn worker processes**, and some default settings to adjust the number of workers based on the current CPU cores automatically. I'll tell you more about it below in [Official Docker Image with Gunicorn - Uvicorn](#official-docker-image-with-gunicorn-uvicorn).
In those cases, you can use the `--workers` command line option to set the number of workers that you want to run:
Here are some examples of when that could make sense:
```{ .dockerfile .annotate }
FROM python:3.9
#### A Simple App
WORKDIR /code
You could want a process manager in the container if your application is **simple enough** that you don't need (at least not yet) to fine-tune the number of processes too much, and you can just use an automated default (with the official Docker image), and you are running it on a **single server**, not a cluster.
COPY ./requirements.txt /code/requirements.txt
#### Docker Compose
RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt
You could be deploying to a **single server** (not a cluster) with **Docker Compose**, so you wouldn't have an easy way to manage replication of containers (with Docker Compose) while preserving the shared network and **load balancing**.
COPY ./app /code/app
Then you could want to have **a single container** with a **process manager** starting **several worker processes** inside.
1. Here we use the `--workers` command line option to set the number of workers to 4.
You could also have **other reasons** that would make it easier to have a **single container** with **multiple processes** instead of having **multiple containers** with **a single process** in each of them.
Here are some examples of when that could make sense:
For example (depending on your setup) you could have some tool like a Prometheus exporter in the same container that should have access to **each of the requests** that come.
#### A Simple App
You could want a process manager in the container if your application is **simple enough** that can run it on a **single server**, not a cluster.
#### Docker Compose
In this case, if you had **multiple containers**, by default, when Prometheus came to **read the metrics**, it would get the ones for **a single container each time** (for the container that handled that particular request), instead of getting the **accumulated metrics** for all the replicated containers.
You could be deploying to a **single server** (not a cluster) with **Docker Compose**, so you wouldn't have an easy way to manage replication of containers (with Docker Compose) while preserving the shared network and **load balancing**.
Then, in that case, it could be simpler to have **one container** with **multiple processes**, and a local tool (e.g. a Prometheus exporter) on the same container collecting Prometheus metrics for all the internal processes and exposing those metrics on that single container.
Then you could want to have **a single container** with a **process manager** starting **several worker processes** inside.
---
@ -541,7 +548,7 @@ And then you can set those same memory limits and requirements in your configura
If your application is **simple**, this will probably **not be a problem**, and you might not need to specify hard memory limits. But if you are **using a lot of memory** (for example with **machine learning** models), you should check how much memory you are consuming and adjust the **number of containers** that runs in **each machine** (and maybe add more machines to your cluster).
If you run **multiple processes per container**(for example with the official Docker image) you will have to make sure that the number of processes started doesn't **consume more memory** than what is available.
If you run **multiple processes per container** you will have to make sure that the number of processes started doesn't **consume more memory** than what is available.
## Previous Steps Before Starting and Containers
@ -561,80 +568,26 @@ If in your use case there's no problem in running those previous steps **multipl
### Single Container
If you have a simple setup, with a **single container** that then starts multiple **worker processes** (or also just one process), then you could run those previous steps in the same container, right before starting the process with the app. The official Docker image supports this internally.
If you have a simple setup, with a **single container** that then starts multiple **worker processes** (or also just one process), then you could run those previous steps in the same container, right before starting the process with the app.
## Official Docker Image with Gunicorn - Uvicorn
### Base Docker Image
There is an official Docker image that includes Gunicorn running with Uvicorn workers, as detailed in a previous chapter: [Server Workers - Gunicorn with Uvicorn](server-workers.md){.internal-link target=_blank}.
There used to be an official FastAPI Docker image: <ahref="https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker"class="external-link"target="_blank">tiangolo/uvicorn-gunicorn-fastapi</a>. But it is now deprecated. ⛔️
This image would be useful mainly in the situations described above in: [Containers with Multiple Processes and Special Cases](#containers-with-multiple-processes-and-special-cases).
There's a high chance that you **don't** need this base image or any other similar one, and would be better off by building the image from scratch as [described above in: Build a Docker Image for FastAPI](#build-a-docker-image-for-fastapi).
///
You should probably **not** use this base Docker image (or any other similar one).
This image has an **auto-tuning** mechanism included to set the **number of worker processes** based on the CPU cores available.
If you are using **Kubernetes** (or others) and you are already setting **replication** at the cluster level, with multiple **containers**. In those cases, you are better off **building an image from scratch** as described above: [Build a Docker Image for FastAPI](#build-a-docker-image-for-fastapi).
It has **sensible defaults**, but you can still change and update all the configurations with **environment variables** or configuration files.
And if you need to have multiple workers, you can simply use the `--workers` command line option.
It also supports running <ahref="https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker#pre_start_path"class="external-link"target="_blank">**previous steps before starting**</a> with a script.
/// note | Technical Details
/// tip
The Docker image was created when Uvicorn didn't support managing and restarting dead workers, so it was needed to use Gunicorn with Uvicorn, which added quite some complexity, just to have Gunicorn manage and restart the Uvicorn worker processes.
To see all the configurations and options, go to the Docker image page: <ahref="https://github.com/tiangolo/uvicorn-gunicorn-fastapi-docker"class="external-link"target="_blank">tiangolo/uvicorn-gunicorn-fastapi</a>.
But now that Uvicorn (and the `fastapi` command) support using `--workers`, there's no reason to use a base Docker image instead of building your own (it's pretty much the same amount of code 😅).
///
### Number of Processes on the Official Docker Image
The **number of processes** on this image is **computed automatically** from the CPU **cores** available.
This means that it will try to **squeeze** as much **performance** from the CPU as possible.
You can also adjust it with the configurations using **environment variables**, etc.
But it also means that as the number of processes depends on the CPU the container is running, the **amount of memory consumed** will also depend on that.
So, if your application consumes a lot of memory (for example with machine learning models), and your server has a lot of CPU cores **but little memory**, then your container could end up trying to use more memory than what is available, and degrading performance a lot (or even crashing). 🚨
### Create a `Dockerfile`
Here's how you would create a `Dockerfile` based on this image:
```Dockerfile
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9
COPY ./requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt
COPY ./app /app
```
### Bigger Applications
If you followed the section about creating [Bigger Applications with Multiple Files](../tutorial/bigger-applications.md){.internal-link target=_blank}, your `Dockerfile` might instead look like:
```Dockerfile hl_lines="7"
FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9
COPY ./requirements.txt /app/requirements.txt
RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt
COPY ./app /app/app
```
### When to Use
You should probably **not** use this official base image (or any other similar one) if you are using **Kubernetes** (or others) and you are already setting **replication** at the cluster level, with multiple **containers**. In those cases, you are better off **building an image from scratch** as described above: [Build a Docker Image for FastAPI](#build-a-docker-image-for-fastapi).
This image would be useful mainly in the special cases described above in [Containers with Multiple Processes and Special Cases](#containers-with-multiple-processes-and-special-cases). For example, if your application is **simple enough** that setting a default number of processes based on the CPU works well, you don't want to bother with manually configuring the replication at the cluster level, and you are not running more than one container with your app. Or if you are deploying with **Docker Compose**, running on a single server, etc.
## Deploy the Container Image
After having a Container (Docker) Image there are several ways to deploy it.
@ -647,98 +600,9 @@ For example:
* With another tool like Nomad
* With a cloud service that takes your container image and deploys it
## Docker Image with Poetry
## Docker Image with `uv`
If you use <ahref="https://python-poetry.org/"class="external-link"target="_blank">Poetry</a> to manage your project's dependencies, you could use Docker multi-stage building:
```{ .dockerfile .annotate }
# (1)
FROM python:3.9 as requirements-stage
# (2)
WORKDIR /tmp
# (3)
RUN pip install poetry
# (4)
COPY ./pyproject.toml ./poetry.lock* /tmp/
# (5)
RUN poetry export -f requirements.txt --output requirements.txt --without-hashes
1. This is the first stage, it is named `requirements-stage`.
2. Set `/tmp` as the current working directory.
Here's where we will generate the file `requirements.txt`
3. Install Poetry in this Docker stage.
4. Copy the `pyproject.toml` and `poetry.lock` files to the `/tmp` directory.
Because it uses `./poetry.lock*` (ending with a `*`), it won't crash if that file is not available yet.
5. Generate the `requirements.txt` file.
6. This is the final stage, anything here will be preserved in the final container image.
7. Set the current working directory to `/code`.
8. Copy the `requirements.txt` file to the `/code` directory.
This file only lives in the previous Docker stage, that's why we use `--from-requirements-stage` to copy it.
9. Install the package dependencies in the generated `requirements.txt` file.
10. Copy the `app` directory to the `/code` directory.
11. Use the `fastapi run` command to run your app.
/// tip
Click the bubble numbers to see what each line does.
///
A **Docker stage** is a part of a `Dockerfile` that works as a **temporary container image** that is only used to generate some files to be used later.
The first stage will only be used to **install Poetry** and to **generate the `requirements.txt`** with your project dependencies from Poetry's `pyproject.toml` file.
This `requirements.txt` file will be used with `pip` later in the **next stage**.
In the final container image **only the final stage** is preserved. The previous stage(s) will be discarded.
When using Poetry, it would make sense to use **Docker multi-stage builds** because you don't really need to have Poetry and its dependencies installed in the final container image, you **only need** to have the generated `requirements.txt` file to install your project dependencies.
Then in the next (and final) stage you would build the image more or less in the same way as described before.
### Behind a TLS Termination Proxy - Poetry
Again, if you are running your container behind a TLS Termination Proxy (load balancer) like Nginx or Traefik, add the option `--proxy-headers` to the command:
If you are using <ahref="https://github.com/astral-sh/uv"class="external-link"target="_blank">uv</a> to install and manage your project, you can follow their <ahref="https://docs.astral.sh/uv/guides/integration/docker/"class="external-link"target="_blank">uv Docker guide</a>.
## Recap
@ -754,5 +618,3 @@ Using container systems (e.g. with **Docker** and **Kubernetes**) it becomes fai
In most cases, you probably won't want to use any base image, and instead **build a container image from scratch** one based on the official Python Docker image.
Taking care of the **order** of instructions in the `Dockerfile` and the **Docker cache** you can **minimize build times**, to maximize your productivity (and avoid boredom). 😎
In certain special cases, you might want to use the official Docker image for FastAPI. 🤓
@ -67,6 +67,8 @@ There are several alternatives, including:
* <ahref="https://www.uvicorn.org/"class="external-link"target="_blank">Uvicorn</a>: a high performance ASGI server.
* <ahref="https://hypercorn.readthedocs.io/"class="external-link"target="_blank">Hypercorn</a>: an ASGI server compatible with HTTP/2 and Trio among other features.
* <ahref="https://github.com/django/daphne"class="external-link"target="_blank">Daphne</a>: the ASGI server built for Django Channels.
* <ahref="https://github.com/emmett-framework/granian"class="external-link"target="_blank">Granian</a>: A Rust HTTP server for Python applications.
* <ahref="https://unit.nginx.org/howto/fastapi/"class="external-link"target="_blank">NGINX Unit</a>: NGINX Unit is a lightweight and versatile web application runtime.
## Server Machine and Server Program
@ -84,11 +86,9 @@ When you install FastAPI, it comes with a production server, Uvicorn, and you ca
But you can also install an ASGI server manually.
Make sure you create a [virtual environment](../virtual-environments.md){.internal-link target=_blank}, activate it, and then you can install the server:
Make sure you create a [virtual environment](../virtual-environments.md){.internal-link target=_blank}, activate it, and then you can install the server application.
//// tab | Uvicorn
* <ahref="https://www.uvicorn.org/"class="external-link"target="_blank">Uvicorn</a>, a lightning-fast ASGI server, built on uvloop and httptools.
A similar process would apply to any other ASGI server program.
/// tip
By adding the `standard`, Uvicorn will install and use some recommended extra dependencies.
@ -110,32 +112,10 @@ When you install FastAPI with something like `pip install "fastapi[standard]"` y
///
////
//// tab | Hypercorn
* <ahref="https://github.com/pgjones/hypercorn"class="external-link"target="_blank">Hypercorn</a>, an ASGI server also compatible with HTTP/2.
<divclass="termy">
```console
$ pip install hypercorn
---> 100%
```
</div>
...or any other ASGI server.
////
## Run the Server Program
If you installed an ASGI server manually, you would normally need to pass an import string in a special format for it to import your FastAPI application:
Running on 0.0.0.0:8080 over http (CTRL + C to quit)
```
</div>
////
/// note
The command `uvicorn main:app` refers to:
@ -177,9 +141,11 @@ from main import app
///
Each alternative ASGI server program would have a similar command, you can read more in their respective documentation.
/// warning
Uvicorn and others support a `--reload` option that is useful during development.
Uvicorn and other servers support a `--reload` option that is useful during development.
The `--reload` option consumes much more resources, is more unstable, etc.
@ -187,43 +153,6 @@ It helps a lot during **development**, but you **shouldn't** use it in **product
///
## Hypercorn with Trio
Starlette and **FastAPI** are based on <ahref="https://anyio.readthedocs.io/en/stable/"class="external-link"target="_blank">AnyIO</a>, which makes them compatible with both Python's standard library <ahref="https://docs.python.org/3/library/asyncio-task.html"class="external-link"target="_blank">asyncio</a> and <ahref="https://trio.readthedocs.io/en/stable/"class="external-link"target="_blank">Trio</a>.
Nevertheless, Uvicorn is currently only compatible with asyncio, and it normally uses <ahref="https://github.com/MagicStack/uvloop"class="external-link"target="_blank">`uvloop`</a>, the high-performance drop-in replacement for `asyncio`.
But if you want to directly use **Trio**, then you can use **Hypercorn** as it supports it. ✨
### Install Hypercorn with Trio
First you need to install Hypercorn with Trio support:
<divclass="termy">
```console
$ pip install "hypercorn[trio]"
---> 100%
```
</div>
### Run with Trio
Then you can pass the command line option `--worker-class` with the value `trio`:
<divclass="termy">
```console
$ hypercorn main:app --worker-class trio
```
</div>
And that will start Hypercorn with your app using Trio as the backend.
Now you can use Trio internally in your app. Or even better, you can use AnyIO, to keep your code compatible with both Trio and asyncio. 🎉
## Deployment Concepts
These examples run the server program (e.g Uvicorn), starting **a single process**, listening on all the IPs (`0.0.0.0`) on a predefined port (e.g. `80`).
Let's check back those deployment concepts from before:
@ -9,125 +9,92 @@ Let's check back those deployment concepts from before:
* Memory
* Previous steps before starting
Up to this point, with all the tutorials in the docs, you have probably been running a **server program** like Uvicorn, running a **single process**.
Up to this point, with all the tutorials in the docs, you have probably been running a **server program**, for example, using the `fastapi` command, that runs Uvicorn, running a **single process**.
When deploying applications you will probably want to have some **replication of processes** to take advantage of **multiple cores** and to be able to handle more requests.
As you saw in the previous chapter about [Deployment Concepts](concepts.md){.internal-link target=_blank}, there are multiple strategies you can use.
Here I'll show you how to use <ahref="https://gunicorn.org/"class="external-link"target="_blank">**Gunicorn**</a> with **Uvicorn worker processes**.
Here I'll show you how to use **Uvicorn** with **worker processes** using the `fastapi` command or the `uvicorn` command directly.
/// info
If you are using containers, for example with Docker or Kubernetes, I'll tell you more about that in the next chapter: [FastAPI in Containers - Docker](docker.md){.internal-link target=_blank}.
In particular, when running on **Kubernetes** you will probably **not** want to use Gunicorn and instead run **a single Uvicorn process per container**, but I'll tell you about it later in that chapter.
In particular, when running on **Kubernetes** you will probably **not** want to use workers and instead run **a single Uvicorn process per container**, but I'll tell you about it later in that chapter.
///
## Gunicorn with Uvicorn Workers
## Multiple Workers
**Gunicorn** is mainly an application server using the **WSGI standard**. That means that Gunicorn can serve applications like Flask and Django. Gunicorn by itself is not compatible with **FastAPI**, as FastAPI uses the newest **<ahref="https://asgi.readthedocs.io/en/latest/"class="external-link"target="_blank">ASGI standard</a>**.
You can start multiple workers with the `--workers` command line option:
But Gunicorn supports working as a **process manager** and allowing users to tell it which specific **worker process class** to use. Then Gunicorn would start one or more **worker processes** using that class.
//// tab | `fastapi`
And **Uvicorn** has a **Gunicorn-compatible worker class**.
Using that combination, Gunicorn would act as a **process manager**, listening on the **port** and the **IP**. And it would **transmit** the communication to the worker processes running the **Uvicorn class**.
And then the Gunicorn-compatible **Uvicorn worker** class would be in charge of converting the data sent by Gunicorn to the ASGI standard for FastAPI to use it.
## Install Gunicorn and Uvicorn
Make sure you create a [virtual environment](../virtual-environments.md){.internal-link target=_blank}, activate it, and then install `gunicorn`:
If you use the `fastapi` command:
<divclass="termy">
```console
$ pip install "uvicorn[standard]" gunicorn
---> 100%
$ <pre><fontcolor="#4E9A06">fastapi</font> run --workers 4 <ustyle="text-decoration-style:single">main.py</u>
<fontcolor="#3465A4">INFO </font> Using path <fontcolor="#3465A4">main.py</font>
[19499] [INFO] Using worker: uvicorn.workers.UvicornWorker
[19511] [INFO] Booting worker with pid: 19511
[19513] [INFO] Booting worker with pid: 19513
[19514] [INFO] Booting worker with pid: 19514
[19515] [INFO] Booting worker with pid: 19515
[19511] [INFO] Started server process [19511]
[19511] [INFO] Waiting for application startup.
[19511] [INFO] Application startup complete.
[19513] [INFO] Started server process [19513]
[19513] [INFO] Waiting for application startup.
[19513] [INFO] Application startup complete.
[19514] [INFO] Started server process [19514]
[19514] [INFO] Waiting for application startup.
[19514] [INFO] Application startup complete.
[19515] [INFO] Started server process [19515]
[19515] [INFO] Waiting for application startup.
[19515] [INFO] Application startup complete.
```
</div>
Let's see what each of those options mean:
* `main:app`: This is the same syntax used by Uvicorn, `main` means the Python module named "`main`", so, a file `main.py`. And `app` is the name of the variable that is the **FastAPI** application.
* You can imagine that `main:app` is equivalent to a Python `import` statement like:
```Python
from main import app
```
* So, the colon in `main:app` would be equivalent to the Python `import` part in `from main import app`.
* `--workers`: The number of worker processes to use, each will run a Uvicorn worker, in this case, 4 workers.
* `--worker-class`: The Gunicorn-compatible worker class to use in the worker processes.
* Here we pass the class that Gunicorn can import and use with:
```Python
import uvicorn.workers.UvicornWorker
```
* `--bind`: This tells Gunicorn the IP and the port to listen to, using a colon (`:`) to separate the IP and the port.
* If you were running Uvicorn directly, instead of `--bind 0.0.0.0:80` (the Gunicorn option) you would use `--host 0.0.0.0` and `--port 80`.
In the output, you can see that it shows the **PID** (process ID) of each process (it's just a number).
You can see that:
* The Gunicorn **process manager** starts with PID `19499` (in your case it will be a different number).
* Then it starts `Listening at: http://0.0.0.0:80`.
* Then it detects that it has to use the worker class at `uvicorn.workers.UvicornWorker`.
* And then it starts **4 workers**, each with its own PID: `19511`, `19513`, `19514`, and `19515`.
Gunicorn would also take care of managing **dead processes** and **restarting** new ones if needed to keep the number of workers. So that helps in part with the **restart** concept from the list above.
Nevertheless, you would probably also want to have something outside making sure to **restart Gunicorn** if necessary, and also to **run it on startup**, etc.
## Uvicorn with Workers
Uvicorn also has an option to start and run several **worker processes**.
Nevertheless, as of now, Uvicorn's capabilities for handling worker processes are more limited than Gunicorn's. So, if you want to have a process manager at this level (at the Python level), then it might be better to try with Gunicorn as the process manager.
In any case, you would run it like this:
If you prefer to use the `uvicorn` command directly:
The only new option here is `--workers` telling Uvicorn to start 4 worker processes.
You can also see that it shows the **PID** of each process, `27365` for the parent process (this is the **process manager**) and one for each worker process: `27368`, `27369`, `27370`, and `27367`.
## Deployment Concepts
Here you saw how to use **Gunicorn** (or Uvicorn) managing **Uvicorn worker processes** to **parallelize** the execution of the application, take advantage of **multiple cores** in the CPU, and be able to serve **more requests**.
Here you saw how to use multiple **workers** to **parallelize** the execution of the application, take advantage of **multiple cores** in the CPU, and be able to serve **more requests**.
From the list of deployment concepts from above, using workers would mainly help with the **replication** part, and a little bit with the **restarts**, but you still need to take care of the others:
@ -172,13 +141,11 @@ From the list of deployment concepts from above, using workers would mainly help
In the next chapter about [FastAPI in Containers - Docker](docker.md){.internal-link target=_blank} I'll tell some strategies you could use to handle the other **deployment concepts**.
I'll also show you the **official Docker image** that includes **Gunicorn with Uvicorn workers** and some default configurations that can be useful for simple cases.
There I'll also show you how to **build your own image from scratch** to run a single Uvicorn process (without Gunicorn). It is a simple process and is probably what you would want to do when using a distributed container management system like **Kubernetes**.
I'll show you how to **build your own image from scratch** to run a single Uvicorn process. It is a simple process and is probably what you would want to do when using a distributed container management system like **Kubernetes**.
## Recap
You can use **Gunicorn** (or also Uvicorn) as a process manager with Uvicorn workers to take advantage of **multi-core CPUs**, to run **multiple processes in parallel**.
You can use multiple worker processes with the `--workers` CLI option with the `fastapi` or `uvicorn` commands to take advantage of **multi-core CPUs**, to run **multiple processes in parallel**.
You could use these tools and ideas if you are setting up **your own deployment system** while taking care of the other deployment concepts yourself.
that would mean that you would use the versions `0.112.0` or above, but less than `0.113.0`, for example, a version `0.112.2` would still be accepted.
If you use any other tool to manage your installations, like Poetry, Pipenv, or others, they all have a way that you can use to define specific versions for your packages.
If you use any other tool to manage your installations, like `uv`, Poetry, Pipenv, or others, they all have a way that you can use to define specific versions for your packages.
Sebastián Ramírez
7 months ago
GitHub