diff --git a/docs/en/docs/deployment/concepts.md b/docs/en/docs/deployment/concepts.md new file mode 100644 index 000000000..d22b53fe0 --- /dev/null +++ b/docs/en/docs/deployment/concepts.md @@ -0,0 +1,311 @@ +# Deployments Concepts + +When deploying a **FastAPI** application, or actually, any type of web API, there are several concepts that you probably care about, and using them you can find the **most appropriate** way to **deploy your application**. + +Some of the important concepts are: + +* Security - HTTPS +* Running on startup +* Restarts +* Replication (the number of processes running) +* Memory +* Previous steps before starting + +We'll see how they would affect **deployments**. + +In the end, the ultimate objective is to be able to **serve your API clients** in a way that is **secure**, to **avoid disruptions**, and to use the **compute resources** (for example remote servers/virtual machines) as efficiently as possible. πŸš€ + +I'll tell you a bit more about these **concepts** here, and that would hopefully give you the **intuition** you would need to decide how to deploy your API in very different environments, possibly even in **future** ones that don't exist yet. + +By considering these concepts, you will be able to **evaluate and design** the best way to deploy **your own APIs**. + +In the next chapters, I'll give you more **concrete recipes** to deploy FastAPI applications. + +But for now, let's check these important **conceptual ideas**. These concepts also apply for any other type of web API. πŸ’‘ + +## Security - HTTPS + +In the [previous chapter about HTTPS](./https.md){.internal-link target=_blank} we learned about how HTTPS provides encryption for your API. + +We also saw that HTTPS is normally provided by a component **external** to your application server, a **TLS Termination Proxy**. + +And there has to be something in charge of **renewing the HTTPS certificates**, it could be the same component or it could be something different. + +### Example Tools for HTTPS + +Some of the tools you could use as a TLS Termination Proxy are: + +* Traefik + * Automatically handles certificates renewals ✨ +* Caddy + * Automatically handles certificates renewals ✨ +* Nginx + * With an external component like Certbot for certificate renewals +* HAProxy + * With an external component like Certbot for certificate renewals +* Kubernetes with an Ingress Controller like Nginx + * With an external component like cert-manager for certificate renewals +* Handled internally by a cloud provider as part of their services (read below πŸ‘‡) + +Another option is that you could use a **cloud service** that does more of the work including setting up HTTPS. It could have some restrictions or charge you more, etc. But in that case you wouldn't have to set up a TLS Termination Proxy yourself. + +I'll show you some concrete examples in the next chapters. + +--- + +Then the next concepts to consider are all about the program running your actual API (e.g. Uvicorn). + +## Program and Process + +We will talk a lot about the running "**process**", so it's useful to have clarity about what it means, and what's the difference with the word "**program**". + +### What is a Program + +The word **program** is commonly used to describe many things: + +* The **code** that you write, the **Python files**. +* The **file** that can be **executed** by the operating system, for example `python`, `python.exe` or `uvicorn`. +* A particular program while it is **running** on the operating system, using the CPU, and storing things on memory. This is also called a **process**. + +### What is a Process + +The word **process** is normally used in a more specific way, only referring to the thing that is running in the operating system (like in the last point above): + +* A particular program while it is **running** on the operating system. + * This doesn't refer to the file, nor to the code, it refers **specifically** to the thing that is being **executed** and managed by the operating system. +* Any program, any code, **can only do things** when it is being **executed**. So, when there's a **process running**. +* The process can be **terminated** (or "killed") by you, or by the operating system. At that point, it stops running/being executed, and it can **no longer do things**. +* Each application that you have running in your computer has some process behind it, each running program, each window, etc. And there are normally many processes running **at the same time** while a computer is on. +* There can be **multiple processes** of the **same program** running at the same time. + +If you check out the "task manager" or "system monitor" (or similar tools) in your operating system, you will be able to see many of those processes running. + +And, for example, you will probably see that there are multiple processes running the same browser program (Firefox, Chrome, Edge, etc). They normally run one process per tab, plus some other extra processes. + + + +--- + +Now that we know the difference between the terms **process** and **program**, let's continue talking about deployments. + +## Running on Startup + +In most cases, when you create a web API, you want it to be **always running**, uninterrupted, so that your clients can always access it. This is of course, unless you have a specific reason why you want it to run only on certain situations, but most of the time you want it constantly running and **available**. + +### In a Remote Server + +When you set up a remote server (a cloud server, a virtual machine, etc.) the simplest thing you can do is to run Uvicorn (or similar) manually, the same way you do when developing locally. + +And it will work, and will be useful **during development**. + +But if your connection to the server is lost, the **running process** will probably die. + +And if the server is restarted (for example after updates, or migrations from the cloud provider) you probably **won't notice it**. And because of that, you won't even know that you have to restart the process manually. So, your API will just stay dead. 😱 + +### Run Automatically on Startup + +In general, you will probably want the server program (e.g. Uvicorn) to be started automatically on server startup, and without needing any **human intervention**, to have a process always running with your API (e.g. Uvicorn running your FastAPI app). + +### Separate Program + +To achieve this, you will normally have a **separate program** that would make sure your application is run on startup. And in many cases it would also make sure other components or applications are also run, for example a database. + +### Example Tools to Run at Startup + +Some examples of the tools that can do this job are: + +* Docker +* Kubernetes +* Docker Compose +* Docker in Swarm Mode +* Systemd +* Supervisor +* Handled internally by a cloud provider as part of their services +* Others... + +I'll give you more concrete examples in the next chapters. + +## Restarts + +Similar to making sure your application is run on startup, you probably also want to make sure it is **restarted** after failures. + +### We Make Mistakes + +We, as humans, make **mistakes**, all the time. Software almost *always* has **bugs** hidden in different places. πŸ› + +And we as developers keep improving the code as we find those bugs and as we implement new features (possibly adding new bugs too πŸ˜…). + +### Small Errors Automatically Handled + +When building web APIs with FastAPI, if there's an error in our code, FastAPI will normally contain it to the single request that triggered the error. πŸ›‘ + +The client will get a **500 Internal Server Error** for that request, but the application will continue working for the next requests instead of just crashing completely. + +### Bigger Errors - Crashes + +Nevertheless, there might be cases where we write some code that **crashes the entire application** making Uvicorn and Python crash. πŸ’₯ + +And still, you would probably not want the application to stay dead because there was an error in one place, you probably want it to **continue running** at least for the *path operations* that are not broken. + +### Restart After Crash + +But in those cases with really bad errors that crash the running **process**, you would want an external component that is in charge of **restarting** the process, at least a couple of times... + +!!! tip + ...Although if the whole application is just **crashing immediately** it probably doesn't make sense to keep restarting it forever. But in those cases, you will probably notice it during development, or at least right after deployment. + + So let's focus on the main cases, where it could crash entirely in some particular cases **in the future**, and it still makes sense to restart it. + +You would probably want to have the thing in charge of restarting your application as an **external component**, because by that point, the same application with Uvicorn and Python already crashed, so there's nothing in the same code of the same app that could do anything about it. + +### Example Tools to Restart Automatically + +In most cases, the same tool that is used to **run the program on startup** is also used to handle automatic **restarts**. + +For example, this could be handled by: + +* Docker +* Kubernetes +* Docker Compose +* Docker in Swarm Mode +* Systemd +* Supervisor +* Handled internally by a cloud provider as part of their services +* Others... + +## Replication - Processes and Memory + +With a FastAPI application, using a server program like Uvicorn, running it once in **one process** can serve multiple clients concurrently. + +But in many cases you will want to run several worker processes at the same time. + +### Multiple Processes - Workers + +If you have more clients than what a single process can handle (for example if the virtual machine is not too big) and you have **multiple cores** in the server's CPU, then you could have **multiple processes** running with the same application at the same time, and distribute all the requests among them. + +When you run **multiple processes** of the same API program, they are commonly called **workers**. + +### Worker Processes and Ports + +Remember from the docs [About HTTPS](./https.md){.internal-link target=_blank} that only one process can be listening on one combination of port and IP address in a server? + +This is still true. + +So, to be able to have **multiple processes** at the same time, there has to be a **single process listening on a port** that then transmits the communication to each worker process in some way. + +### Memory per Process + +Now, when the program loads things in memory, for example, a machine learning model in a variable, or the contents of a large file in a variable, all that **consumes a bit of the memory (RAM)** of the server. + +And multiple processes normally **don't share any memory**. This means that each running process has its own things, its own variables, its own memory. And if you are consuming a large amount of memory in your code, **each process** will consume an equivalent amount of memory. + +### Server Memory + +For example, if your code loads a Machine Learning model with **1 GB in size**, when you run one process with your API, it will consume at least 1 GB or RAM. And if you start **4 processes** (4 workers), each will consume 1 GB of RAM. So, in total your API will consume **4 GB of RAM**. + +And if your remote server or virtual machine only has 3 GB of RAM, trying to load more than 4 GB of RAM will cause problems. 🚨 + +### Multiple Processes - An Example + +In this example, there's a **Manager Process** that starts and controls two **Worker Processes**. + +This Manager Process would probably be the one listening on the **port** in the IP. And it would transmit all the communication to the worker processes. + +Those worker processes would be the ones running your application, they would perform the main computations to receive a **request** and return a **response**, and they would load anything you put in variables in RAM. + + + +And of course, the same machine would probably have **other processes** running as well, apart from your application. + +An interesting detail is that the percentage of the **CPU used** by each process can **vary** a lot over time, but the **memory (RAM)** normally stays more or less **stable**. + +If you have an API that does a comparable amount of computations every time and you have a lot of clients, then the **CPU utilization** will probably *also be stable* (instead of constantly going up and down quickly). + +### Examples of Replication Tools and Strategies + +There can be several approaches to achieve this, and I'll tell you more about specific strategies in the next chapters, for example when talking about Docker and containers. + +The main constraint to consider is that there has to be a **single** component handling the **port** in the **public IP**. And then it has to have a way to **transmit** the communication to the replicated **processes/workers**. + +Here are some possible combinations and strategies: + +* **Gunicorn** managing **Uvicorn workers** + * Gunicorn would be the **process manager** listening on the **IP** and **port**, the replication would be by having **multiple Uvicorn worker processes** +* **Uvicorn** managing **Uvicorn workers** + * One Uvicorn **process manager** would listen on the **IP** and **port**, and it would start **multiple Uvicorn worker processes** +* **Kubernetes** and other distributed **container systems** + * Something in the **Kubernetes** layer would listen on the **IP** and **port**. The replication would be by having **multiple containers**, each with **one Uvicorn process** running +* **Cloud services** that handle this for your + * The cloud service will probably **handle replication for you**. It would possibly let you define **a process to run**, or a **container image** to use, in any case, it would most probably be **a single Uvicorn process**, and the cloud service would be in charge of replicating it. + +!!! tip + Don't worry if some of these items about **containers**, Docker, or Kubernetes don't make a lot of sense yet. + + I'll tell you more about container images, Docker, Kubernetes, etc. in a future chapter: [FastAPI in Containers - Docker](./docker.md){.internal-link target=_blank}. + +## Previous Steps Before Starting + +There are many cases where you want to perform some steps **before starting** your application. + +For example, you might want to run **database migrations**. + +But in most cases, you will want to perform these steps only **once**. + +So, you will want to have a **single process** to perform those **previous steps**, before starting the application. + +And you will have to make sure that it's a single process running those previous steps *even* if afterwards you start **multiple processes** (multiple workers) for the application itself. If those steps were run by **multiple processes**, they would **duplicate** the work by running it on **parallel**, and if the steps were something delicate like a database migration, they could cause conflicts with each other. + +Of course, there are some cases where there's no problem in running the previous steps multiple times, in that case it's a lot easier to handle. + +!!! tip + Also have in mind that depending on your setup, in some cases you **might not even need any previous steps** before starting your application. + + In that case, you wouldn't have to worry about any of this. 🀷 + +### Examples of Previous Steps Strategies + +This will **depend heavily** on the way you **deploy your system**, and it would probably be connected to the way you start programs, handling restarts, etc. + +Here are some possible ideas: + +* An "Init Container" in Kubernetes that runs before your app container +* A bash script that runs the previous steps and then starts your application + * You would still need a way to start/restart *that* bash script, detect errors, etc. + +!!! tip + I'll give you more concrete examples for doing this with containers in a future chapter: [FastAPI in Containers - Docker](./docker.md){.internal-link target=_blank}. + +## Resource Utilization + +Your server(s) is (are) a **resource**, you can consume or **utilize**, with your programs, the computation time on the CPUs, and the RAM memory available. + +How much resources do you want to be consuming/utilizing? It might be easy to think "not much", but in reality, you will probably want to consume **as much as possible without crashing**. + +If you are paying for 3 servers but you are using only a little bit of their RAM and CPU, you are probably **wasting money** πŸ’Έ, and probably **wasting server electric power** 🌎, etc. + +In that case, it could be better to have only 2 servers and use a higher percentage of their resources (CPU, memory, disk, network bandwidth, etc). + +On the other hand, if you have 2 servers and you are using **100% of their CPU and RAM**, at some point one process will ask for more memory, and the server will have to use the disk as "memory" (which can be thousands of times slower), or even **crash**. Or one process might need to do some computation and would have to wait until the CPU is free again. + +In this case, it would be better to get **one extra server** and run some processes on it so that they all have **enough RAM and CPU time**. + +There's also the chance that for some reason you have a **spike** of usage of your API. Maybe it went viral, or maybe some other services or bots start using it. And you might want to have extra resources to be safe in those cases. + +You could put an **arbitrary number** to target, for example something **between 50% to 90%** of resource utilization. The point is that those are probably the main things you will want to measure and use to tweak your deployments. + +You can use simple tools like `htop` to see the CPU and RAM used in your server, or the amount used by each process. Or you can use more complex monitoring tools, maybe distributed across servers, etc. + +## Recap + +You have been reading here some of the main concepts that you would probably need to have in mind when deciding how to deploy your application: + +* Security - HTTPS +* Running on startup +* Restarts +* Replication (the number of processes running) +* Memory +* Previous steps before starting + +Understanding these ideas and how to apply them should give you the intuition necessary to take any decisions when configuring and tweaking your deployments. πŸ€“ + +In the next sections I'll give you more concrete examples of possible strategies you can follow. πŸš€ diff --git a/docs/en/docs/deployment/deta.md b/docs/en/docs/deployment/deta.md index fd4b30a3c..b0d1cb928 100644 --- a/docs/en/docs/deployment/deta.md +++ b/docs/en/docs/deployment/deta.md @@ -238,3 +238,21 @@ You can also edit them and re-play them. At some point you will probably want to store some data for your app in a way that persists through time. For that you can use Deta Base, it also has a generous **free tier**. You can also read more in the Deta Docs. + +## Deployment Concepts + +Coming back to the concepts we discussed in [Deployments Concepts](./concepts.md){.internal-link target=_blank}, here's how each of them would be handled with Deta: + +* **HTTPS**: Handled by Deta, they will give you a subdomain and handle HTTPS automatically. +* **Running on startup**: Handled by Deta, as part of their service. +* **Restarts**: Handled by Deta, as part of their service. +* **Replication**: Handled by Deta, as part of their service. +* **Memory**: Limit predefined by Deta, you could contact them to increase it. +* **Previous steps before starting**: Not directly supported, you could make it work with their Cron system or additional scripts. + +!!! note + Deta is designed to make it easy (and free) to deploy simple applications quickly. + + It can simplify a lot several use cases, but at the same time it doesn't support others, like using external databases (apart from Deta's own NoSQL database system), custom virtual machines, etc. + + You can read more details in the Deta docs to see if it's the right choice for you. diff --git a/docs/en/docs/deployment/docker.md b/docs/en/docs/deployment/docker.md index e32a2969d..e25401f20 100644 --- a/docs/en/docs/deployment/docker.md +++ b/docs/en/docs/deployment/docker.md @@ -1,67 +1,144 @@ -# Deploy with Docker +# FastAPI in Containers - Docker -In this section you'll see instructions and links to guides to know how to: +When deploying FastAPI applications a common approach is to build a **Linux container image**. It's normally done using **Docker**. And then you can deploy that container image in one of different possible ways. -* Make your **FastAPI** application a Docker image/container with maximum performance. In about **5 min**. -* (Optionally) understand what you, as a developer, need to know about HTTPS. -* Set up a Docker Swarm mode cluster with automatic HTTPS, even on a simple $5 USD/month server. In about **20 min**. -* Generate and deploy a full **FastAPI** application, using your Docker Swarm cluster, with HTTPS, etc. In about **10 min**. +Using Linux containers has several advantages including **security**, **replicability**, **simplicity**, and others. -You can use **Docker** for deployment. It has several advantages like security, replicability, development simplicity, etc. - -If you are using Docker, you can use the official Docker image: +!!! tip + In a hurry and already know this stuff? Jump to the [`Dockerfile` below πŸ‘‡](#build-a-docker-image-for-fastapi). -## tiangolo/uvicorn-gunicorn-fastapi +
+Dockerfile Preview πŸ‘€ -This image has an "auto-tuning" mechanism included, so that you can just add your code and get very high performance automatically. And without making sacrifices. +```Dockerfile +FROM python:3.9 -But you can still change and update all the configurations with environment variables or configuration files. +WORKDIR /code -!!! tip - To see all the configurations and options, go to the Docker image page: tiangolo/uvicorn-gunicorn-fastapi. +COPY ./requirements.txt /code/requirements.txt -## Create a `Dockerfile` +RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt -* Go to your project directory. -* Create a `Dockerfile` with: +COPY ./app /code/app -```Dockerfile -FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7 +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"] -COPY ./app /app +# If running behind a proxy like Nginx or Traefik add --proxy-headers +# CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80", "--proxy-headers"] ``` -### Bigger Applications +
-If you followed the section about creating [Bigger Applications with Multiple Files](../tutorial/bigger-applications.md){.internal-link target=_blank}, your `Dockerfile` might instead look like: +## What is a Container -```Dockerfile -FROM tiangolo/uvicorn-gunicorn-fastapi:python3.7 +Containers (mainly Linux containers) are a very **lightweight** way to package applications including all their dependencies and necessary files while keeping them isolated from other containers (other applications or components) in the same system. -COPY ./app /app/app -``` +Linux containers run using the same Linux kernel of the host (machine, virtual machine, cloud server, etc). This just means that they are very lightweight (compared to full virtual machines emulating an entire operating system). -### Raspberry Pi and other architectures +This way, containers consume **little resources**, an amount comparable to running the processes directly (a virtual machine would consume much more). -If you are running Docker in a Raspberry Pi (that has an ARM processor) or any other architecture, you can create a `Dockerfile` from scratch, based on a Python base image (that is multi-architecture) and use Uvicorn alone. +Containers also have their own **isolated** running processes (commonly just one process), file system, and network, simplifying deployment, security, development, etc. -In this case, your `Dockerfile` could look like: +## What is a Container Image -```Dockerfile -FROM python:3.7 +A **container** is run from a **container image**. -RUN pip install fastapi uvicorn +A container image is a **static** version of all the files, environment variables, and the default command/program that should be present in a container. **Static** here means that the container **image** is not running, it's not being executed, it's only the packaged files and metadata. -EXPOSE 80 +In contrast to a "**container image**" that is the stored static contents, a "**container**" normally refers to the running instance, the thing that is being **executed**. -COPY ./app /app +When the **container** is started and running (started from a **container image**) it could create or change files, environment variables, etc. Those changes will exist only in that container, but would not persist in the underlying container image (would not be saved to disk). -CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"] +A container image is comparable to the **program** file and contents, e.g. `python` and some file `main.py`. + +And the **container** itself (in contrast to the **container image**) is the actual running instance of the image, comparable to a **process**. In fact, a container is running only when it has a **process running** (and normally it's only a single process). The container stops when there's no process running in it. + +## Container Images + +Docker has been one of the main tools to create and manage **container images** and **containers**. + +And there's a public Docker Hub with pre-made **official container images** for many tools, environments, databases, and applications. + +For example, there's an official Python Image. + +And there are many other images for different things like databases, for example for: + +* PostgreSQL +* MySQL +* MongoDB +* Redis, etc. + +By using a pre-made container image it's very easy to **combine** and use different tools. For example, to try out a new database. In most cases you can use the **official images**, and just configure them with environment variables. + +That way, in many cases you can learn about containers and Docker and re-use that knowledge with many different tools and components. + +So, you would run **multiple containers** with different things, like a database, a Python application, a web server with a React frontend application, and connect them together via their internal network. + +All the container management systems (like Docker or Kubernetes) have these networking features integrated in them. + +## Containers and Processes + +A **container image** normally includes in its metadata the default program or command that should be run when the **container** is started and the parameters to be passed to that program. Very similar to what would be if it was in the command line. + +When a **container** is started, it will run that command/program (although you can override it and make it run a different command/program). + +A container is running as long as the **main process** (command or program) is running. + +A container normally has a **single process**, but it's also possible to start subprocesses from the main process, and that way have **multiple processes** in the same container. + +But it's not possible to have a running container without **at least one running process**. If the main process stops, the container stops. + +## Build a Docker Image for FastAPI + +Okay, let's build something now! πŸš€ + +I'll show you how to build a **Docker image** for FastAPI **from scratch**, based on the **official Python** image. + +This is what you would want to do in **most cases**, for example: + +* Using **Kubernetes** or similar tools +* When running on a **Raspberry Pi** +* Using a cloud service that would run a container image for you, etc. + +### Package Requirements + +You would normally have the **package requirements** for your application in some file. + +It would depend mainly on the tool you use to **install** those requirements. + +The most common way to do it is to have a file `requirements.txt` with the package names and their versions, one per line. + +You would of course use the same ideas you read in [About FastAPI versions](./versions.md){.internal-link target=_blank} to set the ranges of versions. + +For example, your `requirements.txt` could look like: + +``` +fastapi>=0.68.0,<0.69.0 +pydantic>=1.8.0,<2.0.0 +uvicorn>=0.15.0,<0.16.0 +``` + +And you would normally install those package dependencies with `pip`, for example: + +
+ +```console +$ pip install -r requirements.txt +---> 100% +Successfully installed fastapi pydantic uvicorn ``` -## Create the **FastAPI** Code +
+ +!!! info + There are other formats and tools to define and install package dependencies. + + I'll show you an example using Poetry later in a section below. πŸ‘‡ + +### Create the **FastAPI** Code * Create an `app` directory and enter in it. +* Create an empty file `__init__.py`. * Create a `main.py` file with: ```Python @@ -82,16 +159,126 @@ def read_item(item_id: int, q: Optional[str] = None): return {"item_id": item_id, "q": q} ``` -* You should now have a directory structure like: +### Dockerfile + +Now in the same project directory create a file `Dockerfile` with: + +```{ .dockerfile .annotate } +# (1) +FROM python:3.9 + +# (2) +WORKDIR /code + +# (3) +COPY ./requirements.txt /code/requirements.txt + +# (4) +RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt + +# (5) +COPY ./app /code/app + +# (6) +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"] +``` + +1. Start from the official Python base image. + +2. Set the current working directory to `/code`. + + This is where we'll put the `requirements.txt` file and the `app` directory. + +3. Copy the file with the requirements to the `/code` directory. + + Copy **only** the file with the requirements first, not the rest of the code. + + As this file **doesn't change often**, Docker will detect it and use the **cache** for this step, enabling the cache for the next step too. + +4. Install the package dependencies in the requirements file. + + The `--no-cache-dir` option tells `pip` to not save the downloaded packages locally, as that is only if `pip` was going to be run again to install the same packages, but that's not the case when working with containers. + + !!! note + The `--no-cache-dir` is only related to `pip`, it has nothing to do with Docker or containers. + + The `--upgrade` option tells `pip` to upgrade the packages if they are already installed. + + Because the previous step copying the file could be detected by the **Docker cache**, this step will also **use the Docker cache** when available. + + Using the cache in this step will **save** you a lot of **time** when building the image again and again during development, instead of **downloading and installing** all the dependencies **every time**. + +5. Copy the `./app` directory inside the `/code` directory. + + As this has all the code which is what **changes most frequently** the Docker **cache** won't be used for this or any **following steps** easily. + + So, it's important to put this **near the end** of the `Dockerfile`, to optimize the container image build times. + +6. Set the **command** to run the `uvicorn` server. + + `CMD` takes a list of strings, each of this strings is what you would type in the command line separated by spaces. + + This command will be run from the **current working directory**, the same `/code` directory you set above with `WORKDIR /code`. + + Because the program will be started at `/code` and inside of it is the directory `./app` with your code, **Uvicorn** will be able to see and **import** `app` from `app.main`. + +!!! tip + Review what each line does by clicking each number bubble in the code. πŸ‘† + +You should now have a directory structure like: ``` . β”œβ”€β”€ app +β”‚Β Β  β”œβ”€β”€ __init__.py β”‚ └── main.py -└── Dockerfile +β”œβ”€β”€ Dockerfile +└── requirements.txt +``` + +#### Behind a TLS Termination Proxy + +If you are running your container behind a TLS Termination Proxy (load balancer) like Nginx or Traefik, add the option `--proxy-headers`, this will tell Uvicorn to trust the headers sent by that proxy telling it that the application is running behind HTTPS, etc. + +```Dockerfile +CMD ["uvicorn", "app.main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "80"] +``` + +#### Docker Cache + +There's an important trick in this `Dockerfile`, we first copy the **file with the dependencies alone**, not the rest of the code. Let me tell you why is that. + +```Dockerfile +COPY ./requirements.txt /code/requirements.txt +``` + +Docker and other tools **build** these container images **incrementally**, adding **one layer on top of the other**, starting from the top of the `Dockerfile` and adding any files created by each of the instructions of the `Dockerfile`. + +Docker and similar tools also use an **internal cache** when building the image, if a file hasn't changed since the last time building the container image, then it will **re-use the same layer** created the last time, instead of copying the file again and creating a new layer from scratch. + +Just avoiding the copy of files doesn't necessarily improve things too much, but because it used the cache for that step, it can **use the cache for the next step**. For example, it could use the cache for the instruction that installs dependencies with: + +```Dockerfile +RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt ``` -## Build the Docker image +The file with the package requirements **won't change frequently**. So, by copying only that file, Docker will be able to **use the cache** for that step. + +And then, Docker will be able to **use the cache for the next step** that downloads and install those dependencies. And here's where we **save a lot of time**. ✨ ...and avoid boredom waiting. πŸ˜ͺπŸ˜† + +Downloading and installing the package dependencies **could take minutes**, but using the **cache** would **take seconds** at most. + +And as you would be building the container image again and again during development to check that your code changes are working, there's a lot of accumulated time this would save. + +Then, near the end of the `Dockerfile`, we copy all the code. As this is what **changes most frequently**, we put it near the end, because almost always, anything after this step will not be able to use the cache. + +```Dockerfile +COPY ./app /code/app +``` + +### Build the Docker Image + +Now that all the files are in place, let's build the container image. * Go to the project directory (in where your `Dockerfile` is, containing your `app` directory). * Build your FastAPI image: @@ -106,7 +293,12 @@ $ docker build -t myimage . -## Start the Docker container +!!! tip + Notice the `.` at the end, it's equivalent to `./`, it tells Docker the directory to use to build the container image. + + In this case, it's the same current directory (`.`). + +### Start the Docker Container * Run a container based on your image: @@ -118,8 +310,6 @@ $ docker run -d --name mycontainer -p 80:80 myimage -Now you have an optimized FastAPI server in a Docker container. Auto-tuned for your current server (and number of CPU cores). - ## Check it You should be able to check it in your Docker container's URL, for example: http://192.168.99.100/items/5?q=somequery or http://127.0.0.1/items/5?q=somequery (or equivalent, using your Docker host). @@ -146,34 +336,363 @@ You will see the alternative automatic documentation (provided by Traefik is a high performance reverse proxy / load balancer. It can do the "TLS Termination Proxy" job (apart from other features). +* HTTPS +* Running on startup +* Restarts +* Replication (the number of processes running) +* Memory +* Previous steps before starting -It has integration with Let's Encrypt. So, it can handle all the HTTPS parts, including certificate acquisition and renewal. +## HTTPS -It also has integrations with Docker. So, you can declare your domains in each application configurations and have it read those configurations, generate the HTTPS certificates and serve HTTPS to your application automatically, without requiring any change in its configuration. +If we focus just on the **container image** for a FastAPI application (and later the running **container**), HTTPS normally would be handled **externally** by another tool. + +It could be another container, for example with Traefik, handling **HTTPS** and **automatic** acquisition of **certificates**. + +!!! tip + Traefik has integrations with Docker, Kubernetes, and others, so it's very easy to set up and configure HTTPS for your containers with it. + +Alternatively, HTTPS could be handled by a cloud provider as one of their services (while still running the application in a container). + +## Running on Startup and Restarts + +There is normally another tool in charge of **starting and running** your container. + +It could be **Docker** directly, **Docker Compose**, **Kubernetes**, a **cloud service**, etc. + +In most (or all) cases, there's a simple option to enable running the container on startup and enabling restarts on failures. For example, in Docker, it's the command line option `--restart`. + +Without using containers, making applications run on startup and with restarts can be cumbersome and difficult. But when **working with containers** in most cases that functionality is included by default. ✨ + +## Replication - Number of Processes + +If you have a cluster of machines with **Kubernetes**, Docker Swarm Mode, Nomad, or other similar complex system to manage distributed containers on multiple machines, then you will probably want to **handle replication** at the **cluster level** instead of using a **process manager** (like Gunicorn with workers) in each container. + +One of those distributed container management systems like Kubernetes normally has some integrated way of handling **replication of containers** while still supporting **load balancing** for the incoming requests. All at the **cluster level**. + +In those cases, you would probably want to build a **Docker image from scratch** as [explained above](#dockerfile), installing your dependencies, and running **a single Uvicorn process** instead of running something like Gunicorn with Uvicorn workers. + +### Load Balancer + +When using containers, you would normally have some component **listening on the main port**. It could possibly be another container that is also a **TLS Termination Proxy** to handle **HTTPS** or some similar tool. + +As this component would take the **load** of requests and distribute that among the workers in a (hopefully) **balanced** way, it is also commonly called a **Load Balancer**. + +!!! tip + The same **TLS Termination Proxy** component used for HTTPS would probably also be a **Load Balancer**. + +And when working with containers, the same system you use to start and manage them would already have internal tools to transmit the **network communication** (e.g. HTTP requests) from that **load balancer** (that could also be a **TLS Termination Proxy**) to the container(s) with your app. + +### One Load Balancer - Multiple Worker Containers + +When working with **Kubernetes** or similar distributed container management systems, using their internal networking mechanisms would allow the single **load balancer** that is listening on the main **port** to transmit communication (requests) to possibly **multiple containers** running your app. + +Each of these containers running your app would normally have **just one process** (e.g. a Uvicorn process running your FastAPI application). They would all be **identical containers**, running the same thing, but each with its own process, memory, etc. That way you would take advantage of **parallelization** in **different cores** of the CPU, or even in **different machines**. + +And the distributed container system with the **load balancer** would **distribute the requests** to each one of the containers with your app **in turns**. So, each request could be handled by one of the multiple **replicated containers** running your app. + +And normally this **load balancer** would be able to handle requests that go to *other* apps in your cluster (e.g. to a different domain, or under a different URL path prefix), and would transmit that communication to the right containers for *that other* application running in your cluster. + +### One Process per Container + +In this type of scenario, you probably would want to have **a single (Uvicorn) process per container**, as you would already be handling replication at the cluster level. + +So, in this case, you **would not** want to have a process manager like Gunicorn with Uvicorn workers, or Uvicorn using its own Uvicorn workers. You would want to have just a **single Uvicorn process** per container (but probably multiple containers). + +Having another process manager inside the container (as would be with Gunicorn or Uvicorn managing Uvicorn workers) would only add **unnecessary complexity** that you are most probably already taking care of with your cluster system. + +### Containers with Multiple Processes and Special Cases + +Of course, there are **special cases** where you could want to have **a container** with a **Gunicorn process manager** starting several **Uvicorn worker processes** inside. + +In those cases, you can use the **official Docker image** that includes **Gunicorn** as a process manager running multiple **Uvicorn worker processes**, and some default settings to adjust the number of workers based on the current CPU cores automatically. I'll tell you more about it below in [Official Docker Image with Gunicorn - Uvicorn](#official-docker-image-with-gunicorn-uvicorn). + +Here are some examples of when that could make sense: + +#### A Simple App + +You could want a process manager in the container if your application is **simple enough** that you don't need (at least not yet) to fine-tune the number of processes too much, and you can just use an automated default (with the official Docker image), and you are running it on a **single server**, not a cluster. + +#### Docker Compose + +You could be deploying to a **single server** (not a cluster) with **Docker Compose**, so you wouldn't have an easy way to manage replication of containers (with Docker Compose) while preserving the shared network and **load balancing**. + +Then you could want to have **a single container** with a **process manager** starting **several worker processes** inside. + +#### Prometheus and Other Reasons + +You could also have **other reasons** that would make it easier to have a **single container** with **multiple processes** instead of having **multiple containers** with **a single process** in each of them. + +For example (depending on your setup) you could have some tool like a Prometheus exporter in the same container that should have access to **each of the requests** that come. + +In this case, if you had **multiple containers**, by default, when Prometheus came to **read the metrics**, it would get the ones for **a single container each time** (for the container that handled that particular request), instead of getting the **accumulated metrics** for all the replicated containers. + +Then, in that case, it could be simpler to have **one container** with **multiple processes**, and a local tool (e.g. a Prometheus exporter) on the same container collecting Prometheus metrics for all the internal processes and exposing those metrics on that single container. --- -With this information and tools, continue with the next section to combine everything. +The main point is, **none** of these are **rules written in stone** that you have to blindly follow. You can use these ideas to **evaluate your own use case** and decide what is the best approach for your system, checking out how to manage the concepts of: + +* Security - HTTPS +* Running on startup +* Restarts +* Replication (the number of processes running) +* Memory +* Previous steps before starting + +## Memory + +If you run **a single process per container** you will have a more or less well defined, stable, and limited amount of memory consumed by each of of those containers (more than one if they are replicated). + +And then you can set those same memory limits and requirements in your configurations for your container management system (for example in **Kubernetes**). That way it will be able to **replicate the containers** in the **available machines** taking into account the amount of memory needed by them, and the amount available in the machines in the cluster. + +If your application is **simple**, this will probably **not be a problem**, and you might not need to specify hard memory limits. But if you are **using a lot of memory** (for example with **machine learning** models), you should check how much memory you are consuming and adjust the **number of containers** that runs in **each machine** (and maybe add more machines to your cluster). + +If you run **multiple processes per container** (for example with the official Docker image) you will have to make sure that the number of processes started doesn't **consume more memory** than what is available. + +## Previous Steps Before Starting and Containers + +If you are using containers (e.g. Docker, Kubernetes), then there are two main approaches you can use. + +### Multiple Containers + +If you have **multiple containers**, probably each one running a **single process** (for example, in a **Kubernetes** cluster), then you would probably want to have a **separate container** doing the work of the **previous steps** in a single container, running a single process, **before** running the replicated worker containers. + +!!! info + If you are using Kubernetes, this would probably be an Init Container. + +If in your use case there's no problem in running those previous steps **multiple times in parallel** (for example if you are not running database migrations, but just checking if the database is ready yet), then you could also just put them in each container right before starting the main process. + +### Single Container + +If you have a simple setup, with a **single container** that then starts multiple **worker processes** (or also just one process), then you could run those previous steps in the same container, right before starting the process with the app. The official Docker image supports this internally. + +## Official Docker Image with Gunicorn - Uvicorn + +There is an official Docker image that includes Gunicorn running with Uvicorn workers, as detailed in a previous chapter: [Server Workers - Gunicorn with Uvicorn](./server-workers.md){.internal-link target=_blank}. + +This image would be useful mainly in the situations described above in: [Containers with Multiple Processes and Special Cases](#containers-with-multiple-processes-and-special-cases). + +* tiangolo/uvicorn-gunicorn-fastapi. + +!!! warning + There's a high chance that you **don't** need this base image or any other similar one, and would be better off by building the image from scratch as [described above in: Build a Docker Image for FastAPI](#build-a-docker-image-for-fastapi). + +This image has an **auto-tuning** mechanism included to set the **number of worker processes** based on the CPU cores available. + +It has **sensible defaults**, but you can still change and update all the configurations with **environment variables** or configuration files. + +It also supports running **previous steps before starting** with a script. + +!!! tip + To see all the configurations and options, go to the Docker image page: tiangolo/uvicorn-gunicorn-fastapi. + +### Number of Processes on the Official Docker Image + +The **number of processes** on this image is **computed automatically** from the CPU **cores** available. -## Docker Swarm mode cluster with Traefik and HTTPS +This means that it will try to **squeeze** as much **performance** from the CPU as possible. -You can have a Docker Swarm mode cluster set up in minutes (about 20 min) with a main Traefik handling HTTPS (including certificate acquisition and renewal). +You can also adjust it with the configurations using **environment variables**, etc. -By using Docker Swarm mode, you can start with a "cluster" of a single machine (it can even be a $5 USD / month server) and then you can grow as much as you need adding more servers. +But it also means that as the number of processes depends on the CPU the container is running, the **amount of memory consumed** will also depend on that. -To set up a Docker Swarm Mode cluster with Traefik and HTTPS handling, follow this guide: +So, if your application consumes a lot of memory (for example with machine learning models), and your server has a lot of CPU cores **but little memory**, then your container could end up trying to use more memory than what is available, and degrading performance a lot (or even crashing). 🚨 + +### Create a `Dockerfile` + +Here's how you would create a `Dockerfile` based on this image: + +```Dockerfile +FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9 + +COPY ./requirements.txt /app/requirements.txt + +RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt + +COPY ./app /app +``` + +### Bigger Applications + +If you followed the section about creating [Bigger Applications with Multiple Files](../tutorial/bigger-applications.md){.internal-link target=_blank}, your `Dockerfile` might instead look like: + +```Dockerfile hl_lines="7" +FROM tiangolo/uvicorn-gunicorn-fastapi:python3.9 + +COPY ./requirements.txt /app/requirements.txt + +RUN pip install --no-cache-dir --upgrade -r /app/requirements.txt + +COPY ./app /app/app +``` + +### When to Use + +You should probably **not** use this official base image (or any other similar one) if you are using **Kubernetes** (or others) and you are already setting **replication** at the cluster level, with multiple **containers**. In those cases, you are better off **building an image from scratch** as described above: [Build a Docker Image for FastAPI](#build-a-docker-image-for-fastapi). + +This image would be useful mainly in the special cases described above in [Containers with Multiple Processes and Special Cases](#containers-with-multiple-processes-and-special-cases). For example, if your application is **simple enough** that setting a default number of processes based on the CPU works well, you don't want to bother with manually configuring the replication at the cluster level, and you are not running more than one container with your app. Or if you are deploying with **Docker Compose**, running on a single server, etc. + +## Deploy the Container Image + +After having a Container (Docker) Image there are several ways to deploy it. + +For example: + +* With **Docker Compose** in a single server +* With a **Kubernetes** cluster +* With a Docker Swarm Mode cluster +* With another tool like Nomad +* With a cloud service that takes your container image and deploys it + +## Docker Image with Poetry + +If you use Poetry to manage your project's dependencies, you could use Docker multi-stage building: + +```{ .dockerfile .annotate } +# (1) +FROM python:3.9 as requirements-stage + +# (2) +WORKDIR /tmp + +# (3) +RUN pip install poetry + +# (4) +COPY ./pyproject.toml ./poetry.lock* /tmp/ + +# (5) +RUN poetry export -f requirements.txt --output requirements.txt --without-hashes + +# (6) +FROM python:3.9 + +# (7) +WORKDIR /code + +# (8) +COPY --from=requirements-stage /tmp/requirements.txt /code/requirements.txt + +# (9) +RUN pip install --no-cache-dir --upgrade -r /code/requirements.txt + +# (10) +COPY ./app /code/app + +# (11) +CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "80"] +``` + +1. This is the first stage, it is named `requirements-stage`. + +2. Set `/tmp` as the current working directory. + + Here's where we will generate the file `requirements.txt` + +3. Install Poetry in this Docker stage. + +4. Copy the `pyproject.toml` and `poetry.lock` files to the `/tmp` directory. + + Because it uses `./poetry.lock*` (ending with a `*`), it won't crash if that file is not available yet. + +5. Generate the `requirements.txt` file. + +6. This is the final stage, anything here will be preserved in the final container image. + +7. Set the current working directory to `/code`. + +8. Copy the `requirements.txt` file to the `/code` directory. + + This file only lives in the previous Docker stage, that's why we use `--from-requirements-stage` to copy it. + +9. Install the package dependencies in the generated `requirements.txt` file. + +10. Copy the `app` directory to the `/code` directory. + +11. Run the `uvicorn` command, telling it to use the `app` object imported from `app.main`. + +!!! tip + Click the bubble numbers to see what each line does. + +A **Docker stage** is a part of a `Dockerfile` that works as a **temporary container image** that is only used to generate some files to be used later. + +The first stage will only be used to **install Poetry** and to **generate the `requirements.txt`** with your project dependencies from Poetry's `pyproject.toml` file. + +This `requirements.txt` file will be used with `pip` later in the **next stage**. + +In the final container image **only the final stage** is preserved. The previous stage(s) will be discarded. + +When using Poetry, it would make sense to use **Docker multi-stage builds** because you don't really need to have Poetry and its dependencies installed in the final container image, you **only need** to have the generated `requirements.txt` file to install your project dependencies. + +Then in the next (and final) stage you would build the image more or less in the same way as described before. + +### Behind a TLS Termination Proxy - Poetry + +Again, if you are running your container behind a TLS Termination Proxy (load balancer) like Nginx or Traefik, add the option `--proxy-headers` to the command: + +```Dockerfile +CMD ["uvicorn", "app.main:app", "--proxy-headers", "--host", "0.0.0.0", "--port", "80"] +``` -### Docker Swarm Mode and Traefik for an HTTPS cluster +## Recap -### Deploy a FastAPI application +Using container systems (e.g. with **Docker** and **Kubernetes**) it becomes fairly straightforward to handle all the **deployment concepts**: -The easiest way to set everything up, would be using the [**FastAPI** Project Generators](../project-generation.md){.internal-link target=_blank}. +* HTTPS +* Running on startup +* Restarts +* Replication (the number of processes running) +* Memory +* Previous steps before starting -It is designed to be integrated with this Docker Swarm cluster with Traefik and HTTPS described above. +In most cases, you probably won't want to use any base image, and instead **build a container image from scratch** one based on the official Python Docker image. -You can generate a project in about 2 min. +Taking care of the **order** of instructions in the `Dockerfile` and the **Docker cache** you can **minimize build times**, to maximize your productivity (and avoid boredom). 😎 -The generated project has instructions to deploy it, doing it takes another 2 min. +In certain special cases, you might want to use the official Docker image for FastAPI. πŸ€“ diff --git a/docs/en/docs/deployment/index.md b/docs/en/docs/deployment/index.md index d898cfefc..d1941ad93 100644 --- a/docs/en/docs/deployment/index.md +++ b/docs/en/docs/deployment/index.md @@ -2,6 +2,20 @@ Deploying a **FastAPI** application is relatively easy. +## What Does Deployment Mean + +To **deploy** an application means to perform the necessary steps to make it **available to the users**. + +For a **web API**, it normally involves putting it in a **remote machine**, with a **server program** that provides good performance, stability, etc, so that your **users** can **access** the application efficiently and without interruptions or problems. + +This is in contrast to the the **development** stages, where you are constantly changing the code, breaking it and fixing it, stopping and restarting the development server, etc. + +## Deployment Strategies + There are several ways to do it depending on your specific use case and the tools that you use. -You will see more details to have in mind and some of the techniques to do it in the next sections. +You could **deploy a server** yourself using a combination of tools, you could use a **cloud service** that does part of the work for you, or other possible options. + +I will show you some of the main concepts you should probably have in mind when deploying a **FastAPI** application (although most of it applies to any other type of web application). + +You will see more details to have in mind and some of the techniques to do it in the next sections. ✨ diff --git a/docs/en/docs/deployment/manually.md b/docs/en/docs/deployment/manually.md index daa31a304..80a7df7e6 100644 --- a/docs/en/docs/deployment/manually.md +++ b/docs/en/docs/deployment/manually.md @@ -1,8 +1,26 @@ -# Deploy manually +# Run a Server Manually - Uvicorn -You can deploy **FastAPI** manually as well. +The main thing you need to run a **FastAPI** application in a remote server machine is an ASGI server program like **Uvicorn**. -You just need to install an ASGI compatible server like: +There are 3 main alternatives: + +* Uvicorn: a high performance ASGI server. +* Hypercorn: an ASGI server compatible with HTTP/2 and Trio among other features. +* Daphne: the ASGI server built for Django Channels. + +## Server Machine and Server Program + +There's a small detail about names to have in mind. πŸ’‘ + +The word "**server**" is commonly used to refer to both the remote/cloud computer (the physical or virtual machine) and also the program that is running on that machine (e.g. Uvicorn). + +Just have that in mind when you read "server" in general, it could refer to one of those two things. + +When referring to the remote machine, it's common to call it **server**, but also **machine**, **VM** (virtual machine), **node**. Those all refer to some type of remote machine, normally running Linux, where you run programs. + +## Install the Server Program + +You can install an ASGI compatible server with: === "Uvicorn" @@ -39,7 +57,9 @@ You just need to install an ASGI compatible server like: ...or any other ASGI server. -And run your application the same way you have done in the tutorials, but without the `--reload` option, e.g.: +## Run the Server Program + +You can then your application the same way you have done in the tutorials, but without the `--reload` option, e.g.: === "Uvicorn" @@ -65,10 +85,24 @@ And run your application the same way you have done in the tutorials, but withou -You might want to set up some tooling to make sure it is restarted automatically if it stops. +!!! warning + Remember to remove the `--reload` option if you were using it. + + The `--reload` option consumes much more resources, is more unstable, etc. + + It helps a lot during **development**, but you **shouldn't** use it in **production**. + +## Deployment Concepts + +These examples run the server program (e.g Uvicorn), starting **a single process**, listening on all the IPs (`0.0.0.0`) on a predefined port (e.g. `80`). -You might also want to install Gunicorn and use it as a manager for Uvicorn, or use Hypercorn with multiple workers. +This is the basic idea. But you will probably want to take care of some additional things, like: -Making sure to fine-tune the number of workers, etc. +* Security - HTTPS +* Running on startup +* Restarts +* Replication (the number of processes running) +* Memory +* Previous steps before starting -But if you are doing all that, you might just use the Docker image that does it automatically. +I'll tell you more about each of these concepts, how to think about them, and some concrete examples with strategies to handle them in the next chapters. πŸš€ diff --git a/docs/en/docs/deployment/server-workers.md b/docs/en/docs/deployment/server-workers.md new file mode 100644 index 000000000..2892d5797 --- /dev/null +++ b/docs/en/docs/deployment/server-workers.md @@ -0,0 +1,178 @@ +# Server Workers - Gunicorn with Uvicorn + +Let's check back those deployment concepts from before: + +* Security - HTTPS +* Running on startup +* Restarts +* **Replication (the number of processes running)** +* Memory +* Previous steps before starting + +Up to this point, with all the tutorials in the docs, you have probably been running a **server program** like Uvicorn, running a **single process**. + +When deploying applications you will probably want to have some **replication of processes** to take advantage of **multiple cores** and to be able to handle more requests. + +As you saw in the previous chapter about [Deployment Concepts](./concepts.md){.internal-link target=_blank}, there are multiple strategies you can use. + +Here I'll show you how to use **Gunicorn** with **Uvicorn worker processes**. + +!!! info + If you are using containers, for example with Docker or Kubernetes, I'll tell you more about that in the next chapter: [FastAPI in Containers - Docker](./docker.md){.internal-link target=_blank}. + + In particular, when running on **Kubernetes** you will probably **not** want to use Gunicorn, and instead run **a single Uvicorn process per container**, but I'll tell you about it later in that chapter. + +## Gunicorn with Uvicorn Workers + +**Gunicorn** is mainly an application server using the **WSGI standard**. That means that Gunicorn can serve applications like Flask and Django. Gunicorn by itself is not compatible with **FastAPI**, as FastAPI uses the newest **ASGI standard**. + +But Gunicorn supports working as a **process manager** and allowing users to tell it which specific **worker process class** to use. Then Gunicorn would start one or more **worker processes** using that class. + +And **Uvicorn** has a **Gunicorn-compatible worker class**. + +Using that combination, Gunicorn would act as a **process manager**, listening on the **port** and the **IP**. And it would **transmit** the communication to the worker processes running the **Uvicorn class**. + +And then the Gunicorn-compatible **Uvicorn worker** class would be in charge of converting the data sent by Gunicorn to the ASGI standard for FastAPI to use it. + +## Install Gunicorn and Uvicorn + +
+ +```console +$ pip install uvicorn[standard] gunicorn + +---> 100% +``` + +
+ +That will install both Uvicorn with the `standard` extra packages (to get high performance) and Gunicorn. + +## Run Gunicorn with Uvicorn Workers + +Then you can run Gunicorn with: + +
+ +```console +$ gunicorn main:app --workers 4 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:80 + +[19499] [INFO] Starting gunicorn 20.1.0 +[19499] [INFO] Listening at: http://0.0.0.0:80 (19499) +[19499] [INFO] Using worker: uvicorn.workers.UvicornWorker +[19511] [INFO] Booting worker with pid: 19511 +[19513] [INFO] Booting worker with pid: 19513 +[19514] [INFO] Booting worker with pid: 19514 +[19515] [INFO] Booting worker with pid: 19515 +[19511] [INFO] Started server process [19511] +[19511] [INFO] Waiting for application startup. +[19511] [INFO] Application startup complete. +[19513] [INFO] Started server process [19513] +[19513] [INFO] Waiting for application startup. +[19513] [INFO] Application startup complete. +[19514] [INFO] Started server process [19514] +[19514] [INFO] Waiting for application startup. +[19514] [INFO] Application startup complete. +[19515] [INFO] Started server process [19515] +[19515] [INFO] Waiting for application startup. +[19515] [INFO] Application startup complete. +``` + +
+ +Let's see what each of those options mean: + +* `main:app`: This is the same syntax used by Uvicorn, `main` means the Python module named "`main`", so, a file `main.py`. And `app` is the name of the variable that is the **FastAPI** application. + * You can imagine that `main:app` is equivalent to a Python `import` statement like: + + ```Python + from main import app + ``` + + * So, the colon in `main:app` would be equivalent to the Python `import` part in `from main import app`. +* `--workers`: The number of worker processes to use, each will run a Uvicorn worker, in this case 4 workers. +* `--worker-class`: The Gunicorn-compatible worker class to use in the worker processes. + * Here we pass the class that Gunicorn can import and use with: + + ```Python + import uvicorn.workers.UvicornWorker + ``` + +* `--bind`: This tells Gunicorn the IP and the port to listen to, using a colon (`:`) to separate the IP and the port. + * If you were running Uvicorn directly, instead of `--bind 0.0.0.0:80` (the Gunicorn option) you would use `--host 0.0.0.0` and `--port 80`. + +In the output you can see that it shows the **PID** (process ID) of each process (it's just a number). + +You can see that: + +* The Gunicorn **process manager** starts with PID `19499` (in your case it will be a different number). +* Then it starts `Listening at: http://0.0.0.0:80`. +* Then it detects that it has to use the worker class at `uvicorn.workers.UvicornWorker`. +* And then it starts **4 workers**, each with its own PID: `19511`, `19513`, `19514`, and `19515`. + +Gunicorn would also take care of managing **dead processes** and **restarting** new ones if needed to keep the number of workers. So that helps in part with the **restart** concept from the list above. + +Nevertheless, you would probably also want to have something outside making sure to **restart Gunicorn** if necessary, and also to **run it on startup**, etc. + +## Uvicorn with Workers + +Uvicorn also has an option to start and run several **worker processes**. + +Nevertheless, as of now, Uvicorn's capabilities for handling worker processes are more limited than Gunicorn's. So, if you want to have a process manager at this level (at the Python level), then it might be better to try with Gunicorn as the process manager. + +In any case, you would run it like this: + +
+ +```console +$ uvicorn main:app --host 0.0.0.0 --port 8080 --workers 4 +INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit) +INFO: Started parent process [27365] +INFO: Started server process [27368] +INFO: Waiting for application startup. +INFO: Application startup complete. +INFO: Started server process [27369] +INFO: Waiting for application startup. +INFO: Application startup complete. +INFO: Started server process [27370] +INFO: Waiting for application startup. +INFO: Application startup complete. +INFO: Started server process [27367] +INFO: Waiting for application startup. +INFO: Application startup complete. +``` + +
+ +The only new option here is `--workers` telling Uvicorn to start 4 worker processes. + +You can also see that it shows the **PID** of each process, `27365` for the parent process (this is the **process manager**) and one for each worker process: `27368`, `27369`, `27370`, and `27367`. + +## Deployment Concepts + +Here you saw how to use **Gunicorn** (or Uvicorn) managing **Uvicorn worker processes** to **parallelize** the execution of the application, take advantage of **multiple cores** in the CPU, and be able to serve **more requests**. + +From the list of deployment concepts from above, using workers would mainly help with the **replication** part, and a little bit with the **restarts**, but you still need to take care of the others: + +* **Security - HTTPS** +* **Running on startup** +* ***Restarts*** +* Replication (the number of processes running) +* **Memory** +* **Previous steps before starting** + +## Containers and Docker + +In the next chapter about [FastAPI in Containers - Docker](./docker.md){.internal-link target=_blank} I'll tell some strategies you could use to handle the other **deployment concepts**. + +I'll also show you the **official Docker image** that includes **Gunicorn with Uvicorn workers** and some default configurations that can be useful for simple cases. + +There I'll also show you how to **build your own image from scratch** to run a single Uvicorn process (without Gunicorn). It is a simple process and is probably what you would want to do when using a distributed container management system like **Kubernetes**. + +## Recap + +You can use **Gunicorn** (or also Uvicorn) as a process manager with Uvicorn workers to take advantage of **multi-core CPUs**, to run **multiple processes in parallel**. + +You could use these tools and ideas if you are setting up **your own deployment system** while taking care of the other deployment concepts yourself. + +Check out the next chapter to learn about **FastAPI** with containers (e.g. Docker and Kubernetes). You will see that those tools have simple ways to solve the other **deployment concepts** as well. ✨ diff --git a/docs/en/docs/img/deployment/concepts/image01.png b/docs/en/docs/img/deployment/concepts/image01.png new file mode 100644 index 000000000..fdce75e98 Binary files /dev/null and b/docs/en/docs/img/deployment/concepts/image01.png differ diff --git a/docs/en/docs/img/deployment/concepts/process-ram.drawio b/docs/en/docs/img/deployment/concepts/process-ram.drawio new file mode 100644 index 000000000..51fc30ed3 --- /dev/null +++ b/docs/en/docs/img/deployment/concepts/process-ram.drawio @@ -0,0 +1,106 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/en/docs/img/deployment/concepts/process-ram.svg b/docs/en/docs/img/deployment/concepts/process-ram.svg new file mode 100644 index 000000000..cd086c36b --- /dev/null +++ b/docs/en/docs/img/deployment/concepts/process-ram.svg @@ -0,0 +1,59 @@ +
Server
Server
RAM
RAM +
CPU
CPU +
ProcessΒ Manager
ProcessΒ Manager
Worker Process
Worker Process
Worker Process
Worker Process
Another Process
Another Process
1 GB
1 GB
1 GB
1 GBViewer does not support full SVG 1.1 \ No newline at end of file diff --git a/docs/en/mkdocs.yml b/docs/en/mkdocs.yml index a11ac6871..b32468783 100644 --- a/docs/en/mkdocs.yml +++ b/docs/en/mkdocs.yml @@ -147,9 +147,11 @@ nav: - deployment/index.md - deployment/versions.md - deployment/https.md + - deployment/manually.md + - deployment/concepts.md - deployment/deta.md + - deployment/server-workers.md - deployment/docker.md - - deployment/manually.md - project-generation.md - alternatives.md - history-design-future.md