@ -21,7 +21,7 @@ By considering these concepts, you will be able to **evaluate and design** the b
In the next chapters, I'll give you more **concrete recipes** to deploy FastAPI applications.
But for now, let's check these important **conceptual ideas**. These concepts also apply for any other type of web API. 💡
But for now, let's check these important **conceptual ideas**. These concepts also apply to any other type of web API. 💡
## Security - HTTPS
@ -47,7 +47,7 @@ Some of the tools you could use as a TLS Termination Proxy are:
* With an external component like cert-manager for certificate renewals
* Handled internally by a cloud provider as part of their services (read below 👇)
Another option is that you could use a **cloud service** that does more of the work including setting up HTTPS. It could have some restrictions or charge you more, etc. But in that case you wouldn't have to set up a TLS Termination Proxy yourself.
Another option is that you could use a **cloud service** that does more of the work including setting up HTTPS. It could have some restrictions or charge you more, etc. But in that case, you wouldn't have to set up a TLS Termination Proxy yourself.
I'll show you some concrete examples in the next chapters.
@ -64,7 +64,7 @@ We will talk a lot about the running "**process**", so it's useful to have clari
The word **program** is commonly used to describe many things:
* The **code** that you write, the **Python files**.
* The **file** that can be **executed** by the operating system, for example `python`, `python.exe` or `uvicorn`.
* The **file** that can be **executed** by the operating system, for example:`python`, `python.exe` or `uvicorn`.
* A particular program while it is **running** on the operating system, using the CPU, and storing things on memory. This is also called a **process**.
### What is a Process
@ -75,7 +75,7 @@ The word **process** is normally used in a more specific way, only referring to
* This doesn't refer to the file, nor to the code, it refers **specifically** to the thing that is being **executed** and managed by the operating system.
* Any program, any code, **can only do things** when it is being **executed**. So, when there's a **process running**.
* The process can be **terminated** (or "killed") by you, or by the operating system. At that point, it stops running/being executed, and it can **no longer do things**.
* Each application that you have running in your computer has some process behind it, each running program, each window, etc. And there are normally many processes running **at the same time** while a computer is on.
* Each application that you have running on your computer has some process behind it, each running program, each window, etc. And there are normally many processes running **at the same time** while a computer is on.
* There can be **multiple processes** of the **same program** running at the same time.
If you check out the "task manager" or "system monitor" (or similar tools) in your operating system, you will be able to see many of those processes running.
@ -90,13 +90,13 @@ Now that we know the difference between the terms **process** and **program**, l
## Running on Startup
In most cases, when you create a web API, you want it to be **always running**, uninterrupted, so that your clients can always access it. This is of course, unless you have a specific reason why you want it to run only on certain situations, but most of the time you want it constantly running and **available**.
In most cases, when you create a web API, you want it to be **always running**, uninterrupted, so that your clients can always access it. This is of course, unless you have a specific reason why you want it to run only in certain situations, but most of the time you want it constantly running and **available**.
### In a Remote Server
When you set up a remote server (a cloud server, a virtual machine, etc.) the simplest thing you can do is to run Uvicorn (or similar) manually, the same way you do when developing locally.
And it will work, and will be useful **during development**.
And it will work and will be useful **during development**.
But if your connection to the server is lost, the **running process** will probably die.
@ -108,7 +108,7 @@ In general, you will probably want the server program (e.g. Uvicorn) to be start
### Separate Program
To achieve this, you will normally have a **separate program** that would make sure your application is run on startup. And in many cases it would also make sure other components or applications are also run, for example a database.
To achieve this, you will normally have a **separate program** that would make sure your application is run on startup. And in many cases, it would also make sure other components or applications are also run, for example, a database.
### Example Tools to Run at Startup
@ -177,7 +177,7 @@ For example, this could be handled by:
With a FastAPI application, using a server program like Uvicorn, running it once in **one process** can serve multiple clients concurrently.
But in many cases you will want to run several worker processes at the same time.
But in many cases, you will want to run several worker processes at the same time.
### Multiple Processes - Workers
@ -197,11 +197,11 @@ So, to be able to have **multiple processes** at the same time, there has to be
Now, when the program loads things in memory, for example, a machine learning model in a variable, or the contents of a large file in a variable, all that **consumes a bit of the memory (RAM)** of the server.
And multiple processes normally **don't share any memory**. This means that each running process has its own things, its own variables, its own memory. And if you are consuming a large amount of memory in your code, **each process** will consume an equivalent amount of memory.
And multiple processes normally **don't share any memory**. This means that each running process has its own things, variables, and memory. And if you are consuming a large amount of memory in your code, **each process** will consume an equivalent amount of memory.
### Server Memory
For example, if your code loads a Machine Learning model with **1 GB in size**, when you run one process with your API, it will consume at least 1 GB or RAM. And if you start **4 processes** (4 workers), each will consume 1 GB of RAM. So, in total your API will consume **4 GB of RAM**.
For example, if your code loads a Machine Learning model with **1 GB in size**, when you run one process with your API, it will consume at least 1 GB of RAM. And if you start **4 processes** (4 workers), each will consume 1 GB of RAM. So in total, your API will consume **4 GB of RAM**.
And if your remote server or virtual machine only has 3 GB of RAM, trying to load more than 4 GB of RAM will cause problems. 🚨
@ -253,12 +253,12 @@ But in most cases, you will want to perform these steps only **once**.
So, you will want to have a **single process** to perform those **previous steps**, before starting the application.
And you will have to make sure that it's a single process running those previous steps *even* if afterwards you start **multiple processes** (multiple workers) for the application itself. If those steps were run by **multiple processes**, they would **duplicate** the work by running it on **parallel**, and if the steps were something delicate like a database migration, they could cause conflicts with each other.
And you will have to make sure that it's a single process running those previous steps *even* if afterwards, you start **multiple processes** (multiple workers) for the application itself. If those steps were run by **multiple processes**, they would **duplicate** the work by running it on **parallel**, and if the steps were something delicate like a database migration, they could cause conflicts with each other.
Of course, there are some cases where there's no problem in running the previous steps multiple times, in that case it's a lot easier to handle.
Of course, there are some cases where there's no problem in running the previous steps multiple times, in that case, it's a lot easier to handle.
!!! tip
Also have in mind that depending on your setup, in some cases you **might not even need any previous steps** before starting your application.
Also, have in mind that depending on your setup, in some cases you **might not even need any previous steps** before starting your application.
In that case, you wouldn't have to worry about any of this. 🤷
@ -279,7 +279,7 @@ Here are some possible ideas:
Your server(s) is (are) a **resource**, you can consume or **utilize**, with your programs, the computation time on the CPUs, and the RAM memory available.
How much resources do you want to be consuming/utilizing? It might be easy to think "not much", but in reality, you will probably want to consume **as much as possible without crashing**.
How much of the system resources do you want to be consuming/utilizing? It might be easy to think "not much", but in reality, you will probably want to consume **as much as possible without crashing**.
If you are paying for 3 servers but you are using only a little bit of their RAM and CPU, you are probably **wasting money** 💸, and probably **wasting server electric power** 🌎, etc.
@ -291,9 +291,9 @@ In this case, it would be better to get **one extra server** and run some proces
There's also the chance that for some reason you have a **spike** of usage of your API. Maybe it went viral, or maybe some other services or bots start using it. And you might want to have extra resources to be safe in those cases.
You could put an **arbitrary number** to target, for example something **between 50% to 90%** of resource utilization. The point is that those are probably the main things you will want to measure and use to tweak your deployments.
You could put an **arbitrary number** to target, for example, something **between 50% to 90%** of resource utilization. The point is that those are probably the main things you will want to measure and use to tweak your deployments.
You can use simple tools like `htop` to see the CPU and RAM used in your server, or the amount used by each process. Or you can use more complex monitoring tools, maybe distributed across servers, etc.
You can use simple tools like `htop` to see the CPU and RAM used in your server or the amount used by each process. Or you can use more complex monitoring tools, which maybe distributed across servers, etc.
## Recap
@ -308,4 +308,4 @@ You have been reading here some of the main concepts that you would probably nee
Understanding these ideas and how to apply them should give you the intuition necessary to take any decisions when configuring and tweaking your deployments. 🤓
In the next sections I'll give you more concrete examples of possible strategies you can follow. 🚀
In the next sections, I'll give you more concrete examples of possible strategies you can follow. 🚀
@ -9,7 +9,7 @@ It will take you about **10 minutes**.
## A basic **FastAPI** app
* Create a directory for your app, for example `./fastapideta/` and enter in it.
* Create a directory for your app, for example,`./fastapideta/` and enter into it.
### FastAPI code
@ -213,7 +213,7 @@ Now you can share that URL with anyone and they will be able to access your API.
Congrats! You deployed your FastAPI app to Deta! 🎉 🍰
Also notice that Deta correctly handles HTTPS for you, so you don't have to take care of that and can be sure that your clients will have a secure encrypted connection. ✅ 🔒
Also, notice that Deta correctly handles HTTPS for you, so you don't have to take care of that and can be sure that your clients will have a secure encrypted connection. ✅ 🔒
## Check the Visor
@ -235,7 +235,7 @@ You can also edit them and re-play them.
## Learn more
At some point you will probably want to store some data for your app in a way that persists through time. For that you can use <ahref="https://docs.deta.sh/docs/base/py_tutorial?ref=fastapi"class="external-link"target="_blank">Deta Base</a>, it also has a generous **free tier**.
At some point, you will probably want to store some data for your app in a way that persists through time. For that you can use <ahref="https://docs.deta.sh/docs/base/py_tutorial?ref=fastapi"class="external-link"target="_blank">Deta Base</a>, it also has a generous **free tier**.
You can also read more in the <ahref="https://docs.deta.sh?ref=fastapi"class="external-link"target="_blank">Deta Docs</a>.
@ -253,6 +253,6 @@ Coming back to the concepts we discussed in [Deployments Concepts](./concepts.md
!!! note
Deta is designed to make it easy (and free) to deploy simple applications quickly.
It can simplify a lot several use cases, but at the same time it doesn't support others, like using external databases (apart from Deta's own NoSQL database system), custom virtual machines, etc.
It can simplify several use cases, but at the same time, it doesn't support others, like using external databases (apart from Deta's own NoSQL database system), custom virtual machines, etc.
You can read more details in the <ahref="https://docs.deta.sh/docs/micros/about/"class="external-link"target="_blank">Deta docs</a> to see if it's the right choice for you.
When deploying FastAPI applications a common approach is to build a **Linux container image**. It's normally done using <ahref="https://www.docker.com/"class="external-link"target="_blank">**Docker**</a>. And then you can deploy that container image in one of different possible ways.
When deploying FastAPI applications a common approach is to build a **Linux container image**. It's normally done using <ahref="https://www.docker.com/"class="external-link"target="_blank">**Docker**</a>. You can then deploy that container image in one of a few possible ways.
Using Linux containers has several advantages including **security**, **replicability**, **simplicity**, and others.
@ -68,13 +68,13 @@ And there are many other images for different things like databases, for example
* <ahref="https://hub.docker.com/_/redis"class="external-link"target="_blank">Redis</a>, etc.
By using a pre-made container image it's very easy to **combine** and use different tools. For example, to try out a new database. In most cases you can use the **official images**, and just configure them with environment variables.
By using a pre-made container image it's very easy to **combine** and use different tools. For example, to try out a new database. In most cases, you can use the **official images**, and just configure them with environment variables.
That way, in many cases you can learn about containers and Docker and re-use that knowledge with many different tools and components.
So, you would run **multiple containers** with different things, like a database, a Python application, a web server with a React frontend application, and connect them together via their internal network.
All the container management systems (like Docker or Kubernetes) have these networking features integrated in them.
All the container management systems (like Docker or Kubernetes) have these networking features integrated into them.
## Containers and Processes
@ -84,7 +84,7 @@ When a **container** is started, it will run that command/program (although you
A container is running as long as the **main process** (command or program) is running.
A container normally has a **single process**, but it's also possible to start subprocesses from the main process, and that way have **multiple processes** in the same container.
A container normally has a **single process**, but it's also possible to start subprocesses from the main process, and that way you will have **multiple processes** in the same container.
But it's not possible to have a running container without **at least one running process**. If the main process stops, the container stops.
6. Set the **command** to run the `uvicorn` server.
`CMD` takes a list of strings, each of this strings is what you would type in the command line separated by spaces.
`CMD` takes a list of strings, each of these strings is what you would type in the command line separated by spaces.
This command will be run from the **current working directory**, the same `/code` directory you set above with `WORKDIR /code`.
@ -338,7 +338,7 @@ You will see the alternative automatic documentation (provided by <a href="https
## Build a Docker Image with a Single-File FastAPI
If your FastAPI is a single file, for example `main.py` without an `./app` directory, your file structure could look like:
If your FastAPI is a single file, for example,`main.py` without an `./app` directory, your file structure could look like this:
```
.
@ -411,7 +411,7 @@ Without using containers, making applications run on startup and with restarts c
## Replication - Number of Processes
If you have a <abbrtitle="A group of machines that are configured to be connected and work together in some way.">cluster</abbr> of machines with **Kubernetes**, Docker Swarm Mode, Nomad, or other similar complex system to manage distributed containers on multiple machines, then you will probably want to **handle replication** at the **cluster level** instead of using a **process manager** (like Gunicorn with workers) in each container.
If you have a <abbrtitle="A group of machines that are configured to be connected and work together in some way.">cluster</abbr> of machines with **Kubernetes**, Docker Swarm Mode, Nomad, or another similar complex system to manage distributed containers on multiple machines, then you will probably want to **handle replication** at the **cluster level** instead of using a **process manager** (like Gunicorn with workers) in each container.
One of those distributed container management systems like Kubernetes normally has some integrated way of handling **replication of containers** while still supporting **load balancing** for the incoming requests. All at the **cluster level**.
@ -487,7 +487,7 @@ The main point is, **none** of these are **rules written in stone** that you hav
## Memory
If you run **a single process per container** you will have a more or less welldefined, stable, and limited amount of memory consumed by each of of those containers (more than one if they are replicated).
If you run **a single process per container** you will have a more or less well-defined, stable, and limited amount of memory consumed by each of those containers (more than one if they are replicated).
And then you can set those same memory limits and requirements in your configurations for your container management system (for example in **Kubernetes**). That way it will be able to **replicate the containers** in the **available machines** taking into account the amount of memory needed by them, and the amount available in the machines in the cluster.
@ -187,4 +187,4 @@ Having **HTTPS** is very important, and quite **critical** in most cases. Most o
But once you know the basic information of **HTTPS for developers** you can easily combine and configure different tools to help you manage everything in a simple way.
In some of the next chapters I'll show you several concrete examples of how to set up **HTTPS** for **FastAPI** applications. 🔒
In some of the next chapters, I'll show you several concrete examples of how to set up **HTTPS** for **FastAPI** applications. 🔒
@ -8,7 +8,7 @@ To **deploy** an application means to perform the necessary steps to make it **a
For a **web API**, it normally involves putting it in a **remote machine**, with a **server program** that provides good performance, stability, etc, so that your **users** can **access** the application efficiently and without interruptions or problems.
This is in contrast to the the **development** stages, where you are constantly changing the code, breaking it and fixing it, stopping and restarting the development server, etc.
This is in contrast to the **development** stages, where you are constantly changing the code, breaking it and fixing it, stopping and restarting the development server, etc.
@ -20,7 +20,7 @@ Here I'll show you how to use <a href="https://gunicorn.org/" class="external-li
!!! info
If you are using containers, for example with Docker or Kubernetes, I'll tell you more about that in the next chapter: [FastAPI in Containers - Docker](./docker.md){.internal-link target=_blank}.
In particular, when running on **Kubernetes** you will probably **not** want to use Gunicorn, and instead run **a single Uvicorn process per container**, but I'll tell you about it later in that chapter.
In particular, when running on **Kubernetes** you will probably **not** want to use Gunicorn and instead run **a single Uvicorn process per container**, but I'll tell you about it later in that chapter.
## Gunicorn with Uvicorn Workers
@ -90,7 +90,7 @@ Let's see what each of those options mean:
```
* So, the colon in `main:app` would be equivalent to the Python `import` part in `from main import app`.
* `--workers`: The number of worker processes to use, each will run a Uvicorn worker, in this case 4 workers.
* `--workers`: The number of worker processes to use, each will run a Uvicorn worker, in this case, 4 workers.
* `--worker-class`: The Gunicorn-compatible worker class to use in the worker processes.
* Here we pass the class that Gunicorn can import and use with:
@ -101,7 +101,7 @@ Let's see what each of those options mean:
* `--bind`: This tells Gunicorn the IP and the port to listen to, using a colon (`:`) to separate the IP and the port.
* If you were running Uvicorn directly, instead of `--bind 0.0.0.0:80` (the Gunicorn option) you would use `--host 0.0.0.0` and `--port 80`.
In the output you can see that it shows the **PID** (process ID) of each process (it's just a number).
In the output, you can see that it shows the **PID** (process ID) of each process (it's just a number).
Andy Challis
4 years ago
GitHub