Browse Source

🔥 Remove Celery and Flower, they are currently not used nor recommended (#694)

pull/13907/head
Sebastián Ramírez 1 year ago
committed by GitHub
parent
commit
1e256bce5d
No known key found for this signature in database GPG Key ID: B5690EEEBB952194
  1. 4
      .env
  2. 1
      .github/workflows/deploy-production.yml
  3. 1
      .github/workflows/deploy-staging.yml
  4. 1
      README.md
  5. 0
      backend/Dockerfile
  6. 18
      backend/README.md
  7. 14
      backend/app/api/routes/utils.py
  8. 5
      backend/app/core/celery_app.py
  9. 16
      backend/app/tests/api/routes/test_celery.py
  10. 33
      backend/app/tests/scripts/test_celery_pre_start.py
  11. 12
      backend/app/worker.py
  12. 32
      backend/celeryworker.dockerfile
  13. 215
      backend/poetry.lock
  14. 2
      backend/pyproject.toml
  15. 6
      backend/worker-start.sh
  16. 5
      copier.yml
  17. 5
      deployment.md
  18. 4
      development.md
  19. 26
      docker-compose.override.yml
  20. 69
      docker-compose.yml

4
.env

@ -33,10 +33,6 @@ POSTGRES_PASSWORD=changethis
SENTRY_DSN= SENTRY_DSN=
# Flower
FLOWER_BASIC_AUTH=admin:changethis
# Configure these with your own Docker registry images # Configure these with your own Docker registry images
DOCKER_IMAGE_BACKEND=backend DOCKER_IMAGE_BACKEND=backend
DOCKER_IMAGE_CELERYWORKER=celery
DOCKER_IMAGE_FRONTEND=frontend DOCKER_IMAGE_FRONTEND=frontend

1
.github/workflows/deploy-production.yml

@ -22,7 +22,6 @@ jobs:
EMAILS_FROM_EMAIL: ${{ secrets.EMAILS_FROM_EMAIL }} EMAILS_FROM_EMAIL: ${{ secrets.EMAILS_FROM_EMAIL }}
POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }} POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }} SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
FLOWER_BASIC_AUTH: ${{ secrets.FLOWER_BASIC_AUTH }}
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4

1
.github/workflows/deploy-staging.yml

@ -22,7 +22,6 @@ jobs:
EMAILS_FROM_EMAIL: ${{ secrets.EMAILS_FROM_EMAIL }} EMAILS_FROM_EMAIL: ${{ secrets.EMAILS_FROM_EMAIL }}
POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }} POSTGRES_PASSWORD: ${{ secrets.POSTGRES_PASSWORD }}
SENTRY_DSN: ${{ secrets.SENTRY_DSN }} SENTRY_DSN: ${{ secrets.SENTRY_DSN }}
FLOWER_BASIC_AUTH: ${{ secrets.FLOWER_BASIC_AUTH }}
steps: steps:
- name: Checkout - name: Checkout
uses: actions/checkout@v4 uses: actions/checkout@v4

1
README.md

@ -155,7 +155,6 @@ The input variables, with their default values (some auto generated) are:
- `emails_from_email`: (default: `"info@example.com"`) The email account to send emails from, you can set it later in .env. - `emails_from_email`: (default: `"info@example.com"`) The email account to send emails from, you can set it later in .env.
- `postgres_password`: (default: `"changethis"`) The password for the PostgreSQL database, stored in .env, you can generate one with the method above. - `postgres_password`: (default: `"changethis"`) The password for the PostgreSQL database, stored in .env, you can generate one with the method above.
- `sentry_dsn`: (default: "") The DSN for Sentry, if you are using it, you can set it later in .env. - `sentry_dsn`: (default: "") The DSN for Sentry, if you are using it, you can set it later in .env.
- `flower_basic_auth`: (default: `"admin:changethis"`) The basic auth for Flower, you can set it later in .env.
## Release Notes ## Release Notes

0
backend/backend.dockerfile → backend/Dockerfile

18
backend/README.md

@ -23,8 +23,6 @@ Automatic interactive documentation with Swagger UI (from the OpenAPI backend):
Adminer, database web administration: http://localhost:8080 Adminer, database web administration: http://localhost:8080
Flower, administration of Celery tasks: http://localhost:5555
Traefik UI, to see how the routes are being handled by the proxy: http://localhost:8090 Traefik UI, to see how the routes are being handled by the proxy: http://localhost:8090
**Note**: The first time you start your stack, it might take a minute for it to be ready. While the backend waits for the database to be ready and configures everything. You can check the logs to monitor it. **Note**: The first time you start your stack, it might take a minute for it to be ready. While the backend waits for the database to be ready and configures everything. You can check the logs to monitor it.
@ -65,8 +63,6 @@ Make sure your editor is using the correct Python virtual environment.
Modify or add SQLModel models for data and SQL tables in `./backend/app/models.py`, API endpoints in `./backend/app/api/`, CRUD (Create, Read, Update, Delete) utils in `./backend/app/crud.py`. Modify or add SQLModel models for data and SQL tables in `./backend/app/models.py`, API endpoints in `./backend/app/api/`, CRUD (Create, Read, Update, Delete) utils in `./backend/app/crud.py`.
Add and modify tasks to the Celery worker in `./backend/app/worker.py`.
### VS Code ### VS Code
There are already configurations in place to run the backend through the VS Code debugger, so that you can use breakpoints, pause and explore variables, etc. There are already configurations in place to run the backend through the VS Code debugger, so that you can use breakpoints, pause and explore variables, etc.
@ -157,19 +153,7 @@ docker compose exec backend bash /app/tests-start.sh -x
#### Test Coverage #### Test Coverage
Because the test scripts forward arguments to `pytest`, you can enable test coverage HTML report generation by passing `--cov-report=html`. When the tests are run, a file `htmlcov/index.html` is generated, you can open it in your browser to see the coverage of the tests.
To run the local tests with coverage HTML reports:
```Bash
DOMAIN=backend sh ./scripts/test-local.sh --cov-report=html
```
To run the tests in a running stack with coverage HTML reports:
```bash
docker compose exec backend bash /app/tests-start.sh --cov-report=html
```
### Migrations ### Migrations

14
backend/app/api/routes/utils.py

@ -2,26 +2,12 @@ from fastapi import APIRouter, Depends
from pydantic.networks import EmailStr from pydantic.networks import EmailStr
from app.api.deps import get_current_active_superuser from app.api.deps import get_current_active_superuser
from app.core.celery_app import celery_app
from app.models import Message from app.models import Message
from app.utils import generate_test_email, send_email from app.utils import generate_test_email, send_email
router = APIRouter() router = APIRouter()
@router.post(
"/test-celery/",
dependencies=[Depends(get_current_active_superuser)],
status_code=201,
)
def test_celery(body: Message) -> Message:
"""
Test Celery worker.
"""
celery_app.send_task("app.worker.test_celery", args=[body.message])
return Message(message="Word received")
@router.post( @router.post(
"/test-email/", "/test-email/",
dependencies=[Depends(get_current_active_superuser)], dependencies=[Depends(get_current_active_superuser)],

5
backend/app/core/celery_app.py

@ -1,5 +0,0 @@
from celery import Celery
celery_app = Celery("worker", broker="amqp://guest@queue//")
celery_app.conf.task_routes = {"app.worker.test_celery": "main-queue"}

16
backend/app/tests/api/routes/test_celery.py

@ -1,16 +0,0 @@
from fastapi.testclient import TestClient
from app.core.config import settings
def test_celery_worker_test(
client: TestClient, superuser_token_headers: dict[str, str]
) -> None:
data = {"message": "test"}
r = client.post(
f"{settings.API_V1_STR}/utils/test-celery/",
json=data,
headers=superuser_token_headers,
)
response = r.json()
assert response["message"] == "Word received"

33
backend/app/tests/scripts/test_celery_pre_start.py

@ -1,33 +0,0 @@
from unittest.mock import MagicMock
from pytest_mock import MockerFixture
from sqlmodel import select
from app.celeryworker_pre_start import init, logger
def test_init_successful_connection(mocker: MockerFixture) -> None:
engine_mock = MagicMock()
session_mock = MagicMock()
exec_mock = MagicMock(return_value=True)
session_mock.configure_mock(**{"exec.return_value": exec_mock})
mocker.patch("sqlmodel.Session", return_value=session_mock)
mocker.patch.object(logger, "info")
mocker.patch.object(logger, "error")
mocker.patch.object(logger, "warn")
try:
init(engine_mock)
connection_successful = True
except Exception:
connection_successful = False
assert (
connection_successful
), "The database connection should be successful and not raise an exception."
assert session_mock.exec.called_once_with(
select(1)
), "The session should execute a select statement once."

12
backend/app/worker.py

@ -1,12 +0,0 @@
import sentry_sdk
from app.core.celery_app import celery_app
from app.core.config import settings
if settings.SENTRY_DSN:
sentry_sdk.init(dsn=str(settings.SENTRY_DSN))
@celery_app.task(acks_late=True)
def test_celery(word: str) -> str:
return f"test task return {word}"

32
backend/celeryworker.dockerfile

@ -1,32 +0,0 @@
FROM python:3.10
ENV PYTHONDONTWRITEBYTECODE=1
WORKDIR /app/
# Install Poetry
RUN curl -sSL https://install.python-poetry.org | POETRY_HOME=/opt/poetry python && \
cd /usr/local/bin && \
ln -s /opt/poetry/bin/poetry && \
poetry config virtualenvs.create false
# Copy poetry.lock* in case it doesn't exist in the repo
COPY ./pyproject.toml ./poetry.lock* /app/
# Allow installing dev dependencies to run tests
ARG INSTALL_DEV=false
RUN bash -c "if [ $INSTALL_DEV == 'true' ] ; then poetry install --no-root ; else poetry install --no-root --only main ; fi"
ENV C_FORCE_ROOT=1
ENV PYTHONPATH=/app
COPY ./alembic.ini /app/
COPY ./worker-start.sh /worker-start.sh
COPY ./app /app/app
RUN chmod +x /worker-start.sh
CMD ["bash", "/worker-start.sh"]

215
backend/poetry.lock

@ -1,4 +1,4 @@
# This file is automatically @generated by Poetry 1.7.1 and should not be changed by hand. # This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand.
[[package]] [[package]]
name = "alembic" name = "alembic"
@ -19,20 +19,6 @@ typing-extensions = ">=4"
[package.extras] [package.extras]
tz = ["backports.zoneinfo"] tz = ["backports.zoneinfo"]
[[package]]
name = "amqp"
version = "5.2.0"
description = "Low-level AMQP client for Python (fork of amqplib)."
optional = false
python-versions = ">=3.6"
files = [
{file = "amqp-5.2.0-py3-none-any.whl", hash = "sha256:827cb12fb0baa892aad844fd95258143bce4027fdac4fccddbc43330fd281637"},
{file = "amqp-5.2.0.tar.gz", hash = "sha256:a1ecff425ad063ad42a486c902807d1482311481c8ad95a72694b2975e75f7fd"},
]
[package.dependencies]
vine = ">=5.0.0,<6.0.0"
[[package]] [[package]]
name = "annotated-types" name = "annotated-types"
version = "0.6.0" version = "0.6.0"
@ -99,17 +85,6 @@ files = [
tests = ["pytest (>=3.2.1,!=3.3.0)"] tests = ["pytest (>=3.2.1,!=3.3.0)"]
typecheck = ["mypy"] typecheck = ["mypy"]
[[package]]
name = "billiard"
version = "4.2.0"
description = "Python multiprocessing fork with improvements and bugfixes"
optional = false
python-versions = ">=3.7"
files = [
{file = "billiard-4.2.0-py3-none-any.whl", hash = "sha256:07aa978b308f334ff8282bd4a746e681b3513db5c9a514cbdd810cbbdc19714d"},
{file = "billiard-4.2.0.tar.gz", hash = "sha256:9a3c3184cb275aa17a732f93f65b20c525d3d9f253722d26a82194803ade5a2c"},
]
[[package]] [[package]]
name = "cachetools" name = "cachetools"
version = "5.3.3" version = "5.3.3"
@ -121,75 +96,6 @@ files = [
{file = "cachetools-5.3.3.tar.gz", hash = "sha256:ba29e2dfa0b8b556606f097407ed1aa62080ee108ab0dc5ec9d6a723a007d105"}, {file = "cachetools-5.3.3.tar.gz", hash = "sha256:ba29e2dfa0b8b556606f097407ed1aa62080ee108ab0dc5ec9d6a723a007d105"},
] ]
[[package]]
name = "celery"
version = "5.3.6"
description = "Distributed Task Queue."
optional = false
python-versions = ">=3.8"
files = [
{file = "celery-5.3.6-py3-none-any.whl", hash = "sha256:9da4ea0118d232ce97dff5ed4974587fb1c0ff5c10042eb15278487cdd27d1af"},
{file = "celery-5.3.6.tar.gz", hash = "sha256:870cc71d737c0200c397290d730344cc991d13a057534353d124c9380267aab9"},
]
[package.dependencies]
billiard = ">=4.2.0,<5.0"
click = ">=8.1.2,<9.0"
click-didyoumean = ">=0.3.0"
click-plugins = ">=1.1.1"
click-repl = ">=0.2.0"
kombu = ">=5.3.4,<6.0"
python-dateutil = ">=2.8.2"
tzdata = ">=2022.7"
vine = ">=5.1.0,<6.0"
[package.extras]
arangodb = ["pyArango (>=2.0.2)"]
auth = ["cryptography (==41.0.5)"]
azureblockblob = ["azure-storage-blob (>=12.15.0)"]
brotli = ["brotli (>=1.0.0)", "brotlipy (>=0.7.0)"]
cassandra = ["cassandra-driver (>=3.25.0,<4)"]
consul = ["python-consul2 (==0.1.5)"]
cosmosdbsql = ["pydocumentdb (==2.3.5)"]
couchbase = ["couchbase (>=3.0.0)"]
couchdb = ["pycouchdb (==1.14.2)"]
django = ["Django (>=2.2.28)"]
dynamodb = ["boto3 (>=1.26.143)"]
elasticsearch = ["elastic-transport (<=8.10.0)", "elasticsearch (<=8.11.0)"]
eventlet = ["eventlet (>=0.32.0)"]
gevent = ["gevent (>=1.5.0)"]
librabbitmq = ["librabbitmq (>=2.0.0)"]
memcache = ["pylibmc (==1.6.3)"]
mongodb = ["pymongo[srv] (>=4.0.2)"]
msgpack = ["msgpack (==1.0.7)"]
pymemcache = ["python-memcached (==1.59)"]
pyro = ["pyro4 (==4.82)"]
pytest = ["pytest-celery (==0.0.0)"]
redis = ["redis (>=4.5.2,!=4.5.5,<6.0.0)"]
s3 = ["boto3 (>=1.26.143)"]
slmq = ["softlayer-messaging (>=1.0.3)"]
solar = ["ephem (==4.1.5)"]
sqlalchemy = ["sqlalchemy (>=1.4.48,<2.1)"]
sqs = ["boto3 (>=1.26.143)", "kombu[sqs] (>=5.3.0)", "pycurl (>=7.43.0.5)", "urllib3 (>=1.26.16)"]
tblib = ["tblib (>=1.3.0)", "tblib (>=1.5.0)"]
yaml = ["PyYAML (>=3.10)"]
zookeeper = ["kazoo (>=1.3.1)"]
zstd = ["zstandard (==0.22.0)"]
[[package]]
name = "celery-types"
version = "0.22.0"
description = "Type stubs for Celery and its related packages"
optional = false
python-versions = ">=3.9,<4.0"
files = [
{file = "celery_types-0.22.0-py3-none-any.whl", hash = "sha256:79a66637d1d6af5992d1dc80259d9538869941325e966006f1e795220519b9ac"},
{file = "celery_types-0.22.0.tar.gz", hash = "sha256:0ecad2fa5a6eded0a1f919e5e1e381cc2ff0635fe4b21db53b4661b6876d5b30"},
]
[package.dependencies]
typing-extensions = ">=4.9.0,<5.0.0"
[[package]] [[package]]
name = "certifi" name = "certifi"
version = "2024.2.2" version = "2024.2.2"
@ -400,55 +306,6 @@ files = [
[package.dependencies] [package.dependencies]
colorama = {version = "*", markers = "platform_system == \"Windows\""} colorama = {version = "*", markers = "platform_system == \"Windows\""}
[[package]]
name = "click-didyoumean"
version = "0.3.0"
description = "Enables git-like *did-you-mean* feature in click"
optional = false
python-versions = ">=3.6.2,<4.0.0"
files = [
{file = "click-didyoumean-0.3.0.tar.gz", hash = "sha256:f184f0d851d96b6d29297354ed981b7dd71df7ff500d82fa6d11f0856bee8035"},
{file = "click_didyoumean-0.3.0-py3-none-any.whl", hash = "sha256:a0713dc7a1de3f06bc0df5a9567ad19ead2d3d5689b434768a6145bff77c0667"},
]
[package.dependencies]
click = ">=7"
[[package]]
name = "click-plugins"
version = "1.1.1"
description = "An extension module for click to enable registering CLI commands via setuptools entry-points."
optional = false
python-versions = "*"
files = [
{file = "click-plugins-1.1.1.tar.gz", hash = "sha256:46ab999744a9d831159c3411bb0c79346d94a444df9a3a3742e9ed63645f264b"},
{file = "click_plugins-1.1.1-py2.py3-none-any.whl", hash = "sha256:5d262006d3222f5057fd81e1623d4443e41dcda5dc815c06b442aa3c02889fc8"},
]
[package.dependencies]
click = ">=4.0"
[package.extras]
dev = ["coveralls", "pytest (>=3.6)", "pytest-cov", "wheel"]
[[package]]
name = "click-repl"
version = "0.3.0"
description = "REPL plugin for Click"
optional = false
python-versions = ">=3.6"
files = [
{file = "click-repl-0.3.0.tar.gz", hash = "sha256:17849c23dba3d667247dc4defe1757fff98694e90fe37474f3feebb69ced26a9"},
{file = "click_repl-0.3.0-py3-none-any.whl", hash = "sha256:fb7e06deb8da8de86180a33a9da97ac316751c094c6899382da7feeeeb51b812"},
]
[package.dependencies]
click = ">=7.0"
prompt-toolkit = ">=3.0.36"
[package.extras]
testing = ["pytest (>=7.2.1)", "pytest-cov (>=4.0.0)", "tox (>=4.4.3)"]
[[package]] [[package]]
name = "colorama" name = "colorama"
version = "0.4.6" version = "0.4.6"
@ -987,38 +844,6 @@ MarkupSafe = ">=2.0"
[package.extras] [package.extras]
i18n = ["Babel (>=2.7)"] i18n = ["Babel (>=2.7)"]
[[package]]
name = "kombu"
version = "5.3.5"
description = "Messaging library for Python."
optional = false
python-versions = ">=3.8"
files = [
{file = "kombu-5.3.5-py3-none-any.whl", hash = "sha256:0eac1bbb464afe6fb0924b21bf79460416d25d8abc52546d4f16cad94f789488"},
{file = "kombu-5.3.5.tar.gz", hash = "sha256:30e470f1a6b49c70dc6f6d13c3e4cc4e178aa6c469ceb6bcd55645385fc84b93"},
]
[package.dependencies]
amqp = ">=5.1.1,<6.0.0"
vine = "*"
[package.extras]
azureservicebus = ["azure-servicebus (>=7.10.0)"]
azurestoragequeues = ["azure-identity (>=1.12.0)", "azure-storage-queue (>=12.6.0)"]
confluentkafka = ["confluent-kafka (>=2.2.0)"]
consul = ["python-consul2"]
librabbitmq = ["librabbitmq (>=2.0.0)"]
mongodb = ["pymongo (>=4.1.1)"]
msgpack = ["msgpack"]
pyro = ["pyro4"]
qpid = ["qpid-python (>=0.26)", "qpid-tools (>=0.26)"]
redis = ["redis (>=4.5.2,!=4.5.5,<6.0.0)"]
slmq = ["softlayer-messaging (>=1.0.3)"]
sqlalchemy = ["sqlalchemy (>=1.4.48,<2.1)"]
sqs = ["boto3 (>=1.26.143)", "pycurl (>=7.43.0.5)", "urllib3 (>=1.26.16)"]
yaml = ["PyYAML (>=3.10)"]
zookeeper = ["kazoo (>=2.8.0)"]
[[package]] [[package]]
name = "lxml" name = "lxml"
version = "5.1.0" version = "5.1.0"
@ -1373,20 +1198,6 @@ requests = "*"
dev = ["black", "flake8", "therapist", "tox", "twine", "wheel"] dev = ["black", "flake8", "therapist", "tox", "twine", "wheel"]
test = ["mock", "nose"] test = ["mock", "nose"]
[[package]]
name = "prompt-toolkit"
version = "3.0.43"
description = "Library for building powerful interactive command lines in Python"
optional = false
python-versions = ">=3.7.0"
files = [
{file = "prompt_toolkit-3.0.43-py3-none-any.whl", hash = "sha256:a11a29cb3bf0a28a387fe5122cdb649816a957cd9261dcedf8c9f1fef33eacf6"},
{file = "prompt_toolkit-3.0.43.tar.gz", hash = "sha256:3527b7af26106cbc65a040bcc84839a3566ec1b051bb0bfe953631e704b0ff7d"},
]
[package.dependencies]
wcwidth = "*"
[[package]] [[package]]
name = "psycopg" name = "psycopg"
version = "3.1.18" version = "3.1.18"
@ -2251,17 +2062,6 @@ files = [
docs = ["Sphinx (>=4.1.2,<4.2.0)", "sphinx-rtd-theme (>=0.5.2,<0.6.0)", "sphinxcontrib-asyncio (>=0.3.0,<0.4.0)"] docs = ["Sphinx (>=4.1.2,<4.2.0)", "sphinx-rtd-theme (>=0.5.2,<0.6.0)", "sphinxcontrib-asyncio (>=0.3.0,<0.4.0)"]
test = ["Cython (>=0.29.36,<0.30.0)", "aiohttp (==3.9.0b0)", "aiohttp (>=3.8.1)", "flake8 (>=5.0,<6.0)", "mypy (>=0.800)", "psutil", "pyOpenSSL (>=23.0.0,<23.1.0)", "pycodestyle (>=2.9.0,<2.10.0)"] test = ["Cython (>=0.29.36,<0.30.0)", "aiohttp (==3.9.0b0)", "aiohttp (>=3.8.1)", "flake8 (>=5.0,<6.0)", "mypy (>=0.800)", "psutil", "pyOpenSSL (>=23.0.0,<23.1.0)", "pycodestyle (>=2.9.0,<2.10.0)"]
[[package]]
name = "vine"
version = "5.1.0"
description = "Python promises."
optional = false
python-versions = ">=3.6"
files = [
{file = "vine-5.1.0-py3-none-any.whl", hash = "sha256:40fdf3c48b2cfe1c38a49e9ae2da6fda88e4794c810050a728bd7413811fb1dc"},
{file = "vine-5.1.0.tar.gz", hash = "sha256:8b62e981d35c41049211cf62a0a1242d8c1ee9bd15bb196ce38aefd6799e61e0"},
]
[[package]] [[package]]
name = "virtualenv" name = "virtualenv"
version = "20.25.1" version = "20.25.1"
@ -2369,17 +2169,6 @@ files = [
[package.dependencies] [package.dependencies]
anyio = ">=3.0.0" anyio = ">=3.0.0"
[[package]]
name = "wcwidth"
version = "0.2.13"
description = "Measures the displayed width of unicode strings in a terminal"
optional = false
python-versions = "*"
files = [
{file = "wcwidth-0.2.13-py2.py3-none-any.whl", hash = "sha256:3da69048e4540d84af32131829ff948f1e022c1c6bdb8d6102117aac784f6859"},
{file = "wcwidth-0.2.13.tar.gz", hash = "sha256:72ea0c06399eb286d978fdedb6923a9eb47e1c486ce63e9b4e64fc18303972b5"},
]
[[package]] [[package]]
name = "websockets" name = "websockets"
version = "12.0" version = "12.0"
@ -2464,4 +2253,4 @@ files = [
[metadata] [metadata]
lock-version = "2.0" lock-version = "2.0"
python-versions = "^3.10" python-versions = "^3.10"
content-hash = "83804a2bfa0cb1d08b45695bd9bfe9e9a79764f4622ab1902aaed0805f77ae25" content-hash = "d1bf7cce2d9eb6d62051719a80e14eeb127722d116af04938775db6cbafe40cf"

2
backend/pyproject.toml

@ -10,7 +10,6 @@ uvicorn = {extras = ["standard"], version = "^0.24.0.post1"}
fastapi = "^0.109.1" fastapi = "^0.109.1"
python-multipart = "^0.0.7" python-multipart = "^0.0.7"
email-validator = "^2.1.0.post1" email-validator = "^2.1.0.post1"
celery = "^5.3.5"
passlib = {extras = ["bcrypt"], version = "^1.7.4"} passlib = {extras = ["bcrypt"], version = "^1.7.4"}
tenacity = "^8.2.3" tenacity = "^8.2.3"
pydantic = ">2.0" pydantic = ">2.0"
@ -37,7 +36,6 @@ pre-commit = "^3.6.2"
pytest-mock = "^3.12.0" pytest-mock = "^3.12.0"
types-python-jose = "^3.3.4.20240106" types-python-jose = "^3.3.4.20240106"
types-passlib = "^1.7.7.20240106" types-passlib = "^1.7.7.20240106"
celery-types = "^0.22.0"
[tool.isort] [tool.isort]
multi_line_output = 3 multi_line_output = 3

6
backend/worker-start.sh

@ -1,6 +0,0 @@
#! /usr/bin/env bash
set -e
python /app/app/celeryworker_pre_start.py
celery -A app.worker worker -l info -Q main-queue -c 1

5
copier.yml

@ -59,11 +59,6 @@ sentry_dsn:
help: The DSN for Sentry, if you are using it, you can set it later in .env help: The DSN for Sentry, if you are using it, you can set it later in .env
default: "" default: ""
flower_basic_auth:
type: str
help: The basic auth for Flower, you can set it later in .env
default: "admin:changethis"
_exclude: _exclude:
# Global # Global
- .vscode - .vscode

5
deployment.md

@ -142,7 +142,6 @@ You can set several variables, like:
* `POSTGRES_USER`: The Postgres user, you can leave the default. * `POSTGRES_USER`: The Postgres user, you can leave the default.
* `POSTGRES_DB`: The database name to use for this application. You can leave the default of `app`. * `POSTGRES_DB`: The database name to use for this application. You can leave the default of `app`.
* `SENTRY_DSN`: The DSN for Sentry, if you are using it. * `SENTRY_DSN`: The DSN for Sentry, if you are using it.
* `FLOWER_BASIC_AUTH`: The HTTP Basic Auth for Flower, like `admin:changethis`.
### Generate secret keys ### Generate secret keys
@ -281,8 +280,6 @@ Backend API base URL: `https://fastapi-project.example.com/api/`
Adminer: `https://adminer.fastapi-project.example.com` Adminer: `https://adminer.fastapi-project.example.com`
Flower: `https://flower.fastapi-project.example.com`
### Staging ### Staging
Frontend: `https://staging.fastapi-project.example.com` Frontend: `https://staging.fastapi-project.example.com`
@ -292,5 +289,3 @@ Backend API docs: `https://staging.fastapi-project.example.com/docs`
Backend API base URL: `https://staging.fastapi-project.example.com/api/` Backend API base URL: `https://staging.fastapi-project.example.com/api/`
Adminer: `https://staging.adminer.fastapi-project.example.com` Adminer: `https://staging.adminer.fastapi-project.example.com`
Flower: `https://staging.flower.fastapi-project.example.com`

4
development.md

@ -109,8 +109,6 @@ Automatic Alternative Docs (ReDoc): https://localhost/redoc
Adminer: http://localhost:8080 Adminer: http://localhost:8080
Flower: http://localhost:5555
Traefik UI: http://localhost:8090 Traefik UI: http://localhost:8090
### Development in localhost with a custom domain URLs ### Development in localhost with a custom domain URLs
@ -127,6 +125,4 @@ Automatic Alternative Docs (ReDoc): https://localhost.tiangolo.com/redoc
Adminer: http://localhost.tiangolo.com:8080 Adminer: http://localhost.tiangolo.com:8080
Flower: http://localhost.tiangolo.com:5555
Traefik UI: http://localhost.tiangolo.com:8090 Traefik UI: http://localhost.tiangolo.com:8090

26
docker-compose.override.yml

@ -48,19 +48,6 @@ services:
ports: ports:
- "8080:8080" - "8080:8080"
queue:
restart: "no"
ports:
- "5671:5671"
- "5672:5672"
- "15672:15672"
- "15671:15671"
flower:
restart: "no"
ports:
- "5555:5555"
backend: backend:
restart: "no" restart: "no"
ports: ports:
@ -69,24 +56,11 @@ services:
- ./backend/:/app - ./backend/:/app
build: build:
context: ./backend context: ./backend
dockerfile: backend.dockerfile
args: args:
INSTALL_DEV: ${INSTALL_DEV-true} INSTALL_DEV: ${INSTALL_DEV-true}
# command: sleep infinity # Infinite loop to keep container alive doing nothing # command: sleep infinity # Infinite loop to keep container alive doing nothing
command: /start-reload.sh command: /start-reload.sh
celeryworker:
restart: "no"
volumes:
- ./backend/:/app
environment:
- RUN=celery worker -A app.worker -l info -Q main-queue -c 1
build:
context: ./backend
dockerfile: celeryworker.dockerfile
args:
INSTALL_DEV: ${INSTALL_DEV-true}
frontend: frontend:
restart: "no" restart: "no"
build: build:

69
docker-compose.yml

@ -36,44 +36,6 @@ services:
- traefik.http.routers.${STACK_NAME?Variable not set}-adminer-https.tls.certresolver=le - traefik.http.routers.${STACK_NAME?Variable not set}-adminer-https.tls.certresolver=le
- traefik.http.services.${STACK_NAME?Variable not set}-adminer.loadbalancer.server.port=8080 - traefik.http.services.${STACK_NAME?Variable not set}-adminer.loadbalancer.server.port=8080
queue:
image: rabbitmq:3
# Using the below image instead is required to enable the "Broker" tab in the flower UI:
# image: rabbitmq:3-management
#
# You also have to change the flower command
restart: always
flower:
image: mher/flower:2.0
restart: always
networks:
- traefik-public
- default
env_file:
- .env
environment:
- FLOWER_BASIC_AUTH=${FLOWER_BASIC_AUTH}
command:
- celery
- "--broker=amqp://guest@queue:5672//"
- flower
# For the "Broker" tab to work in the flower UI, uncomment the following command argument,
# and change the queue service's image as well
# - "--broker_api=http://guest:guest@queue:15672/api//"
labels:
- traefik.enable=true
- traefik.docker.network=traefik-public
- traefik.constraint-label=traefik-public
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-http.rule=Host(`flower.${DOMAIN?Variable not set}`)
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-http.entrypoints=http
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-http.middlewares=https-redirect
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.rule=Host(`flower.${DOMAIN?Variable not set}`)
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.entrypoints=https
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.tls=true
- traefik.http.routers.${STACK_NAME?Variable not set}-flower-https.tls.certresolver=le
- traefik.http.services.${STACK_NAME?Variable not set}-flower.loadbalancer.server.port=5555
backend: backend:
image: '${DOCKER_IMAGE_BACKEND?Variable not set}:${TAG-latest}' image: '${DOCKER_IMAGE_BACKEND?Variable not set}:${TAG-latest}'
restart: always restart: always
@ -105,7 +67,6 @@ services:
build: build:
context: ./backend context: ./backend
dockerfile: backend.dockerfile
args: args:
INSTALL_DEV: ${INSTALL_DEV-false} INSTALL_DEV: ${INSTALL_DEV-false}
labels: labels:
@ -125,36 +86,6 @@ services:
- traefik.http.routers.${STACK_NAME?Variable not set}-backend-http.middlewares=https-redirect,${STACK_NAME?Variable not set}-www-redirect - traefik.http.routers.${STACK_NAME?Variable not set}-backend-http.middlewares=https-redirect,${STACK_NAME?Variable not set}-www-redirect
- traefik.http.routers.${STACK_NAME?Variable not set}-backend-https.middlewares=${STACK_NAME?Variable not set}-www-redirect - traefik.http.routers.${STACK_NAME?Variable not set}-backend-https.middlewares=${STACK_NAME?Variable not set}-www-redirect
celeryworker:
image: '${DOCKER_IMAGE_CELERYWORKER?Variable not set}:${TAG-latest}'
restart: always
depends_on:
- db
- queue
env_file:
- .env
environment:
- DOMAIN=${DOMAIN}
- ENVIRONMENT=${ENVIRONMENT}
- BACKEND_CORS_ORIGINS=${BACKEND_CORS_ORIGINS}
- SECRET_KEY=${SECRET_KEY?Variable not set}
- FIRST_SUPERUSER=${FIRST_SUPERUSER?Variable not set}
- FIRST_SUPERUSER_PASSWORD=${FIRST_SUPERUSER_PASSWORD?Variable not set}
- USERS_OPEN_REGISTRATION=${USERS_OPEN_REGISTRATION}
- SMTP_HOST=${SMTP_HOST}
- SMTP_USER=${SMTP_USER}
- SMTP_PASSWORD=${SMTP_PASSWORD}
- EMAILS_FROM_EMAIL=${EMAILS_FROM_EMAIL}
- POSTGRES_SERVER=db
- POSTGRES_DB=${POSTGRES_DB}
- POSTGRES_USER=${POSTGRES_USER?Variable not set}
- POSTGRES_PASSWORD=${POSTGRES_PASSWORD?Variable not set}
- SENTRY_DSN=${SENTRY_DSN}
build:
context: ./backend
dockerfile: celeryworker.dockerfile
args:
INSTALL_DEV: ${INSTALL_DEV-false}
frontend: frontend:
image: '${DOCKER_IMAGE_FRONTEND?Variable not set}:${TAG-latest}' image: '${DOCKER_IMAGE_FRONTEND?Variable not set}:${TAG-latest}'

Loading…
Cancel
Save