Skip to content
DalgoT4DPublic

About

Dalgo — the open-source data stack built for the social sector: ingest, orchestrate, transform, analyse.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Dalgo Monorepo

Monorepo for all Dalgo services. Service code lives in top-level directories; all Docker configuration is centralised under docker/.

License

The entire repository — code, configuration, and documentation — is licensed under the GNU Affero General Public License v3.0 (see LICENSE). If you run a modified version of Dalgo as a networked service, you must make the modified source available to its users under the same license.

Coverage

Service Coverage
Backend backend coverage
Webapp webapp coverage
Prefect Proxy prefect-proxy coverage
AI LLM Service ai-llm-service coverage
dalgo/
├── backend/           Django/Ninja API + Celery workers
├── webapp/            Next.js frontend
├── prefect-proxy/     FastAPI proxy for Prefect orchestration
├── ai-llm-service/    FastAPI LLM service + Celery worker
├── docker/
│   ├── Dockerfile.backend
│   ├── Dockerfile.backend.dev
│   ├── Dockerfile.webapp
│   ├── Dockerfile.prefect-proxy
│   ├── Dockerfile.prefect-proxy.dev
│   ├── Dockerfile.ai-llm-service
│   ├── docker-compose.yml        ← production
│   └── docker-compose.dev.yml    ← local dev
└── scripts/
    ├── install-hooks.sh  ← run once after cloning to install git hooks
    └── hooks/pre-commit  ← the hook (black + prettier + eslint on staged files)

Contributing

One-time setup after cloning

./scripts/install-hooks.sh

This installs a git pre-commit hook that runs automatically on every git commit:

Service Tool Behaviour
backend/ black auto-formats and re-stages
prefect-proxy/ black auto-formats and re-stages
webapp/ prettier auto-formats and re-stages
webapp/ eslint check only — fix errors manually

Auto-formatting: black and prettier rewrite files in place and re-stage them, so your commit always lands clean. If prettier or black reformats something, just re-run git commit — the second attempt will pass.

ESLint errors: fix them manually, then commit again. Warnings are also blocked (--max-warnings=0) to stay in sync with CI.

Requirements

  • uv — for backend/prefect-proxy black (install)
  • node + npm — for webapp prettier/eslint (node_modules must be installed: cd webapp && npm install)

Docker setup

All compose commands run from the docker/ directory.

Production

cd docker/

# Copy and fill in env files for each service
cp ../backend/.env.template ../backend/.env        # edit with prod values
cp ../prefect-proxy/.env.template ../prefect-proxy/.env
cp ../webapp/.env.example ../webapp/.env
cp ../ai-llm-service/.env.example ../ai-llm-service/.env

# Build and start
docker compose build
docker compose up -d

# Single service
docker compose up -d backend
docker compose logs -f backend

Local dev (hot reload)

cd docker/

# Same env setup as above (point DBHOST at your local Postgres)
docker compose -f docker-compose.dev.yml build
docker compose -f docker-compose.dev.yml up -d

# Rebuild after a dependency change (pyproject.toml / uv.lock)
docker compose -f docker-compose.dev.yml build backend
docker compose -f docker-compose.dev.yml up -d --no-deps backend

Notes:

  • Postgres runs on the host; backend containers reach it via host.docker.internal.
  • Prefect server starts as a compose service in dev; in prod it is external.
  • Next.js webapp is built as a standalone image in both modes (no hot reload — rebuild on code change).

Adding / changing Docker config

All Dockerfiles and compose files live in docker/. Changes here do not affect the individual repos.

File Purpose
Dockerfile.backend Production image for backend + celery workers
Dockerfile.backend.dev Dev image (venv at /opt/venv, bind-mount code)
Dockerfile.prefect-proxy Production prefect-proxy image
Dockerfile.prefect-proxy.dev Dev image with hot reload
Dockerfile.webapp Next.js standalone image (used in both prod and dev)
Dockerfile.ai-llm-service Production + dev image for LLM service
docker-compose.yml Full prod stack
docker-compose.dev.yml Dev stack with bind mounts and Prefect server

About

Dalgo — the open-source data stack built for the social sector: ingest, orchestrate, transform, analyse.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages