Skip to content

backend.md

Latest commit

 

History

History
379 lines (321 loc) · 12.7 KB

backend.md

File metadata and controls

379 lines (321 loc) · 12.7 KB
Raw

Backend Documentation

Overview

The backend is a FastAPI application providing REST APIs for the Photobook system. It handles authentication, image management, Knowledge Graph operations, and orchestrates background processing via external ML services.

Directory Structure

backend/
├── main.py                  # FastAPI app factory and health endpoints
├── alembic.ini             # Database migration config
├── requirements.txt        # Python dependencies
├── pytest.ini              # Test configuration
│
├── api/
│   └── v1/
│       ├── router.py       # Main API router aggregating all sub-routers
│       ├── auth.py         # Authentication endpoints (login, register, token)
│       └── settings.py     # Application settings endpoints
│
├── core/
│   ├── config.py           # Pydantic Settings for env vars
│   ├── logging.py          # Structured logging with structlog
│   ├── security.py         # JWT authentication utilities
│   ├── circuit_breaker.py  # Circuit breaker pattern for external services
│   └── retry.py            # Exponential backoff retry logic
│
├── db/
│   ├── base.py             # SQLAlchemy Base class
│   ├── models.py           # User and Photobook models
│   └── session.py          # Database session management (async)
│
├── images/
│   ├── router.py           # Image upload/list/delete endpoints
│   ├── service.py          # Image business logic (save, thumbnail, EXIF)
│   └── schemas.py          # Pydantic models for requests/responses
│
├── photobooks/
│   ├── router.py           # Photobook CRUD endpoints
│   ├── service.py          # Photobook business logic
│   └── schemas.py          # Pydantic schemas
│
├── persons/
│   ├── router.py           # Person CRUD, face association endpoints
│   ├── service.py          # Person clustering, face matching logic
│   └── schemas.py          # Person schemas
│
├── objects/
│   ├── router.py           # Object query endpoints
│   ├── service.py          # Object detection aggregation
│   └── schemas.py          # Object schemas
│
├── detections/
│   ├── router.py           # Raw detection endpoints
│   ├── service.py          # Detection operations
│   └── schemas.py          # Detection schemas
│
├── captions/
│   ├── router.py           # Caption generation endpoints
│   ├── service.py          # Caption logic with VLM
│   └── schemas.py          # Caption schemas
│
├── kg/                      # Knowledge Graph module
│   ├── models.py           # Node, Claim, Evidence, UserAction models
│   ├── enums.py            # NodeType, ClaimStatus, SourceType enums
│   ├── service.py          # KG CRUD operations
│   ├── router.py           # KG query/mutation endpoints
│   └── schemas.py          # KG Pydantic schemas
│
├── jobs/                    # Background job system
│   ├── queue.py            # arq job queue setup
│   ├── worker.py           # Worker configuration and settings
│   ├── router.py           # Job status endpoints
│   ├── monitor.py          # Job monitoring utilities
│   └── tasks/
│       ├── vlm.py          # VLM analysis tasks
│       ├── location.py     # Geo resolution tasks
│       └── face.py         # Face embedding/matching tasks
│
└── alembic/
    ├── env.py              # Alembic environment
    └── versions/           # Database migrations

API Endpoints

Authentication (/api/v1/auth)

Method Path Description
POST /register Create new user account
POST /login Get access token (JWT)
GET /me Get current user info

Images (/api/v1/images)

Method Path Description
POST / Upload image(s)
GET / List user's images
GET /{id} Get image details
DELETE /{id} Delete image and related data
POST /{id}/analyze Trigger VLM analysis
POST /{id}/caption Generate caption for image

Photobooks (/api/v1/photobooks)

Method Path Description
POST / Create photobook
GET / List photobooks
GET /{id} Get photobook details
PUT /{id} Update photobook
DELETE /{id} Delete photobook
POST /{id}/images Add images to photobook
DELETE /{id}/images/{img} Remove image from photobook

Persons (/api/v1/persons)

Method Path Description
GET / List known persons
GET /{id} Get person details
PUT /{id} Update person (name)
DELETE /{id} Delete person
GET /{id}/faces Get all face crops for person
GET /{id}/images Get image IDs containing person
POST /merge Merge two persons
GET /unidentified-faces Get unassigned face detections
POST /assign-face Assign face to person

Objects (/api/v1/objects)

Method Path Description
GET / List detected objects
GET /{id} Get object details
GET /by-type/{type} Get objects by category
GET /image/{image_id} Get objects in specific image

Knowledge Graph (/api/v1/kg)

Method Path Description
GET /nodes List nodes (paginated)
GET /nodes/{id} Get node with claims
PUT /nodes/{id} Update node
GET /claims List claims
PUT /claims/{id} Update claim
POST /claims/{id}/accept Accept a claim
POST /claims/{id}/reject Reject a claim
GET /graph Get graph data for visualization

Jobs (/api/v1/jobs)

Method Path Description
GET / List jobs (pending/completed)
GET /{job_id} Get job status
POST /cancel/{job_id} Cancel pending job
GET /stats Get job queue statistics

Core Components

Configuration (core/config.py)

Uses Pydantic Settings with .env file support:

from core.config import settings

# Access settings
db_url = settings.database_url
vlm_url = settings.vlm_service_url

Key settings:

  • DATABASE_URL - PostgreSQL connection string
  • REDIS_URL - Redis for job queue
  • SECRET_KEY - JWT signing key
  • VLM_SERVICE_URL / GEO_SERVICE_URL / FACE_SERVICE_URL - ML service URLs
  • GOOGLE_PLACES_API_KEY - For geo resolution

Authentication (core/security.py)

JWT-based authentication:

from core.security import get_current_user

@router.get("/protected")
async def protected_route(user: User = Depends(get_current_user)):
    return {"user_id": user.id}

Circuit Breaker (core/circuit_breaker.py)

Protects external service calls:

from core.circuit_breaker import get_vlm_circuit

circuit = get_vlm_circuit()
if circuit.is_open:
    # Fast-fail, service is unavailable
    raise ServiceUnavailable()

try:
    result = await call_vlm_service()
    circuit.record_success()
except Exception as e:
    circuit.record_failure(e)
    raise

Configuration per circuit:

  • failure_threshold - Failures before opening
  • recovery_timeout - Seconds to wait before half-open
  • half_open_max_calls - Test calls in half-open state

Retry Logic (core/retry.py)

Exponential backoff for transient failures:

from core.retry import RetryWithBackoff

retrier = RetryWithBackoff(
    max_retries=3,
    base_delay=1.0,
    max_delay=30.0,
    jitter=0.2,
)

for attempt in retrier:
    try:
        result = await some_operation()
        break
    except TransientError as e:
        retrier.failed(e)
        if retrier.should_retry:
            await retrier.wait()
        else:
            raise

Database Session

Async SQLAlchemy with connection pooling:

from db.session import get_session

@router.get("/items")
async def get_items(session: AsyncSession = Depends(get_session)):
    result = await session.execute(select(Node))
    return result.scalars().all()

Background Jobs

Enqueueing Jobs

from jobs.queue import get_job_queue

async def enqueue_analysis(image_node_id: str, image_path: str, user_id: int):
    queue = await get_job_queue()
    await queue.enqueue_job(
        "vlm_extract_all_task",
        image_node_id,
        image_path,
        user_id,
    )

Worker Process

Run the worker:

cd backend
python -m arq jobs.worker.WorkerSettings

Available Tasks

Task Description
vlm_extract_all_task Unified VLM extraction (preferred)
vlm_scene_graph_task Scene graph relationship extraction
vlm_detect_objects_task Object bounding box detection
vlm_detect_people_task People bounding box detection
vlm_detect_faces_task Face bounding box detection
location_match_task GPS → Place resolution
face_embed_task Face embedding extraction
face_match_task Match face to known persons
face_cluster_task Cluster unknown faces

Service Layer Pattern

Each domain has a service class encapsulating business logic:

# images/service.py
class ImageService:
    def __init__(self, session: AsyncSession):
        self.session = session

    async def save_upload(self, file: UploadFile, user_id: int) -> tuple:
        # 1. Generate unique filename
        # 2. Save to disk
        # 3. Generate thumbnail
        # 4. Extract EXIF
        return filename, file_path, thumb_path

    async def create_image_node(self, user_id: int, ...) -> Node:
        # Create IMAGE node in KG
        ...

Error Handling

Standard HTTPException patterns:

from fastapi import HTTPException, status

# Not found
raise HTTPException(
    status_code=status.HTTP_404_NOT_FOUND,
    detail="Image not found"
)

# Validation error
raise HTTPException(
    status_code=status.HTTP_400_BAD_REQUEST,
    detail="Invalid image format"
)

# Authentication error
raise HTTPException(
    status_code=status.HTTP_401_UNAUTHORIZED,
    detail="Invalid credentials"
)

Testing

Run tests:

cd backend
pytest
pytest tests/api/  # API tests only
pytest -v --tb=short  # Verbose with short tracebacks

Test fixtures in tests/conftest.py:

  • db_session - Test database session
  • client - TestClient for API tests
  • auth_headers - Authenticated request headers
  • sample_image - Test image fixture

Development

Running

cd backend
source .venv/bin/activate
uvicorn main:app --reload --port 8000

Database Migrations

# Create migration
alembic revision --autogenerate -m "Add new table"

# Apply migrations
alembic upgrade head

# Rollback
alembic downgrade -1

Adding a New Endpoint

  1. Create/update {module}/router.py with route handlers
  2. Create/update {module}/schemas.py for Pydantic models
  3. Create/update {module}/service.py for business logic
  4. Register router in api/v1/router.py: 
    from mymodule.router import router as mymodule_router
api_router.include_router(mymodule_router)