Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 35 additions & 2 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,40 @@ If any hooks fail, please fix the issues before committing. You can manually run
prek run --all-files
```

### Tasks (via poe)

Every routine check, build and test step is a [poe](https://poethepoet.natn.io/) task.
Tasks run through poe's `uv` executor, so they resolve their own environment - no `uv run --group ...` prefix needed.
The bare name is the variant CI runs; a colon suffix selects a mode (`:fix` for auto-fix, `:dev` for the local loop):

```bash
uv sync # create the environment
uv run poe check # format + lint + typecheck + lint-imports + deptry (what CI runs)
uv run poe format # ruff format --check
uv run poe format:fix # ruff format
uv run poe lint # ruff check
uv run poe lint:fix # ruff check --fix
uv run poe lint:md # markdownlint over all Markdown (via prek)
uv run poe typecheck # ty check
uv run poe lint-imports # import-linter layer contracts
uv run poe deptry # dependency hygiene
uv run poe tests # pytest with coverage
uv run poe docs # strict docs site build (what docs CI runs)
uv run poe docs:dev # serve the docs site with live reload
```

Run `uv run poe check` and `uv run poe tests` before pushing - they are the same checks the pull request runs.
Markdown is not in `poe check` - the prek hook lints it on every commit and in CI; `uv run poe lint:md` runs it on demand.
`poe check` does not build the documentation site either, so run `uv run poe docs` when the change touches `docs/`.

### Tests

`uv run poe tests` runs the suite; CI runs it on Python 3.13, 3.14 and 3.15.
`tests/conftest.py` patches the pyinfra context so rune and operation calls are no-ops - molds, smithy helpers, runes, and templates unit-test without a real target.
End-to-end correctness is still validated by deploying to a real host.

The suite also enforces the [conventions](https://wlix13.github.io/NullForge/contributing/conventions/).

## Commits

All commits are expected to follow the conventional commits specification.
Expand Down Expand Up @@ -113,8 +147,7 @@ message — please replace that with something descriptive. Examples:
- `Improve CI/CD caching and test matrix`
- `Fix swap sizing on small hosts`

A good title reads cleanly in the PR list and the project history. The individual
**commits** inside the PR still follow Conventional Commits (see [Commits](#commits));
The individual **commits** inside the PR still follow Conventional Commits (see [Commits](#commits));
because PRs are merged with a merge/rebase strategy, those commit messages — not the
title — drive the release changelog, so the title itself does not need a `type:` prefix.

Expand Down
9 changes: 6 additions & 3 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,8 @@ updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
interval: "cron"
cronjob: "0 6 1,15 * *" # twice a month, closest dependabot gets to every two weeks
commit-message:
prefix: "chore"
include: "scope"
Expand All @@ -17,7 +18,8 @@ updates:
- package-ecosystem: "uv"
directory: "/"
schedule:
interval: "weekly"
interval: "cron"
cronjob: "0 6 1,15 * *" # twice a month, closest dependabot gets to every two weeks
commit-message:
prefix: "chore"
include: "scope"
Expand All @@ -31,7 +33,8 @@ updates:
- package-ecosystem: "pre-commit"
directory: "/"
schedule:
interval: "weekly"
interval: "cron"
cronjob: "0 6 1,15 * *" # twice a month, closest dependabot gets to every two weeks
commit-message:
prefix: "chore"
include: "scope"
Expand Down
22 changes: 4 additions & 18 deletions .github/workflows/ci-code-quality.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,27 +25,13 @@ jobs:
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: true
cache-dependency-glob: uv.lock

- run: uv sync --all-groups

- name: Check formatting
run: uv run ruff format --check .

- name: Lint code
run: uv run ruff check .

- name: Type check
run: uv run ty check

- name: Check import contracts
run: uv run lint-imports

- name: Check dependency hygiene
run: uv run deptry .
- name: Run quality checks
run: uv run poe check

hooks:
name: 🪝 prek hooks
Expand All @@ -56,7 +42,7 @@ jobs:
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: true
cache-dependency-glob: uv.lock
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/ci-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,10 +28,10 @@ jobs:
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: true
cache-dependency-glob: uv.lock

- name: Build documentation
run: uv run --group docs zensical build --clean --strict
run: uv run poe docs
17 changes: 11 additions & 6 deletions .github/workflows/ci-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,27 @@ concurrency:

jobs:
tests:
name: 🧪 Tests
name: 🧪 Tests (py${{ matrix.python-version }})
runs-on: ubuntu-latest
timeout-minutes: 15
strategy:
fail-fast: false
matrix:
python-version: ["3.13", "3.14", "3.15"]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: true
cache-dependency-glob: uv.lock
python-version: "3.13"
cache-suffix: py${{ matrix.python-version }}
python-version: ${{ matrix.python-version }}

- name: Run tests with coverage
run: uv run --group tests pytest
- name: Run tests
run: uv run poe tests

build:
name: 📦 Package build check
Expand All @@ -43,7 +48,7 @@ jobs:
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: true
cache-dependency-glob: uv.lock
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy-docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,13 @@ jobs:
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: true
cache-dependency-glob: uv.lock

- name: Build documentation
run: uv run --group docs zensical build --clean --strict
run: uv run poe docs

- uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0

Expand Down
38 changes: 36 additions & 2 deletions .github/workflows/release-please.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
with:
persist-credentials: false

- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: false

Expand Down Expand Up @@ -111,7 +111,9 @@ jobs:
permissions:
id-token: write
steps:
- uses: astral-sh/setup-uv@20cfd1bf945f4377ade1205e4dbc17946fc9a30d # v10.0.1
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: false

- name: Download distribution artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
Expand All @@ -122,6 +124,38 @@ jobs:
- name: Publish to PyPI
run: uv publish --trusted-publishing always

publish-photon:
name: 📤 Publish to photon-manifest
needs: [release-please, build, attest-artifacts]
if: needs.release-please.outputs.release_created == 'true'
runs-on: ubuntu-latest
timeout-minutes: 10
environment:
name: photon
url: https://py.wlix13.dev/simple/nullforge/
permissions:
contents: read
steps:
- uses: astral-sh/setup-uv@bec219d24cd3e171d82865faccec33120bb574f4 # v10.1.0
with:
enable-cache: false

- name: Download distribution artifacts
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: python-package-distributions
path: dist/

- name: Publish to photon-manifest
env:
UV_PUBLISH_TOKEN: ${{ secrets.PHOTON_PUBLISH_TOKEN }}
run: |
uv auth login https://py.wlix13.dev/simple/ --token "$UV_PUBLISH_TOKEN"
uv publish \
--publish-url https://py.wlix13.dev/legacy/ \
--check-url https://py.wlix13.dev/simple/ \
--no-attestations

finalize-release:
name: 🚀 Attach assets and publish
needs: [release-please, build, attest-artifacts, publish-pypi]
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ default_install_hook_types:
- post-rewrite
repos:
- repo: https://github.com/tox-dev/pyproject-fmt
rev: v2.29.3
rev: v2.29.4
hooks:
- id: pyproject-fmt

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# NullForge

[![PyPI](https://img.shields.io/pypi/v/nullforge?logo=pypi&logoColor=white)](https://pypi.org/project/nullforge/)
![Python](https://img.shields.io/badge/python-3.13-blue?logo=python&logoColor=white)
![Python](https://img.shields.io/badge/python-3.13%20%7C%203.14%20%7C%203.15-blue?logo=python&logoColor=white)
![Build](https://img.shields.io/github/actions/workflow/status/wlix13/NullForge/ci-tests.yaml?label=build&logo=github)
![Lint](https://img.shields.io/github/actions/workflow/status/wlix13/NullForge/ci-code-quality.yaml?label=lint&logo=github)
![License](https://img.shields.io/badge/license-MIT-green)
Expand Down
2 changes: 1 addition & 1 deletion docs/concepts/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,4 @@ The [conventions](../contributing/conventions.md) around `host.loop` and change
## The CLI wrapper

`nullforge cast` is a thin planner around pyinfra: it resolves the [cast stages](../getting-started/cli.md#stages) and hands them to pyinfra.
Everything else - connections, facts, operations, parallelism - is stock pyinfra, which is why unknown CLI options are proxied through verbatim.
Everything else is stock pyinfra, which is why unknown CLI options are proxied through verbatim.
17 changes: 2 additions & 15 deletions docs/concepts/inventories.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,20 +44,15 @@ common = (
features = merge_features(BASE_FEATURES, *common, {"dns": {"ecs": True}})
```

The set of allowed sub-mold types is derived from `FeaturesMold.model_fields`, so a new feature is mergeable without touching the merge logic.

!!! warning "Deep-merge is per-key"

Dictionaries merge recursively, but any non-dict value - including lists - replaces the previous value outright.
A layer that sets `netsec.firewall_rules` replaces the whole rule list; extend `BASE_FEATURES.netsec.firewall_rules` in Python if you want "default rules plus mine".

Because inventories are Python, you can factor shared preset tuples, per-environment modules, or host loops however you like - the only contract is the final `hosts` list.

## Validation and coercion

The foundry coerces whatever the inventory provided - `None`, a `dict`, or a mold instance - into validated `SystemMold` / `FeaturesMold` objects before any rune runs (`ensure_system` / `ensure_features`).
The foundry coerces whatever the inventory provided - `None`, a `dict`, or a mold instance - into validated `SystemMold` / `FeaturesMold` objects before any rune runs.
Missing keys fall back to mold defaults; unknown keys are rejected because every mold forbids extra fields.
A typo like `{"warp": {"instal": True}}` fails the cast at validation time, before anything touches a host.

## Scaling to a fleet

Expand Down Expand Up @@ -143,20 +138,12 @@ for zone in ZONE_FEATURES:
nullforge cast -i inventory/main.py --limit web --dry
```

Details worth stealing:

- `entry.get("overrides")` is either a dict fragment or `None` - both are valid `merge_features` layers, so per-host overrides cost one line and still go through mold validation.
- Extra data keys (like `zone` above) ride along on `host.data` untouched; [custom runes](../guides/custom-runes.md) can branch on them.
- Adding a host is a YAML edit, reviewable in a PR and scriptable from CI.

This is the pattern behind the deploy repo of the **Conglomerate** proxy fleet: zone presets over NullForge molds, a YAML host registry edited from CI workflows, and per-zone `--limit` casts.

## Secrets in inventories

Inventories are code; secrets in them (tunnel tokens, proxy user secrets) end up on disk.
Keep real inventories out of public repos, or load secrets from the environment.

Mold fields marked sensitive (user password, Zero Trust token, Nezha secrets, Telemt users) are redacted as `***` in pyinfra's `--debug-inventory` output, so inspecting a plan does not leak them.
Sensitive mold fields are redacted as `***` in pyinfra's `--debug-inventory` output, so inspecting a plan does not leak them.

## Debugging

Expand Down
21 changes: 2 additions & 19 deletions docs/concepts/molds.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,31 +9,15 @@ Every mold extends `BaseMold` (`nullforge/molds/base_mold.py`), which sets the s

- **`extra="forbid"`** - unknown keys are validation errors, so typos fail the cast instead of silently deploying defaults.
- **`is_active`** - the activation protocol used for [rune dispatch](runes.md#dispatch); each feature sub-mold implements it.
- **`to_json()`** - JSON-mode serialization for pyinfra's `--debug-inventory`, with `_sensitive_fields` values redacted as `***` (recursively, through nested molds, lists and dicts).
- **`to_json()`** - JSON-mode serialization for pyinfra's `--debug-inventory`, with `_sensitive_fields` values redacted as `***`.

## The top-level molds

Two molds cover a host:

- **`SystemMold`** - base system state: packages, locales, timezone, hostname, swap, IPv6.
Consumed by the always-on [base rune](../features/base.md).
- **`FeaturesMold`** - one field per feature, each a sub-mold with its own defaults:

```python
class FeaturesMold(BaseMold):
warp: WarpMold = Field(default_factory=WarpMold)
dns: DnsMold = Field(default_factory=DnsMold)
users: UserMold = Field(default_factory=UserMold)
netsec: NetSecMold = Field(default_factory=NetSecMold)
profiles: ProfilesMold = Field(default_factory=ProfilesMold)
zerotrust: ZeroTrustTunnelMold = Field(default_factory=ZeroTrustTunnelMold)
containers: ContainersMold = Field(default_factory=ContainersMold)
monitoring: MonitoringMold = Field(default_factory=MonitoringMold)
haproxy: HaproxyMold = Field(default_factory=HaproxyMold)
xray: XrayCoreMold = Field(default_factory=XrayCoreMold)
tor: TorMold = Field(default_factory=TorMold)
telemt: TelemtMold = Field(default_factory=TelemtMold)
```
- **`FeaturesMold`** - one field per feature, each a sub-mold with its own defaults.

Field order is deploy order.
Everything else is derived from these fields - the allowed [merge layers](inventories.md), the feature-to-mold mapping, and rune dispatch - so adding a feature means adding a field, not editing plumbing.
Expand Down Expand Up @@ -66,4 +50,3 @@ dns = DnsMold(mode=DnsMode.DOT_RESOLVED)
```

`nullforge.molds` exports only the molds.
The split is enforced by an import contract: models import nothing from the rest of the package.
Loading
Loading