From 4e8e67dec44e4461556e0cfcf3cbb5827ab34677 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:17:57 +0600 Subject: [PATCH 001/509] docs: add bottom-up docker ecosystem master plan --- .../00-master-bottom-up-plan.md | 252 ++++++++++++++++++ 1 file changed, 252 insertions(+) create mode 100644 docs/plans/docker-ecosystem/00-master-bottom-up-plan.md diff --git a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md new file mode 100644 index 00000000..94f3d3d3 --- /dev/null +++ b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md @@ -0,0 +1,252 @@ +# LocalDevStack Docker Ecosystem — Bottom-Up Development Plan + +## Status + +Planning branch: `plan/docker-ecosystem-bottom-up` + +This branch is planning-only. No runtime behavior should be changed until the relevant repository plan is accepted and implementation work moves into that repository. + +## Product Definition + +LocalDevStack is a Docker-based local development platform for PHP and Node.js: a modern, modular XAMPP alternative with local domains/TLS, multiple PHP/Node runtimes, databases, admin UIs, mail, schedulers/workers, developer tools, and optional local AI. + +The system is intentionally split across repositories instead of becoming one monolithic image. + +## Repository Stack + +Bottom to top: + +1. `infocyph/Scriptomatic` — reusable bootstrap/entrypoint/setup scripts used while building developer/runtime images. +2. `infocyph/Toolset` — reusable standalone developer utilities consumed by images (`gitx`, `chromacat`, `sqlitex`, `netx`, etc.). +3. `infocyph/docker-runner` — background process infrastructure: Supervisor, cron, logrotate, helper execution. +4. `infocyph/docker-nginx` — primary HTTP/TLS edge and local routing. +5. `infocyph/docker-apache` — optional Apache backend for PHP/vhost compatibility. +6. `infocyph/docker-tools` — LocalDevStack control plane: domain/TLS/config generation, service/profile helpers, monitoring, admin panel, environment/secrets tooling. +7. `infocyph/docker-llm-sm` — published optional local-AI capability; already follows the newer image publication model. +8. `infocyph/LocalDevStack` — product/orchestrator: `lds`, Compose topology, PHP/Node runtime Dockerfiles, generated configuration, user-facing workflow, platform integration. + +## Core Architectural Rules + +### Published infrastructure vs generated runtimes + +Published images remain independently versioned infrastructure: + +- `infocyph/runner` +- `infocyph/nginx` +- `infocyph/apache` +- `infocyph/tools` +- `infocyph/llm-sm` + +PHP and Node remain locally generated runtime images because they are customized by selected runtime version, host UID/GID, extensions/packages, and project-level needs: + +- `localdevstack-php:` +- `localdevstack-node:` + +Do not convert PHP/Node into one-size-fits-all published images unless a later design proves that customization can be retained without increasing user complexity. + +### Image immutability contract + +For every published `docker-*` repository: + +- release event publishes `` and `latest`; +- release-version tags are immutable after first publication; +- scheduled rebuild checks out the latest published release source but refreshes `latest` only; +- source release and resulting image revision must be identifiable through OCI metadata/provenance; +- Docker Hub and GHCR should receive the same built digest for the same variant. + +`docker-llm-sm` is the reference implementation for this contract. + +### Dependency reproducibility + +Do not fetch executable build dependencies from floating `main`, `master`, or `latest` URLs without an explicit reason. + +For Scriptomatic/Toolset files consumed during image builds: + +- prefer a release tag when a release lifecycle exists; +- otherwise pin an immutable commit SHA; +- expose the selected ref as an explicit build argument where useful; +- record the selected ref in image labels/build metadata; +- verify downloaded content when practical. + +Scheduled `latest` image rebuilds may consume updated base-image patches, but they must not mutate an already published version tag. + +### Network contract + +Service discovery should use Docker DNS/service names, not static container IPv4 addresses. + +The current `172.28.0.0/24`, `172.29.0.0/24`, and `172.30.0.0/24` fixed LocalDevStack subnets are orchestration details, not image contracts. No supporting `docker-*` repository currently requires those addresses. + +Migration target: + +- retain logical networks (`frontend`, `backend`, `datastore`); +- remove fixed `ipv4_address` declarations; +- remove hard-coded IPAM subnets/gateways after validation; +- use service names/aliases for all cross-container communication; +- reassess `lds vpn-fix` after static-subnet removal. + +### Docker socket contract + +`/var/run/docker.sock` is equivalent to powerful host Docker control. It must not be mounted merely for convenience. + +During LocalDevStack integration review: + +- inventory each Docker API operation used by `server-tools` and `runner`; +- decide whether both containers still require direct socket access; +- preserve required local-development functionality; +- document the trust boundary clearly; +- do not add a socket proxy unless it demonstrably reduces permissions without breaking the developer workflow. + +### Cross-platform contract + +Preserve Linux, macOS, WSL/Git Bash, and Windows Docker Desktop support where currently intended. + +`lds.bat` remains a Windows bridge into the Bash control plane. Do not replace it with a Linux-only installation model. + +## Execution Order + +### Phase 0 — Shared foundations + +Plan: `01-shared-foundations-plan.md` + +Stabilize Scriptomatic and Toolset consumption before rebuilding dependent Docker images. The purpose is not to redesign those repositories wholesale; it is to establish immutable dependency references, validation, and reusable contracts required by the Docker stack. + +Exit gate: + +- every downstream image can select immutable Scriptomatic/Toolset revisions; +- required helper scripts have syntax/smoke validation; +- no downstream migration depends on floating helper content. + +### Phase 1 — Leaf infrastructure images + +Execute independently where possible: + +- `02-docker-runner-plan.md` +- `03-docker-nginx-plan.md` +- `04-docker-apache-plan.md` + +Goals: + +- modernize CI/publication; +- pin external helper inputs; +- validate scripts/configs before publication; +- keep each image narrowly focused; +- publish immutable release tags and scheduled `latest` only. + +Exit gate for each image: + +- PR validation green; +- actual image build green; +- health/smoke check green; +- release workflow follows immutable-version contract; +- LocalDevStack can consume the resulting release without behavior regression. + +### Phase 2 — Control plane + +Plan: `05-docker-tools-plan.md` + +This is the largest supporting image and must be handled after foundational/leaf contracts are stable. + +Primary goals: + +- establish strong CI around shell tools/templates/admin panel; +- pin all downloaded dependencies; +- make machine-readable contracts explicit; +- reduce duplicated platform configuration with LocalDevStack; +- preserve domain/TLS/secrets/monitoring/admin workflows; +- avoid turning `docker-tools` into the owner of LocalDevStack orchestration policy. + +### Phase 3 — Local AI capability + +Plan: `06-docker-llm-sm-plan.md` + +`docker-llm-sm` is already published and is primarily a compatibility/integration reference, not a rewrite target. + +Primary goals: + +- keep its newer release model intact; +- define how LocalDevStack optionally enables CPU/NVIDIA/AMD tags; +- persist `/root/.ollama`; +- optionally mount project workspace for repo-aware AI commands; +- allow Graphify or other clients to use its Ollama endpoint without coupling them into the image. + +### Phase 4 — Product/orchestrator + +Plan: `07-localdevstack-integration-plan.md` + +Only after lower layers have stable contracts: + +- add comprehensive CI; +- modularize the large `lds` control plane without changing the public CLI unnecessarily; +- establish one canonical service/profile catalog; +- migrate static networking to Docker DNS; +- reconcile generated-state/storage documentation with actual named volumes/bind mounts; +- consume versioned infrastructure images; +- integrate optional `llm-sm` cleanly; +- review Docker socket exposure; +- keep PHP/Node runtime generation flexible. + +## Cross-Repository Release Strategy + +Each infrastructure repository should independently release. LocalDevStack should consume explicit compatibility-tested versions rather than assuming every `latest` across the ecosystem changes safely together. + +During development, `latest` remains useful. For LocalDevStack releases, prefer explicit image versions in the release manifest/default environment so a LocalDevStack release can be reproduced. + +A later automated dependency update workflow may propose image version bumps after integration CI passes; do not couple repositories through automatic mutable `latest` behavior alone. + +## Shared CI Baseline + +Every `docker-*` repository should converge on an appropriate subset of: + +- shell syntax checks (`bash -n` / `sh -n` as appropriate); +- ShellCheck for maintained shell scripts; +- Dockerfile/build validation via real Buildx build; +- container smoke/health check; +- configuration validation (`nginx -t`, `httpd -t`, `supervisorctl`, etc.); +- Docker metadata/provenance; +- Buildx cache with repository/variant-safe scopes; +- concurrency protection for publication; +- timeout limits for stuck builds; +- SBOM and vulnerability visibility where practical; +- scheduled `latest` refresh without version-tag overwrite. + +Do not add CI that cannot exercise the image’s actual contract merely to increase check count. + +## Planning Rules for File-by-File Drafts + +Each repository plan below identifies: + +- existing files to modify; +- existing files to validate but intentionally keep; +- new files/workflows/tests to create; +- cross-repository prerequisites; +- acceptance criteria. + +Implementation should follow the listed order inside each repository unless a dependency discovered during coding requires adjustment. + +## Non-Goals + +This program is not intended to: + +- replace Docker Compose; +- collapse all services into one container; +- convert LocalDevStack into Kubernetes; +- publish customized PHP/Node combinations for every possible user selection; +- remove Apache merely because Nginx is the default edge; +- make local-development containers production-hardening equivalents; +- introduce a new programming language for the CLI solely for refactoring aesthetics; +- add AI requirements to users who do not enable the AI capability. + +## Definition of Completion + +The ecosystem work is complete when: + +1. Shared helper dependencies are reproducible and tested. +2. All published infrastructure images have modern immutable release workflows and meaningful CI. +3. `docker-tools` exposes stable machine-readable contracts instead of duplicating orchestration state where avoidable. +4. LocalDevStack has integration CI covering its major profile/runtime combinations. +5. Static IP dependence is removed unless a documented unavoidable case remains. +6. PHP and Node runtime generation remains version-flexible and user-customizable. +7. Local domains/TLS, mail, DB/admin clients, cron/supervisor, secrets, and diagnostics still work. +8. Optional local AI can be enabled through `infocyph/llm-sm` without contaminating the default stack. +9. Documentation matches the actual storage/network/release behavior. +10. A clean install on supported host categories can reach a working PHP or Node local domain with TLS using the documented workflow. From 0ef577476115d9fa1057ebc0ee1b6b4ecf6ec7ec Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:18:31 +0600 Subject: [PATCH 002/509] docs: add shared foundations plan --- .../01-shared-foundations-plan.md | 249 ++++++++++++++++++ 1 file changed, 249 insertions(+) create mode 100644 docs/plans/docker-ecosystem/01-shared-foundations-plan.md diff --git a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md new file mode 100644 index 00000000..d90bcd34 --- /dev/null +++ b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md @@ -0,0 +1,249 @@ +# Shared Foundations Plan — Scriptomatic + Toolset + +## Purpose + +Stabilize the two shared repositories consumed by LocalDevStack Docker images before changing the image layer itself. + +This is a dependency-contract plan, not a wholesale redesign of Scriptomatic or Toolset. + +## Repositories + +- `infocyph/Scriptomatic` +- `infocyph/Toolset` + +## Primary Problems to Solve + +1. Docker builds currently fetch helpers from mutable `main`/`master` URLs. +2. A rebuild of unchanged Docker source can therefore silently receive different helper code. +3. Shell/bootstrap behavior is consumed by several images without a shared compatibility contract. +4. Some helper scripts are large enough that syntax-only confidence is insufficient. + +## Contract to Establish + +Every downstream image must be able to select immutable revisions through explicit build arguments, for example: + +- `SCRIPTOMATIC_REF=` +- `TOOLSET_REF=` + +Preferred source format: + +`https://raw.githubusercontent.com/infocyph///` + +If release tags are used, treat released helper content as immutable. If the repos do not yet maintain releases consistently, use commit SHAs until that lifecycle exists. + +## Scriptomatic — File-by-File Plan + +### `.github/` + +Create validation workflow(s) for shell scripts. + +Required checks: + +- `bash -n bash/*.sh` where Bash is required; +- `sh -n` only for scripts explicitly POSIX-shell compatible; +- ShellCheck with documented intentional suppressions only; +- smoke tests for PHP and Node setup scripts using disposable containers where practical. + +Do not add a release/publish workflow unless Scriptomatic is intentionally moved to a release-tag lifecycle. + +### `bash/php-cli-setup.sh` + +Current role: builds LocalDevStack PHP developer-runtime behavior. + +Plan: + +- preserve PHP extension/package customization inputs; +- preserve UID/GID user creation, Composer home isolation, FPM setup, Mailpit/msmtp, Git configuration and shell helpers; +- replace floating Toolset/Scriptomatic helper URLs with ref-driven URLs supplied through environment/build context; +- fail clearly when a required helper cannot be fetched; +- avoid `latest`-style helper installers where an immutable version/checksum can be selected; +- validate generated PHP/FPM configuration in smoke CI; +- ensure cleanup does not remove runtime-required assets; +- document inputs used by LocalDevStack Dockerfile (`PHP_EXT`, `PHP_EXT_VERSIONED`, `LINUX_PKG`, `LINUX_PKG_VERSIONED`, UID/GID, profile key). + +### `bash/node-cli-setup.sh` + +Current role: builds LocalDevStack Node developer-runtime behavior. + +Plan: + +- preserve UID/GID reuse/rename behavior for the upstream `node` user; +- preserve npm cache/global prefix and Corepack behavior; +- make Toolset/Scriptomatic helper downloads immutable/ref-driven; +- review whether unconditional `npm@latest` upgrade is desirable for reproducible runtime images; prefer explicit npm policy/version input if retained; +- validate generated user/home/global-package permissions; +- smoke-test a generated Node runtime as non-root. + +### `bash/php-entry.sh` + +Plan: + +- syntax check; +- validate signal/exec semantics; +- ensure mounted CA/config initialization remains idempotent; +- keep entrypoint small; do not move build-time setup into runtime. + +### `bash/node-entry.sh` + +Plan: + +- syntax check; +- validate command forwarding and signal semantics; +- ensure runtime setup is idempotent; +- keep project command execution as the final `exec` path. + +### `bash/banner.sh` + +Plan: + +- treat as shared presentation helper only; +- syntax/ShellCheck validation; +- no runtime-critical logic should depend on banner rendering succeeding; +- downstream images should pin its revision rather than fetch `master`. + +### `bash/alias-maker.sh` + +Plan: + +- validate idempotency; +- ensure aliases do not hide core commands in non-interactive execution; +- document which aliases are relied on by LocalDevStack developer shells. + +### `bash/docknotify.sh` + +Plan: + +- validate behavior when notification transport is unavailable; +- never make normal PHP/Node process startup depend on desktop notification success; +- add minimal smoke coverage for no-listener behavior. + +### `bash/owners.sh` + +Plan: + +- confirm whether current LocalDevStack images still consume it; +- if unused by all current Dockerfiles, mark as standalone Scriptomatic utility rather than part of the LocalDevStack compatibility contract. + +### `bash/mongo-replica.sh` + +Plan: + +- determine whether LocalDevStack currently calls it; +- if retained for future Mongo replica support, add syntax validation and document expected container/network assumptions; +- do not introduce replica-set complexity into default LocalDevStack profiles as part of this program. + +### `bash/certbot-hook.sh` / `bash/certbot-renew.sh` + +Plan: + +- explicitly classify as production/server helpers, not LocalDevStack local-TLS dependencies; +- leave out of LocalDevStack compatibility gating unless direct usage is found. + +### `bash/alias-maker.sh`, `banner.sh`, `docknotify.sh`, `php-entry.sh`, `node-entry.sh` + +Add a compact compatibility test matrix because these are directly pulled into runtime images. + +## Toolset — File-by-File Plan + +Only utilities directly consumed by the Docker ecosystem are in the critical path. + +### `Git/gitx` + +Current consumers: `docker-tools`, PHP runtime setup, Node runtime setup. + +Plan: + +- establish an immutable ref used by images; +- run Bash syntax/ShellCheck if compatible with its implementation style; +- add smoke tests for non-destructive commands needed inside dev containers; +- do not couple `gitx` to `llm-sm`; AI commit behavior already lives independently in `docker-llm-sm`. + +### `ChromaCat/chromacat` + +Current consumers: tools/runner/nginx/apache/PHP/Node developer shells. + +Plan: + +- pin downstream consumption; +- validate no-color/non-TTY behavior; +- make banner/color failures non-critical; +- keep it purely presentational. + +### `Sqlite/sqlitex` + +Current consumer: `docker-tools`. + +Plan: + +- pin downstream consumption; +- smoke-test basic database open/query behavior against temporary SQLite data; +- document runtime package dependency expectations. + +### `Network/netx` + +Current consumer: `docker-tools`. + +Plan: + +- pin downstream consumption; +- smoke-test basic local network inspection without requiring privileged host operations; +- ensure failures are diagnostic rather than destructive. + +### `Docker/dockex` + +Plan: + +- inspect whether LocalDevStack or `docker-tools` currently installs/calls it; +- if unused, do not add it merely for symmetry; +- if later adopted, treat Docker-socket permission requirements explicitly. + +### `PHP/phpx` + +Plan: + +- inspect whether current PHP wrappers already provide the required behavior; +- do not introduce it into LocalDevStack runtime images unless it replaces duplicated functionality with a clear compatibility win. + +### `Clean/cleanx` + +Plan: + +- keep out of default container images unless a concrete LocalDevStack command adopts it; +- destructive cleanup remains explicit and host-controlled. + +### Toolset docs/README files + +Update only when dependency/release guarantees change. Do not rewrite unrelated documentation during Docker ecosystem work. + +## New Shared Validation Artifacts + +If missing, add lightweight test directories/workflows in Scriptomatic and Toolset rather than embedding compatibility tests into downstream Dockerfiles. + +Suggested test categories: + +- syntax; +- non-interactive execution; +- no-color mode; +- temporary HOME/user paths; +- read-only Git/config mounts; +- expected failure behavior when optional host integrations are absent. + +## Downstream Consumption Pattern + +Each downstream Dockerfile should stop hard-coding: + +- `Toolset/main/...` +- `Scriptomatic/master/...` + +and move to explicit build args/defaults tied to accepted immutable revisions. + +Do not duplicate Scriptomatic/Toolset source into every Docker repo unless GitHub availability at build time becomes an unacceptable dependency and a vendoring decision is made intentionally. + +## Acceptance Criteria + +1. Required Scriptomatic scripts pass syntax + ShellCheck policy. +2. PHP and Node setup smoke tests pass in disposable upstream base containers. +3. Toolset utilities used by the Docker ecosystem have smoke validation. +4. Every downstream image can point to immutable Scriptomatic/Toolset refs. +5. Rebuilding the same Docker release source with the same base image/ref inputs does not silently receive newer helper scripts. +6. Optional presentation/notification helpers cannot prevent the primary container service from starting. From e4313b532d970c667409049254c6677f34dc23c8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:18:58 +0600 Subject: [PATCH 003/509] docs: add docker-runner file-by-file plan --- .../docker-ecosystem/02-docker-runner-plan.md | 169 ++++++++++++++++++ 1 file changed, 169 insertions(+) create mode 100644 docs/plans/docker-ecosystem/02-docker-runner-plan.md diff --git a/docs/plans/docker-ecosystem/02-docker-runner-plan.md b/docs/plans/docker-ecosystem/02-docker-runner-plan.md new file mode 100644 index 00000000..9250fbce --- /dev/null +++ b/docs/plans/docker-ecosystem/02-docker-runner-plan.md @@ -0,0 +1,169 @@ +# docker-runner — File-by-File Development Plan + +## Role + +`infocyph/docker-runner` is the background execution image for LocalDevStack. It owns Supervisor, cron, log rotation, and small helper wrappers. It should remain intentionally narrow. + +## Invariants + +- Supervisor stays PID 1. +- Cron and logrotate continue to be supervised services. +- User-provided supervisor/cron definitions remain mountable from LocalDevStack. +- Log handling must remain resilient when some service log directories are absent. +- The image may need Docker CLI access, but Docker socket mounting is a LocalDevStack orchestration decision, not an image default. + +## Existing Files + +### `.github/workflows/docker.publish.yml` + +Replace the legacy publication workflow with the current ecosystem contract: + +- current GitHub Action majors matching `docker-llm-sm` where compatible; +- release event: publish immutable `` + `latest`; +- scheduled event: resolve latest published release source but publish only `latest`; +- never overwrite a release-version tag on schedule; +- Buildx setup + cache; +- Docker Hub + GHCR from the same build digest; +- provenance attestation; +- add concurrency guard; +- add sensible `timeout-minutes`; +- preserve the existing every-two-weeks cadence unless we intentionally standardize schedules later. + +### New `.github/workflows/check.yml` + +Add PR/push validation: + +- `bash -n` for Bash scripts; +- ShellCheck; +- validate Supervisor configuration inside a built image; +- real Docker image build; +- start container and wait for health; +- verify `supervisorctl status` reports `cron` and `logrotate` RUNNING; +- verify stop signal shuts down cleanly. + +### `Dockerfile` + +Plan: + +- parameterize the Alpine base version instead of relying blindly on `alpine:latest`; +- add immutable Scriptomatic/Toolset refs for `banner.sh` and `chromacat`; +- replace floating `ADD raw.githubusercontent.com/.../master|main` with explicit ref-driven fetch/copy; +- keep package list minimal: bash, curl, CA, supervisor, docker-cli, logrotate, cronie, tzdata, presentation dependencies; +- verify whether `gawk`/other current packages are actually required before removing anything; +- preserve healthcheck against Supervisor; +- preserve `STOPSIGNAL SIGTERM`; +- add OCI revision/version metadata through workflow rather than hard-coded release values; +- keep root user if required for cron/logrotate/Docker access; do not force non-root and break the role. + +### `scripts/supervisord.conf` + +Plan: + +- keep `nodaemon=true` and UNIX control socket; +- validate included `/etc/supervisor/conf.d/*.conf` behavior when directory is empty; +- keep cron/logrotate stdout/stderr on container streams; +- add/verify `stopasgroup`/`killasgroup` only if child-process tests show signal leakage; +- do not add unrelated application workers to the base image. + +### `scripts/logrotate-worker.sh` + +Plan: + +- validate `LOGROTATE_INTERVAL` as a positive integer before sleeping; +- keep configurable state file; +- preserve `/etc/logrotate.conf` preference and per-file fallback; +- ensure one invalid optional logrotate fragment does not create an uncontrolled tight restart loop; +- expose useful failure logs through stderr/stdout; +- test a temporary log file rotation end-to-end. + +### `scripts/pexe.sh` + +Plan: + +- review argument quoting/TTY forwarding; +- keep helper single-purpose; +- ensure target process exit code is propagated; +- add shell smoke coverage. + +### `scripts/dexe.sh` + +Plan: + +- same quoting/exit-code review as `pexe.sh`; +- confirm Docker CLI/container-name assumptions are still required by LocalDevStack runner workflows; +- do not duplicate `lds exec` features into this script. + +### `loggables/daily` + +Plan: + +- validate with `logrotate -d`/debug mode in CI; +- confirm paths match LocalDevStack-mounted `/global/log` layout; +- preserve retention policy unless a bug is found. + +### `loggables/dailyold` + +Plan: + +- determine active use vs legacy compatibility; +- if still used, validate separately; +- if unused across LocalDevStack, deprecate first rather than silently delete. + +### `loggables/supervisord` + +Plan: + +- validate configured path and ownership against the current Supervisor logfile path; +- ensure rotation does not break the active file descriptor/process. + +### `README.md` + +Update after code is stable: + +- define the image as LocalDevStack background-process infrastructure; +- document mounted supervisor/cron/log paths; +- document required Docker socket only as an integration choice; +- document health behavior and environment variables; +- document immutable release tags + `latest` semantics. + +### `.dockerignore` + +Review against actual build context and keep only files required by Docker build. No cosmetic expansion. + +### `.gitignore` / `.gitattributes` + +Keep unless validation finds missing generated files/line-ending rules. + +### `LICENSE` + +No change. + +## New Tests + +Suggested `tests/` files: + +- `tests/shell-check.sh` — syntax entrypoint for local/CI use; +- `tests/supervisor-smoke.sh` — run built container, verify supervisor/cron/logrotate; +- `tests/logrotate-smoke.sh` — temporary log + forced rotation validation. + +Tests should be executable from CI and locally without LocalDevStack running. + +## Integration Follow-Up in LocalDevStack + +After release: + +- pin/raise `infocyph/runner` version in LocalDevStack; +- confirm mounted cron/supervisor definitions work; +- inventory why Runner receives Docker socket; +- remove socket mount later only if no required runner workflow uses it. + +## Acceptance Criteria + +1. ShellCheck/syntax green. +2. Real image build green. +3. Container reaches healthy state. +4. Supervisor sees cron/logrotate running. +5. Log rotation smoke test passes. +6. Release workflow cannot overwrite immutable version tags from schedule. +7. Scriptomatic/Toolset dependencies are immutable/ref-driven. +8. Existing LocalDevStack scheduler/supervisor behavior remains compatible. From f5bd684def5ebf5ea368808bf9322395d72c088a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:19:27 +0600 Subject: [PATCH 004/509] docs: add docker-nginx file-by-file plan --- .../docker-ecosystem/03-docker-nginx-plan.md | 151 ++++++++++++++++++ 1 file changed, 151 insertions(+) create mode 100644 docs/plans/docker-ecosystem/03-docker-nginx-plan.md diff --git a/docs/plans/docker-ecosystem/03-docker-nginx-plan.md b/docs/plans/docker-ecosystem/03-docker-nginx-plan.md new file mode 100644 index 00000000..1105bc7d --- /dev/null +++ b/docs/plans/docker-ecosystem/03-docker-nginx-plan.md @@ -0,0 +1,151 @@ +# docker-nginx — File-by-File Development Plan + +## Role + +`infocyph/docker-nginx` is the LocalDevStack edge proxy. It terminates local TLS, serves generated vhosts, routes directly to PHP-FPM or to Apache/Node/service backends, and exposes the LocalDevStack convenience hosts such as admin/mail/database UIs. + +## Invariants + +- Nginx remains the primary listener on host HTTP/HTTPS ports. +- Generated vhost files remain hot-reloadable. +- PHP fastcgi, Apache proxy, Node proxy, WebSocket/SSE behavior must remain compatible. +- LocalDevStack certificates remain mounted rather than regenerated independently by Nginx. +- Docker DNS/service names are the routing contract; static container IPs are not required. + +## Existing Files + +### `.github/workflows/docker.publish.yml` + +Replace with modern publication flow: + +- current Action majors; +- release -> `` + `latest`; +- scheduled rebuild -> `latest` only; +- Buildx cache, provenance, concurrency and timeout; +- same digest pushed to Docker Hub and GHCR; +- never mutate release-version tags during scheduled rebuilds. + +### New `.github/workflows/check.yml` + +Validate: + +- `sh -n` for POSIX shell scripts; +- ShellCheck in `sh` mode; +- Docker image build; +- `nginx -t` on default image; +- entrypoint startup with temporary certificate/config fixtures; +- generated `locals.conf` syntax; +- bad-vhost quarantine/restore behavior; +- clean SIGTERM shutdown. + +### `Dockerfile` + +Plan: + +- parameterize/pin the upstream Nginx Alpine base line rather than uncontrolled `nginx:alpine` drift for release reproducibility; +- use immutable Scriptomatic/Toolset refs for banner/chromacat downloads; +- retain timezone, locale and minimal shell tooling required by entrypoint scripts; +- keep `fcgi-params.sh`, `proxy-params.sh`, `render-locals.sh`, `nginx-entrypoint.sh` copied from this repository; +- keep image-owned healthcheck based on `nginx -t`; +- let workflow supply OCI version/revision/created metadata; +- avoid installing development-only utilities into the edge image. + +### `scripts/fcgi-params.sh` + +Plan: + +- review every generated include path against current PHP vhost templates; +- keep idempotent file generation; +- validate output through a synthetic PHP vhost and `nginx -t`; +- ensure modern forwarded headers/fastcgi parameters are complete without leaking host-specific assumptions; +- preserve Unix-socket and TCP upstream compatibility if both are currently generated by tools. + +### `scripts/proxy-params.sh` + +Plan: + +- validate HTTP proxy, WebSocket, streaming/SSE, timeout and buffer include files; +- keep generated files deterministic/idempotent; +- review forwarded scheme/host/real-IP headers; +- test Node and Apache routes with `nginx -t` and a minimal upstream smoke container; +- avoid over-large default timeouts/buffers unless required for local dev uploads/debugging. + +### `scripts/render-locals.sh` + +Current role: builds convenience-host routing (`admin.localhost`, `webmail.localhost`, DB UIs, etc.). + +Plan: + +- preserve predefined LocalDevStack routes and additive `LOCALHOST_ROUTES` support; +- move predefined route data toward an explicit data block/config contract if LocalDevStack needs to own the service catalog later; +- keep user routes unable to override reserved system routes unless a future explicit override flag is added; +- validate host/upstream input strictly; +- keep Docker resolver `127.0.0.11` only where variable `proxy_pass` requires it; +- verify TLS config against current Nginx/OpenSSL defaults and remove legacy cipher material only after compatibility testing; +- preserve SSE `/api/tail` streaming behavior used by admin/log tooling; +- test generated file when some optional upstream services are not running (config must remain valid even if upstream DNS resolves only at request time). + +### `scripts/nginx-entrypoint.sh` + +Plan: + +- preserve timezone setup, local-route rendering and pre-start validation; +- review the auto-disable-invalid-vhost feature carefully because it mutates mounted config names; +- keep invalid vhost isolation if it materially improves the dev experience, but add deterministic tests and clear logs; +- prevent endless disable/restore oscillation for permanently invalid configuration; +- ensure background auto-restore loop terminates with the main Nginx process/container; +- validate numeric inputs (`MAX_DISABLE_ATTEMPTS`, restore interval); +- preserve final `exec "$@"` signal semantics; +- consider moving state about disabled configs outside the shared config directory only if current mutation creates cross-container/user confusion. + +### `.dockerignore` + +Review minimal build context; retain only Dockerfile/scripts/workflow-independent runtime inputs. + +### `.gitignore` / `.gitattributes` + +Keep unless generated test fixtures require ignores. Preserve LF shell scripts. + +### `LICENSE` + +No change. + +### New `README.md` (if absent) or repository documentation + +The current tree has no README. Add one after behavior stabilizes, covering: + +- image purpose and LocalDevStack relationship; +- expected mounts (`/etc/nginx/conf.d`, cert/root CA, PHP-FPM sockets, logs); +- environment variables controlling invalid-config restore/local routes; +- published tags and release semantics; +- standalone smoke example without presenting this image as a complete LocalDevStack replacement. + +## New Tests / Fixtures + +Create a small `tests/` tree: + +- `tests/nginx-config-smoke.sh`; +- `tests/render-locals-smoke.sh`; +- `tests/invalid-vhost-smoke.sh`; +- `tests/fixtures/certs/` generated during CI, not committed private material; +- `tests/fixtures/conf.d/` with valid/invalid minimal vhosts. + +## Integration Follow-Up in LocalDevStack + +After a compatible release: + +- pin/raise `infocyph/nginx` version; +- validate PHP direct-FPM, PHP-via-Apache, Node, Mailpit, CloudBeaver, RedisInsight, Mongo Express, Kibana and admin-panel routes; +- remove static Nginx IPv4 assignment when LocalDevStack networking migration executes; +- confirm `lds http reload`, domain create/delete and TLS flows still work. + +## Acceptance Criteria + +1. Shell validation green. +2. Real image build green. +3. `nginx -t` green with default and representative generated configs. +4. Node/WebSocket/SSE and PHP/Apache routing smoke tests pass. +5. Invalid-vhost quarantine/restore cannot loop uncontrollably. +6. Release tags remain immutable; schedule refreshes only `latest`. +7. External helper refs are immutable. +8. LocalDevStack can route all supported local service classes by Docker DNS without fixed IP dependencies. From ed1b597f1a57f1fac0efd09f4f27c3d89c6d0faa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:19:50 +0600 Subject: [PATCH 005/509] docs: add docker-apache file-by-file plan --- .../docker-ecosystem/04-docker-apache-plan.md | 138 ++++++++++++++++++ 1 file changed, 138 insertions(+) create mode 100644 docs/plans/docker-ecosystem/04-docker-apache-plan.md diff --git a/docs/plans/docker-ecosystem/04-docker-apache-plan.md b/docs/plans/docker-ecosystem/04-docker-apache-plan.md new file mode 100644 index 00000000..3de97887 --- /dev/null +++ b/docs/plans/docker-ecosystem/04-docker-apache-plan.md @@ -0,0 +1,138 @@ +# docker-apache — File-by-File Development Plan + +## Role + +`infocyph/docker-apache` is the optional Apache backend used when a LocalDevStack project needs Apache semantics or compatibility while Nginx remains the public edge. + +## Invariants + +- Apache is optional, not a second mandatory edge. +- Nginx may reverse-proxy to Apache over Docker DNS. +- Generated vhosts remain externally mounted. +- FastCGI/PHP-FPM compatibility remains available. +- Local TLS compatibility may be retained for internal/direct Apache use, but host-facing TLS ownership remains with LocalDevStack’s edge design. + +## Existing Files + +### `.github/workflows/docker.publish.yml` + +Replace the legacy workflow with the ecosystem release contract: + +- current GitHub Action majors; +- release event publishes immutable `` + `latest`; +- schedule rebuilds latest release source and publishes `latest` only; +- Buildx cache; +- Docker Hub + GHCR from same digest; +- provenance; +- concurrency guard and timeout; +- scheduled builds must never overwrite release-version tags. + +### New `.github/workflows/check.yml` + +Validate: + +- shell syntax and ShellCheck; +- real Docker build; +- `httpd -t` after image construction; +- container startup and healthcheck; +- mounted vhost loading; +- representative reverse-proxy/FastCGI config if feasible with a small fixture; +- clean signal shutdown. + +### `Dockerfile` + +Plan: + +- parameterize upstream `httpd` Alpine base line for controlled updates; +- replace mutable Scriptomatic/Toolset downloads with immutable/ref-driven dependencies; +- retain only packages needed by update/entrypoint/healthcheck and local TLS/FastCGI support; +- preserve `apache-mod-fcgid`, curl/CA, shell/locale requirements where still used; +- keep `WORKDIR /app` and vhost directory contract; +- let CI/workflow provide dynamic OCI revision/version metadata; +- do not make Apache own LocalDevStack certificate generation; +- verify whether exposing 443 internally is still useful; keep until routing tests prove it can be removed safely. + +### `scripts/update_httpd.sh` + +Current role: normalizes upstream `httpd.conf`, enables proxy/FastCGI/rewrite/SSL/HTTP2/headers/deflate and adds the mounted vhost include. + +Plan: + +- keep operation deterministic/idempotent; +- replace brittle line-edit logic only where tests demonstrate risk; +- validate every enabled module exists in the selected upstream image; +- ensure repeated image-build execution would not duplicate config lines; +- validate resulting `httpd.conf` with `httpd -t` during build/CI; +- review whether both `Listen 80` and `Listen 443` are required for current Nginx-to-Apache modes; +- keep `IncludeOptional conf/vhosts/*.conf` as the generated-vhost contract; +- remove self-deletion (`rm -f -- "$0"`) only if retaining the helper at runtime is useful for diagnostics; otherwise document that it is a build-only helper. + +### `scripts/entrypoint.sh` + +Plan: + +- preserve minimal runtime setup and final `exec` semantics; +- validate mounted vhost permissions/readability before launch where useful; +- fail with clear output when core config is invalid instead of looping; +- avoid runtime mutation of user-generated vhosts unless explicitly required. + +### `scripts/healthcheck.sh` + +Plan: + +- verify health check tests the actual Apache process/config rather than a route that depends on an optional project; +- prefer `httpd -t` plus local process/listener validation; +- keep healthcheck fast and independent of Nginx or PHP project availability. + +### `README.md` + +Update after implementation: + +- describe Apache as an optional LocalDevStack backend; +- document mounted vhost path and expected upstream PHP-FPM relationship; +- document ports/network use without implying users should bind Apache directly on host by default; +- document tag/release semantics and healthcheck. + +### `.dockerignore` + +Keep build context minimal and verify scripts are included. + +### `.gitignore` / `.gitattributes` + +Keep unless test/generated artifacts require updates. Preserve LF shell files. + +### `LICENSE` + +No change. + +## New Tests / Fixtures + +Suggested: + +- `tests/apache-config-smoke.sh`; +- `tests/vhost-smoke.sh`; +- `tests/fixtures/vhosts/basic.conf`; +- optional PHP-FPM mock/upstream fixture for proxy/FastCGI validation. + +Do not require a full LocalDevStack environment for repository-level CI. + +## Integration Follow-Up in LocalDevStack + +After release: + +- pin/raise `infocyph/apache` version; +- validate Nginx -> Apache HTTP and HTTPS/internal paths still used by generated templates; +- validate `.htaccess`/rewrite use cases that justify Apache mode; +- remove Apache fixed IPv4 assignment with the broader DNS networking migration; +- ensure Apache is enabled only by the relevant generated profile/domain configuration. + +## Acceptance Criteria + +1. Shell validation green. +2. Image builds successfully. +3. `httpd -t` green after generated base configuration. +4. Mounted representative vhost loads successfully. +5. Healthcheck is project-independent. +6. Release-version tags are immutable and schedule updates `latest` only. +7. Scriptomatic/Toolset inputs are immutable/ref-driven. +8. LocalDevStack Apache-mode domains continue working behind Nginx via Docker DNS. From 1d374cd188170f2f470cba1fcca5eb2b55584b3c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:20:56 +0600 Subject: [PATCH 006/509] docs: add docker-tools file-by-file plan --- .../docker-ecosystem/05-docker-tools-plan.md | 414 ++++++++++++++++++ 1 file changed, 414 insertions(+) create mode 100644 docs/plans/docker-ecosystem/05-docker-tools-plan.md diff --git a/docs/plans/docker-ecosystem/05-docker-tools-plan.md b/docs/plans/docker-ecosystem/05-docker-tools-plan.md new file mode 100644 index 00000000..be100f79 --- /dev/null +++ b/docs/plans/docker-ecosystem/05-docker-tools-plan.md @@ -0,0 +1,414 @@ +# docker-tools — File-by-File Development Plan + +## Role + +`infocyph/docker-tools` is the LocalDevStack control-plane image. It owns domain/vhost generation, local certificate generation, profile/runtime selection helpers, secrets/environment tooling, monitoring, diagnostics, admin UI, reusable templates and the bridge between user intent and generated LocalDevStack configuration. + +This is the largest supporting image and should be changed only after Scriptomatic/Toolset contracts plus Runner/Nginx/Apache image contracts are stable. + +## Architectural Boundary + +`docker-tools` should implement control-plane mechanisms. LocalDevStack should remain the owner of product orchestration policy. + +Target split: + +- Tools: render/validate/generate/inspect. +- LocalDevStack: decide which services/profiles/images are enabled and persist the canonical product configuration. + +Avoid a second independent copy of service defaults inside Tools when LocalDevStack can provide a machine-readable catalog. + +## Existing Top-Level Files + +### `.github/workflows/docker.publish.yml` + +Replace legacy publishing with the modern contract: + +- current Action majors; +- release -> immutable `` + `latest`; +- schedule -> latest published release source, `latest` tag only; +- Buildx cache; +- provenance; +- concurrency guard and timeout; +- same digest to Docker Hub/GHCR; +- no scheduled overwrite of version tags. + +### New `.github/workflows/check.yml` + +Add layered validation rather than one monolithic smoke command: + +1. shell syntax/ShellCheck; +2. PHP syntax for admin-panel PHP files; +3. template fixture rendering; +4. existing `senv-smoke.sh`; +5. Docker image build; +6. container startup/health/control-plane smoke; +7. machine-readable command output validation (`--json` contracts); +8. admin panel endpoint smoke. + +### `Dockerfile` + +Plan: + +- parameterize Alpine base version; +- replace mutable Scriptomatic/Toolset `ADD` sources with immutable/ref-driven fetches; +- pin/verify mkcert, lazydocker and other downloaded executable sources where practical; +- review runtime-version discovery from endoflife.date: keep generated snapshot useful, but make build failure behavior intentional if external API is unavailable; +- preserve tools actually needed by shell commands/admin panel; +- remove duplicate packages only after command-to-package inventory; +- keep Docker CLI/Compose because control-plane commands currently inspect/manage stack containers; +- keep PHP runtime because admin panel is PHP-based; +- generate/store OCI source/version/revision metadata; +- add an image healthcheck covering the primary daemon/admin endpoint if one is stable; +- document Docker-socket trust expectation without attempting to solve host orchestration inside the Dockerfile. + +### `README.md` + +Rewrite only after behavior stabilizes. Document: + +- control-plane responsibility; +- command inventory grouped by domain/TLS/env/monitoring/admin/runtime generation; +- required mounts and Docker socket implications; +- machine-readable interfaces consumed by LocalDevStack; +- release/tag policy; +- standalone limitations vs full LocalDevStack. + +### `.dockerignore` + +Verify admin assets/templates/tests required during build are not accidentally excluded. Keep build context minimal. + +### `.gitignore` / `.gitattributes` + +Preserve LF shell policy and add generated test artifacts only if needed. + +### `LICENSE` + +No change. + +## Shell Command Files + +### `scripts/shells/entrypoint.sh` + +- define one clear primary process lifecycle; +- start notifier/admin services deterministically; +- propagate signals and child failures correctly; +- validate writable/mounted state directories before starting; +- avoid silently creating product configuration that should be owned by LocalDevStack. + +### `scripts/shells/certify.sh` + +- preserve mkcert/local CA workflow; +- make inputs/outputs explicit and idempotent; +- validate SAN/domain lists; +- keep generated CA/server cert paths compatible with Nginx/Apache/Mailpit/PHP trust mounts; +- add smoke generation + verification with `openssl`. + +### `scripts/shells/mkhost.sh` + +This is a critical large script. + +- freeze current CLI/output contracts with fixtures before refactor; +- separate parsing/state/render orchestration internally only after tests exist; +- consume a canonical runtime/service catalog supplied by LocalDevStack instead of hard-coded duplicate product defaults where possible; +- validate project/document-root paths; +- generate PHP/Apache/Nginx/Node artifacts atomically; +- keep `--JSON`/state output stable for `lds`; +- make repeated creation idempotent; +- add PHP direct-FPM, PHP+Apache and Node fixture tests. + +### `scripts/shells/rmhost.sh` + +- mirror `mkhost` state semantics; +- delete only artifacts owned by the selected host; +- preserve unrelated shared config; +- make deletion idempotent; +- test create -> remove -> recreate lifecycle. + +### `scripts/shells/domain-which.sh` + +- keep read-only diagnostics; +- validate all supported generated config locations; +- machine-readable output where already supported must remain stable; +- no mutation side effects. + +### `scripts/shells/env-store.sh` + +- formalize JSON file schema/version; +- atomic writes + locking where concurrent admin/CLI access can occur; +- explicit error on malformed state instead of silent data loss; +- stable `get/set/unset/get-json/set-json` contracts; +- unit-style smoke tests with temporary files. + +### `scripts/shells/profile-chooser.sh` + +Current issue: duplicates LocalDevStack database profile/default definitions. + +Target: + +- accept a service catalog path/environment input; +- keep only generic selection/state mechanics in Tools; +- provide fallback built-in catalog only if standalone compatibility requires it; +- keep JSON/profiles/services/env outputs stable during migration; +- LocalDevStack becomes canonical source for profile names/default environment values. + +### `scripts/shells/composer-setup.sh` + +- pin Composer installer/source validation; +- verify checksum/signature flow if not already present; +- keep build-only responsibility narrow. + +### `scripts/shells/git-default.sh` + +- preserve mounted global Git config behavior; +- avoid overwriting explicit user config unexpectedly; +- support missing name/email cleanly; +- test idempotency. + +### `scripts/shells/init-php-dirs.sh` + +- keep directory initialization idempotent; +- validate permission model against PHP runtime UID/GID strategy; +- avoid world-writable defaults unless required by cross-UID volume behavior and documented. + +### `scripts/shells/senv.sh` + +- keep SOPS/Age operations explicit; +- preserve current smoke test; +- add malformed/missing-key behavior tests; +- never print decrypted secret material in diagnostics by default. + +### `scripts/shells/es-policy.sh` + +- validate Elasticsearch version/API compatibility; +- keep bootstrap idempotent; +- do not make Elasticsearch profile startup depend on Kibana availability unless necessary. + +### `scripts/shells/notifierd.sh` + +- treat notifier as optional support service; +- failures must not take down unrelated control-plane functions; +- validate FIFO/TCP/token behavior and clean shutdown. + +### `scripts/shells/notify.sh` + +- graceful no-listener behavior; +- strict input escaping; +- preserve existing host-notification protocol. + +### `scripts/shells/status.sh` + +- make service status read-only; +- prefer Docker labels/service names over static container IPs; +- stable human + machine-readable output if exposed. + +### Monitoring scripts + +Files: + +- `monitor-flows.sh` +- `monitor-runtime.sh` +- `monitor-tls.sh` +- `monitor-db.sh` +- `monitor-volumes.sh` +- `monitor-queue.sh` +- `monitor-slo.sh` +- `monitor-log-heatmap.sh` +- `monitor-drift.sh` +- `monitor-alerts.sh` + +For every file: + +- classify required Docker/socket/log mounts; +- keep operation read-only unless the command explicitly advertises remediation; +- add timeout/error handling around Docker/network calls; +- return predictable exit status for automation/admin panel; +- support missing optional services without false critical errors; +- emit structured output for admin-panel consumption where practical; +- add fixture/smoke coverage per monitor category. + +## Docker Runtime Templates + +### `scripts/docker-templates/php.compose.yaml` + +- preserve local image build (`localdevstack-php:` + `pull_policy: never`); +- keep UID/GID/version/extensions/packages customization; +- fix any default GID inconsistency (`root` vs numeric) during implementation after validation; +- consume canonical paths/env from LocalDevStack; +- remove static IP assumptions; +- ensure healthcheck reflects PHP-FPM readiness; +- keep shared composer/Git/CA/FPM mounts. + +### `scripts/docker-templates/node.compose.yaml` + +- preserve local image build (`localdevstack-node:`); +- keep UID/GID/version/package/global-package customization; +- keep Node app command/port/host generation explicit; +- remove static IP assumptions; +- validate healthcheck for apps that intentionally have delayed startup; +- keep Git/CA/SSH/project mounts. + +## FPM Templates + +Files: + +- `scripts/fpm-templates/local-dynamic-warm.conf.tpl` +- `scripts/fpm-templates/local-ondemand.conf.tpl` +- `scripts/fpm-templates/local-static-high.conf.tpl` + +Plan for each: + +- render with fixture values; +- validate with `php-fpm -t` in representative PHP versions; +- document intended workload profile; +- keep local-dev defaults bounded to avoid excessive idle processes; +- ensure socket/user/group permissions match generated runtime containers and Nginx/Apache readers. + +## HTTP Templates + +Directories under `scripts/http-templates/` are contract-critical generated artifacts. + +For every template file: + +- render from a fixture replacing all placeholders; +- reject unresolved `{{...}}` tokens; +- Nginx templates must pass `nginx -t` in the target image; +- Apache templates must pass `httpd -t` in the target image; +- use Docker service/container DNS names rather than fixed IPs; +- preserve PHP direct-FPM, Apache proxy, Node proxy and fixed-IP/external-proxy modes only where still intentional; +- centralize common timeout/header snippets in image-level includes instead of duplicating large blocks across templates; +- keep TLS paths aligned with `certify.sh` output. + +Known critical families include Node Nginx HTTP/HTTPS, PHP Nginx direct/proxy HTTP/HTTPS, Apache vhosts and fixed/external proxy templates. + +## Admin Panel + +### `scripts/admin-panel/app/bootstrap.php` + +- centralize environment/path/container-name configuration; +- add safe wrappers for shell/Docker execution; +- enforce output escaping helpers and timeouts; +- keep no-framework deployment unless complexity justifies otherwise. + +### `scripts/admin-panel/app/index.php` and `scripts/admin-panel/index.php` + +- keep routing/front-controller behavior minimal; +- explicit 404/invalid page handling; +- no arbitrary file inclusion from request input. + +### Layout files + +- `_layout_top.php` +- `_layout_bottom.php` + +Plan: + +- centralize navigation/asset loading; +- keep all user/container data escaped; +- expose product/image version info for diagnostics. + +### Operational pages + +Files: + +- `automation_cron.php` +- `automation_manager.php` +- `automation_supervisor.php` +- `dashboard.php` +- `db_health.php` +- `docker_logs.php` +- `drift_monitor.php` +- `host_manager.php` +- `live_stats.php` +- `logs.php` +- `queue_health.php` +- `slo_view.php` +- `tls_monitor.php` +- `volume_monitor.php` + +For every page: + +- separate data collection/action execution from HTML rendering where practical; +- add command timeout/error handling; +- validate/allowlist any container/service/path/action parameters; +- escape all output; +- require explicit confirmation/POST semantics for mutating actions; +- keep read-only monitoring pages non-mutating; +- align with the corresponding shell monitor/helper as the backend contract rather than duplicating logic in PHP; +- add PHP syntax checks and HTTP smoke tests. + +### `scripts/admin-panel/public/css/core.css` / `panel.css` + +- no redesign during hardening; +- remove duplication only when tied to admin panel maintainability; +- preserve offline assets. + +### `scripts/admin-panel/public/js/core.js` / `panel.js` + +- identify whether `core.js` is vendored/minified third-party content vs project source; +- do not hand-edit generated/vendor bundles without a source/update path; +- validate SSE/log-tail reconnect behavior; +- sanitize DOM insertion of runtime/container data. + +### `scripts/admin-panel/public/vendor/**` + +- record upstream package/version/license; +- keep vendored assets offline-capable; +- update only through an explicit dependency update step, not ad hoc edits. + +### `scripts/admin-panel/README.md` + +Update after contracts stabilize with page/backend architecture and standalone diagnostic instructions. + +## Tests + +### Existing `scripts/tests/senv-smoke.sh` + +Keep and run in CI. + +### New tests + +Add focused tests instead of one giant integration script: + +- env-store schema/atomicity; +- profile-chooser with injected catalog; +- mkhost/rmhost lifecycle fixtures; +- certificate generation/inspection; +- PHP/Node Compose template rendering; +- FPM template rendering + `php-fpm -t`; +- HTTP template rendering + Nginx/Apache config validation; +- monitor command no-service/degraded-service behavior; +- admin-panel PHP syntax + endpoint smoke; +- entrypoint/notifier startup. + +## Canonical Service Catalog Migration + +Create a machine-readable catalog contract owned by LocalDevStack, likely JSON/YAML in the LocalDevStack repository. + +`docker-tools` should read the mounted catalog for: + +- profile slug; +- service display name; +- default image/version variable; +- setup environment prompts/defaults; +- optional admin/convenience route metadata where appropriate. + +Migration order: + +1. Tools supports external catalog while retaining current fallback. +2. LocalDevStack adds canonical catalog and mounts it. +3. `lds` and Tools both consume it. +4. Remove duplicated hard-coded catalog only after compatibility tests pass. + +## Acceptance Criteria + +1. All maintained shell files pass syntax/ShellCheck policy. +2. Admin PHP files pass syntax and smoke tests. +3. Image builds and starts successfully. +4. External Scriptomatic/Toolset/helper downloads are immutable/ref-driven. +5. PHP/Node Compose templates render valid Compose. +6. FPM/HTTP templates render without unresolved placeholders and validate in target runtimes. +7. mkhost/rmhost create/delete lifecycle passes fixture tests. +8. cert generation and TLS inspection pass. +9. monitor/admin paths handle missing optional services correctly. +10. scheduled publication updates only `latest`; release tags remain immutable. +11. Tools can consume a LocalDevStack-supplied canonical service catalog. +12. No control-plane feature requires LocalDevStack fixed container IPs. From afcb92a0db06c17430d59af5ad7c98b814de2435 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:21:26 +0600 Subject: [PATCH 007/509] docs: add docker-llm-sm integration plan --- .../docker-ecosystem/06-docker-llm-sm-plan.md | 209 ++++++++++++++++++ 1 file changed, 209 insertions(+) create mode 100644 docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md diff --git a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md new file mode 100644 index 00000000..3c272363 --- /dev/null +++ b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md @@ -0,0 +1,209 @@ +# docker-llm-sm — File-by-File Compatibility + Integration Plan + +## Role + +`infocyph/docker-llm-sm` is the optional local-AI service for the LocalDevStack ecosystem. It is already published and already uses the newer single-repository tag model: + +- standard CPU/NVIDIA: `latest`, `` +- AMD ROCm: `amd-latest`, `amd-` + +This plan is intentionally conservative. The image should remain independently useful outside LocalDevStack. + +## Invariants + +- LocalDevStack integration must remain optional. +- The default model remains replaceable by the user. +- `/root/.ollama` must be persisted by the consumer/orchestrator. +- CLI commands bundled in the image remain available; users do not install/remove the CLI separately. +- CPU/NVIDIA and AMD ROCm remain separate tag families within the same registry repository. +- LocalDevStack must not rebuild `docker-llm-sm` locally. +- Graphify or other AI clients consume the Ollama endpoint; they are not baked into this image merely because they can use it. + +## Existing Files + +### `.github/workflows/docker.publish.yml` + +Treat as the reference workflow for the older Docker repos. + +Plan: + +- keep single-repository standard/AMD tag families; +- keep release tags immutable; +- keep scheduled builds refreshing moving `latest`/`amd-latest` tags only; +- retain separate cache scopes/variant builds; +- retain Docker Hub + GHCR publication and attestations; +- only change if a later ecosystem-wide workflow improvement (SBOM/scanning/action major) is adopted consistently. + +### `.github/workflows/cli.check.yml` + +Keep as the CLI/Compose validation baseline. + +Potential additions only if needed by integration: + +- verify image-bundled CLI layout; +- verify persistent `/root/.ollama` contract; +- verify repo-aware command behavior against `/workspace` mount; +- verify no LocalDevStack-specific dependency enters the image. + +### `Dockerfile` + +Plan: + +- keep Ollama-based build and baked default model architecture; +- keep build-time model pull separated from runtime cloud-disable policy; +- preserve fixed in-image CLI installation; +- preserve healthcheck; +- do not add LocalDevStack-specific orchestration scripts; +- if ecosystem image metadata conventions are standardized, align labels without changing runtime behavior. + +### `.env.example` + +Keep standalone variables for image users. + +LocalDevStack should define its own integration variables rather than requiring users to copy this file. + +### `compose.yml` + +Keep as standalone CPU/default example. + +Do not alter it merely to match LocalDevStack’s profile layout. + +### `examples/compose/cpu.yml` + +Keep published-image/persistent-volume CPU example. + +### `examples/compose/nvidia.yml` + +Keep published-image/persistent-volume NVIDIA example and GPU request semantics. + +### `examples/compose/amd.yml` + +Keep `amd-latest`/ROCm example and `/dev/kfd` + `/dev/dri` device requirements. + +### `scripts/llm-sm` + +Keep as fixed dispatcher installed in the image. + +Integration requirement: + +- commands invoked through `docker exec` must work regardless of LocalDevStack container name chosen by profile; +- no host installer/uninstaller lifecycle. + +### `scripts/lib/core.sh` + +- keep model/API/config helpers generic; +- support environment overrides LocalDevStack can pass; +- no knowledge of LocalDevStack networks/profile names. + +### `scripts/lib/ollama.sh` + +- preserve internal Ollama CLI/API helpers; +- ensure endpoint assumptions work inside the same container. + +### `scripts/lib/commit.sh` + +- preserve Git workspace/safe-directory handling for mounted repositories; +- keep staged-diff behavior self-contained. + +### Command files + +Files include: + +- `commands/ai-commit.sh` +- `api.sh` +- `ask.sh` +- `chat.sh` +- `code.sh` +- `help.sh` +- `json.sh` +- `logs.sh` +- `models.sh` +- `ollama.sh` +- `prompt.sh` +- `ps.sh` +- `pull.sh` +- `restart.sh` +- `review.sh` +- `rm.sh` +- `run.sh` +- `show.sh` +- `start.sh` +- `status.sh` +- `stop.sh` +- `unload.sh` +- `version.sh` + +Plan for all command files: + +- keep image-local responsibilities clear; +- commands that manage models/prompts/API remain first-class; +- any start/stop/restart/log/status command must reflect the actual in-container Ollama process model and not imply Docker host lifecycle control; +- preserve exit codes suitable for `docker exec`; +- keep `ai-commit` able to use a mounted `/workspace` Git repository; +- keep stdin fallback where already supported; +- do not add LocalDevStack wrappers into this repository. + +### `scripts/prompts/ai-commit.txt` + +- keep bundled/self-contained prompt; +- version prompt changes with image releases; +- LocalDevStack must not override it by default. + +### `README.md` + +Keep standalone documentation authoritative for direct image users. + +Add LocalDevStack mention only after integration exists, and only as a short interoperability note/link. + +### `.dockerignore`, `.gitignore`, `.gitattributes`, `LICENSE` + +No LocalDevStack-driven change expected. + +## LocalDevStack Integration Contract + +LocalDevStack should add an optional `llm`/`ai` profile that consumes the published image. + +Suggested variables: + +- `LLM_SM_IMAGE=infocyph/llm-sm:latest` or pinned release; +- `LLM_SM_MODEL=` only when overriding image default/use selection; +- `LLM_SM_VOLUME=`; +- optional GPU mode selection: CPU/NVIDIA standard tag vs AMD tag; +- optional workspace mount path. + +Persistence: + +- mount named volume to `/root/.ollama`; +- preserve user-pulled models across container recreation/upgrades. + +Workspace: + +- optional `${PROJECT_DIR}` or selected project path -> `/workspace`; +- working directory `/workspace` where repo-aware commands are desired; +- mount must be declared at container creation time; +- do not require workspace mount for normal inference/API use. + +Networking: + +- join the appropriate LocalDevStack internal network by service name; +- expose `11434` to host only if local host tools (Graphify/editor integrations/etc.) need it; +- if exposed, default host binding should remain loopback-oriented for a local dev stack; +- other containers should use service DNS, e.g. `http://llm-sm:11434`. + +Graphify/client integration: + +- point client to Ollama/OpenAI-compatible endpoint exposed by `llm-sm`; +- keep `qwen2.5:3b` usable as the default small model; +- allow users to pull/select larger models without changing LocalDevStack image definitions. + +## Acceptance Criteria + +1. Existing published `docker-llm-sm` behavior remains standalone and stable. +2. LocalDevStack can enable it without building locally. +3. Standard and AMD tags are selectable. +4. Named volume persists pulled models across container recreation. +5. Other LocalDevStack containers can reach Ollama by service DNS. +6. Host clients can reach it through an explicitly configured loopback port when enabled. +7. `docker exec llm-sm ...` works under LocalDevStack. +8. Mounted project repo supports `llm-sm ai-commit` without host installation. +9. No Graphify/LocalDevStack-specific package is added to the image solely for integration. From 3ea5f93778e87f771476173d6b68e9e36ed91ca3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 11:22:35 +0600 Subject: [PATCH 008/509] docs: add LocalDevStack integration plan --- .../07-localdevstack-integration-plan.md | 497 ++++++++++++++++++ 1 file changed, 497 insertions(+) create mode 100644 docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md new file mode 100644 index 00000000..721bae02 --- /dev/null +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -0,0 +1,497 @@ +# LocalDevStack — File-by-File Integration + Product Plan + +## Role + +`infocyph/LocalDevStack` is the product/orchestrator above all supporting Docker images. It should present a simple XAMPP-like local-development experience while hiding most Docker complexity. + +This phase executes only after lower-level image contracts are stable. + +## Product Invariants + +- `lds` remains the single primary CLI. +- `lds.bat` keeps Windows/Git Bash interoperability. +- PHP and Node remain locally generated/customizable runtimes. +- Nginx remains the default HTTP/TLS edge. +- Apache remains optional. +- databases/admin clients/mail/runner remain selectively enabled through profiles/configuration. +- local TLS and multiple domains remain first-class. +- AI remains optional. +- persisted DB/cache/mail/model data survives container recreation. +- project source remains host-mounted, not copied into infrastructure images. + +## Top-Level Files + +### New `.github/workflows/check.yml` + +Add product-level CI before major refactoring. + +Required stages: + +1. `bash -n` for `lds` and Bash wrappers; +2. ShellCheck for maintained scripts with explicit suppressions; +3. validate `lds.bat` static expectations where Windows runners are practical; +4. Compose configuration validation with representative profile sets; +5. generated PHP runtime Compose fixture; +6. generated Node runtime Compose fixture; +7. canonical service-catalog schema validation; +8. smoke `lds help`, `lds config`, profile parsing, env handling without destructive host operations; +9. integration job using released supporting images for core Nginx/Tools/Runner flow; +10. optional heavier domain/TLS/runtime smoke job. + +Use Linux CI as baseline; add Windows CI for bridge/path-specific behavior rather than trying to run every container integration twice. + +### `.gitignore` + +Current file is allowlist-oriented. + +Plan: + +- keep the allowlist model if intentional; +- whitelist new static product resources such as `docker/catalog/**`, tests and CI fixtures; +- keep generated runtime `.env`, certificates, secrets, logs and user-generated Compose artifacts ignored; +- make the tracked-vs-generated boundary explicit. + +### `.gitattributes` + +Preserve LF shell scripts and Windows batch compatibility. Add path-specific rules only if needed. + +### `README.md` + +Rewrite after implementation to present the product as a Docker-based XAMPP alternative for PHP + Node. + +Quickstart should prioritize: + +- install Docker; +- clone/setup `lds`; +- select services/runtime; +- create domain; +- trust local CA; +- work through `lds php`, `lds composer`, Node/npm commands and service shortcuts. + +Keep advanced internals in docs, not the first screen. + +### `LICENSE` + +No change. + +## `lds` Modularization + +### Existing `lds` + +Do not rewrite behavior in one step. + +Migration sequence: + +1. add characterization tests around current commands; +2. identify stable command/public-output contracts; +3. extract internal modules while leaving `lds` as bootstrap/dispatcher; +4. only then simplify duplicated code. + +Target layout: + +- `lds` — bootstrap, global flag parsing, command dispatch; +- `lib/core.sh` — errors, output, command/tool lookup, OS detection; +- `lib/compose.sh` — Compose wrapper, service resolution, profiles/extras; +- `lib/env.sh` — dotenv read/write/quoting and product defaults; +- `lib/catalog.sh` — canonical service/runtime catalog reader; +- `lib/profiles.sh` — service/profile selection and persistence; +- `lib/hosts.sh` — mkhost/rmhost integration and reload lifecycle; +- `lib/certificates.sh` — host trust-store install/uninstall; +- `lib/runtime.sh` — PHP/Node/runtime-image generation/rebuild integration; +- `lib/diagnostics.sh` — doctor/diag/sniff/status helpers; +- `lib/maintenance.sh` — clean/disk/events/rebuild operations; +- `lib/platform.sh` — Windows/macOS/Linux path/platform helpers when separation is useful. + +Rules: + +- sourced modules are not user-facing commands; +- no module should execute work merely when sourced; +- keep `set -euo pipefail` behavior intentional; +- avoid global mutable state where function-local state works; +- preserve existing command aliases until a documented deprecation. + +### `lds.bat` + +Plan: + +- preserve Git-for-Windows Bash bridge; +- preserve caller working directory and Windows->Unix path conversion; +- validate Docker installed/running errors; +- add CI/static smoke for quoting paths containing spaces; +- avoid duplicating `lds` command semantics in batch. + +## Canonical Service Catalog + +### New `docker/catalog/services.json` + +Create one canonical product catalog consumed by `lds` and mounted/read by `docker-tools`. + +Initial schema should describe at least: + +- service/profile key; +- display name; +- image/version env variable; +- default version/value; +- setup prompt fields/defaults; +- related admin client profile/service where applicable; +- convenience route metadata where useful; +- persistence volume identifier; +- optional health/dependency metadata only when needed by orchestration. + +Initial catalog covers: + +- PostgreSQL; +- MySQL; +- MariaDB; +- MongoDB; +- Redis; +- Elasticsearch; +- optional AI service metadata in a separate capability section or same versioned schema. + +Do not put secrets directly in catalog. Default dev credentials may be expressed as defaults but user values live in env state. + +### Remove duplicated profile defaults + +After `docker-tools` supports external catalog: + +- replace hard-coded `SERVICES`/`PROFILE_ENV` duplication in `lds` with catalog reads; +- mount catalog into `server-tools`; +- remove Tools fallback duplication only after compatibility period/tests. + +## `bin/` Wrappers + +### `bin/tool-runner` + +- keep common container execution/path/TTY logic centralized; +- make it the reusable primitive for thin service wrappers where possible; +- ensure Windows path conversion and UID/GID behavior are tested; +- propagate exit codes and signals. + +### `bin/php` + +- preserve runtime selection/highest-version fallback; +- keep explicit `--php/-V` selection; +- preserve ad-hoc execution using the selected runtime image; +- validate bind mount and user mapping on Linux/macOS/Windows; +- keep `serve` mode if still useful, but separate it cleanly from normal CLI execution; +- align Git safe-directory behavior for mounted projects. + +### `bin/composer` + +- reduce duplicated runtime-resolution code by consuming `tool-runner`/shared helper where possible; +- preserve PHP-version selection and versioned Composer home; +- test install/update/global operations against mounted project. + +### `bin/my` + +- preserve MySQL login/query/export/import helper behavior; +- resolve service by Compose name/labels, not static IP; +- keep credentials from environment; +- validate quoting and database/file paths. + +### `bin/maria` + +Same principles as `bin/my`, using MariaDB client/service contract. + +### `bin/pg` + +- preserve PostgreSQL login/query/dump/restore helpers; +- DNS service name only; +- environment-driven credentials; +- test dump/restore with temporary DB. + +### `bin/mongo` + +- preserve mongosh/login/import/export helpers; +- DNS service name only; +- environment-driven auth; +- no replica-set assumption in default flow. + +### `bin/redis-cli` + +- preserve thin Redis CLI behavior; +- DNS service name and optional auth env; +- keep wrapper small. + +### `bin/es` + +- preserve Elasticsearch API/helper behavior; +- use service name rather than fixed IP; +- validate ES 9.x/current compatibility via version-aware requests; +- no hidden Kibana dependency for core ES commands. + +## Docker Compose Files + +### `docker/compose/main.yaml` + +Major network migration: + +- keep logical `frontend`, `backend`, `datastore` bridge networks; +- remove hard-coded subnets/gateways after integration validation; +- remove need for fixed container IPv4 values from included files; +- keep named persistent volumes; +- add AI model volume only when AI profile integration lands; +- consider namespacing globally fixed volume/network names if multi-stack coexistence needs it; do not break existing data without migration guidance. + +### `docker/compose/companion.yaml` + +Services: `server-tools`, `runner`, `mailpit`. + +Plan: + +- replace static IPv4 declarations with networks/service DNS; +- move infrastructure image references from hard-coded `:latest` to environment-controlled compatibility-tested versions; +- mount canonical service catalog into `server-tools` read-only; +- inventory Docker socket usage for both Tools and Runner; +- preserve project/config/scheduler/SOPS/SSH/SSL/log mounts; +- preserve Mailpit persistent data and local TLS; +- add health/dependency conditions only where they improve deterministic startup without deadlocks. + +### `docker/compose/http.yaml` + +Services: Nginx + optional Apache. + +Plan: + +- remove static IPs; +- version image refs through LocalDevStack defaults/env; +- keep Nginx host 80/443 binding configurable; +- preserve project/vhost/cert/rootCA/FPM socket/log mounts; +- preserve Apache as internal backend; +- use service names for dependency/routing; +- reconsider `restart: always` vs consistent `unless-stopped` behavior for local dev. + +### `docker/compose/db.yaml` + +Services: Redis, PostgreSQL, MySQL, MongoDB, MariaDB, Elasticsearch. + +Plan: + +- remove all static IPv4 assignments; +- source default version/env metadata from canonical catalog; +- retain named volumes; +- review data-directory mount correctness per current upstream images (especially PostgreSQL version changes); +- retain healthchecks but correct variable-name mismatches (`POSTGRES_DATABASE` vs `POSTGRES_DB`, etc.); +- avoid `latest` defaults for compatibility-sensitive DB majors where a stable major is preferable; +- keep local-development credentials configurable and clearly non-production. + +### `docker/compose/db-client.yaml` + +Services: RedisInsight, CloudBeaver, Mongo Express, Kibana, Filebeat. + +Plan: + +- remove static IP assignments; +- use datastore service DNS; +- keep profile coupling explicit; +- version client images where breaking major drift is possible; +- validate persistent workspace/data volumes; +- ensure clients tolerate target DB starting later (health/retry rather than immediate failure where possible). + +## PHP / Node Runtime Dockerfiles + +### `docker/dockerfiles/php.Dockerfile` + +- keep `ARG PHP_VERSION` + upstream `php:-fpm-alpine` model; +- add `SCRIPTOMATIC_REF` and `TOOLSET_REF`/related immutable dependency inputs; +- stop fetching setup script from floating Scriptomatic master; +- preserve UID/GID, extension/package and profile-key build args; +- validate image as non-root developer user and PHP-FPM service; +- keep build local because combinations are user-selected. + +### `docker/dockerfiles/node.Dockerfile` + +- keep selectable upstream Node Alpine version; +- pin Scriptomatic/Toolset helper revisions; +- preserve UID/GID/packages/globals customization; +- preserve non-root user; +- validate npm/corepack and project startup behavior; +- keep build local. + +## Docker Config Files + +### `docker/conf/filebeat.yml` + +- validate against current Filebeat/Elasticsearch major; +- paths must match mounted LocalDevStack log layout; +- no fixed IP endpoints. + +### `docker/conf/openssl.cnf` + +- validate local dev compatibility with generated certificates/current OpenSSL; +- avoid weakening global TLS unnecessarily; +- document why overrides exist. + +### `docker/conf/pg_hba.conf` + +- review trust/auth scope for isolated local Docker network; +- preserve password auth expectations; +- no fixed subnet assumptions after network migration. + +### `docker/conf/postgresql.conf` + +- decide whether it remains intentionally unused/commented or should become active; +- if unused, avoid presenting it as active configuration in docs; +- if enabled, version-test it. + +### `docker/conf/www-php.conf` + +- validate FPM pool include/listen behavior against generated per-domain pools; +- ensure user/group/socket permissions align with Nginx/Apache access. + +### `docker/conf/www.conf` + +- classify as reference/upstream-derived vs active file; +- remove/deprecate if no runtime path consumes it, but only after search/tests. + +## Generated/User Configuration Directories + +### `configuration/compose/` + +Continue as generated Compose overrides. Add validation that generated files are valid and stale overrides can be detected/cleaned safely. + +### `configuration/php/php.ini` + +Remain user override file. Provide documented defaults/examples rather than overwriting user edits during update. + +### `configuration/scheduler/cron-jobs/` + +Remain user/generated scheduler definitions. Validate filename/permissions/content before Runner consumes them. + +### `configuration/scheduler/supervisor/` + +Remain user/generated supervisor definitions. Validate configs before stack restart. + +### `configuration/sops/config`, `global`, `keys` + +Keep sensitive/state files ignored. Tighten permissions via setup without committing contents. + +### `configuration/ssh/` + +Keep optional read-only mount. Never copy private keys into images. + +### `configuration/ssl/` + +Keep generated cert artifacts/state according to the chosen named-volume/bind-mount model. Reconcile docs with actual source of truth. + +## Logs + +### `logs/` + +- keep host-visible logs where that is a deliberate developer feature; +- avoid `chmod -R 777` if cross-platform/container UID tests show a safer workable model; +- if permissive mode remains necessary, document local-only rationale; +- ensure rotation behavior matches Runner configuration. + +## Optional AI Compose Integration + +### New `docker/compose/ai.yaml` + +Add only after core networking/catalog changes are stable. + +Service design: + +- profile `ai` (or `llm`, decide one canonical public name); +- default published image `infocyph/llm-sm:` through env variable; +- AMD selectable through `amd-` tag, not second repository; +- named volume -> `/root/.ollama`; +- optional project/workspace mount -> `/workspace`; +- loopback host port optional/configurable; +- join network for service-DNS access from other containers; +- GPU options handled through explicit override/profile rather than auto-detect magic that makes Compose unreliable across hosts. + +Graphify/editor/other clients remain external consumers of the endpoint. + +## Image Version Defaults + +### New `docker/images.env` or equivalent committed defaults + +Define compatibility-tested infrastructure image versions centrally, for example conceptual keys: + +- `LDS_TOOLS_IMAGE=infocyph/tools:` +- `LDS_RUNNER_IMAGE=infocyph/runner:` +- `LDS_NGINX_IMAGE=infocyph/nginx:` +- `LDS_APACHE_IMAGE=infocyph/apache:` +- `LDS_LLM_IMAGE=infocyph/llm-sm:` when AI enabled. + +User `.env` may override them. A LocalDevStack release should not depend solely on whatever `latest` means that day. + +## Documentation Files + +### `docs/concepts/architecture.rst` + +Update architecture diagram/responsibility boundaries, canonical catalog, optional AI and DNS-based networking. + +### `docs/concepts/profiles-and-env.rst` + +Document canonical catalog, env override precedence, infrastructure image versions and generated runtime profiles. + +### `docs/concepts/storage-layout.rst` + +Reconcile named volumes vs host `configuration/` directories. Clearly distinguish: + +- persisted Docker named volumes; +- host-generated config; +- project source mounts; +- logs; +- secrets/SSH; +- optional AI model volume. + +### `docs/quickstart.rst` + +Update after final CLI flow is stable; keep XAMPP-like beginner path concise. + +### `docs/guides/domain-setup.rst` + +Document Docker-DNS routing and domain creation without static IP assumptions. + +### `docs/guides/tls-and-certificates.rst` + +Keep cross-platform trust instructions synchronized with actual `lds certificate` behavior. + +### `docs/guides/secrets-sops-age.rst` + +Validate against current Tools `senv` contract. + +### `docs/guides/notifications.rst` + +Validate notifier contract and clarify optional nature. + +### `.readthedocs.yaml` / `docs/conf.py` / `docs/requirements.txt` + +Pin/document docs dependencies enough for reproducible docs builds; validate Read the Docs build in CI if useful. + +## Static Networking Migration Sequence + +1. Add CI/tests resolving all services by DNS name. +2. Search all product/support repos for fixed `172.28/29/30` dependencies. +3. Remove per-service `ipv4_address` declarations from Compose files. +4. Remove IPAM subnet/gateway blocks from `main.yaml`. +5. Run PHP/Node/DB/admin/domain/TLS integration tests. +6. Re-evaluate `lds vpn-fix`: + - delete/deprecate if its only purpose was static-subnet conflict; + - retain only independently useful VPN behavior with updated docs. + +## Docker Socket Review Sequence + +1. Trace every `docker` command in Tools/Runner and admin panel. +2. Categorize read vs write operations. +3. Determine if Runner requires socket directly or only specific mounted jobs do. +4. Determine if Tools requires full socket for domain/profile/admin functionality. +5. Keep required access for local-dev UX; remove redundant mounts. +6. Document trust boundary prominently. + +## Acceptance Criteria + +1. Product CI exists and covers CLI/Compose/runtime generation. +2. `lds` public command surface remains compatible after modularization. +3. Canonical service catalog is consumed by both LocalDevStack and Tools. +4. Infrastructure image defaults are compatibility-tested/pinnable instead of unconditional `latest`. +5. PHP and Node remain dynamically customizable local builds. +6. All core service communication works through Docker DNS without fixed IPv4 assignments. +7. Local domain/TLS flows work on supported host classes. +8. DB/cache/mail/admin clients persist data appropriately. +9. Runner/scheduler/supervisor workflows remain functional. +10. Docker socket mounts are justified and minimized. +11. Optional `llm-sm` profile works with persistent models and published images only. +12. Documentation matches actual networking/storage/runtime behavior. From b53b1d271790588b65c2a1b32c19fce7bbcac92d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 17:19:46 +0600 Subject: [PATCH 009/509] docs: align shared foundations with hardened Scriptomatic contract --- .../01-shared-foundations-plan.md | 340 +++++++++--------- 1 file changed, 170 insertions(+), 170 deletions(-) diff --git a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md index d90bcd34..5af22700 100644 --- a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md +++ b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md @@ -2,248 +2,248 @@ ## Purpose -Stabilize the two shared repositories consumed by LocalDevStack Docker images before changing the image layer itself. +Stabilize the shared Scriptomatic/Toolset dependency contract consumed by LocalDevStack Docker images before changing the image layer itself. -This is a dependency-contract plan, not a wholesale redesign of Scriptomatic or Toolset. +This remains a dependency-contract plan. Scriptomatic owns reusable bootstrap/runtime behavior; Toolset owns released CLI utilities; LocalDevStack owns image composition/orchestration. -## Repositories +## Repositories and accepted upstream contracts - `infocyph/Scriptomatic` + - canonical source: `main` + - reproducible consumers: `SCRIPTOMATIC_REF=` + - no Scriptomatic tag/release lifecycle required + - all sibling Scriptomatic helpers use the same selected ref - `infocyph/Toolset` + - stable Docker-ecosystem dependency: `TOOLSET_REF=2.0` + - released executables/checksum contract; do not consume Toolset `main`/`master` -## Primary Problems to Solve +## Current LocalDevStack mismatch to remove -1. Docker builds currently fetch helpers from mutable `main`/`master` URLs. -2. A rebuild of unchanged Docker source can therefore silently receive different helper code. -3. Shell/bootstrap behavior is consumed by several images without a shared compatibility contract. -4. Some helper scripts are large enough that syntax-only confidence is insufficient. +The current PHP and Node Dockerfiles still contain: -## Contract to Establish +```text +https://raw.githubusercontent.com/infocyph/Scriptomatic/master/... +``` -Every downstream image must be able to select immutable revisions through explicit build arguments, for example: +This must be removed during the LocalDevStack implementation phase. -- `SCRIPTOMATIC_REF=` -- `TOOLSET_REF=` +Do not replace it with another hard-coded mutable URL. Add explicit build arguments and propagate the same selected ref into the bootstrap script. -Preferred source format: +Required downstream inputs: -`https://raw.githubusercontent.com/infocyph///` +```text +SCRIPTOMATIC_REF=main +SCRIPTOMATIC_BASE_URL=https://raw.githubusercontent.com/infocyph/Scriptomatic +TOOLSET_REF=2.0 +SCRIPTOMATIC_UID= +SCRIPTOMATIC_GID= +``` -If release tags are used, treat released helper content as immutable. If the repos do not yet maintain releases consistently, use commit SHAs until that lifecycle exists. +For immutable/reproducible image builds, `SCRIPTOMATIC_REF` must be the accepted Scriptomatic commit SHA rather than `main`. -## Scriptomatic — File-by-File Plan +## Initial Scriptomatic bootstrap acquisition -### `.github/` +The Dockerfile must acquire the initial `php-cli-setup.sh` / `node-cli-setup.sh` through the selected ref with bounded download behavior. Do not use remote `ADD` against `master`. -Create validation workflow(s) for shell scripts. +Recommended shape: -Required checks: +```dockerfile +ARG SCRIPTOMATIC_REF=main +ARG SCRIPTOMATIC_BASE_URL=https://raw.githubusercontent.com/infocyph/Scriptomatic +ARG TOOLSET_REF=2.0 -- `bash -n bash/*.sh` where Bash is required; -- `sh -n` only for scripts explicitly POSIX-shell compatible; -- ShellCheck with documented intentional suppressions only; -- smoke tests for PHP and Node setup scripts using disposable containers where practical. +RUN apk add --no-cache bash curl ca-certificates && \ + curl --fail --location --silent --show-error \ + --connect-timeout 5 --max-time 90 --retry 3 \ + "${SCRIPTOMATIC_BASE_URL}/${SCRIPTOMATIC_REF}/bash/php-cli-setup.sh" \ + -o /usr/local/bin/cli-setup.sh && \ + SCRIPTOMATIC_REF="${SCRIPTOMATIC_REF}" \ + SCRIPTOMATIC_BASE_URL="${SCRIPTOMATIC_BASE_URL}" \ + TOOLSET_REF="${TOOLSET_REF}" \ + SCRIPTOMATIC_UID="${UID}" \ + SCRIPTOMATIC_GID="${GID}" \ + bash /usr/local/bin/cli-setup.sh "${USERNAME}" "${PHP_VERSION}" +``` -Do not add a release/publish workflow unless Scriptomatic is intentionally moved to a release-tag lifecycle. +Use the analogous path for Node. -### `bash/php-cli-setup.sh` +## Scriptomatic accepted behavior -Current role: builds LocalDevStack PHP developer-runtime behavior. +### PHP build/runtime -Plan: +`php-cli-setup.sh` now provides: -- preserve PHP extension/package customization inputs; -- preserve UID/GID user creation, Composer home isolation, FPM setup, Mailpit/msmtp, Git configuration and shell helpers; -- replace floating Toolset/Scriptomatic helper URLs with ref-driven URLs supplied through environment/build context; -- fail clearly when a required helper cannot be fetched; -- avoid `latest`-style helper installers where an immutable version/checksum can be selected; -- validate generated PHP/FPM configuration in smoke CI; -- ensure cleanup does not remove runtime-required assets; -- document inputs used by LocalDevStack Dockerfile (`PHP_EXT`, `PHP_EXT_VERSIONED`, `LINUX_PKG`, `LINUX_PKG_VERSIONED`, UID/GID, profile key). +- validated package/extension inputs; +- Alpine official-PHP-image capability checks; +- pinned/verified PHP extension installer; +- no implicit Composer self-update; +- Toolset `2.0` helper installation from released/checksummed assets; +- same-ref Scriptomatic helper installation; +- root-owned `/usr/local/bin` helpers; +- generated PHP/FPM validation; +- repeat/idempotent bootstrap coverage; +- no broad shared-temp cleanup/self-delete. -### `bash/node-cli-setup.sh` +`php-entry.sh` provides content-aware mounted-root-CA refresh and transparent `exec docker-php-entrypoint "$@"` semantics. -Current role: builds LocalDevStack Node developer-runtime behavior. +### Node build/runtime -Plan: +`node-cli-setup.sh` now provides: -- preserve UID/GID reuse/rename behavior for the upstream `node` user; -- preserve npm cache/global prefix and Corepack behavior; -- make Toolset/Scriptomatic helper downloads immutable/ref-driven; -- review whether unconditional `npm@latest` upgrade is desirable for reproducible runtime images; prefer explicit npm policy/version input if retained; -- validate generated user/home/global-package permissions; -- smoke-test a generated Node runtime as non-root. +- validated package/global-package inputs; +- verified upstream UID reuse/rename behavior; +- optional exact npm version rather than `npm@latest`/`npm@next`; +- reproducible global-package mode; +- Toolset `2.0` and same-ref Scriptomatic helpers; +- root-owned shared executables. -### `bash/php-entry.sh` +`node-entry.sh` defaults to: -Plan: +```text +NODE_LOG_ENABLED=0 +NODE_KEEPALIVE_ON_FAIL=0 +NODE_AUTO_INSTALL=0 +NODE_ALLOW_LOCKFILE_FALLBACK=0 +``` -- syntax check; -- validate signal/exec semantics; -- ensure mounted CA/config initialization remains idempotent; -- keep entrypoint small; do not move build-time setup into runtime. +It selects one final command and `exec`s it. Direct argv is preferred; `NODE_CMD` is only a trusted compatibility escape hatch. -### `bash/node-entry.sh` +## Trusted development sudo / root CA -Plan: +Scriptomatic now defaults passwordless sudo off: -- syntax check; -- validate command forwarding and signal semantics; -- ensure runtime setup is idempotent; -- keep project command execution as the final `exec` path. +```text +SCRIPTOMATIC_PASSWORDLESS_SUDO=0 +``` -### `bash/banner.sh` +LocalDevStack PHP/Node images run as non-root at runtime. If the mounted root CA must be copied/refreshed during entrypoint startup, the trusted development image must explicitly build with: -Plan: +```text +SCRIPTOMATIC_PASSWORDLESS_SUDO=1 +``` -- treat as shared presentation helper only; -- syntax/ShellCheck validation; -- no runtime-critical logic should depend on banner rendering succeeding; -- downstream images should pin its revision rather than fetch `master`. +Do not make this implicit in Scriptomatic. LocalDevStack owns this trust choice. -### `bash/alias-maker.sh` +Use `ROOTCA_REQUIRED=1` only where inability to install the mounted CA should make startup fail. -Plan: +## Shared utility contract -- validate idempotency; -- ensure aliases do not hide core commands in non-interactive execution; -- document which aliases are relied on by LocalDevStack developer shells. +### `alias-maker.sh` -### `bash/docknotify.sh` +- managed/idempotent `.bashrc` block; +- does not interfere with non-interactive application execution; +- optional aliases degrade cleanly. -Plan: +### `banner.sh` -- validate behavior when notification transport is unavailable; -- never make normal PHP/Node process startup depend on desktop notification success; -- add minimal smoke coverage for no-listener behavior. +- presentation-only; +- safe fallback without `figlet`/`chromacat`; +- non-TTY / `NO_COLOR` safe; +- banner failure cannot prevent shell/container startup. -### `bash/owners.sh` +### `docknotify.sh` -Plan: +LocalDevStack-compatible defaults remain: -- confirm whether current LocalDevStack images still consume it; -- if unused by all current Dockerfiles, mark as standalone Scriptomatic utility rather than part of the LocalDevStack compatibility contract. +```text +NOTIFY_HOST=SERVER_TOOLS +NOTIFY_TCP_PORT=9901 +DOCKNOTIFY_STRICT=0 +``` -### `bash/mongo-replica.sh` +Notification is best-effort unless strict mode is explicitly requested. The protocol is one tab-separated newline-terminated record and token data is not emitted in diagnostics. -Plan: +### `owners.sh` -- determine whether LocalDevStack currently calls it; -- if retained for future Mongo replica support, add syntax validation and document expected container/network assumptions; -- do not introduce replica-set complexity into default LocalDevStack profiles as part of this program. +Standalone repository utility; not part of the critical LocalDevStack runtime contract unless adopted explicitly. -### `bash/certbot-hook.sh` / `bash/certbot-renew.sh` +## Service-helper contract -Plan: +### Certbot -- explicitly classify as production/server helpers, not LocalDevStack local-TLS dependencies; -- leave out of LocalDevStack compatibility gating unless direct usage is found. +`certbot-hook.sh` is Docker-control-plane behavior: -### `bash/alias-maker.sh`, `banner.sh`, `docknotify.sh`, `php-entry.sh`, `node-entry.sh` +- exact container inspection; +- no TTY; +- configurable Nginx/Apache container names; +- bounded reload; +- missing optional target skips; +- stopped/reload-failed configured target fails. -Add a compact compatibility test matrix because these are directly pulled into runtime images. +Do not mount the Docker socket into ordinary PHP/Node application containers just to support this helper. -## Toolset — File-by-File Plan +`certbot-renew.sh` is a signal-aware foreground service loop with interval/jitter/failure-threshold controls. -Only utilities directly consumed by the Docker ecosystem are in the critical path. +### Mongo replica bootstrap -### `Git/gitx` +`mongo-replica.sh`: -Current consumers: `docker-tools`, PHP runtime setup, Node runtime setup. +- uses bounded readiness rather than fixed sleeps; +- prefers `mongosh` with legacy `mongo` fallback; +- separates connection URI from advertised replica members; +- defaults advertised members to Docker DNS names; +- is idempotent for a matching topology; +- initializes only when uninitialized; +- refuses conflicting existing topology. -Plan: +Default advertised topology: -- establish an immutable ref used by images; -- run Bash syntax/ShellCheck if compatible with its implementation style; -- add smoke tests for non-destructive commands needed inside dev containers; -- do not couple `gitx` to `llm-sm`; AI commit behavior already lives independently in `docker-llm-sm`. +```text +mongo-primary:27017 +mongo-secondary1:27017 +mongo-secondary2:27017 +``` -### `ChromaCat/chromacat` +Do not replace service names with static `172.x` addresses. -Current consumers: tools/runner/nginx/apache/PHP/Node developer shells. +## Shell compatibility boundary -Plan: +LocalDevStack developer workflows may use `bash`, `sh`, and `sh -l`. -- pin downstream consumption; -- validate no-color/non-TTY behavior; -- make banner/color failures non-critical; -- keep it purely presentational. +- standalone helpers under `/usr/local/bin` have their own shebangs; +- Bash-specific profile/alias behavior belongs to Bash/login presentation paths; +- non-login `sh` is not required to source Bash-only configuration; +- PHP/Node application entrypoints do not depend on interactive shell startup. -### `Sqlite/sqlitex` +## Toolset accepted dependency contract -Current consumer: `docker-tools`. +The Docker ecosystem consumes Toolset stable `2.0` rather than mutable repository branches. -Plan: +Critical current utilities remain: -- pin downstream consumption; -- smoke-test basic database open/query behavior against temporary SQLite data; -- document runtime package dependency expectations. +- `gitx` +- `chromacat` +- `sqlitex` where docker-tools requires it +- `netx` where docker-tools requires it -### `Network/netx` +Do not add `dockex`, `phpx`, or `cleanx` to LocalDevStack merely for symmetry; adoption requires a concrete downstream need. -Current consumer: `docker-tools`. +## Permanent upstream validation -Plan: +Scriptomatic CI now covers: -- pin downstream consumption; -- smoke-test basic local network inspection without requiring privileged host operations; -- ensure failures are diagnostic rather than destructive. +- syntax + ShellCheck; +- repository-wide security audit; +- PHP Alpine bootstrap + repeated execution; +- Node Alpine bootstrap including UID reuse/fresh-user paths; +- entrypoint exit/signal behavior; +- shared utility fixtures; +- Certbot/Mongo deterministic service-helper fixtures; +- aggregate gate. -### `Docker/dockex` +Toolset keeps its own permanent release/utility gates. -Plan: +Do not duplicate these upstream suites inside LocalDevStack. LocalDevStack should add consumer/integration tests that prove its Dockerfiles pass the correct refs/options and that generated images start correctly. -- inspect whether LocalDevStack or `docker-tools` currently installs/calls it; -- if unused, do not add it merely for symmetry; -- if later adopted, treat Docker-socket permission requirements explicitly. +## LocalDevStack implementation acceptance criteria -### `PHP/phpx` - -Plan: - -- inspect whether current PHP wrappers already provide the required behavior; -- do not introduce it into LocalDevStack runtime images unless it replaces duplicated functionality with a clear compatibility win. - -### `Clean/cleanx` - -Plan: - -- keep out of default container images unless a concrete LocalDevStack command adopts it; -- destructive cleanup remains explicit and host-controlled. - -### Toolset docs/README files - -Update only when dependency/release guarantees change. Do not rewrite unrelated documentation during Docker ecosystem work. - -## New Shared Validation Artifacts - -If missing, add lightweight test directories/workflows in Scriptomatic and Toolset rather than embedding compatibility tests into downstream Dockerfiles. - -Suggested test categories: - -- syntax; -- non-interactive execution; -- no-color mode; -- temporary HOME/user paths; -- read-only Git/config mounts; -- expected failure behavior when optional host integrations are absent. - -## Downstream Consumption Pattern - -Each downstream Dockerfile should stop hard-coding: - -- `Toolset/main/...` -- `Scriptomatic/master/...` - -and move to explicit build args/defaults tied to accepted immutable revisions. - -Do not duplicate Scriptomatic/Toolset source into every Docker repo unless GitHub availability at build time becomes an unacceptable dependency and a vendoring decision is made intentionally. - -## Acceptance Criteria - -1. Required Scriptomatic scripts pass syntax + ShellCheck policy. -2. PHP and Node setup smoke tests pass in disposable upstream base containers. -3. Toolset utilities used by the Docker ecosystem have smoke validation. -4. Every downstream image can point to immutable Scriptomatic/Toolset refs. -5. Rebuilding the same Docker release source with the same base image/ref inputs does not silently receive newer helper scripts. -6. Optional presentation/notification helpers cannot prevent the primary container service from starting. +1. No Dockerfile consumes `Scriptomatic/master` or `Toolset/main`/`master`. +2. PHP/Node image builds expose `SCRIPTOMATIC_REF` and `TOOLSET_REF` build inputs. +3. Reproducible builds can pin Scriptomatic by commit SHA and Toolset by accepted stable release. +4. Explicit `SCRIPTOMATIC_UID`/`SCRIPTOMATIC_GID` are passed into setup. +5. Trusted development sudo is enabled only where the runtime CA workflow needs it. +6. PHP/Node images remain non-root at runtime and preserve entrypoint `exec` semantics. +7. `bash`, `sh`, and `sh -l` remain usable for their intended roles. +8. `docknotify` can reach `SERVER_TOOLS:9901` when enabled and remains non-critical when unavailable. +9. service-to-service references use Docker DNS/service names, not static IP assumptions. +10. consumer CI builds representative PHP and Node images using the pinned shared-foundation refs. From beff19a38fdb24ad4dab941cff9fdd685829bb90 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 18:01:43 +0600 Subject: [PATCH 010/509] docs: align shared foundations with preserved Scriptomatic behavior --- .../01-shared-foundations-plan.md | 45 ++++++++++--------- 1 file changed, 25 insertions(+), 20 deletions(-) diff --git a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md index 5af22700..3796790a 100644 --- a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md +++ b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md @@ -76,12 +76,14 @@ Use the analogous path for Node. - validated package/extension inputs; - Alpine official-PHP-image capability checks; - pinned/verified PHP extension installer; -- no implicit Composer self-update; +- Composer available by default through pinned `COMPOSER_VERSION=2.10.3`, without the old floating self-update; - Toolset `2.0` helper installation from released/checksummed assets; - same-ref Scriptomatic helper installation; - root-owned `/usr/local/bin` helpers; - generated PHP/FPM validation; - repeat/idempotent bootstrap coverage; +- preserved trusted-development shell defaults (passwordless sudo and Oh My Bash) with explicit opt-out controls; +- preserved setup progress/banner presentation; - no broad shared-temp cleanup/self-delete. `php-entry.sh` provides content-aware mounted-root-CA refresh and transparent `exec docker-php-entrypoint "$@"` semantics. @@ -95,35 +97,37 @@ Use the analogous path for Node. - optional exact npm version rather than `npm@latest`/`npm@next`; - reproducible global-package mode; - Toolset `2.0` and same-ref Scriptomatic helpers; -- root-owned shared executables. +- root-owned shared executables; +- preserved trusted-development shell defaults (passwordless sudo and Oh My Bash) with explicit opt-out controls; +- preserved setup progress/banner presentation. -`node-entry.sh` defaults to: +`node-entry.sh` keeps the established LocalDevStack developer-container defaults: ```text -NODE_LOG_ENABLED=0 -NODE_KEEPALIVE_ON_FAIL=0 -NODE_AUTO_INSTALL=0 -NODE_ALLOW_LOCKFILE_FALLBACK=0 +NODE_LOG_ENABLED=1 +NODE_KEEPALIVE_ON_FAIL=1 +NODE_AUTO_INSTALL=1 +NODE_ALLOW_LOCKFILE_FALLBACK=1 ``` -It selects one final command and `exec`s it. Direct argv is preferred; `NODE_CMD` is only a trusted compatibility escape hatch. +Each can be set to `0` for stricter/production-like behavior. The entrypoint selects one final command and `exec`s it. Direct argv is preferred; `NODE_CMD` remains a trusted compatibility escape hatch. ## Trusted development sudo / root CA -Scriptomatic now defaults passwordless sudo off: +Scriptomatic keeps the historical trusted-development default: ```text -SCRIPTOMATIC_PASSWORDLESS_SUDO=0 +SCRIPTOMATIC_PASSWORDLESS_SUDO=1 ``` -LocalDevStack PHP/Node images run as non-root at runtime. If the mounted root CA must be copied/refreshed during entrypoint startup, the trusted development image must explicitly build with: +LocalDevStack PHP/Node images run as non-root at runtime, and their developer shells historically include passwordless sudo. This also provides the privilege path needed when a mounted root CA must be copied/refreshed during entrypoint startup. + +For stricter images that do not need runtime sudo/CA mutation, explicitly set: ```text -SCRIPTOMATIC_PASSWORDLESS_SUDO=1 +SCRIPTOMATIC_PASSWORDLESS_SUDO=0 ``` -Do not make this implicit in Scriptomatic. LocalDevStack owns this trust choice. - Use `ROOTCA_REQUIRED=1` only where inability to install the mounted CA should make startup fail. ## Shared utility contract @@ -136,6 +140,7 @@ Use `ROOTCA_REQUIRED=1` only where inability to install the mounted CA should ma ### `banner.sh` +- keeps the established INFOCYPH centered presentation, rotating credit pool, ChromaCat box-style pool, and three-row description box; - presentation-only; - safe fallback without `figlet`/`chromacat`; - non-TTY / `NO_COLOR` safe; @@ -151,11 +156,11 @@ NOTIFY_TCP_PORT=9901 DOCKNOTIFY_STRICT=0 ``` -Notification is best-effort unless strict mode is explicitly requested. The protocol is one tab-separated newline-terminated record and token data is not emitted in diagnostics. +Notification is best-effort unless strict mode is explicitly requested. Optional tuning values retain permissive compatibility behavior. The protocol is one tab-separated newline-terminated record and token data is not emitted in diagnostics. ### `owners.sh` -Standalone repository utility; not part of the critical LocalDevStack runtime contract unless adopted explicitly. +Standalone repository utility; not part of the critical LocalDevStack runtime contract unless adopted explicitly. Its established human-readable output shape is preserved while Git path enumeration is hardened. ## Service-helper contract @@ -167,12 +172,12 @@ Standalone repository utility; not part of the critical LocalDevStack runtime co - no TTY; - configurable Nginx/Apache container names; - bounded reload; -- missing optional target skips; -- stopped/reload-failed configured target fails. +- missing or stopped optional targets skip, preserving the original reload-if-running behavior; +- an attempted reload failure propagates non-zero. Do not mount the Docker socket into ordinary PHP/Node application containers just to support this helper. -`certbot-renew.sh` is a signal-aware foreground service loop with interval/jitter/failure-threshold controls. +`certbot-renew.sh` is a signal-aware foreground service loop with interval/jitter/backoff controls. Unlimited retry remains the compatibility default (`CERTBOT_RENEW_MAX_FAILURES=0`); a positive threshold can be configured when repeated failures should terminate the container. ### Mongo replica bootstrap @@ -241,7 +246,7 @@ Do not duplicate these upstream suites inside LocalDevStack. LocalDevStack shoul 2. PHP/Node image builds expose `SCRIPTOMATIC_REF` and `TOOLSET_REF` build inputs. 3. Reproducible builds can pin Scriptomatic by commit SHA and Toolset by accepted stable release. 4. Explicit `SCRIPTOMATIC_UID`/`SCRIPTOMATIC_GID` are passed into setup. -5. Trusted development sudo is enabled only where the runtime CA workflow needs it. +5. Trusted developer images preserve the established Scriptomatic sudo/Oh My Bash defaults; stricter images may explicitly disable them. 6. PHP/Node images remain non-root at runtime and preserve entrypoint `exec` semantics. 7. `bash`, `sh`, and `sh -l` remain usable for their intended roles. 8. `docknotify` can reach `SERVER_TOOLS:9901` when enabled and remains non-critical when unavailable. From aa00a6299620a4511084c962fa6f62db83a90d46 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Wed, 16 Sep 2026 18:31:13 +0600 Subject: [PATCH 011/509] docs: align shared foundations with Scriptomatic main contract --- .../01-shared-foundations-plan.md | 268 ++++++------------ 1 file changed, 80 insertions(+), 188 deletions(-) diff --git a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md index 3796790a..84398355 100644 --- a/docs/plans/docker-ecosystem/01-shared-foundations-plan.md +++ b/docs/plans/docker-ecosystem/01-shared-foundations-plan.md @@ -2,253 +2,145 @@ ## Purpose -Stabilize the shared Scriptomatic/Toolset dependency contract consumed by LocalDevStack Docker images before changing the image layer itself. +Define the Scriptomatic/Toolset dependency contract LocalDevStack should consume without redesigning the behavior of either upstream repository. -This remains a dependency-contract plan. Scriptomatic owns reusable bootstrap/runtime behavior; Toolset owns released CLI utilities; LocalDevStack owns image composition/orchestration. +## Accepted upstream contracts -## Repositories and accepted upstream contracts +### Scriptomatic -- `infocyph/Scriptomatic` - - canonical source: `main` - - reproducible consumers: `SCRIPTOMATIC_REF=` - - no Scriptomatic tag/release lifecycle required - - all sibling Scriptomatic helpers use the same selected ref -- `infocyph/Toolset` - - stable Docker-ecosystem dependency: `TOOLSET_REF=2.0` - - released executables/checksum contract; do not consume Toolset `main`/`master` +- canonical source: `main` +- optional reproducible pin: `SCRIPTOMATIC_REF=` +- no Scriptomatic tag/release lifecycle +- PHP/Node bootstrap keeps the existing public inputs and developer-container behavior from Scriptomatic `main` -## Current LocalDevStack mismatch to remove +### Toolset -The current PHP and Node Dockerfiles still contain: +- stable dependency: `TOOLSET_REF=2.0` +- use release assets/checksums rather than Toolset `main`/`master` -```text -https://raw.githubusercontent.com/infocyph/Scriptomatic/master/... -``` - -This must be removed during the LocalDevStack implementation phase. - -Do not replace it with another hard-coded mutable URL. Add explicit build arguments and propagate the same selected ref into the bootstrap script. +## Current LocalDevStack migration -Required downstream inputs: +The current PHP and Node Dockerfiles still bootstrap from: ```text -SCRIPTOMATIC_REF=main -SCRIPTOMATIC_BASE_URL=https://raw.githubusercontent.com/infocyph/Scriptomatic -TOOLSET_REF=2.0 -SCRIPTOMATIC_UID= -SCRIPTOMATIC_GID= +https://raw.githubusercontent.com/infocyph/Scriptomatic/master/... ``` -For immutable/reproducible image builds, `SCRIPTOMATIC_REF` must be the accepted Scriptomatic commit SHA rather than `main`. - -## Initial Scriptomatic bootstrap acquisition - -The Dockerfile must acquire the initial `php-cli-setup.sh` / `node-cli-setup.sh` through the selected ref with bounded download behavior. Do not use remote `ADD` against `master`. - -Recommended shape: +During the LocalDevStack implementation phase, replace that hard-coded `master` source with an explicit Scriptomatic ref: ```dockerfile ARG SCRIPTOMATIC_REF=main -ARG SCRIPTOMATIC_BASE_URL=https://raw.githubusercontent.com/infocyph/Scriptomatic ARG TOOLSET_REF=2.0 - -RUN apk add --no-cache bash curl ca-certificates && \ - curl --fail --location --silent --show-error \ - --connect-timeout 5 --max-time 90 --retry 3 \ - "${SCRIPTOMATIC_BASE_URL}/${SCRIPTOMATIC_REF}/bash/php-cli-setup.sh" \ - -o /usr/local/bin/cli-setup.sh && \ - SCRIPTOMATIC_REF="${SCRIPTOMATIC_REF}" \ - SCRIPTOMATIC_BASE_URL="${SCRIPTOMATIC_BASE_URL}" \ - TOOLSET_REF="${TOOLSET_REF}" \ - SCRIPTOMATIC_UID="${UID}" \ - SCRIPTOMATIC_GID="${GID}" \ - bash /usr/local/bin/cli-setup.sh "${USERNAME}" "${PHP_VERSION}" ``` -Use the analogous path for Node. +Fetch `php-cli-setup.sh` / `node-cli-setup.sh` from the selected Scriptomatic ref and pass `SCRIPTOMATIC_REF` plus `TOOLSET_REF` into the setup process so sibling helpers use the same Scriptomatic ref and Toolset helpers use stable `2.0`. -## Scriptomatic accepted behavior +For a reproducible build, LocalDevStack may set `SCRIPTOMATIC_REF` to the accepted Scriptomatic commit SHA. -### PHP build/runtime +## Preserve the existing Docker build inputs -`php-cli-setup.sh` now provides: - -- validated package/extension inputs; -- Alpine official-PHP-image capability checks; -- pinned/verified PHP extension installer; -- Composer available by default through pinned `COMPOSER_VERSION=2.10.3`, without the old floating self-update; -- Toolset `2.0` helper installation from released/checksummed assets; -- same-ref Scriptomatic helper installation; -- root-owned `/usr/local/bin` helpers; -- generated PHP/FPM validation; -- repeat/idempotent bootstrap coverage; -- preserved trusted-development shell defaults (passwordless sudo and Oh My Bash) with explicit opt-out controls; -- preserved setup progress/banner presentation; -- no broad shared-temp cleanup/self-delete. - -`php-entry.sh` provides content-aware mounted-root-CA refresh and transparent `exec docker-php-entrypoint "$@"` semantics. - -### Node build/runtime - -`node-cli-setup.sh` now provides: - -- validated package/global-package inputs; -- verified upstream UID reuse/rename behavior; -- optional exact npm version rather than `npm@latest`/`npm@next`; -- reproducible global-package mode; -- Toolset `2.0` and same-ref Scriptomatic helpers; -- root-owned shared executables; -- preserved trusted-development shell defaults (passwordless sudo and Oh My Bash) with explicit opt-out controls; -- preserved setup progress/banner presentation. - -`node-entry.sh` keeps the established LocalDevStack developer-container defaults: +Do not rename the existing PHP/Node identity inputs. The setup scripts continue to consume the Docker build environment: ```text -NODE_LOG_ENABLED=1 -NODE_KEEPALIVE_ON_FAIL=1 -NODE_AUTO_INSTALL=1 -NODE_ALLOW_LOCKFILE_FALLBACK=1 +UID +GID ``` -Each can be set to `0` for stricter/production-like behavior. The entrypoint selects one final command and `exec`s it. Direct argv is preferred; `NODE_CMD` remains a trusted compatibility escape hatch. - -## Trusted development sudo / root CA +along with the existing package/runtime inputs. -Scriptomatic keeps the historical trusted-development default: +PHP: ```text -SCRIPTOMATIC_PASSWORDLESS_SUDO=1 +LINUX_PKG +LINUX_PKG_VERSIONED +PHP_EXT +PHP_EXT_VERSIONED +MSMTP_FROM ``` -LocalDevStack PHP/Node images run as non-root at runtime, and their developer shells historically include passwordless sudo. This also provides the privilege path needed when a mounted root CA must be copied/refreshed during entrypoint startup. - -For stricter images that do not need runtime sudo/CA mutation, explicitly set: +Node: ```text -SCRIPTOMATIC_PASSWORDLESS_SUDO=0 +LINUX_PKG +LINUX_PKG_VERSIONED +NODE_GLOBAL +NODE_GLOBAL_VERSIONED +NODE_LOG_DIR ``` -Use `ROOTCA_REQUIRED=1` only where inability to install the mounted CA should make startup fail. +Do not introduce `SCRIPTOMATIC_UID`, `SCRIPTOMATIC_GID`, Composer-version, PHP-extension-installer-version, npm-version, reproducibility-mode, sudo-mode or Oh-My-Bash-mode inputs merely for the shared-foundations migration. -## Shared utility contract +## PHP behavior LocalDevStack should expect -### `alias-maker.sh` +Scriptomatic preserves the existing PHP development-image behavior: -- managed/idempotent `.bashrc` block; -- does not interfere with non-interactive application execution; -- optional aliases degrade cleanly. +- Alpine official PHP/FPM conventions; +- Composer installed through `install-php-extensions @composer`; +- requested PHP extensions; +- passwordless sudo for the developer user; +- Oh My Bash with the existing `lambda` theme/plugin set; +- PHP/FPM, msmtp, Composer home, banner and aliases; +- non-root runtime through `php-entry`. -### `banner.sh` +Hardening is underneath that behavior: safer argv handling, bounded/private downloads, same-ref Scriptomatic helpers, Toolset `2.0` checksum verification, root-owned shared executables, idempotent config and content-aware root-CA refresh. -- keeps the established INFOCYPH centered presentation, rotating credit pool, ChromaCat box-style pool, and three-row description box; -- presentation-only; -- safe fallback without `figlet`/`chromacat`; -- non-TTY / `NO_COLOR` safe; -- banner failure cannot prevent shell/container startup. +## Node behavior LocalDevStack should expect -### `docknotify.sh` +Scriptomatic preserves the existing Node development-image behavior: -LocalDevStack-compatible defaults remain: +- upstream UID-1000 user reuse/rename when applicable; +- passwordless sudo; +- Oh My Bash and aliases; +- user npm cache/global prefix; +- build-time `npm install -g npm@latest || npm install -g npm@next || true`; +- optional global package inputs; +- non-root runtime through `node-entry`. -```text -NOTIFY_HOST=SERVER_TOOLS -NOTIFY_TCP_PORT=9901 -DOCKNOTIFY_STRICT=0 -``` - -Notification is best-effort unless strict mode is explicitly requested. Optional tuning values retain permissive compatibility behavior. The protocol is one tab-separated newline-terminated record and token data is not emitted in diagnostics. - -### `owners.sh` +The Node entrypoint keeps its existing logging, automatic dependency installation/fallback, `NODE_CMD`, host/port and keepalive behavior. Hardening only fixes stale CA-state handling and the generic-dev double-execution bug. -Standalone repository utility; not part of the critical LocalDevStack runtime contract unless adopted explicitly. Its established human-readable output shape is preserved while Git path enumeration is hardened. +## Root CA -## Service-helper contract +The existing entrypoint input remains: -### Certbot +```text +ROOTCA_PATH +``` -`certbot-hook.sh` is Docker-control-plane behavior: +System CA installation remains best-effort at the existing fixed destination. No new strictness/destination policy inputs are required from LocalDevStack. -- exact container inspection; -- no TTY; -- configurable Nginx/Apache container names; -- bounded reload; -- missing or stopped optional targets skip, preserving the original reload-if-running behavior; -- an attempted reload failure propagates non-zero. +## Shared utilities -Do not mount the Docker socket into ordinary PHP/Node application containers just to support this helper. +- `alias-maker.sh`: keep the aliases/functions from Scriptomatic `main`. +- `banner.sh`: keep the centered INFOCYPH presentation, three-row description box, rotating credits and ChromaCat styles; fallback only when presentation capabilities are unavailable. +- `docknotify.sh`: keep `SERVER_TOOLS:9901`/best-effort behavior; protocol framing is corrected underneath. +- `owners.sh`: keep its original human output shape while using safe Git filename enumeration. -`certbot-renew.sh` is a signal-aware foreground service loop with interval/jitter/backoff controls. Unlimited retry remains the compatibility default (`CERTBOT_RENEW_MAX_FAILURES=0`); a positive threshold can be configured when repeated failures should terminate the container. +## Certbot -### Mongo replica bootstrap +`certbot-hook.sh` keeps fixed `NGINX` / `APACHE` targets and reload-if-running behavior. Exact inspection and non-TTY exec are implementation fixes. -`mongo-replica.sh`: +`certbot-renew.sh` remains the existing infinite 12-hour renewal loop. LocalDevStack should not pass interval/jitter/failure-threshold configuration that Scriptomatic does not expose. -- uses bounded readiness rather than fixed sleeps; -- prefers `mongosh` with legacy `mongo` fallback; -- separates connection URI from advertised replica members; -- defaults advertised members to Docker DNS names; -- is idempotent for a matching topology; -- initializes only when uninitialized; -- refuses conflicting existing topology. +## Mongo -Default advertised topology: +`mongo-replica.sh` retains its fixed topology: ```text +rs0 mongo-primary:27017 mongo-secondary1:27017 mongo-secondary2:27017 ``` -Do not replace service names with static `172.x` addresses. - -## Shell compatibility boundary - -LocalDevStack developer workflows may use `bash`, `sh`, and `sh -l`. - -- standalone helpers under `/usr/local/bin` have their own shebangs; -- Bash-specific profile/alias behavior belongs to Bash/login presentation paths; -- non-login `sh` is not required to source Bash-only configuration; -- PHP/Node application entrypoints do not depend on interactive shell startup. - -## Toolset accepted dependency contract - -The Docker ecosystem consumes Toolset stable `2.0` rather than mutable repository branches. - -Critical current utilities remain: - -- `gitx` -- `chromacat` -- `sqlitex` where docker-tools requires it -- `netx` where docker-tools requires it - -Do not add `dockex`, `phpx`, or `cleanx` to LocalDevStack merely for symmetry; adoption requires a concrete downstream need. - -## Permanent upstream validation - -Scriptomatic CI now covers: - -- syntax + ShellCheck; -- repository-wide security audit; -- PHP Alpine bootstrap + repeated execution; -- Node Alpine bootstrap including UID reuse/fresh-user paths; -- entrypoint exit/signal behavior; -- shared utility fixtures; -- Certbot/Mongo deterministic service-helper fixtures; -- aggregate gate. - -Toolset keeps its own permanent release/utility gates. - -Do not duplicate these upstream suites inside LocalDevStack. LocalDevStack should add consumer/integration tests that prove its Dockerfiles pass the correct refs/options and that generated images start correctly. +The hardening is readiness/idempotency/conflict handling only. No new Mongo topology environment contract is required from LocalDevStack. ## LocalDevStack implementation acceptance criteria -1. No Dockerfile consumes `Scriptomatic/master` or `Toolset/main`/`master`. -2. PHP/Node image builds expose `SCRIPTOMATIC_REF` and `TOOLSET_REF` build inputs. -3. Reproducible builds can pin Scriptomatic by commit SHA and Toolset by accepted stable release. -4. Explicit `SCRIPTOMATIC_UID`/`SCRIPTOMATIC_GID` are passed into setup. -5. Trusted developer images preserve the established Scriptomatic sudo/Oh My Bash defaults; stricter images may explicitly disable them. -6. PHP/Node images remain non-root at runtime and preserve entrypoint `exec` semantics. -7. `bash`, `sh`, and `sh -l` remain usable for their intended roles. -8. `docknotify` can reach `SERVER_TOOLS:9901` when enabled and remains non-critical when unavailable. -9. service-to-service references use Docker DNS/service names, not static IP assumptions. -10. consumer CI builds representative PHP and Node images using the pinned shared-foundation refs. +1. No PHP/Node Dockerfile downloads Scriptomatic from `master`. +2. No LocalDevStack consumer downloads Toolset helpers from a mutable branch. +3. PHP/Node Dockerfiles expose `SCRIPTOMATIC_REF` and `TOOLSET_REF` while preserving existing `UID`, `GID` and package/runtime inputs. +4. PHP and Node developer-image behavior remains unchanged from the existing LocalDevStack experience. +5. PHP/Node runtime remains non-root and their entrypoints preserve the existing command semantics. +6. `docknotify` still interoperates with the LocalDevStack notification service. +7. Service-to-service names continue to use Docker DNS rather than static IP addresses. +8. Consumer CI builds representative PHP and Node images against the accepted Scriptomatic ref and Toolset `2.0`. From b02f0716ec8313abe358924851eb32e9669f8b26 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 10:30:46 +0600 Subject: [PATCH 012/509] docs: finalize LocalDevStack integration plan --- .../07-localdevstack-integration-plan.md | 2054 ++++++++++++++--- 1 file changed, 1724 insertions(+), 330 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 721bae02..afb0414c 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1,144 +1,840 @@ -# LocalDevStack — File-by-File Integration + Product Plan +# LocalDevStack — Final Integration, Hardening & Productization Plan + +## Status + +Planning branch: `plan/docker-ecosystem-bottom-up` + +Product repository: `infocyph/LocalDevStack` + +Baseline: + +- LocalDevStack `main`: `008c3266313d89dbc4e8243e0c1515447fdceaa9` +- Product role: Docker-based XAMPP/MAMP/LAMP alternative for PHP + Node.js development +- Primary CLI: `lds` +- Windows bridge: `lds.bat` +- Default HTTP/TLS edge: Nginx +- Optional compatibility backend: Apache +- PHP and Node runtimes: locally generated/customized images +- Infrastructure images: published, independently versioned images +- Local AI: optional, provider-backed, never required for the default stack + +All lower-layer ecosystem work is complete and published. This file is now the authoritative implementation plan for the LocalDevStack phase and supersedes the earlier exploratory version of this plan. + +## Published compatibility baseline + +LocalDevStack implementation must begin against this tested ecosystem set: + +| Layer | Contract | +| --- | --- | +| Scriptomatic | hardened `main`; downstreams may use `main` or an explicit full SHA | +| Toolset | stable `2.0` release/installer contract | +| Runner | `infocyph/runner:0.5` | +| Nginx | `infocyph/nginx:0.4.1` | +| Apache | `infocyph/apache:0.4.2` | +| Tools | `infocyph/tools:0.23.2` | +| LLM standard | `infocyph/llm-sm:0.03` | +| LLM AMD | `infocyph/llm-sm:amd-0.03` | + +Do not silently replace those defaults with moving `:latest` tags during the first LocalDevStack integration release. + +Users may override image references explicitly, but a released LocalDevStack version must have a reproducible compatibility baseline. + +--- + +# 1. Product definition + +LocalDevStack is not a generic production orchestrator. + +It is a local developer workstation product intended to provide the convenience of XAMPP/MAMP/LAMP while retaining Docker isolation and modern PHP/Node workflows. + +The product should make these workflows easy: + +1. install/initialize once; +2. choose services and runtimes; +3. create a local domain; +4. get trusted local HTTPS; +5. run PHP/Composer and Node/npm tooling; +6. use databases and their admin clients; +7. use local mail; +8. run cron/Supervisor workers; +9. inspect status/logs/health; +10. optionally enable local AI without changing the core stack. + +Docker remains the implementation mechanism, not the user experience. + +--- + +# 2. Final architecture + +Target architecture: + +```text +Host + │ + ├── lds / lds.bat + │ │ + │ ├── setup / profiles / domains / certificates + │ ├── runtime generation + │ ├── service wrappers + │ └── Compose orchestration + │ + └── Docker + │ + ├── nginx 0.4.1 + │ ├── project.localhost -> PHP / Apache / Node + │ ├── admin.localhost -> server-tools:9911 + │ ├── webmail.localhost -> mailpit:8025 + │ ├── db/ri/me/kibana convenience routes + │ └── llm.localhost -> llm-sm:11434 + │ + ├── apache 0.4.2 (optional backend) + ├── PHP runtimes (local builds) + ├── Node runtimes (local builds) + ├── server-tools 0.23.2 + │ ├── host/domain/TLS/config control plane + │ ├── monitoring/admin panel + │ ├── askai / aiops / gitx AI consumer paths + │ └── http://llm-sm:11434 + │ + ├── runner 0.5 + │ └── Supervisor / cron / logrotate / sibling exec + │ + ├── mailpit + ├── databases and admin clients + │ + └── llm-sm 0.03 (optional) + ├── qwen2.5:3b baked default + ├── persistent /root/.ollama + └── Ollama API :11434 +``` + +Service-to-service AI traffic must use: + +```text +http://llm-sm:11434 +``` + +Host/user-facing AI traffic must use: + +```text +https://llm.localhost +``` + +Do not route Tools -> LLM traffic through Nginx. + +--- + +# 3. Product invariants + +These are hard constraints for the LocalDevStack phase. + +## 3.1 CLI and platform + +- `lds` remains the primary user-facing CLI. +- `lds.bat` remains the Windows/Git-Bash bridge. +- Existing public commands/aliases should remain compatible unless a command is proven obsolete. +- Refactoring the implementation must not force users to learn Docker Compose internals. +- Linux, macOS, WSL/Git Bash and Windows Docker Desktop behavior must remain intentionally supported where the current product already targets them. + +## 3.2 Runtime model + +- PHP remains locally generated from official `php:-fpm-alpine`. +- Node remains locally generated from official `node:-alpine`. +- User-selected packages/extensions/globals remain supported. +- Host UID/GID alignment remains supported. +- Project source remains bind-mounted; it is never baked into infrastructure images. + +## 3.3 HTTP model + +- Nginx remains the only default host-facing HTTP/TLS edge. +- Apache remains optional. +- Local certificates remain generated/owned by LocalDevStack/Tools, not Nginx or Apache. +- Project routing and infrastructure routing must use Docker DNS/service names. +- Static container IPs are not product contracts. -## Role +## 3.4 State model -`infocyph/LocalDevStack` is the product/orchestrator above all supporting Docker images. It should present a simple XAMPP-like local-development experience while hiding most Docker complexity. +The following must survive container recreation: -This phase executes only after lower-level image contracts are stable. +- database data; +- Redis data where persistence is enabled; +- Mailpit data; +- LocalDevStack generated/shared state; +- Composer global state where currently persisted; +- Git config state where currently persisted; +- AI model state under `/root/.ollama`. -## Product Invariants +Deleting volumes must remain an explicit destructive action. -- `lds` remains the single primary CLI. -- `lds.bat` keeps Windows/Git Bash interoperability. -- PHP and Node remain locally generated/customizable runtimes. -- Nginx remains the default HTTP/TLS edge. -- Apache remains optional. -- databases/admin clients/mail/runner remain selectively enabled through profiles/configuration. -- local TLS and multiple domains remain first-class. -- AI remains optional. -- persisted DB/cache/mail/model data survives container recreation. -- project source remains host-mounted, not copied into infrastructure images. +## 3.5 AI model + +- AI is optional. +- `docker-tools` is an AI consumer, never an Ollama runtime. +- `docker-llm-sm` is the only LocalDevStack Ollama/model runtime. +- LocalDevStack must remain fully usable when `llm-sm` is absent. +- No automatic execution of model-generated shell, SQL or code is introduced. +- No external/cloud AI fallback is added by LocalDevStack. +- No Docker socket is mounted into `llm-sm`. +- No repository/workspace is mounted into `llm-sm` by default. + +## 3.6 Single-stack compatibility + +The current product uses fixed container names and globally named volumes. + +For this release: + +- preserve those names unless a concrete bug requires change; +- do not combine the networking migration with a container/volume naming migration; +- document that the current architecture is optimized for one LocalDevStack installation per Docker engine. + +Multi-stack namespacing may be a later project. It must not risk existing user data in this integration release. + +--- + +# 4. Implementation strategy + +The LocalDevStack work should be implemented bottom-up inside this repository. + +Do not begin by splitting the 100+ KB `lds` file. + +First establish CI and compatibility characterization, then change orchestration, then modularize code. + +Each implementation batch has three phases: + +1. **Implement** — make the bounded change. +2. **Validate** — static/unit/Compose/runtime checks. +3. **Integrate** — exercise the change through `lds` and the published images. + +Recommended order: + +- Batch 1 — CI + characterization +- Batch 2 — release/image defaults +- Batch 3 — networking/DNS migration +- Batch 4 — optional AI integration +- Batch 5 — profiles/catalog/runtime defaults +- Batch 6 — PHP/Node build modernization +- Batch 7 — `lds` modularization + wrapper cleanup +- Batch 8 — config/log/socket hardening +- Batch 9 — documentation/QoL/release gate + +A batch should leave the branch usable before moving to the next one. + +--- + +# 5. Batch 1 — CI and behavior characterization + +## 5.1 New `.github/workflows/check.yml` + +This is the first implementation task. + +Required jobs: + +### Static shell validation + +Validate: + +- `lds`; +- executable files under `bin/`; +- new `lib/*.sh` modules once introduced; +- maintained shell fixtures. + +Use: -## Top-Level Files +- `bash -n`; +- ShellCheck with explicit, documented suppressions only. -### New `.github/workflows/check.yml` +Do not globally ignore broad ShellCheck classes to make CI green. -Add product-level CI before major refactoring. +### Batch/Windows bridge validation -Required stages: +At minimum: -1. `bash -n` for `lds` and Bash wrappers; -2. ShellCheck for maintained scripts with explicit suppressions; -3. validate `lds.bat` static expectations where Windows runners are practical; -4. Compose configuration validation with representative profile sets; -5. generated PHP runtime Compose fixture; -6. generated Node runtime Compose fixture; -7. canonical service-catalog schema validation; -8. smoke `lds help`, `lds config`, profile parsing, env handling without destructive host operations; -9. integration job using released supporting images for core Nginx/Tools/Runner flow; -10. optional heavier domain/TLS/runtime smoke job. +- parse/static-check `lds.bat` expectations; +- validate paths containing spaces; +- validate Git-for-Windows Bash discovery assumptions; +- validate caller working-directory preservation. -Use Linux CI as baseline; add Windows CI for bridge/path-specific behavior rather than trying to run every container integration twice. +A Windows runner should be added for bridge-specific smoke tests when practical. -### `.gitignore` +Do not duplicate the entire Linux Docker integration suite on Windows. -Current file is allowlist-oriented. +### Compose validation -Plan: +Run `docker compose config` for representative matrices: -- keep the allowlist model if intentional; -- whitelist new static product resources such as `docker/catalog/**`, tests and CI fixtures; -- keep generated runtime `.env`, certificates, secrets, logs and user-generated Compose artifacts ignored; -- make the tracked-vs-generated boundary explicit. +- core only; +- core + Apache; +- each DB profile; +- Elasticsearch + Kibana/Filebeat; +- AI CPU; +- AI NVIDIA override config syntax; +- AI AMD override config syntax; +- generated PHP runtime; +- generated Node runtime. -### `.gitattributes` +Compose validation must catch missing variables, duplicate keys, invalid profiles and invalid override merges. -Preserve LF shell scripts and Windows batch compatibility. Add path-specific rules only if needed. +### CLI characterization -### `README.md` +Add non-destructive tests for: -Rewrite after implementation to present the product as a Docker-based XAMPP alternative for PHP + Node. +- `lds help`; +- `lds setup` routing; +- profile parsing; +- dotenv reads/writes; +- compose-file resolution; +- runtime selection; +- aliases; +- status/doctor/config commands; +- domain argument validation; +- unknown-command handling; +- commands that should work without Docker where applicable. -Quickstart should prioritize: +### Published-image core integration -- install Docker; -- clone/setup `lds`; -- select services/runtime; -- create domain; -- trust local CA; -- work through `lds php`, `lds composer`, Node/npm commands and service shortcuts. +Use the exact compatibility baseline: -Keep advanced internals in docs, not the first screen. +```text +infocyph/tools:0.23.2 +infocyph/runner:0.5 +infocyph/nginx:0.4.1 +infocyph/apache:0.4.2 +``` -### `LICENSE` +Validate: -No change. +- Tools reaches healthy state using its own healthcheck; +- Runner reaches healthy state; +- Nginx validates and starts; +- Apache validates and starts when enabled; +- `admin.localhost` routes to Tools; +- core containers resolve each other through Docker DNS; +- clean SIGTERM/Compose down behavior. -## `lds` Modularization +### AI integration smoke without model download -### Existing `lds` +Do not download a 3B model on every LocalDevStack PR. -Do not rewrite behavior in one step. +Use a lightweight fake Ollama-compatible service named `llm-sm` for the normal PR test. -Migration sequence: +Validate: -1. add characterization tests around current commands; -2. identify stable command/public-output contracts; -3. extract internal modules while leaving `lds` as bootstrap/dispatcher; -4. only then simplify duplicated code. +- Tools `askai --status` reaches `http://llm-sm:11434`; +- Tools `aiops provider` works; +- Nginx `llm.localhost` reaches the fake service; +- streamed response is not buffered incorrectly; +- AI absence leaves core services healthy. -Target layout: +A manual/release-gate job may optionally exercise the real published `infocyph/llm-sm:0.03`, because that image already has its own model-bearing runtime gate. -- `lds` — bootstrap, global flag parsing, command dispatch; -- `lib/core.sh` — errors, output, command/tool lookup, OS detection; -- `lib/compose.sh` — Compose wrapper, service resolution, profiles/extras; -- `lib/env.sh` — dotenv read/write/quoting and product defaults; -- `lib/catalog.sh` — canonical service/runtime catalog reader; -- `lib/profiles.sh` — service/profile selection and persistence; -- `lib/hosts.sh` — mkhost/rmhost integration and reload lifecycle; -- `lib/certificates.sh` — host trust-store install/uninstall; -- `lib/runtime.sh` — PHP/Node/runtime-image generation/rebuild integration; -- `lib/diagnostics.sh` — doctor/diag/sniff/status helpers; -- `lib/maintenance.sh` — clean/disk/events/rebuild operations; -- `lib/platform.sh` — Windows/macOS/Linux path/platform helpers when separation is useful. +## 5.2 New `tests/` + +Suggested layout: + +```text +tests/ + lib/ + assertions.sh + fixtures.sh + static.sh + cli-contract.sh + env-contract.sh + compose-contract.sh + networking-contract.sh + runtime-php-contract.sh + runtime-node-contract.sh + ai-contract.sh + release-gate.sh + fixtures/ + env/ + compose/ + projects/ + fake-ollama/ +``` + +Keep fixtures disposable. + +Never commit real certificates, private keys, SOPS keys or user secrets. + +--- + +# 6. Batch 2 — Compatibility image defaults + +## 6.1 New tracked `docker/release.env` + +Create a committed compatibility manifest. + +Initial values: + +```text +LDS_TOOLS_IMAGE=infocyph/tools:0.23.2 +LDS_RUNNER_IMAGE=infocyph/runner:0.5 +LDS_NGINX_IMAGE=infocyph/nginx:0.4.1 +LDS_APACHE_IMAGE=infocyph/apache:0.4.2 +LDS_LLM_IMAGE=infocyph/llm-sm:0.03 +LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-0.03 +``` Rules: -- sourced modules are not user-facing commands; -- no module should execute work merely when sourced; -- keep `set -euo pipefail` behavior intentional; -- avoid global mutable state where function-local state works; -- preserve existing command aliases until a documented deprecation. +- this file is owned by the LocalDevStack release; +- user overrides belong in `docker/.env`; +- user overrides always win; +- `lds` should load release defaults before user overrides; +- do not copy release defaults into a user file on every update; +- upgrades should not overwrite user values. -### `lds.bat` +The exact implementation may use repeated Compose `--env-file` flags or controlled export/merge logic in `lds`, but precedence must be deterministic and covered by tests. -Plan: +## 6.2 Compose image references -- preserve Git-for-Windows Bash bridge; -- preserve caller working directory and Windows->Unix path conversion; -- validate Docker installed/running errors; -- add CI/static smoke for quoting paths containing spaces; -- avoid duplicating `lds` command semantics in batch. +Replace hard-coded: -## Canonical Service Catalog +```text +infocyph/tools:latest +infocyph/runner:latest +infocyph/nginx:latest +infocyph/apache:latest +``` -### New `docker/catalog/services.json` +with: -Create one canonical product catalog consumed by `lds` and mounted/read by `docker-tools`. +```yaml +image: ${LDS_TOOLS_IMAGE:-infocyph/tools:0.23.2} +image: ${LDS_RUNNER_IMAGE:-infocyph/runner:0.5} +image: ${LDS_NGINX_IMAGE:-infocyph/nginx:0.4.1} +image: ${LDS_APACHE_IMAGE:-infocyph/apache:0.4.2} +``` -Initial schema should describe at least: +Do the equivalent for `llm-sm`. -- service/profile key; -- display name; -- image/version env variable; -- default version/value; -- setup prompt fields/defaults; -- related admin client profile/service where applicable; -- convenience route metadata where useful; -- persistence volume identifier; -- optional health/dependency metadata only when needed by orchestration. +## 6.3 Update command / future dependency bumps + +Do not silently mutate release defaults at runtime. + +A future QoL command may report newer releases, for example: + +```text +lds update check +``` + +but it should not rewrite compatibility pins without explicit user action. + +A future CI workflow may test proposed dependency bumps and open a PR, but that is not required to complete this phase. + +--- + +# 7. Batch 3 — Remove static-IP architecture + +This is the largest orchestration cleanup before AI/profile work. + +## 7.1 `docker/compose/main.yaml` + +Keep logical networks: + +- `frontend`; +- `backend`; +- `datastore`. + +Remove: + +- hard-coded `172.28.0.0/24`; +- hard-coded `172.29.0.0/24`; +- hard-coded `172.30.0.0/24`; +- explicit gateway declarations. + +Target: + +```yaml +networks: + frontend: + driver: bridge + backend: + driver: bridge + datastore: + driver: bridge +``` + +Preserve current labels where they are useful for LocalDevStack discovery. + +Do not rename networks in the same migration unless necessary. + +## 7.2 `docker/compose/companion.yaml` + +Remove all `ipv4_address` entries. + +Preserve service DNS names: + +- `server-tools`; +- `runner`; +- `mailpit`. + +Retain the network membership actually required by each service. + +Do not attach services to every network merely for convenience. + +## 7.3 `docker/compose/http.yaml` + +Remove fixed Nginx/Apache addresses. + +Nginx must route by service name only. + +Preserve: + +- host `80/443` bindings; +- vhost volume; +- cert/root-CA mounts; +- FPM socket volume; +- logs; +- `host.docker.internal:host-gateway` only if a real supported path still requires it. + +Normalize Apache restart behavior to `unless-stopped` unless a tested reason requires `always`. + +## 7.4 `docker/compose/db.yaml` + +Remove every datastore `ipv4_address`. + +Keep hostnames/service names as the application contract: + +- `redis`; +- `postgres`; +- `mysql`; +- `mongodb`; +- `mariadb`; +- `elasticsearch`. + +## 7.5 `docker/compose/db-client.yaml` + +Remove fixed addresses on both `frontend` and `datastore`. + +Admin clients must use service DNS. + +## 7.6 `lds vpn-fix` + +Characterize the current behavior before removal. + +If the command only exists to work around collisions created by LocalDevStack's fixed subnets: + +- deprecate it in this release; +- keep a compatibility message for one release if useful; +- remove related routing mutation code after tests prove dynamic Docker networks solve the original problem. + +If some independent VPN behavior is still useful, rename/re-scope the command to that actual function rather than preserving obsolete subnet assumptions. + +## 7.7 Networking acceptance tests + +Validate all of these without fixed IPs: + +- Nginx -> Tools; +- Nginx -> Mailpit; +- Nginx -> DB UIs; +- Nginx -> Apache; +- Nginx -> Node app; +- Nginx -> llm-sm; +- Tools -> DB/service diagnostics; +- Tools -> llm-sm; +- Runner -> PHP/Node sibling execution; +- DB clients -> databases; +- Filebeat -> Elasticsearch; +- generated project vhosts. + +--- + +# 8. Batch 4 — Optional local AI integration + +AI should become a first-class optional LocalDevStack capability while remaining absent from the default stack. + +## 8.1 New `docker/compose/ai.yaml` + +Base service: + +```yaml +services: + llm-sm: + image: ${LDS_LLM_IMAGE:-infocyph/llm-sm:0.03} + restart: unless-stopped + profiles: [ai] + volumes: + - lds_llm:/root/.ollama + networks: + - frontend + - backend +``` + +Important rules: + +- service key must be exactly `llm-sm`; +- do not set a fixed IP; +- do not require `container_name`; +- do not mount Docker socket; +- do not expose `11434` to all interfaces; +- do not add a host port in the base AI file; +- do not add an automatic repository/workspace mount; +- do not add cloud fallback. + +Why both networks: + +- Nginx must reach `llm-sm:11434` for `https://llm.localhost`; +- Tools must reach `llm-sm:11434` directly for AI consumer commands. + +## 8.2 `docker/compose/main.yaml` volume + +Add: + +```yaml +lds_llm: + name: LLMModels +``` + +or a similarly consistent LocalDevStack volume name. + +Mount it to: + +```text +/root/.ollama +``` + +The volume is authoritative runtime model state. + +A fresh volume receives the image-baked `qwen2.5:3b`. + +Existing populated volumes must never be silently replaced/reset during upgrades. + +## 8.3 CPU / NVIDIA / AMD runtime selection + +Do not implement unreliable GPU auto-detection as an orchestration requirement. + +Provide explicit runtime modes: + +```text +cpu +nvidia +amd +``` + +Recommended static overrides: + +```text +docker/compose/ai-nvidia.yaml +docker/compose/ai-amd.yaml +``` + +NVIDIA: -Initial catalog covers: +- keep `LDS_LLM_IMAGE`; +- add GPU access using the Compose mechanism supported by current Docker Desktop/Engine. + +AMD: + +- use `LDS_LLM_AMD_IMAGE`; +- expose `/dev/kfd`; +- expose `/dev/dri`. + +Store the selected mode in LocalDevStack env/state. + +Do not change image variants automatically because a GPU happens to be detected. + +## 8.4 Optional direct host API + +Default LocalDevStack access is: + +```text +https://llm.localhost +``` + +Do not expose `11434` by default. + +If developers explicitly need direct Ollama access, add an optional override such as: + +```text +docker/compose/ai-host-port.yaml +``` + +binding only: + +```text +127.0.0.1:${LLM_SM_PORT:-11434}:11434 +``` + +Never default to `0.0.0.0:11434`. + +## 8.5 Nginx integration + +No new Nginx image changes are required. + +`nginx:0.4.1` already reserves: + +```text +llm.localhost -> llm-sm:11434 +``` + +and uses lazy Docker DNS resolution plus streaming proxy behavior. + +LocalDevStack must validate: + +- certificate coverage for `llm.localhost`; +- HTTP -> HTTPS redirect; +- `/api/tags`; +- `/api/generate`; +- `/api/chat`; +- OpenAI-compatible `/v1/...`; +- stream passthrough; +- expected 502/unavailable behavior when AI profile is disabled, without affecting Nginx startup. + +## 8.6 Tools integration + +Pass the published Tools AI contract into `server-tools`: + +```text +LDS_AI_ENABLED=auto +LDS_AI_PROVIDER=ollama +LDS_AI_URL=http://llm-sm:11434 +LDS_AI_MODEL=qwen2.5:3b +``` + +Allow user overrides. + +Also pass through supported advanced limits only when the user sets them: + +- `LDS_AI_CONNECT_TIMEOUT`; +- `LDS_AI_PREFLIGHT_TIMEOUT`; +- `LDS_AI_TIMEOUT`; +- `LDS_AI_AVAILABILITY_TTL`; +- `LDS_AI_MAX_CONTEXT_BYTES`; +- `LDS_AI_MAX_REQUEST_BYTES`; +- `LDS_AI_MAX_RESPONSE_BYTES`. + +Why LocalDevStack should default `LDS_AI_MODEL=qwen2.5:3b`: + +- `llm-sm:0.03` ships that model; +- Tools intentionally reports ambiguity when multiple models are installed and no model is selected; +- users may pull more models without breaking Tools AI workflows. + +Users can change `LDS_AI_MODEL` explicitly. + +Do not make Tools pull/remove models. + +## 8.7 AI CLI UX + +Add a thin LocalDevStack surface without duplicating provider implementations. + +Recommended commands: + +```text +lds ai status +lds ai ask ... +lds ai explain ... +lds ai troubleshoot ... +lds ai review ... +lds ai repo-review ... +lds ai graphify ... +``` + +Map these to the published Tools commands: + +- `askai`; +- `aiops`. + +Recommended provider/model-management pass-through: + +```text +lds llm models +lds llm ps +lds llm show [model] +lds llm pull +lds llm rm +lds llm unload [model] +lds llm run ... +lds llm chat [model] +lds llm prompt ... +lds llm code ... +lds llm review ... +lds llm json ... +lds llm ai-commit ... +``` + +These should delegate to the bundled `llm-sm` CLI inside the provider container. + +LocalDevStack must not reimplement Ollama/model logic. + +## 8.8 Workspace/repository access + +Default: no project mount into `llm-sm`. + +Tools already has the LocalDevStack project mounted at `/app` and its AI layer applies size/sensitivity/redaction guards. + +For direct `llm-sm` repository-aware commands: + +- prefer stdin/file transfer where practical; +- optionally provide a separate read-only workspace override; +- default the workspace mount to read-only; +- require explicit opt-in for writable workspace access. + +Never automatically mount arbitrary host repositories. + +## 8.9 Graphify + +Do not install Graphify into `llm-sm` or Tools solely for this integration. + +Supported patterns: + +- host Graphify -> `https://llm.localhost/v1`; +- container Graphify on a shared network -> `http://llm-sm:11434/v1`; +- Tools `aiops graphify --file ` -> analyze an explicitly supplied Graphify output file. + +Graphify remains an external consumer/tool. + +## 8.10 AI admin panel + +Do not create a second AI web UI in LocalDevStack. + +Use the AI capability already shipped in `tools:0.23.2` and exposed through: + +```text +https://admin.localhost +``` + +LocalDevStack's responsibility is to provide correct provider env/networking. + +--- + +# 9. Batch 5 — Profiles, defaults and service metadata + +## 9.1 Current duplication + +Today profile/service defaults exist in more than one place: + +- LocalDevStack `lds`; +- Tools `profile-chooser`. + +Because `tools:0.23.2` is already a published lower-layer contract, LocalDevStack must not require another Tools release to complete this phase. + +## 9.2 LocalDevStack canonical host-side catalog + +Add a host-side product catalog such as: + +```text +docker/catalog/services.json +``` + +Use it for new/rewritten LocalDevStack profile logic. + +Schema should support: + +- profile key; +- display name; +- service key; +- image/version env key; +- default values; +- setup prompts; +- optional admin client; +- persistence volume; +- convenience URL; +- category; +- whether the service is optional/default; +- AI runtime metadata where appropriate. + +Initial entries: - PostgreSQL; - MySQL; @@ -146,352 +842,1050 @@ Initial catalog covers: - MongoDB; - Redis; - Elasticsearch; -- optional AI service metadata in a separate capability section or same versioned schema. +- AI. + +Do not place user secrets in the catalog. + +## 9.3 Compatibility with Tools profile chooser + +Do not block LocalDevStack release on making Tools consume the catalog. + +For this release: + +- LocalDevStack host setup uses its own catalog; +- Tools `profile-chooser` remains a compatible standalone Tools capability; +- add a contract test that detects material drift between LocalDevStack defaults and Tools defaults where they overlap. + +A future Tools release may add external-catalog support. That is a later cleanup, not a prerequisite. + +## 9.4 AI profile setup + +Extend `lds setup profile` to include optional AI. + +Prompt only relevant AI settings: + +- enable AI yes/no; +- runtime `cpu|nvidia|amd`; +- preferred model default `qwen2.5:3b`; +- optional direct localhost port yes/no. + +Do not ask users for low-level timeout/byte-limit settings during normal setup. + +Advanced variables remain manual env overrides. -Do not put secrets directly in catalog. Default dev credentials may be expressed as defaults but user values live in env state. +--- -### Remove duplicated profile defaults +# 10. Batch 6 — PHP and Node runtime build modernization -After `docker-tools` supports external catalog: +## 10.1 `docker/dockerfiles/php.Dockerfile` -- replace hard-coded `SERVICES`/`PROFILE_ENV` duplication in `lds` with catalog reads; -- mount catalog into `server-tools`; -- remove Tools fallback duplication only after compatibility period/tests. +Preserve: -## `bin/` Wrappers +```dockerfile +ARG PHP_VERSION=8.4 +FROM php:${PHP_VERSION}-fpm-alpine +``` -### `bin/tool-runner` +Replace stale Scriptomatic `master` download. -- keep common container execution/path/TTY logic centralized; -- make it the reusable primitive for thin service wrappers where possible; -- ensure Windows path conversion and UID/GID behavior are tested; -- propagate exit codes and signals. +Introduce: -### `bin/php` +```dockerfile +ARG SCRIPTOMATIC_REF=main +``` -- preserve runtime selection/highest-version fallback; -- keep explicit `--php/-V` selection; -- preserve ad-hoc execution using the selected runtime image; -- validate bind mount and user mapping on Linux/macOS/Windows; -- keep `serve` mode if still useful, but separate it cleanly from normal CLI execution; -- align Git safe-directory behavior for mounted projects. +Fetch: -### `bin/composer` +```text +https://raw.githubusercontent.com/infocyph/Scriptomatic/${SCRIPTOMATIC_REF}/bash/php-cli-setup.sh +``` -- reduce duplicated runtime-resolution code by consuming `tool-runner`/shared helper where possible; -- preserve PHP-version selection and versioned Composer home; -- test install/update/global operations against mounted project. +Pass the same `SCRIPTOMATIC_REF` into the bootstrap so sibling Scriptomatic helpers come from the same revision. -### `bin/my` +Requirements: -- preserve MySQL login/query/export/import helper behavior; -- resolve service by Compose name/labels, not static IP; -- keep credentials from environment; -- validate quoting and database/file paths. +- bounded curl retries/timeouts; +- verify non-empty script; +- `bash -n` before execution; +- retain UID/GID; +- retain package/extension args; +- retain PHP profile key; +- retain non-root developer user; +- retain FPM entrypoint; +- retain Composer-home separation. -### `bin/maria` +Do not add a direct Toolset download to this Dockerfile. Scriptomatic owns the current Toolset stable installer contract. -Same principles as `bin/my`, using MariaDB client/service contract. +Default may remain `SCRIPTOMATIC_REF=main`. -### `bin/pg` +A LocalDevStack release or user may override it with a full SHA when exact reproducibility is needed. -- preserve PostgreSQL login/query/dump/restore helpers; -- DNS service name only; -- environment-driven credentials; -- test dump/restore with temporary DB. +## 10.2 `docker/dockerfiles/node.Dockerfile` -### `bin/mongo` +Apply the equivalent Scriptomatic contract: -- preserve mongosh/login/import/export helpers; -- DNS service name only; -- environment-driven auth; -- no replica-set assumption in default flow. +```dockerfile +ARG SCRIPTOMATIC_REF=main +``` -### `bin/redis-cli` +Preserve: -- preserve thin Redis CLI behavior; -- DNS service name and optional auth env; -- keep wrapper small. +- selectable Node version; +- UID/GID; +- Linux packages; +- global Node packages; +- non-root user; +- npm/Corepack behavior; +- Node entrypoint. -### `bin/es` +## 10.3 Runtime image identity -- preserve Elasticsearch API/helper behavior; -- use service name rather than fixed IP; -- validate ES 9.x/current compatibility via version-aware requests; -- no hidden Kibana dependency for core ES commands. +Keep: -## Docker Compose Files +```text +localdevstack-php: +localdevstack-node: +``` -### `docker/compose/main.yaml` +Do not publish the combinatorial PHP/Node runtime matrix. -Major network migration: +## 10.4 Build caching -- keep logical `frontend`, `backend`, `datastore` bridge networks; -- remove hard-coded subnets/gateways after integration validation; -- remove need for fixed container IPv4 values from included files; -- keep named persistent volumes; -- add AI model volume only when AI profile integration lands; -- consider namespacing globally fixed volume/network names if multi-stack coexistence needs it; do not break existing data without migration guidance. +Avoid rebuilding unchanged local runtime images unnecessarily. -### `docker/compose/companion.yaml` +The runtime-image identity/hash should account for inputs that affect the image: -Services: `server-tools`, `runner`, `mailpit`. +- runtime version; +- Scriptomatic ref; +- UID/GID; +- selected extensions/packages/globals; +- relevant Dockerfile revision. -Plan: +A later optimization may use labels or a deterministic configuration hash. -- replace static IPv4 declarations with networks/service DNS; -- move infrastructure image references from hard-coded `:latest` to environment-controlled compatibility-tested versions; -- mount canonical service catalog into `server-tools` read-only; -- inventory Docker socket usage for both Tools and Runner; -- preserve project/config/scheduler/SOPS/SSH/SSL/log mounts; -- preserve Mailpit persistent data and local TLS; -- add health/dependency conditions only where they improve deterministic startup without deadlocks. +Do not let caching return a runtime built with different extension/package inputs. -### `docker/compose/http.yaml` +--- -Services: Nginx + optional Apache. +# 11. Batch 7 — Modularize `lds` safely -Plan: +The current `lds` file is over 100 KB. -- remove static IPs; -- version image refs through LocalDevStack defaults/env; -- keep Nginx host 80/443 binding configurable; -- preserve project/vhost/cert/rootCA/FPM socket/log mounts; -- preserve Apache as internal backend; -- use service names for dependency/routing; -- reconsider `restart: always` vs consistent `unless-stopped` behavior for local dev. +Refactor only after Batches 1–6 have tests. -### `docker/compose/db.yaml` +## 11.1 Target layout -Services: Redis, PostgreSQL, MySQL, MongoDB, MariaDB, Elasticsearch. +```text +lds +lib/ + core.sh + platform.sh + env.sh + compose.sh + profiles.sh + catalog.sh + runtime.sh + hosts.sh + certificates.sh + services.sh + ai.sh + diagnostics.sh + maintenance.sh +``` -Plan: +Suggested ownership: -- remove all static IPv4 assignments; -- source default version/env metadata from canonical catalog; -- retain named volumes; -- review data-directory mount correctness per current upstream images (especially PostgreSQL version changes); -- retain healthchecks but correct variable-name mismatches (`POSTGRES_DATABASE` vs `POSTGRES_DB`, etc.); -- avoid `latest` defaults for compatibility-sensitive DB majors where a stable major is preferable; -- keep local-development credentials configurable and clearly non-production. +### `lds` -### `docker/compose/db-client.yaml` +Only: -Services: RedisInsight, CloudBeaver, Mongo Express, Kibana, Filebeat. +- bootstrap; +- global option parsing; +- command dispatch; +- help/version entrypoints. -Plan: +### `lib/core.sh` -- remove static IP assignments; -- use datastore service DNS; -- keep profile coupling explicit; -- version client images where breaking major drift is possible; -- validate persistent workspace/data volumes; -- ensure clients tolerate target DB starting later (health/retry rather than immediate failure where possible). +- output/error helpers; +- command requirements; +- common argument helpers; +- temporary-file helpers. -## PHP / Node Runtime Dockerfiles +### `lib/platform.sh` -### `docker/dockerfiles/php.Dockerfile` +- OS/WSL detection; +- Docker Desktop detection where needed; +- path conversion helpers; +- browser/open helpers. -- keep `ARG PHP_VERSION` + upstream `php:-fpm-alpine` model; -- add `SCRIPTOMATIC_REF` and `TOOLSET_REF`/related immutable dependency inputs; -- stop fetching setup script from floating Scriptomatic master; -- preserve UID/GID, extension/package and profile-key build args; -- validate image as non-root developer user and PHP-FPM service; -- keep build local because combinations are user-selected. +### `lib/env.sh` -### `docker/dockerfiles/node.Dockerfile` +- release env; +- user env; +- dotenv parse/write; +- precedence; +- safe quoting. -- keep selectable upstream Node Alpine version; -- pin Scriptomatic/Toolset helper revisions; -- preserve UID/GID/packages/globals customization; -- preserve non-root user; -- validate npm/corepack and project startup behavior; -- keep build local. +Never `source` arbitrary dotenv files as shell code. -## Docker Config Files +### `lib/compose.sh` -### `docker/conf/filebeat.yml` +- canonical Compose file list; +- profile resolution; +- optional overrides; +- Compose command wrapper; +- service status/health helpers. -- validate against current Filebeat/Elasticsearch major; -- paths must match mounted LocalDevStack log layout; -- no fixed IP endpoints. +### `lib/catalog.sh` -### `docker/conf/openssl.cnf` +- validate/read `services.json`; +- list available services; +- expose setup fields/defaults. -- validate local dev compatibility with generated certificates/current OpenSSL; -- avoid weakening global TLS unnecessarily; -- document why overrides exist. +### `lib/profiles.sh` -### `docker/conf/pg_hba.conf` +- setup menu; +- selected profile persistence; +- AI runtime selection; +- profile/env reconciliation. -- review trust/auth scope for isolated local Docker network; -- preserve password auth expectations; -- no fixed subnet assumptions after network migration. +### `lib/runtime.sh` -### `docker/conf/postgresql.conf` +- PHP/Node version resolution; +- generated Compose fragments; +- local image naming/build/rebuild. -- decide whether it remains intentionally unused/commented or should become active; -- if unused, avoid presenting it as active configuration in docs; -- if enabled, version-test it. +### `lib/hosts.sh` -### `docker/conf/www-php.conf` +- domain create/delete; +- vhost generation calls; +- reload validation. -- validate FPM pool include/listen behavior against generated per-domain pools; -- ensure user/group/socket permissions align with Nginx/Apache access. +### `lib/certificates.sh` -### `docker/conf/www.conf` +- CA creation/trust; +- certificate refresh; +- platform-specific trust store. -- classify as reference/upstream-derived vs active file; -- remove/deprecate if no runtime path consumes it, but only after search/tests. +### `lib/services.sh` -## Generated/User Configuration Directories +- start/stop/restart/status/logs/open; +- convenience service URL mapping. -### `configuration/compose/` +### `lib/ai.sh` -Continue as generated Compose overrides. Add validation that generated files are valid and stale overrides can be detected/cleaned safely. +Only LocalDevStack orchestration/delegation: -### `configuration/php/php.ini` +- profile enabled?; +- compose override selection; +- `lds ai` -> Tools; +- `lds llm` -> llm-sm; +- model/runtime status. -Remain user override file. Provide documented defaults/examples rather than overwriting user edits during update. +No provider implementation. -### `configuration/scheduler/cron-jobs/` +### `lib/diagnostics.sh` -Remain user/generated scheduler definitions. Validate filename/permissions/content before Runner consumes them. +- doctor; +- config; +- health; +- environment diagnostics. -### `configuration/scheduler/supervisor/` +### `lib/maintenance.sh` -Remain user/generated supervisor definitions. Validate configs before stack restart. +- clean; +- disk; +- events; +- rebuild; +- safe destructive confirmations. -### `configuration/sops/config`, `global`, `keys` +## 11.2 Refactor rules -Keep sensitive/state files ignored. Tighten permissions via setup without committing contents. +- no module performs work just because it is sourced; +- avoid hidden mutation of global state; +- keep function-local variables local; +- do not replace clear shell with framework-like abstractions; +- preserve exit codes where wrappers rely on them; +- keep `set -euo pipefail` behavior deliberate; +- characterization tests must pass after every extraction; +- one subsystem at a time. -### `configuration/ssh/` +--- -Keep optional read-only mount. Never copy private keys into images. +# 12. Batch 7 — `bin/` wrapper cleanup -### `configuration/ssl/` +## 12.1 `bin/tool-runner` -Keep generated cert artifacts/state according to the chosen named-volume/bind-mount model. Reconcile docs with actual source of truth. +Treat as the common execution primitive. -## Logs +Validate: -### `logs/` +- TTY forwarding; +- stdin forwarding; +- exit-code propagation; +- path conversion; +- service-running errors; +- working-directory selection. -- keep host-visible logs where that is a deliberate developer feature; -- avoid `chmod -R 777` if cross-platform/container UID tests show a safer workable model; -- if permissive mode remains necessary, document local-only rationale; -- ensure rotation behavior matches Runner configuration. +## 12.2 `bin/php` -## Optional AI Compose Integration +Preserve: -### New `docker/compose/ai.yaml` +- explicit PHP version selection; +- default/highest configured runtime; +- normal CLI; +- current serve behavior where useful; +- mounted project; +- non-root execution. -Add only after core networking/catalog changes are stable. +Add tests for spaces in paths and Git safe-directory behavior. -Service design: +## 12.3 `bin/composer` -- profile `ai` (or `llm`, decide one canonical public name); -- default published image `infocyph/llm-sm:` through env variable; -- AMD selectable through `amd-` tag, not second repository; -- named volume -> `/root/.ollama`; -- optional project/workspace mount -> `/workspace`; -- loopback host port optional/configurable; -- join network for service-DNS access from other containers; -- GPU options handled through explicit override/profile rather than auto-detect magic that makes Compose unreliable across hosts. +Reduce duplicated PHP runtime selection where practical. -Graphify/editor/other clients remain external consumers of the endpoint. +Preserve versioned Composer homes and project mount behavior. -## Image Version Defaults +## 12.4 DB wrappers -### New `docker/images.env` or equivalent committed defaults +Files: -Define compatibility-tested infrastructure image versions centrally, for example conceptual keys: +- `bin/pg`; +- `bin/my`; +- `bin/maria`; +- `bin/mongo`; +- `bin/redis-cli`; +- `bin/es`. -- `LDS_TOOLS_IMAGE=infocyph/tools:` -- `LDS_RUNNER_IMAGE=infocyph/runner:` -- `LDS_NGINX_IMAGE=infocyph/nginx:` -- `LDS_APACHE_IMAGE=infocyph/apache:` -- `LDS_LLM_IMAGE=infocyph/llm-sm:` when AI enabled. +Requirements: -User `.env` may override them. A LocalDevStack release should not depend solely on whatever `latest` means that day. +- use Compose/service names, not IPs; +- no assumptions about static subnets; +- credentials from environment/state; +- safe argument/file quoting; +- accurate exit codes; +- temporary dump/restore integration tests. -## Documentation Files +## 12.5 New AI wrappers -### `docs/concepts/architecture.rst` +Prefer adding: -Update architecture diagram/responsibility boundaries, canonical catalog, optional AI and DNS-based networking. +```text +bin/ai +bin/llm +``` -### `docs/concepts/profiles-and-env.rst` +or equivalent internal helpers invoked by `lds`. -Document canonical catalog, env override precedence, infrastructure image versions and generated runtime profiles. +Keep them thin. -### `docs/concepts/storage-layout.rst` +They must not duplicate `askai`, `aiops` or `llm-sm` logic. -Reconcile named volumes vs host `configuration/` directories. Clearly distinguish: +--- -- persisted Docker named volumes; -- host-generated config; -- project source mounts; +# 13. Batch 8 — Compose/service hardening + +## 13.1 `docker/compose/companion.yaml` + +### `server-tools` + +Use: + +```text +${LDS_TOOLS_IMAGE:-infocyph/tools:0.23.2} +``` + +Preserve: + +- project mount; +- SSL/root CA; +- generated Nginx/Apache/FPM/Composer state; +- Git config; +- scheduler state; +- SOPS state; +- optional SSH mount; - logs; -- secrets/SSH; -- optional AI model volume. +- Docker socket. + +Add AI environment from Section 8. + +Use the Tools-owned healthcheck instead of inventing an external health command. + +### `runner` + +Use: + +```text +${LDS_RUNNER_IMAGE:-infocyph/runner:0.5} +``` + +Preserve scheduler/log mounts. + +Runner still has legitimate Docker access because its `pexe`/`dexe` and mounted jobs can execute commands in sibling containers. + +Do not remove the socket until those use cases are intentionally redesigned. + +### `mailpit` + +Keep persistence and TLS. + +Validate that its certificate paths are ready before requiring STARTTLS. + +## 13.2 `docker/compose/http.yaml` + +### Nginx + +Use: + +```text +${LDS_NGINX_IMAGE:-infocyph/nginx:0.4.1} +``` + +Preserve `80/443`, generated vhosts, certs, FPM sockets and logs. + +### Apache + +Use: + +```text +${LDS_APACHE_IMAGE:-infocyph/apache:0.4.2} +``` + +Keep optional. + +Use `unless-stopped` unless testing proves `always` is required. + +## 13.3 `docker/compose/db.yaml` + +### Redis + +Keep persistent `/data`. + +Review whether `redis/redis-stack-server:latest` remains desirable as a default or should use a compatibility-tested major/tag. + +Do not change data format in the same release without migration guidance. + +### PostgreSQL + +Reconcile variable naming. + +Current service sets: + +```text +POSTGRES_DB=${POSTGRES_DATABASE:-postgres} +``` + +but the healthcheck references `POSTGRES_DB` through Compose interpolation rather than the resulting container env. + +Use one canonical LocalDevStack variable and test it. + +Validate the selected official Postgres image's current data directory contract before changing the volume mount. + +### MySQL/MariaDB/MongoDB + +Keep explicit local-dev credential variables. + +Review moving `latest` defaults separately from the infrastructure-image migration. + +Do not unexpectedly major-upgrade a user's database by changing defaults without documentation. + +### Elasticsearch + +Keep Elasticsearch/Kibana versions aligned. + +Validate Filebeat compatibility with the same stack version. + +## 13.4 `docker/compose/db-client.yaml` + +Preserve: + +- RedisInsight; +- CloudBeaver; +- Mongo Express; +- Kibana; +- Filebeat. + +Improve startup dependencies only when meaningful: + +- prefer service health/retry behavior; +- avoid dependency chains that deadlock optional profiles. + +--- + +# 14. Docker socket trust boundary + +Both Tools and Runner currently require powerful Docker access for real LocalDevStack functionality. + +Do not remove the socket merely to make a security checklist look better. + +Document the actual boundary: + +```text +/var/run/docker.sock == effective host Docker control +``` + +## Tools reasons + +Tools/admin functionality includes container/service inspection and management. + +## Runner reasons + +Runner helpers and user scheduler definitions may execute into sibling containers. + +## Plan + +1. inventory exact Docker commands used; +2. classify read/write/destructive operations; +3. retain required socket mounts; +4. ensure `llm-sm` never receives the socket; +5. do not mount socket into ordinary databases/admin clients; +6. document that LocalDevStack is trusted local developer infrastructure. + +A socket proxy is not required unless a future design demonstrates a useful permission reduction without breaking the product. + +--- + +# 15. Docker config files + +## `docker/conf/filebeat.yml` + +Validate: + +- current Elasticsearch/Filebeat version alignment; +- log paths; +- service DNS endpoint; +- no static IP reference. + +## `docker/conf/openssl.cnf` + +Validate current OpenSSL compatibility. + +Do not weaken TLS globally just to support old clients unless a supported LocalDevStack flow requires it. + +## `docker/conf/pg_hba.conf` + +Remove any fixed-subnet assumptions. + +Keep local Docker-network auth appropriately scoped. + +## `docker/conf/postgresql.conf` + +Currently optional/commented. + +Decide one of: + +- intentionally supported and tested; or +- clearly documented as inactive reference config. + +Do not leave ambiguous pseudo-active config. + +## `docker/conf/www-php.conf` + +Validate generated FPM pool/socket integration. + +## `docker/conf/www.conf` + +Determine whether it is actively consumed. + +If unused, mark/deprecate/remove only after search and runtime tests. + +--- + +# 16. Generated/user-owned configuration + +## `configuration/compose/` + +Continue to hold generated user/project Compose fragments. + +Add safe stale-artifact detection. + +Never blindly delete files not known to LocalDevStack. + +## `configuration/php/` + +Preserve user-edited PHP configuration. + +Updates must not overwrite user customizations. + +## `configuration/scheduler/cron-jobs/` + +Validate generated files before Runner consumes them. + +Account for Windows CRLF. + +## `configuration/scheduler/supervisor/` + +Validate Supervisor syntax before stack restart where possible. + +## SOPS directories + +Paths: + +- `configuration/sops/config`; +- `configuration/sops/global`; +- `configuration/sops/keys`. + +Keep ignored/sensitive. + +Ensure setup permissions remain restrictive. + +## `configuration/ssh/` + +Keep optional and read-only. + +Never bake keys into images. + +## `configuration/ssl/` + +Reconcile actual host-visible state with named certificate volumes. + +Documentation must identify which paths are authoritative. + +--- + +# 17. Logs and rotation + +Keep host-visible logs because they are useful in a workstation stack. + +Validate compatibility with `runner:0.5` logrotate paths. + +Review directory permissions. + +Avoid broad `777` changes when a narrower cross-platform permission model works. + +If permissive permissions are still required for Windows/macOS/Linux interoperability, document the local-development rationale. + +Add a smoke that: + +1. writes a test log; +2. Runner sees it; +3. rotation succeeds; +4. application continues writing. + +--- + +# 18. Environment and precedence contract + +Define one documented precedence order. + +Recommended: + +1. built-in product fallback; +2. tracked `docker/release.env`; +3. user `docker/.env`; +4. command-scoped explicit environment overrides. + +Never shell-source untrusted dotenv content. + +Separate classes of settings: + +## Product/release + +- infrastructure image refs; +- default feature compatibility versions. + +## User stack + +- selected profiles; +- DB credentials; +- ports; +- timezone; +- project directory; +- runtime selections. + +## AI + +- enabled/profile; +- runtime variant; +- model; +- optional host port; +- advanced Tools limits. + +## Generated runtime + +- PHP packages/extensions; +- Node globals/packages; +- UID/GID; +- generated project profiles. + +`lds config` should be able to show effective non-secret configuration and redact secrets. + +--- + +# 19. Cross-platform requirements + +## Windows / Git Bash + +Preserve `lds.bat`. + +Tests must cover: + +- Git executable discovery; +- Git Bash discovery; +- spaces in repo/project paths; +- `cygpath` conversion; +- Docker Desktop unavailable/running errors; +- working-directory preservation; +- CRLF-sensitive generated files. + +## WSL + +Avoid assuming Docker socket path/platform behavior that conflicts with Docker Desktop integration. + +## macOS + +Account for bind-mount UID behavior and browser trust-store commands. + +## Linux + +Preserve UID/GID mapping and native Docker behavior. + +Avoid root-owned host project files after normal `lds` commands. + +--- + +# 20. QoL improvements that belong in this phase + +Implement only after core compatibility is stable. + +## `lds status` + +One concise product view: + +- core services; +- selected profiles; +- health; +- domains; +- URLs; +- AI enabled/provider status. + +## `lds urls` + +Print known convenience URLs: + +```text +https://admin.localhost +https://webmail.localhost +https://db.localhost +https://ri.localhost +https://me.localhost +https://kibana.localhost +https://llm.localhost # when AI enabled +``` + +Only show profile-dependent URLs when relevant. + +## `lds open ` + +Open a known local service in the host browser using existing platform helpers. + +## `lds doctor` + +Check: + +- Docker; +- Compose; +- expected networks; +- volume access; +- certificate state; +- port conflicts; +- image availability; +- selected service health; +- DNS/service resolution; +- optional AI provider status. + +Doctor should diagnose, not mutate, unless the user explicitly chooses a fix action. + +## `lds images` + +Show effective infrastructure image compatibility versions. + +This is useful when troubleshooting a mixed/overridden stack. + +--- + +# 21. Documentation rewrite + +LocalDevStack documentation should now describe the product users actually have. + +## `README.md` + +Lead with: + +> Docker-based XAMPP alternative for PHP and Node.js local development. + +First screen should explain: + +- local domains; +- HTTPS; +- PHP/Node versions; +- databases; +- admin tools; +- mail; +- background workers; +- optional local AI. + +Keep internal architecture below quickstart. + +## `docs/concepts/architecture.rst` + +Update responsibility map with the exact image split and optional AI provider. + +## `docs/concepts/profiles-and-env.rst` + +Document: + +- release env vs user env; +- profile selection; +- runtime variant; +- AI profile; +- override precedence. + +## `docs/concepts/storage-layout.rst` + +Clearly separate: + +- Docker named volumes; +- host configuration; +- host logs; +- project bind mounts; +- secrets; +- AI model volume. + +## `docs/quickstart.rst` + +Target beginner flow: + +```text +lds setup init +lds setup permissions +lds setup profile +lds setup domain +lds up +``` + +Use the actual final command names after implementation. + +## `docs/guides/domain-setup.rst` + +Remove static-IP mental model. + +Explain Docker DNS routing. + +## `docs/guides/tls-and-certificates.rst` + +Ensure `llm.localhost` and convenience-host certificate behavior is covered. + +## New `docs/guides/local-ai.rst` + +Cover: + +- enabling AI; +- CPU/NVIDIA/AMD; +- `https://llm.localhost`; +- `lds ai`; +- `lds llm`; +- model persistence; +- selecting a different model; +- direct host port opt-in; +- privacy boundaries; +- optional workspace access; +- Graphify connection. + +## Existing SOPS/notification docs + +Revalidate against `tools:0.23.2`. + +--- + +# 22. Migration/backward compatibility + +The first integrated LocalDevStack release must handle existing installations deliberately. + +## Existing databases/volumes + +Do not rename volumes in this release. + +## Existing Nginx vhost volume + +Older volumes may contain upstream `default.conf` artifacts. + +Use the Nginx 0.4.1 documented cleanup/migration behavior and test an upgraded volume. + +## Existing fixed networks + +Compose recreation may replace old fixed networks. + +Document that containers may be recreated while named-volume data remains. + +Do not run destructive `docker compose down -v` during migration. + +## Existing `docker/.env` + +Preserve user values. + +New release defaults must not overwrite it. + +## Existing generated runtime images + +Detect/rebuild only when relevant inputs changed. + +## Existing users without AI + +Their stack should not pull `llm-sm`, create the model volume or consume GPU resources unless AI is selected. + +--- + +# 23. Release-readiness matrix + +A LocalDevStack release candidate is not ready until these pass. + +## Core + +- clean install; +- existing-install upgrade; +- `lds help`; +- setup init; +- setup permissions; +- setup profiles; +- domain create/delete; +- trusted HTTPS; +- Nginx core routing; +- Tools admin; +- Mailpit; +- Runner. + +## PHP + +At least: + +- one current PHP runtime build; +- Composer; +- FPM through Nginx; +- FPM through Apache path if supported; +- custom extension/package fixture. + +## Node + +At least: + +- one current Node runtime build; +- npm/npx; +- Node proxy; +- WebSocket/HMR fixture. + +## Databases + +Smoke: + +- PostgreSQL; +- MySQL; +- MariaDB; +- MongoDB; +- Redis; +- Elasticsearch. + +Include admin clients where practical. + +## AI + +With fake provider on normal CI: + +- Tools provider; +- `askai`; +- `aiops`; +- Nginx LLM route; +- streaming. + +With real provider on manual/release gate when feasible: + +- `infocyph/llm-sm:0.03`; +- baked `qwen2.5:3b`; +- persistent model volume; +- Tools generation; +- Nginx `llm.localhost`. -### `docs/quickstart.rst` +## Platforms -Update after final CLI flow is stable; keep XAMPP-like beginner path concise. +At minimum: -### `docs/guides/domain-setup.rst` +- Linux full integration; +- Windows bridge/path validation; +- explicit documentation/manual verification for macOS/WSL if CI environment does not support full Docker Desktop tests. -Document Docker-DNS routing and domain creation without static IP assumptions. +--- -### `docs/guides/tls-and-certificates.rst` +# 24. Must-ship vs follow-up -Keep cross-platform trust instructions synchronized with actual `lds certificate` behavior. +## Must ship -### `docs/guides/secrets-sops-age.rst` +- product CI; +- explicit published infrastructure versions; +- static-IP removal; +- core Compose validation; +- AI profile/provider integration; +- persistent LLM model volume; +- Tools AI env wiring; +- PHP/Node Scriptomatic `main` migration; +- safe `lds` modularization of touched areas; +- DB health/env correctness fixes discovered by CI; +- updated docs; +- release/upgrade smoke. -Validate against current Tools `senv` contract. +## Follow-up allowed -### `docs/guides/notifications.rst` +These do not block the LocalDevStack integration release unless implementation reveals a direct dependency: -Validate notifier contract and clarify optional nature. +- multi-instance container/volume namespacing; +- automatic dependency-update PRs; +- Docker socket proxy; +- Graphify installation; +- browser AI UI beyond Tools admin panel; +- automatic GPU detection; +- automatic model downloads beyond the baked model; +- production-hardening changes unrelated to local development; +- rewriting the CLI in another language. -### `.readthedocs.yaml` / `docs/conf.py` / `docs/requirements.txt` +--- -Pin/document docs dependencies enough for reproducible docs builds; validate Read the Docs build in CI if useful. +# 25. Definition of completion -## Static Networking Migration Sequence +This LocalDevStack phase is complete when all of the following are true: -1. Add CI/tests resolving all services by DNS name. -2. Search all product/support repos for fixed `172.28/29/30` dependencies. -3. Remove per-service `ipv4_address` declarations from Compose files. -4. Remove IPAM subnet/gateway blocks from `main.yaml`. -5. Run PHP/Node/DB/admin/domain/TLS integration tests. -6. Re-evaluate `lds vpn-fix`: - - delete/deprecate if its only purpose was static-subnet conflict; - - retain only independently useful VPN behavior with updated docs. +1. LocalDevStack consumes the published compatibility matrix by explicit default. +2. No core LocalDevStack service requires a hard-coded `172.28/29/30` address. +3. `lds` still presents the existing public workflow while internals are better separated and tested. +4. PHP and Node remain locally customizable runtime builds. +5. Scriptomatic consumption no longer uses stale `master` references. +6. Tools, Runner, Nginx and Apache integrate using their published health/runtime contracts. +7. Local domains/TLS work through service-name routing. +8. Databases and admin clients work through Docker DNS. +9. Mailpit remains persistent and TLS-capable. +10. Runner cron/Supervisor/logrotate workflows still work. +11. AI can be omitted completely with no degradation to the default product. +12. When AI is enabled, `llm-sm` persists models and is reachable internally at `http://llm-sm:11434`. +13. `https://llm.localhost` works through Nginx streaming proxy behavior. +14. Tools `askai`, `aiops` and AI-enabled `gitx` use the separate LLM provider. +15. No LocalDevStack component embeds a second Ollama runtime. +16. No AI component auto-executes model-generated commands. +17. Existing user volumes and env overrides survive upgrade. +18. Docker socket exposure is documented and limited to components that actually require it. +19. Product docs match the implemented storage, network, version and AI behavior. +20. A clean supported workstation can go from clone/setup to a working HTTPS PHP or Node local domain using the documented flow. -## Docker Socket Review Sequence +--- -1. Trace every `docker` command in Tools/Runner and admin panel. -2. Categorize read vs write operations. -3. Determine if Runner requires socket directly or only specific mounted jobs do. -4. Determine if Tools requires full socket for domain/profile/admin functionality. -5. Keep required access for local-dev UX; remove redundant mounts. -6. Document trust boundary prominently. +# 26. First implementation checkpoint -## Acceptance Criteria +Before any broad refactor, the first implementation PR/batch should contain only: -1. Product CI exists and covers CLI/Compose/runtime generation. -2. `lds` public command surface remains compatible after modularization. -3. Canonical service catalog is consumed by both LocalDevStack and Tools. -4. Infrastructure image defaults are compatibility-tested/pinnable instead of unconditional `latest`. -5. PHP and Node remain dynamically customizable local builds. -6. All core service communication works through Docker DNS without fixed IPv4 assignments. -7. Local domain/TLS flows work on supported host classes. -8. DB/cache/mail/admin clients persist data appropriately. -9. Runner/scheduler/supervisor workflows remain functional. -10. Docker socket mounts are justified and minimized. -11. Optional `llm-sm` profile works with persistent models and published images only. -12. Documentation matches actual networking/storage/runtime behavior. +1. permanent LocalDevStack CI foundation; +2. tracked compatibility image defaults; +3. Compose references switched from infrastructure `:latest` to those defaults; +4. characterization tests around current `lds`; +5. no static-IP removal yet; +6. no `lds` modularization yet. + +Once that is green, proceed to the networking migration. + +This gives every later change a reliable regression boundary. From 5c3d062b224f90b170b11d4c364907707508d40c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 10:31:02 +0600 Subject: [PATCH 013/509] docs: mark lower ecosystem phases complete --- .../00-master-bottom-up-plan.md | 35 ++++++++++++++----- 1 file changed, 26 insertions(+), 9 deletions(-) diff --git a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md index 94f3d3d3..6a17147d 100644 --- a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md +++ b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md @@ -4,7 +4,18 @@ Planning branch: `plan/docker-ecosystem-bottom-up` -This branch is planning-only. No runtime behavior should be changed until the relevant repository plan is accepted and implementation work moves into that repository. +Lower-layer work is now complete and published. The active phase is LocalDevStack product integration. + +Completed/published baseline as of 2026-09-18: + +- Shared foundations: Scriptomatic hardened on `main`, Toolset `2.0` +- `infocyph/runner:0.5` +- `infocyph/nginx:0.4.1` +- `infocyph/apache:0.4.2` +- `infocyph/tools:0.23.2` +- `infocyph/llm-sm:0.03` / `amd-0.03` + +The authoritative active implementation plan is now `07-localdevstack-integration-plan.md`. Earlier 01–06 files remain historical planning records for the completed lower layers. ## Product Definition @@ -104,7 +115,7 @@ Preserve Linux, macOS, WSL/Git Bash, and Windows Docker Desktop support where cu ## Execution Order -### Phase 0 — Shared foundations +### Phase 0 — Shared foundations — COMPLETE Plan: `01-shared-foundations-plan.md` @@ -116,9 +127,15 @@ Exit gate: - required helper scripts have syntax/smoke validation; - no downstream migration depends on floating helper content. -### Phase 1 — Leaf infrastructure images +### Phase 1 — Leaf infrastructure images — COMPLETE + +Completed releases: + +- Runner `0.5` +- Nginx `0.4.1` +- Apache `0.4.2` -Execute independently where possible: +Historical plans: - `02-docker-runner-plan.md` - `03-docker-nginx-plan.md` @@ -140,11 +157,11 @@ Exit gate for each image: - release workflow follows immutable-version contract; - LocalDevStack can consume the resulting release without behavior regression. -### Phase 2 — Control plane +### Phase 2 — Control plane — COMPLETE Plan: `05-docker-tools-plan.md` -This is the largest supporting image and must be handled after foundational/leaf contracts are stable. +This control-plane phase is complete and published as `infocyph/tools:0.23.2`. Primary goals: @@ -155,11 +172,11 @@ Primary goals: - preserve domain/TLS/secrets/monitoring/admin workflows; - avoid turning `docker-tools` into the owner of LocalDevStack orchestration policy. -### Phase 3 — Local AI capability +### Phase 3 — Local AI capability — COMPLETE Plan: `06-docker-llm-sm-plan.md` -`docker-llm-sm` is already published and is primarily a compatibility/integration reference, not a rewrite target. +`docker-llm-sm` is published as `0.03` and its provider/runtime contract is complete for this program. Primary goals: @@ -169,7 +186,7 @@ Primary goals: - optionally mount project workspace for repo-aware AI commands; - allow Graphify or other clients to use its Ollama endpoint without coupling them into the image. -### Phase 4 — Product/orchestrator +### Phase 4 — Product/orchestrator — ACTIVE Plan: `07-localdevstack-integration-plan.md` From 98077872587df205248f427e4231d3f3f91bbe96 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:09:52 +0600 Subject: [PATCH 014/509] ci: add LocalDevStack characterization foundation --- .github/workflows/check.yml | 99 +++++++++++++++++++++++++++ .gitignore | 4 ++ tests/ai-contract.sh | 41 +++++++++++ tests/cli-contract.sh | 43 ++++++++++++ tests/compose-contract.sh | 61 +++++++++++++++++ tests/env-contract.sh | 22 ++++++ tests/fixtures/fake-ollama/Dockerfile | 8 +++ tests/fixtures/fake-ollama/server.py | 94 +++++++++++++++++++++++++ tests/lib/assertions.sh | 35 ++++++++++ tests/networking-contract.sh | 37 ++++++++++ tests/published-images.sh | 33 +++++++++ tests/release-gate.sh | 14 ++++ tests/runtime-node-contract.sh | 21 ++++++ tests/runtime-php-contract.sh | 22 ++++++ tests/static.sh | 38 ++++++++++ tests/windows-bridge.ps1 | 47 +++++++++++++ 16 files changed, 619 insertions(+) create mode 100644 .github/workflows/check.yml create mode 100755 tests/ai-contract.sh create mode 100755 tests/cli-contract.sh create mode 100755 tests/compose-contract.sh create mode 100755 tests/env-contract.sh create mode 100644 tests/fixtures/fake-ollama/Dockerfile create mode 100644 tests/fixtures/fake-ollama/server.py create mode 100644 tests/lib/assertions.sh create mode 100755 tests/networking-contract.sh create mode 100755 tests/published-images.sh create mode 100755 tests/release-gate.sh create mode 100755 tests/runtime-node-contract.sh create mode 100755 tests/runtime-php-contract.sh create mode 100755 tests/static.sh create mode 100644 tests/windows-bridge.ps1 diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml new file mode 100644 index 00000000..030314ed --- /dev/null +++ b/.github/workflows/check.yml @@ -0,0 +1,99 @@ +name: Check + +on: + push: + branches: + - main + - "plan/**" + - "feature/**" + pull_request: + workflow_dispatch: + +permissions: + contents: read + +concurrency: + group: check-${{ github.workflow }}-${{ github.ref }} + cancel-in-progress: true + +jobs: + static-and-contracts: + name: Static and contracts + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Install ShellCheck + run: | + sudo apt-get update + sudo apt-get install -y --no-install-recommends shellcheck + + - name: Static validation + run: tests/static.sh + + - name: CLI contract + run: tests/cli-contract.sh + + - name: Environment contract + run: tests/env-contract.sh + + - name: PHP runtime contract + run: tests/runtime-php-contract.sh + + - name: Node runtime contract + run: tests/runtime-node-contract.sh + + - name: Networking characterization + run: tests/networking-contract.sh + + compose: + name: Compose contract + runs-on: ubuntu-latest + timeout-minutes: 15 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Docker versions + run: | + docker version + docker compose version + + - name: Render supported Compose matrices + run: tests/compose-contract.sh + + fake-ollama: + name: Fake Ollama contract + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Validate lightweight Ollama-compatible fixture + run: tests/ai-contract.sh + + published-images: + name: Published image baseline + runs-on: ubuntu-latest + timeout-minutes: 25 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Validate published compatibility images + run: tests/published-images.sh + + windows-bridge: + name: Windows bridge + runs-on: windows-latest + timeout-minutes: 10 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Validate lds.bat contract + shell: pwsh + run: tests/windows-bridge.ps1 diff --git a/.gitignore b/.gitignore index 68191ca9..d4339402 100644 --- a/.gitignore +++ b/.gitignore @@ -7,6 +7,10 @@ !docker/dockerfiles !docs !logs +!.github/ +!.github/** +!tests/ +!tests/** !.gitignore !.gitattributes !.readthedocs.yaml diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh new file mode 100755 index 00000000..607ee689 --- /dev/null +++ b/tests/ai-contract.sh @@ -0,0 +1,41 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +image="lds-fake-ollama:ci" +container="lds-fake-ollama-ci" + +cleanup() { + docker rm -f "$container" >/dev/null 2>&1 || true +} +trap cleanup EXIT + +docker build -q -t "$image" "$ROOT/tests/fixtures/fake-ollama" >/dev/null +docker run -d --name "$container" "$image" >/dev/null + +for _ in {1..20}; do + if docker exec "$container" python -c 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=1).read()' >/dev/null 2>&1; then + break + fi + sleep 1 +done + +tags="$( + docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=2).read().decode())' +)" +assert_contains "$tags" "qwen2.5:3b" + +generate="$( + docker exec "$container" python -c 'import urllib.request; r=urllib.request.Request("http://127.0.0.1:11434/api/generate", data=b"{}", headers={"Content-Type":"application/json"}); print(urllib.request.urlopen(r, timeout=2).read().decode())' +)" +assert_contains "$generate" "LocalDevStack CI" + +models="$( + docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/v1/models", timeout=2).read().decode())' +)" +assert_contains "$models" "qwen2.5:3b" + +pass "fake Ollama tags/generate/OpenAI-compatible contracts" diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh new file mode 100755 index 00000000..9d46e7c5 --- /dev/null +++ b/tests/cli-contract.sh @@ -0,0 +1,43 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +help_output="$("$ROOT/lds" help)" +assert_contains "$help_output" "LocalDevStack" +assert_contains "$help_output" "Stack" +assert_contains "$help_output" "Domain" +assert_contains "$help_output" "Setup" +pass "lds help" + +markdown_output="$("$ROOT/lds" help --markdown)" +assert_contains "$markdown_output" "# LocalDevStack" +assert_contains "$markdown_output" "lds stack up" +pass "lds markdown help" + +global_help="$("$ROOT/lds" --help)" +assert_contains "$global_help" "LocalDevStack" +pass "lds --help" + +code=0 +"$ROOT/lds" >/tmp/lds-noargs.out 2>/tmp/lds-noargs.err || code=$? +[[ "$code" -eq 1 ]] || fail "no-argument invocation must exit 1; got $code" +grep -q "LocalDevStack" /tmp/lds-noargs.out || fail "no-argument invocation must print help" +pass "no-argument behavior" + +tmpbin="$(mktemp -d)" +trap 'rm -rf "$tmpbin" /tmp/lds-noargs.out /tmp/lds-noargs.err' EXIT +cat >"$tmpbin/docker" <<'SH' +#!/usr/bin/env sh +exit 0 +SH +chmod +x "$tmpbin/docker" + +stack_help="$(PATH="$tmpbin:$PATH" "$ROOT/lds" stack help)" +assert_contains "$stack_help" "LocalDevStack" +pass "grouped stack help routing" + +assert_file_contains "$ROOT/lds" 'exec "$DIR/bin/tool-runner" "$cmd" "$@"' +pass "unknown command fallback remains delegated to tool-runner" diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh new file mode 100755 index 00000000..d4fd7537 --- /dev/null +++ b/tests/compose-contract.sh @@ -0,0 +1,61 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +command -v docker >/dev/null 2>&1 || fail "docker is required" +docker compose version >/dev/null 2>&1 || fail "docker compose plugin is required" + +env_file="$ROOT/docker/.env" +created_env=0 +if [[ ! -e "$env_file" ]]; then + created_env=1 + cat >"$env_file" <&2 + exit 1 +} + +pass() { + printf 'PASS: %s\n' "$*" +} + +assert_file() { + local file="$1" + [[ -f "$file" ]] || fail "expected file: $file" +} + +assert_contains() { + local haystack="$1" + local needle="$2" + [[ "$haystack" == *"$needle"* ]] || fail "expected output to contain: $needle" +} + +assert_file_contains() { + local file="$1" + local needle="$2" + grep -Fq -- "$needle" "$file" || fail "$file does not contain: $needle" +} + +assert_exit() { + local expected="$1" + shift + local code=0 + "$@" >/dev/null 2>&1 || code=$? + [[ "$code" -eq "$expected" ]] || fail "expected exit $expected, got $code: $*" +} diff --git a/tests/networking-contract.sh b/tests/networking-contract.sh new file mode 100755 index 00000000..136945bd --- /dev/null +++ b/tests/networking-contract.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +main="$ROOT/docker/compose/main.yaml" +db="$ROOT/docker/compose/db.yaml" +clients="$ROOT/docker/compose/db-client.yaml" +companion="$ROOT/docker/compose/companion.yaml" +http="$ROOT/docker/compose/http.yaml" + +for network in frontend backend datastore; do + grep -Eq "^ ${network}:" "$main" || fail "missing logical network: $network" +done +pass "logical network names" + +assert_file_contains "$clients" '@mongodb:' +assert_file_contains "$clients" 'ELASTICSEARCH_HOSTS=http://elasticsearch:9200' +assert_file_contains "$companion" 'hostname: runner' +assert_file_contains "$http" 'hostname: nginx' +assert_file_contains "$db" 'hostname: postgres' +pass "service-name/Docker-DNS contracts are present" + +static_count="$( + grep -Rhs 'ipv4_address:' "$ROOT/docker/compose" | wc -l | tr -d ' ' +)" +printf 'INFO: baseline contains %s explicit ipv4_address declarations; Batch 3 removes them.\n' "$static_count" + +if [[ "$static_count" -gt 0 ]]; then + grep -Fq '172.28.0.0/24' "$main" || fail "baseline frontend subnet changed unexpectedly" + grep -Fq '172.29.0.0/24' "$main" || fail "baseline backend subnet changed unexpectedly" + grep -Fq '172.30.0.0/24' "$main" || fail "baseline datastore subnet changed unexpectedly" +fi + +pass "static-network baseline characterized" diff --git a/tests/published-images.sh b/tests/published-images.sh new file mode 100755 index 00000000..715dcf80 --- /dev/null +++ b/tests/published-images.sh @@ -0,0 +1,33 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +images=( + "infocyph/tools:0.23.2" + "infocyph/runner:0.5" + "infocyph/nginx:0.4.1" + "infocyph/apache:0.4.2" +) + +for image in "${images[@]}"; do + printf 'Pulling %s\n' "$image" + docker pull "$image" + docker image inspect "$image" >/dev/null +done +pass "published infrastructure compatibility images exist" + +for image in "infocyph/tools:0.23.2" "infocyph/runner:0.5"; do + health="$(docker image inspect "$image" --format '{{json .Config.Healthcheck}}')" + [[ -n "$health" && "$health" != "null" ]] || fail "$image must publish a healthcheck" +done +pass "Tools and Runner publish healthchecks" + +# The model-bearing LLM image is intentionally not pulled on every PR. +# Its exact release reference is still characterized here and is exercised +# by the LocalDevStack real-provider release gate once AI integration lands. +assert_file_contains "$ROOT/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md" 'infocyph/llm-sm:0.03' +assert_file_contains "$ROOT/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md" 'infocyph/llm-sm:amd-0.03' +pass "LLM release references are pinned in the integration plan" diff --git a/tests/release-gate.sh b/tests/release-gate.sh new file mode 100755 index 00000000..70b47724 --- /dev/null +++ b/tests/release-gate.sh @@ -0,0 +1,14 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" + +"$ROOT/tests/static.sh" +"$ROOT/tests/cli-contract.sh" +"$ROOT/tests/env-contract.sh" +"$ROOT/tests/compose-contract.sh" +"$ROOT/tests/networking-contract.sh" +"$ROOT/tests/runtime-php-contract.sh" +"$ROOT/tests/runtime-node-contract.sh" +"$ROOT/tests/ai-contract.sh" +"$ROOT/tests/published-images.sh" diff --git a/tests/runtime-node-contract.sh b/tests/runtime-node-contract.sh new file mode 100755 index 00000000..c83c8d2f --- /dev/null +++ b/tests/runtime-node-contract.sh @@ -0,0 +1,21 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +file="$ROOT/docker/dockerfiles/node.Dockerfile" +assert_file "$file" +assert_file_contains "$file" 'ARG NODE_VERSION=current' +assert_file_contains "$file" 'FROM node:${NODE_VERSION}-alpine' +assert_file_contains "$file" 'ARG LINUX_PKG' +assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED' +assert_file_contains "$file" 'ARG NODE_GLOBAL' +assert_file_contains "$file" 'ARG NODE_GLOBAL_VERSIONED' +assert_file_contains "$file" 'ARG UID=1000' +assert_file_contains "$file" 'ARG GID=1000' +assert_file_contains "$file" 'node-cli-setup.sh' +assert_file_contains "$file" 'EXPOSE 3000' +assert_file_contains "$file" 'ENTRYPOINT ["/usr/local/bin/node-entry"]' +pass "Node runtime customization contract" diff --git a/tests/runtime-php-contract.sh b/tests/runtime-php-contract.sh new file mode 100755 index 00000000..12e837fd --- /dev/null +++ b/tests/runtime-php-contract.sh @@ -0,0 +1,22 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +file="$ROOT/docker/dockerfiles/php.Dockerfile" +assert_file "$file" +assert_file_contains "$file" 'ARG PHP_VERSION=8.4' +assert_file_contains "$file" 'FROM php:${PHP_VERSION}-fpm-alpine' +assert_file_contains "$file" 'ARG PHP_PROFILE_KEY=84' +assert_file_contains "$file" 'ARG LINUX_PKG' +assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED' +assert_file_contains "$file" 'ARG PHP_EXT' +assert_file_contains "$file" 'ARG PHP_EXT_VERSIONED' +assert_file_contains "$file" 'ARG UID=1000' +assert_file_contains "$file" 'ARG GID=1000' +assert_file_contains "$file" 'php-cli-setup.sh' +assert_file_contains "$file" 'ENTRYPOINT ["/usr/local/bin/php-entry"]' +assert_file_contains "$file" 'CMD ["php-fpm"]' +pass "PHP runtime customization contract" diff --git a/tests/static.sh b/tests/static.sh new file mode 100755 index 00000000..d553be07 --- /dev/null +++ b/tests/static.sh @@ -0,0 +1,38 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +mapfile -t shell_files < <( + { + printf '%s\n' "$ROOT/lds" + find "$ROOT/bin" -maxdepth 1 -type f -print + find "$ROOT/tests" -type f -name '*.sh' -print + } | sort -u +) + +(("${#shell_files[@]}" > 0)) || fail "no shell files found" + +for file in "${shell_files[@]}"; do + bash -n "$file" +done +pass "bash syntax" + +if command -v shellcheck >/dev/null 2>&1; then + shellcheck --severity=error -x "${shell_files[@]}" + pass "ShellCheck error-level gate" + + mapfile -t test_shell_files < <(find "$ROOT/tests" -type f -name '*.sh' -print | sort) + shellcheck --severity=warning -x "${test_shell_files[@]}" + pass "ShellCheck warning-level gate for tests" +fi + +while IFS= read -r file; do + if grep -Iq . "$file" && grep -q $'\r$' "$file"; then + fail "CRLF detected in shell file: $file" + fi +done < <(printf '%s\n' "${shell_files[@]}") + +pass "shell files use LF endings" diff --git a/tests/windows-bridge.ps1 b/tests/windows-bridge.ps1 new file mode 100644 index 00000000..de114503 --- /dev/null +++ b/tests/windows-bridge.ps1 @@ -0,0 +1,47 @@ +$ErrorActionPreference = "Stop" + +$root = Split-Path -Parent $PSScriptRoot +$batPath = Join-Path $root "lds.bat" + +if (-not (Test-Path $batPath)) { + throw "lds.bat not found" +} + +$content = Get-Content -Raw -Path $batPath + +$required = @( + 'set "DEVHOME=%~dp0"', + 'set "WORKDIR=%CD%"', + 'where git.exe', + 'Get-Command git', + '\bin\bash.exe', + '\usr\bin\bash.exe', + 'cygpath -u', + '--__win_workdir', + '"%DEVHOME%" "%WORKDIR%" %*' +) + +foreach ($needle in $required) { + if (-not $content.Contains($needle)) { + throw "lds.bat is missing expected bridge contract: $needle" + } +} + +# Characterize the existing Docker preflight. Batch 7 will move Docker +# availability checks behind commands that actually require Docker. +if (-not $content.Contains('docker info')) { + Write-Host "INFO: unconditional Docker preflight is already absent" +} else { + Write-Host "INFO: current bridge still performs the known unconditional Docker preflight" +} + +$tempParent = Join-Path $env:RUNNER_TEMP "Local Dev Stack" +New-Item -ItemType Directory -Force -Path $tempParent | Out-Null +Copy-Item -Path $batPath -Destination (Join-Path $tempParent "lds.bat") -Force + +$copied = Get-Content -Raw -Path (Join-Path $tempParent "lds.bat") +if (-not $copied.Contains('set "DEVHOME=%~dp0"')) { + throw "bridge contract was not preserved when copied under a path containing spaces" +} + +Write-Host "PASS: Windows bridge quoting/discovery contract" From 8516be5f04f8d76aafe3235499deaaf151abc4fb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:10:37 +0600 Subject: [PATCH 015/509] ci: fix php wrapper shellcheck expansion --- bin/php | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/bin/php b/bin/php index cee67318..ecdcc30e 100755 --- a/bin/php +++ b/bin/php @@ -130,11 +130,11 @@ port_in_use() { return 1 fi if command -v ss >/dev/null 2>&1; then - ss -lnt 2>/dev/null | grep -qE "[:.]$port[[:space:]]" && return 0 + ss -lnt 2>/dev/null | grep -qE "[:.]${port}[[:space:]]" && return 0 return 1 fi if command -v netstat >/dev/null 2>&1; then - netstat -lnt 2>/dev/null | grep -qE "[:.]$port[[:space:]]" && return 0 + netstat -lnt 2>/dev/null | grep -qE "[:.]${port}[[:space:]]" && return 0 return 1 fi return 1 From 5bc43b637e7abef09bb6f4cf9241755df23bdfb9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:15:17 +0600 Subject: [PATCH 016/509] build: pin LocalDevStack compatibility images --- .gitignore | 1 + docker/compose/companion.yaml | 4 +-- docker/compose/http.yaml | 4 +-- docker/release.env | 8 ++++++ lds | 10 ++++++- tests/compose-contract.sh | 54 ++++++++++++++++++++++++++--------- tests/env-contract.sh | 28 ++++++++++++++++-- tests/published-images.sh | 31 +++++++++++++------- 8 files changed, 108 insertions(+), 32 deletions(-) create mode 100644 docker/release.env diff --git a/.gitignore b/.gitignore index d4339402..c224cecb 100644 --- a/.gitignore +++ b/.gitignore @@ -5,6 +5,7 @@ !docker/compose !docker/conf !docker/dockerfiles +!docker/release.env !docs !logs !.github/ diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 20ac3cdb..80667beb 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -2,7 +2,7 @@ services: server-tools: container_name: SERVER_TOOLS hostname: local-dock - image: infocyph/tools:latest + image: ${LDS_TOOLS_IMAGE:-infocyph/tools:0.23.2} restart: unless-stopped environment: - TZ=${TZ:-} @@ -47,7 +47,7 @@ services: runner: container_name: RUNNER hostname: runner - image: infocyph/runner:latest + image: ${LDS_RUNNER_IMAGE:-infocyph/runner:0.5} restart: unless-stopped environment: - TZ=${TZ:-} diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 6091018c..f71a6012 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -2,7 +2,7 @@ services: nginx: container_name: NGINX hostname: nginx - image: infocyph/nginx:latest + image: ${LDS_NGINX_IMAGE:-infocyph/nginx:0.4.1} restart: unless-stopped environment: - TZ=${TZ:-} @@ -27,7 +27,7 @@ services: apache: container_name: APACHE hostname: apache - image: infocyph/apache:latest + image: ${LDS_APACHE_IMAGE:-infocyph/apache:0.4.2} restart: always environment: - TZ=${TZ:-} diff --git a/docker/release.env b/docker/release.env new file mode 100644 index 00000000..15e1ef3f --- /dev/null +++ b/docker/release.env @@ -0,0 +1,8 @@ +# LocalDevStack release-owned compatibility baseline. +# User overrides belong in docker/.env. Do not edit this file for local settings. +LDS_TOOLS_IMAGE=infocyph/tools:0.23.2 +LDS_RUNNER_IMAGE=infocyph/runner:0.5 +LDS_NGINX_IMAGE=infocyph/nginx:0.4.1 +LDS_APACHE_IMAGE=infocyph/apache:0.4.2 +LDS_LLM_IMAGE=infocyph/llm-sm:0.03 +LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-0.03 diff --git a/lds b/lds index 545070c1..fcd1296b 100755 --- a/lds +++ b/lds @@ -303,6 +303,7 @@ DIR="$(dirname -- "$(_realpath "$0")")" CFG="$DIR/docker" ENV_MAIN="$DIR/.env" ENV_DOCKER="$CFG/.env" +ENV_RELEASE="$CFG/release.env" COMPOSE_FILE="$CFG/compose/main.yaml" EXTRAS_DIR="$DIR/configuration/compose" @@ -422,6 +423,13 @@ docker_compose() { fi fi + [[ -r "$ENV_RELEASE" ]] || die "Missing release compatibility manifest: $ENV_RELEASE" + + # Release defaults are loaded first; user docker/.env overrides them. + # Shell variables remain higher-precedence Compose interpolation inputs. + local -a env_files=(--env-file "$ENV_RELEASE") + [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") + # Build -f list (stable order; later overrides earlier) local -a extra_f=() f for f in "${__EXTRA_FILES[@]:-}"; do @@ -437,7 +445,7 @@ docker_compose() { --project-directory "$DIR" \ -f "$COMPOSE_FILE" \ "${extra_f[@]}" \ - --env-file "$ENV_DOCKER" \ + "${env_files[@]}" \ "$@" } diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index d4fd7537..abe2ac9f 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -8,27 +8,40 @@ source "$ROOT/tests/lib/assertions.sh" command -v docker >/dev/null 2>&1 || fail "docker is required" docker compose version >/dev/null 2>&1 || fail "docker compose plugin is required" -env_file="$ROOT/docker/.env" -created_env=0 -if [[ ! -e "$env_file" ]]; then - created_env=1 - cat >"$env_file" <"$user_env" <>"$user_env" +user_override="$("${compose[@]}" config)" +assert_contains "$user_override" "image: example.invalid/tools:user-override" +pass "docker/.env overrides release defaults" + +shell_override="$( + LDS_TOOLS_IMAGE=example.invalid/tools:shell-override "${compose[@]}" config +)" +assert_contains "$shell_override" "image: example.invalid/tools:shell-override" +pass "shell override wins over user and release env files" # Apache is currently an unconditional service in the baseline. This is # characterized here; Batch 5 will make the product's optional-HTTP contract -# explicit rather than silently changing it in the CI foundation batch. +# explicit rather than silently changing it in the compatibility batch. assert_contains "$resolved" "apache:" pass "current Apache compose presence characterized" diff --git a/tests/env-contract.sh b/tests/env-contract.sh index b3e19df3..bab92ea1 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -5,16 +5,38 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # shellcheck source=tests/lib/assertions.sh source "$ROOT/tests/lib/assertions.sh" +release_env="$ROOT/docker/release.env" + +assert_file "$release_env" assert_file_contains "$ROOT/lds" 'ENV_MAIN="$DIR/.env"' assert_file_contains "$ROOT/lds" 'ENV_DOCKER="$CFG/.env"' +assert_file_contains "$ROOT/lds" 'ENV_RELEASE="$CFG/release.env"' assert_file_contains "$ROOT/lds" 'COMPOSE_FILE="$CFG/compose/main.yaml"' -assert_file_contains "$ROOT/lds" '--env-file "$ENV_DOCKER"' +assert_file_contains "$ROOT/lds" 'local -a env_files=(--env-file "$ENV_RELEASE")' +assert_file_contains "$ROOT/lds" 'env_files+=(--env-file "$ENV_DOCKER")' +assert_file_contains "$ROOT/lds" '"${env_files[@]}"' assert_file_contains "$ROOT/lds" 'var=COMPOSE_PROFILES' -pass "environment file and profile locations" +pass "environment file and precedence wiring" git -C "$ROOT" check-ignore -q docker/.env || fail "docker/.env must remain ignored user state" git -C "$ROOT" check-ignore -q .env || fail ".env must remain ignored user state" -pass "user env files are not tracked" +if git -C "$ROOT" check-ignore -q docker/release.env; then + fail "docker/release.env must be tracked release state" +fi +pass "release and user env ownership boundaries" + +expected=( + 'LDS_TOOLS_IMAGE=infocyph/tools:0.23.2' + 'LDS_RUNNER_IMAGE=infocyph/runner:0.5' + 'LDS_NGINX_IMAGE=infocyph/nginx:0.4.1' + 'LDS_APACHE_IMAGE=infocyph/apache:0.4.2' + 'LDS_LLM_IMAGE=infocyph/llm-sm:0.03' + 'LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-0.03' +) +for entry in "${expected[@]}"; do + assert_file_contains "$release_env" "$entry" +done +pass "published compatibility manifest" for key in POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS; do grep -Fq "[$key]=" "$ROOT/lds" || fail "missing profile catalog entry: $key" diff --git a/tests/published-images.sh b/tests/published-images.sh index 715dcf80..41dd02f0 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -5,29 +5,38 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # shellcheck source=tests/lib/assertions.sh source "$ROOT/tests/lib/assertions.sh" +release_env="$ROOT/docker/release.env" +assert_file "$release_env" + +declare -A release=() +while IFS='=' read -r key value; do + [[ -n "$key" && "$key" != \#* ]] || continue + release["$key"]="$value" +done <"$release_env" + images=( - "infocyph/tools:0.23.2" - "infocyph/runner:0.5" - "infocyph/nginx:0.4.1" - "infocyph/apache:0.4.2" + "${release[LDS_TOOLS_IMAGE]:-}" + "${release[LDS_RUNNER_IMAGE]:-}" + "${release[LDS_NGINX_IMAGE]:-}" + "${release[LDS_APACHE_IMAGE]:-}" ) for image in "${images[@]}"; do + [[ -n "$image" ]] || fail "empty infrastructure image in release manifest" printf 'Pulling %s\n' "$image" docker pull "$image" docker image inspect "$image" >/dev/null done pass "published infrastructure compatibility images exist" -for image in "infocyph/tools:0.23.2" "infocyph/runner:0.5"; do +for image in "${release[LDS_TOOLS_IMAGE]}" "${release[LDS_RUNNER_IMAGE]}"; do health="$(docker image inspect "$image" --format '{{json .Config.Healthcheck}}')" [[ -n "$health" && "$health" != "null" ]] || fail "$image must publish a healthcheck" done pass "Tools and Runner publish healthchecks" -# The model-bearing LLM image is intentionally not pulled on every PR. -# Its exact release reference is still characterized here and is exercised -# by the LocalDevStack real-provider release gate once AI integration lands. -assert_file_contains "$ROOT/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md" 'infocyph/llm-sm:0.03' -assert_file_contains "$ROOT/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md" 'infocyph/llm-sm:amd-0.03' -pass "LLM release references are pinned in the integration plan" +[[ "${release[LDS_LLM_IMAGE]:-}" == "infocyph/llm-sm:0.03" ]] || + fail "unexpected LLM compatibility image" +[[ "${release[LDS_LLM_AMD_IMAGE]:-}" == "infocyph/llm-sm:amd-0.03" ]] || + fail "unexpected AMD LLM compatibility image" +pass "LLM compatibility references are release-pinned" From 0df2e283a8e8acc225184921025144ac5778a7f8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:14 +0600 Subject: [PATCH 017/509] network: add safe legacy network migration --- lds | 77 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 76 insertions(+), 1 deletion(-) diff --git a/lds b/lds index fcd1296b..806a3442 100755 --- a/lds +++ b/lds @@ -1444,9 +1444,83 @@ compose_image_for_service() { ############################################################################### # 6. STACK COMMANDS (CLI) ############################################################################### -cmd_up() { dc_up "$@"; } + +# One-time migration from the historical fixed 172.28/29/30 /24 networks. +# Only known LocalDevStack-owned networks are touched, and named volumes are +# never removed. New networks carry com.infocyph.network-schema=dynamic-v1. +declare -a __LDS_LEGACY_NETWORK_NAMES=(Frontend Backend DataStore) + +legacy_network_expected_subnet() { + case "\${1:-}" in + Frontend) printf '%s' '172.28.0.0/24' ;; + Backend) printf '%s' '172.29.0.0/24' ;; + DataStore) printf '%s' '172.30.0.0/24' ;; + *) return 1 ;; + esac +} + +migrate_legacy_networks() { + local network expected subnets stack_label project_label schema_label ctr ctr_project attachments + local -a legacy=() + + for network in "\${__LDS_LEGACY_NETWORK_NAMES[@]}"; do + docker network inspect "$network" >/dev/null 2>&1 || continue + + schema_label="$(docker network inspect -f '{{index .Labels "com.infocyph.network-schema"}}' "$network" 2>/dev/null || true)" + [[ "$schema_label" != "dynamic-v1" ]] || continue + + expected="$(legacy_network_expected_subnet "$network")" + subnets="$(docker network inspect -f '{{range .IPAM.Config}}{{println .Subnet}}{{end}}' "$network" 2>/dev/null || true)" + grep -Fxq "$expected" <<<"$subnets" || continue + + stack_label="$(docker network inspect -f '{{index .Labels "com.infocyph.stack"}}' "$network" 2>/dev/null || true)" + project_label="$(docker network inspect -f '{{index .Labels "com.docker.compose.project"}}' "$network" 2>/dev/null || true)" + if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "LocalDevStack" ]]; then + die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack. Remove or rename that network manually." + fi + + while IFS= read -r ctr; do + [[ -n "$ctr" ]] || continue + ctr_project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$ctr" 2>/dev/null || true)" + if [[ "$ctr_project" != "LocalDevStack" ]]; then + die "Refusing to migrate '$network': container '$ctr' is not owned by the LocalDevStack Compose project." + fi + done < <(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true) + + legacy+=("$network") + done + + (("\${#legacy[@]}" > 0)) || return 0 + + warn "Legacy fixed LocalDevStack network(s) detected: \${legacy[*]}" + warn "Recreating stack networks dynamically; named volumes and persisted data are preserved." + + # Stop/remove only LocalDevStack Compose containers and networks. Never use -v. + docker_compose down --remove-orphans + + for network in "\${legacy[@]}"; do + docker network inspect "$network" >/dev/null 2>&1 || continue + attachments="$(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true)" + [[ -z "$attachments" ]] || + die "Cannot remove legacy network '$network': attached container(s) remain: $(tr '\n' ' ' <<<"$attachments")" + docker network rm "$network" >/dev/null + done + + ok "Legacy fixed networks removed; Compose will recreate dynamic bridge networks." +} + +cmd_vpn_fix() { + warn "vpn-fix is deprecated: LocalDevStack no longer owns fixed Docker subnets." + warn "If a VPN conflict remains after dynamic-network migration, diagnose the VPN/Docker route directly." +} + +cmd_up() { + migrate_legacy_networks + dc_up "$@" +} cmd_start() { + migrate_legacy_networks dc_up -d "$@" http_reload } @@ -3461,6 +3535,7 @@ main() { redis | redis-cli) exec "$DIR/bin/redis-cli" "$@" ;; es | elastic | elasticsearch) exec "$DIR/bin/es" "$@" ;; mongo | mongodb | mongosh | mongoimport | mongoexport) exec "$DIR/bin/mongo" "$@" ;; + vpn-fix) cmd_vpn_fix "$@" ;; *) if declare -F "cmd_$cmd" >/dev/null 2>&1; then "cmd_$cmd" "$@" From 1d9a4afdef7bf7e25773b1be1a83fcbc2d0a3bda Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:38 +0600 Subject: [PATCH 018/509] network: remove fixed network IPAM --- docker/compose/main.yaml | 34 ++++++++++++++++++++++------------ 1 file changed, 22 insertions(+), 12 deletions(-) diff --git a/docker/compose/main.yaml b/docker/compose/main.yaml index 2e189244..1403f0d8 100644 --- a/docker/compose/main.yaml +++ b/docker/compose/main.yaml @@ -7,10 +7,7 @@ networks: com.docker.compose.project: "LocalDevStack" com.docker.compose.network: "frontend" com.infocyph.stack: "LocalDevStack" - ipam: - config: - - subnet: 172.28.0.0/24 - gateway: 172.28.0.1 + com.infocyph.network-schema: "dynamic-v1" backend: name: Backend driver: bridge @@ -18,10 +15,7 @@ networks: com.docker.compose.project: "LocalDevStack" com.docker.compose.network: "backend" com.infocyph.stack: "LocalDevStack" - ipam: - config: - - subnet: 172.29.0.0/24 - gateway: 172.29.0.1 + com.infocyph.network-schema: "dynamic-v1" datastore: name: DataStore driver: bridge @@ -29,124 +23,140 @@ networks: com.docker.compose.project: "LocalDevStack" com.docker.compose.network: "datastore" com.infocyph.stack: "LocalDevStack" - ipam: - config: - - subnet: 172.30.0.0/24 - gateway: 172.30.0.1 + com.infocyph.network-schema: "dynamic-v1" volumes: lds_fpm_sock: name: FPMSocks labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "PHP-FPM Sockets" lds_fpm_pools: name: FPMPools labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "PHP-FPM Pools" lds_composer_global: name: ComposerGlobal labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Composer Global Home" lds_ssl_keys: name: SSLKeys labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "SSL Keys" lds_ssl_roots: name: SSLRootCA labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "SSL Root CA" lds_git_config: name: GitConfig labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Git Config" lds_nginx_host: name: NginxHosts labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Nginx Host Configs" lds_apache_host: name: ApacheHosts labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Apache Host Configs" lds_mail: name: EmailStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Email Store" lds_redis: name: RedisStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Redis Data" lds_ri: name: RedisInsightStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Redis Insight Data" lds_pg: name: PostgresStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "PostgreSQL Data" lds_my: name: MySQLStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "MySQL Data" lds_maria: name: MariaDBStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "MariaDB Data" lds_mongo: name: MongoDBStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "MongoDB Data" lds_es: name: ElasticSearchStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "ElasticSearch Data" lds_kb: name: KibanaStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Kibana Data" lds_cb: name: CloudBeaverStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "CloudBeaver Data" lds_fbeat: name: FilebeatStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" + com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Filebeat Data" include: - docker/compose/companion.yaml From a8637fed7d1b9e7f309ed106e3b843c9d2b9a8b4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:40 +0600 Subject: [PATCH 019/509] network: use service DNS for companion services --- docker/compose/companion.yaml | 18 ++++++------------ 1 file changed, 6 insertions(+), 12 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 80667beb..2dca19b0 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -37,12 +37,9 @@ services: - ../../logs:/global/log:ro - /var/run/docker.sock:/var/run/docker.sock networks: - frontend: - ipv4_address: 172.28.0.13 - backend: - ipv4_address: 172.29.0.10 - datastore: - ipv4_address: 172.30.0.10 + - frontend + - backend + - datastore runner: container_name: RUNNER @@ -71,8 +68,7 @@ services: depends_on: - server-tools networks: - backend: - ipv4_address: 172.29.0.11 + - backend mailpit: container_name: MAILPIT @@ -93,10 +89,8 @@ services: depends_on: - server-tools networks: - frontend: - ipv4_address: 172.28.0.12 - backend: - ipv4_address: 172.29.0.12 + - frontend + - backend healthcheck: test: [ "CMD", "wget", "-qO-", "http://127.0.0.1:8025/" ] interval: 10s From bdfe21efdb8d9ae79f93125d4180734bd4a4c44c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:42 +0600 Subject: [PATCH 020/509] network: use dynamic HTTP service networking --- docker/compose/http.yaml | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index f71a6012..ad88961f 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -16,11 +16,8 @@ services: - lds_ssl_roots:/etc/share/rootCA:ro - lds_fpm_sock:/run/php-fpm:ro - ../../logs/nginx:/var/log/nginx - extra_hosts: - - "host.docker.internal:host-gateway" networks: - frontend: - ipv4_address: 172.28.0.10 + - frontend depends_on: - server-tools @@ -28,7 +25,7 @@ services: container_name: APACHE hostname: apache image: ${LDS_APACHE_IMAGE:-infocyph/apache:0.4.2} - restart: always + restart: unless-stopped environment: - TZ=${TZ:-} volumes: @@ -41,5 +38,4 @@ services: depends_on: - nginx networks: - frontend: - ipv4_address: 172.28.0.11 + - frontend From 17e0bcd017eb48fcd1dff5568183e1c04679f8da Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:45 +0600 Subject: [PATCH 021/509] network: remove fixed database addresses --- docker/compose/db.yaml | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index 9af78c1d..1e730fa2 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -21,9 +21,6 @@ services: interval: 30s timeout: 10s retries: 3 - networks: - datastore: - ipv4_address: 172.30.0.100 postgres: <<: *db-service @@ -56,9 +53,6 @@ services: interval: 30s timeout: 10s retries: 5 - networks: - datastore: - ipv4_address: 172.30.0.101 mysql: <<: *db-service @@ -80,9 +74,6 @@ services: interval: 30s timeout: 10s retries: 3 - networks: - datastore: - ipv4_address: 172.30.0.102 mongodb: <<: *db-service @@ -106,9 +97,6 @@ services: interval: 30s timeout: 10s retries: 3 - networks: - datastore: - ipv4_address: 172.30.0.103 mariadb: <<: *db-service @@ -130,9 +118,6 @@ services: interval: 30s timeout: 10s retries: 3 - networks: - datastore: - ipv4_address: 172.30.0.104 elasticsearch: <<: *db-service @@ -154,6 +139,3 @@ services: interval: 30s timeout: 10s retries: 3 - networks: - datastore: - ipv4_address: 172.30.0.105 From 8ad8e9fce2eb15af4eb7d5b89c077b69d4def638 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:47 +0600 Subject: [PATCH 022/509] network: remove fixed database client addresses --- docker/compose/db-client.yaml | 23 +---------------------- 1 file changed, 1 insertion(+), 22 deletions(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index 13764616..99ab89df 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -18,11 +18,6 @@ services: volumes: - lds_ri:/data - ../../logs/redis-insight:/var/log/redis-insight - networks: - datastore: - ipv4_address: 172.30.0.150 - frontend: - ipv4_address: 172.28.0.150 cloudbeaver: <<: *db-client-service @@ -40,11 +35,6 @@ services: interval: 30s timeout: 10s retries: 3 - networks: - datastore: - ipv4_address: 172.30.0.151 - frontend: - ipv4_address: 172.28.0.151 mongo-express: <<: *db-client-service @@ -62,11 +52,6 @@ services: - ME_CONFIG_MONGODB_URL=mongodb://${MONGODB_ROOT_USERNAME:-root}:${MONGODB_ROOT_PASSWORD:-12345}@mongodb:${MONGODB_PORT:-27017}/admin volumes: - ../../logs/mongo-express:/var/log/mongo-express - networks: - datastore: - ipv4_address: 172.30.0.152 - frontend: - ipv4_address: 172.28.0.152 kibana: <<: *db-client-service @@ -82,11 +67,6 @@ services: volumes: - lds_kb:/usr/share/kibana/data - ../../logs/kibana:/usr/share/kibana/logs - networks: - datastore: - ipv4_address: 172.30.0.153 - frontend: - ipv4_address: 172.28.0.153 filebeat: image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.3.0} @@ -102,5 +82,4 @@ services: - ../../logs:/global/log:ro command: ["--strict.perms=false"] networks: - datastore: - ipv4_address: 172.30.0.154 + - datastore From aeed10447b1ad8773460e33a98e8b9e86108d2c6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:20:49 +0600 Subject: [PATCH 023/509] network: allow postgres clients on attached networks --- docker/conf/pg_hba.conf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/conf/pg_hba.conf b/docker/conf/pg_hba.conf index 5534f88c..01d95332 100644 --- a/docker/conf/pg_hba.conf +++ b/docker/conf/pg_hba.conf @@ -11,8 +11,8 @@ host all all 127.0.0.1/32 scram-sha-256 # 3. IPv6 loopback host all all ::1/128 scram-sha-256 -# 4. Docker “datastore” network -host all all 172.30.0.0/24 scram-sha-256 +# 4. Docker-attached networks (dynamic bridge subnet) +host all all samenet scram-sha-256 # 5. (Optional) pgAdmin or external tools # If you really need to allow any host, you can—but it’s safer to lock this From 3ff709ca419e6a2e1f4835ca32898a131f3a54b8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:21:08 +0600 Subject: [PATCH 024/509] test: enforce dynamic network contracts --- tests/networking-contract.sh | 37 ++++++++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 10 deletions(-) diff --git a/tests/networking-contract.sh b/tests/networking-contract.sh index 136945bd..d69e606f 100755 --- a/tests/networking-contract.sh +++ b/tests/networking-contract.sh @@ -10,28 +10,45 @@ db="$ROOT/docker/compose/db.yaml" clients="$ROOT/docker/compose/db-client.yaml" companion="$ROOT/docker/compose/companion.yaml" http="$ROOT/docker/compose/http.yaml" +pghba="$ROOT/docker/conf/pg_hba.conf" for network in frontend backend datastore; do grep -Eq "^ ${network}:" "$main" || fail "missing logical network: $network" done pass "logical network names" +for runtime_file in "$ROOT"/docker/compose/*.yaml "$pghba"; do + if grep -Eq 'ipv4_address:|172\.28\.0\.|172\.29\.0\.|172\.30\.0\.' "$runtime_file"; then + fail "static LocalDevStack network dependency remains in $runtime_file" + fi +done +pass "fixed LocalDevStack addresses removed" + +schema_count="$(grep -c 'com.infocyph.network-schema: "dynamic-v1"' "$main")" +[[ "$schema_count" -eq 3 ]] || fail "expected dynamic-v1 schema label on all three networks" +if grep -q '^[[:space:]]*ipam:' "$main"; then + fail "main compose must not define fixed IPAM" +fi +pass "dynamic network schema" + assert_file_contains "$clients" '@mongodb:' assert_file_contains "$clients" 'ELASTICSEARCH_HOSTS=http://elasticsearch:9200' assert_file_contains "$companion" 'hostname: runner' assert_file_contains "$http" 'hostname: nginx' assert_file_contains "$db" 'hostname: postgres' +assert_file_contains "$pghba" 'samenet' pass "service-name/Docker-DNS contracts are present" -static_count="$( - grep -Rhs 'ipv4_address:' "$ROOT/docker/compose" | wc -l | tr -d ' ' -)" -printf 'INFO: baseline contains %s explicit ipv4_address declarations; Batch 3 removes them.\n' "$static_count" - -if [[ "$static_count" -gt 0 ]]; then - grep -Fq '172.28.0.0/24' "$main" || fail "baseline frontend subnet changed unexpectedly" - grep -Fq '172.29.0.0/24' "$main" || fail "baseline backend subnet changed unexpectedly" - grep -Fq '172.30.0.0/24' "$main" || fail "baseline datastore subnet changed unexpectedly" +if grep -Fq 'host.docker.internal:host-gateway' "$http"; then + fail "unused host-gateway mapping should not remain in the HTTP layer" fi +assert_file_contains "$http" 'restart: unless-stopped' +pass "HTTP network/restart cleanup" -pass "static-network baseline characterized" +assert_file_contains "$ROOT/lds" 'migrate_legacy_networks()' +assert_file_contains "$ROOT/lds" 'docker_compose down --remove-orphans' +assert_file_contains "$ROOT/lds" 'com.infocyph.network-schema' +assert_file_contains "$ROOT/lds" 'cmd_vpn_fix()' +assert_file_contains "$ROOT/lds" 'vpn-fix) cmd_vpn_fix "$@"' +assert_file_contains "$ROOT/lds" 'migrate_legacy_networks' +pass "safe legacy-network migration and vpn-fix deprecation" From fbbff75dadad55df842dc97a3ae0f528281ef34e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:21:11 +0600 Subject: [PATCH 025/509] test: reject fixed addresses in resolved compose --- tests/compose-contract.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index abe2ac9f..e1971a26 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -69,7 +69,13 @@ assert_contains "$resolved" "image: infocyph/tools:0.23.2" assert_contains "$resolved" "image: infocyph/runner:0.5" assert_contains "$resolved" "image: infocyph/nginx:0.4.1" assert_contains "$resolved" "image: infocyph/apache:0.4.2" -pass "release compatibility defaults resolve" +if grep -Eq 'ipv4_address:|172\\.28\\.0\\.|172\\.29\\.0\\.|172\\.30\\.0\\.' <<<"$resolved"; then + fail "resolved Compose config still contains fixed LocalDevStack addresses" +fi +assert_contains "$resolved" "name: Frontend" +assert_contains "$resolved" "name: Backend" +assert_contains "$resolved" "name: DataStore" +pass "release compatibility defaults and dynamic networks resolve" printf '%s\n' 'LDS_TOOLS_IMAGE=example.invalid/tools:user-override' >>"$user_env" user_override="$("${compose[@]}" config)" From c048bf72679bbb44cb085f7d81e3f48b61f05c13 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:22:06 +0600 Subject: [PATCH 026/509] network: scope schema labels to bridge networks --- docker/compose/main.yaml | 19 ------------------- 1 file changed, 19 deletions(-) diff --git a/docker/compose/main.yaml b/docker/compose/main.yaml index 1403f0d8..73adc0e4 100644 --- a/docker/compose/main.yaml +++ b/docker/compose/main.yaml @@ -30,133 +30,114 @@ volumes: labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "PHP-FPM Sockets" lds_fpm_pools: name: FPMPools labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "PHP-FPM Pools" lds_composer_global: name: ComposerGlobal labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Composer Global Home" lds_ssl_keys: name: SSLKeys labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "SSL Keys" lds_ssl_roots: name: SSLRootCA labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "SSL Root CA" lds_git_config: name: GitConfig labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Git Config" lds_nginx_host: name: NginxHosts labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Nginx Host Configs" lds_apache_host: name: ApacheHosts labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Apache Host Configs" lds_mail: name: EmailStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Email Store" lds_redis: name: RedisStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Redis Data" lds_ri: name: RedisInsightStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Redis Insight Data" lds_pg: name: PostgresStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "PostgreSQL Data" lds_my: name: MySQLStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "MySQL Data" lds_maria: name: MariaDBStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "MariaDB Data" lds_mongo: name: MongoDBStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "MongoDB Data" lds_es: name: ElasticSearchStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "ElasticSearch Data" lds_kb: name: KibanaStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Kibana Data" lds_cb: name: CloudBeaverStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "CloudBeaver Data" lds_fbeat: name: FilebeatStore labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.network-schema: "dynamic-v1" com.infocyph.purpose: "Filebeat Data" include: - docker/compose/companion.yaml From 416c7a283cc9b61f46efbc0d2bc2b1fb31e5002c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:25:54 +0600 Subject: [PATCH 027/509] ai: add persistent LLM service state --- docker/compose/main.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docker/compose/main.yaml b/docker/compose/main.yaml index 73adc0e4..83ec4765 100644 --- a/docker/compose/main.yaml +++ b/docker/compose/main.yaml @@ -139,8 +139,15 @@ volumes: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" com.infocyph.purpose: "Filebeat Data" + lds_llm: + name: LLMModels + labels: + com.infocyph.lds: "1" + com.infocyph.stack: "LocalDevStack" + com.infocyph.purpose: "LLM Model Store" include: - docker/compose/companion.yaml - docker/compose/db.yaml - docker/compose/db-client.yaml - docker/compose/http.yaml + - docker/compose/ai.yaml From 3f7b9a27b8017661b56abd981b9abf832861e212 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:25:56 +0600 Subject: [PATCH 028/509] ai: configure Tools provider contract --- docker/compose/companion.yaml | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 2dca19b0..e9d53160 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -16,6 +16,17 @@ services: - SOPS_CFG_DIR=/etc/share/sops/config - SOPS_GLOBAL_DIR=/etc/share/sops/global - SOPS_REPO_DIR=/etc/share/vhosts/sops + - LDS_AI_ENABLED=${LDS_AI_ENABLED:-auto} + - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-ollama} + - LDS_AI_URL=${LDS_AI_URL:-http://llm-sm:11434} + - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} + - LDS_AI_CONNECT_TIMEOUT + - LDS_AI_PREFLIGHT_TIMEOUT + - LDS_AI_TIMEOUT + - LDS_AI_AVAILABILITY_TTL + - LDS_AI_MAX_CONTEXT_BYTES + - LDS_AI_MAX_REQUEST_BYTES + - LDS_AI_MAX_RESPONSE_BYTES volumes: - "${PROJECT_DIR:-./../../../application}:/app" - lds_ssl_roots:/etc/share/rootCA From 28c7550a7548c3c344011747933718518fc1d65b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:25:59 +0600 Subject: [PATCH 029/509] ai: add optional llm-sm provider --- docker/compose/ai.yaml | 10 ++++++++++ 1 file changed, 10 insertions(+) create mode 100644 docker/compose/ai.yaml diff --git a/docker/compose/ai.yaml b/docker/compose/ai.yaml new file mode 100644 index 00000000..17b540ff --- /dev/null +++ b/docker/compose/ai.yaml @@ -0,0 +1,10 @@ +services: + llm-sm: + image: ${LDS_LLM_IMAGE:-infocyph/llm-sm:0.03} + restart: unless-stopped + profiles: [ai] + volumes: + - lds_llm:/root/.ollama + networks: + - frontend + - backend From ba32ce5badc7d870c86fb76d48a53e8cdc562e73 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:26:02 +0600 Subject: [PATCH 030/509] ai: add NVIDIA runtime override --- docker/compose/ai-nvidia.yaml | 3 +++ 1 file changed, 3 insertions(+) create mode 100644 docker/compose/ai-nvidia.yaml diff --git a/docker/compose/ai-nvidia.yaml b/docker/compose/ai-nvidia.yaml new file mode 100644 index 00000000..addc4687 --- /dev/null +++ b/docker/compose/ai-nvidia.yaml @@ -0,0 +1,3 @@ +services: + llm-sm: + gpus: all From 7e481985946e2324353a513af10f8e309b501471 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:26:04 +0600 Subject: [PATCH 031/509] ai: add AMD runtime override --- docker/compose/ai-amd.yaml | 6 ++++++ 1 file changed, 6 insertions(+) create mode 100644 docker/compose/ai-amd.yaml diff --git a/docker/compose/ai-amd.yaml b/docker/compose/ai-amd.yaml new file mode 100644 index 00000000..bc1c79c6 --- /dev/null +++ b/docker/compose/ai-amd.yaml @@ -0,0 +1,6 @@ +services: + llm-sm: + image: ${LDS_LLM_AMD_IMAGE:-infocyph/llm-sm:amd-0.03} + devices: + - /dev/kfd:/dev/kfd + - /dev/dri:/dev/dri From 26cd71a7fba44344c361715ad6af8b0f6f6186f3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:26:06 +0600 Subject: [PATCH 032/509] ai: add loopback host API override --- docker/compose/ai-host-port.yaml | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 docker/compose/ai-host-port.yaml diff --git a/docker/compose/ai-host-port.yaml b/docker/compose/ai-host-port.yaml new file mode 100644 index 00000000..46fc02c5 --- /dev/null +++ b/docker/compose/ai-host-port.yaml @@ -0,0 +1,4 @@ +services: + llm-sm: + ports: + - "127.0.0.1:${LLM_SM_PORT:-11434}:11434" From 932652c0bee3d09b2e48880be999bd421d3673df Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:26:56 +0600 Subject: [PATCH 033/509] ai: add consumer and provider CLI routing --- lds | 3402 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 3402 insertions(+) diff --git a/lds b/lds index 806a3442..c683ea6c 100755 --- a/lds +++ b/lds @@ -307,6 +307,3408 @@ ENV_RELEASE="$CFG/release.env" COMPOSE_FILE="$CFG/compose/main.yaml" EXTRAS_DIR="$DIR/configuration/compose" +# Read a single dotenv value without sourcing/evaluating the file. +# This is intentionally limited to literal KEY=VALUE state used by LDS itself. +dotenv_value() { + local file="${1:-}" key="${2:-}" line value first last + [[ -r "$file" && -n "$key" ]] || return 1 + + line="$(grep -E "^${key}=" "$file" 2>/dev/null | tail -n1 || true)" + [[ -n "$line" ]] || return 1 + + value="${line#*=}" + value="${value% + export WORKDIR_WIN="${2:-}" + shift 2 +fi + +COLOR() { printf '[%sm' "$1"; } +############################################################################### +# Colors + UI (higher contrast; aligned with mkhost.sh) +############################################################################### +# Color control: +# - If stdout isn't a TTY, disable colors by default. +# - If NO_COLOR is set, disable colors. +# - Set LDS_FORCE_COLOR=1 to force colors. +_is_tty() { [[ -t 1 ]]; } + +_use_color=1 +if [[ "${LDS_FORCE_COLOR:-0}" != "1" ]]; then + if [[ -n "${NO_COLOR:-}" ]] || ! _is_tty; then + _use_color=0 + fi +fi + +if ((_use_color)); then + BOLD=$'\033[1m' + DIM=$'\033[2m' + RED=$'\033[1;31m' + GREEN=$'\033[1;32m' + CYAN=$'\033[1;36m' + YELLOW=$'\033[1;33m' + BLUE=$'\033[1;34m' + MAGENTA=$'\033[1;35m' + NC=$'\033[0m' +else + BOLD='' DIM='' RED='' GREEN='' CYAN='' YELLOW='' BLUE='' MAGENTA='' NC='' +fi + +# Output control: +# - --quiet suppresses non-error output +QUIET=0 + +say() { ((QUIET)) || printf '%b\n' "$*"; } +ok() { ((QUIET)) || printf '%b\n' "${GREEN}$*${NC}"; } +warn() { ((QUIET)) || printf '%b\n' "${YELLOW}$*${NC}"; } +err() { printf '%b\n' "${RED}$*${NC}" >&2; } + +# Default behavior: QUIET +VERBOSE=0 + +#─────────────────────────────────────────────────────────────────────────────── +# 0a. GLOBAL ERROR HANDLER +#─────────────────────────────────────────────────────────────────────────────── +command_not_found_handle() { + local unknown="$1" + [[ $unknown == cmd_* ]] && unknown=${unknown#cmd_} + printf "\n%bError:%b Unknown command '%b'\n\n" "$RED" "$NC" "$unknown" + cmd_help + exit 1 +} + +trap 'on_error $? $LINENO "$BASH_COMMAND"' ERR +on_error() { + local code="$1" line="$2" cmd="$3" + local fn="${FUNCNAME[1]:-main}" + local src="${BASH_SOURCE[1]:-$0}" + + printf "\n%bError:%b %s:%s in %s() (exit %d)\n" "$RED" "$NC" "$src" "$line" "$fn" "$code" >&2 + printf "%bCommand:%b %s\n" "$RED" "$NC" "$cmd" >&2 + + if ((VERBOSE)); then + printf "%bStack:%b\n" "$DIM" "$NC" >&2 + local i=1 + while caller "$i" >/dev/null 2>&1; do + caller "$i" >&2 + i=$((i + 1)) + done + fi + printf "\n" >&2 + exit "$code" +} + +############################################################################### +# 1a. DOCKER COMPOSE WRAPPER +############################################################################### + +# ── compose extras (docker/extras/*.y{a,}ml) ──────────────────────────────── +__EXTRAS_LOADED=0 +declare -a __EXTRA_FILES=() + +load_extras() { + # Set LDS_EXTRAS_RELOAD=1 (or global --reload-extras) to re-scan templates every call. + if [[ "${LDS_EXTRAS_RELOAD:-0}" == "1" ]]; then + __EXTRAS_LOADED=0 + fi + + ((__EXTRAS_LOADED)) && return 0 + __EXTRAS_LOADED=1 + + [[ -d "$EXTRAS_DIR" ]] || return 0 + + mapfile -t __EXTRA_FILES < <( + find "$EXTRAS_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -print 2>/dev/null | sort | sed '/^[[:space:]]*$/d' + ) +} + +docker_compose() { + load_extras + # Create required runtime files only when docker/compose operations are invoked. + ((EUID == 0)) || ensure_files_exist "/docker/.env" "/configuration/php/php.ini" "/.env" + if [[ -z "${__LDS_DC_BIN:-}" ]]; then + if docker compose version >/dev/null 2>&1; then + __LDS_DC_BIN=(docker compose) + else + __LDS_DC_BIN=(docker-compose) + fi + fi + + [[ -r "$ENV_RELEASE" ]] || die "Missing release compatibility manifest: $ENV_RELEASE" + + # Release defaults are loaded first; user docker/.env overrides them. + # Shell variables remain higher-precedence Compose interpolation inputs. + local -a env_files=(--env-file "$ENV_RELEASE") + [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") + + # Static LocalDevStack overrides are selected from controlled state. + local ai_runtime ai_host_port + local -a static_f=() + ai_runtime="$(compose_control_value LDS_AI_RUNTIME cpu)" + case "${ai_runtime,,}" in + "" | cpu) ;; + nvidia) static_f+=(-f "$CFG/compose/ai-nvidia.yaml") ;; + amd) static_f+=(-f "$CFG/compose/ai-amd.yaml") ;; + *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd)" ;; + esac + + ai_host_port="$(compose_control_value LDS_LLM_HOST_PORT 0)" + case "${ai_host_port,,}" in + "" | 0 | false | no | off) ;; + 1 | true | yes | on) static_f+=(-f "$CFG/compose/ai-host-port.yaml") ;; + *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; + esac + + # Build -f list (stable order; product overrides before user extras). + local -a extra_f=() f + for f in "${__EXTRA_FILES[@]:-}"; do + [[ -f "$f" ]] || continue + case "$f" in + *.yml | *.yaml) extra_f+=(-f "$f") ;; + esac + done + + local host_os="${HOST_OS:-$(detect_host_os)}" + + HOST_OS="$host_os" "${__LDS_DC_BIN[@]}" \ + --project-directory "$DIR" \ + -f "$COMPOSE_FILE" \ + "${static_f[@]}" \ + "${extra_f[@]}" \ + "${env_files[@]}" \ + "$@" +} + +# helper: print project name +lds_project() { printf '%s' "${__LDS_PROJECT:-$(basename -- "$DIR")}"; } + +# (QUIET by default) ──────────────────────────────── +# Centralize quiet/verbose handling for compose subcommands. +# Usage: dc_cmd [args...] +dc_cmd() { + local sub="${1:-}" + shift || true + + local -a quiet=() + if ((VERBOSE == 0)); then + case "$sub" in + up) quiet+=(--quiet-pull) ;; + pull) quiet+=(-q) ;; + build) quiet+=(--quiet) ;; + esac + fi + + docker_compose "$sub" "${quiet[@]}" "$@" +} + +dc_up() { dc_cmd up "$@"; } +dc_pull() { dc_cmd pull "$@"; } +dc_build() { dc_cmd build "$@"; } + +# helper for our own minimal logging (still shows in quiet mode) +logv() { ((VERBOSE)) && printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2 || true; } +logq() { printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2; } + +############################################################################### +# 1b. PROMPTS + DOTENV HELPERS +############################################################################### + +# Unified prompt helper (used by env_init + profiles) +tty_readline() { + # Robust prompt/read across Linux/macOS/WSL/Windows Git Bash. + # Prefer stdin when it is a TTY (normal interactive use). If stdin is not a TTY, + # fall back to /dev/tty when available. + local __var_name="$1" __prompt="$2" __line + + if [[ -t 0 ]]; then + # Interactive: show prompt on stderr (so it is never swallowed) and read stdin. + printf '%s' "$__prompt" >&2 + IFS= read -r __line || return 1 + elif [[ -r /dev/tty ]]; then + # Non-interactive stdin (piped) but we still have a controlling terminal. + printf '%s' "$__prompt" >/dev/tty + IFS= read -r __line keep as-is + if [[ "$v" =~ ^\".*\"$ || "$v" =~ ^\'.*\'$ ]]; then + printf '%s' "$v" + return 0 + fi + + # Leading/trailing whitespace or any internal whitespace or # or quotes => quote + if [[ "$v" =~ ^[[:space:]] || "$v" =~ [[:space:]]$ || "$v" == *$'\t'* || "$v" == *" "* || "$v" == *"#"* || "$v" == *"\""* ]]; then + env_quote "$v" + return 0 + fi + + printf '%s' "$v" +} + +# Escape replacement for sed (delimiter '|') +sed_escape_repl() { + local s=${1-} + s=${s//\\/\\\\} + s=${s//&/\\&} + s=${s//|/\\|} + printf '%s' "$s" +} + +update_env() { + local file=$1 var=$2 val=${3-} + mkdir -p "$(dirname "$file")" + [[ -f "$file" ]] || { + printf "%bFile '%s' not found. Creating one.%b\n" "$YELLOW" "$file" "$NC" + : >"$file" + } + + # Apply quoting only when needed (spaces etc.) + val="$(env_quote_if_needed "$val")" + + # Sed-safe replacement + local val_sed + val_sed="$(sed_escape_repl "$val")" + + var=$(echo "$var" | sed 's/[]\/$*.^|[]/\\&/g') + if grep -qE "^[# ]*$var=" "$file" 2>/dev/null; then + sed -Ei "s|^[# ]*($var)=.*|\1=$val_sed|" "$file" + else + printf "%s=%s\n" "$var" "$val" >>"$file" + fi +} + +############################################################################### +# 1c. HTTP / WEB SERVER HELPERS +############################################################################### + +http_reload() { + printf "%bReloading HTTP...%b" "$MAGENTA" "$NC" + docker ps -qf name=NGINX &>/dev/null && docker exec NGINX nginx -s reload &>/dev/null || true + docker ps -qf name=APACHE &>/dev/null && docker exec APACHE apachectl graceful &>/dev/null || true + printf "\r%bHTTP reloaded! %b\n" "$GREEN" "$NC" +} + +############################################################################### +# 2. INSTALL / PERMISSIONS (HOST) +############################################################################### +add_to_windows_path() { + [[ "$OSTYPE" =~ (msys|cygwin) ]] || return 0 + has_cmd cygpath || return 0 + + # Only add if lds.bat exists where we think it is + [[ -f "$DIR/lds.bat" ]] || return 0 + + local win_repo + win_repo="$(cygpath -w "$DIR")" + + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " + \$t = '$win_repo' + \$cur = [Environment]::GetEnvironmentVariable('Path','User') + if ([string]::IsNullOrWhiteSpace(\$cur)) { \$cur = '' } + + # Normalize (trim + case-insensitive compare) to avoid duplicates + \$parts = \$cur -split ';' | ForEach-Object { \$_.Trim() } | Where-Object { \$_ } + \$exists = \$false + foreach (\$p in \$parts) { if (\$p.ToLowerInvariant() -eq \$t.ToLowerInvariant()) { \$exists = \$true; break } } + + if (-not \$exists) { + \$new = (\$parts + \$t) -join ';' + [Environment]::SetEnvironmentVariable('Path', \$new, 'User') + } + " >/dev/null 2>&1 || true +} + +fix_perms() { + if [[ "$OSTYPE" =~ (msys|cygwin) ]]; then + add_to_windows_path + printf "%bWindows PATH configured.%b\n" "$GREEN" "$NC" + return 0 + fi + + ((EUID == 0)) || die "Please run with sudo." + + chmod 755 "$DIR" + chmod 2775 "$DIR/configuration" + find "$DIR/configuration" -type f ! -perm 664 -exec chmod 664 {} + + + chmod 755 "$DIR/docker" + find "$DIR/docker" -type f ! -perm 644 -exec chmod 644 {} + + + chmod -R 777 "$DIR/logs" + chown -R "$USER:docker" "$DIR/logs" + + chmod 755 "$DIR/bin" + find "$DIR/bin" -type f -exec chmod +x {} + + chmod +x "$DIR/lds" + + ln -fs "$DIR/lds" /usr/local/bin/lds + printf "%bPermissions assigned.%b\n" "$GREEN" "$NC" +} + +############################################################################### +# 3. DOMAIN / PROFILE INTEGRATION +############################################################################### +mkhost() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" mkhost "$@" +} +rmhost() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" rmhost "$@" +} + +setup_domain() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + mkhost --RESET + docker exec -it "$ctr" mkhost + local mk_state svr_prof + mk_state="$(mkhost --JSON || true)" + if has_tool jq; then + svr_prof="$(printf '%s' "$mk_state" | jq -r '.state.apache_active // empty' 2>/dev/null || true)" + else + svr_prof="$(printf '%s' "$mk_state" | tr -d '\r\n' | sed -n 's/.*"apache_active"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" + fi + [[ -n $svr_prof ]] && modify_profiles add "$svr_prof" + mkhost --RESET + cmd_reboot +} + +delete_domain() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + rmhost --RESET + + # interactive delete + docker exec -it "$ctr" rmhost "$@" + + local rm_state apache_cont + rm_state="$(rmhost --JSON || true)" + if has_tool jq; then + apache_cont="$(printf '%s' "$rm_state" | jq -r '.state.apache_delete // empty' 2>/dev/null || true)" + else + apache_cont="$(printf '%s' "$rm_state" | tr -d '\r\n' | sed -n 's/.*"apache_delete"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" + fi + [[ -n "$apache_cont" ]] && modify_profiles remove "$apache_cont" + + rmhost --RESET + cmd_reboot +} + +modify_profiles() { + local action=$1 + shift + local file=$ENV_DOCKER var=COMPOSE_PROFILES + local -a existing updated + + if [[ -r $file ]]; then + local line value + line=$(grep -E "^${var}=" "$file" | tail -n1 || true) + value=${line#*=} + IFS=',' read -r -a existing <<<"$value" + fi + + case $action in + add) + local p + for p; do + [[ -n $p && ! " ${existing[*]} " =~ " $p " ]] && updated+=("$p") + done + updated+=("${existing[@]}") + ;; + remove) + local old + for old in "${existing[@]}"; do + [[ ! " $* " =~ " $old " ]] && updated+=("$old") + done + ;; + *) die "modify_profiles: invalid action '$action'" ;; + esac + + update_env "$file" "$var" "$( + IFS=, + echo "${updated[*]}" + )" +} + +# ───────────────────────────────────────────────────────────────────────────── +# Profiles +# ───────────────────────────────────────────────────────────────────────────── + +############################################################################### +# 3a. PROFILES: DEFINITIONS + SETUP FLOW +############################################################################### + +declare -A SERVICES=( + [POSTGRESQL]="postgresql" + [MYSQL]="mysql" + [MARIADB]="mariadb" + [ELASTICSEARCH]="elasticsearch" + [MONGODB]="mongodb" + [REDIS]="redis" +) + +declare -a SERVICE_ORDER=(POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS) + +declare -A PROFILE_ENV=( + [elasticsearch]="ELASTICSEARCH_VERSION=9.3.0" + [mysql]="MYSQL_VERSION=latest MYSQL_ROOT_PASSWORD=12345 MYSQL_USER=infocyph MYSQL_PASSWORD=12345 MYSQL_DATABASE=localdb" + [mariadb]="MARIADB_VERSION=latest MARIADB_ROOT_PASSWORD=12345 MARIADB_USER=infocyph MARIADB_PASSWORD=12345 MARIADB_DATABASE=localdb" + [mongodb]="MONGODB_VERSION=latest MONGODB_ROOT_USERNAME=root MONGODB_ROOT_PASSWORD=12345" + [redis]="REDIS_VERSION=latest" + [postgresql]="POSTGRES_VERSION=latest POSTGRES_USER=postgres POSTGRES_PASSWORD=postgres POSTGRES_DATABASE=postgres" +) + +declare -a PENDING_ENVS=() +declare -a PENDING_PROFILES=() + +queue_env() { PENDING_ENVS+=("$1"); } +queue_profile() { PENDING_PROFILES+=("$1"); } + +flush_envs() { + local env_file="$ENV_DOCKER" kv key val + for kv in "${PENDING_ENVS[@]}"; do + IFS='=' read -r key val <<<"$kv" + update_env "$env_file" "$key" "$val" + done +} + +flush_profiles() { + local profile + for profile in "${PENDING_PROFILES[@]}"; do + modify_profiles add "$profile" + done +} + +# ── setup menu (selection-first) ────────────────────────────────────────────── + +setup_menu_print() { + # Print menu to stderr to avoid stdout buffering in some Windows wrappers. + { + printf "\n%bSetup profiles%b (will replace previous configuration, if exists):\n\n" "$CYAN" "$NC" + local i=1 key slug + for key in "${SERVICE_ORDER[@]}"; do + slug="${SERVICES[$key]}" + printf " %2d) %-12s (%s)\n" "$i" "$key" "$slug" + i=$((i + 1)) + done + printf "\n a) ALL\n" + printf " n) NONE / Back\n\n" + } >&2 +} + +# Parse user selection into indices or ALL/NONE (prints one token per line) +setup_menu_parse() { + local input="${1//[[:space:]]/}" + [[ -n "$input" ]] || return 1 + input="${input//;/,}" + + echo "$input" | tr ',' '\n' | awk ' + BEGIN { ok=1 } + /^[0-9]+-[0-9]+$/ { + split($0,a,"-") + if (a[1] > a[2]) { t=a[1]; a[1]=a[2]; a[2]=t } + for (i=a[1]; i<=a[2]; i++) print i + next + } + /^[0-9]+$/ { print $0; next } + /^[aA]$/ { print "ALL"; next } + /^[nN]$/ { print "NONE"; next } + { ok=0 } + END { if (!ok) exit 2 } + ' +} + +# Outputs: newline-separated service KEYS from SERVICE_ORDER (e.g. MYSQL, REDIS) +setup_choose_services() { + local ans parsed + while :; do + setup_menu_print + tty_readline ans "Select (e.g. 1,3,5 or 2-4 or a): " || return 1 + + if ! parsed="$(setup_menu_parse "$ans" 2>/dev/null)"; then + printf "%bInvalid selection.%b Try again.\n" "$YELLOW" "$NC" + continue + fi + + if grep -qx "NONE" <<<"$parsed"; then + return 1 + fi + + if grep -qx "ALL" <<<"$parsed"; then + printf "%s\n" "${SERVICE_ORDER[@]}" + return 0 + fi + + # Indices -> keys (de-dupe, preserve order) + local -A seen=() + local out=() + local idx key + while IFS= read -r idx; do + [[ "$idx" =~ ^[0-9]+$ ]] || continue + ((idx >= 1 && idx <= ${#SERVICE_ORDER[@]})) || continue + key="${SERVICE_ORDER[idx - 1]}" + [[ -n "${seen[$key]:-}" ]] && continue + seen[$key]=1 + out+=("$key") + done <<<"$parsed" + + if ((${#out[@]} == 0)); then + printf "%bNo valid items selected.%b\n" "$YELLOW" "$NC" + continue + fi + + printf "%s\n" "${out[@]}" + return 0 + done +} + +setup_service() { + local service="$1" + local profile="${SERVICES[$service]:-}" + [[ -n "$profile" ]] || die "Unknown service: $service" + + printf "\n%b→ %s%b\n" "$YELLOW" "$service" "$NC" + queue_profile "$profile" + + printf "%bEnter value(s) for %s:%b\n" "$BLUE" "$service" "$NC" + local pair key def val + for pair in ${PROFILE_ENV[$profile]}; do + IFS='=' read -r key def <<<"$pair" + val=$(read_default "$key" "$def") + queue_env "$key=$val" + done +} + +process_all() { + local selected + if ! selected="$(setup_choose_services)"; then + printf "\n%bSetup cancelled.%b\n" "$YELLOW" "$NC" + return 0 + fi + + printf "\n%bWill configure:%b\n" "$CYAN" "$NC" + while IFS= read -r svc; do + printf " - %s (%s)\n" "$svc" "${SERVICES[$svc]}" + done <<<"$selected" + echo + + local svc + while IFS= read -r svc; do + setup_service "$svc" + done <<<"$selected" + + flush_envs + flush_profiles + printf "\n%b✅ Selected services configured!%b\n" "$GREEN" "$NC" +} + +############################################################################### +# 5. ENVIRONMENT + CERT / CA +############################################################################### +detect_timezone() { + if has_cmd timedatectl; then + timedatectl show -p Timezone --value + elif [[ -n ${TZ-} ]]; then + printf '%s' "$TZ" + elif [[ -r /etc/timezone ]]; then + /dev/null | tr -d '\r' + else + date +%Z + fi +} + +env_init() { + local env_file="$ENV_DOCKER" + printf "%bBootstrapping environment defaults…%b\n" "$YELLOW" "$NC" + + local default_tz tz + default_tz="$(detect_timezone)" + tz="$(read_default "Timezone (TZ)" "$default_tz")" + + local default_git_name default_git_email git_name git_email + default_git_name="$(git config --global --get user.name 2>/dev/null || true)" + default_git_email="$(git config --global --get user.email 2>/dev/null || true)" + git_name="$(read_default "Git user.name (GIT_USER_NAME)" "$default_git_name")" + git_email="$(read_default "Git user.email (GIT_USER_EMAIL)" "$default_git_email")" + + # update_env now quotes automatically when needed + update_env "$env_file" "TZ" "$tz" + update_env "$env_file" "GIT_USER_NAME" "$git_name" + update_env "$env_file" "GIT_USER_EMAIL" "$git_email" + + printf "%bConfiguration saved!%b\n" "$GREEN" "$NC" +} + +# ───────────────────────────────────────────────────────────────────────────── +# Root CA helpers (cross-distro) +# ───────────────────────────────────────────────────────────────────────────── + +# Unique identity (avoid conflicts with other mkcert/dev CAs) +CA_BASENAME="localdevstack-rootca" +CA_NICK="LocalDevStack Root CA" + +detect_os_family() { + # Output: "id|like" + # Must never fail under set -e + if [[ "${OSTYPE:-}" =~ (msys|cygwin|win32) ]]; then + echo "windows|windows" + return 0 + fi + + local id like + id="unknown" + like="unknown" + + if [[ -r /etc/os-release ]]; then + # shellcheck disable=SC1091 + . /etc/os-release || true + id="${ID:-unknown}" + like="${ID_LIKE:-unknown}" + elif has_cmd uname; then + # fallback for macOS / other unix + case "$(uname -s 2>/dev/null || true)" in + Darwin) + id="macos" + like="darwin" + ;; + Linux) + id="linux" + like="linux" + ;; + esac + fi + + echo "$id|$like" +} + +# Decide destination path + update mechanism. Echo: family|dest|updater +ca_plan() { + local os_id os_like + IFS='|' read -r os_id os_like < <(detect_os_family) + + case " $os_id $os_like " in + *" debian "* | *" ubuntu "* | *" linuxmint "* | *" pop "* | *" raspbian "*) + printf "debian|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" + ;; + *" alpine "*) + printf "alpine|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" + ;; + *" fedora "* | *" rhel "* | *" redhat "* | *" centos "* | *" rocky "* | *" alma "* | *" amzn "* | *" amazon "* | *" sles "* | *" suse "*) + printf "rhel|/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt|update-ca-trust\n" + ;; + *" arch "* | *" manjaro "*) + printf "arch|/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt|trust\n" + ;; + *) + # best default: Debian-style location (works on many distros even if updater differs) + printf "fallback|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|\n" + ;; + esac +} + +is_windows_shell() { + [[ "${OSTYPE:-}" =~ (msys|cygwin) ]] || [[ -n "${WORKDIR_WIN:-}" ]] +} + +need_windows_tools() { + has_cmd cygpath || die "Windows certificate install needs 'cygpath' (Git Bash)." + has_cmd powershell.exe || die "Windows certificate install needs 'powershell.exe' on PATH." +} + +# Import CA into the invoking user's NSS DB (Chrome/Chromium/Firefox on many Linux setups) +install_ca_nss_user() { + local ca_file="$1" + has_cmd certutil || return 0 + + local user="${SUDO_USER:-}" + [[ -n "$user" && "$user" != "root" ]] || return 0 + + local home + home="$(getent passwd "$user" | cut -d: -f6)" + [[ -n "$home" && -d "$home" ]] || return 0 + + local nssdb="sql:${home}/.pki/nssdb" + sudo -u "$user" mkdir -p "${home}/.pki/nssdb" >/dev/null 2>&1 || true + + if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then + printf "%b✔ NSS already has CA%b (%s)\n" "$GREEN" "$NC" "$user" + return 0 + fi + + if sudo -u "$user" certutil -d "$nssdb" -A -n "$CA_NICK" -t "C,," -i "$ca_file" >/dev/null 2>&1; then + printf "%b✔ Imported CA into NSS%b (%s)\n" "$GREEN" "$NC" "$user" + else + printf "%bWARN%b: NSS import failed (certutil).\n" "$YELLOW" "$NC" >&2 + fi +} + +install_ca_windows() { + need_windows_tools + + local src_ca="$DIR/configuration/rootCA/rootCA.pem" + [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + + local win_ca + win_ca="$(cygpath -w "$src_ca")" + + printf "%bInstalling root CA into Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" + + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " + \$ErrorActionPreference = 'Stop' + \$path = '$win_ca' + \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) + \$cert.FriendlyName = '$CA_NICK' + + \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') + \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) + + \$exists = \$store.Certificates | Where-Object { \$_.Thumbprint -eq \$cert.Thumbprint } + if (-not \$exists) { \$store.Add(\$cert) } + + \$store.Close() + " >/dev/null 2>&1 || die "Windows certificate install failed (PowerShell import)." + + printf "%bRoot CA installed on Windows%b (CurrentUser\\Root) as %s\n" "$GREEN" "$NC" "$CA_NICK" + printf "%bNote:%b restart browsers if they still show trust errors.\n" "$YELLOW" "$NC" +} + +install_ca() { + if is_windows_shell; then + install_ca_windows + return 0 + fi + + local src_ca="$DIR/configuration/rootCA/rootCA.pem" + [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate install requires sudo" + [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + + local family dest updater os_id os_like + IFS='|' read -r os_id os_like < <(detect_os_family) + IFS='|' read -r family dest updater < <(ca_plan) + + printf "%bInstalling root CA…%b\n" "$CYAN" "$NC" + printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" + + install -d -m 755 "$(dirname "$dest")" + install -m 644 "$src_ca" "$dest" + printf "%b✔ Copied%b → %s\n" "$GREEN" "$NC" "$dest" + + case "$family" in + debian | alpine) + if has_cmd update-ca-certificates; then + printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" + if update-ca-certificates; then + printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" + printf "%bNote:%b If you see \"rehash: skipping ca-certificates.crt…\", that’s normal (it’s a bundle).\n" "$YELLOW" "$NC" + else + printf "%bWARN%b: update-ca-certificates failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bWARN%b: update-ca-certificates not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 + fi + + # Optional p11-kit sync: best-effort only (can be missing helper on minimal installs) + if has_cmd trust; then + printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" + if trust extract-compat >/dev/null 2>&1; then + printf "%b✔ p11-kit trust synced%b\n" "$GREEN" "$NC" + else + printf "%bWARN%b: trust extract-compat failed (helper missing on some installs). Skipping.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bINFO%b: 'trust' not found — skipping p11-kit sync.\n" "$YELLOW" "$NC" + fi + ;; + rhel) + if has_cmd update-ca-trust; then + printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" + if update-ca-trust extract; then + printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" + else + printf "%bWARN%b: update-ca-trust extract failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bWARN%b: update-ca-trust not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 + fi + ;; + arch) + if has_cmd trust; then + printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" + if trust extract-compat >/dev/null 2>&1; then + printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" + else + printf "%bWARN%b: trust extract-compat failed. CA is installed, but trust sync may be incomplete.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bWARN%b: 'trust' not found. CA is installed, but trust sync is unavailable.\n" "$YELLOW" "$NC" >&2 + fi + ;; + *) + printf "%bINFO%b: Unknown distro; CA copied to %s.\n" "$YELLOW" "$NC" "$dest" + printf "%bINFO%b: You may need to update trust store manually for your OS.\n" "$YELLOW" "$NC" + ;; + esac + + # Extra: ensure browsers that rely on NSS trust pick it up + install_ca_nss_user "$src_ca" + + printf "%bRoot CA installed%b → %s (%s)\n" "$GREEN" "$NC" "$dest" "$CA_NICK" +} + +uninstall_ca_windows() { + need_windows_tools + + local src_ca="$DIR/configuration/rootCA/rootCA.pem" + [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + + local win_ca + win_ca="$(cygpath -w "$src_ca")" + + printf "%bUninstalling root CA from Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" + + local removed + removed="$(powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " + \$ErrorActionPreference = 'Stop' + \$path = '$win_ca' + \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) + \$thumb = \$cert.Thumbprint + + \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') + \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) + + \$matches = @(\$store.Certificates | Where-Object { \$_.Thumbprint -eq \$thumb }) + foreach (\$c in \$matches) { \$store.Remove(\$c) } + + \$store.Close() + [string]\$matches.Count + " 2>/dev/null || true)" + + removed="${removed//[$'\r\n\t ']/}" + if [[ "${removed:-0}" =~ ^[0-9]+$ ]] && ((removed > 0)); then + printf "%bRoot CA uninstalled on Windows%b (removed %s cert)\n" "$GREEN" "$NC" "$removed" + else + printf "%bRoot CA already absent on Windows%b (no matching cert)\n" "$YELLOW" "$NC" + fi +} + +uninstall_ca_nss_user() { + has_cmd certutil || return 0 + local user="${SUDO_USER:-}" + [[ -n "$user" && "$user" != "root" ]] || return 0 + + local home + home="$(getent passwd "$user" | cut -d: -f6)" + [[ -n "$home" && -d "$home" ]] || return 0 + + local nssdb="sql:${home}/.pki/nssdb" + if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then + sudo -u "$user" certutil -d "$nssdb" -D -n "$CA_NICK" >/dev/null 2>&1 || true + printf "%b✔ Removed CA from NSS%b (%s)\n" "$GREEN" "$NC" "$user" + fi +} + +uninstall_ca() { + if is_windows_shell; then + uninstall_ca_windows + return 0 + fi + + [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate uninstall requires sudo" + + local all=0 + if [[ "${1:-}" == "--all" ]]; then + all=1 + shift + fi + + local family dest updater os_id os_like + IFS='|' read -r os_id os_like < <(detect_os_family) + IFS='|' read -r family dest updater < <(ca_plan) + + printf "%bUninstalling root CA…%b\n" "$CYAN" "$NC" + printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" + + local removed=0 + + if [[ -e "$dest" ]]; then + rm -f "$dest" + removed=$((removed + 1)) + printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$dest" + else + printf "%bINFO%b: CA file not found at %s (nothing to remove)\n" "$YELLOW" "$NC" "$dest" + fi + + if ((all)); then + printf "%bScanning all known CA anchor paths…%b\n" "$CYAN" "$NC" + local f + for f in \ + "/usr/local/share/ca-certificates/${CA_BASENAME}.crt" \ + "/usr/local/share/ca-certificates/${CA_BASENAME}.pem" \ + "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt" \ + "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.pem" \ + "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt" \ + "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.pem"; do + [[ "$f" == "$dest" ]] && continue + if [[ -e "$f" ]]; then + rm -f "$f" + removed=$((removed + 1)) + printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$f" + fi + done + fi + + case "$family" in + debian | alpine) + if has_cmd update-ca-certificates; then + printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" + update-ca-certificates || printf "%bWARN%b: update-ca-certificates failed.\n" "$YELLOW" "$NC" >&2 + else + printf "%bWARN%b: update-ca-certificates not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 + fi + + if has_cmd trust; then + printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" + trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed. Skipping.\n" "$YELLOW" "$NC" >&2 + fi + ;; + rhel) + if has_cmd update-ca-trust; then + printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" + update-ca-trust extract || printf "%bWARN%b: update-ca-trust extract failed.\n" "$YELLOW" "$NC" >&2 + else + printf "%bWARN%b: update-ca-trust not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 + fi + ;; + arch) + if has_cmd trust; then + printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" + trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed.\n" "$YELLOW" "$NC" >&2 + else + printf "%bWARN%b: 'trust' not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 + fi + ;; + *) + if has_cmd update-ca-certificates; then + printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" + update-ca-certificates || true + fi + if has_cmd update-ca-trust; then + printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" + update-ca-trust extract || true + fi + if has_cmd trust; then + printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" + trust extract-compat >/dev/null 2>&1 || true + fi + printf "%bINFO%b: Unknown distro; removed CA file(s) if present. Refresh trust store manually if needed.\n" "$YELLOW" "$NC" + ;; + esac + + uninstall_ca_nss_user + + if ((removed)); then + printf "%bRoot CA uninstalled%b (removed %d file(s))\n" "$GREEN" "$NC" "$removed" + else + printf "%bRoot CA already absent%b (no files removed)\n" "$YELLOW" "$NC" + fi +} + +add_required_env() { + update_env "$ENV_DOCKER" WORKING_DIR "$DIR" + ((EUID == 0)) && return 0 + update_env "$ENV_DOCKER" USER "$(id -un)" + update_env "$ENV_DOCKER" UID "$(id -u)" + update_env "$ENV_DOCKER" GID "$(id -g)" +} + +############################################################################### +# Compose helpers for rebuild (robust: supports service key OR container name) +############################################################################### +__COMPOSE_CFG_JSON="" +__COMPOSE_CFG_YAML="" +__COMPOSE_SVCS_LOADED=0 +declare -a __COMPOSE_SVCS=() + +compose_cfg_json() { + if [[ -z "${__COMPOSE_CFG_JSON}" ]]; then + __COMPOSE_CFG_JSON="$(docker_compose config --format json 2>/dev/null || true)" + fi + printf '%s' "${__COMPOSE_CFG_JSON}" +} + +compose_cfg_yaml() { + if [[ -z "${__COMPOSE_CFG_YAML}" ]]; then + __COMPOSE_CFG_YAML="$(docker_compose config 2>/dev/null || true)" + fi + printf '%s' "${__COMPOSE_CFG_YAML}" +} + +compose_services_load() { + ((__COMPOSE_SVCS_LOADED)) && return 0 + mapfile -t __COMPOSE_SVCS < <(docker_compose config --services 2>/dev/null || true) + __COMPOSE_SVCS_LOADED=1 +} + +compose_service_exists() { + local want="${1:-}" s + [[ -n "$want" ]] || return 1 + compose_services_load + for s in "${__COMPOSE_SVCS[@]}"; do + [[ "$s" == "$want" ]] && return 0 + done + return 1 +} + +resolve_service() { + local raw="${1:-}" norm svc + raw="${raw//[[:space:]]/}" + [[ -n "$raw" ]] || { + printf '' + return 0 + } + + compose_service_exists "$raw" && { + printf '%s' "$raw" + return 0 + } + + norm="$(normalize_service "$raw")" + compose_service_exists "$norm" && { + printf '%s' "$norm" + return 0 + } + + if docker inspect "$raw" >/dev/null 2>&1; then + svc="$(docker inspect -f '{{ index .Config.Labels "com.docker.compose.service" }}' "$raw" 2>/dev/null || true)" + if [[ -n "$svc" ]] && compose_service_exists "$svc"; then + printf '%s' "$svc" + return 0 + fi + fi + + printf '%s' "$norm" +} + +compose_has_build() { + local svc="$1" json + json="$(compose_cfg_json)" + if [[ -n "$json" ]]; then + if has_tool jq; then + jq -e --arg s "$svc" '.services[$s].build != null' >/dev/null <<<"$json" + return $? + fi + fi + + compose_cfg_yaml | awk -v s="$svc" ' + $1=="services:" {in_services=1; next} + in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} + in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 1} + in_svc && $0 ~ /^ build:/ {exit 0} + END {exit 1} + ' +} + +compose_image_for_service() { + local svc="$1" json + json="$(compose_cfg_json)" + if [[ -n "$json" ]]; then + if has_tool jq; then + jq -r --arg s "$svc" '.services[$s].image // empty' <<<"$json" + return 0 + fi + fi + + compose_cfg_yaml | awk -v s="$svc" ' + $1=="services:" {in_services=1; next} + in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} + in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 0} + in_svc && $0 ~ /^ image:/ { + sub(/^ image:[[:space:]]*/, "", $0) + print $0 + exit 0 + } + ' +} + +############################################################################### +# 6. STACK COMMANDS (CLI) +############################################################################### + +# One-time migration from the historical fixed 172.28/29/30 /24 networks. +# Only known LocalDevStack-owned networks are touched, and named volumes are +# never removed. New networks carry com.infocyph.network-schema=dynamic-v1. +declare -a __LDS_LEGACY_NETWORK_NAMES=(Frontend Backend DataStore) + +legacy_network_expected_subnet() { + case "\${1:-}" in + Frontend) printf '%s' '172.28.0.0/24' ;; + Backend) printf '%s' '172.29.0.0/24' ;; + DataStore) printf '%s' '172.30.0.0/24' ;; + *) return 1 ;; + esac +} + +migrate_legacy_networks() { + local network expected subnets stack_label project_label schema_label ctr ctr_project attachments + local -a legacy=() + + for network in "\${__LDS_LEGACY_NETWORK_NAMES[@]}"; do + docker network inspect "$network" >/dev/null 2>&1 || continue + + schema_label="$(docker network inspect -f '{{index .Labels "com.infocyph.network-schema"}}' "$network" 2>/dev/null || true)" + [[ "$schema_label" != "dynamic-v1" ]] || continue + + expected="$(legacy_network_expected_subnet "$network")" + subnets="$(docker network inspect -f '{{range .IPAM.Config}}{{println .Subnet}}{{end}}' "$network" 2>/dev/null || true)" + grep -Fxq "$expected" <<<"$subnets" || continue + + stack_label="$(docker network inspect -f '{{index .Labels "com.infocyph.stack"}}' "$network" 2>/dev/null || true)" + project_label="$(docker network inspect -f '{{index .Labels "com.docker.compose.project"}}' "$network" 2>/dev/null || true)" + if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "LocalDevStack" ]]; then + die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack. Remove or rename that network manually." + fi + + while IFS= read -r ctr; do + [[ -n "$ctr" ]] || continue + ctr_project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$ctr" 2>/dev/null || true)" + if [[ "$ctr_project" != "LocalDevStack" ]]; then + die "Refusing to migrate '$network': container '$ctr' is not owned by the LocalDevStack Compose project." + fi + done < <(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true) + + legacy+=("$network") + done + + (("\${#legacy[@]}" > 0)) || return 0 + + warn "Legacy fixed LocalDevStack network(s) detected: \${legacy[*]}" + warn "Recreating stack networks dynamically; named volumes and persisted data are preserved." + + # Stop/remove only LocalDevStack Compose containers and networks. Never use -v. + docker_compose down --remove-orphans + + for network in "\${legacy[@]}"; do + docker network inspect "$network" >/dev/null 2>&1 || continue + attachments="$(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true)" + [[ -z "$attachments" ]] || + die "Cannot remove legacy network '$network': attached container(s) remain: $(tr '\n' ' ' <<<"$attachments")" + docker network rm "$network" >/dev/null + done + + ok "Legacy fixed networks removed; Compose will recreate dynamic bridge networks." +} + +cmd_vpn_fix() { + warn "vpn-fix is deprecated: LocalDevStack no longer owns fixed Docker subnets." + warn "If a VPN conflict remains after dynamic-network migration, diagnose the VPN/Docker route directly." +} + +cmd_up() { + migrate_legacy_networks + dc_up "$@" +} + +cmd_start() { + migrate_legacy_networks + dc_up -d "$@" + http_reload +} + +cmd_stop() { docker_compose down; } + +cmd_down() { + # Safety rails: + # lds down --volumes requires --yes + local yes=0 vols=0 + local -a args=() + while [[ "${1:-}" ]]; do + case "$1" in + --yes | -y) + yes=1 + shift + ;; + --volumes | -v) + vols=1 + args+=("--volumes") + shift + ;; + --remove-orphans) + args+=("--remove-orphans") + shift + ;; + *) + args+=("$1") + shift + ;; + esac + done + if ((vols)) && ((yes == 0)); then + die "Refusing: down --volumes requires --yes" + fi + docker_compose down "${args[@]}" +} + +cmd_restart() { + cmd_stop + cmd_start +} +cmd_reboot() { cmd_restart; } + +# ───────────────────────────────────────────────────────────────────────────── +# 6a. STATUS / PS / STATS +# ───────────────────────────────────────────────────────────────────────────── +cmd_ps() { + if (($#)); then + docker_compose ps "$@" + else + docker_compose ps + fi +} + +cmd_status() { + local ctr project + project="$(lds_project)" + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container not found or not running for project: $project" + + local -a flags=() + [[ -t 1 ]] && flags+=(-t) + docker exec "${flags[@]}" "$ctr" status "$@" +} +# ───────────────────────────────────────────────────────────────────────────── +# 6b. LOGS / OPEN +# ───────────────────────────────────────────────────────────────────────────── +cmd_logs() { + local svc="" follow=0 since="" grep_pat="" + while [[ "${1:-}" ]]; do + case "$1" in + -f | --follow) + follow=1 + shift + ;; + --since) + since="${2:-}" + shift 2 + ;; + --grep) + grep_pat="${2:-}" + shift 2 + ;; + *) + svc="${1:-}" + shift + ;; + esac + done + + local -a args=() + ((follow)) && args+=("-f") + [[ -n "$since" ]] && args+=("--since" "$since") + + if [[ -n "$svc" ]]; then + local s + s="$(resolve_service "$svc" || true)" + [[ -n "$s" ]] || die "Unknown service: $svc" + if [[ -n "$grep_pat" ]]; then + docker_compose logs "${args[@]}" "$s" 2>&1 | text_grep "$grep_pat" + else + docker_compose logs "${args[@]}" "$s" + fi + else + if [[ -n "$grep_pat" ]]; then + docker_compose logs "${args[@]}" 2>&1 | text_grep "$grep_pat" + else + docker_compose logs "${args[@]}" + fi + fi +} + +cmd_open() { + local target="${1:-}" + [[ -n "$target" ]] || die "open " + local url="" + case "${target,,}" in + mail | mailpit | webmail) url="https://webmail.localhost" ;; + db | cloudbeaver) url="https://db.localhost" ;; + redis | redisinsight | redis-insight | rds) url="http://ri.localhost" ;; + mongo | me | mongoexpress | mongo-express) url="http://me.localhost" ;; + kibana | kbn) url="http://kibana.localhost" ;; + *) + url="https://${target}" + ;; + esac + open_url "$url" +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6c. PROFILES +# ───────────────────────────────────────────────────────────────────────────── +_known_profile() { + local p="${1:-}" + [[ -n "$p" ]] || return 1 + + # Prefer compose-config JSON for exact profile membership. + local json + json="$(compose_cfg_json)" + if [[ -n "$json" ]] && has_tool jq; then + printf '%s' "$json" | jq -e --arg p "$p" ' + [ .services[]? | (.profiles // [])[] ] | index($p) != null + ' >/dev/null 2>&1 + return $? + fi + + # Fallback: text scan when jq/json path is unavailable. + local f + for f in "$COMPOSE_FILE" "${__EXTRA_FILES[@]:-}"; do + [[ -r "$f" ]] || continue + grep -Fq -- "$p" "$f" && return 0 + done + return 1 +} + +cmd_profiles() { + local action="${1:-list}" + shift || true + case "${action,,}" in + list | "") + local cur="" + [[ -r "$ENV_DOCKER" ]] && cur="$(grep -E '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1 | cut -d= -f2- | tr -d '\r' || true)" + printf "%bEnabled profiles:%b %s +" "$CYAN" "$NC" "${cur:-}" + printf "%bAvailable profiles:%b +" "$CYAN" "$NC" + printf ' - %s +' "${SERVICES[@]}" | LC_ALL=C sort -u + # warn if enabled profile has no mention in compose + if [[ -n "$cur" ]]; then + local p + IFS=',' read -r -a __ps <<<"$cur" + for p in "${__ps[@]}"; do + p="${p//[[:space:]]/}" + [[ -n "$p" ]] || continue + _known_profile "$p" || printf "%b[warn]%b enabled profile '%s' has no matching services in compose +" "$YELLOW" "$NC" "$p" + done + fi + ;; + add) + [[ $# -gt 0 ]] || die "profiles add " + for p in "$@"; do + modify_profiles add "$p" + done + ;; + remove | rm | del) + [[ $# -gt 0 ]] || die "profiles remove " + modify_profiles remove "$@" + ;; + *) + die "profiles " + ;; + esac +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6d. DIAG / SNIFF +# ───────────────────────────────────────────────────────────────────────────── +_tools_exec() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + # NOTE: pass a SINGLE command string; do not pass arrays here. + docker exec -i "$ctr" sh -lc "$*" +} + +_tools_exec_argv() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + local -a flags=(-i) + [[ -t 0 && -t 1 ]] && flags+=(-t) + docker exec "${flags[@]}" "$ctr" "$@" +} + +cmd_ai() { + local sub="${1:-status}" + shift || true + case "${sub,,}" in + status | provider) + _tools_exec_argv aiops provider "$@" + ;; + ask) + _tools_exec_argv askai "$@" + ;; + explain | troubleshoot | review | repo-review | graphify) + _tools_exec_argv aiops "${sub,,}" "$@" + ;; + *) + die "ai [args...]" + ;; + esac +} + +_llm_exec() { + local ctr + ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" + [[ -n "$ctr" ]] || die "llm-sm is not running. Enable the ai profile and start the stack first." + docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || + die "llm-sm container exists but is not running." + + local -a exec_args=(exec) + [[ -t 0 && -t 1 ]] || exec_args+=(-T) + docker_compose "${exec_args[@]}" llm-sm llm-sm "$@" +} + +cmd_llm() { + local sub="${1:-models}" + shift || true + + case "${sub,,}" in + runtime) + local mode="${1:-}" + if [[ -z "$mode" ]]; then + printf '%s\n' "$(compose_control_value LDS_AI_RUNTIME cpu)" + return 0 + fi + case "${mode,,}" in + cpu | nvidia | amd) + update_env "$ENV_DOCKER" LDS_AI_RUNTIME "${mode,,}" + ok "LLM runtime set to ${mode,,}. Recreate llm-sm to apply the change." + ;; + *) die "llm runtime " ;; + esac + ;; + host-port) + local state="${1:-status}" + case "${state,,}" in + status) + printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" + ;; + on | enable | enabled | 1) + update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 1 + ok "Direct LLM API enabled on loopback only. Recreate llm-sm to apply." + ;; + off | disable | disabled | 0) + update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 0 + ok "Direct LLM host API disabled. Recreate llm-sm to apply." + ;; + *) die "llm host-port " ;; + esac + ;; + models | ps | show | pull | rm | unload | run | ask | chat | prompt | code | review | json | ai-commit | ollama | api | version) + _llm_exec "${sub,,}" "$@" + ;; + help | -h | --help) + printf '%s\n' "llm " + printf '%s\n' "llm runtime " + printf '%s\n' "llm host-port " + ;; + *) + die "llm " + ;; + esac +} + +_shq() { printf '%q' "$1"; } + +cmd_diag() { + local sub="${1:-}" + shift || true + + case "${sub,,}" in + dns) + local dom="${1:-}" + [[ -n "$dom" ]] || die "diag dns " + local qdom + qdom="$(_shq "$dom")" + _tools_exec "dig +short $qdom; echo; nslookup $qdom 2>/dev/null || true; echo; getent hosts $qdom 2>/dev/null || true" + ;; + route | net) + _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || netstat -tulpen 2>/dev/null || true" + ;; + tcp) + local h="${1:-}" + local p="${2:-}" + [[ -n "$h" && -n "$p" ]] || die "diag tcp " + _tools_exec "nc -vz -w2 $(_shq "$h") $(_shq "$p")" + ;; + http) + local url="${1:-}" + shift || true + [[ -n "$url" ]] || die "diag http [curl-args...]" + local -a qargs=() + local a + for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done + _tools_exec "curl -vkI $(_shq "$url") ${qargs[*]}" + ;; + tls) + local dom="${1:-}" + [[ -n "$dom" ]] || die "diag tls " + local qdom + qdom="$(_shq "$dom")" + _tools_exec "echo | openssl s_client -connect ${qdom}:443 -servername $qdom -showcerts 2>/dev/null | sed -n '1,60p'" + ;; + *) + die "diag " + ;; + esac +} + +cmd_sniff() { + local url="${1:-}" + shift || true + [[ -n "$url" ]] || die "sniff [curl-args...]" + local -a qargs=() + local a + for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done + _tools_exec "curl -vk -D - $(_shq "$url") ${qargs[*]} | (command -v jq >/dev/null 2>&1 && jq . 2>/dev/null || cat)" +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6e. SECRETS / CERT / HOST / UI +# ───────────────────────────────────────────────────────────────────────────── +cmd_secrets() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" senv "$@" +} + +cmd_cert() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" certify "$@" +} + +cmd_host() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + add) + setup_domain + ;; + rm | remove | del | delete) + delete_domain "$@" + ;; + list) + shopt -s nullglob + for f in "$DIR/configuration/nginx/"*.conf; do + printf '%s +' "$(basename -- "$f" .conf)" + done + shopt -u nullglob + ;; + *) + die "host " + ;; + esac +} + +cmd_ui() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" lazydocker +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6f. EXEC / EVENTS / CLEAN / DISK +# ───────────────────────────────────────────────────────────────────────────── +cmd_exec() { + local svc="${1:-}" + shift || true + [[ -n "$svc" ]] || die "exec [cmd...]" + local s + s="$(resolve_service "$svc" || true)" + [[ -n "$s" ]] || die "Unknown service: $svc" + if [[ $# -gt 0 ]]; then + docker_compose exec "$s" "$@" + else + docker_compose exec "$s" sh -lc 'command -v bash >/dev/null 2>&1 && exec bash || exec sh' + fi +} + +cmd_events() { + local since="${1:-1h}" + local project + project="$(lds_project)" + docker events --since "$since" --filter "label=com.docker.compose.project=$project" +} + +cmd_clean() { + local yes=0 vols=0 + while [[ "${1:-}" ]]; do + case "$1" in + --yes | -y) + yes=1 + shift + ;; + --volumes | -v) + vols=1 + shift + ;; + *) + die "clean [--yes|-y] [--volumes|-v]" + ;; + esac + done + + ((yes)) || die "clean requires --yes" + + printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" + docker container prune -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" + docker network prune -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" + docker image prune -a -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" + docker builder prune -a -f >/dev/null 2>&1 || true + + if ((vols)); then + printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" + docker volume prune -f >/dev/null 2>&1 || true + fi + + printf "%b[clean]%b done\n" "$GREEN" "$NC" +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6g. HELP MARKDOWN +# ───────────────────────────────────────────────────────────────────────────── + +normalize_service() { + local raw="${1:-}" + local s="${raw//[[:space:]]/}" + [[ -n "$s" ]] || { + printf '%s' "" + return 0 + } + + local low="${s,,}" + + local key="${low//_/}" + key="${key//-/}" + if [[ "$key" =~ ^php ]]; then + local ver="${key#php}" + ver="${ver//[^0-9]/}" + if [[ "$ver" =~ ^([0-9])([0-9]).* ]]; then + printf 'php%s%s' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" + return 0 + fi + printf 'php' + return 0 + fi + + low="${low//_/-}" + while [[ "$low" == *"--"* ]]; do low="${low//--/-}"; done + printf '%s' "$low" +} + +cmd_rebuild() { + local -a targets=() all_svcs=() + local arg svc img + declare -A seen=() + + # ----------------------------- + # helper: add a service once + # ----------------------------- + _add_target() { + local s="$1" + [[ -n "$s" ]] || return 0 + [[ -n "${seen[$s]:-}" ]] && return 0 + seen[$s]=1 + targets+=("$s") + } + + # ----------------------------- + # helper: trim + # ----------------------------- + _trim() { + local s="$1" + s="${s#"${s%%[![:space:]]*}"}" + s="${s%"${s##*[![:space:]]}"}" + printf '%s' "$s" + } + + # ----------------------------- + # helper: interactive selection (comma separated, supports ranges) + # accepts: "all" or "1,3,5-7" or mix with names "nginx,2,5-6" + # ----------------------------- + _pick_targets_interactive() { + compose_services_load + all_svcs=("${__COMPOSE_SVCS[@]}") + if ((${#all_svcs[@]})); then + mapfile -t all_svcs < <(printf '%s\n' "${all_svcs[@]}" | LC_ALL=C sort -f -u) + fi + [[ ${#all_svcs[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" + + echo + echo "Select services to rebuild (comma separated; ranges allowed)." + echo "Examples: 1,3,5-7 | nginx,2,5-6 | all" + echo + + local i + for i in "${!all_svcs[@]}"; do + printf " %2d) %s\n" "$((i + 1))" "${all_svcs[$i]}" + done + + echo + local sel + read -r -p "Pick: " sel + sel="$(_trim "${sel:-}")" + [[ -n "$sel" ]] || die "No selection provided." + + if [[ "${sel,,}" == "all" ]]; then + for svc in "${all_svcs[@]}"; do _add_target "$svc"; done + return 0 + fi + + # split by comma + local IFS=, + for arg in $sel; do + arg="$(_trim "$arg")" + [[ -n "$arg" ]] || continue + + # range like 3-7 + if [[ "$arg" =~ ^[0-9]+-[0-9]+$ ]]; then + local a b + a="${arg%-*}" + b="${arg#*-}" + ((a >= 1)) || continue + ((b >= 1)) || continue + ((a <= b)) || { + local t="$a" + a="$b" + b="$t" + } + + local n + for ((n = a; n <= b; n++)); do + ((n >= 1 && n <= ${#all_svcs[@]})) || continue + _add_target "${all_svcs[$((n - 1))]}" + done + continue + fi + + # single index + if [[ "$arg" =~ ^[0-9]+$ ]]; then + local n="$arg" + ((n >= 1 && n <= ${#all_svcs[@]})) || continue + _add_target "${all_svcs[$((n - 1))]}" + continue + fi + + # treat as service/container name + svc="$(resolve_service "$arg")" + [[ -n "$svc" ]] && _add_target "$svc" + done + + [[ ${#targets[@]} -gt 0 ]] || die "No valid services selected." + } + + # ----------------------------- + # build target list + # ----------------------------- + if (($# == 0)); then + _pick_targets_interactive + elif [[ "${1,,}" == "all" ]]; then + compose_services_load + targets=("${__COMPOSE_SVCS[@]}") + [[ ${#targets[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" + else + for arg in "$@"; do + svc="$(resolve_service "$arg")" + [[ -n "$svc" ]] || continue + _add_target "$svc" + done + [[ ${#targets[@]} -gt 0 ]] || die "No valid services provided." + fi + + # ----------------------------- + # rebuild each target + # ----------------------------- + for svc in "${targets[@]}"; do + [[ -n "$svc" ]] || continue + compose_service_exists "$svc" || die "Unknown service/container: '$svc'" + + if compose_has_build "$svc"; then + logq rebuild "build/recreate $svc" + dc_build --no-cache --pull "$svc" + dc_up -d --no-deps --force-recreate "$svc" + continue + fi + + img="$(compose_image_for_service "$svc")" + logq rebuild "pull/recreate $svc${img:+ ($img)}" + + docker_compose rm -sf "$svc" >/dev/null 2>&1 || true + + if [[ -n "${img:-}" ]]; then + docker rmi -f "$img" >/dev/null 2>&1 || true + dc_pull "$svc" || true + else + dc_build --no-cache --pull "$svc" >/dev/null 2>&1 || true + fi + + dc_up -d --no-deps --force-recreate "$svc" + done + logq reboot "Rebooting stacks" + cmd_reboot +} + +cmd_config() { docker_compose config; } + +docker_shell() { + local c="${1:-}" + [[ -n "$c" ]] || die "container name required" + if docker exec "$c" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then + exec docker exec -it "$c" bash + else + exec docker exec -it "$c" sh + fi +} +cmd_tools() { + local sub="${1:-sh}" + shift || true + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + case "${sub,,}" in + sh | shell | "") + docker_shell "$ctr" + ;; + exec) + [[ $# -gt 0 ]] || die "tools exec " + docker exec -it "$ctr" sh -lc "$*" + ;; + file) + local p="${1:-}" + [[ -n "$p" ]] || die "tools file " + docker exec -it "$ctr" sh -lc "ls -la -- \"$p\" 2>/dev/null || true; echo; sed -n '1,200p' -- \"$p\" 2>/dev/null || true" + ;; + *) + die "tools " + ;; + esac +} +cmd_http() { [[ ${1:-} == reload ]] && http_reload; } +cmd_cli() { + local ctr="${1:-}" + shift || true + + [[ -n "$ctr" ]] || die "Usage: lds cli [cmd...]" + + docker inspect "$ctr" >/dev/null 2>&1 || die "Container not found: $ctr" + docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "Container not running: $ctr" + + # If user provided a command, run it; otherwise open an interactive shell. + if [[ "$#" -gt 0 ]]; then + local cmd="$*" + docker exec -it "$ctr" sh -lc ' + if command -v bash >/dev/null 2>&1; then + exec bash --login -lc "$1" + fi + exec sh -lc "$1" + ' sh "$cmd" + return + fi + + docker exec -it "$ctr" sh -lc ' + if command -v bash >/dev/null 2>&1; then + exec bash --login + fi + exec sh + ' +} + +cmd_core() { + # Usage: + # lds core -> open correct container for that domain (PHP/Node) + # lds core -> open a shell in that container + # lds core -> list domains and let user pick + + local target="${1:-}" + + # domain regex (same as domain-which/mkhost family) + local re='^([a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9])?\.)+(localhost|local|test|loc|[a-zA-Z]{2,})$' + + # If no target -> prompt from domain-which list + if [[ -z "$target" ]]; then + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + local -a domains=() + mapfile -t domains < <(docker exec "$tools_ctr" domain-which --list-domains 2>/dev/null | sed '/^[[:space:]]*$/d' || true) + + ((${#domains[@]} > 0)) || die "No domains found" + + # stable ordering + IFS=$'\n' domains=($(printf '%s\n' "${domains[@]}" | LC_ALL=C sort -u)) + + if ((${#domains[@]} == 1)); then + target="${domains[0]}" + else + if [[ ! -t 0 ]]; then + printf "%b[core]%b No domain provided. Available domains:\n" "$YELLOW" "$NC" >&2 + local i=1 + local d + for d in "${domains[@]}"; do + printf " %2d) %s\n" "$i" "$d" >&2 + ((i++)) + done + die "No TTY to prompt. Use: lds core " + fi + + printf "%bSelect domain:%b\n" "$CYAN" "$NC" >&2 + local i=1 d + for d in "${domains[@]}"; do + printf " %b%2d)%b %s\n" "$CYAN" "$i" "$NC" "$d" >&2 + ((i++)) + done + + local ans="" + while true; do + read -r -p "Enter number (1-${#domains[@]}): " ans + ans="$(echo "$ans" | xargs)" + [[ "$ans" =~ ^[0-9]+$ ]] || { + printf "%bInvalid input.%b\n" "$YELLOW" "$NC" >&2 + continue + } + ((ans >= 1 && ans <= ${#domains[@]})) || { + printf "%bOut of range.%b\n" "$YELLOW" "$NC" >&2 + continue + } + target="${domains[$((ans - 1))]}" + break + done + fi + fi + + # If target looks like a domain -> resolve via domain-which then shell in + if [[ "$target" =~ $re ]]; then + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + local app container wd + app="$(docker exec "$tools_ctr" domain-which --app --quiet "$target" 2>/dev/null)" || die "Unknown domain: $target" + container="$(docker exec "$tools_ctr" domain-which --container --quiet "$target" 2>/dev/null)" || die "No container resolved for: $target" + wd="$(docker exec "$tools_ctr" domain-which --docroot --quiet "$target" 2>/dev/null)" || true + [[ -n "${container:-}" ]] || die "No container resolved for: $target" + + # Node apps should always land at /app. Others follow resolved docroot. + if [[ "${app:-}" == "node" ]]; then + wd="/app" + fi + [[ -n "${wd:-}" ]] || wd="/app" + + docker exec -it "$container" bash -lc "cd \"$wd\" 2>/dev/null || cd /app 2>/dev/null || cd /; exec bash" + return 0 + fi + + # Otherwise treat target as a container name + docker exec -it "$(printf '%s' "$target" | tr '[:lower:]' '[:upper:]')" sh -lc 'exec bash -i || exec sh' +} + +cmd_setup() { + add_required_env + case ${1:-} in + init) env_init ;; + permission | permissions | perms | perm) fix_perms ;; + domain) setup_domain ;; + profiles | profile) process_all ;; + *) die "setup " ;; + esac +} + +cmd_certificate() { + case ${1:-} in + install) + shift || true + install_ca + ;; + uninstall | remove | rm) + shift || true + uninstall_ca "${@:-}" + ;; + *) + die "certificate " + ;; + esac +} + +############################################################################### +# NOTIFY +############################################################################### +notify_watch() { + local container="${1:-}" + if [[ -z "$container" ]]; then + container="$(_project_tools_container_running || true)" + [[ -n "$container" ]] || die "server-tools container is not running for project: $(lds_project)" + fi + local prefix="__HOST_NOTIFY__" + + need docker + + local _disp="${DISPLAY-}" + local _dbus="${DBUS_SESSION_BUS_ADDRESS-}" + + # Args: timeout(ms) urgency title body + _host_notify() { + local timeout="${1:-2500}" urgency="${2:-normal}" title="${3:-Notification}" body="${4:-}" + + # Linux desktop (or WSLg) + if has_cmd notify-send; then + (env DISPLAY="${_disp-}" DBUS_SESSION_BUS_ADDRESS="${_dbus-}" \ + setsid -f notify-send -u "$urgency" -t "$timeout" "$title" "$body" \ + >/dev/null 2>&1 || true) & + return 0 + fi + + # Windows toast (Git Bash) / WSL-on-Windows + if has_cmd powershell.exe; then + # Pass values as args to avoid quoting issues entirely. + # Note: urgency/timeout not used by toast api here; kept for parity. + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command \ + 'param([string]$t,[string]$b) + try { + [Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null + [Windows.Data.Xml.Dom.XmlDocument, Windows.Data.Xml.Dom.XmlDocument, ContentType = WindowsRuntime] > $null + + function Esc([string]$s) { + if ($null -eq $s) { return "" } + return ($s -replace "&","&" -replace "<","<" -replace ">",">" -replace "\"",""" -replace "'\''","'") + } + + $title = Esc $t + $body = Esc $b + + $xml = New-Object Windows.Data.Xml.Dom.XmlDocument + $xml.LoadXml("$title$body") + $toast = New-Object Windows.UI.Notifications.ToastNotification $xml + [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier("Devtainer").Show($toast) + } catch { }' \ + --% "$title" "$body" >/dev/null 2>&1 || true + + return 0 + fi + + # Fallback + printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 + return 0 + } + + trap - ERR + set +e + set +o pipefail + + local _stop=0 + + _watcher_notify() { + local urgency="${1:-critical}" title="${2:-Notifier}" body="${3:-Watcher event}" + _host_notify 2500 "$urgency" "$title" "$body" + } + + _watcher_int_term() { + _stop=1 + _watcher_notify critical "Notifier" "Notification watcher interrupted/exiting" + printf "%b[watcher]%b Notification watcher interrupted/exiting\n" "$RED" "$NC" >&2 + } + trap _watcher_int_term INT TERM + + local grep_cmd=(grep -a --line-buffered -E "^${prefix}([[:space:]]|$)") + has_cmd stdbuf && grep_cmd=(stdbuf -oL -eL "${grep_cmd[@]}") + + printf "%bNotify Watch:%b monitoring is active. Ctrl+C to stop.\n" "$GREEN" "$NC" + + while ((_stop == 0)); do + if ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then + _watcher_notify critical "Notifier" "Watcher stopped: $container is not running" + printf "%b[watcher]%b %s is not running; exiting.\n" "$RED" "$NC" "$container" >&2 + break + fi + + docker logs -f --tail 0 "$container" 2>&1 | + ("${grep_cmd[@]}" || true) | + while IFS=$'\t' read -r _ f1 f2 f3 f4 rest; do + local timeout urgency title body + + if [[ "${f1:-}" =~ ^[0-9]{1,6}$ ]]; then + timeout="$f1" + urgency="${f2:-normal}" + title="${f3:-Notification}" + body="${f4:-}" + else + timeout="2500" + urgency="${f1:-normal}" + title="${f2:-Notification}" + body="${f3:-}" + fi + + [[ -n "${rest:-}" ]] && body+=$'\t'"${rest}" + case "$urgency" in low | normal | critical) ;; *) urgency="normal" ;; esac + + _host_notify "$timeout" "$urgency" "$title" "$body" + printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 + done + + ((_stop)) && break + + if docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then + _watcher_notify critical "Notifier" "Watcher lost log stream (docker logs ended). Reconnecting…" + printf "%b[watcher]%b docker logs ended; reconnecting...\n" "$YELLOW" "$NC" >&2 + sleep 1 + continue + fi + + _watcher_notify critical "Notifier" "Watcher stopped: $container stopped" + printf "%b[watcher]%b %s stopped; exiting.\n" "$RED" "$NC" "$container" >&2 + break + done + + trap - INT TERM + set -euo pipefail + + ((_stop)) && return 130 + return 0 +} + +notify_test() { + local title="${1:-Notifier OK}" + local body="${2:-Hello from host via project server-tools container}" + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" notify -t 2500 -u normal "$title" "$body" +} + +cmd_notify() { + case ${1:-watch} in + watch) notify_watch "${2:-}" ;; + test) notify_test "${2:-Notifier OK}" "${3:-Hello from host}" ;; + *) die "notify " ;; + esac +} + +open_url() { + local url="${1:-}" + [[ -n "$url" ]] || return 0 + + # WSL/Windows helpers first when available + if grep -qi microsoft /proc/version 2>/dev/null; then + if has_cmd powershell.exe; then + powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 || true + return 0 + fi + if has_cmd cmd.exe; then + cmd.exe /c start "" "$url" >/dev/null 2>&1 || true + return 0 + fi + fi + + if has_cmd xdg-open; then + (xdg-open "$url" >/dev/null 2>&1 &) + return 0 + fi + if has_cmd open; then + (open "$url" >/dev/null 2>&1 &) + return 0 + fi + if has_cmd powershell; then + (powershell -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 &) + return 0 + fi + + printf "%bINFO%b: open this URL manually → %s\n" "$YELLOW" "$NC" "$url" +} + +############################################################################### +# RUN (ad-hoc Dockerfile runner) +############################################################################### +hash_short() { + local s="$1" + if has_cmd sha1sum; then + printf '%s' "$s" | sha1sum | cut -c1-8 + elif has_cmd shasum; then + printf '%s' "$s" | shasum -a 1 | cut -c1-8 + else + # POSIX fallback; stable (not cryptographic) + printf '%s' "$s" | cksum | awk '{print $1}' + fi +} + +run_slug() { + local dir="$1" base hash + base="$(basename "$dir" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9._-' '-')" + hash="$(hash_short "$dir")" + printf '%s-%s' "$base" "$hash" +} + +run_plan() { + local dir="$1" slug + slug="$(run_slug "$dir")" + printf '%s|%s|%s\n' \ + "lds-run-${slug}" \ + "${slug}:local" \ + "$dir" +} + +detect_host_os() { + if is_windows_shell || grep -qi microsoft /proc/version 2>/dev/null; then + printf 'windows' + return 0 + fi + + if has_cmd uname; then + case "$(uname -s 2>/dev/null || true)" in + Darwin) + printf 'macos' + ;; + Linux) + printf 'linux' + ;; + *) + printf 'unknown' + ;; + esac + else + printf 'unknown' + fi +} + +run_find_container() { + local dir="$1" + docker ps -a --filter "label=com.infocyph.lds.run=1" \ + --filter "label=com.infocyph.lds.dir=${dir}" \ + --format '{{.Names}}' | head -n 1 +} + +run_build() { + local tag="$1" dir="$2" + + # Build only if the image doesn't already exist. + if docker image inspect "$tag" >/dev/null 2>&1; then + printf "%b[run]%b Image exists, skipping build: %b%s%b +" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" + return 0 + fi + + printf "%b[run]%b Building image %b%s%b from %s +" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" "$dir" + docker build -t "$tag" "$dir" +} + +run_start() { + local name="$1" tag="$2" dir="$3" keepalive="$4" sock="$5" host_os="$6" + shift 6 || true + + # Remaining args are split by a "--" sentinel: + # - before "--" : publish specs (HOST:CONT), repeatable + # - after "--" : mount specs (HOST[:CONT]), repeatable + local -a pubs=() mounts=() + local seen_delim=0 x + for x in "$@"; do + if [[ "$x" == "--" ]]; then + seen_delim=1 + continue + fi + if ((seen_delim)); then + mounts+=("$x") + else + pubs+=("$x") + fi + done + + # Normalize project dir (POSIX absolute). + # On Windows Git Bash, /e/... is OK; docker.exe will receive converted path automatically. + local dir_posix + dir_posix="$(cd "$dir" 2>/dev/null && pwd -P)" || die "invalid dir: $dir" + + # MSYS-safe container paths: use '//' prefix to prevent path conversion. + # Docker interprets //path as /path inside container. + local WDIR="//workspace" + local WDIR_MOUNT="${dir_posix}://workspace" + + local -a args=(docker run -d --name "$name" + --label "com.infocyph.lds.run=1" + --label "com.infocyph.lds.dir=$dir_posix" + --label "com.infocyph.lds.tag=$tag" + -w "$WDIR" + -v "$WDIR_MOUNT" + ) + + if [[ -n "$host_os" ]]; then + args+=(-e "HOST_OS=$host_os") + fi + + # Mount extra directories/files (HOST[:CONT]). + # - If container path missing, mounts under /mnt/. + # - HOST may be relative to the run directory. + if ((${#mounts[@]})); then + local spec host cont base + for spec in "${mounts[@]}"; do + [[ -n "$spec" ]] || continue + host="$spec" + cont="" + + # Split as HOST:CONT ONLY if suffix after last ':' looks like a container absolute path (/...) + # (safe for Windows drive letters like E:\... because tail won't start with '/') + if [[ "$spec" == *:* ]]; then + local tail="${spec##*:}" + if [[ "$tail" == /* ]]; then + host="${spec%:*}" + cont="$tail" + fi + fi + + # Resolve host to absolute (POSIX) for checks + if [[ "$host" != /* && "$host" != ~* && ! "$host" =~ ^[A-Za-z]:[\\/].* ]]; then + host="${dir_posix%/}/$host" + fi + + # If user provided Windows path (E:\...), convert to POSIX for existence check + if [[ "$host" =~ ^[A-Za-z]:[\\/].* ]] && has_cmd cygpath; then + host="$(cygpath -u "$host")" + fi + + host="$(cd "${host%/*}" 2>/dev/null && pwd -P)/${host##*/}" || { + printf "%b[run]%b Warning: cannot resolve mount path: %s\n" "$YELLOW" "$NC" "$spec" >&2 + continue + } + + [[ -e "$host" ]] || { + printf "%b[run]%b Warning: mount path does not exist: %s\n" "$YELLOW" "$NC" "$host" >&2 + continue + } + + if [[ -z "$cont" ]]; then + base="${host##*/}" + cont="/mnt/${base}" + fi + [[ "$cont" == /* ]] || cont="/mnt/${cont}" + + # Prevent MSYS conversion for container side by using '//' prefix + cont="//${cont#/}" + + args+=(-v "${host}:${cont}") + done + fi + + # Optional docker sock + if [[ "${sock:-0}" == 1 ]]; then + args+=(-v "/var/run/docker.sock:/var/run/docker.sock") + fi + + # Publish ports + local pub + for pub in "${pubs[@]}"; do + [[ -n "$pub" ]] || continue + args+=(-p "$pub") + done + + if [[ "$keepalive" == 1 ]]; then + # Keepalive mode replaces the image command; disable image healthcheck to avoid false "unhealthy". + args+=(--no-healthcheck --entrypoint sh "$tag" -c "trap : TERM INT; sleep infinity & wait") + else + args+=("$tag") + fi + + printf "%b[run]%b Starting container %b%s%b\n" "$CYAN" "$NC" "$BLUE" "$name" "$NC" + + # IMPORTANT: don't hide errors; if it fails, you need to see why + if ! "${args[@]}"; then + printf "%b[run]%b docker run failed.\n" "$RED" "$NC" >&2 + return 1 + fi + printf "\n" +} + +run_exec_shell() { + local name="$1" + if docker exec "$name" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then + exec docker exec -it "$name" bash + else + exec docker exec -it "$name" sh + fi +} + +cmd_run() { + local action="*" dir="$PWD" name="" tag="" nobuild=0 keepalive=1 sock=0 host_os="" + local -a publish=() mounts=() + local open_port="" open_path="/" open_proto="http" + + while [[ $# -gt 0 ]]; do + case "$1" in + stop | rm | ps | shell | logs | open | "*") + action="$1" + shift + ;; + build) + action="*" + shift + ;; + --name) + name="${2:-}" + shift 2 + ;; + --tag) + tag="${2:-}" + shift 2 + ;; + --no-build) + nobuild=1 + shift + ;; + --no-keepalive) + keepalive=0 + shift + ;; + --sock) + sock=1 + shift + ;; + --host-os) + host_os="${2:-}" + shift 2 + ;; + -p | --publish) + publish+=("${2:-}") + shift 2 + ;; + --mount) + mounts+=("${2:-}") + shift 2 + ;; + --port) + open_port="${2:-}" + shift 2 + ;; + --path) + open_path="${2:-/}" + shift 2 + ;; + --https) + open_proto="https" + shift + ;; + --http) + open_proto="http" + shift + ;; + *) break ;; + esac + done + + # Host path (POSIX) for planning/labels + local dir_posix + dir_posix="$(cd "$dir" && pwd -P)" + + if [[ -z "$host_os" ]]; then + host_os="${HOST_OS:-$(detect_host_os)}" + fi + + # Docker path (may need Windows form for docker.exe) + local dir_docker="$dir_posix" + + # Windows Git Bash/MSYS hardening: + # - stop MSYS rewriting container paths (/workspace -> D:/Program Files/Git/workspace) + # - but still feed docker.exe Windows-absolute host paths for build/run contexts + if is_windows_shell; then + export MSYS_NO_PATHCONV=1 + export MSYS2_ARG_CONV_EXCL='*' + if has_cmd cygpath; then + dir_docker="$(cygpath -w "$dir_posix")" + fi + fi + + # Plan/name/tag should be based on the real project identity (POSIX dir) + IFS='|' read -r def_name def_tag _def_dir < <(run_plan "$dir_posix") + name="${name:-$def_name}" + + # Tag rules: + # - Default tag is ":local" (from run_plan) + # - If user passes --tag without ":", append ":local" + if [[ -n "${tag:-}" ]]; then + if [[ "$tag" != *:* ]]; then + tag="${tag}:local" + fi + else + tag="$def_tag" + fi + + _find_for_dir() { + local found + found="$(run_find_container "$dir_posix" || true)" + if [[ -n "$found" ]]; then + printf '%s' "$found" + return 0 + fi + if docker inspect "$name" >/dev/null 2>&1; then + printf '%s' "$name" + return 0 + fi + return 1 + } + + _run_build_summary() { + local img="$1" build_dir="$2" cname="$3" + local tag_only="${img##*:}" + + printf "\n%b[run]%b Build summary\n" "$CYAN" "$NC" + printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" + printf " %bTag:%b %s\n" "$BOLD" "$NC" "$tag_only" + printf " %bDir:%b %s\n" "$BOLD" "$NC" "$build_dir" + printf " %bName:%b %s\n" "$BOLD" "$NC" "$cname" + printf " %bKeepalive:%b %s\n" "$BOLD" "$NC" "$keepalive" + printf " %bSock:%b %s\n" "$BOLD" "$NC" "$sock" + printf " %bHost OS:%b %s\n" "$BOLD" "$NC" "$host_os" + + if ((${#publish[@]})); then + printf " %bPublish:%b %s\n" "$BOLD" "$NC" "${publish[*]}" + else + printf " %bPublish:%b (none)\n" "$BOLD" "$NC" + fi + + if ((${#mounts[@]})); then + printf " %bMounts:%b %s\n" "$BOLD" "$NC" "${mounts[*]}" + else + printf " %bMounts:%b (none)\n" "$BOLD" "$NC" + fi + printf "\n" + } + + _run_runtime_summary() { + local cname="$1" + local id img state ports + id="$(docker inspect -f '{{.Id}}' "$cname" 2>/dev/null | cut -c1-12 || true)" + img="$(docker inspect -f '{{.Config.Image}}' "$cname" 2>/dev/null || true)" + state="$(docker inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || true)" + ports="$(docker port "$cname" 2>/dev/null | sed '/^[[:space:]]*$/d' | tr '\n' '; ' | sed 's/; $//' || true)" + + printf "%b[run]%b Runtime summary\n" "$CYAN" "$NC" + printf " %bContainer:%b %s\n" "$BOLD" "$NC" "${cname}${id:+ ($id)}" + [[ -n "$img" ]] && printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" + [[ -n "$state" ]] && printf " %bState:%b %s\n" "$BOLD" "$NC" "$state" + if [[ -n "$ports" ]]; then + printf " %bPorts:%b %s\n" "$BOLD" "$NC" "$ports" + else + printf " %bPorts:%b (none published)\n" "$BOLD" "$NC" + fi + printf "%b\n[run]%b Example Usage (in Composer)\n" "$CYAN" "$NC" + printf " %bimage:%b %s\n" "$BOLD" "$NC" "$img" + printf " %bpull_policy:%b never\n" "$BOLD" "$NC" + printf "\n" + } + + case "$action" in + ps) + docker ps -a --filter "label=com.infocyph.lds.run=1" \ + --format 'table {{.Names}} {{.Image}} {{.Status}} {{.Labels}}' + return 0 + ;; + stop) + local existing + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + docker stop "$existing" >/dev/null + printf "%b[run]%b Stopped %s\n" "$GREEN" "$NC" "$existing" + return 0 + ;; + logs) + local existing + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + exec docker logs -f "$existing" + ;; + open) + local existing line addr hp url + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + + [[ -n "$open_path" ]] || open_path="/" + [[ "$open_path" == /* ]] || open_path="/$open_path" + + if [[ -n "$open_port" ]]; then + line="$(docker port "$existing" "$open_port" 2>/dev/null | head -n 1 || true)" + [[ -n "$line" ]] || line="$(docker port "$existing" "${open_port}/tcp" 2>/dev/null | head -n 1 || true)" + else + line="$(docker port "$existing" 2>/dev/null | head -n 1 || true)" + fi + + if [[ -z "$line" ]]; then + printf "%b[run]%b No published ports found.\n" "$YELLOW" "$NC" + printf "%b[run]%b Tip: start with %blds run --publish 8025:8025%b then %blds run open%b\n" \ + "$YELLOW" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" + return 1 + fi + + addr="${line##*-> }" + hp="${addr##*:}" + url="${open_proto}://localhost:${hp}${open_path}" + open_url "$url" + printf "%b[run]%b Opened: %s\n" "$GREEN" "$NC" "$url" + return 0 + ;; + rm) + local existing img + existing="$(_find_for_dir)" || true + if [[ -n "${existing:-}" ]]; then + img="$(docker inspect -f '{{.Config.Image}}' "$existing" 2>/dev/null || true)" + docker stop "$existing" >/dev/null 2>&1 || true + docker rm "$existing" >/dev/null 2>&1 || true + printf "%b[run]%b Removed container %s\n" "$GREEN" "$NC" "$existing" + if [[ -n "${img:-}" ]]; then + docker rmi -f "$img" >/dev/null 2>&1 || true + printf "%b[run]%b Removed image %s\n" "$GREEN" "$NC" "$img" + fi + else + printf "%b[run]%b No container found for %s\n" "$YELLOW" "$NC" "$dir_posix" + fi + return 0 + ;; + shell | "*") + if ((nobuild == 0)); then + # Build needs docker.exe-friendly path on Windows + run_build "$tag" "$dir_docker" + else + printf "%b[run]%b Skipping build (--no-build)\n" "$YELLOW" "$NC" + fi + + _run_build_summary "$tag" "$dir_posix" "$name" + + if docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null | grep -q true; then + printf "%b[run]%b Container already running: %s\n\n" "$GREEN" "$NC" "$name" + else + if docker inspect "$name" >/dev/null 2>&1; then + docker rm -f "$name" >/dev/null 2>&1 || true + fi + + # Keep mounts as user gave them (POSIX/relative); run_start should validate POSIX + # and convert host-side to Windows only at docker run time. + run_start "$name" "$tag" "$dir_docker" "$keepalive" "$sock" "$host_os" \ + "${publish[@]}" -- "${mounts[@]}" + fi + + _run_runtime_summary "$name" + + # "shell" enters the container; "*" / "build" does not. + if [[ "$action" == "shell" ]]; then + run_exec_shell "$name" + else + printf "%b[run]%b Built/started. Use %blds run shell%b to enter, %blds run logs%b to follow logs.\n" \ + "$GREEN" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" + return 0 + fi + ;; + esac +} + +############################################################################### +# 6w. NEW FEATURES: stack diff | support trace +############################################################################### + +# stack diff: show what would run (compose) vs what's running (docker) +cmd_stack_diff() { + local json=0 + local show_config=0 + while [[ "${1:-}" ]]; do + case "$1" in + --json) + json=1 + shift + ;; + --config) + show_config=1 + shift + ;; + *) break ;; + esac + done + + local project + project="$(lds_project)" + local cfg_json="" + + if docker_compose config --format json >/dev/null 2>&1; then + cfg_json="$(docker_compose config --format json)" + else + # fallback: best-effort text config + cfg_json="" + fi + + # running: service -> image + declare -A running=() + local line + while IFS= read -r line; do + [[ -n "$line" ]] || continue + local svc="${line%%|*}" + local img="${line#*|}" + running["$svc"]="$img" + done < <(docker ps \ + --filter "label=com.docker.compose.project=$project" \ + --format '{{index .Labels "com.docker.compose.service"}}|{{.Image}}' 2>/dev/null || true) + + # desired: service -> image/build context (best-effort) + declare -A desired_img=() + declare -A desired_ctx=() + declare -A desired_df=() + + if [[ -n "$cfg_json" ]]; then + if has_tool jq; then + while IFS= read -r line; do + local svc="${line%%|*}" + local img="${line#*|}" + desired_img["$svc"]="$img" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"') + while IFS= read -r line; do + local svc="${line%%|*}" + local ctx="${line#*|}" + desired_ctx["$svc"]="$ctx" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"') + while IFS= read -r line; do + local svc="${line%%|*}" + local df="${line#*|}" + desired_df["$svc"]="$df" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"') + elif _server_tools_has jq; then + # Fallback: parse via project tools container jq through stdin (no shell re-quoting of JSON payload). + local ctr + ctr="$(_project_tools_container_running || true)" + if [[ -n "$ctr" ]]; then + while IFS= read -r line; do + local svc="${line%%|*}" + local img="${line#*|}" + desired_img["$svc"]="$img" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"' 2>/dev/null || true) + while IFS= read -r line; do + local svc="${line%%|*}" + local ctx="${line#*|}" + desired_ctx["$svc"]="$ctx" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"' 2>/dev/null || true) + while IFS= read -r line; do + local svc="${line%%|*}" + local df="${line#*|}" + desired_df["$svc"]="$df" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"' 2>/dev/null || true) + fi + fi + fi + + # Build result object + if ((json)); then + if has_tool jq; then + # assemble in bash -> jq + local tmp + tmp="$(mktemp)" + { + printf '{' + printf '"project":%s,' "$(printf '%s' "$project" | jq -Rsa .)" + printf '"compose_file":%s,' "$(printf '%s' "$COMPOSE_FILE" | jq -Rsa .)" + printf '"running":{' + local first=1 k + for k in "${!running[@]}"; do + ((first)) || printf ',' + first=0 + printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${running[$k]}" | jq -R .)" + done + printf '},' + printf '"desired":{' + first=1 + for k in "${!desired_img[@]}"; do + ((first)) || printf ',' + first=0 + printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${desired_img[$k]}" | jq -R .)" + done + printf '},' + printf '"diff":[' + first=1 + # union keys + declare -A seen=() + for k in "${!running[@]}"; do seen["$k"]=1; done + for k in "${!desired_img[@]}"; do seen["$k"]=1; done + for k in "${!seen[@]}"; do + local r="${running[$k]:-}" + local d="${desired_img[$k]:-}" + if [[ "$r" != "$d" ]]; then + ((first)) || printf ',' + first=0 + printf '{"service":%s,"running":%s,"desired":%s}' \ + "$(printf '%s' "$k" | jq -R .)" \ + "$(printf '%s' "$r" | jq -R .)" \ + "$(printf '%s' "$d" | jq -R .)" + fi + done + printf ']' + printf '}\n' + } >"$tmp" + cat "$tmp" | jq . + rm -f "$tmp" + else + die "jq required for --json (or run inside project server-tools container)" + fi + return 0 + fi + + printf "%bStack diff%b (project=%s)\n" "$CYAN" "$NC" "$project" + printf "%bCompose file:%b %s\n" "$DIM" "$NC" "$COMPOSE_FILE" + + if ((show_config)); then + if [[ -n "$cfg_json" ]]; then + printf "\n%bEffective compose config (json):%b\n" "$DIM" "$NC" + printf '%s\n' "$cfg_json" + else + printf "\n%bEffective compose config:%b\n" "$DIM" "$NC" + docker_compose config || true + fi + fi + + # union services + declare -A all=() + local svc + for svc in "${!running[@]}"; do all["$svc"]=1; done + for svc in "${!desired_img[@]}"; do all["$svc"]=1; done + + printf "\n%-22s %-40s %-40s %s\n" "SERVICE" "RUNNING" "DESIRED" "STATUS" + printf "%-22s %-40s %-40s %s\n" "------" "-------" "-------" "------" + for svc in $(printf '%s\n' "${!all[@]}" | sort); do + local r="${running[$svc]:-}" + local d="${desired_img[$svc]:-}" + local st + if [[ -z "$r" ]]; then + st="(not running)" + elif [[ -z "$d" ]]; then + st="(not in config)" + elif [[ "$r" == "$d" ]]; then + st="OK" + else + st="DIFF" + fi + printf "%-22s %-40.40s %-40.40s %s\n" "$svc" "$r" "$d" "$st" + done + + printf "\n%bNotes:%b\n" "$DIM" "$NC" + printf " - Desired image is derived from 'docker compose config'. If a service uses only 'build:' and no 'image:', desired may be empty.\n" + printf " - Use: lds stack diff --config (to print resolved compose config)\n" +} + +# support trace: quick end-to-end trace for a domain +cmd_support_trace() { + local dom="${1:-}" + [[ -n "$dom" ]] || die "support trace " + + local nconf="$DIR/configuration/nginx/$dom.conf" + printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" + + # 1) DNS + if _server_tools_running; then + printf "\n%b[DNS]%b\n" "$DIM" "$NC" + _tools_exec "dig +short $(_shq "$dom") || true; getent hosts $(_shq "$dom") 2>/dev/null || true" + else + printf "\n%b[DNS]%b\n" "$DIM" "$NC" + (has_cmd dig && dig +short "$dom") || true + (has_cmd getent && getent hosts "$dom") || true + fi + + # 2) TLS certificate + printf "\n%b[TLS]%b\n" "$DIM" "$NC" + if _server_tools_running; then + _tools_exec "echo | openssl s_client -connect $(_shq "$dom"):443 -servername $(_shq "$dom") -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true" + else + echo | openssl s_client -connect "${dom}:443" -servername "$dom" -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true + fi + + # 3) HTTP probe (timings) + printf "\n%b[HTTP]%b\n" "$DIM" "$NC" + if _server_tools_running; then + _tools_exec "curl -sk -o /dev/null -D - -w 'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' https://$(_shq "$dom") | sed -n '1,30p'" + else + curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' + fi + + # 4) Upstream inference from nginx conf (if exists) + printf "\n%b[Upstream]%b\n" "$DIM" "$NC" + if [[ -r "$nconf" ]]; then + if grep -q fastcgi_pass "$nconf"; then + local php + php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" + elif grep -q proxy_pass "$nconf"; then + local up + up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" + else + printf "type=static\n" + fi + else + printf "nginx_conf=%s (missing)\n" "$nconf" + fi + + # 5) Recent nginx logs (compose) + printf "\n%b[Recent nginx logs]%b\n" "$DIM" "$NC" + docker_compose logs --no-color --tail 120 nginx 2>/dev/null | text_grep -i "$dom" || docker_compose logs --no-color --tail 120 nginx 2>/dev/null || true + + printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" +} + +############################################################################### +# 6x. GROUPED COMMAND ROUTERS (stack/domain/support) + backward-compatible aliases +############################################################################### + +cmd_stack() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + "" | help | -h | --help) cmd_help stack ;; + up) cmd_up "$@" ;; + start) cmd_start "$@" ;; + down | stop) cmd_down "$@" ;; + restart | reboot) cmd_restart "$@" ;; + status) cmd_status "$@" ;; + ps) cmd_ps "$@" ;; + logs) cmd_logs "$@" ;; + exec) cmd_exec "$@" ;; + events) cmd_events "$@" ;; + clean) cmd_clean "$@" ;; + config) cmd_config "$@" ;; diff) cmd_stack_diff "$@" ;; + + http) cmd_http "$@" ;; + *) + die "stack " + ;; + esac +} + +# Canonical: domain. Legacy: host. +cmd_domain() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + "" | help | -h | --help) die "domain " ;; + add) cmd_host add "$@" ;; + rm | remove | del | delete) cmd_host rm "$@" ;; + ls | list) cmd_host list "$@" ;; + *) + die "domain " + ;; + esac +} + +cmd_support() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + "" | help | -h | --help) die "support " ;; + open) cmd_open "$@" ;; + bundle) cmd_bundle "$@" ;; + notify) cmd_notify "$@" ;; + ui) cmd_ui "$@" ;; + *) + die "support " + ;; + esac +} + +# Minimal support bundle (shareable diagnostics zip) +# Modes: +# --redact : remove secrets-like env lines from captured env/config outputs (best-effort) +# --full : include more logs/inspect output (can be large) +cmd_bundle() { + local mode="redact" + local out="${1:-}" + while [[ "${1:-}" ]]; do + case "$1" in + --redact) + mode="redact" + shift + ;; + --full) + mode="full" + shift + ;; + *.zip) + out="$1" + shift + ;; + *) break ;; + esac + done + + need zip + local ts + ts="$(date +%Y%m%d_%H%M%S)" + local project + project="$(lds_project)" + local tmp + tmp="$(mktemp -d "${TMPDIR:-/tmp}/lds_bundle.XXXXXX")" + local base="lds_bundle_${project}_${ts}" + [[ -n "$out" ]] || out="$PWD/${base}.zip" + + cleanup_bundle() { rm -rf "$tmp" 2>/dev/null || true; } + trap cleanup_bundle RETURN + + { + echo "project=$project" + echo "dir=$DIR" + echo "time=$ts" + echo "mode=$mode" + } >"$tmp/meta.txt" + + # compose config (effective) + { + echo "# docker compose config" + docker_compose config 2>&1 || true + } >"$tmp/compose.config.txt" + + # ps + networks + docker_compose ps >"$tmp/compose.ps.txt" 2>&1 || true + docker network ls >"$tmp/docker.networks.txt" 2>&1 || true + + # container list + inspect (scoped) + docker ps --filter "label=com.docker.compose.project=$project" --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}' \ + >"$tmp/docker.ps.txt" 2>&1 || true + + if [[ "$mode" == "full" ]]; then + docker inspect $(docker ps -q --filter "label=com.docker.compose.project=$project" 2>/dev/null) \ + >"$tmp/docker.inspect.json" 2>/dev/null || true + fi + + # recent logs (tail) + docker_compose logs --no-color --tail 400 >"$tmp/compose.logs.txt" 2>&1 || true + + # vhost configs + if [[ -d "$DIR/configuration/nginx" ]]; then + mkdir -p "$tmp/nginx" + cp -a "$DIR/configuration/nginx/." "$tmp/nginx/" 2>/dev/null || true + fi + + # env files (redacted best-effort) + mkdir -p "$tmp/env" + if [[ -r "$ENV_MAIN" ]]; then + if [[ "$mode" == "redact" ]]; then + sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_MAIN" >"$tmp/env/.env" + else + cp -a "$ENV_MAIN" "$tmp/env/.env" 2>/dev/null || true + fi + fi + if [[ -r "$ENV_DOCKER" ]]; then + if [[ "$mode" == "redact" ]]; then + sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_DOCKER" >"$tmp/env/docker.env" + else + cp -a "$ENV_DOCKER" "$tmp/env/docker.env" 2>/dev/null || true + fi + fi + + # tools-side quick diagnostics (inside network) + { + echo "# ip r / ip a / ss" + _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || true" + } >"$tmp/tools.net.txt" 2>&1 || true + + # pack + (cd "$tmp" && zip -qr "$out" .) || die "Failed to write bundle: $out" + ok "Bundle created: $out" +} + +cmd_help() { + + if [[ "${1:-}" == "--markdown" ]]; then + cat <<'MD' +# LocalDevStack (lds) — Command Reference + +## Stack (compose) +- `lds stack up` *(aliases: `up`)* +- `lds stack start` *(aliases: `start`)* +- `lds stack down [--volumes --yes]` *(aliases: `down`, `stop`)* +- `lds stack restart [svc]` *(aliases: `restart`, `reboot`)* +- `lds stack status [status-args…]` *(alias: `status`; forwards args to tools `status`)* +- `lds stack ps` *(alias: `ps`)* +- `lds stack logs [svc] [--follow] [--since ] [--grep ]` *(alias: `logs`)* +- `lds stack exec [cmd…]` *(alias: `exec`)* +- `lds stack events [--since ]` *(alias: `events`)* +- `lds stack clean --yes [--volumes]` *(alias: `clean`)* +- `lds stack diff [--config] [--json]` *(shows desired vs running images)* + + +## Domain (vhost lifecycle + routing) +- `lds domain add …` +- `lds domain rm …` +- `lds domain ls` + +Legacy alias: `lds host …` → same subcommands as `domain`. + +## Certificates (TLS) +- `lds cert status [domain|all]` +- `lds cert regen [domain|all] [--yes]` +- `lds cert diagnose ` +- `lds certificate install` +- `lds certificate uninstall [--all]` + +## Diagnostics +- `lds diag dns ` +- `lds diag net` +- `lds diag tcp ` +- `lds diag http ` *(alias: `sniff `)* +- `lds diag tls ` + +## Config +- `lds config show [--json]` +- `lds config services` +- `lds config profiles` +- `lds config env-used` +- `lds config validate` + +## Support +- `lds support open + support trace ` +- `lds support bundle [--redact|--full]` +- `lds support notify …` +- `lds support ui` + +Shortcuts: `open`, `bundle`, `notify`, `ui` map to `support …`. + +## Secrets (senv) +- `lds secrets …` + +## Tools (project server-tools container) +- `lds tools sh` +- `lds tools exec ""` +- `lds tools file ` + +## AI +- `lds ai status|ask|explain|troubleshoot|review|repo-review|graphify …` +- `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit …` +- `lds llm runtime ` +- `lds llm host-port ` + +## Setup +- `lds setup init|permissions|domain|profiles` + +## Runner (ad‑hoc Dockerfile runner) +- `lds run` *(default: build+start only)* / `lds run shell` *(build+start+enter)* / `lds run *` *(same as default)* (+ `ps|logs|stop|rm|open` and flags: `--publish|-p`, `--no-keepalive`, `--mount`, `--sock`, `--host-os`, `--tag`, `--name`) + +## Other +- `lds rebuild [all|]` +- `lds core [domain]` + +## Short client wrappers +- `lds php|composer|node|npm|npx …` +- `lds my|mysql|mysqldump …` +- `lds maria|mariadb|mariadb-dump …` +- `lds pg|psql|pg_dump|pg_restore …` +- `lds redis|redis-cli …` +MD + return 0 + fi + + cat < + clean Alias of: stack <...> + +${CYAN}Domain (vhosts + routing):${NC} + domain add|rm|ls + host add|rm|list Legacy alias of: domain <...> + +${CYAN}Certificates (TLS):${NC} + cert status|regen|diagnose + certificate install|uninstall [--all] + +${CYAN}Diagnostics:${NC} + diag dns|net|tcp|http|tls + sniff Alias of: diag http + +${CYAN}Config:${NC} + config show|services|profiles|env-used|validate + +${CYAN}Support:${NC} + support open + support trace + support bundle [--redact|--full] + support notify ... + support ui + open|bundle|notify|ui Shortcuts → support <...> + +${CYAN}Secrets:${NC} + secrets + +${CYAN}Tools (project server-tools container):${NC} + tools sh|exec|file + +${CYAN}AI:${NC} + ai status|ask|explain|troubleshoot|review|repo-review|graphify + llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit + llm runtime + llm host-port + +${CYAN}Setup:${NC} + setup init|permissions|domain|profiles + +${CYAN}Runner (ad-hoc Dockerfile runner):${NC} + run [ps|logs|stop|rm|open] [--publish|-p A:B] [--no-keepalive] [--mount HOST[:CONT]] [--sock] [--host-os VALUE] + +${CYAN}Other:${NC} + rebuild [all|] + core [domain] + +${CYAN}Help:${NC} + help [--markdown] + +EOF +} + +############################################################################### +# 7. MAIN +############################################################################### +main() { + [[ $# -gt 0 ]] || { + cmd_help + exit 1 + } + + while [[ $# -gt 0 ]]; do + case "$1" in + -v | --verbose) + VERBOSE=1 + QUIET=0 + shift + ;; + -q | --quiet) + QUIET=1 + VERBOSE=0 + shift + ;; + --reload-extras) + export LDS_EXTRAS_RELOAD=1 + shift + ;; + --) + shift + break + ;; + -h | --help) + cmd_help + exit 0 + ;; + -*) die "Unknown global option: $1" ;; + *) break ;; + esac + done + + [[ $# -gt 0 ]] || { + cmd_help + exit 1 + } + + local cmd="${1,,}" + shift || true + + if [[ "$cmd" == "help" ]]; then + cmd_help "$@" + exit 0 + fi + + # Do not require docker for pure help output; everything else expects the stack. + need docker + + case "$cmd" in + php | composer | node | npm | npx) exec "$DIR/bin/$cmd" "$@" ;; + pg | psql | pg_restore | pg-restore | pgrestore | pg_dump | pgdump | pg-dump) exec "$DIR/bin/pg" "$@" ;; + maria | mariadb | mariadbdump | mariadb-dump | mariadb_dump) exec "$DIR/bin/maria" "$@" ;; + my | mysql | mysqldump | mysql-dump | mysql_dump) exec "$DIR/bin/my" "$@" ;; + redis | redis-cli) exec "$DIR/bin/redis-cli" "$@" ;; + es | elastic | elasticsearch) exec "$DIR/bin/es" "$@" ;; + mongo | mongodb | mongosh | mongoimport | mongoexport) exec "$DIR/bin/mongo" "$@" ;; + ai) cmd_ai "$@" ;; + llm) cmd_llm "$@" ;; + vpn-fix) cmd_vpn_fix "$@" ;; + *) + if declare -F "cmd_$cmd" >/dev/null 2>&1; then + "cmd_$cmd" "$@" + else + exec "$DIR/bin/tool-runner" "$cmd" "$@" + fi + ;; + esac +} + +main "$@" +\r'}" + if (("${#value}" >= 2)); then + first="${value:0:1}" + last="${value: -1}" + if [[ ("$first" == '"' && "$last" == '"') || ("$first" == "'" && "$last" == "'") ]]; then + value="${value:1:${#value}-2}" + fi + fi + printf '%s' "$value" +} + +# Resolution mirrors Compose intent: shell > user docker/.env > release.env > fallback. +compose_control_value() { + local key="${1:?}" fallback="${2-}" value + if [[ -v "$key" ]]; then + printf '%s' "${!key}" + return 0 + fi + if value="$(dotenv_value "$ENV_DOCKER" "$key")"; then + printf '%s' "$value" + return 0 + fi + if value="$(dotenv_value "$ENV_RELEASE" "$key")"; then + printf '%s' "$value" + return 0 + fi + printf '%s' "$fallback" +} + if [[ "${1:-}" == "--__win_workdir" ]]; then export WORKDIR_WIN="${2:-}" shift 2 From 700e6f22fcdd08b77b14062b252ba03fe3e92baf Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:28:20 +0600 Subject: [PATCH 034/509] test: cover LocalDevStack AI compose modes --- tests/compose-contract.sh | 62 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index e1971a26..2366c9f0 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -59,6 +59,19 @@ render mongodb --profile mongodb render redis --profile redis render elasticsearch --profile elasticsearch render elasticsearch-filebeat --profile elasticsearch --profile filebeat +render ai --profile ai +docker compose --project-directory "$ROOT" \ + -f "$ROOT/docker/compose/main.yaml" \ + -f "$ROOT/docker/compose/ai-nvidia.yaml" \ + --env-file "$release_env" --env-file "$user_env" --profile ai config --quiet +docker compose --project-directory "$ROOT" \ + -f "$ROOT/docker/compose/main.yaml" \ + -f "$ROOT/docker/compose/ai-amd.yaml" \ + --env-file "$release_env" --env-file "$user_env" --profile ai config --quiet +docker compose --project-directory "$ROOT" \ + -f "$ROOT/docker/compose/main.yaml" \ + -f "$ROOT/docker/compose/ai-host-port.yaml" \ + --env-file "$release_env" --env-file "$user_env" --profile ai config --quiet resolved="$("${compose[@]}" --profile mysql config)" assert_contains "$resolved" "server-tools:" @@ -93,3 +106,52 @@ pass "shell override wins over user and release env files" # explicit rather than silently changing it in the compatibility batch. assert_contains "$resolved" "apache:" pass "current Apache compose presence characterized" + +core_json="$("${compose[@]}" config --format json)" +python3 -c 'import json,sys; d=json.load(sys.stdin); assert "llm-sm" not in d.get("services", {})' <<<"$core_json" +pass "AI provider remains outside the default profile" + +ai_json="$("${compose[@]}" --profile ai config --format json)" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +s=d["services"]["llm-sm"] +assert s["image"] == "infocyph/llm-sm:0.03" +assert "container_name" not in s +assert not s.get("ports") +assert set(s["networks"]) == {"frontend","backend"} +targets={v["target"] for v in s["volumes"]} +assert targets == {"/root/.ollama"} +assert d["volumes"]["lds_llm"]["name"] == "LLMModels" +tools=d["services"]["server-tools"]["environment"] +assert tools["LDS_AI_ENABLED"] == "auto" +assert tools["LDS_AI_PROVIDER"] == "ollama" +assert tools["LDS_AI_URL"] == "http://llm-sm:11434" +assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" +' <<<"$ai_json" +pass "base AI profile is internal-only and deterministic" + +amd_json="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-amd.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config --format json)" +python3 -c ' +import json,sys +s=json.load(sys.stdin)["services"]["llm-sm"] +assert s["image"] == "infocyph/llm-sm:amd-0.03" +devices=" ".join(str(x) for x in s.get("devices", [])) +assert "/dev/kfd" in devices and "/dev/dri" in devices +' <<<"$amd_json" +pass "AMD AI override" + +nvidia_yaml="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-nvidia.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config)" +assert_contains "$nvidia_yaml" "gpus:" +pass "NVIDIA AI override" + +host_json="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-host-port.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config --format json)" +python3 -c ' +import json,sys +ports=json.load(sys.stdin)["services"]["llm-sm"]["ports"] +assert len(ports) == 1 +p=ports[0] +assert p["host_ip"] == "127.0.0.1" +assert int(p["target"]) == 11434 and int(p["published"]) == 11434 +' <<<"$host_json" +pass "direct Ollama port is explicit loopback-only" From 866f6720c9542ac7c56d1e7ff20f9565e3ab76d2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:28:22 +0600 Subject: [PATCH 035/509] test: cover AI CLI and control state --- tests/env-contract.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index bab92ea1..2b9b676f 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -16,6 +16,10 @@ assert_file_contains "$ROOT/lds" 'local -a env_files=(--env-file "$ENV_RELEASE") assert_file_contains "$ROOT/lds" 'env_files+=(--env-file "$ENV_DOCKER")' assert_file_contains "$ROOT/lds" '"${env_files[@]}"' assert_file_contains "$ROOT/lds" 'var=COMPOSE_PROFILES' +assert_file_contains "$ROOT/lds" 'compose_control_value()' +assert_file_contains "$ROOT/lds" 'dotenv_value()' +assert_file_contains "$ROOT/lds" 'LDS_AI_RUNTIME cpu' +assert_file_contains "$ROOT/lds" 'LDS_LLM_HOST_PORT 0' pass "environment file and precedence wiring" git -C "$ROOT" check-ignore -q docker/.env || fail "docker/.env must remain ignored user state" @@ -42,3 +46,10 @@ for key in POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS; do grep -Fq "[$key]=" "$ROOT/lds" || fail "missing profile catalog entry: $key" done pass "current profile catalog entries" + +assert_file_contains "$ROOT/lds" 'cmd_ai()' +assert_file_contains "$ROOT/lds" 'cmd_llm()' +assert_file_contains "$ROOT/lds" '_tools_exec_argv()' +assert_file_contains "$ROOT/lds" 'ai) cmd_ai "$@"' +assert_file_contains "$ROOT/lds" 'llm) cmd_llm "$@"' +pass "AI/LLM CLI routing contract" From 698aa0d14c912d6044791e478d9fa1a8fba33b90 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:28:43 +0600 Subject: [PATCH 036/509] test: extend fake Ollama version endpoint --- tests/fixtures/fake-ollama/server.py | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/fixtures/fake-ollama/server.py b/tests/fixtures/fake-ollama/server.py index 40c67098..9b7baf72 100644 --- a/tests/fixtures/fake-ollama/server.py +++ b/tests/fixtures/fake-ollama/server.py @@ -14,6 +14,10 @@ def _json(self, status, payload): self.wfile.write(body) def do_GET(self): + if self.path == "/api/version": + self._json(200, {"version": "0.0.0-ci"}) + return + if self.path == "/api/tags": self._json( 200, From 8e0bfc8032cae1fe9bc043ef15ca26093c651da8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:28:45 +0600 Subject: [PATCH 037/509] test: validate Tools to llm-sm provider integration --- tests/ai-contract.sh | 24 ++++++++++++++++++++++-- 1 file changed, 22 insertions(+), 2 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 607ee689..6c07ef0e 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -7,14 +7,17 @@ source "$ROOT/tests/lib/assertions.sh" image="lds-fake-ollama:ci" container="lds-fake-ollama-ci" +network="lds-ai-ci" cleanup() { docker rm -f "$container" >/dev/null 2>&1 || true + docker network rm "$network" >/dev/null 2>&1 || true } trap cleanup EXIT +docker network create "$network" >/dev/null docker build -q -t "$image" "$ROOT/tests/fixtures/fake-ollama" >/dev/null -docker run -d --name "$container" "$image" >/dev/null +docker run -d --name "$container" --network "$network" --network-alias llm-sm "$image" >/dev/null for _ in {1..20}; do if docker exec "$container" python -c 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=1).read()' >/dev/null 2>&1; then @@ -37,5 +40,22 @@ models="$( docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/v1/models", timeout=2).read().decode())' )" assert_contains "$models" "qwen2.5:3b" - pass "fake Ollama tags/generate/OpenAI-compatible contracts" + +docker pull infocyph/tools:0.23.2 >/dev/null +provider_status="$( + docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-sm:11434 -e LDS_AI_MODEL=qwen2.5:3b infocyph/tools:0.23.2 --status +)" +assert_contains "$provider_status" "available=1" +assert_contains "$provider_status" "model=qwen2.5:3b" +pass "Tools 0.23.2 reaches the separate provider contract" + +for file in ai.yaml ai-nvidia.yaml ai-amd.yaml ai-host-port.yaml; do + assert_file "$ROOT/docker/compose/$file" +done +assert_file_contains "$ROOT/docker/compose/ai.yaml" 'profiles: [ai]' +assert_file_contains "$ROOT/docker/compose/ai.yaml" 'lds_llm:/root/.ollama' +if grep -Eq '/var/run/docker.sock|PROJECT_DIR|/app' "$ROOT/docker/compose/ai.yaml"; then + fail "base llm-sm service must not receive Docker socket or project mounts" +fi +pass "base AI service trust boundary" From b6cdfc7ac83767717f145641020092f1a8416648 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:30:00 +0600 Subject: [PATCH 038/509] ai: repair safe dotenv control parser --- lds | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/lds b/lds index c683ea6c..db1f0cb9 100755 --- a/lds +++ b/lds @@ -308,7 +308,7 @@ COMPOSE_FILE="$CFG/compose/main.yaml" EXTRAS_DIR="$DIR/configuration/compose" # Read a single dotenv value without sourcing/evaluating the file. -# This is intentionally limited to literal KEY=VALUE state used by LDS itself. +# Only literal KEY=VALUE state is accepted; nothing is executed. dotenv_value() { local file="${1:-}" key="${2:-}" line value first last [[ -r "$file" && -n "$key" ]] || return 1 @@ -317,7 +317,37 @@ dotenv_value() { [[ -n "$line" ]] || return 1 value="${line#*=}" - value="${value% + if (("${#value}" >= 2)); then + first="${value:0:1}" + last="${value: -1}" + if [[ "$first" == '"' && "$last" == '"' ]]; then + value="${value:1:${#value}-2}" + elif [[ "$first" == "'" && "$last" == "'" ]]; then + value="${value:1:${#value}-2}" + fi + fi + printf '%s' "$value" +} + +# Resolution mirrors Compose intent: shell > user docker/.env > release.env > fallback. +compose_control_value() { + local key="${1:?}" fallback="${2-}" value + if [[ -v "$key" ]]; then + printf '%s' "${!key}" + return 0 + fi + if value="$(dotenv_value "$ENV_DOCKER" "$key")"; then + printf '%s' "$value" + return 0 + fi + if value="$(dotenv_value "$ENV_RELEASE" "$key")"; then + printf '%s' "$value" + return 0 + fi + printf '%s' "$fallback" +} + +if [[ "${1:-}" == "--__win_workdir" ]]; then export WORKDIR_WIN="${2:-}" shift 2 fi From a85ebc9f43a910df0922cca168fed99d2eafc9b8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:31:02 +0600 Subject: [PATCH 039/509] ai: remove stale parser tail --- lds | 5020 ----------------------------------------------------------- 1 file changed, 5020 deletions(-) diff --git a/lds b/lds index db1f0cb9..294936a0 100755 --- a/lds +++ b/lds @@ -1959,5023 +1959,3 @@ cmd_host() { ;; rm | remove | del | delete) delete_domain "$@" - ;; - list) - shopt -s nullglob - for f in "$DIR/configuration/nginx/"*.conf; do - printf '%s -' "$(basename -- "$f" .conf)" - done - shopt -u nullglob - ;; - *) - die "host " - ;; - esac -} - -cmd_ui() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" lazydocker -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6f. EXEC / EVENTS / CLEAN / DISK -# ───────────────────────────────────────────────────────────────────────────── -cmd_exec() { - local svc="${1:-}" - shift || true - [[ -n "$svc" ]] || die "exec [cmd...]" - local s - s="$(resolve_service "$svc" || true)" - [[ -n "$s" ]] || die "Unknown service: $svc" - if [[ $# -gt 0 ]]; then - docker_compose exec "$s" "$@" - else - docker_compose exec "$s" sh -lc 'command -v bash >/dev/null 2>&1 && exec bash || exec sh' - fi -} - -cmd_events() { - local since="${1:-1h}" - local project - project="$(lds_project)" - docker events --since "$since" --filter "label=com.docker.compose.project=$project" -} - -cmd_clean() { - local yes=0 vols=0 - while [[ "${1:-}" ]]; do - case "$1" in - --yes | -y) - yes=1 - shift - ;; - --volumes | -v) - vols=1 - shift - ;; - *) - die "clean [--yes|-y] [--volumes|-v]" - ;; - esac - done - - ((yes)) || die "clean requires --yes" - - printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" - docker container prune -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" - docker network prune -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" - docker image prune -a -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" - docker builder prune -a -f >/dev/null 2>&1 || true - - if ((vols)); then - printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" - docker volume prune -f >/dev/null 2>&1 || true - fi - - printf "%b[clean]%b done\n" "$GREEN" "$NC" -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6g. HELP MARKDOWN -# ───────────────────────────────────────────────────────────────────────────── - -normalize_service() { - local raw="${1:-}" - local s="${raw//[[:space:]]/}" - [[ -n "$s" ]] || { - printf '%s' "" - return 0 - } - - local low="${s,,}" - - local key="${low//_/}" - key="${key//-/}" - if [[ "$key" =~ ^php ]]; then - local ver="${key#php}" - ver="${ver//[^0-9]/}" - if [[ "$ver" =~ ^([0-9])([0-9]).* ]]; then - printf 'php%s%s' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" - return 0 - fi - printf 'php' - return 0 - fi - - low="${low//_/-}" - while [[ "$low" == *"--"* ]]; do low="${low//--/-}"; done - printf '%s' "$low" -} - -cmd_rebuild() { - local -a targets=() all_svcs=() - local arg svc img - declare -A seen=() - - # ----------------------------- - # helper: add a service once - # ----------------------------- - _add_target() { - local s="$1" - [[ -n "$s" ]] || return 0 - [[ -n "${seen[$s]:-}" ]] && return 0 - seen[$s]=1 - targets+=("$s") - } - - # ----------------------------- - # helper: trim - # ----------------------------- - _trim() { - local s="$1" - s="${s#"${s%%[![:space:]]*}"}" - s="${s%"${s##*[![:space:]]}"}" - printf '%s' "$s" - } - - # ----------------------------- - # helper: interactive selection (comma separated, supports ranges) - # accepts: "all" or "1,3,5-7" or mix with names "nginx,2,5-6" - # ----------------------------- - _pick_targets_interactive() { - compose_services_load - all_svcs=("${__COMPOSE_SVCS[@]}") - if ((${#all_svcs[@]})); then - mapfile -t all_svcs < <(printf '%s\n' "${all_svcs[@]}" | LC_ALL=C sort -f -u) - fi - [[ ${#all_svcs[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" - - echo - echo "Select services to rebuild (comma separated; ranges allowed)." - echo "Examples: 1,3,5-7 | nginx,2,5-6 | all" - echo - - local i - for i in "${!all_svcs[@]}"; do - printf " %2d) %s\n" "$((i + 1))" "${all_svcs[$i]}" - done - - echo - local sel - read -r -p "Pick: " sel - sel="$(_trim "${sel:-}")" - [[ -n "$sel" ]] || die "No selection provided." - - if [[ "${sel,,}" == "all" ]]; then - for svc in "${all_svcs[@]}"; do _add_target "$svc"; done - return 0 - fi - - # split by comma - local IFS=, - for arg in $sel; do - arg="$(_trim "$arg")" - [[ -n "$arg" ]] || continue - - # range like 3-7 - if [[ "$arg" =~ ^[0-9]+-[0-9]+$ ]]; then - local a b - a="${arg%-*}" - b="${arg#*-}" - ((a >= 1)) || continue - ((b >= 1)) || continue - ((a <= b)) || { - local t="$a" - a="$b" - b="$t" - } - - local n - for ((n = a; n <= b; n++)); do - ((n >= 1 && n <= ${#all_svcs[@]})) || continue - _add_target "${all_svcs[$((n - 1))]}" - done - continue - fi - - # single index - if [[ "$arg" =~ ^[0-9]+$ ]]; then - local n="$arg" - ((n >= 1 && n <= ${#all_svcs[@]})) || continue - _add_target "${all_svcs[$((n - 1))]}" - continue - fi - - # treat as service/container name - svc="$(resolve_service "$arg")" - [[ -n "$svc" ]] && _add_target "$svc" - done - - [[ ${#targets[@]} -gt 0 ]] || die "No valid services selected." - } - - # ----------------------------- - # build target list - # ----------------------------- - if (($# == 0)); then - _pick_targets_interactive - elif [[ "${1,,}" == "all" ]]; then - compose_services_load - targets=("${__COMPOSE_SVCS[@]}") - [[ ${#targets[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" - else - for arg in "$@"; do - svc="$(resolve_service "$arg")" - [[ -n "$svc" ]] || continue - _add_target "$svc" - done - [[ ${#targets[@]} -gt 0 ]] || die "No valid services provided." - fi - - # ----------------------------- - # rebuild each target - # ----------------------------- - for svc in "${targets[@]}"; do - [[ -n "$svc" ]] || continue - compose_service_exists "$svc" || die "Unknown service/container: '$svc'" - - if compose_has_build "$svc"; then - logq rebuild "build/recreate $svc" - dc_build --no-cache --pull "$svc" - dc_up -d --no-deps --force-recreate "$svc" - continue - fi - - img="$(compose_image_for_service "$svc")" - logq rebuild "pull/recreate $svc${img:+ ($img)}" - - docker_compose rm -sf "$svc" >/dev/null 2>&1 || true - - if [[ -n "${img:-}" ]]; then - docker rmi -f "$img" >/dev/null 2>&1 || true - dc_pull "$svc" || true - else - dc_build --no-cache --pull "$svc" >/dev/null 2>&1 || true - fi - - dc_up -d --no-deps --force-recreate "$svc" - done - logq reboot "Rebooting stacks" - cmd_reboot -} - -cmd_config() { docker_compose config; } - -docker_shell() { - local c="${1:-}" - [[ -n "$c" ]] || die "container name required" - if docker exec "$c" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then - exec docker exec -it "$c" bash - else - exec docker exec -it "$c" sh - fi -} -cmd_tools() { - local sub="${1:-sh}" - shift || true - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - case "${sub,,}" in - sh | shell | "") - docker_shell "$ctr" - ;; - exec) - [[ $# -gt 0 ]] || die "tools exec " - docker exec -it "$ctr" sh -lc "$*" - ;; - file) - local p="${1:-}" - [[ -n "$p" ]] || die "tools file " - docker exec -it "$ctr" sh -lc "ls -la -- \"$p\" 2>/dev/null || true; echo; sed -n '1,200p' -- \"$p\" 2>/dev/null || true" - ;; - *) - die "tools " - ;; - esac -} -cmd_http() { [[ ${1:-} == reload ]] && http_reload; } -cmd_cli() { - local ctr="${1:-}" - shift || true - - [[ -n "$ctr" ]] || die "Usage: lds cli [cmd...]" - - docker inspect "$ctr" >/dev/null 2>&1 || die "Container not found: $ctr" - docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "Container not running: $ctr" - - # If user provided a command, run it; otherwise open an interactive shell. - if [[ "$#" -gt 0 ]]; then - local cmd="$*" - docker exec -it "$ctr" sh -lc ' - if command -v bash >/dev/null 2>&1; then - exec bash --login -lc "$1" - fi - exec sh -lc "$1" - ' sh "$cmd" - return - fi - - docker exec -it "$ctr" sh -lc ' - if command -v bash >/dev/null 2>&1; then - exec bash --login - fi - exec sh - ' -} - -cmd_core() { - # Usage: - # lds core -> open correct container for that domain (PHP/Node) - # lds core -> open a shell in that container - # lds core -> list domains and let user pick - - local target="${1:-}" - - # domain regex (same as domain-which/mkhost family) - local re='^([a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9])?\.)+(localhost|local|test|loc|[a-zA-Z]{2,})$' - - # If no target -> prompt from domain-which list - if [[ -z "$target" ]]; then - local tools_ctr - tools_ctr="$(_project_tools_container_running || true)" - [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - local -a domains=() - mapfile -t domains < <(docker exec "$tools_ctr" domain-which --list-domains 2>/dev/null | sed '/^[[:space:]]*$/d' || true) - - ((${#domains[@]} > 0)) || die "No domains found" - - # stable ordering - IFS=$'\n' domains=($(printf '%s\n' "${domains[@]}" | LC_ALL=C sort -u)) - - if ((${#domains[@]} == 1)); then - target="${domains[0]}" - else - if [[ ! -t 0 ]]; then - printf "%b[core]%b No domain provided. Available domains:\n" "$YELLOW" "$NC" >&2 - local i=1 - local d - for d in "${domains[@]}"; do - printf " %2d) %s\n" "$i" "$d" >&2 - ((i++)) - done - die "No TTY to prompt. Use: lds core " - fi - - printf "%bSelect domain:%b\n" "$CYAN" "$NC" >&2 - local i=1 d - for d in "${domains[@]}"; do - printf " %b%2d)%b %s\n" "$CYAN" "$i" "$NC" "$d" >&2 - ((i++)) - done - - local ans="" - while true; do - read -r -p "Enter number (1-${#domains[@]}): " ans - ans="$(echo "$ans" | xargs)" - [[ "$ans" =~ ^[0-9]+$ ]] || { - printf "%bInvalid input.%b\n" "$YELLOW" "$NC" >&2 - continue - } - ((ans >= 1 && ans <= ${#domains[@]})) || { - printf "%bOut of range.%b\n" "$YELLOW" "$NC" >&2 - continue - } - target="${domains[$((ans - 1))]}" - break - done - fi - fi - - # If target looks like a domain -> resolve via domain-which then shell in - if [[ "$target" =~ $re ]]; then - local tools_ctr - tools_ctr="$(_project_tools_container_running || true)" - [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - local app container wd - app="$(docker exec "$tools_ctr" domain-which --app --quiet "$target" 2>/dev/null)" || die "Unknown domain: $target" - container="$(docker exec "$tools_ctr" domain-which --container --quiet "$target" 2>/dev/null)" || die "No container resolved for: $target" - wd="$(docker exec "$tools_ctr" domain-which --docroot --quiet "$target" 2>/dev/null)" || true - [[ -n "${container:-}" ]] || die "No container resolved for: $target" - - # Node apps should always land at /app. Others follow resolved docroot. - if [[ "${app:-}" == "node" ]]; then - wd="/app" - fi - [[ -n "${wd:-}" ]] || wd="/app" - - docker exec -it "$container" bash -lc "cd \"$wd\" 2>/dev/null || cd /app 2>/dev/null || cd /; exec bash" - return 0 - fi - - # Otherwise treat target as a container name - docker exec -it "$(printf '%s' "$target" | tr '[:lower:]' '[:upper:]')" sh -lc 'exec bash -i || exec sh' -} - -cmd_setup() { - add_required_env - case ${1:-} in - init) env_init ;; - permission | permissions | perms | perm) fix_perms ;; - domain) setup_domain ;; - profiles | profile) process_all ;; - *) die "setup " ;; - esac -} - -cmd_certificate() { - case ${1:-} in - install) - shift || true - install_ca - ;; - uninstall | remove | rm) - shift || true - uninstall_ca "${@:-}" - ;; - *) - die "certificate " - ;; - esac -} - -############################################################################### -# NOTIFY -############################################################################### -notify_watch() { - local container="${1:-}" - if [[ -z "$container" ]]; then - container="$(_project_tools_container_running || true)" - [[ -n "$container" ]] || die "server-tools container is not running for project: $(lds_project)" - fi - local prefix="__HOST_NOTIFY__" - - need docker - - local _disp="${DISPLAY-}" - local _dbus="${DBUS_SESSION_BUS_ADDRESS-}" - - # Args: timeout(ms) urgency title body - _host_notify() { - local timeout="${1:-2500}" urgency="${2:-normal}" title="${3:-Notification}" body="${4:-}" - - # Linux desktop (or WSLg) - if has_cmd notify-send; then - (env DISPLAY="${_disp-}" DBUS_SESSION_BUS_ADDRESS="${_dbus-}" \ - setsid -f notify-send -u "$urgency" -t "$timeout" "$title" "$body" \ - >/dev/null 2>&1 || true) & - return 0 - fi - - # Windows toast (Git Bash) / WSL-on-Windows - if has_cmd powershell.exe; then - # Pass values as args to avoid quoting issues entirely. - # Note: urgency/timeout not used by toast api here; kept for parity. - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command \ - 'param([string]$t,[string]$b) - try { - [Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null - [Windows.Data.Xml.Dom.XmlDocument, Windows.Data.Xml.Dom.XmlDocument, ContentType = WindowsRuntime] > $null - - function Esc([string]$s) { - if ($null -eq $s) { return "" } - return ($s -replace "&","&" -replace "<","<" -replace ">",">" -replace "\"",""" -replace "'\''","'") - } - - $title = Esc $t - $body = Esc $b - - $xml = New-Object Windows.Data.Xml.Dom.XmlDocument - $xml.LoadXml("$title$body") - $toast = New-Object Windows.UI.Notifications.ToastNotification $xml - [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier("Devtainer").Show($toast) - } catch { }' \ - --% "$title" "$body" >/dev/null 2>&1 || true - - return 0 - fi - - # Fallback - printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 - return 0 - } - - trap - ERR - set +e - set +o pipefail - - local _stop=0 - - _watcher_notify() { - local urgency="${1:-critical}" title="${2:-Notifier}" body="${3:-Watcher event}" - _host_notify 2500 "$urgency" "$title" "$body" - } - - _watcher_int_term() { - _stop=1 - _watcher_notify critical "Notifier" "Notification watcher interrupted/exiting" - printf "%b[watcher]%b Notification watcher interrupted/exiting\n" "$RED" "$NC" >&2 - } - trap _watcher_int_term INT TERM - - local grep_cmd=(grep -a --line-buffered -E "^${prefix}([[:space:]]|$)") - has_cmd stdbuf && grep_cmd=(stdbuf -oL -eL "${grep_cmd[@]}") - - printf "%bNotify Watch:%b monitoring is active. Ctrl+C to stop.\n" "$GREEN" "$NC" - - while ((_stop == 0)); do - if ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then - _watcher_notify critical "Notifier" "Watcher stopped: $container is not running" - printf "%b[watcher]%b %s is not running; exiting.\n" "$RED" "$NC" "$container" >&2 - break - fi - - docker logs -f --tail 0 "$container" 2>&1 | - ("${grep_cmd[@]}" || true) | - while IFS=$'\t' read -r _ f1 f2 f3 f4 rest; do - local timeout urgency title body - - if [[ "${f1:-}" =~ ^[0-9]{1,6}$ ]]; then - timeout="$f1" - urgency="${f2:-normal}" - title="${f3:-Notification}" - body="${f4:-}" - else - timeout="2500" - urgency="${f1:-normal}" - title="${f2:-Notification}" - body="${f3:-}" - fi - - [[ -n "${rest:-}" ]] && body+=$'\t'"${rest}" - case "$urgency" in low | normal | critical) ;; *) urgency="normal" ;; esac - - _host_notify "$timeout" "$urgency" "$title" "$body" - printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 - done - - ((_stop)) && break - - if docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then - _watcher_notify critical "Notifier" "Watcher lost log stream (docker logs ended). Reconnecting…" - printf "%b[watcher]%b docker logs ended; reconnecting...\n" "$YELLOW" "$NC" >&2 - sleep 1 - continue - fi - - _watcher_notify critical "Notifier" "Watcher stopped: $container stopped" - printf "%b[watcher]%b %s stopped; exiting.\n" "$RED" "$NC" "$container" >&2 - break - done - - trap - INT TERM - set -euo pipefail - - ((_stop)) && return 130 - return 0 -} - -notify_test() { - local title="${1:-Notifier OK}" - local body="${2:-Hello from host via project server-tools container}" - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" notify -t 2500 -u normal "$title" "$body" -} - -cmd_notify() { - case ${1:-watch} in - watch) notify_watch "${2:-}" ;; - test) notify_test "${2:-Notifier OK}" "${3:-Hello from host}" ;; - *) die "notify " ;; - esac -} - -open_url() { - local url="${1:-}" - [[ -n "$url" ]] || return 0 - - # WSL/Windows helpers first when available - if grep -qi microsoft /proc/version 2>/dev/null; then - if has_cmd powershell.exe; then - powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 || true - return 0 - fi - if has_cmd cmd.exe; then - cmd.exe /c start "" "$url" >/dev/null 2>&1 || true - return 0 - fi - fi - - if has_cmd xdg-open; then - (xdg-open "$url" >/dev/null 2>&1 &) - return 0 - fi - if has_cmd open; then - (open "$url" >/dev/null 2>&1 &) - return 0 - fi - if has_cmd powershell; then - (powershell -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 &) - return 0 - fi - - printf "%bINFO%b: open this URL manually → %s\n" "$YELLOW" "$NC" "$url" -} - -############################################################################### -# RUN (ad-hoc Dockerfile runner) -############################################################################### -hash_short() { - local s="$1" - if has_cmd sha1sum; then - printf '%s' "$s" | sha1sum | cut -c1-8 - elif has_cmd shasum; then - printf '%s' "$s" | shasum -a 1 | cut -c1-8 - else - # POSIX fallback; stable (not cryptographic) - printf '%s' "$s" | cksum | awk '{print $1}' - fi -} - -run_slug() { - local dir="$1" base hash - base="$(basename "$dir" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9._-' '-')" - hash="$(hash_short "$dir")" - printf '%s-%s' "$base" "$hash" -} - -run_plan() { - local dir="$1" slug - slug="$(run_slug "$dir")" - printf '%s|%s|%s\n' \ - "lds-run-${slug}" \ - "${slug}:local" \ - "$dir" -} - -detect_host_os() { - if is_windows_shell || grep -qi microsoft /proc/version 2>/dev/null; then - printf 'windows' - return 0 - fi - - if has_cmd uname; then - case "$(uname -s 2>/dev/null || true)" in - Darwin) - printf 'macos' - ;; - Linux) - printf 'linux' - ;; - *) - printf 'unknown' - ;; - esac - else - printf 'unknown' - fi -} - -run_find_container() { - local dir="$1" - docker ps -a --filter "label=com.infocyph.lds.run=1" \ - --filter "label=com.infocyph.lds.dir=${dir}" \ - --format '{{.Names}}' | head -n 1 -} - -run_build() { - local tag="$1" dir="$2" - - # Build only if the image doesn't already exist. - if docker image inspect "$tag" >/dev/null 2>&1; then - printf "%b[run]%b Image exists, skipping build: %b%s%b -" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" - return 0 - fi - - printf "%b[run]%b Building image %b%s%b from %s -" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" "$dir" - docker build -t "$tag" "$dir" -} - -run_start() { - local name="$1" tag="$2" dir="$3" keepalive="$4" sock="$5" host_os="$6" - shift 6 || true - - # Remaining args are split by a "--" sentinel: - # - before "--" : publish specs (HOST:CONT), repeatable - # - after "--" : mount specs (HOST[:CONT]), repeatable - local -a pubs=() mounts=() - local seen_delim=0 x - for x in "$@"; do - if [[ "$x" == "--" ]]; then - seen_delim=1 - continue - fi - if ((seen_delim)); then - mounts+=("$x") - else - pubs+=("$x") - fi - done - - # Normalize project dir (POSIX absolute). - # On Windows Git Bash, /e/... is OK; docker.exe will receive converted path automatically. - local dir_posix - dir_posix="$(cd "$dir" 2>/dev/null && pwd -P)" || die "invalid dir: $dir" - - # MSYS-safe container paths: use '//' prefix to prevent path conversion. - # Docker interprets //path as /path inside container. - local WDIR="//workspace" - local WDIR_MOUNT="${dir_posix}://workspace" - - local -a args=(docker run -d --name "$name" - --label "com.infocyph.lds.run=1" - --label "com.infocyph.lds.dir=$dir_posix" - --label "com.infocyph.lds.tag=$tag" - -w "$WDIR" - -v "$WDIR_MOUNT" - ) - - if [[ -n "$host_os" ]]; then - args+=(-e "HOST_OS=$host_os") - fi - - # Mount extra directories/files (HOST[:CONT]). - # - If container path missing, mounts under /mnt/. - # - HOST may be relative to the run directory. - if ((${#mounts[@]})); then - local spec host cont base - for spec in "${mounts[@]}"; do - [[ -n "$spec" ]] || continue - host="$spec" - cont="" - - # Split as HOST:CONT ONLY if suffix after last ':' looks like a container absolute path (/...) - # (safe for Windows drive letters like E:\... because tail won't start with '/') - if [[ "$spec" == *:* ]]; then - local tail="${spec##*:}" - if [[ "$tail" == /* ]]; then - host="${spec%:*}" - cont="$tail" - fi - fi - - # Resolve host to absolute (POSIX) for checks - if [[ "$host" != /* && "$host" != ~* && ! "$host" =~ ^[A-Za-z]:[\\/].* ]]; then - host="${dir_posix%/}/$host" - fi - - # If user provided Windows path (E:\...), convert to POSIX for existence check - if [[ "$host" =~ ^[A-Za-z]:[\\/].* ]] && has_cmd cygpath; then - host="$(cygpath -u "$host")" - fi - - host="$(cd "${host%/*}" 2>/dev/null && pwd -P)/${host##*/}" || { - printf "%b[run]%b Warning: cannot resolve mount path: %s\n" "$YELLOW" "$NC" "$spec" >&2 - continue - } - - [[ -e "$host" ]] || { - printf "%b[run]%b Warning: mount path does not exist: %s\n" "$YELLOW" "$NC" "$host" >&2 - continue - } - - if [[ -z "$cont" ]]; then - base="${host##*/}" - cont="/mnt/${base}" - fi - [[ "$cont" == /* ]] || cont="/mnt/${cont}" - - # Prevent MSYS conversion for container side by using '//' prefix - cont="//${cont#/}" - - args+=(-v "${host}:${cont}") - done - fi - - # Optional docker sock - if [[ "${sock:-0}" == 1 ]]; then - args+=(-v "/var/run/docker.sock:/var/run/docker.sock") - fi - - # Publish ports - local pub - for pub in "${pubs[@]}"; do - [[ -n "$pub" ]] || continue - args+=(-p "$pub") - done - - if [[ "$keepalive" == 1 ]]; then - # Keepalive mode replaces the image command; disable image healthcheck to avoid false "unhealthy". - args+=(--no-healthcheck --entrypoint sh "$tag" -c "trap : TERM INT; sleep infinity & wait") - else - args+=("$tag") - fi - - printf "%b[run]%b Starting container %b%s%b\n" "$CYAN" "$NC" "$BLUE" "$name" "$NC" - - # IMPORTANT: don't hide errors; if it fails, you need to see why - if ! "${args[@]}"; then - printf "%b[run]%b docker run failed.\n" "$RED" "$NC" >&2 - return 1 - fi - printf "\n" -} - -run_exec_shell() { - local name="$1" - if docker exec "$name" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then - exec docker exec -it "$name" bash - else - exec docker exec -it "$name" sh - fi -} - -cmd_run() { - local action="*" dir="$PWD" name="" tag="" nobuild=0 keepalive=1 sock=0 host_os="" - local -a publish=() mounts=() - local open_port="" open_path="/" open_proto="http" - - while [[ $# -gt 0 ]]; do - case "$1" in - stop | rm | ps | shell | logs | open | "*") - action="$1" - shift - ;; - build) - action="*" - shift - ;; - --name) - name="${2:-}" - shift 2 - ;; - --tag) - tag="${2:-}" - shift 2 - ;; - --no-build) - nobuild=1 - shift - ;; - --no-keepalive) - keepalive=0 - shift - ;; - --sock) - sock=1 - shift - ;; - --host-os) - host_os="${2:-}" - shift 2 - ;; - -p | --publish) - publish+=("${2:-}") - shift 2 - ;; - --mount) - mounts+=("${2:-}") - shift 2 - ;; - --port) - open_port="${2:-}" - shift 2 - ;; - --path) - open_path="${2:-/}" - shift 2 - ;; - --https) - open_proto="https" - shift - ;; - --http) - open_proto="http" - shift - ;; - *) break ;; - esac - done - - # Host path (POSIX) for planning/labels - local dir_posix - dir_posix="$(cd "$dir" && pwd -P)" - - if [[ -z "$host_os" ]]; then - host_os="${HOST_OS:-$(detect_host_os)}" - fi - - # Docker path (may need Windows form for docker.exe) - local dir_docker="$dir_posix" - - # Windows Git Bash/MSYS hardening: - # - stop MSYS rewriting container paths (/workspace -> D:/Program Files/Git/workspace) - # - but still feed docker.exe Windows-absolute host paths for build/run contexts - if is_windows_shell; then - export MSYS_NO_PATHCONV=1 - export MSYS2_ARG_CONV_EXCL='*' - if has_cmd cygpath; then - dir_docker="$(cygpath -w "$dir_posix")" - fi - fi - - # Plan/name/tag should be based on the real project identity (POSIX dir) - IFS='|' read -r def_name def_tag _def_dir < <(run_plan "$dir_posix") - name="${name:-$def_name}" - - # Tag rules: - # - Default tag is ":local" (from run_plan) - # - If user passes --tag without ":", append ":local" - if [[ -n "${tag:-}" ]]; then - if [[ "$tag" != *:* ]]; then - tag="${tag}:local" - fi - else - tag="$def_tag" - fi - - _find_for_dir() { - local found - found="$(run_find_container "$dir_posix" || true)" - if [[ -n "$found" ]]; then - printf '%s' "$found" - return 0 - fi - if docker inspect "$name" >/dev/null 2>&1; then - printf '%s' "$name" - return 0 - fi - return 1 - } - - _run_build_summary() { - local img="$1" build_dir="$2" cname="$3" - local tag_only="${img##*:}" - - printf "\n%b[run]%b Build summary\n" "$CYAN" "$NC" - printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" - printf " %bTag:%b %s\n" "$BOLD" "$NC" "$tag_only" - printf " %bDir:%b %s\n" "$BOLD" "$NC" "$build_dir" - printf " %bName:%b %s\n" "$BOLD" "$NC" "$cname" - printf " %bKeepalive:%b %s\n" "$BOLD" "$NC" "$keepalive" - printf " %bSock:%b %s\n" "$BOLD" "$NC" "$sock" - printf " %bHost OS:%b %s\n" "$BOLD" "$NC" "$host_os" - - if ((${#publish[@]})); then - printf " %bPublish:%b %s\n" "$BOLD" "$NC" "${publish[*]}" - else - printf " %bPublish:%b (none)\n" "$BOLD" "$NC" - fi - - if ((${#mounts[@]})); then - printf " %bMounts:%b %s\n" "$BOLD" "$NC" "${mounts[*]}" - else - printf " %bMounts:%b (none)\n" "$BOLD" "$NC" - fi - printf "\n" - } - - _run_runtime_summary() { - local cname="$1" - local id img state ports - id="$(docker inspect -f '{{.Id}}' "$cname" 2>/dev/null | cut -c1-12 || true)" - img="$(docker inspect -f '{{.Config.Image}}' "$cname" 2>/dev/null || true)" - state="$(docker inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || true)" - ports="$(docker port "$cname" 2>/dev/null | sed '/^[[:space:]]*$/d' | tr '\n' '; ' | sed 's/; $//' || true)" - - printf "%b[run]%b Runtime summary\n" "$CYAN" "$NC" - printf " %bContainer:%b %s\n" "$BOLD" "$NC" "${cname}${id:+ ($id)}" - [[ -n "$img" ]] && printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" - [[ -n "$state" ]] && printf " %bState:%b %s\n" "$BOLD" "$NC" "$state" - if [[ -n "$ports" ]]; then - printf " %bPorts:%b %s\n" "$BOLD" "$NC" "$ports" - else - printf " %bPorts:%b (none published)\n" "$BOLD" "$NC" - fi - printf "%b\n[run]%b Example Usage (in Composer)\n" "$CYAN" "$NC" - printf " %bimage:%b %s\n" "$BOLD" "$NC" "$img" - printf " %bpull_policy:%b never\n" "$BOLD" "$NC" - printf "\n" - } - - case "$action" in - ps) - docker ps -a --filter "label=com.infocyph.lds.run=1" \ - --format 'table {{.Names}} {{.Image}} {{.Status}} {{.Labels}}' - return 0 - ;; - stop) - local existing - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - docker stop "$existing" >/dev/null - printf "%b[run]%b Stopped %s\n" "$GREEN" "$NC" "$existing" - return 0 - ;; - logs) - local existing - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - exec docker logs -f "$existing" - ;; - open) - local existing line addr hp url - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - - [[ -n "$open_path" ]] || open_path="/" - [[ "$open_path" == /* ]] || open_path="/$open_path" - - if [[ -n "$open_port" ]]; then - line="$(docker port "$existing" "$open_port" 2>/dev/null | head -n 1 || true)" - [[ -n "$line" ]] || line="$(docker port "$existing" "${open_port}/tcp" 2>/dev/null | head -n 1 || true)" - else - line="$(docker port "$existing" 2>/dev/null | head -n 1 || true)" - fi - - if [[ -z "$line" ]]; then - printf "%b[run]%b No published ports found.\n" "$YELLOW" "$NC" - printf "%b[run]%b Tip: start with %blds run --publish 8025:8025%b then %blds run open%b\n" \ - "$YELLOW" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" - return 1 - fi - - addr="${line##*-> }" - hp="${addr##*:}" - url="${open_proto}://localhost:${hp}${open_path}" - open_url "$url" - printf "%b[run]%b Opened: %s\n" "$GREEN" "$NC" "$url" - return 0 - ;; - rm) - local existing img - existing="$(_find_for_dir)" || true - if [[ -n "${existing:-}" ]]; then - img="$(docker inspect -f '{{.Config.Image}}' "$existing" 2>/dev/null || true)" - docker stop "$existing" >/dev/null 2>&1 || true - docker rm "$existing" >/dev/null 2>&1 || true - printf "%b[run]%b Removed container %s\n" "$GREEN" "$NC" "$existing" - if [[ -n "${img:-}" ]]; then - docker rmi -f "$img" >/dev/null 2>&1 || true - printf "%b[run]%b Removed image %s\n" "$GREEN" "$NC" "$img" - fi - else - printf "%b[run]%b No container found for %s\n" "$YELLOW" "$NC" "$dir_posix" - fi - return 0 - ;; - shell | "*") - if ((nobuild == 0)); then - # Build needs docker.exe-friendly path on Windows - run_build "$tag" "$dir_docker" - else - printf "%b[run]%b Skipping build (--no-build)\n" "$YELLOW" "$NC" - fi - - _run_build_summary "$tag" "$dir_posix" "$name" - - if docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null | grep -q true; then - printf "%b[run]%b Container already running: %s\n\n" "$GREEN" "$NC" "$name" - else - if docker inspect "$name" >/dev/null 2>&1; then - docker rm -f "$name" >/dev/null 2>&1 || true - fi - - # Keep mounts as user gave them (POSIX/relative); run_start should validate POSIX - # and convert host-side to Windows only at docker run time. - run_start "$name" "$tag" "$dir_docker" "$keepalive" "$sock" "$host_os" \ - "${publish[@]}" -- "${mounts[@]}" - fi - - _run_runtime_summary "$name" - - # "shell" enters the container; "*" / "build" does not. - if [[ "$action" == "shell" ]]; then - run_exec_shell "$name" - else - printf "%b[run]%b Built/started. Use %blds run shell%b to enter, %blds run logs%b to follow logs.\n" \ - "$GREEN" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" - return 0 - fi - ;; - esac -} - -############################################################################### -# 6w. NEW FEATURES: stack diff | support trace -############################################################################### - -# stack diff: show what would run (compose) vs what's running (docker) -cmd_stack_diff() { - local json=0 - local show_config=0 - while [[ "${1:-}" ]]; do - case "$1" in - --json) - json=1 - shift - ;; - --config) - show_config=1 - shift - ;; - *) break ;; - esac - done - - local project - project="$(lds_project)" - local cfg_json="" - - if docker_compose config --format json >/dev/null 2>&1; then - cfg_json="$(docker_compose config --format json)" - else - # fallback: best-effort text config - cfg_json="" - fi - - # running: service -> image - declare -A running=() - local line - while IFS= read -r line; do - [[ -n "$line" ]] || continue - local svc="${line%%|*}" - local img="${line#*|}" - running["$svc"]="$img" - done < <(docker ps \ - --filter "label=com.docker.compose.project=$project" \ - --format '{{index .Labels "com.docker.compose.service"}}|{{.Image}}' 2>/dev/null || true) - - # desired: service -> image/build context (best-effort) - declare -A desired_img=() - declare -A desired_ctx=() - declare -A desired_df=() - - if [[ -n "$cfg_json" ]]; then - if has_tool jq; then - while IFS= read -r line; do - local svc="${line%%|*}" - local img="${line#*|}" - desired_img["$svc"]="$img" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"') - while IFS= read -r line; do - local svc="${line%%|*}" - local ctx="${line#*|}" - desired_ctx["$svc"]="$ctx" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"') - while IFS= read -r line; do - local svc="${line%%|*}" - local df="${line#*|}" - desired_df["$svc"]="$df" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"') - elif _server_tools_has jq; then - # Fallback: parse via project tools container jq through stdin (no shell re-quoting of JSON payload). - local ctr - ctr="$(_project_tools_container_running || true)" - if [[ -n "$ctr" ]]; then - while IFS= read -r line; do - local svc="${line%%|*}" - local img="${line#*|}" - desired_img["$svc"]="$img" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"' 2>/dev/null || true) - while IFS= read -r line; do - local svc="${line%%|*}" - local ctx="${line#*|}" - desired_ctx["$svc"]="$ctx" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"' 2>/dev/null || true) - while IFS= read -r line; do - local svc="${line%%|*}" - local df="${line#*|}" - desired_df["$svc"]="$df" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"' 2>/dev/null || true) - fi - fi - fi - - # Build result object - if ((json)); then - if has_tool jq; then - # assemble in bash -> jq - local tmp - tmp="$(mktemp)" - { - printf '{' - printf '"project":%s,' "$(printf '%s' "$project" | jq -Rsa .)" - printf '"compose_file":%s,' "$(printf '%s' "$COMPOSE_FILE" | jq -Rsa .)" - printf '"running":{' - local first=1 k - for k in "${!running[@]}"; do - ((first)) || printf ',' - first=0 - printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${running[$k]}" | jq -R .)" - done - printf '},' - printf '"desired":{' - first=1 - for k in "${!desired_img[@]}"; do - ((first)) || printf ',' - first=0 - printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${desired_img[$k]}" | jq -R .)" - done - printf '},' - printf '"diff":[' - first=1 - # union keys - declare -A seen=() - for k in "${!running[@]}"; do seen["$k"]=1; done - for k in "${!desired_img[@]}"; do seen["$k"]=1; done - for k in "${!seen[@]}"; do - local r="${running[$k]:-}" - local d="${desired_img[$k]:-}" - if [[ "$r" != "$d" ]]; then - ((first)) || printf ',' - first=0 - printf '{"service":%s,"running":%s,"desired":%s}' \ - "$(printf '%s' "$k" | jq -R .)" \ - "$(printf '%s' "$r" | jq -R .)" \ - "$(printf '%s' "$d" | jq -R .)" - fi - done - printf ']' - printf '}\n' - } >"$tmp" - cat "$tmp" | jq . - rm -f "$tmp" - else - die "jq required for --json (or run inside project server-tools container)" - fi - return 0 - fi - - printf "%bStack diff%b (project=%s)\n" "$CYAN" "$NC" "$project" - printf "%bCompose file:%b %s\n" "$DIM" "$NC" "$COMPOSE_FILE" - - if ((show_config)); then - if [[ -n "$cfg_json" ]]; then - printf "\n%bEffective compose config (json):%b\n" "$DIM" "$NC" - printf '%s\n' "$cfg_json" - else - printf "\n%bEffective compose config:%b\n" "$DIM" "$NC" - docker_compose config || true - fi - fi - - # union services - declare -A all=() - local svc - for svc in "${!running[@]}"; do all["$svc"]=1; done - for svc in "${!desired_img[@]}"; do all["$svc"]=1; done - - printf "\n%-22s %-40s %-40s %s\n" "SERVICE" "RUNNING" "DESIRED" "STATUS" - printf "%-22s %-40s %-40s %s\n" "------" "-------" "-------" "------" - for svc in $(printf '%s\n' "${!all[@]}" | sort); do - local r="${running[$svc]:-}" - local d="${desired_img[$svc]:-}" - local st - if [[ -z "$r" ]]; then - st="(not running)" - elif [[ -z "$d" ]]; then - st="(not in config)" - elif [[ "$r" == "$d" ]]; then - st="OK" - else - st="DIFF" - fi - printf "%-22s %-40.40s %-40.40s %s\n" "$svc" "$r" "$d" "$st" - done - - printf "\n%bNotes:%b\n" "$DIM" "$NC" - printf " - Desired image is derived from 'docker compose config'. If a service uses only 'build:' and no 'image:', desired may be empty.\n" - printf " - Use: lds stack diff --config (to print resolved compose config)\n" -} - -# support trace: quick end-to-end trace for a domain -cmd_support_trace() { - local dom="${1:-}" - [[ -n "$dom" ]] || die "support trace " - - local nconf="$DIR/configuration/nginx/$dom.conf" - printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" - - # 1) DNS - if _server_tools_running; then - printf "\n%b[DNS]%b\n" "$DIM" "$NC" - _tools_exec "dig +short $(_shq "$dom") || true; getent hosts $(_shq "$dom") 2>/dev/null || true" - else - printf "\n%b[DNS]%b\n" "$DIM" "$NC" - (has_cmd dig && dig +short "$dom") || true - (has_cmd getent && getent hosts "$dom") || true - fi - - # 2) TLS certificate - printf "\n%b[TLS]%b\n" "$DIM" "$NC" - if _server_tools_running; then - _tools_exec "echo | openssl s_client -connect $(_shq "$dom"):443 -servername $(_shq "$dom") -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true" - else - echo | openssl s_client -connect "${dom}:443" -servername "$dom" -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true - fi - - # 3) HTTP probe (timings) - printf "\n%b[HTTP]%b\n" "$DIM" "$NC" - if _server_tools_running; then - _tools_exec "curl -sk -o /dev/null -D - -w 'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' https://$(_shq "$dom") | sed -n '1,30p'" - else - curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' - fi - - # 4) Upstream inference from nginx conf (if exists) - printf "\n%b[Upstream]%b\n" "$DIM" "$NC" - if [[ -r "$nconf" ]]; then - if grep -q fastcgi_pass "$nconf"; then - local php - php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" - printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" - elif grep -q proxy_pass "$nconf"; then - local up - up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" - printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" - else - printf "type=static\n" - fi - else - printf "nginx_conf=%s (missing)\n" "$nconf" - fi - - # 5) Recent nginx logs (compose) - printf "\n%b[Recent nginx logs]%b\n" "$DIM" "$NC" - docker_compose logs --no-color --tail 120 nginx 2>/dev/null | text_grep -i "$dom" || docker_compose logs --no-color --tail 120 nginx 2>/dev/null || true - - printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" -} - -############################################################################### -# 6x. GROUPED COMMAND ROUTERS (stack/domain/support) + backward-compatible aliases -############################################################################### - -cmd_stack() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - "" | help | -h | --help) cmd_help stack ;; - up) cmd_up "$@" ;; - start) cmd_start "$@" ;; - down | stop) cmd_down "$@" ;; - restart | reboot) cmd_restart "$@" ;; - status) cmd_status "$@" ;; - ps) cmd_ps "$@" ;; - logs) cmd_logs "$@" ;; - exec) cmd_exec "$@" ;; - events) cmd_events "$@" ;; - clean) cmd_clean "$@" ;; - config) cmd_config "$@" ;; diff) cmd_stack_diff "$@" ;; - - http) cmd_http "$@" ;; - *) - die "stack " - ;; - esac -} - -# Canonical: domain. Legacy: host. -cmd_domain() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - "" | help | -h | --help) die "domain " ;; - add) cmd_host add "$@" ;; - rm | remove | del | delete) cmd_host rm "$@" ;; - ls | list) cmd_host list "$@" ;; - *) - die "domain " - ;; - esac -} - -cmd_support() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - "" | help | -h | --help) die "support " ;; - open) cmd_open "$@" ;; - bundle) cmd_bundle "$@" ;; - notify) cmd_notify "$@" ;; - ui) cmd_ui "$@" ;; - *) - die "support " - ;; - esac -} - -# Minimal support bundle (shareable diagnostics zip) -# Modes: -# --redact : remove secrets-like env lines from captured env/config outputs (best-effort) -# --full : include more logs/inspect output (can be large) -cmd_bundle() { - local mode="redact" - local out="${1:-}" - while [[ "${1:-}" ]]; do - case "$1" in - --redact) - mode="redact" - shift - ;; - --full) - mode="full" - shift - ;; - *.zip) - out="$1" - shift - ;; - *) break ;; - esac - done - - need zip - local ts - ts="$(date +%Y%m%d_%H%M%S)" - local project - project="$(lds_project)" - local tmp - tmp="$(mktemp -d "${TMPDIR:-/tmp}/lds_bundle.XXXXXX")" - local base="lds_bundle_${project}_${ts}" - [[ -n "$out" ]] || out="$PWD/${base}.zip" - - cleanup_bundle() { rm -rf "$tmp" 2>/dev/null || true; } - trap cleanup_bundle RETURN - - { - echo "project=$project" - echo "dir=$DIR" - echo "time=$ts" - echo "mode=$mode" - } >"$tmp/meta.txt" - - # compose config (effective) - { - echo "# docker compose config" - docker_compose config 2>&1 || true - } >"$tmp/compose.config.txt" - - # ps + networks - docker_compose ps >"$tmp/compose.ps.txt" 2>&1 || true - docker network ls >"$tmp/docker.networks.txt" 2>&1 || true - - # container list + inspect (scoped) - docker ps --filter "label=com.docker.compose.project=$project" --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}' \ - >"$tmp/docker.ps.txt" 2>&1 || true - - if [[ "$mode" == "full" ]]; then - docker inspect $(docker ps -q --filter "label=com.docker.compose.project=$project" 2>/dev/null) \ - >"$tmp/docker.inspect.json" 2>/dev/null || true - fi - - # recent logs (tail) - docker_compose logs --no-color --tail 400 >"$tmp/compose.logs.txt" 2>&1 || true - - # vhost configs - if [[ -d "$DIR/configuration/nginx" ]]; then - mkdir -p "$tmp/nginx" - cp -a "$DIR/configuration/nginx/." "$tmp/nginx/" 2>/dev/null || true - fi - - # env files (redacted best-effort) - mkdir -p "$tmp/env" - if [[ -r "$ENV_MAIN" ]]; then - if [[ "$mode" == "redact" ]]; then - sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_MAIN" >"$tmp/env/.env" - else - cp -a "$ENV_MAIN" "$tmp/env/.env" 2>/dev/null || true - fi - fi - if [[ -r "$ENV_DOCKER" ]]; then - if [[ "$mode" == "redact" ]]; then - sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_DOCKER" >"$tmp/env/docker.env" - else - cp -a "$ENV_DOCKER" "$tmp/env/docker.env" 2>/dev/null || true - fi - fi - - # tools-side quick diagnostics (inside network) - { - echo "# ip r / ip a / ss" - _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || true" - } >"$tmp/tools.net.txt" 2>&1 || true - - # pack - (cd "$tmp" && zip -qr "$out" .) || die "Failed to write bundle: $out" - ok "Bundle created: $out" -} - -cmd_help() { - - if [[ "${1:-}" == "--markdown" ]]; then - cat <<'MD' -# LocalDevStack (lds) — Command Reference - -## Stack (compose) -- `lds stack up` *(aliases: `up`)* -- `lds stack start` *(aliases: `start`)* -- `lds stack down [--volumes --yes]` *(aliases: `down`, `stop`)* -- `lds stack restart [svc]` *(aliases: `restart`, `reboot`)* -- `lds stack status [status-args…]` *(alias: `status`; forwards args to tools `status`)* -- `lds stack ps` *(alias: `ps`)* -- `lds stack logs [svc] [--follow] [--since ] [--grep ]` *(alias: `logs`)* -- `lds stack exec [cmd…]` *(alias: `exec`)* -- `lds stack events [--since ]` *(alias: `events`)* -- `lds stack clean --yes [--volumes]` *(alias: `clean`)* -- `lds stack diff [--config] [--json]` *(shows desired vs running images)* - - -## Domain (vhost lifecycle + routing) -- `lds domain add …` -- `lds domain rm …` -- `lds domain ls` - -Legacy alias: `lds host …` → same subcommands as `domain`. - -## Certificates (TLS) -- `lds cert status [domain|all]` -- `lds cert regen [domain|all] [--yes]` -- `lds cert diagnose ` -- `lds certificate install` -- `lds certificate uninstall [--all]` - -## Diagnostics -- `lds diag dns ` -- `lds diag net` -- `lds diag tcp ` -- `lds diag http ` *(alias: `sniff `)* -- `lds diag tls ` - -## Config -- `lds config show [--json]` -- `lds config services` -- `lds config profiles` -- `lds config env-used` -- `lds config validate` - -## Support -- `lds support open - support trace ` -- `lds support bundle [--redact|--full]` -- `lds support notify …` -- `lds support ui` - -Shortcuts: `open`, `bundle`, `notify`, `ui` map to `support …`. - -## Secrets (senv) -- `lds secrets …` - -## Tools (project server-tools container) -- `lds tools sh` -- `lds tools exec ""` -- `lds tools file ` - -## AI -- `lds ai status|ask|explain|troubleshoot|review|repo-review|graphify …` -- `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit …` -- `lds llm runtime ` -- `lds llm host-port ` - -## Setup -- `lds setup init|permissions|domain|profiles` - -## Runner (ad‑hoc Dockerfile runner) -- `lds run` *(default: build+start only)* / `lds run shell` *(build+start+enter)* / `lds run *` *(same as default)* (+ `ps|logs|stop|rm|open` and flags: `--publish|-p`, `--no-keepalive`, `--mount`, `--sock`, `--host-os`, `--tag`, `--name`) - -## Other -- `lds rebuild [all|]` -- `lds core [domain]` - -## Short client wrappers -- `lds php|composer|node|npm|npx …` -- `lds my|mysql|mysqldump …` -- `lds maria|mariadb|mariadb-dump …` -- `lds pg|psql|pg_dump|pg_restore …` -- `lds redis|redis-cli …` -MD - return 0 - fi - - cat < - clean Alias of: stack <...> - -${CYAN}Domain (vhosts + routing):${NC} - domain add|rm|ls - host add|rm|list Legacy alias of: domain <...> - -${CYAN}Certificates (TLS):${NC} - cert status|regen|diagnose - certificate install|uninstall [--all] - -${CYAN}Diagnostics:${NC} - diag dns|net|tcp|http|tls - sniff Alias of: diag http - -${CYAN}Config:${NC} - config show|services|profiles|env-used|validate - -${CYAN}Support:${NC} - support open - support trace - support bundle [--redact|--full] - support notify ... - support ui - open|bundle|notify|ui Shortcuts → support <...> - -${CYAN}Secrets:${NC} - secrets - -${CYAN}Tools (project server-tools container):${NC} - tools sh|exec|file - -${CYAN}AI:${NC} - ai status|ask|explain|troubleshoot|review|repo-review|graphify - llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit - llm runtime - llm host-port - -${CYAN}Setup:${NC} - setup init|permissions|domain|profiles - -${CYAN}Runner (ad-hoc Dockerfile runner):${NC} - run [ps|logs|stop|rm|open] [--publish|-p A:B] [--no-keepalive] [--mount HOST[:CONT]] [--sock] [--host-os VALUE] - -${CYAN}Other:${NC} - rebuild [all|] - core [domain] - -${CYAN}Help:${NC} - help [--markdown] - -EOF -} - -############################################################################### -# 7. MAIN -############################################################################### -main() { - [[ $# -gt 0 ]] || { - cmd_help - exit 1 - } - - while [[ $# -gt 0 ]]; do - case "$1" in - -v | --verbose) - VERBOSE=1 - QUIET=0 - shift - ;; - -q | --quiet) - QUIET=1 - VERBOSE=0 - shift - ;; - --reload-extras) - export LDS_EXTRAS_RELOAD=1 - shift - ;; - --) - shift - break - ;; - -h | --help) - cmd_help - exit 0 - ;; - -*) die "Unknown global option: $1" ;; - *) break ;; - esac - done - - [[ $# -gt 0 ]] || { - cmd_help - exit 1 - } - - local cmd="${1,,}" - shift || true - - if [[ "$cmd" == "help" ]]; then - cmd_help "$@" - exit 0 - fi - - # Do not require docker for pure help output; everything else expects the stack. - need docker - - case "$cmd" in - php | composer | node | npm | npx) exec "$DIR/bin/$cmd" "$@" ;; - pg | psql | pg_restore | pg-restore | pgrestore | pg_dump | pgdump | pg-dump) exec "$DIR/bin/pg" "$@" ;; - maria | mariadb | mariadbdump | mariadb-dump | mariadb_dump) exec "$DIR/bin/maria" "$@" ;; - my | mysql | mysqldump | mysql-dump | mysql_dump) exec "$DIR/bin/my" "$@" ;; - redis | redis-cli) exec "$DIR/bin/redis-cli" "$@" ;; - es | elastic | elasticsearch) exec "$DIR/bin/es" "$@" ;; - mongo | mongodb | mongosh | mongoimport | mongoexport) exec "$DIR/bin/mongo" "$@" ;; - ai) cmd_ai "$@" ;; - llm) cmd_llm "$@" ;; - vpn-fix) cmd_vpn_fix "$@" ;; - *) - if declare -F "cmd_$cmd" >/dev/null 2>&1; then - "cmd_$cmd" "$@" - else - exec "$DIR/bin/tool-runner" "$cmd" "$@" - fi - ;; - esac -} - -main "$@" -\r'}" - if (("${#value}" >= 2)); then - first="${value:0:1}" - last="${value: -1}" - if [[ ("$first" == '"' && "$last" == '"') || ("$first" == "'" && "$last" == "'") ]]; then - value="${value:1:${#value}-2}" - fi - fi - printf '%s' "$value" -} - -# Resolution mirrors Compose intent: shell > user docker/.env > release.env > fallback. -compose_control_value() { - local key="${1:?}" fallback="${2-}" value - if [[ -v "$key" ]]; then - printf '%s' "${!key}" - return 0 - fi - if value="$(dotenv_value "$ENV_DOCKER" "$key")"; then - printf '%s' "$value" - return 0 - fi - if value="$(dotenv_value "$ENV_RELEASE" "$key")"; then - printf '%s' "$value" - return 0 - fi - printf '%s' "$fallback" -} - -if [[ "${1:-}" == "--__win_workdir" ]]; then - export WORKDIR_WIN="${2:-}" - shift 2 -fi - -COLOR() { printf '[%sm' "$1"; } -############################################################################### -# Colors + UI (higher contrast; aligned with mkhost.sh) -############################################################################### -# Color control: -# - If stdout isn't a TTY, disable colors by default. -# - If NO_COLOR is set, disable colors. -# - Set LDS_FORCE_COLOR=1 to force colors. -_is_tty() { [[ -t 1 ]]; } - -_use_color=1 -if [[ "${LDS_FORCE_COLOR:-0}" != "1" ]]; then - if [[ -n "${NO_COLOR:-}" ]] || ! _is_tty; then - _use_color=0 - fi -fi - -if ((_use_color)); then - BOLD=$'\033[1m' - DIM=$'\033[2m' - RED=$'\033[1;31m' - GREEN=$'\033[1;32m' - CYAN=$'\033[1;36m' - YELLOW=$'\033[1;33m' - BLUE=$'\033[1;34m' - MAGENTA=$'\033[1;35m' - NC=$'\033[0m' -else - BOLD='' DIM='' RED='' GREEN='' CYAN='' YELLOW='' BLUE='' MAGENTA='' NC='' -fi - -# Output control: -# - --quiet suppresses non-error output -QUIET=0 - -say() { ((QUIET)) || printf '%b\n' "$*"; } -ok() { ((QUIET)) || printf '%b\n' "${GREEN}$*${NC}"; } -warn() { ((QUIET)) || printf '%b\n' "${YELLOW}$*${NC}"; } -err() { printf '%b\n' "${RED}$*${NC}" >&2; } - -# Default behavior: QUIET -VERBOSE=0 - -#─────────────────────────────────────────────────────────────────────────────── -# 0a. GLOBAL ERROR HANDLER -#─────────────────────────────────────────────────────────────────────────────── -command_not_found_handle() { - local unknown="$1" - [[ $unknown == cmd_* ]] && unknown=${unknown#cmd_} - printf "\n%bError:%b Unknown command '%b'\n\n" "$RED" "$NC" "$unknown" - cmd_help - exit 1 -} - -trap 'on_error $? $LINENO "$BASH_COMMAND"' ERR -on_error() { - local code="$1" line="$2" cmd="$3" - local fn="${FUNCNAME[1]:-main}" - local src="${BASH_SOURCE[1]:-$0}" - - printf "\n%bError:%b %s:%s in %s() (exit %d)\n" "$RED" "$NC" "$src" "$line" "$fn" "$code" >&2 - printf "%bCommand:%b %s\n" "$RED" "$NC" "$cmd" >&2 - - if ((VERBOSE)); then - printf "%bStack:%b\n" "$DIM" "$NC" >&2 - local i=1 - while caller "$i" >/dev/null 2>&1; do - caller "$i" >&2 - i=$((i + 1)) - done - fi - printf "\n" >&2 - exit "$code" -} - -############################################################################### -# 1a. DOCKER COMPOSE WRAPPER -############################################################################### - -# ── compose extras (docker/extras/*.y{a,}ml) ──────────────────────────────── -__EXTRAS_LOADED=0 -declare -a __EXTRA_FILES=() - -load_extras() { - # Set LDS_EXTRAS_RELOAD=1 (or global --reload-extras) to re-scan templates every call. - if [[ "${LDS_EXTRAS_RELOAD:-0}" == "1" ]]; then - __EXTRAS_LOADED=0 - fi - - ((__EXTRAS_LOADED)) && return 0 - __EXTRAS_LOADED=1 - - [[ -d "$EXTRAS_DIR" ]] || return 0 - - mapfile -t __EXTRA_FILES < <( - find "$EXTRAS_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -print 2>/dev/null | sort | sed '/^[[:space:]]*$/d' - ) -} - -docker_compose() { - load_extras - # Create required runtime files only when docker/compose operations are invoked. - ((EUID == 0)) || ensure_files_exist "/docker/.env" "/configuration/php/php.ini" "/.env" - if [[ -z "${__LDS_DC_BIN:-}" ]]; then - if docker compose version >/dev/null 2>&1; then - __LDS_DC_BIN=(docker compose) - else - __LDS_DC_BIN=(docker-compose) - fi - fi - - [[ -r "$ENV_RELEASE" ]] || die "Missing release compatibility manifest: $ENV_RELEASE" - - # Release defaults are loaded first; user docker/.env overrides them. - # Shell variables remain higher-precedence Compose interpolation inputs. - local -a env_files=(--env-file "$ENV_RELEASE") - [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - - # Build -f list (stable order; later overrides earlier) - local -a extra_f=() f - for f in "${__EXTRA_FILES[@]:-}"; do - [[ -f "$f" ]] || continue - case "$f" in - *.yml | *.yaml) extra_f+=(-f "$f") ;; - esac - done - - local host_os="${HOST_OS:-$(detect_host_os)}" - - HOST_OS="$host_os" "${__LDS_DC_BIN[@]}" \ - --project-directory "$DIR" \ - -f "$COMPOSE_FILE" \ - "${extra_f[@]}" \ - "${env_files[@]}" \ - "$@" -} - -# helper: print project name -lds_project() { printf '%s' "${__LDS_PROJECT:-$(basename -- "$DIR")}"; } - -# (QUIET by default) ──────────────────────────────── -# Centralize quiet/verbose handling for compose subcommands. -# Usage: dc_cmd [args...] -dc_cmd() { - local sub="${1:-}" - shift || true - - local -a quiet=() - if ((VERBOSE == 0)); then - case "$sub" in - up) quiet+=(--quiet-pull) ;; - pull) quiet+=(-q) ;; - build) quiet+=(--quiet) ;; - esac - fi - - docker_compose "$sub" "${quiet[@]}" "$@" -} - -dc_up() { dc_cmd up "$@"; } -dc_pull() { dc_cmd pull "$@"; } -dc_build() { dc_cmd build "$@"; } - -# helper for our own minimal logging (still shows in quiet mode) -logv() { ((VERBOSE)) && printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2 || true; } -logq() { printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2; } - -############################################################################### -# 1b. PROMPTS + DOTENV HELPERS -############################################################################### - -# Unified prompt helper (used by env_init + profiles) -tty_readline() { - # Robust prompt/read across Linux/macOS/WSL/Windows Git Bash. - # Prefer stdin when it is a TTY (normal interactive use). If stdin is not a TTY, - # fall back to /dev/tty when available. - local __var_name="$1" __prompt="$2" __line - - if [[ -t 0 ]]; then - # Interactive: show prompt on stderr (so it is never swallowed) and read stdin. - printf '%s' "$__prompt" >&2 - IFS= read -r __line || return 1 - elif [[ -r /dev/tty ]]; then - # Non-interactive stdin (piped) but we still have a controlling terminal. - printf '%s' "$__prompt" >/dev/tty - IFS= read -r __line keep as-is - if [[ "$v" =~ ^\".*\"$ || "$v" =~ ^\'.*\'$ ]]; then - printf '%s' "$v" - return 0 - fi - - # Leading/trailing whitespace or any internal whitespace or # or quotes => quote - if [[ "$v" =~ ^[[:space:]] || "$v" =~ [[:space:]]$ || "$v" == *$'\t'* || "$v" == *" "* || "$v" == *"#"* || "$v" == *"\""* ]]; then - env_quote "$v" - return 0 - fi - - printf '%s' "$v" -} - -# Escape replacement for sed (delimiter '|') -sed_escape_repl() { - local s=${1-} - s=${s//\\/\\\\} - s=${s//&/\\&} - s=${s//|/\\|} - printf '%s' "$s" -} - -update_env() { - local file=$1 var=$2 val=${3-} - mkdir -p "$(dirname "$file")" - [[ -f "$file" ]] || { - printf "%bFile '%s' not found. Creating one.%b\n" "$YELLOW" "$file" "$NC" - : >"$file" - } - - # Apply quoting only when needed (spaces etc.) - val="$(env_quote_if_needed "$val")" - - # Sed-safe replacement - local val_sed - val_sed="$(sed_escape_repl "$val")" - - var=$(echo "$var" | sed 's/[]\/$*.^|[]/\\&/g') - if grep -qE "^[# ]*$var=" "$file" 2>/dev/null; then - sed -Ei "s|^[# ]*($var)=.*|\1=$val_sed|" "$file" - else - printf "%s=%s\n" "$var" "$val" >>"$file" - fi -} - -############################################################################### -# 1c. HTTP / WEB SERVER HELPERS -############################################################################### - -http_reload() { - printf "%bReloading HTTP...%b" "$MAGENTA" "$NC" - docker ps -qf name=NGINX &>/dev/null && docker exec NGINX nginx -s reload &>/dev/null || true - docker ps -qf name=APACHE &>/dev/null && docker exec APACHE apachectl graceful &>/dev/null || true - printf "\r%bHTTP reloaded! %b\n" "$GREEN" "$NC" -} - -############################################################################### -# 2. INSTALL / PERMISSIONS (HOST) -############################################################################### -add_to_windows_path() { - [[ "$OSTYPE" =~ (msys|cygwin) ]] || return 0 - has_cmd cygpath || return 0 - - # Only add if lds.bat exists where we think it is - [[ -f "$DIR/lds.bat" ]] || return 0 - - local win_repo - win_repo="$(cygpath -w "$DIR")" - - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " - \$t = '$win_repo' - \$cur = [Environment]::GetEnvironmentVariable('Path','User') - if ([string]::IsNullOrWhiteSpace(\$cur)) { \$cur = '' } - - # Normalize (trim + case-insensitive compare) to avoid duplicates - \$parts = \$cur -split ';' | ForEach-Object { \$_.Trim() } | Where-Object { \$_ } - \$exists = \$false - foreach (\$p in \$parts) { if (\$p.ToLowerInvariant() -eq \$t.ToLowerInvariant()) { \$exists = \$true; break } } - - if (-not \$exists) { - \$new = (\$parts + \$t) -join ';' - [Environment]::SetEnvironmentVariable('Path', \$new, 'User') - } - " >/dev/null 2>&1 || true -} - -fix_perms() { - if [[ "$OSTYPE" =~ (msys|cygwin) ]]; then - add_to_windows_path - printf "%bWindows PATH configured.%b\n" "$GREEN" "$NC" - return 0 - fi - - ((EUID == 0)) || die "Please run with sudo." - - chmod 755 "$DIR" - chmod 2775 "$DIR/configuration" - find "$DIR/configuration" -type f ! -perm 664 -exec chmod 664 {} + - - chmod 755 "$DIR/docker" - find "$DIR/docker" -type f ! -perm 644 -exec chmod 644 {} + - - chmod -R 777 "$DIR/logs" - chown -R "$USER:docker" "$DIR/logs" - - chmod 755 "$DIR/bin" - find "$DIR/bin" -type f -exec chmod +x {} + - chmod +x "$DIR/lds" - - ln -fs "$DIR/lds" /usr/local/bin/lds - printf "%bPermissions assigned.%b\n" "$GREEN" "$NC" -} - -############################################################################### -# 3. DOMAIN / PROFILE INTEGRATION -############################################################################### -mkhost() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" mkhost "$@" -} -rmhost() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" rmhost "$@" -} - -setup_domain() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - mkhost --RESET - docker exec -it "$ctr" mkhost - local mk_state svr_prof - mk_state="$(mkhost --JSON || true)" - if has_tool jq; then - svr_prof="$(printf '%s' "$mk_state" | jq -r '.state.apache_active // empty' 2>/dev/null || true)" - else - svr_prof="$(printf '%s' "$mk_state" | tr -d '\r\n' | sed -n 's/.*"apache_active"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" - fi - [[ -n $svr_prof ]] && modify_profiles add "$svr_prof" - mkhost --RESET - cmd_reboot -} - -delete_domain() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - rmhost --RESET - - # interactive delete - docker exec -it "$ctr" rmhost "$@" - - local rm_state apache_cont - rm_state="$(rmhost --JSON || true)" - if has_tool jq; then - apache_cont="$(printf '%s' "$rm_state" | jq -r '.state.apache_delete // empty' 2>/dev/null || true)" - else - apache_cont="$(printf '%s' "$rm_state" | tr -d '\r\n' | sed -n 's/.*"apache_delete"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" - fi - [[ -n "$apache_cont" ]] && modify_profiles remove "$apache_cont" - - rmhost --RESET - cmd_reboot -} - -modify_profiles() { - local action=$1 - shift - local file=$ENV_DOCKER var=COMPOSE_PROFILES - local -a existing updated - - if [[ -r $file ]]; then - local line value - line=$(grep -E "^${var}=" "$file" | tail -n1 || true) - value=${line#*=} - IFS=',' read -r -a existing <<<"$value" - fi - - case $action in - add) - local p - for p; do - [[ -n $p && ! " ${existing[*]} " =~ " $p " ]] && updated+=("$p") - done - updated+=("${existing[@]}") - ;; - remove) - local old - for old in "${existing[@]}"; do - [[ ! " $* " =~ " $old " ]] && updated+=("$old") - done - ;; - *) die "modify_profiles: invalid action '$action'" ;; - esac - - update_env "$file" "$var" "$( - IFS=, - echo "${updated[*]}" - )" -} - -# ───────────────────────────────────────────────────────────────────────────── -# Profiles -# ───────────────────────────────────────────────────────────────────────────── - -############################################################################### -# 3a. PROFILES: DEFINITIONS + SETUP FLOW -############################################################################### - -declare -A SERVICES=( - [POSTGRESQL]="postgresql" - [MYSQL]="mysql" - [MARIADB]="mariadb" - [ELASTICSEARCH]="elasticsearch" - [MONGODB]="mongodb" - [REDIS]="redis" -) - -declare -a SERVICE_ORDER=(POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS) - -declare -A PROFILE_ENV=( - [elasticsearch]="ELASTICSEARCH_VERSION=9.3.0" - [mysql]="MYSQL_VERSION=latest MYSQL_ROOT_PASSWORD=12345 MYSQL_USER=infocyph MYSQL_PASSWORD=12345 MYSQL_DATABASE=localdb" - [mariadb]="MARIADB_VERSION=latest MARIADB_ROOT_PASSWORD=12345 MARIADB_USER=infocyph MARIADB_PASSWORD=12345 MARIADB_DATABASE=localdb" - [mongodb]="MONGODB_VERSION=latest MONGODB_ROOT_USERNAME=root MONGODB_ROOT_PASSWORD=12345" - [redis]="REDIS_VERSION=latest" - [postgresql]="POSTGRES_VERSION=latest POSTGRES_USER=postgres POSTGRES_PASSWORD=postgres POSTGRES_DATABASE=postgres" -) - -declare -a PENDING_ENVS=() -declare -a PENDING_PROFILES=() - -queue_env() { PENDING_ENVS+=("$1"); } -queue_profile() { PENDING_PROFILES+=("$1"); } - -flush_envs() { - local env_file="$ENV_DOCKER" kv key val - for kv in "${PENDING_ENVS[@]}"; do - IFS='=' read -r key val <<<"$kv" - update_env "$env_file" "$key" "$val" - done -} - -flush_profiles() { - local profile - for profile in "${PENDING_PROFILES[@]}"; do - modify_profiles add "$profile" - done -} - -# ── setup menu (selection-first) ────────────────────────────────────────────── - -setup_menu_print() { - # Print menu to stderr to avoid stdout buffering in some Windows wrappers. - { - printf "\n%bSetup profiles%b (will replace previous configuration, if exists):\n\n" "$CYAN" "$NC" - local i=1 key slug - for key in "${SERVICE_ORDER[@]}"; do - slug="${SERVICES[$key]}" - printf " %2d) %-12s (%s)\n" "$i" "$key" "$slug" - i=$((i + 1)) - done - printf "\n a) ALL\n" - printf " n) NONE / Back\n\n" - } >&2 -} - -# Parse user selection into indices or ALL/NONE (prints one token per line) -setup_menu_parse() { - local input="${1//[[:space:]]/}" - [[ -n "$input" ]] || return 1 - input="${input//;/,}" - - echo "$input" | tr ',' '\n' | awk ' - BEGIN { ok=1 } - /^[0-9]+-[0-9]+$/ { - split($0,a,"-") - if (a[1] > a[2]) { t=a[1]; a[1]=a[2]; a[2]=t } - for (i=a[1]; i<=a[2]; i++) print i - next - } - /^[0-9]+$/ { print $0; next } - /^[aA]$/ { print "ALL"; next } - /^[nN]$/ { print "NONE"; next } - { ok=0 } - END { if (!ok) exit 2 } - ' -} - -# Outputs: newline-separated service KEYS from SERVICE_ORDER (e.g. MYSQL, REDIS) -setup_choose_services() { - local ans parsed - while :; do - setup_menu_print - tty_readline ans "Select (e.g. 1,3,5 or 2-4 or a): " || return 1 - - if ! parsed="$(setup_menu_parse "$ans" 2>/dev/null)"; then - printf "%bInvalid selection.%b Try again.\n" "$YELLOW" "$NC" - continue - fi - - if grep -qx "NONE" <<<"$parsed"; then - return 1 - fi - - if grep -qx "ALL" <<<"$parsed"; then - printf "%s\n" "${SERVICE_ORDER[@]}" - return 0 - fi - - # Indices -> keys (de-dupe, preserve order) - local -A seen=() - local out=() - local idx key - while IFS= read -r idx; do - [[ "$idx" =~ ^[0-9]+$ ]] || continue - ((idx >= 1 && idx <= ${#SERVICE_ORDER[@]})) || continue - key="${SERVICE_ORDER[idx - 1]}" - [[ -n "${seen[$key]:-}" ]] && continue - seen[$key]=1 - out+=("$key") - done <<<"$parsed" - - if ((${#out[@]} == 0)); then - printf "%bNo valid items selected.%b\n" "$YELLOW" "$NC" - continue - fi - - printf "%s\n" "${out[@]}" - return 0 - done -} - -setup_service() { - local service="$1" - local profile="${SERVICES[$service]:-}" - [[ -n "$profile" ]] || die "Unknown service: $service" - - printf "\n%b→ %s%b\n" "$YELLOW" "$service" "$NC" - queue_profile "$profile" - - printf "%bEnter value(s) for %s:%b\n" "$BLUE" "$service" "$NC" - local pair key def val - for pair in ${PROFILE_ENV[$profile]}; do - IFS='=' read -r key def <<<"$pair" - val=$(read_default "$key" "$def") - queue_env "$key=$val" - done -} - -process_all() { - local selected - if ! selected="$(setup_choose_services)"; then - printf "\n%bSetup cancelled.%b\n" "$YELLOW" "$NC" - return 0 - fi - - printf "\n%bWill configure:%b\n" "$CYAN" "$NC" - while IFS= read -r svc; do - printf " - %s (%s)\n" "$svc" "${SERVICES[$svc]}" - done <<<"$selected" - echo - - local svc - while IFS= read -r svc; do - setup_service "$svc" - done <<<"$selected" - - flush_envs - flush_profiles - printf "\n%b✅ Selected services configured!%b\n" "$GREEN" "$NC" -} - -############################################################################### -# 5. ENVIRONMENT + CERT / CA -############################################################################### -detect_timezone() { - if has_cmd timedatectl; then - timedatectl show -p Timezone --value - elif [[ -n ${TZ-} ]]; then - printf '%s' "$TZ" - elif [[ -r /etc/timezone ]]; then - /dev/null | tr -d '\r' - else - date +%Z - fi -} - -env_init() { - local env_file="$ENV_DOCKER" - printf "%bBootstrapping environment defaults…%b\n" "$YELLOW" "$NC" - - local default_tz tz - default_tz="$(detect_timezone)" - tz="$(read_default "Timezone (TZ)" "$default_tz")" - - local default_git_name default_git_email git_name git_email - default_git_name="$(git config --global --get user.name 2>/dev/null || true)" - default_git_email="$(git config --global --get user.email 2>/dev/null || true)" - git_name="$(read_default "Git user.name (GIT_USER_NAME)" "$default_git_name")" - git_email="$(read_default "Git user.email (GIT_USER_EMAIL)" "$default_git_email")" - - # update_env now quotes automatically when needed - update_env "$env_file" "TZ" "$tz" - update_env "$env_file" "GIT_USER_NAME" "$git_name" - update_env "$env_file" "GIT_USER_EMAIL" "$git_email" - - printf "%bConfiguration saved!%b\n" "$GREEN" "$NC" -} - -# ───────────────────────────────────────────────────────────────────────────── -# Root CA helpers (cross-distro) -# ───────────────────────────────────────────────────────────────────────────── - -# Unique identity (avoid conflicts with other mkcert/dev CAs) -CA_BASENAME="localdevstack-rootca" -CA_NICK="LocalDevStack Root CA" - -detect_os_family() { - # Output: "id|like" - # Must never fail under set -e - if [[ "${OSTYPE:-}" =~ (msys|cygwin|win32) ]]; then - echo "windows|windows" - return 0 - fi - - local id like - id="unknown" - like="unknown" - - if [[ -r /etc/os-release ]]; then - # shellcheck disable=SC1091 - . /etc/os-release || true - id="${ID:-unknown}" - like="${ID_LIKE:-unknown}" - elif has_cmd uname; then - # fallback for macOS / other unix - case "$(uname -s 2>/dev/null || true)" in - Darwin) - id="macos" - like="darwin" - ;; - Linux) - id="linux" - like="linux" - ;; - esac - fi - - echo "$id|$like" -} - -# Decide destination path + update mechanism. Echo: family|dest|updater -ca_plan() { - local os_id os_like - IFS='|' read -r os_id os_like < <(detect_os_family) - - case " $os_id $os_like " in - *" debian "* | *" ubuntu "* | *" linuxmint "* | *" pop "* | *" raspbian "*) - printf "debian|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" - ;; - *" alpine "*) - printf "alpine|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" - ;; - *" fedora "* | *" rhel "* | *" redhat "* | *" centos "* | *" rocky "* | *" alma "* | *" amzn "* | *" amazon "* | *" sles "* | *" suse "*) - printf "rhel|/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt|update-ca-trust\n" - ;; - *" arch "* | *" manjaro "*) - printf "arch|/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt|trust\n" - ;; - *) - # best default: Debian-style location (works on many distros even if updater differs) - printf "fallback|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|\n" - ;; - esac -} - -is_windows_shell() { - [[ "${OSTYPE:-}" =~ (msys|cygwin) ]] || [[ -n "${WORKDIR_WIN:-}" ]] -} - -need_windows_tools() { - has_cmd cygpath || die "Windows certificate install needs 'cygpath' (Git Bash)." - has_cmd powershell.exe || die "Windows certificate install needs 'powershell.exe' on PATH." -} - -# Import CA into the invoking user's NSS DB (Chrome/Chromium/Firefox on many Linux setups) -install_ca_nss_user() { - local ca_file="$1" - has_cmd certutil || return 0 - - local user="${SUDO_USER:-}" - [[ -n "$user" && "$user" != "root" ]] || return 0 - - local home - home="$(getent passwd "$user" | cut -d: -f6)" - [[ -n "$home" && -d "$home" ]] || return 0 - - local nssdb="sql:${home}/.pki/nssdb" - sudo -u "$user" mkdir -p "${home}/.pki/nssdb" >/dev/null 2>&1 || true - - if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then - printf "%b✔ NSS already has CA%b (%s)\n" "$GREEN" "$NC" "$user" - return 0 - fi - - if sudo -u "$user" certutil -d "$nssdb" -A -n "$CA_NICK" -t "C,," -i "$ca_file" >/dev/null 2>&1; then - printf "%b✔ Imported CA into NSS%b (%s)\n" "$GREEN" "$NC" "$user" - else - printf "%bWARN%b: NSS import failed (certutil).\n" "$YELLOW" "$NC" >&2 - fi -} - -install_ca_windows() { - need_windows_tools - - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" - - local win_ca - win_ca="$(cygpath -w "$src_ca")" - - printf "%bInstalling root CA into Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" - - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " - \$ErrorActionPreference = 'Stop' - \$path = '$win_ca' - \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) - \$cert.FriendlyName = '$CA_NICK' - - \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') - \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) - - \$exists = \$store.Certificates | Where-Object { \$_.Thumbprint -eq \$cert.Thumbprint } - if (-not \$exists) { \$store.Add(\$cert) } - - \$store.Close() - " >/dev/null 2>&1 || die "Windows certificate install failed (PowerShell import)." - - printf "%bRoot CA installed on Windows%b (CurrentUser\\Root) as %s\n" "$GREEN" "$NC" "$CA_NICK" - printf "%bNote:%b restart browsers if they still show trust errors.\n" "$YELLOW" "$NC" -} - -install_ca() { - if is_windows_shell; then - install_ca_windows - return 0 - fi - - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate install requires sudo" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" - - local family dest updater os_id os_like - IFS='|' read -r os_id os_like < <(detect_os_family) - IFS='|' read -r family dest updater < <(ca_plan) - - printf "%bInstalling root CA…%b\n" "$CYAN" "$NC" - printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" - - install -d -m 755 "$(dirname "$dest")" - install -m 644 "$src_ca" "$dest" - printf "%b✔ Copied%b → %s\n" "$GREEN" "$NC" "$dest" - - case "$family" in - debian | alpine) - if has_cmd update-ca-certificates; then - printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" - if update-ca-certificates; then - printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" - printf "%bNote:%b If you see \"rehash: skipping ca-certificates.crt…\", that’s normal (it’s a bundle).\n" "$YELLOW" "$NC" - else - printf "%bWARN%b: update-ca-certificates failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bWARN%b: update-ca-certificates not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 - fi - - # Optional p11-kit sync: best-effort only (can be missing helper on minimal installs) - if has_cmd trust; then - printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" - if trust extract-compat >/dev/null 2>&1; then - printf "%b✔ p11-kit trust synced%b\n" "$GREEN" "$NC" - else - printf "%bWARN%b: trust extract-compat failed (helper missing on some installs). Skipping.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bINFO%b: 'trust' not found — skipping p11-kit sync.\n" "$YELLOW" "$NC" - fi - ;; - rhel) - if has_cmd update-ca-trust; then - printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" - if update-ca-trust extract; then - printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" - else - printf "%bWARN%b: update-ca-trust extract failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bWARN%b: update-ca-trust not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 - fi - ;; - arch) - if has_cmd trust; then - printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" - if trust extract-compat >/dev/null 2>&1; then - printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" - else - printf "%bWARN%b: trust extract-compat failed. CA is installed, but trust sync may be incomplete.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bWARN%b: 'trust' not found. CA is installed, but trust sync is unavailable.\n" "$YELLOW" "$NC" >&2 - fi - ;; - *) - printf "%bINFO%b: Unknown distro; CA copied to %s.\n" "$YELLOW" "$NC" "$dest" - printf "%bINFO%b: You may need to update trust store manually for your OS.\n" "$YELLOW" "$NC" - ;; - esac - - # Extra: ensure browsers that rely on NSS trust pick it up - install_ca_nss_user "$src_ca" - - printf "%bRoot CA installed%b → %s (%s)\n" "$GREEN" "$NC" "$dest" "$CA_NICK" -} - -uninstall_ca_windows() { - need_windows_tools - - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" - - local win_ca - win_ca="$(cygpath -w "$src_ca")" - - printf "%bUninstalling root CA from Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" - - local removed - removed="$(powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " - \$ErrorActionPreference = 'Stop' - \$path = '$win_ca' - \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) - \$thumb = \$cert.Thumbprint - - \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') - \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) - - \$matches = @(\$store.Certificates | Where-Object { \$_.Thumbprint -eq \$thumb }) - foreach (\$c in \$matches) { \$store.Remove(\$c) } - - \$store.Close() - [string]\$matches.Count - " 2>/dev/null || true)" - - removed="${removed//[$'\r\n\t ']/}" - if [[ "${removed:-0}" =~ ^[0-9]+$ ]] && ((removed > 0)); then - printf "%bRoot CA uninstalled on Windows%b (removed %s cert)\n" "$GREEN" "$NC" "$removed" - else - printf "%bRoot CA already absent on Windows%b (no matching cert)\n" "$YELLOW" "$NC" - fi -} - -uninstall_ca_nss_user() { - has_cmd certutil || return 0 - local user="${SUDO_USER:-}" - [[ -n "$user" && "$user" != "root" ]] || return 0 - - local home - home="$(getent passwd "$user" | cut -d: -f6)" - [[ -n "$home" && -d "$home" ]] || return 0 - - local nssdb="sql:${home}/.pki/nssdb" - if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then - sudo -u "$user" certutil -d "$nssdb" -D -n "$CA_NICK" >/dev/null 2>&1 || true - printf "%b✔ Removed CA from NSS%b (%s)\n" "$GREEN" "$NC" "$user" - fi -} - -uninstall_ca() { - if is_windows_shell; then - uninstall_ca_windows - return 0 - fi - - [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate uninstall requires sudo" - - local all=0 - if [[ "${1:-}" == "--all" ]]; then - all=1 - shift - fi - - local family dest updater os_id os_like - IFS='|' read -r os_id os_like < <(detect_os_family) - IFS='|' read -r family dest updater < <(ca_plan) - - printf "%bUninstalling root CA…%b\n" "$CYAN" "$NC" - printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" - - local removed=0 - - if [[ -e "$dest" ]]; then - rm -f "$dest" - removed=$((removed + 1)) - printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$dest" - else - printf "%bINFO%b: CA file not found at %s (nothing to remove)\n" "$YELLOW" "$NC" "$dest" - fi - - if ((all)); then - printf "%bScanning all known CA anchor paths…%b\n" "$CYAN" "$NC" - local f - for f in \ - "/usr/local/share/ca-certificates/${CA_BASENAME}.crt" \ - "/usr/local/share/ca-certificates/${CA_BASENAME}.pem" \ - "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt" \ - "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.pem" \ - "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt" \ - "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.pem"; do - [[ "$f" == "$dest" ]] && continue - if [[ -e "$f" ]]; then - rm -f "$f" - removed=$((removed + 1)) - printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$f" - fi - done - fi - - case "$family" in - debian | alpine) - if has_cmd update-ca-certificates; then - printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" - update-ca-certificates || printf "%bWARN%b: update-ca-certificates failed.\n" "$YELLOW" "$NC" >&2 - else - printf "%bWARN%b: update-ca-certificates not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 - fi - - if has_cmd trust; then - printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" - trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed. Skipping.\n" "$YELLOW" "$NC" >&2 - fi - ;; - rhel) - if has_cmd update-ca-trust; then - printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" - update-ca-trust extract || printf "%bWARN%b: update-ca-trust extract failed.\n" "$YELLOW" "$NC" >&2 - else - printf "%bWARN%b: update-ca-trust not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 - fi - ;; - arch) - if has_cmd trust; then - printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" - trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed.\n" "$YELLOW" "$NC" >&2 - else - printf "%bWARN%b: 'trust' not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 - fi - ;; - *) - if has_cmd update-ca-certificates; then - printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" - update-ca-certificates || true - fi - if has_cmd update-ca-trust; then - printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" - update-ca-trust extract || true - fi - if has_cmd trust; then - printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" - trust extract-compat >/dev/null 2>&1 || true - fi - printf "%bINFO%b: Unknown distro; removed CA file(s) if present. Refresh trust store manually if needed.\n" "$YELLOW" "$NC" - ;; - esac - - uninstall_ca_nss_user - - if ((removed)); then - printf "%bRoot CA uninstalled%b (removed %d file(s))\n" "$GREEN" "$NC" "$removed" - else - printf "%bRoot CA already absent%b (no files removed)\n" "$YELLOW" "$NC" - fi -} - -add_required_env() { - update_env "$ENV_DOCKER" WORKING_DIR "$DIR" - ((EUID == 0)) && return 0 - update_env "$ENV_DOCKER" USER "$(id -un)" - update_env "$ENV_DOCKER" UID "$(id -u)" - update_env "$ENV_DOCKER" GID "$(id -g)" -} - -############################################################################### -# Compose helpers for rebuild (robust: supports service key OR container name) -############################################################################### -__COMPOSE_CFG_JSON="" -__COMPOSE_CFG_YAML="" -__COMPOSE_SVCS_LOADED=0 -declare -a __COMPOSE_SVCS=() - -compose_cfg_json() { - if [[ -z "${__COMPOSE_CFG_JSON}" ]]; then - __COMPOSE_CFG_JSON="$(docker_compose config --format json 2>/dev/null || true)" - fi - printf '%s' "${__COMPOSE_CFG_JSON}" -} - -compose_cfg_yaml() { - if [[ -z "${__COMPOSE_CFG_YAML}" ]]; then - __COMPOSE_CFG_YAML="$(docker_compose config 2>/dev/null || true)" - fi - printf '%s' "${__COMPOSE_CFG_YAML}" -} - -compose_services_load() { - ((__COMPOSE_SVCS_LOADED)) && return 0 - mapfile -t __COMPOSE_SVCS < <(docker_compose config --services 2>/dev/null || true) - __COMPOSE_SVCS_LOADED=1 -} - -compose_service_exists() { - local want="${1:-}" s - [[ -n "$want" ]] || return 1 - compose_services_load - for s in "${__COMPOSE_SVCS[@]}"; do - [[ "$s" == "$want" ]] && return 0 - done - return 1 -} - -resolve_service() { - local raw="${1:-}" norm svc - raw="${raw//[[:space:]]/}" - [[ -n "$raw" ]] || { - printf '' - return 0 - } - - compose_service_exists "$raw" && { - printf '%s' "$raw" - return 0 - } - - norm="$(normalize_service "$raw")" - compose_service_exists "$norm" && { - printf '%s' "$norm" - return 0 - } - - if docker inspect "$raw" >/dev/null 2>&1; then - svc="$(docker inspect -f '{{ index .Config.Labels "com.docker.compose.service" }}' "$raw" 2>/dev/null || true)" - if [[ -n "$svc" ]] && compose_service_exists "$svc"; then - printf '%s' "$svc" - return 0 - fi - fi - - printf '%s' "$norm" -} - -compose_has_build() { - local svc="$1" json - json="$(compose_cfg_json)" - if [[ -n "$json" ]]; then - if has_tool jq; then - jq -e --arg s "$svc" '.services[$s].build != null' >/dev/null <<<"$json" - return $? - fi - fi - - compose_cfg_yaml | awk -v s="$svc" ' - $1=="services:" {in_services=1; next} - in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} - in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 1} - in_svc && $0 ~ /^ build:/ {exit 0} - END {exit 1} - ' -} - -compose_image_for_service() { - local svc="$1" json - json="$(compose_cfg_json)" - if [[ -n "$json" ]]; then - if has_tool jq; then - jq -r --arg s "$svc" '.services[$s].image // empty' <<<"$json" - return 0 - fi - fi - - compose_cfg_yaml | awk -v s="$svc" ' - $1=="services:" {in_services=1; next} - in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} - in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 0} - in_svc && $0 ~ /^ image:/ { - sub(/^ image:[[:space:]]*/, "", $0) - print $0 - exit 0 - } - ' -} - -############################################################################### -# 6. STACK COMMANDS (CLI) -############################################################################### - -# One-time migration from the historical fixed 172.28/29/30 /24 networks. -# Only known LocalDevStack-owned networks are touched, and named volumes are -# never removed. New networks carry com.infocyph.network-schema=dynamic-v1. -declare -a __LDS_LEGACY_NETWORK_NAMES=(Frontend Backend DataStore) - -legacy_network_expected_subnet() { - case "\${1:-}" in - Frontend) printf '%s' '172.28.0.0/24' ;; - Backend) printf '%s' '172.29.0.0/24' ;; - DataStore) printf '%s' '172.30.0.0/24' ;; - *) return 1 ;; - esac -} - -migrate_legacy_networks() { - local network expected subnets stack_label project_label schema_label ctr ctr_project attachments - local -a legacy=() - - for network in "\${__LDS_LEGACY_NETWORK_NAMES[@]}"; do - docker network inspect "$network" >/dev/null 2>&1 || continue - - schema_label="$(docker network inspect -f '{{index .Labels "com.infocyph.network-schema"}}' "$network" 2>/dev/null || true)" - [[ "$schema_label" != "dynamic-v1" ]] || continue - - expected="$(legacy_network_expected_subnet "$network")" - subnets="$(docker network inspect -f '{{range .IPAM.Config}}{{println .Subnet}}{{end}}' "$network" 2>/dev/null || true)" - grep -Fxq "$expected" <<<"$subnets" || continue - - stack_label="$(docker network inspect -f '{{index .Labels "com.infocyph.stack"}}' "$network" 2>/dev/null || true)" - project_label="$(docker network inspect -f '{{index .Labels "com.docker.compose.project"}}' "$network" 2>/dev/null || true)" - if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "LocalDevStack" ]]; then - die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack. Remove or rename that network manually." - fi - - while IFS= read -r ctr; do - [[ -n "$ctr" ]] || continue - ctr_project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$ctr" 2>/dev/null || true)" - if [[ "$ctr_project" != "LocalDevStack" ]]; then - die "Refusing to migrate '$network': container '$ctr' is not owned by the LocalDevStack Compose project." - fi - done < <(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true) - - legacy+=("$network") - done - - (("\${#legacy[@]}" > 0)) || return 0 - - warn "Legacy fixed LocalDevStack network(s) detected: \${legacy[*]}" - warn "Recreating stack networks dynamically; named volumes and persisted data are preserved." - - # Stop/remove only LocalDevStack Compose containers and networks. Never use -v. - docker_compose down --remove-orphans - - for network in "\${legacy[@]}"; do - docker network inspect "$network" >/dev/null 2>&1 || continue - attachments="$(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true)" - [[ -z "$attachments" ]] || - die "Cannot remove legacy network '$network': attached container(s) remain: $(tr '\n' ' ' <<<"$attachments")" - docker network rm "$network" >/dev/null - done - - ok "Legacy fixed networks removed; Compose will recreate dynamic bridge networks." -} - -cmd_vpn_fix() { - warn "vpn-fix is deprecated: LocalDevStack no longer owns fixed Docker subnets." - warn "If a VPN conflict remains after dynamic-network migration, diagnose the VPN/Docker route directly." -} - -cmd_up() { - migrate_legacy_networks - dc_up "$@" -} - -cmd_start() { - migrate_legacy_networks - dc_up -d "$@" - http_reload -} - -cmd_stop() { docker_compose down; } - -cmd_down() { - # Safety rails: - # lds down --volumes requires --yes - local yes=0 vols=0 - local -a args=() - while [[ "${1:-}" ]]; do - case "$1" in - --yes | -y) - yes=1 - shift - ;; - --volumes | -v) - vols=1 - args+=("--volumes") - shift - ;; - --remove-orphans) - args+=("--remove-orphans") - shift - ;; - *) - args+=("$1") - shift - ;; - esac - done - if ((vols)) && ((yes == 0)); then - die "Refusing: down --volumes requires --yes" - fi - docker_compose down "${args[@]}" -} - -cmd_restart() { - cmd_stop - cmd_start -} -cmd_reboot() { cmd_restart; } - -# ───────────────────────────────────────────────────────────────────────────── -# 6a. STATUS / PS / STATS -# ───────────────────────────────────────────────────────────────────────────── -cmd_ps() { - if (($#)); then - docker_compose ps "$@" - else - docker_compose ps - fi -} - -cmd_status() { - local ctr project - project="$(lds_project)" - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container not found or not running for project: $project" - - local -a flags=() - [[ -t 1 ]] && flags+=(-t) - docker exec "${flags[@]}" "$ctr" status "$@" -} -# ───────────────────────────────────────────────────────────────────────────── -# 6b. LOGS / OPEN -# ───────────────────────────────────────────────────────────────────────────── -cmd_logs() { - local svc="" follow=0 since="" grep_pat="" - while [[ "${1:-}" ]]; do - case "$1" in - -f | --follow) - follow=1 - shift - ;; - --since) - since="${2:-}" - shift 2 - ;; - --grep) - grep_pat="${2:-}" - shift 2 - ;; - *) - svc="${1:-}" - shift - ;; - esac - done - - local -a args=() - ((follow)) && args+=("-f") - [[ -n "$since" ]] && args+=("--since" "$since") - - if [[ -n "$svc" ]]; then - local s - s="$(resolve_service "$svc" || true)" - [[ -n "$s" ]] || die "Unknown service: $svc" - if [[ -n "$grep_pat" ]]; then - docker_compose logs "${args[@]}" "$s" 2>&1 | text_grep "$grep_pat" - else - docker_compose logs "${args[@]}" "$s" - fi - else - if [[ -n "$grep_pat" ]]; then - docker_compose logs "${args[@]}" 2>&1 | text_grep "$grep_pat" - else - docker_compose logs "${args[@]}" - fi - fi -} - -cmd_open() { - local target="${1:-}" - [[ -n "$target" ]] || die "open " - local url="" - case "${target,,}" in - mail | mailpit | webmail) url="https://webmail.localhost" ;; - db | cloudbeaver) url="https://db.localhost" ;; - redis | redisinsight | redis-insight | rds) url="http://ri.localhost" ;; - mongo | me | mongoexpress | mongo-express) url="http://me.localhost" ;; - kibana | kbn) url="http://kibana.localhost" ;; - *) - url="https://${target}" - ;; - esac - open_url "$url" -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6c. PROFILES -# ───────────────────────────────────────────────────────────────────────────── -_known_profile() { - local p="${1:-}" - [[ -n "$p" ]] || return 1 - - # Prefer compose-config JSON for exact profile membership. - local json - json="$(compose_cfg_json)" - if [[ -n "$json" ]] && has_tool jq; then - printf '%s' "$json" | jq -e --arg p "$p" ' - [ .services[]? | (.profiles // [])[] ] | index($p) != null - ' >/dev/null 2>&1 - return $? - fi - - # Fallback: text scan when jq/json path is unavailable. - local f - for f in "$COMPOSE_FILE" "${__EXTRA_FILES[@]:-}"; do - [[ -r "$f" ]] || continue - grep -Fq -- "$p" "$f" && return 0 - done - return 1 -} - -cmd_profiles() { - local action="${1:-list}" - shift || true - case "${action,,}" in - list | "") - local cur="" - [[ -r "$ENV_DOCKER" ]] && cur="$(grep -E '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1 | cut -d= -f2- | tr -d '\r' || true)" - printf "%bEnabled profiles:%b %s -" "$CYAN" "$NC" "${cur:-}" - printf "%bAvailable profiles:%b -" "$CYAN" "$NC" - printf ' - %s -' "${SERVICES[@]}" | LC_ALL=C sort -u - # warn if enabled profile has no mention in compose - if [[ -n "$cur" ]]; then - local p - IFS=',' read -r -a __ps <<<"$cur" - for p in "${__ps[@]}"; do - p="${p//[[:space:]]/}" - [[ -n "$p" ]] || continue - _known_profile "$p" || printf "%b[warn]%b enabled profile '%s' has no matching services in compose -" "$YELLOW" "$NC" "$p" - done - fi - ;; - add) - [[ $# -gt 0 ]] || die "profiles add " - for p in "$@"; do - modify_profiles add "$p" - done - ;; - remove | rm | del) - [[ $# -gt 0 ]] || die "profiles remove " - modify_profiles remove "$@" - ;; - *) - die "profiles " - ;; - esac -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6d. DIAG / SNIFF -# ───────────────────────────────────────────────────────────────────────────── -_tools_exec() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - # NOTE: pass a SINGLE command string; do not pass arrays here. - docker exec -i "$ctr" sh -lc "$*" -} - -_shq() { printf '%q' "$1"; } - -cmd_diag() { - local sub="${1:-}" - shift || true - - case "${sub,,}" in - dns) - local dom="${1:-}" - [[ -n "$dom" ]] || die "diag dns " - local qdom - qdom="$(_shq "$dom")" - _tools_exec "dig +short $qdom; echo; nslookup $qdom 2>/dev/null || true; echo; getent hosts $qdom 2>/dev/null || true" - ;; - route | net) - _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || netstat -tulpen 2>/dev/null || true" - ;; - tcp) - local h="${1:-}" - local p="${2:-}" - [[ -n "$h" && -n "$p" ]] || die "diag tcp " - _tools_exec "nc -vz -w2 $(_shq "$h") $(_shq "$p")" - ;; - http) - local url="${1:-}" - shift || true - [[ -n "$url" ]] || die "diag http [curl-args...]" - local -a qargs=() - local a - for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done - _tools_exec "curl -vkI $(_shq "$url") ${qargs[*]}" - ;; - tls) - local dom="${1:-}" - [[ -n "$dom" ]] || die "diag tls " - local qdom - qdom="$(_shq "$dom")" - _tools_exec "echo | openssl s_client -connect ${qdom}:443 -servername $qdom -showcerts 2>/dev/null | sed -n '1,60p'" - ;; - *) - die "diag " - ;; - esac -} - -cmd_sniff() { - local url="${1:-}" - shift || true - [[ -n "$url" ]] || die "sniff [curl-args...]" - local -a qargs=() - local a - for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done - _tools_exec "curl -vk -D - $(_shq "$url") ${qargs[*]} | (command -v jq >/dev/null 2>&1 && jq . 2>/dev/null || cat)" -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6e. SECRETS / CERT / HOST / UI -# ───────────────────────────────────────────────────────────────────────────── -cmd_secrets() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" senv "$@" -} - -cmd_cert() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" certify "$@" -} - -cmd_host() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - add) - setup_domain - ;; - rm | remove | del | delete) - delete_domain "$@" - ;; - list) - shopt -s nullglob - for f in "$DIR/configuration/nginx/"*.conf; do - printf '%s -' "$(basename -- "$f" .conf)" - done - shopt -u nullglob - ;; - *) - die "host " - ;; - esac -} - -cmd_ui() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" lazydocker -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6f. EXEC / EVENTS / CLEAN / DISK -# ───────────────────────────────────────────────────────────────────────────── -cmd_exec() { - local svc="${1:-}" - shift || true - [[ -n "$svc" ]] || die "exec [cmd...]" - local s - s="$(resolve_service "$svc" || true)" - [[ -n "$s" ]] || die "Unknown service: $svc" - if [[ $# -gt 0 ]]; then - docker_compose exec "$s" "$@" - else - docker_compose exec "$s" sh -lc 'command -v bash >/dev/null 2>&1 && exec bash || exec sh' - fi -} - -cmd_events() { - local since="${1:-1h}" - local project - project="$(lds_project)" - docker events --since "$since" --filter "label=com.docker.compose.project=$project" -} - -cmd_clean() { - local yes=0 vols=0 - while [[ "${1:-}" ]]; do - case "$1" in - --yes | -y) - yes=1 - shift - ;; - --volumes | -v) - vols=1 - shift - ;; - *) - die "clean [--yes|-y] [--volumes|-v]" - ;; - esac - done - - ((yes)) || die "clean requires --yes" - - printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" - docker container prune -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" - docker network prune -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" - docker image prune -a -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" - docker builder prune -a -f >/dev/null 2>&1 || true - - if ((vols)); then - printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" - docker volume prune -f >/dev/null 2>&1 || true - fi - - printf "%b[clean]%b done\n" "$GREEN" "$NC" -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6g. HELP MARKDOWN -# ───────────────────────────────────────────────────────────────────────────── - -normalize_service() { - local raw="${1:-}" - local s="${raw//[[:space:]]/}" - [[ -n "$s" ]] || { - printf '%s' "" - return 0 - } - - local low="${s,,}" - - local key="${low//_/}" - key="${key//-/}" - if [[ "$key" =~ ^php ]]; then - local ver="${key#php}" - ver="${ver//[^0-9]/}" - if [[ "$ver" =~ ^([0-9])([0-9]).* ]]; then - printf 'php%s%s' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" - return 0 - fi - printf 'php' - return 0 - fi - - low="${low//_/-}" - while [[ "$low" == *"--"* ]]; do low="${low//--/-}"; done - printf '%s' "$low" -} - -cmd_rebuild() { - local -a targets=() all_svcs=() - local arg svc img - declare -A seen=() - - # ----------------------------- - # helper: add a service once - # ----------------------------- - _add_target() { - local s="$1" - [[ -n "$s" ]] || return 0 - [[ -n "${seen[$s]:-}" ]] && return 0 - seen[$s]=1 - targets+=("$s") - } - - # ----------------------------- - # helper: trim - # ----------------------------- - _trim() { - local s="$1" - s="${s#"${s%%[![:space:]]*}"}" - s="${s%"${s##*[![:space:]]}"}" - printf '%s' "$s" - } - - # ----------------------------- - # helper: interactive selection (comma separated, supports ranges) - # accepts: "all" or "1,3,5-7" or mix with names "nginx,2,5-6" - # ----------------------------- - _pick_targets_interactive() { - compose_services_load - all_svcs=("${__COMPOSE_SVCS[@]}") - if ((${#all_svcs[@]})); then - mapfile -t all_svcs < <(printf '%s\n' "${all_svcs[@]}" | LC_ALL=C sort -f -u) - fi - [[ ${#all_svcs[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" - - echo - echo "Select services to rebuild (comma separated; ranges allowed)." - echo "Examples: 1,3,5-7 | nginx,2,5-6 | all" - echo - - local i - for i in "${!all_svcs[@]}"; do - printf " %2d) %s\n" "$((i + 1))" "${all_svcs[$i]}" - done - - echo - local sel - read -r -p "Pick: " sel - sel="$(_trim "${sel:-}")" - [[ -n "$sel" ]] || die "No selection provided." - - if [[ "${sel,,}" == "all" ]]; then - for svc in "${all_svcs[@]}"; do _add_target "$svc"; done - return 0 - fi - - # split by comma - local IFS=, - for arg in $sel; do - arg="$(_trim "$arg")" - [[ -n "$arg" ]] || continue - - # range like 3-7 - if [[ "$arg" =~ ^[0-9]+-[0-9]+$ ]]; then - local a b - a="${arg%-*}" - b="${arg#*-}" - ((a >= 1)) || continue - ((b >= 1)) || continue - ((a <= b)) || { - local t="$a" - a="$b" - b="$t" - } - - local n - for ((n = a; n <= b; n++)); do - ((n >= 1 && n <= ${#all_svcs[@]})) || continue - _add_target "${all_svcs[$((n - 1))]}" - done - continue - fi - - # single index - if [[ "$arg" =~ ^[0-9]+$ ]]; then - local n="$arg" - ((n >= 1 && n <= ${#all_svcs[@]})) || continue - _add_target "${all_svcs[$((n - 1))]}" - continue - fi - - # treat as service/container name - svc="$(resolve_service "$arg")" - [[ -n "$svc" ]] && _add_target "$svc" - done - - [[ ${#targets[@]} -gt 0 ]] || die "No valid services selected." - } - - # ----------------------------- - # build target list - # ----------------------------- - if (($# == 0)); then - _pick_targets_interactive - elif [[ "${1,,}" == "all" ]]; then - compose_services_load - targets=("${__COMPOSE_SVCS[@]}") - [[ ${#targets[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" - else - for arg in "$@"; do - svc="$(resolve_service "$arg")" - [[ -n "$svc" ]] || continue - _add_target "$svc" - done - [[ ${#targets[@]} -gt 0 ]] || die "No valid services provided." - fi - - # ----------------------------- - # rebuild each target - # ----------------------------- - for svc in "${targets[@]}"; do - [[ -n "$svc" ]] || continue - compose_service_exists "$svc" || die "Unknown service/container: '$svc'" - - if compose_has_build "$svc"; then - logq rebuild "build/recreate $svc" - dc_build --no-cache --pull "$svc" - dc_up -d --no-deps --force-recreate "$svc" - continue - fi - - img="$(compose_image_for_service "$svc")" - logq rebuild "pull/recreate $svc${img:+ ($img)}" - - docker_compose rm -sf "$svc" >/dev/null 2>&1 || true - - if [[ -n "${img:-}" ]]; then - docker rmi -f "$img" >/dev/null 2>&1 || true - dc_pull "$svc" || true - else - dc_build --no-cache --pull "$svc" >/dev/null 2>&1 || true - fi - - dc_up -d --no-deps --force-recreate "$svc" - done - logq reboot "Rebooting stacks" - cmd_reboot -} - -cmd_config() { docker_compose config; } - -docker_shell() { - local c="${1:-}" - [[ -n "$c" ]] || die "container name required" - if docker exec "$c" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then - exec docker exec -it "$c" bash - else - exec docker exec -it "$c" sh - fi -} -cmd_tools() { - local sub="${1:-sh}" - shift || true - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - case "${sub,,}" in - sh | shell | "") - docker_shell "$ctr" - ;; - exec) - [[ $# -gt 0 ]] || die "tools exec " - docker exec -it "$ctr" sh -lc "$*" - ;; - file) - local p="${1:-}" - [[ -n "$p" ]] || die "tools file " - docker exec -it "$ctr" sh -lc "ls -la -- \"$p\" 2>/dev/null || true; echo; sed -n '1,200p' -- \"$p\" 2>/dev/null || true" - ;; - *) - die "tools " - ;; - esac -} -cmd_http() { [[ ${1:-} == reload ]] && http_reload; } -cmd_cli() { - local ctr="${1:-}" - shift || true - - [[ -n "$ctr" ]] || die "Usage: lds cli [cmd...]" - - docker inspect "$ctr" >/dev/null 2>&1 || die "Container not found: $ctr" - docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "Container not running: $ctr" - - # If user provided a command, run it; otherwise open an interactive shell. - if [[ "$#" -gt 0 ]]; then - local cmd="$*" - docker exec -it "$ctr" sh -lc ' - if command -v bash >/dev/null 2>&1; then - exec bash --login -lc "$1" - fi - exec sh -lc "$1" - ' sh "$cmd" - return - fi - - docker exec -it "$ctr" sh -lc ' - if command -v bash >/dev/null 2>&1; then - exec bash --login - fi - exec sh - ' -} - -cmd_core() { - # Usage: - # lds core -> open correct container for that domain (PHP/Node) - # lds core -> open a shell in that container - # lds core -> list domains and let user pick - - local target="${1:-}" - - # domain regex (same as domain-which/mkhost family) - local re='^([a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9])?\.)+(localhost|local|test|loc|[a-zA-Z]{2,})$' - - # If no target -> prompt from domain-which list - if [[ -z "$target" ]]; then - local tools_ctr - tools_ctr="$(_project_tools_container_running || true)" - [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - local -a domains=() - mapfile -t domains < <(docker exec "$tools_ctr" domain-which --list-domains 2>/dev/null | sed '/^[[:space:]]*$/d' || true) - - ((${#domains[@]} > 0)) || die "No domains found" - - # stable ordering - IFS=$'\n' domains=($(printf '%s\n' "${domains[@]}" | LC_ALL=C sort -u)) - - if ((${#domains[@]} == 1)); then - target="${domains[0]}" - else - if [[ ! -t 0 ]]; then - printf "%b[core]%b No domain provided. Available domains:\n" "$YELLOW" "$NC" >&2 - local i=1 - local d - for d in "${domains[@]}"; do - printf " %2d) %s\n" "$i" "$d" >&2 - ((i++)) - done - die "No TTY to prompt. Use: lds core " - fi - - printf "%bSelect domain:%b\n" "$CYAN" "$NC" >&2 - local i=1 d - for d in "${domains[@]}"; do - printf " %b%2d)%b %s\n" "$CYAN" "$i" "$NC" "$d" >&2 - ((i++)) - done - - local ans="" - while true; do - read -r -p "Enter number (1-${#domains[@]}): " ans - ans="$(echo "$ans" | xargs)" - [[ "$ans" =~ ^[0-9]+$ ]] || { - printf "%bInvalid input.%b\n" "$YELLOW" "$NC" >&2 - continue - } - ((ans >= 1 && ans <= ${#domains[@]})) || { - printf "%bOut of range.%b\n" "$YELLOW" "$NC" >&2 - continue - } - target="${domains[$((ans - 1))]}" - break - done - fi - fi - - # If target looks like a domain -> resolve via domain-which then shell in - if [[ "$target" =~ $re ]]; then - local tools_ctr - tools_ctr="$(_project_tools_container_running || true)" - [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - local app container wd - app="$(docker exec "$tools_ctr" domain-which --app --quiet "$target" 2>/dev/null)" || die "Unknown domain: $target" - container="$(docker exec "$tools_ctr" domain-which --container --quiet "$target" 2>/dev/null)" || die "No container resolved for: $target" - wd="$(docker exec "$tools_ctr" domain-which --docroot --quiet "$target" 2>/dev/null)" || true - [[ -n "${container:-}" ]] || die "No container resolved for: $target" - - # Node apps should always land at /app. Others follow resolved docroot. - if [[ "${app:-}" == "node" ]]; then - wd="/app" - fi - [[ -n "${wd:-}" ]] || wd="/app" - - docker exec -it "$container" bash -lc "cd \"$wd\" 2>/dev/null || cd /app 2>/dev/null || cd /; exec bash" - return 0 - fi - - # Otherwise treat target as a container name - docker exec -it "$(printf '%s' "$target" | tr '[:lower:]' '[:upper:]')" sh -lc 'exec bash -i || exec sh' -} - -cmd_setup() { - add_required_env - case ${1:-} in - init) env_init ;; - permission | permissions | perms | perm) fix_perms ;; - domain) setup_domain ;; - profiles | profile) process_all ;; - *) die "setup " ;; - esac -} - -cmd_certificate() { - case ${1:-} in - install) - shift || true - install_ca - ;; - uninstall | remove | rm) - shift || true - uninstall_ca "${@:-}" - ;; - *) - die "certificate " - ;; - esac -} - -############################################################################### -# NOTIFY -############################################################################### -notify_watch() { - local container="${1:-}" - if [[ -z "$container" ]]; then - container="$(_project_tools_container_running || true)" - [[ -n "$container" ]] || die "server-tools container is not running for project: $(lds_project)" - fi - local prefix="__HOST_NOTIFY__" - - need docker - - local _disp="${DISPLAY-}" - local _dbus="${DBUS_SESSION_BUS_ADDRESS-}" - - # Args: timeout(ms) urgency title body - _host_notify() { - local timeout="${1:-2500}" urgency="${2:-normal}" title="${3:-Notification}" body="${4:-}" - - # Linux desktop (or WSLg) - if has_cmd notify-send; then - (env DISPLAY="${_disp-}" DBUS_SESSION_BUS_ADDRESS="${_dbus-}" \ - setsid -f notify-send -u "$urgency" -t "$timeout" "$title" "$body" \ - >/dev/null 2>&1 || true) & - return 0 - fi - - # Windows toast (Git Bash) / WSL-on-Windows - if has_cmd powershell.exe; then - # Pass values as args to avoid quoting issues entirely. - # Note: urgency/timeout not used by toast api here; kept for parity. - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command \ - 'param([string]$t,[string]$b) - try { - [Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null - [Windows.Data.Xml.Dom.XmlDocument, Windows.Data.Xml.Dom.XmlDocument, ContentType = WindowsRuntime] > $null - - function Esc([string]$s) { - if ($null -eq $s) { return "" } - return ($s -replace "&","&" -replace "<","<" -replace ">",">" -replace "\"",""" -replace "'\''","'") - } - - $title = Esc $t - $body = Esc $b - - $xml = New-Object Windows.Data.Xml.Dom.XmlDocument - $xml.LoadXml("$title$body") - $toast = New-Object Windows.UI.Notifications.ToastNotification $xml - [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier("Devtainer").Show($toast) - } catch { }' \ - --% "$title" "$body" >/dev/null 2>&1 || true - - return 0 - fi - - # Fallback - printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 - return 0 - } - - trap - ERR - set +e - set +o pipefail - - local _stop=0 - - _watcher_notify() { - local urgency="${1:-critical}" title="${2:-Notifier}" body="${3:-Watcher event}" - _host_notify 2500 "$urgency" "$title" "$body" - } - - _watcher_int_term() { - _stop=1 - _watcher_notify critical "Notifier" "Notification watcher interrupted/exiting" - printf "%b[watcher]%b Notification watcher interrupted/exiting\n" "$RED" "$NC" >&2 - } - trap _watcher_int_term INT TERM - - local grep_cmd=(grep -a --line-buffered -E "^${prefix}([[:space:]]|$)") - has_cmd stdbuf && grep_cmd=(stdbuf -oL -eL "${grep_cmd[@]}") - - printf "%bNotify Watch:%b monitoring is active. Ctrl+C to stop.\n" "$GREEN" "$NC" - - while ((_stop == 0)); do - if ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then - _watcher_notify critical "Notifier" "Watcher stopped: $container is not running" - printf "%b[watcher]%b %s is not running; exiting.\n" "$RED" "$NC" "$container" >&2 - break - fi - - docker logs -f --tail 0 "$container" 2>&1 | - ("${grep_cmd[@]}" || true) | - while IFS=$'\t' read -r _ f1 f2 f3 f4 rest; do - local timeout urgency title body - - if [[ "${f1:-}" =~ ^[0-9]{1,6}$ ]]; then - timeout="$f1" - urgency="${f2:-normal}" - title="${f3:-Notification}" - body="${f4:-}" - else - timeout="2500" - urgency="${f1:-normal}" - title="${f2:-Notification}" - body="${f3:-}" - fi - - [[ -n "${rest:-}" ]] && body+=$'\t'"${rest}" - case "$urgency" in low | normal | critical) ;; *) urgency="normal" ;; esac - - _host_notify "$timeout" "$urgency" "$title" "$body" - printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 - done - - ((_stop)) && break - - if docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then - _watcher_notify critical "Notifier" "Watcher lost log stream (docker logs ended). Reconnecting…" - printf "%b[watcher]%b docker logs ended; reconnecting...\n" "$YELLOW" "$NC" >&2 - sleep 1 - continue - fi - - _watcher_notify critical "Notifier" "Watcher stopped: $container stopped" - printf "%b[watcher]%b %s stopped; exiting.\n" "$RED" "$NC" "$container" >&2 - break - done - - trap - INT TERM - set -euo pipefail - - ((_stop)) && return 130 - return 0 -} - -notify_test() { - local title="${1:-Notifier OK}" - local body="${2:-Hello from host via project server-tools container}" - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" notify -t 2500 -u normal "$title" "$body" -} - -cmd_notify() { - case ${1:-watch} in - watch) notify_watch "${2:-}" ;; - test) notify_test "${2:-Notifier OK}" "${3:-Hello from host}" ;; - *) die "notify " ;; - esac -} - -open_url() { - local url="${1:-}" - [[ -n "$url" ]] || return 0 - - # WSL/Windows helpers first when available - if grep -qi microsoft /proc/version 2>/dev/null; then - if has_cmd powershell.exe; then - powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 || true - return 0 - fi - if has_cmd cmd.exe; then - cmd.exe /c start "" "$url" >/dev/null 2>&1 || true - return 0 - fi - fi - - if has_cmd xdg-open; then - (xdg-open "$url" >/dev/null 2>&1 &) - return 0 - fi - if has_cmd open; then - (open "$url" >/dev/null 2>&1 &) - return 0 - fi - if has_cmd powershell; then - (powershell -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 &) - return 0 - fi - - printf "%bINFO%b: open this URL manually → %s\n" "$YELLOW" "$NC" "$url" -} - -############################################################################### -# RUN (ad-hoc Dockerfile runner) -############################################################################### -hash_short() { - local s="$1" - if has_cmd sha1sum; then - printf '%s' "$s" | sha1sum | cut -c1-8 - elif has_cmd shasum; then - printf '%s' "$s" | shasum -a 1 | cut -c1-8 - else - # POSIX fallback; stable (not cryptographic) - printf '%s' "$s" | cksum | awk '{print $1}' - fi -} - -run_slug() { - local dir="$1" base hash - base="$(basename "$dir" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9._-' '-')" - hash="$(hash_short "$dir")" - printf '%s-%s' "$base" "$hash" -} - -run_plan() { - local dir="$1" slug - slug="$(run_slug "$dir")" - printf '%s|%s|%s\n' \ - "lds-run-${slug}" \ - "${slug}:local" \ - "$dir" -} - -detect_host_os() { - if is_windows_shell || grep -qi microsoft /proc/version 2>/dev/null; then - printf 'windows' - return 0 - fi - - if has_cmd uname; then - case "$(uname -s 2>/dev/null || true)" in - Darwin) - printf 'macos' - ;; - Linux) - printf 'linux' - ;; - *) - printf 'unknown' - ;; - esac - else - printf 'unknown' - fi -} - -run_find_container() { - local dir="$1" - docker ps -a --filter "label=com.infocyph.lds.run=1" \ - --filter "label=com.infocyph.lds.dir=${dir}" \ - --format '{{.Names}}' | head -n 1 -} - -run_build() { - local tag="$1" dir="$2" - - # Build only if the image doesn't already exist. - if docker image inspect "$tag" >/dev/null 2>&1; then - printf "%b[run]%b Image exists, skipping build: %b%s%b -" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" - return 0 - fi - - printf "%b[run]%b Building image %b%s%b from %s -" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" "$dir" - docker build -t "$tag" "$dir" -} - -run_start() { - local name="$1" tag="$2" dir="$3" keepalive="$4" sock="$5" host_os="$6" - shift 6 || true - - # Remaining args are split by a "--" sentinel: - # - before "--" : publish specs (HOST:CONT), repeatable - # - after "--" : mount specs (HOST[:CONT]), repeatable - local -a pubs=() mounts=() - local seen_delim=0 x - for x in "$@"; do - if [[ "$x" == "--" ]]; then - seen_delim=1 - continue - fi - if ((seen_delim)); then - mounts+=("$x") - else - pubs+=("$x") - fi - done - - # Normalize project dir (POSIX absolute). - # On Windows Git Bash, /e/... is OK; docker.exe will receive converted path automatically. - local dir_posix - dir_posix="$(cd "$dir" 2>/dev/null && pwd -P)" || die "invalid dir: $dir" - - # MSYS-safe container paths: use '//' prefix to prevent path conversion. - # Docker interprets //path as /path inside container. - local WDIR="//workspace" - local WDIR_MOUNT="${dir_posix}://workspace" - - local -a args=(docker run -d --name "$name" - --label "com.infocyph.lds.run=1" - --label "com.infocyph.lds.dir=$dir_posix" - --label "com.infocyph.lds.tag=$tag" - -w "$WDIR" - -v "$WDIR_MOUNT" - ) - - if [[ -n "$host_os" ]]; then - args+=(-e "HOST_OS=$host_os") - fi - - # Mount extra directories/files (HOST[:CONT]). - # - If container path missing, mounts under /mnt/. - # - HOST may be relative to the run directory. - if ((${#mounts[@]})); then - local spec host cont base - for spec in "${mounts[@]}"; do - [[ -n "$spec" ]] || continue - host="$spec" - cont="" - - # Split as HOST:CONT ONLY if suffix after last ':' looks like a container absolute path (/...) - # (safe for Windows drive letters like E:\... because tail won't start with '/') - if [[ "$spec" == *:* ]]; then - local tail="${spec##*:}" - if [[ "$tail" == /* ]]; then - host="${spec%:*}" - cont="$tail" - fi - fi - - # Resolve host to absolute (POSIX) for checks - if [[ "$host" != /* && "$host" != ~* && ! "$host" =~ ^[A-Za-z]:[\\/].* ]]; then - host="${dir_posix%/}/$host" - fi - - # If user provided Windows path (E:\...), convert to POSIX for existence check - if [[ "$host" =~ ^[A-Za-z]:[\\/].* ]] && has_cmd cygpath; then - host="$(cygpath -u "$host")" - fi - - host="$(cd "${host%/*}" 2>/dev/null && pwd -P)/${host##*/}" || { - printf "%b[run]%b Warning: cannot resolve mount path: %s\n" "$YELLOW" "$NC" "$spec" >&2 - continue - } - - [[ -e "$host" ]] || { - printf "%b[run]%b Warning: mount path does not exist: %s\n" "$YELLOW" "$NC" "$host" >&2 - continue - } - - if [[ -z "$cont" ]]; then - base="${host##*/}" - cont="/mnt/${base}" - fi - [[ "$cont" == /* ]] || cont="/mnt/${cont}" - - # Prevent MSYS conversion for container side by using '//' prefix - cont="//${cont#/}" - - args+=(-v "${host}:${cont}") - done - fi - - # Optional docker sock - if [[ "${sock:-0}" == 1 ]]; then - args+=(-v "/var/run/docker.sock:/var/run/docker.sock") - fi - - # Publish ports - local pub - for pub in "${pubs[@]}"; do - [[ -n "$pub" ]] || continue - args+=(-p "$pub") - done - - if [[ "$keepalive" == 1 ]]; then - # Keepalive mode replaces the image command; disable image healthcheck to avoid false "unhealthy". - args+=(--no-healthcheck --entrypoint sh "$tag" -c "trap : TERM INT; sleep infinity & wait") - else - args+=("$tag") - fi - - printf "%b[run]%b Starting container %b%s%b\n" "$CYAN" "$NC" "$BLUE" "$name" "$NC" - - # IMPORTANT: don't hide errors; if it fails, you need to see why - if ! "${args[@]}"; then - printf "%b[run]%b docker run failed.\n" "$RED" "$NC" >&2 - return 1 - fi - printf "\n" -} - -run_exec_shell() { - local name="$1" - if docker exec "$name" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then - exec docker exec -it "$name" bash - else - exec docker exec -it "$name" sh - fi -} - -cmd_run() { - local action="*" dir="$PWD" name="" tag="" nobuild=0 keepalive=1 sock=0 host_os="" - local -a publish=() mounts=() - local open_port="" open_path="/" open_proto="http" - - while [[ $# -gt 0 ]]; do - case "$1" in - stop | rm | ps | shell | logs | open | "*") - action="$1" - shift - ;; - build) - action="*" - shift - ;; - --name) - name="${2:-}" - shift 2 - ;; - --tag) - tag="${2:-}" - shift 2 - ;; - --no-build) - nobuild=1 - shift - ;; - --no-keepalive) - keepalive=0 - shift - ;; - --sock) - sock=1 - shift - ;; - --host-os) - host_os="${2:-}" - shift 2 - ;; - -p | --publish) - publish+=("${2:-}") - shift 2 - ;; - --mount) - mounts+=("${2:-}") - shift 2 - ;; - --port) - open_port="${2:-}" - shift 2 - ;; - --path) - open_path="${2:-/}" - shift 2 - ;; - --https) - open_proto="https" - shift - ;; - --http) - open_proto="http" - shift - ;; - *) break ;; - esac - done - - # Host path (POSIX) for planning/labels - local dir_posix - dir_posix="$(cd "$dir" && pwd -P)" - - if [[ -z "$host_os" ]]; then - host_os="${HOST_OS:-$(detect_host_os)}" - fi - - # Docker path (may need Windows form for docker.exe) - local dir_docker="$dir_posix" - - # Windows Git Bash/MSYS hardening: - # - stop MSYS rewriting container paths (/workspace -> D:/Program Files/Git/workspace) - # - but still feed docker.exe Windows-absolute host paths for build/run contexts - if is_windows_shell; then - export MSYS_NO_PATHCONV=1 - export MSYS2_ARG_CONV_EXCL='*' - if has_cmd cygpath; then - dir_docker="$(cygpath -w "$dir_posix")" - fi - fi - - # Plan/name/tag should be based on the real project identity (POSIX dir) - IFS='|' read -r def_name def_tag _def_dir < <(run_plan "$dir_posix") - name="${name:-$def_name}" - - # Tag rules: - # - Default tag is ":local" (from run_plan) - # - If user passes --tag without ":", append ":local" - if [[ -n "${tag:-}" ]]; then - if [[ "$tag" != *:* ]]; then - tag="${tag}:local" - fi - else - tag="$def_tag" - fi - - _find_for_dir() { - local found - found="$(run_find_container "$dir_posix" || true)" - if [[ -n "$found" ]]; then - printf '%s' "$found" - return 0 - fi - if docker inspect "$name" >/dev/null 2>&1; then - printf '%s' "$name" - return 0 - fi - return 1 - } - - _run_build_summary() { - local img="$1" build_dir="$2" cname="$3" - local tag_only="${img##*:}" - - printf "\n%b[run]%b Build summary\n" "$CYAN" "$NC" - printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" - printf " %bTag:%b %s\n" "$BOLD" "$NC" "$tag_only" - printf " %bDir:%b %s\n" "$BOLD" "$NC" "$build_dir" - printf " %bName:%b %s\n" "$BOLD" "$NC" "$cname" - printf " %bKeepalive:%b %s\n" "$BOLD" "$NC" "$keepalive" - printf " %bSock:%b %s\n" "$BOLD" "$NC" "$sock" - printf " %bHost OS:%b %s\n" "$BOLD" "$NC" "$host_os" - - if ((${#publish[@]})); then - printf " %bPublish:%b %s\n" "$BOLD" "$NC" "${publish[*]}" - else - printf " %bPublish:%b (none)\n" "$BOLD" "$NC" - fi - - if ((${#mounts[@]})); then - printf " %bMounts:%b %s\n" "$BOLD" "$NC" "${mounts[*]}" - else - printf " %bMounts:%b (none)\n" "$BOLD" "$NC" - fi - printf "\n" - } - - _run_runtime_summary() { - local cname="$1" - local id img state ports - id="$(docker inspect -f '{{.Id}}' "$cname" 2>/dev/null | cut -c1-12 || true)" - img="$(docker inspect -f '{{.Config.Image}}' "$cname" 2>/dev/null || true)" - state="$(docker inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || true)" - ports="$(docker port "$cname" 2>/dev/null | sed '/^[[:space:]]*$/d' | tr '\n' '; ' | sed 's/; $//' || true)" - - printf "%b[run]%b Runtime summary\n" "$CYAN" "$NC" - printf " %bContainer:%b %s\n" "$BOLD" "$NC" "${cname}${id:+ ($id)}" - [[ -n "$img" ]] && printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" - [[ -n "$state" ]] && printf " %bState:%b %s\n" "$BOLD" "$NC" "$state" - if [[ -n "$ports" ]]; then - printf " %bPorts:%b %s\n" "$BOLD" "$NC" "$ports" - else - printf " %bPorts:%b (none published)\n" "$BOLD" "$NC" - fi - printf "%b\n[run]%b Example Usage (in Composer)\n" "$CYAN" "$NC" - printf " %bimage:%b %s\n" "$BOLD" "$NC" "$img" - printf " %bpull_policy:%b never\n" "$BOLD" "$NC" - printf "\n" - } - - case "$action" in - ps) - docker ps -a --filter "label=com.infocyph.lds.run=1" \ - --format 'table {{.Names}} {{.Image}} {{.Status}} {{.Labels}}' - return 0 - ;; - stop) - local existing - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - docker stop "$existing" >/dev/null - printf "%b[run]%b Stopped %s\n" "$GREEN" "$NC" "$existing" - return 0 - ;; - logs) - local existing - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - exec docker logs -f "$existing" - ;; - open) - local existing line addr hp url - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - - [[ -n "$open_path" ]] || open_path="/" - [[ "$open_path" == /* ]] || open_path="/$open_path" - - if [[ -n "$open_port" ]]; then - line="$(docker port "$existing" "$open_port" 2>/dev/null | head -n 1 || true)" - [[ -n "$line" ]] || line="$(docker port "$existing" "${open_port}/tcp" 2>/dev/null | head -n 1 || true)" - else - line="$(docker port "$existing" 2>/dev/null | head -n 1 || true)" - fi - - if [[ -z "$line" ]]; then - printf "%b[run]%b No published ports found.\n" "$YELLOW" "$NC" - printf "%b[run]%b Tip: start with %blds run --publish 8025:8025%b then %blds run open%b\n" \ - "$YELLOW" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" - return 1 - fi - - addr="${line##*-> }" - hp="${addr##*:}" - url="${open_proto}://localhost:${hp}${open_path}" - open_url "$url" - printf "%b[run]%b Opened: %s\n" "$GREEN" "$NC" "$url" - return 0 - ;; - rm) - local existing img - existing="$(_find_for_dir)" || true - if [[ -n "${existing:-}" ]]; then - img="$(docker inspect -f '{{.Config.Image}}' "$existing" 2>/dev/null || true)" - docker stop "$existing" >/dev/null 2>&1 || true - docker rm "$existing" >/dev/null 2>&1 || true - printf "%b[run]%b Removed container %s\n" "$GREEN" "$NC" "$existing" - if [[ -n "${img:-}" ]]; then - docker rmi -f "$img" >/dev/null 2>&1 || true - printf "%b[run]%b Removed image %s\n" "$GREEN" "$NC" "$img" - fi - else - printf "%b[run]%b No container found for %s\n" "$YELLOW" "$NC" "$dir_posix" - fi - return 0 - ;; - shell | "*") - if ((nobuild == 0)); then - # Build needs docker.exe-friendly path on Windows - run_build "$tag" "$dir_docker" - else - printf "%b[run]%b Skipping build (--no-build)\n" "$YELLOW" "$NC" - fi - - _run_build_summary "$tag" "$dir_posix" "$name" - - if docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null | grep -q true; then - printf "%b[run]%b Container already running: %s\n\n" "$GREEN" "$NC" "$name" - else - if docker inspect "$name" >/dev/null 2>&1; then - docker rm -f "$name" >/dev/null 2>&1 || true - fi - - # Keep mounts as user gave them (POSIX/relative); run_start should validate POSIX - # and convert host-side to Windows only at docker run time. - run_start "$name" "$tag" "$dir_docker" "$keepalive" "$sock" "$host_os" \ - "${publish[@]}" -- "${mounts[@]}" - fi - - _run_runtime_summary "$name" - - # "shell" enters the container; "*" / "build" does not. - if [[ "$action" == "shell" ]]; then - run_exec_shell "$name" - else - printf "%b[run]%b Built/started. Use %blds run shell%b to enter, %blds run logs%b to follow logs.\n" \ - "$GREEN" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" - return 0 - fi - ;; - esac -} - -############################################################################### -# 6w. NEW FEATURES: stack diff | support trace -############################################################################### - -# stack diff: show what would run (compose) vs what's running (docker) -cmd_stack_diff() { - local json=0 - local show_config=0 - while [[ "${1:-}" ]]; do - case "$1" in - --json) - json=1 - shift - ;; - --config) - show_config=1 - shift - ;; - *) break ;; - esac - done - - local project - project="$(lds_project)" - local cfg_json="" - - if docker_compose config --format json >/dev/null 2>&1; then - cfg_json="$(docker_compose config --format json)" - else - # fallback: best-effort text config - cfg_json="" - fi - - # running: service -> image - declare -A running=() - local line - while IFS= read -r line; do - [[ -n "$line" ]] || continue - local svc="${line%%|*}" - local img="${line#*|}" - running["$svc"]="$img" - done < <(docker ps \ - --filter "label=com.docker.compose.project=$project" \ - --format '{{index .Labels "com.docker.compose.service"}}|{{.Image}}' 2>/dev/null || true) - - # desired: service -> image/build context (best-effort) - declare -A desired_img=() - declare -A desired_ctx=() - declare -A desired_df=() - - if [[ -n "$cfg_json" ]]; then - if has_tool jq; then - while IFS= read -r line; do - local svc="${line%%|*}" - local img="${line#*|}" - desired_img["$svc"]="$img" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"') - while IFS= read -r line; do - local svc="${line%%|*}" - local ctx="${line#*|}" - desired_ctx["$svc"]="$ctx" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"') - while IFS= read -r line; do - local svc="${line%%|*}" - local df="${line#*|}" - desired_df["$svc"]="$df" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"') - elif _server_tools_has jq; then - # Fallback: parse via project tools container jq through stdin (no shell re-quoting of JSON payload). - local ctr - ctr="$(_project_tools_container_running || true)" - if [[ -n "$ctr" ]]; then - while IFS= read -r line; do - local svc="${line%%|*}" - local img="${line#*|}" - desired_img["$svc"]="$img" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"' 2>/dev/null || true) - while IFS= read -r line; do - local svc="${line%%|*}" - local ctx="${line#*|}" - desired_ctx["$svc"]="$ctx" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"' 2>/dev/null || true) - while IFS= read -r line; do - local svc="${line%%|*}" - local df="${line#*|}" - desired_df["$svc"]="$df" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"' 2>/dev/null || true) - fi - fi - fi - - # Build result object - if ((json)); then - if has_tool jq; then - # assemble in bash -> jq - local tmp - tmp="$(mktemp)" - { - printf '{' - printf '"project":%s,' "$(printf '%s' "$project" | jq -Rsa .)" - printf '"compose_file":%s,' "$(printf '%s' "$COMPOSE_FILE" | jq -Rsa .)" - printf '"running":{' - local first=1 k - for k in "${!running[@]}"; do - ((first)) || printf ',' - first=0 - printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${running[$k]}" | jq -R .)" - done - printf '},' - printf '"desired":{' - first=1 - for k in "${!desired_img[@]}"; do - ((first)) || printf ',' - first=0 - printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${desired_img[$k]}" | jq -R .)" - done - printf '},' - printf '"diff":[' - first=1 - # union keys - declare -A seen=() - for k in "${!running[@]}"; do seen["$k"]=1; done - for k in "${!desired_img[@]}"; do seen["$k"]=1; done - for k in "${!seen[@]}"; do - local r="${running[$k]:-}" - local d="${desired_img[$k]:-}" - if [[ "$r" != "$d" ]]; then - ((first)) || printf ',' - first=0 - printf '{"service":%s,"running":%s,"desired":%s}' \ - "$(printf '%s' "$k" | jq -R .)" \ - "$(printf '%s' "$r" | jq -R .)" \ - "$(printf '%s' "$d" | jq -R .)" - fi - done - printf ']' - printf '}\n' - } >"$tmp" - cat "$tmp" | jq . - rm -f "$tmp" - else - die "jq required for --json (or run inside project server-tools container)" - fi - return 0 - fi - - printf "%bStack diff%b (project=%s)\n" "$CYAN" "$NC" "$project" - printf "%bCompose file:%b %s\n" "$DIM" "$NC" "$COMPOSE_FILE" - - if ((show_config)); then - if [[ -n "$cfg_json" ]]; then - printf "\n%bEffective compose config (json):%b\n" "$DIM" "$NC" - printf '%s\n' "$cfg_json" - else - printf "\n%bEffective compose config:%b\n" "$DIM" "$NC" - docker_compose config || true - fi - fi - - # union services - declare -A all=() - local svc - for svc in "${!running[@]}"; do all["$svc"]=1; done - for svc in "${!desired_img[@]}"; do all["$svc"]=1; done - - printf "\n%-22s %-40s %-40s %s\n" "SERVICE" "RUNNING" "DESIRED" "STATUS" - printf "%-22s %-40s %-40s %s\n" "------" "-------" "-------" "------" - for svc in $(printf '%s\n' "${!all[@]}" | sort); do - local r="${running[$svc]:-}" - local d="${desired_img[$svc]:-}" - local st - if [[ -z "$r" ]]; then - st="(not running)" - elif [[ -z "$d" ]]; then - st="(not in config)" - elif [[ "$r" == "$d" ]]; then - st="OK" - else - st="DIFF" - fi - printf "%-22s %-40.40s %-40.40s %s\n" "$svc" "$r" "$d" "$st" - done - - printf "\n%bNotes:%b\n" "$DIM" "$NC" - printf " - Desired image is derived from 'docker compose config'. If a service uses only 'build:' and no 'image:', desired may be empty.\n" - printf " - Use: lds stack diff --config (to print resolved compose config)\n" -} - -# support trace: quick end-to-end trace for a domain -cmd_support_trace() { - local dom="${1:-}" - [[ -n "$dom" ]] || die "support trace " - - local nconf="$DIR/configuration/nginx/$dom.conf" - printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" - - # 1) DNS - if _server_tools_running; then - printf "\n%b[DNS]%b\n" "$DIM" "$NC" - _tools_exec "dig +short $(_shq "$dom") || true; getent hosts $(_shq "$dom") 2>/dev/null || true" - else - printf "\n%b[DNS]%b\n" "$DIM" "$NC" - (has_cmd dig && dig +short "$dom") || true - (has_cmd getent && getent hosts "$dom") || true - fi - - # 2) TLS certificate - printf "\n%b[TLS]%b\n" "$DIM" "$NC" - if _server_tools_running; then - _tools_exec "echo | openssl s_client -connect $(_shq "$dom"):443 -servername $(_shq "$dom") -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true" - else - echo | openssl s_client -connect "${dom}:443" -servername "$dom" -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true - fi - - # 3) HTTP probe (timings) - printf "\n%b[HTTP]%b\n" "$DIM" "$NC" - if _server_tools_running; then - _tools_exec "curl -sk -o /dev/null -D - -w 'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' https://$(_shq "$dom") | sed -n '1,30p'" - else - curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' - fi - - # 4) Upstream inference from nginx conf (if exists) - printf "\n%b[Upstream]%b\n" "$DIM" "$NC" - if [[ -r "$nconf" ]]; then - if grep -q fastcgi_pass "$nconf"; then - local php - php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" - printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" - elif grep -q proxy_pass "$nconf"; then - local up - up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" - printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" - else - printf "type=static\n" - fi - else - printf "nginx_conf=%s (missing)\n" "$nconf" - fi - - # 5) Recent nginx logs (compose) - printf "\n%b[Recent nginx logs]%b\n" "$DIM" "$NC" - docker_compose logs --no-color --tail 120 nginx 2>/dev/null | text_grep -i "$dom" || docker_compose logs --no-color --tail 120 nginx 2>/dev/null || true - - printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" -} - -############################################################################### -# 6x. GROUPED COMMAND ROUTERS (stack/domain/support) + backward-compatible aliases -############################################################################### - -cmd_stack() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - "" | help | -h | --help) cmd_help stack ;; - up) cmd_up "$@" ;; - start) cmd_start "$@" ;; - down | stop) cmd_down "$@" ;; - restart | reboot) cmd_restart "$@" ;; - status) cmd_status "$@" ;; - ps) cmd_ps "$@" ;; - logs) cmd_logs "$@" ;; - exec) cmd_exec "$@" ;; - events) cmd_events "$@" ;; - clean) cmd_clean "$@" ;; - config) cmd_config "$@" ;; diff) cmd_stack_diff "$@" ;; - - http) cmd_http "$@" ;; - *) - die "stack " - ;; - esac -} - -# Canonical: domain. Legacy: host. -cmd_domain() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - "" | help | -h | --help) die "domain " ;; - add) cmd_host add "$@" ;; - rm | remove | del | delete) cmd_host rm "$@" ;; - ls | list) cmd_host list "$@" ;; - *) - die "domain " - ;; - esac -} - -cmd_support() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - "" | help | -h | --help) die "support " ;; - open) cmd_open "$@" ;; - bundle) cmd_bundle "$@" ;; - notify) cmd_notify "$@" ;; - ui) cmd_ui "$@" ;; - *) - die "support " - ;; - esac -} - -# Minimal support bundle (shareable diagnostics zip) -# Modes: -# --redact : remove secrets-like env lines from captured env/config outputs (best-effort) -# --full : include more logs/inspect output (can be large) -cmd_bundle() { - local mode="redact" - local out="${1:-}" - while [[ "${1:-}" ]]; do - case "$1" in - --redact) - mode="redact" - shift - ;; - --full) - mode="full" - shift - ;; - *.zip) - out="$1" - shift - ;; - *) break ;; - esac - done - - need zip - local ts - ts="$(date +%Y%m%d_%H%M%S)" - local project - project="$(lds_project)" - local tmp - tmp="$(mktemp -d "${TMPDIR:-/tmp}/lds_bundle.XXXXXX")" - local base="lds_bundle_${project}_${ts}" - [[ -n "$out" ]] || out="$PWD/${base}.zip" - - cleanup_bundle() { rm -rf "$tmp" 2>/dev/null || true; } - trap cleanup_bundle RETURN - - { - echo "project=$project" - echo "dir=$DIR" - echo "time=$ts" - echo "mode=$mode" - } >"$tmp/meta.txt" - - # compose config (effective) - { - echo "# docker compose config" - docker_compose config 2>&1 || true - } >"$tmp/compose.config.txt" - - # ps + networks - docker_compose ps >"$tmp/compose.ps.txt" 2>&1 || true - docker network ls >"$tmp/docker.networks.txt" 2>&1 || true - - # container list + inspect (scoped) - docker ps --filter "label=com.docker.compose.project=$project" --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}' \ - >"$tmp/docker.ps.txt" 2>&1 || true - - if [[ "$mode" == "full" ]]; then - docker inspect $(docker ps -q --filter "label=com.docker.compose.project=$project" 2>/dev/null) \ - >"$tmp/docker.inspect.json" 2>/dev/null || true - fi - - # recent logs (tail) - docker_compose logs --no-color --tail 400 >"$tmp/compose.logs.txt" 2>&1 || true - - # vhost configs - if [[ -d "$DIR/configuration/nginx" ]]; then - mkdir -p "$tmp/nginx" - cp -a "$DIR/configuration/nginx/." "$tmp/nginx/" 2>/dev/null || true - fi - - # env files (redacted best-effort) - mkdir -p "$tmp/env" - if [[ -r "$ENV_MAIN" ]]; then - if [[ "$mode" == "redact" ]]; then - sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_MAIN" >"$tmp/env/.env" - else - cp -a "$ENV_MAIN" "$tmp/env/.env" 2>/dev/null || true - fi - fi - if [[ -r "$ENV_DOCKER" ]]; then - if [[ "$mode" == "redact" ]]; then - sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_DOCKER" >"$tmp/env/docker.env" - else - cp -a "$ENV_DOCKER" "$tmp/env/docker.env" 2>/dev/null || true - fi - fi - - # tools-side quick diagnostics (inside network) - { - echo "# ip r / ip a / ss" - _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || true" - } >"$tmp/tools.net.txt" 2>&1 || true - - # pack - (cd "$tmp" && zip -qr "$out" .) || die "Failed to write bundle: $out" - ok "Bundle created: $out" -} - -cmd_help() { - - if [[ "${1:-}" == "--markdown" ]]; then - cat <<'MD' -# LocalDevStack (lds) — Command Reference - -## Stack (compose) -- `lds stack up` *(aliases: `up`)* -- `lds stack start` *(aliases: `start`)* -- `lds stack down [--volumes --yes]` *(aliases: `down`, `stop`)* -- `lds stack restart [svc]` *(aliases: `restart`, `reboot`)* -- `lds stack status [status-args…]` *(alias: `status`; forwards args to tools `status`)* -- `lds stack ps` *(alias: `ps`)* -- `lds stack logs [svc] [--follow] [--since ] [--grep ]` *(alias: `logs`)* -- `lds stack exec [cmd…]` *(alias: `exec`)* -- `lds stack events [--since ]` *(alias: `events`)* -- `lds stack clean --yes [--volumes]` *(alias: `clean`)* -- `lds stack diff [--config] [--json]` *(shows desired vs running images)* - - -## Domain (vhost lifecycle + routing) -- `lds domain add …` -- `lds domain rm …` -- `lds domain ls` - -Legacy alias: `lds host …` → same subcommands as `domain`. - -## Certificates (TLS) -- `lds cert status [domain|all]` -- `lds cert regen [domain|all] [--yes]` -- `lds cert diagnose ` -- `lds certificate install` -- `lds certificate uninstall [--all]` - -## Diagnostics -- `lds diag dns ` -- `lds diag net` -- `lds diag tcp ` -- `lds diag http ` *(alias: `sniff `)* -- `lds diag tls ` - -## Config -- `lds config show [--json]` -- `lds config services` -- `lds config profiles` -- `lds config env-used` -- `lds config validate` - -## Support -- `lds support open - support trace ` -- `lds support bundle [--redact|--full]` -- `lds support notify …` -- `lds support ui` - -Shortcuts: `open`, `bundle`, `notify`, `ui` map to `support …`. - -## Secrets (senv) -- `lds secrets …` - -## Tools (project server-tools container) -- `lds tools sh` -- `lds tools exec ""` -- `lds tools file ` - -## Setup -- `lds setup init|permissions|domain|profiles` - -## Runner (ad‑hoc Dockerfile runner) -- `lds run` *(default: build+start only)* / `lds run shell` *(build+start+enter)* / `lds run *` *(same as default)* (+ `ps|logs|stop|rm|open` and flags: `--publish|-p`, `--no-keepalive`, `--mount`, `--sock`, `--host-os`, `--tag`, `--name`) - -## Other -- `lds rebuild [all|]` -- `lds core [domain]` - -## Short client wrappers -- `lds php|composer|node|npm|npx …` -- `lds my|mysql|mysqldump …` -- `lds maria|mariadb|mariadb-dump …` -- `lds pg|psql|pg_dump|pg_restore …` -- `lds redis|redis-cli …` -MD - return 0 - fi - - cat < - clean Alias of: stack <...> - -${CYAN}Domain (vhosts + routing):${NC} - domain add|rm|ls - host add|rm|list Legacy alias of: domain <...> - -${CYAN}Certificates (TLS):${NC} - cert status|regen|diagnose - certificate install|uninstall [--all] - -${CYAN}Diagnostics:${NC} - diag dns|net|tcp|http|tls - sniff Alias of: diag http - -${CYAN}Config:${NC} - config show|services|profiles|env-used|validate - -${CYAN}Support:${NC} - support open - support trace - support bundle [--redact|--full] - support notify ... - support ui - open|bundle|notify|ui Shortcuts → support <...> - -${CYAN}Secrets:${NC} - secrets - -${CYAN}Tools (project server-tools container):${NC} - tools sh|exec|file - -${CYAN}Setup:${NC} - setup init|permissions|domain|profiles - -${CYAN}Runner (ad-hoc Dockerfile runner):${NC} - run [ps|logs|stop|rm|open] [--publish|-p A:B] [--no-keepalive] [--mount HOST[:CONT]] [--sock] [--host-os VALUE] - -${CYAN}Other:${NC} - rebuild [all|] - core [domain] - -${CYAN}Help:${NC} - help [--markdown] - -EOF -} - -############################################################################### -# 7. MAIN -############################################################################### -main() { - [[ $# -gt 0 ]] || { - cmd_help - exit 1 - } - - while [[ $# -gt 0 ]]; do - case "$1" in - -v | --verbose) - VERBOSE=1 - QUIET=0 - shift - ;; - -q | --quiet) - QUIET=1 - VERBOSE=0 - shift - ;; - --reload-extras) - export LDS_EXTRAS_RELOAD=1 - shift - ;; - --) - shift - break - ;; - -h | --help) - cmd_help - exit 0 - ;; - -*) die "Unknown global option: $1" ;; - *) break ;; - esac - done - - [[ $# -gt 0 ]] || { - cmd_help - exit 1 - } - - local cmd="${1,,}" - shift || true - - if [[ "$cmd" == "help" ]]; then - cmd_help "$@" - exit 0 - fi - - # Do not require docker for pure help output; everything else expects the stack. - need docker - - case "$cmd" in - php | composer | node | npm | npx) exec "$DIR/bin/$cmd" "$@" ;; - pg | psql | pg_restore | pg-restore | pgrestore | pg_dump | pgdump | pg-dump) exec "$DIR/bin/pg" "$@" ;; - maria | mariadb | mariadbdump | mariadb-dump | mariadb_dump) exec "$DIR/bin/maria" "$@" ;; - my | mysql | mysqldump | mysql-dump | mysql_dump) exec "$DIR/bin/my" "$@" ;; - redis | redis-cli) exec "$DIR/bin/redis-cli" "$@" ;; - es | elastic | elasticsearch) exec "$DIR/bin/es" "$@" ;; - mongo | mongodb | mongosh | mongoimport | mongoexport) exec "$DIR/bin/mongo" "$@" ;; - vpn-fix) cmd_vpn_fix "$@" ;; - *) - if declare -F "cmd_$cmd" >/dev/null 2>&1; then - "cmd_$cmd" "$@" - else - exec "$DIR/bin/tool-runner" "$cmd" "$@" - fi - ;; - esac -} - -main "$@" From eb3947bf06333daed34972bbc45b12e8012b96df Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:32:19 +0600 Subject: [PATCH 040/509] ai: rebuild CLI changes on clean network baseline --- lds | 1782 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 1761 insertions(+), 21 deletions(-) diff --git a/lds b/lds index 294936a0..51b939d0 100755 --- a/lds +++ b/lds @@ -307,16 +307,15 @@ ENV_RELEASE="$CFG/release.env" COMPOSE_FILE="$CFG/compose/main.yaml" EXTRAS_DIR="$DIR/configuration/compose" -# Read a single dotenv value without sourcing/evaluating the file. -# Only literal KEY=VALUE state is accepted; nothing is executed. +# Read a literal dotenv KEY=VALUE without sourcing/evaluating the file. dotenv_value() { local file="${1:-}" key="${2:-}" line value first last [[ -r "$file" && -n "$key" ]] || return 1 line="$(grep -E "^${key}=" "$file" 2>/dev/null | tail -n1 || true)" [[ -n "$line" ]] || return 1 - value="${line#*=}" + if (("${#value}" >= 2)); then first="${value:0:1}" last="${value: -1}" @@ -329,7 +328,7 @@ dotenv_value() { printf '%s' "$value" } -# Resolution mirrors Compose intent: shell > user docker/.env > release.env > fallback. +# Control-state precedence: shell > user docker/.env > release.env > fallback. compose_control_value() { local key="${1:?}" fallback="${2-}" value if [[ -v "$key" ]]; then @@ -470,9 +469,9 @@ docker_compose() { local -a env_files=(--env-file "$ENV_RELEASE") [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - # Static LocalDevStack overrides are selected from controlled state. local ai_runtime ai_host_port local -a static_f=() + ai_runtime="$(compose_control_value LDS_AI_RUNTIME cpu)" case "${ai_runtime,,}" in "" | cpu) ;; @@ -488,7 +487,7 @@ docker_compose() { *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; esac - # Build -f list (stable order; product overrides before user extras). + # Product overrides are applied before user-provided compose extras. local -a extra_f=() f for f in "${__EXTRA_FILES[@]:-}"; do [[ -f "$f" ]] || continue @@ -1801,18 +1800,12 @@ cmd_ai() { local sub="${1:-status}" shift || true case "${sub,,}" in - status | provider) - _tools_exec_argv aiops provider "$@" - ;; - ask) - _tools_exec_argv askai "$@" - ;; + status | provider) _tools_exec_argv aiops provider "$@" ;; + ask) _tools_exec_argv askai "$@" ;; explain | troubleshoot | review | repo-review | graphify) _tools_exec_argv aiops "${sub,,}" "$@" ;; - *) - die "ai [args...]" - ;; + *) die "ai [args...]" ;; esac } @@ -1850,9 +1843,7 @@ cmd_llm() { host-port) local state="${1:-status}" case "${state,,}" in - status) - printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" - ;; + status) printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" ;; on | enable | enabled | 1) update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 1 ok "Direct LLM API enabled on loopback only. Recreate llm-sm to apply." @@ -1872,9 +1863,7 @@ cmd_llm() { printf '%s\n' "llm runtime " printf '%s\n' "llm host-port " ;; - *) - die "llm " - ;; + *) die "llm " ;; esac } @@ -1959,3 +1948,1754 @@ cmd_host() { ;; rm | remove | del | delete) delete_domain "$@" + ;; + list) + shopt -s nullglob + for f in "$DIR/configuration/nginx/"*.conf; do + printf '%s +' "$(basename -- "$f" .conf)" + done + shopt -u nullglob + ;; + *) + die "host " + ;; + esac +} + +cmd_ui() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" lazydocker +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6f. EXEC / EVENTS / CLEAN / DISK +# ───────────────────────────────────────────────────────────────────────────── +cmd_exec() { + local svc="${1:-}" + shift || true + [[ -n "$svc" ]] || die "exec [cmd...]" + local s + s="$(resolve_service "$svc" || true)" + [[ -n "$s" ]] || die "Unknown service: $svc" + if [[ $# -gt 0 ]]; then + docker_compose exec "$s" "$@" + else + docker_compose exec "$s" sh -lc 'command -v bash >/dev/null 2>&1 && exec bash || exec sh' + fi +} + +cmd_events() { + local since="${1:-1h}" + local project + project="$(lds_project)" + docker events --since "$since" --filter "label=com.docker.compose.project=$project" +} + +cmd_clean() { + local yes=0 vols=0 + while [[ "${1:-}" ]]; do + case "$1" in + --yes | -y) + yes=1 + shift + ;; + --volumes | -v) + vols=1 + shift + ;; + *) + die "clean [--yes|-y] [--volumes|-v]" + ;; + esac + done + + ((yes)) || die "clean requires --yes" + + printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" + docker container prune -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" + docker network prune -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" + docker image prune -a -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" + docker builder prune -a -f >/dev/null 2>&1 || true + + if ((vols)); then + printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" + docker volume prune -f >/dev/null 2>&1 || true + fi + + printf "%b[clean]%b done\n" "$GREEN" "$NC" +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6g. HELP MARKDOWN +# ───────────────────────────────────────────────────────────────────────────── + +normalize_service() { + local raw="${1:-}" + local s="${raw//[[:space:]]/}" + [[ -n "$s" ]] || { + printf '%s' "" + return 0 + } + + local low="${s,,}" + + local key="${low//_/}" + key="${key//-/}" + if [[ "$key" =~ ^php ]]; then + local ver="${key#php}" + ver="${ver//[^0-9]/}" + if [[ "$ver" =~ ^([0-9])([0-9]).* ]]; then + printf 'php%s%s' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" + return 0 + fi + printf 'php' + return 0 + fi + + low="${low//_/-}" + while [[ "$low" == *"--"* ]]; do low="${low//--/-}"; done + printf '%s' "$low" +} + +cmd_rebuild() { + local -a targets=() all_svcs=() + local arg svc img + declare -A seen=() + + # ----------------------------- + # helper: add a service once + # ----------------------------- + _add_target() { + local s="$1" + [[ -n "$s" ]] || return 0 + [[ -n "${seen[$s]:-}" ]] && return 0 + seen[$s]=1 + targets+=("$s") + } + + # ----------------------------- + # helper: trim + # ----------------------------- + _trim() { + local s="$1" + s="${s#"${s%%[![:space:]]*}"}" + s="${s%"${s##*[![:space:]]}"}" + printf '%s' "$s" + } + + # ----------------------------- + # helper: interactive selection (comma separated, supports ranges) + # accepts: "all" or "1,3,5-7" or mix with names "nginx,2,5-6" + # ----------------------------- + _pick_targets_interactive() { + compose_services_load + all_svcs=("${__COMPOSE_SVCS[@]}") + if ((${#all_svcs[@]})); then + mapfile -t all_svcs < <(printf '%s\n' "${all_svcs[@]}" | LC_ALL=C sort -f -u) + fi + [[ ${#all_svcs[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" + + echo + echo "Select services to rebuild (comma separated; ranges allowed)." + echo "Examples: 1,3,5-7 | nginx,2,5-6 | all" + echo + + local i + for i in "${!all_svcs[@]}"; do + printf " %2d) %s\n" "$((i + 1))" "${all_svcs[$i]}" + done + + echo + local sel + read -r -p "Pick: " sel + sel="$(_trim "${sel:-}")" + [[ -n "$sel" ]] || die "No selection provided." + + if [[ "${sel,,}" == "all" ]]; then + for svc in "${all_svcs[@]}"; do _add_target "$svc"; done + return 0 + fi + + # split by comma + local IFS=, + for arg in $sel; do + arg="$(_trim "$arg")" + [[ -n "$arg" ]] || continue + + # range like 3-7 + if [[ "$arg" =~ ^[0-9]+-[0-9]+$ ]]; then + local a b + a="${arg%-*}" + b="${arg#*-}" + ((a >= 1)) || continue + ((b >= 1)) || continue + ((a <= b)) || { + local t="$a" + a="$b" + b="$t" + } + + local n + for ((n = a; n <= b; n++)); do + ((n >= 1 && n <= ${#all_svcs[@]})) || continue + _add_target "${all_svcs[$((n - 1))]}" + done + continue + fi + + # single index + if [[ "$arg" =~ ^[0-9]+$ ]]; then + local n="$arg" + ((n >= 1 && n <= ${#all_svcs[@]})) || continue + _add_target "${all_svcs[$((n - 1))]}" + continue + fi + + # treat as service/container name + svc="$(resolve_service "$arg")" + [[ -n "$svc" ]] && _add_target "$svc" + done + + [[ ${#targets[@]} -gt 0 ]] || die "No valid services selected." + } + + # ----------------------------- + # build target list + # ----------------------------- + if (($# == 0)); then + _pick_targets_interactive + elif [[ "${1,,}" == "all" ]]; then + compose_services_load + targets=("${__COMPOSE_SVCS[@]}") + [[ ${#targets[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" + else + for arg in "$@"; do + svc="$(resolve_service "$arg")" + [[ -n "$svc" ]] || continue + _add_target "$svc" + done + [[ ${#targets[@]} -gt 0 ]] || die "No valid services provided." + fi + + # ----------------------------- + # rebuild each target + # ----------------------------- + for svc in "${targets[@]}"; do + [[ -n "$svc" ]] || continue + compose_service_exists "$svc" || die "Unknown service/container: '$svc'" + + if compose_has_build "$svc"; then + logq rebuild "build/recreate $svc" + dc_build --no-cache --pull "$svc" + dc_up -d --no-deps --force-recreate "$svc" + continue + fi + + img="$(compose_image_for_service "$svc")" + logq rebuild "pull/recreate $svc${img:+ ($img)}" + + docker_compose rm -sf "$svc" >/dev/null 2>&1 || true + + if [[ -n "${img:-}" ]]; then + docker rmi -f "$img" >/dev/null 2>&1 || true + dc_pull "$svc" || true + else + dc_build --no-cache --pull "$svc" >/dev/null 2>&1 || true + fi + + dc_up -d --no-deps --force-recreate "$svc" + done + logq reboot "Rebooting stacks" + cmd_reboot +} + +cmd_config() { docker_compose config; } + +docker_shell() { + local c="${1:-}" + [[ -n "$c" ]] || die "container name required" + if docker exec "$c" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then + exec docker exec -it "$c" bash + else + exec docker exec -it "$c" sh + fi +} +cmd_tools() { + local sub="${1:-sh}" + shift || true + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + case "${sub,,}" in + sh | shell | "") + docker_shell "$ctr" + ;; + exec) + [[ $# -gt 0 ]] || die "tools exec " + docker exec -it "$ctr" sh -lc "$*" + ;; + file) + local p="${1:-}" + [[ -n "$p" ]] || die "tools file " + docker exec -it "$ctr" sh -lc "ls -la -- \"$p\" 2>/dev/null || true; echo; sed -n '1,200p' -- \"$p\" 2>/dev/null || true" + ;; + *) + die "tools " + ;; + esac +} +cmd_http() { [[ ${1:-} == reload ]] && http_reload; } +cmd_cli() { + local ctr="${1:-}" + shift || true + + [[ -n "$ctr" ]] || die "Usage: lds cli [cmd...]" + + docker inspect "$ctr" >/dev/null 2>&1 || die "Container not found: $ctr" + docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "Container not running: $ctr" + + # If user provided a command, run it; otherwise open an interactive shell. + if [[ "$#" -gt 0 ]]; then + local cmd="$*" + docker exec -it "$ctr" sh -lc ' + if command -v bash >/dev/null 2>&1; then + exec bash --login -lc "$1" + fi + exec sh -lc "$1" + ' sh "$cmd" + return + fi + + docker exec -it "$ctr" sh -lc ' + if command -v bash >/dev/null 2>&1; then + exec bash --login + fi + exec sh + ' +} + +cmd_core() { + # Usage: + # lds core -> open correct container for that domain (PHP/Node) + # lds core -> open a shell in that container + # lds core -> list domains and let user pick + + local target="${1:-}" + + # domain regex (same as domain-which/mkhost family) + local re='^([a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9])?\.)+(localhost|local|test|loc|[a-zA-Z]{2,})$' + + # If no target -> prompt from domain-which list + if [[ -z "$target" ]]; then + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + local -a domains=() + mapfile -t domains < <(docker exec "$tools_ctr" domain-which --list-domains 2>/dev/null | sed '/^[[:space:]]*$/d' || true) + + ((${#domains[@]} > 0)) || die "No domains found" + + # stable ordering + IFS=$'\n' domains=($(printf '%s\n' "${domains[@]}" | LC_ALL=C sort -u)) + + if ((${#domains[@]} == 1)); then + target="${domains[0]}" + else + if [[ ! -t 0 ]]; then + printf "%b[core]%b No domain provided. Available domains:\n" "$YELLOW" "$NC" >&2 + local i=1 + local d + for d in "${domains[@]}"; do + printf " %2d) %s\n" "$i" "$d" >&2 + ((i++)) + done + die "No TTY to prompt. Use: lds core " + fi + + printf "%bSelect domain:%b\n" "$CYAN" "$NC" >&2 + local i=1 d + for d in "${domains[@]}"; do + printf " %b%2d)%b %s\n" "$CYAN" "$i" "$NC" "$d" >&2 + ((i++)) + done + + local ans="" + while true; do + read -r -p "Enter number (1-${#domains[@]}): " ans + ans="$(echo "$ans" | xargs)" + [[ "$ans" =~ ^[0-9]+$ ]] || { + printf "%bInvalid input.%b\n" "$YELLOW" "$NC" >&2 + continue + } + ((ans >= 1 && ans <= ${#domains[@]})) || { + printf "%bOut of range.%b\n" "$YELLOW" "$NC" >&2 + continue + } + target="${domains[$((ans - 1))]}" + break + done + fi + fi + + # If target looks like a domain -> resolve via domain-which then shell in + if [[ "$target" =~ $re ]]; then + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + local app container wd + app="$(docker exec "$tools_ctr" domain-which --app --quiet "$target" 2>/dev/null)" || die "Unknown domain: $target" + container="$(docker exec "$tools_ctr" domain-which --container --quiet "$target" 2>/dev/null)" || die "No container resolved for: $target" + wd="$(docker exec "$tools_ctr" domain-which --docroot --quiet "$target" 2>/dev/null)" || true + [[ -n "${container:-}" ]] || die "No container resolved for: $target" + + # Node apps should always land at /app. Others follow resolved docroot. + if [[ "${app:-}" == "node" ]]; then + wd="/app" + fi + [[ -n "${wd:-}" ]] || wd="/app" + + docker exec -it "$container" bash -lc "cd \"$wd\" 2>/dev/null || cd /app 2>/dev/null || cd /; exec bash" + return 0 + fi + + # Otherwise treat target as a container name + docker exec -it "$(printf '%s' "$target" | tr '[:lower:]' '[:upper:]')" sh -lc 'exec bash -i || exec sh' +} + +cmd_setup() { + add_required_env + case ${1:-} in + init) env_init ;; + permission | permissions | perms | perm) fix_perms ;; + domain) setup_domain ;; + profiles | profile) process_all ;; + *) die "setup " ;; + esac +} + +cmd_certificate() { + case ${1:-} in + install) + shift || true + install_ca + ;; + uninstall | remove | rm) + shift || true + uninstall_ca "${@:-}" + ;; + *) + die "certificate " + ;; + esac +} + +############################################################################### +# NOTIFY +############################################################################### +notify_watch() { + local container="${1:-}" + if [[ -z "$container" ]]; then + container="$(_project_tools_container_running || true)" + [[ -n "$container" ]] || die "server-tools container is not running for project: $(lds_project)" + fi + local prefix="__HOST_NOTIFY__" + + need docker + + local _disp="${DISPLAY-}" + local _dbus="${DBUS_SESSION_BUS_ADDRESS-}" + + # Args: timeout(ms) urgency title body + _host_notify() { + local timeout="${1:-2500}" urgency="${2:-normal}" title="${3:-Notification}" body="${4:-}" + + # Linux desktop (or WSLg) + if has_cmd notify-send; then + (env DISPLAY="${_disp-}" DBUS_SESSION_BUS_ADDRESS="${_dbus-}" \ + setsid -f notify-send -u "$urgency" -t "$timeout" "$title" "$body" \ + >/dev/null 2>&1 || true) & + return 0 + fi + + # Windows toast (Git Bash) / WSL-on-Windows + if has_cmd powershell.exe; then + # Pass values as args to avoid quoting issues entirely. + # Note: urgency/timeout not used by toast api here; kept for parity. + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command \ + 'param([string]$t,[string]$b) + try { + [Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null + [Windows.Data.Xml.Dom.XmlDocument, Windows.Data.Xml.Dom.XmlDocument, ContentType = WindowsRuntime] > $null + + function Esc([string]$s) { + if ($null -eq $s) { return "" } + return ($s -replace "&","&" -replace "<","<" -replace ">",">" -replace "\"",""" -replace "'\''","'") + } + + $title = Esc $t + $body = Esc $b + + $xml = New-Object Windows.Data.Xml.Dom.XmlDocument + $xml.LoadXml("$title$body") + $toast = New-Object Windows.UI.Notifications.ToastNotification $xml + [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier("Devtainer").Show($toast) + } catch { }' \ + --% "$title" "$body" >/dev/null 2>&1 || true + + return 0 + fi + + # Fallback + printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 + return 0 + } + + trap - ERR + set +e + set +o pipefail + + local _stop=0 + + _watcher_notify() { + local urgency="${1:-critical}" title="${2:-Notifier}" body="${3:-Watcher event}" + _host_notify 2500 "$urgency" "$title" "$body" + } + + _watcher_int_term() { + _stop=1 + _watcher_notify critical "Notifier" "Notification watcher interrupted/exiting" + printf "%b[watcher]%b Notification watcher interrupted/exiting\n" "$RED" "$NC" >&2 + } + trap _watcher_int_term INT TERM + + local grep_cmd=(grep -a --line-buffered -E "^${prefix}([[:space:]]|$)") + has_cmd stdbuf && grep_cmd=(stdbuf -oL -eL "${grep_cmd[@]}") + + printf "%bNotify Watch:%b monitoring is active. Ctrl+C to stop.\n" "$GREEN" "$NC" + + while ((_stop == 0)); do + if ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then + _watcher_notify critical "Notifier" "Watcher stopped: $container is not running" + printf "%b[watcher]%b %s is not running; exiting.\n" "$RED" "$NC" "$container" >&2 + break + fi + + docker logs -f --tail 0 "$container" 2>&1 | + ("${grep_cmd[@]}" || true) | + while IFS=$'\t' read -r _ f1 f2 f3 f4 rest; do + local timeout urgency title body + + if [[ "${f1:-}" =~ ^[0-9]{1,6}$ ]]; then + timeout="$f1" + urgency="${f2:-normal}" + title="${f3:-Notification}" + body="${f4:-}" + else + timeout="2500" + urgency="${f1:-normal}" + title="${f2:-Notification}" + body="${f3:-}" + fi + + [[ -n "${rest:-}" ]] && body+=$'\t'"${rest}" + case "$urgency" in low | normal | critical) ;; *) urgency="normal" ;; esac + + _host_notify "$timeout" "$urgency" "$title" "$body" + printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 + done + + ((_stop)) && break + + if docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then + _watcher_notify critical "Notifier" "Watcher lost log stream (docker logs ended). Reconnecting…" + printf "%b[watcher]%b docker logs ended; reconnecting...\n" "$YELLOW" "$NC" >&2 + sleep 1 + continue + fi + + _watcher_notify critical "Notifier" "Watcher stopped: $container stopped" + printf "%b[watcher]%b %s stopped; exiting.\n" "$RED" "$NC" "$container" >&2 + break + done + + trap - INT TERM + set -euo pipefail + + ((_stop)) && return 130 + return 0 +} + +notify_test() { + local title="${1:-Notifier OK}" + local body="${2:-Hello from host via project server-tools container}" + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" notify -t 2500 -u normal "$title" "$body" +} + +cmd_notify() { + case ${1:-watch} in + watch) notify_watch "${2:-}" ;; + test) notify_test "${2:-Notifier OK}" "${3:-Hello from host}" ;; + *) die "notify " ;; + esac +} + +open_url() { + local url="${1:-}" + [[ -n "$url" ]] || return 0 + + # WSL/Windows helpers first when available + if grep -qi microsoft /proc/version 2>/dev/null; then + if has_cmd powershell.exe; then + powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 || true + return 0 + fi + if has_cmd cmd.exe; then + cmd.exe /c start "" "$url" >/dev/null 2>&1 || true + return 0 + fi + fi + + if has_cmd xdg-open; then + (xdg-open "$url" >/dev/null 2>&1 &) + return 0 + fi + if has_cmd open; then + (open "$url" >/dev/null 2>&1 &) + return 0 + fi + if has_cmd powershell; then + (powershell -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 &) + return 0 + fi + + printf "%bINFO%b: open this URL manually → %s\n" "$YELLOW" "$NC" "$url" +} + +############################################################################### +# RUN (ad-hoc Dockerfile runner) +############################################################################### +hash_short() { + local s="$1" + if has_cmd sha1sum; then + printf '%s' "$s" | sha1sum | cut -c1-8 + elif has_cmd shasum; then + printf '%s' "$s" | shasum -a 1 | cut -c1-8 + else + # POSIX fallback; stable (not cryptographic) + printf '%s' "$s" | cksum | awk '{print $1}' + fi +} + +run_slug() { + local dir="$1" base hash + base="$(basename "$dir" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9._-' '-')" + hash="$(hash_short "$dir")" + printf '%s-%s' "$base" "$hash" +} + +run_plan() { + local dir="$1" slug + slug="$(run_slug "$dir")" + printf '%s|%s|%s\n' \ + "lds-run-${slug}" \ + "${slug}:local" \ + "$dir" +} + +detect_host_os() { + if is_windows_shell || grep -qi microsoft /proc/version 2>/dev/null; then + printf 'windows' + return 0 + fi + + if has_cmd uname; then + case "$(uname -s 2>/dev/null || true)" in + Darwin) + printf 'macos' + ;; + Linux) + printf 'linux' + ;; + *) + printf 'unknown' + ;; + esac + else + printf 'unknown' + fi +} + +run_find_container() { + local dir="$1" + docker ps -a --filter "label=com.infocyph.lds.run=1" \ + --filter "label=com.infocyph.lds.dir=${dir}" \ + --format '{{.Names}}' | head -n 1 +} + +run_build() { + local tag="$1" dir="$2" + + # Build only if the image doesn't already exist. + if docker image inspect "$tag" >/dev/null 2>&1; then + printf "%b[run]%b Image exists, skipping build: %b%s%b +" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" + return 0 + fi + + printf "%b[run]%b Building image %b%s%b from %s +" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" "$dir" + docker build -t "$tag" "$dir" +} + +run_start() { + local name="$1" tag="$2" dir="$3" keepalive="$4" sock="$5" host_os="$6" + shift 6 || true + + # Remaining args are split by a "--" sentinel: + # - before "--" : publish specs (HOST:CONT), repeatable + # - after "--" : mount specs (HOST[:CONT]), repeatable + local -a pubs=() mounts=() + local seen_delim=0 x + for x in "$@"; do + if [[ "$x" == "--" ]]; then + seen_delim=1 + continue + fi + if ((seen_delim)); then + mounts+=("$x") + else + pubs+=("$x") + fi + done + + # Normalize project dir (POSIX absolute). + # On Windows Git Bash, /e/... is OK; docker.exe will receive converted path automatically. + local dir_posix + dir_posix="$(cd "$dir" 2>/dev/null && pwd -P)" || die "invalid dir: $dir" + + # MSYS-safe container paths: use '//' prefix to prevent path conversion. + # Docker interprets //path as /path inside container. + local WDIR="//workspace" + local WDIR_MOUNT="${dir_posix}://workspace" + + local -a args=(docker run -d --name "$name" + --label "com.infocyph.lds.run=1" + --label "com.infocyph.lds.dir=$dir_posix" + --label "com.infocyph.lds.tag=$tag" + -w "$WDIR" + -v "$WDIR_MOUNT" + ) + + if [[ -n "$host_os" ]]; then + args+=(-e "HOST_OS=$host_os") + fi + + # Mount extra directories/files (HOST[:CONT]). + # - If container path missing, mounts under /mnt/. + # - HOST may be relative to the run directory. + if ((${#mounts[@]})); then + local spec host cont base + for spec in "${mounts[@]}"; do + [[ -n "$spec" ]] || continue + host="$spec" + cont="" + + # Split as HOST:CONT ONLY if suffix after last ':' looks like a container absolute path (/...) + # (safe for Windows drive letters like E:\... because tail won't start with '/') + if [[ "$spec" == *:* ]]; then + local tail="${spec##*:}" + if [[ "$tail" == /* ]]; then + host="${spec%:*}" + cont="$tail" + fi + fi + + # Resolve host to absolute (POSIX) for checks + if [[ "$host" != /* && "$host" != ~* && ! "$host" =~ ^[A-Za-z]:[\\/].* ]]; then + host="${dir_posix%/}/$host" + fi + + # If user provided Windows path (E:\...), convert to POSIX for existence check + if [[ "$host" =~ ^[A-Za-z]:[\\/].* ]] && has_cmd cygpath; then + host="$(cygpath -u "$host")" + fi + + host="$(cd "${host%/*}" 2>/dev/null && pwd -P)/${host##*/}" || { + printf "%b[run]%b Warning: cannot resolve mount path: %s\n" "$YELLOW" "$NC" "$spec" >&2 + continue + } + + [[ -e "$host" ]] || { + printf "%b[run]%b Warning: mount path does not exist: %s\n" "$YELLOW" "$NC" "$host" >&2 + continue + } + + if [[ -z "$cont" ]]; then + base="${host##*/}" + cont="/mnt/${base}" + fi + [[ "$cont" == /* ]] || cont="/mnt/${cont}" + + # Prevent MSYS conversion for container side by using '//' prefix + cont="//${cont#/}" + + args+=(-v "${host}:${cont}") + done + fi + + # Optional docker sock + if [[ "${sock:-0}" == 1 ]]; then + args+=(-v "/var/run/docker.sock:/var/run/docker.sock") + fi + + # Publish ports + local pub + for pub in "${pubs[@]}"; do + [[ -n "$pub" ]] || continue + args+=(-p "$pub") + done + + if [[ "$keepalive" == 1 ]]; then + # Keepalive mode replaces the image command; disable image healthcheck to avoid false "unhealthy". + args+=(--no-healthcheck --entrypoint sh "$tag" -c "trap : TERM INT; sleep infinity & wait") + else + args+=("$tag") + fi + + printf "%b[run]%b Starting container %b%s%b\n" "$CYAN" "$NC" "$BLUE" "$name" "$NC" + + # IMPORTANT: don't hide errors; if it fails, you need to see why + if ! "${args[@]}"; then + printf "%b[run]%b docker run failed.\n" "$RED" "$NC" >&2 + return 1 + fi + printf "\n" +} + +run_exec_shell() { + local name="$1" + if docker exec "$name" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then + exec docker exec -it "$name" bash + else + exec docker exec -it "$name" sh + fi +} + +cmd_run() { + local action="*" dir="$PWD" name="" tag="" nobuild=0 keepalive=1 sock=0 host_os="" + local -a publish=() mounts=() + local open_port="" open_path="/" open_proto="http" + + while [[ $# -gt 0 ]]; do + case "$1" in + stop | rm | ps | shell | logs | open | "*") + action="$1" + shift + ;; + build) + action="*" + shift + ;; + --name) + name="${2:-}" + shift 2 + ;; + --tag) + tag="${2:-}" + shift 2 + ;; + --no-build) + nobuild=1 + shift + ;; + --no-keepalive) + keepalive=0 + shift + ;; + --sock) + sock=1 + shift + ;; + --host-os) + host_os="${2:-}" + shift 2 + ;; + -p | --publish) + publish+=("${2:-}") + shift 2 + ;; + --mount) + mounts+=("${2:-}") + shift 2 + ;; + --port) + open_port="${2:-}" + shift 2 + ;; + --path) + open_path="${2:-/}" + shift 2 + ;; + --https) + open_proto="https" + shift + ;; + --http) + open_proto="http" + shift + ;; + *) break ;; + esac + done + + # Host path (POSIX) for planning/labels + local dir_posix + dir_posix="$(cd "$dir" && pwd -P)" + + if [[ -z "$host_os" ]]; then + host_os="${HOST_OS:-$(detect_host_os)}" + fi + + # Docker path (may need Windows form for docker.exe) + local dir_docker="$dir_posix" + + # Windows Git Bash/MSYS hardening: + # - stop MSYS rewriting container paths (/workspace -> D:/Program Files/Git/workspace) + # - but still feed docker.exe Windows-absolute host paths for build/run contexts + if is_windows_shell; then + export MSYS_NO_PATHCONV=1 + export MSYS2_ARG_CONV_EXCL='*' + if has_cmd cygpath; then + dir_docker="$(cygpath -w "$dir_posix")" + fi + fi + + # Plan/name/tag should be based on the real project identity (POSIX dir) + IFS='|' read -r def_name def_tag _def_dir < <(run_plan "$dir_posix") + name="${name:-$def_name}" + + # Tag rules: + # - Default tag is ":local" (from run_plan) + # - If user passes --tag without ":", append ":local" + if [[ -n "${tag:-}" ]]; then + if [[ "$tag" != *:* ]]; then + tag="${tag}:local" + fi + else + tag="$def_tag" + fi + + _find_for_dir() { + local found + found="$(run_find_container "$dir_posix" || true)" + if [[ -n "$found" ]]; then + printf '%s' "$found" + return 0 + fi + if docker inspect "$name" >/dev/null 2>&1; then + printf '%s' "$name" + return 0 + fi + return 1 + } + + _run_build_summary() { + local img="$1" build_dir="$2" cname="$3" + local tag_only="${img##*:}" + + printf "\n%b[run]%b Build summary\n" "$CYAN" "$NC" + printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" + printf " %bTag:%b %s\n" "$BOLD" "$NC" "$tag_only" + printf " %bDir:%b %s\n" "$BOLD" "$NC" "$build_dir" + printf " %bName:%b %s\n" "$BOLD" "$NC" "$cname" + printf " %bKeepalive:%b %s\n" "$BOLD" "$NC" "$keepalive" + printf " %bSock:%b %s\n" "$BOLD" "$NC" "$sock" + printf " %bHost OS:%b %s\n" "$BOLD" "$NC" "$host_os" + + if ((${#publish[@]})); then + printf " %bPublish:%b %s\n" "$BOLD" "$NC" "${publish[*]}" + else + printf " %bPublish:%b (none)\n" "$BOLD" "$NC" + fi + + if ((${#mounts[@]})); then + printf " %bMounts:%b %s\n" "$BOLD" "$NC" "${mounts[*]}" + else + printf " %bMounts:%b (none)\n" "$BOLD" "$NC" + fi + printf "\n" + } + + _run_runtime_summary() { + local cname="$1" + local id img state ports + id="$(docker inspect -f '{{.Id}}' "$cname" 2>/dev/null | cut -c1-12 || true)" + img="$(docker inspect -f '{{.Config.Image}}' "$cname" 2>/dev/null || true)" + state="$(docker inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || true)" + ports="$(docker port "$cname" 2>/dev/null | sed '/^[[:space:]]*$/d' | tr '\n' '; ' | sed 's/; $//' || true)" + + printf "%b[run]%b Runtime summary\n" "$CYAN" "$NC" + printf " %bContainer:%b %s\n" "$BOLD" "$NC" "${cname}${id:+ ($id)}" + [[ -n "$img" ]] && printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" + [[ -n "$state" ]] && printf " %bState:%b %s\n" "$BOLD" "$NC" "$state" + if [[ -n "$ports" ]]; then + printf " %bPorts:%b %s\n" "$BOLD" "$NC" "$ports" + else + printf " %bPorts:%b (none published)\n" "$BOLD" "$NC" + fi + printf "%b\n[run]%b Example Usage (in Composer)\n" "$CYAN" "$NC" + printf " %bimage:%b %s\n" "$BOLD" "$NC" "$img" + printf " %bpull_policy:%b never\n" "$BOLD" "$NC" + printf "\n" + } + + case "$action" in + ps) + docker ps -a --filter "label=com.infocyph.lds.run=1" \ + --format 'table {{.Names}} {{.Image}} {{.Status}} {{.Labels}}' + return 0 + ;; + stop) + local existing + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + docker stop "$existing" >/dev/null + printf "%b[run]%b Stopped %s\n" "$GREEN" "$NC" "$existing" + return 0 + ;; + logs) + local existing + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + exec docker logs -f "$existing" + ;; + open) + local existing line addr hp url + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + + [[ -n "$open_path" ]] || open_path="/" + [[ "$open_path" == /* ]] || open_path="/$open_path" + + if [[ -n "$open_port" ]]; then + line="$(docker port "$existing" "$open_port" 2>/dev/null | head -n 1 || true)" + [[ -n "$line" ]] || line="$(docker port "$existing" "${open_port}/tcp" 2>/dev/null | head -n 1 || true)" + else + line="$(docker port "$existing" 2>/dev/null | head -n 1 || true)" + fi + + if [[ -z "$line" ]]; then + printf "%b[run]%b No published ports found.\n" "$YELLOW" "$NC" + printf "%b[run]%b Tip: start with %blds run --publish 8025:8025%b then %blds run open%b\n" \ + "$YELLOW" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" + return 1 + fi + + addr="${line##*-> }" + hp="${addr##*:}" + url="${open_proto}://localhost:${hp}${open_path}" + open_url "$url" + printf "%b[run]%b Opened: %s\n" "$GREEN" "$NC" "$url" + return 0 + ;; + rm) + local existing img + existing="$(_find_for_dir)" || true + if [[ -n "${existing:-}" ]]; then + img="$(docker inspect -f '{{.Config.Image}}' "$existing" 2>/dev/null || true)" + docker stop "$existing" >/dev/null 2>&1 || true + docker rm "$existing" >/dev/null 2>&1 || true + printf "%b[run]%b Removed container %s\n" "$GREEN" "$NC" "$existing" + if [[ -n "${img:-}" ]]; then + docker rmi -f "$img" >/dev/null 2>&1 || true + printf "%b[run]%b Removed image %s\n" "$GREEN" "$NC" "$img" + fi + else + printf "%b[run]%b No container found for %s\n" "$YELLOW" "$NC" "$dir_posix" + fi + return 0 + ;; + shell | "*") + if ((nobuild == 0)); then + # Build needs docker.exe-friendly path on Windows + run_build "$tag" "$dir_docker" + else + printf "%b[run]%b Skipping build (--no-build)\n" "$YELLOW" "$NC" + fi + + _run_build_summary "$tag" "$dir_posix" "$name" + + if docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null | grep -q true; then + printf "%b[run]%b Container already running: %s\n\n" "$GREEN" "$NC" "$name" + else + if docker inspect "$name" >/dev/null 2>&1; then + docker rm -f "$name" >/dev/null 2>&1 || true + fi + + # Keep mounts as user gave them (POSIX/relative); run_start should validate POSIX + # and convert host-side to Windows only at docker run time. + run_start "$name" "$tag" "$dir_docker" "$keepalive" "$sock" "$host_os" \ + "${publish[@]}" -- "${mounts[@]}" + fi + + _run_runtime_summary "$name" + + # "shell" enters the container; "*" / "build" does not. + if [[ "$action" == "shell" ]]; then + run_exec_shell "$name" + else + printf "%b[run]%b Built/started. Use %blds run shell%b to enter, %blds run logs%b to follow logs.\n" \ + "$GREEN" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" + return 0 + fi + ;; + esac +} + +############################################################################### +# 6w. NEW FEATURES: stack diff | support trace +############################################################################### + +# stack diff: show what would run (compose) vs what's running (docker) +cmd_stack_diff() { + local json=0 + local show_config=0 + while [[ "${1:-}" ]]; do + case "$1" in + --json) + json=1 + shift + ;; + --config) + show_config=1 + shift + ;; + *) break ;; + esac + done + + local project + project="$(lds_project)" + local cfg_json="" + + if docker_compose config --format json >/dev/null 2>&1; then + cfg_json="$(docker_compose config --format json)" + else + # fallback: best-effort text config + cfg_json="" + fi + + # running: service -> image + declare -A running=() + local line + while IFS= read -r line; do + [[ -n "$line" ]] || continue + local svc="${line%%|*}" + local img="${line#*|}" + running["$svc"]="$img" + done < <(docker ps \ + --filter "label=com.docker.compose.project=$project" \ + --format '{{index .Labels "com.docker.compose.service"}}|{{.Image}}' 2>/dev/null || true) + + # desired: service -> image/build context (best-effort) + declare -A desired_img=() + declare -A desired_ctx=() + declare -A desired_df=() + + if [[ -n "$cfg_json" ]]; then + if has_tool jq; then + while IFS= read -r line; do + local svc="${line%%|*}" + local img="${line#*|}" + desired_img["$svc"]="$img" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"') + while IFS= read -r line; do + local svc="${line%%|*}" + local ctx="${line#*|}" + desired_ctx["$svc"]="$ctx" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"') + while IFS= read -r line; do + local svc="${line%%|*}" + local df="${line#*|}" + desired_df["$svc"]="$df" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"') + elif _server_tools_has jq; then + # Fallback: parse via project tools container jq through stdin (no shell re-quoting of JSON payload). + local ctr + ctr="$(_project_tools_container_running || true)" + if [[ -n "$ctr" ]]; then + while IFS= read -r line; do + local svc="${line%%|*}" + local img="${line#*|}" + desired_img["$svc"]="$img" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"' 2>/dev/null || true) + while IFS= read -r line; do + local svc="${line%%|*}" + local ctx="${line#*|}" + desired_ctx["$svc"]="$ctx" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"' 2>/dev/null || true) + while IFS= read -r line; do + local svc="${line%%|*}" + local df="${line#*|}" + desired_df["$svc"]="$df" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"' 2>/dev/null || true) + fi + fi + fi + + # Build result object + if ((json)); then + if has_tool jq; then + # assemble in bash -> jq + local tmp + tmp="$(mktemp)" + { + printf '{' + printf '"project":%s,' "$(printf '%s' "$project" | jq -Rsa .)" + printf '"compose_file":%s,' "$(printf '%s' "$COMPOSE_FILE" | jq -Rsa .)" + printf '"running":{' + local first=1 k + for k in "${!running[@]}"; do + ((first)) || printf ',' + first=0 + printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${running[$k]}" | jq -R .)" + done + printf '},' + printf '"desired":{' + first=1 + for k in "${!desired_img[@]}"; do + ((first)) || printf ',' + first=0 + printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${desired_img[$k]}" | jq -R .)" + done + printf '},' + printf '"diff":[' + first=1 + # union keys + declare -A seen=() + for k in "${!running[@]}"; do seen["$k"]=1; done + for k in "${!desired_img[@]}"; do seen["$k"]=1; done + for k in "${!seen[@]}"; do + local r="${running[$k]:-}" + local d="${desired_img[$k]:-}" + if [[ "$r" != "$d" ]]; then + ((first)) || printf ',' + first=0 + printf '{"service":%s,"running":%s,"desired":%s}' \ + "$(printf '%s' "$k" | jq -R .)" \ + "$(printf '%s' "$r" | jq -R .)" \ + "$(printf '%s' "$d" | jq -R .)" + fi + done + printf ']' + printf '}\n' + } >"$tmp" + cat "$tmp" | jq . + rm -f "$tmp" + else + die "jq required for --json (or run inside project server-tools container)" + fi + return 0 + fi + + printf "%bStack diff%b (project=%s)\n" "$CYAN" "$NC" "$project" + printf "%bCompose file:%b %s\n" "$DIM" "$NC" "$COMPOSE_FILE" + + if ((show_config)); then + if [[ -n "$cfg_json" ]]; then + printf "\n%bEffective compose config (json):%b\n" "$DIM" "$NC" + printf '%s\n' "$cfg_json" + else + printf "\n%bEffective compose config:%b\n" "$DIM" "$NC" + docker_compose config || true + fi + fi + + # union services + declare -A all=() + local svc + for svc in "${!running[@]}"; do all["$svc"]=1; done + for svc in "${!desired_img[@]}"; do all["$svc"]=1; done + + printf "\n%-22s %-40s %-40s %s\n" "SERVICE" "RUNNING" "DESIRED" "STATUS" + printf "%-22s %-40s %-40s %s\n" "------" "-------" "-------" "------" + for svc in $(printf '%s\n' "${!all[@]}" | sort); do + local r="${running[$svc]:-}" + local d="${desired_img[$svc]:-}" + local st + if [[ -z "$r" ]]; then + st="(not running)" + elif [[ -z "$d" ]]; then + st="(not in config)" + elif [[ "$r" == "$d" ]]; then + st="OK" + else + st="DIFF" + fi + printf "%-22s %-40.40s %-40.40s %s\n" "$svc" "$r" "$d" "$st" + done + + printf "\n%bNotes:%b\n" "$DIM" "$NC" + printf " - Desired image is derived from 'docker compose config'. If a service uses only 'build:' and no 'image:', desired may be empty.\n" + printf " - Use: lds stack diff --config (to print resolved compose config)\n" +} + +# support trace: quick end-to-end trace for a domain +cmd_support_trace() { + local dom="${1:-}" + [[ -n "$dom" ]] || die "support trace " + + local nconf="$DIR/configuration/nginx/$dom.conf" + printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" + + # 1) DNS + if _server_tools_running; then + printf "\n%b[DNS]%b\n" "$DIM" "$NC" + _tools_exec "dig +short $(_shq "$dom") || true; getent hosts $(_shq "$dom") 2>/dev/null || true" + else + printf "\n%b[DNS]%b\n" "$DIM" "$NC" + (has_cmd dig && dig +short "$dom") || true + (has_cmd getent && getent hosts "$dom") || true + fi + + # 2) TLS certificate + printf "\n%b[TLS]%b\n" "$DIM" "$NC" + if _server_tools_running; then + _tools_exec "echo | openssl s_client -connect $(_shq "$dom"):443 -servername $(_shq "$dom") -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true" + else + echo | openssl s_client -connect "${dom}:443" -servername "$dom" -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true + fi + + # 3) HTTP probe (timings) + printf "\n%b[HTTP]%b\n" "$DIM" "$NC" + if _server_tools_running; then + _tools_exec "curl -sk -o /dev/null -D - -w 'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' https://$(_shq "$dom") | sed -n '1,30p'" + else + curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' + fi + + # 4) Upstream inference from nginx conf (if exists) + printf "\n%b[Upstream]%b\n" "$DIM" "$NC" + if [[ -r "$nconf" ]]; then + if grep -q fastcgi_pass "$nconf"; then + local php + php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" + elif grep -q proxy_pass "$nconf"; then + local up + up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" + else + printf "type=static\n" + fi + else + printf "nginx_conf=%s (missing)\n" "$nconf" + fi + + # 5) Recent nginx logs (compose) + printf "\n%b[Recent nginx logs]%b\n" "$DIM" "$NC" + docker_compose logs --no-color --tail 120 nginx 2>/dev/null | text_grep -i "$dom" || docker_compose logs --no-color --tail 120 nginx 2>/dev/null || true + + printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" +} + +############################################################################### +# 6x. GROUPED COMMAND ROUTERS (stack/domain/support) + backward-compatible aliases +############################################################################### + +cmd_stack() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + "" | help | -h | --help) cmd_help stack ;; + up) cmd_up "$@" ;; + start) cmd_start "$@" ;; + down | stop) cmd_down "$@" ;; + restart | reboot) cmd_restart "$@" ;; + status) cmd_status "$@" ;; + ps) cmd_ps "$@" ;; + logs) cmd_logs "$@" ;; + exec) cmd_exec "$@" ;; + events) cmd_events "$@" ;; + clean) cmd_clean "$@" ;; + config) cmd_config "$@" ;; diff) cmd_stack_diff "$@" ;; + + http) cmd_http "$@" ;; + *) + die "stack " + ;; + esac +} + +# Canonical: domain. Legacy: host. +cmd_domain() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + "" | help | -h | --help) die "domain " ;; + add) cmd_host add "$@" ;; + rm | remove | del | delete) cmd_host rm "$@" ;; + ls | list) cmd_host list "$@" ;; + *) + die "domain " + ;; + esac +} + +cmd_support() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + "" | help | -h | --help) die "support " ;; + open) cmd_open "$@" ;; + bundle) cmd_bundle "$@" ;; + notify) cmd_notify "$@" ;; + ui) cmd_ui "$@" ;; + *) + die "support " + ;; + esac +} + +# Minimal support bundle (shareable diagnostics zip) +# Modes: +# --redact : remove secrets-like env lines from captured env/config outputs (best-effort) +# --full : include more logs/inspect output (can be large) +cmd_bundle() { + local mode="redact" + local out="${1:-}" + while [[ "${1:-}" ]]; do + case "$1" in + --redact) + mode="redact" + shift + ;; + --full) + mode="full" + shift + ;; + *.zip) + out="$1" + shift + ;; + *) break ;; + esac + done + + need zip + local ts + ts="$(date +%Y%m%d_%H%M%S)" + local project + project="$(lds_project)" + local tmp + tmp="$(mktemp -d "${TMPDIR:-/tmp}/lds_bundle.XXXXXX")" + local base="lds_bundle_${project}_${ts}" + [[ -n "$out" ]] || out="$PWD/${base}.zip" + + cleanup_bundle() { rm -rf "$tmp" 2>/dev/null || true; } + trap cleanup_bundle RETURN + + { + echo "project=$project" + echo "dir=$DIR" + echo "time=$ts" + echo "mode=$mode" + } >"$tmp/meta.txt" + + # compose config (effective) + { + echo "# docker compose config" + docker_compose config 2>&1 || true + } >"$tmp/compose.config.txt" + + # ps + networks + docker_compose ps >"$tmp/compose.ps.txt" 2>&1 || true + docker network ls >"$tmp/docker.networks.txt" 2>&1 || true + + # container list + inspect (scoped) + docker ps --filter "label=com.docker.compose.project=$project" --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}' \ + >"$tmp/docker.ps.txt" 2>&1 || true + + if [[ "$mode" == "full" ]]; then + docker inspect $(docker ps -q --filter "label=com.docker.compose.project=$project" 2>/dev/null) \ + >"$tmp/docker.inspect.json" 2>/dev/null || true + fi + + # recent logs (tail) + docker_compose logs --no-color --tail 400 >"$tmp/compose.logs.txt" 2>&1 || true + + # vhost configs + if [[ -d "$DIR/configuration/nginx" ]]; then + mkdir -p "$tmp/nginx" + cp -a "$DIR/configuration/nginx/." "$tmp/nginx/" 2>/dev/null || true + fi + + # env files (redacted best-effort) + mkdir -p "$tmp/env" + if [[ -r "$ENV_MAIN" ]]; then + if [[ "$mode" == "redact" ]]; then + sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_MAIN" >"$tmp/env/.env" + else + cp -a "$ENV_MAIN" "$tmp/env/.env" 2>/dev/null || true + fi + fi + if [[ -r "$ENV_DOCKER" ]]; then + if [[ "$mode" == "redact" ]]; then + sed -E 's/^([A-Za-z0-9_]+)=.*/\1=REDACTED/' "$ENV_DOCKER" >"$tmp/env/docker.env" + else + cp -a "$ENV_DOCKER" "$tmp/env/docker.env" 2>/dev/null || true + fi + fi + + # tools-side quick diagnostics (inside network) + { + echo "# ip r / ip a / ss" + _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || true" + } >"$tmp/tools.net.txt" 2>&1 || true + + # pack + (cd "$tmp" && zip -qr "$out" .) || die "Failed to write bundle: $out" + ok "Bundle created: $out" +} + +cmd_help() { + + if [[ "${1:-}" == "--markdown" ]]; then + cat <<'MD' +# LocalDevStack (lds) — Command Reference + +## Stack (compose) +- `lds stack up` *(aliases: `up`)* +- `lds stack start` *(aliases: `start`)* +- `lds stack down [--volumes --yes]` *(aliases: `down`, `stop`)* +- `lds stack restart [svc]` *(aliases: `restart`, `reboot`)* +- `lds stack status [status-args…]` *(alias: `status`; forwards args to tools `status`)* +- `lds stack ps` *(alias: `ps`)* +- `lds stack logs [svc] [--follow] [--since ] [--grep ]` *(alias: `logs`)* +- `lds stack exec [cmd…]` *(alias: `exec`)* +- `lds stack events [--since ]` *(alias: `events`)* +- `lds stack clean --yes [--volumes]` *(alias: `clean`)* +- `lds stack diff [--config] [--json]` *(shows desired vs running images)* + + +## Domain (vhost lifecycle + routing) +- `lds domain add …` +- `lds domain rm …` +- `lds domain ls` + +Legacy alias: `lds host …` → same subcommands as `domain`. + +## Certificates (TLS) +- `lds cert status [domain|all]` +- `lds cert regen [domain|all] [--yes]` +- `lds cert diagnose ` +- `lds certificate install` +- `lds certificate uninstall [--all]` + +## Diagnostics +- `lds diag dns ` +- `lds diag net` +- `lds diag tcp ` +- `lds diag http ` *(alias: `sniff `)* +- `lds diag tls ` + +## Config +- `lds config show [--json]` +- `lds config services` +- `lds config profiles` +- `lds config env-used` +- `lds config validate` + +## Support +- `lds support open + support trace ` +- `lds support bundle [--redact|--full]` +- `lds support notify …` +- `lds support ui` + +Shortcuts: `open`, `bundle`, `notify`, `ui` map to `support …`. + +## Secrets (senv) +- `lds secrets …` + +## Tools (project server-tools container) +- `lds tools sh` +- `lds tools exec ""` +- `lds tools file ` + +## AI +- `lds ai status|ask|explain|troubleshoot|review|repo-review|graphify …` +- `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit …` +- `lds llm runtime ` +- `lds llm host-port ` + +## Setup +- `lds setup init|permissions|domain|profiles` + +## Runner (ad‑hoc Dockerfile runner) +- `lds run` *(default: build+start only)* / `lds run shell` *(build+start+enter)* / `lds run *` *(same as default)* (+ `ps|logs|stop|rm|open` and flags: `--publish|-p`, `--no-keepalive`, `--mount`, `--sock`, `--host-os`, `--tag`, `--name`) + +## Other +- `lds rebuild [all|]` +- `lds core [domain]` + +## Short client wrappers +- `lds php|composer|node|npm|npx …` +- `lds my|mysql|mysqldump …` +- `lds maria|mariadb|mariadb-dump …` +- `lds pg|psql|pg_dump|pg_restore …` +- `lds redis|redis-cli …` +MD + return 0 + fi + + cat < + clean Alias of: stack <...> + +${CYAN}Domain (vhosts + routing):${NC} + domain add|rm|ls + host add|rm|list Legacy alias of: domain <...> + +${CYAN}Certificates (TLS):${NC} + cert status|regen|diagnose + certificate install|uninstall [--all] + +${CYAN}Diagnostics:${NC} + diag dns|net|tcp|http|tls + sniff Alias of: diag http + +${CYAN}Config:${NC} + config show|services|profiles|env-used|validate + +${CYAN}Support:${NC} + support open + support trace + support bundle [--redact|--full] + support notify ... + support ui + open|bundle|notify|ui Shortcuts → support <...> + +${CYAN}Secrets:${NC} + secrets + +${CYAN}Tools (project server-tools container):${NC} + tools sh|exec|file + +${CYAN}AI:${NC} + ai status|ask|explain|troubleshoot|review|repo-review|graphify + llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit + llm runtime + llm host-port + +${CYAN}Setup:${NC} + setup init|permissions|domain|profiles + +${CYAN}Runner (ad-hoc Dockerfile runner):${NC} + run [ps|logs|stop|rm|open] [--publish|-p A:B] [--no-keepalive] [--mount HOST[:CONT]] [--sock] [--host-os VALUE] + +${CYAN}Other:${NC} + rebuild [all|] + core [domain] + +${CYAN}Help:${NC} + help [--markdown] + +EOF +} + +############################################################################### +# 7. MAIN +############################################################################### +main() { + [[ $# -gt 0 ]] || { + cmd_help + exit 1 + } + + while [[ $# -gt 0 ]]; do + case "$1" in + -v | --verbose) + VERBOSE=1 + QUIET=0 + shift + ;; + -q | --quiet) + QUIET=1 + VERBOSE=0 + shift + ;; + --reload-extras) + export LDS_EXTRAS_RELOAD=1 + shift + ;; + --) + shift + break + ;; + -h | --help) + cmd_help + exit 0 + ;; + -*) die "Unknown global option: $1" ;; + *) break ;; + esac + done + + [[ $# -gt 0 ]] || { + cmd_help + exit 1 + } + + local cmd="${1,,}" + shift || true + + if [[ "$cmd" == "help" ]]; then + cmd_help "$@" + exit 0 + fi + + # Do not require docker for pure help output; everything else expects the stack. + need docker + + case "$cmd" in + php | composer | node | npm | npx) exec "$DIR/bin/$cmd" "$@" ;; + pg | psql | pg_restore | pg-restore | pgrestore | pg_dump | pgdump | pg-dump) exec "$DIR/bin/pg" "$@" ;; + maria | mariadb | mariadbdump | mariadb-dump | mariadb_dump) exec "$DIR/bin/maria" "$@" ;; + my | mysql | mysqldump | mysql-dump | mysql_dump) exec "$DIR/bin/my" "$@" ;; + redis | redis-cli) exec "$DIR/bin/redis-cli" "$@" ;; + es | elastic | elasticsearch) exec "$DIR/bin/es" "$@" ;; + mongo | mongodb | mongosh | mongoimport | mongoexport) exec "$DIR/bin/mongo" "$@" ;; + ai) cmd_ai "$@" ;; + llm) cmd_llm "$@" ;; + vpn-fix) cmd_vpn_fix "$@" ;; + *) + if declare -F "cmd_$cmd" >/dev/null 2>&1; then + "cmd_$cmd" "$@" + else + exec "$DIR/bin/tool-runner" "$cmd" "$@" + fi + ;; + esac +} + +main "$@" From a2ad6c61e211abcdfb5158f0135bf85f9e26638d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:35:14 +0600 Subject: [PATCH 041/509] catalog: track LocalDevStack service metadata --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitignore b/.gitignore index c224cecb..5b1bd229 100644 --- a/.gitignore +++ b/.gitignore @@ -3,6 +3,8 @@ !configuration !data/.gitignore !docker/compose +!docker/catalog/ +!docker/catalog/** !docker/conf !docker/dockerfiles !docker/release.env From 939c8997747e3c2c2e99341c70c41e4c7f3808a1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:35:17 +0600 Subject: [PATCH 042/509] catalog: add canonical host service catalog --- docker/catalog/services.psv | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 docker/catalog/services.psv diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv new file mode 100644 index 00000000..3aac4193 --- /dev/null +++ b/docker/catalog/services.psv @@ -0,0 +1,8 @@ +# key|profile|display_name|service_key|version_env|setup_defaults|setup_prompts|admin_client|volume|url|category|optional|default_enabled|runtime_modes +POSTGRESQL|postgresql|PostgreSQL|postgres|POSTGRES_VERSION|POSTGRES_VERSION=latest;POSTGRES_USER=postgres;POSTGRES_PASSWORD=postgres;POSTGRES_DATABASE=postgres|Version;User;Password;Database|cloudbeaver|PostgresStore|https://db.localhost|database|1|0| +MYSQL|mysql|MySQL|mysql|MYSQL_VERSION|MYSQL_VERSION=latest;MYSQL_ROOT_PASSWORD=12345;MYSQL_USER=infocyph;MYSQL_PASSWORD=12345;MYSQL_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MySQLStore|https://db.localhost|database|1|0| +MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_ROOT_PASSWORD=12345;MARIADB_USER=infocyph;MARIADB_PASSWORD=12345;MARIADB_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MariaDBStore|https://db.localhost|database|1|0| +ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.3.0|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| +MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| +REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| +AI|ai|Local AI|llm-sm|LDS_LLM_IMAGE|LDS_AI_RUNTIME=cpu;LDS_AI_MODEL=qwen2.5:3b;LDS_LLM_HOST_PORT=no|Runtime (cpu/nvidia/amd);Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd From fdc65837b2359d761f559def8d112194583ef270 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:35:55 +0600 Subject: [PATCH 043/509] catalog: drive profile setup from canonical metadata --- lds | 116 ++++++++++++++++++++++++++++++++++++++++++++++-------------- 1 file changed, 89 insertions(+), 27 deletions(-) diff --git a/lds b/lds index 51b939d0..6a1040e6 100755 --- a/lds +++ b/lds @@ -805,25 +805,56 @@ modify_profiles() { # 3a. PROFILES: DEFINITIONS + SETUP FLOW ############################################################################### -declare -A SERVICES=( - [POSTGRESQL]="postgresql" - [MYSQL]="mysql" - [MARIADB]="mariadb" - [ELASTICSEARCH]="elasticsearch" - [MONGODB]="mongodb" - [REDIS]="redis" -) - -declare -a SERVICE_ORDER=(POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS) - -declare -A PROFILE_ENV=( - [elasticsearch]="ELASTICSEARCH_VERSION=9.3.0" - [mysql]="MYSQL_VERSION=latest MYSQL_ROOT_PASSWORD=12345 MYSQL_USER=infocyph MYSQL_PASSWORD=12345 MYSQL_DATABASE=localdb" - [mariadb]="MARIADB_VERSION=latest MARIADB_ROOT_PASSWORD=12345 MARIADB_USER=infocyph MARIADB_PASSWORD=12345 MARIADB_DATABASE=localdb" - [mongodb]="MONGODB_VERSION=latest MONGODB_ROOT_USERNAME=root MONGODB_ROOT_PASSWORD=12345" - [redis]="REDIS_VERSION=latest" - [postgresql]="POSTGRES_VERSION=latest POSTGRES_USER=postgres POSTGRES_PASSWORD=postgres POSTGRES_DATABASE=postgres" -) +CATALOG_FILE="$CFG/catalog/services.psv" + +declare -A SERVICES=() +declare -A SERVICE_DISPLAY=() +declare -A SERVICE_KEY=() +declare -A SERVICE_VERSION_ENV=() +declare -A PROFILE_ENV=() +declare -A PROFILE_PROMPTS=() +declare -A SERVICE_ADMIN_CLIENT=() +declare -A SERVICE_VOLUME=() +declare -A SERVICE_URL=() +declare -A SERVICE_CATEGORY=() +declare -A SERVICE_OPTIONAL=() +declare -A SERVICE_DEFAULT_ENABLED=() +declare -A SERVICE_RUNTIME_MODES=() +declare -a SERVICE_ORDER=() + +load_service_catalog() { + [[ -r "$CATALOG_FILE" ]] || die "Missing service catalog: $CATALOG_FILE" + + local key profile display service_key version_env defaults prompts + local admin_client volume url category optional default_enabled runtime_modes + + while IFS='|' read -r key profile display service_key version_env defaults prompts admin_client volume url category optional default_enabled runtime_modes; do + [[ -n "$key" && "$key" != \#* ]] || continue + [[ -n "$profile" && -n "$display" && -n "$service_key" ]] || + die "Invalid service catalog row for: $key" + [[ -z "${SERVICES[$key]+x}" ]] || die "Duplicate service catalog key: $key" + [[ -z "${PROFILE_ENV[$profile]+x}" ]] || die "Duplicate service catalog profile: $profile" + + SERVICE_ORDER+=("$key") + SERVICES["$key"]="$profile" + SERVICE_DISPLAY["$key"]="$display" + SERVICE_KEY["$key"]="$service_key" + SERVICE_VERSION_ENV["$key"]="$version_env" + PROFILE_ENV["$profile"]="$defaults" + PROFILE_PROMPTS["$profile"]="$prompts" + SERVICE_ADMIN_CLIENT["$key"]="$admin_client" + SERVICE_VOLUME["$key"]="$volume" + SERVICE_URL["$key"]="$url" + SERVICE_CATEGORY["$key"]="$category" + SERVICE_OPTIONAL["$key"]="$optional" + SERVICE_DEFAULT_ENABLED["$key"]="$default_enabled" + SERVICE_RUNTIME_MODES["$key"]="$runtime_modes" + done <"$CATALOG_FILE" + + (("${#SERVICE_ORDER[@]}" > 0)) || die "Service catalog is empty: $CATALOG_FILE" +} + +load_service_catalog declare -a PENDING_ENVS=() declare -a PENDING_PROFILES=() @@ -852,10 +883,11 @@ setup_menu_print() { # Print menu to stderr to avoid stdout buffering in some Windows wrappers. { printf "\n%bSetup profiles%b (will replace previous configuration, if exists):\n\n" "$CYAN" "$NC" - local i=1 key slug + local i=1 key slug display for key in "${SERVICE_ORDER[@]}"; do slug="${SERVICES[$key]}" - printf " %2d) %-12s (%s)\n" "$i" "$key" "$slug" + display="${SERVICE_DISPLAY[$key]}" + printf " %2d) %-16s (%s)\n" "$i" "$display" "$slug" i=$((i + 1)) done printf "\n a) ALL\n" @@ -932,16 +964,46 @@ setup_choose_services() { setup_service() { local service="$1" local profile="${SERVICES[$service]:-}" + local display="${SERVICE_DISPLAY[$service]:-$service}" [[ -n "$profile" ]] || die "Unknown service: $service" - printf "\n%b→ %s%b\n" "$YELLOW" "$service" "$NC" + printf "\n%b→ %s%b\n" "$YELLOW" "$display" "$NC" queue_profile "$profile" - printf "%bEnter value(s) for %s:%b\n" "$BLUE" "$service" "$NC" - local pair key def val - for pair in ${PROFILE_ENV[$profile]}; do + local defaults="${PROFILE_ENV[$profile]:-}" + [[ -n "$defaults" ]] || return 0 + + printf "%bEnter value(s) for %s:%b\n" "$BLUE" "$display" "$NC" + + local -a pairs=() prompts=() + IFS=';' read -r -a pairs <<<"$defaults" + IFS=';' read -r -a prompts <<<"${PROFILE_PROMPTS[$profile]:-}" + + local i pair key def val prompt + for i in "${!pairs[@]}"; do + pair="${pairs[$i]}" + [[ -n "$pair" ]] || continue IFS='=' read -r key def <<<"$pair" - val=$(read_default "$key" "$def") + prompt="${prompts[$i]:-$key}" + val="$(read_default "$prompt" "$def")" + + case "$key" in + LDS_AI_RUNTIME) + val="${val,,}" + case "$val" in + cpu | nvidia | amd) ;; + *) die "AI runtime must be cpu, nvidia, or amd" ;; + esac + ;; + LDS_LLM_HOST_PORT) + case "${val,,}" in + 1 | y | yes | true | on) val=1 ;; + 0 | n | no | false | off) val=0 ;; + *) die "Direct localhost API must be yes or no" ;; + esac + ;; + esac + queue_env "$key=$val" done } @@ -955,7 +1017,7 @@ process_all() { printf "\n%bWill configure:%b\n" "$CYAN" "$NC" while IFS= read -r svc; do - printf " - %s (%s)\n" "$svc" "${SERVICES[$svc]}" + printf " - %s (%s)\n" "${SERVICE_DISPLAY[$svc]:-$svc}" "${SERVICES[$svc]}" done <<<"$selected" echo From 63519e9e0ef9bd2d10c092f4c5d0d0883f4f97bc Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:36:22 +0600 Subject: [PATCH 044/509] test: validate catalog-driven profile setup --- tests/env-contract.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index 2b9b676f..a0416d19 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -42,10 +42,10 @@ for entry in "${expected[@]}"; do done pass "published compatibility manifest" -for key in POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS; do - grep -Fq "[$key]=" "$ROOT/lds" || fail "missing profile catalog entry: $key" -done -pass "current profile catalog entries" +assert_file_contains "$ROOT/lds" 'CATALOG_FILE="$CFG/catalog/services.psv"' +assert_file_contains "$ROOT/lds" 'load_service_catalog()' +assert_file_contains "$ROOT/lds" 'load_service_catalog' +pass "profile setup loads the tracked host catalog" assert_file_contains "$ROOT/lds" 'cmd_ai()' assert_file_contains "$ROOT/lds" 'cmd_llm()' From bc29298c0347584420db14c86071f6035a362fda Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:36:26 +0600 Subject: [PATCH 045/509] test: detect Tools profile catalog drift --- tests/published-images.sh | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/tests/published-images.sh b/tests/published-images.sh index 41dd02f0..c683cd0b 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -40,3 +40,20 @@ pass "Tools and Runner publish healthchecks" [[ "${release[LDS_LLM_AMD_IMAGE]:-}" == "infocyph/llm-sm:amd-0.03" ]] || fail "unexpected AMD LLM compatibility image" pass "LLM compatibility references are release-pinned" + +tools_profile_chooser="$( + docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /usr/local/bin/profile-chooser +)" +catalog="$ROOT/docker/catalog/services.psv" + +while IFS='|' read -r key profile _display _service_key _version_env defaults _prompts _admin _volume _url _category _optional _default_enabled _runtime_modes; do + [[ -n "$key" && "$key" != \#* ]] || continue + [[ "$profile" != "ai" ]] || continue + + expected_defaults="${defaults//;/ }" + grep -Fq "[$key]=\"$profile\"" <<<"$tools_profile_chooser" || + fail "Tools profile-chooser service mapping drift: $key -> $profile" + grep -Fq "[$profile]=\"$expected_defaults\"" <<<"$tools_profile_chooser" || + fail "Tools profile-chooser defaults drift for profile: $profile" +done <"$catalog" +pass "LocalDevStack catalog matches overlapping Tools 0.23.2 profile defaults" From 575e8b3af0d8624848b11702b6559de635515fed Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:36:28 +0600 Subject: [PATCH 046/509] test: add canonical service catalog contract --- tests/catalog-contract.sh | 63 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 63 insertions(+) create mode 100644 tests/catalog-contract.sh diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh new file mode 100644 index 00000000..5c487db8 --- /dev/null +++ b/tests/catalog-contract.sh @@ -0,0 +1,63 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +catalog="$ROOT/docker/catalog/services.psv" +assert_file "$catalog" + +declare -A seen_keys=() +declare -A seen_profiles=() +declare -a order=() + +while IFS='|' read -r key profile display service_key version_env defaults prompts admin_client volume url category optional default_enabled runtime_modes; do + [[ -n "$key" && "$key" != \#* ]] || continue + + [[ -n "$profile" ]] || fail "catalog row missing profile: $key" + [[ -n "$display" ]] || fail "catalog row missing display name: $key" + [[ -n "$service_key" ]] || fail "catalog row missing service key: $key" + [[ -n "$version_env" ]] || fail "catalog row missing version env: $key" + [[ -n "$category" ]] || fail "catalog row missing category: $key" + [[ "$optional" =~ ^[01]$ ]] || fail "catalog optional flag invalid: $key" + [[ "$default_enabled" =~ ^[01]$ ]] || fail "catalog default_enabled flag invalid: $key" + [[ -z "${seen_keys[$key]:-}" ]] || fail "duplicate catalog key: $key" + [[ -z "${seen_profiles[$profile]:-}" ]] || fail "duplicate catalog profile: $profile" + + seen_keys["$key"]=1 + seen_profiles["$profile"]=1 + order+=("$key") + + [[ -n "$defaults" ]] || fail "catalog setup defaults missing: $key" + [[ -n "$prompts" ]] || fail "catalog setup prompts missing: $key" + + IFS=';' read -r -a default_items <<<"$defaults" + IFS=';' read -r -a prompt_items <<<"$prompts" + [[ "${#default_items[@]}" -eq "${#prompt_items[@]}" ]] || + fail "catalog prompts/defaults count mismatch: $key" + + if [[ -n "$url" && "$url" != https://*.localhost ]]; then + fail "catalog convenience URL must use local HTTPS: $key" + fi + + case "$key" in + POSTGRESQL|MYSQL|MARIADB|MONGODB|REDIS|ELASTICSEARCH) + [[ -n "$volume" ]] || fail "persistent service missing volume metadata: $key" + ;; + AI) + [[ "$profile" == "ai" ]] || fail "AI profile must be ai" + [[ "$service_key" == "llm-sm" ]] || fail "AI service key must be llm-sm" + [[ "$runtime_modes" == "cpu,nvidia,amd" ]] || fail "AI runtime metadata drift" + [[ "$defaults" == *"LDS_AI_MODEL=qwen2.5:3b"* ]] || fail "AI model default drift" + [[ "$defaults" == *"LDS_AI_RUNTIME=cpu"* ]] || fail "AI runtime default drift" + [[ "$defaults" == *"LDS_LLM_HOST_PORT=no"* ]] || fail "AI host-port default drift" + ;; + esac +done <"$catalog" + +expected=(POSTGRESQL MYSQL MARIADB ELASTICSEARCH MONGODB REDIS AI) +[[ "${order[*]}" == "${expected[*]}" ]] || + fail "catalog order/coverage drift: ${order[*]}" + +pass "canonical host service catalog schema and coverage" From 031c6de1c932dd439e788b130a4551377a639be1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:36:40 +0600 Subject: [PATCH 047/509] ci: enforce service catalog contract --- .github/workflows/check.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 030314ed..5889fc6c 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -39,6 +39,9 @@ jobs: - name: Environment contract run: tests/env-contract.sh + - name: Catalog contract + run: tests/catalog-contract.sh + - name: PHP runtime contract run: tests/runtime-php-contract.sh From 4e19f6dd4545b92d897f43d0245d81313192285c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:36:43 +0600 Subject: [PATCH 048/509] test: add catalog to release gate --- tests/release-gate.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/release-gate.sh b/tests/release-gate.sh index 70b47724..75ed984c 100755 --- a/tests/release-gate.sh +++ b/tests/release-gate.sh @@ -6,6 +6,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" "$ROOT/tests/static.sh" "$ROOT/tests/cli-contract.sh" "$ROOT/tests/env-contract.sh" +"$ROOT/tests/catalog-contract.sh" "$ROOT/tests/compose-contract.sh" "$ROOT/tests/networking-contract.sh" "$ROOT/tests/runtime-php-contract.sh" From 42a1d68c0a613f58b793ad2dcddbd4fb270b4cd3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:37:41 +0600 Subject: [PATCH 049/509] test: validate catalog admin-client metadata --- tests/catalog-contract.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 5c487db8..4f3466b2 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -20,6 +20,8 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ -n "$service_key" ]] || fail "catalog row missing service key: $key" [[ -n "$version_env" ]] || fail "catalog row missing version env: $key" [[ -n "$category" ]] || fail "catalog row missing category: $key" + [[ -z "$admin_client" || "$admin_client" =~ ^[a-z0-9-]+$ ]] || + fail "catalog admin client invalid: $key" [[ "$optional" =~ ^[01]$ ]] || fail "catalog optional flag invalid: $key" [[ "$default_enabled" =~ ^[01]$ ]] || fail "catalog default_enabled flag invalid: $key" [[ -z "${seen_keys[$key]:-}" ]] || fail "duplicate catalog key: $key" From e8acc803e6d6cdbc328c714baccf199b1d3157f3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:39:41 +0600 Subject: [PATCH 050/509] images: move LocalDevStack infrastructure defaults to latest --- docker/release.env | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docker/release.env b/docker/release.env index 15e1ef3f..dd2b18d9 100644 --- a/docker/release.env +++ b/docker/release.env @@ -1,8 +1,8 @@ # LocalDevStack release-owned compatibility baseline. # User overrides belong in docker/.env. Do not edit this file for local settings. -LDS_TOOLS_IMAGE=infocyph/tools:0.23.2 -LDS_RUNNER_IMAGE=infocyph/runner:0.5 -LDS_NGINX_IMAGE=infocyph/nginx:0.4.1 -LDS_APACHE_IMAGE=infocyph/apache:0.4.2 -LDS_LLM_IMAGE=infocyph/llm-sm:0.03 -LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-0.03 +LDS_TOOLS_IMAGE=infocyph/tools:latest +LDS_RUNNER_IMAGE=infocyph/runner:latest +LDS_NGINX_IMAGE=infocyph/nginx:latest +LDS_APACHE_IMAGE=infocyph/apache:latest +LDS_LLM_IMAGE=infocyph/llm-sm:latest +LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest From 41eac3375826fdfef7845d7db6f7291c8b61c14b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:39:44 +0600 Subject: [PATCH 051/509] images: use latest Tools and Runner defaults --- docker/compose/companion.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index e9d53160..faee5794 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -2,7 +2,7 @@ services: server-tools: container_name: SERVER_TOOLS hostname: local-dock - image: ${LDS_TOOLS_IMAGE:-infocyph/tools:0.23.2} + image: ${LDS_TOOLS_IMAGE:-infocyph/tools:latest} restart: unless-stopped environment: - TZ=${TZ:-} @@ -55,7 +55,7 @@ services: runner: container_name: RUNNER hostname: runner - image: ${LDS_RUNNER_IMAGE:-infocyph/runner:0.5} + image: ${LDS_RUNNER_IMAGE:-infocyph/runner:latest} restart: unless-stopped environment: - TZ=${TZ:-} From c5334a97e45390ac6f91170ea55e77f6ac693c8c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:39:48 +0600 Subject: [PATCH 052/509] images: use latest HTTP infrastructure defaults --- docker/compose/http.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index ad88961f..9b319c97 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -2,7 +2,7 @@ services: nginx: container_name: NGINX hostname: nginx - image: ${LDS_NGINX_IMAGE:-infocyph/nginx:0.4.1} + image: ${LDS_NGINX_IMAGE:-infocyph/nginx:latest} restart: unless-stopped environment: - TZ=${TZ:-} @@ -24,7 +24,7 @@ services: apache: container_name: APACHE hostname: apache - image: ${LDS_APACHE_IMAGE:-infocyph/apache:0.4.2} + image: ${LDS_APACHE_IMAGE:-infocyph/apache:latest} restart: unless-stopped environment: - TZ=${TZ:-} From 002703e91aff54bd91d6fdb5af36d68ee678a534 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:39:52 +0600 Subject: [PATCH 053/509] images: use latest standard LLM runtime --- docker/compose/ai.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/ai.yaml b/docker/compose/ai.yaml index 17b540ff..c7ee9844 100644 --- a/docker/compose/ai.yaml +++ b/docker/compose/ai.yaml @@ -1,6 +1,6 @@ services: llm-sm: - image: ${LDS_LLM_IMAGE:-infocyph/llm-sm:0.03} + image: ${LDS_LLM_IMAGE:-infocyph/llm-sm:latest} restart: unless-stopped profiles: [ai] volumes: From 4b97e90674c6f025512ba9c6026de78570af85d4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:39:55 +0600 Subject: [PATCH 054/509] images: use latest AMD LLM runtime --- docker/compose/ai-amd.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/ai-amd.yaml b/docker/compose/ai-amd.yaml index bc1c79c6..9aa98d41 100644 --- a/docker/compose/ai-amd.yaml +++ b/docker/compose/ai-amd.yaml @@ -1,6 +1,6 @@ services: llm-sm: - image: ${LDS_LLM_AMD_IMAGE:-infocyph/llm-sm:amd-0.03} + image: ${LDS_LLM_AMD_IMAGE:-infocyph/llm-sm:amd-latest} devices: - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri From 936d13eecc8287229ebb2182e0c6e28ab1774618 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:39:58 +0600 Subject: [PATCH 055/509] images: align database defaults with latest policy --- docker/compose/db.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index 1e730fa2..bdd615e2 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -26,7 +26,7 @@ services: <<: *db-service container_name: POSTGRESQL hostname: postgres - image: postgres:${POSTGRES_VERSION:-alpine} + image: postgres:${POSTGRES_VERSION:-latest} profiles: [postgresql] environment: - TZ=${TZ:-} @@ -123,7 +123,7 @@ services: <<: *db-service container_name: ELASTICSEARCH hostname: elasticsearch - image: elasticsearch:${ELASTICSEARCH_VERSION:-9.3.0} + image: elasticsearch:${ELASTICSEARCH_VERSION:-9.5.4} profiles: [elasticsearch] environment: - TZ=${TZ:-} From 7cf28563b7c54876c25aded84f3c3af88a62f1a4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:40:01 +0600 Subject: [PATCH 056/509] images: align Elastic clients on current stable release --- docker/compose/db-client.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index 99ab89df..9d0c525e 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -57,7 +57,7 @@ services: <<: *db-client-service container_name: KIBANA hostname: kibana - image: kibana:${ELASTICSEARCH_VERSION:-9.3.0} + image: kibana:${ELASTICSEARCH_VERSION:-9.5.4} profiles: [elasticsearch] depends_on: - elasticsearch @@ -69,7 +69,7 @@ services: - ../../logs/kibana:/usr/share/kibana/logs filebeat: - image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.3.0} + image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.5.4} container_name: FILEBEAT restart: unless-stopped user: root From adedda262aeadc664ac74b9d59821ead5be7180d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:40:07 +0600 Subject: [PATCH 057/509] catalog: advance Elastic default to current stable --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index 3aac4193..0c522b32 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -2,7 +2,7 @@ POSTGRESQL|postgresql|PostgreSQL|postgres|POSTGRES_VERSION|POSTGRES_VERSION=latest;POSTGRES_USER=postgres;POSTGRES_PASSWORD=postgres;POSTGRES_DATABASE=postgres|Version;User;Password;Database|cloudbeaver|PostgresStore|https://db.localhost|database|1|0| MYSQL|mysql|MySQL|mysql|MYSQL_VERSION|MYSQL_VERSION=latest;MYSQL_ROOT_PASSWORD=12345;MYSQL_USER=infocyph;MYSQL_PASSWORD=12345;MYSQL_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MySQLStore|https://db.localhost|database|1|0| MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_ROOT_PASSWORD=12345;MARIADB_USER=infocyph;MARIADB_PASSWORD=12345;MARIADB_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MariaDBStore|https://db.localhost|database|1|0| -ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.3.0|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| +ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.4|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| AI|ai|Local AI|llm-sm|LDS_LLM_IMAGE|LDS_AI_RUNTIME=cpu;LDS_AI_MODEL=qwen2.5:3b;LDS_LLM_HOST_PORT=no|Runtime (cpu/nvidia/amd);Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd From f4d832d5fe1ed05fe174e352372ebe5c7984ea1f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:40:37 +0600 Subject: [PATCH 058/509] test: enforce latest infrastructure image policy --- tests/env-contract.sh | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index a0416d19..e4080947 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -30,17 +30,17 @@ fi pass "release and user env ownership boundaries" expected=( - 'LDS_TOOLS_IMAGE=infocyph/tools:0.23.2' - 'LDS_RUNNER_IMAGE=infocyph/runner:0.5' - 'LDS_NGINX_IMAGE=infocyph/nginx:0.4.1' - 'LDS_APACHE_IMAGE=infocyph/apache:0.4.2' - 'LDS_LLM_IMAGE=infocyph/llm-sm:0.03' - 'LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-0.03' + 'LDS_TOOLS_IMAGE=infocyph/tools:latest' + 'LDS_RUNNER_IMAGE=infocyph/runner:latest' + 'LDS_NGINX_IMAGE=infocyph/nginx:latest' + 'LDS_APACHE_IMAGE=infocyph/apache:latest' + 'LDS_LLM_IMAGE=infocyph/llm-sm:latest' + 'LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest' ) for entry in "${expected[@]}"; do assert_file_contains "$release_env" "$entry" done -pass "published compatibility manifest" +pass "moving latest image manifest" assert_file_contains "$ROOT/lds" 'CATALOG_FILE="$CFG/catalog/services.psv"' assert_file_contains "$ROOT/lds" 'load_service_catalog()' From 83585f7b36fd260a7c1ab359d85a6325d83fd2d1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:40:40 +0600 Subject: [PATCH 059/509] test: resolve moving latest infrastructure images --- tests/compose-contract.sh | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 2366c9f0..c85f4c30 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -78,10 +78,10 @@ assert_contains "$resolved" "server-tools:" assert_contains "$resolved" "nginx:" assert_contains "$resolved" "mysql:" assert_contains "$resolved" "cloudbeaver:" -assert_contains "$resolved" "image: infocyph/tools:0.23.2" -assert_contains "$resolved" "image: infocyph/runner:0.5" -assert_contains "$resolved" "image: infocyph/nginx:0.4.1" -assert_contains "$resolved" "image: infocyph/apache:0.4.2" +assert_contains "$resolved" "image: infocyph/tools:latest" +assert_contains "$resolved" "image: infocyph/runner:latest" +assert_contains "$resolved" "image: infocyph/nginx:latest" +assert_contains "$resolved" "image: infocyph/apache:latest" if grep -Eq 'ipv4_address:|172\\.28\\.0\\.|172\\.29\\.0\\.|172\\.30\\.0\\.' <<<"$resolved"; then fail "resolved Compose config still contains fixed LocalDevStack addresses" fi @@ -116,7 +116,7 @@ python3 -c ' import json,sys d=json.load(sys.stdin) s=d["services"]["llm-sm"] -assert s["image"] == "infocyph/llm-sm:0.03" +assert s["image"] == "infocyph/llm-sm:latest" assert "container_name" not in s assert not s.get("ports") assert set(s["networks"]) == {"frontend","backend"} @@ -135,7 +135,7 @@ amd_json="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/ python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] -assert s["image"] == "infocyph/llm-sm:amd-0.03" +assert s["image"] == "infocyph/llm-sm:amd-latest" devices=" ".join(str(x) for x in s.get("devices", [])) assert "/dev/kfd" in devices and "/dev/dri" in devices ' <<<"$amd_json" From b86bf9a79baaa49b371a9a711033d803e0e00ded Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:40:43 +0600 Subject: [PATCH 060/509] test: validate AI against latest Tools image --- tests/ai-contract.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 6c07ef0e..d8279b0a 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -42,13 +42,13 @@ models="$( assert_contains "$models" "qwen2.5:3b" pass "fake Ollama tags/generate/OpenAI-compatible contracts" -docker pull infocyph/tools:0.23.2 >/dev/null +docker pull infocyph/tools:latest >/dev/null provider_status="$( - docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-sm:11434 -e LDS_AI_MODEL=qwen2.5:3b infocyph/tools:0.23.2 --status + docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-sm:11434 -e LDS_AI_MODEL=qwen2.5:3b infocyph/tools:latest --status )" assert_contains "$provider_status" "available=1" assert_contains "$provider_status" "model=qwen2.5:3b" -pass "Tools 0.23.2 reaches the separate provider contract" +pass "latest Tools reaches the separate provider contract" for file in ai.yaml ai-nvidia.yaml ai-amd.yaml ai-host-port.yaml; do assert_file "$ROOT/docker/compose/$file" From d9ad0ed6f5e7490f6005f971d55ac3571236917a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:40:47 +0600 Subject: [PATCH 061/509] test: pin Elastic exception to current stable --- tests/catalog-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 4f3466b2..b6cb6fda 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -55,6 +55,9 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$defaults" == *"LDS_AI_RUNTIME=cpu"* ]] || fail "AI runtime default drift" [[ "$defaults" == *"LDS_LLM_HOST_PORT=no"* ]] || fail "AI host-port default drift" ;; + ELASTICSEARCH) + [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.4"* ]] || fail "Elastic stable default drift" + ;; esac done <"$catalog" From 82f491968905c039e64fdf65ea0283d708387604 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:41:11 +0600 Subject: [PATCH 062/509] test: validate Elastic current-stable exception --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index b6cb6fda..134d30a5 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -44,7 +44,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp fi case "$key" in - POSTGRESQL|MYSQL|MARIADB|MONGODB|REDIS|ELASTICSEARCH) + POSTGRESQL|MYSQL|MARIADB|MONGODB|REDIS) [[ -n "$volume" ]] || fail "persistent service missing volume metadata: $key" ;; AI) From 2116ce0cfa9dd7c39e925f0358187d769e4a9574 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:41:13 +0600 Subject: [PATCH 063/509] test: follow latest image and Tools contracts --- tests/published-images.sh | 37 +++++++++++++++++++++++++++---------- 1 file changed, 27 insertions(+), 10 deletions(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index c683cd0b..742e9707 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -35,25 +35,42 @@ for image in "${release[LDS_TOOLS_IMAGE]}" "${release[LDS_RUNNER_IMAGE]}"; do done pass "Tools and Runner publish healthchecks" -[[ "${release[LDS_LLM_IMAGE]:-}" == "infocyph/llm-sm:0.03" ]] || - fail "unexpected LLM compatibility image" -[[ "${release[LDS_LLM_AMD_IMAGE]:-}" == "infocyph/llm-sm:amd-0.03" ]] || - fail "unexpected AMD LLM compatibility image" -pass "LLM compatibility references are release-pinned" +[[ "${release[LDS_LLM_IMAGE]:-}" == "infocyph/llm-sm:latest" ]] || + fail "unexpected standard LLM moving image" +[[ "${release[LDS_LLM_AMD_IMAGE]:-}" == "infocyph/llm-sm:amd-latest" ]] || + fail "unexpected AMD LLM moving image" +pass "LLM image references follow latest-tag policy" tools_profile_chooser="$( docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /usr/local/bin/profile-chooser )" catalog="$ROOT/docker/catalog/services.psv" -while IFS='|' read -r key profile _display _service_key _version_env defaults _prompts _admin _volume _url _category _optional _default_enabled _runtime_modes; do +while IFS='|' read -r key profile _display _service_key version_env defaults _prompts _admin _volume _url _category _optional _default_enabled _runtime_modes; do [[ -n "$key" && "$key" != \#* ]] || continue [[ "$profile" != "ai" ]] || continue - expected_defaults="${defaults//;/ }" grep -Fq "[$key]=\"$profile\"" <<<"$tools_profile_chooser" || fail "Tools profile-chooser service mapping drift: $key -> $profile" - grep -Fq "[$profile]=\"$expected_defaults\"" <<<"$tools_profile_chooser" || - fail "Tools profile-chooser defaults drift for profile: $profile" + + IFS=';' read -r -a catalog_defaults <<<"$defaults" + for kv in "${catalog_defaults[@]}"; do + [[ -n "$kv" ]] || continue + k="${kv%%=*}" + [[ "$k" == "$version_env" ]] && continue + grep -Fq "$kv" <<<"$tools_profile_chooser" || + fail "Tools profile-chooser non-version default drift for $profile: $kv" + done + + tools_version="$( + grep -oE "\[$profile\]=\"[^\"]+\"" <<<"$tools_profile_chooser" | + grep -oE "(${version_env})=[^ \"]+" | head -n1 || true + )" + catalog_version="$( + printf '%s\n' "${catalog_defaults[@]}" | grep -E "^${version_env}=" | head -n1 || true + )" + if [[ -n "$tools_version" && "$tools_version" != "$catalog_version" ]]; then + printf 'INFO: intentional image-version drift for %s: Tools=%s LocalDevStack=%s\n' "$profile" "$tools_version" "$catalog_version" + fi done <"$catalog" -pass "LocalDevStack catalog matches overlapping Tools 0.23.2 profile defaults" +pass "LocalDevStack catalog matches latest Tools non-version profile contract" From 01c63d37a80fe547bbd5c868a5ad9db8865f5233 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:41:28 +0600 Subject: [PATCH 064/509] test: enforce latest image policy and Elastic exception --- tests/compose-contract.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index c85f4c30..bf1af9a9 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -90,6 +90,24 @@ assert_contains "$resolved" "name: Backend" assert_contains "$resolved" "name: DataStore" pass "release compatibility defaults and dynamic networks resolve" +postgres_json="$("${compose[@]}" --profile postgresql config --format json)" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +assert d["services"]["postgres"]["image"] == "postgres:latest" +' <<<"$postgres_json" +pass "PostgreSQL follows latest-tag policy" + +elastic_json="$("${compose[@]}" --profile elasticsearch --profile filebeat config --format json)" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +assert d["services"]["elasticsearch"]["image"] == "elasticsearch:9.5.4" +assert d["services"]["kibana"]["image"] == "kibana:9.5.4" +assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9.5.4" +' <<<"$elastic_json" +pass "Elastic stack uses aligned current-stable tags because latest is unsupported" + printf '%s\n' 'LDS_TOOLS_IMAGE=example.invalid/tools:user-override' >>"$user_env" user_override="$("${compose[@]}" config)" assert_contains "$user_override" "image: example.invalid/tools:user-override" From a5267aaeac8b4016ce35e416f47ebb82672b19b0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:41:31 +0600 Subject: [PATCH 065/509] images: document moving latest baseline --- docker/release.env | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docker/release.env b/docker/release.env index dd2b18d9..331f0eba 100644 --- a/docker/release.env +++ b/docker/release.env @@ -1,4 +1,5 @@ -# LocalDevStack release-owned compatibility baseline. +# LocalDevStack release-owned moving-image baseline. +# Use latest tags wherever the upstream image family supports them. # User overrides belong in docker/.env. Do not edit this file for local settings. LDS_TOOLS_IMAGE=infocyph/tools:latest LDS_RUNNER_IMAGE=infocyph/runner:latest From 3eaf3406516a8ad77680a720cd619dd584c79f9b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:41:50 +0600 Subject: [PATCH 066/509] plan: record latest image tag policy override --- .../07-localdevstack-integration-plan.md | 49 +++++++++++++++++++ 1 file changed, 49 insertions(+) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index afb0414c..514e8dad 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1889,3 +1889,52 @@ Before any broad refactor, the first implementation PR/batch should contain only Once that is green, proceed to the networking migration. This gives every later change a reliable regression boundary. + + +--- + +# Appendix — User-directed latest-tag policy override (2026-09-18) + +This section is an explicit product-direction override and supersedes earlier image-pinning recommendations in this plan wherever they conflict. + +## Image default policy + +Use moving `latest` tags wherever the upstream image family publishes/supports them. + +LocalDevStack release defaults: + +```text +LDS_TOOLS_IMAGE=infocyph/tools:latest +LDS_RUNNER_IMAGE=infocyph/runner:latest +LDS_NGINX_IMAGE=infocyph/nginx:latest +LDS_APACHE_IMAGE=infocyph/apache:latest +LDS_LLM_IMAGE=infocyph/llm-sm:latest +LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest +``` + +Other runtime defaults should likewise use `latest` where supported, including PostgreSQL, MySQL, MariaDB, MongoDB, Redis/Redis Insight, CloudBeaver, Mongo Express and Mailpit. + +## Elastic exception + +Elasticsearch and Kibana explicitly do not support a `latest` tag, and the Elastic Filebeat registry publishes versioned releases rather than a supported moving `latest` contract. + +Keep Elasticsearch, Kibana and Filebeat on one aligned current-stable version. At implementation time this is: + +```text +9.5.4 +``` + +When this Elastic exception is advanced, update all three together and update the LocalDevStack catalog/contract tests in the same change. + +## Override precedence + +User values in `docker/.env` and command-scoped shell environment still override these defaults. + +The latest-tag policy changes default image selection only; it does not weaken: + +- profile isolation; +- AI trust boundaries; +- dynamic networking; +- persistent-volume behavior; +- loopback-only optional Ollama host exposure; +- compatibility/contract testing. From dd05ff06633b0353d9f94028f2f8a1120f3014fb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:42:54 +0600 Subject: [PATCH 067/509] ci: run catalog contract through bash --- .github/workflows/check.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 5889fc6c..44b06d03 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -40,7 +40,7 @@ jobs: run: tests/env-contract.sh - name: Catalog contract - run: tests/catalog-contract.sh + run: bash tests/catalog-contract.sh - name: PHP runtime contract run: tests/runtime-php-contract.sh From 3a57cad30b2147b304c9541b82c27fc1c7181888 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:57:46 +0600 Subject: [PATCH 068/509] images: prefer Alpine PostgreSQL and align Elastic stable version --- docker/compose/db.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index bdd615e2..475268fe 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -26,7 +26,7 @@ services: <<: *db-service container_name: POSTGRESQL hostname: postgres - image: postgres:${POSTGRES_VERSION:-latest} + image: postgres:${POSTGRES_VERSION:-alpine} profiles: [postgresql] environment: - TZ=${TZ:-} @@ -123,7 +123,7 @@ services: <<: *db-service container_name: ELASTICSEARCH hostname: elasticsearch - image: elasticsearch:${ELASTICSEARCH_VERSION:-9.5.4} + image: elasticsearch:${ELASTICSEARCH_VERSION:-9.5.3} profiles: [elasticsearch] environment: - TZ=${TZ:-} From 5e1c36ae9ea856f01406028a5d8e5740191b716f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:57:50 +0600 Subject: [PATCH 069/509] images: align Elastic clients with current stable stack --- docker/compose/db-client.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index 9d0c525e..ce4d19cb 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -57,7 +57,7 @@ services: <<: *db-client-service container_name: KIBANA hostname: kibana - image: kibana:${ELASTICSEARCH_VERSION:-9.5.4} + image: kibana:${ELASTICSEARCH_VERSION:-9.5.3} profiles: [elasticsearch] depends_on: - elasticsearch @@ -69,7 +69,7 @@ services: - ../../logs/kibana:/usr/share/kibana/logs filebeat: - image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.5.4} + image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.5.3} container_name: FILEBEAT restart: unless-stopped user: root From 76e9663a75384cc4528494090c288e7d53d13c6e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:57:55 +0600 Subject: [PATCH 070/509] catalog: apply Alpine-first image defaults --- docker/catalog/services.psv | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index 0c522b32..17d7c5d5 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -1,8 +1,8 @@ # key|profile|display_name|service_key|version_env|setup_defaults|setup_prompts|admin_client|volume|url|category|optional|default_enabled|runtime_modes -POSTGRESQL|postgresql|PostgreSQL|postgres|POSTGRES_VERSION|POSTGRES_VERSION=latest;POSTGRES_USER=postgres;POSTGRES_PASSWORD=postgres;POSTGRES_DATABASE=postgres|Version;User;Password;Database|cloudbeaver|PostgresStore|https://db.localhost|database|1|0| +POSTGRESQL|postgresql|PostgreSQL|postgres|POSTGRES_VERSION|POSTGRES_VERSION=alpine;POSTGRES_USER=postgres;POSTGRES_PASSWORD=postgres;POSTGRES_DATABASE=postgres|Version;User;Password;Database|cloudbeaver|PostgresStore|https://db.localhost|database|1|0| MYSQL|mysql|MySQL|mysql|MYSQL_VERSION|MYSQL_VERSION=latest;MYSQL_ROOT_PASSWORD=12345;MYSQL_USER=infocyph;MYSQL_PASSWORD=12345;MYSQL_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MySQLStore|https://db.localhost|database|1|0| MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_ROOT_PASSWORD=12345;MARIADB_USER=infocyph;MARIADB_PASSWORD=12345;MARIADB_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MariaDBStore|https://db.localhost|database|1|0| -ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.4|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| +ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.3|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| AI|ai|Local AI|llm-sm|LDS_LLM_IMAGE|LDS_AI_RUNTIME=cpu;LDS_AI_MODEL=qwen2.5:3b;LDS_LLM_HOST_PORT=no|Runtime (cpu/nvidia/amd);Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd From ef2359a002d00811e91de806438ad9da887a490b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:57:58 +0600 Subject: [PATCH 071/509] test: enforce Alpine-first image policy --- tests/compose-contract.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index bf1af9a9..8527bac7 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -94,17 +94,17 @@ postgres_json="$("${compose[@]}" --profile postgresql config --format json)" python3 -c ' import json,sys d=json.load(sys.stdin) -assert d["services"]["postgres"]["image"] == "postgres:latest" +assert d["services"]["postgres"]["image"] == "postgres:alpine" ' <<<"$postgres_json" -pass "PostgreSQL follows latest-tag policy" +pass "PostgreSQL follows moving Alpine policy" elastic_json="$("${compose[@]}" --profile elasticsearch --profile filebeat config --format json)" python3 -c ' import json,sys d=json.load(sys.stdin) -assert d["services"]["elasticsearch"]["image"] == "elasticsearch:9.5.4" -assert d["services"]["kibana"]["image"] == "kibana:9.5.4" -assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9.5.4" +assert d["services"]["elasticsearch"]["image"] == "elasticsearch:9.5.3" +assert d["services"]["kibana"]["image"] == "kibana:9.5.3" +assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9.5.3" ' <<<"$elastic_json" pass "Elastic stack uses aligned current-stable tags because latest is unsupported" From 5c29f6cb71f74fea8504763f2ed1475fcfae64b1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:58:01 +0600 Subject: [PATCH 072/509] test: align Elastic catalog stable version --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 134d30a5..4764ea06 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -56,7 +56,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$defaults" == *"LDS_LLM_HOST_PORT=no"* ]] || fail "AI host-port default drift" ;; ELASTICSEARCH) - [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.4"* ]] || fail "Elastic stable default drift" + [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.3"* ]] || fail "Elastic stable default drift" ;; esac done <"$catalog" From 857e7858eb61d238281014492588a60aa3fd723a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 11:58:14 +0600 Subject: [PATCH 073/509] plan: clarify Alpine-first moving image policy --- .../07-localdevstack-integration-plan.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 514e8dad..ab270622 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1893,13 +1893,13 @@ This gives every later change a reliable regression boundary. --- -# Appendix — User-directed latest-tag policy override (2026-09-18) +# Appendix — User-directed Alpine-first moving-tag policy override (2026-09-18) This section is an explicit product-direction override and supersedes earlier image-pinning recommendations in this plan wherever they conflict. ## Image default policy -Use moving `latest` tags wherever the upstream image family publishes/supports them. +Prefer the moving Alpine variant when the same image family publishes/supports one; otherwise use its normal moving latest tag. LocalDevStack release defaults: @@ -1912,7 +1912,7 @@ LDS_LLM_IMAGE=infocyph/llm-sm:latest LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest ``` -Other runtime defaults should likewise use `latest` where supported, including PostgreSQL, MySQL, MariaDB, MongoDB, Redis/Redis Insight, CloudBeaver, Mongo Express and Mailpit. +Other runtime defaults follow the same rule. PostgreSQL uses `postgres:alpine`; MySQL, MariaDB, MongoDB, Redis Stack/Redis Insight, CloudBeaver, Mongo Express and Mailpit use their normal moving latest tags because the selected image family does not provide a suitable moving Alpine alias for this stack. ## Elastic exception @@ -1921,7 +1921,7 @@ Elasticsearch and Kibana explicitly do not support a `latest` tag, and the Elast Keep Elasticsearch, Kibana and Filebeat on one aligned current-stable version. At implementation time this is: ```text -9.5.4 +9.5.3 ``` When this Elastic exception is advanced, update all three together and update the LocalDevStack catalog/contract tests in the same change. @@ -1930,7 +1930,7 @@ When this Elastic exception is advanced, update all three together and update th User values in `docker/.env` and command-scoped shell environment still override these defaults. -The latest-tag policy changes default image selection only; it does not weaken: +The Alpine-first moving-tag policy changes default image selection only; it does not weaken: - profile isolation; - AI trust boundaries; From 14579eda91b2845b77be4829c2451ef72dc1f546 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:01:14 +0600 Subject: [PATCH 074/509] test: preserve runtime version selection contract --- tests/published-images.sh | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/tests/published-images.sh b/tests/published-images.sh index 742e9707..cad0949c 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -74,3 +74,25 @@ while IFS='|' read -r key profile _display _service_key version_env defaults _pr fi done <"$catalog" pass "LocalDevStack catalog matches latest Tools non-version profile contract" + + +docker run --rm --entrypoint sh "${release[LDS_TOOLS_IMAGE]}" -lc ' + test -x /usr/local/bin/mkhost + test -s /etc/share/runtime-versions.json + jq -e ".php.active | type == \"array\" and length > 0" /etc/share/runtime-versions.json >/dev/null + jq -e ".node.active | type == \"array\" and length > 0" /etc/share/runtime-versions.json >/dev/null + grep -Fq "RUNTIME_VERSIONS_DB" /usr/local/bin/mkhost +' +pass "latest Tools preserves interactive PHP/Node runtime version catalog" + +php_template="$( + docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /etc/docker-templates/php.compose.yaml +)" +node_template="$( + docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /etc/docker-templates/node.compose.yaml +)" +assert_contains "$php_template" 'PHP_VERSION: {{PHP_VERSION}}' +assert_contains "$php_template" 'image: localdevstack-php:{{PHP_VERSION}}' +assert_contains "$node_template" 'NODE_VERSION: {{NODE_VERSION}}' +assert_contains "$node_template" 'image: localdevstack-node:{{NODE_VERSION}}' +pass "selected runtime versions remain build/image identity inputs" From f379582f5252a5144f8f4ed8cef7636a68ffa639 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:02:44 +0600 Subject: [PATCH 075/509] runtime: harden PHP Scriptomatic bootstrap --- docker/dockerfiles/php.Dockerfile | 32 +++++++++++++++++++++++++++++-- 1 file changed, 30 insertions(+), 2 deletions(-) diff --git a/docker/dockerfiles/php.Dockerfile b/docker/dockerfiles/php.Dockerfile index 9d0ec641..8eb2cb2f 100644 --- a/docker/dockerfiles/php.Dockerfile +++ b/docker/dockerfiles/php.Dockerfile @@ -15,14 +15,42 @@ ARG PHP_EXT ARG PHP_EXT_VERSIONED ARG UID=1000 ARG GID=1000 +ARG SCRIPTOMATIC_REF=main +ARG SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT=10 +ARG SCRIPTOMATIC_DOWNLOAD_MAX_TIME=120 +ARG SCRIPTOMATIC_DOWNLOAD_RETRIES=3 ENV PATH="/usr/local/bin:/usr/bin:/bin:/usr/games:$PATH" \ LANG=en_US.UTF-8 \ LC_ALL=en_US.UTF-8 \ GIT_CONFIG_GLOBAL=/git-config/.gitconfig \ COMPOSER_HOME=/home/${USERNAME}/.composer/php${PHP_PROFILE_KEY} -ADD https://raw.githubusercontent.com/infocyph/Scriptomatic/master/bash/php-cli-setup.sh /usr/local/bin/cli-setup.sh -RUN apk add --no-cache bash && PHP_PROFILE_KEY="${PHP_PROFILE_KEY}" bash /usr/local/bin/cli-setup.sh "${USERNAME}" "${PHP_VERSION}" +RUN set -eux; \ + apk add --no-cache bash curl; \ + bootstrap="$(mktemp /tmp/scriptomatic-php.XXXXXX)"; \ + curl --fail --silent --show-error --location \ + --connect-timeout "${SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT}" \ + --max-time "${SCRIPTOMATIC_DOWNLOAD_MAX_TIME}" \ + --retry "${SCRIPTOMATIC_DOWNLOAD_RETRIES}" \ + --retry-delay 1 \ + --retry-connrefused \ + "https://raw.githubusercontent.com/infocyph/Scriptomatic/${SCRIPTOMATIC_REF}/bash/php-cli-setup.sh" \ + -o "$bootstrap"; \ + test -s "$bootstrap"; \ + bash -n "$bootstrap"; \ + UID="${UID}" \ + GID="${GID}" \ + LINUX_PKG="${LINUX_PKG}" \ + LINUX_PKG_VERSIONED="${LINUX_PKG_VERSIONED}" \ + PHP_EXT="${PHP_EXT}" \ + PHP_EXT_VERSIONED="${PHP_EXT_VERSIONED}" \ + PHP_PROFILE_KEY="${PHP_PROFILE_KEY}" \ + SCRIPTOMATIC_REF="${SCRIPTOMATIC_REF}" \ + SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT="${SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT}" \ + SCRIPTOMATIC_DOWNLOAD_MAX_TIME="${SCRIPTOMATIC_DOWNLOAD_MAX_TIME}" \ + SCRIPTOMATIC_DOWNLOAD_RETRIES="${SCRIPTOMATIC_DOWNLOAD_RETRIES}" \ + bash "$bootstrap" "${USERNAME}" "${PHP_VERSION}"; \ + rm -f "$bootstrap" USER ${USERNAME} WORKDIR /app From 53d05a601460aa6ad1ee6cf8a7a0390317992324 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:02:47 +0600 Subject: [PATCH 076/509] runtime: harden Node Scriptomatic bootstrap --- docker/dockerfiles/node.Dockerfile | 34 ++++++++++++++++++++++++++---- 1 file changed, 30 insertions(+), 4 deletions(-) diff --git a/docker/dockerfiles/node.Dockerfile b/docker/dockerfiles/node.Dockerfile index 5775aa57..ac2f09c6 100644 --- a/docker/dockerfiles/node.Dockerfile +++ b/docker/dockerfiles/node.Dockerfile @@ -14,16 +14,42 @@ ARG LINUX_PKG ARG LINUX_PKG_VERSIONED ARG NODE_GLOBAL ARG NODE_GLOBAL_VERSIONED +ARG SCRIPTOMATIC_REF=main +ARG SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT=10 +ARG SCRIPTOMATIC_DOWNLOAD_MAX_TIME=120 +ARG SCRIPTOMATIC_DOWNLOAD_RETRIES=3 ENV PATH="/usr/local/bin:/usr/bin:/bin:/usr/games:$PATH" \ LANG=en_US.UTF-8 \ LC_ALL=en_US.UTF-8 \ NPM_CONFIG_CACHE=/home/${USERNAME}/.npm \ GIT_CONFIG_GLOBAL=/git-config/.gitconfig -ADD https://raw.githubusercontent.com/infocyph/Scriptomatic/master/bash/node-cli-setup.sh /usr/local/bin/cli-setup.sh -RUN apk add --no-cache bash && \ - NODE_VERSION="$(node -v | sed 's/^v//')" && \ - bash /usr/local/bin/cli-setup.sh "${USERNAME}" "${NODE_VERSION}" +RUN set -eux; \ + apk add --no-cache bash curl; \ + bootstrap="$(mktemp /tmp/scriptomatic-node.XXXXXX)"; \ + curl --fail --silent --show-error --location \ + --connect-timeout "${SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT}" \ + --max-time "${SCRIPTOMATIC_DOWNLOAD_MAX_TIME}" \ + --retry "${SCRIPTOMATIC_DOWNLOAD_RETRIES}" \ + --retry-delay 1 \ + --retry-connrefused \ + "https://raw.githubusercontent.com/infocyph/Scriptomatic/${SCRIPTOMATIC_REF}/bash/node-cli-setup.sh" \ + -o "$bootstrap"; \ + test -s "$bootstrap"; \ + bash -n "$bootstrap"; \ + resolved_node_version="$(node -v | sed 's/^v//')"; \ + UID="${UID}" \ + GID="${GID}" \ + LINUX_PKG="${LINUX_PKG}" \ + LINUX_PKG_VERSIONED="${LINUX_PKG_VERSIONED}" \ + NODE_GLOBAL="${NODE_GLOBAL}" \ + NODE_GLOBAL_VERSIONED="${NODE_GLOBAL_VERSIONED}" \ + SCRIPTOMATIC_REF="${SCRIPTOMATIC_REF}" \ + SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT="${SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT}" \ + SCRIPTOMATIC_DOWNLOAD_MAX_TIME="${SCRIPTOMATIC_DOWNLOAD_MAX_TIME}" \ + SCRIPTOMATIC_DOWNLOAD_RETRIES="${SCRIPTOMATIC_DOWNLOAD_RETRIES}" \ + bash "$bootstrap" "${USERNAME}" "$resolved_node_version"; \ + rm -f "$bootstrap" USER ${USERNAME} WORKDIR /app From b60ef8c8b977879a828ef982b6fb2b5a878c2d91 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:03:31 +0600 Subject: [PATCH 077/509] test: enforce hardened PHP runtime bootstrap --- tests/runtime-php-contract.sh | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/tests/runtime-php-contract.sh b/tests/runtime-php-contract.sh index 12e837fd..8ff6a680 100755 --- a/tests/runtime-php-contract.sh +++ b/tests/runtime-php-contract.sh @@ -16,7 +16,23 @@ assert_file_contains "$file" 'ARG PHP_EXT' assert_file_contains "$file" 'ARG PHP_EXT_VERSIONED' assert_file_contains "$file" 'ARG UID=1000' assert_file_contains "$file" 'ARG GID=1000' -assert_file_contains "$file" 'php-cli-setup.sh' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_REF=main' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT=10' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_DOWNLOAD_MAX_TIME=120' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_DOWNLOAD_RETRIES=3' +assert_file_contains "$file" 'Scriptomatic/${SCRIPTOMATIC_REF}/bash/php-cli-setup.sh' +assert_file_contains "$file" '--connect-timeout "${SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT}"' +assert_file_contains "$file" '--max-time "${SCRIPTOMATIC_DOWNLOAD_MAX_TIME}"' +assert_file_contains "$file" '--retry "${SCRIPTOMATIC_DOWNLOAD_RETRIES}"' +assert_file_contains "$file" 'test -s "$bootstrap"' +assert_file_contains "$file" 'bash -n "$bootstrap"' +assert_file_contains "$file" 'SCRIPTOMATIC_REF="${SCRIPTOMATIC_REF}"' +assert_file_contains "$file" 'bash "$bootstrap" "${USERNAME}" "${PHP_VERSION}"' assert_file_contains "$file" 'ENTRYPOINT ["/usr/local/bin/php-entry"]' assert_file_contains "$file" 'CMD ["php-fpm"]' -pass "PHP runtime customization contract" + +if grep -Eq '^ADD https?://|Scriptomatic/master/' "$file"; then + fail "PHP runtime must not use remote ADD or the stale Scriptomatic master ref" +fi + +pass "PHP selected-version Alpine runtime and Scriptomatic bootstrap contract" From 406c23628a3fb57d21b5a7a9264d9955c3511712 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:03:34 +0600 Subject: [PATCH 078/509] test: enforce hardened Node runtime bootstrap --- tests/runtime-node-contract.sh | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/tests/runtime-node-contract.sh b/tests/runtime-node-contract.sh index c83c8d2f..408b17af 100755 --- a/tests/runtime-node-contract.sh +++ b/tests/runtime-node-contract.sh @@ -15,7 +15,24 @@ assert_file_contains "$file" 'ARG NODE_GLOBAL' assert_file_contains "$file" 'ARG NODE_GLOBAL_VERSIONED' assert_file_contains "$file" 'ARG UID=1000' assert_file_contains "$file" 'ARG GID=1000' -assert_file_contains "$file" 'node-cli-setup.sh' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_REF=main' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT=10' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_DOWNLOAD_MAX_TIME=120' +assert_file_contains "$file" 'ARG SCRIPTOMATIC_DOWNLOAD_RETRIES=3' +assert_file_contains "$file" 'Scriptomatic/${SCRIPTOMATIC_REF}/bash/node-cli-setup.sh' +assert_file_contains "$file" '--connect-timeout "${SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT}"' +assert_file_contains "$file" '--max-time "${SCRIPTOMATIC_DOWNLOAD_MAX_TIME}"' +assert_file_contains "$file" '--retry "${SCRIPTOMATIC_DOWNLOAD_RETRIES}"' +assert_file_contains "$file" 'test -s "$bootstrap"' +assert_file_contains "$file" 'bash -n "$bootstrap"' +assert_file_contains "$file" 'resolved_node_version="$(node -v' +assert_file_contains "$file" 'SCRIPTOMATIC_REF="${SCRIPTOMATIC_REF}"' +assert_file_contains "$file" 'bash "$bootstrap" "${USERNAME}" "$resolved_node_version"' assert_file_contains "$file" 'EXPOSE 3000' assert_file_contains "$file" 'ENTRYPOINT ["/usr/local/bin/node-entry"]' -pass "Node runtime customization contract" + +if grep -Eq '^ADD https?://|Scriptomatic/master/' "$file"; then + fail "Node runtime must not use remote ADD or the stale Scriptomatic master ref" +fi + +pass "Node selected-version Alpine runtime and Scriptomatic bootstrap contract" From 7d1f40f6113cd03bbe02529ab01b821b46f3a1b3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:04:10 +0600 Subject: [PATCH 079/509] runtime: preserve selected PHP version inside build stage --- docker/dockerfiles/php.Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/dockerfiles/php.Dockerfile b/docker/dockerfiles/php.Dockerfile index 8eb2cb2f..44fa9a42 100644 --- a/docker/dockerfiles/php.Dockerfile +++ b/docker/dockerfiles/php.Dockerfile @@ -1,5 +1,6 @@ ARG PHP_VERSION=8.4 FROM php:${PHP_VERSION}-fpm-alpine +ARG PHP_VERSION LABEL org.opencontainers.image.source="https://github.com/infocyph/LocalDevStack" LABEL org.opencontainers.image.description="PHP FPM Alpine" From c321e562ff0fb409bbcd17fa700d9c503bd433f0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:04:15 +0600 Subject: [PATCH 080/509] runtime: preserve selected Node version inside build stage --- docker/dockerfiles/node.Dockerfile | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/dockerfiles/node.Dockerfile b/docker/dockerfiles/node.Dockerfile index ac2f09c6..7a112688 100644 --- a/docker/dockerfiles/node.Dockerfile +++ b/docker/dockerfiles/node.Dockerfile @@ -1,5 +1,6 @@ ARG NODE_VERSION=current FROM node:${NODE_VERSION}-alpine +ARG NODE_VERSION LABEL org.opencontainers.image.source="https://github.com/infocyph/LocalDevStack" LABEL org.opencontainers.image.description="NodeJS Alpine" From f362fa11fad2fa29f1fa964bc34b0841849d3013 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:05:05 +0600 Subject: [PATCH 081/509] runtime: cache rebuilds and thread Scriptomatic ref --- lds | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/lds b/lds index 6a1040e6..a9916d3d 100755 --- a/lds +++ b/lds @@ -531,7 +531,13 @@ dc_cmd() { dc_up() { dc_cmd up "$@"; } dc_pull() { dc_cmd pull "$@"; } -dc_build() { dc_cmd build "$@"; } +dc_build() { + local scriptomatic_ref + scriptomatic_ref="$(compose_control_value SCRIPTOMATIC_REF main)" + [[ "$scriptomatic_ref" == "main" || "$scriptomatic_ref" =~ ^[0-9A-Fa-f]{40}$ ]] || + die "SCRIPTOMATIC_REF must be main or a full 40-character commit SHA" + dc_cmd build --build-arg "SCRIPTOMATIC_REF=$scriptomatic_ref" "$@" +} # helper for our own minimal logging (still shows in quiet mode) logv() { ((VERBOSE)) && printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2 || true; } @@ -2257,7 +2263,7 @@ cmd_rebuild() { if compose_has_build "$svc"; then logq rebuild "build/recreate $svc" - dc_build --no-cache --pull "$svc" + dc_build --pull "$svc" dc_up -d --no-deps --force-recreate "$svc" continue fi @@ -2271,7 +2277,7 @@ cmd_rebuild() { docker rmi -f "$img" >/dev/null 2>&1 || true dc_pull "$svc" || true else - dc_build --no-cache --pull "$svc" >/dev/null 2>&1 || true + dc_build --pull "$svc" >/dev/null 2>&1 || true fi dc_up -d --no-deps --force-recreate "$svc" From 0b688146cbae1198d5fa7bbdca6f9198ec675733 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:05:09 +0600 Subject: [PATCH 082/509] runtime: default Scriptomatic bootstrap ref to main --- docker/release.env | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/release.env b/docker/release.env index 331f0eba..1d39cccf 100644 --- a/docker/release.env +++ b/docker/release.env @@ -7,3 +7,4 @@ LDS_NGINX_IMAGE=infocyph/nginx:latest LDS_APACHE_IMAGE=infocyph/apache:latest LDS_LLM_IMAGE=infocyph/llm-sm:latest LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest +SCRIPTOMATIC_REF=main From 5c8261af2d0989152e7a809060af03d32d423c51 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:05:28 +0600 Subject: [PATCH 083/509] test: enforce cached runtime rebuild contract --- tests/env-contract.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index e4080947..dbb21e04 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -36,6 +36,7 @@ expected=( 'LDS_APACHE_IMAGE=infocyph/apache:latest' 'LDS_LLM_IMAGE=infocyph/llm-sm:latest' 'LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest' + 'SCRIPTOMATIC_REF=main' ) for entry in "${expected[@]}"; do assert_file_contains "$release_env" "$entry" @@ -53,3 +54,10 @@ assert_file_contains "$ROOT/lds" '_tools_exec_argv()' assert_file_contains "$ROOT/lds" 'ai) cmd_ai "$@"' assert_file_contains "$ROOT/lds" 'llm) cmd_llm "$@"' pass "AI/LLM CLI routing contract" + +assert_file_contains "$ROOT/lds" 'dc_cmd build --build-arg "SCRIPTOMATIC_REF=$scriptomatic_ref"' +if grep -Fq 'dc_build --no-cache' "$ROOT/lds"; then + fail "runtime rebuild path must preserve Docker build cache" +fi +assert_file_contains "$ROOT/lds" 'dc_build --pull "$svc"' +pass "runtime rebuilds preserve cache while refreshing selected bases" From e59a93e885d5c5cfb6241ef92f2e84359230bd0a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:05:32 +0600 Subject: [PATCH 084/509] test: preserve selected PHP version in build stage --- tests/runtime-php-contract.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/runtime-php-contract.sh b/tests/runtime-php-contract.sh index 8ff6a680..f86f67a9 100755 --- a/tests/runtime-php-contract.sh +++ b/tests/runtime-php-contract.sh @@ -9,6 +9,8 @@ file="$ROOT/docker/dockerfiles/php.Dockerfile" assert_file "$file" assert_file_contains "$file" 'ARG PHP_VERSION=8.4' assert_file_contains "$file" 'FROM php:${PHP_VERSION}-fpm-alpine' +[[ "$(grep -c '^ARG PHP_VERSION' "$file")" -eq 2 ]] || + fail "PHP_VERSION must be redeclared inside the build stage" assert_file_contains "$file" 'ARG PHP_PROFILE_KEY=84' assert_file_contains "$file" 'ARG LINUX_PKG' assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED' From 0abf89f6ea8797e901cc513821822c31ef345d62 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:05:36 +0600 Subject: [PATCH 085/509] test: preserve selected Node version in build stage --- tests/runtime-node-contract.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/runtime-node-contract.sh b/tests/runtime-node-contract.sh index 408b17af..ad1a2c16 100755 --- a/tests/runtime-node-contract.sh +++ b/tests/runtime-node-contract.sh @@ -9,6 +9,8 @@ file="$ROOT/docker/dockerfiles/node.Dockerfile" assert_file "$file" assert_file_contains "$file" 'ARG NODE_VERSION=current' assert_file_contains "$file" 'FROM node:${NODE_VERSION}-alpine' +[[ "$(grep -c '^ARG NODE_VERSION' "$file")" -eq 2 ]] || + fail "NODE_VERSION must be redeclared inside the build stage" assert_file_contains "$file" 'ARG LINUX_PKG' assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED' assert_file_contains "$file" 'ARG NODE_GLOBAL' From e6dfe0d0bb75f1b6adcce7919707cbb8d3af3125 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:05:51 +0600 Subject: [PATCH 086/509] plan: preserve runtime version selector invariant --- .../07-localdevstack-integration-plan.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index ab270622..50aa949c 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1938,3 +1938,21 @@ The Alpine-first moving-tag policy changes default image selection only; it does - persistent-volume behavior; - loopback-only optional Ollama host exposure; - compatibility/contract testing. + + +## Runtime version-selection invariant + +The Alpine-first moving-image policy does **not** replace the existing PHP/Node runtime version selector. + +LocalDevStack must preserve the published Tools runtime-selection contract: + +- `mkhost` reads `/etc/share/runtime-versions.json`; +- PHP runtime selection remains version-specific; +- Node runtime selection remains version/tag-specific; +- the selected PHP value remains the `PHP_VERSION` build arg and `localdevstack-php:` image identity; +- the selected Node value remains the `NODE_VERSION` build arg and `localdevstack-node:` image identity; +- selected PHP and Node bases continue to use their Alpine variants; +- moving Alpine/latest defaults apply only when the user has not selected/persisted a more specific version; +- explicit user-selected versions always win. + +Batch 6 must not collapse these selectors into a single global PHP or Node version. From aa63fe935add9510ff1e0e3b4b8fdc6dd7b2c43b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:07:57 +0600 Subject: [PATCH 087/509] refactor: extract compose.sh module --- lib/compose.sh | 117 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 117 insertions(+) create mode 100644 lib/compose.sh diff --git a/lib/compose.sh b/lib/compose.sh new file mode 100644 index 00000000..a62d73b1 --- /dev/null +++ b/lib/compose.sh @@ -0,0 +1,117 @@ +############################################################################### +# 1a. DOCKER COMPOSE WRAPPER +############################################################################### + +# ── compose extras (docker/extras/*.y{a,}ml) ──────────────────────────────── +__EXTRAS_LOADED=0 +declare -a __EXTRA_FILES=() + +load_extras() { + # Set LDS_EXTRAS_RELOAD=1 (or global --reload-extras) to re-scan templates every call. + if [[ "${LDS_EXTRAS_RELOAD:-0}" == "1" ]]; then + __EXTRAS_LOADED=0 + fi + + ((__EXTRAS_LOADED)) && return 0 + __EXTRAS_LOADED=1 + + [[ -d "$EXTRAS_DIR" ]] || return 0 + + mapfile -t __EXTRA_FILES < <( + find "$EXTRAS_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -print 2>/dev/null | sort | sed '/^[[:space:]]*$/d' + ) +} + +docker_compose() { + load_extras + # Create required runtime files only when docker/compose operations are invoked. + ((EUID == 0)) || ensure_files_exist "/docker/.env" "/configuration/php/php.ini" "/.env" + if [[ -z "${__LDS_DC_BIN:-}" ]]; then + if docker compose version >/dev/null 2>&1; then + __LDS_DC_BIN=(docker compose) + else + __LDS_DC_BIN=(docker-compose) + fi + fi + + [[ -r "$ENV_RELEASE" ]] || die "Missing release compatibility manifest: $ENV_RELEASE" + + # Release defaults are loaded first; user docker/.env overrides them. + # Shell variables remain higher-precedence Compose interpolation inputs. + local -a env_files=(--env-file "$ENV_RELEASE") + [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") + + local ai_runtime ai_host_port + local -a static_f=() + + ai_runtime="$(compose_control_value LDS_AI_RUNTIME cpu)" + case "${ai_runtime,,}" in + "" | cpu) ;; + nvidia) static_f+=(-f "$CFG/compose/ai-nvidia.yaml") ;; + amd) static_f+=(-f "$CFG/compose/ai-amd.yaml") ;; + *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd)" ;; + esac + + ai_host_port="$(compose_control_value LDS_LLM_HOST_PORT 0)" + case "${ai_host_port,,}" in + "" | 0 | false | no | off) ;; + 1 | true | yes | on) static_f+=(-f "$CFG/compose/ai-host-port.yaml") ;; + *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; + esac + + # Product overrides are applied before user-provided compose extras. + local -a extra_f=() f + for f in "${__EXTRA_FILES[@]:-}"; do + [[ -f "$f" ]] || continue + case "$f" in + *.yml | *.yaml) extra_f+=(-f "$f") ;; + esac + done + + local host_os="${HOST_OS:-$(detect_host_os)}" + + HOST_OS="$host_os" "${__LDS_DC_BIN[@]}" \ + --project-directory "$DIR" \ + -f "$COMPOSE_FILE" \ + "${static_f[@]}" \ + "${extra_f[@]}" \ + "${env_files[@]}" \ + "$@" +} + +# helper: print project name +lds_project() { printf '%s' "${__LDS_PROJECT:-$(basename -- "$DIR")}"; } + +# (QUIET by default) ──────────────────────────────── +# Centralize quiet/verbose handling for compose subcommands. +# Usage: dc_cmd [args...] +dc_cmd() { + local sub="${1:-}" + shift || true + + local -a quiet=() + if ((VERBOSE == 0)); then + case "$sub" in + up) quiet+=(--quiet-pull) ;; + pull) quiet+=(-q) ;; + build) quiet+=(--quiet) ;; + esac + fi + + docker_compose "$sub" "${quiet[@]}" "$@" +} + +dc_up() { dc_cmd up "$@"; } +dc_pull() { dc_cmd pull "$@"; } +dc_build() { + local scriptomatic_ref + scriptomatic_ref="$(compose_control_value SCRIPTOMATIC_REF main)" + [[ "$scriptomatic_ref" == "main" || "$scriptomatic_ref" =~ ^[0-9A-Fa-f]{40}$ ]] || + die "SCRIPTOMATIC_REF must be main or a full 40-character commit SHA" + dc_cmd build --build-arg "SCRIPTOMATIC_REF=$scriptomatic_ref" "$@" +} + +# helper for our own minimal logging (still shows in quiet mode) +logv() { ((VERBOSE)) && printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2 || true; } +logq() { printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2; } + From e25b510b3c8a924aa666e7bf9c2af1fc1e22a6c6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:08:01 +0600 Subject: [PATCH 088/509] refactor: extract env.sh module --- lib/env.sh | 98 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 98 insertions(+) create mode 100644 lib/env.sh diff --git a/lib/env.sh b/lib/env.sh new file mode 100644 index 00000000..4d4c2c02 --- /dev/null +++ b/lib/env.sh @@ -0,0 +1,98 @@ +############################################################################### +# 1b. PROMPTS + DOTENV HELPERS +############################################################################### + +# Unified prompt helper (used by env_init + profiles) +tty_readline() { + # Robust prompt/read across Linux/macOS/WSL/Windows Git Bash. + # Prefer stdin when it is a TTY (normal interactive use). If stdin is not a TTY, + # fall back to /dev/tty when available. + local __var_name="$1" __prompt="$2" __line + + if [[ -t 0 ]]; then + # Interactive: show prompt on stderr (so it is never swallowed) and read stdin. + printf '%s' "$__prompt" >&2 + IFS= read -r __line || return 1 + elif [[ -r /dev/tty ]]; then + # Non-interactive stdin (piped) but we still have a controlling terminal. + printf '%s' "$__prompt" >/dev/tty + IFS= read -r __line keep as-is + if [[ "$v" =~ ^\".*\"$ || "$v" =~ ^\'.*\'$ ]]; then + printf '%s' "$v" + return 0 + fi + + # Leading/trailing whitespace or any internal whitespace or # or quotes => quote + if [[ "$v" =~ ^[[:space:]] || "$v" =~ [[:space:]]$ || "$v" == *$'\t'* || "$v" == *" "* || "$v" == *"#"* || "$v" == *"\""* ]]; then + env_quote "$v" + return 0 + fi + + printf '%s' "$v" +} + +# Escape replacement for sed (delimiter '|') +sed_escape_repl() { + local s=${1-} + s=${s//\\/\\\\} + s=${s//&/\\&} + s=${s//|/\\|} + printf '%s' "$s" +} + +update_env() { + local file=$1 var=$2 val=${3-} + mkdir -p "$(dirname "$file")" + [[ -f "$file" ]] || { + printf "%bFile '%s' not found. Creating one.%b\n" "$YELLOW" "$file" "$NC" + : >"$file" + } + + # Apply quoting only when needed (spaces etc.) + val="$(env_quote_if_needed "$val")" + + # Sed-safe replacement + local val_sed + val_sed="$(sed_escape_repl "$val")" + + var=$(echo "$var" | sed 's/[]\/$*.^|[]/\\&/g') + if grep -qE "^[# ]*$var=" "$file" 2>/dev/null; then + sed -Ei "s|^[# ]*($var)=.*|\1=$val_sed|" "$file" + else + printf "%s=%s\n" "$var" "$val" >>"$file" + fi +} + From 3d172504daff1ad12007bfd888026973525a9592 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:08:05 +0600 Subject: [PATCH 089/509] refactor: extract profiles.sh module --- lib/profiles.sh | 230 ++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 230 insertions(+) create mode 100644 lib/profiles.sh diff --git a/lib/profiles.sh b/lib/profiles.sh new file mode 100644 index 00000000..fce2c95e --- /dev/null +++ b/lib/profiles.sh @@ -0,0 +1,230 @@ +############################################################################### +# 3a. PROFILES: DEFINITIONS + SETUP FLOW +############################################################################### + +CATALOG_FILE="$CFG/catalog/services.psv" + +declare -A SERVICES=() +declare -A SERVICE_DISPLAY=() +declare -A SERVICE_KEY=() +declare -A SERVICE_VERSION_ENV=() +declare -A PROFILE_ENV=() +declare -A PROFILE_PROMPTS=() +declare -A SERVICE_ADMIN_CLIENT=() +declare -A SERVICE_VOLUME=() +declare -A SERVICE_URL=() +declare -A SERVICE_CATEGORY=() +declare -A SERVICE_OPTIONAL=() +declare -A SERVICE_DEFAULT_ENABLED=() +declare -A SERVICE_RUNTIME_MODES=() +declare -a SERVICE_ORDER=() + +load_service_catalog() { + [[ -r "$CATALOG_FILE" ]] || die "Missing service catalog: $CATALOG_FILE" + + local key profile display service_key version_env defaults prompts + local admin_client volume url category optional default_enabled runtime_modes + + while IFS='|' read -r key profile display service_key version_env defaults prompts admin_client volume url category optional default_enabled runtime_modes; do + [[ -n "$key" && "$key" != \#* ]] || continue + [[ -n "$profile" && -n "$display" && -n "$service_key" ]] || + die "Invalid service catalog row for: $key" + [[ -z "${SERVICES[$key]+x}" ]] || die "Duplicate service catalog key: $key" + [[ -z "${PROFILE_ENV[$profile]+x}" ]] || die "Duplicate service catalog profile: $profile" + + SERVICE_ORDER+=("$key") + SERVICES["$key"]="$profile" + SERVICE_DISPLAY["$key"]="$display" + SERVICE_KEY["$key"]="$service_key" + SERVICE_VERSION_ENV["$key"]="$version_env" + PROFILE_ENV["$profile"]="$defaults" + PROFILE_PROMPTS["$profile"]="$prompts" + SERVICE_ADMIN_CLIENT["$key"]="$admin_client" + SERVICE_VOLUME["$key"]="$volume" + SERVICE_URL["$key"]="$url" + SERVICE_CATEGORY["$key"]="$category" + SERVICE_OPTIONAL["$key"]="$optional" + SERVICE_DEFAULT_ENABLED["$key"]="$default_enabled" + SERVICE_RUNTIME_MODES["$key"]="$runtime_modes" + done <"$CATALOG_FILE" + + (("${#SERVICE_ORDER[@]}" > 0)) || die "Service catalog is empty: $CATALOG_FILE" +} + +load_service_catalog + +declare -a PENDING_ENVS=() +declare -a PENDING_PROFILES=() + +queue_env() { PENDING_ENVS+=("$1"); } +queue_profile() { PENDING_PROFILES+=("$1"); } + +flush_envs() { + local env_file="$ENV_DOCKER" kv key val + for kv in "${PENDING_ENVS[@]}"; do + IFS='=' read -r key val <<<"$kv" + update_env "$env_file" "$key" "$val" + done +} + +flush_profiles() { + local profile + for profile in "${PENDING_PROFILES[@]}"; do + modify_profiles add "$profile" + done +} + +# ── setup menu (selection-first) ────────────────────────────────────────────── + +setup_menu_print() { + # Print menu to stderr to avoid stdout buffering in some Windows wrappers. + { + printf "\n%bSetup profiles%b (will replace previous configuration, if exists):\n\n" "$CYAN" "$NC" + local i=1 key slug display + for key in "${SERVICE_ORDER[@]}"; do + slug="${SERVICES[$key]}" + display="${SERVICE_DISPLAY[$key]}" + printf " %2d) %-16s (%s)\n" "$i" "$display" "$slug" + i=$((i + 1)) + done + printf "\n a) ALL\n" + printf " n) NONE / Back\n\n" + } >&2 +} + +# Parse user selection into indices or ALL/NONE (prints one token per line) +setup_menu_parse() { + local input="${1//[[:space:]]/}" + [[ -n "$input" ]] || return 1 + input="${input//;/,}" + + echo "$input" | tr ',' '\n' | awk ' + BEGIN { ok=1 } + /^[0-9]+-[0-9]+$/ { + split($0,a,"-") + if (a[1] > a[2]) { t=a[1]; a[1]=a[2]; a[2]=t } + for (i=a[1]; i<=a[2]; i++) print i + next + } + /^[0-9]+$/ { print $0; next } + /^[aA]$/ { print "ALL"; next } + /^[nN]$/ { print "NONE"; next } + { ok=0 } + END { if (!ok) exit 2 } + ' +} + +# Outputs: newline-separated service KEYS from SERVICE_ORDER (e.g. MYSQL, REDIS) +setup_choose_services() { + local ans parsed + while :; do + setup_menu_print + tty_readline ans "Select (e.g. 1,3,5 or 2-4 or a): " || return 1 + + if ! parsed="$(setup_menu_parse "$ans" 2>/dev/null)"; then + printf "%bInvalid selection.%b Try again.\n" "$YELLOW" "$NC" + continue + fi + + if grep -qx "NONE" <<<"$parsed"; then + return 1 + fi + + if grep -qx "ALL" <<<"$parsed"; then + printf "%s\n" "${SERVICE_ORDER[@]}" + return 0 + fi + + # Indices -> keys (de-dupe, preserve order) + local -A seen=() + local out=() + local idx key + while IFS= read -r idx; do + [[ "$idx" =~ ^[0-9]+$ ]] || continue + ((idx >= 1 && idx <= ${#SERVICE_ORDER[@]})) || continue + key="${SERVICE_ORDER[idx - 1]}" + [[ -n "${seen[$key]:-}" ]] && continue + seen[$key]=1 + out+=("$key") + done <<<"$parsed" + + if ((${#out[@]} == 0)); then + printf "%bNo valid items selected.%b\n" "$YELLOW" "$NC" + continue + fi + + printf "%s\n" "${out[@]}" + return 0 + done +} + +setup_service() { + local service="$1" + local profile="${SERVICES[$service]:-}" + local display="${SERVICE_DISPLAY[$service]:-$service}" + [[ -n "$profile" ]] || die "Unknown service: $service" + + printf "\n%b→ %s%b\n" "$YELLOW" "$display" "$NC" + queue_profile "$profile" + + local defaults="${PROFILE_ENV[$profile]:-}" + [[ -n "$defaults" ]] || return 0 + + printf "%bEnter value(s) for %s:%b\n" "$BLUE" "$display" "$NC" + + local -a pairs=() prompts=() + IFS=';' read -r -a pairs <<<"$defaults" + IFS=';' read -r -a prompts <<<"${PROFILE_PROMPTS[$profile]:-}" + + local i pair key def val prompt + for i in "${!pairs[@]}"; do + pair="${pairs[$i]}" + [[ -n "$pair" ]] || continue + IFS='=' read -r key def <<<"$pair" + prompt="${prompts[$i]:-$key}" + val="$(read_default "$prompt" "$def")" + + case "$key" in + LDS_AI_RUNTIME) + val="${val,,}" + case "$val" in + cpu | nvidia | amd) ;; + *) die "AI runtime must be cpu, nvidia, or amd" ;; + esac + ;; + LDS_LLM_HOST_PORT) + case "${val,,}" in + 1 | y | yes | true | on) val=1 ;; + 0 | n | no | false | off) val=0 ;; + *) die "Direct localhost API must be yes or no" ;; + esac + ;; + esac + + queue_env "$key=$val" + done +} + +process_all() { + local selected + if ! selected="$(setup_choose_services)"; then + printf "\n%bSetup cancelled.%b\n" "$YELLOW" "$NC" + return 0 + fi + + printf "\n%bWill configure:%b\n" "$CYAN" "$NC" + while IFS= read -r svc; do + printf " - %s (%s)\n" "${SERVICE_DISPLAY[$svc]:-$svc}" "${SERVICES[$svc]}" + done <<<"$selected" + echo + + local svc + while IFS= read -r svc; do + setup_service "$svc" + done <<<"$selected" + + flush_envs + flush_profiles + printf "\n%b✅ Selected services configured!%b\n" "$GREEN" "$NC" +} + From 016822685009b2698adda3682e3ed22438ed4737 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:08:08 +0600 Subject: [PATCH 090/509] refactor: source compose env and profile modules --- lds | 439 +----------------------------------------------------------- 1 file changed, 6 insertions(+), 433 deletions(-) diff --git a/lds b/lds index a9916d3d..039b5af9 100755 --- a/lds +++ b/lds @@ -429,217 +429,14 @@ on_error() { ############################################################################### # 1a. DOCKER COMPOSE WRAPPER ############################################################################### - -# ── compose extras (docker/extras/*.y{a,}ml) ──────────────────────────────── -__EXTRAS_LOADED=0 -declare -a __EXTRA_FILES=() - -load_extras() { - # Set LDS_EXTRAS_RELOAD=1 (or global --reload-extras) to re-scan templates every call. - if [[ "${LDS_EXTRAS_RELOAD:-0}" == "1" ]]; then - __EXTRAS_LOADED=0 - fi - - ((__EXTRAS_LOADED)) && return 0 - __EXTRAS_LOADED=1 - - [[ -d "$EXTRAS_DIR" ]] || return 0 - - mapfile -t __EXTRA_FILES < <( - find "$EXTRAS_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -print 2>/dev/null | sort | sed '/^[[:space:]]*$/d' - ) -} - -docker_compose() { - load_extras - # Create required runtime files only when docker/compose operations are invoked. - ((EUID == 0)) || ensure_files_exist "/docker/.env" "/configuration/php/php.ini" "/.env" - if [[ -z "${__LDS_DC_BIN:-}" ]]; then - if docker compose version >/dev/null 2>&1; then - __LDS_DC_BIN=(docker compose) - else - __LDS_DC_BIN=(docker-compose) - fi - fi - - [[ -r "$ENV_RELEASE" ]] || die "Missing release compatibility manifest: $ENV_RELEASE" - - # Release defaults are loaded first; user docker/.env overrides them. - # Shell variables remain higher-precedence Compose interpolation inputs. - local -a env_files=(--env-file "$ENV_RELEASE") - [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - - local ai_runtime ai_host_port - local -a static_f=() - - ai_runtime="$(compose_control_value LDS_AI_RUNTIME cpu)" - case "${ai_runtime,,}" in - "" | cpu) ;; - nvidia) static_f+=(-f "$CFG/compose/ai-nvidia.yaml") ;; - amd) static_f+=(-f "$CFG/compose/ai-amd.yaml") ;; - *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd)" ;; - esac - - ai_host_port="$(compose_control_value LDS_LLM_HOST_PORT 0)" - case "${ai_host_port,,}" in - "" | 0 | false | no | off) ;; - 1 | true | yes | on) static_f+=(-f "$CFG/compose/ai-host-port.yaml") ;; - *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; - esac - - # Product overrides are applied before user-provided compose extras. - local -a extra_f=() f - for f in "${__EXTRA_FILES[@]:-}"; do - [[ -f "$f" ]] || continue - case "$f" in - *.yml | *.yaml) extra_f+=(-f "$f") ;; - esac - done - - local host_os="${HOST_OS:-$(detect_host_os)}" - - HOST_OS="$host_os" "${__LDS_DC_BIN[@]}" \ - --project-directory "$DIR" \ - -f "$COMPOSE_FILE" \ - "${static_f[@]}" \ - "${extra_f[@]}" \ - "${env_files[@]}" \ - "$@" -} - -# helper: print project name -lds_project() { printf '%s' "${__LDS_PROJECT:-$(basename -- "$DIR")}"; } - -# (QUIET by default) ──────────────────────────────── -# Centralize quiet/verbose handling for compose subcommands. -# Usage: dc_cmd [args...] -dc_cmd() { - local sub="${1:-}" - shift || true - - local -a quiet=() - if ((VERBOSE == 0)); then - case "$sub" in - up) quiet+=(--quiet-pull) ;; - pull) quiet+=(-q) ;; - build) quiet+=(--quiet) ;; - esac - fi - - docker_compose "$sub" "${quiet[@]}" "$@" -} - -dc_up() { dc_cmd up "$@"; } -dc_pull() { dc_cmd pull "$@"; } -dc_build() { - local scriptomatic_ref - scriptomatic_ref="$(compose_control_value SCRIPTOMATIC_REF main)" - [[ "$scriptomatic_ref" == "main" || "$scriptomatic_ref" =~ ^[0-9A-Fa-f]{40}$ ]] || - die "SCRIPTOMATIC_REF must be main or a full 40-character commit SHA" - dc_cmd build --build-arg "SCRIPTOMATIC_REF=$scriptomatic_ref" "$@" -} - -# helper for our own minimal logging (still shows in quiet mode) -logv() { ((VERBOSE)) && printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2 || true; } -logq() { printf "%b[%s]%b %s\n" "$CYAN" "${1:-info}" "$NC" "${2:-}" >&2; } +# shellcheck source=lib/compose.sh +source "$DIR/lib/compose.sh" ############################################################################### # 1b. PROMPTS + DOTENV HELPERS ############################################################################### - -# Unified prompt helper (used by env_init + profiles) -tty_readline() { - # Robust prompt/read across Linux/macOS/WSL/Windows Git Bash. - # Prefer stdin when it is a TTY (normal interactive use). If stdin is not a TTY, - # fall back to /dev/tty when available. - local __var_name="$1" __prompt="$2" __line - - if [[ -t 0 ]]; then - # Interactive: show prompt on stderr (so it is never swallowed) and read stdin. - printf '%s' "$__prompt" >&2 - IFS= read -r __line || return 1 - elif [[ -r /dev/tty ]]; then - # Non-interactive stdin (piped) but we still have a controlling terminal. - printf '%s' "$__prompt" >/dev/tty - IFS= read -r __line keep as-is - if [[ "$v" =~ ^\".*\"$ || "$v" =~ ^\'.*\'$ ]]; then - printf '%s' "$v" - return 0 - fi - - # Leading/trailing whitespace or any internal whitespace or # or quotes => quote - if [[ "$v" =~ ^[[:space:]] || "$v" =~ [[:space:]]$ || "$v" == *$'\t'* || "$v" == *" "* || "$v" == *"#"* || "$v" == *"\""* ]]; then - env_quote "$v" - return 0 - fi - - printf '%s' "$v" -} - -# Escape replacement for sed (delimiter '|') -sed_escape_repl() { - local s=${1-} - s=${s//\\/\\\\} - s=${s//&/\\&} - s=${s//|/\\|} - printf '%s' "$s" -} - -update_env() { - local file=$1 var=$2 val=${3-} - mkdir -p "$(dirname "$file")" - [[ -f "$file" ]] || { - printf "%bFile '%s' not found. Creating one.%b\n" "$YELLOW" "$file" "$NC" - : >"$file" - } - - # Apply quoting only when needed (spaces etc.) - val="$(env_quote_if_needed "$val")" - - # Sed-safe replacement - local val_sed - val_sed="$(sed_escape_repl "$val")" - - var=$(echo "$var" | sed 's/[]\/$*.^|[]/\\&/g') - if grep -qE "^[# ]*$var=" "$file" 2>/dev/null; then - sed -Ei "s|^[# ]*($var)=.*|\1=$val_sed|" "$file" - else - printf "%s=%s\n" "$var" "$val" >>"$file" - fi -} +# shellcheck source=lib/env.sh +source "$DIR/lib/env.sh" ############################################################################### # 1c. HTTP / WEB SERVER HELPERS @@ -810,232 +607,8 @@ modify_profiles() { ############################################################################### # 3a. PROFILES: DEFINITIONS + SETUP FLOW ############################################################################### - -CATALOG_FILE="$CFG/catalog/services.psv" - -declare -A SERVICES=() -declare -A SERVICE_DISPLAY=() -declare -A SERVICE_KEY=() -declare -A SERVICE_VERSION_ENV=() -declare -A PROFILE_ENV=() -declare -A PROFILE_PROMPTS=() -declare -A SERVICE_ADMIN_CLIENT=() -declare -A SERVICE_VOLUME=() -declare -A SERVICE_URL=() -declare -A SERVICE_CATEGORY=() -declare -A SERVICE_OPTIONAL=() -declare -A SERVICE_DEFAULT_ENABLED=() -declare -A SERVICE_RUNTIME_MODES=() -declare -a SERVICE_ORDER=() - -load_service_catalog() { - [[ -r "$CATALOG_FILE" ]] || die "Missing service catalog: $CATALOG_FILE" - - local key profile display service_key version_env defaults prompts - local admin_client volume url category optional default_enabled runtime_modes - - while IFS='|' read -r key profile display service_key version_env defaults prompts admin_client volume url category optional default_enabled runtime_modes; do - [[ -n "$key" && "$key" != \#* ]] || continue - [[ -n "$profile" && -n "$display" && -n "$service_key" ]] || - die "Invalid service catalog row for: $key" - [[ -z "${SERVICES[$key]+x}" ]] || die "Duplicate service catalog key: $key" - [[ -z "${PROFILE_ENV[$profile]+x}" ]] || die "Duplicate service catalog profile: $profile" - - SERVICE_ORDER+=("$key") - SERVICES["$key"]="$profile" - SERVICE_DISPLAY["$key"]="$display" - SERVICE_KEY["$key"]="$service_key" - SERVICE_VERSION_ENV["$key"]="$version_env" - PROFILE_ENV["$profile"]="$defaults" - PROFILE_PROMPTS["$profile"]="$prompts" - SERVICE_ADMIN_CLIENT["$key"]="$admin_client" - SERVICE_VOLUME["$key"]="$volume" - SERVICE_URL["$key"]="$url" - SERVICE_CATEGORY["$key"]="$category" - SERVICE_OPTIONAL["$key"]="$optional" - SERVICE_DEFAULT_ENABLED["$key"]="$default_enabled" - SERVICE_RUNTIME_MODES["$key"]="$runtime_modes" - done <"$CATALOG_FILE" - - (("${#SERVICE_ORDER[@]}" > 0)) || die "Service catalog is empty: $CATALOG_FILE" -} - -load_service_catalog - -declare -a PENDING_ENVS=() -declare -a PENDING_PROFILES=() - -queue_env() { PENDING_ENVS+=("$1"); } -queue_profile() { PENDING_PROFILES+=("$1"); } - -flush_envs() { - local env_file="$ENV_DOCKER" kv key val - for kv in "${PENDING_ENVS[@]}"; do - IFS='=' read -r key val <<<"$kv" - update_env "$env_file" "$key" "$val" - done -} - -flush_profiles() { - local profile - for profile in "${PENDING_PROFILES[@]}"; do - modify_profiles add "$profile" - done -} - -# ── setup menu (selection-first) ────────────────────────────────────────────── - -setup_menu_print() { - # Print menu to stderr to avoid stdout buffering in some Windows wrappers. - { - printf "\n%bSetup profiles%b (will replace previous configuration, if exists):\n\n" "$CYAN" "$NC" - local i=1 key slug display - for key in "${SERVICE_ORDER[@]}"; do - slug="${SERVICES[$key]}" - display="${SERVICE_DISPLAY[$key]}" - printf " %2d) %-16s (%s)\n" "$i" "$display" "$slug" - i=$((i + 1)) - done - printf "\n a) ALL\n" - printf " n) NONE / Back\n\n" - } >&2 -} - -# Parse user selection into indices or ALL/NONE (prints one token per line) -setup_menu_parse() { - local input="${1//[[:space:]]/}" - [[ -n "$input" ]] || return 1 - input="${input//;/,}" - - echo "$input" | tr ',' '\n' | awk ' - BEGIN { ok=1 } - /^[0-9]+-[0-9]+$/ { - split($0,a,"-") - if (a[1] > a[2]) { t=a[1]; a[1]=a[2]; a[2]=t } - for (i=a[1]; i<=a[2]; i++) print i - next - } - /^[0-9]+$/ { print $0; next } - /^[aA]$/ { print "ALL"; next } - /^[nN]$/ { print "NONE"; next } - { ok=0 } - END { if (!ok) exit 2 } - ' -} - -# Outputs: newline-separated service KEYS from SERVICE_ORDER (e.g. MYSQL, REDIS) -setup_choose_services() { - local ans parsed - while :; do - setup_menu_print - tty_readline ans "Select (e.g. 1,3,5 or 2-4 or a): " || return 1 - - if ! parsed="$(setup_menu_parse "$ans" 2>/dev/null)"; then - printf "%bInvalid selection.%b Try again.\n" "$YELLOW" "$NC" - continue - fi - - if grep -qx "NONE" <<<"$parsed"; then - return 1 - fi - - if grep -qx "ALL" <<<"$parsed"; then - printf "%s\n" "${SERVICE_ORDER[@]}" - return 0 - fi - - # Indices -> keys (de-dupe, preserve order) - local -A seen=() - local out=() - local idx key - while IFS= read -r idx; do - [[ "$idx" =~ ^[0-9]+$ ]] || continue - ((idx >= 1 && idx <= ${#SERVICE_ORDER[@]})) || continue - key="${SERVICE_ORDER[idx - 1]}" - [[ -n "${seen[$key]:-}" ]] && continue - seen[$key]=1 - out+=("$key") - done <<<"$parsed" - - if ((${#out[@]} == 0)); then - printf "%bNo valid items selected.%b\n" "$YELLOW" "$NC" - continue - fi - - printf "%s\n" "${out[@]}" - return 0 - done -} - -setup_service() { - local service="$1" - local profile="${SERVICES[$service]:-}" - local display="${SERVICE_DISPLAY[$service]:-$service}" - [[ -n "$profile" ]] || die "Unknown service: $service" - - printf "\n%b→ %s%b\n" "$YELLOW" "$display" "$NC" - queue_profile "$profile" - - local defaults="${PROFILE_ENV[$profile]:-}" - [[ -n "$defaults" ]] || return 0 - - printf "%bEnter value(s) for %s:%b\n" "$BLUE" "$display" "$NC" - - local -a pairs=() prompts=() - IFS=';' read -r -a pairs <<<"$defaults" - IFS=';' read -r -a prompts <<<"${PROFILE_PROMPTS[$profile]:-}" - - local i pair key def val prompt - for i in "${!pairs[@]}"; do - pair="${pairs[$i]}" - [[ -n "$pair" ]] || continue - IFS='=' read -r key def <<<"$pair" - prompt="${prompts[$i]:-$key}" - val="$(read_default "$prompt" "$def")" - - case "$key" in - LDS_AI_RUNTIME) - val="${val,,}" - case "$val" in - cpu | nvidia | amd) ;; - *) die "AI runtime must be cpu, nvidia, or amd" ;; - esac - ;; - LDS_LLM_HOST_PORT) - case "${val,,}" in - 1 | y | yes | true | on) val=1 ;; - 0 | n | no | false | off) val=0 ;; - *) die "Direct localhost API must be yes or no" ;; - esac - ;; - esac - - queue_env "$key=$val" - done -} - -process_all() { - local selected - if ! selected="$(setup_choose_services)"; then - printf "\n%bSetup cancelled.%b\n" "$YELLOW" "$NC" - return 0 - fi - - printf "\n%bWill configure:%b\n" "$CYAN" "$NC" - while IFS= read -r svc; do - printf " - %s (%s)\n" "${SERVICE_DISPLAY[$svc]:-$svc}" "${SERVICES[$svc]}" - done <<<"$selected" - echo - - local svc - while IFS= read -r svc; do - setup_service "$svc" - done <<<"$selected" - - flush_envs - flush_profiles - printf "\n%b✅ Selected services configured!%b\n" "$GREEN" "$NC" -} +# shellcheck source=lib/profiles.sh +source "$DIR/lib/profiles.sh" ############################################################################### # 5. ENVIRONMENT + CERT / CA From 5d42d318723bcdd05ec4ca45288db5b3527a4b5e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:08:25 +0600 Subject: [PATCH 091/509] refactor: track LocalDevStack shell modules --- .gitignore | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.gitignore b/.gitignore index 5b1bd229..74c02d64 100644 --- a/.gitignore +++ b/.gitignore @@ -10,6 +10,8 @@ !docker/release.env !docs !logs +!lib/ +!lib/** !.github/ !.github/** !tests/ From 13c99f7ea9e7c3c85910cbf9f6d3ed6ab74cac6f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:08:29 +0600 Subject: [PATCH 092/509] test: lint extracted LocalDevStack modules --- tests/static.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/static.sh b/tests/static.sh index d553be07..513a2849 100755 --- a/tests/static.sh +++ b/tests/static.sh @@ -9,6 +9,7 @@ mapfile -t shell_files < <( { printf '%s\n' "$ROOT/lds" find "$ROOT/bin" -maxdepth 1 -type f -print + find "$ROOT/lib" -type f -name '*.sh' -print find "$ROOT/tests" -type f -name '*.sh' -print } | sort -u ) From 2bfbbd18632b8b87527053d110db029c36f84110 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:09:27 +0600 Subject: [PATCH 093/509] refactor: annotate compose.sh as sourced Bash --- lib/compose.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/compose.sh b/lib/compose.sh index a62d73b1..c215e5cf 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -1,3 +1,4 @@ +# shellcheck shell=bash ############################################################################### # 1a. DOCKER COMPOSE WRAPPER ############################################################################### From df7781c32b1d538ec068fdd3aab79a2e4df73f0d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:09:32 +0600 Subject: [PATCH 094/509] refactor: annotate env.sh as sourced Bash --- lib/env.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/env.sh b/lib/env.sh index 4d4c2c02..93d56339 100644 --- a/lib/env.sh +++ b/lib/env.sh @@ -1,3 +1,4 @@ +# shellcheck shell=bash ############################################################################### # 1b. PROMPTS + DOTENV HELPERS ############################################################################### From 5bc5f9a232c62f818ced994ee7e2ff92f77b767c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:09:39 +0600 Subject: [PATCH 095/509] refactor: annotate profiles.sh as sourced Bash --- lib/profiles.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/profiles.sh b/lib/profiles.sh index fce2c95e..9702eb50 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -1,3 +1,4 @@ +# shellcheck shell=bash ############################################################################### # 3a. PROFILES: DEFINITIONS + SETUP FLOW ############################################################################### From bf78687d95a76b76c179510a613845c9e01fe0e3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:12:09 +0600 Subject: [PATCH 096/509] refactor: extract hosts.sh module --- lib/hosts.sh | 107 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 107 insertions(+) create mode 100644 lib/hosts.sh diff --git a/lib/hosts.sh b/lib/hosts.sh new file mode 100644 index 00000000..cf043e14 --- /dev/null +++ b/lib/hosts.sh @@ -0,0 +1,107 @@ +# shellcheck shell=bash +############################################################################### +# 1c. HTTP / WEB SERVER HELPERS +############################################################################### + +http_reload() { + printf "%bReloading HTTP...%b" "$MAGENTA" "$NC" + docker ps -qf name=NGINX &>/dev/null && docker exec NGINX nginx -s reload &>/dev/null || true + docker ps -qf name=APACHE &>/dev/null && docker exec APACHE apachectl graceful &>/dev/null || true + printf "\r%bHTTP reloaded! %b\n" "$GREEN" "$NC" +} + + +############################################################################### +# 3. DOMAIN / PROFILE INTEGRATION +############################################################################### +mkhost() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" mkhost "$@" +} +rmhost() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" rmhost "$@" +} + +setup_domain() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + mkhost --RESET + docker exec -it "$ctr" mkhost + local mk_state svr_prof + mk_state="$(mkhost --JSON || true)" + if has_tool jq; then + svr_prof="$(printf '%s' "$mk_state" | jq -r '.state.apache_active // empty' 2>/dev/null || true)" + else + svr_prof="$(printf '%s' "$mk_state" | tr -d '\r\n' | sed -n 's/.*"apache_active"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" + fi + [[ -n $svr_prof ]] && modify_profiles add "$svr_prof" + mkhost --RESET + cmd_reboot +} + +delete_domain() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + rmhost --RESET + + # interactive delete + docker exec -it "$ctr" rmhost "$@" + + local rm_state apache_cont + rm_state="$(rmhost --JSON || true)" + if has_tool jq; then + apache_cont="$(printf '%s' "$rm_state" | jq -r '.state.apache_delete // empty' 2>/dev/null || true)" + else + apache_cont="$(printf '%s' "$rm_state" | tr -d '\r\n' | sed -n 's/.*"apache_delete"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" + fi + [[ -n "$apache_cont" ]] && modify_profiles remove "$apache_cont" + + rmhost --RESET + cmd_reboot +} + +modify_profiles() { + local action=$1 + shift + local file=$ENV_DOCKER var=COMPOSE_PROFILES + local -a existing updated + + if [[ -r $file ]]; then + local line value + line=$(grep -E "^${var}=" "$file" | tail -n1 || true) + value=${line#*=} + IFS=',' read -r -a existing <<<"$value" + fi + + case $action in + add) + local p + for p; do + [[ -n $p && ! " ${existing[*]} " =~ " $p " ]] && updated+=("$p") + done + updated+=("${existing[@]}") + ;; + remove) + local old + for old in "${existing[@]}"; do + [[ ! " $* " =~ " $old " ]] && updated+=("$old") + done + ;; + *) die "modify_profiles: invalid action '$action'" ;; + esac + + update_env "$file" "$var" "$( + IFS=, + echo "${updated[*]}" + )" +} + From 218247095df7a542437d58c5c7311d59978be68c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:12:13 +0600 Subject: [PATCH 097/509] refactor: extract platform.sh module --- lib/platform.sh | 58 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 58 insertions(+) create mode 100644 lib/platform.sh diff --git a/lib/platform.sh b/lib/platform.sh new file mode 100644 index 00000000..9a6ebaf3 --- /dev/null +++ b/lib/platform.sh @@ -0,0 +1,58 @@ +# shellcheck shell=bash +############################################################################### +# 2. INSTALL / PERMISSIONS (HOST) +############################################################################### +add_to_windows_path() { + [[ "$OSTYPE" =~ (msys|cygwin) ]] || return 0 + has_cmd cygpath || return 0 + + # Only add if lds.bat exists where we think it is + [[ -f "$DIR/lds.bat" ]] || return 0 + + local win_repo + win_repo="$(cygpath -w "$DIR")" + + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " + \$t = '$win_repo' + \$cur = [Environment]::GetEnvironmentVariable('Path','User') + if ([string]::IsNullOrWhiteSpace(\$cur)) { \$cur = '' } + + # Normalize (trim + case-insensitive compare) to avoid duplicates + \$parts = \$cur -split ';' | ForEach-Object { \$_.Trim() } | Where-Object { \$_ } + \$exists = \$false + foreach (\$p in \$parts) { if (\$p.ToLowerInvariant() -eq \$t.ToLowerInvariant()) { \$exists = \$true; break } } + + if (-not \$exists) { + \$new = (\$parts + \$t) -join ';' + [Environment]::SetEnvironmentVariable('Path', \$new, 'User') + } + " >/dev/null 2>&1 || true +} + +fix_perms() { + if [[ "$OSTYPE" =~ (msys|cygwin) ]]; then + add_to_windows_path + printf "%bWindows PATH configured.%b\n" "$GREEN" "$NC" + return 0 + fi + + ((EUID == 0)) || die "Please run with sudo." + + chmod 755 "$DIR" + chmod 2775 "$DIR/configuration" + find "$DIR/configuration" -type f ! -perm 664 -exec chmod 664 {} + + + chmod 755 "$DIR/docker" + find "$DIR/docker" -type f ! -perm 644 -exec chmod 644 {} + + + chmod -R 777 "$DIR/logs" + chown -R "$USER:docker" "$DIR/logs" + + chmod 755 "$DIR/bin" + find "$DIR/bin" -type f -exec chmod +x {} + + chmod +x "$DIR/lds" + + ln -fs "$DIR/lds" /usr/local/bin/lds + printf "%bPermissions assigned.%b\n" "$GREEN" "$NC" +} + From 38847e39e0ca922230630c9d33b0240278be3bb9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:12:17 +0600 Subject: [PATCH 098/509] refactor: extract certificates.sh module --- lib/certificates.sh | 422 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 422 insertions(+) create mode 100644 lib/certificates.sh diff --git a/lib/certificates.sh b/lib/certificates.sh new file mode 100644 index 00000000..74d692f6 --- /dev/null +++ b/lib/certificates.sh @@ -0,0 +1,422 @@ +# shellcheck shell=bash +############################################################################### +# 5. ENVIRONMENT + CERT / CA +############################################################################### +detect_timezone() { + if has_cmd timedatectl; then + timedatectl show -p Timezone --value + elif [[ -n ${TZ-} ]]; then + printf '%s' "$TZ" + elif [[ -r /etc/timezone ]]; then + /dev/null | tr -d '\r' + else + date +%Z + fi +} + +env_init() { + local env_file="$ENV_DOCKER" + printf "%bBootstrapping environment defaults…%b\n" "$YELLOW" "$NC" + + local default_tz tz + default_tz="$(detect_timezone)" + tz="$(read_default "Timezone (TZ)" "$default_tz")" + + local default_git_name default_git_email git_name git_email + default_git_name="$(git config --global --get user.name 2>/dev/null || true)" + default_git_email="$(git config --global --get user.email 2>/dev/null || true)" + git_name="$(read_default "Git user.name (GIT_USER_NAME)" "$default_git_name")" + git_email="$(read_default "Git user.email (GIT_USER_EMAIL)" "$default_git_email")" + + # update_env now quotes automatically when needed + update_env "$env_file" "TZ" "$tz" + update_env "$env_file" "GIT_USER_NAME" "$git_name" + update_env "$env_file" "GIT_USER_EMAIL" "$git_email" + + printf "%bConfiguration saved!%b\n" "$GREEN" "$NC" +} + +# ───────────────────────────────────────────────────────────────────────────── +# Root CA helpers (cross-distro) +# ───────────────────────────────────────────────────────────────────────────── + +# Unique identity (avoid conflicts with other mkcert/dev CAs) +CA_BASENAME="localdevstack-rootca" +CA_NICK="LocalDevStack Root CA" + +detect_os_family() { + # Output: "id|like" + # Must never fail under set -e + if [[ "${OSTYPE:-}" =~ (msys|cygwin|win32) ]]; then + echo "windows|windows" + return 0 + fi + + local id like + id="unknown" + like="unknown" + + if [[ -r /etc/os-release ]]; then + # shellcheck disable=SC1091 + . /etc/os-release || true + id="${ID:-unknown}" + like="${ID_LIKE:-unknown}" + elif has_cmd uname; then + # fallback for macOS / other unix + case "$(uname -s 2>/dev/null || true)" in + Darwin) + id="macos" + like="darwin" + ;; + Linux) + id="linux" + like="linux" + ;; + esac + fi + + echo "$id|$like" +} + +# Decide destination path + update mechanism. Echo: family|dest|updater +ca_plan() { + local os_id os_like + IFS='|' read -r os_id os_like < <(detect_os_family) + + case " $os_id $os_like " in + *" debian "* | *" ubuntu "* | *" linuxmint "* | *" pop "* | *" raspbian "*) + printf "debian|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" + ;; + *" alpine "*) + printf "alpine|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" + ;; + *" fedora "* | *" rhel "* | *" redhat "* | *" centos "* | *" rocky "* | *" alma "* | *" amzn "* | *" amazon "* | *" sles "* | *" suse "*) + printf "rhel|/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt|update-ca-trust\n" + ;; + *" arch "* | *" manjaro "*) + printf "arch|/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt|trust\n" + ;; + *) + # best default: Debian-style location (works on many distros even if updater differs) + printf "fallback|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|\n" + ;; + esac +} + +is_windows_shell() { + [[ "${OSTYPE:-}" =~ (msys|cygwin) ]] || [[ -n "${WORKDIR_WIN:-}" ]] +} + +need_windows_tools() { + has_cmd cygpath || die "Windows certificate install needs 'cygpath' (Git Bash)." + has_cmd powershell.exe || die "Windows certificate install needs 'powershell.exe' on PATH." +} + +# Import CA into the invoking user's NSS DB (Chrome/Chromium/Firefox on many Linux setups) +install_ca_nss_user() { + local ca_file="$1" + has_cmd certutil || return 0 + + local user="${SUDO_USER:-}" + [[ -n "$user" && "$user" != "root" ]] || return 0 + + local home + home="$(getent passwd "$user" | cut -d: -f6)" + [[ -n "$home" && -d "$home" ]] || return 0 + + local nssdb="sql:${home}/.pki/nssdb" + sudo -u "$user" mkdir -p "${home}/.pki/nssdb" >/dev/null 2>&1 || true + + if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then + printf "%b✔ NSS already has CA%b (%s)\n" "$GREEN" "$NC" "$user" + return 0 + fi + + if sudo -u "$user" certutil -d "$nssdb" -A -n "$CA_NICK" -t "C,," -i "$ca_file" >/dev/null 2>&1; then + printf "%b✔ Imported CA into NSS%b (%s)\n" "$GREEN" "$NC" "$user" + else + printf "%bWARN%b: NSS import failed (certutil).\n" "$YELLOW" "$NC" >&2 + fi +} + +install_ca_windows() { + need_windows_tools + + local src_ca="$DIR/configuration/rootCA/rootCA.pem" + [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + + local win_ca + win_ca="$(cygpath -w "$src_ca")" + + printf "%bInstalling root CA into Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" + + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " + \$ErrorActionPreference = 'Stop' + \$path = '$win_ca' + \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) + \$cert.FriendlyName = '$CA_NICK' + + \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') + \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) + + \$exists = \$store.Certificates | Where-Object { \$_.Thumbprint -eq \$cert.Thumbprint } + if (-not \$exists) { \$store.Add(\$cert) } + + \$store.Close() + " >/dev/null 2>&1 || die "Windows certificate install failed (PowerShell import)." + + printf "%bRoot CA installed on Windows%b (CurrentUser\\Root) as %s\n" "$GREEN" "$NC" "$CA_NICK" + printf "%bNote:%b restart browsers if they still show trust errors.\n" "$YELLOW" "$NC" +} + +install_ca() { + if is_windows_shell; then + install_ca_windows + return 0 + fi + + local src_ca="$DIR/configuration/rootCA/rootCA.pem" + [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate install requires sudo" + [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + + local family dest updater os_id os_like + IFS='|' read -r os_id os_like < <(detect_os_family) + IFS='|' read -r family dest updater < <(ca_plan) + + printf "%bInstalling root CA…%b\n" "$CYAN" "$NC" + printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" + + install -d -m 755 "$(dirname "$dest")" + install -m 644 "$src_ca" "$dest" + printf "%b✔ Copied%b → %s\n" "$GREEN" "$NC" "$dest" + + case "$family" in + debian | alpine) + if has_cmd update-ca-certificates; then + printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" + if update-ca-certificates; then + printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" + printf "%bNote:%b If you see \"rehash: skipping ca-certificates.crt…\", that’s normal (it’s a bundle).\n" "$YELLOW" "$NC" + else + printf "%bWARN%b: update-ca-certificates failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bWARN%b: update-ca-certificates not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 + fi + + # Optional p11-kit sync: best-effort only (can be missing helper on minimal installs) + if has_cmd trust; then + printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" + if trust extract-compat >/dev/null 2>&1; then + printf "%b✔ p11-kit trust synced%b\n" "$GREEN" "$NC" + else + printf "%bWARN%b: trust extract-compat failed (helper missing on some installs). Skipping.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bINFO%b: 'trust' not found — skipping p11-kit sync.\n" "$YELLOW" "$NC" + fi + ;; + rhel) + if has_cmd update-ca-trust; then + printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" + if update-ca-trust extract; then + printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" + else + printf "%bWARN%b: update-ca-trust extract failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bWARN%b: update-ca-trust not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 + fi + ;; + arch) + if has_cmd trust; then + printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" + if trust extract-compat >/dev/null 2>&1; then + printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" + else + printf "%bWARN%b: trust extract-compat failed. CA is installed, but trust sync may be incomplete.\n" "$YELLOW" "$NC" >&2 + fi + else + printf "%bWARN%b: 'trust' not found. CA is installed, but trust sync is unavailable.\n" "$YELLOW" "$NC" >&2 + fi + ;; + *) + printf "%bINFO%b: Unknown distro; CA copied to %s.\n" "$YELLOW" "$NC" "$dest" + printf "%bINFO%b: You may need to update trust store manually for your OS.\n" "$YELLOW" "$NC" + ;; + esac + + # Extra: ensure browsers that rely on NSS trust pick it up + install_ca_nss_user "$src_ca" + + printf "%bRoot CA installed%b → %s (%s)\n" "$GREEN" "$NC" "$dest" "$CA_NICK" +} + +uninstall_ca_windows() { + need_windows_tools + + local src_ca="$DIR/configuration/rootCA/rootCA.pem" + [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + + local win_ca + win_ca="$(cygpath -w "$src_ca")" + + printf "%bUninstalling root CA from Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" + + local removed + removed="$(powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " + \$ErrorActionPreference = 'Stop' + \$path = '$win_ca' + \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) + \$thumb = \$cert.Thumbprint + + \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') + \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) + + \$matches = @(\$store.Certificates | Where-Object { \$_.Thumbprint -eq \$thumb }) + foreach (\$c in \$matches) { \$store.Remove(\$c) } + + \$store.Close() + [string]\$matches.Count + " 2>/dev/null || true)" + + removed="${removed//[$'\r\n\t ']/}" + if [[ "${removed:-0}" =~ ^[0-9]+$ ]] && ((removed > 0)); then + printf "%bRoot CA uninstalled on Windows%b (removed %s cert)\n" "$GREEN" "$NC" "$removed" + else + printf "%bRoot CA already absent on Windows%b (no matching cert)\n" "$YELLOW" "$NC" + fi +} + +uninstall_ca_nss_user() { + has_cmd certutil || return 0 + local user="${SUDO_USER:-}" + [[ -n "$user" && "$user" != "root" ]] || return 0 + + local home + home="$(getent passwd "$user" | cut -d: -f6)" + [[ -n "$home" && -d "$home" ]] || return 0 + + local nssdb="sql:${home}/.pki/nssdb" + if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then + sudo -u "$user" certutil -d "$nssdb" -D -n "$CA_NICK" >/dev/null 2>&1 || true + printf "%b✔ Removed CA from NSS%b (%s)\n" "$GREEN" "$NC" "$user" + fi +} + +uninstall_ca() { + if is_windows_shell; then + uninstall_ca_windows + return 0 + fi + + [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate uninstall requires sudo" + + local all=0 + if [[ "${1:-}" == "--all" ]]; then + all=1 + shift + fi + + local family dest updater os_id os_like + IFS='|' read -r os_id os_like < <(detect_os_family) + IFS='|' read -r family dest updater < <(ca_plan) + + printf "%bUninstalling root CA…%b\n" "$CYAN" "$NC" + printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" + + local removed=0 + + if [[ -e "$dest" ]]; then + rm -f "$dest" + removed=$((removed + 1)) + printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$dest" + else + printf "%bINFO%b: CA file not found at %s (nothing to remove)\n" "$YELLOW" "$NC" "$dest" + fi + + if ((all)); then + printf "%bScanning all known CA anchor paths…%b\n" "$CYAN" "$NC" + local f + for f in \ + "/usr/local/share/ca-certificates/${CA_BASENAME}.crt" \ + "/usr/local/share/ca-certificates/${CA_BASENAME}.pem" \ + "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt" \ + "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.pem" \ + "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt" \ + "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.pem"; do + [[ "$f" == "$dest" ]] && continue + if [[ -e "$f" ]]; then + rm -f "$f" + removed=$((removed + 1)) + printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$f" + fi + done + fi + + case "$family" in + debian | alpine) + if has_cmd update-ca-certificates; then + printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" + update-ca-certificates || printf "%bWARN%b: update-ca-certificates failed.\n" "$YELLOW" "$NC" >&2 + else + printf "%bWARN%b: update-ca-certificates not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 + fi + + if has_cmd trust; then + printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" + trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed. Skipping.\n" "$YELLOW" "$NC" >&2 + fi + ;; + rhel) + if has_cmd update-ca-trust; then + printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" + update-ca-trust extract || printf "%bWARN%b: update-ca-trust extract failed.\n" "$YELLOW" "$NC" >&2 + else + printf "%bWARN%b: update-ca-trust not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 + fi + ;; + arch) + if has_cmd trust; then + printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" + trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed.\n" "$YELLOW" "$NC" >&2 + else + printf "%bWARN%b: 'trust' not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 + fi + ;; + *) + if has_cmd update-ca-certificates; then + printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" + update-ca-certificates || true + fi + if has_cmd update-ca-trust; then + printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" + update-ca-trust extract || true + fi + if has_cmd trust; then + printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" + trust extract-compat >/dev/null 2>&1 || true + fi + printf "%bINFO%b: Unknown distro; removed CA file(s) if present. Refresh trust store manually if needed.\n" "$YELLOW" "$NC" + ;; + esac + + uninstall_ca_nss_user + + if ((removed)); then + printf "%bRoot CA uninstalled%b (removed %d file(s))\n" "$GREEN" "$NC" "$removed" + else + printf "%bRoot CA already absent%b (no files removed)\n" "$YELLOW" "$NC" + fi +} + +add_required_env() { + update_env "$ENV_DOCKER" WORKING_DIR "$DIR" + ((EUID == 0)) && return 0 + update_env "$ENV_DOCKER" USER "$(id -un)" + update_env "$ENV_DOCKER" UID "$(id -u)" + update_env "$ENV_DOCKER" GID "$(id -g)" +} + From 510e40233539cf3cad95f0b94f428808d5e62a19 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:12:20 +0600 Subject: [PATCH 099/509] refactor: source host platform and certificate modules --- lds | 574 +----------------------------------------------------------- 1 file changed, 7 insertions(+), 567 deletions(-) diff --git a/lds b/lds index 039b5af9..ef866311 100755 --- a/lds +++ b/lds @@ -441,164 +441,19 @@ source "$DIR/lib/env.sh" ############################################################################### # 1c. HTTP / WEB SERVER HELPERS ############################################################################### - -http_reload() { - printf "%bReloading HTTP...%b" "$MAGENTA" "$NC" - docker ps -qf name=NGINX &>/dev/null && docker exec NGINX nginx -s reload &>/dev/null || true - docker ps -qf name=APACHE &>/dev/null && docker exec APACHE apachectl graceful &>/dev/null || true - printf "\r%bHTTP reloaded! %b\n" "$GREEN" "$NC" -} +# shellcheck source=lib/hosts.sh +source "$DIR/lib/hosts.sh" ############################################################################### # 2. INSTALL / PERMISSIONS (HOST) ############################################################################### -add_to_windows_path() { - [[ "$OSTYPE" =~ (msys|cygwin) ]] || return 0 - has_cmd cygpath || return 0 - - # Only add if lds.bat exists where we think it is - [[ -f "$DIR/lds.bat" ]] || return 0 - - local win_repo - win_repo="$(cygpath -w "$DIR")" - - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " - \$t = '$win_repo' - \$cur = [Environment]::GetEnvironmentVariable('Path','User') - if ([string]::IsNullOrWhiteSpace(\$cur)) { \$cur = '' } - - # Normalize (trim + case-insensitive compare) to avoid duplicates - \$parts = \$cur -split ';' | ForEach-Object { \$_.Trim() } | Where-Object { \$_ } - \$exists = \$false - foreach (\$p in \$parts) { if (\$p.ToLowerInvariant() -eq \$t.ToLowerInvariant()) { \$exists = \$true; break } } - - if (-not \$exists) { - \$new = (\$parts + \$t) -join ';' - [Environment]::SetEnvironmentVariable('Path', \$new, 'User') - } - " >/dev/null 2>&1 || true -} - -fix_perms() { - if [[ "$OSTYPE" =~ (msys|cygwin) ]]; then - add_to_windows_path - printf "%bWindows PATH configured.%b\n" "$GREEN" "$NC" - return 0 - fi - - ((EUID == 0)) || die "Please run with sudo." - - chmod 755 "$DIR" - chmod 2775 "$DIR/configuration" - find "$DIR/configuration" -type f ! -perm 664 -exec chmod 664 {} + - - chmod 755 "$DIR/docker" - find "$DIR/docker" -type f ! -perm 644 -exec chmod 644 {} + - - chmod -R 777 "$DIR/logs" - chown -R "$USER:docker" "$DIR/logs" - - chmod 755 "$DIR/bin" - find "$DIR/bin" -type f -exec chmod +x {} + - chmod +x "$DIR/lds" - - ln -fs "$DIR/lds" /usr/local/bin/lds - printf "%bPermissions assigned.%b\n" "$GREEN" "$NC" -} +# shellcheck source=lib/platform.sh +source "$DIR/lib/platform.sh" ############################################################################### # 3. DOMAIN / PROFILE INTEGRATION ############################################################################### -mkhost() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" mkhost "$@" -} -rmhost() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" rmhost "$@" -} - -setup_domain() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - mkhost --RESET - docker exec -it "$ctr" mkhost - local mk_state svr_prof - mk_state="$(mkhost --JSON || true)" - if has_tool jq; then - svr_prof="$(printf '%s' "$mk_state" | jq -r '.state.apache_active // empty' 2>/dev/null || true)" - else - svr_prof="$(printf '%s' "$mk_state" | tr -d '\r\n' | sed -n 's/.*"apache_active"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" - fi - [[ -n $svr_prof ]] && modify_profiles add "$svr_prof" - mkhost --RESET - cmd_reboot -} - -delete_domain() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - rmhost --RESET - - # interactive delete - docker exec -it "$ctr" rmhost "$@" - - local rm_state apache_cont - rm_state="$(rmhost --JSON || true)" - if has_tool jq; then - apache_cont="$(printf '%s' "$rm_state" | jq -r '.state.apache_delete // empty' 2>/dev/null || true)" - else - apache_cont="$(printf '%s' "$rm_state" | tr -d '\r\n' | sed -n 's/.*"apache_delete"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p')" - fi - [[ -n "$apache_cont" ]] && modify_profiles remove "$apache_cont" - - rmhost --RESET - cmd_reboot -} - -modify_profiles() { - local action=$1 - shift - local file=$ENV_DOCKER var=COMPOSE_PROFILES - local -a existing updated - - if [[ -r $file ]]; then - local line value - line=$(grep -E "^${var}=" "$file" | tail -n1 || true) - value=${line#*=} - IFS=',' read -r -a existing <<<"$value" - fi - - case $action in - add) - local p - for p; do - [[ -n $p && ! " ${existing[*]} " =~ " $p " ]] && updated+=("$p") - done - updated+=("${existing[@]}") - ;; - remove) - local old - for old in "${existing[@]}"; do - [[ ! " $* " =~ " $old " ]] && updated+=("$old") - done - ;; - *) die "modify_profiles: invalid action '$action'" ;; - esac - - update_env "$file" "$var" "$( - IFS=, - echo "${updated[*]}" - )" -} +# Host/domain functions are loaded by lib/hosts.sh above. # ───────────────────────────────────────────────────────────────────────────── # Profiles @@ -613,423 +468,8 @@ source "$DIR/lib/profiles.sh" ############################################################################### # 5. ENVIRONMENT + CERT / CA ############################################################################### -detect_timezone() { - if has_cmd timedatectl; then - timedatectl show -p Timezone --value - elif [[ -n ${TZ-} ]]; then - printf '%s' "$TZ" - elif [[ -r /etc/timezone ]]; then - /dev/null | tr -d '\r' - else - date +%Z - fi -} - -env_init() { - local env_file="$ENV_DOCKER" - printf "%bBootstrapping environment defaults…%b\n" "$YELLOW" "$NC" - - local default_tz tz - default_tz="$(detect_timezone)" - tz="$(read_default "Timezone (TZ)" "$default_tz")" - - local default_git_name default_git_email git_name git_email - default_git_name="$(git config --global --get user.name 2>/dev/null || true)" - default_git_email="$(git config --global --get user.email 2>/dev/null || true)" - git_name="$(read_default "Git user.name (GIT_USER_NAME)" "$default_git_name")" - git_email="$(read_default "Git user.email (GIT_USER_EMAIL)" "$default_git_email")" - - # update_env now quotes automatically when needed - update_env "$env_file" "TZ" "$tz" - update_env "$env_file" "GIT_USER_NAME" "$git_name" - update_env "$env_file" "GIT_USER_EMAIL" "$git_email" - - printf "%bConfiguration saved!%b\n" "$GREEN" "$NC" -} - -# ───────────────────────────────────────────────────────────────────────────── -# Root CA helpers (cross-distro) -# ───────────────────────────────────────────────────────────────────────────── - -# Unique identity (avoid conflicts with other mkcert/dev CAs) -CA_BASENAME="localdevstack-rootca" -CA_NICK="LocalDevStack Root CA" - -detect_os_family() { - # Output: "id|like" - # Must never fail under set -e - if [[ "${OSTYPE:-}" =~ (msys|cygwin|win32) ]]; then - echo "windows|windows" - return 0 - fi - - local id like - id="unknown" - like="unknown" - - if [[ -r /etc/os-release ]]; then - # shellcheck disable=SC1091 - . /etc/os-release || true - id="${ID:-unknown}" - like="${ID_LIKE:-unknown}" - elif has_cmd uname; then - # fallback for macOS / other unix - case "$(uname -s 2>/dev/null || true)" in - Darwin) - id="macos" - like="darwin" - ;; - Linux) - id="linux" - like="linux" - ;; - esac - fi - - echo "$id|$like" -} - -# Decide destination path + update mechanism. Echo: family|dest|updater -ca_plan() { - local os_id os_like - IFS='|' read -r os_id os_like < <(detect_os_family) - - case " $os_id $os_like " in - *" debian "* | *" ubuntu "* | *" linuxmint "* | *" pop "* | *" raspbian "*) - printf "debian|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" - ;; - *" alpine "*) - printf "alpine|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|update-ca-certificates\n" - ;; - *" fedora "* | *" rhel "* | *" redhat "* | *" centos "* | *" rocky "* | *" alma "* | *" amzn "* | *" amazon "* | *" sles "* | *" suse "*) - printf "rhel|/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt|update-ca-trust\n" - ;; - *" arch "* | *" manjaro "*) - printf "arch|/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt|trust\n" - ;; - *) - # best default: Debian-style location (works on many distros even if updater differs) - printf "fallback|/usr/local/share/ca-certificates/${CA_BASENAME}.crt|\n" - ;; - esac -} - -is_windows_shell() { - [[ "${OSTYPE:-}" =~ (msys|cygwin) ]] || [[ -n "${WORKDIR_WIN:-}" ]] -} - -need_windows_tools() { - has_cmd cygpath || die "Windows certificate install needs 'cygpath' (Git Bash)." - has_cmd powershell.exe || die "Windows certificate install needs 'powershell.exe' on PATH." -} - -# Import CA into the invoking user's NSS DB (Chrome/Chromium/Firefox on many Linux setups) -install_ca_nss_user() { - local ca_file="$1" - has_cmd certutil || return 0 - - local user="${SUDO_USER:-}" - [[ -n "$user" && "$user" != "root" ]] || return 0 - - local home - home="$(getent passwd "$user" | cut -d: -f6)" - [[ -n "$home" && -d "$home" ]] || return 0 - - local nssdb="sql:${home}/.pki/nssdb" - sudo -u "$user" mkdir -p "${home}/.pki/nssdb" >/dev/null 2>&1 || true - - if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then - printf "%b✔ NSS already has CA%b (%s)\n" "$GREEN" "$NC" "$user" - return 0 - fi - - if sudo -u "$user" certutil -d "$nssdb" -A -n "$CA_NICK" -t "C,," -i "$ca_file" >/dev/null 2>&1; then - printf "%b✔ Imported CA into NSS%b (%s)\n" "$GREEN" "$NC" "$user" - else - printf "%bWARN%b: NSS import failed (certutil).\n" "$YELLOW" "$NC" >&2 - fi -} - -install_ca_windows() { - need_windows_tools - - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" - - local win_ca - win_ca="$(cygpath -w "$src_ca")" - - printf "%bInstalling root CA into Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" - - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " - \$ErrorActionPreference = 'Stop' - \$path = '$win_ca' - \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) - \$cert.FriendlyName = '$CA_NICK' - - \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') - \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) - - \$exists = \$store.Certificates | Where-Object { \$_.Thumbprint -eq \$cert.Thumbprint } - if (-not \$exists) { \$store.Add(\$cert) } - - \$store.Close() - " >/dev/null 2>&1 || die "Windows certificate install failed (PowerShell import)." - - printf "%bRoot CA installed on Windows%b (CurrentUser\\Root) as %s\n" "$GREEN" "$NC" "$CA_NICK" - printf "%bNote:%b restart browsers if they still show trust errors.\n" "$YELLOW" "$NC" -} - -install_ca() { - if is_windows_shell; then - install_ca_windows - return 0 - fi - - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate install requires sudo" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" - - local family dest updater os_id os_like - IFS='|' read -r os_id os_like < <(detect_os_family) - IFS='|' read -r family dest updater < <(ca_plan) - - printf "%bInstalling root CA…%b\n" "$CYAN" "$NC" - printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" - - install -d -m 755 "$(dirname "$dest")" - install -m 644 "$src_ca" "$dest" - printf "%b✔ Copied%b → %s\n" "$GREEN" "$NC" "$dest" - - case "$family" in - debian | alpine) - if has_cmd update-ca-certificates; then - printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" - if update-ca-certificates; then - printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" - printf "%bNote:%b If you see \"rehash: skipping ca-certificates.crt…\", that’s normal (it’s a bundle).\n" "$YELLOW" "$NC" - else - printf "%bWARN%b: update-ca-certificates failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bWARN%b: update-ca-certificates not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 - fi - - # Optional p11-kit sync: best-effort only (can be missing helper on minimal installs) - if has_cmd trust; then - printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" - if trust extract-compat >/dev/null 2>&1; then - printf "%b✔ p11-kit trust synced%b\n" "$GREEN" "$NC" - else - printf "%bWARN%b: trust extract-compat failed (helper missing on some installs). Skipping.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bINFO%b: 'trust' not found — skipping p11-kit sync.\n" "$YELLOW" "$NC" - fi - ;; - rhel) - if has_cmd update-ca-trust; then - printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" - if update-ca-trust extract; then - printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" - else - printf "%bWARN%b: update-ca-trust extract failed. CA is installed but may not be active yet.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bWARN%b: update-ca-trust not found. CA is installed but auto-update is unavailable.\n" "$YELLOW" "$NC" >&2 - fi - ;; - arch) - if has_cmd trust; then - printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" - if trust extract-compat >/dev/null 2>&1; then - printf "%b✔ Trust store updated%b\n" "$GREEN" "$NC" - else - printf "%bWARN%b: trust extract-compat failed. CA is installed, but trust sync may be incomplete.\n" "$YELLOW" "$NC" >&2 - fi - else - printf "%bWARN%b: 'trust' not found. CA is installed, but trust sync is unavailable.\n" "$YELLOW" "$NC" >&2 - fi - ;; - *) - printf "%bINFO%b: Unknown distro; CA copied to %s.\n" "$YELLOW" "$NC" "$dest" - printf "%bINFO%b: You may need to update trust store manually for your OS.\n" "$YELLOW" "$NC" - ;; - esac - - # Extra: ensure browsers that rely on NSS trust pick it up - install_ca_nss_user "$src_ca" - - printf "%bRoot CA installed%b → %s (%s)\n" "$GREEN" "$NC" "$dest" "$CA_NICK" -} - -uninstall_ca_windows() { - need_windows_tools - - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" - - local win_ca - win_ca="$(cygpath -w "$src_ca")" - - printf "%bUninstalling root CA from Windows trust store (CurrentUser\\Root)…%b\n" "$CYAN" "$NC" - - local removed - removed="$(powershell.exe -NoProfile -ExecutionPolicy Bypass -Command " - \$ErrorActionPreference = 'Stop' - \$path = '$win_ca' - \$cert = New-Object System.Security.Cryptography.X509Certificates.X509Certificate2(\$path) - \$thumb = \$cert.Thumbprint - - \$store = New-Object System.Security.Cryptography.X509Certificates.X509Store('Root','CurrentUser') - \$store.Open([System.Security.Cryptography.X509Certificates.OpenFlags]::ReadWrite) - - \$matches = @(\$store.Certificates | Where-Object { \$_.Thumbprint -eq \$thumb }) - foreach (\$c in \$matches) { \$store.Remove(\$c) } - - \$store.Close() - [string]\$matches.Count - " 2>/dev/null || true)" - - removed="${removed//[$'\r\n\t ']/}" - if [[ "${removed:-0}" =~ ^[0-9]+$ ]] && ((removed > 0)); then - printf "%bRoot CA uninstalled on Windows%b (removed %s cert)\n" "$GREEN" "$NC" "$removed" - else - printf "%bRoot CA already absent on Windows%b (no matching cert)\n" "$YELLOW" "$NC" - fi -} - -uninstall_ca_nss_user() { - has_cmd certutil || return 0 - local user="${SUDO_USER:-}" - [[ -n "$user" && "$user" != "root" ]] || return 0 - - local home - home="$(getent passwd "$user" | cut -d: -f6)" - [[ -n "$home" && -d "$home" ]] || return 0 - - local nssdb="sql:${home}/.pki/nssdb" - if sudo -u "$user" certutil -d "$nssdb" -L 2>/dev/null | grep -Fq "$CA_NICK"; then - sudo -u "$user" certutil -d "$nssdb" -D -n "$CA_NICK" >/dev/null 2>&1 || true - printf "%b✔ Removed CA from NSS%b (%s)\n" "$GREEN" "$NC" "$user" - fi -} - -uninstall_ca() { - if is_windows_shell; then - uninstall_ca_windows - return 0 - fi - - [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate uninstall requires sudo" - - local all=0 - if [[ "${1:-}" == "--all" ]]; then - all=1 - shift - fi - - local family dest updater os_id os_like - IFS='|' read -r os_id os_like < <(detect_os_family) - IFS='|' read -r family dest updater < <(ca_plan) - - printf "%bUninstalling root CA…%b\n" "$CYAN" "$NC" - printf "%bDetected OS%b: id=%s like=%s → %s\n" "$CYAN" "$NC" "$os_id" "$os_like" "$family" - - local removed=0 - - if [[ -e "$dest" ]]; then - rm -f "$dest" - removed=$((removed + 1)) - printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$dest" - else - printf "%bINFO%b: CA file not found at %s (nothing to remove)\n" "$YELLOW" "$NC" "$dest" - fi - - if ((all)); then - printf "%bScanning all known CA anchor paths…%b\n" "$CYAN" "$NC" - local f - for f in \ - "/usr/local/share/ca-certificates/${CA_BASENAME}.crt" \ - "/usr/local/share/ca-certificates/${CA_BASENAME}.pem" \ - "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.crt" \ - "/etc/pki/ca-trust/source/anchors/${CA_BASENAME}.pem" \ - "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.crt" \ - "/etc/ca-certificates/trust-source/anchors/${CA_BASENAME}.pem"; do - [[ "$f" == "$dest" ]] && continue - if [[ -e "$f" ]]; then - rm -f "$f" - removed=$((removed + 1)) - printf "%b✔ Removed%b → %s\n" "$GREEN" "$NC" "$f" - fi - done - fi - - case "$family" in - debian | alpine) - if has_cmd update-ca-certificates; then - printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" - update-ca-certificates || printf "%bWARN%b: update-ca-certificates failed.\n" "$YELLOW" "$NC" >&2 - else - printf "%bWARN%b: update-ca-certificates not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 - fi - - if has_cmd trust; then - printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" - trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed. Skipping.\n" "$YELLOW" "$NC" >&2 - fi - ;; - rhel) - if has_cmd update-ca-trust; then - printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" - update-ca-trust extract || printf "%bWARN%b: update-ca-trust extract failed.\n" "$YELLOW" "$NC" >&2 - else - printf "%bWARN%b: update-ca-trust not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 - fi - ;; - arch) - if has_cmd trust; then - printf "%bUpdating trust store%b (trust extract-compat)…\n" "$CYAN" "$NC" - trust extract-compat >/dev/null 2>&1 || printf "%bWARN%b: trust extract-compat failed.\n" "$YELLOW" "$NC" >&2 - else - printf "%bWARN%b: 'trust' not found; trust store not refreshed.\n" "$YELLOW" "$NC" >&2 - fi - ;; - *) - if has_cmd update-ca-certificates; then - printf "%bUpdating trust store%b (update-ca-certificates)…\n" "$CYAN" "$NC" - update-ca-certificates || true - fi - if has_cmd update-ca-trust; then - printf "%bUpdating trust store%b (update-ca-trust extract)…\n" "$CYAN" "$NC" - update-ca-trust extract || true - fi - if has_cmd trust; then - printf "%bSyncing p11-kit%b (trust extract-compat)…\n" "$CYAN" "$NC" - trust extract-compat >/dev/null 2>&1 || true - fi - printf "%bINFO%b: Unknown distro; removed CA file(s) if present. Refresh trust store manually if needed.\n" "$YELLOW" "$NC" - ;; - esac - - uninstall_ca_nss_user - - if ((removed)); then - printf "%bRoot CA uninstalled%b (removed %d file(s))\n" "$GREEN" "$NC" "$removed" - else - printf "%bRoot CA already absent%b (no files removed)\n" "$YELLOW" "$NC" - fi -} - -add_required_env() { - update_env "$ENV_DOCKER" WORKING_DIR "$DIR" - ((EUID == 0)) && return 0 - update_env "$ENV_DOCKER" USER "$(id -un)" - update_env "$ENV_DOCKER" UID "$(id -u)" - update_env "$ENV_DOCKER" GID "$(id -g)" -} +# shellcheck source=lib/certificates.sh +source "$DIR/lib/certificates.sh" ############################################################################### # Compose helpers for rebuild (robust: supports service key OR container name) From 48ba9af5ffbebf70ff6e0f2b6f909d0460eb2106 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:12:58 +0600 Subject: [PATCH 100/509] refactor: extract services.sh module --- lib/services.sh | 1048 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 1048 insertions(+) create mode 100644 lib/services.sh diff --git a/lib/services.sh b/lib/services.sh new file mode 100644 index 00000000..8d3155f1 --- /dev/null +++ b/lib/services.sh @@ -0,0 +1,1048 @@ +# shellcheck shell=bash +############################################################################### +# Compose helpers for rebuild (robust: supports service key OR container name) +############################################################################### +__COMPOSE_CFG_JSON="" +__COMPOSE_CFG_YAML="" +__COMPOSE_SVCS_LOADED=0 +declare -a __COMPOSE_SVCS=() + +compose_cfg_json() { + if [[ -z "${__COMPOSE_CFG_JSON}" ]]; then + __COMPOSE_CFG_JSON="$(docker_compose config --format json 2>/dev/null || true)" + fi + printf '%s' "${__COMPOSE_CFG_JSON}" +} + +compose_cfg_yaml() { + if [[ -z "${__COMPOSE_CFG_YAML}" ]]; then + __COMPOSE_CFG_YAML="$(docker_compose config 2>/dev/null || true)" + fi + printf '%s' "${__COMPOSE_CFG_YAML}" +} + +compose_services_load() { + ((__COMPOSE_SVCS_LOADED)) && return 0 + mapfile -t __COMPOSE_SVCS < <(docker_compose config --services 2>/dev/null || true) + __COMPOSE_SVCS_LOADED=1 +} + +compose_service_exists() { + local want="${1:-}" s + [[ -n "$want" ]] || return 1 + compose_services_load + for s in "${__COMPOSE_SVCS[@]}"; do + [[ "$s" == "$want" ]] && return 0 + done + return 1 +} + +resolve_service() { + local raw="${1:-}" norm svc + raw="${raw//[[:space:]]/}" + [[ -n "$raw" ]] || { + printf '' + return 0 + } + + compose_service_exists "$raw" && { + printf '%s' "$raw" + return 0 + } + + norm="$(normalize_service "$raw")" + compose_service_exists "$norm" && { + printf '%s' "$norm" + return 0 + } + + if docker inspect "$raw" >/dev/null 2>&1; then + svc="$(docker inspect -f '{{ index .Config.Labels "com.docker.compose.service" }}' "$raw" 2>/dev/null || true)" + if [[ -n "$svc" ]] && compose_service_exists "$svc"; then + printf '%s' "$svc" + return 0 + fi + fi + + printf '%s' "$norm" +} + +compose_has_build() { + local svc="$1" json + json="$(compose_cfg_json)" + if [[ -n "$json" ]]; then + if has_tool jq; then + jq -e --arg s "$svc" '.services[$s].build != null' >/dev/null <<<"$json" + return $? + fi + fi + + compose_cfg_yaml | awk -v s="$svc" ' + $1=="services:" {in_services=1; next} + in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} + in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 1} + in_svc && $0 ~ /^ build:/ {exit 0} + END {exit 1} + ' +} + +compose_image_for_service() { + local svc="$1" json + json="$(compose_cfg_json)" + if [[ -n "$json" ]]; then + if has_tool jq; then + jq -r --arg s "$svc" '.services[$s].image // empty' <<<"$json" + return 0 + fi + fi + + compose_cfg_yaml | awk -v s="$svc" ' + $1=="services:" {in_services=1; next} + in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} + in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 0} + in_svc && $0 ~ /^ image:/ { + sub(/^ image:[[:space:]]*/, "", $0) + print $0 + exit 0 + } + ' +} + +############################################################################### +# 6. STACK COMMANDS (CLI) +############################################################################### + +# One-time migration from the historical fixed 172.28/29/30 /24 networks. +# Only known LocalDevStack-owned networks are touched, and named volumes are +# never removed. New networks carry com.infocyph.network-schema=dynamic-v1. +declare -a __LDS_LEGACY_NETWORK_NAMES=(Frontend Backend DataStore) + +legacy_network_expected_subnet() { + case "\${1:-}" in + Frontend) printf '%s' '172.28.0.0/24' ;; + Backend) printf '%s' '172.29.0.0/24' ;; + DataStore) printf '%s' '172.30.0.0/24' ;; + *) return 1 ;; + esac +} + +migrate_legacy_networks() { + local network expected subnets stack_label project_label schema_label ctr ctr_project attachments + local -a legacy=() + + for network in "\${__LDS_LEGACY_NETWORK_NAMES[@]}"; do + docker network inspect "$network" >/dev/null 2>&1 || continue + + schema_label="$(docker network inspect -f '{{index .Labels "com.infocyph.network-schema"}}' "$network" 2>/dev/null || true)" + [[ "$schema_label" != "dynamic-v1" ]] || continue + + expected="$(legacy_network_expected_subnet "$network")" + subnets="$(docker network inspect -f '{{range .IPAM.Config}}{{println .Subnet}}{{end}}' "$network" 2>/dev/null || true)" + grep -Fxq "$expected" <<<"$subnets" || continue + + stack_label="$(docker network inspect -f '{{index .Labels "com.infocyph.stack"}}' "$network" 2>/dev/null || true)" + project_label="$(docker network inspect -f '{{index .Labels "com.docker.compose.project"}}' "$network" 2>/dev/null || true)" + if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "LocalDevStack" ]]; then + die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack. Remove or rename that network manually." + fi + + while IFS= read -r ctr; do + [[ -n "$ctr" ]] || continue + ctr_project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$ctr" 2>/dev/null || true)" + if [[ "$ctr_project" != "LocalDevStack" ]]; then + die "Refusing to migrate '$network': container '$ctr' is not owned by the LocalDevStack Compose project." + fi + done < <(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true) + + legacy+=("$network") + done + + (("\${#legacy[@]}" > 0)) || return 0 + + warn "Legacy fixed LocalDevStack network(s) detected: \${legacy[*]}" + warn "Recreating stack networks dynamically; named volumes and persisted data are preserved." + + # Stop/remove only LocalDevStack Compose containers and networks. Never use -v. + docker_compose down --remove-orphans + + for network in "\${legacy[@]}"; do + docker network inspect "$network" >/dev/null 2>&1 || continue + attachments="$(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true)" + [[ -z "$attachments" ]] || + die "Cannot remove legacy network '$network': attached container(s) remain: $(tr '\n' ' ' <<<"$attachments")" + docker network rm "$network" >/dev/null + done + + ok "Legacy fixed networks removed; Compose will recreate dynamic bridge networks." +} + +cmd_vpn_fix() { + warn "vpn-fix is deprecated: LocalDevStack no longer owns fixed Docker subnets." + warn "If a VPN conflict remains after dynamic-network migration, diagnose the VPN/Docker route directly." +} + +cmd_up() { + migrate_legacy_networks + dc_up "$@" +} + +cmd_start() { + migrate_legacy_networks + dc_up -d "$@" + http_reload +} + +cmd_stop() { docker_compose down; } + +cmd_down() { + # Safety rails: + # lds down --volumes requires --yes + local yes=0 vols=0 + local -a args=() + while [[ "${1:-}" ]]; do + case "$1" in + --yes | -y) + yes=1 + shift + ;; + --volumes | -v) + vols=1 + args+=("--volumes") + shift + ;; + --remove-orphans) + args+=("--remove-orphans") + shift + ;; + *) + args+=("$1") + shift + ;; + esac + done + if ((vols)) && ((yes == 0)); then + die "Refusing: down --volumes requires --yes" + fi + docker_compose down "${args[@]}" +} + +cmd_restart() { + cmd_stop + cmd_start +} +cmd_reboot() { cmd_restart; } + +# ───────────────────────────────────────────────────────────────────────────── +# 6a. STATUS / PS / STATS +# ───────────────────────────────────────────────────────────────────────────── +cmd_ps() { + if (($#)); then + docker_compose ps "$@" + else + docker_compose ps + fi +} + +cmd_status() { + local ctr project + project="$(lds_project)" + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container not found or not running for project: $project" + + local -a flags=() + [[ -t 1 ]] && flags+=(-t) + docker exec "${flags[@]}" "$ctr" status "$@" +} +# ───────────────────────────────────────────────────────────────────────────── +# 6b. LOGS / OPEN +# ───────────────────────────────────────────────────────────────────────────── +cmd_logs() { + local svc="" follow=0 since="" grep_pat="" + while [[ "${1:-}" ]]; do + case "$1" in + -f | --follow) + follow=1 + shift + ;; + --since) + since="${2:-}" + shift 2 + ;; + --grep) + grep_pat="${2:-}" + shift 2 + ;; + *) + svc="${1:-}" + shift + ;; + esac + done + + local -a args=() + ((follow)) && args+=("-f") + [[ -n "$since" ]] && args+=("--since" "$since") + + if [[ -n "$svc" ]]; then + local s + s="$(resolve_service "$svc" || true)" + [[ -n "$s" ]] || die "Unknown service: $svc" + if [[ -n "$grep_pat" ]]; then + docker_compose logs "${args[@]}" "$s" 2>&1 | text_grep "$grep_pat" + else + docker_compose logs "${args[@]}" "$s" + fi + else + if [[ -n "$grep_pat" ]]; then + docker_compose logs "${args[@]}" 2>&1 | text_grep "$grep_pat" + else + docker_compose logs "${args[@]}" + fi + fi +} + +cmd_open() { + local target="${1:-}" + [[ -n "$target" ]] || die "open " + local url="" + case "${target,,}" in + mail | mailpit | webmail) url="https://webmail.localhost" ;; + db | cloudbeaver) url="https://db.localhost" ;; + redis | redisinsight | redis-insight | rds) url="http://ri.localhost" ;; + mongo | me | mongoexpress | mongo-express) url="http://me.localhost" ;; + kibana | kbn) url="http://kibana.localhost" ;; + *) + url="https://${target}" + ;; + esac + open_url "$url" +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6c. PROFILES +# ───────────────────────────────────────────────────────────────────────────── +_known_profile() { + local p="${1:-}" + [[ -n "$p" ]] || return 1 + + # Prefer compose-config JSON for exact profile membership. + local json + json="$(compose_cfg_json)" + if [[ -n "$json" ]] && has_tool jq; then + printf '%s' "$json" | jq -e --arg p "$p" ' + [ .services[]? | (.profiles // [])[] ] | index($p) != null + ' >/dev/null 2>&1 + return $? + fi + + # Fallback: text scan when jq/json path is unavailable. + local f + for f in "$COMPOSE_FILE" "${__EXTRA_FILES[@]:-}"; do + [[ -r "$f" ]] || continue + grep -Fq -- "$p" "$f" && return 0 + done + return 1 +} + +cmd_profiles() { + local action="${1:-list}" + shift || true + case "${action,,}" in + list | "") + local cur="" + [[ -r "$ENV_DOCKER" ]] && cur="$(grep -E '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1 | cut -d= -f2- | tr -d '\r' || true)" + printf "%bEnabled profiles:%b %s +" "$CYAN" "$NC" "${cur:-}" + printf "%bAvailable profiles:%b +" "$CYAN" "$NC" + printf ' - %s +' "${SERVICES[@]}" | LC_ALL=C sort -u + # warn if enabled profile has no mention in compose + if [[ -n "$cur" ]]; then + local p + IFS=',' read -r -a __ps <<<"$cur" + for p in "${__ps[@]}"; do + p="${p//[[:space:]]/}" + [[ -n "$p" ]] || continue + _known_profile "$p" || printf "%b[warn]%b enabled profile '%s' has no matching services in compose +" "$YELLOW" "$NC" "$p" + done + fi + ;; + add) + [[ $# -gt 0 ]] || die "profiles add " + for p in "$@"; do + modify_profiles add "$p" + done + ;; + remove | rm | del) + [[ $# -gt 0 ]] || die "profiles remove " + modify_profiles remove "$@" + ;; + *) + die "profiles " + ;; + esac +} + + +# ───────────────────────────────────────────────────────────────────────────── +# 6e. SECRETS / CERT / HOST / UI +# ───────────────────────────────────────────────────────────────────────────── +cmd_secrets() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" senv "$@" +} + +cmd_cert() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" certify "$@" +} + +cmd_host() { + local sub="${1:-}" + shift || true + case "${sub,,}" in + add) + setup_domain + ;; + rm | remove | del | delete) + delete_domain "$@" + ;; + list) + shopt -s nullglob + for f in "$DIR/configuration/nginx/"*.conf; do + printf '%s +' "$(basename -- "$f" .conf)" + done + shopt -u nullglob + ;; + *) + die "host " + ;; + esac +} + +cmd_ui() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec -it "$ctr" lazydocker +} + +# ───────────────────────────────────────────────────────────────────────────── +# 6f. EXEC / EVENTS / CLEAN / DISK +# ───────────────────────────────────────────────────────────────────────────── +cmd_exec() { + local svc="${1:-}" + shift || true + [[ -n "$svc" ]] || die "exec [cmd...]" + local s + s="$(resolve_service "$svc" || true)" + [[ -n "$s" ]] || die "Unknown service: $svc" + if [[ $# -gt 0 ]]; then + docker_compose exec "$s" "$@" + else + docker_compose exec "$s" sh -lc 'command -v bash >/dev/null 2>&1 && exec bash || exec sh' + fi +} + +cmd_events() { + local since="${1:-1h}" + local project + project="$(lds_project)" + docker events --since "$since" --filter "label=com.docker.compose.project=$project" +} + +cmd_clean() { + local yes=0 vols=0 + while [[ "${1:-}" ]]; do + case "$1" in + --yes | -y) + yes=1 + shift + ;; + --volumes | -v) + vols=1 + shift + ;; + *) + die "clean [--yes|-y] [--volumes|-v]" + ;; + esac + done + + ((yes)) || die "clean requires --yes" + + printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" + docker container prune -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" + docker network prune -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" + docker image prune -a -f >/dev/null 2>&1 || true + + printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" + docker builder prune -a -f >/dev/null 2>&1 || true + + if ((vols)); then + printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" + docker volume prune -f >/dev/null 2>&1 || true + fi + + printf "%b[clean]%b done\n" "$GREEN" "$NC" +} + + +normalize_service() { + local raw="${1:-}" + local s="${raw//[[:space:]]/}" + [[ -n "$s" ]] || { + printf '%s' "" + return 0 + } + + local low="${s,,}" + + local key="${low//_/}" + key="${key//-/}" + if [[ "$key" =~ ^php ]]; then + local ver="${key#php}" + ver="${ver//[^0-9]/}" + if [[ "$ver" =~ ^([0-9])([0-9]).* ]]; then + printf 'php%s%s' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" + return 0 + fi + printf 'php' + return 0 + fi + + low="${low//_/-}" + while [[ "$low" == *"--"* ]]; do low="${low//--/-}"; done + printf '%s' "$low" +} + +cmd_rebuild() { + local -a targets=() all_svcs=() + local arg svc img + declare -A seen=() + + # ----------------------------- + # helper: add a service once + # ----------------------------- + _add_target() { + local s="$1" + [[ -n "$s" ]] || return 0 + [[ -n "${seen[$s]:-}" ]] && return 0 + seen[$s]=1 + targets+=("$s") + } + + # ----------------------------- + # helper: trim + # ----------------------------- + _trim() { + local s="$1" + s="${s#"${s%%[![:space:]]*}"}" + s="${s%"${s##*[![:space:]]}"}" + printf '%s' "$s" + } + + # ----------------------------- + # helper: interactive selection (comma separated, supports ranges) + # accepts: "all" or "1,3,5-7" or mix with names "nginx,2,5-6" + # ----------------------------- + _pick_targets_interactive() { + compose_services_load + all_svcs=("${__COMPOSE_SVCS[@]}") + if ((${#all_svcs[@]})); then + mapfile -t all_svcs < <(printf '%s\n' "${all_svcs[@]}" | LC_ALL=C sort -f -u) + fi + [[ ${#all_svcs[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" + + echo + echo "Select services to rebuild (comma separated; ranges allowed)." + echo "Examples: 1,3,5-7 | nginx,2,5-6 | all" + echo + + local i + for i in "${!all_svcs[@]}"; do + printf " %2d) %s\n" "$((i + 1))" "${all_svcs[$i]}" + done + + echo + local sel + read -r -p "Pick: " sel + sel="$(_trim "${sel:-}")" + [[ -n "$sel" ]] || die "No selection provided." + + if [[ "${sel,,}" == "all" ]]; then + for svc in "${all_svcs[@]}"; do _add_target "$svc"; done + return 0 + fi + + # split by comma + local IFS=, + for arg in $sel; do + arg="$(_trim "$arg")" + [[ -n "$arg" ]] || continue + + # range like 3-7 + if [[ "$arg" =~ ^[0-9]+-[0-9]+$ ]]; then + local a b + a="${arg%-*}" + b="${arg#*-}" + ((a >= 1)) || continue + ((b >= 1)) || continue + ((a <= b)) || { + local t="$a" + a="$b" + b="$t" + } + + local n + for ((n = a; n <= b; n++)); do + ((n >= 1 && n <= ${#all_svcs[@]})) || continue + _add_target "${all_svcs[$((n - 1))]}" + done + continue + fi + + # single index + if [[ "$arg" =~ ^[0-9]+$ ]]; then + local n="$arg" + ((n >= 1 && n <= ${#all_svcs[@]})) || continue + _add_target "${all_svcs[$((n - 1))]}" + continue + fi + + # treat as service/container name + svc="$(resolve_service "$arg")" + [[ -n "$svc" ]] && _add_target "$svc" + done + + [[ ${#targets[@]} -gt 0 ]] || die "No valid services selected." + } + + # ----------------------------- + # build target list + # ----------------------------- + if (($# == 0)); then + _pick_targets_interactive + elif [[ "${1,,}" == "all" ]]; then + compose_services_load + targets=("${__COMPOSE_SVCS[@]}") + [[ ${#targets[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" + else + for arg in "$@"; do + svc="$(resolve_service "$arg")" + [[ -n "$svc" ]] || continue + _add_target "$svc" + done + [[ ${#targets[@]} -gt 0 ]] || die "No valid services provided." + fi + + # ----------------------------- + # rebuild each target + # ----------------------------- + for svc in "${targets[@]}"; do + [[ -n "$svc" ]] || continue + compose_service_exists "$svc" || die "Unknown service/container: '$svc'" + + if compose_has_build "$svc"; then + logq rebuild "build/recreate $svc" + dc_build --pull "$svc" + dc_up -d --no-deps --force-recreate "$svc" + continue + fi + + img="$(compose_image_for_service "$svc")" + logq rebuild "pull/recreate $svc${img:+ ($img)}" + + docker_compose rm -sf "$svc" >/dev/null 2>&1 || true + + if [[ -n "${img:-}" ]]; then + docker rmi -f "$img" >/dev/null 2>&1 || true + dc_pull "$svc" || true + else + dc_build --pull "$svc" >/dev/null 2>&1 || true + fi + + dc_up -d --no-deps --force-recreate "$svc" + done + logq reboot "Rebooting stacks" + cmd_reboot +} + +cmd_config() { docker_compose config; } + +docker_shell() { + local c="${1:-}" + [[ -n "$c" ]] || die "container name required" + if docker exec "$c" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then + exec docker exec -it "$c" bash + else + exec docker exec -it "$c" sh + fi +} +cmd_tools() { + local sub="${1:-sh}" + shift || true + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + case "${sub,,}" in + sh | shell | "") + docker_shell "$ctr" + ;; + exec) + [[ $# -gt 0 ]] || die "tools exec " + docker exec -it "$ctr" sh -lc "$*" + ;; + file) + local p="${1:-}" + [[ -n "$p" ]] || die "tools file " + docker exec -it "$ctr" sh -lc "ls -la -- \"$p\" 2>/dev/null || true; echo; sed -n '1,200p' -- \"$p\" 2>/dev/null || true" + ;; + *) + die "tools " + ;; + esac +} +cmd_http() { [[ ${1:-} == reload ]] && http_reload; } +cmd_cli() { + local ctr="${1:-}" + shift || true + + [[ -n "$ctr" ]] || die "Usage: lds cli [cmd...]" + + docker inspect "$ctr" >/dev/null 2>&1 || die "Container not found: $ctr" + docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "Container not running: $ctr" + + # If user provided a command, run it; otherwise open an interactive shell. + if [[ "$#" -gt 0 ]]; then + local cmd="$*" + docker exec -it "$ctr" sh -lc ' + if command -v bash >/dev/null 2>&1; then + exec bash --login -lc "$1" + fi + exec sh -lc "$1" + ' sh "$cmd" + return + fi + + docker exec -it "$ctr" sh -lc ' + if command -v bash >/dev/null 2>&1; then + exec bash --login + fi + exec sh + ' +} + +cmd_core() { + # Usage: + # lds core -> open correct container for that domain (PHP/Node) + # lds core -> open a shell in that container + # lds core -> list domains and let user pick + + local target="${1:-}" + + # domain regex (same as domain-which/mkhost family) + local re='^([a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9])?\.)+(localhost|local|test|loc|[a-zA-Z]{2,})$' + + # If no target -> prompt from domain-which list + if [[ -z "$target" ]]; then + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + local -a domains=() + mapfile -t domains < <(docker exec "$tools_ctr" domain-which --list-domains 2>/dev/null | sed '/^[[:space:]]*$/d' || true) + + ((${#domains[@]} > 0)) || die "No domains found" + + # stable ordering + IFS=$'\n' domains=($(printf '%s\n' "${domains[@]}" | LC_ALL=C sort -u)) + + if ((${#domains[@]} == 1)); then + target="${domains[0]}" + else + if [[ ! -t 0 ]]; then + printf "%b[core]%b No domain provided. Available domains:\n" "$YELLOW" "$NC" >&2 + local i=1 + local d + for d in "${domains[@]}"; do + printf " %2d) %s\n" "$i" "$d" >&2 + ((i++)) + done + die "No TTY to prompt. Use: lds core " + fi + + printf "%bSelect domain:%b\n" "$CYAN" "$NC" >&2 + local i=1 d + for d in "${domains[@]}"; do + printf " %b%2d)%b %s\n" "$CYAN" "$i" "$NC" "$d" >&2 + ((i++)) + done + + local ans="" + while true; do + read -r -p "Enter number (1-${#domains[@]}): " ans + ans="$(echo "$ans" | xargs)" + [[ "$ans" =~ ^[0-9]+$ ]] || { + printf "%bInvalid input.%b\n" "$YELLOW" "$NC" >&2 + continue + } + ((ans >= 1 && ans <= ${#domains[@]})) || { + printf "%bOut of range.%b\n" "$YELLOW" "$NC" >&2 + continue + } + target="${domains[$((ans - 1))]}" + break + done + fi + fi + + # If target looks like a domain -> resolve via domain-which then shell in + if [[ "$target" =~ $re ]]; then + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + + local app container wd + app="$(docker exec "$tools_ctr" domain-which --app --quiet "$target" 2>/dev/null)" || die "Unknown domain: $target" + container="$(docker exec "$tools_ctr" domain-which --container --quiet "$target" 2>/dev/null)" || die "No container resolved for: $target" + wd="$(docker exec "$tools_ctr" domain-which --docroot --quiet "$target" 2>/dev/null)" || true + [[ -n "${container:-}" ]] || die "No container resolved for: $target" + + # Node apps should always land at /app. Others follow resolved docroot. + if [[ "${app:-}" == "node" ]]; then + wd="/app" + fi + [[ -n "${wd:-}" ]] || wd="/app" + + docker exec -it "$container" bash -lc "cd \"$wd\" 2>/dev/null || cd /app 2>/dev/null || cd /; exec bash" + return 0 + fi + + # Otherwise treat target as a container name + docker exec -it "$(printf '%s' "$target" | tr '[:lower:]' '[:upper:]')" sh -lc 'exec bash -i || exec sh' +} + +cmd_setup() { + add_required_env + case ${1:-} in + init) env_init ;; + permission | permissions | perms | perm) fix_perms ;; + domain) setup_domain ;; + profiles | profile) process_all ;; + *) die "setup " ;; + esac +} + +cmd_certificate() { + case ${1:-} in + install) + shift || true + install_ca + ;; + uninstall | remove | rm) + shift || true + uninstall_ca "${@:-}" + ;; + *) + die "certificate " + ;; + esac +} + +############################################################################### +# NOTIFY +############################################################################### +notify_watch() { + local container="${1:-}" + if [[ -z "$container" ]]; then + container="$(_project_tools_container_running || true)" + [[ -n "$container" ]] || die "server-tools container is not running for project: $(lds_project)" + fi + local prefix="__HOST_NOTIFY__" + + need docker + + local _disp="${DISPLAY-}" + local _dbus="${DBUS_SESSION_BUS_ADDRESS-}" + + # Args: timeout(ms) urgency title body + _host_notify() { + local timeout="${1:-2500}" urgency="${2:-normal}" title="${3:-Notification}" body="${4:-}" + + # Linux desktop (or WSLg) + if has_cmd notify-send; then + (env DISPLAY="${_disp-}" DBUS_SESSION_BUS_ADDRESS="${_dbus-}" \ + setsid -f notify-send -u "$urgency" -t "$timeout" "$title" "$body" \ + >/dev/null 2>&1 || true) & + return 0 + fi + + # Windows toast (Git Bash) / WSL-on-Windows + if has_cmd powershell.exe; then + # Pass values as args to avoid quoting issues entirely. + # Note: urgency/timeout not used by toast api here; kept for parity. + powershell.exe -NoProfile -ExecutionPolicy Bypass -Command \ + 'param([string]$t,[string]$b) + try { + [Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null + [Windows.Data.Xml.Dom.XmlDocument, Windows.Data.Xml.Dom.XmlDocument, ContentType = WindowsRuntime] > $null + + function Esc([string]$s) { + if ($null -eq $s) { return "" } + return ($s -replace "&","&" -replace "<","<" -replace ">",">" -replace "\"",""" -replace "'\''","'") + } + + $title = Esc $t + $body = Esc $b + + $xml = New-Object Windows.Data.Xml.Dom.XmlDocument + $xml.LoadXml("$title$body") + $toast = New-Object Windows.UI.Notifications.ToastNotification $xml + [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier("Devtainer").Show($toast) + } catch { }' \ + --% "$title" "$body" >/dev/null 2>&1 || true + + return 0 + fi + + # Fallback + printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 + return 0 + } + + trap - ERR + set +e + set +o pipefail + + local _stop=0 + + _watcher_notify() { + local urgency="${1:-critical}" title="${2:-Notifier}" body="${3:-Watcher event}" + _host_notify 2500 "$urgency" "$title" "$body" + } + + _watcher_int_term() { + _stop=1 + _watcher_notify critical "Notifier" "Notification watcher interrupted/exiting" + printf "%b[watcher]%b Notification watcher interrupted/exiting\n" "$RED" "$NC" >&2 + } + trap _watcher_int_term INT TERM + + local grep_cmd=(grep -a --line-buffered -E "^${prefix}([[:space:]]|$)") + has_cmd stdbuf && grep_cmd=(stdbuf -oL -eL "${grep_cmd[@]}") + + printf "%bNotify Watch:%b monitoring is active. Ctrl+C to stop.\n" "$GREEN" "$NC" + + while ((_stop == 0)); do + if ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then + _watcher_notify critical "Notifier" "Watcher stopped: $container is not running" + printf "%b[watcher]%b %s is not running; exiting.\n" "$RED" "$NC" "$container" >&2 + break + fi + + docker logs -f --tail 0 "$container" 2>&1 | + ("${grep_cmd[@]}" || true) | + while IFS=$'\t' read -r _ f1 f2 f3 f4 rest; do + local timeout urgency title body + + if [[ "${f1:-}" =~ ^[0-9]{1,6}$ ]]; then + timeout="$f1" + urgency="${f2:-normal}" + title="${f3:-Notification}" + body="${f4:-}" + else + timeout="2500" + urgency="${f1:-normal}" + title="${f2:-Notification}" + body="${f3:-}" + fi + + [[ -n "${rest:-}" ]] && body+=$'\t'"${rest}" + case "$urgency" in low | normal | critical) ;; *) urgency="normal" ;; esac + + _host_notify "$timeout" "$urgency" "$title" "$body" + printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 + done + + ((_stop)) && break + + if docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then + _watcher_notify critical "Notifier" "Watcher lost log stream (docker logs ended). Reconnecting…" + printf "%b[watcher]%b docker logs ended; reconnecting...\n" "$YELLOW" "$NC" >&2 + sleep 1 + continue + fi + + _watcher_notify critical "Notifier" "Watcher stopped: $container stopped" + printf "%b[watcher]%b %s stopped; exiting.\n" "$RED" "$NC" "$container" >&2 + break + done + + trap - INT TERM + set -euo pipefail + + ((_stop)) && return 130 + return 0 +} + +notify_test() { + local title="${1:-Notifier OK}" + local body="${2:-Hello from host via project server-tools container}" + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" notify -t 2500 -u normal "$title" "$body" +} + +cmd_notify() { + case ${1:-watch} in + watch) notify_watch "${2:-}" ;; + test) notify_test "${2:-Notifier OK}" "${3:-Hello from host}" ;; + *) die "notify " ;; + esac +} + +open_url() { + local url="${1:-}" + [[ -n "$url" ]] || return 0 + + # WSL/Windows helpers first when available + if grep -qi microsoft /proc/version 2>/dev/null; then + if has_cmd powershell.exe; then + powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 || true + return 0 + fi + if has_cmd cmd.exe; then + cmd.exe /c start "" "$url" >/dev/null 2>&1 || true + return 0 + fi + fi + + if has_cmd xdg-open; then + (xdg-open "$url" >/dev/null 2>&1 &) + return 0 + fi + if has_cmd open; then + (open "$url" >/dev/null 2>&1 &) + return 0 + fi + if has_cmd powershell; then + (powershell -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 &) + return 0 + fi + + printf "%bINFO%b: open this URL manually → %s\n" "$YELLOW" "$NC" "$url" +} + From 97ddf79522c45fc2b41364fd935571c1a57a0294 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:13:02 +0600 Subject: [PATCH 101/509] refactor: extract ai.sh module --- lib/ai.sh | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 89 insertions(+) create mode 100644 lib/ai.sh diff --git a/lib/ai.sh b/lib/ai.sh new file mode 100644 index 00000000..88ffcbd5 --- /dev/null +++ b/lib/ai.sh @@ -0,0 +1,89 @@ +# shellcheck shell=bash +_tools_exec() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + # NOTE: pass a SINGLE command string; do not pass arrays here. + docker exec -i "$ctr" sh -lc "$*" +} + +_tools_exec_argv() { + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + local -a flags=(-i) + [[ -t 0 && -t 1 ]] && flags+=(-t) + docker exec "${flags[@]}" "$ctr" "$@" +} + +cmd_ai() { + local sub="${1:-status}" + shift || true + case "${sub,,}" in + status | provider) _tools_exec_argv aiops provider "$@" ;; + ask) _tools_exec_argv askai "$@" ;; + explain | troubleshoot | review | repo-review | graphify) + _tools_exec_argv aiops "${sub,,}" "$@" + ;; + *) die "ai [args...]" ;; + esac +} + +_llm_exec() { + local ctr + ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" + [[ -n "$ctr" ]] || die "llm-sm is not running. Enable the ai profile and start the stack first." + docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || + die "llm-sm container exists but is not running." + + local -a exec_args=(exec) + [[ -t 0 && -t 1 ]] || exec_args+=(-T) + docker_compose "${exec_args[@]}" llm-sm llm-sm "$@" +} + +cmd_llm() { + local sub="${1:-models}" + shift || true + + case "${sub,,}" in + runtime) + local mode="${1:-}" + if [[ -z "$mode" ]]; then + printf '%s\n' "$(compose_control_value LDS_AI_RUNTIME cpu)" + return 0 + fi + case "${mode,,}" in + cpu | nvidia | amd) + update_env "$ENV_DOCKER" LDS_AI_RUNTIME "${mode,,}" + ok "LLM runtime set to ${mode,,}. Recreate llm-sm to apply the change." + ;; + *) die "llm runtime " ;; + esac + ;; + host-port) + local state="${1:-status}" + case "${state,,}" in + status) printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" ;; + on | enable | enabled | 1) + update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 1 + ok "Direct LLM API enabled on loopback only. Recreate llm-sm to apply." + ;; + off | disable | disabled | 0) + update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 0 + ok "Direct LLM host API disabled. Recreate llm-sm to apply." + ;; + *) die "llm host-port " ;; + esac + ;; + models | ps | show | pull | rm | unload | run | ask | chat | prompt | code | review | json | ai-commit | ollama | api | version) + _llm_exec "${sub,,}" "$@" + ;; + help | -h | --help) + printf '%s\n' "llm " + printf '%s\n' "llm runtime " + printf '%s\n' "llm host-port " + ;; + *) die "llm " ;; + esac +} + From dc430d128c4bd691ace4f0c1129a0c2212bf3d57 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:13:06 +0600 Subject: [PATCH 102/509] refactor: extract diagnostics.sh module --- lib/diagnostics.sh | 56 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) create mode 100644 lib/diagnostics.sh diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh new file mode 100644 index 00000000..4402c342 --- /dev/null +++ b/lib/diagnostics.sh @@ -0,0 +1,56 @@ +# shellcheck shell=bash +_shq() { printf '%q' "$1"; } + +cmd_diag() { + local sub="${1:-}" + shift || true + + case "${sub,,}" in + dns) + local dom="${1:-}" + [[ -n "$dom" ]] || die "diag dns " + local qdom + qdom="$(_shq "$dom")" + _tools_exec "dig +short $qdom; echo; nslookup $qdom 2>/dev/null || true; echo; getent hosts $qdom 2>/dev/null || true" + ;; + route | net) + _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || netstat -tulpen 2>/dev/null || true" + ;; + tcp) + local h="${1:-}" + local p="${2:-}" + [[ -n "$h" && -n "$p" ]] || die "diag tcp " + _tools_exec "nc -vz -w2 $(_shq "$h") $(_shq "$p")" + ;; + http) + local url="${1:-}" + shift || true + [[ -n "$url" ]] || die "diag http [curl-args...]" + local -a qargs=() + local a + for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done + _tools_exec "curl -vkI $(_shq "$url") ${qargs[*]}" + ;; + tls) + local dom="${1:-}" + [[ -n "$dom" ]] || die "diag tls " + local qdom + qdom="$(_shq "$dom")" + _tools_exec "echo | openssl s_client -connect ${qdom}:443 -servername $qdom -showcerts 2>/dev/null | sed -n '1,60p'" + ;; + *) + die "diag " + ;; + esac +} + +cmd_sniff() { + local url="${1:-}" + shift || true + [[ -n "$url" ]] || die "sniff [curl-args...]" + local -a qargs=() + local a + for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done + _tools_exec "curl -vk -D - $(_shq "$url") ${qargs[*]} | (command -v jq >/dev/null 2>&1 && jq . 2>/dev/null || cat)" +} + From 4a051e397569bbbd79430ef7df3295f750377807 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:13:10 +0600 Subject: [PATCH 103/509] refactor: source service AI and diagnostics modules --- lds | 1190 +---------------------------------------------------------- 1 file changed, 10 insertions(+), 1180 deletions(-) diff --git a/lds b/lds index ef866311..70c8c348 100755 --- a/lds +++ b/lds @@ -472,1199 +472,29 @@ source "$DIR/lib/profiles.sh" source "$DIR/lib/certificates.sh" ############################################################################### -# Compose helpers for rebuild (robust: supports service key OR container name) +# SERVICE / STACK OPERATIONS ############################################################################### -__COMPOSE_CFG_JSON="" -__COMPOSE_CFG_YAML="" -__COMPOSE_SVCS_LOADED=0 -declare -a __COMPOSE_SVCS=() - -compose_cfg_json() { - if [[ -z "${__COMPOSE_CFG_JSON}" ]]; then - __COMPOSE_CFG_JSON="$(docker_compose config --format json 2>/dev/null || true)" - fi - printf '%s' "${__COMPOSE_CFG_JSON}" -} - -compose_cfg_yaml() { - if [[ -z "${__COMPOSE_CFG_YAML}" ]]; then - __COMPOSE_CFG_YAML="$(docker_compose config 2>/dev/null || true)" - fi - printf '%s' "${__COMPOSE_CFG_YAML}" -} - -compose_services_load() { - ((__COMPOSE_SVCS_LOADED)) && return 0 - mapfile -t __COMPOSE_SVCS < <(docker_compose config --services 2>/dev/null || true) - __COMPOSE_SVCS_LOADED=1 -} - -compose_service_exists() { - local want="${1:-}" s - [[ -n "$want" ]] || return 1 - compose_services_load - for s in "${__COMPOSE_SVCS[@]}"; do - [[ "$s" == "$want" ]] && return 0 - done - return 1 -} - -resolve_service() { - local raw="${1:-}" norm svc - raw="${raw//[[:space:]]/}" - [[ -n "$raw" ]] || { - printf '' - return 0 - } - - compose_service_exists "$raw" && { - printf '%s' "$raw" - return 0 - } - - norm="$(normalize_service "$raw")" - compose_service_exists "$norm" && { - printf '%s' "$norm" - return 0 - } - - if docker inspect "$raw" >/dev/null 2>&1; then - svc="$(docker inspect -f '{{ index .Config.Labels "com.docker.compose.service" }}' "$raw" 2>/dev/null || true)" - if [[ -n "$svc" ]] && compose_service_exists "$svc"; then - printf '%s' "$svc" - return 0 - fi - fi - - printf '%s' "$norm" -} - -compose_has_build() { - local svc="$1" json - json="$(compose_cfg_json)" - if [[ -n "$json" ]]; then - if has_tool jq; then - jq -e --arg s "$svc" '.services[$s].build != null' >/dev/null <<<"$json" - return $? - fi - fi - - compose_cfg_yaml | awk -v s="$svc" ' - $1=="services:" {in_services=1; next} - in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} - in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 1} - in_svc && $0 ~ /^ build:/ {exit 0} - END {exit 1} - ' -} - -compose_image_for_service() { - local svc="$1" json - json="$(compose_cfg_json)" - if [[ -n "$json" ]]; then - if has_tool jq; then - jq -r --arg s "$svc" '.services[$s].image // empty' <<<"$json" - return 0 - fi - fi - - compose_cfg_yaml | awk -v s="$svc" ' - $1=="services:" {in_services=1; next} - in_services && $0 ~ ("^ " s ":$") {in_svc=1; next} - in_svc && $0 ~ /^ [A-Za-z0-9_.-]+:$/ {exit 0} - in_svc && $0 ~ /^ image:/ { - sub(/^ image:[[:space:]]*/, "", $0) - print $0 - exit 0 - } - ' -} - -############################################################################### -# 6. STACK COMMANDS (CLI) -############################################################################### - -# One-time migration from the historical fixed 172.28/29/30 /24 networks. -# Only known LocalDevStack-owned networks are touched, and named volumes are -# never removed. New networks carry com.infocyph.network-schema=dynamic-v1. -declare -a __LDS_LEGACY_NETWORK_NAMES=(Frontend Backend DataStore) - -legacy_network_expected_subnet() { - case "\${1:-}" in - Frontend) printf '%s' '172.28.0.0/24' ;; - Backend) printf '%s' '172.29.0.0/24' ;; - DataStore) printf '%s' '172.30.0.0/24' ;; - *) return 1 ;; - esac -} - -migrate_legacy_networks() { - local network expected subnets stack_label project_label schema_label ctr ctr_project attachments - local -a legacy=() - - for network in "\${__LDS_LEGACY_NETWORK_NAMES[@]}"; do - docker network inspect "$network" >/dev/null 2>&1 || continue - - schema_label="$(docker network inspect -f '{{index .Labels "com.infocyph.network-schema"}}' "$network" 2>/dev/null || true)" - [[ "$schema_label" != "dynamic-v1" ]] || continue - - expected="$(legacy_network_expected_subnet "$network")" - subnets="$(docker network inspect -f '{{range .IPAM.Config}}{{println .Subnet}}{{end}}' "$network" 2>/dev/null || true)" - grep -Fxq "$expected" <<<"$subnets" || continue - - stack_label="$(docker network inspect -f '{{index .Labels "com.infocyph.stack"}}' "$network" 2>/dev/null || true)" - project_label="$(docker network inspect -f '{{index .Labels "com.docker.compose.project"}}' "$network" 2>/dev/null || true)" - if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "LocalDevStack" ]]; then - die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack. Remove or rename that network manually." - fi - - while IFS= read -r ctr; do - [[ -n "$ctr" ]] || continue - ctr_project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$ctr" 2>/dev/null || true)" - if [[ "$ctr_project" != "LocalDevStack" ]]; then - die "Refusing to migrate '$network': container '$ctr' is not owned by the LocalDevStack Compose project." - fi - done < <(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true) - - legacy+=("$network") - done - - (("\${#legacy[@]}" > 0)) || return 0 - - warn "Legacy fixed LocalDevStack network(s) detected: \${legacy[*]}" - warn "Recreating stack networks dynamically; named volumes and persisted data are preserved." - - # Stop/remove only LocalDevStack Compose containers and networks. Never use -v. - docker_compose down --remove-orphans - - for network in "\${legacy[@]}"; do - docker network inspect "$network" >/dev/null 2>&1 || continue - attachments="$(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true)" - [[ -z "$attachments" ]] || - die "Cannot remove legacy network '$network': attached container(s) remain: $(tr '\n' ' ' <<<"$attachments")" - docker network rm "$network" >/dev/null - done - - ok "Legacy fixed networks removed; Compose will recreate dynamic bridge networks." -} - -cmd_vpn_fix() { - warn "vpn-fix is deprecated: LocalDevStack no longer owns fixed Docker subnets." - warn "If a VPN conflict remains after dynamic-network migration, diagnose the VPN/Docker route directly." -} - -cmd_up() { - migrate_legacy_networks - dc_up "$@" -} - -cmd_start() { - migrate_legacy_networks - dc_up -d "$@" - http_reload -} - -cmd_stop() { docker_compose down; } - -cmd_down() { - # Safety rails: - # lds down --volumes requires --yes - local yes=0 vols=0 - local -a args=() - while [[ "${1:-}" ]]; do - case "$1" in - --yes | -y) - yes=1 - shift - ;; - --volumes | -v) - vols=1 - args+=("--volumes") - shift - ;; - --remove-orphans) - args+=("--remove-orphans") - shift - ;; - *) - args+=("$1") - shift - ;; - esac - done - if ((vols)) && ((yes == 0)); then - die "Refusing: down --volumes requires --yes" - fi - docker_compose down "${args[@]}" -} - -cmd_restart() { - cmd_stop - cmd_start -} -cmd_reboot() { cmd_restart; } - -# ───────────────────────────────────────────────────────────────────────────── -# 6a. STATUS / PS / STATS -# ───────────────────────────────────────────────────────────────────────────── -cmd_ps() { - if (($#)); then - docker_compose ps "$@" - else - docker_compose ps - fi -} - -cmd_status() { - local ctr project - project="$(lds_project)" - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container not found or not running for project: $project" - - local -a flags=() - [[ -t 1 ]] && flags+=(-t) - docker exec "${flags[@]}" "$ctr" status "$@" -} -# ───────────────────────────────────────────────────────────────────────────── -# 6b. LOGS / OPEN -# ───────────────────────────────────────────────────────────────────────────── -cmd_logs() { - local svc="" follow=0 since="" grep_pat="" - while [[ "${1:-}" ]]; do - case "$1" in - -f | --follow) - follow=1 - shift - ;; - --since) - since="${2:-}" - shift 2 - ;; - --grep) - grep_pat="${2:-}" - shift 2 - ;; - *) - svc="${1:-}" - shift - ;; - esac - done - - local -a args=() - ((follow)) && args+=("-f") - [[ -n "$since" ]] && args+=("--since" "$since") - - if [[ -n "$svc" ]]; then - local s - s="$(resolve_service "$svc" || true)" - [[ -n "$s" ]] || die "Unknown service: $svc" - if [[ -n "$grep_pat" ]]; then - docker_compose logs "${args[@]}" "$s" 2>&1 | text_grep "$grep_pat" - else - docker_compose logs "${args[@]}" "$s" - fi - else - if [[ -n "$grep_pat" ]]; then - docker_compose logs "${args[@]}" 2>&1 | text_grep "$grep_pat" - else - docker_compose logs "${args[@]}" - fi - fi -} - -cmd_open() { - local target="${1:-}" - [[ -n "$target" ]] || die "open " - local url="" - case "${target,,}" in - mail | mailpit | webmail) url="https://webmail.localhost" ;; - db | cloudbeaver) url="https://db.localhost" ;; - redis | redisinsight | redis-insight | rds) url="http://ri.localhost" ;; - mongo | me | mongoexpress | mongo-express) url="http://me.localhost" ;; - kibana | kbn) url="http://kibana.localhost" ;; - *) - url="https://${target}" - ;; - esac - open_url "$url" -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6c. PROFILES -# ───────────────────────────────────────────────────────────────────────────── -_known_profile() { - local p="${1:-}" - [[ -n "$p" ]] || return 1 - - # Prefer compose-config JSON for exact profile membership. - local json - json="$(compose_cfg_json)" - if [[ -n "$json" ]] && has_tool jq; then - printf '%s' "$json" | jq -e --arg p "$p" ' - [ .services[]? | (.profiles // [])[] ] | index($p) != null - ' >/dev/null 2>&1 - return $? - fi - - # Fallback: text scan when jq/json path is unavailable. - local f - for f in "$COMPOSE_FILE" "${__EXTRA_FILES[@]:-}"; do - [[ -r "$f" ]] || continue - grep -Fq -- "$p" "$f" && return 0 - done - return 1 -} - -cmd_profiles() { - local action="${1:-list}" - shift || true - case "${action,,}" in - list | "") - local cur="" - [[ -r "$ENV_DOCKER" ]] && cur="$(grep -E '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1 | cut -d= -f2- | tr -d '\r' || true)" - printf "%bEnabled profiles:%b %s -" "$CYAN" "$NC" "${cur:-}" - printf "%bAvailable profiles:%b -" "$CYAN" "$NC" - printf ' - %s -' "${SERVICES[@]}" | LC_ALL=C sort -u - # warn if enabled profile has no mention in compose - if [[ -n "$cur" ]]; then - local p - IFS=',' read -r -a __ps <<<"$cur" - for p in "${__ps[@]}"; do - p="${p//[[:space:]]/}" - [[ -n "$p" ]] || continue - _known_profile "$p" || printf "%b[warn]%b enabled profile '%s' has no matching services in compose -" "$YELLOW" "$NC" "$p" - done - fi - ;; - add) - [[ $# -gt 0 ]] || die "profiles add " - for p in "$@"; do - modify_profiles add "$p" - done - ;; - remove | rm | del) - [[ $# -gt 0 ]] || die "profiles remove " - modify_profiles remove "$@" - ;; - *) - die "profiles " - ;; - esac -} +# shellcheck source=lib/services.sh +source "$DIR/lib/services.sh" # ───────────────────────────────────────────────────────────────────────────── # 6d. DIAG / SNIFF # ───────────────────────────────────────────────────────────────────────────── -_tools_exec() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - # NOTE: pass a SINGLE command string; do not pass arrays here. - docker exec -i "$ctr" sh -lc "$*" -} - -_tools_exec_argv() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - local -a flags=(-i) - [[ -t 0 && -t 1 ]] && flags+=(-t) - docker exec "${flags[@]}" "$ctr" "$@" -} - -cmd_ai() { - local sub="${1:-status}" - shift || true - case "${sub,,}" in - status | provider) _tools_exec_argv aiops provider "$@" ;; - ask) _tools_exec_argv askai "$@" ;; - explain | troubleshoot | review | repo-review | graphify) - _tools_exec_argv aiops "${sub,,}" "$@" - ;; - *) die "ai [args...]" ;; - esac -} - -_llm_exec() { - local ctr - ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" - [[ -n "$ctr" ]] || die "llm-sm is not running. Enable the ai profile and start the stack first." - docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || - die "llm-sm container exists but is not running." - - local -a exec_args=(exec) - [[ -t 0 && -t 1 ]] || exec_args+=(-T) - docker_compose "${exec_args[@]}" llm-sm llm-sm "$@" -} - -cmd_llm() { - local sub="${1:-models}" - shift || true - - case "${sub,,}" in - runtime) - local mode="${1:-}" - if [[ -z "$mode" ]]; then - printf '%s\n' "$(compose_control_value LDS_AI_RUNTIME cpu)" - return 0 - fi - case "${mode,,}" in - cpu | nvidia | amd) - update_env "$ENV_DOCKER" LDS_AI_RUNTIME "${mode,,}" - ok "LLM runtime set to ${mode,,}. Recreate llm-sm to apply the change." - ;; - *) die "llm runtime " ;; - esac - ;; - host-port) - local state="${1:-status}" - case "${state,,}" in - status) printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" ;; - on | enable | enabled | 1) - update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 1 - ok "Direct LLM API enabled on loopback only. Recreate llm-sm to apply." - ;; - off | disable | disabled | 0) - update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 0 - ok "Direct LLM host API disabled. Recreate llm-sm to apply." - ;; - *) die "llm host-port " ;; - esac - ;; - models | ps | show | pull | rm | unload | run | ask | chat | prompt | code | review | json | ai-commit | ollama | api | version) - _llm_exec "${sub,,}" "$@" - ;; - help | -h | --help) - printf '%s\n' "llm " - printf '%s\n' "llm runtime " - printf '%s\n' "llm host-port " - ;; - *) die "llm " ;; - esac -} - -_shq() { printf '%q' "$1"; } - -cmd_diag() { - local sub="${1:-}" - shift || true - - case "${sub,,}" in - dns) - local dom="${1:-}" - [[ -n "$dom" ]] || die "diag dns " - local qdom - qdom="$(_shq "$dom")" - _tools_exec "dig +short $qdom; echo; nslookup $qdom 2>/dev/null || true; echo; getent hosts $qdom 2>/dev/null || true" - ;; - route | net) - _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || netstat -tulpen 2>/dev/null || true" - ;; - tcp) - local h="${1:-}" - local p="${2:-}" - [[ -n "$h" && -n "$p" ]] || die "diag tcp " - _tools_exec "nc -vz -w2 $(_shq "$h") $(_shq "$p")" - ;; - http) - local url="${1:-}" - shift || true - [[ -n "$url" ]] || die "diag http [curl-args...]" - local -a qargs=() - local a - for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done - _tools_exec "curl -vkI $(_shq "$url") ${qargs[*]}" - ;; - tls) - local dom="${1:-}" - [[ -n "$dom" ]] || die "diag tls " - local qdom - qdom="$(_shq "$dom")" - _tools_exec "echo | openssl s_client -connect ${qdom}:443 -servername $qdom -showcerts 2>/dev/null | sed -n '1,60p'" - ;; - *) - die "diag " - ;; - esac -} - -cmd_sniff() { - local url="${1:-}" - shift || true - [[ -n "$url" ]] || die "sniff [curl-args...]" - local -a qargs=() - local a - for a in "$@"; do qargs+=("$(printf '%q' "$a")"); done - _tools_exec "curl -vk -D - $(_shq "$url") ${qargs[*]} | (command -v jq >/dev/null 2>&1 && jq . 2>/dev/null || cat)" -} - -# ───────────────────────────────────────────────────────────────────────────── -# 6e. SECRETS / CERT / HOST / UI -# ───────────────────────────────────────────────────────────────────────────── -cmd_secrets() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" senv "$@" -} - -cmd_cert() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" certify "$@" -} - -cmd_host() { - local sub="${1:-}" - shift || true - case "${sub,,}" in - add) - setup_domain - ;; - rm | remove | del | delete) - delete_domain "$@" - ;; - list) - shopt -s nullglob - for f in "$DIR/configuration/nginx/"*.conf; do - printf '%s -' "$(basename -- "$f" .conf)" - done - shopt -u nullglob - ;; - *) - die "host " - ;; - esac -} - -cmd_ui() { - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec -it "$ctr" lazydocker -} +# shellcheck source=lib/ai.sh +source "$DIR/lib/ai.sh" +# shellcheck source=lib/diagnostics.sh +source "$DIR/lib/diagnostics.sh" # ───────────────────────────────────────────────────────────────────────────── -# 6f. EXEC / EVENTS / CLEAN / DISK +# 6e/6f. SERVICE / HOST / MAINTENANCE COMMANDS # ───────────────────────────────────────────────────────────────────────────── -cmd_exec() { - local svc="${1:-}" - shift || true - [[ -n "$svc" ]] || die "exec [cmd...]" - local s - s="$(resolve_service "$svc" || true)" - [[ -n "$s" ]] || die "Unknown service: $svc" - if [[ $# -gt 0 ]]; then - docker_compose exec "$s" "$@" - else - docker_compose exec "$s" sh -lc 'command -v bash >/dev/null 2>&1 && exec bash || exec sh' - fi -} - -cmd_events() { - local since="${1:-1h}" - local project - project="$(lds_project)" - docker events --since "$since" --filter "label=com.docker.compose.project=$project" -} - -cmd_clean() { - local yes=0 vols=0 - while [[ "${1:-}" ]]; do - case "$1" in - --yes | -y) - yes=1 - shift - ;; - --volumes | -v) - vols=1 - shift - ;; - *) - die "clean [--yes|-y] [--volumes|-v]" - ;; - esac - done - - ((yes)) || die "clean requires --yes" - - printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" - docker container prune -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" - docker network prune -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" - docker image prune -a -f >/dev/null 2>&1 || true - - printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" - docker builder prune -a -f >/dev/null 2>&1 || true - - if ((vols)); then - printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" - docker volume prune -f >/dev/null 2>&1 || true - fi - - printf "%b[clean]%b done\n" "$GREEN" "$NC" -} +# Implementations are loaded by lib/services.sh above. # ───────────────────────────────────────────────────────────────────────────── # 6g. HELP MARKDOWN # ───────────────────────────────────────────────────────────────────────────── -normalize_service() { - local raw="${1:-}" - local s="${raw//[[:space:]]/}" - [[ -n "$s" ]] || { - printf '%s' "" - return 0 - } - - local low="${s,,}" - - local key="${low//_/}" - key="${key//-/}" - if [[ "$key" =~ ^php ]]; then - local ver="${key#php}" - ver="${ver//[^0-9]/}" - if [[ "$ver" =~ ^([0-9])([0-9]).* ]]; then - printf 'php%s%s' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" - return 0 - fi - printf 'php' - return 0 - fi - - low="${low//_/-}" - while [[ "$low" == *"--"* ]]; do low="${low//--/-}"; done - printf '%s' "$low" -} - -cmd_rebuild() { - local -a targets=() all_svcs=() - local arg svc img - declare -A seen=() - - # ----------------------------- - # helper: add a service once - # ----------------------------- - _add_target() { - local s="$1" - [[ -n "$s" ]] || return 0 - [[ -n "${seen[$s]:-}" ]] && return 0 - seen[$s]=1 - targets+=("$s") - } - - # ----------------------------- - # helper: trim - # ----------------------------- - _trim() { - local s="$1" - s="${s#"${s%%[![:space:]]*}"}" - s="${s%"${s##*[![:space:]]}"}" - printf '%s' "$s" - } - - # ----------------------------- - # helper: interactive selection (comma separated, supports ranges) - # accepts: "all" or "1,3,5-7" or mix with names "nginx,2,5-6" - # ----------------------------- - _pick_targets_interactive() { - compose_services_load - all_svcs=("${__COMPOSE_SVCS[@]}") - if ((${#all_svcs[@]})); then - mapfile -t all_svcs < <(printf '%s\n' "${all_svcs[@]}" | LC_ALL=C sort -f -u) - fi - [[ ${#all_svcs[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" - - echo - echo "Select services to rebuild (comma separated; ranges allowed)." - echo "Examples: 1,3,5-7 | nginx,2,5-6 | all" - echo - - local i - for i in "${!all_svcs[@]}"; do - printf " %2d) %s\n" "$((i + 1))" "${all_svcs[$i]}" - done - - echo - local sel - read -r -p "Pick: " sel - sel="$(_trim "${sel:-}")" - [[ -n "$sel" ]] || die "No selection provided." - - if [[ "${sel,,}" == "all" ]]; then - for svc in "${all_svcs[@]}"; do _add_target "$svc"; done - return 0 - fi - - # split by comma - local IFS=, - for arg in $sel; do - arg="$(_trim "$arg")" - [[ -n "$arg" ]] || continue - - # range like 3-7 - if [[ "$arg" =~ ^[0-9]+-[0-9]+$ ]]; then - local a b - a="${arg%-*}" - b="${arg#*-}" - ((a >= 1)) || continue - ((b >= 1)) || continue - ((a <= b)) || { - local t="$a" - a="$b" - b="$t" - } - - local n - for ((n = a; n <= b; n++)); do - ((n >= 1 && n <= ${#all_svcs[@]})) || continue - _add_target "${all_svcs[$((n - 1))]}" - done - continue - fi - - # single index - if [[ "$arg" =~ ^[0-9]+$ ]]; then - local n="$arg" - ((n >= 1 && n <= ${#all_svcs[@]})) || continue - _add_target "${all_svcs[$((n - 1))]}" - continue - fi - - # treat as service/container name - svc="$(resolve_service "$arg")" - [[ -n "$svc" ]] && _add_target "$svc" - done - - [[ ${#targets[@]} -gt 0 ]] || die "No valid services selected." - } - - # ----------------------------- - # build target list - # ----------------------------- - if (($# == 0)); then - _pick_targets_interactive - elif [[ "${1,,}" == "all" ]]; then - compose_services_load - targets=("${__COMPOSE_SVCS[@]}") - [[ ${#targets[@]} -gt 0 ]] || die "No services found (docker compose config --services failed?)" - else - for arg in "$@"; do - svc="$(resolve_service "$arg")" - [[ -n "$svc" ]] || continue - _add_target "$svc" - done - [[ ${#targets[@]} -gt 0 ]] || die "No valid services provided." - fi - - # ----------------------------- - # rebuild each target - # ----------------------------- - for svc in "${targets[@]}"; do - [[ -n "$svc" ]] || continue - compose_service_exists "$svc" || die "Unknown service/container: '$svc'" - - if compose_has_build "$svc"; then - logq rebuild "build/recreate $svc" - dc_build --pull "$svc" - dc_up -d --no-deps --force-recreate "$svc" - continue - fi - - img="$(compose_image_for_service "$svc")" - logq rebuild "pull/recreate $svc${img:+ ($img)}" - - docker_compose rm -sf "$svc" >/dev/null 2>&1 || true - - if [[ -n "${img:-}" ]]; then - docker rmi -f "$img" >/dev/null 2>&1 || true - dc_pull "$svc" || true - else - dc_build --pull "$svc" >/dev/null 2>&1 || true - fi - - dc_up -d --no-deps --force-recreate "$svc" - done - logq reboot "Rebooting stacks" - cmd_reboot -} - -cmd_config() { docker_compose config; } - -docker_shell() { - local c="${1:-}" - [[ -n "$c" ]] || die "container name required" - if docker exec "$c" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then - exec docker exec -it "$c" bash - else - exec docker exec -it "$c" sh - fi -} -cmd_tools() { - local sub="${1:-sh}" - shift || true - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - case "${sub,,}" in - sh | shell | "") - docker_shell "$ctr" - ;; - exec) - [[ $# -gt 0 ]] || die "tools exec " - docker exec -it "$ctr" sh -lc "$*" - ;; - file) - local p="${1:-}" - [[ -n "$p" ]] || die "tools file " - docker exec -it "$ctr" sh -lc "ls -la -- \"$p\" 2>/dev/null || true; echo; sed -n '1,200p' -- \"$p\" 2>/dev/null || true" - ;; - *) - die "tools " - ;; - esac -} -cmd_http() { [[ ${1:-} == reload ]] && http_reload; } -cmd_cli() { - local ctr="${1:-}" - shift || true - - [[ -n "$ctr" ]] || die "Usage: lds cli [cmd...]" - - docker inspect "$ctr" >/dev/null 2>&1 || die "Container not found: $ctr" - docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "Container not running: $ctr" - - # If user provided a command, run it; otherwise open an interactive shell. - if [[ "$#" -gt 0 ]]; then - local cmd="$*" - docker exec -it "$ctr" sh -lc ' - if command -v bash >/dev/null 2>&1; then - exec bash --login -lc "$1" - fi - exec sh -lc "$1" - ' sh "$cmd" - return - fi - - docker exec -it "$ctr" sh -lc ' - if command -v bash >/dev/null 2>&1; then - exec bash --login - fi - exec sh - ' -} - -cmd_core() { - # Usage: - # lds core -> open correct container for that domain (PHP/Node) - # lds core -> open a shell in that container - # lds core -> list domains and let user pick - - local target="${1:-}" - - # domain regex (same as domain-which/mkhost family) - local re='^([a-zA-Z0-9]([-a-zA-Z0-9]{0,61}[a-zA-Z0-9])?\.)+(localhost|local|test|loc|[a-zA-Z]{2,})$' - - # If no target -> prompt from domain-which list - if [[ -z "$target" ]]; then - local tools_ctr - tools_ctr="$(_project_tools_container_running || true)" - [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - local -a domains=() - mapfile -t domains < <(docker exec "$tools_ctr" domain-which --list-domains 2>/dev/null | sed '/^[[:space:]]*$/d' || true) - - ((${#domains[@]} > 0)) || die "No domains found" - - # stable ordering - IFS=$'\n' domains=($(printf '%s\n' "${domains[@]}" | LC_ALL=C sort -u)) - - if ((${#domains[@]} == 1)); then - target="${domains[0]}" - else - if [[ ! -t 0 ]]; then - printf "%b[core]%b No domain provided. Available domains:\n" "$YELLOW" "$NC" >&2 - local i=1 - local d - for d in "${domains[@]}"; do - printf " %2d) %s\n" "$i" "$d" >&2 - ((i++)) - done - die "No TTY to prompt. Use: lds core " - fi - - printf "%bSelect domain:%b\n" "$CYAN" "$NC" >&2 - local i=1 d - for d in "${domains[@]}"; do - printf " %b%2d)%b %s\n" "$CYAN" "$i" "$NC" "$d" >&2 - ((i++)) - done - - local ans="" - while true; do - read -r -p "Enter number (1-${#domains[@]}): " ans - ans="$(echo "$ans" | xargs)" - [[ "$ans" =~ ^[0-9]+$ ]] || { - printf "%bInvalid input.%b\n" "$YELLOW" "$NC" >&2 - continue - } - ((ans >= 1 && ans <= ${#domains[@]})) || { - printf "%bOut of range.%b\n" "$YELLOW" "$NC" >&2 - continue - } - target="${domains[$((ans - 1))]}" - break - done - fi - fi - - # If target looks like a domain -> resolve via domain-which then shell in - if [[ "$target" =~ $re ]]; then - local tools_ctr - tools_ctr="$(_project_tools_container_running || true)" - [[ -n "$tools_ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - - local app container wd - app="$(docker exec "$tools_ctr" domain-which --app --quiet "$target" 2>/dev/null)" || die "Unknown domain: $target" - container="$(docker exec "$tools_ctr" domain-which --container --quiet "$target" 2>/dev/null)" || die "No container resolved for: $target" - wd="$(docker exec "$tools_ctr" domain-which --docroot --quiet "$target" 2>/dev/null)" || true - [[ -n "${container:-}" ]] || die "No container resolved for: $target" - - # Node apps should always land at /app. Others follow resolved docroot. - if [[ "${app:-}" == "node" ]]; then - wd="/app" - fi - [[ -n "${wd:-}" ]] || wd="/app" - - docker exec -it "$container" bash -lc "cd \"$wd\" 2>/dev/null || cd /app 2>/dev/null || cd /; exec bash" - return 0 - fi - - # Otherwise treat target as a container name - docker exec -it "$(printf '%s' "$target" | tr '[:lower:]' '[:upper:]')" sh -lc 'exec bash -i || exec sh' -} - -cmd_setup() { - add_required_env - case ${1:-} in - init) env_init ;; - permission | permissions | perms | perm) fix_perms ;; - domain) setup_domain ;; - profiles | profile) process_all ;; - *) die "setup " ;; - esac -} - -cmd_certificate() { - case ${1:-} in - install) - shift || true - install_ca - ;; - uninstall | remove | rm) - shift || true - uninstall_ca "${@:-}" - ;; - *) - die "certificate " - ;; - esac -} - -############################################################################### -# NOTIFY -############################################################################### -notify_watch() { - local container="${1:-}" - if [[ -z "$container" ]]; then - container="$(_project_tools_container_running || true)" - [[ -n "$container" ]] || die "server-tools container is not running for project: $(lds_project)" - fi - local prefix="__HOST_NOTIFY__" - - need docker - - local _disp="${DISPLAY-}" - local _dbus="${DBUS_SESSION_BUS_ADDRESS-}" - - # Args: timeout(ms) urgency title body - _host_notify() { - local timeout="${1:-2500}" urgency="${2:-normal}" title="${3:-Notification}" body="${4:-}" - - # Linux desktop (or WSLg) - if has_cmd notify-send; then - (env DISPLAY="${_disp-}" DBUS_SESSION_BUS_ADDRESS="${_dbus-}" \ - setsid -f notify-send -u "$urgency" -t "$timeout" "$title" "$body" \ - >/dev/null 2>&1 || true) & - return 0 - fi - - # Windows toast (Git Bash) / WSL-on-Windows - if has_cmd powershell.exe; then - # Pass values as args to avoid quoting issues entirely. - # Note: urgency/timeout not used by toast api here; kept for parity. - powershell.exe -NoProfile -ExecutionPolicy Bypass -Command \ - 'param([string]$t,[string]$b) - try { - [Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime] > $null - [Windows.Data.Xml.Dom.XmlDocument, Windows.Data.Xml.Dom.XmlDocument, ContentType = WindowsRuntime] > $null - - function Esc([string]$s) { - if ($null -eq $s) { return "" } - return ($s -replace "&","&" -replace "<","<" -replace ">",">" -replace "\"",""" -replace "'\''","'") - } - - $title = Esc $t - $body = Esc $b - - $xml = New-Object Windows.Data.Xml.Dom.XmlDocument - $xml.LoadXml("$title$body") - $toast = New-Object Windows.UI.Notifications.ToastNotification $xml - [Windows.UI.Notifications.ToastNotificationManager]::CreateToastNotifier("Devtainer").Show($toast) - } catch { }' \ - --% "$title" "$body" >/dev/null 2>&1 || true - - return 0 - fi - - # Fallback - printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 - return 0 - } - - trap - ERR - set +e - set +o pipefail - - local _stop=0 - - _watcher_notify() { - local urgency="${1:-critical}" title="${2:-Notifier}" body="${3:-Watcher event}" - _host_notify 2500 "$urgency" "$title" "$body" - } - - _watcher_int_term() { - _stop=1 - _watcher_notify critical "Notifier" "Notification watcher interrupted/exiting" - printf "%b[watcher]%b Notification watcher interrupted/exiting\n" "$RED" "$NC" >&2 - } - trap _watcher_int_term INT TERM - - local grep_cmd=(grep -a --line-buffered -E "^${prefix}([[:space:]]|$)") - has_cmd stdbuf && grep_cmd=(stdbuf -oL -eL "${grep_cmd[@]}") - - printf "%bNotify Watch:%b monitoring is active. Ctrl+C to stop.\n" "$GREEN" "$NC" - - while ((_stop == 0)); do - if ! docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then - _watcher_notify critical "Notifier" "Watcher stopped: $container is not running" - printf "%b[watcher]%b %s is not running; exiting.\n" "$RED" "$NC" "$container" >&2 - break - fi - - docker logs -f --tail 0 "$container" 2>&1 | - ("${grep_cmd[@]}" || true) | - while IFS=$'\t' read -r _ f1 f2 f3 f4 rest; do - local timeout urgency title body - - if [[ "${f1:-}" =~ ^[0-9]{1,6}$ ]]; then - timeout="$f1" - urgency="${f2:-normal}" - title="${f3:-Notification}" - body="${f4:-}" - else - timeout="2500" - urgency="${f1:-normal}" - title="${f2:-Notification}" - body="${f3:-}" - fi - - [[ -n "${rest:-}" ]] && body+=$'\t'"${rest}" - case "$urgency" in low | normal | critical) ;; *) urgency="normal" ;; esac - - _host_notify "$timeout" "$urgency" "$title" "$body" - printf "%s [%s] %s - %s\n" "$(date '+%Y-%m-%d %H:%M:%S')" "$urgency" "$title" "$body" >&2 - done - - ((_stop)) && break - - if docker inspect -f '{{.State.Running}}' "$container" 2>/dev/null | grep -q true; then - _watcher_notify critical "Notifier" "Watcher lost log stream (docker logs ended). Reconnecting…" - printf "%b[watcher]%b docker logs ended; reconnecting...\n" "$YELLOW" "$NC" >&2 - sleep 1 - continue - fi - - _watcher_notify critical "Notifier" "Watcher stopped: $container stopped" - printf "%b[watcher]%b %s stopped; exiting.\n" "$RED" "$NC" "$container" >&2 - break - done - - trap - INT TERM - set -euo pipefail - - ((_stop)) && return 130 - return 0 -} - -notify_test() { - local title="${1:-Notifier OK}" - local body="${2:-Hello from host via project server-tools container}" - local ctr - ctr="$(_project_tools_container_running || true)" - [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" - docker exec "$ctr" notify -t 2500 -u normal "$title" "$body" -} - -cmd_notify() { - case ${1:-watch} in - watch) notify_watch "${2:-}" ;; - test) notify_test "${2:-Notifier OK}" "${3:-Hello from host}" ;; - *) die "notify " ;; - esac -} - -open_url() { - local url="${1:-}" - [[ -n "$url" ]] || return 0 - - # WSL/Windows helpers first when available - if grep -qi microsoft /proc/version 2>/dev/null; then - if has_cmd powershell.exe; then - powershell.exe -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 || true - return 0 - fi - if has_cmd cmd.exe; then - cmd.exe /c start "" "$url" >/dev/null 2>&1 || true - return 0 - fi - fi - - if has_cmd xdg-open; then - (xdg-open "$url" >/dev/null 2>&1 &) - return 0 - fi - if has_cmd open; then - (open "$url" >/dev/null 2>&1 &) - return 0 - fi - if has_cmd powershell; then - (powershell -NoProfile -Command "Start-Process '$url'" >/dev/null 2>&1 &) - return 0 - fi - - printf "%bINFO%b: open this URL manually → %s\n" "$YELLOW" "$NC" "$url" -} +# Service/rebuild/core/notify implementations are loaded by lib/services.sh. ############################################################################### # RUN (ad-hoc Dockerfile runner) From 682b63552c3f2632708d52dc799381ad2f3bc7c9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:14:36 +0600 Subject: [PATCH 104/509] refactor: extract maintenance runner module --- lib/maintenance.sh | 479 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 479 insertions(+) create mode 100644 lib/maintenance.sh diff --git a/lib/maintenance.sh b/lib/maintenance.sh new file mode 100644 index 00000000..2a32f1ec --- /dev/null +++ b/lib/maintenance.sh @@ -0,0 +1,479 @@ +# shellcheck shell=bash +############################################################################### +# RUN (ad-hoc Dockerfile runner) +############################################################################### +hash_short() { + local s="$1" + if has_cmd sha1sum; then + printf '%s' "$s" | sha1sum | cut -c1-8 + elif has_cmd shasum; then + printf '%s' "$s" | shasum -a 1 | cut -c1-8 + else + # POSIX fallback; stable (not cryptographic) + printf '%s' "$s" | cksum | awk '{print $1}' + fi +} + +run_slug() { + local dir="$1" base hash + base="$(basename "$dir" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9._-' '-')" + hash="$(hash_short "$dir")" + printf '%s-%s' "$base" "$hash" +} + +run_plan() { + local dir="$1" slug + slug="$(run_slug "$dir")" + printf '%s|%s|%s\n' \ + "lds-run-${slug}" \ + "${slug}:local" \ + "$dir" +} + +detect_host_os() { + if is_windows_shell || grep -qi microsoft /proc/version 2>/dev/null; then + printf 'windows' + return 0 + fi + + if has_cmd uname; then + case "$(uname -s 2>/dev/null || true)" in + Darwin) + printf 'macos' + ;; + Linux) + printf 'linux' + ;; + *) + printf 'unknown' + ;; + esac + else + printf 'unknown' + fi +} + +run_find_container() { + local dir="$1" + docker ps -a --filter "label=com.infocyph.lds.run=1" \ + --filter "label=com.infocyph.lds.dir=${dir}" \ + --format '{{.Names}}' | head -n 1 +} + +run_build() { + local tag="$1" dir="$2" + + # Build only if the image doesn't already exist. + if docker image inspect "$tag" >/dev/null 2>&1; then + printf "%b[run]%b Image exists, skipping build: %b%s%b +" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" + return 0 + fi + + printf "%b[run]%b Building image %b%s%b from %s +" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" "$dir" + docker build -t "$tag" "$dir" +} + +run_start() { + local name="$1" tag="$2" dir="$3" keepalive="$4" sock="$5" host_os="$6" + shift 6 || true + + # Remaining args are split by a "--" sentinel: + # - before "--" : publish specs (HOST:CONT), repeatable + # - after "--" : mount specs (HOST[:CONT]), repeatable + local -a pubs=() mounts=() + local seen_delim=0 x + for x in "$@"; do + if [[ "$x" == "--" ]]; then + seen_delim=1 + continue + fi + if ((seen_delim)); then + mounts+=("$x") + else + pubs+=("$x") + fi + done + + # Normalize project dir (POSIX absolute). + # On Windows Git Bash, /e/... is OK; docker.exe will receive converted path automatically. + local dir_posix + dir_posix="$(cd "$dir" 2>/dev/null && pwd -P)" || die "invalid dir: $dir" + + # MSYS-safe container paths: use '//' prefix to prevent path conversion. + # Docker interprets //path as /path inside container. + local WDIR="//workspace" + local WDIR_MOUNT="${dir_posix}://workspace" + + local -a args=(docker run -d --name "$name" + --label "com.infocyph.lds.run=1" + --label "com.infocyph.lds.dir=$dir_posix" + --label "com.infocyph.lds.tag=$tag" + -w "$WDIR" + -v "$WDIR_MOUNT" + ) + + if [[ -n "$host_os" ]]; then + args+=(-e "HOST_OS=$host_os") + fi + + # Mount extra directories/files (HOST[:CONT]). + # - If container path missing, mounts under /mnt/. + # - HOST may be relative to the run directory. + if ((${#mounts[@]})); then + local spec host cont base + for spec in "${mounts[@]}"; do + [[ -n "$spec" ]] || continue + host="$spec" + cont="" + + # Split as HOST:CONT ONLY if suffix after last ':' looks like a container absolute path (/...) + # (safe for Windows drive letters like E:\... because tail won't start with '/') + if [[ "$spec" == *:* ]]; then + local tail="${spec##*:}" + if [[ "$tail" == /* ]]; then + host="${spec%:*}" + cont="$tail" + fi + fi + + # Resolve host to absolute (POSIX) for checks + if [[ "$host" != /* && "$host" != ~* && ! "$host" =~ ^[A-Za-z]:[\\/].* ]]; then + host="${dir_posix%/}/$host" + fi + + # If user provided Windows path (E:\...), convert to POSIX for existence check + if [[ "$host" =~ ^[A-Za-z]:[\\/].* ]] && has_cmd cygpath; then + host="$(cygpath -u "$host")" + fi + + host="$(cd "${host%/*}" 2>/dev/null && pwd -P)/${host##*/}" || { + printf "%b[run]%b Warning: cannot resolve mount path: %s\n" "$YELLOW" "$NC" "$spec" >&2 + continue + } + + [[ -e "$host" ]] || { + printf "%b[run]%b Warning: mount path does not exist: %s\n" "$YELLOW" "$NC" "$host" >&2 + continue + } + + if [[ -z "$cont" ]]; then + base="${host##*/}" + cont="/mnt/${base}" + fi + [[ "$cont" == /* ]] || cont="/mnt/${cont}" + + # Prevent MSYS conversion for container side by using '//' prefix + cont="//${cont#/}" + + args+=(-v "${host}:${cont}") + done + fi + + # Optional docker sock + if [[ "${sock:-0}" == 1 ]]; then + args+=(-v "/var/run/docker.sock:/var/run/docker.sock") + fi + + # Publish ports + local pub + for pub in "${pubs[@]}"; do + [[ -n "$pub" ]] || continue + args+=(-p "$pub") + done + + if [[ "$keepalive" == 1 ]]; then + # Keepalive mode replaces the image command; disable image healthcheck to avoid false "unhealthy". + args+=(--no-healthcheck --entrypoint sh "$tag" -c "trap : TERM INT; sleep infinity & wait") + else + args+=("$tag") + fi + + printf "%b[run]%b Starting container %b%s%b\n" "$CYAN" "$NC" "$BLUE" "$name" "$NC" + + # IMPORTANT: don't hide errors; if it fails, you need to see why + if ! "${args[@]}"; then + printf "%b[run]%b docker run failed.\n" "$RED" "$NC" >&2 + return 1 + fi + printf "\n" +} + +run_exec_shell() { + local name="$1" + if docker exec "$name" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then + exec docker exec -it "$name" bash + else + exec docker exec -it "$name" sh + fi +} + +cmd_run() { + local action="*" dir="$PWD" name="" tag="" nobuild=0 keepalive=1 sock=0 host_os="" + local -a publish=() mounts=() + local open_port="" open_path="/" open_proto="http" + + while [[ $# -gt 0 ]]; do + case "$1" in + stop | rm | ps | shell | logs | open | "*") + action="$1" + shift + ;; + build) + action="*" + shift + ;; + --name) + name="${2:-}" + shift 2 + ;; + --tag) + tag="${2:-}" + shift 2 + ;; + --no-build) + nobuild=1 + shift + ;; + --no-keepalive) + keepalive=0 + shift + ;; + --sock) + sock=1 + shift + ;; + --host-os) + host_os="${2:-}" + shift 2 + ;; + -p | --publish) + publish+=("${2:-}") + shift 2 + ;; + --mount) + mounts+=("${2:-}") + shift 2 + ;; + --port) + open_port="${2:-}" + shift 2 + ;; + --path) + open_path="${2:-/}" + shift 2 + ;; + --https) + open_proto="https" + shift + ;; + --http) + open_proto="http" + shift + ;; + *) break ;; + esac + done + + # Host path (POSIX) for planning/labels + local dir_posix + dir_posix="$(cd "$dir" && pwd -P)" + + if [[ -z "$host_os" ]]; then + host_os="${HOST_OS:-$(detect_host_os)}" + fi + + # Docker path (may need Windows form for docker.exe) + local dir_docker="$dir_posix" + + # Windows Git Bash/MSYS hardening: + # - stop MSYS rewriting container paths (/workspace -> D:/Program Files/Git/workspace) + # - but still feed docker.exe Windows-absolute host paths for build/run contexts + if is_windows_shell; then + export MSYS_NO_PATHCONV=1 + export MSYS2_ARG_CONV_EXCL='*' + if has_cmd cygpath; then + dir_docker="$(cygpath -w "$dir_posix")" + fi + fi + + # Plan/name/tag should be based on the real project identity (POSIX dir) + IFS='|' read -r def_name def_tag _def_dir < <(run_plan "$dir_posix") + name="${name:-$def_name}" + + # Tag rules: + # - Default tag is ":local" (from run_plan) + # - If user passes --tag without ":", append ":local" + if [[ -n "${tag:-}" ]]; then + if [[ "$tag" != *:* ]]; then + tag="${tag}:local" + fi + else + tag="$def_tag" + fi + + _find_for_dir() { + local found + found="$(run_find_container "$dir_posix" || true)" + if [[ -n "$found" ]]; then + printf '%s' "$found" + return 0 + fi + if docker inspect "$name" >/dev/null 2>&1; then + printf '%s' "$name" + return 0 + fi + return 1 + } + + _run_build_summary() { + local img="$1" build_dir="$2" cname="$3" + local tag_only="${img##*:}" + + printf "\n%b[run]%b Build summary\n" "$CYAN" "$NC" + printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" + printf " %bTag:%b %s\n" "$BOLD" "$NC" "$tag_only" + printf " %bDir:%b %s\n" "$BOLD" "$NC" "$build_dir" + printf " %bName:%b %s\n" "$BOLD" "$NC" "$cname" + printf " %bKeepalive:%b %s\n" "$BOLD" "$NC" "$keepalive" + printf " %bSock:%b %s\n" "$BOLD" "$NC" "$sock" + printf " %bHost OS:%b %s\n" "$BOLD" "$NC" "$host_os" + + if ((${#publish[@]})); then + printf " %bPublish:%b %s\n" "$BOLD" "$NC" "${publish[*]}" + else + printf " %bPublish:%b (none)\n" "$BOLD" "$NC" + fi + + if ((${#mounts[@]})); then + printf " %bMounts:%b %s\n" "$BOLD" "$NC" "${mounts[*]}" + else + printf " %bMounts:%b (none)\n" "$BOLD" "$NC" + fi + printf "\n" + } + + _run_runtime_summary() { + local cname="$1" + local id img state ports + id="$(docker inspect -f '{{.Id}}' "$cname" 2>/dev/null | cut -c1-12 || true)" + img="$(docker inspect -f '{{.Config.Image}}' "$cname" 2>/dev/null || true)" + state="$(docker inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || true)" + ports="$(docker port "$cname" 2>/dev/null | sed '/^[[:space:]]*$/d' | tr '\n' '; ' | sed 's/; $//' || true)" + + printf "%b[run]%b Runtime summary\n" "$CYAN" "$NC" + printf " %bContainer:%b %s\n" "$BOLD" "$NC" "${cname}${id:+ ($id)}" + [[ -n "$img" ]] && printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" + [[ -n "$state" ]] && printf " %bState:%b %s\n" "$BOLD" "$NC" "$state" + if [[ -n "$ports" ]]; then + printf " %bPorts:%b %s\n" "$BOLD" "$NC" "$ports" + else + printf " %bPorts:%b (none published)\n" "$BOLD" "$NC" + fi + printf "%b\n[run]%b Example Usage (in Composer)\n" "$CYAN" "$NC" + printf " %bimage:%b %s\n" "$BOLD" "$NC" "$img" + printf " %bpull_policy:%b never\n" "$BOLD" "$NC" + printf "\n" + } + + case "$action" in + ps) + docker ps -a --filter "label=com.infocyph.lds.run=1" \ + --format 'table {{.Names}} {{.Image}} {{.Status}} {{.Labels}}' + return 0 + ;; + stop) + local existing + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + docker stop "$existing" >/dev/null + printf "%b[run]%b Stopped %s\n" "$GREEN" "$NC" "$existing" + return 0 + ;; + logs) + local existing + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + exec docker logs -f "$existing" + ;; + open) + local existing line addr hp url + existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" + + [[ -n "$open_path" ]] || open_path="/" + [[ "$open_path" == /* ]] || open_path="/$open_path" + + if [[ -n "$open_port" ]]; then + line="$(docker port "$existing" "$open_port" 2>/dev/null | head -n 1 || true)" + [[ -n "$line" ]] || line="$(docker port "$existing" "${open_port}/tcp" 2>/dev/null | head -n 1 || true)" + else + line="$(docker port "$existing" 2>/dev/null | head -n 1 || true)" + fi + + if [[ -z "$line" ]]; then + printf "%b[run]%b No published ports found.\n" "$YELLOW" "$NC" + printf "%b[run]%b Tip: start with %blds run --publish 8025:8025%b then %blds run open%b\n" \ + "$YELLOW" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" + return 1 + fi + + addr="${line##*-> }" + hp="${addr##*:}" + url="${open_proto}://localhost:${hp}${open_path}" + open_url "$url" + printf "%b[run]%b Opened: %s\n" "$GREEN" "$NC" "$url" + return 0 + ;; + rm) + local existing img + existing="$(_find_for_dir)" || true + if [[ -n "${existing:-}" ]]; then + img="$(docker inspect -f '{{.Config.Image}}' "$existing" 2>/dev/null || true)" + docker stop "$existing" >/dev/null 2>&1 || true + docker rm "$existing" >/dev/null 2>&1 || true + printf "%b[run]%b Removed container %s\n" "$GREEN" "$NC" "$existing" + if [[ -n "${img:-}" ]]; then + docker rmi -f "$img" >/dev/null 2>&1 || true + printf "%b[run]%b Removed image %s\n" "$GREEN" "$NC" "$img" + fi + else + printf "%b[run]%b No container found for %s\n" "$YELLOW" "$NC" "$dir_posix" + fi + return 0 + ;; + shell | "*") + if ((nobuild == 0)); then + # Build needs docker.exe-friendly path on Windows + run_build "$tag" "$dir_docker" + else + printf "%b[run]%b Skipping build (--no-build)\n" "$YELLOW" "$NC" + fi + + _run_build_summary "$tag" "$dir_posix" "$name" + + if docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null | grep -q true; then + printf "%b[run]%b Container already running: %s\n\n" "$GREEN" "$NC" "$name" + else + if docker inspect "$name" >/dev/null 2>&1; then + docker rm -f "$name" >/dev/null 2>&1 || true + fi + + # Keep mounts as user gave them (POSIX/relative); run_start should validate POSIX + # and convert host-side to Windows only at docker run time. + run_start "$name" "$tag" "$dir_docker" "$keepalive" "$sock" "$host_os" \ + "${publish[@]}" -- "${mounts[@]}" + fi + + _run_runtime_summary "$name" + + # "shell" enters the container; "*" / "build" does not. + if [[ "$action" == "shell" ]]; then + run_exec_shell "$name" + else + printf "%b[run]%b Built/started. Use %blds run shell%b to enter, %blds run logs%b to follow logs.\n" \ + "$GREEN" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" + return 0 + fi + ;; + esac +} + From f42d3225581e0775152b6e2b796bc7bf2dfebc8a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:14:40 +0600 Subject: [PATCH 105/509] refactor: move diff and support diagnostics into module --- lib/diagnostics.sh | 248 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 248 insertions(+) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 4402c342..c5a7f9b2 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -54,3 +54,251 @@ cmd_sniff() { _tools_exec "curl -vk -D - $(_shq "$url") ${qargs[*]} | (command -v jq >/dev/null 2>&1 && jq . 2>/dev/null || cat)" } + +############################################################################### +# 6w. NEW FEATURES: stack diff | support trace +############################################################################### + +# stack diff: show what would run (compose) vs what's running (docker) +cmd_stack_diff() { + local json=0 + local show_config=0 + while [[ "${1:-}" ]]; do + case "$1" in + --json) + json=1 + shift + ;; + --config) + show_config=1 + shift + ;; + *) break ;; + esac + done + + local project + project="$(lds_project)" + local cfg_json="" + + if docker_compose config --format json >/dev/null 2>&1; then + cfg_json="$(docker_compose config --format json)" + else + # fallback: best-effort text config + cfg_json="" + fi + + # running: service -> image + declare -A running=() + local line + while IFS= read -r line; do + [[ -n "$line" ]] || continue + local svc="${line%%|*}" + local img="${line#*|}" + running["$svc"]="$img" + done < <(docker ps \ + --filter "label=com.docker.compose.project=$project" \ + --format '{{index .Labels "com.docker.compose.service"}}|{{.Image}}' 2>/dev/null || true) + + # desired: service -> image/build context (best-effort) + declare -A desired_img=() + declare -A desired_ctx=() + declare -A desired_df=() + + if [[ -n "$cfg_json" ]]; then + if has_tool jq; then + while IFS= read -r line; do + local svc="${line%%|*}" + local img="${line#*|}" + desired_img["$svc"]="$img" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"') + while IFS= read -r line; do + local svc="${line%%|*}" + local ctx="${line#*|}" + desired_ctx["$svc"]="$ctx" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"') + while IFS= read -r line; do + local svc="${line%%|*}" + local df="${line#*|}" + desired_df["$svc"]="$df" + done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"') + elif _server_tools_has jq; then + # Fallback: parse via project tools container jq through stdin (no shell re-quoting of JSON payload). + local ctr + ctr="$(_project_tools_container_running || true)" + if [[ -n "$ctr" ]]; then + while IFS= read -r line; do + local svc="${line%%|*}" + local img="${line#*|}" + desired_img["$svc"]="$img" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"' 2>/dev/null || true) + while IFS= read -r line; do + local svc="${line%%|*}" + local ctx="${line#*|}" + desired_ctx["$svc"]="$ctx" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"' 2>/dev/null || true) + while IFS= read -r line; do + local svc="${line%%|*}" + local df="${line#*|}" + desired_df["$svc"]="$df" + done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"' 2>/dev/null || true) + fi + fi + fi + + # Build result object + if ((json)); then + if has_tool jq; then + # assemble in bash -> jq + local tmp + tmp="$(mktemp)" + { + printf '{' + printf '"project":%s,' "$(printf '%s' "$project" | jq -Rsa .)" + printf '"compose_file":%s,' "$(printf '%s' "$COMPOSE_FILE" | jq -Rsa .)" + printf '"running":{' + local first=1 k + for k in "${!running[@]}"; do + ((first)) || printf ',' + first=0 + printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${running[$k]}" | jq -R .)" + done + printf '},' + printf '"desired":{' + first=1 + for k in "${!desired_img[@]}"; do + ((first)) || printf ',' + first=0 + printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${desired_img[$k]}" | jq -R .)" + done + printf '},' + printf '"diff":[' + first=1 + # union keys + declare -A seen=() + for k in "${!running[@]}"; do seen["$k"]=1; done + for k in "${!desired_img[@]}"; do seen["$k"]=1; done + for k in "${!seen[@]}"; do + local r="${running[$k]:-}" + local d="${desired_img[$k]:-}" + if [[ "$r" != "$d" ]]; then + ((first)) || printf ',' + first=0 + printf '{"service":%s,"running":%s,"desired":%s}' \ + "$(printf '%s' "$k" | jq -R .)" \ + "$(printf '%s' "$r" | jq -R .)" \ + "$(printf '%s' "$d" | jq -R .)" + fi + done + printf ']' + printf '}\n' + } >"$tmp" + cat "$tmp" | jq . + rm -f "$tmp" + else + die "jq required for --json (or run inside project server-tools container)" + fi + return 0 + fi + + printf "%bStack diff%b (project=%s)\n" "$CYAN" "$NC" "$project" + printf "%bCompose file:%b %s\n" "$DIM" "$NC" "$COMPOSE_FILE" + + if ((show_config)); then + if [[ -n "$cfg_json" ]]; then + printf "\n%bEffective compose config (json):%b\n" "$DIM" "$NC" + printf '%s\n' "$cfg_json" + else + printf "\n%bEffective compose config:%b\n" "$DIM" "$NC" + docker_compose config || true + fi + fi + + # union services + declare -A all=() + local svc + for svc in "${!running[@]}"; do all["$svc"]=1; done + for svc in "${!desired_img[@]}"; do all["$svc"]=1; done + + printf "\n%-22s %-40s %-40s %s\n" "SERVICE" "RUNNING" "DESIRED" "STATUS" + printf "%-22s %-40s %-40s %s\n" "------" "-------" "-------" "------" + for svc in $(printf '%s\n' "${!all[@]}" | sort); do + local r="${running[$svc]:-}" + local d="${desired_img[$svc]:-}" + local st + if [[ -z "$r" ]]; then + st="(not running)" + elif [[ -z "$d" ]]; then + st="(not in config)" + elif [[ "$r" == "$d" ]]; then + st="OK" + else + st="DIFF" + fi + printf "%-22s %-40.40s %-40.40s %s\n" "$svc" "$r" "$d" "$st" + done + + printf "\n%bNotes:%b\n" "$DIM" "$NC" + printf " - Desired image is derived from 'docker compose config'. If a service uses only 'build:' and no 'image:', desired may be empty.\n" + printf " - Use: lds stack diff --config (to print resolved compose config)\n" +} + +# support trace: quick end-to-end trace for a domain +cmd_support_trace() { + local dom="${1:-}" + [[ -n "$dom" ]] || die "support trace " + + local nconf="$DIR/configuration/nginx/$dom.conf" + printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" + + # 1) DNS + if _server_tools_running; then + printf "\n%b[DNS]%b\n" "$DIM" "$NC" + _tools_exec "dig +short $(_shq "$dom") || true; getent hosts $(_shq "$dom") 2>/dev/null || true" + else + printf "\n%b[DNS]%b\n" "$DIM" "$NC" + (has_cmd dig && dig +short "$dom") || true + (has_cmd getent && getent hosts "$dom") || true + fi + + # 2) TLS certificate + printf "\n%b[TLS]%b\n" "$DIM" "$NC" + if _server_tools_running; then + _tools_exec "echo | openssl s_client -connect $(_shq "$dom"):443 -servername $(_shq "$dom") -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true" + else + echo | openssl s_client -connect "${dom}:443" -servername "$dom" -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true + fi + + # 3) HTTP probe (timings) + printf "\n%b[HTTP]%b\n" "$DIM" "$NC" + if _server_tools_running; then + _tools_exec "curl -sk -o /dev/null -D - -w 'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' https://$(_shq "$dom") | sed -n '1,30p'" + else + curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' + fi + + # 4) Upstream inference from nginx conf (if exists) + printf "\n%b[Upstream]%b\n" "$DIM" "$NC" + if [[ -r "$nconf" ]]; then + if grep -q fastcgi_pass "$nconf"; then + local php + php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" + elif grep -q proxy_pass "$nconf"; then + local up + up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" + else + printf "type=static\n" + fi + else + printf "nginx_conf=%s (missing)\n" "$nconf" + fi + + # 5) Recent nginx logs (compose) + printf "\n%b[Recent nginx logs]%b\n" "$DIM" "$NC" + docker_compose logs --no-color --tail 120 nginx 2>/dev/null | text_grep -i "$dom" || docker_compose logs --no-color --tail 120 nginx 2>/dev/null || true + + printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" +} + From d73580e04efe1a39b2704389b53e259d9ff042c9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:14:45 +0600 Subject: [PATCH 106/509] refactor: source maintenance and extended diagnostics modules --- lds | 725 +----------------------------------------------------------- 1 file changed, 4 insertions(+), 721 deletions(-) diff --git a/lds b/lds index 70c8c348..0bb82643 100755 --- a/lds +++ b/lds @@ -497,729 +497,12 @@ source "$DIR/lib/diagnostics.sh" # Service/rebuild/core/notify implementations are loaded by lib/services.sh. ############################################################################### -# RUN (ad-hoc Dockerfile runner) +# RUN / MAINTENANCE ############################################################################### -hash_short() { - local s="$1" - if has_cmd sha1sum; then - printf '%s' "$s" | sha1sum | cut -c1-8 - elif has_cmd shasum; then - printf '%s' "$s" | shasum -a 1 | cut -c1-8 - else - # POSIX fallback; stable (not cryptographic) - printf '%s' "$s" | cksum | awk '{print $1}' - fi -} - -run_slug() { - local dir="$1" base hash - base="$(basename "$dir" | tr '[:upper:]' '[:lower:]' | tr -cs 'a-z0-9._-' '-')" - hash="$(hash_short "$dir")" - printf '%s-%s' "$base" "$hash" -} - -run_plan() { - local dir="$1" slug - slug="$(run_slug "$dir")" - printf '%s|%s|%s\n' \ - "lds-run-${slug}" \ - "${slug}:local" \ - "$dir" -} - -detect_host_os() { - if is_windows_shell || grep -qi microsoft /proc/version 2>/dev/null; then - printf 'windows' - return 0 - fi - - if has_cmd uname; then - case "$(uname -s 2>/dev/null || true)" in - Darwin) - printf 'macos' - ;; - Linux) - printf 'linux' - ;; - *) - printf 'unknown' - ;; - esac - else - printf 'unknown' - fi -} - -run_find_container() { - local dir="$1" - docker ps -a --filter "label=com.infocyph.lds.run=1" \ - --filter "label=com.infocyph.lds.dir=${dir}" \ - --format '{{.Names}}' | head -n 1 -} - -run_build() { - local tag="$1" dir="$2" - - # Build only if the image doesn't already exist. - if docker image inspect "$tag" >/dev/null 2>&1; then - printf "%b[run]%b Image exists, skipping build: %b%s%b -" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" - return 0 - fi - - printf "%b[run]%b Building image %b%s%b from %s -" "$CYAN" "$NC" "$BLUE" "$tag" "$NC" "$dir" - docker build -t "$tag" "$dir" -} - -run_start() { - local name="$1" tag="$2" dir="$3" keepalive="$4" sock="$5" host_os="$6" - shift 6 || true - - # Remaining args are split by a "--" sentinel: - # - before "--" : publish specs (HOST:CONT), repeatable - # - after "--" : mount specs (HOST[:CONT]), repeatable - local -a pubs=() mounts=() - local seen_delim=0 x - for x in "$@"; do - if [[ "$x" == "--" ]]; then - seen_delim=1 - continue - fi - if ((seen_delim)); then - mounts+=("$x") - else - pubs+=("$x") - fi - done - - # Normalize project dir (POSIX absolute). - # On Windows Git Bash, /e/... is OK; docker.exe will receive converted path automatically. - local dir_posix - dir_posix="$(cd "$dir" 2>/dev/null && pwd -P)" || die "invalid dir: $dir" - - # MSYS-safe container paths: use '//' prefix to prevent path conversion. - # Docker interprets //path as /path inside container. - local WDIR="//workspace" - local WDIR_MOUNT="${dir_posix}://workspace" - - local -a args=(docker run -d --name "$name" - --label "com.infocyph.lds.run=1" - --label "com.infocyph.lds.dir=$dir_posix" - --label "com.infocyph.lds.tag=$tag" - -w "$WDIR" - -v "$WDIR_MOUNT" - ) - - if [[ -n "$host_os" ]]; then - args+=(-e "HOST_OS=$host_os") - fi - - # Mount extra directories/files (HOST[:CONT]). - # - If container path missing, mounts under /mnt/. - # - HOST may be relative to the run directory. - if ((${#mounts[@]})); then - local spec host cont base - for spec in "${mounts[@]}"; do - [[ -n "$spec" ]] || continue - host="$spec" - cont="" - - # Split as HOST:CONT ONLY if suffix after last ':' looks like a container absolute path (/...) - # (safe for Windows drive letters like E:\... because tail won't start with '/') - if [[ "$spec" == *:* ]]; then - local tail="${spec##*:}" - if [[ "$tail" == /* ]]; then - host="${spec%:*}" - cont="$tail" - fi - fi - - # Resolve host to absolute (POSIX) for checks - if [[ "$host" != /* && "$host" != ~* && ! "$host" =~ ^[A-Za-z]:[\\/].* ]]; then - host="${dir_posix%/}/$host" - fi - - # If user provided Windows path (E:\...), convert to POSIX for existence check - if [[ "$host" =~ ^[A-Za-z]:[\\/].* ]] && has_cmd cygpath; then - host="$(cygpath -u "$host")" - fi - - host="$(cd "${host%/*}" 2>/dev/null && pwd -P)/${host##*/}" || { - printf "%b[run]%b Warning: cannot resolve mount path: %s\n" "$YELLOW" "$NC" "$spec" >&2 - continue - } - - [[ -e "$host" ]] || { - printf "%b[run]%b Warning: mount path does not exist: %s\n" "$YELLOW" "$NC" "$host" >&2 - continue - } - - if [[ -z "$cont" ]]; then - base="${host##*/}" - cont="/mnt/${base}" - fi - [[ "$cont" == /* ]] || cont="/mnt/${cont}" - - # Prevent MSYS conversion for container side by using '//' prefix - cont="//${cont#/}" - - args+=(-v "${host}:${cont}") - done - fi - - # Optional docker sock - if [[ "${sock:-0}" == 1 ]]; then - args+=(-v "/var/run/docker.sock:/var/run/docker.sock") - fi - - # Publish ports - local pub - for pub in "${pubs[@]}"; do - [[ -n "$pub" ]] || continue - args+=(-p "$pub") - done - - if [[ "$keepalive" == 1 ]]; then - # Keepalive mode replaces the image command; disable image healthcheck to avoid false "unhealthy". - args+=(--no-healthcheck --entrypoint sh "$tag" -c "trap : TERM INT; sleep infinity & wait") - else - args+=("$tag") - fi - - printf "%b[run]%b Starting container %b%s%b\n" "$CYAN" "$NC" "$BLUE" "$name" "$NC" - - # IMPORTANT: don't hide errors; if it fails, you need to see why - if ! "${args[@]}"; then - printf "%b[run]%b docker run failed.\n" "$RED" "$NC" >&2 - return 1 - fi - printf "\n" -} - -run_exec_shell() { - local name="$1" - if docker exec "$name" sh -lc 'command -v bash >/dev/null 2>&1' >/dev/null 2>&1; then - exec docker exec -it "$name" bash - else - exec docker exec -it "$name" sh - fi -} - -cmd_run() { - local action="*" dir="$PWD" name="" tag="" nobuild=0 keepalive=1 sock=0 host_os="" - local -a publish=() mounts=() - local open_port="" open_path="/" open_proto="http" - - while [[ $# -gt 0 ]]; do - case "$1" in - stop | rm | ps | shell | logs | open | "*") - action="$1" - shift - ;; - build) - action="*" - shift - ;; - --name) - name="${2:-}" - shift 2 - ;; - --tag) - tag="${2:-}" - shift 2 - ;; - --no-build) - nobuild=1 - shift - ;; - --no-keepalive) - keepalive=0 - shift - ;; - --sock) - sock=1 - shift - ;; - --host-os) - host_os="${2:-}" - shift 2 - ;; - -p | --publish) - publish+=("${2:-}") - shift 2 - ;; - --mount) - mounts+=("${2:-}") - shift 2 - ;; - --port) - open_port="${2:-}" - shift 2 - ;; - --path) - open_path="${2:-/}" - shift 2 - ;; - --https) - open_proto="https" - shift - ;; - --http) - open_proto="http" - shift - ;; - *) break ;; - esac - done - - # Host path (POSIX) for planning/labels - local dir_posix - dir_posix="$(cd "$dir" && pwd -P)" - - if [[ -z "$host_os" ]]; then - host_os="${HOST_OS:-$(detect_host_os)}" - fi - - # Docker path (may need Windows form for docker.exe) - local dir_docker="$dir_posix" - - # Windows Git Bash/MSYS hardening: - # - stop MSYS rewriting container paths (/workspace -> D:/Program Files/Git/workspace) - # - but still feed docker.exe Windows-absolute host paths for build/run contexts - if is_windows_shell; then - export MSYS_NO_PATHCONV=1 - export MSYS2_ARG_CONV_EXCL='*' - if has_cmd cygpath; then - dir_docker="$(cygpath -w "$dir_posix")" - fi - fi - - # Plan/name/tag should be based on the real project identity (POSIX dir) - IFS='|' read -r def_name def_tag _def_dir < <(run_plan "$dir_posix") - name="${name:-$def_name}" - - # Tag rules: - # - Default tag is ":local" (from run_plan) - # - If user passes --tag without ":", append ":local" - if [[ -n "${tag:-}" ]]; then - if [[ "$tag" != *:* ]]; then - tag="${tag}:local" - fi - else - tag="$def_tag" - fi - - _find_for_dir() { - local found - found="$(run_find_container "$dir_posix" || true)" - if [[ -n "$found" ]]; then - printf '%s' "$found" - return 0 - fi - if docker inspect "$name" >/dev/null 2>&1; then - printf '%s' "$name" - return 0 - fi - return 1 - } - - _run_build_summary() { - local img="$1" build_dir="$2" cname="$3" - local tag_only="${img##*:}" - - printf "\n%b[run]%b Build summary\n" "$CYAN" "$NC" - printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" - printf " %bTag:%b %s\n" "$BOLD" "$NC" "$tag_only" - printf " %bDir:%b %s\n" "$BOLD" "$NC" "$build_dir" - printf " %bName:%b %s\n" "$BOLD" "$NC" "$cname" - printf " %bKeepalive:%b %s\n" "$BOLD" "$NC" "$keepalive" - printf " %bSock:%b %s\n" "$BOLD" "$NC" "$sock" - printf " %bHost OS:%b %s\n" "$BOLD" "$NC" "$host_os" - - if ((${#publish[@]})); then - printf " %bPublish:%b %s\n" "$BOLD" "$NC" "${publish[*]}" - else - printf " %bPublish:%b (none)\n" "$BOLD" "$NC" - fi - - if ((${#mounts[@]})); then - printf " %bMounts:%b %s\n" "$BOLD" "$NC" "${mounts[*]}" - else - printf " %bMounts:%b (none)\n" "$BOLD" "$NC" - fi - printf "\n" - } - - _run_runtime_summary() { - local cname="$1" - local id img state ports - id="$(docker inspect -f '{{.Id}}' "$cname" 2>/dev/null | cut -c1-12 || true)" - img="$(docker inspect -f '{{.Config.Image}}' "$cname" 2>/dev/null || true)" - state="$(docker inspect -f '{{.State.Status}}' "$cname" 2>/dev/null || true)" - ports="$(docker port "$cname" 2>/dev/null | sed '/^[[:space:]]*$/d' | tr '\n' '; ' | sed 's/; $//' || true)" - - printf "%b[run]%b Runtime summary\n" "$CYAN" "$NC" - printf " %bContainer:%b %s\n" "$BOLD" "$NC" "${cname}${id:+ ($id)}" - [[ -n "$img" ]] && printf " %bImage:%b %s\n" "$BOLD" "$NC" "$img" - [[ -n "$state" ]] && printf " %bState:%b %s\n" "$BOLD" "$NC" "$state" - if [[ -n "$ports" ]]; then - printf " %bPorts:%b %s\n" "$BOLD" "$NC" "$ports" - else - printf " %bPorts:%b (none published)\n" "$BOLD" "$NC" - fi - printf "%b\n[run]%b Example Usage (in Composer)\n" "$CYAN" "$NC" - printf " %bimage:%b %s\n" "$BOLD" "$NC" "$img" - printf " %bpull_policy:%b never\n" "$BOLD" "$NC" - printf "\n" - } - - case "$action" in - ps) - docker ps -a --filter "label=com.infocyph.lds.run=1" \ - --format 'table {{.Names}} {{.Image}} {{.Status}} {{.Labels}}' - return 0 - ;; - stop) - local existing - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - docker stop "$existing" >/dev/null - printf "%b[run]%b Stopped %s\n" "$GREEN" "$NC" "$existing" - return 0 - ;; - logs) - local existing - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - exec docker logs -f "$existing" - ;; - open) - local existing line addr hp url - existing="$(_find_for_dir)" || die "no run container found for: $dir_posix" - - [[ -n "$open_path" ]] || open_path="/" - [[ "$open_path" == /* ]] || open_path="/$open_path" - - if [[ -n "$open_port" ]]; then - line="$(docker port "$existing" "$open_port" 2>/dev/null | head -n 1 || true)" - [[ -n "$line" ]] || line="$(docker port "$existing" "${open_port}/tcp" 2>/dev/null | head -n 1 || true)" - else - line="$(docker port "$existing" 2>/dev/null | head -n 1 || true)" - fi - - if [[ -z "$line" ]]; then - printf "%b[run]%b No published ports found.\n" "$YELLOW" "$NC" - printf "%b[run]%b Tip: start with %blds run --publish 8025:8025%b then %blds run open%b\n" \ - "$YELLOW" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" - return 1 - fi - - addr="${line##*-> }" - hp="${addr##*:}" - url="${open_proto}://localhost:${hp}${open_path}" - open_url "$url" - printf "%b[run]%b Opened: %s\n" "$GREEN" "$NC" "$url" - return 0 - ;; - rm) - local existing img - existing="$(_find_for_dir)" || true - if [[ -n "${existing:-}" ]]; then - img="$(docker inspect -f '{{.Config.Image}}' "$existing" 2>/dev/null || true)" - docker stop "$existing" >/dev/null 2>&1 || true - docker rm "$existing" >/dev/null 2>&1 || true - printf "%b[run]%b Removed container %s\n" "$GREEN" "$NC" "$existing" - if [[ -n "${img:-}" ]]; then - docker rmi -f "$img" >/dev/null 2>&1 || true - printf "%b[run]%b Removed image %s\n" "$GREEN" "$NC" "$img" - fi - else - printf "%b[run]%b No container found for %s\n" "$YELLOW" "$NC" "$dir_posix" - fi - return 0 - ;; - shell | "*") - if ((nobuild == 0)); then - # Build needs docker.exe-friendly path on Windows - run_build "$tag" "$dir_docker" - else - printf "%b[run]%b Skipping build (--no-build)\n" "$YELLOW" "$NC" - fi - - _run_build_summary "$tag" "$dir_posix" "$name" - - if docker inspect -f '{{.State.Running}}' "$name" 2>/dev/null | grep -q true; then - printf "%b[run]%b Container already running: %s\n\n" "$GREEN" "$NC" "$name" - else - if docker inspect "$name" >/dev/null 2>&1; then - docker rm -f "$name" >/dev/null 2>&1 || true - fi - - # Keep mounts as user gave them (POSIX/relative); run_start should validate POSIX - # and convert host-side to Windows only at docker run time. - run_start "$name" "$tag" "$dir_docker" "$keepalive" "$sock" "$host_os" \ - "${publish[@]}" -- "${mounts[@]}" - fi - - _run_runtime_summary "$name" - - # "shell" enters the container; "*" / "build" does not. - if [[ "$action" == "shell" ]]; then - run_exec_shell "$name" - else - printf "%b[run]%b Built/started. Use %blds run shell%b to enter, %blds run logs%b to follow logs.\n" \ - "$GREEN" "$NC" "$BLUE" "$NC" "$BLUE" "$NC" - return 0 - fi - ;; - esac -} - -############################################################################### -# 6w. NEW FEATURES: stack diff | support trace -############################################################################### - -# stack diff: show what would run (compose) vs what's running (docker) -cmd_stack_diff() { - local json=0 - local show_config=0 - while [[ "${1:-}" ]]; do - case "$1" in - --json) - json=1 - shift - ;; - --config) - show_config=1 - shift - ;; - *) break ;; - esac - done +# shellcheck source=lib/maintenance.sh +source "$DIR/lib/maintenance.sh" - local project - project="$(lds_project)" - local cfg_json="" - - if docker_compose config --format json >/dev/null 2>&1; then - cfg_json="$(docker_compose config --format json)" - else - # fallback: best-effort text config - cfg_json="" - fi - - # running: service -> image - declare -A running=() - local line - while IFS= read -r line; do - [[ -n "$line" ]] || continue - local svc="${line%%|*}" - local img="${line#*|}" - running["$svc"]="$img" - done < <(docker ps \ - --filter "label=com.docker.compose.project=$project" \ - --format '{{index .Labels "com.docker.compose.service"}}|{{.Image}}' 2>/dev/null || true) - - # desired: service -> image/build context (best-effort) - declare -A desired_img=() - declare -A desired_ctx=() - declare -A desired_df=() - - if [[ -n "$cfg_json" ]]; then - if has_tool jq; then - while IFS= read -r line; do - local svc="${line%%|*}" - local img="${line#*|}" - desired_img["$svc"]="$img" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"') - while IFS= read -r line; do - local svc="${line%%|*}" - local ctx="${line#*|}" - desired_ctx["$svc"]="$ctx" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"') - while IFS= read -r line; do - local svc="${line%%|*}" - local df="${line#*|}" - desired_df["$svc"]="$df" - done < <(printf '%s' "$cfg_json" | jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"') - elif _server_tools_has jq; then - # Fallback: parse via project tools container jq through stdin (no shell re-quoting of JSON payload). - local ctr - ctr="$(_project_tools_container_running || true)" - if [[ -n "$ctr" ]]; then - while IFS= read -r line; do - local svc="${line%%|*}" - local img="${line#*|}" - desired_img["$svc"]="$img" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.image // "")"' 2>/dev/null || true) - while IFS= read -r line; do - local svc="${line%%|*}" - local ctx="${line#*|}" - desired_ctx["$svc"]="$ctx" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.context // "")"' 2>/dev/null || true) - while IFS= read -r line; do - local svc="${line%%|*}" - local df="${line#*|}" - desired_df["$svc"]="$df" - done < <(printf '%s' "$cfg_json" | docker exec -i "$ctr" jq -r '.services | to_entries[] | "\(.key)|\(.value.build.dockerfile // "")"' 2>/dev/null || true) - fi - fi - fi - - # Build result object - if ((json)); then - if has_tool jq; then - # assemble in bash -> jq - local tmp - tmp="$(mktemp)" - { - printf '{' - printf '"project":%s,' "$(printf '%s' "$project" | jq -Rsa .)" - printf '"compose_file":%s,' "$(printf '%s' "$COMPOSE_FILE" | jq -Rsa .)" - printf '"running":{' - local first=1 k - for k in "${!running[@]}"; do - ((first)) || printf ',' - first=0 - printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${running[$k]}" | jq -R .)" - done - printf '},' - printf '"desired":{' - first=1 - for k in "${!desired_img[@]}"; do - ((first)) || printf ',' - first=0 - printf '%s:%s' "$(printf '%s' "$k" | jq -R .)" "$(printf '%s' "${desired_img[$k]}" | jq -R .)" - done - printf '},' - printf '"diff":[' - first=1 - # union keys - declare -A seen=() - for k in "${!running[@]}"; do seen["$k"]=1; done - for k in "${!desired_img[@]}"; do seen["$k"]=1; done - for k in "${!seen[@]}"; do - local r="${running[$k]:-}" - local d="${desired_img[$k]:-}" - if [[ "$r" != "$d" ]]; then - ((first)) || printf ',' - first=0 - printf '{"service":%s,"running":%s,"desired":%s}' \ - "$(printf '%s' "$k" | jq -R .)" \ - "$(printf '%s' "$r" | jq -R .)" \ - "$(printf '%s' "$d" | jq -R .)" - fi - done - printf ']' - printf '}\n' - } >"$tmp" - cat "$tmp" | jq . - rm -f "$tmp" - else - die "jq required for --json (or run inside project server-tools container)" - fi - return 0 - fi - - printf "%bStack diff%b (project=%s)\n" "$CYAN" "$NC" "$project" - printf "%bCompose file:%b %s\n" "$DIM" "$NC" "$COMPOSE_FILE" - - if ((show_config)); then - if [[ -n "$cfg_json" ]]; then - printf "\n%bEffective compose config (json):%b\n" "$DIM" "$NC" - printf '%s\n' "$cfg_json" - else - printf "\n%bEffective compose config:%b\n" "$DIM" "$NC" - docker_compose config || true - fi - fi - - # union services - declare -A all=() - local svc - for svc in "${!running[@]}"; do all["$svc"]=1; done - for svc in "${!desired_img[@]}"; do all["$svc"]=1; done - - printf "\n%-22s %-40s %-40s %s\n" "SERVICE" "RUNNING" "DESIRED" "STATUS" - printf "%-22s %-40s %-40s %s\n" "------" "-------" "-------" "------" - for svc in $(printf '%s\n' "${!all[@]}" | sort); do - local r="${running[$svc]:-}" - local d="${desired_img[$svc]:-}" - local st - if [[ -z "$r" ]]; then - st="(not running)" - elif [[ -z "$d" ]]; then - st="(not in config)" - elif [[ "$r" == "$d" ]]; then - st="OK" - else - st="DIFF" - fi - printf "%-22s %-40.40s %-40.40s %s\n" "$svc" "$r" "$d" "$st" - done - - printf "\n%bNotes:%b\n" "$DIM" "$NC" - printf " - Desired image is derived from 'docker compose config'. If a service uses only 'build:' and no 'image:', desired may be empty.\n" - printf " - Use: lds stack diff --config (to print resolved compose config)\n" -} - -# support trace: quick end-to-end trace for a domain -cmd_support_trace() { - local dom="${1:-}" - [[ -n "$dom" ]] || die "support trace " - - local nconf="$DIR/configuration/nginx/$dom.conf" - printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" - - # 1) DNS - if _server_tools_running; then - printf "\n%b[DNS]%b\n" "$DIM" "$NC" - _tools_exec "dig +short $(_shq "$dom") || true; getent hosts $(_shq "$dom") 2>/dev/null || true" - else - printf "\n%b[DNS]%b\n" "$DIM" "$NC" - (has_cmd dig && dig +short "$dom") || true - (has_cmd getent && getent hosts "$dom") || true - fi - - # 2) TLS certificate - printf "\n%b[TLS]%b\n" "$DIM" "$NC" - if _server_tools_running; then - _tools_exec "echo | openssl s_client -connect $(_shq "$dom"):443 -servername $(_shq "$dom") -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true" - else - echo | openssl s_client -connect "${dom}:443" -servername "$dom" -showcerts 2>/dev/null | openssl x509 -noout -subject -issuer -dates 2>/dev/null || true - fi - - # 3) HTTP probe (timings) - printf "\n%b[HTTP]%b\n" "$DIM" "$NC" - if _server_tools_running; then - _tools_exec "curl -sk -o /dev/null -D - -w 'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' https://$(_shq "$dom") | sed -n '1,30p'" - else - curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' - fi - - # 4) Upstream inference from nginx conf (if exists) - printf "\n%b[Upstream]%b\n" "$DIM" "$NC" - if [[ -r "$nconf" ]]; then - if grep -q fastcgi_pass "$nconf"; then - local php - php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" - printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" - elif grep -q proxy_pass "$nconf"; then - local up - up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" - printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" - else - printf "type=static\n" - fi - else - printf "nginx_conf=%s (missing)\n" "$nconf" - fi - - # 5) Recent nginx logs (compose) - printf "\n%b[Recent nginx logs]%b\n" "$DIM" "$NC" - docker_compose logs --no-color --tail 120 nginx 2>/dev/null | text_grep -i "$dom" || docker_compose logs --no-color --tail 120 nginx 2>/dev/null || true - - printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" -} +# Extended stack/support diagnostics are loaded by lib/diagnostics.sh. ############################################################################### # 6x. GROUPED COMMAND ROUTERS (stack/domain/support) + backward-compatible aliases From 836fc60bb57316d21e6ff985f86d7668cffe7c75 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:16:04 +0600 Subject: [PATCH 107/509] test: follow modular LocalDevStack implementation paths --- tests/env-contract.sh | 30 +++++++++++++++--------------- 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index dbb21e04..a751c468 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -12,14 +12,14 @@ assert_file_contains "$ROOT/lds" 'ENV_MAIN="$DIR/.env"' assert_file_contains "$ROOT/lds" 'ENV_DOCKER="$CFG/.env"' assert_file_contains "$ROOT/lds" 'ENV_RELEASE="$CFG/release.env"' assert_file_contains "$ROOT/lds" 'COMPOSE_FILE="$CFG/compose/main.yaml"' -assert_file_contains "$ROOT/lds" 'local -a env_files=(--env-file "$ENV_RELEASE")' -assert_file_contains "$ROOT/lds" 'env_files+=(--env-file "$ENV_DOCKER")' -assert_file_contains "$ROOT/lds" '"${env_files[@]}"' -assert_file_contains "$ROOT/lds" 'var=COMPOSE_PROFILES' +assert_file_contains "$ROOT/lib/compose.sh" 'local -a env_files=(--env-file "$ENV_RELEASE")' +assert_file_contains "$ROOT/lib/compose.sh" 'env_files+=(--env-file "$ENV_DOCKER")' +assert_file_contains "$ROOT/lib/compose.sh" '"${env_files[@]}"' +assert_file_contains "$ROOT/lib/hosts.sh" 'var=COMPOSE_PROFILES' assert_file_contains "$ROOT/lds" 'compose_control_value()' assert_file_contains "$ROOT/lds" 'dotenv_value()' -assert_file_contains "$ROOT/lds" 'LDS_AI_RUNTIME cpu' -assert_file_contains "$ROOT/lds" 'LDS_LLM_HOST_PORT 0' +assert_file_contains "$ROOT/lib/compose.sh" 'LDS_AI_RUNTIME cpu' +assert_file_contains "$ROOT/lib/compose.sh" 'LDS_LLM_HOST_PORT 0' pass "environment file and precedence wiring" git -C "$ROOT" check-ignore -q docker/.env || fail "docker/.env must remain ignored user state" @@ -43,21 +43,21 @@ for entry in "${expected[@]}"; do done pass "moving latest image manifest" -assert_file_contains "$ROOT/lds" 'CATALOG_FILE="$CFG/catalog/services.psv"' -assert_file_contains "$ROOT/lds" 'load_service_catalog()' -assert_file_contains "$ROOT/lds" 'load_service_catalog' +assert_file_contains "$ROOT/lib/profiles.sh" 'CATALOG_FILE="$CFG/catalog/services.psv"' +assert_file_contains "$ROOT/lib/profiles.sh" 'load_service_catalog()' +assert_file_contains "$ROOT/lib/profiles.sh" 'load_service_catalog' pass "profile setup loads the tracked host catalog" -assert_file_contains "$ROOT/lds" 'cmd_ai()' -assert_file_contains "$ROOT/lds" 'cmd_llm()' -assert_file_contains "$ROOT/lds" '_tools_exec_argv()' +assert_file_contains "$ROOT/lib/ai.sh" 'cmd_ai()' +assert_file_contains "$ROOT/lib/ai.sh" 'cmd_llm()' +assert_file_contains "$ROOT/lib/ai.sh" '_tools_exec_argv()' assert_file_contains "$ROOT/lds" 'ai) cmd_ai "$@"' assert_file_contains "$ROOT/lds" 'llm) cmd_llm "$@"' pass "AI/LLM CLI routing contract" -assert_file_contains "$ROOT/lds" 'dc_cmd build --build-arg "SCRIPTOMATIC_REF=$scriptomatic_ref"' -if grep -Fq 'dc_build --no-cache' "$ROOT/lds"; then +assert_file_contains "$ROOT/lib/compose.sh" 'dc_cmd build --build-arg "SCRIPTOMATIC_REF=$scriptomatic_ref"' +if grep -R -Fq 'dc_build --no-cache' "$ROOT/lds" "$ROOT/lib"; then fail "runtime rebuild path must preserve Docker build cache" fi -assert_file_contains "$ROOT/lds" 'dc_build --pull "$svc"' +assert_file_contains "$ROOT/lib/services.sh" 'dc_build --pull "$svc"' pass "runtime rebuilds preserve cache while refreshing selected bases" From 5ad29c41c9158fb0d04650abeb8abd84d498e0f7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:17:01 +0600 Subject: [PATCH 108/509] test: follow modular network migration implementation --- tests/networking-contract.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/networking-contract.sh b/tests/networking-contract.sh index d69e606f..9590d4d3 100755 --- a/tests/networking-contract.sh +++ b/tests/networking-contract.sh @@ -45,10 +45,10 @@ fi assert_file_contains "$http" 'restart: unless-stopped' pass "HTTP network/restart cleanup" -assert_file_contains "$ROOT/lds" 'migrate_legacy_networks()' -assert_file_contains "$ROOT/lds" 'docker_compose down --remove-orphans' -assert_file_contains "$ROOT/lds" 'com.infocyph.network-schema' -assert_file_contains "$ROOT/lds" 'cmd_vpn_fix()' +assert_file_contains "$ROOT/lib/services.sh" 'migrate_legacy_networks()' +assert_file_contains "$ROOT/lib/services.sh" 'docker_compose down --remove-orphans' +assert_file_contains "$ROOT/lib/services.sh" 'com.infocyph.network-schema' +assert_file_contains "$ROOT/lib/services.sh" 'cmd_vpn_fix()' assert_file_contains "$ROOT/lds" 'vpn-fix) cmd_vpn_fix "$@"' -assert_file_contains "$ROOT/lds" 'migrate_legacy_networks' +assert_file_contains "$ROOT/lib/services.sh" 'migrate_legacy_networks' pass "safe legacy-network migration and vpn-fix deprecation" From 8f820ee5404d77b4ccec5065ea07419c122f785d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:18:48 +0600 Subject: [PATCH 109/509] windows: defer Docker checks to Docker commands --- lds.bat | 16 ---------------- 1 file changed, 16 deletions(-) diff --git a/lds.bat b/lds.bat index e838a65d..955baa08 100644 --- a/lds.bat +++ b/lds.bat @@ -41,22 +41,6 @@ if not exist "%DEVHOME%\lds" ( exit /b 4 ) -where docker.exe >nul 2>&1 -if errorlevel 1 goto :no_docker - -docker info >nul 2>&1 -if errorlevel 1 goto :docker_not_running - -goto :run - -:no_docker -echo %WARN% docker.exe not found on Windows PATH -exit /b 10 - -:docker_not_running -echo %WARN% Docker is installed but NOT running/reachable (docker info failed) -exit /b 11 - :run "%BASH_EXE%" -lc "set -euo pipefail; export TERM=xterm-256color; DEVHOME_WIN=\"$1\"; CALLER_WIN=\"$2\"; DEVHOME=$(cygpath -u \"$DEVHOME_WIN\"); CALLER=$(cygpath -u \"$CALLER_WIN\"); cd \"$DEVHOME\"; chmod +x ./lds >/dev/null 2>&1 || true; cd \"$CALLER\"; shift 2; exec \"$DEVHOME/lds\" --__win_workdir \"$CALLER_WIN\" \"$@\"" bash "%DEVHOME%" "%WORKDIR%" %* From 57bc355fbede00345a76c652d5021bb69d9a9fb1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:18:54 +0600 Subject: [PATCH 110/509] test: require offline-safe Windows CLI bridge --- tests/windows-bridge.ps1 | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-) diff --git a/tests/windows-bridge.ps1 b/tests/windows-bridge.ps1 index de114503..7f3a2e4b 100644 --- a/tests/windows-bridge.ps1 +++ b/tests/windows-bridge.ps1 @@ -27,13 +27,19 @@ foreach ($needle in $required) { } } -# Characterize the existing Docker preflight. Batch 7 will move Docker -# availability checks behind commands that actually require Docker. -if (-not $content.Contains('docker info')) { - Write-Host "INFO: unconditional Docker preflight is already absent" -} else { - Write-Host "INFO: current bridge still performs the known unconditional Docker preflight" +if ($content.Contains('docker info') -or $content.Contains('where docker.exe')) { + throw "lds.bat must not require Docker for offline-safe commands" } +Write-Host "PASS: Windows bridge has no unconditional Docker preflight" + +$helpOutput = & cmd.exe /d /c ('"' + $batPath + '" help') 2>&1 +if ($LASTEXITCODE -ne 0) { + throw "lds.bat help failed with exit code $LASTEXITCODE: $helpOutput" +} +if (($helpOutput -join "`n") -notmatch 'LocalDevStack') { + throw "lds.bat help did not reach the Bash CLI" +} +Write-Host "PASS: lds.bat help works without wrapper-level Docker checks" $tempParent = Join-Path $env:RUNNER_TEMP "Local Dev Stack" New-Item -ItemType Directory -Force -Path $tempParent | Out-Null From 9cb4d723b493ceebb0b81c9009f68b4ebc298c99 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:21:46 +0600 Subject: [PATCH 111/509] test: characterize LocalDevStack wrapper contracts --- tests/wrappers-contract.sh | 70 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 tests/wrappers-contract.sh diff --git a/tests/wrappers-contract.sh b/tests/wrappers-contract.sh new file mode 100644 index 00000000..771364e9 --- /dev/null +++ b/tests/wrappers-contract.sh @@ -0,0 +1,70 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +wrappers=( + tool-runner php composer pg my maria mongo redis-cli es +) + +for name in "${wrappers[@]}"; do + file="$ROOT/bin/$name" + assert_file "$file" + bash -n "$file" + if grep -Eq '172\.(28|29|30)\.' "$file"; then + fail "$name contains a removed fixed-network address" + fi +done +pass "wrapper syntax and Docker-DNS independence" + +runner="$ROOT/bin/tool-runner" +assert_file_contains "$runner" '[[ -t 0 ]] && flags+=(-i)' +assert_file_contains "$runner" '[[ -t 1 ]] && flags+=(-t)' +assert_file_contains "$runner" 'MSYS_NO_PATHCONV=1' +assert_file_contains "$runner" 'MSYS2_ARG_CONV_EXCL=' +assert_file_contains "$runner" '--network "container:$SERVER_TOOLS_CONTAINER"' +assert_file_contains "$runner" '--volumes-from "$SERVER_TOOLS_CONTAINER"' +assert_file_contains "$runner" 'exec "$(bin_path docker)" run' +pass "tool-runner preserves TTY, path, namespace and exit-code contracts" + +php="$ROOT/bin/php" +assert_file_contains "$php" '-V|--v|--php)' +assert_file_contains "$php" 'pick_highest_php_container()' +assert_file_contains "$php" 'SERVER_TOOLS_VERSION="$(server_tools_php_version || true)"' +assert_file_contains "$php" '--network "container:${NETWORK_SOURCE_CONTAINER}"' +assert_file_contains "$php" 'MSYS_NO_PATHCONV=1' +pass "PHP wrapper preserves explicit/highest runtime selection" + +composer="$ROOT/bin/composer" +assert_file_contains "$composer" 'select_runtime()' +assert_file_contains "$composer" 'running_php_versions()' +assert_file_contains "$composer" 'server_tools_php_version()' +assert_file_contains "$composer" '--network "container:$TARGET_CONTAINER"' +assert_file_contains "$composer" 'MSYS_NO_PATHCONV=1' +pass "Composer wrapper preserves PHP runtime resolution" + +assert_file_contains "$ROOT/bin/pg" 'SERVICE="${POSTGRESQL_CONTAINER:-${POSTGRES_CONTAINER:-POSTGRESQL}}"' +assert_file_contains "$ROOT/bin/my" 'SERVICE="${MYSQL_CONTAINER:-MYSQL}"' +assert_file_contains "$ROOT/bin/maria" 'SERVICE="${MARIADB_CONTAINER:-${MYSQL_CONTAINER:-MARIADB}}"' +assert_file_contains "$ROOT/bin/mongo" 'SERVICE_DEFAULT="MONGODB"' +assert_file_contains "$ROOT/bin/redis-cli" 'SERVICE="${REDIS_CONTAINER:-REDIS}"' +assert_file_contains "$ROOT/bin/es" 'SERVICE="${ELASTICSEARCH_SERVICE:-ELASTICSEARCH}"' +pass "database wrappers resolve logical service/container identities" + +for name in pg my maria mongo redis-cli es; do + file="$ROOT/bin/$name" + if grep -Eq '172\.(28|29|30)\.|--host[= ]172\.' "$file"; then + fail "$name still assumes a LocalDevStack bridge address" + fi +done +pass "database wrappers have no static subnet assumptions" + +# Loopback inside the selected database/container namespace is intentional. +assert_file_contains "$ROOT/bin/pg" '-h127.0.0.1' +assert_file_contains "$ROOT/bin/my" 'host=127.0.0.1' +assert_file_contains "$ROOT/bin/maria" 'host=127.0.0.1' +assert_file_contains "$ROOT/bin/mongo" '@127.0.0.1:27017' +assert_file_contains "$ROOT/bin/es" 'ES_HOST="${ELASTICSEARCH_HOST:-127.0.0.1}"' +pass "database loopback use remains container-local, not bridge addressing" From 55ea202555e80cfef109df381fc438bbef898bb3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:21:51 +0600 Subject: [PATCH 112/509] ci: enforce wrapper contracts --- .github/workflows/check.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 44b06d03..bf6896d6 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -51,6 +51,9 @@ jobs: - name: Networking characterization run: tests/networking-contract.sh + - name: Wrapper contract + run: tests/wrappers-contract.sh + compose: name: Compose contract runs-on: ubuntu-latest From ec4777d61b27ae594786a64744b3c99c1d1405f6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:21:55 +0600 Subject: [PATCH 113/509] test: add wrappers to release gate --- tests/release-gate.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/release-gate.sh b/tests/release-gate.sh index 75ed984c..2e6c0e86 100755 --- a/tests/release-gate.sh +++ b/tests/release-gate.sh @@ -9,6 +9,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" "$ROOT/tests/catalog-contract.sh" "$ROOT/tests/compose-contract.sh" "$ROOT/tests/networking-contract.sh" +"$ROOT/tests/wrappers-contract.sh" "$ROOT/tests/runtime-php-contract.sh" "$ROOT/tests/runtime-node-contract.sh" "$ROOT/tests/ai-contract.sh" From d41340691f3d846c25df16c6ddd0eaee758fa9ce Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:22:35 +0600 Subject: [PATCH 114/509] test: fix Windows bridge failure formatting --- tests/windows-bridge.ps1 | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/windows-bridge.ps1 b/tests/windows-bridge.ps1 index 7f3a2e4b..6ead17b6 100644 --- a/tests/windows-bridge.ps1 +++ b/tests/windows-bridge.ps1 @@ -34,7 +34,7 @@ Write-Host "PASS: Windows bridge has no unconditional Docker preflight" $helpOutput = & cmd.exe /d /c ('"' + $batPath + '" help') 2>&1 if ($LASTEXITCODE -ne 0) { - throw "lds.bat help failed with exit code $LASTEXITCODE: $helpOutput" + throw "lds.bat help failed with exit code ${LASTEXITCODE}: $helpOutput" } if (($helpOutput -join "`n") -notmatch 'LocalDevStack') { throw "lds.bat help did not reach the Bash CLI" From e32dc1a4882a5dc0343f8b2362b4fb07b6eb67e9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:23:59 +0600 Subject: [PATCH 115/509] test: fix Windows bridge exit-code interpolation From 95822feb952acaf1f89406f25ba97e2e9aef1bfa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:25:19 +0600 Subject: [PATCH 116/509] test: make wrapper contract executable --- tests/wrappers-contract.sh | 0 1 file changed, 0 insertions(+), 0 deletions(-) mode change 100644 => 100755 tests/wrappers-contract.sh diff --git a/tests/wrappers-contract.sh b/tests/wrappers-contract.sh old mode 100644 new mode 100755 From 1493ba5fcbffa170c256395863dcc2d05b167980 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:28:41 +0600 Subject: [PATCH 117/509] compose: harden database health probes --- docker/compose/db.yaml | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index 475268fe..e17adf6e 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -37,18 +37,16 @@ services: - lds_pg:/var/lib/postgresql - ../../logs/postgresql:/var/log/postgresql - ../conf/pg_hba.conf:/etc/postgresql/pg_hba.conf -# - ../conf/postgresql.conf:/etc/postgresql/postgresql.conf command: [ "postgres", "-c", "hba_file=/etc/postgresql/pg_hba.conf", -# "-c", "config_file=/etc/postgresql/postgresql.conf", ] healthcheck: test: [ "CMD-SHELL", - "PGPASSWORD=${POSTGRES_PASSWORD:-postgres} pg_isready -U ${POSTGRES_USER:-postgres} -h localhost -d ${POSTGRES_DB:-postgres}" + "PGPASSWORD=\"${POSTGRES_PASSWORD}\" pg_isready -U \"${POSTGRES_USER}\" -h 127.0.0.1 -d \"${POSTGRES_DB}\"" ] interval: 30s timeout: 10s @@ -70,7 +68,7 @@ services: - lds_my:/var/lib/mysql - ../../logs/mysql:/var/log/mysql healthcheck: - test: ["CMD-SHELL", "mysqladmin ping -h127.0.0.1 -u root -p${MYSQL_ROOT_PASSWORD:-12345}"] + test: ["CMD-SHELL", "mysqladmin ping -h127.0.0.1 -u root -p\"${MYSQL_ROOT_PASSWORD}\""] interval: 30s timeout: 10s retries: 3 @@ -92,7 +90,7 @@ services: test: [ "CMD-SHELL", - "mongosh \"mongodb://${MONGODB_ROOT_USERNAME:-root}:${MONGODB_ROOT_PASSWORD:-12345}@localhost:${MONGODB_PORT:-27017}/admin\" --eval \"db.adminCommand('ping')\" --quiet || exit 1" + "mongosh --host 127.0.0.1 --username \"${MONGO_INITDB_ROOT_USERNAME}\" --password \"${MONGO_INITDB_ROOT_PASSWORD}\" --authenticationDatabase admin --eval \"db.adminCommand('ping')\" --quiet || exit 1" ] interval: 30s timeout: 10s @@ -114,7 +112,7 @@ services: - lds_maria:/var/lib/mysql - ../../logs/mariadb:/var/log/mysql healthcheck: - test: ["CMD-SHELL", "mysqladmin ping -h localhost -u root -p${MARIADB_ROOT_PASSWORD:-12345}"] + test: ["CMD-SHELL", "mysqladmin ping -h127.0.0.1 -u root -p\"${MARIADB_ROOT_PASSWORD}\""] interval: 30s timeout: 10s retries: 3 From db76820a35ad7da5dd31b204c8446b840be1555b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:28:43 +0600 Subject: [PATCH 118/509] compose: wait for healthy data services --- docker/compose/db-client.yaml | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index ce4d19cb..57c65544 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -12,7 +12,8 @@ services: image: redis/redisinsight:latest profiles: [redis] depends_on: - - redis + redis: + condition: service_healthy environment: - RI_REDIS_HOST=redis volumes: @@ -43,7 +44,8 @@ services: image: mongo-express:${MONGO_EXPRESS_VERSION:-latest} profiles: [mongodb] depends_on: - - mongodb + mongodb: + condition: service_healthy environment: - TZ=${TZ:-} - ME_CONFIG_BASICAUTH=false @@ -60,7 +62,8 @@ services: image: kibana:${ELASTICSEARCH_VERSION:-9.5.3} profiles: [elasticsearch] depends_on: - - elasticsearch + elasticsearch: + condition: service_healthy environment: - TZ=${TZ:-} - "ELASTICSEARCH_HOSTS=http://elasticsearch:9200" From 60c256287cebea8c7dde7158cc05b06bf17d7d63 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:28:46 +0600 Subject: [PATCH 119/509] compose: gate companions on Tools health --- docker/compose/companion.yaml | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index faee5794..dfbdcd51 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -77,7 +77,8 @@ services: - ../../logs/redis-insight:/global/log/redis-insight - /var/run/docker.sock:/var/run/docker.sock depends_on: - - server-tools + server-tools: + condition: service_healthy networks: - backend @@ -98,7 +99,8 @@ services: - lds_ssl_keys:/certs:ro - lds_mail:/data depends_on: - - server-tools + server-tools: + condition: service_healthy networks: - frontend - backend From 8f4296f57f5c300ffa63bb40da22590d35511f46 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:28:48 +0600 Subject: [PATCH 120/509] compose: gate HTTP startup on Tools health --- docker/compose/http.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 9b319c97..0beb9898 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -19,7 +19,8 @@ services: networks: - frontend depends_on: - - server-tools + server-tools: + condition: service_healthy apache: container_name: APACHE From c70e221dd82564c31a9b7e6e5b197de1e1cdce89 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:28:50 +0600 Subject: [PATCH 121/509] config: mark PostgreSQL tuning file inactive --- docker/conf/postgresql.conf | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docker/conf/postgresql.conf b/docker/conf/postgresql.conf index 83ad60d0..e1799637 100644 --- a/docker/conf/postgresql.conf +++ b/docker/conf/postgresql.conf @@ -1,3 +1,9 @@ +# INACTIVE REFERENCE CONFIGURATION +# LocalDevStack does not mount or activate this file by default. +# The supported PostgreSQL runtime currently uses the image defaults plus +# docker/conf/pg_hba.conf. Any future activation of this file requires explicit +# version-specific validation and a Compose contract test. +# ########################## # CONNECTIONS & AUTHENTICATION ########################## From 3ed23a4cd9641575c7c733f207b4b0d3b5f74da2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:29:12 +0600 Subject: [PATCH 122/509] compose: health-gate remaining Elastic consumers --- docker/compose/db-client.yaml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index 57c65544..da50d33c 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -78,7 +78,8 @@ services: user: root profiles: [filebeat] depends_on: - - elasticsearch + elasticsearch: + condition: service_healthy volumes: - lds_fbeat:/usr/share/filebeat/data - ../conf/filebeat.yml:/usr/share/filebeat/filebeat.yml:ro From 709382c47fba4f00fdf87f83dac24371c968e788 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:29:16 +0600 Subject: [PATCH 123/509] config: remove unused legacy FPM pool reference --- docker/conf/www.conf | 417 ------------------------------------------- 1 file changed, 417 deletions(-) delete mode 100644 docker/conf/www.conf diff --git a/docker/conf/www.conf b/docker/conf/www.conf deleted file mode 100644 index 70bc610a..00000000 --- a/docker/conf/www.conf +++ /dev/null @@ -1,417 +0,0 @@ -[global] -error_log = /proc/self/fd/2 -; Start a new pool named 'www'. -; the variable $pool can be used in any directive and will be replaced by the -; pool name ('www' here) -[www] - - - -; Per pool prefix -; It only applies on the following directives: -; - 'access.log' -; - 'slowlog' -; - 'listen' (unixsocket) -; - 'chroot' -; - 'chdir' -; - 'php_values' -; - 'php_admin_values' -; When not set, the global prefix (or NONE) applies instead. -; Note: This directive can also be relative to the global prefix. -; Default Value: none -;prefix = /path/to/pools/$pool - -; Unix user/group of processes -; Note: The user is mandatory. If the group is not set, the default user's group -; will be used. -user = www-data -group = www-data - -; The address on which to accept FastCGI requests. -; Valid syntaxes are: -; 'ip.add.re.ss:port' - to listen on a TCP socket to a specific IPv4 address on -; a specific port; -; '[ip:6:addr:ess]:port' - to listen on a TCP socket to a specific IPv6 address on -; a specific port; -; 'port' - to listen on a TCP socket to all addresses -; (IPv6 and IPv4-mapped) on a specific port; -; '/path/to/unix/socket' - to listen on a unix socket. -; Note: This value is mandatory. -listen = 0.0.0.0:9000 - -; Set listen(2) backlog. -; Default Value: 511 (-1 on FreeBSD and OpenBSD) -;listen.backlog = 511 - -; Set permissions for unix socket, if one is used. In Linux, read/write -; permissions must be set in order to allow connections from a web server. Many -; BSD-derived systems allow connections regardless of permissions. -; Default Values: user and group are set as the running user -; mode is set to 0660 -;listen.owner = www-data -;listen.group = www-data -;listen.mode = 0660 -; When POSIX Access Control Lists are supported you can set them using -; these options, value is a comma separated list of user/group names. -; When set, listen.owner and listen.group are ignored -;listen.acl_users = -;listen.acl_groups = - -; List of addresses (IPv4/IPv6) of FastCGI clients which are allowed to connect. -; Equivalent to the FCGI_WEB_SERVER_ADDRS environment variable in the original -; PHP FCGI (5.2.2+). Makes sense only with a tcp listening socket. Each address -; must be separated by a comma. If this value is left blank, connections will be -; accepted from any ip address. -; Default Value: any -;listen.allowed_clients = 127.0.0.1 - -; Specify the nice(2) priority to apply to the pool processes (only if set) -; The value can vary from -19 (highest priority) to 20 (lower priority) -; Note: - It will only work if the FPM master process is launched as root -; - The pool processes will inherit the master process priority -; unless it specified otherwise -; Default Value: no set -; process.priority = -19 - -; Choose how the process manager will control the number of child processes. -; Possible Values: -; static - a fixed number (pm.max_children) of child processes; -; dynamic - the number of child processes are set dynamically based on the -; following directives. With this process management, there will be -; always at least 1 children. -; pm.max_children - the maximum number of children that can -; be alive at the same time. -; pm.start_servers - the number of children created on startup. -; pm.min_spare_servers - the minimum number of children in 'idle' -; state (waiting to process). If the number -; of 'idle' processes is less than this -; number then some children will be created. -; pm.max_spare_servers - the maximum number of children in 'idle' -; state (waiting to process). If the number -; of 'idle' processes is greater than this -; number then some children will be killed. -; ondemand - no children are created at startup. Children will be forked when -; new requests will connect. The following parameter are used: -; pm.max_children - the maximum number of children that -; can be alive at the same time. -; pm.process_idle_timeout - The number of seconds after which -; an idle process will be killed. -; Note: This value is mandatory. -pm = static - -; The number of child processes to be created when pm is set to 'static' and the -; maximum number of child processes when pm is set to 'dynamic' or 'ondemand'. -; This value sets the limit on the number of simultaneous requests that will be -; served. Equivalent to the ApacheMaxClients directive with mpm_prefork. -; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP -; CGI. The below defaults are based on a server without much resources. Don't -; forget to tweak pm.* to fit your needs. -; Note: Used when pm is set to 'static', 'dynamic' or 'ondemand' -; Note: This value is mandatory. -pm.max_children = 300 - -; The number of child processes created on startup. -; Note: Used only when pm is set to 'dynamic' -; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2 -pm.start_servers = 12 - -; The desired minimum number of idle server processes. -; Note: Used only when pm is set to 'dynamic' -; Note: Mandatory when pm is set to 'dynamic' -pm.min_spare_servers = 6 - -; The desired maximum number of idle server processes. -; Note: Used only when pm is set to 'dynamic' -; Note: Mandatory when pm is set to 'dynamic' -pm.max_spare_servers = 12 - -; The number of seconds after which an idle process will be killed. -; Note: Used only when pm is set to 'ondemand' -; Default Value: 10s -pm.process_idle_timeout = 8s; - -; The number of requests each child process should execute before respawning. -; This can be useful to work around memory leaks in 3rd party libraries. For -; endless request processing specify '0'. Equivalent to PHP_FCGI_MAX_REQUESTS. -; Default Value: 0 -pm.max_requests = 300 - -; The URI to view the FPM status page. If this value is not set, no URI will be -; recognized as a status page. It shows the following informations: -; pool - the name of the pool; -; process manager - static, dynamic or ondemand; -; start time - the date and time FPM has started; -; start since - number of seconds since FPM has started; -; accepted conn - the number of request accepted by the pool; -; listen queue - the number of request in the queue of pending -; connections (see backlog in listen(2)); -; max listen queue - the maximum number of requests in the queue -; of pending connections since FPM has started; -; listen queue len - the size of the socket queue of pending connections; -; idle processes - the number of idle processes; -; active processes - the number of active processes; -; total processes - the number of idle + active processes; -; max active processes - the maximum number of active processes since FPM -; has started; -; max children reached - number of times, the process limit has been reached, -; when pm tries to start more children (works only for -; pm 'dynamic' and 'ondemand'); -; Value are updated in real time. -; Example output: -; pool: www -; process manager: static -; start time: 01/Jul/2011:17:53:49 +0200 -; start since: 62636 -; accepted conn: 190460 -; listen queue: 0 -; max listen queue: 1 -; listen queue len: 42 -; idle processes: 4 -; active processes: 11 -; total processes: 15 -; max active processes: 12 -; max children reached: 0 -; -; By default the status page output is formatted as text/plain. Passing either -; 'html', 'xml' or 'json' in the query string will return the corresponding -; output syntax. Example: -; http://www.foo.bar/status -; http://www.foo.bar/status?json -; http://www.foo.bar/status?html -; http://www.foo.bar/status?xml -; -; By default the status page only outputs short status. Passing 'full' in the -; query string will also return status for each pool process. -; Example: -; http://www.foo.bar/status?full -; http://www.foo.bar/status?json&full -; http://www.foo.bar/status?html&full -; http://www.foo.bar/status?xml&full -; The Full status returns for each process: -; pid - the PID of the process; -; state - the state of the process (Idle, Running, ...); -; start time - the date and time the process has started; -; start since - the number of seconds since the process has started; -; requests - the number of requests the process has served; -; request duration - the duration in µs of the requests; -; request method - the request method (GET, POST, ...); -; request URI - the request URI with the query string; -; content length - the content length of the request (only with POST); -; user - the user (PHP_AUTH_USER) (or '-' if not set); -; script - the main script called (or '-' if not set); -; last request cpu - the %cpu the last request consumed -; it's always 0 if the process is not in Idle state -; because CPU calculation is done when the request -; processing has terminated; -; last request memory - the max amount of memory the last request consumed -; it's always 0 if the process is not in Idle state -; because memory calculation is done when the request -; processing has terminated; -; If the process is in Idle state, then informations are related to the -; last request the process has served. Otherwise informations are related to -; the current request being served. -; Example output: -; ************************ -; pid: 31330 -; state: Running -; start time: 01/Jul/2011:17:53:49 +0200 -; start since: 63087 -; requests: 12808 -; request duration: 1250261 -; request method: GET -; request URI: /test_mem.php?N=10000 -; content length: 0 -; user: - -; script: /home/fat/web/docs/php/test_mem.php -; last request cpu: 0.00 -; last request memory: 0 -; -; Note: There is a real-time FPM status monitoring sample web page available -; It's available in: /usr/local/share/php/fpm/status.html -; -; Note: The value must start with a leading slash (/). The value can be -; anything, but it may not be a good idea to use the .php extension or it -; may conflict with a real PHP file. -; Default Value: not set -pm.status_path = /php_status - -; The ping URI to call the monitoring page of FPM. If this value is not set, no -; URI will be recognized as a ping page. This could be used to test from outside -; that FPM is alive and responding, or to -; - create a graph of FPM availability (rrd or such); -; - remove a server from a group if it is not responding (load balancing); -; - trigger alerts for the operating team (24/7). -; Note: The value must start with a leading slash (/). The value can be -; anything, but it may not be a good idea to use the .php extension or it -; may conflict with a real PHP file. -; Default Value: not set -;ping.path = /ping - -; This directive may be used to customize the response of a ping request. The -; response is formatted as text/plain with a 200 response code. -; Default Value: pong -;ping.response = pong - -; The access log file -; Default: not set -;access.log = log/$pool.access.log - -; The access log format. -; The following syntax is allowed -; %%: the '%' character -; %C: %CPU used by the request -; it can accept the following format: -; - %{user}C for user CPU only -; - %{system}C for system CPU only -; - %{total}C for user + system CPU (default) -; %d: time taken to serve the request -; it can accept the following format: -; - %{seconds}d (default) -; - %{miliseconds}d -; - %{mili}d -; - %{microseconds}d -; - %{micro}d -; %e: an environment variable (same as $_ENV or $_SERVER) -; it must be associated with embraces to specify the name of the env -; variable. Some exemples: -; - server specifics like: %{REQUEST_METHOD}e or %{SERVER_PROTOCOL}e -; - HTTP headers like: %{HTTP_HOST}e or %{HTTP_USER_AGENT}e -; %f: script filename -; %l: content-length of the request (for POST request only) -; %m: request method -; %M: peak of memory allocated by PHP -; it can accept the following format: -; - %{bytes}M (default) -; - %{kilobytes}M -; - %{kilo}M -; - %{megabytes}M -; - %{mega}M -; %n: pool name -; %o: output header -; it must be associated with embraces to specify the name of the header: -; - %{Content-Type}o -; - %{X-Powered-By}o -; - %{Transfert-Encoding}o -; - .... -; %p: PID of the child that serviced the request -; %P: PID of the parent of the child that serviced the request -; %q: the query string -; %Q: the '?' character if query string exists -; %r: the request URI (without the query string, see %q and %Q) -; %R: remote IP address -; %s: status (response code) -; %t: server time the request was received -; it can accept a strftime(3) format: -; %d/%b/%Y:%H:%M:%S %z (default) -; The strftime(3) format must be encapsuled in a %{}t tag -; e.g. for a ISO8601 formatted timestring, use: %{%Y-%m-%dT%H:%M:%S%z}t -; %T: time the log has been written (the request has finished) -; it can accept a strftime(3) format: -; %d/%b/%Y:%H:%M:%S %z (default) -; The strftime(3) format must be encapsuled in a %{}t tag -; e.g. for a ISO8601 formatted timestring, use: %{%Y-%m-%dT%H:%M:%S%z}t -; %u: remote user -; -; Default: "%R - %u %t \"%m %r\" %s" -;access.format = "%R - %u %t \"%m %r%Q%q\" %s %f %{mili}d %{kilo}M %C%%" - -; The log file for slow requests -; Default Value: not set -; Note: slowlog is mandatory if request_slowlog_timeout is set -;slowlog = log/$pool.log.slow - -; The timeout for serving a single request after which a PHP backtrace will be -; dumped to the 'slowlog' file. A value of '0s' means 'off'. -; Available units: s(econds)(default), m(inutes), h(ours), or d(ays) -; Default Value: 0 -;request_slowlog_timeout = 0 - -; The timeout for serving a single request after which the worker process will -; be killed. This option should be used when the 'max_execution_time' ini option -; does not stop script execution for some reason. A value of '0' means 'off'. -; Available units: s(econds)(default), m(inutes), h(ours), or d(ays) -; Default Value: 0 -;request_terminate_timeout = 0 - -; Set open file descriptor rlimit. -; Default Value: system defined value -;rlimit_files = 1024 - -; Set max core size rlimit. -; Possible Values: 'unlimited' or an integer greater or equal to 0 -; Default Value: system defined value -;rlimit_core = 0 - -; Chroot to this directory at the start. This value must be defined as an -; absolute path. When this value is not set, chroot is not used. -; Note: you can prefix with '$prefix' to chroot to the pool prefix or one -; of its subdirectories. If the pool prefix is not set, the global prefix -; will be used instead. -; Note: chrooting is a great security feature and should be used whenever -; possible. However, all PHP paths will be relative to the chroot -; (error_log, sessions.save_path, ...). -; Default Value: not set -;chroot = - -; Chdir to this directory at the start. -; Note: relative path can be used. -; Default Value: current directory or / when chroot -;chdir = /var/www - -; Redirect worker stdout and stderr into main error log. If not set, stdout and -; stderr will be redirected to /dev/null according to FastCGI specs. -; Note: on highloaded environement, this can cause some delay in the page -; process time (several ms). -; Default Value: no -;catch_workers_output = yes - -; Clear environment in FPM workers -; Prevents arbitrary environment variables from reaching FPM worker processes -; by clearing the environment in workers before env vars specified in this -; pool configuration are added. -; Setting to "no" will make all environment variables available to PHP code -; via getenv(), $_ENV and $_SERVER. -; Default Value: yes -;clear_env = no - -; Limits the extensions of the main script FPM will allow to parse. This can -; prevent configuration mistakes on the web server side. You should only limit -; FPM to .php extensions to prevent malicious users to use other extensions to -; execute php code. -; Note: set an empty value to allow all extensions. -; Default Value: .php -;security.limit_extensions = .php .php3 .php4 .php5 .php7 - -; Pass environment variables like LD_LIBRARY_PATH. All $VARIABLEs are taken from -; the current environment. -; Default Value: clean env -;env[HOSTNAME] = $HOSTNAME -;env[PATH] = /usr/local/bin:/usr/bin:/bin -;env[TMP] = /tmp -;env[TMPDIR] = /tmp -;env[TEMP] = /tmp - -; Additional php.ini defines, specific to this pool of workers. These settings -; overwrite the values previously defined in the php.ini. The directives are the -; same as the PHP SAPI: -; php_value/php_flag - you can set classic ini defines which can -; be overwritten from PHP call 'ini_set'. -; php_admin_value/php_admin_flag - these directives won't be overwritten by -; PHP call 'ini_set' -; For php_*flag, valid values are on, off, 1, 0, true, false, yes or no. - -; Defining 'extension' will load the corresponding shared extension from -; extension_dir. Defining 'disable_functions' or 'disable_classes' will not -; overwrite previously defined php.ini values, but will append the new value -; instead. - -; Note: path INI options can be relative and will be expanded with the prefix -; (pool, global or /usr/local) - -; Default Value: nothing is defined by default except the values in php.ini and -; specified at startup with the -d argument -;php_admin_value[sendmail_path] = /usr/sbin/sendmail -t -i -f www@my.domain.com -;php_flag[display_errors] = on -;php_admin_value[error_log] = /var/log/fpm-php.www.log -;php_admin_flag[log_errors] = on -;php_admin_value[memory_limit] = 32M From ad1c80f3723570ad75c9cbbb037b931d0f3bc03b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:30:49 +0600 Subject: [PATCH 124/509] test: add Compose service hardening contract --- tests/service-hardening-contract.sh | 47 +++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 tests/service-hardening-contract.sh diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh new file mode 100644 index 00000000..1fbde3e6 --- /dev/null +++ b/tests/service-hardening-contract.sh @@ -0,0 +1,47 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +companion="$ROOT/docker/compose/companion.yaml" +http="$ROOT/docker/compose/http.yaml" +db="$ROOT/docker/compose/db.yaml" +clients="$ROOT/docker/compose/db-client.yaml" +filebeat="$ROOT/docker/conf/filebeat.yml" +pg_ref="$ROOT/docker/conf/postgresql.conf" + +socket_count="$(grep -RhsF '/var/run/docker.sock:/var/run/docker.sock' "$ROOT/docker/compose" | wc -l | tr -d ' ')" +[[ "$socket_count" -eq 2 ]] || fail "Docker socket must be mounted only by Tools and Runner; found $socket_count compose mounts" +assert_file_contains "$companion" '/var/run/docker.sock:/var/run/docker.sock' +if grep -RqsF '/var/run/docker.sock' "$ROOT/docker/compose/ai.yaml" "$ROOT/docker/compose/db.yaml" "$ROOT/docker/compose/db-client.yaml" "$ROOT/docker/compose/http.yaml"; then + fail "Docker socket escaped the trusted Tools/Runner boundary" +fi +pass "Docker socket trust boundary" + +assert_file_contains "$db" 'PGPASSWORD="$${POSTGRES_PASSWORD}" pg_isready -U "$${POSTGRES_USER}" -h 127.0.0.1 -d "$${POSTGRES_DB}"' +assert_file_contains "$db" 'mysqladmin ping -h127.0.0.1 -u root -p"$${MYSQL_ROOT_PASSWORD}"' +assert_file_contains "$db" 'mysqladmin ping -h127.0.0.1 -u root -p"$${MARIADB_ROOT_PASSWORD}"' +assert_file_contains "$db" '--username "$${MONGO_INITDB_ROOT_USERNAME}" --password "$${MONGO_INITDB_ROOT_PASSWORD}"' +if grep -Fq '${POSTGRES_DB:-postgres}' "$db"; then + fail "PostgreSQL healthcheck must probe the container POSTGRES_DB value" +fi +pass "database health probes use container runtime state" + +for file in "$companion" "$http" "$clients"; do + assert_file_contains "$file" 'condition: service_healthy' +done +pass "service consumers wait for declared dependency health" + +assert_file_contains "$filebeat" 'hosts: ["http://elasticsearch:9200"]' +assert_file_contains "$filebeat" 'host: "http://kibana:5601"' +pass "Filebeat uses service DNS" + +assert_file_contains "$pg_ref" '# INACTIVE REFERENCE CONFIGURATION' +if grep -Fq 'postgresql.conf:/etc/postgresql/postgresql.conf' "$db"; then + fail "inactive postgresql.conf must not be pseudo-wired in Compose" +fi +[[ ! -e "$ROOT/docker/conf/www.conf" ]] || fail "unused legacy docker/conf/www.conf must remain removed" +assert_file "$ROOT/docker/conf/www-php.conf" +pass "Docker config ownership is explicit" From dcfe15a92729f662d5a6f0f8b1a72e633ce14a34 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:32:16 +0600 Subject: [PATCH 125/509] ci: enforce service hardening contracts --- .github/workflows/check.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index bf6896d6..dcc0e0ea 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -54,6 +54,9 @@ jobs: - name: Wrapper contract run: tests/wrappers-contract.sh + - name: Service hardening contract + run: bash tests/service-hardening-contract.sh + compose: name: Compose contract runs-on: ubuntu-latest From a22d31325a0af555513e577c48eca0d76b49ca5c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:32:19 +0600 Subject: [PATCH 126/509] test: add service hardening to release gate --- tests/release-gate.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/release-gate.sh b/tests/release-gate.sh index 2e6c0e86..ba3f3c70 100755 --- a/tests/release-gate.sh +++ b/tests/release-gate.sh @@ -10,6 +10,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" "$ROOT/tests/compose-contract.sh" "$ROOT/tests/networking-contract.sh" "$ROOT/tests/wrappers-contract.sh" +bash "$ROOT/tests/service-hardening-contract.sh" "$ROOT/tests/runtime-php-contract.sh" "$ROOT/tests/runtime-node-contract.sh" "$ROOT/tests/ai-contract.sh" From b3a945d233490c20143550f6360bbe5f705d9450 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:32:21 +0600 Subject: [PATCH 127/509] test: validate Runner and active FPM config contracts --- tests/published-images.sh | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/tests/published-images.sh b/tests/published-images.sh index cad0949c..cd000a20 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -96,3 +96,14 @@ assert_contains "$php_template" 'image: localdevstack-php:{{PHP_VERSION}}' assert_contains "$node_template" 'NODE_VERSION: {{NODE_VERSION}}' assert_contains "$node_template" 'image: localdevstack-node:{{NODE_VERSION}}' pass "selected runtime versions remain build/image identity inputs" + +docker run --rm --entrypoint sh "${release[LDS_RUNNER_IMAGE]}" -ec ' + test -x /usr/local/bin/logrotate-worker.sh + test -x /usr/local/bin/runner-healthcheck.sh + test -f /etc/logrotate.d/daily + test -f /etc/logrotate.d/supervisord +' +pass "latest Runner preserves logrotate/health contract" + +assert_contains "$php_template" './docker/conf/www-php.conf:/usr/local/etc/php-fpm.d/www.conf' +pass "generated PHP runtime uses the maintained FPM pool config" From dd558924ccbf6bc673d4069e4f7a790fe7c17926 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:34:46 +0600 Subject: [PATCH 128/509] compose: probe database container env in healthchecks From 2ee4e5fa802dd28cac91a46d29f47d63f8008576 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:34:49 +0600 Subject: [PATCH 129/509] test: match published Runner healthcheck path --- tests/published-images.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index cd000a20..49b050a5 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -99,7 +99,7 @@ pass "selected runtime versions remain build/image identity inputs" docker run --rm --entrypoint sh "${release[LDS_RUNNER_IMAGE]}" -ec ' test -x /usr/local/bin/logrotate-worker.sh - test -x /usr/local/bin/runner-healthcheck.sh + test -x /usr/local/bin/runner-healthcheck test -f /etc/logrotate.d/daily test -f /etc/logrotate.d/supervisord ' From aa5c340bd8c6d156cac6677d7ba19e4dd9c8388a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:36:43 +0600 Subject: [PATCH 130/509] compose: make database readiness probes credential-free --- docker/compose/db.yaml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index e17adf6e..022aadd0 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -46,7 +46,7 @@ services: test: [ "CMD-SHELL", - "PGPASSWORD=\"${POSTGRES_PASSWORD}\" pg_isready -U \"${POSTGRES_USER}\" -h 127.0.0.1 -d \"${POSTGRES_DB}\"" + "pg_isready -h 127.0.0.1"${POSTGRES_USER}\" -h 127.0.0.1 -d \"${POSTGRES_DB}\"" ] interval: 30s timeout: 10s @@ -68,7 +68,7 @@ services: - lds_my:/var/lib/mysql - ../../logs/mysql:/var/log/mysql healthcheck: - test: ["CMD-SHELL", "mysqladmin ping -h127.0.0.1 -u root -p\"${MYSQL_ROOT_PASSWORD}\""] + test: ["CMD", "mysqladmin", "ping", "-h127.0.0.1", "--silent"] interval: 30s timeout: 10s retries: 3 @@ -90,7 +90,7 @@ services: test: [ "CMD-SHELL", - "mongosh --host 127.0.0.1 --username \"${MONGO_INITDB_ROOT_USERNAME}\" --password \"${MONGO_INITDB_ROOT_PASSWORD}\" --authenticationDatabase admin --eval \"db.adminCommand('ping')\" --quiet || exit 1" + "mongosh --host 127.0.0.1 --quiet --eval \"db.adminCommand(\'ping\')\" || exit 1" ] interval: 30s timeout: 10s @@ -112,7 +112,7 @@ services: - lds_maria:/var/lib/mysql - ../../logs/mariadb:/var/log/mysql healthcheck: - test: ["CMD-SHELL", "mysqladmin ping -h127.0.0.1 -u root -p\"${MARIADB_ROOT_PASSWORD}\""] + test: ["CMD", "mysqladmin", "ping", "-h127.0.0.1", "--silent"] interval: 30s timeout: 10s retries: 3 From b392171b748507ab1b9718cdfed61ec48fd96ebd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:36:47 +0600 Subject: [PATCH 131/509] test: enforce credential-free database readiness probes --- tests/service-hardening-contract.sh | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index 1fbde3e6..8c3db59a 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -20,14 +20,14 @@ if grep -RqsF '/var/run/docker.sock' "$ROOT/docker/compose/ai.yaml" "$ROOT/docke fi pass "Docker socket trust boundary" -assert_file_contains "$db" 'PGPASSWORD="$${POSTGRES_PASSWORD}" pg_isready -U "$${POSTGRES_USER}" -h 127.0.0.1 -d "$${POSTGRES_DB}"' -assert_file_contains "$db" 'mysqladmin ping -h127.0.0.1 -u root -p"$${MYSQL_ROOT_PASSWORD}"' -assert_file_contains "$db" 'mysqladmin ping -h127.0.0.1 -u root -p"$${MARIADB_ROOT_PASSWORD}"' -assert_file_contains "$db" '--username "$${MONGO_INITDB_ROOT_USERNAME}" --password "$${MONGO_INITDB_ROOT_PASSWORD}"' -if grep -Fq '${POSTGRES_DB:-postgres}' "$db"; then - fail "PostgreSQL healthcheck must probe the container POSTGRES_DB value" +assert_file_contains "$db" 'pg_isready -h 127.0.0.1' +assert_file_contains "$db" 'test: ["CMD", "mysqladmin", "ping", "-h127.0.0.1", "--silent"]' +assert_file_contains "$db" 'mongosh --host 127.0.0.1 --quiet --eval' +if grep -E 'healthcheck:|PGPASSWORD=|MYSQL_ROOT_PASSWORD|MARIADB_ROOT_PASSWORD|MONGO_INITDB_ROOT_PASSWORD' "$db" | + grep -E 'PGPASSWORD=|MYSQL_ROOT_PASSWORD|MARIADB_ROOT_PASSWORD|MONGO_INITDB_ROOT_PASSWORD' >/dev/null; then + fail "database readiness probes must not embed credentials" fi -pass "database health probes use container runtime state" +pass "database health probes are credential-free local readiness checks" for file in "$companion" "$http" "$clients"; do assert_file_contains "$file" 'condition: service_healthy' From 348a675a32d108e0d905b73669088b845a8df8b4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:37:14 +0600 Subject: [PATCH 132/509] compose: use exec-form database readiness probes --- docker/compose/db.yaml | 12 ++---------- 1 file changed, 2 insertions(+), 10 deletions(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index 022aadd0..1e9b0270 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -43,11 +43,7 @@ services: "-c", "hba_file=/etc/postgresql/pg_hba.conf", ] healthcheck: - test: - [ - "CMD-SHELL", - "pg_isready -h 127.0.0.1"${POSTGRES_USER}\" -h 127.0.0.1 -d \"${POSTGRES_DB}\"" - ] + test: ["CMD", "pg_isready", "-h", "127.0.0.1"] interval: 30s timeout: 10s retries: 5 @@ -87,11 +83,7 @@ services: - lds_mongo:/data/db - ../../logs/mongodb:/var/log/mongodb healthcheck: - test: - [ - "CMD-SHELL", - "mongosh --host 127.0.0.1 --quiet --eval \"db.adminCommand(\'ping\')\" || exit 1" - ] + test: ["CMD", "mongosh", "--host", "127.0.0.1", "--quiet", "--eval", "db.adminCommand('ping')"] interval: 30s timeout: 10s retries: 3 From 1150dc3c7eab36a6d814903b74bb33ffc5276955 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:37:36 +0600 Subject: [PATCH 133/509] test: match exec-form database readiness probes --- tests/service-hardening-contract.sh | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index 8c3db59a..0e40a8f3 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -20,11 +20,10 @@ if grep -RqsF '/var/run/docker.sock' "$ROOT/docker/compose/ai.yaml" "$ROOT/docke fi pass "Docker socket trust boundary" -assert_file_contains "$db" 'pg_isready -h 127.0.0.1' +assert_file_contains "$db" 'test: ["CMD", "pg_isready", "-h", "127.0.0.1"]' assert_file_contains "$db" 'test: ["CMD", "mysqladmin", "ping", "-h127.0.0.1", "--silent"]' -assert_file_contains "$db" 'mongosh --host 127.0.0.1 --quiet --eval' -if grep -E 'healthcheck:|PGPASSWORD=|MYSQL_ROOT_PASSWORD|MARIADB_ROOT_PASSWORD|MONGO_INITDB_ROOT_PASSWORD' "$db" | - grep -E 'PGPASSWORD=|MYSQL_ROOT_PASSWORD|MARIADB_ROOT_PASSWORD|MONGO_INITDB_ROOT_PASSWORD' >/dev/null; then +assert_file_contains "$db" 'test: ["CMD", "mongosh", "--host", "127.0.0.1", "--quiet", "--eval", "db.adminCommand('\''ping'\'')"]' +if grep -Fq 'PGPASSWORD=' "$db" || grep -Fq ' --password ' "$db"; then fail "database readiness probes must not embed credentials" fi pass "database health probes are credential-free local readiness checks" From 34549adaa2f80d5dbebd02717d1602033f6ea108 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:41:25 +0600 Subject: [PATCH 134/509] permissions: replace broad log 777 with scoped modes --- lib/platform.sh | 45 ++++++++++++++++++++++++++++++++++----------- 1 file changed, 34 insertions(+), 11 deletions(-) diff --git a/lib/platform.sh b/lib/platform.sh index 9a6ebaf3..67eb9280 100644 --- a/lib/platform.sh +++ b/lib/platform.sh @@ -38,21 +38,44 @@ fix_perms() { ((EUID == 0)) || die "Please run with sudo." + local owner group + owner="${SUDO_USER:-${USER:-}}" + [[ -n "$owner" ]] || owner="$(id -un)" + id "$owner" >/dev/null 2>&1 || die "Cannot resolve permission owner: $owner" + + if getent group docker >/dev/null 2>&1; then + group=docker + else + group="$(id -gn "$owner")" + fi + chmod 755 "$DIR" - chmod 2775 "$DIR/configuration" - find "$DIR/configuration" -type f ! -perm 664 -exec chmod 664 {} + - chmod 755 "$DIR/docker" - find "$DIR/docker" -type f ! -perm 644 -exec chmod 644 {} + + chown -R "$owner:$group" "$DIR/configuration" "$DIR/logs" + find "$DIR/configuration" -type d -exec chmod 2775 {} + + find "$DIR/configuration" -type f -exec chmod 0664 {} + + find "$DIR/logs" -type d -exec chmod 2775 {} + + find "$DIR/logs" -type f -exec chmod 0664 {} + - chmod -R 777 "$DIR/logs" - chown -R "$USER:docker" "$DIR/logs" + # Secret-bearing host directories stay private to the workstation owner. + for private_dir in "$DIR/configuration/ssh" "$DIR/configuration/sops/keys"; do + [[ -d "$private_dir" ]] || continue + chown -R "$owner:$group" "$private_dir" + find "$private_dir" -type d -exec chmod 0700 {} + + find "$private_dir" -type f -exec chmod 0600 {} + + done - chmod 755 "$DIR/bin" - find "$DIR/bin" -type f -exec chmod +x {} + - chmod +x "$DIR/lds" + find "$DIR/docker" -type d -exec chmod 0755 {} + + find "$DIR/docker" -type f -exec chmod 0644 {} + + + chmod 0755 "$DIR/bin" + find "$DIR/bin" -type f -exec chmod 0755 {} + + if [[ -d "$DIR/lib" ]]; then + find "$DIR/lib" -type d -exec chmod 0755 {} + + find "$DIR/lib" -type f -exec chmod 0644 {} + + fi + chmod 0755 "$DIR/lds" ln -fs "$DIR/lds" /usr/local/bin/lds - printf "%bPermissions assigned.%b\n" "$GREEN" "$NC" + printf "%bPermissions assigned to %s:%s.%b\n" "$GREEN" "$owner" "$group" "$NC" } - From bab6e4f994054f7d692b0664cce98ac92e6237e8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:41:50 +0600 Subject: [PATCH 135/509] qol: add profile-aware URLs and HTTPS open targets --- lib/services.sh | 51 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 4 deletions(-) diff --git a/lib/services.sh b/lib/services.sh index 8d3155f1..4a8b81af 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -301,16 +301,59 @@ cmd_logs() { fi } +_enabled_profiles_csv() { + compose_control_value COMPOSE_PROFILES "" +} + +profile_enabled() { + local wanted="${1:-}" csv p + [[ -n "$wanted" ]] || return 1 + csv="$(_enabled_profiles_csv)" + IFS=',' read -r -a __lds_profiles <<<"$csv" + for p in "${__lds_profiles[@]}"; do + p="${p//[[:space:]]/}" + [[ "$p" == "$wanted" ]] && return 0 + done + return 1 +} + +cmd_urls() { + local -A seen=() + local url key profile + + _print_url() { + local value="${1:-}" + [[ -n "$value" ]] || return 0 + [[ -z "${seen[$value]:-}" ]] || return 0 + seen["$value"]=1 + printf '%s\n' "$value" + } + + _print_url "https://admin.localhost" + _print_url "https://webmail.localhost" + + for key in "${SERVICE_ORDER[@]}"; do + profile="${SERVICES[$key]:-}" + url="${SERVICE_URL[$key]:-}" + [[ -n "$profile" && -n "$url" ]] || continue + profile_enabled "$profile" || continue + _print_url "$url" + done +} + cmd_open() { local target="${1:-}" - [[ -n "$target" ]] || die "open " + [[ -n "$target" ]] || die "open " local url="" case "${target,,}" in + http://* | https://*) url="$target" ;; + admin | tools) url="https://admin.localhost" ;; mail | mailpit | webmail) url="https://webmail.localhost" ;; db | cloudbeaver) url="https://db.localhost" ;; - redis | redisinsight | redis-insight | rds) url="http://ri.localhost" ;; - mongo | me | mongoexpress | mongo-express) url="http://me.localhost" ;; - kibana | kbn) url="http://kibana.localhost" ;; + redis | redisinsight | redis-insight | rds) url="https://ri.localhost" ;; + mongo | me | mongoexpress | mongo-express) url="https://me.localhost" ;; + kibana | kbn) url="https://kibana.localhost" ;; + ai | llm | llm-sm) url="https://llm.localhost" ;; *) url="https://${target}" ;; From db8e8c08469aed19b78c568ecbaf25988671a5ce Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:43:08 +0600 Subject: [PATCH 136/509] qol: add redacted config images and doctor commands --- lib/diagnostics.sh | 207 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 207 insertions(+) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index c5a7f9b2..8202f516 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -302,3 +302,210 @@ cmd_support_trace() { printf "\n%bDone.%b If this still looks wrong, run: lds diag tls %s\n" "$GREEN" "$NC" "$dom" } + + +############################################################################### +# PRODUCT CONFIG / IMAGES / DOCTOR +############################################################################### + +_redact_effective_config() { + sed -E \ + -e 's/^([[:space:]]*[A-Z0-9_]*(PASSWORD|SECRET|TOKEN|PRIVATE_KEY|API_KEY|ACCESS_KEY)[A-Z0-9_]*:[[:space:]]*).*$/\1"***REDACTED***"/' \ + -e 's/^([[:space:]]*(ME_CONFIG_MONGODB_URL|DATABASE_URL):[[:space:]]*).*$/\1"***REDACTED***"/' \ + -e 's/("[A-Z0-9_]*(PASSWORD|SECRET|TOKEN|PRIVATE_KEY|API_KEY|ACCESS_KEY)[A-Z0-9_]*"[[:space:]]*:[[:space:]]*)"[^"]*"/\1"***REDACTED***"/g' \ + -e 's/("(ME_CONFIG_MONGODB_URL|DATABASE_URL)"[[:space:]]*:[[:space:]]*)"[^"]*"/\1"***REDACTED***"/g' +} + +_env_key_list() { + local file="${1:-}" source="${2:-}" + [[ -r "$file" ]] || return 0 + awk -F= -v source="$source" ' + /^[[:space:]]*#/ || /^[[:space:]]*$/ { next } + /^[A-Za-z_][A-Za-z0-9_]*=/ { print source "\t" $1 } + ' "$file" +} + +_validate_scheduler_text() { + local file failed=0 + while IFS= read -r file; do + [[ -f "$file" ]] || continue + if grep -Iq . "$file" && grep -q $'\r$' "$file"; then + printf '%b[fail]%b CRLF scheduler file: %s\n' "$RED" "$NC" "$file" >&2 + failed=1 + fi + done < <(find "$DIR/configuration/scheduler" -type f ! -name '.gitignore' -print 2>/dev/null | sort) + return "$failed" +} + +cmd_config() { + local sub="${1:-show}" + shift || true + + case "${sub,,}" in + show | "") + local format="" raw=0 + while [[ "${1:-}" ]]; do + case "$1" in + --json) format=json; shift ;; + --raw) raw=1; shift ;; + *) die "config show [--json] [--raw]" ;; + esac + done + + local -a args=(config) + [[ "$format" == json ]] && args+=(--format json) + if ((raw)); then + warn "Printing raw effective configuration; secret values may be visible." + docker_compose "${args[@]}" + else + docker_compose "${args[@]}" | _redact_effective_config + fi + ;; + services) + docker_compose config --services + ;; + profiles) + docker_compose config --profiles + ;; + env-used) + { + _env_key_list "$ENV_RELEASE" release + _env_key_list "$ENV_DOCKER" user + } | LC_ALL=C sort -k2,2 -k1,1 + ;; + validate) + docker_compose config --quiet + _validate_scheduler_text || die "Scheduler files contain CRLF; convert them to LF before Runner consumes them." + + local runner + runner="$(docker_compose ps -q runner 2>/dev/null | sed -n '1p' || true)" + if [[ -n "$runner" ]] && docker inspect -f '{{.State.Running}}' "$runner" 2>/dev/null | grep -qx true; then + docker exec "$runner" supervisord -t -c /etc/supervisor/supervisord.conf >/dev/null + ok "Compose and mounted Supervisor configuration validate." + else + ok "Compose configuration validates." + warn "Runner is not running; Supervisor syntax check skipped." + fi + + local -a fragments=() + mapfile -t fragments < <(find "$EXTRAS_DIR" -maxdepth 1 -type f \( -name '*.yaml' -o -name '*.yml' \) -print 2>/dev/null | sort) + if (("${#fragments[@]}" > 0)); then + printf '%bGenerated Compose fragments:%b\n' "$CYAN" "$NC" + printf ' %s\n' "${fragments[@]}" + fi + ;; + *) + die "config " + ;; + esac +} + +cmd_images() { + local elastic + elastic="$(compose_control_value ELASTICSEARCH_VERSION 9.5.3)" + + printf '%-16s %s\n' "Tools" "$(compose_control_value LDS_TOOLS_IMAGE infocyph/tools:latest)" + printf '%-16s %s\n' "Runner" "$(compose_control_value LDS_RUNNER_IMAGE infocyph/runner:latest)" + printf '%-16s %s\n' "Nginx" "$(compose_control_value LDS_NGINX_IMAGE infocyph/nginx:latest)" + printf '%-16s %s\n' "Apache" "$(compose_control_value LDS_APACHE_IMAGE infocyph/apache:latest)" + printf '%-16s %s\n' "LLM" "$(compose_control_value LDS_LLM_IMAGE infocyph/llm-sm:latest)" + printf '%-16s %s\n' "LLM AMD" "$(compose_control_value LDS_LLM_AMD_IMAGE infocyph/llm-sm:amd-latest)" + printf '%-16s postgres:%s\n' "PostgreSQL" "$(compose_control_value POSTGRES_VERSION alpine)" + printf '%-16s mysql:%s\n' "MySQL" "$(compose_control_value MYSQL_VERSION latest)" + printf '%-16s mariadb:%s\n' "MariaDB" "$(compose_control_value MARIADB_VERSION latest)" + printf '%-16s mongo:%s\n' "MongoDB" "$(compose_control_value MONGODB_VERSION latest)" + printf '%-16s redis/redis-stack-server:%s\n' "Redis" "$(compose_control_value REDIS_VERSION latest)" + printf '%-16s elasticsearch:%s\n' "Elasticsearch" "$elastic" + printf '%-16s kibana:%s\n' "Kibana" "$elastic" + printf '%-16s docker.elastic.co/beats/filebeat:%s\n' "Filebeat" "$elastic" + printf '%-16s %s\n' "PHP runtimes" "localdevstack-php: (Alpine)" + printf '%-16s %s\n' "Node runtimes" "localdevstack-node: (Alpine)" +} + +_doctor_ok() { printf '%b[ok]%b %s\n' "$GREEN" "$NC" "$*"; } +_doctor_warn() { printf '%b[warn]%b %s\n' "$YELLOW" "$NC" "$*"; } +_doctor_fail() { printf '%b[fail]%b %s\n' "$RED" "$NC" "$*" >&2; } + +cmd_doctor() { + local failures=0 warnings=0 project ctr status name + project="$(lds_project)" + + if ! has_bin docker; then + _doctor_fail "Docker CLI is not installed." + return 1 + fi + _doctor_ok "Docker CLI: $(docker --version 2>/dev/null || printf unknown)" + + if ! docker info >/dev/null 2>&1; then + _doctor_fail "Docker daemon is unavailable." + return 1 + fi + _doctor_ok "Docker daemon is reachable." + + if docker compose version >/dev/null 2>&1 || has_bin docker-compose; then + _doctor_ok "Docker Compose is available." + else + _doctor_fail "Docker Compose is unavailable." + failures=$((failures + 1)) + fi + + if docker_compose config --quiet >/dev/null 2>&1; then + _doctor_ok "Effective Compose configuration validates." + else + _doctor_fail "Effective Compose configuration is invalid." + failures=$((failures + 1)) + fi + + printf '%bProfiles:%b %s\n' "$CYAN" "$NC" "$(_enabled_profiles_csv | sed 's/^$//')" + + for name in Frontend Backend DataStore; do + if docker network inspect "$name" >/dev/null 2>&1; then + _doctor_ok "Network present: $name" + else + _doctor_warn "Network not created yet: $name" + warnings=$((warnings + 1)) + fi + done + + ctr="$(_project_tools_container_running || true)" + if [[ -n "$ctr" ]]; then + _doctor_ok "Tools container is running: $ctr" + if docker exec "$ctr" sh -ec 'test -s /etc/mkcert/lds-server.pem && test -s /etc/mkcert/lds-server-key.pem' >/dev/null 2>&1; then + _doctor_ok "Shared TLS certificate/key are present." + else + _doctor_warn "Shared TLS certificate/key are not ready." + warnings=$((warnings + 1)) + fi + else + _doctor_warn "Tools container is not running." + warnings=$((warnings + 1)) + fi + + while IFS='|' read -r name status; do + [[ -n "$name" ]] || continue + if [[ "$status" == *"(unhealthy)"* ]]; then + _doctor_fail "$name: $status" + failures=$((failures + 1)) + else + _doctor_ok "$name: $status" + fi + done < <( + docker ps \ + --filter "label=com.docker.compose.project=$project" \ + --format '{{.Names}}|{{.Status}}' 2>/dev/null || true + ) + + if profile_enabled ai; then + local llm + llm="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" + if [[ -n "$llm" ]] && docker inspect -f '{{.State.Running}}' "$llm" 2>/dev/null | grep -qx true; then + _doctor_ok "AI provider container is running." + else + _doctor_warn "AI profile is selected but llm-sm is not running." + warnings=$((warnings + 1)) + fi + fi + + printf '%bDoctor summary:%b %d failure(s), %d warning(s)\n' "$CYAN" "$NC" "$failures" "$warnings" + ((failures == 0)) +} From 4414b743e3937ff8ff8569be86450f8063689998 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:43:26 +0600 Subject: [PATCH 137/509] qol: expose offline-safe images urls and config metadata --- lds | 27 +++++++++++++++++++++------ 1 file changed, 21 insertions(+), 6 deletions(-) diff --git a/lds b/lds index 0bb82643..0ca22af2 100755 --- a/lds +++ b/lds @@ -698,6 +698,7 @@ Legacy alias: `lds host …` → same subcommands as `domain`. - `lds certificate uninstall [--all]` ## Diagnostics +- `lds doctor` - `lds diag dns ` - `lds diag net` - `lds diag tcp ` @@ -705,15 +706,17 @@ Legacy alias: `lds host …` → same subcommands as `domain`. - `lds diag tls ` ## Config -- `lds config show [--json]` +- `lds config show [--json] [--raw]` *(redacted by default)* - `lds config services` - `lds config profiles` - `lds config env-used` - `lds config validate` +- `lds images` +- `lds urls` ## Support -- `lds support open - support trace ` +- `lds support open ` +- `lds support trace ` - `lds support bundle [--redact|--full]` - `lds support notify …` - `lds support ui` @@ -775,14 +778,17 @@ ${CYAN}Certificates (TLS):${NC} certificate install|uninstall [--all] ${CYAN}Diagnostics:${NC} + doctor diag dns|net|tcp|http|tls sniff Alias of: diag http ${CYAN}Config:${NC} config show|services|profiles|env-used|validate + images + urls ${CYAN}Support:${NC} - support open + support open support trace support bundle [--redact|--full] support notify ... @@ -868,8 +874,17 @@ main() { exit 0 fi - # Do not require docker for pure help output; everything else expects the stack. - need docker + # Offline-safe read-only commands do not require a running Docker daemon. + case "$cmd" in + images | urls) + ;; + config) + [[ "${1:-}" == "env-used" ]] || need docker + ;; + *) + need docker + ;; + esac case "$cmd" in php | composer | node | npm | npx) exec "$DIR/bin/$cmd" "$@" ;; From f4d3c6d09dfe622c0fafbc0c5d6bbd3a2b6e255b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:43:55 +0600 Subject: [PATCH 138/509] test: add QoL command and redaction contract --- tests/qol-contract.sh | 78 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 tests/qol-contract.sh diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh new file mode 100644 index 00000000..90a50af1 --- /dev/null +++ b/tests/qol-contract.sh @@ -0,0 +1,78 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +user_env="$ROOT/docker/.env" +backup="" +had_env=0 +if [[ -e "$user_env" ]]; then + had_env=1 + backup="$(mktemp)" + cp "$user_env" "$backup" +fi +cleanup() { + if ((had_env)); then + cp "$backup" "$user_env" + rm -f "$backup" + else + rm -f "$user_env" + fi +} +trap cleanup EXIT + +cat >"$user_env" <"$tmpbin/docker" <<'SH' +#!/usr/bin/env sh +echo "docker should not be executed for this command" >&2 +exit 97 +SH +chmod +x "$tmpbin/docker" + +images="$(PATH="$tmpbin:$PATH" "$ROOT/lds" images)" +assert_contains "$images" "example.invalid/tools:custom" +assert_contains "$images" "postgres:alpine" +assert_contains "$images" "elasticsearch:9.5.3" +assert_contains "$images" "localdevstack-php: (Alpine)" +pass "images is offline-safe and reflects effective overrides" + +urls="$(PATH="$tmpbin:$PATH" "$ROOT/lds" urls)" +assert_contains "$urls" "https://admin.localhost" +assert_contains "$urls" "https://webmail.localhost" +assert_contains "$urls" "https://db.localhost" +assert_contains "$urls" "https://ri.localhost" +assert_contains "$urls" "https://llm.localhost" +if grep -Fq "https://kibana.localhost" <<<"$urls"; then + fail "urls must not show disabled Elasticsearch profile URL" +fi +pass "urls is profile-aware and offline-safe" + +env_used="$(PATH="$tmpbin:$PATH" "$ROOT/lds" config env-used)" +assert_contains "$env_used" $'user\tCOMPOSE_PROFILES' +assert_contains "$env_used" $'release\tLDS_TOOLS_IMAGE' +if grep -Fq "supersecret-ci-value" <<<"$env_used"; then + fail "config env-used leaked a value" +fi +pass "config env-used reports keys only" + +redacted="$("$ROOT/lds" config show)" +if grep -Fq "supersecret-ci-value" <<<"$redacted"; then + fail "config show leaked MYSQL_ROOT_PASSWORD" +fi +assert_contains "$redacted" "***REDACTED***" +pass "config show redacts effective secrets by default" + +help="$("$ROOT/lds" help)" +assert_contains "$help" "doctor" +assert_contains "$help" "images" +assert_contains "$help" "urls" +pass "QoL commands are discoverable" From dab6fbd298f4a498cc819deaaef3737f0c9fbd39 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:43:58 +0600 Subject: [PATCH 139/509] test: add host permission hardening contract --- tests/permissions-contract.sh | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 tests/permissions-contract.sh diff --git a/tests/permissions-contract.sh b/tests/permissions-contract.sh new file mode 100644 index 00000000..a097f9f1 --- /dev/null +++ b/tests/permissions-contract.sh @@ -0,0 +1,19 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +file="$ROOT/lib/platform.sh" +assert_file "$file" +if grep -Eq 'chmod[[:space:]]+-R[[:space:]]+777|chmod[[:space:]]+777' "$file"; then + fail "broad world-writable permission logic returned" +fi +assert_file_contains "$file" 'owner="${SUDO_USER:-${USER:-}}"' +assert_file_contains "$file" 'find "$DIR/configuration" -type d -exec chmod 2775 {} +' +assert_file_contains "$file" 'find "$DIR/logs" -type d -exec chmod 2775 {} +' +assert_file_contains "$file" 'find "$private_dir" -type d -exec chmod 0700 {} +' +assert_file_contains "$file" 'find "$private_dir" -type f -exec chmod 0600 {} +' +assert_file_contains "$file" '"$DIR/configuration/ssh" "$DIR/configuration/sops/keys"' +pass "host permissions are scoped and secret directories remain private" From 350acba65f12cdbbfe2dd8f59c679d9a2b8f5406 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:44:13 +0600 Subject: [PATCH 140/509] ci: enforce final QoL and permission contracts --- .github/workflows/check.yml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index dcc0e0ea..38096a65 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -57,6 +57,12 @@ jobs: - name: Service hardening contract run: bash tests/service-hardening-contract.sh + - name: QoL contract + run: bash tests/qol-contract.sh + + - name: Permission contract + run: bash tests/permissions-contract.sh + compose: name: Compose contract runs-on: ubuntu-latest From 42c3c93b3df58db0db1ba4c26546d4d06ac6c1af Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:44:16 +0600 Subject: [PATCH 141/509] test: add QoL and permissions to release gate --- tests/release-gate.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/release-gate.sh b/tests/release-gate.sh index ba3f3c70..e1e8ce2a 100755 --- a/tests/release-gate.sh +++ b/tests/release-gate.sh @@ -11,6 +11,8 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" "$ROOT/tests/networking-contract.sh" "$ROOT/tests/wrappers-contract.sh" bash "$ROOT/tests/service-hardening-contract.sh" +bash "$ROOT/tests/qol-contract.sh" +bash "$ROOT/tests/permissions-contract.sh" "$ROOT/tests/runtime-php-contract.sh" "$ROOT/tests/runtime-node-contract.sh" "$ROOT/tests/ai-contract.sh" From 4950a27cd1d232242618d01f90852a2d790c6d23 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 12:45:31 +0600 Subject: [PATCH 142/509] docs: rewrite README for integrated LocalDevStack --- README.md | 277 ++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 173 insertions(+), 104 deletions(-) diff --git a/README.md b/README.md index 03939553..531315ef 100644 --- a/README.md +++ b/README.md @@ -1,177 +1,246 @@ # LocalDevStack -LocalDevStack provides an easy-to-use Docker-based development environment for your projects. -All modules are **selective** and can be enabled via environment settings (Compose profiles). -Supports **multiple domains** and local TLS. +**Docker-based XAMPP alternative for PHP and Node.js local development.** -> 1) Local development only. -> 2) Your domain(s) must be resolvable on your host: -> - add entries to your hosts file: ` `, **or** -> - use a DNS that resolves to your machine, **or** -> - use `*.localhost` (no hosts entry required in many setups) +LocalDevStack gives you local domains, trusted HTTPS, selectable PHP/Node runtimes, databases and admin UIs, Mailpit, background jobs, and optional local AI behind one `lds` CLI. -## Prerequisites (Docker) +## What you get -Install docker on your system first. If you already have docker installed, you can skip this step. -- It is recommended to use [Docker Engine](https://docs.docker.com/engine/install/). -- If Docker Engine not supported in your OS, use [Docker Desktop](https://docs.docker.com/desktop/) (although you can also install this on linux as well). +- Nginx as the front door on ports 80/443, with optional Apache behind it. +- Interactive PHP and Node runtime selection per domain. +- PostgreSQL, MySQL, MariaDB, MongoDB, Redis, and Elasticsearch profiles. +- CloudBeaver, RedisInsight, Mongo Express, and Kibana where applicable. +- Persistent Mailpit with local SMTP TLS. +- Runner-based cron, Supervisor, and log rotation. +- Optional local AI via `llm-sm`, available internally at `http://llm-sm:11434` and through `https://llm.localhost`. +- Dynamic Docker networking through service DNS; no fixed `172.28/29/30` addresses. -## Supported Project Languages +LocalDevStack is for **trusted local development infrastructure**, not production deployment. -- PHP -- NodeJs +## Prerequisites ---- +Install Docker first: -## Quickstart +- Docker Engine is preferred on Linux. +- Docker Desktop is supported on Windows/macOS and can also be used on Linux. +- Windows CLI access uses `lds.bat` + Git Bash. -### 0) Default Layout +## Recommended layout -``` +```text project-root/ ├─ application/ │ ├─ site1/ │ ├─ site2/ │ └─ ... -└─ LocalDevStack/ (this repository) +└─ LocalDevStack/ ``` -This layout is flexible. If you want a different projects folder, set `PROJECT_DIR` in your env. +Set `PROJECT_DIR` in `docker/.env` when your project directory lives elsewhere. -```bash -# supports relative/absolute path (recommended to use absolute path for less confusion) -PROJECT_DIR=/path/to/your/projects -``` +## Quick start -### 1) Clone ```bash git clone https://github.com/infocyph/LocalDevStack.git cd LocalDevStack -```` - -### 2) Setup global shortcut and Permissions -On Linux/Mac, -```bash chmod +x ./lds 2>/dev/null || true -sudo ./lds setup permissions -``` -On Windows, -```cmd -./lds setup permissions +sudo ./lds setup permissions # Linux/macOS +./lds setup init +./lds setup profile +./lds up +./lds setup domain ``` -Once ran, it will add a globally available shortcut (lds). And necessary permissions(linux/mac) will be assigned as well. +On Windows, run the equivalent commands through `lds.bat` or Git Bash. The permissions command configures the wrapper path and does not apply Unix chmod logic. -### 3) Start the stack +Install the generated LocalDevStack root CA when you want browser-trusted HTTPS: ```bash -lds start +sudo lds certificate install ``` -### 4) Add a domain (vhost wizard) +## Common commands ```bash +lds help + +lds up +lds down +lds restart +lds status +lds logs [service] +lds rebuild [all|service...] + +lds setup profile lds setup domain -``` +lds profiles list -### 5) Optional: trust HTTPS locally (Root CA) +lds urls +lds images +lds doctor -```bash -sudo lds certificate install +lds config show +lds config show --json +lds config show --raw # explicitly shows secret-bearing effective config +lds config env-used +lds config validate ``` +`lds config show` is **redacted by default**. -## Command hints (quick reference) +## Runtime selection +The domain wizard keeps the runtime selector as the source of truth: -```bash -lds help +- PHP: select the PHP version for that domain. +- Node: select a Node version/tag for that domain. + +Generated local images remain version-specific: + +```text +localdevstack-php: +localdevstack-node: ``` -### Core stack +Selected PHP and Node images use Alpine variants. -```bash -lds up # start stack -lds start # alias of up -lds stop # stop stack (down) -lds down # alias of stop -lds restart # stop + up + HTTP reload -lds reload # recreate + HTTP reload -lds rebuild all # rebuild/pull images -lds config # show resolved docker compose config -lds http reload # reload the HTTP load balancer (nginx/apache) -lds tools # shell into SERVER_TOOLS container -lds doctor # host diagnostics +The local build path consumes Scriptomatic from `main` by default. Advanced users/releases may set `SCRIPTOMATIC_REF` to a full 40-character commit SHA. + +## Service image policy + +Default image selection follows this rule: + +> Prefer the moving Alpine variant when that image family publishes a suitable one; otherwise use its normal moving latest tag. + +Examples: + +- PostgreSQL: `postgres:alpine` +- Tools / Runner / Nginx / Apache: their published `:latest` images (already built on their intended Alpine bases) +- LLM: `infocyph/llm-sm:latest` +- AMD LLM: `infocyph/llm-sm:amd-latest` + +Elasticsearch, Kibana, and Filebeat are kept on one aligned version because those image families do not provide a supported moving `latest` contract for this stack. + +Run `lds images` to see the effective image set after release, user, and shell overrides. + +## Profiles and environment + +Tracked product defaults live in: + +```text +docker/release.env ``` -### Setup +User stack settings live in: -```bash -lds setup init -lds setup permissions -lds setup profile # (or: lds setup profiles) choose which services to configure -lds setup domain +```text +docker/.env ``` -### Certificates +Precedence is: -```bash -lds certificate install -lds certificate uninstall -lds certificate uninstall --all +```text +built-in fallback + < docker/release.env + < docker/.env + < command-scoped shell environment ``` -### Run (ad-hoc Dockerfile runner) +LocalDevStack never shell-sources `docker/.env` as executable code. + +## Optional local AI + +Enable the `ai` profile from the profile setup flow, then choose the runtime mode: ```bash -lds run -lds run --publish 8025:8025 -lds run ps -lds run logs -lds run stop -lds run rm -lds run open 8025 +lds llm runtime cpu +lds llm runtime nvidia +lds llm runtime amd ``` -### Shortcuts +Useful commands: ```bash -# Run your hosts file using container -lds php -v -lds composer install -lds node -v -lds npm i -lds npx - -# Login into the service containers -lds my --login -lds maria --login -lds pg --login -lds redis --login - -# if You are not getting access to certain IP that is used via vpn -lds vpn-fix +lds ai status +lds ai ask "explain this error" +lds ai troubleshoot ... +lds llm models +lds llm pull +lds llm chat ... ``` -> Tip: If you forget anything, `lds help` is the source of truth. +Models persist in the `LLMModels` named volume. -## CLI help (built-in “man”) +Default access is through: + +```text +https://llm.localhost +``` + +Direct Ollama host access is disabled by default. When explicitly enabled: ```bash -lds help +lds llm host-port on ``` ---- +it binds only to: -## Documentation +```text +127.0.0.1:11434 +``` -This README stays intentionally short. +The LLM container does **not** receive the Docker socket or a project mount by default. -* Full documentation: https://docs.infocyph.com/projects/LocalDevStack -* Quick reference: `lds help ...` +## Storage and trust boundaries + +Important named volumes include: + +- `NginxHosts` / `ApacheHosts` +- `SSLKeys` / `SSLRootCA` +- `FPMPools` / `FPMSocks` +- database/admin-client stores +- `EmailStore` +- `LLMModels` + +Host-owned configuration remains under `configuration/`, including PHP overrides, scheduler files, SOPS data, optional SSH material, and generated Compose fragments. + +`server-tools` and `runner` intentionally mount `/var/run/docker.sock`. That socket is equivalent to powerful host Docker control and is limited to those trusted control-plane components. Ordinary databases/admin clients and `llm-sm` do not receive it. + +## Networking + +Core networks are still logically separated as: + +```text +Frontend +Backend +DataStore +``` + +Docker assigns their subnets dynamically. Services communicate by Docker DNS names such as: + +```text +server-tools +runner +mailpit +postgres +mysql +mariadb +mongodb +redis +elasticsearch +llm-sm +``` + +The legacy `lds vpn-fix` command is deprecated because LocalDevStack no longer owns fixed private subnets. + +## Documentation ---- +- Full docs: https://docs.infocyph.com/projects/LocalDevStack +- Quick reference: `lds help` +- Local AI: `docs/guides/local-ai.rst` +- Profiles/env: `docs/concepts/profiles-and-env.rst` +- Storage: `docs/concepts/storage-layout.rst` ## License From 1fe099345b037778879d32b7fa93d664d2531ddd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:04:44 +0600 Subject: [PATCH 143/509] tls: use current exported root CA path with legacy fallback --- lib/certificates.sh | 25 +++++++++++++++++++++---- 1 file changed, 21 insertions(+), 4 deletions(-) diff --git a/lib/certificates.sh b/lib/certificates.sh index 74d692f6..a2075ee1 100644 --- a/lib/certificates.sh +++ b/lib/certificates.sh @@ -46,6 +46,21 @@ env_init() { CA_BASENAME="localdevstack-rootca" CA_NICK="LocalDevStack Root CA" +host_root_ca_path() { + local current="$DIR/configuration/ssl/rootCA.pem" + local legacy="$DIR/configuration/rootCA/rootCA.pem" + + if [[ -r "$current" ]]; then + printf '%s' "$current" + return 0 + fi + if [[ -r "$legacy" ]]; then + printf '%s' "$legacy" + return 0 + fi + return 1 +} + detect_os_family() { # Output: "id|like" # Must never fail under set -e @@ -144,8 +159,9 @@ install_ca_nss_user() { install_ca_windows() { need_windows_tools - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + local src_ca + src_ca="$(host_root_ca_path || true)" + [[ -n "$src_ca" ]] || die "certificate not found: expected configuration/ssl/rootCA.pem (legacy: configuration/rootCA/rootCA.pem)" local win_ca win_ca="$(cygpath -w "$src_ca")" @@ -257,8 +273,9 @@ install_ca() { uninstall_ca_windows() { need_windows_tools - local src_ca="$DIR/configuration/rootCA/rootCA.pem" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + local src_ca + src_ca="$(host_root_ca_path || true)" + [[ -n "$src_ca" ]] || die "certificate not found: expected configuration/ssl/rootCA.pem (legacy: configuration/rootCA/rootCA.pem)" local win_ca win_ca="$(cygpath -w "$src_ca")" From 6c7f91c05e820d7b81756b7d711e0c04793325d0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:05:10 +0600 Subject: [PATCH 144/509] test: enforce public root CA export bridge --- tests/service-hardening-contract.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index 0e40a8f3..0aeb9d4f 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -44,3 +44,10 @@ fi [[ ! -e "$ROOT/docker/conf/www.conf" ]] || fail "unused legacy docker/conf/www.conf must remain removed" assert_file "$ROOT/docker/conf/www-php.conf" pass "Docker config ownership is explicit" + +certs="$ROOT/lib/certificates.sh" +assert_file_contains "$companion" '../../configuration/ssl:/etc/share/certs' +assert_file_contains "$certs" 'local current="$DIR/configuration/ssl/rootCA.pem"' +assert_file_contains "$certs" 'local legacy="$DIR/configuration/rootCA/rootCA.pem"' +assert_file_contains "$certs" 'src_ca="$(host_root_ca_path || true)"' +pass "certificate export bridge uses the current public host path with legacy fallback" From 77b26a4d9bc3cf4be7179572d29c967de8ced1ce Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:05:12 +0600 Subject: [PATCH 145/509] test: enforce Tools public TLS export contract --- tests/published-images.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/tests/published-images.sh b/tests/published-images.sh index 49b050a5..22af80bb 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -107,3 +107,11 @@ pass "latest Runner preserves logrotate/health contract" assert_contains "$php_template" './docker/conf/www-php.conf:/usr/local/etc/php-fpm.d/www.conf' pass "generated PHP runtime uses the maintained FPM pool config" + +tools_certify="$( + docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /usr/local/bin/certify +)" +assert_contains "$tools_certify" 'EXPORT_DIR="${EXPORT_DIR:-/etc/share/certs}"' +assert_contains "$tools_certify" 'EXPORT_ROOTCA_NAME="${EXPORT_ROOTCA_NAME:-rootCA.pem}"' +assert_contains "$tools_certify" 'atomic_install 0644 "$root_ca" "$EXPORT_DIR/$EXPORT_ROOTCA_NAME"' +pass "latest Tools public TLS export contract" From 591f92deffce6cde986de94634ad62e54c4814ba Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:08:51 +0600 Subject: [PATCH 146/509] docs: refresh quickstart --- docs/quickstart.rst | 153 ++++++++++++++++++-------------------------- 1 file changed, 61 insertions(+), 92 deletions(-) diff --git a/docs/quickstart.rst b/docs/quickstart.rst index 2d4d294e..cf71bae9 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -1,137 +1,106 @@ Getting Started =============== -Lets start with simple basics first without too many fuzz. LocalDevStack is a modular Docker-based -local dev stack orchestrated by the ``lds`` CLI and powered by Compose profiles. +LocalDevStack is a Docker-based XAMPP alternative for PHP and Node.js local development. +The lds CLI manages Compose profiles, local domains, TLS, runtime builds, databases, +admin tools, background jobs, and optional local AI. Prerequisites ------------- -Install Docker on your system first. If you already have Docker installed, you can skip this step. +Install Docker first. -- Recommended: Docker Engine (Linux) for best performance and lowest overhead. -- If Docker Engine is not supported on your OS, use Docker Desktop (Windows/macOS; can also be used on Linux). +- Docker Engine is preferred on Linux. +- Docker Desktop is supported on Windows and macOS. +- Windows CLI access uses lds.bat with Git Bash. Quick Start ----------- -1. Make ``lds`` executable and apply permissions +Clone the repository and enter it:: - Linux/macOS:: + git clone https://github.com/infocyph/LocalDevStack.git + cd LocalDevStack - chmod +x ./lds - sudo ./lds setup permissions +On Linux/macOS, prepare the CLI and host permissions:: - Notes: + chmod +x ./lds + sudo ./lds setup permissions - - On Linux, the permissions step is recommended to avoid common volume/UID permission issues. - - On Windows, you typically run the wrapper (example: ``lds.bat``) and may need to add the project root directory - to your Environment PATH if you want ``lds`` usable from any directory. +Then initialize LocalDevStack and choose optional services:: -2. Start the stack:: + ./lds setup init + ./lds setup profile - lds start +Start the stack:: -3. Add a domain (generates vhosts and updates stack selection):: + ./lds up - lds setup domain +Create your first local domain:: -4. (Optional) Generate and trust TLS certificates + ./lds setup domain - If you enabled HTTPS vhosts and want browser trust, use the TLS workflow described in: +The domain wizard asks for the application type and runtime version. PHP and Node +selection remains version-specific; the selected value becomes the runtime image identity. - - :doc:`guides/tls-and-certificates` +For browser-trusted HTTPS, install the exported LocalDevStack root CA:: -Directory Structure -------------------- - -Recommended project layout -~~~~~~~~~~~~~~~~~~~~~~~~~ - -By default, keep your projects in a sibling directory (simple and predictable):: - - project-root/ - ├─ application/ - │ ├─ site1/ - │ ├─ site2/ - │ └─ ... - └─ LocalDevStack/ (this repository) - -This layout is flexible. If you want a different projects folder, set ``PROJECT_DIR`` in your env. + sudo ./lds certificate install -Example:: +On Windows, run the same workflow through lds.bat or Git Bash. The permissions +command configures the wrapper path and does not apply Unix chmod logic. - PROJECT_DIR=../path/to/your/projects # supports relative/absolute path (recommended to use absolute path for less confusion) - -LocalDevStack layout -~~~~~~~~~~~~~~~~~~~~ - -Where things live:: - - LocalDevStack/ - ├─ bin/ # optional helper binaries / shims (Don't touch) - ├─ configuration/ # These are created and persisted according to the process you follow (generated automatically) - │ ├─ apache/ # Generated apache vhost configs (if you use apache mode) - │ ├─ nginx/ # Generated nginx vhost configs (primary entry in most setups) - │ ├─ php/ # php.ini and php overrides (The php.ini you see here can be modified) - │ ├─ ssh/ # ssh keys (optional; useful for git over ssh inside containers) - │ ├─ ssl/ # generated TLS certs (can be used for trusting systemwide) - │ └─ rootCA/ # local CA store (The generated root certificate) - ├─ docker/ # internal stack definition (Don't touch) - │ ├─ compose/ # main.yaml + service fragments (http/php/db/tools/... etc) - │ ├─ conf/ # container configuration templates/snippets - │ ├─ data/ # persistent service data (db volumes etc.) - │ └─ logs/ # container logs (if your stack writes here) - ├─ .env # Docker only env (generated by tools, Don't touch) - ├─ lds # main CLI (Linux/macOS) (Don't touch) - └─ lds.bat # Windows wrapper (Don't touch) - -Run the server (the easy way) ------------------------------ +Useful First Checks +------------------- -1. Create or update env files (minimum: profiles + project dir). +Show the active convenience URLs:: - Typical locations used by this stack: + lds urls - - ``docker/.env`` (stack settings / profiles) - - ``.env`` (project-level env, exposed to your projects) +Show effective infrastructure/runtime image defaults:: -2. Start the stack:: + lds images - lds start +Run non-destructive diagnostics:: -3. Add domains via the wizard (recommended):: + lds doctor - lds setup domain +Validate the effective Compose and scheduler configuration:: -Usage ------ + lds config validate -Common commands:: +Show effective Compose configuration with secret values redacted:: - lds start - lds stop - lds reload - lds restart - lds rebuild + lds config show -HTTP utilities:: +Use lds config show --raw only when you intentionally need the unredacted output. - lds http reload +Project Layout +-------------- -Shells:: +A common layout is:: - lds core + project-root/ + ├─ application/ + │ ├─ site1/ + │ ├─ site2/ + │ └─ ... + └─ LocalDevStack/ -Notes: +Set PROJECT_DIR in docker/.env when your application directory is elsewhere. -- ``lds core `` is intended to open the right runtime container shell for that domain. +LocalDevStack keeps host-managed state under configuration/ and logs/ while +runtime vhosts, certificates, databases, and other service data primarily live in +named Docker volumes. -Next steps +Next Steps ---------- -- Domain and vhosts: :doc:`guides/domain-setup` -- Local TLS (mkcert + certify): :doc:`guides/tls-and-certificates` -- Node apps behind Nginx: :doc:`guides/node-apps` -- Encrypted secrets (SOPS + Age): :doc:`guides/secrets-sops-age` -- Notifications: :doc:`guides/notifications` +- Profiles and environment: concepts/profiles-and-env +- Architecture: concepts/architecture +- Storage: concepts/storage-layout +- Domain setup: guides/domain-setup +- TLS: guides/tls-and-certificates +- Local AI: guides/local-ai +- Encrypted secrets: guides/secrets-sops-age +- Notifications: guides/notifications From 295388c9290b85e4b8ddc64505c0deea2ab160bb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:08:55 +0600 Subject: [PATCH 147/509] docs: refresh architecture --- docs/concepts/architecture.rst | 118 +++++++++++++++++++++++++++------ 1 file changed, 96 insertions(+), 22 deletions(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index b7d15a1d..af2b42e0 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -1,33 +1,107 @@ Architecture ============ -LocalDevStack is composed of: +LocalDevStack is split into small, purpose-specific layers. -- The **orchestrator**: ``lds`` / ``lds.bat`` (selects profiles, runs Compose, common workflows) -- The **HTTP layer**: Nginx (front proxy) and optionally Apache (backend HTTP) depending on your stack choice -- The **runtimes**: PHP (FPM) and Node (and future stacks) -- The **control plane**: the **server-tools** image (domain/vhost generation, TLS automation, secrets helpers) -- The **runner**: supervisord + cron + logrotate and helper exec wrappers +Core Components +--------------- -Key idea --------- +lds / lds.bat + Host-side orchestrator for setup, profiles, Compose, diagnostics, runtime rebuilds, + domains, TLS, and convenience commands. -Instead of a monolithic "one container does everything" model, LocalDevStack uses: +Nginx + Front door for local HTTP/HTTPS traffic. It routes PHP-FPM, optional Apache, + Node applications, admin UIs, Mailpit, and llm.localhost. -- Compose profiles to enable only what you need -- Generated configuration artifacts (vhosts, certificates) persisted on the host -- Stable container names/hostnames to keep local routing predictable +Apache + Optional HTTP backend for projects that explicitly choose the Apache path. -How containers cooperate ------------------------- +PHP / Node runtimes + Locally built, version-specific runtime images. The domain wizard preserves the + selected PHP/Node version and uses Alpine variants. -1. You generate vhost configs (via ``lds setup domain``). -2. The Tools container can scan all vhosts and generate certificates. -3. Nginx loads hosts and routes requests either: +Tools + Trusted control plane for vhost generation, certificates, admin UI, secrets, + Git helpers, monitoring, and AI-consumer commands. - - directly to PHP-FPM (fastcgi) or - - to Apache (reverse proxy) when Apache mode is enabled or - - to a Node service (reverse proxy). +Runner + Background execution layer for Supervisor, cron, and log rotation. -4. The Runner handles background services (cron/logrotate) and gives you a consistent place for helper utilities. -5. You also get following services: EMail, DB, Caching +llm-sm + Optional local AI provider. It is enabled only through the ai profile and is + separate from Tools. + +Networking +---------- + +LocalDevStack keeps three logical networks:: + + Frontend + Backend + DataStore + +Docker assigns their address ranges dynamically. Core services do not depend on +hard-coded 172.28/29/30 addresses. + +Service-to-service communication uses Docker DNS names such as:: + + server-tools + runner + mailpit + nginx + apache + postgres + mysql + mariadb + mongodb + redis + elasticsearch + llm-sm + +The legacy lds vpn-fix workflow is deprecated because LocalDevStack no longer owns +fixed private subnets. + +Runtime Flow +------------ + +A normal web-domain flow is: + +1. lds setup domain delegates domain/runtime generation to Tools. +2. Tools writes HTTP vhosts into persistent named volumes. +3. Tools writes runtime Compose fragments under configuration/compose/. +4. Nginx routes by Docker service name or PHP-FPM socket. +5. Selected PHP/Node runtime images are built only for the chosen versions. + +For optional AI: + +1. the ai profile starts llm-sm; +2. Tools consumes http://llm-sm:11434 internally; +3. Nginx exposes https://llm.localhost; +4. lds ai delegates operational AI to Tools; +5. lds llm delegates provider/model management to llm-sm. + +Trust Boundaries +---------------- + +server-tools and runner intentionally receive /var/run/docker.sock because their +supported workflows control sibling containers. Docker socket access is equivalent +to powerful host Docker control. + +The Docker socket is not mounted into: + +- llm-sm; +- databases; +- database admin clients; +- Nginx/Apache; +- ordinary runtime services. + +llm-sm also receives no project/repository mount by default. AI output is not +automatically executed as shell, SQL, or code. + +Persistence +----------- + +Runtime-generated vhosts, certificate material, databases, Mailpit state, runtime +sockets/pools, and AI models use named volumes. Host-editable/generated configuration, +logs, SOPS state, optional SSH material, and public TLS exports remain under the repository. From 0f88b9ba7f44d5165bf42e87abc701779214cbf3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:08:58 +0600 Subject: [PATCH 148/509] docs: refresh profiles-and-env --- docs/concepts/profiles-and-env.rst | 142 +++++++++++++++++++++++++---- 1 file changed, 123 insertions(+), 19 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index 310978e5..c1ca4992 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -1,37 +1,141 @@ Profiles and Environment ======================== -LocalDevStack uses Docker Compose **profiles** so you can enable only the services you want for a given project. +LocalDevStack uses Docker Compose profiles so optional services are selected explicitly. -Setting up Environment usable by Your Projects ----------------------- +Guided Profile Setup +-------------------- -- Use root ``.env`` (project-level overrides) -- These are common & directly shared to projects you host -- If you want project specific only a single project you need to handle that inside your project +Use:: -Guided setup for on demands services (DB, Cache) ------------- + lds setup profile + +The host-side service catalog is tracked at:: + + docker/catalog/services.psv + +It describes the service/profile name, setup defaults, prompts, convenience URL, +persistent volume metadata, and supported AI runtime modes. + +The current optional service profiles include: + +- PostgreSQL; +- MySQL; +- MariaDB; +- MongoDB; +- Redis; +- Elasticsearch; +- local AI. + +PHP and Node domain runtimes are selected separately by the domain wizard. Their +version selector is not replaced by the moving-image policy. + +Environment Ownership +--------------------- + +Tracked product defaults live in:: + + docker/release.env + +User stack settings live in:: + + docker/.env + +The repository root .env remains project/application-facing state where applicable; +it is not the LocalDevStack release-default file. + +Effective precedence is:: + + built-in fallback + < docker/release.env + < docker/.env + < command-scoped shell environment + +LocalDevStack reads dotenv values as data. It does not blindly shell-source docker/.env. + +Useful Commands +--------------- + +Show which environment keys are defined without printing values:: + + lds config env-used + +Show effective Compose config with secrets redacted:: + + lds config show + +Show raw effective Compose config only when intentionally needed:: + + lds config show --raw + +Validate Compose and mounted scheduler configuration:: + + lds config validate + +Show selected profiles:: + + lds profiles list + +Image Defaults +-------------- + +The default policy is: + + Prefer the moving Alpine variant when the image family provides a suitable one; + otherwise use its normal moving latest tag. + +Examples: + +- PostgreSQL defaults to postgres:alpine. +- Tools, Runner, Nginx, and Apache use their published :latest aliases. +- Standard local AI uses infocyph/llm-sm:latest. +- AMD local AI uses infocyph/llm-sm:amd-latest. +- Elasticsearch, Kibana, and Filebeat stay on one aligned version because their + stack does not provide the required moving latest contract. + +Explicit user values in docker/.env or the shell override these defaults. + +Runtime Version Selection +------------------------- + +Tools publishes the runtime catalog used by mkhost. The user still chooses the +runtime version per domain. + +PHP selection becomes: + +- PHP_VERSION build input; +- localdevstack-php: image identity; +- Alpine PHP-FPM base. -The ``lds`` CLI typically includes helpers like: +Node selection becomes: -- ``lds setup profiles`` +- NODE_VERSION build input; +- localdevstack-node: image identity; +- Alpine Node base. -These helpers are opinionated: they try to keep profiles and generated configs consistent. +Runtime rebuilds preserve Docker build cache while using --pull to refresh the +selected base. -Manual setup (Don't use unless you are fully aware of internals) +Scriptomatic ------------ -Enable PHP 8.4 + MariaDB + Redis: +Runtime builds consume Scriptomatic from main by default:: -.. code-block:: none + SCRIPTOMATIC_REF=main - COMPOSE_PROFILES=php84,mariadb,redis +A full 40-character commit SHA is also accepted for release/debug reproducibility. -Enable Apache mode (Nginx -> Apache -> PHP-FPM): +AI Settings +----------- -.. code-block:: none +Important AI settings include:: - COMPOSE_PROFILES=apache,php84 + LDS_AI_ENABLED=auto + LDS_AI_PROVIDER=ollama + LDS_AI_URL=http://llm-sm:11434 + LDS_AI_MODEL=qwen2.5:3b + LDS_AI_RUNTIME=cpu + LDS_LLM_HOST_PORT=0 -These are just some of the samples. +Use lds llm runtime and lds llm host-port instead of editing these manually for +normal workflows. From 9cf931fa75f0eda45677871ac262074767397eb6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:09:03 +0600 Subject: [PATCH 149/509] docs: refresh storage-layout --- docs/concepts/storage-layout.rst | 114 ++++++++++++++++++++++++++----- 1 file changed, 96 insertions(+), 18 deletions(-) diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 592c548f..3703720a 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -1,26 +1,104 @@ Storage Layout ============== -To keep LocalDevStack reproducible, generated artifacts are persisted on the host and mounted into containers. +LocalDevStack deliberately separates Docker-managed runtime state from host-managed +configuration and exports. -Directories ------------------------- +Named Volumes +------------- -LocalDevStack uses a ``configuration/`` root (mounted into the stack). Keep all user-managed and generated -artifacts here: +Important named volumes include: -- ``configuration/nginx``: Nginx host configs (primary entry in most setups) -- ``configuration/apache``: Apache host configs (only if Apache mode is enabled) -- ``configuration/ssl``: generated certificates -- ``configuration/rootCA``: Root CA store (persist this to keep browser trust stable, see :doc:`tls-and-certificates`.) -- ``configuration/php``: PHP runtime ini overrides (e.g., ``php.ini``) -- ``configuration/ssh``: optional SSH mount (useful for private repos, git over SSH or tooling) -- ``configuration/sops``: optional SOPS/Age keys + config (if you use the secrets workflow) +NginxHosts / ApacheHosts + Generated HTTP vhost configuration consumed by Nginx/Apache. -Why this matters ----------------- +SSLKeys / SSLRootCA + Runtime certificate/key material and the mkcert CA store. -- Keeping hosts stable keeps host and certificate generation stable. -- Persisting the Root CA avoids repeated trust resets and browser warnings. -- Persisting ``php.ini`` overrides keeps runtime behavior consistent between rebuilds. -- Persisting SOPS/Age configuration avoids re-creating keys and keeps secrets workflows predictable. +FPMPools / FPMSocks + Generated PHP-FPM pool configuration and runtime sockets. + +ComposerGlobal / GitConfig + Shared runtime state for Composer and Git integration. + +Database/admin volumes + Persistent PostgreSQL, MySQL, MariaDB, MongoDB, Redis, Elasticsearch, Kibana, + RedisInsight, CloudBeaver, and Filebeat state. + +EmailStore + Mailpit persistence. + +LLMModels + Ollama/local-model persistence when the ai profile is enabled. + +Named volumes are intentionally not renamed during this integration release so existing +developer data can survive upgrades. + +Host Configuration +------------------ + +configuration/compose/ + LocalDevStack-generated runtime Compose fragments. The CLI discovers only YAML files + in this directory and validates the effective Compose graph before use. + +configuration/php/ + User-editable PHP configuration. Normal updates must not overwrite user customizations. + +configuration/scheduler/cron-jobs/ + Runner cron definitions. + +configuration/scheduler/supervisor/ + Runner Supervisor definitions. + +configuration/sops/config/ +configuration/sops/global/ +configuration/sops/keys/ + SOPS/Age configuration and sensitive key material. + +configuration/ssh/ + Optional SSH material mounted read-only into supported runtime/tool flows. + +configuration/ssl/ + User-facing TLS exports from Tools. The public root CA is exported here as + rootCA.pem. Optional password-protected mTLS user artifacts may also appear here. + +logs/ + Host-visible service logs consumed by Runner log rotation and diagnostics. + +TLS Authority +------------- + +Runtime TLS state is owned by the named volumes: + +- SSLKeys for server/client certificate material; +- SSLRootCA for the mkcert CA store. + +Tools exports the public root certificate to:: + + configuration/ssl/rootCA.pem + +lds certificate install uses that current path. Older installations using +configuration/rootCA/rootCA.pem remain readable as a migration fallback. + +The private CA key is not intended as a public host export. + +Permissions +----------- + +On Unix-like hosts, lds setup permissions uses group-writable setgid directories +for configuration/ and logs/ rather than broad world-writable 777 modes. + +Sensitive host directories are restricted: + +- configuration/ssh/ directories: 0700; files: 0600; +- configuration/sops/keys/ directories: 0700; files: 0600. + +Windows keeps its platform-specific wrapper/permission behavior. + +Project Mount +------------- + +PROJECT_DIR controls the application bind mount. A common layout is a sibling +application/ directory, but absolute or relative alternatives are supported. + +Local AI receives no application/project bind mount by default. From e598ccded78ff78a50fbb3331000d32890597c2a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:09:07 +0600 Subject: [PATCH 150/509] docs: refresh domain-setup --- docs/guides/domain-setup.rst | 116 ++++++++++++++++++++++------------- 1 file changed, 73 insertions(+), 43 deletions(-) diff --git a/docs/guides/domain-setup.rst b/docs/guides/domain-setup.rst index 78467787..46a7ac95 100644 --- a/docs/guides/domain-setup.rst +++ b/docs/guides/domain-setup.rst @@ -1,68 +1,98 @@ Domain Setup ============ -LocalDevStack creates domains and vhosts, when you run ``lds setup domain``. +Use the interactive domain wizard:: -How ``lds setup domain`` works ------------------------------- + lds setup domain -1. Run the interactive wizard. -2. Enable the selected profiles. -3. Bring the stack up and reload HTTP. +The wizard delegates generation to the Tools mkhost workflow while LocalDevStack +owns the surrounding Compose/profile/runtime orchestration. -Wizard flow (what the user answers) ------------------------------------ +Wizard Flow +----------- -``lds setup domain`` runs an interactive 8-step flow: +The wizard collects the information needed for the selected application type, including: -1. Domain name -2. App type (PHP or NodeJs) -3. Runtime version (PHP Major.Minor, or Node major/tags) -4. Server type (PHP: Nginx or Apache; Node: Nginx forced + optional Node start command) -5. Protocol (HTTP only / HTTPS only / both + optional redirect) -6. Document root (relative path mapped under ``/app``) -7. Client max body size -8. Mutual TLS toggle (only available when HTTPS is enabled; this requires client side certificate) +1. domain name; +2. PHP or Node application type; +3. runtime version; +4. HTTP server mode where applicable; +5. HTTP/HTTPS behavior; +6. document root; +7. request/body limits; +8. optional mutual TLS settings. -What it generates +Runtime Version Selection ------------------------- -Vhost configs -~~~~~~~~~~~~~~~~~~ +Runtime selection is interactive and version-specific. -Writes generated vhost files: +For PHP, the selected version remains the PHP_VERSION build input and produces:: -- Nginx vhost: - ``configuration/nginx/.conf`` + localdevstack-php: -- Apache vhost (only when Apache mode is selected): - ``configuration/apache/.conf`` +For Node, the selected version/tag remains the NODE_VERSION build input and produces:: -TLS handling (HTTPS) -~~~~~~~~~~~~~~~~~~~~ + localdevstack-node: -If you select HTTPS in the wizard, after writing the HTTPS config; -this generates/refreshes certificates for all known hosts. +Both runtime families use Alpine variants. -See: :doc:`tls-and-certificates` +Generated State +--------------- -Node apps (optional) -~~~~~~~~~~~~~~~~~~~~ +The current architecture does not write active Nginx/Apache vhosts to +configuration/nginx or configuration/apache. -If you choose **NodeJs** app type: +Instead: -- It generates a Node compose fragment: +- Nginx vhosts persist in the NginxHosts named volume; +- Apache vhosts persist in the ApacheHosts named volume; +- PHP-FPM pool state persists in FPMPools; +- generated runtime Compose fragments are written under configuration/compose/. - ``docker/extras/.yaml`` +Use:: -The token is derived from the domain (slugified). -This compose fragment defines a Node service (internal port is always ``3000``) and sets a profile like: + lds config validate -- ``node_`` +to validate the effective Compose graph and mounted scheduler configuration. -Tips ----- +Routing +------- -- Prefer a consistent domain scheme (e.g., ``project.localhost``) so your routing stays predictable. -- After any vhost/cert changes, ``lds`` will run ``lds http reload`` automatically as part of setup; - you can also run it manually when you edit configs yourself. +LocalDevStack uses Docker DNS/service names instead of fixed bridge addresses. Generated +HTTP configuration routes to logical runtime service names or PHP-FPM sockets. + +The three logical networks remain Frontend, Backend, and DataStore, but Docker chooses +their address ranges dynamically. + +TLS +--- + +When HTTPS is selected, Tools refreshes the shared LocalDevStack certificate set. The +certificate SAN set always includes localhost, *.localhost, 127.0.0.1, and ::1 in +addition to generated domains/service-derived hosts. + +This means convenience hosts such as admin.localhost, webmail.localhost, and +llm.localhost can use the same LocalDevStack trust chain. + +Convenience Commands +-------------------- + +List active convenience URLs:: + + lds urls + +Open a known UI/domain:: + + lds open admin + lds open mail + lds open db + lds open redis + lds open mongo + lds open kibana + lds open ai + lds open project.localhost + +Run diagnostics without mutating the stack:: + + lds doctor From 76ff007c139b809df0e813d65f3cc1d32c5b3e49 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:09:11 +0600 Subject: [PATCH 151/509] docs: refresh tls-and-certificates --- docs/guides/tls-and-certificates.rst | 170 +++++++++++++-------------- 1 file changed, 79 insertions(+), 91 deletions(-) diff --git a/docs/guides/tls-and-certificates.rst b/docs/guides/tls-and-certificates.rst index ffe9153b..a980acc0 100644 --- a/docs/guides/tls-and-certificates.rst +++ b/docs/guides/tls-and-certificates.rst @@ -1,128 +1,116 @@ TLS and Certificates ==================== -LocalDevStack uses **mkcert-based local TLS** for development. +LocalDevStack uses mkcert-based local TLS for development. Tools owns certificate +generation; Nginx, Apache, Mailpit, and runtime consumers receive only the mounts needed +by the LocalDevStack runtime contract. -At first run, it creates a **local Root CA** and issues development certificates. -After that, it scans your vhost configs and (re)generates certificates for all detected domains. +Runtime TLS State +----------------- -Certificates are generated and persisted under the host-mounted ``configuration/`` tree so they survive rebuilds. +The active runtime state is stored in Docker named volumes: -Generated files (host) ----------------------- +SSLRootCA + mkcert CA store exposed to trusted LocalDevStack consumers at /etc/share/rootCA. -LocalDevStack persists TLS artifacts in this layout:: - - configuration/ - ├── rootCA - │ ├── rootCA-key.pem - │ └── rootCA.pem - └── ssl - ├── apache-client-key.pem - ├── apache-client.pem - ├── apache-server-key.pem - ├── apache-server.pem - ├── local-key.pem - ├── local.pem - ├── nginx-client-key.pem - ├── nginx-client.p12 - ├── nginx-client.pem - ├── nginx-proxy-key.pem - ├── nginx-proxy.pem - ├── nginx-server-key.pem - └── nginx-server.pem - -Notes: - -- ``configuration/rootCA/rootCA.pem`` is your local development CA certificate. -- The ``*-server*.pem`` pairs are used by Nginx/Apache for HTTPS. -- The ``*-client*.pem`` pairs are used when **mutual TLS** is enabled for a domain. -- ``nginx-client.p12`` is provided for convenient browser import when mutual TLS is enabled. - -Domain discovery ----------------- - -Certificate generation scans all ``*.conf`` files under the shared vhost directory (mounted from your host). -From those filenames/configs, LocalDevStack derives domain names and generates SAN certificates -covering all detected domains. - -This keeps certs aligned with your active vhost set: add/remove a domain, regenerate, done. - -Trusting the Root CA --------------------- +SSLKeys + generated server/client certificate material exposed at /etc/mkcert where needed. -To trust your local CA on your host system, run:: +Tools automatically refreshes certificates through certify. - sudo lds certificate install +Public Host Export +------------------ -This installs ``configuration/rootCA/rootCA.pem`` into your OS trust store (where supported). +Tools exports user-facing certificate artifacts to the host-mounted directory:: -Manual install is also possible: + configuration/ssl/ -- Import ``configuration/rootCA/rootCA.pem`` into your OS trust store using your system UI/tools. -- This is useful in locked-down environments where automated install is restricted. +The public root CA is:: -Mutual TLS (Client certificates) --------------------------------- + configuration/ssl/rootCA.pem -If you enable **mutual TLS** for any domain: +lds certificate install uses this path. For upgrades, the legacy +configuration/rootCA/rootCA.pem path remains a read fallback. -- You must install the client certificate in your browser. -- Recommended: import ``configuration/ssl/nginx-client.p12`` into the browser certificate store. +The public CA export is safe to install into the host trust store. The private CA key +is not intended as a user-facing export. -After importing, the browser will present the client certificate when accessing mTLS-protected domains. +Optional user mTLS export is disabled by default. When explicitly enabled in Tools, +a password-protected user P12 is exported under configuration/ssl/. + +Certificate Coverage +-------------------- -Uninstalling the Root CA ------------------------- +The generated certificate SAN set includes at least:: -If you previously trusted the LocalDevStack Root CA and want to remove it from your system trust store, use:: + localhost + *.localhost + 127.0.0.1 + ::1 - sudo lds certificate uninstall +Tools also discovers generated vhost/service domains. The *.localhost entry covers +built-in convenience endpoints such as: + +- admin.localhost; +- webmail.localhost; +- db.localhost; +- ri.localhost; +- me.localhost; +- kibana.localhost; +- llm.localhost. + +Installing the Root CA +---------------------- -This removes the installed CA file from the detected OS trust anchor location and then refreshes the system trust store -(best-effort). +Linux/macOS where supported:: -Remove from all known locations (cleanup mode) -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + sudo lds certificate install -If you changed distros, moved CA paths, or installed it manually in different locations, use:: +Windows through Git Bash/lds.bat uses the CurrentUser root certificate store and does +not require the Unix sudo path. - sudo lds certificate uninstall --all +After installation, restart browsers that cache trust decisions. -This additionally scans common CA anchor locations and removes any leftover ``rootCA`` entries it finds, then refreshes -the trust store. +Uninstalling +------------ -Notes -~~~~~ +Remove the LocalDevStack trust anchor:: -- Uninstall requires sudo/admin privileges. -- Trust store refresh is best-effort; on uncommon distributions you may need to refresh trust manually after removal. -- This only removes the *installed* OS trust anchor. It does not delete your generated CA files under - ``configuration/rootCA`` (those are part of your project persistence). + sudo lds certificate uninstall +Scan/remove known legacy anchor locations too:: + + sudo lds certificate uninstall --all + +This removes the host trust-store entry. It does not destroy the LocalDevStack named +certificate volumes. + +Mutual TLS +---------- + +When a domain enables mutual TLS, browser/user certificate material must be imported +separately. User-facing P12 export is intentionally opt-in and password-protected. + +Internal Nginx-to-Apache mTLS material remains runtime state and is not the same as the +user-facing client certificate. Troubleshooting --------------- -Browser still shows “Not Secure” -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -- Confirm the Root CA is trusted: +Check LocalDevStack health and TLS readiness:: - - Run ``lds certificate install`` again, or - - Manually install ``configuration/rootCA/rootCA.pem`` into your OS trust store. + lds doctor -- Restart the browser after installing the CA (some browsers cache trust decisions). +Inspect a domain TLS handshake:: -Certificate mismatch after changing domains -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + lds diag tls project.localhost -- If you renamed/removed domains, regenerate certs so SANs match the current vhost set. -- Ensure the vhost files under ``configuration/nginx`` / ``configuration/apache`` reflect the current domains. +Regenerate/diagnose through the existing certificate commands:: -Mutual TLS enabled but browser doesn’t prompt / request fails -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + lds cert status + lds cert regen all + lds cert diagnose project.localhost -- Import the client cert (recommended): ``configuration/ssl/nginx-client.p12``. -- Verify the client cert is imported into the *correct* browser profile. -- If you have multiple client certs, remove old ones and retry to avoid wrong-certificate selection. +If the host export is missing, ensure server-tools is running and certificate +generation has completed. The current public export should appear at +configuration/ssl/rootCA.pem. From ce0d131b40da0a3c46daef3a36c616816d6497d0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:09:14 +0600 Subject: [PATCH 152/509] docs: refresh index --- docs/index.rst | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index 37c7cc3b..dd8624bd 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,11 +1,9 @@ LocalDevStack Documentation ========================== -LocalDevStack is a modular, Docker-based local development stack designed to replace traditional local bundles -(XAMPP/MAMP/LAMP) with a reproducible, profile-driven setup. - -It is built around a small orchestrator (the ``lds`` CLI + Compose profiles) and a set of purpose-built images -that work together (tools, HTTP, runner). +LocalDevStack is a Docker-based XAMPP alternative for PHP and Node.js local development. +It provides profile-driven services, local domains and TLS, selectable runtimes, +background jobs, admin UIs, and optional local AI behind the lds CLI. .. toctree:: :maxdepth: 2 @@ -27,5 +25,6 @@ that work together (tools, HTTP, runner). guides/domain-setup guides/tls-and-certificates + guides/local-ai guides/secrets-sops-age guides/notifications From 2c03329159ed27fc2075fac346d1a83394c6f02a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:09:17 +0600 Subject: [PATCH 153/509] docs: add local AI guide --- docs/guides/local-ai.rst | 153 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 153 insertions(+) create mode 100644 docs/guides/local-ai.rst diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst new file mode 100644 index 00000000..1427ddef --- /dev/null +++ b/docs/guides/local-ai.rst @@ -0,0 +1,153 @@ +Local AI +======== + +LocalDevStack can run a local Ollama-compatible provider as an optional profile. The +provider/runtime is llm-sm; Tools remains an AI consumer and never embeds a second +Ollama runtime. + +Enable AI +--------- + +Use the guided service setup:: + + lds setup profile + +Select Local AI / the ai profile. + +The default provider contract is:: + + LDS_AI_ENABLED=auto + LDS_AI_PROVIDER=ollama + LDS_AI_URL=http://llm-sm:11434 + LDS_AI_MODEL=qwen2.5:3b + +The explicit LocalDevStack model default avoids ambiguity when several models are +installed in the persistent Ollama store. + +Runtime Modes +------------- + +Select the provider runtime explicitly:: + + lds llm runtime cpu + lds llm runtime nvidia + lds llm runtime amd + +CPU and NVIDIA use the standard infocyph/llm-sm:latest image. AMD/ROCm uses +infocyph/llm-sm:amd-latest. + +LocalDevStack does not attempt unreliable automatic GPU detection. + +Access +------ + +Container-to-container provider endpoint:: + + http://llm-sm:11434 + +User-facing HTTPS endpoint:: + + https://llm.localhost + +Nginx owns the host-facing route and preserves streaming behavior for Ollama/OpenAI-style +API calls. + +Direct host Ollama access is disabled by default. To opt in:: + + lds llm host-port on + +The direct binding is loopback-only:: + + 127.0.0.1:11434 + +Disable it again with:: + + lds llm host-port off + +AI vs LLM Commands +------------------ + +lds ai is for Tools-owned operational intelligence:: + + lds ai status + lds ai ask "explain this error" + lds ai explain ... + lds ai troubleshoot ... + lds ai review ... + lds ai repo-review ... + lds ai graphify ... + +lds llm is for provider/model runtime management:: + + lds llm models + lds llm ps + lds llm show + lds llm pull + lds llm rm + lds llm run + lds llm chat ... + lds llm version + +This separation keeps application/operational AI in Tools and model lifecycle inside +the provider image. + +Model Persistence +----------------- + +Ollama state persists in the LLMModels named volume mounted at /root/.ollama. +Removing/recreating the container does not remove installed models unless the volume is +explicitly deleted. + +Do not use destructive docker compose down -v during normal LocalDevStack upgrades. + +Selecting a Different Model +--------------------------- + +Install the model through the provider:: + + lds llm pull + +Then set LDS_AI_MODEL in docker/.env to make Tools use that model by default. +Shell environment overrides still have the highest precedence. + +Privacy and Trust Boundaries +---------------------------- + +By default, llm-sm receives: + +- no Docker socket; +- no project/repository bind mount; +- no host port; +- only its model volume and the LocalDevStack networks needed for provider access. + +Tools may send bounded/sanitized context to the local provider when the user invokes an +AI feature. Deterministic monitoring and system checks remain the source of truth. + +LocalDevStack does not: + +- fall back silently to a cloud AI provider; +- automatically execute model-generated shell commands; +- automatically execute model-generated SQL; +- automatically execute generated code. + +Graphify +-------- + +lds ai graphify delegates Graphify-assisted analysis to the Tools AI layer. The +LocalDevStack AI provider remains only the model runtime; Graphify integration does not +grant llm-sm direct repository access. + +Diagnostics +----------- + +Show provider state through Tools:: + + lds ai status + +Check the LocalDevStack stack non-destructively:: + + lds doctor + +List convenience endpoints:: + + lds urls From 804cda7cb5fb0ca8b1d018591fab1c4fca2e818c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:10:23 +0600 Subject: [PATCH 154/509] test: add documentation contract --- tests/docs-contract.sh | 76 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 76 insertions(+) create mode 100644 tests/docs-contract.sh diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh new file mode 100644 index 00000000..b65c7b03 --- /dev/null +++ b/tests/docs-contract.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +# shellcheck source=tests/lib/assertions.sh +source "$ROOT/tests/lib/assertions.sh" + +index="$ROOT/docs/index.rst" +readme="$ROOT/README.md" +quick="$ROOT/docs/quickstart.rst" +arch="$ROOT/docs/concepts/architecture.rst" +profiles="$ROOT/docs/concepts/profiles-and-env.rst" +storage="$ROOT/docs/concepts/storage-layout.rst" +domain="$ROOT/docs/guides/domain-setup.rst" +tls="$ROOT/docs/guides/tls-and-certificates.rst" +ai="$ROOT/docs/guides/local-ai.rst" + +for file in "$index" "$readme" "$quick" "$arch" "$profiles" "$storage" "$domain" "$tls" "$ai"; do + assert_file "$file" +done + +assert_file_contains "$index" 'guides/local-ai' +assert_file_contains "$readme" 'Dynamic Docker networking through service DNS' +assert_file_contains "$arch" 'Docker assigns their address ranges dynamically' +assert_file_contains "$domain" 'generated runtime Compose fragments are written under configuration/compose/' +pass "docs describe dynamic DNS/runtime generation" + +assert_file_contains "$profiles" 'docker/release.env' +assert_file_contains "$profiles" 'docker/.env' +assert_file_contains "$profiles" 'SCRIPTOMATIC_REF=main' +assert_file_contains "$profiles" 'localdevstack-php:' +assert_file_contains "$profiles" 'localdevstack-node:' +pass "docs preserve env precedence and runtime version selection" + +assert_file_contains "$storage" 'configuration/ssl/rootCA.pem' +assert_file_contains "$tls" 'configuration/ssl/rootCA.pem' +assert_file_contains "$tls" 'configuration/rootCA/rootCA.pem' +assert_file_contains "$storage" 'SSLKeys / SSLRootCA' +pass "docs distinguish runtime TLS state from public host exports" + +assert_file_contains "$ai" 'http://llm-sm:11434' +assert_file_contains "$ai" 'https://llm.localhost' +assert_file_contains "$ai" '127.0.0.1:11434' +assert_file_contains "$ai" 'no Docker socket' +assert_file_contains "$ai" 'no project/repository bind mount' +assert_file_contains "$ai" 'automatically execute model-generated shell commands' +pass "local AI trust boundary and access paths are documented" + +for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-sm:0.03'; do + if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then + fail "user-facing docs contain stale compatibility reference: $stale" + fi +done +pass "user-facing docs contain no superseded pinned infrastructure defaults" + +for subnet in 172.28.0.0 172.29.0.0 172.30.0.0; do + if grep -RqsF "$subnet" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then + fail "user-facing docs contain legacy fixed subnet: $subnet" + fi +done +pass "user-facing docs contain no legacy fixed subnets" + +while IFS= read -r target; do + [[ -n "$target" ]] || continue + [[ -f "$ROOT/docs/$target.rst" ]] || + fail "docs/index.rst references missing page: $target.rst" +done < <( + awk ' + /^[[:space:]]{3}[A-Za-z0-9_./-]+$/ { + line=$0 + sub(/^[[:space:]]+/, "", line) + if (line !~ /^:/) print line + } + ' "$index" +) +pass "documentation toctree targets exist" From c08c0bd11e56453a082b84d5a70a8dc9090a1686 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:10:29 +0600 Subject: [PATCH 155/509] ci: enforce documentation contract --- .github/workflows/check.yml | 3 +++ 1 file changed, 3 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 38096a65..a23536f7 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -63,6 +63,9 @@ jobs: - name: Permission contract run: bash tests/permissions-contract.sh + - name: Documentation contract + run: bash tests/docs-contract.sh + compose: name: Compose contract runs-on: ubuntu-latest From 4a854accceb65357b04d91d92af4053a832f7e04 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:10:32 +0600 Subject: [PATCH 156/509] test: add docs to release gate --- tests/release-gate.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/release-gate.sh b/tests/release-gate.sh index e1e8ce2a..f339012a 100755 --- a/tests/release-gate.sh +++ b/tests/release-gate.sh @@ -13,6 +13,7 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" bash "$ROOT/tests/service-hardening-contract.sh" bash "$ROOT/tests/qol-contract.sh" bash "$ROOT/tests/permissions-contract.sh" +bash "$ROOT/tests/docs-contract.sh" "$ROOT/tests/runtime-php-contract.sh" "$ROOT/tests/runtime-node-contract.sh" "$ROOT/tests/ai-contract.sh" From ca8391d11cedbaa884acc429c26fd556b4e114d2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:12:10 +0600 Subject: [PATCH 157/509] ci: build Sphinx docs with warnings as errors --- .github/workflows/check.yml | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index a23536f7..0a2075c2 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -115,3 +115,25 @@ jobs: - name: Validate lds.bat contract shell: pwsh run: tests/windows-bridge.ps1 + + + docs: + name: Documentation build + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Set up Python + uses: actions/setup-python@v6 + with: + python-version: "3.13" + cache: pip + cache-dependency-path: docs/requirements.txt + + - name: Install documentation dependencies + run: python -m pip install -r docs/requirements.txt + + - name: Build documentation + run: sphinx-build -W --keep-going -b html docs docs/_build/html From 610a994b7bef95bf71eab57012bc117263dd1c86 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:13:53 +0600 Subject: [PATCH 158/509] qol: let doctor own Docker availability diagnostics --- lds | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lds b/lds index 0ca22af2..c4c52071 100755 --- a/lds +++ b/lds @@ -876,7 +876,7 @@ main() { # Offline-safe read-only commands do not require a running Docker daemon. case "$cmd" in - images | urls) + images | urls | doctor) ;; config) [[ "${1:-}" == "env-used" ]] || need docker From 62c6b5fc2c14e57f3b3b7a0135bade4f657667cb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:13:56 +0600 Subject: [PATCH 159/509] refactor: remove shadowed legacy config command --- lib/services.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/services.sh b/lib/services.sh index 4a8b81af..aaeed5f0 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -722,7 +722,6 @@ cmd_rebuild() { cmd_reboot } -cmd_config() { docker_compose config; } docker_shell() { local c="${1:-}" From 71b9e943b1f4527597b870fdf2646cf2337b2620 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:14:20 +0600 Subject: [PATCH 160/509] test: keep doctor outside the global Docker preflight --- tests/qol-contract.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 90a50af1..14313425 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -76,3 +76,7 @@ assert_contains "$help" "doctor" assert_contains "$help" "images" assert_contains "$help" "urls" pass "QoL commands are discoverable" + +assert_file_contains "$ROOT/lds" 'images | urls | doctor)' +assert_file_contains "$ROOT/lib/diagnostics.sh" 'Docker daemon is unavailable.' +pass "doctor owns Docker availability diagnostics" From 1a7d553634c7e1ea20e7daf30eb43140eced0091 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:14:48 +0600 Subject: [PATCH 161/509] plan: mark LocalDevStack integration batches complete --- .../07-localdevstack-integration-plan.md | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 50aa949c..22ff4aaf 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1956,3 +1956,100 @@ LocalDevStack must preserve the published Tools runtime-selection contract: - explicit user-selected versions always win. Batch 6 must not collapse these selectors into a single global PHP or Node version. + + +--- + +# Implementation completion record — 2026-09-18 + +All planned LocalDevStack integration batches are implemented on branch `plan/docker-ecosystem-bottom-up`. + +## Completed batches + +1. **Batch 1 — CI + characterization** + - permanent CI foundation; + - CLI/env/runtime/network characterization; + - Windows bridge coverage; + - published-image checks. + +2. **Batch 2 — image/default policy** + - tracked `docker/release.env`; + - deterministic fallback/release/user/shell precedence; + - user-directed Alpine-first moving image policy; + - explicit Elastic aligned-version exception. + +3. **Batch 3 — networking/DNS migration** + - static `172.28/29/30` addresses removed; + - Docker DNS/service-name routing; + - deterministic legacy-network migration; + - `vpn-fix` deprecated. + +4. **Batch 4 — optional AI integration** + - optional `ai` profile; + - persistent `LLMModels`; + - CPU/NVIDIA/AMD modes; + - `https://llm.localhost`; + - loopback-only optional direct host port; + - Tools consumer wiring; + - `lds ai` / `lds llm` separation; + - fake-provider integration test. + +5. **Batch 5 — profiles/catalog/runtime defaults** + - canonical host service catalog; + - profile setup driven from catalog metadata; + - AI setup fields; + - published Tools profile-drift checks; + - interactive runtime version selection preserved. + +6. **Batch 6 — PHP/Node runtime modernization** + - Scriptomatic `main` / full-SHA contract; + - bounded download + non-empty/syntax validation; + - selected `PHP_VERSION` / `NODE_VERSION` retained inside build stage; + - Alpine runtime bases retained; + - rebuild cache preserved with `--pull`. + +7. **Batch 7 — CLI modularization + wrapper cleanup** + - `lds` split into focused `lib/*.sh` modules; + - Windows wrapper Docker preflight boundary corrected; + - wrapper/runtime/database contracts added; + - fixed-IP assumptions removed from wrappers. + +8. **Batch 8 — Compose/service hardening** + - health-gated service dependencies; + - credential-free database readiness checks; + - Docker socket limited to Tools/Runner; + - inactive PostgreSQL tuning config made explicit; + - unused legacy FPM config removed; + - Runner health/logrotate contract validated; + - public TLS export bridge corrected to `configuration/ssl/rootCA.pem` with legacy fallback. + +9. **Batch 9 — permissions/QoL/docs/release gate** + - broad `chmod -R 777` removed; + - SSH/SOPS key directories hardened; + - `lds urls`, `lds images`, redacted grouped `lds config`, and diagnostic-only `lds doctor`; + - Sphinx user documentation rewritten for the implemented architecture; + - dedicated local-AI guide; + - docs contract + warning-as-error Sphinx CI; + - release-gate script contains all Linux/runtime/AI/published-image contracts. + +## Final policy clarifications + +- Prefer a moving Alpine variant when an image family provides a suitable one; otherwise use its normal moving latest alias. +- PostgreSQL defaults to `postgres:alpine`. +- Tools, Runner, Nginx and Apache consume their published `:latest` aliases. +- Standard LLM uses `infocyph/llm-sm:latest`; AMD/ROCm uses `infocyph/llm-sm:amd-latest`. +- Elasticsearch, Kibana and Filebeat remain version-aligned on the tested stable version because their required image contract does not provide a suitable moving `latest` alias. +- PHP/Node runtime selection remains user-driven and version-specific. +- Existing named volumes and container names remain intentionally stable for this release. + +## Non-blocking follow-ups retained from the plan + +These remain future work rather than release blockers: + +- multi-instance container/volume namespacing; +- automatic dependency-update PRs; +- Docker socket proxy if it can reduce privilege without breaking supported workflows; +- automatic GPU detection; +- Graphify installation/management; +- additional browser AI UI; +- automatic model downloads beyond the provider defaults. From e538b24864c64a9a59b7a4836bc3f04bf076b573 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:16:25 +0600 Subject: [PATCH 162/509] docs: fix SOPS storage definition formatting --- docs/concepts/storage-layout.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 3703720a..43b5c4df 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -50,9 +50,7 @@ configuration/scheduler/cron-jobs/ configuration/scheduler/supervisor/ Runner Supervisor definitions. -configuration/sops/config/ -configuration/sops/global/ -configuration/sops/keys/ +configuration/sops/config/, configuration/sops/global/, configuration/sops/keys/ SOPS/Age configuration and sensitive key material. configuration/ssh/ From c63e765bd7ec74789a2737de0badaf5eb3a5d2b2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:16:28 +0600 Subject: [PATCH 163/509] docs: escape wildcard localhost in domain guide --- docs/guides/domain-setup.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/domain-setup.rst b/docs/guides/domain-setup.rst index 46a7ac95..4eac5ab7 100644 --- a/docs/guides/domain-setup.rst +++ b/docs/guides/domain-setup.rst @@ -69,7 +69,7 @@ TLS --- When HTTPS is selected, Tools refreshes the shared LocalDevStack certificate set. The -certificate SAN set always includes localhost, *.localhost, 127.0.0.1, and ::1 in +certificate SAN set always includes localhost, \*.localhost, 127.0.0.1, and ::1 in addition to generated domains/service-derived hosts. This means convenience hosts such as admin.localhost, webmail.localhost, and From 4a03c1ef0d987d8f73fbce32ee440b6a347588a0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:16:32 +0600 Subject: [PATCH 164/509] docs: escape wildcard localhost in TLS guide --- docs/guides/tls-and-certificates.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/tls-and-certificates.rst b/docs/guides/tls-and-certificates.rst index a980acc0..e1d3e4e2 100644 --- a/docs/guides/tls-and-certificates.rst +++ b/docs/guides/tls-and-certificates.rst @@ -48,7 +48,7 @@ The generated certificate SAN set includes at least:: 127.0.0.1 ::1 -Tools also discovers generated vhost/service domains. The *.localhost entry covers +Tools also discovers generated vhost/service domains. The \*.localhost entry covers built-in convenience endpoints such as: - admin.localhost; From 472e99677eb5ef64e05ca7d647b0ecb769f4311e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:16:35 +0600 Subject: [PATCH 165/509] docs: fix index title underline --- docs/index.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/index.rst b/docs/index.rst index dd8624bd..3b188267 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,5 +1,5 @@ LocalDevStack Documentation -========================== +=========================== LocalDevStack is a Docker-based XAMPP alternative for PHP and Node.js local development. It provides profile-driven services, local domains and TLS, selectable runtimes, From 8664654d938b7350bf444312fe2b94a0e5831135 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 13:16:39 +0600 Subject: [PATCH 166/509] docs: exclude internal plans and broken PHP inventory --- docs/conf.py | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-) diff --git a/docs/conf.py b/docs/conf.py index e46463dc..ea884f54 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -31,6 +31,9 @@ def get_version() -> str: # Sphinx 8 root document root_doc = "index" +# Internal implementation plans are repository artifacts, not public Sphinx pages. +exclude_patterns = ["_build", "plans/**"] + # --- Syntax highlighting (PHP) ---------------------------------------------- from pygments.lexers.web import PhpLexer from sphinx.highlighting import lexers @@ -43,7 +46,6 @@ def get_version() -> str: "myst_parser", "sphinx.ext.todo", "sphinx.ext.autosectionlabel", - "sphinx.ext.intersphinx", "sphinx_copybutton", "sphinx_design", "sphinxcontrib.phpdomain", # Essential for PHP projects @@ -65,11 +67,6 @@ def get_version() -> str: # Autodoc/Napoleon are for Python, so they are omitted. autosectionlabel_prefix_document = True -# Intersphinx: Link to PHP manual -intersphinx_mapping = { - "php": ("https://www.php.net/manual/en/", None), -} - # extlinks shortcut for PHP manual extlinks = { "php": ("https://www.php.net/manual/en/%s.php", "%s"), From 3eecb4e5b2e1a7aa857a9977de6767e8468bbb9d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:08:09 +0600 Subject: [PATCH 167/509] compat: persist Tools durable state --- docker/compose/main.yaml | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docker/compose/main.yaml b/docker/compose/main.yaml index 83ec4765..3ed4cbc2 100644 --- a/docker/compose/main.yaml +++ b/docker/compose/main.yaml @@ -145,6 +145,12 @@ volumes: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" com.infocyph.purpose: "LLM Model Store" + lds_tools_state: + name: ToolsState + labels: + com.infocyph.lds: "1" + com.infocyph.stack: "LocalDevStack" + com.infocyph.purpose: "Tools Durable State" include: - docker/compose/companion.yaml - docker/compose/db.yaml From f58da9cc0e30a1eedbabc3efcc7402687d78c99a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:08:12 +0600 Subject: [PATCH 168/509] compat: mount persistent Tools state --- docker/compose/companion.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index dfbdcd51..52996120 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -36,6 +36,7 @@ services: - lds_fpm_pools:/etc/share/vhosts/fpm - lds_composer_global:/etc/share/vhosts/composer - lds_git_config:/git-config + - lds_tools_state:/etc/share/state - ../../configuration/compose:/etc/share/vhosts/docker-compose - ../../configuration/scheduler/cron-jobs:/etc/share/scheduler/cron-jobs - ../../configuration/scheduler/supervisor:/etc/share/scheduler/supervisor From 4d8d755f0c39608690b7d05efef021f5baaf368b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:08:15 +0600 Subject: [PATCH 169/509] compat: make Apache backend profile-driven --- docker/compose/http.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 0beb9898..3f614757 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -27,6 +27,7 @@ services: hostname: apache image: ${LDS_APACHE_IMAGE:-infocyph/apache:latest} restart: unless-stopped + profiles: [apache] environment: - TZ=${TZ:-} volumes: From 2741cdd1ef99559df9aed5b08056dacd43b7655b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:08:19 +0600 Subject: [PATCH 170/509] tls: use exported root CA on Unix installs --- lib/certificates.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/certificates.sh b/lib/certificates.sh index a2075ee1..41d580ef 100644 --- a/lib/certificates.sh +++ b/lib/certificates.sh @@ -193,9 +193,10 @@ install_ca() { return 0 fi - local src_ca="$DIR/configuration/rootCA/rootCA.pem" + local src_ca + src_ca="$(host_root_ca_path || true)" [[ ${EUID:-$(id -u)} -eq 0 ]] || die "certificate install requires sudo" - [[ -r "$src_ca" ]] || die "certificate not found: $src_ca" + [[ -n "$src_ca" ]] || die "certificate not found: expected configuration/ssl/rootCA.pem (legacy: configuration/rootCA/rootCA.pem)" local family dest updater os_id os_like IFS='|' read -r os_id os_like < <(detect_os_family) From 4ed6a73f769412f332758bddf3ca191be3a1307c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:08:55 +0600 Subject: [PATCH 171/509] test: enforce Apache profile and Tools state persistence --- tests/compose-contract.sh | 32 +++++++++++++++++++++++--------- 1 file changed, 23 insertions(+), 9 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 8527bac7..ae3a06ed 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -59,6 +59,7 @@ render mongodb --profile mongodb render redis --profile redis render elasticsearch --profile elasticsearch render elasticsearch-filebeat --profile elasticsearch --profile filebeat +render apache --profile apache render ai --profile ai docker compose --project-directory "$ROOT" \ -f "$ROOT/docker/compose/main.yaml" \ @@ -81,7 +82,6 @@ assert_contains "$resolved" "cloudbeaver:" assert_contains "$resolved" "image: infocyph/tools:latest" assert_contains "$resolved" "image: infocyph/runner:latest" assert_contains "$resolved" "image: infocyph/nginx:latest" -assert_contains "$resolved" "image: infocyph/apache:latest" if grep -Eq 'ipv4_address:|172\\.28\\.0\\.|172\\.29\\.0\\.|172\\.30\\.0\\.' <<<"$resolved"; then fail "resolved Compose config still contains fixed LocalDevStack addresses" fi @@ -119,15 +119,29 @@ shell_override="$( assert_contains "$shell_override" "image: example.invalid/tools:shell-override" pass "shell override wins over user and release env files" -# Apache is currently an unconditional service in the baseline. This is -# characterized here; Batch 5 will make the product's optional-HTTP contract -# explicit rather than silently changing it in the compatibility batch. -assert_contains "$resolved" "apache:" -pass "current Apache compose presence characterized" - core_json="$("${compose[@]}" config --format json)" -python3 -c 'import json,sys; d=json.load(sys.stdin); assert "llm-sm" not in d.get("services", {})' <<<"$core_json" -pass "AI provider remains outside the default profile" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +assert "apache" not in d.get("services", {}) +assert "llm-sm" not in d.get("services", {}) +assert d["volumes"]["lds_tools_state"]["name"] == "ToolsState" +tools=d["services"]["server-tools"] +targets={v["target"] for v in tools["volumes"]} +assert "/etc/share/state" in targets +' <<<"$core_json" +pass "optional Apache/AI stay outside default stack and Tools state is persistent" + +apache_json="$("${compose[@]}" --profile apache config --format json)" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +s=d["services"]["apache"] +assert s["image"] == "infocyph/apache:latest" +assert s["profiles"] == ["apache"] +assert "nginx" in s["depends_on"] +' <<<"$apache_json" +pass "Apache backend is activated only through its profile" ai_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' From b24450924b76bc8648a5d1195ff46aa4f2c45e2a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:08:58 +0600 Subject: [PATCH 172/509] test: enforce cross-image compatibility fixes --- tests/service-hardening-contract.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index 0aeb9d4f..0d42ee4f 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -51,3 +51,21 @@ assert_file_contains "$certs" 'local current="$DIR/configuration/ssl/rootCA.pem" assert_file_contains "$certs" 'local legacy="$DIR/configuration/rootCA/rootCA.pem"' assert_file_contains "$certs" 'src_ca="$(host_root_ca_path || true)"' pass "certificate export bridge uses the current public host path with legacy fallback" + +main="$ROOT/docker/compose/main.yaml" +assert_file_contains "$main" 'name: ToolsState' +assert_file_contains "$companion" 'lds_tools_state:/etc/share/state' +pass "Tools durable state persistence" + +assert_file_contains "$http" 'profiles: [apache]' +assert_file_contains "$ROOT/lib/hosts.sh" 'modify_profiles add "$svr_prof"' +assert_file_contains "$ROOT/lib/hosts.sh" 'modify_profiles remove "$apache_cont"' +pass "Apache domain profile lifecycle is wired end to end" + +cert_helper_uses="$(grep -c 'src_ca="$(host_root_ca_path || true)"' "$certs")" +[[ "$cert_helper_uses" -ge 3 ]] || + fail "Windows install/uninstall and Unix install must all use host_root_ca_path" +if grep -Fq 'local src_ca="$DIR/configuration/rootCA/rootCA.pem"' "$certs"; then + fail "Unix CA install regressed to the legacy-only path" +fi +pass "all CA install paths use current export with legacy fallback" From f01a44fa7d1d6084f21596c55ad3660168667f25 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:09:02 +0600 Subject: [PATCH 173/509] test: enforce Tools durable state ABI --- tests/published-images.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/published-images.sh b/tests/published-images.sh index 22af80bb..e4514a55 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -115,3 +115,12 @@ assert_contains "$tools_certify" 'EXPORT_DIR="${EXPORT_DIR:-/etc/share/certs}"' assert_contains "$tools_certify" 'EXPORT_ROOTCA_NAME="${EXPORT_ROOTCA_NAME:-rootCA.pem}"' assert_contains "$tools_certify" 'atomic_install 0644 "$root_ca" "$EXPORT_DIR/$EXPORT_ROOTCA_NAME"' pass "latest Tools public TLS export contract" + +docker run --rm --entrypoint sh "${release[LDS_TOOLS_IMAGE]}" -ec ' + test -d /etc/share/state + test -x /usr/local/bin/env-store + grep -Fq "/etc/share/state/env-store.json" /usr/local/bin/env-store + grep -Fq "/etc/share/state" /usr/local/bin/monitor-alerts + grep -Fq "/etc/share/state" /usr/local/bin/monitor-slo +' +pass "latest Tools durable state ABI" From 778e48d13226b993bfd4278206b37c3a748292d9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:11:22 +0600 Subject: [PATCH 174/509] compat: preserve Admin Panel Apache host behavior --- docker/compose/http.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 3f614757..0beb9898 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -27,7 +27,6 @@ services: hostname: apache image: ${LDS_APACHE_IMAGE:-infocyph/apache:latest} restart: unless-stopped - profiles: [apache] environment: - TZ=${TZ:-} volumes: From b07e99bdc9ec4e210433fc1890ddc950ed5b54d1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:11:25 +0600 Subject: [PATCH 175/509] test: preserve Apache Admin Panel compatibility --- tests/compose-contract.sh | 16 ++-------------- 1 file changed, 2 insertions(+), 14 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index ae3a06ed..f125a382 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -59,7 +59,6 @@ render mongodb --profile mongodb render redis --profile redis render elasticsearch --profile elasticsearch render elasticsearch-filebeat --profile elasticsearch --profile filebeat -render apache --profile apache render ai --profile ai docker compose --project-directory "$ROOT" \ -f "$ROOT/docker/compose/main.yaml" \ @@ -123,25 +122,14 @@ core_json="$("${compose[@]}" config --format json)" python3 -c ' import json,sys d=json.load(sys.stdin) -assert "apache" not in d.get("services", {}) +assert "apache" in d.get("services", {}) assert "llm-sm" not in d.get("services", {}) assert d["volumes"]["lds_tools_state"]["name"] == "ToolsState" tools=d["services"]["server-tools"] targets={v["target"] for v in tools["volumes"]} assert "/etc/share/state" in targets ' <<<"$core_json" -pass "optional Apache/AI stay outside default stack and Tools state is persistent" - -apache_json="$("${compose[@]}" --profile apache config --format json)" -python3 -c ' -import json,sys -d=json.load(sys.stdin) -s=d["services"]["apache"] -assert s["image"] == "infocyph/apache:latest" -assert s["profiles"] == ["apache"] -assert "nginx" in s["depends_on"] -' <<<"$apache_json" -pass "Apache backend is activated only through its profile" +pass "Apache remains compatibility-safe for CLI/Admin hosts, AI stays optional, and Tools state is persistent" ai_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' From d25376599b3a7789c51e7dac45dbeb85c042e3f8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:11:28 +0600 Subject: [PATCH 176/509] test: guard Apache host lifecycle compatibility --- tests/service-hardening-contract.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index 0d42ee4f..63451a70 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -57,10 +57,12 @@ assert_file_contains "$main" 'name: ToolsState' assert_file_contains "$companion" 'lds_tools_state:/etc/share/state' pass "Tools durable state persistence" -assert_file_contains "$http" 'profiles: [apache]' assert_file_contains "$ROOT/lib/hosts.sh" 'modify_profiles add "$svr_prof"' assert_file_contains "$ROOT/lib/hosts.sh" 'modify_profiles remove "$apache_cont"' -pass "Apache domain profile lifecycle is wired end to end" +if grep -Fq 'profiles: [apache]' "$http"; then + fail "Apache cannot become profile-only until Admin Panel host lifecycle can manage the profile" +fi +pass "Apache remains always available so CLI and Admin Panel host creation retain parity" cert_helper_uses="$(grep -c 'src_ca="$(host_root_ca_path || true)"' "$certs")" [[ "$cert_helper_uses" -ge 3 ]] || From 1e152f8412b0a379ccfeb4768c7efe86ea245aeb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:01 +0600 Subject: [PATCH 177/509] compat: expose active profiles to Tools status --- docker/compose/companion.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 52996120..ad31dc14 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -16,6 +16,7 @@ services: - SOPS_CFG_DIR=/etc/share/sops/config - SOPS_GLOBAL_DIR=/etc/share/sops/global - SOPS_REPO_DIR=/etc/share/vhosts/sops + - COMPOSE_PROFILES=${COMPOSE_PROFILES:-} - LDS_AI_ENABLED=${LDS_AI_ENABLED:-auto} - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-ollama} - LDS_AI_URL=${LDS_AI_URL:-http://llm-sm:11434} From b33ed9aba85a8d0551a7e7a24d5f0434c6ff161c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:05 +0600 Subject: [PATCH 178/509] ci: recheck moving latest images weekly --- .github/workflows/check.yml | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 0a2075c2..30afb658 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -7,6 +7,10 @@ on: - "plan/**" - "feature/**" pull_request: + schedule: + # Related moving latest images refresh weekly; re-run the full compatibility + # suite after that window even when LocalDevStack source has not changed. + - cron: "30 2 * * 1" workflow_dispatch: permissions: From e1e6fb69d8f6ea89337569f198a37d366b3105d8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:08 +0600 Subject: [PATCH 179/509] test: preserve Tools profile visibility --- tests/service-hardening-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index 63451a70..bc36b893 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -71,3 +71,6 @@ if grep -Fq 'local src_ca="$DIR/configuration/rootCA/rootCA.pem"' "$certs"; then fail "Unix CA install regressed to the legacy-only path" fi pass "all CA install paths use current export with legacy fallback" + +assert_file_contains "$companion" 'COMPOSE_PROFILES=${COMPOSE_PROFILES:-}' +pass "Tools profile visibility follows LocalDevStack profile selection" From 8cab2d42118481ee205c4cc77833d0be2f705739 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:32 +0600 Subject: [PATCH 180/509] docs: document persistent Tools control-plane state --- docs/concepts/storage-layout.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 43b5c4df..38207036 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -21,6 +21,11 @@ FPMPools / FPMSocks ComposerGlobal / GitConfig Shared runtime state for Composer and Git integration. +ToolsState + Durable Tools control-plane state under /etc/share/state, including host-manager + state, profile/runtime metadata, monitor history, alert acknowledgements, and + other state owned by the Tools image. + Database/admin volumes Persistent PostgreSQL, MySQL, MariaDB, MongoDB, Redis, Elasticsearch, Kibana, RedisInsight, CloudBeaver, and Filebeat state. From a9f7a00b1eb5a020e6e2e0ed886eb9e7842aa932 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:39 +0600 Subject: [PATCH 181/509] docs: document Tools state ownership --- docs/concepts/architecture.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index af2b42e0..5957a9a3 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -23,7 +23,8 @@ PHP / Node runtimes Tools Trusted control plane for vhost generation, certificates, admin UI, secrets, - Git helpers, monitoring, and AI-consumer commands. + Git helpers, monitoring, and AI-consumer commands. Tools control-plane state is + persisted separately in the ToolsState named volume. Runner Background execution layer for Supervisor, cron, and log rotation. From 24406a3a90b8261b6c2362414df9c561069a8edf Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:45 +0600 Subject: [PATCH 182/509] docs: clarify local AI platform and workspace boundaries --- docs/guides/local-ai.rst | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 1427ddef..baede4fc 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -151,3 +151,31 @@ Check the LocalDevStack stack non-destructively:: List convenience endpoints:: lds urls + + +Platform Availability +--------------------- + +The current llm-sm 0.03 publication is linux/amd64 only. LocalDevStack itself remains +usable on arm64 with the ai profile disabled. Native arm64 local-AI support should only +be advertised after the llm-sm image publishes and validates a native arm64 runtime. + +Direct File and Repository Context +---------------------------------- + +LocalDevStack intentionally does not mount the project/repository into llm-sm by default. + +This means normal provider/model commands work directly, including ask, chat, model +management, API access, and stdin-based prompts. Repository-aware analysis should use +the Tools consumer layer by default:: + + lds ai review ... + lds ai repo-review ... + +For direct llm-sm commands, file/PDF/image paths must exist inside the provider +container. Git-diff based ai-commit can be used without a repository mount by piping +the diff through stdin to the provider CLI. + +The upstream llm-sm image provides an optional workspace override for explicit +repository mounts, but LocalDevStack does not enable it automatically. That preserves +the default no-repository-ingestion trust boundary. From 3bf8a004bcf759a4522788a579f1cb4b01617087 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:14:57 +0600 Subject: [PATCH 183/509] test: lock current AI and Tools state documentation --- tests/docs-contract.sh | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index b65c7b03..6dfa3634 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -36,7 +36,8 @@ assert_file_contains "$storage" 'configuration/ssl/rootCA.pem' assert_file_contains "$tls" 'configuration/ssl/rootCA.pem' assert_file_contains "$tls" 'configuration/rootCA/rootCA.pem' assert_file_contains "$storage" 'SSLKeys / SSLRootCA' -pass "docs distinguish runtime TLS state from public host exports" +assert_file_contains "$storage" 'ToolsState' +pass "docs distinguish persistent runtime/control state and public TLS exports" assert_file_contains "$ai" 'http://llm-sm:11434' assert_file_contains "$ai" 'https://llm.localhost' @@ -44,7 +45,9 @@ assert_file_contains "$ai" '127.0.0.1:11434' assert_file_contains "$ai" 'no Docker socket' assert_file_contains "$ai" 'no project/repository bind mount' assert_file_contains "$ai" 'automatically execute model-generated shell commands' -pass "local AI trust boundary and access paths are documented" +assert_file_contains "$ai" 'linux/amd64 only' +assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by default' +pass "local AI trust boundary, platform and workspace limits are documented" for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-sm:0.03'; do if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then From 5dd940283b76ce52cd74cdde39cc7701b35589ad Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:18:18 +0600 Subject: [PATCH 184/509] permissions: preserve private TLS export modes --- lib/platform.sh | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/lib/platform.sh b/lib/platform.sh index 67eb9280..8826d3b3 100644 --- a/lib/platform.sh +++ b/lib/platform.sh @@ -65,6 +65,14 @@ fix_perms() { find "$private_dir" -type f -exec chmod 0600 {} + done + # Public CA exports may stay readable, but password-protected/user key + # artifacts must retain the restrictive mode Tools assigns to them. + if [[ -d "$DIR/configuration/ssl" ]]; then + find "$DIR/configuration/ssl" -type d -exec chmod 0755 {} + + find "$DIR/configuration/ssl" -type f -exec chmod 0644 {} + + find "$DIR/configuration/ssl" -type f \( -name '*.p12' -o -name '*.pfx' -o -name '*.key' -o -name '*-key.pem' \) -exec chmod 0600 {} + + fi + find "$DIR/docker" -type d -exec chmod 0755 {} + find "$DIR/docker" -type f -exec chmod 0644 {} + From 8fdf01cc68d4660b2abdb95ac4b8584f5eb4768d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:18:22 +0600 Subject: [PATCH 185/509] test: enforce private TLS export permissions --- tests/permissions-contract.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/permissions-contract.sh b/tests/permissions-contract.sh index a097f9f1..b0e09419 100644 --- a/tests/permissions-contract.sh +++ b/tests/permissions-contract.sh @@ -17,3 +17,9 @@ assert_file_contains "$file" 'find "$private_dir" -type d -exec chmod 0700 {} +' assert_file_contains "$file" 'find "$private_dir" -type f -exec chmod 0600 {} +' assert_file_contains "$file" '"$DIR/configuration/ssh" "$DIR/configuration/sops/keys"' pass "host permissions are scoped and secret directories remain private" + +assert_file_contains "$file" 'find "$DIR/configuration/ssl" -type f -exec chmod 0644 {} +' +assert_file_contains "$file" "-name '*.p12'" +assert_file_contains "$file" "-name '*-key.pem'" +assert_file_contains "$file" ') -exec chmod 0600 {} +' +pass "TLS public exports remain readable while private/P12 artifacts stay 0600" From 6c293262796b9b49772650791f738d8e3e7a70f1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:20:34 +0600 Subject: [PATCH 186/509] test: add scheduled real runtime build smoke --- tests/runtime-build-smoke.sh | 49 ++++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+) create mode 100644 tests/runtime-build-smoke.sh diff --git a/tests/runtime-build-smoke.sh b/tests/runtime-build-smoke.sh new file mode 100644 index 00000000..096cb7b0 --- /dev/null +++ b/tests/runtime-build-smoke.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +set -euo pipefail + +ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" +tools_image="infocyph/tools:latest" + +docker pull "$tools_image" >/dev/null +versions="$(docker run --rm --entrypoint cat "$tools_image" /etc/share/runtime-versions.json)" + +php_version="$(jq -r '.php.active[0] // empty' <<<"$versions")" +node_version="$(jq -r '.node.active[0] // empty' <<<"$versions")" +[[ "$php_version" =~ ^[0-9]+\.[0-9]+$ ]] || { + printf 'Invalid active PHP version from Tools: %s\n' "$php_version" >&2 + exit 1 +} +[[ "$node_version" =~ ^[0-9]+([.][0-9]+([.][0-9]+)?)?$ ]] || { + printf 'Invalid active Node version from Tools: %s\n' "$node_version" >&2 + exit 1 +} + +php_key="${php_version/./}" +php_image="localdevstack-php:compat-smoke" +node_image="localdevstack-node:compat-smoke" + +printf 'Building selected PHP runtime: %s\n' "$php_version" +docker build --pull -f "$ROOT/docker/dockerfiles/php.Dockerfile" --build-arg "PHP_VERSION=$php_version" --build-arg "PHP_PROFILE_KEY=$php_key" --build-arg "UID=1000" --build-arg "GID=1000" --build-arg "USERNAME=dockery" --build-arg "SCRIPTOMATIC_REF=main" -t "$php_image" "$ROOT/docker/dockerfiles" + +docker run --rm --entrypoint sh "$php_image" -ec ' + php -v + php-fpm -t + command -v gitx >/dev/null + command -v chromacat >/dev/null + gitx --version + chromacat --version +' + +printf 'Building selected Node runtime: %s\n' "$node_version" +docker build --pull -f "$ROOT/docker/dockerfiles/node.Dockerfile" --build-arg "NODE_VERSION=$node_version" --build-arg "UID=1000" --build-arg "GID=1000" --build-arg "USERNAME=dockery" --build-arg "SCRIPTOMATIC_REF=main" -t "$node_image" "$ROOT/docker/dockerfiles" + +docker run --rm --entrypoint sh "$node_image" -ec ' + node --version + npm --version + command -v gitx >/dev/null + command -v chromacat >/dev/null + gitx --version + chromacat --version +' + +printf 'Runtime build smoke passed: PHP %s / Node %s\n' "$php_version" "$node_version" From 2ded253704a15dbd300f2886343691b02d7ee5b9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:20:40 +0600 Subject: [PATCH 187/509] ci: smoke current Scriptomatic and runtime selections weekly --- .github/workflows/check.yml | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index 30afb658..f24edc1c 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -141,3 +141,16 @@ jobs: - name: Build documentation run: sphinx-build -W --keep-going -b html docs docs/_build/html + + + runtime-build: + name: Runtime build compatibility + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + runs-on: ubuntu-latest + timeout-minutes: 45 + steps: + - name: Checkout + uses: actions/checkout@v7 + + - name: Build current selected PHP/Node runtimes + run: bash tests/runtime-build-smoke.sh From 1bcba4b9bf4f12ccf8f241cda90d8855532ae665 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:22:40 +0600 Subject: [PATCH 188/509] plan: record final cross-image compatibility audit --- .../07-localdevstack-integration-plan.md | 128 ++++++++++++++++++ 1 file changed, 128 insertions(+) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 22ff4aaf..4d69c2b9 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -2053,3 +2053,131 @@ These remain future work rather than release blockers: - Graphify installation/management; - additional browser AI UI; - automatic model downloads beyond the provider defaults. + + +--- + +# Post-implementation cross-image audit — 2026-09-18 + +Audit window: approximately **2026-09-16 13:53 Asia/Dhaka through 2026-09-18**. + +Compared LocalDevStack against the current related releases/main contracts: + +- Tools **0.23.2** +- Runner **0.5** +- Nginx **0.4.1** +- Apache **0.4.2** +- LLM-SM **0.03** +- Toolset **2.0** +- Scriptomatic current `main` + +The final published/check runs for those release heads are successful. + +## Feature-parity result + +No legacy LocalDevStack command/service/storage feature was removed: + +- all **60** old public/support `lds` functions still exist after modularization; +- all **9** old `bin/*` wrappers remain; +- all **16** old Compose services remain; +- all **19** old named volumes remain. + +New runtime additions are additive: + +- `llm-sm`; +- `LLMModels`; +- `ToolsState`; +- AI/QoL/diagnostic commands. + +## Cross-image issues found and corrected + +### Tools durable state + +Tools 0.23.2 owns mutable state under `/etc/share/state` for host-manager/env-store, +profile/runtime state, monitor history and alert acknowledgement state. + +LocalDevStack now persists that directory through the global named volume: + +```text +ToolsState -> /etc/share/state +``` + +This prevents Tools control-plane state from disappearing when `server-tools` is recreated. + +### Root CA export bridge + +Tools exports the public CA to: + +```text +configuration/ssl/rootCA.pem +``` + +Unix and Windows LocalDevStack certificate-install paths now use that current export, +with `configuration/rootCA/rootCA.pem` retained only as a legacy read fallback. + +### TLS export permissions + +Tools deliberately exports opt-in user P12 material with mode `0600`. +`lds setup permissions` now preserves restrictive modes for P12/PFX/key artifacts +instead of widening all files under `configuration/ssl` to group-readable mode. + +### Tools profile visibility + +`COMPOSE_PROFILES` is now passed into `server-tools` so the current Tools status/Admin +diagnostic layer can report LocalDevStack's active profile selection. + +### Moving-image drift + +Because LocalDevStack intentionally follows moving image aliases, the compatibility +workflow now runs weekly even when LocalDevStack source has not changed. + +A scheduled/manual runtime-build smoke also builds the current selected PHP and Node +versions from Tools' runtime catalog against Scriptomatic `main` and verifies the +runtime plus Toolset helper surface. + +## Apache compatibility decision + +Apache 0.4.2 is architecturally an optional backend, and the LocalDevStack CLI already +tracks `APACHE_ACTIVE` / `APACHE_DELETE` state. + +However, the new Tools Admin Panel Host Manager can create/edit Apache hosts directly +and currently does not control LocalDevStack's host-side `COMPOSE_PROFILES` lifecycle. + +Therefore Apache remains an always-created compatibility service in this release. +Making the container profile-only before adding a proper Admin Panel↔LocalDevStack +profile bridge would regress Admin Panel-created Apache hosts. + +This is a non-blocking follow-up, not a release defect. + +## LLM-SM capability boundaries + +The current LLM-SM 0.03 image is **linux/amd64 only**. LocalDevStack remains usable on +arm64 with the `ai` profile disabled. + +LocalDevStack intentionally does not mount a repository/workspace into `llm-sm` by +default. Direct model/API/chat/stdin workflows are supported. Repository-aware analysis +is supported through the Tools consumer layer (`lds ai review`, `repo-review`). + +The upstream optional workspace override is not automatically enabled because doing so +would weaken the no-repository-ingestion default and still requires explicit Git +safe-directory/identity decisions for writable repository operations. + +## Lower-layer compatibility notes + +- Tools' own feature-parity contract preserves its pre-hardening CLI/Admin surface. +- Tools' template ABI explicitly targets Nginx 0.4.1, Apache 0.4.2 and Runner 0.5. +- Runner 0.5 has a LocalDevStack-shaped integration smoke. +- Nginx routes use lazy Docker DNS and do not require optional profile services to exist + at Nginx startup. +- Scriptomatic still installs the same PHP/Node runtime Toolset helper surface + (`gitx` + `chromacat`); acquisition changed to the checksum-verified latest-stable + Toolset release installer. +- The PHP template's historical `GID:-root` fallback predates this audit window. + Supported LocalDevStack setup writes a numeric UID/GID before runtime generation, so + it is not a current LocalDevStack release blocker. + +## Readiness conclusion + +After the corrections above, there is no identified legacy feature loss or current +cross-image release blocker. The remaining items are explicit optional/future capability +work rather than regressions. From 86fc32d4aa026b19027a37da670b487f0b2f5491 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:23:47 +0600 Subject: [PATCH 189/509] ci: execute runtime build audit on integration branch --- .github/workflows/check.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index f24edc1c..ebcf9c8b 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -145,7 +145,7 @@ jobs: runtime-build: name: Runtime build compatibility - if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' + if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch' || (github.event_name == 'push' && github.ref == 'refs/heads/plan/docker-ecosystem-bottom-up') runs-on: ubuntu-latest timeout-minutes: 45 steps: From 3b50cb7a6e3daf7b86c19052a5a48efc6dd620ad Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:24:45 +0600 Subject: [PATCH 190/509] test: read version objects from Tools runtime catalog --- tests/runtime-build-smoke.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/runtime-build-smoke.sh b/tests/runtime-build-smoke.sh index 096cb7b0..e857e9f0 100644 --- a/tests/runtime-build-smoke.sh +++ b/tests/runtime-build-smoke.sh @@ -7,8 +7,8 @@ tools_image="infocyph/tools:latest" docker pull "$tools_image" >/dev/null versions="$(docker run --rm --entrypoint cat "$tools_image" /etc/share/runtime-versions.json)" -php_version="$(jq -r '.php.active[0] // empty' <<<"$versions")" -node_version="$(jq -r '.node.active[0] // empty' <<<"$versions")" +php_version="$(jq -r '.php.active[0].version // empty' <<<"$versions")" +node_version="$(jq -r '.node.active[0].version // empty' <<<"$versions")" [[ "$php_version" =~ ^[0-9]+\.[0-9]+$ ]] || { printf 'Invalid active PHP version from Tools: %s\n' "$php_version" >&2 exit 1 From 75f307fe185bcb14dba0bc4237bfe0ea48756a99 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:27:28 +0600 Subject: [PATCH 191/509] runtime: default optional PHP bootstrap args to empty --- docker/dockerfiles/php.Dockerfile | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docker/dockerfiles/php.Dockerfile b/docker/dockerfiles/php.Dockerfile index 44fa9a42..5636d6e1 100644 --- a/docker/dockerfiles/php.Dockerfile +++ b/docker/dockerfiles/php.Dockerfile @@ -10,10 +10,10 @@ LABEL org.opencontainers.image.authors="infocyph,abmmhasan" ARG USERNAME=dockery ENV USERNAME=${USERNAME} ARG PHP_PROFILE_KEY=84 -ARG LINUX_PKG -ARG LINUX_PKG_VERSIONED -ARG PHP_EXT -ARG PHP_EXT_VERSIONED +ARG LINUX_PKG="" +ARG LINUX_PKG_VERSIONED="" +ARG PHP_EXT="" +ARG PHP_EXT_VERSIONED="" ARG UID=1000 ARG GID=1000 ARG SCRIPTOMATIC_REF=main From eb499c2d6fedc632f211fc75ac44b3419a9e2b19 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:27:33 +0600 Subject: [PATCH 192/509] runtime: default optional Node bootstrap args to empty --- docker/dockerfiles/node.Dockerfile | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docker/dockerfiles/node.Dockerfile b/docker/dockerfiles/node.Dockerfile index 7a112688..4e41d09f 100644 --- a/docker/dockerfiles/node.Dockerfile +++ b/docker/dockerfiles/node.Dockerfile @@ -11,10 +11,10 @@ ARG USERNAME=dockery ENV USERNAME=${USERNAME} ARG UID=1000 ARG GID=1000 -ARG LINUX_PKG -ARG LINUX_PKG_VERSIONED -ARG NODE_GLOBAL -ARG NODE_GLOBAL_VERSIONED +ARG LINUX_PKG="" +ARG LINUX_PKG_VERSIONED="" +ARG NODE_GLOBAL="" +ARG NODE_GLOBAL_VERSIONED="" ARG SCRIPTOMATIC_REF=main ARG SCRIPTOMATIC_DOWNLOAD_CONNECT_TIMEOUT=10 ARG SCRIPTOMATIC_DOWNLOAD_MAX_TIME=120 From 853da945e7c1757e5ed97fb16d28699d4aeafac3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:27:47 +0600 Subject: [PATCH 193/509] test: require safe PHP optional arg defaults --- tests/runtime-php-contract.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/runtime-php-contract.sh b/tests/runtime-php-contract.sh index f86f67a9..1e117c12 100755 --- a/tests/runtime-php-contract.sh +++ b/tests/runtime-php-contract.sh @@ -12,10 +12,10 @@ assert_file_contains "$file" 'FROM php:${PHP_VERSION}-fpm-alpine' [[ "$(grep -c '^ARG PHP_VERSION' "$file")" -eq 2 ]] || fail "PHP_VERSION must be redeclared inside the build stage" assert_file_contains "$file" 'ARG PHP_PROFILE_KEY=84' -assert_file_contains "$file" 'ARG LINUX_PKG' -assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED' -assert_file_contains "$file" 'ARG PHP_EXT' -assert_file_contains "$file" 'ARG PHP_EXT_VERSIONED' +assert_file_contains "$file" 'ARG LINUX_PKG=""' +assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED=""' +assert_file_contains "$file" 'ARG PHP_EXT=""' +assert_file_contains "$file" 'ARG PHP_EXT_VERSIONED=""' assert_file_contains "$file" 'ARG UID=1000' assert_file_contains "$file" 'ARG GID=1000' assert_file_contains "$file" 'ARG SCRIPTOMATIC_REF=main' From b8e8d552fdd875e3ced0ac646182938ab37317e1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 14:27:51 +0600 Subject: [PATCH 194/509] test: require safe Node optional arg defaults --- tests/runtime-node-contract.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/runtime-node-contract.sh b/tests/runtime-node-contract.sh index ad1a2c16..5f244031 100755 --- a/tests/runtime-node-contract.sh +++ b/tests/runtime-node-contract.sh @@ -11,10 +11,10 @@ assert_file_contains "$file" 'ARG NODE_VERSION=current' assert_file_contains "$file" 'FROM node:${NODE_VERSION}-alpine' [[ "$(grep -c '^ARG NODE_VERSION' "$file")" -eq 2 ]] || fail "NODE_VERSION must be redeclared inside the build stage" -assert_file_contains "$file" 'ARG LINUX_PKG' -assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED' -assert_file_contains "$file" 'ARG NODE_GLOBAL' -assert_file_contains "$file" 'ARG NODE_GLOBAL_VERSIONED' +assert_file_contains "$file" 'ARG LINUX_PKG=""' +assert_file_contains "$file" 'ARG LINUX_PKG_VERSIONED=""' +assert_file_contains "$file" 'ARG NODE_GLOBAL=""' +assert_file_contains "$file" 'ARG NODE_GLOBAL_VERSIONED=""' assert_file_contains "$file" 'ARG UID=1000' assert_file_contains "$file" 'ARG GID=1000' assert_file_contains "$file" 'ARG SCRIPTOMATIC_REF=main' From 502c45ca7427901dc37230d7a251ab2cadfc4ac5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:01:03 +0600 Subject: [PATCH 195/509] fix: harden runtime state and support paths --- README.md | 17 +++- docs/concepts/storage-layout.rst | 4 +- docs/guides/domain-setup.rst | 9 ++- lds | 61 +++++++++----- lib/compose.sh | 10 ++- lib/diagnostics.sh | 34 ++++++-- lib/profiles.sh | 36 ++++++++- lib/services.sh | 132 +++++++++++++++++++++++-------- tests/docs-contract.sh | 5 +- tests/env-contract.sh | 33 ++++++++ tests/networking-contract.sh | 47 +++++++++++ tests/qol-contract.sh | 27 +++++++ 12 files changed, 347 insertions(+), 68 deletions(-) diff --git a/README.md b/README.md index 531315ef..f532e8a3 100644 --- a/README.md +++ b/README.md @@ -68,6 +68,7 @@ lds help lds up lds down lds restart +lds restart nginx # restart only selected service(s) lds status lds logs [service] lds rebuild [all|service...] @@ -85,9 +86,17 @@ lds config show --json lds config show --raw # explicitly shows secret-bearing effective config lds config env-used lds config validate + +lds support trace project.localhost +lds support bundle --redact + +lds clean --yes # LocalDevStack-scoped cleanup +lds clean --yes --volumes # also remove unused LocalDevStack volumes +lds clean --global --yes # explicit host-wide Docker prune ``` -`lds config show` is **redacted by default**. +`lds config show` is **redacted by default**. Shareable support bundles are also +redacted by default; `--full` is intentionally raw and may contain secrets. ## Runtime selection @@ -149,6 +158,10 @@ built-in fallback LocalDevStack never shell-sources `docker/.env` as executable code. +Re-running `lds setup profile` replaces the catalog-managed service selection +(database/cache/search/AI profiles) while preserving generated domain/runtime +profiles. + ## Optional local AI Enable the `ai` profile from the profile setup flow, then choose the runtime mode: @@ -203,7 +216,7 @@ Important named volumes include: - `EmailStore` - `LLMModels` -Host-owned configuration remains under `configuration/`, including PHP overrides, scheduler files, SOPS data, optional SSH material, and generated Compose fragments. +Host-owned configuration remains under `configuration/`, including PHP overrides, scheduler files, SOPS data, optional SSH material, and generated Compose fragments. Generated Nginx/Apache vhosts remain Docker-managed state in `NginxHosts`/`ApacheHosts`; `lds domain ls`, tracing, and support bundles inspect those persisted volumes through the control plane. `server-tools` and `runner` intentionally mount `/var/run/docker.sock`. That socket is equivalent to powerful host Docker control and is limited to those trusted control-plane components. Ordinary databases/admin clients and `llm-sm` do not receive it. diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 38207036..8c2d9cfd 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -37,7 +37,9 @@ LLMModels Ollama/local-model persistence when the ai profile is enabled. Named volumes are intentionally not renamed during this integration release so existing -developer data can survive upgrades. +developer data can survive upgrades. Domain listing, support traces, and support bundles +inspect generated Nginx/Apache vhost state through these named volumes instead of stale +host-side vhost paths. Host Configuration ------------------ diff --git a/docs/guides/domain-setup.rst b/docs/guides/domain-setup.rst index 4eac5ab7..48134866 100644 --- a/docs/guides/domain-setup.rst +++ b/docs/guides/domain-setup.rst @@ -52,9 +52,12 @@ Instead: Use:: + lds domain ls lds config validate -to validate the effective Compose graph and mounted scheduler configuration. +to list persisted domains and validate the effective Compose graph and mounted +scheduler configuration. Domain listing reads the NginxHosts named volume through +server-tools rather than relying on a host-side vhost directory. Routing ------- @@ -96,3 +99,7 @@ Open a known UI/domain:: Run diagnostics without mutating the stack:: lds doctor + lds support trace project.localhost + +The support trace reads the persisted Nginx vhost from NginxHosts (or the running +Nginx mount as a fallback), so upstream inference matches the active generated state. diff --git a/lds b/lds index c4c52071..404caa08 100755 --- a/lds +++ b/lds @@ -527,7 +527,7 @@ cmd_stack() { http) cmd_http "$@" ;; *) - die "stack " + die "stack " ;; esac } @@ -555,6 +555,7 @@ cmd_support() { open) cmd_open "$@" ;; bundle) cmd_bundle "$@" ;; notify) cmd_notify "$@" ;; + trace) cmd_support_trace "$@" ;; ui) cmd_ui "$@" ;; *) die "support " @@ -607,11 +608,19 @@ cmd_bundle() { echo "mode=$mode" } >"$tmp/meta.txt" - # compose config (effective) - { - echo "# docker compose config" - docker_compose config 2>&1 || true - } >"$tmp/compose.config.txt" + # compose config (effective). Redacted bundles must never contain interpolated + # secrets from docker/.env. + if [[ "$mode" == "redact" ]]; then + { + echo "# docker compose config" + docker_compose config 2>&1 || true + } | _redact_support_text >"$tmp/compose.config.txt" + else + { + echo "# docker compose config" + docker_compose config 2>&1 || true + } >"$tmp/compose.config.txt" + fi # ps + networks docker_compose ps >"$tmp/compose.ps.txt" 2>&1 || true @@ -626,13 +635,22 @@ cmd_bundle() { >"$tmp/docker.inspect.json" 2>/dev/null || true fi - # recent logs (tail) - docker_compose logs --no-color --tail 400 >"$tmp/compose.logs.txt" 2>&1 || true + # recent logs (tail). Apply best-effort secret-pattern redaction in the + # default shareable bundle. + if [[ "$mode" == "redact" ]]; then + docker_compose logs --no-color --tail 400 2>&1 | _redact_support_text >"$tmp/compose.logs.txt" || true + else + docker_compose logs --no-color --tail 400 >"$tmp/compose.logs.txt" 2>&1 || true + fi - # vhost configs - if [[ -d "$DIR/configuration/nginx" ]]; then - mkdir -p "$tmp/nginx" - cp -a "$DIR/configuration/nginx/." "$tmp/nginx/" 2>/dev/null || true + # Generated vhosts live in Docker named volumes. Capture them through the + # server-tools mount rather than stale host paths. + local tools_ctr + tools_ctr="$(_project_tools_container_running || true)" + if [[ -n "$tools_ctr" ]]; then + mkdir -p "$tmp/vhosts/nginx" "$tmp/vhosts/apache" + docker cp "$tools_ctr:/etc/share/vhosts/nginx/." "$tmp/vhosts/nginx/" >/dev/null 2>&1 || true + docker cp "$tools_ctr:/etc/share/vhosts/apache/." "$tmp/vhosts/apache/" >/dev/null 2>&1 || true fi # env files (redacted best-effort) @@ -652,11 +670,16 @@ cmd_bundle() { fi fi - # tools-side quick diagnostics (inside network) - { - echo "# ip r / ip a / ss" - _tools_exec "ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || true" - } >"$tmp/tools.net.txt" 2>&1 || true + # tools-side quick diagnostics (inside network). A support bundle remains + # usable even when server-tools itself is down. + if [[ -n "$tools_ctr" ]]; then + { + echo "# ip r / ip a / ss" + docker exec -i "$tools_ctr" sh -lc 'ip r; echo; ip a; echo; ss -tulpen 2>/dev/null || true' + } >"$tmp/tools.net.txt" 2>&1 || true + else + printf '%s\n' "server-tools is not running; in-container network diagnostics unavailable." >"$tmp/tools.net.txt" + fi # pack (cd "$tmp" && zip -qr "$out" .) || die "Failed to write bundle: $out" @@ -679,7 +702,7 @@ cmd_help() { - `lds stack logs [svc] [--follow] [--since ] [--grep ]` *(alias: `logs`)* - `lds stack exec [cmd…]` *(alias: `exec`)* - `lds stack events [--since ]` *(alias: `events`)* -- `lds stack clean --yes [--volumes]` *(alias: `clean`)* +- `lds stack clean --yes [--volumes] [--global]` *(alias: `clean`; scoped by default)* - `lds stack diff [--config] [--json]` *(shows desired vs running images)* @@ -767,7 +790,7 @@ ${CYAN}Stack (compose):${NC} down|stop Alias of: stack down restart|reboot Alias of: stack restart status|ps|logs|exec|events Alias of: stack <...> - clean Alias of: stack <...> + clean --yes [--volumes] [--global] Alias of: stack clean (scoped by default) ${CYAN}Domain (vhosts + routing):${NC} domain add|rm|ls diff --git a/lib/compose.sh b/lib/compose.sh index c215e5cf..73b60c73 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -80,8 +80,14 @@ docker_compose() { "$@" } -# helper: print project name -lds_project() { printf '%s' "${__LDS_PROJECT:-$(basename -- "$DIR")}"; } +# helper: print the effective Compose project name. +# main.yaml owns the LocalDevStack default; COMPOSE_PROJECT_NAME remains an +# explicit override and must be reflected by label-scoped diagnostics. +lds_project() { + local project + project="$(compose_control_value COMPOSE_PROJECT_NAME LocalDevStack)" + printf '%s' "${__LDS_PROJECT:-$project}" +} # (QUIET by default) ──────────────────────────────── # Centralize quiet/verbose handling for compose subcommands. diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 8202f516..f734b22e 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -248,7 +248,20 @@ cmd_support_trace() { local dom="${1:-}" [[ -n "$dom" ]] || die "support trace " - local nconf="$DIR/configuration/nginx/$dom.conf" + local nconf="/etc/share/vhosts/nginx/$dom.conf" + local nconf_source="server-tools:$nconf" + local nconf_text="" ctr nginx_ctr + ctr="$(_project_tools_container_running || true)" + if [[ -n "$ctr" ]]; then + nconf_text="$(docker exec "$ctr" sh -c 'cat "$1" 2>/dev/null || true' sh "$nconf" 2>/dev/null || true)" + else + nginx_ctr="$(docker_compose ps -q nginx 2>/dev/null | sed -n '1p' || true)" + if [[ -n "$nginx_ctr" ]] && docker inspect -f '{{.State.Running}}' "$nginx_ctr" 2>/dev/null | grep -qx true; then + nconf="/etc/nginx/conf.d/$dom.conf" + nconf_source="nginx:$nconf" + nconf_text="$(docker exec "$nginx_ctr" sh -c 'cat "$1" 2>/dev/null || true' sh "$nconf" 2>/dev/null || true)" + fi + fi printf "%bTrace%b: %s\n" "$CYAN" "$NC" "$dom" # 1) DNS @@ -277,22 +290,22 @@ cmd_support_trace() { curl -sk -o /dev/null -D - -w $'time_namelookup=%{time_namelookup}\ntime_connect=%{time_connect}\ntime_appconnect=%{time_appconnect}\ntime_starttransfer=%{time_starttransfer}\ntime_total=%{time_total}\nhttp_code=%{http_code}\n' "https://$dom" | sed -n '1,30p' fi - # 4) Upstream inference from nginx conf (if exists) + # 4) Upstream inference from the persisted NginxHosts state. printf "\n%b[Upstream]%b\n" "$DIM" "$NC" - if [[ -r "$nconf" ]]; then - if grep -q fastcgi_pass "$nconf"; then + if [[ -n "$nconf_text" ]]; then + if grep -q fastcgi_pass <<<"$nconf_text"; then local php - php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + php="$(grep -Eo 'fastcgi_pass[[:space:]]+[^;]+' <<<"$nconf_text" | awk '{print $2}' | head -n1 || true)" printf "type=php\nfastcgi_pass=%s\n" "${php:-unknown}" - elif grep -q proxy_pass "$nconf"; then + elif grep -q proxy_pass <<<"$nconf_text"; then local up - up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' "$nconf" | awk '{print $2}' | head -n1 || true)" + up="$(grep -m1 -Eo 'proxy_pass[[:space:]]+http[s]?://[^;]+' <<<"$nconf_text" | awk '{print $2}' | head -n1 || true)" printf "type=proxy\nproxy_pass=%s\n" "${up:-unknown}" else printf "type=static\n" fi else - printf "nginx_conf=%s (missing)\n" "$nconf" + printf "nginx_conf=%s (missing or unavailable)\n" "$nconf_source" fi # 5) Recent nginx logs (compose) @@ -316,6 +329,11 @@ _redact_effective_config() { -e 's/("(ME_CONFIG_MONGODB_URL|DATABASE_URL)"[[:space:]]*:[[:space:]]*)"[^"]*"/\1"***REDACTED***"/g' } +_redact_support_text() { + _redact_effective_config | sed -E \ + -e 's/((password|secret|token|api[_-]?key|access[_-]?key)[=:][[:space:]]*)[^[:space:]]+/\1***REDACTED***/Ig' +} + _env_key_list() { local file="${1:-}" source="${2:-}" [[ -r "$file" ]] || return 0 diff --git a/lib/profiles.sh b/lib/profiles.sh index 9702eb50..632fedd7 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -69,10 +69,40 @@ flush_envs() { } flush_profiles() { - local profile + local current key profile + local -A managed=() seen=() + local -a existing=() updated=() + + # The setup wizard owns only catalog-managed service profiles. Generated + # runtime/domain profiles (for example apache/php/node fragments) must survive + # a service-profile reselection. + for key in "${SERVICE_ORDER[@]}"; do + profile="${SERVICES[$key]:-}" + [[ -n "$profile" ]] && managed["$profile"]=1 + done + for profile in "${PENDING_PROFILES[@]}"; do - modify_profiles add "$profile" + [[ -n "$profile" && -z "${seen[$profile]:-}" ]] || continue + updated+=("$profile") + seen["$profile"]=1 + done + + current="$(dotenv_value "$ENV_DOCKER" COMPOSE_PROFILES 2>/dev/null || true)" + IFS=',' read -r -a existing <<<"$current" + for profile in "${existing[@]}"; do + profile="${profile//[[:space:]]/}" + [[ -n "$profile" ]] || continue + [[ -n "${managed[$profile]:-}" ]] && continue + [[ -n "${seen[$profile]:-}" ]] && continue + updated+=("$profile") + seen["$profile"]=1 done + + local joined="" + if (("${#updated[@]}" > 0)); then + joined="$(IFS=,; printf '%s' "${updated[*]}")" + fi + update_env "$ENV_DOCKER" COMPOSE_PROFILES "$joined" } # ── setup menu (selection-first) ────────────────────────────────────────────── @@ -80,7 +110,7 @@ flush_profiles() { setup_menu_print() { # Print menu to stderr to avoid stdout buffering in some Windows wrappers. { - printf "\n%bSetup profiles%b (will replace previous configuration, if exists):\n\n" "$CYAN" "$NC" + printf "\n%bSetup profiles%b (replaces catalog-managed service profiles; generated runtime/domain profiles are preserved):\n\n" "$CYAN" "$NC" local i=1 key slug display for key in "${SERVICE_ORDER[@]}"; do slug="${SERVICES[$key]}" diff --git a/lib/services.sh b/lib/services.sh index aaeed5f0..6858193d 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -118,7 +118,7 @@ compose_image_for_service() { declare -a __LDS_LEGACY_NETWORK_NAMES=(Frontend Backend DataStore) legacy_network_expected_subnet() { - case "\${1:-}" in + case "${1:-}" in Frontend) printf '%s' '172.28.0.0/24' ;; Backend) printf '%s' '172.29.0.0/24' ;; DataStore) printf '%s' '172.30.0.0/24' ;; @@ -127,10 +127,11 @@ legacy_network_expected_subnet() { } migrate_legacy_networks() { - local network expected subnets stack_label project_label schema_label ctr ctr_project attachments + local network expected subnets stack_label project_label schema_label ctr ctr_project attachments project local -a legacy=() + project="$(lds_project)" - for network in "\${__LDS_LEGACY_NETWORK_NAMES[@]}"; do + for network in "${__LDS_LEGACY_NETWORK_NAMES[@]}"; do docker network inspect "$network" >/dev/null 2>&1 || continue schema_label="$(docker network inspect -f '{{index .Labels "com.infocyph.network-schema"}}' "$network" 2>/dev/null || true)" @@ -142,30 +143,30 @@ migrate_legacy_networks() { stack_label="$(docker network inspect -f '{{index .Labels "com.infocyph.stack"}}' "$network" 2>/dev/null || true)" project_label="$(docker network inspect -f '{{index .Labels "com.docker.compose.project"}}' "$network" 2>/dev/null || true)" - if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "LocalDevStack" ]]; then - die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack. Remove or rename that network manually." + if [[ "$stack_label" != "LocalDevStack" || "$project_label" != "$project" ]]; then + die "Legacy subnet detected on '$network', but ownership labels do not prove it belongs to LocalDevStack project '$project'. Remove or rename that network manually." fi while IFS= read -r ctr; do [[ -n "$ctr" ]] || continue ctr_project="$(docker inspect -f '{{index .Config.Labels "com.docker.compose.project"}}' "$ctr" 2>/dev/null || true)" - if [[ "$ctr_project" != "LocalDevStack" ]]; then - die "Refusing to migrate '$network': container '$ctr' is not owned by the LocalDevStack Compose project." + if [[ "$ctr_project" != "$project" ]]; then + die "Refusing to migrate '$network': container '$ctr' is not owned by LocalDevStack project '$project'." fi done < <(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true) legacy+=("$network") done - (("\${#legacy[@]}" > 0)) || return 0 + (("${#legacy[@]}" > 0)) || return 0 - warn "Legacy fixed LocalDevStack network(s) detected: \${legacy[*]}" + warn "Legacy fixed LocalDevStack network(s) detected: ${legacy[*]}" warn "Recreating stack networks dynamically; named volumes and persisted data are preserved." # Stop/remove only LocalDevStack Compose containers and networks. Never use -v. docker_compose down --remove-orphans - for network in "\${legacy[@]}"; do + for network in "${legacy[@]}"; do docker network inspect "$network" >/dev/null 2>&1 || continue attachments="$(docker network inspect -f '{{range .Containers}}{{println .Name}}{{end}}' "$network" 2>/dev/null || true)" [[ -z "$attachments" ]] || @@ -227,8 +228,21 @@ cmd_down() { } cmd_restart() { - cmd_stop - cmd_start + if (($# == 0)); then + cmd_stop + cmd_start + return 0 + fi + + local arg svc + local -a services=() + for arg in "$@"; do + svc="$(resolve_service "$arg")" + compose_service_exists "$svc" || die "Unknown service: $arg" + services+=("$svc") + done + + docker_compose restart "${services[@]}" } cmd_reboot() { cmd_restart; } @@ -457,12 +471,15 @@ cmd_host() { delete_domain "$@" ;; list) - shopt -s nullglob - for f in "$DIR/configuration/nginx/"*.conf; do - printf '%s -' "$(basename -- "$f" .conf)" - done - shopt -u nullglob + local ctr + ctr="$(_project_tools_container_running || true)" + [[ -n "$ctr" ]] || die "server-tools container is not running for project: $(lds_project)" + docker exec "$ctr" sh -lc ' + for f in /etc/share/vhosts/nginx/*.conf; do + [ -e "$f" ] || continue + basename "$f" .conf + done + ' | LC_ALL=C sort ;; *) die "host " @@ -502,7 +519,7 @@ cmd_events() { } cmd_clean() { - local yes=0 vols=0 + local yes=0 vols=0 global=0 while [[ "${1:-}" ]]; do case "$1" in --yes | -y) @@ -513,32 +530,85 @@ cmd_clean() { vols=1 shift ;; + --global) + global=1 + shift + ;; *) - die "clean [--yes|-y] [--volumes|-v]" + die "clean [--yes|-y] [--volumes|-v] [--global]" ;; esac done ((yes)) || die "clean requires --yes" - printf "%b[clean]%b pruning stopped containers...\n" "$CYAN" "$NC" - docker container prune -f >/dev/null 2>&1 || true + if ((global)); then + warn "Global Docker cleanup requested; unrelated stopped containers, images, networks, build cache, and optionally volumes may be removed." + + printf "%b[clean]%b globally pruning stopped containers...\n" "$CYAN" "$NC" + docker container prune -f >/dev/null 2>&1 || true + printf "%b[clean]%b globally pruning unused networks...\n" "$CYAN" "$NC" + docker network prune -f >/dev/null 2>&1 || true + printf "%b[clean]%b globally pruning unused images...\n" "$CYAN" "$NC" + docker image prune -a -f >/dev/null 2>&1 || true + printf "%b[clean]%b globally pruning build cache...\n" "$CYAN" "$NC" + docker builder prune -a -f >/dev/null 2>&1 || true + if ((vols)); then + printf "%b[clean]%b globally pruning unused volumes...\n" "$CYAN" "$NC" + docker volume prune -f >/dev/null 2>&1 || true + fi + printf "%b[clean]%b global cleanup done\n" "$GREEN" "$NC" + return 0 + fi + + local project id net refs image + local -a ids=() networks=() volumes=() images=() + project="$(lds_project)" - printf "%b[clean]%b pruning unused networks...\n" "$CYAN" "$NC" - docker network prune -f >/dev/null 2>&1 || true + mapfile -t ids < <( + { + docker ps -aq --filter "label=com.docker.compose.project=$project" --filter status=created + docker ps -aq --filter "label=com.docker.compose.project=$project" --filter status=exited + docker ps -aq --filter "label=com.docker.compose.project=$project" --filter status=dead + } 2>/dev/null | awk 'NF' | sort -u + ) + if (("${#ids[@]}" > 0)); then + printf "%b[clean]%b removing stopped LocalDevStack containers...\n" "$CYAN" "$NC" + docker rm "${ids[@]}" >/dev/null 2>&1 || true + fi - printf "%b[clean]%b pruning unused images...\n" "$CYAN" "$NC" - docker image prune -a -f >/dev/null 2>&1 || true + mapfile -t networks < <( + docker network ls -q --filter "label=com.infocyph.stack=LocalDevStack" --filter "label=com.docker.compose.project=$project" 2>/dev/null || true + ) + for net in "${networks[@]}"; do + [[ -n "$net" ]] || continue + refs="$(docker network inspect -f '{{len .Containers}}' "$net" 2>/dev/null || printf '1')" + [[ "$refs" == "0" ]] || continue + docker network rm "$net" >/dev/null 2>&1 || true + done - printf "%b[clean]%b pruning build cache...\n" "$CYAN" "$NC" - docker builder prune -a -f >/dev/null 2>&1 || true + mapfile -t images < <( + { + docker images -q --filter 'reference=localdevstack-php:*' + docker images -q --filter 'reference=localdevstack-node:*' + } 2>/dev/null | awk 'NF' | sort -u + ) + for image in "${images[@]}"; do + docker image rm "$image" >/dev/null 2>&1 || true + done if ((vols)); then - printf "%b[clean]%b pruning unused volumes...\n" "$CYAN" "$NC" - docker volume prune -f >/dev/null 2>&1 || true + mapfile -t volumes < <( + docker volume ls -q --filter "label=com.infocyph.lds=1" --filter "label=com.infocyph.stack=LocalDevStack" 2>/dev/null || true + ) + for id in "${volumes[@]}"; do + [[ -n "$id" ]] || continue + docker volume rm "$id" >/dev/null 2>&1 || true + done fi - printf "%b[clean]%b done\n" "$GREEN" "$NC" + printf "%b[clean]%b LocalDevStack-scoped cleanup done\n" "$GREEN" "$NC" + warn "Docker build cache is intentionally untouched by scoped cleanup; use --global for host-wide pruning." } diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 6dfa3634..1e2bc289 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -23,7 +23,10 @@ assert_file_contains "$index" 'guides/local-ai' assert_file_contains "$readme" 'Dynamic Docker networking through service DNS' assert_file_contains "$arch" 'Docker assigns their address ranges dynamically' assert_file_contains "$domain" 'generated runtime Compose fragments are written under configuration/compose/' -pass "docs describe dynamic DNS/runtime generation" +assert_file_contains "$domain" 'Domain listing reads the NginxHosts named volume' +assert_file_contains "$storage" 'support traces, and support bundles' +assert_file_contains "$readme" 'lds clean --global --yes' +pass "docs describe dynamic DNS/runtime generation and safe operational boundaries" assert_file_contains "$profiles" 'docker/release.env' assert_file_contains "$profiles" 'docker/.env' diff --git a/tests/env-contract.sh b/tests/env-contract.sh index a751c468..6f2df465 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -61,3 +61,36 @@ if grep -R -Fq 'dc_build --no-cache' "$ROOT/lds" "$ROOT/lib"; then fi assert_file_contains "$ROOT/lib/services.sh" 'dc_build --pull "$svc"' pass "runtime rebuilds preserve cache while refreshing selected bases" + + +# Profile setup replaces catalog-managed service profiles while preserving +# generated runtime/domain profiles. +profile_tmp="$(mktemp -d)" +( + set -euo pipefail + CFG="$ROOT/docker" + ENV_DOCKER="$profile_tmp/docker.env" + CYAN="" NC="" BLUE="" YELLOW="" GREEN="" + die() { printf 'die: %s\n' "$*" >&2; exit 1; } + dotenv_value() { + local file="$1" key="$2" line + line="$(grep -E "^${key}=" "$file" 2>/dev/null | tail -n1 || true)" + [[ -n "$line" ]] || return 1 + printf '%s' "${line#*=}" + } + # shellcheck source=lib/env.sh + source "$ROOT/lib/env.sh" + # shellcheck source=lib/profiles.sh + source "$ROOT/lib/profiles.sh" + printf '%s\n' 'COMPOSE_PROFILES=mysql,redis,ai,apache,php84' >"$ENV_DOCKER" + PENDING_PROFILES=(postgresql) + flush_profiles + actual="$(grep '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1)" + [[ "$actual" == 'COMPOSE_PROFILES=postgresql,apache,php84' ]] || + fail "profile reselection drifted: $actual" +) +rm -rf "$profile_tmp" +pass "profile setup replaces managed selections and preserves generated profiles" + +assert_file_contains "$ROOT/lib/compose.sh" 'compose_control_value COMPOSE_PROJECT_NAME LocalDevStack' +pass "CLI project identity follows the Compose project contract" diff --git a/tests/networking-contract.sh b/tests/networking-contract.sh index 9590d4d3..e0d05dad 100755 --- a/tests/networking-contract.sh +++ b/tests/networking-contract.sh @@ -52,3 +52,50 @@ assert_file_contains "$ROOT/lib/services.sh" 'cmd_vpn_fix()' assert_file_contains "$ROOT/lds" 'vpn-fix) cmd_vpn_fix "$@"' assert_file_contains "$ROOT/lib/services.sh" 'migrate_legacy_networks' pass "safe legacy-network migration and vpn-fix deprecation" + + +# Exercise the migration path rather than only checking for its source text. +migration_log="$(mktemp)" +( + set -euo pipefail + RED="" GREEN="" CYAN="" YELLOW="" NC="" + die() { printf 'die: %s\n' "$*" >&2; exit 1; } + warn() { :; } + ok() { :; } + lds_project() { printf '%s' "LocalDevStack"; } + docker_compose() { printf 'compose %s\n' "$*" >>"$migration_log"; } + docker() { + printf 'docker %s\n' "$*" >>"$migration_log" + if [[ "${1:-}" == "network" && "${2:-}" == "inspect" ]]; then + local network="${!#}" + [[ "$network" == "Frontend" ]] || return 1 + if [[ "${3:-}" != "-f" ]]; then + return 0 + fi + case "${4:-}" in + *com.infocyph.network-schema*) return 0 ;; + *Subnet*) printf '%s\n' '172.28.0.0/24' ;; + *com.infocyph.stack*) printf '%s\n' 'LocalDevStack' ;; + *com.docker.compose.project*) printf '%s\n' 'LocalDevStack' ;; + *Containers*) return 0 ;; + *) return 0 ;; + esac + return 0 + fi + if [[ "${1:-}" == "network" && "${2:-}" == "rm" ]]; then + return 0 + fi + return 0 + } + # shellcheck source=lib/services.sh + source "$ROOT/lib/services.sh" + migrate_legacy_networks +) +assert_file_contains "$migration_log" 'docker network inspect Frontend' +assert_file_contains "$migration_log" 'compose down --remove-orphans' +assert_file_contains "$migration_log" 'docker network rm Frontend' +rm -f "$migration_log" +if grep -Fq '\${' "$ROOT/lib/services.sh"; then + fail "legacy migration contains escaped parameter expansion" +fi +pass "legacy network migration executes against real network names" diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 14313425..97ecf592 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -71,12 +71,39 @@ fi assert_contains "$redacted" "***REDACTED***" pass "config show redacts effective secrets by default" +bundle="$(mktemp --suffix=.zip)" +"$ROOT/lds" support bundle --redact "$bundle" >/dev/null +python3 - "$bundle" "supersecret-ci-value" <<'PY' +import sys, zipfile +path, secret = sys.argv[1:] +with zipfile.ZipFile(path) as z: + for name in z.namelist(): + data = z.read(name) + if secret.encode() in data: + raise SystemExit(f"support bundle leaked secret in {name}") +PY +rm -f "$bundle" +pass "support bundle redacts interpolated secrets" + help="$("$ROOT/lds" help)" assert_contains "$help" "doctor" assert_contains "$help" "images" assert_contains "$help" "urls" +assert_contains "$help" "support trace" +assert_contains "$help" "--global" pass "QoL commands are discoverable" assert_file_contains "$ROOT/lds" 'images | urls | doctor)' assert_file_contains "$ROOT/lib/diagnostics.sh" 'Docker daemon is unavailable.' pass "doctor owns Docker availability diagnostics" + + +assert_file_contains "$ROOT/lds" 'trace) cmd_support_trace "$@" ;;' +assert_file_contains "$ROOT/lib/services.sh" '/etc/share/vhosts/nginx/*.conf' +if grep -Fq '$DIR/configuration/nginx/' "$ROOT/lib/services.sh" "$ROOT/lib/diagnostics.sh"; then + fail "domain inspection must use persisted named-volume vhosts" +fi +assert_file_contains "$ROOT/lib/services.sh" 'docker_compose restart "${services[@]}"' +assert_file_contains "$ROOT/lib/services.sh" '--global' +assert_file_contains "$ROOT/lib/services.sh" 'label=com.docker.compose.project=$project' +pass "trace, domain listing, targeted restart, and scoped cleanup contracts" From 040d02aa4b98f7d6849b3e960f845e8272b01869 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:02:31 +0600 Subject: [PATCH 196/509] fix: keep support bundle default output safe --- lds | 2 +- tests/qol-contract.sh | 10 ++++++++++ 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/lds b/lds index 404caa08..f51f43c1 100755 --- a/lds +++ b/lds @@ -569,7 +569,7 @@ cmd_support() { # --full : include more logs/inspect output (can be large) cmd_bundle() { local mode="redact" - local out="${1:-}" + local out="" while [[ "${1:-}" ]]; do case "$1" in --redact) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 97ecf592..5f416e83 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -85,6 +85,16 @@ PY rm -f "$bundle" pass "support bundle redacts interpolated secrets" +default_bundle_dir="$(mktemp -d)" +( + cd "$default_bundle_dir" + "$ROOT/lds" support bundle --redact >/dev/null +) +default_bundle="$(find "$default_bundle_dir" -maxdepth 1 -type f -name 'lds_bundle_*.zip' -print -quit)" +[[ -n "$default_bundle" ]] || fail "support bundle option-only invocation did not create a default zip" +rm -rf "$default_bundle_dir" +pass "support bundle option-only invocation uses a generated filename" + help="$("$ROOT/lds" help)" assert_contains "$help" "doctor" assert_contains "$help" "images" From 036c596165a8283464e91f8449ea74966fc5b598 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:03:57 +0600 Subject: [PATCH 197/509] fix: package support bundles after staging --- lds | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/lds b/lds index f51f43c1..90b057ec 100755 --- a/lds +++ b/lds @@ -598,9 +598,6 @@ cmd_bundle() { local base="lds_bundle_${project}_${ts}" [[ -n "$out" ]] || out="$PWD/${base}.zip" - cleanup_bundle() { rm -rf "$tmp" 2>/dev/null || true; } - trap cleanup_bundle RETURN - { echo "project=$project" echo "dir=$DIR" @@ -682,7 +679,11 @@ cmd_bundle() { fi # pack - (cd "$tmp" && zip -qr "$out" .) || die "Failed to write bundle: $out" + if ! (cd "$tmp" && zip -qr "$out" .); then + rm -rf "$tmp" 2>/dev/null || true + die "Failed to write bundle: $out" + fi + rm -rf "$tmp" 2>/dev/null || true ok "Bundle created: $out" } From 74a86fe30247020215167e771f74f41b67c6de8b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:05:59 +0600 Subject: [PATCH 198/509] test: use fresh support bundle output path --- tests/qol-contract.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 5f416e83..85855976 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -71,7 +71,8 @@ fi assert_contains "$redacted" "***REDACTED***" pass "config show redacts effective secrets by default" -bundle="$(mktemp --suffix=.zip)" +bundle_dir="$(mktemp -d)" +bundle="$bundle_dir/bundle.zip" "$ROOT/lds" support bundle --redact "$bundle" >/dev/null python3 - "$bundle" "supersecret-ci-value" <<'PY' import sys, zipfile @@ -82,7 +83,7 @@ with zipfile.ZipFile(path) as z: if secret.encode() in data: raise SystemExit(f"support bundle leaked secret in {name}") PY -rm -f "$bundle" +rm -rf "$bundle_dir" pass "support bundle redacts interpolated secrets" default_bundle_dir="$(mktemp -d)" From 56c2e9b47fc57ff2537af763a617619a1f00afc7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:20:35 +0600 Subject: [PATCH 199/509] docs: complete LocalDevStack user documentation --- README.md | 357 ++++++++++++++++--------- docs/concepts/architecture.rst | 201 ++++++++++---- docs/concepts/profiles-and-env.rst | 195 +++++++++++--- docs/concepts/storage-layout.rst | 163 +++++++---- docs/guides/ad-hoc-runner.rst | 161 +++++++++++ docs/guides/databases-and-clients.rst | 187 +++++++++++++ docs/guides/domain-setup.rst | 156 ++++++++--- docs/guides/local-ai.rst | 142 +++++----- docs/guides/notifications.rst | 341 ++++------------------- docs/guides/operations-and-support.rst | 249 +++++++++++++++++ docs/guides/secrets-sops-age.rst | 78 +++++- docs/guides/tls-and-certificates.rst | 133 +++++---- docs/index.rst | 16 +- docs/quickstart.rst | 156 +++++++---- docs/reference/cli.rst | 339 +++++++++++++++++++++++ lds | 203 +++++++------- tests/docs-contract.sh | 31 ++- 17 files changed, 2243 insertions(+), 865 deletions(-) create mode 100644 docs/guides/ad-hoc-runner.rst create mode 100644 docs/guides/databases-and-clients.rst create mode 100644 docs/guides/operations-and-support.rst create mode 100644 docs/reference/cli.rst diff --git a/README.md b/README.md index f532e8a3..bbe04b30 100644 --- a/README.md +++ b/README.md @@ -1,29 +1,56 @@ # LocalDevStack -**Docker-based XAMPP alternative for PHP and Node.js local development.** +**A Docker-based XAMPP alternative for modern PHP and Node.js local development.** -LocalDevStack gives you local domains, trusted HTTPS, selectable PHP/Node runtimes, databases and admin UIs, Mailpit, background jobs, and optional local AI behind one `lds` CLI. +LocalDevStack provides local domains, trusted HTTPS, selectable PHP and Node runtimes, databases and admin UIs, Mailpit, background workers, developer utilities, diagnostics, and optional local AI behind one `lds` command. -## What you get +It is designed for **trusted local development infrastructure**. It is not a production deployment stack. -- Nginx as the front door on ports 80/443, with optional Apache behind it. -- Interactive PHP and Node runtime selection per domain. -- PostgreSQL, MySQL, MariaDB, MongoDB, Redis, and Elasticsearch profiles. -- CloudBeaver, RedisInsight, Mongo Express, and Kibana where applicable. -- Persistent Mailpit with local SMTP TLS. -- Runner-based cron, Supervisor, and log rotation. -- Optional local AI via `llm-sm`, available internally at `http://llm-sm:11434` and through `https://llm.localhost`. -- Dynamic Docker networking through service DNS; no fixed `172.28/29/30` addresses. +## Highlights -LocalDevStack is for **trusted local development infrastructure**, not production deployment. +- Nginx is the local HTTP/HTTPS front door on ports 80/443. +- Apache is always available as an alternate HTTP backend for domains that need it. +- PHP and Node runtimes are selected per domain and built as version-specific Alpine images. +- PostgreSQL, MySQL, MariaDB, MongoDB, Redis, and Elasticsearch are profile-driven. +- CloudBeaver, RedisInsight, Mongo Express, and Kibana are enabled with their related profiles. +- Mailpit is part of the core stack and persists captured mail. +- Runner provides cron, Supervisor, and log-rotation support. +- Tools owns domains, certificates, the admin UI, developer helpers, secrets integration, diagnostics, monitoring, and AI-consumer features. +- Optional `llm-sm` provides a small local Ollama runtime with persistent models. +- Dynamic Docker networking through service DNS removes fixed subnet dependencies. +- Linux, Windows/Git Bash, WSL, and Docker Desktop workflows are supported by the host CLI. + +## Architecture at a glance + +```text +Host + └─ lds / lds.bat + │ + ├─ Docker Compose + │ ├─ server-tools + │ ├─ runner + │ ├─ mailpit + │ ├─ nginx + │ ├─ apache + │ ├─ optional databases/admin clients + │ ├─ generated PHP/Node runtimes + │ └─ optional llm-sm + │ + ├─ configuration/ + └─ logs/ +``` + +LocalDevStack uses three logical Docker networks: `Frontend`, `Backend`, and `DataStore`. Docker assigns their address ranges dynamically. Internal communication uses service names such as `nginx`, `postgres`, `redis`, `server-tools`, and `llm-sm`. ## Prerequisites Install Docker first: -- Docker Engine is preferred on Linux. -- Docker Desktop is supported on Windows/macOS and can also be used on Linux. -- Windows CLI access uses `lds.bat` + Git Bash. +- **Linux:** Docker Engine is preferred; Docker Desktop is also usable. +- **Windows:** Docker Desktop plus Git Bash. `lds.bat` bridges into the Bash CLI. +- **macOS:** Docker Desktop. + +Useful host tools such as `jq`, `yq`, `rg`, `fd`, `tree`, and `shellcheck` can be proxied through the running `server-tools` container when they are not installed on the host. Docker itself always remains a host requirement. ## Recommended layout @@ -36,7 +63,7 @@ project-root/ └─ LocalDevStack/ ``` -Set `PROJECT_DIR` in `docker/.env` when your project directory lives elsewhere. +The default application bind mount is the sibling `application/` directory. Set `PROJECT_DIR` in `docker/.env` when your projects live elsewhere. ## Quick start @@ -45,109 +72,112 @@ git clone https://github.com/infocyph/LocalDevStack.git cd LocalDevStack chmod +x ./lds 2>/dev/null || true -sudo ./lds setup permissions # Linux/macOS +sudo ./lds setup permissions # Linux/macOS ./lds setup init ./lds setup profile ./lds up -./lds setup domain ``` -On Windows, run the equivalent commands through `lds.bat` or Git Bash. The permissions command configures the wrapper path and does not apply Unix chmod logic. - -Install the generated LocalDevStack root CA when you want browser-trusted HTTPS: +The first `up` starts the control plane and web stack. Then create a domain: ```bash -sudo lds certificate install +./lds setup domain ``` -## Common commands +For browser-trusted HTTPS, install the generated root CA: ```bash -lds help +sudo ./lds certificate install # Linux +``` -lds up -lds down -lds restart -lds restart nginx # restart only selected service(s) -lds status -lds logs [service] -lds rebuild [all|service...] +On Windows, use `lds.bat` or Git Bash. Certificate installation targets the current user's Windows root store and does not require the Unix `sudo` path. -lds setup profile -lds setup domain -lds profiles list +Run these checks after setup: -lds urls -lds images +```bash lds doctor - -lds config show -lds config show --json -lds config show --raw # explicitly shows secret-bearing effective config -lds config env-used +lds urls lds config validate +``` -lds support trace project.localhost -lds support bundle --redact +## Built-in endpoints -lds clean --yes # LocalDevStack-scoped cleanup -lds clean --yes --volumes # also remove unused LocalDevStack volumes -lds clean --global --yes # explicit host-wide Docker prune +The exact optional endpoints shown by `lds urls` depend on enabled profiles. + +| Purpose | URL | +| --- | --- | +| Tools/Admin | `https://admin.localhost` | +| Mailpit | `https://webmail.localhost` | +| CloudBeaver | `https://db.localhost` | +| RedisInsight | `https://ri.localhost` | +| Mongo Express | `https://me.localhost` | +| Kibana | `https://kibana.localhost` | +| Local AI | `https://llm.localhost` | + +Use `lds open admin`, `lds open mail`, `lds open db`, `lds open redis`, `lds open mongo`, `lds open kibana`, or `lds open ai` to open a known endpoint. + +## Profiles + +The guided profile selector is: + +```bash +lds setup profile ``` -`lds config show` is **redacted by default**. Shareable support bundles are also -redacted by default; `--full` is intentionally raw and may contain secrets. +Catalog-managed optional profiles are `postgresql`, `mysql`, `mariadb`, `mongodb`, `redis`, `elasticsearch`, and `ai`. -## Runtime selection +Re-running the wizard **replaces the catalog-managed selection** while preserving generated domain/runtime profiles. -The domain wizard keeps the runtime selector as the source of truth: +Manual profile operations remain available: -- PHP: select the PHP version for that domain. -- Node: select a Node version/tag for that domain. +```bash +lds profiles list +lds profiles add redis +lds profiles remove redis +``` -Generated local images remain version-specific: +## PHP and Node runtimes + +The domain wizard keeps runtime version choice in the user's hands. A selected version becomes a version-specific local image: ```text localdevstack-php: localdevstack-node: ``` -Selected PHP and Node images use Alpine variants. +Both runtime families use Alpine variants. Runtime builds consume Scriptomatic from `main` by default. For reproducible debugging/release work, `SCRIPTOMATIC_REF` also accepts a full 40-character commit SHA. + +Rebuild selected services with normal Docker cache preserved: -The local build path consumes Scriptomatic from `main` by default. Advanced users/releases may set `SCRIPTOMATIC_REF` to a full 40-character commit SHA. +```bash +lds rebuild php84 +lds rebuild nginx +lds rebuild all +``` ## Service image policy -Default image selection follows this rule: +Release-owned defaults live in `docker/release.env`. -> Prefer the moving Alpine variant when that image family publishes a suitable one; otherwise use its normal moving latest tag. +> Prefer the moving Alpine variant when the image family provides a suitable Alpine variant; otherwise use the normal moving latest tag. Examples: - PostgreSQL: `postgres:alpine` -- Tools / Runner / Nginx / Apache: their published `:latest` images (already built on their intended Alpine bases) -- LLM: `infocyph/llm-sm:latest` -- AMD LLM: `infocyph/llm-sm:amd-latest` - -Elasticsearch, Kibana, and Filebeat are kept on one aligned version because those image families do not provide a supported moving `latest` contract for this stack. +- Tools / Runner / Nginx / Apache: published `:latest` +- MySQL / MariaDB / MongoDB / Redis: their supported moving defaults +- local AI: `infocyph/llm-sm:latest` +- AMD local AI: `infocyph/llm-sm:amd-latest` -Run `lds images` to see the effective image set after release, user, and shell overrides. +Elasticsearch, Kibana, and Filebeat stay on one aligned Elastic version. -## Profiles and environment +Inspect the effective defaults with `lds images`. -Tracked product defaults live in: +## Environment ownership and precedence -```text -docker/release.env -``` +Tracked release defaults live in `docker/release.env`; user LocalDevStack settings live in `docker/.env`; the repository-root `.env` remains application-facing state where applicable. -User stack settings live in: - -```text -docker/.env -``` - -Precedence is: +LocalDevStack control precedence is: ```text built-in fallback @@ -156,15 +186,55 @@ built-in fallback < command-scoped shell environment ``` -LocalDevStack never shell-sources `docker/.env` as executable code. +`docker/.env` is read as dotenv data; it is not shell-sourced as executable code. + +Useful inspection commands: + +```bash +lds config env-used +lds config show +lds config show --json +lds config show --raw +lds config services +lds config profiles +lds config validate +``` + +`config show` is redacted by default. `--raw` can expose credentials. + +## Databases and clients + +Applications connect through Docker DNS names, not fixed IP addresses: `postgres`, `mysql`, `mariadb`, `mongodb`, `redis`, and `elasticsearch`. + +Host-side wrappers forward into the appropriate LocalDevStack service: -Re-running `lds setup profile` replaces the catalog-managed service selection -(database/cache/search/AI profiles) while preserving generated domain/runtime -profiles. +```bash +lds pg ... +lds psql ... +lds my ... +lds mysql ... +lds maria ... +lds mariadb ... +lds redis-cli ... +lds mongo ... +lds mongosh ... +lds es ... +``` + +See `docs/guides/databases-and-clients.rst` for the profile/client map. ## Optional local AI -Enable the `ai` profile from the profile setup flow, then choose the runtime mode: +Enable the `ai` profile through `lds setup profile`. + +```text +Tools consumer -> http://llm-sm:11434 +User HTTPS -> https://llm.localhost +Default model -> qwen2.5:3b +Model store -> LLMModels +``` + +Choose the runtime explicitly: ```bash lds llm runtime cpu @@ -172,88 +242,129 @@ lds llm runtime nvidia lds llm runtime amd ``` -Useful commands: +CPU/NVIDIA use `infocyph/llm-sm:latest`; AMD uses `infocyph/llm-sm:amd-latest`. + +Common commands: ```bash lds ai status -lds ai ask "explain this error" +lds ai ask "Explain this error" lds ai troubleshoot ... +lds ai review ... +lds ai repo-review ... + lds llm models lds llm pull +lds llm show lds llm chat ... ``` -Models persist in the `LLMModels` named volume. +Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. -Default access is through: +The `llm-sm` container receives **no Docker socket and no project/repository bind mount** by default. Current published `llm-sm` images are `linux/amd64`; the rest of LocalDevStack can still run on arm64 with the AI profile disabled. -```text -https://llm.localhost +## Storage and trust boundaries + +Important named volumes include `NginxHosts`, `ApacheHosts`, `SSLKeys`, `SSLRootCA`, `FPMPools`, `FPMSocks`, `ComposerGlobal`, `GitConfig`, `ToolsState`, database/admin stores, `EmailStore`, and `LLMModels`. + +Generated Nginx/Apache vhosts are Docker-managed state. `lds domain ls`, support tracing, and support bundles read the persisted vhost state through the control plane. + +`server-tools` and `runner` intentionally receive `/var/run/docker.sock`. Docker socket access is equivalent to powerful host Docker control. Ordinary databases, admin clients, Nginx/Apache, generated runtimes, and `llm-sm` do not receive it unless a user explicitly opts into a separate ad-hoc runner `--sock` flow. + +## Operations and support + +```bash +lds up +lds down +lds restart +lds restart nginx +lds status +lds ps +lds logs nginx --follow +lds stack diff ``` -Direct Ollama host access is disabled by default. When explicitly enabled: +A normal restart does not pull fresh images. Use `lds rebuild ` when you want to refresh/recreate a service. + +The first `up` or `start` after upgrading from the historical fixed-network layout safely migrates proven LocalDevStack legacy networks to dynamic bridges while preserving named volumes. + +Support tools: ```bash -lds llm host-port on +lds doctor +lds support trace project.localhost +lds support bundle --redact ``` -it binds only to: +Redacted support bundles are the default. `--full` is intentionally raw and can contain credentials or other sensitive material. -```text -127.0.0.1:11434 +Cleanup is scoped by default: + +```bash +lds clean --yes +lds clean --yes --volumes ``` -The LLM container does **not** receive the Docker socket or a project mount by default. +Host-wide Docker pruning is a separate explicit action: -## Storage and trust boundaries +```bash +lds clean --global --yes +``` + +That global mode can remove unrelated stopped containers, unused images/networks, build cache, and optionally volumes. `lds down --volumes --yes` is also destructive and should not be used for normal upgrades. -Important named volumes include: +## Ad-hoc Dockerfile runner -- `NginxHosts` / `ApacheHosts` -- `SSLKeys` / `SSLRootCA` -- `FPMPools` / `FPMSocks` -- database/admin-client stores -- `EmailStore` -- `LLMModels` +From a directory containing a Dockerfile: -Host-owned configuration remains under `configuration/`, including PHP overrides, scheduler files, SOPS data, optional SSH material, and generated Compose fragments. Generated Nginx/Apache vhosts remain Docker-managed state in `NginxHosts`/`ApacheHosts`; `lds domain ls`, tracing, and support bundles inspect those persisted volumes through the control plane. +```bash +lds run +lds run shell +lds run logs +lds run stop +lds run rm +``` -`server-tools` and `runner` intentionally mount `/var/run/docker.sock`. That socket is equivalent to powerful host Docker control and is limited to those trusted control-plane components. Ordinary databases/admin clients and `llm-sm` do not receive it. +The current directory is mounted at `/workspace`. -## Networking +```bash +lds run --publish 8080:8080 +lds run --mount ./data:/data +``` -Core networks are still logically separated as: +`--sock` deliberately grants the ad-hoc container the host Docker socket and should be used only for trusted images/code. -```text -Frontend -Backend -DataStore +## Notifications + +```bash +lds notify watch +lds notify test "LocalDevStack" "Notifications work" ``` -Docker assigns their subnets dynamically. Services communicate by Docker DNS names such as: +Linux/WSLg uses `notify-send` when available. Windows/Git Bash and compatible WSL environments use PowerShell toast support. Other hosts fall back to terminal output. -```text -server-tools -runner -mailpit -postgres -mysql -mariadb -mongodb -redis -elasticsearch -llm-sm +## Command reference + +```bash +lds help +lds help --markdown ``` -The legacy `lds vpn-fix` command is deprecated because LocalDevStack no longer owns fixed private subnets. +The complete reference is maintained in `docs/reference/cli.rst`. ## Documentation - Full docs: https://docs.infocyph.com/projects/LocalDevStack -- Quick reference: `lds help` -- Local AI: `docs/guides/local-ai.rst` +- Getting started: `docs/quickstart.rst` +- Architecture: `docs/concepts/architecture.rst` - Profiles/env: `docs/concepts/profiles-and-env.rst` - Storage: `docs/concepts/storage-layout.rst` +- Domain setup: `docs/guides/domain-setup.rst` +- Databases/clients: `docs/guides/databases-and-clients.rst` +- Local AI: `docs/guides/local-ai.rst` +- Operations/support: `docs/guides/operations-and-support.rst` +- Ad-hoc runner: `docs/guides/ad-hoc-runner.rst` +- CLI reference: `docs/reference/cli.rst` ## License diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index 5957a9a3..ea4e2fb3 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -1,37 +1,80 @@ Architecture ============ -LocalDevStack is split into small, purpose-specific layers. +LocalDevStack separates host orchestration, trusted control-plane duties, web routing, +application runtimes, persistent service state, and optional local AI. -Core Components ---------------- +Host Orchestration +------------------ -lds / lds.bat - Host-side orchestrator for setup, profiles, Compose, diagnostics, runtime rebuilds, - domains, TLS, and convenience commands. +``lds`` / ``lds.bat`` + Host-side orchestration for setup, profiles, Compose, diagnostics, rebuilds, domains, + TLS installation, support tooling, and convenience wrappers. -Nginx - Front door for local HTTP/HTTPS traffic. It routes PHP-FPM, optional Apache, - Node applications, admin UIs, Mailpit, and llm.localhost. +``docker/release.env`` + Tracked release-owned defaults such as infrastructure image aliases. -Apache - Optional HTTP backend for projects that explicitly choose the Apache path. +``docker/.env`` + User-owned LocalDevStack configuration. -PHP / Node runtimes - Locally built, version-specific runtime images. The domain wizard preserves the - selected PHP/Node version and uses Alpine variants. +``configuration/compose/`` + Generated runtime Compose fragments discovered and merged into the effective stack. -Tools - Trusted control plane for vhost generation, certificates, admin UI, secrets, - Git helpers, monitoring, and AI-consumer commands. Tools control-plane state is - persisted separately in the ToolsState named volume. +Core Services +------------- -Runner - Background execution layer for Supervisor, cron, and log rotation. +``server-tools`` + Trusted control plane. Tools owns vhost generation, certificate generation, durable + domain/runtime metadata, admin UI behavior, secrets helpers, Git/dev utilities, + monitoring, and AI-consumer commands. -llm-sm - Optional local AI provider. It is enabled only through the ai profile and is - separate from Tools. +``runner`` + Background execution layer for Supervisor, cron definitions, and log rotation. + +``mailpit`` + Persistent local mail capture with LocalDevStack TLS material. + +``nginx`` + Host-facing HTTP/HTTPS front door. Ports 80/443 are published here. + +``apache`` + Always-available alternate HTTP backend. Individual domains decide whether to route + through Apache; the container remains part of the core stack so CLI and Admin Panel + domain creation retain the same capabilities. + +Application Runtimes +-------------------- + +PHP and Node runtimes are generated per selected version. The domain wizard preserves +the user's explicit version choice. + +Generated image identities are:: + + localdevstack-php: + localdevstack-node: + +Both runtime families use Alpine variants. Scriptomatic supplies common runtime bootstrap +behavior and installs Toolset according to its own current contract. + +Optional Data Services +---------------------- + +Catalog-managed profiles currently include:: + + postgresql + mysql + mariadb + mongodb + redis + elasticsearch + ai + +Related admin clients are enabled through the same profiles where applicable: +CloudBeaver, RedisInsight, Mongo Express, and Kibana. + +An advanced ``filebeat`` profile exists in Compose for Elastic log ingestion. It is not +part of the normal guided catalog and should be enabled deliberately alongside +Elasticsearch. Networking ---------- @@ -42,10 +85,10 @@ LocalDevStack keeps three logical networks:: Backend DataStore -Docker assigns their address ranges dynamically. Core services do not depend on -hard-coded 172.28/29/30 addresses. +Docker assigns their address ranges dynamically. Core services do not rely on fixed +private subnet addresses. -Service-to-service communication uses Docker DNS names such as:: +Service-to-service traffic uses Docker DNS names such as:: server-tools runner @@ -60,49 +103,103 @@ Service-to-service communication uses Docker DNS names such as:: elasticsearch llm-sm -The legacy lds vpn-fix workflow is deprecated because LocalDevStack no longer owns -fixed private subnets. +The historical ``lds vpn-fix`` command remains only as a deprecated compatibility +message because LocalDevStack no longer owns fixed bridge subnets. + +Legacy Network Migration +------------------------ -Runtime Flow ------------- +``lds up`` and ``lds start`` call the legacy-network migration guard before starting the +stack. -A normal web-domain flow is: +The migration is deliberately conservative: -1. lds setup domain delegates domain/runtime generation to Tools. -2. Tools writes HTTP vhosts into persistent named volumes. -3. Tools writes runtime Compose fragments under configuration/compose/. -4. Nginx routes by Docker service name or PHP-FPM socket. -5. Selected PHP/Node runtime images are built only for the chosen versions. +1. only the known historical network names are considered; +2. the expected historical subnet must match; +3. ownership labels must prove the network belongs to LocalDevStack; +4. attached containers must also belong to the effective Compose project; +5. Compose is brought down without ``-v``; +6. only proven legacy networks are removed. -For optional AI: +Persistent named volumes are not removed by this migration. -1. the ai profile starts llm-sm; -2. Tools consumes http://llm-sm:11434 internally; -3. Nginx exposes https://llm.localhost; -4. lds ai delegates operational AI to Tools; -5. lds llm delegates provider/model management to llm-sm. +Domain Flow +----------- + +A normal domain flow is: + +1. ``lds setup domain`` delegates domain/runtime generation to Tools; +2. Tools writes Nginx/Apache vhosts into persistent named volumes; +3. Tools writes runtime Compose fragments under ``configuration/compose/``; +4. generated runtime/server profiles are updated; +5. LocalDevStack recreates the effective stack; +6. Nginx routes through Docker service names or PHP-FPM sockets. + +Removing a domain follows the corresponding Tools ``rmhost`` state and removes generated +profiles before recreating the stack. + +AI Flow +------- + +When the ``ai`` profile is enabled: + +1. ``llm-sm`` provides the Ollama runtime and persistent model store; +2. Tools consumes ``http://llm-sm:11434`` internally; +3. Nginx exposes ``https://llm.localhost`` to the user; +4. ``lds ai`` delegates higher-level/operational AI to Tools; +5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-sm`` CLI. + +The standard CPU/NVIDIA provider and AMD/ROCm provider remain separate image variants. + +Project Identity +---------------- + +The Compose project contract defaults to ``LocalDevStack``. Label-scoped commands such +as diagnostics, events, support bundles, and cleanup use the effective Compose project +name instead of deriving identity from the checkout directory. + +An explicit ``COMPOSE_PROJECT_NAME`` override therefore remains coherent with these +operations. + +Tool Proxying +------------- + +Some non-privileged developer utilities can be resolved from ``server-tools`` when they +are unavailable on the host. Host binaries win first. Docker and other host-control +commands are never proxied. + +Disable proxying for a command with:: + + LDS_PROXY_TOOLS=0 lds Trust Boundaries ---------------- -server-tools and runner intentionally receive /var/run/docker.sock because their -supported workflows control sibling containers. Docker socket access is equivalent -to powerful host Docker control. +``server-tools`` and ``runner`` intentionally receive ``/var/run/docker.sock`` because +their supported workflows control sibling containers. Docker socket access is +equivalent to powerful host Docker control. The Docker socket is not mounted into: -- llm-sm; +- ``llm-sm``; - databases; - database admin clients; -- Nginx/Apache; -- ordinary runtime services. +- Nginx or Apache; +- ordinary generated runtimes. + +The separate ad-hoc ``lds run --sock`` option is an explicit opt-in and should be used +only with trusted Dockerfiles/code. -llm-sm also receives no project/repository mount by default. AI output is not +``llm-sm`` also receives no project/repository mount by default. AI output is not automatically executed as shell, SQL, or code. Persistence ----------- -Runtime-generated vhosts, certificate material, databases, Mailpit state, runtime -sockets/pools, and AI models use named volumes. Host-editable/generated configuration, -logs, SOPS state, optional SSH material, and public TLS exports remain under the repository. +Runtime-generated vhosts, certificate material, databases, Mailpit data, PHP-FPM state, +Tools control state, and AI models use named Docker volumes. + +Host-editable/generated configuration, logs, SOPS/Age state, optional SSH material, and +public TLS exports remain under the repository. + +See :doc:`storage-layout` for ownership details. diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index c1ca4992..c275abef 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -1,7 +1,8 @@ Profiles and Environment ======================== -LocalDevStack uses Docker Compose profiles so optional services are selected explicitly. +LocalDevStack uses Docker Compose profiles to keep optional services explicit while the +core development/control plane remains available. Guided Profile Setup -------------------- @@ -14,49 +15,112 @@ The host-side service catalog is tracked at:: docker/catalog/services.psv -It describes the service/profile name, setup defaults, prompts, convenience URL, -persistent volume metadata, and supported AI runtime modes. +It describes service/profile names, setup defaults/prompts, convenience URLs, +persistent-volume metadata, and supported AI runtime modes. -The current optional service profiles include: +Catalog-managed optional profiles are: -- PostgreSQL; -- MySQL; -- MariaDB; -- MongoDB; -- Redis; -- Elasticsearch; -- local AI. +- ``postgresql``; +- ``mysql``; +- ``mariadb``; +- ``mongodb``; +- ``redis``; +- ``elasticsearch``; +- ``ai``. -PHP and Node domain runtimes are selected separately by the domain wizard. Their -version selector is not replaced by the moving-image policy. +PHP/Node domain runtimes are selected separately by the domain wizard. + +Replacement Semantics +--------------------- + +Re-running ``lds setup profile`` replaces only the catalog-managed service selection. +Generated domain/runtime profiles are preserved. + +For example, if the existing profile set is conceptually:: + + mysql,redis,ai,apache,php84 + +and the guided selector is re-run with only PostgreSQL selected, the resulting set keeps +the generated runtime/server profiles:: + + postgresql,apache,php84 + +while removing the previously selected catalog services. + +Manual profile operations remain available:: + + lds profiles list + lds profiles add redis + lds profiles remove redis + +The advanced Compose-only ``filebeat`` profile can be enabled manually when required. +It should be used with Elasticsearch. Environment Ownership --------------------- -Tracked product defaults live in:: +Tracked product/release defaults live in:: docker/release.env -User stack settings live in:: +User LocalDevStack settings live in:: docker/.env -The repository root .env remains project/application-facing state where applicable; -it is not the LocalDevStack release-default file. +The repository-root ``.env`` remains project/application-facing state where applicable; +it is not the release-default manifest. -Effective precedence is:: +Effective LocalDevStack control precedence is:: built-in fallback < docker/release.env < docker/.env < command-scoped shell environment -LocalDevStack reads dotenv values as data. It does not blindly shell-source docker/.env. +LocalDevStack reads dotenv values as data. It does not blindly shell-source +``docker/.env``. + +Initialization +-------------- + +Run:: + + lds setup init + +to initialize workstation defaults such as: + +- ``TZ``; +- ``GIT_USER_NAME``; +- ``GIT_USER_EMAIL``. + +Setup also maintains LocalDevStack host identity values such as ``WORKING_DIR`` and, +on non-root Unix invocation, ``USER``, ``UID``, and ``GID``. + +Common Configuration Keys +------------------------- + +Frequently used user overrides include:: + + PROJECT_DIR=/path/to/application + HTTP_PORT=80 + HTTPS_PORT=443 + COMPOSE_PROFILES=... + COMPOSE_PROJECT_NAME=LocalDevStack + +Infrastructure image overrides include:: -Useful Commands ---------------- + LDS_TOOLS_IMAGE=infocyph/tools:latest + LDS_RUNNER_IMAGE=infocyph/runner:latest + LDS_NGINX_IMAGE=infocyph/nginx:latest + LDS_APACHE_IMAGE=infocyph/apache:latest -Show which environment keys are defined without printing values:: +The moving values above are release defaults; normal users usually leave them in +``docker/release.env`` rather than copying them to ``docker/.env``. + +Useful Inspection Commands +-------------------------- + +Show environment keys without printing values:: lds config env-used @@ -64,15 +128,24 @@ Show effective Compose config with secrets redacted:: lds config show +JSON form:: + + lds config show --json + Show raw effective Compose config only when intentionally needed:: lds config show --raw +Show resolved services/profiles:: + + lds config services + lds config profiles + Validate Compose and mounted scheduler configuration:: lds config validate -Show selected profiles:: +Show selected/catalog profiles:: lds profiles list @@ -86,44 +159,48 @@ The default policy is: Examples: -- PostgreSQL defaults to postgres:alpine. -- Tools, Runner, Nginx, and Apache use their published :latest aliases. -- Standard local AI uses infocyph/llm-sm:latest. -- AMD local AI uses infocyph/llm-sm:amd-latest. -- Elasticsearch, Kibana, and Filebeat stay on one aligned version because their - stack does not provide the required moving latest contract. +- PostgreSQL defaults to ``postgres:alpine``. +- Tools, Runner, Nginx, and Apache use their published ``:latest`` aliases. +- Standard local AI uses ``infocyph/llm-sm:latest``. +- AMD local AI uses ``infocyph/llm-sm:amd-latest``. +- Elasticsearch, Kibana, and Filebeat stay on one aligned version. -Explicit user values in docker/.env or the shell override these defaults. +Run:: + + lds images + +to inspect the effective image set after release defaults and user/shell overrides. Runtime Version Selection ------------------------- -Tools publishes the runtime catalog used by mkhost. The user still chooses the -runtime version per domain. +Tools publishes the runtime catalog used by ``mkhost``. The user still selects a runtime +version per domain. PHP selection becomes: -- PHP_VERSION build input; -- localdevstack-php: image identity; +- ``PHP_VERSION`` build input; +- ``localdevstack-php:`` image identity; - Alpine PHP-FPM base. Node selection becomes: -- NODE_VERSION build input; -- localdevstack-node: image identity; +- ``NODE_VERSION`` build input; +- ``localdevstack-node:`` image identity; - Alpine Node base. -Runtime rebuilds preserve Docker build cache while using --pull to refresh the +Runtime rebuilds keep normal Docker build cache while using ``--pull`` to refresh the selected base. Scriptomatic ------------ -Runtime builds consume Scriptomatic from main by default:: +Runtime builds consume Scriptomatic from ``main`` by default:: SCRIPTOMATIC_REF=main -A full 40-character commit SHA is also accepted for release/debug reproducibility. +A full 40-character commit SHA is also accepted for reproducible debugging/release +builds. Other arbitrary refs are rejected. AI Settings ----------- @@ -137,5 +214,41 @@ Important AI settings include:: LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=0 -Use lds llm runtime and lds llm host-port instead of editing these manually for -normal workflows. +Use:: + + lds llm runtime + lds llm host-port + +instead of editing the corresponding runtime flags manually in normal workflows. + +Tools also accepts optional timeout/context limits through ``LDS_AI_CONNECT_TIMEOUT``, +``LDS_AI_PREFLIGHT_TIMEOUT``, ``LDS_AI_TIMEOUT``, ``LDS_AI_AVAILABILITY_TTL``, +``LDS_AI_MAX_CONTEXT_BYTES``, ``LDS_AI_MAX_REQUEST_BYTES``, and +``LDS_AI_MAX_RESPONSE_BYTES``. + +Compose Extras +-------------- + +LocalDevStack discovers ``*.yaml`` and ``*.yml`` files under:: + + configuration/compose/ + +and merges them after the built-in product Compose files. This is where generated runtime +fragments live. + +Use the global option:: + + lds --reload-extras + +when a workflow must force an extras rescan before the command. + +Tool Proxy Control +------------------ + +Proxy-safe utilities can fall back to ``server-tools`` when unavailable on the host. +Disable that behavior with:: + + LDS_PROXY_TOOLS=0 lds + +Host-control utilities such as Docker, mount operations, chmod/chown, and system service +control are never proxied. diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 8c2d9cfd..ac2a2572 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -1,109 +1,170 @@ Storage Layout ============== -LocalDevStack deliberately separates Docker-managed runtime state from host-managed -configuration and exports. +LocalDevStack separates Docker-managed persistent runtime state from host-managed +configuration, exports, and logs. Named Volumes ------------- Important named volumes include: -NginxHosts / ApacheHosts +``NginxHosts`` / ``ApacheHosts`` Generated HTTP vhost configuration consumed by Nginx/Apache. -SSLKeys / SSLRootCA +``SSLKeys`` / ``SSLRootCA`` Runtime certificate/key material and the mkcert CA store. -FPMPools / FPMSocks +``FPMPools`` / ``FPMSocks`` Generated PHP-FPM pool configuration and runtime sockets. -ComposerGlobal / GitConfig - Shared runtime state for Composer and Git integration. +``ComposerGlobal`` / ``GitConfig`` + Shared Composer/Git runtime state. -ToolsState - Durable Tools control-plane state under /etc/share/state, including host-manager - state, profile/runtime metadata, monitor history, alert acknowledgements, and - other state owned by the Tools image. +``ToolsState`` + Durable Tools control-plane state under ``/etc/share/state``, including domain/runtime + metadata, monitor history, alert acknowledgements, and other Tools-owned state. -Database/admin volumes - Persistent PostgreSQL, MySQL, MariaDB, MongoDB, Redis, Elasticsearch, Kibana, - RedisInsight, CloudBeaver, and Filebeat state. +Database volumes + ``PostgresStore``, ``MySQLStore``, ``MariaDBStore``, ``MongoDBStore``, + ``RedisStore``, and ``ElasticSearchStore``. -EmailStore +Admin/observability volumes + ``RedisInsightStore``, ``CloudBeaverStore``, ``KibanaStore``, and ``FilebeatStore``. + +``EmailStore`` Mailpit persistence. -LLMModels - Ollama/local-model persistence when the ai profile is enabled. +``LLMModels`` + Ollama/local-model persistence when the ``ai`` profile is enabled. + +These named volumes are intentionally stable so developer data can survive container and +image replacement. + +Generated Vhost State +--------------------- + +Active Nginx/Apache vhosts do **not** live under host-side +``configuration/nginx``/``configuration/apache`` directories. -Named volumes are intentionally not renamed during this integration release so existing -developer data can survive upgrades. Domain listing, support traces, and support bundles -inspect generated Nginx/Apache vhost state through these named volumes instead of stale -host-side vhost paths. +They are persisted in ``NginxHosts`` / ``ApacheHosts`` and are available to Tools and +the web servers through their mounted paths. + +Accordingly: + +- ``lds domain ls`` reads persisted Nginx vhost state through ``server-tools``; +- ``lds support trace`` reads the named-volume vhost, with the running Nginx mount as a + fallback; +- support bundles copy generated Nginx/Apache vhost state through ``server-tools``. Host Configuration ------------------ -configuration/compose/ - LocalDevStack-generated runtime Compose fragments. The CLI discovers only YAML files - in this directory and validates the effective Compose graph before use. +``configuration/compose/`` + LocalDevStack-generated/runtime Compose fragments. Only YAML files are discovered. -configuration/php/ - User-editable PHP configuration. Normal updates must not overwrite user customizations. +``configuration/php/`` + User-editable PHP configuration. -configuration/scheduler/cron-jobs/ +``configuration/scheduler/cron-jobs/`` Runner cron definitions. -configuration/scheduler/supervisor/ +``configuration/scheduler/supervisor/`` Runner Supervisor definitions. -configuration/sops/config/, configuration/sops/global/, configuration/sops/keys/ - SOPS/Age configuration and sensitive key material. +``configuration/sops/config/``, ``configuration/sops/global/``, +``configuration/sops/keys/`` + SOPS/Age configuration, global secret data, and sensitive key material. -configuration/ssh/ - Optional SSH material mounted read-only into supported runtime/tool flows. +``configuration/ssh/`` + Optional SSH material mounted read-only into supported Tools/runtime flows. -configuration/ssl/ - User-facing TLS exports from Tools. The public root CA is exported here as - rootCA.pem. Optional password-protected mTLS user artifacts may also appear here. +``configuration/ssl/`` + User-facing TLS exports from Tools. The public root CA is ``rootCA.pem``. Optional + password-protected user mTLS artifacts may also be exported here. -logs/ +``logs/`` Host-visible service logs consumed by Runner log rotation and diagnostics. TLS Authority ------------- -Runtime TLS state is owned by the named volumes: +Runtime TLS state is owned by: -- SSLKeys for server/client certificate material; -- SSLRootCA for the mkcert CA store. +- ``SSLKeys`` for server/client certificate material; +- ``SSLRootCA`` for the mkcert CA store. Tools exports the public root certificate to:: configuration/ssl/rootCA.pem -lds certificate install uses that current path. Older installations using -configuration/rootCA/rootCA.pem remain readable as a migration fallback. +``lds certificate install`` uses that current path. Older installations using:: + + configuration/rootCA/rootCA.pem + +remain readable as a migration fallback. The private CA key is not intended as a public host export. +Application Mount +----------------- + +``PROJECT_DIR`` controls the application bind mount. A sibling ``application/`` directory +is the default, but absolute or relative alternatives are supported. + +Local AI receives no application/project bind mount by default. + +Docker Socket Boundary +---------------------- + +``server-tools`` and ``runner`` intentionally mount:: + + /var/run/docker.sock + +because their supported workflows need Docker control. + +Persistent databases, admin clients, Nginx/Apache, and ``llm-sm`` do not receive the +socket by default. + +``lds run --sock`` is a separate explicit opt-in for an ad-hoc container. + Permissions ----------- -On Unix-like hosts, lds setup permissions uses group-writable setgid directories -for configuration/ and logs/ rather than broad world-writable 777 modes. +On Unix-like hosts, ``lds setup permissions`` uses group-writable setgid directories for +``configuration/`` and ``logs/`` rather than broad world-writable modes. Sensitive host directories are restricted: -- configuration/ssh/ directories: 0700; files: 0600; -- configuration/sops/keys/ directories: 0700; files: 0600. +- ``configuration/ssh/`` directories: ``0700``; files: ``0600``; +- ``configuration/sops/keys/`` directories: ``0700``; files: ``0600``; +- exported P12/PFX/private-key-style files under ``configuration/ssl/``: ``0600``. -Windows keeps its platform-specific wrapper/permission behavior. +Public certificate exports remain readable. -Project Mount -------------- +Cleanup and Data Safety +----------------------- -PROJECT_DIR controls the application bind mount. A common layout is a sibling -application/ directory, but absolute or relative alternatives are supported. +Normal stack shutdown does not delete named volumes:: -Local AI receives no application/project bind mount by default. + lds down + +Explicit destructive volume removal requires confirmation:: + + lds down --volumes --yes + +Scoped cleanup is safer for routine maintenance:: + + lds clean --yes + +Adding ``--volumes`` attempts to remove LocalDevStack-labelled volumes that are not in +use:: + + lds clean --yes --volumes + +Host-wide Docker pruning requires the explicit ``--global`` flag and may affect unrelated +projects:: + + lds clean --global --yes + +Do not use destructive volume cleanup as part of a normal upgrade. diff --git a/docs/guides/ad-hoc-runner.rst b/docs/guides/ad-hoc-runner.rst new file mode 100644 index 00000000..ca03a36e --- /dev/null +++ b/docs/guides/ad-hoc-runner.rst @@ -0,0 +1,161 @@ +Ad-hoc Dockerfile Runner +======================== + +``lds run`` is a small host-side utility for building and running a Dockerfile from the +current directory without adding that project to the main LocalDevStack Compose graph. + +It is separate from generated PHP/Node domain runtimes. + +Basic Flow +---------- + +From a directory containing a Dockerfile:: + + cd /path/to/project + lds run + +The default action: + +1. derives a deterministic project slug from the directory name plus a short path hash; +2. uses ``:local`` as the default image tag; +3. builds the image only when that tag is not already present; +4. starts a labelled container; +5. mounts the current directory at ``/workspace``; +6. uses ``/workspace`` as the working directory; +7. keeps the container alive with a shell/sleep command by default. + +Open a shell:: + + lds run shell + +The shell command builds/starts first when required, then opens Bash when available or +falls back to ``sh``. + +Container Identity +------------------ + +Default container names look conceptually like:: + + lds-run-- + +The path hash prevents same-named projects in different directories from colliding. + +Override the name or image tag explicitly:: + + lds run --name my-local-container + lds run --tag my-image + lds run --tag my-image:dev + +When a custom tag has no colon, ``:local`` is appended. + +Build Control +------------- + +Skip the build step when the required image already exists:: + + lds run --no-build + +Force a rebuild by removing the managed container/image first:: + + lds run rm + lds run + +``lds run rm`` removes the matching managed container and its image. + +Container Command vs Keepalive +------------------------------ + +Default mode replaces the image command with a keepalive shell and disables the image +healthcheck so an unrelated healthcheck does not mark the sleeping development +container unhealthy. + +Run the image's normal entrypoint/command instead:: + + lds run --no-keepalive + +Published Ports +--------------- + +Publish one or more ports:: + + lds run --publish 8080:8080 + lds run -p 8080:8080 -p 9229:9229 + +Open the first published port in the host browser:: + + lds run open + +Select a published container port/path:: + + lds run open --port 8080 --path /health + +Use HTTPS when the exposed service expects it:: + + lds run open --port 8443 --https + +Extra Mounts +------------ + +The project directory is always mounted at ``/workspace``. + +Add another mount:: + + lds run --mount ./data:/data + +When the container path is omitted, LocalDevStack mounts the path below ``/mnt`` using +its basename. + +Relative host paths are resolved against the current run directory. + +Docker Socket Opt-in +-------------------- + +Grant the container the host Docker socket only when deliberately required:: + + lds run --sock + +This mounts:: + + /var/run/docker.sock:/var/run/docker.sock + +Docker socket access is equivalent to powerful host Docker control. Use ``--sock`` only +with trusted Dockerfiles and code. + +Host OS +------- + +LocalDevStack automatically detects ``linux``, ``macos``, or ``windows`` and exports it +as ``HOST_OS`` inside the ad-hoc container. + +Override it when required:: + + lds run --host-os linux + +Windows/Git Bash path conversion is handled explicitly so host paths and container +``/workspace`` paths are not confused by MSYS rewriting. + +Lifecycle Commands +------------------ + +List all ad-hoc containers managed by this feature:: + + lds run ps + +Follow logs:: + + lds run logs + +Stop the current-directory container:: + + lds run stop + +Remove its container and image:: + + lds run rm + +Open a shell:: + + lds run shell + +The ad-hoc runner is intentionally independent from ``lds clean`` and the main +LocalDevStack Compose project. diff --git a/docs/guides/databases-and-clients.rst b/docs/guides/databases-and-clients.rst new file mode 100644 index 00000000..f9ac335e --- /dev/null +++ b/docs/guides/databases-and-clients.rst @@ -0,0 +1,187 @@ +Databases and Admin Clients +=========================== + +Database services are profile-driven and communicate through Docker DNS names. Applications +inside LocalDevStack should use service names rather than container IP addresses. + +Service Map +----------- + +.. list-table:: + :header-rows: 1 + :widths: 18 18 24 24 + + * - Profile + - Service DNS + - Admin client + - Convenience URL + * - ``postgresql`` + - ``postgres`` + - CloudBeaver + - ``https://db.localhost`` + * - ``mysql`` + - ``mysql`` + - CloudBeaver + - ``https://db.localhost`` + * - ``mariadb`` + - ``mariadb`` + - CloudBeaver + - ``https://db.localhost`` + * - ``mongodb`` + - ``mongodb`` + - Mongo Express + - ``https://me.localhost`` + * - ``redis`` + - ``redis`` + - RedisInsight + - ``https://ri.localhost`` + * - ``elasticsearch`` + - ``elasticsearch`` + - Kibana + - ``https://kibana.localhost`` + +Enable services through:: + + lds setup profile + +or manage profiles manually:: + + lds profiles add postgresql redis + lds profiles remove redis + +Persistence +----------- + +Each primary datastore has a stable named volume: + +- PostgreSQL: ``PostgresStore``; +- MySQL: ``MySQLStore``; +- MariaDB: ``MariaDBStore``; +- MongoDB: ``MongoDBStore``; +- Redis: ``RedisStore``; +- Elasticsearch: ``ElasticSearchStore``. + +Admin/observability state is persisted separately where the upstream application needs +it. + +Connection Settings +------------------- + +The guided profile flow writes service settings to ``docker/.env``. + +Important keys include: + +PostgreSQL + ``POSTGRES_VERSION``, ``POSTGRES_USER``, ``POSTGRES_PASSWORD``, + ``POSTGRES_DATABASE``. + +MySQL + ``MYSQL_VERSION``, ``MYSQL_ROOT_PASSWORD``, ``MYSQL_USER``, ``MYSQL_PASSWORD``, + ``MYSQL_DATABASE``. + +MariaDB + ``MARIADB_VERSION``, ``MARIADB_ROOT_PASSWORD``, ``MARIADB_USER``, + ``MARIADB_PASSWORD``, ``MARIADB_DATABASE``. + +MongoDB + ``MONGODB_VERSION``, ``MONGODB_ROOT_USERNAME``, ``MONGODB_ROOT_PASSWORD``. + +Redis + ``REDIS_VERSION``. + +Elasticsearch + ``ELASTICSEARCH_VERSION``. The same version is used for Elasticsearch, Kibana, and + Filebeat so the Elastic stack stays aligned. + +The wizard defaults are intended for local development convenience. Change credentials +when a project or workstation policy requires stronger local isolation. + +CLI Wrappers +------------ + +LocalDevStack includes short wrappers so host commands can be routed into the relevant +service/runtime. + +PostgreSQL:: + + lds pg ... + lds psql ... + lds pg_dump ... + lds pg_restore ... + +MySQL:: + + lds my ... + lds mysql ... + lds mysqldump ... + +MariaDB:: + + lds maria ... + lds mariadb ... + lds mariadb-dump ... + +Redis:: + + lds redis ... + lds redis-cli ... + +MongoDB:: + + lds mongo ... + lds mongosh ... + lds mongoimport ... + lds mongoexport ... + +Elasticsearch:: + + lds es ... + lds elasticsearch ... + +The wrappers preserve the service-name based architecture and avoid depending on fixed +bridge addresses. + +Health and Dependencies +----------------------- + +Databases use local readiness probes. Related admin clients wait for their dependency +where the Compose contract declares a health condition. + +Check the whole stack with:: + + lds doctor + +Inspect a service directly with:: + + lds logs postgres + lds stack exec postgres + lds restart postgres + +Admin UIs +--------- + +Use:: + + lds urls + +to show only convenience endpoints whose related profile is enabled. + +Open them directly with:: + + lds open db + lds open redis + lds open mongo + lds open kibana + +Filebeat +-------- + +An advanced ``filebeat`` Compose profile is available for Elastic log ingestion. It is +not included in the normal guided catalog. + +Enable it deliberately alongside Elasticsearch, for example:: + + lds profiles add elasticsearch filebeat + +Filebeat reads the shared host log tree read-only and sends to the ``elasticsearch`` +service over Docker DNS. diff --git a/docs/guides/domain-setup.rst b/docs/guides/domain-setup.rst index 48134866..57a09684 100644 --- a/docs/guides/domain-setup.rst +++ b/docs/guides/domain-setup.rst @@ -1,63 +1,98 @@ Domain Setup ============ -Use the interactive domain wizard:: +LocalDevStack uses Tools for domain/vhost generation while the host ``lds`` CLI owns the +surrounding profile, Compose, and runtime orchestration. + +Create a Domain +--------------- + +Start the stack first so ``server-tools`` is available:: + + lds up + +Then run the interactive wizard:: lds setup domain -The wizard delegates generation to the Tools mkhost workflow while LocalDevStack -owns the surrounding Compose/profile/runtime orchestration. +or the canonical domain command:: -Wizard Flow ------------ + lds domain add -The wizard collects the information needed for the selected application type, including: +The wizard delegates to Tools ``mkhost`` and collects the details needed for the chosen +application type, including: 1. domain name; -2. PHP or Node application type; -3. runtime version; -4. HTTP server mode where applicable; +2. PHP, Node, or supported static/backend application type; +3. runtime version where applicable; +4. HTTP server path where applicable; 5. HTTP/HTTPS behavior; 6. document root; 7. request/body limits; 8. optional mutual TLS settings. +After generation, LocalDevStack reads Tools state, adds any required generated +server/runtime profile, clears temporary mkhost state, and recreates the stack. + Runtime Version Selection ------------------------- -Runtime selection is interactive and version-specific. +Runtime selection remains explicit and version-specific. -For PHP, the selected version remains the PHP_VERSION build input and produces:: +For PHP, the selected version produces:: localdevstack-php: -For Node, the selected version/tag remains the NODE_VERSION build input and produces:: +For Node, the selected version produces:: localdevstack-node: -Both runtime families use Alpine variants. +Both runtime families use Alpine variants. The version selector is intentionally not +replaced by the moving infrastructure-image policy. Generated State --------------- -The current architecture does not write active Nginx/Apache vhosts to -configuration/nginx or configuration/apache. +Active vhosts are Docker-managed state: -Instead: +- Nginx vhosts persist in ``NginxHosts``; +- Apache vhosts persist in ``ApacheHosts``; +- PHP-FPM pool state persists in ``FPMPools``; +- PHP-FPM sockets use ``FPMSocks``; +- generated runtime Compose fragments are written under ``configuration/compose/``. -- Nginx vhosts persist in the NginxHosts named volume; -- Apache vhosts persist in the ApacheHosts named volume; -- PHP-FPM pool state persists in FPMPools; -- generated runtime Compose fragments are written under configuration/compose/. +There is no active host-side ``configuration/nginx`` source of truth. -Use:: +List Domains +------------ + +List persisted Nginx domains:: lds domain ls - lds config validate -to list persisted domains and validate the effective Compose graph and mounted -scheduler configuration. Domain listing reads the NginxHosts named volume through -server-tools rather than relying on a host-side vhost directory. +The list is read through ``server-tools`` from the ``NginxHosts`` named volume. + +Remove a Domain +--------------- + +Use:: + + lds domain rm + +or pass arguments supported by the underlying Tools removal flow:: + + lds domain rm + +LocalDevStack delegates removal to Tools ``rmhost``, removes any generated server profile +reported by that operation, resets temporary removal state, and recreates the stack. + +The legacy command group remains available:: + + lds host add + lds host rm + lds host list + +but ``domain`` is the canonical interface. Routing ------- @@ -65,23 +100,70 @@ Routing LocalDevStack uses Docker DNS/service names instead of fixed bridge addresses. Generated HTTP configuration routes to logical runtime service names or PHP-FPM sockets. -The three logical networks remain Frontend, Backend, and DataStore, but Docker chooses -their address ranges dynamically. +Nginx is always the host-facing front door. Apache is always available as an alternate +backend for domains that choose that mode. + +The three logical networks remain ``Frontend``, ``Backend``, and ``DataStore`` while +Docker chooses their address ranges dynamically. TLS --- -When HTTPS is selected, Tools refreshes the shared LocalDevStack certificate set. The -certificate SAN set always includes localhost, \*.localhost, 127.0.0.1, and ::1 in -addition to generated domains/service-derived hosts. +When HTTPS is selected, Tools refreshes the shared LocalDevStack certificate set. + +The certificate SAN set includes at least:: + + localhost + *.localhost + 127.0.0.1 + ::1 -This means convenience hosts such as admin.localhost, webmail.localhost, and -llm.localhost can use the same LocalDevStack trust chain. +and Tools can include generated domain/service names. + +The wildcard covers built-in convenience endpoints such as ``admin.localhost``, +``webmail.localhost``, and ``llm.localhost``. + +Working in a Domain Container +----------------------------- + +Resolve a domain to its application/runtime container and open a shell:: + + lds core project.localhost + +When no domain is supplied, ``lds core`` lists known domains and prompts on an +interactive terminal. + +A direct container name can also be supplied to ``lds core``. + +For a generic container shell/command flow use:: + + lds cli + lds cli + +Diagnostics +----------- + +Validate the effective stack:: + + lds config validate + +Inspect a domain end to end:: + + lds support trace project.localhost + +The trace checks DNS, TLS, HTTP timing, generated Nginx upstream configuration, and +recent Nginx logs. + +Additional probes include:: + + lds diag dns project.localhost + lds diag tls project.localhost + lds diag http https://project.localhost Convenience Commands -------------------- -List active convenience URLs:: +List active built-in URLs:: lds urls @@ -95,11 +177,3 @@ Open a known UI/domain:: lds open kibana lds open ai lds open project.localhost - -Run diagnostics without mutating the stack:: - - lds doctor - lds support trace project.localhost - -The support trace reads the persisted Nginx vhost from NginxHosts (or the running -Nginx mount as a fallback), so upstream inference matches the active generated state. diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index baede4fc..da4b9db4 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -1,28 +1,28 @@ Local AI ======== -LocalDevStack can run a local Ollama-compatible provider as an optional profile. The -provider/runtime is llm-sm; Tools remains an AI consumer and never embeds a second -Ollama runtime. +LocalDevStack can run a local Ollama-compatible provider as an optional profile. +``llm-sm`` owns the provider/runtime; Tools remains the higher-level AI consumer and +never embeds a second Ollama runtime. Enable AI --------- -Use the guided service setup:: +Use the guided profile setup:: lds setup profile -Select Local AI / the ai profile. +Select Local AI / the ``ai`` profile. -The default provider contract is:: +The default consumer contract is:: LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama LDS_AI_URL=http://llm-sm:11434 LDS_AI_MODEL=qwen2.5:3b -The explicit LocalDevStack model default avoids ambiguity when several models are -installed in the persistent Ollama store. +The explicit model default avoids ambiguity when multiple models are installed in the +persistent Ollama store. Runtime Modes ------------- @@ -33,15 +33,20 @@ Select the provider runtime explicitly:: lds llm runtime nvidia lds llm runtime amd -CPU and NVIDIA use the standard infocyph/llm-sm:latest image. AMD/ROCm uses -infocyph/llm-sm:amd-latest. +CPU and NVIDIA use ``infocyph/llm-sm:latest``. AMD/ROCm uses +``infocyph/llm-sm:amd-latest``. + +NVIDIA mode adds Docker GPU access. AMD mode exposes ``/dev/kfd`` and ``/dev/dri``. LocalDevStack does not attempt unreliable automatic GPU detection. +Changing runtime mode updates ``docker/.env``. Recreate/start the service afterward so +the Compose override changes take effect. + Access ------ -Container-to-container provider endpoint:: +Internal provider endpoint:: http://llm-sm:11434 @@ -49,25 +54,28 @@ User-facing HTTPS endpoint:: https://llm.localhost -Nginx owns the host-facing route and preserves streaming behavior for Ollama/OpenAI-style -API calls. +Internal Docker consumers should use the service endpoint directly rather than routing +through Nginx. -Direct host Ollama access is disabled by default. To opt in:: +Direct host Ollama access is disabled by default. Opt in with:: lds llm host-port on -The direct binding is loopback-only:: +The default direct binding is loopback-only:: 127.0.0.1:11434 -Disable it again with:: +Show/disable it with:: + lds llm host-port status lds llm host-port off +The port can be changed through ``LLM_SM_PORT`` when direct host access is enabled. + AI vs LLM Commands ------------------ -lds ai is for Tools-owned operational intelligence:: +``lds ai`` is for Tools-owned operational/developer intelligence:: lds ai status lds ai ask "explain this error" @@ -77,105 +85,113 @@ lds ai is for Tools-owned operational intelligence:: lds ai repo-review ... lds ai graphify ... -lds llm is for provider/model runtime management:: +``lds llm`` is for the provider/model CLI:: lds llm models lds llm ps lds llm show lds llm pull lds llm rm + lds llm unload lds llm run + lds llm ask ... lds llm chat ... + lds llm prompt ... + lds llm code ... + lds llm review ... + lds llm json ... + lds llm ai-commit ... + lds llm ollama ... + lds llm api ... lds llm version -This separation keeps application/operational AI in Tools and model lifecycle inside +This separation keeps application/operational AI in Tools and model/runtime behavior in the provider image. Model Persistence ----------------- -Ollama state persists in the LLMModels named volume mounted at /root/.ollama. -Removing/recreating the container does not remove installed models unless the volume is -explicitly deleted. +Ollama state persists in the ``LLMModels`` named volume mounted at +``/root/.ollama``. + +Container recreation/image replacement does not delete user-pulled models while the +volume is retained. -Do not use destructive docker compose down -v during normal LocalDevStack upgrades. +Do not use ``lds down --volumes --yes`` or destructive volume cleanup during a normal +upgrade. Selecting a Different Model --------------------------- -Install the model through the provider:: +Pull a model explicitly:: lds llm pull -Then set LDS_AI_MODEL in docker/.env to make Tools use that model by default. -Shell environment overrides still have the highest precedence. +Then set ``LDS_AI_MODEL`` in ``docker/.env`` if Tools should use that model by default. +A command-scoped shell value remains the highest-precedence override. + +The provider CLI does not silently download a missing model for an unrelated command. Privacy and Trust Boundaries ---------------------------- -By default, llm-sm receives: +By default, ``llm-sm`` receives: - no Docker socket; - no project/repository bind mount; - no host port; -- only its model volume and the LocalDevStack networks needed for provider access. +- only the model volume and LocalDevStack networks needed for provider access. Tools may send bounded/sanitized context to the local provider when the user invokes an -AI feature. Deterministic monitoring and system checks remain the source of truth. +AI feature. Deterministic monitoring/system checks remain the source of truth. LocalDevStack does not: -- fall back silently to a cloud AI provider; +- silently fall back to a cloud AI provider; - automatically execute model-generated shell commands; - automatically execute model-generated SQL; - automatically execute generated code. +Repository Context +------------------ + +LocalDevStack intentionally does not mount the project/repository into ``llm-sm`` by +default. + +Repository-aware analysis should normally use the Tools consumer layer:: + + lds ai review ... + lds ai repo-review ... + lds ai graphify ... + +For direct provider commands, file/PDF/image paths must exist inside the provider +container. The upstream ``llm-sm`` CLI also supports stdin-based flows such as +``ai-commit --diff-stdin`` when explicitly invoked. + Graphify -------- -lds ai graphify delegates Graphify-assisted analysis to the Tools AI layer. The -LocalDevStack AI provider remains only the model runtime; Graphify integration does not -grant llm-sm direct repository access. +``lds ai graphify`` delegates Graphify-assisted analysis to the Tools AI layer. +``llm-sm`` remains only the model provider and does not gain direct repository access +from this integration. Diagnostics ----------- -Show provider state through Tools:: +:: lds ai status - -Check the LocalDevStack stack non-destructively:: - + lds llm models lds doctor - -List convenience endpoints:: - lds urls - + lds logs llm-sm Platform Availability --------------------- -The current llm-sm 0.03 publication is linux/amd64 only. LocalDevStack itself remains -usable on arm64 with the ai profile disabled. Native arm64 local-AI support should only -be advertised after the llm-sm image publishes and validates a native arm64 runtime. - -Direct File and Repository Context ----------------------------------- - -LocalDevStack intentionally does not mount the project/repository into llm-sm by default. - -This means normal provider/model commands work directly, including ask, chat, model -management, API access, and stdin-based prompts. Repository-aware analysis should use -the Tools consumer layer by default:: - - lds ai review ... - lds ai repo-review ... - -For direct llm-sm commands, file/PDF/image paths must exist inside the provider -container. Git-diff based ai-commit can be used without a repository mount by piping -the diff through stdin to the provider CLI. +The current published ``llm-sm`` standard and AMD images are validated/published as +``linux/amd64`` only. -The upstream llm-sm image provides an optional workspace override for explicit -repository mounts, but LocalDevStack does not enable it automatically. That preserves -the default no-repository-ingestion trust boundary. +LocalDevStack itself can still be used on arm64 with the ``ai`` profile disabled. Native +arm64 local-AI support should only be advertised after the provider publishes and +validates that platform. diff --git a/docs/guides/notifications.rst b/docs/guides/notifications.rst index 7e8214af..5f1d0137 100644 --- a/docs/guides/notifications.rst +++ b/docs/guides/notifications.rst @@ -1,322 +1,81 @@ Notifications ============= -LocalDevStack can optionally emit notifications from containers to the host (non-Windows). +LocalDevStack can forward notification events emitted inside the trusted Tools container +to the host desktop or terminal. -The idea is simple: +Host Watcher +------------ -- Your host runs a watcher that listens for notification events. -- Containers fire-and-forget messages using a tiny client binary (``docknotify``). -- The watcher turns those events into desktop notifications (toast / notify-send / etc.). +Start the watcher:: -Host usage (non-Windows) ------------------------- + lds notify watch -Start watching (recommended during development):: +or through the grouped support command:: - lds notify watch + lds support notify watch -Send a one-off test notification:: +By default the watcher follows the current project's running ``server-tools`` container. - lds notify test "T" "B" +An explicit container can be supplied for advanced/debug use:: -From inside containers (docknotify) ------------------------------------ + lds notify watch -Inside LocalDevStack containers, trigger a notification by calling ``docknotify``:: +Test the path:: - docknotify -t 2500 -u normal some_title some_body >/dev/null 2>&1 & + lds notify test "LocalDevStack" "Notification channel works" -Options: +Host Behavior +------------- -- ``-t``: timeout in milliseconds (example: ``2500``) -- ``-u``: urgency (example: ``low``, ``normal``, ``critical``) -- The final two arguments are: ``title`` and ``body`` +Linux / WSLg + Uses ``notify-send`` when available. -The redirection + ``&`` makes it fire-and-forget so it never blocks your request or job. +Windows/Git Bash or compatible WSL environment + Uses ``powershell.exe`` to create a Windows toast. -Common pattern --------------- +Other environments + Falls back to a timestamped terminal message when no supported desktop notifier is + available. -- You keep ``lds notify watch`` running on the host. -- Your apps/services inside containers call ``docknotify`` when something noteworthy happens - (errors, deploy events, background jobs, long tasks, etc.). +The watcher reconnects if the Docker log stream ends while the container remains +running. It exits when the watched container stops. -Message format --------------- +Container-side Events +--------------------- -Notifications are transmitted as a single-line payload (safe for log streaming and easy parsing). -Implementations typically use a tab-separated payload like: +Tools-side notification helpers emit a line beginning with:: -- token -- timeout -- urgency -- source -- title -- body + __HOST_NOTIFY__ -This is intentionally simple: it survives log streaming and is easy to parse reliably. +The host watcher consumes those log events and supports timeout, urgency, title, and +body fields. -PHP example: forward all PHP errors to notifications ----------------------------------------------------- +Normal urgency values are:: -Below is a minimal helper you can drop into any PHP project to log everything to a file and optionally -emit desktop notifications via ``docknotify`` (when running inside LocalDevStack containers). + low + normal + critical -.. code-block:: php +Application Helpers +------------------- - /dev/null 2>&1 & - $notifyFn = static function (string $title, string $body) use ($notify): void { - if (!$notify) { - return; - } +Applications should treat notification delivery as optional observability, not as a +critical execution dependency. - // Require docknotify in PATH - $bin = \trim((string)@\shell_exec('command -v docknotify 2>/dev/null')); - if ($bin === '') { - return; - } +Operational Notes +----------------- - // Keep short + safe; remove newlines/tabs to keep one-line protocol stable - $title = (string)(\preg_replace('/\s+/', ' ', $title) ?? 'PHP Error'); - $body = (string)(\preg_replace('/\s+/', ' ', $body) ?? ''); - - $title = \substr($title, 0, 80); - $body = \substr($body, 0, 220); - - // Escape args (no injection) - $t = \escapeshellarg($title); - $b = \escapeshellarg($body); - - // Send as "normal" urgency, 2500ms timeout; fire-and-forget - @\shell_exec($bin . ' -t 2500 -u normal ' . $t . ' ' . $b . ' >/dev/null 2>&1 &'); - }; - - $map = [ - E_ERROR => 'E_ERROR', - E_WARNING => 'E_WARNING', - E_PARSE => 'E_PARSE', - E_NOTICE => 'E_NOTICE', - E_CORE_ERROR => 'E_CORE_ERROR', - E_CORE_WARNING => 'E_CORE_WARNING', - E_COMPILE_ERROR => 'E_COMPILE_ERROR', - E_COMPILE_WARNING => 'E_COMPILE_WARNING', - E_USER_ERROR => 'E_USER_ERROR', - E_USER_WARNING => 'E_USER_WARNING', - E_USER_NOTICE => 'E_USER_NOTICE', - E_RECOVERABLE_ERROR => 'E_RECOVERABLE_ERROR', - E_DEPRECATED => 'E_DEPRECATED', - E_USER_DEPRECATED => 'E_USER_DEPRECATED', - ]; - - // Log non-fatal errors (warnings/notices/deprecations, etc.) - \set_error_handler( - static function (int $severity, string $message, string $file, int $line) use ($logFile, $map, $notifyFn): bool { - // Respect @ suppression - if (!(error_reporting() & $severity)) { - return true; - } - - $label = $map[$severity] ?? ('E_' . (string)$severity); - $ts = \date('Y-m-d H:i:s'); - - @\file_put_contents( - $logFile, - $ts . ' [' . $label . '] ' . $message . ' in ' . $file . ':' . $line . PHP_EOL, - FILE_APPEND | LOCK_EX - ); - - $notifyFn($label, $message . ' (' . \basename($file) . ':' . $line . ')'); - - // We handled it; do not let PHP print/log elsewhere - return true; - } - ); - - // Log uncaught exceptions / TypeErrors, etc. - \set_exception_handler( - static function (\Throwable $e) use ($logFile, $notifyFn): void { - $ts = \date('Y-m-d H:i:s'); - $type = \get_class($e); - - $msg = $ts - . ' [UNCAUGHT ' . $type . '] ' - . $e->getMessage() - . ' in ' . $e->getFile() . ':' . $e->getLine() - . PHP_EOL - . $e->getTraceAsString() - . PHP_EOL; - - @\file_put_contents($logFile, $msg . PHP_EOL, FILE_APPEND | LOCK_EX); - - $notifyFn( - 'UNCAUGHT ' . $type, - $e->getMessage() . ' (' . \basename($e->getFile()) . ':' . $e->getLine() . ')' - ); - - exit(255); - } - ); - - // Log fatal errors (E_ERROR, E_PARSE, E_COMPILE_ERROR, etc.) - \register_shutdown_function( - static function () use ($logFile, $notifyFn, $map): void { - $err = \error_get_last(); - if ($err === null) { - return; - } - - $fatalTypes = [E_ERROR, E_PARSE, E_CORE_ERROR, E_COMPILE_ERROR, E_USER_ERROR]; - $type = (int)($err['type'] ?? 0); - - if (!\in_array($type, $fatalTypes, true)) { - return; - } - - $label = $map[$type] ?? ('E_' . (string)$type); - $ts = \date('Y-m-d H:i:s'); - - $message = (string)($err['message'] ?? ''); - $file = (string)($err['file'] ?? ''); - $line = (int)($err['line'] ?? 0); - - @\file_put_contents( - $logFile, - $ts . ' [' . $label . '] ' . $message . ' in ' . $file . ':' . $line . PHP_EOL, - FILE_APPEND | LOCK_EX - ); - - $notifyFn($label, $message . ' (' . \basename($file) . ':' . $line . ')'); - } - ); - - // Ensure nothing is printed to screen by PHP itself - \ini_set('display_errors', '0'); - \ini_set('log_errors', '0'); - } - - // Usage: - $log = __DIR__ . '/php-upg-err-' . \date('Ymd') . '.log'; - registerAllErrorsToFile($log, true); - -Node.js example: send notifications using docknotify ----------------------------------------------------- - -LocalDevStack ships ``docknotify`` inside Node containers too, so Node apps can emit host notifications -without extra dependencies. - -Below is a small helper that: - -- checks ``docknotify`` exists -- strips newlines/tabs (keeps the one-line protocol stable) -- sends a fire-and-forget notification (non-blocking) - -.. code-block:: js - - // docknotify.js - const { spawnSync, spawn } = require("node:child_process"); - - // Cache existence check so we don't run it per request - const HAS_DOCKNOTIFY = (() => { - const r = spawnSync("sh", ["-lc", "command -v docknotify >/dev/null 2>&1"], { stdio: "ignore" }); - return r.status === 0; - })(); - - function clean(s, max) { - return String(s ?? "") - .replace(/[\t\r\n]+/g, " ") - .replace(/\s+/g, " ") - .trim() - .slice(0, max); - } - - function notify(title, body, { timeout = 2500, urgency = "normal" } = {}) { - if (!HAS_DOCKNOTIFY) return; - - const t = clean(title, 80) || "Node"; - const b = clean(body, 220); - - // Fire-and-forget: no stdout/stderr, detached, unref - const child = spawn( - "docknotify", - ["-t", String(timeout), "-u", urgency, t, b], - { stdio: "ignore", detached: true } - ); - - child.on("error", () => {}); - child.unref(); - } - - module.exports = { notify }; - -.. code-block:: js - - // example usage (Express) - const express = require("express"); - const { notify } = require("./docknotify"); - - const app = express(); - - process.on("unhandledRejection", (err) => { - notify("Unhandled Rejection", err?.stack || String(err)); - }); - - process.on("uncaughtException", (err) => { - notify("Uncaught Exception", err?.stack || String(err), { urgency: "critical" }); - // process.exit(1); - }); - - app.get("/", (req, res) => res.json({ ok: true })); - - // test route - app.get("/boom", () => { - throw new Error("Test error from /boom"); - }); - - // eslint-disable-next-line no-unused-vars - app.use((err, req, res, next) => { - notify("Express Error", `${err.message} (${req.method} ${req.originalUrl})`); - res.status(500).json({ error: "Internal Server Error" }); - }); - - const port = process.env.PORT || 3000; - app.listen(port, () => { - notify("Node Started", `Listening on :${port}`, { timeout: 1500, urgency: "low" }); - }); - -Quick test -~~~~~~~~~~ - -1. On host, start watcher:: - - lds notify watch - -2. Trigger the test route:: - - curl -sS http://your-domain.localhost/boom >/dev/null - -Practical workflow ------------------- - -1. On your host, keep this running in a terminal:: - - lds notify watch - -2. In your PHP/Node apps (inside containers), trigger ``docknotify`` on important events - (errors, failed jobs, timeouts, etc.). - -This gives you immediate feedback without tailing logs all day. +- The watcher requires the target container to be running. +- Docker remains a host-side requirement. +- Desktop delivery depends on the host session and notifier availability. +- Notification payloads travel through container logs; avoid placing credentials, + tokens, or other sensitive values in titles/bodies. +- Ctrl+C stops the watcher and returns the normal interrupted exit status. diff --git a/docs/guides/operations-and-support.rst b/docs/guides/operations-and-support.rst new file mode 100644 index 00000000..24146f5d --- /dev/null +++ b/docs/guides/operations-and-support.rst @@ -0,0 +1,249 @@ +Operations and Support +====================== + +This guide covers routine stack operations, upgrades, diagnostics, support bundles, and +cleanup boundaries. + +Stack Lifecycle +--------------- + +Start in the foreground/default Compose mode:: + + lds up + +Start detached:: + + lds start + +Stop/remove Compose containers while preserving named volumes:: + + lds down + +Restart the entire stack:: + + lds restart + +Restart only selected services:: + + lds restart nginx + lds restart nginx server-tools + +A service-specific restart uses Compose ``restart`` and does not perform the full +stop/start path. + +Status and Logs +--------------- + +:: + + lds status + lds ps + lds logs + lds logs nginx + lds logs nginx --follow + lds logs nginx --since 10m + lds logs nginx --grep project.localhost + +``lds status`` delegates the richer status view to Tools. ``lds ps`` uses Compose +directly. + +Execute a command in a service:: + + lds stack exec nginx nginx -t + +Open an interactive shell when no command is supplied:: + + lds stack exec nginx + +Stack Diff +---------- + +Compare desired Compose images with running container images:: + + lds stack diff + +Show the effective config alongside the diff:: + + lds stack diff --config + +Machine-readable diff output requires ``jq``:: + + lds stack diff --json + +Rebuild and Refresh +------------------- + +A normal restart does not intentionally pull newer moving tags. + +Refresh/recreate selected services with:: + + lds rebuild nginx + lds rebuild nginx runner + +With no arguments, ``lds rebuild`` presents an interactive service selector. +``lds rebuild all`` resolves all active Compose services. + +For locally built runtime services, normal Docker build cache is preserved while +``--pull`` refreshes the selected base. + +For published images, LocalDevStack removes the selected local image, pulls the resolved +service image, recreates it, and then reboots the effective stack. + +Legacy Network Migration +------------------------ + +Older LocalDevStack versions used fixed bridge subnets. + +``lds up`` and ``lds start`` automatically check the known legacy networks before +starting. Migration only proceeds when subnet and ownership checks prove that the +network belongs to the current LocalDevStack project. + +The migration uses Compose ``down --remove-orphans`` without ``-v`` and preserves named +volumes. + +The old command:: + + lds vpn-fix + +is deprecated and no longer changes host routes/subnets. + +Diagnostics +----------- + +Run the main non-destructive health check:: + + lds doctor + +Useful targeted probes:: + + lds diag dns project.localhost + lds diag net + lds diag tcp postgres 5432 + lds diag http https://project.localhost + lds diag tls project.localhost + +``lds sniff `` is an alias for the HTTP diagnostic path. + +Domain Trace +------------ + +Run an end-to-end domain trace:: + + lds support trace project.localhost + +The trace covers: + +- DNS; +- TLS certificate details; +- HTTP timing/status; +- persisted Nginx upstream inference; +- recent Nginx logs. + +Support Bundles +--------------- + +Create a shareable redacted ZIP in the current directory:: + + lds support bundle + +or choose an explicit path:: + + lds support bundle --redact ./localdevstack-support.zip + +The default bundle includes effective Compose configuration, Compose/container state, +networks, recent logs, generated Nginx/Apache vhosts where available, redacted env +files, and Tools-side network diagnostics. + +Redaction is best-effort and is designed to remove common password/secret/token/API-key +fields and secret-bearing connection strings. + +For the intentionally raw form:: + + lds support bundle --full + +``--full`` also includes scoped Docker inspect output and does not apply normal +redaction. Treat it as sensitive data. + +The host ``zip`` command is required for bundle creation. + +Events +------ + +Follow Docker events for the effective LocalDevStack Compose project:: + + lds events + +The default ``--since`` value is ``1h``. A custom value can be supplied:: + + lds events 10m + +Project Identity +---------------- + +The default Compose project identity is ``LocalDevStack``. + +An explicit ``COMPOSE_PROJECT_NAME`` override is respected by project-scoped +diagnostics, events, bundles, and cleanup. + +Cleanup +------- + +Scoped cleanup requires explicit confirmation:: + + lds clean --yes + +It targets stopped LocalDevStack project containers, unused labelled LocalDevStack +networks, and generated ``localdevstack-php:*`` / ``localdevstack-node:*`` images. + +Attempt LocalDevStack-labelled volume removal too:: + + lds clean --yes --volumes + +In-use volumes are left to Docker's normal safety rules. + +Host-wide Docker prune is deliberately separate:: + + lds clean --global --yes + +With ``--global``, unrelated stopped containers, unused networks/images, and build +cache can be removed. Add ``--volumes`` only when host-wide unused-volume pruning is +also intended. + +Destructive Compose Down +------------------------ + +Normal:: + + lds down + +preserves persistent data. + +This requires explicit confirmation and removes Compose volumes:: + + lds down --volumes --yes + +Do not use that destructive form for routine upgrades. + +Config Validation +----------------- + +:: + + lds config validate + +This validates the effective Compose graph and checks scheduler files for CRLF. If Runner +is already running, LocalDevStack also asks Supervisor to validate its mounted +configuration. + +Generated Compose fragments under ``configuration/compose/`` are listed when present. + +Tools UI +-------- + +Open the terminal Tools UI (lazydocker) inside the project control plane:: + + lds support ui + +or use the shortcut:: + + lds ui diff --git a/docs/guides/secrets-sops-age.rst b/docs/guides/secrets-sops-age.rst index bb137304..377b9edd 100644 --- a/docs/guides/secrets-sops-age.rst +++ b/docs/guides/secrets-sops-age.rst @@ -1,22 +1,74 @@ Secrets with SOPS and Age ========================= -LocalDevStack can integrate an encrypted secrets workflow using **SOPS** and **Age**. +LocalDevStack provides the host/storage wiring for a SOPS/Age-based encrypted secrets +workflow through the trusted Tools control plane. -The goal is simple: +The goal is to keep plaintext secrets out of Git while making deliberate local +decryption available to development workflows. -- Keep ``.env``-like secrets encrypted in Git -- Decrypt only when needed (locally) into runtime containers or build steps +Host Layout +----------- -Typical workflow (high level) ------------------------------ +LocalDevStack mounts these host paths into ``server-tools``: -1. Store secrets as ``*.enc.env`` (or similar) in a repo. -2. Keep Age private keys outside the repo (mounted into Tools container). -3. Use a helper (often called ``senv``) to decrypt into a target env file. +``configuration/sops/config/`` + SOPS configuration. -Safety notes ------------- +``configuration/sops/global/`` + Shared encrypted/global secret material. -- Prefer read-only mounts for secrets repos. -- Do not bake private keys into images. +``configuration/sops/keys/`` + Sensitive Age/key material. Unix setup permissions restrict this directory and its + files. + +An optional external secrets repository can be mounted through ``SOP_REPO`` and is +available to Tools under its SOPS/vhost integration path. + +CLI Access +---------- + +The LocalDevStack command:: + + lds secrets + +delegates directly to the Tools ``senv`` command inside the running ``server-tools`` +container. + +Use:: + + lds tools sh + +when you need to inspect the trusted Tools environment interactively before running a +manual SOPS/Age operation. + +Trust Boundary +-------------- + +SOPS/Age processing belongs in the trusted Tools control plane, not in ordinary +database, web, or AI provider containers. + +``server-tools`` already has powerful Docker access, so access to its shell and mounted +secret material should be treated as privileged workstation access. + +Safety Guidelines +----------------- + +- Keep Age private keys out of application repositories. +- Do not bake private keys into Docker images. +- Keep ``configuration/sops/keys/`` private. +- Prefer encrypted files in Git and decrypt only for the required local workflow. +- Review permissions after moving/copying key material. +- Do not place secret-bearing raw output into support bundles unless deliberately using + and protecting ``--full`` output. + +Unix Permissions +---------------- + +``lds setup permissions`` applies: + +- ``0700`` to directories under ``configuration/sops/keys/``; +- ``0600`` to files under ``configuration/sops/keys/``. + +This deliberately overrides the broader group-writable configuration-directory policy +for key material. diff --git a/docs/guides/tls-and-certificates.rst b/docs/guides/tls-and-certificates.rst index e1d3e4e2..35ce58c6 100644 --- a/docs/guides/tls-and-certificates.rst +++ b/docs/guides/tls-and-certificates.rst @@ -2,26 +2,27 @@ TLS and Certificates ==================== LocalDevStack uses mkcert-based local TLS for development. Tools owns certificate -generation; Nginx, Apache, Mailpit, and runtime consumers receive only the mounts needed -by the LocalDevStack runtime contract. +generation; the host ``lds`` CLI owns trust-store installation/removal. Runtime TLS State ----------------- The active runtime state is stored in Docker named volumes: -SSLRootCA - mkcert CA store exposed to trusted LocalDevStack consumers at /etc/share/rootCA. +``SSLRootCA`` + mkcert CA store exposed to trusted LocalDevStack consumers at + ``/etc/share/rootCA``. -SSLKeys - generated server/client certificate material exposed at /etc/mkcert where needed. +``SSLKeys`` + Generated server/client certificate material exposed at ``/etc/mkcert`` where + needed. -Tools automatically refreshes certificates through certify. +Tools refreshes certificates through its ``certify`` command family. Public Host Export ------------------ -Tools exports user-facing certificate artifacts to the host-mounted directory:: +Tools exports user-facing certificate artifacts to:: configuration/ssl/ @@ -29,88 +30,128 @@ The public root CA is:: configuration/ssl/rootCA.pem -lds certificate install uses this path. For upgrades, the legacy -configuration/rootCA/rootCA.pem path remains a read fallback. +For upgrades, the legacy path remains a read fallback:: -The public CA export is safe to install into the host trust store. The private CA key -is not intended as a user-facing export. + configuration/rootCA/rootCA.pem -Optional user mTLS export is disabled by default. When explicitly enabled in Tools, -a password-protected user P12 is exported under configuration/ssl/. +The public CA certificate is safe to install into the host trust store. The private CA +key is not intended as a public export. + +Optional password-protected user mTLS artifacts may also be exported under +``configuration/ssl/``. Certificate Coverage -------------------- -The generated certificate SAN set includes at least:: +The generated SAN set includes at least:: localhost *.localhost 127.0.0.1 ::1 -Tools also discovers generated vhost/service domains. The \*.localhost entry covers -built-in convenience endpoints such as: +Tools can also discover generated vhost/service domains. + +The wildcard covers built-in endpoints such as: -- admin.localhost; -- webmail.localhost; -- db.localhost; -- ri.localhost; -- me.localhost; -- kibana.localhost; -- llm.localhost. +- ``admin.localhost``; +- ``webmail.localhost``; +- ``db.localhost``; +- ``ri.localhost``; +- ``me.localhost``; +- ``kibana.localhost``; +- ``llm.localhost``. -Installing the Root CA ----------------------- +Install the Root CA +------------------- -Linux/macOS where supported:: +Linux:: sudo lds certificate install -Windows through Git Bash/lds.bat uses the CurrentUser root certificate store and does -not require the Unix sudo path. +The installer detects common Linux families and uses their normal trust-store location/ +refresh mechanism where available: + +- Debian/Ubuntu-style ``update-ca-certificates``; +- RHEL/Fedora-style ``update-ca-trust``; +- Arch-style p11-kit/trust handling. + +When ``certutil`` is available, the invoking user's NSS database is also updated on +supported Unix flows. + +Windows/Git Bash:: + + lds.bat certificate install -After installation, restart browsers that cache trust decisions. +Windows imports the CA into ``CurrentUser\\Root`` through PowerShell. + +macOS + LocalDevStack can run through Docker Desktop, but automatic macOS Keychain import is + not currently implemented by the host installer. Trust + ``configuration/ssl/rootCA.pem`` manually in Keychain when required. + +Restart browsers that cache trust results after changing the CA. Uninstalling ------------ -Remove the LocalDevStack trust anchor:: +Linux:: sudo lds certificate uninstall -Scan/remove known legacy anchor locations too:: +Scan/remove all known LocalDevStack anchor locations too:: sudo lds certificate uninstall --all -This removes the host trust-store entry. It does not destroy the LocalDevStack named -certificate volumes. +Windows/Git Bash:: + + lds.bat certificate uninstall + +The uninstall operation removes the host trust anchor. It does not delete the +``SSLRootCA`` / ``SSLKeys`` Docker volumes. + +Tools Certificate Commands +-------------------------- + +The Tools certificate workflow is exposed separately through:: + + lds cert status + lds cert regen all + lds cert diagnose project.localhost + +``lds certificate ...`` manages host trust; ``lds cert ...`` delegates certificate +generation/status/diagnostics to Tools. Mutual TLS ---------- When a domain enables mutual TLS, browser/user certificate material must be imported -separately. User-facing P12 export is intentionally opt-in and password-protected. +separately. User-facing P12/PFX exports are intentionally opt-in and password-protected. -Internal Nginx-to-Apache mTLS material remains runtime state and is not the same as the -user-facing client certificate. +Internal Nginx-to-Apache mTLS material remains runtime state and is not the same as a +user-facing browser certificate. + +Permissions +----------- + +``lds setup permissions`` keeps public CA exports readable while applying restrictive +permissions to exported private-key-style files (including P12/PFX/key artifacts). Troubleshooting --------------- -Check LocalDevStack health and TLS readiness:: +Check TLS readiness:: lds doctor -Inspect a domain TLS handshake:: +Inspect a domain handshake:: lds diag tls project.localhost -Regenerate/diagnose through the existing certificate commands:: +Run the end-to-end trace:: - lds cert status - lds cert regen all - lds cert diagnose project.localhost + lds support trace project.localhost -If the host export is missing, ensure server-tools is running and certificate -generation has completed. The current public export should appear at -configuration/ssl/rootCA.pem. +If the public export is missing, ensure ``server-tools`` is running and certificate +generation has completed. The current export should appear at +``configuration/ssl/rootCA.pem``. diff --git a/docs/index.rst b/docs/index.rst index 3b188267..f496b401 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -2,8 +2,11 @@ LocalDevStack Documentation =========================== LocalDevStack is a Docker-based XAMPP alternative for PHP and Node.js local development. -It provides profile-driven services, local domains and TLS, selectable runtimes, -background jobs, admin UIs, and optional local AI behind the lds CLI. +It combines local domains, trusted TLS, selectable runtimes, databases and admin tools, +background jobs, developer utilities, diagnostics, and optional local AI behind the +``lds`` CLI. + +LocalDevStack is designed for trusted developer workstations, not production deployment. .. toctree:: :maxdepth: 2 @@ -24,7 +27,16 @@ background jobs, admin UIs, and optional local AI behind the lds CLI. :caption: Guides guides/domain-setup + guides/databases-and-clients guides/tls-and-certificates guides/local-ai + guides/operations-and-support + guides/ad-hoc-runner guides/secrets-sops-age guides/notifications + +.. toctree:: + :maxdepth: 2 + :caption: Reference + + reference/cli diff --git a/docs/quickstart.rst b/docs/quickstart.rst index cf71bae9..60022cde 100644 --- a/docs/quickstart.rst +++ b/docs/quickstart.rst @@ -2,105 +2,167 @@ Getting Started =============== LocalDevStack is a Docker-based XAMPP alternative for PHP and Node.js local development. -The lds CLI manages Compose profiles, local domains, TLS, runtime builds, databases, -admin tools, background jobs, and optional local AI. +The ``lds`` CLI manages Compose profiles, local domains, TLS, runtime builds, databases, +admin tools, background jobs, diagnostics, and optional local AI. Prerequisites ------------- Install Docker first. -- Docker Engine is preferred on Linux. -- Docker Desktop is supported on Windows and macOS. -- Windows CLI access uses lds.bat with Git Bash. +Linux + Docker Engine is preferred. Docker Desktop is also usable. -Quick Start ------------ +Windows + Docker Desktop plus Git Bash. ``lds.bat`` bridges Windows invocation into the Bash CLI. -Clone the repository and enter it:: +macOS + Docker Desktop. + +Docker is always a host-side requirement. Some developer utilities such as ``jq``, +``yq``, ``rg``, ``fd``, ``tree``, and ``shellcheck`` can be proxied through a running +``server-tools`` container when they are not installed on the host. + +Recommended Layout +------------------ + +A common layout is:: + + project-root/ + ├─ application/ + │ ├─ site1/ + │ ├─ site2/ + │ └─ ... + └─ LocalDevStack/ + +The default project mount points at the sibling ``application/`` directory. Set +``PROJECT_DIR`` in ``docker/.env`` when the application directory lives elsewhere. + +First-Time Setup +---------------- + +Clone the repository:: git clone https://github.com/infocyph/LocalDevStack.git cd LocalDevStack -On Linux/macOS, prepare the CLI and host permissions:: +On Linux/macOS, prepare permissions and the ``lds`` symlink:: chmod +x ./lds sudo ./lds setup permissions -Then initialize LocalDevStack and choose optional services:: +Initialize workstation defaults:: ./lds setup init + +Choose optional service profiles:: + ./lds setup profile +The selector covers PostgreSQL, MySQL, MariaDB, MongoDB, Redis, Elasticsearch, and Local +AI. Re-running it replaces those catalog-managed choices while preserving generated +domain/runtime profiles. + Start the stack:: ./lds up -Create your first local domain:: +The core stack includes ``server-tools``, Runner, Mailpit, Nginx, and Apache. Apache is +always available so a domain can choose it as its backend when needed. + +Create the First Domain +----------------------- + +The domain wizard requires the control plane to be running:: ./lds setup domain -The domain wizard asks for the application type and runtime version. PHP and Node -selection remains version-specific; the selected value becomes the runtime image identity. +The wizard asks for application type, runtime version and routing details. PHP and Node +version selection remains explicit. + +List generated domains and validate the graph:: -For browser-trusted HTTPS, install the exported LocalDevStack root CA:: + lds domain ls + lds config validate + +Trusted HTTPS +------------- + +Once Tools has exported the LocalDevStack root CA, install it on the host. + +Linux:: sudo ./lds certificate install -On Windows, run the same workflow through lds.bat or Git Bash. The permissions -command configures the wrapper path and does not apply Unix chmod logic. +Windows/Git Bash:: + + lds.bat certificate install + +Windows installs into ``CurrentUser\\Root``. Linux handles supported distro trust stores +and can also update the invoking user's NSS database when ``certutil`` is available. + +macOS users may need to trust ``configuration/ssl/rootCA.pem`` manually in Keychain; +automatic Keychain import is not currently part of the host installer. Useful First Checks ------------------- -Show the active convenience URLs:: +:: + lds doctor lds urls - -Show effective infrastructure/runtime image defaults:: - lds images + lds config show + lds config validate + lds status + lds ps -Run non-destructive diagnostics:: +``lds config show`` is redacted by default. Use ``--raw`` only when unredacted output is +deliberately required. - lds doctor +Updating an Existing Installation +--------------------------------- -Validate the effective Compose and scheduler configuration:: +On ``lds up`` or ``lds start``, known historical fixed LocalDevStack networks are safely +migrated to dynamic bridge networks when ownership can be proven. Named volumes are +preserved. +After an upgrade:: + + lds doctor lds config validate -Show effective Compose configuration with secret values redacted:: +Refresh/recreate a selected service with:: - lds config show + lds rebuild nginx -Use lds config show --raw only when you intentionally need the unredacted output. +or all resolved services with:: -Project Layout --------------- + lds rebuild all -A common layout is:: +A normal ``lds restart`` does not intentionally pull newer moving image tags. - project-root/ - ├─ application/ - │ ├─ site1/ - │ ├─ site2/ - │ └─ ... - └─ LocalDevStack/ +Safety Notes +------------ + +``lds support bundle`` is redacted by default; ``--full`` is intentionally raw. -Set PROJECT_DIR in docker/.env when your application directory is elsewhere. +``lds clean --yes`` is LocalDevStack-scoped. ``lds clean --global --yes`` performs +host-wide Docker pruning and can affect unrelated projects. -LocalDevStack keeps host-managed state under configuration/ and logs/ while -runtime vhosts, certificates, databases, and other service data primarily live in -named Docker volumes. +``lds down --volumes --yes`` removes persistent Compose data and should not be part of a +normal update. Next Steps ---------- -- Profiles and environment: concepts/profiles-and-env -- Architecture: concepts/architecture -- Storage: concepts/storage-layout -- Domain setup: guides/domain-setup -- TLS: guides/tls-and-certificates -- Local AI: guides/local-ai -- Encrypted secrets: guides/secrets-sops-age -- Notifications: guides/notifications +- Architecture: :doc:`concepts/architecture` +- Profiles/environment: :doc:`concepts/profiles-and-env` +- Storage: :doc:`concepts/storage-layout` +- Domains: :doc:`guides/domain-setup` +- Databases/clients: :doc:`guides/databases-and-clients` +- TLS: :doc:`guides/tls-and-certificates` +- Local AI: :doc:`guides/local-ai` +- Operations/support: :doc:`guides/operations-and-support` +- Ad-hoc runner: :doc:`guides/ad-hoc-runner` +- CLI reference: :doc:`reference/cli` diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst new file mode 100644 index 00000000..dd3c68aa --- /dev/null +++ b/docs/reference/cli.rst @@ -0,0 +1,339 @@ +CLI Reference +============= + +``lds`` is the canonical LocalDevStack host CLI. ``lds.bat`` bridges Windows/Git Bash +invocation into the same command surface. + +Global Options +-------------- + +``-v`` / ``--verbose`` + Enable verbose command/error output. + +``-q`` / ``--quiet`` + Suppress non-error LocalDevStack output where supported. + +``--reload-extras`` + Force a rescan of ``configuration/compose/*.yaml`` / ``*.yml`` before the command. + +``-h`` / ``--help`` + Show help. + +The machine-copyable Markdown command summary is:: + + lds help --markdown + +Stack +----- + +:: + + lds stack up + lds stack start + lds stack down [--volumes --yes] + lds stack restart [service...] + lds stack status [status-args...] + lds stack ps + lds stack logs [service] [--follow] [--since ] [--grep ] + lds stack exec [command...] + lds stack events [since] + lds stack clean --yes [--volumes] [--global] + lds stack diff [--config] [--json] + lds stack config + lds stack http reload + +Common top-level aliases are:: + + lds up + lds start + lds down + lds stop + lds restart + lds reboot + lds status + lds ps + lds logs + lds exec + lds events + lds clean + lds config + +Domains +------- + +:: + + lds domain add + lds domain rm [args...] + lds domain ls + +Legacy aliases:: + + lds host add + lds host rm + lds host list + +Setup +----- + +:: + + lds setup init + lds setup permissions + lds setup domain + lds setup profile + lds setup profiles + +Profiles +-------- + +:: + + lds profiles list + lds profiles add + lds profiles remove + +Configuration +------------- + +:: + + lds config show [--json] [--raw] + lds config services + lds config profiles + lds config env-used + lds config validate + lds images + lds urls + +``config show`` is redacted by default. + +Certificates +------------ + +Tools certificate operations:: + + lds cert status [domain|all] + lds cert regen [domain|all] [--yes] + lds cert diagnose + +Host trust-store operations:: + + lds certificate install + lds certificate uninstall [--all] + +Diagnostics +----------- + +:: + + lds doctor + lds diag dns + lds diag net + lds diag tcp + lds diag http [curl-args...] + lds diag tls + lds sniff [curl-args...] + +``sniff`` is the HTTP diagnostic shortcut. + +Support +------- + +:: + + lds support open + lds support trace + lds support bundle [--redact|--full] [output.zip] + lds support notify ... + lds support ui + +Shortcuts:: + + lds open ... + lds bundle ... + lds notify ... + lds ui + +Tools Control Plane +------------------- + +:: + + lds tools sh + lds tools exec "" + lds tools file + +Open a generic container shell or run a command:: + + lds cli + lds cli + +Resolve a domain/container to its application shell:: + + lds core [domain|container] + +Secrets +------- + +:: + + lds secrets + +This delegates to Tools ``senv``. + +AI Consumer +----------- + +:: + + lds ai status + lds ai ask ... + lds ai explain ... + lds ai troubleshoot ... + lds ai review ... + lds ai repo-review ... + lds ai graphify ... + +``status`` maps to the Tools provider-status flow. + +LLM Provider +------------ + +:: + + lds llm models + lds llm ps + lds llm show ... + lds llm pull ... + lds llm rm ... + lds llm unload ... + lds llm run ... + lds llm ask ... + lds llm chat ... + lds llm prompt ... + lds llm code ... + lds llm review ... + lds llm json ... + lds llm ai-commit ... + lds llm ollama ... + lds llm api ... + lds llm version + +Runtime selection:: + + lds llm runtime + lds llm runtime + +Direct host-port control:: + + lds llm host-port status + lds llm host-port on + lds llm host-port off + +Rebuild +------- + +:: + + lds rebuild + lds rebuild all + lds rebuild + +No-argument rebuild uses an interactive selector. + +Ad-hoc Dockerfile Runner +------------------------ + +:: + + lds run + lds run shell + lds run ps + lds run logs + lds run stop + lds run rm + lds run open + +Useful flags include:: + + --name + --tag + --no-build + --no-keepalive + --sock + --host-os + --publish + -p + --mount + --port + --path + --http + --https + +See :doc:`../guides/ad-hoc-runner`. + +Runtime and Client Wrappers +--------------------------- + +PHP/Node:: + + lds php ... + lds composer ... + lds node ... + lds npm ... + lds npx ... + +PostgreSQL:: + + lds pg ... + lds psql ... + lds pg_dump ... + lds pg_restore ... + +MySQL:: + + lds my ... + lds mysql ... + lds mysqldump ... + +MariaDB:: + + lds maria ... + lds mariadb ... + lds mariadb-dump ... + +Redis:: + + lds redis ... + lds redis-cli ... + +MongoDB:: + + lds mongo ... + lds mongodb ... + lds mongosh ... + lds mongoimport ... + lds mongoexport ... + +Elasticsearch:: + + lds es ... + lds elastic ... + lds elasticsearch ... + +Unknown Command Fallback +------------------------ + +If a command is not implemented by ``lds`` itself, LocalDevStack delegates it to +``bin/tool-runner``. This preserves the existing Tools/Toolset command extension surface. + +Deprecated +---------- + +:: + + lds vpn-fix + +The command now explains that fixed-subnet manipulation is obsolete because networking +is dynamically assigned. diff --git a/lds b/lds index 90b057ec..026f3f34 100755 --- a/lds +++ b/lds @@ -688,49 +688,67 @@ cmd_bundle() { } cmd_help() { - if [[ "${1:-}" == "--markdown" ]]; then cat <<'MD' # LocalDevStack (lds) — Command Reference -## Stack (compose) -- `lds stack up` *(aliases: `up`)* -- `lds stack start` *(aliases: `start`)* -- `lds stack down [--volumes --yes]` *(aliases: `down`, `stop`)* -- `lds stack restart [svc]` *(aliases: `restart`, `reboot`)* -- `lds stack status [status-args…]` *(alias: `status`; forwards args to tools `status`)* -- `lds stack ps` *(alias: `ps`)* -- `lds stack logs [svc] [--follow] [--since ] [--grep ]` *(alias: `logs`)* -- `lds stack exec [cmd…]` *(alias: `exec`)* -- `lds stack events [--since ]` *(alias: `events`)* -- `lds stack clean --yes [--volumes] [--global]` *(alias: `clean`; scoped by default)* -- `lds stack diff [--config] [--json]` *(shows desired vs running images)* - - -## Domain (vhost lifecycle + routing) -- `lds domain add …` -- `lds domain rm …` +## Global options +- `-v`, `--verbose` +- `-q`, `--quiet` +- `--reload-extras` +- `-h`, `--help` + +## Stack +- `lds stack up` *(alias: `up`)* +- `lds stack start` *(alias: `start`)* +- `lds stack down [--volumes --yes]` *(aliases: `down`, `stop`)* +- `lds stack restart [service...]` *(aliases: `restart`, `reboot`)* +- `lds stack status [args...]` *(alias: `status`)* +- `lds stack ps` *(alias: `ps`)* +- `lds stack logs [service] [--follow] [--since ] [--grep ]` *(alias: `logs`)* +- `lds stack exec [cmd...]` *(alias: `exec`)* +- `lds stack events [since]` *(alias: `events`)* +- `lds stack clean --yes [--volumes] [--global]` *(alias: `clean`; scoped by default)* +- `lds stack diff [--config] [--json]` +- `lds stack config ` +- `lds stack http reload` + +## Domains +- `lds domain add` +- `lds domain rm [args...]` - `lds domain ls` +- Legacy: `lds host add|rm|list` -Legacy alias: `lds host …` → same subcommands as `domain`. +## Setup and profiles +- `lds setup init|permissions|domain|profile|profiles` +- `lds profiles list` +- `lds profiles add ` +- `lds profiles remove ` -## Certificates (TLS) +## Certificates - `lds cert status [domain|all]` - `lds cert regen [domain|all] [--yes]` - `lds cert diagnose ` - `lds certificate install` - `lds certificate uninstall [--all]` -## Diagnostics +## Diagnostics and support - `lds doctor` - `lds diag dns ` - `lds diag net` - `lds diag tcp ` -- `lds diag http ` *(alias: `sniff `)* +- `lds diag http [curl-args...]` - `lds diag tls ` +- `lds sniff [curl-args...]` +- `lds support open ` +- `lds support trace ` +- `lds support bundle [--redact|--full] [output.zip]` +- `lds support notify ...` +- `lds support ui` +- Shortcuts: `open`, `bundle`, `notify`, `ui` ## Config -- `lds config show [--json] [--raw]` *(redacted by default)* +- `lds config show [--json] [--raw]` - `lds config services` - `lds config profiles` - `lds config env-used` @@ -738,45 +756,43 @@ Legacy alias: `lds host …` → same subcommands as `domain`. - `lds images` - `lds urls` -## Support -- `lds support open ` -- `lds support trace ` -- `lds support bundle [--redact|--full]` -- `lds support notify …` -- `lds support ui` - -Shortcuts: `open`, `bundle`, `notify`, `ui` map to `support …`. - -## Secrets (senv) -- `lds secrets …` - -## Tools (project server-tools container) +## Tools / shells / secrets - `lds tools sh` - `lds tools exec ""` - `lds tools file ` +- `lds cli [cmd...]` +- `lds core [domain|container]` +- `lds secrets ` -## AI -- `lds ai status|ask|explain|troubleshoot|review|repo-review|graphify …` -- `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit …` -- `lds llm runtime ` -- `lds llm host-port ` - -## Setup -- `lds setup init|permissions|domain|profiles` +## AI consumer +- `lds ai status|ask|explain|troubleshoot|review|repo-review|graphify ...` -## Runner (ad‑hoc Dockerfile runner) -- `lds run` *(default: build+start only)* / `lds run shell` *(build+start+enter)* / `lds run *` *(same as default)* (+ `ps|logs|stop|rm|open` and flags: `--publish|-p`, `--no-keepalive`, `--mount`, `--sock`, `--host-os`, `--tag`, `--name`) - -## Other -- `lds rebuild [all|]` -- `lds core [domain]` +## LLM provider +- `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit|ollama|api|version ...` +- `lds llm runtime [cpu|nvidia|amd]` +- `lds llm host-port ` -## Short client wrappers -- `lds php|composer|node|npm|npx …` -- `lds my|mysql|mysqldump …` -- `lds maria|mariadb|mariadb-dump …` -- `lds pg|psql|pg_dump|pg_restore …` -- `lds redis|redis-cli …` +## Rebuild +- `lds rebuild` +- `lds rebuild all` +- `lds rebuild ` + +## Ad-hoc Dockerfile runner +- `lds run` +- `lds run shell|ps|logs|stop|rm|open` +- flags: `--name`, `--tag`, `--no-build`, `--no-keepalive`, `--sock`, `--host-os`, `--publish|-p`, `--mount`, `--port`, `--path`, `--http`, `--https` + +## Client/runtime wrappers +- PHP/Node: `php`, `composer`, `node`, `npm`, `npx` +- PostgreSQL: `pg`, `psql`, `pg_dump`, `pg_restore` +- MySQL: `my`, `mysql`, `mysqldump` +- MariaDB: `maria`, `mariadb`, `mariadb-dump` +- Redis: `redis`, `redis-cli` +- MongoDB: `mongo`, `mongodb`, `mongosh`, `mongoimport`, `mongoexport` +- Elasticsearch: `es`, `elastic`, `elasticsearch` + +## Deprecated +- `lds vpn-fix` — fixed-subnet manipulation is no longer required. MD return 0 fi @@ -784,69 +800,70 @@ MD cat < - clean --yes [--volumes] [--global] Alias of: stack clean (scoped by default) +${CYAN}Global:${NC} + -v|--verbose -q|--quiet --reload-extras -h|--help + +${CYAN}Stack:${NC} + stack up|start|down|restart|status|ps|logs|exec|events|clean|config|diff|http + up|start|down|stop|restart|reboot|status|ps|logs|exec|events + clean --yes [--volumes] [--global] (LocalDevStack-scoped unless --global) -${CYAN}Domain (vhosts + routing):${NC} +${CYAN}Domains:${NC} domain add|rm|ls - host add|rm|list Legacy alias of: domain <...> + host add|rm|list Legacy aliases + core [domain|container] + cli [cmd...] -${CYAN}Certificates (TLS):${NC} +${CYAN}Setup / Profiles:${NC} + setup init|permissions|domain|profile|profiles + profiles list|add|remove + +${CYAN}Certificates:${NC} cert status|regen|diagnose certificate install|uninstall [--all] -${CYAN}Diagnostics:${NC} +${CYAN}Diagnostics / Support:${NC} doctor diag dns|net|tcp|http|tls - sniff Alias of: diag http + sniff + support open|trace|bundle|notify|ui + open|bundle|notify|ui Shortcuts ${CYAN}Config:${NC} config show|services|profiles|env-used|validate images urls -${CYAN}Support:${NC} - support open - support trace - support bundle [--redact|--full] - support notify ... - support ui - open|bundle|notify|ui Shortcuts → support <...> - -${CYAN}Secrets:${NC} - secrets - -${CYAN}Tools (project server-tools container):${NC} +${CYAN}Tools / Secrets:${NC} tools sh|exec|file + secrets ${CYAN}AI:${NC} ai status|ask|explain|troubleshoot|review|repo-review|graphify - llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit - llm runtime + llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit|ollama|api|version + llm runtime [cpu|nvidia|amd] llm host-port -${CYAN}Setup:${NC} - setup init|permissions|domain|profiles - -${CYAN}Runner (ad-hoc Dockerfile runner):${NC} - run [ps|logs|stop|rm|open] [--publish|-p A:B] [--no-keepalive] [--mount HOST[:CONT]] [--sock] [--host-os VALUE] +${CYAN}Maintenance:${NC} + rebuild [all|] + run [shell|ps|logs|stop|rm|open] [runner flags...] -${CYAN}Other:${NC} - rebuild [all|] - core [domain] +${CYAN}Wrappers:${NC} + php|composer|node|npm|npx + pg|psql|pg_dump|pg_restore + my|mysql|mysqldump + maria|mariadb|mariadb-dump + redis|redis-cli + mongo|mongodb|mongosh|mongoimport|mongoexport + es|elastic|elasticsearch ${CYAN}Help:${NC} - help [--markdown] - + help + help --markdown EOF } + ############################################################################### # 7. MAIN ############################################################################### diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 1e2bc289..b3d6c0af 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -14,19 +14,37 @@ storage="$ROOT/docs/concepts/storage-layout.rst" domain="$ROOT/docs/guides/domain-setup.rst" tls="$ROOT/docs/guides/tls-and-certificates.rst" ai="$ROOT/docs/guides/local-ai.rst" +databases="$ROOT/docs/guides/databases-and-clients.rst" +ops="$ROOT/docs/guides/operations-and-support.rst" +runner="$ROOT/docs/guides/ad-hoc-runner.rst" +notify="$ROOT/docs/guides/notifications.rst" +secrets="$ROOT/docs/guides/secrets-sops-age.rst" +cli="$ROOT/docs/reference/cli.rst" -for file in "$index" "$readme" "$quick" "$arch" "$profiles" "$storage" "$domain" "$tls" "$ai"; do +for file in "$index" "$readme" "$quick" "$arch" "$profiles" "$storage" "$domain" "$tls" "$ai" "$databases" "$ops" "$runner" "$notify" "$secrets" "$cli"; do assert_file "$file" done assert_file_contains "$index" 'guides/local-ai' +assert_file_contains "$index" 'guides/databases-and-clients' +assert_file_contains "$index" 'guides/operations-and-support' +assert_file_contains "$index" 'guides/ad-hoc-runner' +assert_file_contains "$index" 'reference/cli' assert_file_contains "$readme" 'Dynamic Docker networking through service DNS' assert_file_contains "$arch" 'Docker assigns their address ranges dynamically' assert_file_contains "$domain" 'generated runtime Compose fragments are written under configuration/compose/' assert_file_contains "$domain" 'Domain listing reads the NginxHosts named volume' assert_file_contains "$storage" 'support traces, and support bundles' assert_file_contains "$readme" 'lds clean --global --yes' -pass "docs describe dynamic DNS/runtime generation and safe operational boundaries" +assert_file_contains "$arch" 'Apache' +assert_file_contains "$arch" 'COMPOSE_PROJECT_NAME' +assert_file_contains "$databases" 'CloudBeaver' +assert_file_contains "$ops" 'lds support bundle' +assert_file_contains "$ops" 'lds clean --global --yes' +assert_file_contains "$runner" 'lds run --sock' +assert_file_contains "$notify" 'Windows/Git Bash' +assert_file_contains "$cli" 'lds llm host-port' +pass "docs describe current architecture, operations, and complete user surfaces" assert_file_contains "$profiles" 'docker/release.env' assert_file_contains "$profiles" 'docker/.env' @@ -50,6 +68,8 @@ assert_file_contains "$ai" 'no project/repository bind mount' assert_file_contains "$ai" 'automatically execute model-generated shell commands' assert_file_contains "$ai" 'linux/amd64 only' assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by default' +assert_file_contains "$ai" 'infocyph/llm-sm:amd-latest' +assert_file_contains "$ai" 'LDS_AI_MODEL=qwen2.5:3b' pass "local AI trust boundary, platform and workspace limits are documented" for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-sm:0.03'; do @@ -80,3 +100,10 @@ done < <( ' "$index" ) pass "documentation toctree targets exist" + + +help_md="$("$ROOT/lds" help --markdown)" +for required in 'lds profiles add ' 'lds support trace ' 'lds support bundle [--redact|--full] [output.zip]' 'lds cli [cmd...]' 'lds run shell|ps|logs|stop|rm|open' 'MongoDB:' 'Elasticsearch:'; do + assert_contains "$help_md" "$required" +done +pass "embedded CLI help covers documented command groups" From aeeafd240fe3c21d487c6c6c6f073894d958b599 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:23:38 +0600 Subject: [PATCH 200/509] docs: fix reference contracts and Sphinx layout --- docs/concepts/storage-layout.rst | 8 +++++--- docs/guides/domain-setup.rst | 4 ++-- docs/guides/local-ai.rst | 7 +++---- lds | 6 +++++- 4 files changed, 15 insertions(+), 10 deletions(-) diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index ac2a2572..305502ad 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -50,6 +50,9 @@ Active Nginx/Apache vhosts do **not** live under host-side They are persisted in ``NginxHosts`` / ``ApacheHosts`` and are available to Tools and the web servers through their mounted paths. +Domain listing, support traces, and support bundles all read this persisted state rather +than a stale host-side vhost directory. + Accordingly: - ``lds domain ls`` reads persisted Nginx vhost state through ``server-tools``; @@ -72,8 +75,7 @@ Host Configuration ``configuration/scheduler/supervisor/`` Runner Supervisor definitions. -``configuration/sops/config/``, ``configuration/sops/global/``, -``configuration/sops/keys/`` +``configuration/sops/config/`` / ``configuration/sops/global/`` / ``configuration/sops/keys/`` SOPS/Age configuration, global secret data, and sensitive key material. ``configuration/ssh/`` @@ -89,7 +91,7 @@ Host Configuration TLS Authority ------------- -Runtime TLS state is owned by: +Runtime TLS state is owned by the SSLKeys / SSLRootCA volume pair: - ``SSLKeys`` for server/client certificate material; - ``SSLRootCA`` for the mkcert CA store. diff --git a/docs/guides/domain-setup.rst b/docs/guides/domain-setup.rst index 57a09684..6c1912bc 100644 --- a/docs/guides/domain-setup.rst +++ b/docs/guides/domain-setup.rst @@ -59,7 +59,7 @@ Active vhosts are Docker-managed state: - Apache vhosts persist in ``ApacheHosts``; - PHP-FPM pool state persists in ``FPMPools``; - PHP-FPM sockets use ``FPMSocks``; -- generated runtime Compose fragments are written under ``configuration/compose/``. +- generated runtime Compose fragments are written under configuration/compose/. There is no active host-side ``configuration/nginx`` source of truth. @@ -70,7 +70,7 @@ List persisted Nginx domains:: lds domain ls -The list is read through ``server-tools`` from the ``NginxHosts`` named volume. +Domain listing reads the NginxHosts named volume through ``server-tools``. Remove a Domain --------------- diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index da4b9db4..5898c902 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -155,8 +155,7 @@ LocalDevStack does not: Repository Context ------------------ -LocalDevStack intentionally does not mount the project/repository into ``llm-sm`` by -default. +LocalDevStack intentionally does not mount the project/repository into llm-sm by default. Repository-aware analysis should normally use the Tools consumer layer:: @@ -189,8 +188,8 @@ Diagnostics Platform Availability --------------------- -The current published ``llm-sm`` standard and AMD images are validated/published as -``linux/amd64`` only. +The current published ``llm-sm`` standard and AMD images have a linux/amd64 only +platform contract. LocalDevStack itself can still be used on arm64 with the ``ai`` profile disabled. Native arm64 local-AI support should only be advertised after the provider publishes and diff --git a/lds b/lds index 026f3f34..1d140e4b 100755 --- a/lds +++ b/lds @@ -826,7 +826,11 @@ ${CYAN}Diagnostics / Support:${NC} doctor diag dns|net|tcp|http|tls sniff - support open|trace|bundle|notify|ui + support open + support trace + support bundle [--redact|--full] [output.zip] + support notify ... + support ui open|bundle|notify|ui Shortcuts ${CYAN}Config:${NC} From a300b4c32579d3196258516e4876e7288a25ee59 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 21:59:11 +0600 Subject: [PATCH 201/509] refactor: simplify local AI runtime selection --- README.md | 14 +++-- docker/catalog/services.psv | 2 +- docker/compose/ai-amd.yaml | 1 - docker/compose/ai.yaml | 10 ---- docker/compose/companion.yaml | 10 ++++ docker/compose/main.yaml | 1 - docker/release.env | 3 +- docs/concepts/architecture.rst | 2 +- docs/concepts/profiles-and-env.rst | 7 ++- docs/guides/local-ai.rst | 32 ++++++++---- .../07-localdevstack-integration-plan.md | 25 ++++----- lib/ai.sh | 2 +- lib/certificates.sh | 9 ++++ lib/compose.sh | 11 ++-- lib/diagnostics.sh | 8 ++- lib/platform.sh | 32 ++++++++++++ lib/profiles.sh | 7 +++ tests/ai-contract.sh | 16 +++--- tests/catalog-contract.sh | 3 +- tests/compose-contract.sh | 9 +++- tests/docs-contract.sh | 3 +- tests/env-contract.sh | 52 +++++++++++++++++-- tests/published-images.sh | 12 +++-- tests/service-hardening-contract.sh | 7 ++- 24 files changed, 207 insertions(+), 71 deletions(-) delete mode 100644 docker/compose/ai.yaml diff --git a/README.md b/README.md index bbe04b30..fa65231a 100644 --- a/README.md +++ b/README.md @@ -234,7 +234,17 @@ Default model -> qwen2.5:3b Model store -> LLMModels ``` -Choose the runtime explicitly: +LocalDevStack auto-detects the preferred AI runtime during setup: NVIDIA when `nvidia-smi` is usable, AMD only when the Linux ROCm device nodes `/dev/kfd` and `/dev/dri` are present, otherwise CPU. An AMD CPU by itself does not select the AMD image. + +The provider uses one image/tag contract: + +```text +infocyph/llm-sm:${LDS_LLM_ARCH} +``` + +CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. + +Override detection explicitly when needed: ```bash lds llm runtime cpu @@ -242,8 +252,6 @@ lds llm runtime nvidia lds llm runtime amd ``` -CPU/NVIDIA use `infocyph/llm-sm:latest`; AMD uses `infocyph/llm-sm:amd-latest`. - Common commands: ```bash diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index 17d7c5d5..f307d340 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.3|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm-sm|LDS_LLM_IMAGE|LDS_AI_RUNTIME=cpu;LDS_AI_MODEL=qwen2.5:3b;LDS_LLM_HOST_PORT=no|Runtime (cpu/nvidia/amd);Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd +AI|ai|Local AI|llm-sm|LDS_LLM_ARCH|LDS_AI_MODEL=qwen2.5:3b;LDS_LLM_HOST_PORT=no|Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd diff --git a/docker/compose/ai-amd.yaml b/docker/compose/ai-amd.yaml index 9aa98d41..8b864c73 100644 --- a/docker/compose/ai-amd.yaml +++ b/docker/compose/ai-amd.yaml @@ -1,6 +1,5 @@ services: llm-sm: - image: ${LDS_LLM_AMD_IMAGE:-infocyph/llm-sm:amd-latest} devices: - /dev/kfd:/dev/kfd - /dev/dri:/dev/dri diff --git a/docker/compose/ai.yaml b/docker/compose/ai.yaml deleted file mode 100644 index c7ee9844..00000000 --- a/docker/compose/ai.yaml +++ /dev/null @@ -1,10 +0,0 @@ -services: - llm-sm: - image: ${LDS_LLM_IMAGE:-infocyph/llm-sm:latest} - restart: unless-stopped - profiles: [ai] - volumes: - - lds_llm:/root/.ollama - networks: - - frontend - - backend diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index ad31dc14..d070d137 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -54,6 +54,16 @@ services: - backend - datastore + llm-sm: + image: infocyph/llm-sm:${LDS_LLM_ARCH} + restart: unless-stopped + profiles: [ai] + volumes: + - lds_llm:/root/.ollama + networks: + - frontend + - backend + runner: container_name: RUNNER hostname: runner diff --git a/docker/compose/main.yaml b/docker/compose/main.yaml index 3ed4cbc2..7cfac84e 100644 --- a/docker/compose/main.yaml +++ b/docker/compose/main.yaml @@ -156,4 +156,3 @@ include: - docker/compose/db.yaml - docker/compose/db-client.yaml - docker/compose/http.yaml - - docker/compose/ai.yaml diff --git a/docker/release.env b/docker/release.env index 1d39cccf..bae66bfe 100644 --- a/docker/release.env +++ b/docker/release.env @@ -5,6 +5,5 @@ LDS_TOOLS_IMAGE=infocyph/tools:latest LDS_RUNNER_IMAGE=infocyph/runner:latest LDS_NGINX_IMAGE=infocyph/nginx:latest LDS_APACHE_IMAGE=infocyph/apache:latest -LDS_LLM_IMAGE=infocyph/llm-sm:latest -LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest +LDS_LLM_ARCH=latest SCRIPTOMATIC_REF=main diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index ea4e2fb3..899ff393 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -149,7 +149,7 @@ When the ``ai`` profile is enabled: 4. ``lds ai`` delegates higher-level/operational AI to Tools; 5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-sm`` CLI. -The standard CPU/NVIDIA provider and AMD/ROCm provider remain separate image variants. +The provider is one ``llm-sm`` service declared with the core companion services and enabled only by the ``ai`` profile. Its image is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. Hardware-specific Compose overlays add only NVIDIA GPU access or AMD ``/dev/kfd`` + ``/dev/dri`` device mappings. Project Identity ---------------- diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index c275abef..b989aec6 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -211,15 +211,18 @@ Important AI settings include:: LDS_AI_PROVIDER=ollama LDS_AI_URL=http://llm-sm:11434 LDS_AI_MODEL=qwen2.5:3b - LDS_AI_RUNTIME=cpu + LDS_AI_RUNTIME= + LDS_LLM_ARCH=latest LDS_LLM_HOST_PORT=0 +During setup LocalDevStack detects the preferred runtime. NVIDIA is selected only when ``nvidia-smi`` is usable; AMD is selected only when the ROCm Linux device nodes ``/dev/kfd`` and ``/dev/dri`` are present; otherwise CPU is selected. The corresponding image tag is persisted as ``LDS_LLM_ARCH`` (``latest`` for CPU/NVIDIA, ``amd-latest`` for AMD). + Use:: lds llm runtime lds llm host-port -instead of editing the corresponding runtime flags manually in normal workflows. +to override the detected runtime or host-port behavior. Tools also accepts optional timeout/context limits through ``LDS_AI_CONNECT_TIMEOUT``, ``LDS_AI_PREFLIGHT_TIMEOUT``, ``LDS_AI_TIMEOUT``, ``LDS_AI_AVAILABILITY_TTL``, diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 5898c902..60e7178b 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -27,21 +27,31 @@ persistent Ollama store. Runtime Modes ------------- -Select the provider runtime explicitly:: +LocalDevStack detects the preferred runtime during environment setup. Detection is intentionally capability-based rather than CPU-vendor based: - lds llm runtime cpu - lds llm runtime nvidia - lds llm runtime amd +- NVIDIA when ``nvidia-smi`` is usable; +- AMD only when both ``/dev/kfd`` and ``/dev/dri`` are present for ROCm; +- CPU otherwise. -CPU and NVIDIA use ``infocyph/llm-sm:latest``. AMD/ROCm uses -``infocyph/llm-sm:amd-latest``. +An AMD CPU alone does **not** select the AMD image. -NVIDIA mode adds Docker GPU access. AMD mode exposes ``/dev/kfd`` and ``/dev/dri``. +The single provider service uses:: -LocalDevStack does not attempt unreliable automatic GPU detection. + image: infocyph/llm-sm:${LDS_LLM_ARCH} -Changing runtime mode updates ``docker/.env``. Recreate/start the service afterward so -the Compose override changes take effect. +The persisted mapping is: + +- ``cpu`` -> ``LDS_LLM_ARCH=latest``; +- ``nvidia`` -> ``LDS_LLM_ARCH=latest`` plus the NVIDIA GPU overlay; +- ``amd`` -> ``LDS_LLM_ARCH=amd-latest`` plus the AMD device overlay. + +Override detection explicitly when required:: + + lds llm runtime cpu + lds llm runtime nvidia + lds llm runtime amd + +Changing runtime mode updates both ``LDS_AI_RUNTIME`` and ``LDS_LLM_ARCH`` in ``docker/.env``. Recreate/start the service afterward so the Compose override changes take effect. Access ------ @@ -135,6 +145,8 @@ The provider CLI does not silently download a missing model for an unrelated com Privacy and Trust Boundaries ---------------------------- +The base ``llm-sm`` service lives in ``docker/compose/companion.yaml`` and is gated by the ``ai`` profile. Hardware-only overlays add GPU/device access without duplicating the service definition. + By default, ``llm-sm`` receives: - no Docker socket; diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 4d69c2b9..d0641e6b 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -544,14 +544,14 @@ Validate all of these without fixed IPs: AI should become a first-class optional LocalDevStack capability while remaining absent from the default stack. -## 8.1 New `docker/compose/ai.yaml` +## 8.1 Single `llm-sm` service in `docker/compose/companion.yaml` Base service: ```yaml services: llm-sm: - image: ${LDS_LLM_IMAGE:-infocyph/llm-sm:0.03} + image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] volumes: @@ -568,7 +568,7 @@ Important rules: - do not require `container_name`; - do not mount Docker socket; - do not expose `11434` to all interfaces; -- do not add a host port in the base AI file; +- do not add a host port in the base companion service; - do not add an automatic repository/workspace mount; - do not add cloud fallback. @@ -602,7 +602,7 @@ Existing populated volumes must never be silently replaced/reset during upgrades ## 8.3 CPU / NVIDIA / AMD runtime selection -Do not implement unreliable GPU auto-detection as an orchestration requirement. +Use conservative capability detection to choose the initial runtime, while keeping explicit user override authoritative. Provide explicit runtime modes: @@ -621,18 +621,16 @@ docker/compose/ai-amd.yaml NVIDIA: -- keep `LDS_LLM_IMAGE`; +- set `LDS_LLM_ARCH=latest`; - add GPU access using the Compose mechanism supported by current Docker Desktop/Engine. AMD: -- use `LDS_LLM_AMD_IMAGE`; +- set `LDS_LLM_ARCH=amd-latest`; - expose `/dev/kfd`; - expose `/dev/dri`. -Store the selected mode in LocalDevStack env/state. - -Do not change image variants automatically because a GPU happens to be detected. +Store both the selected runtime (`LDS_AI_RUNTIME`) and derived tag (`LDS_LLM_ARCH`) in LocalDevStack env/state. Detection rules are: usable `nvidia-smi` -> NVIDIA; both `/dev/kfd` and `/dev/dri` -> AMD/ROCm; otherwise CPU. An AMD CPU alone never selects the AMD/ROCm image. `lds llm runtime ...` remains the explicit override. ## 8.4 Optional direct host API @@ -865,7 +863,7 @@ Extend `lds setup profile` to include optional AI. Prompt only relevant AI settings: - enable AI yes/no; -- runtime `cpu|nvidia|amd`; +- detected runtime is shown, not prompted by default; - preferred model default `qwen2.5:3b`; - optional direct localhost port yes/no. @@ -1841,7 +1839,7 @@ These do not block the LocalDevStack integration release unless implementation r - Docker socket proxy; - Graphify installation; - browser AI UI beyond Tools admin panel; -- automatic GPU detection; +- conservative GPU/runtime detection with explicit override; - automatic model downloads beyond the baked model; - production-hardening changes unrelated to local development; - rewriting the CLI in another language. @@ -1908,8 +1906,7 @@ LDS_TOOLS_IMAGE=infocyph/tools:latest LDS_RUNNER_IMAGE=infocyph/runner:latest LDS_NGINX_IMAGE=infocyph/nginx:latest LDS_APACHE_IMAGE=infocyph/apache:latest -LDS_LLM_IMAGE=infocyph/llm-sm:latest -LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest +LDS_LLM_ARCH=latest ``` Other runtime defaults follow the same rule. PostgreSQL uses `postgres:alpine`; MySQL, MariaDB, MongoDB, Redis Stack/Redis Insight, CloudBeaver, Mongo Express and Mailpit use their normal moving latest tags because the selected image family does not provide a suitable moving Alpine alias for this stack. @@ -2037,7 +2034,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - Prefer a moving Alpine variant when an image family provides a suitable one; otherwise use its normal moving latest alias. - PostgreSQL defaults to `postgres:alpine`. - Tools, Runner, Nginx and Apache consume their published `:latest` aliases. -- Standard LLM uses `infocyph/llm-sm:latest`; AMD/ROCm uses `infocyph/llm-sm:amd-latest`. +- One LLM service uses `infocyph/llm-sm:${LDS_LLM_ARCH}`; CPU/NVIDIA map to `latest`, AMD/ROCm maps to `amd-latest`. - Elasticsearch, Kibana and Filebeat remain version-aligned on the tested stable version because their required image contract does not provide a suitable moving `latest` alias. - PHP/Node runtime selection remains user-driven and version-specific. - Existing named volumes and container names remain intentionally stable for this release. diff --git a/lib/ai.sh b/lib/ai.sh index 88ffcbd5..157bb26f 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -49,7 +49,7 @@ cmd_llm() { runtime) local mode="${1:-}" if [[ -z "$mode" ]]; then - printf '%s\n' "$(compose_control_value LDS_AI_RUNTIME cpu)" + printf '%s\n' "$(compose_control_value LDS_AI_RUNTIME "$(detect_ai_runtime)")" return 0 fi case "${mode,,}" in diff --git a/lib/certificates.sh b/lib/certificates.sh index 41d580ef..329e8bf7 100644 --- a/lib/certificates.sh +++ b/lib/certificates.sh @@ -432,6 +432,15 @@ uninstall_ca() { add_required_env() { update_env "$ENV_DOCKER" WORKING_DIR "$DIR" + + local ai_runtime + ai_runtime="$(dotenv_value "$ENV_DOCKER" LDS_AI_RUNTIME 2>/dev/null || true)" + if [[ -z "$ai_runtime" ]]; then + ai_runtime="$(detect_ai_runtime)" + update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$ai_runtime" + fi + update_env "$ENV_DOCKER" LDS_LLM_ARCH "$(llm_arch_for_runtime "$ai_runtime")" + ((EUID == 0)) && return 0 update_env "$ENV_DOCKER" USER "$(id -un)" update_env "$ENV_DOCKER" UID "$(id -u)" diff --git a/lib/compose.sh b/lib/compose.sh index 73b60c73..34b7b4d6 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -45,14 +45,19 @@ docker_compose() { local ai_runtime ai_host_port local -a static_f=() - ai_runtime="$(compose_control_value LDS_AI_RUNTIME cpu)" + ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" + [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" case "${ai_runtime,,}" in - "" | cpu) ;; + cpu) ;; nvidia) static_f+=(-f "$CFG/compose/ai-nvidia.yaml") ;; amd) static_f+=(-f "$CFG/compose/ai-amd.yaml") ;; *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd)" ;; esac + local llm_arch + llm_arch="$(llm_arch_for_runtime "$ai_runtime")" || + die "Cannot resolve LLM image tag for runtime: $ai_runtime" + ai_host_port="$(compose_control_value LDS_LLM_HOST_PORT 0)" case "${ai_host_port,,}" in "" | 0 | false | no | off) ;; @@ -71,7 +76,7 @@ docker_compose() { local host_os="${HOST_OS:-$(detect_host_os)}" - HOST_OS="$host_os" "${__LDS_DC_BIN[@]}" \ + HOST_OS="$host_os" LDS_LLM_ARCH="$llm_arch" "${__LDS_DC_BIN[@]}" \ --project-directory "$DIR" \ -f "$COMPOSE_FILE" \ "${static_f[@]}" \ diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index f734b22e..e8e14215 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -426,8 +426,12 @@ cmd_images() { printf '%-16s %s\n' "Runner" "$(compose_control_value LDS_RUNNER_IMAGE infocyph/runner:latest)" printf '%-16s %s\n' "Nginx" "$(compose_control_value LDS_NGINX_IMAGE infocyph/nginx:latest)" printf '%-16s %s\n' "Apache" "$(compose_control_value LDS_APACHE_IMAGE infocyph/apache:latest)" - printf '%-16s %s\n' "LLM" "$(compose_control_value LDS_LLM_IMAGE infocyph/llm-sm:latest)" - printf '%-16s %s\n' "LLM AMD" "$(compose_control_value LDS_LLM_AMD_IMAGE infocyph/llm-sm:amd-latest)" + local ai_runtime llm_arch + ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" + [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" + llm_arch="$(llm_arch_for_runtime "$ai_runtime")" + printf '%-16s %s\n' "LLM" "infocyph/llm-sm:$llm_arch" + printf '%-16s %s\n' "LLM runtime" "$ai_runtime" printf '%-16s postgres:%s\n' "PostgreSQL" "$(compose_control_value POSTGRES_VERSION alpine)" printf '%-16s mysql:%s\n' "MySQL" "$(compose_control_value MYSQL_VERSION latest)" printf '%-16s mariadb:%s\n' "MariaDB" "$(compose_control_value MARIADB_VERSION latest)" diff --git a/lib/platform.sh b/lib/platform.sh index 8826d3b3..f2f5a473 100644 --- a/lib/platform.sh +++ b/lib/platform.sh @@ -1,4 +1,36 @@ # shellcheck shell=bash + +# Detect the preferred local-AI runtime from host GPU capability. +# NVIDIA is preferred on hybrid hosts because the standard image supports +# CPU/NVIDIA, while the AMD image is specifically the ROCm variant. +detect_ai_runtime() { + if has_cmd nvidia-smi && nvidia-smi -L >/dev/null 2>&1; then + printf '%s' nvidia + return 0 + fi + if has_cmd nvidia-smi.exe && nvidia-smi.exe -L >/dev/null 2>&1; then + printf '%s' nvidia + return 0 + fi + + # The AMD Compose override requires the Linux ROCm device nodes, so merely + # having an AMD CPU/GPU name is not sufficient. + if [[ -e /dev/kfd && -d /dev/dri ]]; then + printf '%s' amd + return 0 + fi + + printf '%s' cpu +} + +llm_arch_for_runtime() { + case "${1,,}" in + amd) printf '%s' amd-latest ;; + "" | cpu | nvidia) printf '%s' latest ;; + *) return 1 ;; + esac +} + ############################################################################### # 2. INSTALL / PERMISSIONS (HOST) ############################################################################### diff --git a/lib/profiles.sh b/lib/profiles.sh index 632fedd7..a5aea5b6 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -198,6 +198,13 @@ setup_service() { printf "\n%b→ %s%b\n" "$YELLOW" "$display" "$NC" queue_profile "$profile" + if [[ "$service" == "AI" ]]; then + local detected_runtime detected_arch + detected_runtime="$(compose_control_value LDS_AI_RUNTIME "$(detect_ai_runtime)")" + detected_arch="$(llm_arch_for_runtime "$detected_runtime")" + printf "%bDetected local-AI runtime:%b %s (%s)\n" "$CYAN" "$NC" "$detected_runtime" "infocyph/llm-sm:$detected_arch" + fi + local defaults="${PROFILE_ENV[$profile]:-}" [[ -n "$defaults" ]] || return 0 diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index d8279b0a..6ef1bc37 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -50,12 +50,14 @@ assert_contains "$provider_status" "available=1" assert_contains "$provider_status" "model=qwen2.5:3b" pass "latest Tools reaches the separate provider contract" -for file in ai.yaml ai-nvidia.yaml ai-amd.yaml ai-host-port.yaml; do +for file in ai-nvidia.yaml ai-amd.yaml ai-host-port.yaml; do assert_file "$ROOT/docker/compose/$file" done -assert_file_contains "$ROOT/docker/compose/ai.yaml" 'profiles: [ai]' -assert_file_contains "$ROOT/docker/compose/ai.yaml" 'lds_llm:/root/.ollama' -if grep -Eq '/var/run/docker.sock|PROJECT_DIR|/app' "$ROOT/docker/compose/ai.yaml"; then - fail "base llm-sm service must not receive Docker socket or project mounts" -fi -pass "base AI service trust boundary" +[[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must be consolidated into companion.yaml" +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' +assert_file_contains "$ROOT/docker/compose/ai-nvidia.yaml" 'gpus: all' +assert_file_contains "$ROOT/docker/compose/ai-amd.yaml" '/dev/kfd:/dev/kfd' +assert_file_contains "$ROOT/docker/compose/ai-amd.yaml" '/dev/dri:/dev/dri' +pass "single companion AI service plus hardware-only overlays" diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 4764ea06..16ba2b4c 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -52,7 +52,8 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$service_key" == "llm-sm" ]] || fail "AI service key must be llm-sm" [[ "$runtime_modes" == "cpu,nvidia,amd" ]] || fail "AI runtime metadata drift" [[ "$defaults" == *"LDS_AI_MODEL=qwen2.5:3b"* ]] || fail "AI model default drift" - [[ "$defaults" == *"LDS_AI_RUNTIME=cpu"* ]] || fail "AI runtime default drift" + [[ "$version_env" == "LDS_LLM_ARCH" ]] || fail "AI image selector must be LDS_LLM_ARCH" + [[ "$defaults" != *"LDS_AI_RUNTIME="* ]] || fail "AI profile wizard must not prompt for runtime" [[ "$defaults" == *"LDS_LLM_HOST_PORT=no"* ]] || fail "AI host-port default drift" ;; ELASTICSEARCH) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index f125a382..c0b6158e 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -149,9 +149,9 @@ assert tools["LDS_AI_PROVIDER"] == "ollama" assert tools["LDS_AI_URL"] == "http://llm-sm:11434" assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" ' <<<"$ai_json" -pass "base AI profile is internal-only and deterministic" +pass "companion-owned AI profile is internal-only and deterministic" -amd_json="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-amd.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config --format json)" +amd_json="$(LDS_LLM_ARCH=amd-latest docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-amd.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config --format json)" python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] @@ -175,3 +175,8 @@ assert p["host_ip"] == "127.0.0.1" assert int(p["target"]) == 11434 and int(p["published"]) == 11434 ' <<<"$host_json" pass "direct Ollama port is explicit loopback-only" + + +[[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must live in companion.yaml, not ai.yaml" +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' +pass "LLM service uses one tag selector in companion.yaml" diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index b3d6c0af..02b4c367 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -68,7 +68,8 @@ assert_file_contains "$ai" 'no project/repository bind mount' assert_file_contains "$ai" 'automatically execute model-generated shell commands' assert_file_contains "$ai" 'linux/amd64 only' assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by default' -assert_file_contains "$ai" 'infocyph/llm-sm:amd-latest' +assert_file_contains "$ai" 'infocyph/llm-sm:${LDS_LLM_ARCH}' +assert_file_contains "$ai" 'LDS_LLM_ARCH=amd-latest' assert_file_contains "$ai" 'LDS_AI_MODEL=qwen2.5:3b' pass "local AI trust boundary, platform and workspace limits are documented" diff --git a/tests/env-contract.sh b/tests/env-contract.sh index 6f2df465..f7bea792 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -18,7 +18,8 @@ assert_file_contains "$ROOT/lib/compose.sh" '"${env_files[@]}"' assert_file_contains "$ROOT/lib/hosts.sh" 'var=COMPOSE_PROFILES' assert_file_contains "$ROOT/lds" 'compose_control_value()' assert_file_contains "$ROOT/lds" 'dotenv_value()' -assert_file_contains "$ROOT/lib/compose.sh" 'LDS_AI_RUNTIME cpu' +assert_file_contains "$ROOT/lib/compose.sh" 'compose_control_value LDS_AI_RUNTIME ""' +assert_file_contains "$ROOT/lib/compose.sh" 'LDS_LLM_ARCH="$llm_arch"' assert_file_contains "$ROOT/lib/compose.sh" 'LDS_LLM_HOST_PORT 0' pass "environment file and precedence wiring" @@ -34,8 +35,7 @@ expected=( 'LDS_RUNNER_IMAGE=infocyph/runner:latest' 'LDS_NGINX_IMAGE=infocyph/nginx:latest' 'LDS_APACHE_IMAGE=infocyph/apache:latest' - 'LDS_LLM_IMAGE=infocyph/llm-sm:latest' - 'LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-latest' + 'LDS_LLM_ARCH=latest' 'SCRIPTOMATIC_REF=main' ) for entry in "${expected[@]}"; do @@ -94,3 +94,49 @@ pass "profile setup replaces managed selections and preserves generated profiles assert_file_contains "$ROOT/lib/compose.sh" 'compose_control_value COMPOSE_PROJECT_NAME LocalDevStack' pass "CLI project identity follows the Compose project contract" + + +( + set -euo pipefail + has_cmd() { return 1; } + source "$ROOT/lib/platform.sh" + [[ "$(llm_arch_for_runtime cpu)" == "latest" ]] || fail "CPU LLM tag drift" + [[ "$(llm_arch_for_runtime nvidia)" == "latest" ]] || fail "NVIDIA LLM tag drift" + [[ "$(llm_arch_for_runtime amd)" == "amd-latest" ]] || fail "AMD LLM tag drift" +) +pass "LLM runtime maps to the single LDS_LLM_ARCH tag selector" + +ai_env_tmp="$(mktemp -d)" +( + set -euo pipefail + DIR="$ROOT" + CFG="$ROOT/docker" + ENV_RELEASE="$ROOT/docker/release.env" + ENV_DOCKER="$ai_env_tmp/docker.env" + YELLOW="" NC="" + die() { printf "die: %s\n" "$*" >&2; exit 1; } + compose_control_value() { + local key="$1" fallback="${2:-}" value="" + if [[ -n "${!key+x}" ]]; then printf "%s" "${!key}"; return 0; fi + value="$(dotenv_value "$ENV_DOCKER" "$key" 2>/dev/null || true)" + [[ -n "$value" ]] && { printf "%s" "$value"; return 0; } + value="$(dotenv_value "$ENV_RELEASE" "$key" 2>/dev/null || true)" + [[ -n "$value" ]] && { printf "%s" "$value"; return 0; } + printf "%s" "$fallback" + } + source "$ROOT/lib/env.sh" + source "$ROOT/lib/platform.sh" + source "$ROOT/lib/certificates.sh" + detect_ai_runtime() { printf "%s" amd; } + add_required_env + grep -Fxq "LDS_AI_RUNTIME=amd" "$ENV_DOCKER" || fail "detected AI runtime was not persisted" + grep -Fxq "LDS_LLM_ARCH=amd-latest" "$ENV_DOCKER" || fail "detected AMD tag was not persisted" + + update_env "$ENV_DOCKER" LDS_AI_RUNTIME nvidia + detect_ai_runtime() { printf "%s" amd; } + add_required_env + grep -Fxq "LDS_AI_RUNTIME=nvidia" "$ENV_DOCKER" || fail "explicit runtime must win over detection" + grep -Fxq "LDS_LLM_ARCH=latest" "$ENV_DOCKER" || fail "NVIDIA runtime must use standard latest tag" +) +rm -rf "$ai_env_tmp" +pass "setup bootstrap persists detection without overriding an explicit runtime" diff --git a/tests/published-images.sh b/tests/published-images.sh index e4514a55..d4898cb4 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -35,11 +35,13 @@ for image in "${release[LDS_TOOLS_IMAGE]}" "${release[LDS_RUNNER_IMAGE]}"; do done pass "Tools and Runner publish healthchecks" -[[ "${release[LDS_LLM_IMAGE]:-}" == "infocyph/llm-sm:latest" ]] || - fail "unexpected standard LLM moving image" -[[ "${release[LDS_LLM_AMD_IMAGE]:-}" == "infocyph/llm-sm:amd-latest" ]] || - fail "unexpected AMD LLM moving image" -pass "LLM image references follow latest-tag policy" +[[ "${release[LDS_LLM_ARCH]:-}" == "latest" ]] || + fail "unexpected default LLM tag selector" +grep -Fq 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' "$ROOT/docker/compose/companion.yaml" || + fail "LLM service must use the single LDS_LLM_ARCH selector" +grep -Fq "amd) printf '%s' amd-latest" "$ROOT/lib/platform.sh" || + fail "AMD runtime must map to amd-latest" +pass "LLM image selection follows the single latest/amd-latest tag contract" tools_profile_chooser="$( docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /usr/local/bin/profile-chooser diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index bc36b893..ec4835de 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -15,7 +15,7 @@ pg_ref="$ROOT/docker/conf/postgresql.conf" socket_count="$(grep -RhsF '/var/run/docker.sock:/var/run/docker.sock' "$ROOT/docker/compose" | wc -l | tr -d ' ')" [[ "$socket_count" -eq 2 ]] || fail "Docker socket must be mounted only by Tools and Runner; found $socket_count compose mounts" assert_file_contains "$companion" '/var/run/docker.sock:/var/run/docker.sock' -if grep -RqsF '/var/run/docker.sock' "$ROOT/docker/compose/ai.yaml" "$ROOT/docker/compose/db.yaml" "$ROOT/docker/compose/db-client.yaml" "$ROOT/docker/compose/http.yaml"; then +if grep -RqsF '/var/run/docker.sock' "$ROOT/docker/compose/db.yaml" "$ROOT/docker/compose/db-client.yaml" "$ROOT/docker/compose/http.yaml"; then fail "Docker socket escaped the trusted Tools/Runner boundary" fi pass "Docker socket trust boundary" @@ -74,3 +74,8 @@ pass "all CA install paths use current export with legacy fallback" assert_file_contains "$companion" 'COMPOSE_PROFILES=${COMPOSE_PROFILES:-}' pass "Tools profile visibility follows LocalDevStack profile selection" + +if awk '/^ llm-sm:/ { in_llm=1; next } in_llm && /^ [a-zA-Z0-9_-]+:/ { in_llm=0 } in_llm { print }' "$companion" | grep -Eq '/var/run/docker.sock|PROJECT_DIR|/app'; then + fail "llm-sm must not receive Docker socket or project mounts" +fi +pass "companion-owned llm-sm keeps the AI trust boundary" From 7856672f7009eb1d9be4112e4da697916513d191 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 22:00:56 +0600 Subject: [PATCH 202/509] fix: preserve AI runtime precedence during setup --- lib/certificates.sh | 2 +- tests/env-contract.sh | 9 ++++++++- 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/certificates.sh b/lib/certificates.sh index 329e8bf7..13d0c440 100644 --- a/lib/certificates.sh +++ b/lib/certificates.sh @@ -434,7 +434,7 @@ add_required_env() { update_env "$ENV_DOCKER" WORKING_DIR "$DIR" local ai_runtime - ai_runtime="$(dotenv_value "$ENV_DOCKER" LDS_AI_RUNTIME 2>/dev/null || true)" + ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" if [[ -z "$ai_runtime" ]]; then ai_runtime="$(detect_ai_runtime)" update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$ai_runtime" diff --git a/tests/env-contract.sh b/tests/env-contract.sh index f7bea792..f039d3fa 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -114,7 +114,14 @@ ai_env_tmp="$(mktemp -d)" ENV_RELEASE="$ROOT/docker/release.env" ENV_DOCKER="$ai_env_tmp/docker.env" YELLOW="" NC="" - die() { printf "die: %s\n" "$*" >&2; exit 1; } + die() { printf "die: %s +" "$*" >&2; exit 1; } + dotenv_value() { + local file="$1" key="$2" line + line="$(grep -E "^${key}=" "$file" 2>/dev/null | tail -n1 || true)" + [[ -n "$line" ]] || return 1 + printf "%s" "${line#*=}" + } compose_control_value() { local key="$1" fallback="${2:-}" value="" if [[ -n "${!key+x}" ]]; then printf "%s" "${!key}"; return 0; fi From 753bbfed8a53032a56c3ee0c6253c0f39c3622e1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 22:20:57 +0600 Subject: [PATCH 203/509] refactor: consolidate LLM runtime orchestration --- README.md | 4 +- docker/compose/ai-amd.yaml | 5 -- docker/compose/ai-host-port.yaml | 4 -- docker/compose/ai-nvidia.yaml | 3 - docker/compose/companion.yaml | 4 +- docker/compose/http.yaml | 4 +- docker/release.env | 10 +-- docs/concepts/architecture.rst | 4 +- docs/concepts/profiles-and-env.rst | 16 ++--- docs/guides/local-ai.rst | 2 +- .../07-localdevstack-integration-plan.md | 68 +++++++++++++++++++ lib/ai.sh | 8 ++- lib/compose.sh | 51 +++++++++++--- lib/diagnostics.sh | 8 +-- tests/ai-contract.sh | 17 ++--- tests/compose-contract.sh | 59 ++++++++-------- tests/docs-contract.sh | 9 +++ tests/env-contract.sh | 20 ++++-- tests/published-images.sh | 36 +++++----- 19 files changed, 214 insertions(+), 118 deletions(-) delete mode 100644 docker/compose/ai-amd.yaml delete mode 100644 docker/compose/ai-host-port.yaml delete mode 100644 docker/compose/ai-nvidia.yaml diff --git a/README.md b/README.md index fa65231a..77c9b417 100644 --- a/README.md +++ b/README.md @@ -157,7 +157,7 @@ lds rebuild all ## Service image policy -Release-owned defaults live in `docker/release.env`. +Fixed infrastructure images are declared directly in Compose. `docker/release.env` is reserved for genuinely variable release/build defaults such as `SCRIPTOMATIC_REF`. > Prefer the moving Alpine variant when the image family provides a suitable Alpine variant; otherwise use the normal moving latest tag. @@ -175,7 +175,7 @@ Inspect the effective defaults with `lds images`. ## Environment ownership and precedence -Tracked release defaults live in `docker/release.env`; user LocalDevStack settings live in `docker/.env`; the repository-root `.env` remains application-facing state where applicable. +Tracked release/build defaults live in `docker/release.env`; user LocalDevStack settings live in `docker/.env`; the repository-root `.env` remains application-facing state where applicable. Fixed Tools/Runner/Nginx/Apache image names are not env-overridable because they have no runtime variant choice. LocalDevStack control precedence is: diff --git a/docker/compose/ai-amd.yaml b/docker/compose/ai-amd.yaml deleted file mode 100644 index 8b864c73..00000000 --- a/docker/compose/ai-amd.yaml +++ /dev/null @@ -1,5 +0,0 @@ -services: - llm-sm: - devices: - - /dev/kfd:/dev/kfd - - /dev/dri:/dev/dri diff --git a/docker/compose/ai-host-port.yaml b/docker/compose/ai-host-port.yaml deleted file mode 100644 index 46fc02c5..00000000 --- a/docker/compose/ai-host-port.yaml +++ /dev/null @@ -1,4 +0,0 @@ -services: - llm-sm: - ports: - - "127.0.0.1:${LLM_SM_PORT:-11434}:11434" diff --git a/docker/compose/ai-nvidia.yaml b/docker/compose/ai-nvidia.yaml deleted file mode 100644 index addc4687..00000000 --- a/docker/compose/ai-nvidia.yaml +++ /dev/null @@ -1,3 +0,0 @@ -services: - llm-sm: - gpus: all diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index d070d137..8bd919f7 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -2,7 +2,7 @@ services: server-tools: container_name: SERVER_TOOLS hostname: local-dock - image: ${LDS_TOOLS_IMAGE:-infocyph/tools:latest} + image: infocyph/tools:latest restart: unless-stopped environment: - TZ=${TZ:-} @@ -67,7 +67,7 @@ services: runner: container_name: RUNNER hostname: runner - image: ${LDS_RUNNER_IMAGE:-infocyph/runner:latest} + image: infocyph/runner:latest restart: unless-stopped environment: - TZ=${TZ:-} diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 0beb9898..1b15ea93 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -2,7 +2,7 @@ services: nginx: container_name: NGINX hostname: nginx - image: ${LDS_NGINX_IMAGE:-infocyph/nginx:latest} + image: infocyph/nginx:latest restart: unless-stopped environment: - TZ=${TZ:-} @@ -25,7 +25,7 @@ services: apache: container_name: APACHE hostname: apache - image: ${LDS_APACHE_IMAGE:-infocyph/apache:latest} + image: infocyph/apache:latest restart: unless-stopped environment: - TZ=${TZ:-} diff --git a/docker/release.env b/docker/release.env index bae66bfe..10c037ba 100644 --- a/docker/release.env +++ b/docker/release.env @@ -1,9 +1,3 @@ -# LocalDevStack release-owned moving-image baseline. -# Use latest tags wherever the upstream image family supports them. -# User overrides belong in docker/.env. Do not edit this file for local settings. -LDS_TOOLS_IMAGE=infocyph/tools:latest -LDS_RUNNER_IMAGE=infocyph/runner:latest -LDS_NGINX_IMAGE=infocyph/nginx:latest -LDS_APACHE_IMAGE=infocyph/apache:latest -LDS_LLM_ARCH=latest +# LocalDevStack release-owned defaults that genuinely vary at build/runtime. +# Fixed infrastructure images are declared directly in Compose. SCRIPTOMATIC_REF=main diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index 899ff393..043eef75 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -12,7 +12,7 @@ Host Orchestration TLS installation, support tooling, and convenience wrappers. ``docker/release.env`` - Tracked release-owned defaults such as infrastructure image aliases. + Tracked release/build defaults that genuinely vary, such as ``SCRIPTOMATIC_REF``. Fixed infrastructure image names live directly in Compose. ``docker/.env`` User-owned LocalDevStack configuration. @@ -149,7 +149,7 @@ When the ``ai`` profile is enabled: 4. ``lds ai`` delegates higher-level/operational AI to Tools; 5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-sm`` CLI. -The provider is one ``llm-sm`` service declared with the core companion services and enabled only by the ``ai`` profile. Its image is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. Hardware-specific Compose overlays add only NVIDIA GPU access or AMD ``/dev/kfd`` + ``/dev/dri`` device mappings. +The provider is one ``llm-sm`` service declared with the core companion services and enabled only by the ``ai`` profile. Its image is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. No AI-specific Compose files are tracked; ``lds`` generates a temporary Compose fragment only when NVIDIA GPU access, AMD device mappings, or loopback host-port exposure is required, then removes it after the Compose command. Project Identity ---------------- diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index b989aec6..02b9b068 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -107,15 +107,15 @@ Frequently used user overrides include:: COMPOSE_PROFILES=... COMPOSE_PROJECT_NAME=LocalDevStack -Infrastructure image overrides include:: +Fixed Infrastructure Images +--------------------------- - LDS_TOOLS_IMAGE=infocyph/tools:latest - LDS_RUNNER_IMAGE=infocyph/runner:latest - LDS_NGINX_IMAGE=infocyph/nginx:latest - LDS_APACHE_IMAGE=infocyph/apache:latest +Tools, Runner, Nginx, and Apache are fixed LocalDevStack product components and are +declared directly in Compose as their published ``:latest`` images. LocalDevStack does +not expose redundant ``LDS_*_IMAGE`` overrides for them. -The moving values above are release defaults; normal users usually leave them in -``docker/release.env`` rather than copying them to ``docker/.env``. +``docker/release.env`` is reserved for defaults that genuinely vary, such as +``SCRIPTOMATIC_REF``. Useful Inspection Commands -------------------------- @@ -169,7 +169,7 @@ Run:: lds images -to inspect the effective image set after release defaults and user/shell overrides. +to inspect the effective image set. Fixed product images are reported directly; database/runtime/LLM selections reflect their actual configurable state. Runtime Version Selection ------------------------- diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 60e7178b..cd31c2b5 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -145,7 +145,7 @@ The provider CLI does not silently download a missing model for an unrelated com Privacy and Trust Boundaries ---------------------------- -The base ``llm-sm`` service lives in ``docker/compose/companion.yaml`` and is gated by the ``ai`` profile. Hardware-only overlays add GPU/device access without duplicating the service definition. +The base ``llm-sm`` service lives in ``docker/compose/companion.yaml`` and is gated by the ``ai`` profile. There are no tracked ``ai-*.yaml`` files. When NVIDIA, AMD/ROCm, or direct loopback access is selected, ``lds`` writes a small temporary Compose fragment under ``docker/.runtime/``, uses it for that Compose invocation, and removes it immediately afterward. By default, ``llm-sm`` receives: diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index d0641e6b..da625384 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -2178,3 +2178,71 @@ safe-directory/identity decisions for writable repository operations. After the corrections above, there is no identified legacy feature loss or current cross-image release blocker. The remaining items are explicit optional/future capability work rather than regressions. + + +--- + +# Final implementation simplification — single LLM service and fixed infrastructure images + +This section supersedes earlier planning text in this file wherever it conflicts with the final implementation. + +## Fixed infrastructure images + +Tools, Runner, Nginx, and Apache have no LocalDevStack runtime image variants. Their image names are therefore declared directly in Compose: + +```text +infocyph/tools:latest +infocyph/runner:latest +infocyph/nginx:latest +infocyph/apache:latest +``` + +Do not add `LDS_TOOLS_IMAGE`, `LDS_RUNNER_IMAGE`, `LDS_NGINX_IMAGE`, or `LDS_APACHE_IMAGE` indirection. `docker/release.env` is reserved for release/build defaults that genuinely vary, currently including `SCRIPTOMATIC_REF`. + +## Single LLM service + +The only tracked LLM service definition lives in: + +```text +docker/compose/companion.yaml +``` + +with: + +```yaml +llm-sm: + image: infocyph/llm-sm:${LDS_LLM_ARCH} + profiles: [ai] +``` + +There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. + +Runtime/tag mapping is: + +```text +cpu -> LDS_LLM_ARCH=latest +nvidia -> LDS_LLM_ARCH=latest +amd -> LDS_LLM_ARCH=amd-latest +``` + +Initial runtime detection is conservative: + +- usable `nvidia-smi` / `nvidia-smi.exe` -> `nvidia`; +- both `/dev/kfd` and `/dev/dri` present -> `amd`; +- otherwise -> `cpu`. + +An AMD CPU alone does not select the ROCm image. + +Explicit `LDS_AI_RUNTIME` / `lds llm runtime ...` selection remains authoritative. + +## Ephemeral hardware/host-port Compose augmentation + +Hardware and optional direct-port settings are generated by `lds` only for the current Compose invocation: + +- NVIDIA -> `gpus: all`; +- AMD -> `/dev/kfd` and `/dev/dri`; +- direct host API -> `127.0.0.1:${LLM_SM_PORT:-11434}:11434`. + +The generated fragment is temporary, is not part of `configuration/compose/`, and is removed after the Compose command completes. + +This keeps the repository at one LLM service definition while still avoiding invalid GPU/device declarations on unsupported hosts. diff --git a/lib/ai.sh b/lib/ai.sh index 157bb26f..cd0c59f0 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -54,8 +54,12 @@ cmd_llm() { fi case "${mode,,}" in cpu | nvidia | amd) - update_env "$ENV_DOCKER" LDS_AI_RUNTIME "${mode,,}" - ok "LLM runtime set to ${mode,,}. Recreate llm-sm to apply the change." + local normalized arch + normalized="${mode,,}" + arch="$(llm_arch_for_runtime "$normalized")" + update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" + update_env "$ENV_DOCKER" LDS_LLM_ARCH "$arch" + ok "LLM runtime set to $normalized (infocyph/llm-sm:$arch). Recreate llm-sm to apply the change." ;; *) die "llm runtime " ;; esac diff --git a/lib/compose.sh b/lib/compose.sh index 34b7b4d6..fa3ec17f 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -42,30 +42,57 @@ docker_compose() { local -a env_files=(--env-file "$ENV_RELEASE") [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - local ai_runtime ai_host_port - local -a static_f=() + local ai_runtime ai_host_port llm_arch llm_port runtime_override="" + local ai_host_port_enabled=0 + local -a runtime_f=() ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" case "${ai_runtime,,}" in - cpu) ;; - nvidia) static_f+=(-f "$CFG/compose/ai-nvidia.yaml") ;; - amd) static_f+=(-f "$CFG/compose/ai-amd.yaml") ;; + cpu | nvidia | amd) ;; *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd)" ;; esac + ai_runtime="${ai_runtime,,}" - local llm_arch llm_arch="$(llm_arch_for_runtime "$ai_runtime")" || die "Cannot resolve LLM image tag for runtime: $ai_runtime" ai_host_port="$(compose_control_value LDS_LLM_HOST_PORT 0)" case "${ai_host_port,,}" in "" | 0 | false | no | off) ;; - 1 | true | yes | on) static_f+=(-f "$CFG/compose/ai-host-port.yaml") ;; + 1 | true | yes | on) ai_host_port_enabled=1 ;; *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; esac - # Product overrides are applied before user-provided compose extras. + llm_port="$(compose_control_value LLM_SM_PORT 11434)" + [[ "$llm_port" =~ ^[0-9]+$ ]] && ((llm_port >= 1 && llm_port <= 65535)) || + die "Invalid LLM_SM_PORT: $llm_port (expected 1-65535)" + + if [[ "$ai_runtime" != "cpu" || "$ai_host_port_enabled" == "1" ]]; then + mkdir -p "$CFG/.runtime" + runtime_override="$(mktemp "$CFG/.runtime/ai.XXXXXX")" || + die "Unable to create temporary AI Compose override" + + { + printf '%s\n' 'services:' ' llm-sm:' + case "$ai_runtime" in + nvidia) + printf '%s\n' ' gpus: all' + ;; + amd) + printf '%s\n' ' devices:' ' - /dev/kfd:/dev/kfd' ' - /dev/dri:/dev/dri' + ;; + esac + if ((ai_host_port_enabled)); then + printf '%s\n' ' ports:' + printf ' - "127.0.0.1:%s:11434"\n' "$llm_port" + fi + } >"$runtime_override" + + runtime_f=(-f "$runtime_override") + fi + + # Runtime-generated product overrides are applied before user-provided extras. local -a extra_f=() f for f in "${__EXTRA_FILES[@]:-}"; do [[ -f "$f" ]] || continue @@ -76,13 +103,17 @@ docker_compose() { local host_os="${HOST_OS:-$(detect_host_os)}" + local rc=0 HOST_OS="$host_os" LDS_LLM_ARCH="$llm_arch" "${__LDS_DC_BIN[@]}" \ --project-directory "$DIR" \ -f "$COMPOSE_FILE" \ - "${static_f[@]}" \ + "${runtime_f[@]}" \ "${extra_f[@]}" \ "${env_files[@]}" \ - "$@" + "$@" || rc=$? + + [[ -z "$runtime_override" ]] || rm -f "$runtime_override" + return "$rc" } # helper: print the effective Compose project name. diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index e8e14215..81b34e7e 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -422,10 +422,10 @@ cmd_images() { local elastic elastic="$(compose_control_value ELASTICSEARCH_VERSION 9.5.3)" - printf '%-16s %s\n' "Tools" "$(compose_control_value LDS_TOOLS_IMAGE infocyph/tools:latest)" - printf '%-16s %s\n' "Runner" "$(compose_control_value LDS_RUNNER_IMAGE infocyph/runner:latest)" - printf '%-16s %s\n' "Nginx" "$(compose_control_value LDS_NGINX_IMAGE infocyph/nginx:latest)" - printf '%-16s %s\n' "Apache" "$(compose_control_value LDS_APACHE_IMAGE infocyph/apache:latest)" + printf '%-16s %s\n' "Tools" "infocyph/tools:latest" + printf '%-16s %s\n' "Runner" "infocyph/runner:latest" + printf '%-16s %s\n' "Nginx" "infocyph/nginx:latest" + printf '%-16s %s\n' "Apache" "infocyph/apache:latest" local ai_runtime llm_arch ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 6ef1bc37..96cd1bbb 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -50,14 +50,15 @@ assert_contains "$provider_status" "available=1" assert_contains "$provider_status" "model=qwen2.5:3b" pass "latest Tools reaches the separate provider contract" -for file in ai-nvidia.yaml ai-amd.yaml ai-host-port.yaml; do - assert_file "$ROOT/docker/compose/$file" -done -[[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must be consolidated into companion.yaml" +[[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must remain consolidated into companion.yaml" +if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -quit | grep -q .; then + fail "AI-specific Compose overlays must be generated ephemerally" +fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' -assert_file_contains "$ROOT/docker/compose/ai-nvidia.yaml" 'gpus: all' -assert_file_contains "$ROOT/docker/compose/ai-amd.yaml" '/dev/kfd:/dev/kfd' -assert_file_contains "$ROOT/docker/compose/ai-amd.yaml" '/dev/dri:/dev/dri' -pass "single companion AI service plus hardware-only overlays" +assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" +assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" +assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" +assert_file_contains "$ROOT/lib/compose.sh" '"127.0.0.1:%s:11434"' +pass "single companion AI service with ephemeral hardware/port augmentation" diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index c0b6158e..d17cf814 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -35,6 +35,7 @@ USER=$(id -un) UID=$(id -u) GID=$(id -g) PROJECT_DIR=$ROOT +LDS_LLM_ARCH=latest EOF compose=(docker compose @@ -60,18 +61,6 @@ render redis --profile redis render elasticsearch --profile elasticsearch render elasticsearch-filebeat --profile elasticsearch --profile filebeat render ai --profile ai -docker compose --project-directory "$ROOT" \ - -f "$ROOT/docker/compose/main.yaml" \ - -f "$ROOT/docker/compose/ai-nvidia.yaml" \ - --env-file "$release_env" --env-file "$user_env" --profile ai config --quiet -docker compose --project-directory "$ROOT" \ - -f "$ROOT/docker/compose/main.yaml" \ - -f "$ROOT/docker/compose/ai-amd.yaml" \ - --env-file "$release_env" --env-file "$user_env" --profile ai config --quiet -docker compose --project-directory "$ROOT" \ - -f "$ROOT/docker/compose/main.yaml" \ - -f "$ROOT/docker/compose/ai-host-port.yaml" \ - --env-file "$release_env" --env-file "$user_env" --profile ai config --quiet resolved="$("${compose[@]}" --profile mysql config)" assert_contains "$resolved" "server-tools:" @@ -107,16 +96,13 @@ assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9 ' <<<"$elastic_json" pass "Elastic stack uses aligned current-stable tags because latest is unsupported" -printf '%s\n' 'LDS_TOOLS_IMAGE=example.invalid/tools:user-override' >>"$user_env" -user_override="$("${compose[@]}" config)" -assert_contains "$user_override" "image: example.invalid/tools:user-override" -pass "docker/.env overrides release defaults" - -shell_override="$( - LDS_TOOLS_IMAGE=example.invalid/tools:shell-override "${compose[@]}" config -)" -assert_contains "$shell_override" "image: example.invalid/tools:shell-override" -pass "shell override wins over user and release env files" +printf '%s\n' 'LDS_TOOLS_IMAGE=example.invalid/tools:ignored' >>"$user_env" +fixed_images="$("${compose[@]}" config)" +assert_contains "$fixed_images" "image: infocyph/tools:latest" +if grep -Fq 'example.invalid/tools' <<<"$fixed_images"; then + fail "fixed infrastructure images must not be user-overridable" +fi +pass "fixed infrastructure images are declared directly in Compose" core_json="$("${compose[@]}" config --format json)" python3 -c ' @@ -151,7 +137,7 @@ assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" -amd_json="$(LDS_LLM_ARCH=amd-latest docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-amd.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config --format json)" +amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" config show --json --raw 2>/dev/null)" python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] @@ -159,13 +145,18 @@ assert s["image"] == "infocyph/llm-sm:amd-latest" devices=" ".join(str(x) for x in s.get("devices", [])) assert "/dev/kfd" in devices and "/dev/dri" in devices ' <<<"$amd_json" -pass "AMD AI override" +pass "AMD AI runtime is generated dynamically" -nvidia_yaml="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-nvidia.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config)" -assert_contains "$nvidia_yaml" "gpus:" -pass "NVIDIA AI override" +nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" config show --json --raw 2>/dev/null)" +python3 -c ' +import json,sys +s=json.load(sys.stdin)["services"]["llm-sm"] +assert s["image"] == "infocyph/llm-sm:latest" +assert s.get("gpus") +' <<<"$nvidia_json" +pass "NVIDIA AI runtime is generated dynamically" -host_json="$(docker compose --project-directory "$ROOT" -f "$ROOT/docker/compose/main.yaml" -f "$ROOT/docker/compose/ai-host-port.yaml" --env-file "$release_env" --env-file "$user_env" --profile ai config --format json)" +host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" config show --json --raw 2>/dev/null)" python3 -c ' import json,sys ports=json.load(sys.stdin)["services"]["llm-sm"]["ports"] @@ -174,9 +165,13 @@ p=ports[0] assert p["host_ip"] == "127.0.0.1" assert int(p["target"]) == 11434 and int(p["published"]) == 11434 ' <<<"$host_json" -pass "direct Ollama port is explicit loopback-only" - +pass "direct Ollama port is generated dynamically and loopback-only" -[[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must live in companion.yaml, not ai.yaml" +if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -quit | grep -q .; then + fail "AI-specific Compose files must not exist" +fi +if [[ -d "$ROOT/docker/.runtime" ]] && find "$ROOT/docker/.runtime" -type f -print -quit | grep -q .; then + fail "temporary AI Compose overrides were not cleaned up" +fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' -pass "LLM service uses one tag selector in companion.yaml" +pass "single LLM service plus ephemeral hardware/port overrides" diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 02b4c367..5322b44a 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -108,3 +108,12 @@ for required in 'lds profiles add ' 'lds support trace ' assert_contains "$help_md" "$required" done pass "embedded CLI help covers documented command groups" + + +for stale in LDS_TOOLS_IMAGE LDS_RUNNER_IMAGE LDS_NGINX_IMAGE LDS_APACHE_IMAGE; do + if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then + fail "user-facing docs expose obsolete fixed-image variable: $stale" + fi +done +assert_file_contains "$ai" 'There are no tracked ``ai-*.yaml`` files' +pass "docs reflect fixed infrastructure images and ephemeral AI overrides" diff --git a/tests/env-contract.sh b/tests/env-contract.sh index f039d3fa..ab30e31d 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -31,17 +31,12 @@ fi pass "release and user env ownership boundaries" expected=( - 'LDS_TOOLS_IMAGE=infocyph/tools:latest' - 'LDS_RUNNER_IMAGE=infocyph/runner:latest' - 'LDS_NGINX_IMAGE=infocyph/nginx:latest' - 'LDS_APACHE_IMAGE=infocyph/apache:latest' - 'LDS_LLM_ARCH=latest' 'SCRIPTOMATIC_REF=main' ) for entry in "${expected[@]}"; do assert_file_contains "$release_env" "$entry" done -pass "moving latest image manifest" +pass "release env contains only genuinely variable release defaults" assert_file_contains "$ROOT/lib/profiles.sh" 'CATALOG_FILE="$CFG/catalog/services.psv"' assert_file_contains "$ROOT/lib/profiles.sh" 'load_service_catalog()' @@ -147,3 +142,16 @@ ai_env_tmp="$(mktemp -d)" ) rm -rf "$ai_env_tmp" pass "setup bootstrap persists detection without overriding an explicit runtime" + +for stale in LDS_TOOLS_IMAGE LDS_RUNNER_IMAGE LDS_NGINX_IMAGE LDS_APACHE_IMAGE; do + if grep -RqsF "$stale" "$ROOT/docker/compose" "$ROOT/docker/release.env" "$ROOT/lib"; then + fail "fixed infrastructure image still has unnecessary variable: $stale" + fi +done +pass "fixed Tools/Runner/Nginx/Apache images have no env indirection" + +for stale in ai-nvidia.yaml ai-amd.yaml ai-host-port.yaml; do + [[ ! -e "$ROOT/docker/compose/$stale" ]] || fail "stale AI Compose overlay remains: $stale" +done +assert_file_contains "$ROOT/lib/compose.sh" 'mktemp "$CFG/.runtime/ai.XXXXXX"' +pass "AI runtime overrides are ephemeral rather than tracked Compose files" diff --git a/tests/published-images.sh b/tests/published-images.sh index d4898cb4..4648a983 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -7,18 +7,18 @@ source "$ROOT/tests/lib/assertions.sh" release_env="$ROOT/docker/release.env" assert_file "$release_env" +assert_file_contains "$release_env" 'SCRIPTOMATIC_REF=main' -declare -A release=() -while IFS='=' read -r key value; do - [[ -n "$key" && "$key" != \#* ]] || continue - release["$key"]="$value" -done <"$release_env" +tools_image="infocyph/tools:latest" +runner_image="infocyph/runner:latest" +nginx_image="infocyph/nginx:latest" +apache_image="infocyph/apache:latest" images=( - "${release[LDS_TOOLS_IMAGE]:-}" - "${release[LDS_RUNNER_IMAGE]:-}" - "${release[LDS_NGINX_IMAGE]:-}" - "${release[LDS_APACHE_IMAGE]:-}" + "$tools_image" + "$runner_image" + "$nginx_image" + "$apache_image" ) for image in "${images[@]}"; do @@ -29,14 +29,12 @@ for image in "${images[@]}"; do done pass "published infrastructure compatibility images exist" -for image in "${release[LDS_TOOLS_IMAGE]}" "${release[LDS_RUNNER_IMAGE]}"; do +for image in "$tools_image" "$runner_image"; do health="$(docker image inspect "$image" --format '{{json .Config.Healthcheck}}')" [[ -n "$health" && "$health" != "null" ]] || fail "$image must publish a healthcheck" done pass "Tools and Runner publish healthchecks" -[[ "${release[LDS_LLM_ARCH]:-}" == "latest" ]] || - fail "unexpected default LLM tag selector" grep -Fq 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' "$ROOT/docker/compose/companion.yaml" || fail "LLM service must use the single LDS_LLM_ARCH selector" grep -Fq "amd) printf '%s' amd-latest" "$ROOT/lib/platform.sh" || @@ -44,7 +42,7 @@ grep -Fq "amd) printf '%s' amd-latest" "$ROOT/lib/platform.sh" || pass "LLM image selection follows the single latest/amd-latest tag contract" tools_profile_chooser="$( - docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /usr/local/bin/profile-chooser + docker run --rm --entrypoint cat "$tools_image" /usr/local/bin/profile-chooser )" catalog="$ROOT/docker/catalog/services.psv" @@ -78,7 +76,7 @@ done <"$catalog" pass "LocalDevStack catalog matches latest Tools non-version profile contract" -docker run --rm --entrypoint sh "${release[LDS_TOOLS_IMAGE]}" -lc ' +docker run --rm --entrypoint sh "$tools_image" -lc ' test -x /usr/local/bin/mkhost test -s /etc/share/runtime-versions.json jq -e ".php.active | type == \"array\" and length > 0" /etc/share/runtime-versions.json >/dev/null @@ -88,10 +86,10 @@ docker run --rm --entrypoint sh "${release[LDS_TOOLS_IMAGE]}" -lc ' pass "latest Tools preserves interactive PHP/Node runtime version catalog" php_template="$( - docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /etc/docker-templates/php.compose.yaml + docker run --rm --entrypoint cat "$tools_image" /etc/docker-templates/php.compose.yaml )" node_template="$( - docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /etc/docker-templates/node.compose.yaml + docker run --rm --entrypoint cat "$tools_image" /etc/docker-templates/node.compose.yaml )" assert_contains "$php_template" 'PHP_VERSION: {{PHP_VERSION}}' assert_contains "$php_template" 'image: localdevstack-php:{{PHP_VERSION}}' @@ -99,7 +97,7 @@ assert_contains "$node_template" 'NODE_VERSION: {{NODE_VERSION}}' assert_contains "$node_template" 'image: localdevstack-node:{{NODE_VERSION}}' pass "selected runtime versions remain build/image identity inputs" -docker run --rm --entrypoint sh "${release[LDS_RUNNER_IMAGE]}" -ec ' +docker run --rm --entrypoint sh "$runner_image" -ec ' test -x /usr/local/bin/logrotate-worker.sh test -x /usr/local/bin/runner-healthcheck test -f /etc/logrotate.d/daily @@ -111,14 +109,14 @@ assert_contains "$php_template" './docker/conf/www-php.conf:/usr/local/etc/php-f pass "generated PHP runtime uses the maintained FPM pool config" tools_certify="$( - docker run --rm --entrypoint cat "${release[LDS_TOOLS_IMAGE]}" /usr/local/bin/certify + docker run --rm --entrypoint cat "$tools_image" /usr/local/bin/certify )" assert_contains "$tools_certify" 'EXPORT_DIR="${EXPORT_DIR:-/etc/share/certs}"' assert_contains "$tools_certify" 'EXPORT_ROOTCA_NAME="${EXPORT_ROOTCA_NAME:-rootCA.pem}"' assert_contains "$tools_certify" 'atomic_install 0644 "$root_ca" "$EXPORT_DIR/$EXPORT_ROOTCA_NAME"' pass "latest Tools public TLS export contract" -docker run --rm --entrypoint sh "${release[LDS_TOOLS_IMAGE]}" -ec ' +docker run --rm --entrypoint sh "$tools_image" -ec ' test -d /etc/share/state test -x /usr/local/bin/env-store grep -Fq "/etc/share/state/env-store.json" /usr/local/bin/env-store From 95291e8f6a8bb451d9c927786dcd2767a11498ff Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 22:23:43 +0600 Subject: [PATCH 204/509] test: align contracts with fixed images and quiet config output --- tests/compose-contract.sh | 6 ++-- tests/qol-contract.sh | 67 ++++++++++++++++++++++++++++++++++++--- 2 files changed, 66 insertions(+), 7 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index d17cf814..b55568f9 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -137,7 +137,7 @@ assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" -amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" config show --json --raw 2>/dev/null)" +amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null)" python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] @@ -147,7 +147,7 @@ assert "/dev/kfd" in devices and "/dev/dri" in devices ' <<<"$amd_json" pass "AMD AI runtime is generated dynamically" -nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" config show --json --raw 2>/dev/null)" +nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" --quiet config show --json --raw 2>/dev/null)" python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] @@ -156,7 +156,7 @@ assert s.get("gpus") ' <<<"$nvidia_json" pass "NVIDIA AI runtime is generated dynamically" -host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" config show --json --raw 2>/dev/null)" +host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" --quiet config show --json --raw 2>/dev/null)" python3 -c ' import json,sys ports=json.load(sys.stdin)["services"]["llm-sm"]["ports"] diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 85855976..9ac1602c 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -25,7 +25,6 @@ trap cleanup EXIT cat >"$user_env" < (Alpine)" -pass "images is offline-safe and reflects effective overrides" +pass "images is offline-safe and reports fixed plus configurable image selections" urls="$(PATH="$tmpbin:$PATH" "$ROOT/lds" urls)" assert_contains "$urls" "https://admin.localhost" @@ -58,7 +57,67 @@ pass "urls is profile-aware and offline-safe" env_used="$(PATH="$tmpbin:$PATH" "$ROOT/lds" config env-used)" assert_contains "$env_used" $'user\tCOMPOSE_PROFILES' -assert_contains "$env_used" $'release\tLDS_TOOLS_IMAGE' +assert_contains "$env_used" +if grep -Fq "supersecret-ci-value" <<<"$env_used"; then + fail "config env-used leaked a value" +fi +pass "config env-used reports keys only" + +redacted="$("$ROOT/lds" config show)" +if grep -Fq "supersecret-ci-value" <<<"$redacted"; then + fail "config show leaked MYSQL_ROOT_PASSWORD" +fi +assert_contains "$redacted" "***REDACTED***" +pass "config show redacts effective secrets by default" + +bundle_dir="$(mktemp -d)" +bundle="$bundle_dir/bundle.zip" +"$ROOT/lds" support bundle --redact "$bundle" >/dev/null +python3 - "$bundle" "supersecret-ci-value" <<'PY' +import sys, zipfile +path, secret = sys.argv[1:] +with zipfile.ZipFile(path) as z: + for name in z.namelist(): + data = z.read(name) + if secret.encode() in data: + raise SystemExit(f"support bundle leaked secret in {name}") +PY +rm -rf "$bundle_dir" +pass "support bundle redacts interpolated secrets" + +default_bundle_dir="$(mktemp -d)" +( + cd "$default_bundle_dir" + "$ROOT/lds" support bundle --redact >/dev/null +) +default_bundle="$(find "$default_bundle_dir" -maxdepth 1 -type f -name 'lds_bundle_*.zip' -print -quit)" +[[ -n "$default_bundle" ]] || fail "support bundle option-only invocation did not create a default zip" +rm -rf "$default_bundle_dir" +pass "support bundle option-only invocation uses a generated filename" + +help="$("$ROOT/lds" help)" +assert_contains "$help" "doctor" +assert_contains "$help" "images" +assert_contains "$help" "urls" +assert_contains "$help" "support trace" +assert_contains "$help" "--global" +pass "QoL commands are discoverable" + +assert_file_contains "$ROOT/lds" 'images | urls | doctor)' +assert_file_contains "$ROOT/lib/diagnostics.sh" 'Docker daemon is unavailable.' +pass "doctor owns Docker availability diagnostics" + + +assert_file_contains "$ROOT/lds" 'trace) cmd_support_trace "$@" ;;' +assert_file_contains "$ROOT/lib/services.sh" '/etc/share/vhosts/nginx/*.conf' +if grep -Fq '$DIR/configuration/nginx/' "$ROOT/lib/services.sh" "$ROOT/lib/diagnostics.sh"; then + fail "domain inspection must use persisted named-volume vhosts" +fi +assert_file_contains "$ROOT/lib/services.sh" 'docker_compose restart "${services[@]}"' +assert_file_contains "$ROOT/lib/services.sh" '--global' +assert_file_contains "$ROOT/lib/services.sh" 'label=com.docker.compose.project=$project' +pass "trace, domain listing, targeted restart, and scoped cleanup contracts" +release\tSCRIPTOMATIC_REF' if grep -Fq "supersecret-ci-value" <<<"$env_used"; then fail "config env-used leaked a value" fi From 50cb53ea9464bfac2bd948c28df3e57996d33146 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Fri, 18 Sep 2026 22:25:39 +0600 Subject: [PATCH 205/509] test: stabilize dynamic AI compose verification --- tests/compose-contract.sh | 6 ++-- tests/qol-contract.sh | 66 +++------------------------------------ 2 files changed, 7 insertions(+), 65 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index b55568f9..5b225324 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -137,7 +137,7 @@ assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" -amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null)" +amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] @@ -147,7 +147,7 @@ assert "/dev/kfd" in devices and "/dev/dri" in devices ' <<<"$amd_json" pass "AMD AI runtime is generated dynamically" -nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" --quiet config show --json --raw 2>/dev/null)" +nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] @@ -156,7 +156,7 @@ assert s.get("gpus") ' <<<"$nvidia_json" pass "NVIDIA AI runtime is generated dynamically" -host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" --quiet config show --json --raw 2>/dev/null)" +host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys ports=json.load(sys.stdin)["services"]["llm-sm"]["ports"] diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 9ac1602c..b1d9fe64 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -39,6 +39,9 @@ chmod +x "$tmpbin/docker" images="$(PATH="$tmpbin:$PATH" "$ROOT/lds" images)" assert_contains "$images" "infocyph/tools:latest" +assert_contains "$images" "infocyph/runner:latest" +assert_contains "$images" "infocyph/nginx:latest" +assert_contains "$images" "infocyph/apache:latest" assert_contains "$images" "postgres:alpine" assert_contains "$images" "elasticsearch:9.5.3" assert_contains "$images" "localdevstack-php: (Alpine)" @@ -57,7 +60,7 @@ pass "urls is profile-aware and offline-safe" env_used="$(PATH="$tmpbin:$PATH" "$ROOT/lds" config env-used)" assert_contains "$env_used" $'user\tCOMPOSE_PROFILES' -assert_contains "$env_used" +assert_contains "$env_used" $'release\tSCRIPTOMATIC_REF' if grep -Fq "supersecret-ci-value" <<<"$env_used"; then fail "config env-used leaked a value" fi @@ -107,67 +110,6 @@ assert_file_contains "$ROOT/lds" 'images | urls | doctor)' assert_file_contains "$ROOT/lib/diagnostics.sh" 'Docker daemon is unavailable.' pass "doctor owns Docker availability diagnostics" - -assert_file_contains "$ROOT/lds" 'trace) cmd_support_trace "$@" ;;' -assert_file_contains "$ROOT/lib/services.sh" '/etc/share/vhosts/nginx/*.conf' -if grep -Fq '$DIR/configuration/nginx/' "$ROOT/lib/services.sh" "$ROOT/lib/diagnostics.sh"; then - fail "domain inspection must use persisted named-volume vhosts" -fi -assert_file_contains "$ROOT/lib/services.sh" 'docker_compose restart "${services[@]}"' -assert_file_contains "$ROOT/lib/services.sh" '--global' -assert_file_contains "$ROOT/lib/services.sh" 'label=com.docker.compose.project=$project' -pass "trace, domain listing, targeted restart, and scoped cleanup contracts" -release\tSCRIPTOMATIC_REF' -if grep -Fq "supersecret-ci-value" <<<"$env_used"; then - fail "config env-used leaked a value" -fi -pass "config env-used reports keys only" - -redacted="$("$ROOT/lds" config show)" -if grep -Fq "supersecret-ci-value" <<<"$redacted"; then - fail "config show leaked MYSQL_ROOT_PASSWORD" -fi -assert_contains "$redacted" "***REDACTED***" -pass "config show redacts effective secrets by default" - -bundle_dir="$(mktemp -d)" -bundle="$bundle_dir/bundle.zip" -"$ROOT/lds" support bundle --redact "$bundle" >/dev/null -python3 - "$bundle" "supersecret-ci-value" <<'PY' -import sys, zipfile -path, secret = sys.argv[1:] -with zipfile.ZipFile(path) as z: - for name in z.namelist(): - data = z.read(name) - if secret.encode() in data: - raise SystemExit(f"support bundle leaked secret in {name}") -PY -rm -rf "$bundle_dir" -pass "support bundle redacts interpolated secrets" - -default_bundle_dir="$(mktemp -d)" -( - cd "$default_bundle_dir" - "$ROOT/lds" support bundle --redact >/dev/null -) -default_bundle="$(find "$default_bundle_dir" -maxdepth 1 -type f -name 'lds_bundle_*.zip' -print -quit)" -[[ -n "$default_bundle" ]] || fail "support bundle option-only invocation did not create a default zip" -rm -rf "$default_bundle_dir" -pass "support bundle option-only invocation uses a generated filename" - -help="$("$ROOT/lds" help)" -assert_contains "$help" "doctor" -assert_contains "$help" "images" -assert_contains "$help" "urls" -assert_contains "$help" "support trace" -assert_contains "$help" "--global" -pass "QoL commands are discoverable" - -assert_file_contains "$ROOT/lds" 'images | urls | doctor)' -assert_file_contains "$ROOT/lib/diagnostics.sh" 'Docker daemon is unavailable.' -pass "doctor owns Docker availability diagnostics" - - assert_file_contains "$ROOT/lds" 'trace) cmd_support_trace "$@" ;;' assert_file_contains "$ROOT/lib/services.sh" '/etc/share/vhosts/nginx/*.conf' if grep -Fq '$DIR/configuration/nginx/' "$ROOT/lib/services.sh" "$ROOT/lib/diagnostics.sh"; then From f02056628ddfbda3ba3aa31fc9a332e54c74b463 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 07:52:37 +0600 Subject: [PATCH 206/509] chores --- docker/compose/companion.yaml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 8bd919f7..1f3c2e4a 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -55,6 +55,8 @@ services: - datastore llm-sm: + container_name: LLM_SM + hostname: llm-sm image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] From de2ce7a2e32ff779ee1421c058a80c86eaa5115e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:14:20 +0600 Subject: [PATCH 207/509] docs: clarify LocalDevStack LLM invocation --- README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/README.md b/README.md index 77c9b417..9b4683e6 100644 --- a/README.md +++ b/README.md @@ -264,11 +264,14 @@ lds ai repo-review ... lds llm models lds llm pull lds llm show +lds llm ask "Explain dependency injection briefly" lds llm chat ... ``` Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. +Use the `lds` wrapper for provider commands. LocalDevStack intentionally has no repository-root `compose.yml`; its effective Compose project is assembled from `docker/compose/main.yaml`, release/user env files, optional extras, and temporary runtime-specific overrides. Therefore a bare command such as `docker compose exec llm-sm ...` from the LocalDevStack root is not equivalent and fails before reaching the container. Use `lds llm ...` instead. + The `llm-sm` container receives **no Docker socket and no project/repository bind mount** by default. Current published `llm-sm` images are `linux/amd64`; the rest of LocalDevStack can still run on arm64 with the AI profile disabled. ## Storage and trust boundaries From b176d8f03143ed6b6946a3894e1be5ff41c6fb8b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:14:45 +0600 Subject: [PATCH 208/509] docs: explain LocalDevStack provider invocation --- docs/guides/local-ai.rst | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index cd31c2b5..782e5ef0 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -104,7 +104,7 @@ AI vs LLM Commands lds llm rm lds llm unload lds llm run - lds llm ask ... + lds llm ask "Explain dependency injection briefly" lds llm chat ... lds llm prompt ... lds llm code ... @@ -118,6 +118,12 @@ AI vs LLM Commands This separation keeps application/operational AI in Tools and model/runtime behavior in the provider image. +Always invoke provider commands through ``lds llm`` inside LocalDevStack. The repository +does not expose a root ``compose.yml`` because ``lds`` assembles the effective Compose +project from the tracked main file, env layers, optional extras, and runtime-specific +temporary overrides. A bare ``docker compose exec llm-sm ...`` from the LocalDevStack +repository root therefore fails at the host Compose layer before ``llm-sm`` runs. + Model Persistence ----------------- From f528c0b7818da16a3b325e0fb188397eac97e16f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:14:48 +0600 Subject: [PATCH 209/509] test: lock LocalDevStack LLM invocation boundary --- tests/docs-contract.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 5322b44a..37079433 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -71,7 +71,10 @@ assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by assert_file_contains "$ai" 'infocyph/llm-sm:${LDS_LLM_ARCH}' assert_file_contains "$ai" 'LDS_LLM_ARCH=amd-latest' assert_file_contains "$ai" 'LDS_AI_MODEL=qwen2.5:3b' -pass "local AI trust boundary, platform and workspace limits are documented" +assert_file_contains "$ai" 'Always invoke provider commands through ``lds llm``' +assert_file_contains "$ai" 'docker compose exec llm-sm' +assert_file_contains "$readme" 'lds llm ask "Explain dependency injection briefly"' +pass "local AI trust boundary, platform, workspace and invocation limits are documented" for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-sm:0.03'; do if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then From 38b06c5cf3a20d77d21b6e430a99c713a625013b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:19:23 +0600 Subject: [PATCH 210/509] fix(ai): align LocalDevStack with llm-sm provider options --- docker/compose/companion.yaml | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 1f3c2e4a..948719db 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -55,11 +55,24 @@ services: - datastore llm-sm: - container_name: LLM_SM hostname: llm-sm image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] + environment: + - LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} + - LLM_SM_INPUT_WARN_BYTES=${LLM_SM_INPUT_WARN_BYTES:-1048576} + - LLM_SM_INPUT_MAX_BYTES=${LLM_SM_INPUT_MAX_BYTES:-0} + - LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216} + - LLM_SM_ATTACHMENTS_MAX_BYTES=${LLM_SM_ATTACHMENTS_MAX_BYTES:-33554432} + - LLM_SM_ATTACHMENT_MAX_COUNT=${LLM_SM_ATTACHMENT_MAX_COUNT:-16} + - LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24} + - LLM_SM_PDF_DPI=${LLM_SM_PDF_DPI:-120} + - LLM_SM_ALLOW_LARGE_INPUT=${LLM_SM_ALLOW_LARGE_INPUT:-0} + - OLLAMA_NUM_PARALLEL=${OLLAMA_NUM_PARALLEL:-1} + - OLLAMA_MAX_LOADED_MODELS=${OLLAMA_MAX_LOADED_MODELS:-1} + - OLLAMA_KEEP_ALIVE=${OLLAMA_KEEP_ALIVE:-5m} + - OLLAMA_NO_CLOUD=${OLLAMA_NO_CLOUD:-1} volumes: - lds_llm:/root/.ollama networks: From 61d8bd338a3e47d70e2c728d3fe05a7d34bfeec8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:19:28 +0600 Subject: [PATCH 211/509] test(ai): lock provider environment bridge --- tests/compose-contract.sh | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 5b225324..8f994d92 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -129,6 +129,20 @@ assert set(s["networks"]) == {"frontend","backend"} targets={v["target"] for v in s["volumes"]} assert targets == {"/root/.ollama"} assert d["volumes"]["lds_llm"]["name"] == "LLMModels" +env=s["environment"] +assert env["LLM_SM_MODEL"] == "qwen2.5:3b" +assert env["LLM_SM_INPUT_WARN_BYTES"] == "1048576" +assert env["LLM_SM_INPUT_MAX_BYTES"] == "0" +assert env["LLM_SM_ATTACHMENT_MAX_BYTES"] == "16777216" +assert env["LLM_SM_ATTACHMENTS_MAX_BYTES"] == "33554432" +assert env["LLM_SM_ATTACHMENT_MAX_COUNT"] == "16" +assert env["LLM_SM_PDF_MAX_PAGES"] == "24" +assert env["LLM_SM_PDF_DPI"] == "120" +assert env["LLM_SM_ALLOW_LARGE_INPUT"] == "0" +assert env["OLLAMA_NUM_PARALLEL"] == "1" +assert env["OLLAMA_MAX_LOADED_MODELS"] == "1" +assert env["OLLAMA_KEEP_ALIVE"] == "5m" +assert env["OLLAMA_NO_CLOUD"] == "1" tools=d["services"]["server-tools"]["environment"] assert tools["LDS_AI_ENABLED"] == "auto" assert tools["LDS_AI_PROVIDER"] == "ollama" From 69375fd40299ef72df10b94cfd3cf347210cce4c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:19:31 +0600 Subject: [PATCH 212/509] test(ai): cover llm-sm provider option bridge --- tests/ai-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 96cd1bbb..77e79e31 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -57,6 +57,9 @@ fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24}' assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" From 7420ee9510a658be328329db1fe132eed12822fa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:20:43 +0600 Subject: [PATCH 213/509] fix(ai): pass default provider system prompt --- docker/compose/companion.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 948719db..52fc6dd9 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -61,6 +61,7 @@ services: profiles: [ai] environment: - LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} + - LLM_SM_SYSTEM=${LLM_SM_SYSTEM:-} - LLM_SM_INPUT_WARN_BYTES=${LLM_SM_INPUT_WARN_BYTES:-1048576} - LLM_SM_INPUT_MAX_BYTES=${LLM_SM_INPUT_MAX_BYTES:-0} - LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216} From 5205d7e3cb6e8bfefb790575af84ed59b08c056a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:21:00 +0600 Subject: [PATCH 214/509] docs(ai): reconcile LLM Compose and provider options --- README.md | 41 +++++++++++++++++++++++++++++++++++++++-- 1 file changed, 39 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 9b4683e6..1d66b81d 100644 --- a/README.md +++ b/README.md @@ -242,7 +242,7 @@ The provider uses one image/tag contract: infocyph/llm-sm:${LDS_LLM_ARCH} ``` -CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. +CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. `LDS_LLM_ARCH` is derived by the LocalDevStack runtime selector; it is not a separate version choice users should maintain manually. Override detection explicitly when needed: @@ -268,7 +268,44 @@ lds llm ask "Explain dependency injection briefly" lds llm chat ... ``` -Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. +Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. Change that loopback port with `LLM_SM_PORT` in `docker/.env`. + +The LocalDevStack LLM Compose layout is intentionally small: + +```text +docker/compose/main.yaml + └─ includes docker/compose/companion.yaml + └─ llm-sm service (profile: ai) + +docker/.runtime/ai.* temporary per-command overlay generated by lds + only when NVIDIA, AMD/ROCm, or host-port settings require it +``` + +There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. AI hardware and host-port additions are generated ephemerally and removed after the Compose command. Files under `configuration/compose/` remain the normal LocalDevStack extras/runtime-fragment area; they are not built-in LLM variant files. + +Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where applicable: + +| Setting | Default | Effect | +|---|---:|---| +| `LDS_AI_MODEL` | `qwen2.5:3b` | Tools model and `lds llm` default model | +| `LDS_AI_RUNTIME` | auto-detected | `cpu`, `nvidia`, or `amd` runtime selection | +| `LDS_LLM_HOST_PORT` | `0` | Enables/disables direct loopback API mapping | +| `LLM_SM_PORT` | `11434` | Host loopback port when direct mapping is enabled | +| `LLM_SM_SYSTEM` | empty | Default system instruction for provider prompts | +| `LLM_SM_INPUT_WARN_BYTES` | `1048576` | Text/diff warning threshold | +| `LLM_SM_INPUT_MAX_BYTES` | `0` | Text/diff hard ceiling; `0` disables it | +| `LLM_SM_ATTACHMENT_MAX_BYTES` | `16777216` | Per attachment/source-file ceiling | +| `LLM_SM_ATTACHMENTS_MAX_BYTES` | `33554432` | Aggregate attachment ceiling | +| `LLM_SM_ATTACHMENT_MAX_COUNT` | `16` | Attachment-count ceiling | +| `LLM_SM_PDF_MAX_PAGES` | `24` | PDF-vision page ceiling | +| `LLM_SM_PDF_DPI` | `120` | PDF-vision render DPI | +| `LLM_SM_ALLOW_LARGE_INPUT` | `0` | Deliberate bypass for configured provider limits | +| `OLLAMA_NUM_PARALLEL` | `1` | Ollama request parallelism | +| `OLLAMA_MAX_LOADED_MODELS` | `1` | Loaded-model limit | +| `OLLAMA_KEEP_ALIVE` | `5m` | Model keep-alive | +| `OLLAMA_NO_CLOUD` | `1` | Keep cloud integration disabled | + +`LDS_AI_CONNECT_TIMEOUT`, `LDS_AI_PREFLIGHT_TIMEOUT`, `LDS_AI_TIMEOUT`, `LDS_AI_AVAILABILITY_TTL`, `LDS_AI_MAX_CONTEXT_BYTES`, `LDS_AI_MAX_REQUEST_BYTES`, and `LDS_AI_MAX_RESPONSE_BYTES` belong to the Tools consumer layer rather than the provider container. Use the `lds` wrapper for provider commands. LocalDevStack intentionally has no repository-root `compose.yml`; its effective Compose project is assembled from `docker/compose/main.yaml`, release/user env files, optional extras, and temporary runtime-specific overrides. Therefore a bare command such as `docker compose exec llm-sm ...` from the LocalDevStack root is not equivalent and fails before reaching the container. Use `lds llm ...` instead. From b627f962484d3217b1ee6cf2d087938b832dff2d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:21:25 +0600 Subject: [PATCH 215/509] docs(ai): document authoritative Compose and provider settings --- docs/guides/local-ai.rst | 76 +++++++++++++++++++++++++++++++++++++--- 1 file changed, 72 insertions(+), 4 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 782e5ef0..067ef385 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -39,12 +39,16 @@ The single provider service uses:: image: infocyph/llm-sm:${LDS_LLM_ARCH} -The persisted mapping is: +The derived image mapping is: - ``cpu`` -> ``LDS_LLM_ARCH=latest``; - ``nvidia`` -> ``LDS_LLM_ARCH=latest`` plus the NVIDIA GPU overlay; - ``amd`` -> ``LDS_LLM_ARCH=amd-latest`` plus the AMD device overlay. +``LDS_LLM_ARCH`` is derived from the selected runtime for each Compose invocation. It is +not an independent image-version selector. ``lds llm runtime ...`` persists the explicit +runtime choice and its matching derived tag for compatibility/inspection. + Override detection explicitly when required:: lds llm runtime cpu @@ -82,6 +86,31 @@ Show/disable it with:: The port can be changed through ``LLM_SM_PORT`` when direct host access is enabled. +Compose Ownership +----------------- + +The tracked provider service exists only in:: + + docker/compose/companion.yaml + +and is included through:: + + docker/compose/main.yaml + +There are no tracked ``ai.yaml``, ``ai-nvidia.yaml``, ``ai-amd.yaml`` or +``ai-host-port.yaml`` files. + +When runtime-specific Compose data is required, ``lds`` creates a temporary fragment +under ``docker/.runtime/`` for the current command only: + +- NVIDIA -> ``gpus: all``; +- AMD/ROCm -> ``/dev/kfd`` and ``/dev/dri``; +- direct host API -> ``127.0.0.1:${LLM_SM_PORT:-11434}:11434``. + +The fragment is removed after the Compose command. ``configuration/compose/`` remains +the normal extras/generated-runtime area and is not the location of built-in LLM +runtime variants. + AI vs LLM Commands ------------------ @@ -143,15 +172,54 @@ Pull a model explicitly:: lds llm pull -Then set ``LDS_AI_MODEL`` in ``docker/.env`` if Tools should use that model by default. -A command-scoped shell value remains the highest-precedence override. +Then set ``LDS_AI_MODEL`` in ``docker/.env`` if it should become the default for both +Tools AI commands and ``lds llm`` provider commands. LocalDevStack forwards that value to +the provider as ``LLM_SM_MODEL``. A command-scoped shell value remains the +highest-precedence LocalDevStack override. The provider CLI does not silently download a missing model for an unrelated command. +Provider Settings +----------------- + +These provider settings can be placed in ``docker/.env`` and are forwarded to +``llm-sm``:: + + LLM_SM_SYSTEM= + LLM_SM_INPUT_WARN_BYTES=1048576 + LLM_SM_INPUT_MAX_BYTES=0 + LLM_SM_ATTACHMENT_MAX_BYTES=16777216 + LLM_SM_ATTACHMENTS_MAX_BYTES=33554432 + LLM_SM_ATTACHMENT_MAX_COUNT=16 + LLM_SM_PDF_MAX_PAGES=24 + LLM_SM_PDF_DPI=120 + LLM_SM_ALLOW_LARGE_INPUT=0 + OLLAMA_NUM_PARALLEL=1 + OLLAMA_MAX_LOADED_MODELS=1 + OLLAMA_KEEP_ALIVE=5m + OLLAMA_NO_CLOUD=1 + +``LLM_SM_INPUT_MAX_BYTES=0`` disables only the hard text/diff ceiling. Attachment and +PDF-vision limits remain active unless their own value is set to ``0``. +``LLM_SM_ALLOW_LARGE_INPUT=1`` is the explicit escape hatch for a deliberate request. + +Consumer settings are separate. The following values configure the Tools AI layer, not +the provider runtime:: + + LDS_AI_CONNECT_TIMEOUT + LDS_AI_PREFLIGHT_TIMEOUT + LDS_AI_TIMEOUT + LDS_AI_AVAILABILITY_TTL + LDS_AI_MAX_CONTEXT_BYTES + LDS_AI_MAX_REQUEST_BYTES + LDS_AI_MAX_RESPONSE_BYTES + Privacy and Trust Boundaries ---------------------------- -The base ``llm-sm`` service lives in ``docker/compose/companion.yaml`` and is gated by the ``ai`` profile. There are no tracked ``ai-*.yaml`` files. When NVIDIA, AMD/ROCm, or direct loopback access is selected, ``lds`` writes a small temporary Compose fragment under ``docker/.runtime/``, uses it for that Compose invocation, and removes it immediately afterward. +The base ``llm-sm`` service lives in ``docker/compose/companion.yaml`` and is gated by +the ``ai`` profile. Runtime-specific hardware/port additions are ephemeral as described +above; no separate tracked AI Compose variants exist. By default, ``llm-sm`` receives: From dc132c92fa1b4a92b0a184fc68eb63cd8751ea91 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:21:39 +0600 Subject: [PATCH 216/509] docs(ai): align runtime and provider environment semantics --- docs/concepts/profiles-and-env.rst | 45 +++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 7 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index 02b9b068..d9633376 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -211,11 +211,19 @@ Important AI settings include:: LDS_AI_PROVIDER=ollama LDS_AI_URL=http://llm-sm:11434 LDS_AI_MODEL=qwen2.5:3b - LDS_AI_RUNTIME= - LDS_LLM_ARCH=latest + LDS_AI_RUNTIME= LDS_LLM_HOST_PORT=0 + LLM_SM_PORT=11434 -During setup LocalDevStack detects the preferred runtime. NVIDIA is selected only when ``nvidia-smi`` is usable; AMD is selected only when the ROCm Linux device nodes ``/dev/kfd`` and ``/dev/dri`` are present; otherwise CPU is selected. The corresponding image tag is persisted as ``LDS_LLM_ARCH`` (``latest`` for CPU/NVIDIA, ``amd-latest`` for AMD). +When ``LDS_AI_RUNTIME`` is not explicitly set, LocalDevStack detects the preferred +runtime for the Compose invocation. NVIDIA is selected only when ``nvidia-smi`` is +usable; AMD is selected only when the ROCm Linux device nodes ``/dev/kfd`` and +``/dev/dri`` are present; otherwise CPU is selected. + +``LDS_LLM_ARCH`` is derived from that runtime (``latest`` for CPU/NVIDIA, +``amd-latest`` for AMD). It is not a separate user-facing image version selector. +Using ``lds llm runtime ...`` persists the explicit runtime choice and matching derived +tag for compatibility/inspection. Use:: @@ -224,10 +232,33 @@ Use:: to override the detected runtime or host-port behavior. -Tools also accepts optional timeout/context limits through ``LDS_AI_CONNECT_TIMEOUT``, -``LDS_AI_PREFLIGHT_TIMEOUT``, ``LDS_AI_TIMEOUT``, ``LDS_AI_AVAILABILITY_TTL``, -``LDS_AI_MAX_CONTEXT_BYTES``, ``LDS_AI_MAX_REQUEST_BYTES``, and -``LDS_AI_MAX_RESPONSE_BYTES``. +The selected ``LDS_AI_MODEL`` is also forwarded to the provider as +``LLM_SM_MODEL``, so Tools and ``lds llm`` share the same default model. + +Provider-side options accepted in ``docker/.env`` include:: + + LLM_SM_SYSTEM= + LLM_SM_INPUT_WARN_BYTES=1048576 + LLM_SM_INPUT_MAX_BYTES=0 + LLM_SM_ATTACHMENT_MAX_BYTES=16777216 + LLM_SM_ATTACHMENTS_MAX_BYTES=33554432 + LLM_SM_ATTACHMENT_MAX_COUNT=16 + LLM_SM_PDF_MAX_PAGES=24 + LLM_SM_PDF_DPI=120 + LLM_SM_ALLOW_LARGE_INPUT=0 + OLLAMA_NUM_PARALLEL=1 + OLLAMA_MAX_LOADED_MODELS=1 + OLLAMA_KEEP_ALIVE=5m + OLLAMA_NO_CLOUD=1 + +Tools consumer settings remain separate and include +``LDS_AI_CONNECT_TIMEOUT``, ``LDS_AI_PREFLIGHT_TIMEOUT``, ``LDS_AI_TIMEOUT``, +``LDS_AI_AVAILABILITY_TTL``, ``LDS_AI_MAX_CONTEXT_BYTES``, +``LDS_AI_MAX_REQUEST_BYTES``, and ``LDS_AI_MAX_RESPONSE_BYTES``. + +The LLM service itself is tracked only in ``docker/compose/companion.yaml``. There are +no tracked AI runtime-variant YAML files; ``lds`` creates temporary fragments under +``docker/.runtime/`` only for NVIDIA, AMD/ROCm, or direct host-port augmentation. Compose Extras -------------- From 8fdec7c4878d593c2b7b8b0d28b0866b66928ed5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:21:53 +0600 Subject: [PATCH 217/509] docs(ai): complete LLM CLI reference --- docs/reference/cli.rst | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index dd3c68aa..b8168595 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -219,6 +219,15 @@ LLM Provider lds llm ollama ... lds llm api ... lds llm version + lds llm help + +These commands execute the bundled provider CLI through LocalDevStack's Compose wrapper. +Do not replace them with bare ``docker compose exec llm-sm ...`` from the repository +root; LocalDevStack has no root ``compose.yml``. + +``LDS_AI_MODEL`` is forwarded to the provider as ``LLM_SM_MODEL``, so +``lds ai`` and ``lds llm`` share the configured default model. Provider input, +attachment, PDF and Ollama runtime knobs are documented in :doc:`../guides/local-ai`. Runtime selection:: @@ -231,6 +240,8 @@ Direct host-port control:: lds llm host-port on lds llm host-port off +The loopback port defaults to 11434 and can be changed with ``LLM_SM_PORT``. + Rebuild ------- From 6d2691e1b8333778dd01b26fca5e67cb2a2355f0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:22:04 +0600 Subject: [PATCH 218/509] docs(ai): align provider architecture contract --- docs/concepts/architecture.rst | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index 043eef75..c8e032b7 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -149,7 +149,17 @@ When the ``ai`` profile is enabled: 4. ``lds ai`` delegates higher-level/operational AI to Tools; 5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-sm`` CLI. -The provider is one ``llm-sm`` service declared with the core companion services and enabled only by the ``ai`` profile. Its image is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. No AI-specific Compose files are tracked; ``lds`` generates a temporary Compose fragment only when NVIDIA GPU access, AMD device mappings, or loopback host-port exposure is required, then removes it after the Compose command. +The provider is one ``llm-sm`` service declared in +``docker/compose/companion.yaml`` and enabled only by the ``ai`` profile. Its image +is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and +AMD/ROCm resolves to ``amd-latest``. ``LDS_LLM_ARCH`` is derived from the effective +runtime rather than maintained as an independent version selector. + +No AI-specific Compose files are tracked. ``lds`` generates a temporary fragment under +``docker/.runtime/`` only when NVIDIA GPU access, AMD device mappings, or loopback +host-port exposure is required, then removes it after the Compose command. The service +also forwards the configured ``LDS_AI_MODEL`` to the provider as ``LLM_SM_MODEL`` and +passes the documented provider safety/tuning settings from ``docker/.env``. Project Identity ---------------- From 09ffcfc2aef246e0268f8ddcb2441e457bf30860 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:22:44 +0600 Subject: [PATCH 219/509] docs(plan): reconcile final LLM integration contract --- .../07-localdevstack-integration-plan.md | 86 ++++++++++++------- 1 file changed, 55 insertions(+), 31 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index da625384..d3fdaec3 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -32,12 +32,14 @@ LocalDevStack implementation must begin against this tested ecosystem set: | Nginx | `infocyph/nginx:0.4.1` | | Apache | `infocyph/apache:0.4.2` | | Tools | `infocyph/tools:0.23.2` | -| LLM standard | `infocyph/llm-sm:0.03` | -| LLM AMD | `infocyph/llm-sm:amd-0.03` | +| LLM standard | `infocyph/llm-sm:latest` | +| LLM AMD | `infocyph/llm-sm:amd-latest` | -Do not silently replace those defaults with moving `:latest` tags during the first LocalDevStack integration release. - -Users may override image references explicitly, but a released LocalDevStack version must have a reproducible compatibility baseline. +The implemented image policy now follows the ecosystem moving aliases for LocalDevStack +infrastructure. Standard LLM uses `latest`; AMD/ROCm uses `amd-latest`. Release +reproducibility is provided by each image repository's immutable release tags, provenance +and compatibility gates rather than by duplicating pinned image versions in +LocalDevStack configuration. --- @@ -102,7 +104,7 @@ Host ├── mailpit ├── databases and admin clients │ - └── llm-sm 0.03 (optional) + └── llm-sm current stable (optional) ├── qwen2.5:3b baked default ├── persistent /root/.ollama └── Ollama API :11434 @@ -325,7 +327,7 @@ Validate: - streamed response is not buffered incorrectly; - AI absence leaves core services healthy. -A manual/release-gate job may optionally exercise the real published `infocyph/llm-sm:0.03`, because that image already has its own model-bearing runtime gate. +A manual/release-gate job may optionally exercise the real published `infocyph/llm-sm:latest`, because that image already has its own model-bearing runtime gate. ## 5.2 New `tests/` @@ -612,25 +614,25 @@ nvidia amd ``` -Recommended static overrides: - -```text -docker/compose/ai-nvidia.yaml -docker/compose/ai-amd.yaml -``` +Do not create tracked runtime-variant Compose files. Keep one service in +`docker/compose/companion.yaml` and let `lds` generate a temporary +`docker/.runtime/ai.*` fragment for the current Compose invocation only. NVIDIA: -- set `LDS_LLM_ARCH=latest`; -- add GPU access using the Compose mechanism supported by current Docker Desktop/Engine. +- derive `LDS_LLM_ARCH=latest`; +- add `gpus: all` in the temporary fragment. AMD: -- set `LDS_LLM_ARCH=amd-latest`; -- expose `/dev/kfd`; -- expose `/dev/dri`. +- derive `LDS_LLM_ARCH=amd-latest`; +- expose `/dev/kfd` and `/dev/dri` in the temporary fragment. -Store both the selected runtime (`LDS_AI_RUNTIME`) and derived tag (`LDS_LLM_ARCH`) in LocalDevStack env/state. Detection rules are: usable `nvidia-smi` -> NVIDIA; both `/dev/kfd` and `/dev/dri` -> AMD/ROCm; otherwise CPU. An AMD CPU alone never selects the AMD/ROCm image. `lds llm runtime ...` remains the explicit override. +`LDS_AI_RUNTIME` is the explicit runtime selector when configured. `LDS_LLM_ARCH` +is derived from the effective runtime for Compose interpolation and is not a separate +user-facing image-version choice. Detection rules are: usable `nvidia-smi` -> NVIDIA; +both `/dev/kfd` and `/dev/dri` -> AMD/ROCm; otherwise CPU. An AMD CPU alone never +selects the AMD/ROCm image. `lds llm runtime ...` remains the explicit override. ## 8.4 Optional direct host API @@ -642,13 +644,9 @@ https://llm.localhost Do not expose `11434` by default. -If developers explicitly need direct Ollama access, add an optional override such as: - -```text -docker/compose/ai-host-port.yaml -``` - -binding only: +If developers explicitly need direct Ollama access, `lds` adds the port mapping to the +same temporary `docker/.runtime/ai.*` fragment used for hardware augmentation, binding +only: ```text 127.0.0.1:${LLM_SM_PORT:-11434}:11434 @@ -704,11 +702,31 @@ Also pass through supported advanced limits only when the user sets them: Why LocalDevStack should default `LDS_AI_MODEL=qwen2.5:3b`: -- `llm-sm:0.03` ships that model; +- `llm-sm:latest` ships that model; - Tools intentionally reports ambiguity when multiple models are installed and no model is selected; - users may pull more models without breaking Tools AI workflows. -Users can change `LDS_AI_MODEL` explicitly. +Users can change `LDS_AI_MODEL` explicitly. LocalDevStack forwards it to the provider +as `LLM_SM_MODEL` so Tools and `lds llm` share the same default model. + +Provider settings accepted through `docker/.env` and forwarded to `llm-sm` are: + +- `LLM_SM_SYSTEM`; +- `LLM_SM_INPUT_WARN_BYTES`; +- `LLM_SM_INPUT_MAX_BYTES`; +- `LLM_SM_ATTACHMENT_MAX_BYTES`; +- `LLM_SM_ATTACHMENTS_MAX_BYTES`; +- `LLM_SM_ATTACHMENT_MAX_COUNT`; +- `LLM_SM_PDF_MAX_PAGES`; +- `LLM_SM_PDF_DPI`; +- `LLM_SM_ALLOW_LARGE_INPUT`; +- `OLLAMA_NUM_PARALLEL`; +- `OLLAMA_MAX_LOADED_MODELS`; +- `OLLAMA_KEEP_ALIVE`; +- `OLLAMA_NO_CLOUD`. + +These are provider/runtime controls and are distinct from the Tools consumer +`LDS_AI_*` timeout/context limits. Do not make Tools pull/remove models. @@ -1797,7 +1815,7 @@ With fake provider on normal CI: With real provider on manual/release gate when feasible: -- `infocyph/llm-sm:0.03`; +- `infocyph/llm-sm:latest`; - baked `qwen2.5:3b`; - persistent model volume; - Tools generation; @@ -2064,7 +2082,7 @@ Compared LocalDevStack against the current related releases/main contracts: - Runner **0.5** - Nginx **0.4.1** - Apache **0.4.2** -- LLM-SM **0.03** +- LLM-SM **current stable** - Toolset **2.0** - Scriptomatic current `main` @@ -2213,9 +2231,13 @@ with: llm-sm: image: infocyph/llm-sm:${LDS_LLM_ARCH} profiles: [ai] + environment: + LLM_SM_MODEL: ${LDS_AI_MODEL:-qwen2.5:3b} + # provider input/PDF/Ollama tuning values are forwarded from docker/.env ``` -There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. +There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or +`ai-host-port.yaml` files. Do not reintroduce them. Runtime/tag mapping is: @@ -2234,6 +2256,8 @@ Initial runtime detection is conservative: An AMD CPU alone does not select the ROCm image. Explicit `LDS_AI_RUNTIME` / `lds llm runtime ...` selection remains authoritative. +`LDS_LLM_ARCH` is derived from that effective runtime and must not become an +independent manual image-version selector. ## Ephemeral hardware/host-port Compose augmentation From 1ea96776c949ec1cd502e1e0bad5b4730be0f96a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:23:02 +0600 Subject: [PATCH 220/509] docs(plan): align llm-sm integration with final LocalDevStack contract --- .../docker-ecosystem/06-docker-llm-sm-plan.md | 61 +++++++++++-------- 1 file changed, 37 insertions(+), 24 deletions(-) diff --git a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md index 3c272363..be16dc2a 100644 --- a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md +++ b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md @@ -161,40 +161,53 @@ No LocalDevStack-driven change expected. ## LocalDevStack Integration Contract -LocalDevStack should add an optional `llm`/`ai` profile that consumes the published image. +This lower-layer plan is implemented through the authoritative LocalDevStack integration +plan in `07-localdevstack-integration-plan.md`. -Suggested variables: +LocalDevStack consumes the published provider through the optional `ai` profile. It does +not rebuild `docker-llm-sm` locally and does not consume this repository's standalone +Compose files. -- `LLM_SM_IMAGE=infocyph/llm-sm:latest` or pinned release; -- `LLM_SM_MODEL=` only when overriding image default/use selection; -- `LLM_SM_VOLUME=`; -- optional GPU mode selection: CPU/NVIDIA standard tag vs AMD tag; -- optional workspace mount path. +Current LocalDevStack controls: + +- `LDS_AI_MODEL` selects the shared Tools/provider default model and is forwarded as + `LLM_SM_MODEL`; +- `LDS_AI_RUNTIME` selects `cpu`, `nvidia`, or `amd` when explicitly configured; +- `LDS_LLM_ARCH` is derived from the effective runtime (`latest` for CPU/NVIDIA, + `amd-latest` for AMD/ROCm); +- `LDS_LLM_HOST_PORT` controls optional direct loopback exposure; +- `LLM_SM_PORT` controls that loopback host port; +- provider input/PDF/Ollama tuning values are forwarded from LocalDevStack + `docker/.env`. + +Compose ownership: + +- one tracked `llm-sm` service in `docker/compose/companion.yaml`; +- no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or + `ai-host-port.yaml`; +- NVIDIA, AMD/ROCm, and host-port additions are generated temporarily under + `docker/.runtime/` by `lds`. Persistence: -- mount named volume to `/root/.ollama`; -- preserve user-pulled models across container recreation/upgrades. +- LocalDevStack's `LLMModels` named volume mounts at `/root/.ollama`; +- user-pulled models survive container recreation/upgrades. Workspace: -- optional `${PROJECT_DIR}` or selected project path -> `/workspace`; -- working directory `/workspace` where repo-aware commands are desired; -- mount must be declared at container creation time; -- do not require workspace mount for normal inference/API use. +- LocalDevStack intentionally does not mount a project/repository into `llm-sm` by + default; +- repository-aware analysis normally uses the Tools consumer layer; +- provider stdin flows remain available without weakening the default trust boundary. Networking: -- join the appropriate LocalDevStack internal network by service name; -- expose `11434` to host only if local host tools (Graphify/editor integrations/etc.) need it; -- if exposed, default host binding should remain loopback-oriented for a local dev stack; -- other containers should use service DNS, e.g. `http://llm-sm:11434`. - -Graphify/client integration: +- internal consumers use `http://llm-sm:11434`; +- Nginx exposes `https://llm.localhost`; +- direct host `11434`-style access is opt-in and loopback-only. -- point client to Ollama/OpenAI-compatible endpoint exposed by `llm-sm`; -- keep `qwen2.5:3b` usable as the default small model; -- allow users to pull/select larger models without changing LocalDevStack image definitions. +User/provider commands are invoked through `lds llm ...`, not a bare +`docker compose exec` from the LocalDevStack repository root. ## Acceptance Criteria @@ -204,6 +217,6 @@ Graphify/client integration: 4. Named volume persists pulled models across container recreation. 5. Other LocalDevStack containers can reach Ollama by service DNS. 6. Host clients can reach it through an explicitly configured loopback port when enabled. -7. `docker exec llm-sm ...` works under LocalDevStack. -8. Mounted project repo supports `llm-sm ai-commit` without host installation. +7. `lds llm ...` delegates to the bundled provider CLI through LocalDevStack's Compose wrapper. +8. No project/repository bind mount is required for normal provider operation. 9. No Graphify/LocalDevStack-specific package is added to the image solely for integration. From fe157bd040c0865b3c67deeb80fc0d25990d4fdd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:23:17 +0600 Subject: [PATCH 221/509] test(ai): verify provider option overrides --- tests/compose-contract.sh | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 8f994d92..d7d4d351 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -131,6 +131,7 @@ assert targets == {"/root/.ollama"} assert d["volumes"]["lds_llm"]["name"] == "LLMModels" env=s["environment"] assert env["LLM_SM_MODEL"] == "qwen2.5:3b" +assert env["LLM_SM_SYSTEM"] == "" assert env["LLM_SM_INPUT_WARN_BYTES"] == "1048576" assert env["LLM_SM_INPUT_MAX_BYTES"] == "0" assert env["LLM_SM_ATTACHMENT_MAX_BYTES"] == "16777216" @@ -151,6 +152,18 @@ assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" +printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_SM_PDF_MAX_PAGES=12' 'LLM_SM_SYSTEM=Answer briefly.' >>"$user_env" +ai_override_json="$("${compose[@]}" --profile ai config --format json)" +python3 -c ' +import json,sys +s=json.load(sys.stdin)["services"]["llm-sm"] +env=s["environment"] +assert env["LLM_SM_MODEL"] == "qwen2.5:1.5b" +assert env["LLM_SM_PDF_MAX_PAGES"] == "12" +assert env["LLM_SM_SYSTEM"] == "Answer briefly." +' <<<"$ai_override_json" +pass "LocalDevStack forwards configured model and provider options into llm-sm" + amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys From 675abfb1b847b8db74e2a96e3be5df734dfd7e67 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:23:20 +0600 Subject: [PATCH 222/509] test(docs): lock current LLM options and Compose ownership --- tests/docs-contract.sh | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 37079433..ee4e7de8 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -71,10 +71,18 @@ assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by assert_file_contains "$ai" 'infocyph/llm-sm:${LDS_LLM_ARCH}' assert_file_contains "$ai" 'LDS_LLM_ARCH=amd-latest' assert_file_contains "$ai" 'LDS_AI_MODEL=qwen2.5:3b' +assert_file_contains "$ai" 'LLM_SM_ATTACHMENT_MAX_BYTES=16777216' +assert_file_contains "$ai" 'LLM_SM_PDF_MAX_PAGES=24' +assert_file_contains "$ai" 'docker/compose/companion.yaml' +assert_file_contains "$ai" 'docker/.runtime/' +assert_file_contains "$ai" 'There are no tracked ``ai.yaml``, ``ai-nvidia.yaml``, ``ai-amd.yaml`` or' assert_file_contains "$ai" 'Always invoke provider commands through ``lds llm``' assert_file_contains "$ai" 'docker compose exec llm-sm' +assert_file_contains "$profiles" 'LLM_SM_ALLOW_LARGE_INPUT=0' +assert_file_contains "$profiles" 'LDS_LLM_ARCH' assert_file_contains "$readme" 'lds llm ask "Explain dependency injection briefly"' -pass "local AI trust boundary, platform, workspace and invocation limits are documented" +assert_file_contains "$readme" 'LLM_SM_ATTACHMENTS_MAX_BYTES' +pass "local AI trust boundary, Compose ownership, provider options and invocation limits are documented" for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-sm:0.03'; do if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then @@ -118,5 +126,5 @@ for stale in LDS_TOOLS_IMAGE LDS_RUNNER_IMAGE LDS_NGINX_IMAGE LDS_APACHE_IMAGE; fail "user-facing docs expose obsolete fixed-image variable: $stale" fi done -assert_file_contains "$ai" 'There are no tracked ``ai-*.yaml`` files' +assert_file_contains "$ai" 'no separate tracked AI Compose variants exist' pass "docs reflect fixed infrastructure images and ephemeral AI overrides" From a57b6f3051f28d3dffc25fd28f04e229bc29bc8a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:24:02 +0600 Subject: [PATCH 223/509] docs(plan): remove stale LLM pins and image indirection --- .../07-localdevstack-integration-plan.md | 40 +++++++++---------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index d3fdaec3..0c3d8e3a 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -360,33 +360,33 @@ Never commit real certificates, private keys, SOPS keys or user secrets. --- -# 6. Batch 2 — Compatibility image defaults +# 6. Batch 2 — Release defaults and image policy -## 6.1 New tracked `docker/release.env` +## 6.1 Tracked `docker/release.env` -Create a committed compatibility manifest. - -Initial values: +The implemented release manifest contains only defaults that genuinely vary independently +of the fixed product Compose definitions. Current tracked value: ```text -LDS_TOOLS_IMAGE=infocyph/tools:0.23.2 -LDS_RUNNER_IMAGE=infocyph/runner:0.5 -LDS_NGINX_IMAGE=infocyph/nginx:0.4.1 -LDS_APACHE_IMAGE=infocyph/apache:0.4.2 -LDS_LLM_IMAGE=infocyph/llm-sm:0.03 -LDS_LLM_AMD_IMAGE=infocyph/llm-sm:amd-0.03 +SCRIPTOMATIC_REF=main ``` +Tools, Runner, Nginx and Apache are declared directly as their moving `:latest` product +images. The LLM service is declared directly as +`infocyph/llm-sm:${LDS_LLM_ARCH}`, where `LDS_LLM_ARCH` is derived from the effective +AI runtime (`latest` for CPU/NVIDIA, `amd-latest` for AMD/ROCm). + +Do not add `LDS_TOOLS_IMAGE`, `LDS_RUNNER_IMAGE`, `LDS_NGINX_IMAGE`, +`LDS_APACHE_IMAGE`, `LDS_LLM_IMAGE`, or `LDS_LLM_AMD_IMAGE` indirection. + Rules: -- this file is owned by the LocalDevStack release; -- user overrides belong in `docker/.env`; -- user overrides always win; -- `lds` should load release defaults before user overrides; +- `docker/release.env` is release-owned; +- user LocalDevStack overrides belong in `docker/.env`; +- command-scoped shell values retain the highest interpolation precedence; - do not copy release defaults into a user file on every update; -- upgrades should not overwrite user values. - -The exact implementation may use repeated Compose `--env-file` flags or controlled export/merge logic in `lds`, but precedence must be deterministic and covered by tests. +- upgrades should not overwrite user values; +- precedence must remain deterministic and covered by tests. ## 6.2 Compose image references @@ -2166,8 +2166,8 @@ This is a non-blocking follow-up, not a release defect. ## LLM-SM capability boundaries -The current LLM-SM 0.03 image is **linux/amd64 only**. LocalDevStack remains usable on -arm64 with the `ai` profile disabled. +The current published LLM-SM image contract is **linux/amd64 only**. LocalDevStack +remains usable on arm64 with the `ai` profile disabled. LocalDevStack intentionally does not mount a repository/workspace into `llm-sm` by default. Direct model/API/chat/stdin workflows are supported. Repository-aware analysis From 66c01eea1484c0dd22254d42f87ef5790677fa86 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:26:47 +0600 Subject: [PATCH 224/509] docs(ai): clarify source attachment count semantics --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 1d66b81d..b2a4da9a 100644 --- a/README.md +++ b/README.md @@ -296,7 +296,7 @@ Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where ap | `LLM_SM_INPUT_MAX_BYTES` | `0` | Text/diff hard ceiling; `0` disables it | | `LLM_SM_ATTACHMENT_MAX_BYTES` | `16777216` | Per attachment/source-file ceiling | | `LLM_SM_ATTACHMENTS_MAX_BYTES` | `33554432` | Aggregate attachment ceiling | -| `LLM_SM_ATTACHMENT_MAX_COUNT` | `16` | Attachment-count ceiling | +| `LLM_SM_ATTACHMENT_MAX_COUNT` | `16` | Source-attachment count ceiling | | `LLM_SM_PDF_MAX_PAGES` | `24` | PDF-vision page ceiling | | `LLM_SM_PDF_DPI` | `120` | PDF-vision render DPI | | `LLM_SM_ALLOW_LARGE_INPUT` | `0` | Deliberate bypass for configured provider limits | From 493c279ef722a30e4dedb1d34498944ab9729594 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:26:51 +0600 Subject: [PATCH 225/509] docs(ai): clarify source attachment count semantics From 6065e36fc9dec6718d83fcc6c2235fbd9102f406 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:26:54 +0600 Subject: [PATCH 226/509] docs(ai): clarify source attachment count semantics From 65ad094a929b254a8b1f386398a8d0ee8ea8a3dc Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:46:10 +0600 Subject: [PATCH 227/509] style(compose): normalize volume mount scalars --- docker/compose/companion.yaml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 52fc6dd9..573c8c76 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -29,7 +29,7 @@ services: - LDS_AI_MAX_REQUEST_BYTES - LDS_AI_MAX_RESPONSE_BYTES volumes: - - "${PROJECT_DIR:-./../../../application}:/app" + - ${PROJECT_DIR:-./../../../application}:/app - lds_ssl_roots:/etc/share/rootCA - lds_apache_host:/etc/share/vhosts/apache - lds_nginx_host:/etc/share/vhosts/nginx @@ -44,8 +44,8 @@ services: - ../../configuration/sops/config:/etc/share/sops/config - ../../configuration/sops/global:/etc/share/sops/global - ../../configuration/sops/keys:/etc/share/sops/keys - - "../../configuration/ssh:/home/root/.ssh:ro" - - "${SOP_REPO:-./../../../sops-repo}:/etc/share/vhosts/sops" + - ../../configuration/ssh:/home/root/.ssh:ro + - ${SOP_REPO:-./../../../sops-repo}:/etc/share/vhosts/sops - ../../configuration/ssl:/etc/share/certs - ../../logs:/global/log:ro - /var/run/docker.sock:/var/run/docker.sock From 02d7dc49f9fe12c44385f0600191394c9a87d8dd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:46:14 +0600 Subject: [PATCH 228/509] style(compose): normalize project volume mounts --- docker/compose/http.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 1b15ea93..da9c449b 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -10,7 +10,7 @@ services: - "${HTTP_PORT:-80}:80" - "${HTTPS_PORT:-443}:443" volumes: - - "${PROJECT_DIR:-./../../../application}:/app" + - ${PROJECT_DIR:-./../../../application}:/app - lds_nginx_host:/etc/nginx/conf.d - lds_ssl_keys:/etc/mkcert:ro - lds_ssl_roots:/etc/share/rootCA:ro @@ -30,7 +30,7 @@ services: environment: - TZ=${TZ:-} volumes: - - "${PROJECT_DIR:-./../../../application}:/app" + - ${PROJECT_DIR:-./../../../application}:/app - lds_ssl_keys:/etc/mkcert:ro - lds_apache_host:/usr/local/apache2/conf/vhosts:ro - lds_ssl_roots:/etc/share/rootCA:ro From 6d00544a6e8aad392b7f45cd0145a1bc1decba29 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 08:46:17 +0600 Subject: [PATCH 229/509] test(compose): enforce unquoted volume mount style --- tests/compose-contract.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index d7d4d351..fbb16b7b 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -8,6 +8,24 @@ source "$ROOT/tests/lib/assertions.sh" command -v docker >/dev/null 2>&1 || fail "docker is required" docker compose version >/dev/null 2>&1 || fail "docker compose plugin is required" +for compose_file in "$ROOT"/docker/compose/*.yaml; do + if awk ' + /^ volumes:[[:space:]]*$/ { in_service_volumes=1; next } + in_service_volumes && /^ - / { + entry=$0 + sub(/^[[:space:]]*-[[:space:]]*/, "", entry) + if (entry ~ /^["'\''"]/ ) exit 1 + next + } + in_service_volumes { in_service_volumes=0 } + ' "$compose_file"; then + : + else + fail "service volume mounts must use unquoted short-syntax scalars: $compose_file" + fi +done +pass "service volume mounts use one unquoted short-syntax style" + release_env="$ROOT/docker/release.env" user_env="$ROOT/docker/.env" backup_env="" From 6d73aa0b5b07370c1ee92df9b5c1c8d725068e47 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:20:45 +0600 Subject: [PATCH 230/509] chores --- docker/compose/companion.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 573c8c76..3e8f00ec 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -55,6 +55,7 @@ services: - datastore llm-sm: + container_name: LLM_SM hostname: llm-sm image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped From 304561616de9dd4cc76b0ba81fa5797145b4b3c2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:52:26 +0600 Subject: [PATCH 231/509] fix(ai): raise generation timeout to 30 minutes --- docker/compose/companion.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 3e8f00ec..2f7454a6 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -23,7 +23,7 @@ services: - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} - LDS_AI_CONNECT_TIMEOUT - LDS_AI_PREFLIGHT_TIMEOUT - - LDS_AI_TIMEOUT + - LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800} - LDS_AI_AVAILABILITY_TTL - LDS_AI_MAX_CONTEXT_BYTES - LDS_AI_MAX_REQUEST_BYTES From 6cca4ab1d0cf79d6670309b593faa5bd0725474a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:52:30 +0600 Subject: [PATCH 232/509] fix(ai): align LLM proxy timeout with generation timeout --- docker/compose/http.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index da9c449b..6960f8a1 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -6,6 +6,7 @@ services: restart: unless-stopped environment: - TZ=${TZ:-} + - LLM_PROXY_TIMEOUT_SECONDS=${LDS_AI_TIMEOUT:-1800} ports: - "${HTTP_PORT:-80}:80" - "${HTTPS_PORT:-443}:443" From 86b1094953fb7a212675412289d957daa0798838 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:55:02 +0600 Subject: [PATCH 233/509] fix(ai): remove fixed llm container name --- docker/compose/companion.yaml | 1 - 1 file changed, 1 deletion(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 2f7454a6..a91364e2 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -55,7 +55,6 @@ services: - datastore llm-sm: - container_name: LLM_SM hostname: llm-sm image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped From 61ff96d18a4d2a6aa7e0c788281673285e0ae818 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:55:06 +0600 Subject: [PATCH 234/509] test(ai): lock 30-minute consumer and proxy timeout --- tests/compose-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index fbb16b7b..02cc4609 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -167,6 +167,9 @@ assert tools["LDS_AI_ENABLED"] == "auto" assert tools["LDS_AI_PROVIDER"] == "ollama" assert tools["LDS_AI_URL"] == "http://llm-sm:11434" assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" +assert tools["LDS_AI_TIMEOUT"] == "1800" +nginx=d["services"]["nginx"]["environment"] +assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" From 2a19828874ecb523a8e5b70f027c8115a4515097 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:55:21 +0600 Subject: [PATCH 235/509] fix(ai): make Tools AI defaults deterministic --- docker/compose/companion.yaml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index a91364e2..ea5e2e8b 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -21,13 +21,13 @@ services: - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-ollama} - LDS_AI_URL=${LDS_AI_URL:-http://llm-sm:11434} - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} - - LDS_AI_CONNECT_TIMEOUT - - LDS_AI_PREFLIGHT_TIMEOUT + - LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2} + - LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5} - LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800} - - LDS_AI_AVAILABILITY_TTL - - LDS_AI_MAX_CONTEXT_BYTES - - LDS_AI_MAX_REQUEST_BYTES - - LDS_AI_MAX_RESPONSE_BYTES + - LDS_AI_AVAILABILITY_TTL=${LDS_AI_AVAILABILITY_TTL:-5} + - LDS_AI_MAX_CONTEXT_BYTES=${LDS_AI_MAX_CONTEXT_BYTES:-524288} + - LDS_AI_MAX_REQUEST_BYTES=${LDS_AI_MAX_REQUEST_BYTES:-1048576} + - LDS_AI_MAX_RESPONSE_BYTES=${LDS_AI_MAX_RESPONSE_BYTES:-2097152} volumes: - ${PROJECT_DIR:-./../../../application}:/app - lds_ssl_roots:/etc/share/rootCA From 960a6b27475953b152dd4294ff64febe68aa4555 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:55:25 +0600 Subject: [PATCH 236/509] test(ai): cover deterministic consumer defaults --- tests/compose-contract.sh | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 02cc4609..d9b4487a 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -167,7 +167,13 @@ assert tools["LDS_AI_ENABLED"] == "auto" assert tools["LDS_AI_PROVIDER"] == "ollama" assert tools["LDS_AI_URL"] == "http://llm-sm:11434" assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" +assert tools["LDS_AI_CONNECT_TIMEOUT"] == "2" +assert tools["LDS_AI_PREFLIGHT_TIMEOUT"] == "5" assert tools["LDS_AI_TIMEOUT"] == "1800" +assert tools["LDS_AI_AVAILABILITY_TTL"] == "5" +assert tools["LDS_AI_MAX_CONTEXT_BYTES"] == "524288" +assert tools["LDS_AI_MAX_REQUEST_BYTES"] == "1048576" +assert tools["LDS_AI_MAX_RESPONSE_BYTES"] == "2097152" nginx=d["services"]["nginx"]["environment"] assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" ' <<<"$ai_json" From d52a53b37e69d537d6c174e3f9ae9ddf1f395ea0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:55:58 +0600 Subject: [PATCH 237/509] docs(ai): document layered timeout defaults --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b2a4da9a..16b01791 100644 --- a/README.md +++ b/README.md @@ -305,7 +305,7 @@ Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where ap | `OLLAMA_KEEP_ALIVE` | `5m` | Model keep-alive | | `OLLAMA_NO_CLOUD` | `1` | Keep cloud integration disabled | -`LDS_AI_CONNECT_TIMEOUT`, `LDS_AI_PREFLIGHT_TIMEOUT`, `LDS_AI_TIMEOUT`, `LDS_AI_AVAILABILITY_TTL`, `LDS_AI_MAX_CONTEXT_BYTES`, `LDS_AI_MAX_REQUEST_BYTES`, and `LDS_AI_MAX_RESPONSE_BYTES` belong to the Tools consumer layer rather than the provider container. +`LDS_AI_CONNECT_TIMEOUT=2` and `LDS_AI_PREFLIGHT_TIMEOUT=5` intentionally fail fast. `LDS_AI_TIMEOUT=1800` gives local model loading/inference up to 30 minutes and is also forwarded to the dedicated Nginx LLM route as `LLM_PROXY_TIMEOUT_SECONDS`. `LDS_AI_AVAILABILITY_TTL=5`, `LDS_AI_MAX_CONTEXT_BYTES=524288`, `LDS_AI_MAX_REQUEST_BYTES=1048576`, and `LDS_AI_MAX_RESPONSE_BYTES=2097152` remain Tools consumer controls rather than provider-container limits. Use the `lds` wrapper for provider commands. LocalDevStack intentionally has no repository-root `compose.yml`; its effective Compose project is assembled from `docker/compose/main.yaml`, release/user env files, optional extras, and temporary runtime-specific overrides. Therefore a bare command such as `docker compose exec llm-sm ...` from the LocalDevStack root is not equivalent and fails before reaching the container. Use `lds llm ...` instead. From 8be35d3440e9eb5e68fcc07448a1d517c43288e9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:56:09 +0600 Subject: [PATCH 238/509] docs(ai): document layered timeout defaults From 2b59deddc8ccbf2850a74ceee8a5de2aa0afed88 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:56:18 +0600 Subject: [PATCH 239/509] docs(ai): define long-generation timeout contract --- docs/guides/local-ai.rst | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 067ef385..2f287809 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -206,13 +206,19 @@ PDF-vision limits remain active unless their own value is set to ``0``. Consumer settings are separate. The following values configure the Tools AI layer, not the provider runtime:: - LDS_AI_CONNECT_TIMEOUT - LDS_AI_PREFLIGHT_TIMEOUT - LDS_AI_TIMEOUT - LDS_AI_AVAILABILITY_TTL - LDS_AI_MAX_CONTEXT_BYTES - LDS_AI_MAX_REQUEST_BYTES - LDS_AI_MAX_RESPONSE_BYTES + LDS_AI_CONNECT_TIMEOUT=2 + LDS_AI_PREFLIGHT_TIMEOUT=5 + LDS_AI_TIMEOUT=1800 + LDS_AI_AVAILABILITY_TTL=5 + LDS_AI_MAX_CONTEXT_BYTES=524288 + LDS_AI_MAX_REQUEST_BYTES=1048576 + LDS_AI_MAX_RESPONSE_BYTES=2097152 + +Connection and provider/model preflight remain deliberately short. Generation and +analysis use a separate 30-minute default because first model load, CPU inference, and +larger local prompts can legitimately take much longer. LocalDevStack also forwards +``LDS_AI_TIMEOUT`` to Nginx as ``LLM_PROXY_TIMEOUT_SECONDS`` for the dedicated +``llm.localhost`` route so the edge proxy does not terminate a valid generation earlier. Privacy and Trust Boundaries ---------------------------- From e76cbc541eec9c8c90fa57ea33d0b6c0910beb60 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:56:26 +0600 Subject: [PATCH 240/509] docs(ai): document deterministic consumer timeout defaults --- docs/concepts/profiles-and-env.rst | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index d9633376..1edbe318 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -251,10 +251,19 @@ Provider-side options accepted in ``docker/.env`` include:: OLLAMA_KEEP_ALIVE=5m OLLAMA_NO_CLOUD=1 -Tools consumer settings remain separate and include -``LDS_AI_CONNECT_TIMEOUT``, ``LDS_AI_PREFLIGHT_TIMEOUT``, ``LDS_AI_TIMEOUT``, -``LDS_AI_AVAILABILITY_TTL``, ``LDS_AI_MAX_CONTEXT_BYTES``, -``LDS_AI_MAX_REQUEST_BYTES``, and ``LDS_AI_MAX_RESPONSE_BYTES``. +Tools consumer settings remain separate:: + + LDS_AI_CONNECT_TIMEOUT=2 + LDS_AI_PREFLIGHT_TIMEOUT=5 + LDS_AI_TIMEOUT=1800 + LDS_AI_AVAILABILITY_TTL=5 + LDS_AI_MAX_CONTEXT_BYTES=524288 + LDS_AI_MAX_REQUEST_BYTES=1048576 + LDS_AI_MAX_RESPONSE_BYTES=2097152 + +Only generation/analysis receives the long 1800-second default. Connect and preflight +checks stay fast. The same generation timeout is passed to Nginx as +``LLM_PROXY_TIMEOUT_SECONDS`` for the dedicated LLM proxy path. The LLM service itself is tracked only in ``docker/compose/companion.yaml``. There are no tracked AI runtime-variant YAML files; ``lds`` creates temporary fragments under From 7ecdd3ebb1ca59b29371c83c54db9f41bf0bd3b7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:56:34 +0600 Subject: [PATCH 241/509] docs(plan): align AI timeout layers --- .../07-localdevstack-integration-plan.md | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 0c3d8e3a..bc28f88c 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -692,13 +692,19 @@ Allow user overrides. Also pass through supported advanced limits only when the user sets them: -- `LDS_AI_CONNECT_TIMEOUT`; -- `LDS_AI_PREFLIGHT_TIMEOUT`; -- `LDS_AI_TIMEOUT`; -- `LDS_AI_AVAILABILITY_TTL`; -- `LDS_AI_MAX_CONTEXT_BYTES`; -- `LDS_AI_MAX_REQUEST_BYTES`; -- `LDS_AI_MAX_RESPONSE_BYTES`. +- `LDS_AI_CONNECT_TIMEOUT=2`; +- `LDS_AI_PREFLIGHT_TIMEOUT=5`; +- `LDS_AI_TIMEOUT=1800`; +- `LDS_AI_AVAILABILITY_TTL=5`; +- `LDS_AI_MAX_CONTEXT_BYTES=524288`; +- `LDS_AI_MAX_REQUEST_BYTES=1048576`; +- `LDS_AI_MAX_RESPONSE_BYTES=2097152`. + +Keep connection and preflight bounds short, but give generation/analysis a 30-minute +default. Forward `LDS_AI_TIMEOUT` to the Nginx container as +`LLM_PROXY_TIMEOUT_SECONDS` so the dedicated `llm.localhost` route has the same +long-running request budget. Do not reintroduce a shorter independent UI/process timeout +for Admin AI analysis. Why LocalDevStack should default `LDS_AI_MODEL=qwen2.5:3b`: From c9b500ea819f60e6361c142e8032c69f19e9bab8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 09:56:55 +0600 Subject: [PATCH 242/509] test(ai): lock timeout layering --- tests/ai-contract.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 77e79e31..8fec98ca 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -60,6 +60,10 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollam assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800}' +assert_file_contains "$ROOT/docker/compose/http.yaml" 'LLM_PROXY_TIMEOUT_SECONDS=${LDS_AI_TIMEOUT:-1800}' assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" From abbc227bd29dfb9ec50a1a25770c7e051abbdb88 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:00:23 +0600 Subject: [PATCH 243/509] fix(ai): preserve LLM_SM container name --- docker/compose/companion.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index ea5e2e8b..17097a45 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -55,6 +55,7 @@ services: - datastore llm-sm: + container_name: LLM_SM hostname: llm-sm image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped From c6e9d8b4c42481428b9942b58aea416aae0364df Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:00:26 +0600 Subject: [PATCH 244/509] test(ai): preserve fixed LLM container name --- tests/compose-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index d9b4487a..10e9733e 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -141,7 +141,7 @@ import json,sys d=json.load(sys.stdin) s=d["services"]["llm-sm"] assert s["image"] == "infocyph/llm-sm:latest" -assert "container_name" not in s +assert s["container_name"] == "LLM_SM" assert not s.get("ports") assert set(s["networks"]) == {"frontend","backend"} targets={v["target"] for v in s["volumes"]} From ab6395f1ef40186085e31b0efcdfd49f50ff6157 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:00:30 +0600 Subject: [PATCH 245/509] docs(plan): preserve LLM_SM single-stack compatibility --- .../docker-ecosystem/07-localdevstack-integration-plan.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index bc28f88c..5f69c387 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -553,6 +553,7 @@ Base service: ```yaml services: llm-sm: + container_name: LLM_SM image: infocyph/llm-sm:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] @@ -566,8 +567,9 @@ services: Important rules: - service key must be exactly `llm-sm`; +- preserve the existing fixed container name `LLM_SM` for single-stack compatibility; +- route internally by the Compose service/hostname `llm-sm`, not by the fixed container name; - do not set a fixed IP; -- do not require `container_name`; - do not mount Docker socket; - do not expose `11434` to all interfaces; - do not add a host port in the base companion service; @@ -2235,6 +2237,7 @@ with: ```yaml llm-sm: + container_name: LLM_SM image: infocyph/llm-sm:${LDS_LLM_ARCH} profiles: [ai] environment: From 1e22f21279ee6face4ff271634db2e4cb85a65b4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:00:50 +0600 Subject: [PATCH 246/509] docs(ai): document persistent LLM_SM container name --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 16b01791..8d0432d0 100644 --- a/README.md +++ b/README.md @@ -40,7 +40,7 @@ Host └─ logs/ ``` -LocalDevStack uses three logical Docker networks: `Frontend`, `Backend`, and `DataStore`. Docker assigns their address ranges dynamically. Internal communication uses service names such as `nginx`, `postgres`, `redis`, `server-tools`, and `llm-sm`. +LocalDevStack uses three logical Docker networks: `Frontend`, `Backend`, and `DataStore`. Docker assigns their address ranges dynamically. Internal communication uses service names such as `nginx`, `postgres`, `redis`, `server-tools`, and `llm-sm`. For single-stack compatibility the LLM service keeps the fixed container name `LLM_SM`; internal routing still uses the Compose service/hostname `llm-sm`, never the fixed container name. ## Prerequisites From dcd5569be9f912fa9ca7d46c69d72e964e25a272 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:00:53 +0600 Subject: [PATCH 247/509] docs(ai): preserve fixed LLM_SM compatibility name --- docs/guides/local-ai.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 2f287809..09622b14 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -93,6 +93,10 @@ The tracked provider service exists only in:: docker/compose/companion.yaml +The Compose service key/hostname is ``llm-sm`` and the fixed container name remains +``LLM_SM`` for LocalDevStack single-stack compatibility. Internal consumers must continue +to route through ``llm-sm`` rather than depending on the container name. + and is included through:: docker/compose/main.yaml From 09b3b1eac19d517dd26c8f063b6fe47f31aca734 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:00:57 +0600 Subject: [PATCH 248/509] docs(ai): record LLM_SM naming contract --- docs/concepts/architecture.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index c8e032b7..8ec2bd79 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -150,7 +150,9 @@ When the ``ai`` profile is enabled: 5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-sm`` CLI. The provider is one ``llm-sm`` service declared in -``docker/compose/companion.yaml`` and enabled only by the ``ai`` profile. Its image +``docker/compose/companion.yaml`` and enabled only by the ``ai`` profile. The service +keeps ``container_name: LLM_SM`` for current single-stack compatibility, while all +internal routing continues to use the Compose service/hostname ``llm-sm``. Its image is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. ``LDS_LLM_ARCH`` is derived from the effective runtime rather than maintained as an independent version selector. From 341846dc086351a413612fd3fdf1b9a9a83dd3d7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:01:01 +0600 Subject: [PATCH 249/509] test(ai): lock persistent LLM_SM container name --- tests/ai-contract.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 8fec98ca..3c23b4a8 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -57,6 +57,7 @@ fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_SM' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24}' From f48535d97643ac9ccf38c708a95ffef811d8cf29 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:05:08 +0600 Subject: [PATCH 250/509] test(ai): propagate custom generation timeout --- tests/compose-contract.sh | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 10e9733e..434db351 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -179,17 +179,21 @@ assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" -printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_SM_PDF_MAX_PAGES=12' 'LLM_SM_SYSTEM=Answer briefly.' >>"$user_env" +printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_SM_PDF_MAX_PAGES=12' 'LLM_SM_SYSTEM=Answer briefly.' 'LDS_AI_TIMEOUT=2400' >>"$user_env" ai_override_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' import json,sys -s=json.load(sys.stdin)["services"]["llm-sm"] -env=s["environment"] -assert env["LLM_SM_MODEL"] == "qwen2.5:1.5b" -assert env["LLM_SM_PDF_MAX_PAGES"] == "12" -assert env["LLM_SM_SYSTEM"] == "Answer briefly." +d=json.load(sys.stdin) +llm=d["services"]["llm-sm"]["environment"] +assert llm["LLM_SM_MODEL"] == "qwen2.5:1.5b" +assert llm["LLM_SM_PDF_MAX_PAGES"] == "12" +assert llm["LLM_SM_SYSTEM"] == "Answer briefly." +tools=d["services"]["server-tools"]["environment"] +assert tools["LDS_AI_TIMEOUT"] == "2400" +nginx=d["services"]["nginx"]["environment"] +assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "2400" ' <<<"$ai_override_json" -pass "LocalDevStack forwards configured model and provider options into llm-sm" +pass "LocalDevStack forwards configured provider and generation-timeout options" amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' From 1738d87179dd96ab666f2a5e8a6e66affbd4a804 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:45:18 +0600 Subject: [PATCH 251/509] feat(ai): add host Graphify workflow command --- lib/ai.sh | 68 +++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) diff --git a/lib/ai.sh b/lib/ai.sh index cd0c59f0..66a39a2b 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -29,6 +29,74 @@ cmd_ai() { esac } +_graphify_local_base_url() { + local ctr published host_port + ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" + [[ -n "$ctr" ]] || + die "llm-sm is not running. Enable the ai profile and start the stack first." + + docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || + die "llm-sm container exists but is not running." + + published="$( + docker inspect -f '{{with (index .NetworkSettings.Ports "11434/tcp")}}{{(index . 0).HostPort}}{{end}}' "$ctr" 2>/dev/null || true + )" + [[ "$published" =~ ^[0-9]+$ ]] || + die "Graphify needs the llm-sm loopback API. Run: lds llm host-port on && lds up -d llm-sm" + + host_port="$published" + printf 'http://127.0.0.1:%s/v1' "$host_port" +} + +cmd_graphify() { + need_bin graphify "install the Graphify CLI on the host first" + + local target="${1:-.}" + [[ $# -eq 0 ]] || shift + [[ -e "$target" ]] || die "Graphify target does not exist: $target" + + local base_url timeout model graphify_bin arg next_is_model=0 + base_url="${OLLAMA_BASE_URL:-$(_graphify_local_base_url)}" + timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" + model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen2.5:3b)}" + + [[ "$timeout" =~ ^[0-9]+$ ]] && ((timeout >= 1)) || + die "GRAPHIFY_API_TIMEOUT must be a positive integer" + + # Keep an explicit --model override consistent across extraction and clustering. + for arg in "$@"; do + if ((next_is_model)); then + model="$arg" + next_is_model=0 + continue + fi + case "$arg" in + --model) next_is_model=1 ;; + --model=*) model="${arg#--model=}" ;; + --backend | --backend=*) + die "lds graphify owns --backend=ollama; do not pass --backend" + ;; + --no-cluster) + die "lds graphify already separates extraction and clustering; do not pass --no-cluster" + ;; + esac + done + ((next_is_model == 0)) || die "--model requires a value" + [[ -n "$model" ]] || die "Graphify model cannot be empty" + + graphify_bin="$(bin_path graphify)" + + OLLAMA_BASE_URL="$base_url" \ + OLLAMA_MODEL="$model" \ + GRAPHIFY_API_TIMEOUT="$timeout" \ + "$graphify_bin" extract "$target" --backend ollama --no-cluster "$@" + + OLLAMA_BASE_URL="$base_url" \ + OLLAMA_MODEL="$model" \ + GRAPHIFY_API_TIMEOUT="$timeout" \ + "$graphify_bin" cluster-only "$target" --backend ollama +} + _llm_exec() { local ctr ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" From 3bf0599250a8487ee1a51ee91b0f12205bae23fe Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:45:37 +0600 Subject: [PATCH 252/509] fix(ai): keep Graphify phase overrides consistent --- lib/ai.sh | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 66a39a2b..92d7b2e0 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -55,24 +55,28 @@ cmd_graphify() { [[ $# -eq 0 ]] || shift [[ -e "$target" ]] || die "Graphify target does not exist: $target" - local base_url timeout model graphify_bin arg next_is_model=0 + local base_url timeout model graphify_bin arg next_is_model=0 next_is_timeout=0 base_url="${OLLAMA_BASE_URL:-$(_graphify_local_base_url)}" timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen2.5:3b)}" - [[ "$timeout" =~ ^[0-9]+$ ]] && ((timeout >= 1)) || - die "GRAPHIFY_API_TIMEOUT must be a positive integer" - - # Keep an explicit --model override consistent across extraction and clustering. + # Keep explicit model/timeout overrides consistent across extraction and clustering. for arg in "$@"; do if ((next_is_model)); then model="$arg" next_is_model=0 continue fi + if ((next_is_timeout)); then + timeout="$arg" + next_is_timeout=0 + continue + fi case "$arg" in --model) next_is_model=1 ;; --model=*) model="${arg#--model=}" ;; + --api-timeout) next_is_timeout=1 ;; + --api-timeout=*) timeout="${arg#--api-timeout=}" ;; --backend | --backend=*) die "lds graphify owns --backend=ollama; do not pass --backend" ;; @@ -82,7 +86,10 @@ cmd_graphify() { esac done ((next_is_model == 0)) || die "--model requires a value" + ((next_is_timeout == 0)) || die "--api-timeout requires a value" [[ -n "$model" ]] || die "Graphify model cannot be empty" + [[ "$timeout" =~ ^[0-9]+$ ]] && ((timeout >= 1)) || + die "GRAPHIFY_API_TIMEOUT/--api-timeout must be a positive integer" graphify_bin="$(bin_path graphify)" From b215905ed4a983ed3de58f461a2b722827d8523b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:46:34 +0600 Subject: [PATCH 253/509] docs(cli): expose top-level Graphify workflow --- lds | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/lds b/lds index 1d140e4b..63c2598b 100755 --- a/lds +++ b/lds @@ -767,6 +767,9 @@ cmd_help() { ## AI consumer - `lds ai status|ask|explain|troubleshoot|review|repo-review|graphify ...` +## Host Graphify workflow +- `lds graphify [path] [graphify-extract-options...]` + ## LLM provider - `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit|ollama|api|version ...` - `lds llm runtime [cpu|nvidia|amd]` @@ -844,6 +847,7 @@ ${CYAN}Tools / Secrets:${NC} ${CYAN}AI:${NC} ai status|ask|explain|troubleshoot|review|repo-review|graphify + graphify [path] [graphify-extract-options...] llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit|ollama|api|version llm runtime [cpu|nvidia|amd] llm host-port From 0ed7e812401d1a8c66cc781190c12099f56be723 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:46:45 +0600 Subject: [PATCH 254/509] docs(ai): add host Graphify workflow --- README.md | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/README.md b/README.md index 8d0432d0..ae008bb1 100644 --- a/README.md +++ b/README.md @@ -261,6 +261,10 @@ lds ai troubleshoot ... lds ai review ... lds ai repo-review ... +# Host Graphify + LocalDevStack Ollama +lds graphify +lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 + lds llm models lds llm pull lds llm show @@ -270,6 +274,18 @@ lds llm chat ... Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. Change that loopback port with `LLM_SM_PORT` in `docker/.env`. +`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets the actual loopback-published `llm-sm` API. The command runs extraction with `--backend ollama --no-cluster`, then runs `cluster-only` for the same path if extraction succeeds. This avoids clustering twice. + +Before first use, publish the provider port and apply that Compose change: + +```bash +lds llm host-port on +lds up -d llm-sm +lds graphify +``` + +You may override `OLLAMA_BASE_URL`, `OLLAMA_MODEL`, or `GRAPHIFY_API_TIMEOUT` for a one-off run. A CLI `--model` or `--api-timeout` override is kept consistent across both Graphify phases. + The LocalDevStack LLM Compose layout is intentionally small: ```text From 7c5ac7cfaf9e649fcf67f047c61eb9fdb3988775 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:46:55 +0600 Subject: [PATCH 255/509] docs(ai): document top-level Graphify workflow --- docs/reference/cli.rst | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index b8168595..c0a694e5 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -197,6 +197,32 @@ AI Consumer ``status`` maps to the Tools provider-status flow. +Host Graphify Workflow +---------------------- + +:: + + lds graphify + lds graphify ./your-project + lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 + +This command runs the host ``graphify`` CLI against LocalDevStack's Ollama provider. +It performs ``extract --backend ollama --no-cluster`` followed by +``cluster-only --backend ollama``, so clustering happens once. + +By default it derives: + +- ``OLLAMA_BASE_URL`` from the actual loopback-published ``llm-sm`` port; +- ``OLLAMA_MODEL`` from ``LDS_AI_MODEL``; +- ``GRAPHIFY_API_TIMEOUT`` from ``LDS_AI_TIMEOUT``. + +The host port must already be applied with:: + + lds llm host-port on + lds up -d llm-sm + +An explicitly supplied ``OLLAMA_BASE_URL`` bypasses that host-port lookup. + LLM Provider ------------ From a68692167592cdef05289f67a1a7f595e46c9f2a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:47:10 +0600 Subject: [PATCH 256/509] docs(ai): distinguish Graphify execution and analysis --- docs/guides/local-ai.rst | 32 +++++++++++++++++++++++++++++--- 1 file changed, 29 insertions(+), 3 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 09622b14..5c41bb82 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -266,9 +266,35 @@ container. The upstream ``llm-sm`` CLI also supports stdin-based flows such as Graphify -------- -``lds ai graphify`` delegates Graphify-assisted analysis to the Tools AI layer. -``llm-sm`` remains only the model provider and does not gain direct repository access -from this integration. +Two Graphify flows are intentionally separate. + +``lds ai graphify`` analyzes an already-produced Graphify output file through the Tools +AI layer. + +``lds graphify [path] [extract-options...]`` runs the host Graphify CLI against the +LocalDevStack Ollama provider. The default path is the current directory. It derives the +model from ``LDS_AI_MODEL``, derives ``GRAPHIFY_API_TIMEOUT`` from +``LDS_AI_TIMEOUT``, and uses the actual loopback-published provider port unless +``OLLAMA_BASE_URL`` is already set. + +Enable and apply the direct provider port before the default workflow:: + + lds llm host-port on + lds up -d llm-sm + lds graphify + +Internally the workflow runs extraction with ``--backend ollama --no-cluster`` and, +only after a successful extraction, runs ``cluster-only`` against the same path. This +preserves the requested two-phase flow without performing the default extraction +clustering and then immediately clustering a second time. + +Example:: + + lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 + +A one-off ``--model`` or ``--api-timeout`` option is propagated to the clustering +phase through the corresponding Graphify environment value. ``llm-sm`` remains only +the model provider and still receives no repository bind mount. Diagnostics ----------- From 0fe93079e53edeb82f1507f95a94ed589085e9a8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:47:19 +0600 Subject: [PATCH 257/509] test(cli): cover host Graphify workflow --- tests/cli-contract.sh | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index 9d46e7c5..cf66e81f 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -38,6 +38,30 @@ chmod +x "$tmpbin/docker" stack_help="$(PATH="$tmpbin:$PATH" "$ROOT/lds" stack help)" assert_contains "$stack_help" "LocalDevStack" pass "grouped stack help routing" +assert_contains "$help_output" "graphify [path]" +assert_contains "$markdown_output" "lds graphify" +pass "Graphify workflow help" + +graphify_log="$(mktemp)" +cat >"$tmpbin/graphify" <<'SH' +#!/usr/bin/env sh +printf 'base=%s model=%s timeout=%s args=%s\n' "$OLLAMA_BASE_URL" "$OLLAMA_MODEL" "$GRAPHIFY_API_TIMEOUT" "$*" >>"$GRAPHIFY_TEST_LOG" +SH +chmod +x "$tmpbin/graphify" + +PATH="$tmpbin:$PATH" \ +OLLAMA_BASE_URL=http://127.0.0.1:11434/v1 \ +LDS_AI_MODEL=qwen2.5:3b \ +LDS_AI_TIMEOUT=1800 \ +GRAPHIFY_TEST_LOG="$graphify_log" \ + "$ROOT/lds" graphify . --model test-model --api-timeout 42 --mode deep >/dev/null + +grep -Fq 'base=http://127.0.0.1:11434/v1 model=test-model timeout=42 args=extract . --backend ollama --no-cluster --model test-model --api-timeout 42 --mode deep' "$graphify_log" || + fail "Graphify extract wrapper contract failed" +grep -Fq 'base=http://127.0.0.1:11434/v1 model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || + fail "Graphify cluster wrapper contract failed" +rm -f "$graphify_log" +pass "Graphify host workflow wrapper" assert_file_contains "$ROOT/lds" 'exec "$DIR/bin/tool-runner" "$cmd" "$@"' pass "unknown command fallback remains delegated to tool-runner" From 484ad6a18a633a71832015b9e2f4853979208207 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:47:28 +0600 Subject: [PATCH 258/509] test(ai): lock Graphify Ollama workflow --- tests/ai-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 3c23b4a8..39af1d25 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -65,6 +65,9 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_CONNECT_TIMEO assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800}' assert_file_contains "$ROOT/docker/compose/http.yaml" 'LLM_PROXY_TIMEOUT_SECONDS=${LDS_AI_TIMEOUT:-1800}' +assert_file_contains "$ROOT/lib/ai.sh" 'cmd_graphify()' +assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" extract "$target" --backend ollama --no-cluster' +assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" cluster-only "$target" --backend ollama' assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" From b9a3739380550632e6fe2f1104ad95b626dcaca9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:47:56 +0600 Subject: [PATCH 259/509] docs(plan): add LDS Graphify host workflow --- .../07-localdevstack-integration-plan.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 5f69c387..d1e85d57 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -802,11 +802,18 @@ Do not install Graphify into `llm-sm` or Tools solely for this integration. Supported patterns: -- host Graphify -> `https://llm.localhost/v1`; +- `lds graphify [path] [extract-options...]` -> host Graphify workflow against the + loopback-published provider, using `LDS_AI_MODEL` and `LDS_AI_TIMEOUT` by default; +- explicit host Graphify may use `https://llm.localhost/v1` or an overridden + `OLLAMA_BASE_URL`; - container Graphify on a shared network -> `http://llm-sm:11434/v1`; - Tools `aiops graphify --file ` -> analyze an explicitly supplied Graphify output file. -Graphify remains an external consumer/tool. +The `lds graphify` workflow runs `extract --backend ollama --no-cluster` followed by +`cluster-only --backend ollama` so the requested two-phase flow clusters +once rather than re-clustering immediately after the extraction command's default +clustering pass. Graphify remains a host/external consumer and is not installed into +`llm-sm` or Tools. ## 8.10 AI admin panel From 6546fb0b810e19ffa212863644318dbb3026e0dd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:52:50 +0600 Subject: [PATCH 260/509] fix(ai): set local Graphify Ollama API key --- lib/ai.sh | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/ai.sh b/lib/ai.sh index 92d7b2e0..e33f98a4 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -55,10 +55,11 @@ cmd_graphify() { [[ $# -eq 0 ]] || shift [[ -e "$target" ]] || die "Graphify target does not exist: $target" - local base_url timeout model graphify_bin arg next_is_model=0 next_is_timeout=0 + local base_url timeout model api_key graphify_bin arg next_is_model=0 next_is_timeout=0 base_url="${OLLAMA_BASE_URL:-$(_graphify_local_base_url)}" timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen2.5:3b)}" + api_key="${OLLAMA_API_KEY:-local}" # Keep explicit model/timeout overrides consistent across extraction and clustering. for arg in "$@"; do @@ -94,11 +95,13 @@ cmd_graphify() { graphify_bin="$(bin_path graphify)" OLLAMA_BASE_URL="$base_url" \ + OLLAMA_API_KEY="$api_key" \ OLLAMA_MODEL="$model" \ GRAPHIFY_API_TIMEOUT="$timeout" \ "$graphify_bin" extract "$target" --backend ollama --no-cluster "$@" OLLAMA_BASE_URL="$base_url" \ + OLLAMA_API_KEY="$api_key" \ OLLAMA_MODEL="$model" \ GRAPHIFY_API_TIMEOUT="$timeout" \ "$graphify_bin" cluster-only "$target" --backend ollama From 694f4042a79aeac935cc4282fcb61b0510afa801 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:52:56 +0600 Subject: [PATCH 261/509] test(cli): assert local Graphify API key --- tests/cli-contract.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index cf66e81f..1e11eda0 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -45,7 +45,7 @@ pass "Graphify workflow help" graphify_log="$(mktemp)" cat >"$tmpbin/graphify" <<'SH' #!/usr/bin/env sh -printf 'base=%s model=%s timeout=%s args=%s\n' "$OLLAMA_BASE_URL" "$OLLAMA_MODEL" "$GRAPHIFY_API_TIMEOUT" "$*" >>"$GRAPHIFY_TEST_LOG" +printf 'base=%s key=%s model=%s timeout=%s args=%s\n' "$OLLAMA_BASE_URL" "$OLLAMA_API_KEY" "$OLLAMA_MODEL" "$GRAPHIFY_API_TIMEOUT" "$*" >>"$GRAPHIFY_TEST_LOG" SH chmod +x "$tmpbin/graphify" @@ -56,9 +56,9 @@ LDS_AI_TIMEOUT=1800 \ GRAPHIFY_TEST_LOG="$graphify_log" \ "$ROOT/lds" graphify . --model test-model --api-timeout 42 --mode deep >/dev/null -grep -Fq 'base=http://127.0.0.1:11434/v1 model=test-model timeout=42 args=extract . --backend ollama --no-cluster --model test-model --api-timeout 42 --mode deep' "$graphify_log" || +grep -Fq 'base=http://127.0.0.1:11434/v1 key=local model=test-model timeout=42 args=extract . --backend ollama --no-cluster --model test-model --api-timeout 42 --mode deep' "$graphify_log" || fail "Graphify extract wrapper contract failed" -grep -Fq 'base=http://127.0.0.1:11434/v1 model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || +grep -Fq 'base=http://127.0.0.1:11434/v1 key=local model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || fail "Graphify cluster wrapper contract failed" rm -f "$graphify_log" pass "Graphify host workflow wrapper" From f471f1153372af930538aca74326e13a4eaed223 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:52:59 +0600 Subject: [PATCH 262/509] test(ai): lock Graphify local API key --- tests/ai-contract.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 39af1d25..5e088d34 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -66,6 +66,7 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PREFLIGHT_TIM assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800}' assert_file_contains "$ROOT/docker/compose/http.yaml" 'LLM_PROXY_TIMEOUT_SECONDS=${LDS_AI_TIMEOUT:-1800}' assert_file_contains "$ROOT/lib/ai.sh" 'cmd_graphify()' +assert_file_contains "$ROOT/lib/ai.sh" 'OLLAMA_API_KEY="$api_key"' assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" extract "$target" --backend ollama --no-cluster' assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" cluster-only "$target" --backend ollama' assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" From 64218ce65c8e72485476e1e57410811b85d9c433 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:53:02 +0600 Subject: [PATCH 263/509] docs(ai): explain Graphify local API key --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index ae008bb1..084798a5 100644 --- a/README.md +++ b/README.md @@ -284,7 +284,9 @@ lds up -d llm-sm lds graphify ``` -You may override `OLLAMA_BASE_URL`, `OLLAMA_MODEL`, or `GRAPHIFY_API_TIMEOUT` for a one-off run. A CLI `--model` or `--api-timeout` override is kept consistent across both Graphify phases. +`lds graphify` also sets `OLLAMA_API_KEY=local` when no key is supplied. Ollama does not require authentication for this local loopback endpoint; the non-empty placeholder only satisfies Graphify's Ollama-backend warning check. An explicitly supplied `OLLAMA_API_KEY` is preserved. + +You may override `OLLAMA_BASE_URL`, `OLLAMA_API_KEY`, `OLLAMA_MODEL`, or `GRAPHIFY_API_TIMEOUT` for a one-off run. A CLI `--model` or `--api-timeout` override is kept consistent across both Graphify phases. The LocalDevStack LLM Compose layout is intentionally small: From 2f0208494f3d62527d27ce4cef56b603a6009b80 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:53:06 +0600 Subject: [PATCH 264/509] docs(ai): document Graphify API key placeholder --- docs/guides/local-ai.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 5c41bb82..f07dac83 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -292,6 +292,11 @@ Example:: lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 +``lds graphify`` sets ``OLLAMA_API_KEY=local`` when no key is supplied. Ollama does +not require authentication on this loopback endpoint; the non-empty placeholder only +suppresses Graphify's Ollama-backend warning. An explicitly supplied +``OLLAMA_API_KEY`` is preserved. + A one-off ``--model`` or ``--api-timeout`` option is propagated to the clustering phase through the corresponding Graphify environment value. ``llm-sm`` remains only the model provider and still receives no repository bind mount. From aaf44181af15c21b41027cda31cd5bde5def8e2f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 10:59:25 +0600 Subject: [PATCH 265/509] refactor(ai): scope Graphify environment once --- lib/ai.sh | 20 +++++++++----------- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index e33f98a4..64608987 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -94,17 +94,15 @@ cmd_graphify() { graphify_bin="$(bin_path graphify)" - OLLAMA_BASE_URL="$base_url" \ - OLLAMA_API_KEY="$api_key" \ - OLLAMA_MODEL="$model" \ - GRAPHIFY_API_TIMEOUT="$timeout" \ - "$graphify_bin" extract "$target" --backend ollama --no-cluster "$@" - - OLLAMA_BASE_URL="$base_url" \ - OLLAMA_API_KEY="$api_key" \ - OLLAMA_MODEL="$model" \ - GRAPHIFY_API_TIMEOUT="$timeout" \ - "$graphify_bin" cluster-only "$target" --backend ollama + ( + export OLLAMA_BASE_URL="$base_url" + export OLLAMA_API_KEY="$api_key" + export OLLAMA_MODEL="$model" + export GRAPHIFY_API_TIMEOUT="$timeout" + + "$graphify_bin" extract "$target" --backend ollama --no-cluster "$@" && + "$graphify_bin" cluster-only "$target" --backend ollama + ) } _llm_exec() { From b3d67bd3c179976537bc5e2ae6089b54dce1806b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 19:59:22 +0600 Subject: [PATCH 266/509] feat(ai): detect AMD CPU iGPU preference --- lib/platform.sh | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/lib/platform.sh b/lib/platform.sh index f2f5a473..dce320a9 100644 --- a/lib/platform.sh +++ b/lib/platform.sh @@ -23,6 +23,28 @@ detect_ai_runtime() { printf '%s' cpu } +host_cpu_is_amd() { + if [[ -r /proc/cpuinfo ]] && + grep -qiE '^[[:space:]]*vendor_id[[:space:]]*:[[:space:]]*AuthenticAMD([[:space:]]|$)' /proc/cpuinfo; then + return 0 + fi + + if has_cmd lscpu && + lscpu 2>/dev/null | grep -qiE '^Vendor ID:[[:space:]]*AuthenticAMD([[:space:]]|$)'; then + return 0 + fi + + return 1 +} + +ai_igpu_default_for_runtime() { + if [[ "${1,,}" == "amd" ]] && host_cpu_is_amd; then + printf '%s' 1 + else + printf '%s' 0 + fi +} + llm_arch_for_runtime() { case "${1,,}" in amd) printf '%s' amd-latest ;; From 7dadfb0b0842b373d5c248fd02200657e2b1374f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 19:59:35 +0600 Subject: [PATCH 267/509] feat(ai): persist AMD iGPU preference during setup --- lib/certificates.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/lib/certificates.sh b/lib/certificates.sh index 13d0c440..743ee0b3 100644 --- a/lib/certificates.sh +++ b/lib/certificates.sh @@ -433,7 +433,7 @@ uninstall_ca() { add_required_env() { update_env "$ENV_DOCKER" WORKING_DIR "$DIR" - local ai_runtime + local ai_runtime ai_igpu_enable ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" if [[ -z "$ai_runtime" ]]; then ai_runtime="$(detect_ai_runtime)" @@ -441,6 +441,12 @@ add_required_env() { fi update_env "$ENV_DOCKER" LDS_LLM_ARCH "$(llm_arch_for_runtime "$ai_runtime")" + ai_igpu_enable="$(compose_control_value LDS_AI_IGPU_ENABLE "")" + if [[ -z "$ai_igpu_enable" ]]; then + ai_igpu_enable="$(ai_igpu_default_for_runtime "$ai_runtime")" + update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$ai_igpu_enable" + fi + ((EUID == 0)) && return 0 update_env "$ENV_DOCKER" USER "$(id -un)" update_env "$ENV_DOCKER" UID "$(id -u)" From 0f491efd407c87598def65af8cd2850d8f2008f5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 19:59:47 +0600 Subject: [PATCH 268/509] feat(ai): align qwen3 default and AMD iGPU runtime state --- lib/ai.sh | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 64608987..563a9c45 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -58,7 +58,7 @@ cmd_graphify() { local base_url timeout model api_key graphify_bin arg next_is_model=0 next_is_timeout=0 base_url="${OLLAMA_BASE_URL:-$(_graphify_local_base_url)}" timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" - model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen2.5:3b)}" + model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen3:14b)}" api_key="${OLLAMA_API_KEY:-local}" # Keep explicit model/timeout overrides consistent across extraction and clustering. @@ -130,12 +130,14 @@ cmd_llm() { fi case "${mode,,}" in cpu | nvidia | amd) - local normalized arch + local normalized arch igpu_enable normalized="${mode,,}" arch="$(llm_arch_for_runtime "$normalized")" + igpu_enable="$(ai_igpu_default_for_runtime "$normalized")" update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" update_env "$ENV_DOCKER" LDS_LLM_ARCH "$arch" - ok "LLM runtime set to $normalized (infocyph/llm-sm:$arch). Recreate llm-sm to apply the change." + update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable" + ok "LLM runtime set to $normalized (infocyph/llm-sm:$arch, iGPU=$igpu_enable). Recreate llm-sm to apply the change." ;; *) die "llm runtime " ;; esac From f98a8f74ce9a811e853f9f8ff4e901e4545c3741 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 19:59:59 +0600 Subject: [PATCH 269/509] feat(ai): forward AMD iGPU enablement to Ollama --- docker/compose/companion.yaml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 17097a45..dedf4fa4 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -20,7 +20,7 @@ services: - LDS_AI_ENABLED=${LDS_AI_ENABLED:-auto} - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-ollama} - LDS_AI_URL=${LDS_AI_URL:-http://llm-sm:11434} - - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} + - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen3:14b} - LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2} - LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5} - LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800} @@ -61,7 +61,7 @@ services: restart: unless-stopped profiles: [ai] environment: - - LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b} + - LLM_SM_MODEL=${LDS_AI_MODEL:-qwen3:14b} - LLM_SM_SYSTEM=${LLM_SM_SYSTEM:-} - LLM_SM_INPUT_WARN_BYTES=${LLM_SM_INPUT_WARN_BYTES:-1048576} - LLM_SM_INPUT_MAX_BYTES=${LLM_SM_INPUT_MAX_BYTES:-0} @@ -71,6 +71,7 @@ services: - LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24} - LLM_SM_PDF_DPI=${LLM_SM_PDF_DPI:-120} - LLM_SM_ALLOW_LARGE_INPUT=${LLM_SM_ALLOW_LARGE_INPUT:-0} + - OLLAMA_IGPU_ENABLE=${LDS_AI_IGPU_ENABLE:-0} - OLLAMA_NUM_PARALLEL=${OLLAMA_NUM_PARALLEL:-1} - OLLAMA_MAX_LOADED_MODELS=${OLLAMA_MAX_LOADED_MODELS:-1} - OLLAMA_KEEP_ALIVE=${OLLAMA_KEEP_ALIVE:-5m} From 91f2e76eb07aca5a03581d40d9330bc3b10bcef1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:00:25 +0600 Subject: [PATCH 270/509] feat(ai): default profile model to qwen3 14b --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index f307d340..dd1ac952 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.3|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm-sm|LDS_LLM_ARCH|LDS_AI_MODEL=qwen2.5:3b;LDS_LLM_HOST_PORT=no|Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd +AI|ai|Local AI|llm-sm|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b;LDS_LLM_HOST_PORT=no|Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd From 111eef6a142c70d606ea04083a746d7d6b1f180c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:00:42 +0600 Subject: [PATCH 271/509] test(ai): make fake Ollama advertise qwen3 14b --- tests/fixtures/fake-ollama/server.py | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/fixtures/fake-ollama/server.py b/tests/fixtures/fake-ollama/server.py index 9b7baf72..e0bff7e9 100644 --- a/tests/fixtures/fake-ollama/server.py +++ b/tests/fixtures/fake-ollama/server.py @@ -24,8 +24,8 @@ def do_GET(self): { "models": [ { - "name": "qwen2.5:3b", - "model": "qwen2.5:3b", + "name": "qwen3:14b", + "model": "qwen3:14b", "size": 1, "digest": "ci-fixture", } @@ -41,7 +41,7 @@ def do_GET(self): "object": "list", "data": [ { - "id": "qwen2.5:3b", + "id": "qwen3:14b", "object": "model", "owned_by": "local", } @@ -61,7 +61,7 @@ def do_POST(self): self._json( 200, { - "model": "qwen2.5:3b", + "model": "qwen3:14b", "response": "LocalDevStack CI", "message": {"role": "assistant", "content": "LocalDevStack CI"}, "done": True, From ad72142e405fe178a92bd7683bf56b6a79647bf4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:00:51 +0600 Subject: [PATCH 272/509] test(ai): align CLI contract with qwen3 14b --- tests/cli-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index 1e11eda0..1c84fd00 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -51,7 +51,7 @@ chmod +x "$tmpbin/graphify" PATH="$tmpbin:$PATH" \ OLLAMA_BASE_URL=http://127.0.0.1:11434/v1 \ -LDS_AI_MODEL=qwen2.5:3b \ +LDS_AI_MODEL=qwen3:14b \ LDS_AI_TIMEOUT=1800 \ GRAPHIFY_TEST_LOG="$graphify_log" \ "$ROOT/lds" graphify . --model test-model --api-timeout 42 --mode deep >/dev/null From c27d94431138b73078425b60aed0edeaec87e6d1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:00:55 +0600 Subject: [PATCH 273/509] test(ai): align provider contract with qwen3 14b --- tests/ai-contract.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 5e088d34..df50d606 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -29,7 +29,7 @@ done tags="$( docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=2).read().decode())' )" -assert_contains "$tags" "qwen2.5:3b" +assert_contains "$tags" "qwen3:14b" generate="$( docker exec "$container" python -c 'import urllib.request; r=urllib.request.Request("http://127.0.0.1:11434/api/generate", data=b"{}", headers={"Content-Type":"application/json"}); print(urllib.request.urlopen(r, timeout=2).read().decode())' @@ -39,15 +39,15 @@ assert_contains "$generate" "LocalDevStack CI" models="$( docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/v1/models", timeout=2).read().decode())' )" -assert_contains "$models" "qwen2.5:3b" +assert_contains "$models" "qwen3:14b" pass "fake Ollama tags/generate/OpenAI-compatible contracts" docker pull infocyph/tools:latest >/dev/null provider_status="$( - docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-sm:11434 -e LDS_AI_MODEL=qwen2.5:3b infocyph/tools:latest --status + docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-sm:11434 -e LDS_AI_MODEL=qwen3:14b infocyph/tools:latest --status )" assert_contains "$provider_status" "available=1" -assert_contains "$provider_status" "model=qwen2.5:3b" +assert_contains "$provider_status" "model=qwen3:14b" pass "latest Tools reaches the separate provider contract" [[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must remain consolidated into companion.yaml" @@ -58,7 +58,7 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm- assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_SM' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen2.5:3b}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen3:14b}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2}' From 23d06f1896ca406d3362a22f2114dc4fe594ef29 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:00:58 +0600 Subject: [PATCH 274/509] test(docs): expect qwen3 14b AI default --- tests/docs-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index ee4e7de8..7bcae8fa 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -70,7 +70,7 @@ assert_file_contains "$ai" 'linux/amd64 only' assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by default' assert_file_contains "$ai" 'infocyph/llm-sm:${LDS_LLM_ARCH}' assert_file_contains "$ai" 'LDS_LLM_ARCH=amd-latest' -assert_file_contains "$ai" 'LDS_AI_MODEL=qwen2.5:3b' +assert_file_contains "$ai" 'LDS_AI_MODEL=qwen3:14b' assert_file_contains "$ai" 'LLM_SM_ATTACHMENT_MAX_BYTES=16777216' assert_file_contains "$ai" 'LLM_SM_PDF_MAX_PAGES=24' assert_file_contains "$ai" 'docker/compose/companion.yaml' From 4497f2442d89180a6c8f0e38a7b240d6184e3794 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:01:01 +0600 Subject: [PATCH 275/509] test(ai): cover qwen3 default and iGPU forwarding --- tests/compose-contract.sh | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 434db351..b9849450 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -148,7 +148,7 @@ targets={v["target"] for v in s["volumes"]} assert targets == {"/root/.ollama"} assert d["volumes"]["lds_llm"]["name"] == "LLMModels" env=s["environment"] -assert env["LLM_SM_MODEL"] == "qwen2.5:3b" +assert env["LLM_SM_MODEL"] == "qwen3:14b" assert env["LLM_SM_SYSTEM"] == "" assert env["LLM_SM_INPUT_WARN_BYTES"] == "1048576" assert env["LLM_SM_INPUT_MAX_BYTES"] == "0" @@ -162,11 +162,12 @@ assert env["OLLAMA_NUM_PARALLEL"] == "1" assert env["OLLAMA_MAX_LOADED_MODELS"] == "1" assert env["OLLAMA_KEEP_ALIVE"] == "5m" assert env["OLLAMA_NO_CLOUD"] == "1" +assert env["OLLAMA_IGPU_ENABLE"] == "0" tools=d["services"]["server-tools"]["environment"] assert tools["LDS_AI_ENABLED"] == "auto" assert tools["LDS_AI_PROVIDER"] == "ollama" assert tools["LDS_AI_URL"] == "http://llm-sm:11434" -assert tools["LDS_AI_MODEL"] == "qwen2.5:3b" +assert tools["LDS_AI_MODEL"] == "qwen3:14b" assert tools["LDS_AI_CONNECT_TIMEOUT"] == "2" assert tools["LDS_AI_PREFLIGHT_TIMEOUT"] == "5" assert tools["LDS_AI_TIMEOUT"] == "1800" @@ -179,7 +180,7 @@ assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" -printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_SM_PDF_MAX_PAGES=12' 'LLM_SM_SYSTEM=Answer briefly.' 'LDS_AI_TIMEOUT=2400' >>"$user_env" +printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_SM_PDF_MAX_PAGES=12' 'LLM_SM_SYSTEM=Answer briefly.' 'LDS_AI_TIMEOUT=2400' 'LDS_AI_IGPU_ENABLE=1' >>"$user_env" ai_override_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' import json,sys @@ -188,6 +189,7 @@ llm=d["services"]["llm-sm"]["environment"] assert llm["LLM_SM_MODEL"] == "qwen2.5:1.5b" assert llm["LLM_SM_PDF_MAX_PAGES"] == "12" assert llm["LLM_SM_SYSTEM"] == "Answer briefly." +assert llm["OLLAMA_IGPU_ENABLE"] == "1" tools=d["services"]["server-tools"]["environment"] assert tools["LDS_AI_TIMEOUT"] == "2400" nginx=d["services"]["nginx"]["environment"] From 8f5a1adea1681a1d8274a64f55f152b454873f98 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:01:26 +0600 Subject: [PATCH 276/509] test(ai): cover AMD CPU iGPU persistence --- tests/env-contract.sh | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index ab30e31d..8353053e 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -98,8 +98,14 @@ pass "CLI project identity follows the Compose project contract" [[ "$(llm_arch_for_runtime cpu)" == "latest" ]] || fail "CPU LLM tag drift" [[ "$(llm_arch_for_runtime nvidia)" == "latest" ]] || fail "NVIDIA LLM tag drift" [[ "$(llm_arch_for_runtime amd)" == "amd-latest" ]] || fail "AMD LLM tag drift" + host_cpu_is_amd() { return 0; } + [[ "$(ai_igpu_default_for_runtime amd)" == "1" ]] || fail "AMD CPU + AMD runtime must enable iGPU" + [[ "$(ai_igpu_default_for_runtime cpu)" == "0" ]] || fail "CPU runtime must not enable iGPU" + [[ "$(ai_igpu_default_for_runtime nvidia)" == "0" ]] || fail "NVIDIA runtime must not enable AMD iGPU" + host_cpu_is_amd() { return 1; } + [[ "$(ai_igpu_default_for_runtime amd)" == "0" ]] || fail "non-AMD CPU must not auto-enable iGPU" ) -pass "LLM runtime maps to the single LDS_LLM_ARCH tag selector" +pass "LLM runtime maps image tags and AMD CPU iGPU defaults deterministically" ai_env_tmp="$(mktemp -d)" ( @@ -130,15 +136,19 @@ ai_env_tmp="$(mktemp -d)" source "$ROOT/lib/platform.sh" source "$ROOT/lib/certificates.sh" detect_ai_runtime() { printf "%s" amd; } + host_cpu_is_amd() { return 0; } add_required_env grep -Fxq "LDS_AI_RUNTIME=amd" "$ENV_DOCKER" || fail "detected AI runtime was not persisted" grep -Fxq "LDS_LLM_ARCH=amd-latest" "$ENV_DOCKER" || fail "detected AMD tag was not persisted" + grep -Fxq "LDS_AI_IGPU_ENABLE=1" "$ENV_DOCKER" || fail "AMD CPU iGPU preference was not persisted" update_env "$ENV_DOCKER" LDS_AI_RUNTIME nvidia + update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE 0 detect_ai_runtime() { printf "%s" amd; } add_required_env grep -Fxq "LDS_AI_RUNTIME=nvidia" "$ENV_DOCKER" || fail "explicit runtime must win over detection" grep -Fxq "LDS_LLM_ARCH=latest" "$ENV_DOCKER" || fail "NVIDIA runtime must use standard latest tag" + grep -Fxq "LDS_AI_IGPU_ENABLE=0" "$ENV_DOCKER" || fail "explicit iGPU preference must be preserved" ) rm -rf "$ai_env_tmp" pass "setup bootstrap persists detection without overriding an explicit runtime" From a323dd346ccb282174935daa0182dbfcc96744c6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:01:44 +0600 Subject: [PATCH 277/509] test(ai): lock iGPU forwarding and runtime persistence --- tests/ai-contract.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index df50d606..3a05e453 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -59,6 +59,7 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_SM' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen3:14b}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'OLLAMA_IGPU_ENABLE=${LDS_AI_IGPU_ENABLE:-0}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2}' @@ -72,5 +73,6 @@ assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" cluster-only "$target" --b assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" +assert_file_contains "$ROOT/lib/ai.sh" 'update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable"' assert_file_contains "$ROOT/lib/compose.sh" '"127.0.0.1:%s:11434"' pass "single companion AI service with ephemeral hardware/port augmentation" From b42c4d587897e1adad3f8b919f00c58dd567b428 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:02:12 +0600 Subject: [PATCH 278/509] docs(ai): document qwen3 default and AMD iGPU enablement --- README.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 084798a5..35a25282 100644 --- a/README.md +++ b/README.md @@ -230,11 +230,11 @@ Enable the `ai` profile through `lds setup profile`. ```text Tools consumer -> http://llm-sm:11434 User HTTPS -> https://llm.localhost -Default model -> qwen2.5:3b +Default model -> qwen3:14b Model store -> LLMModels ``` -LocalDevStack auto-detects the preferred AI runtime during setup: NVIDIA when `nvidia-smi` is usable, AMD only when the Linux ROCm device nodes `/dev/kfd` and `/dev/dri` are present, otherwise CPU. An AMD CPU by itself does not select the AMD image. +LocalDevStack auto-detects the preferred AI runtime during setup: NVIDIA when `nvidia-smi` is usable, AMD only when the Linux ROCm device nodes `/dev/kfd` and `/dev/dri` are present, otherwise CPU. An AMD CPU by itself does not select the AMD image. When the effective runtime is AMD and the host CPU vendor is AMD, setup persists `LDS_AI_IGPU_ENABLE=1`; the provider forwards it as `OLLAMA_IGPU_ENABLE=1` so Ollama does not discard the integrated Radeon GPU. The provider uses one image/tag contract: @@ -242,7 +242,7 @@ The provider uses one image/tag contract: infocyph/llm-sm:${LDS_LLM_ARCH} ``` -CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. `LDS_LLM_ARCH` is derived by the LocalDevStack runtime selector; it is not a separate version choice users should maintain manually. +CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. `LDS_LLM_ARCH` is derived by the LocalDevStack runtime selector; it is not a separate version choice users should maintain manually. `lds llm runtime ...` also refreshes `LDS_AI_IGPU_ENABLE` to match the selected runtime and host CPU. Override detection explicitly when needed: @@ -305,8 +305,9 @@ Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where ap | Setting | Default | Effect | |---|---:|---| -| `LDS_AI_MODEL` | `qwen2.5:3b` | Tools model and `lds llm` default model | +| `LDS_AI_MODEL` | `qwen3:14b` | Tools model and `lds llm` default model | | `LDS_AI_RUNTIME` | auto-detected | `cpu`, `nvidia`, or `amd` runtime selection | +| `LDS_AI_IGPU_ENABLE` | `1` for AMD CPU + AMD runtime, otherwise `0` | Forwarded to Ollama as `OLLAMA_IGPU_ENABLE` | | `LDS_LLM_HOST_PORT` | `0` | Enables/disables direct loopback API mapping | | `LLM_SM_PORT` | `11434` | Host loopback port when direct mapping is enabled | | `LLM_SM_SYSTEM` | empty | Default system instruction for provider prompts | From f1949f6eb6062cb5eff0dc4ba369719c62b005cf Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:02:29 +0600 Subject: [PATCH 279/509] docs(ai): align local runtime guide with qwen3 and AMD iGPU --- docs/guides/local-ai.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index f07dac83..fa7c9def 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -19,7 +19,7 @@ The default consumer contract is:: LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama LDS_AI_URL=http://llm-sm:11434 - LDS_AI_MODEL=qwen2.5:3b + LDS_AI_MODEL=qwen3:14b The explicit model default avoids ambiguity when multiple models are installed in the persistent Ollama store. @@ -33,7 +33,7 @@ LocalDevStack detects the preferred runtime during environment setup. Detection - AMD only when both ``/dev/kfd`` and ``/dev/dri`` are present for ROCm; - CPU otherwise. -An AMD CPU alone does **not** select the AMD image. +An AMD CPU alone does **not** select the AMD image. When the effective runtime is ``amd`` and the host CPU vendor is AMD, LocalDevStack persists ``LDS_AI_IGPU_ENABLE=1`` and forwards it to Ollama as ``OLLAMA_IGPU_ENABLE=1`` so integrated Radeon GPUs are not discarded by Ollama's default iGPU filter. The single provider service uses:: @@ -55,7 +55,7 @@ Override detection explicitly when required:: lds llm runtime nvidia lds llm runtime amd -Changing runtime mode updates both ``LDS_AI_RUNTIME`` and ``LDS_LLM_ARCH`` in ``docker/.env``. Recreate/start the service afterward so the Compose override changes take effect. +Changing runtime mode updates ``LDS_AI_RUNTIME``, ``LDS_LLM_ARCH`` and the derived ``LDS_AI_IGPU_ENABLE`` value in ``docker/.env``. Recreate/start the service afterward so the Compose override and provider environment changes take effect. Access ------ @@ -198,11 +198,14 @@ These provider settings can be placed in ``docker/.env`` and are forwarded to LLM_SM_PDF_MAX_PAGES=24 LLM_SM_PDF_DPI=120 LLM_SM_ALLOW_LARGE_INPUT=0 + LDS_AI_IGPU_ENABLE=1 # auto-persisted for AMD CPU + AMD runtime; otherwise 0 OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_LOADED_MODELS=1 OLLAMA_KEEP_ALIVE=5m OLLAMA_NO_CLOUD=1 +``LDS_AI_IGPU_ENABLE`` is forwarded as ``OLLAMA_IGPU_ENABLE``. The automatic value is ``1`` only for an AMD CPU with the AMD runtime selected; users may override the persisted value deliberately. + ``LLM_SM_INPUT_MAX_BYTES=0`` disables only the hard text/diff ceiling. Attachment and PDF-vision limits remain active unless their own value is set to ``0``. ``LLM_SM_ALLOW_LARGE_INPUT=1`` is the explicit escape hatch for a deliberate request. From e0e4059f28b410d4cabc7b172568c6f9f2c28bb5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:02:52 +0600 Subject: [PATCH 280/509] docs(plan): sync qwen3 default and AMD iGPU behavior --- .../07-localdevstack-integration-plan.md | 28 ++++++++++++------- 1 file changed, 18 insertions(+), 10 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index d1e85d57..6a513842 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -105,7 +105,7 @@ Host ├── databases and admin clients │ └── llm-sm current stable (optional) - ├── qwen2.5:3b baked default + ├── qwen3:14b baked default ├── persistent /root/.ollama └── Ollama API :11434 ``` @@ -600,7 +600,7 @@ Mount it to: The volume is authoritative runtime model state. -A fresh volume receives the image-baked `qwen2.5:3b`. +A fresh volume receives the image-baked `qwen3:14b`. Existing populated volumes must never be silently replaced/reset during upgrades. @@ -687,7 +687,7 @@ Pass the published Tools AI contract into `server-tools`: LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama LDS_AI_URL=http://llm-sm:11434 -LDS_AI_MODEL=qwen2.5:3b +LDS_AI_MODEL=qwen3:14b ``` Allow user overrides. @@ -708,9 +708,10 @@ default. Forward `LDS_AI_TIMEOUT` to the Nginx container as long-running request budget. Do not reintroduce a shorter independent UI/process timeout for Admin AI analysis. -Why LocalDevStack should default `LDS_AI_MODEL=qwen2.5:3b`: +Why LocalDevStack should default `LDS_AI_MODEL=qwen3:14b`: - `llm-sm:latest` ships that model; +- the 14B Qwen3 default provides materially stronger instruction/structured-output behavior than the previous 3B model while remaining practical on modern 32 GB unified-memory developer hosts; - Tools intentionally reports ambiguity when multiple models are installed and no model is selected; - users may pull more models without breaking Tools AI workflows. @@ -897,7 +898,7 @@ Prompt only relevant AI settings: - enable AI yes/no; - detected runtime is shown, not prompted by default; -- preferred model default `qwen2.5:3b`; +- preferred model default `qwen3:14b`; - optional direct localhost port yes/no. Do not ask users for low-level timeout/byte-limit settings during normal setup. @@ -1831,7 +1832,7 @@ With fake provider on normal CI: With real provider on manual/release gate when feasible: - `infocyph/llm-sm:latest`; -- baked `qwen2.5:3b`; +- baked `qwen3:14b`; - persistent model volume; - Tools generation; - Nginx `llm.localhost`. @@ -2248,7 +2249,8 @@ llm-sm: image: infocyph/llm-sm:${LDS_LLM_ARCH} profiles: [ai] environment: - LLM_SM_MODEL: ${LDS_AI_MODEL:-qwen2.5:3b} + LLM_SM_MODEL: ${LDS_AI_MODEL:-qwen3:14b} + OLLAMA_IGPU_ENABLE: ${LDS_AI_IGPU_ENABLE:-0} # provider input/PDF/Ollama tuning values are forwarded from docker/.env ``` @@ -2269,11 +2271,17 @@ Initial runtime detection is conservative: - both `/dev/kfd` and `/dev/dri` present -> `amd`; - otherwise -> `cpu`. -An AMD CPU alone does not select the ROCm image. +An AMD CPU alone does not select the ROCm image. When the effective runtime is `amd` +and the host CPU vendor is AMD, persist `LDS_AI_IGPU_ENABLE=1` and forward it to the +provider as `OLLAMA_IGPU_ENABLE=1`; otherwise the derived default is `0`. This avoids +Ollama's integrated-GPU filter silently dropping Ryzen integrated Radeon devices that are +otherwise available through the ROCm device nodes. Explicit `LDS_AI_RUNTIME` / `lds llm runtime ...` selection remains authoritative. -`LDS_LLM_ARCH` is derived from that effective runtime and must not become an -independent manual image-version selector. +`LDS_LLM_ARCH` and the automatic `LDS_AI_IGPU_ENABLE` value are derived from that +effective runtime/host combination and must not become independent image-version +selectors. A deliberately persisted iGPU override remains user-controlled until the +runtime selector is invoked again. ## Ephemeral hardware/host-port Compose augmentation From 6878f5f709358a530a6fa0edd265eeaf70017598 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:03:53 +0600 Subject: [PATCH 281/509] docs(ai): describe qwen3 and AMD iGPU profile settings --- docs/concepts/profiles-and-env.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index 1edbe318..e5a53994 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -210,8 +210,9 @@ Important AI settings include:: LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama LDS_AI_URL=http://llm-sm:11434 - LDS_AI_MODEL=qwen2.5:3b + LDS_AI_MODEL=qwen3:14b LDS_AI_RUNTIME= + LDS_AI_IGPU_ENABLE= LDS_LLM_HOST_PORT=0 LLM_SM_PORT=11434 @@ -222,8 +223,10 @@ usable; AMD is selected only when the ROCm Linux device nodes ``/dev/kfd`` and ``LDS_LLM_ARCH`` is derived from that runtime (``latest`` for CPU/NVIDIA, ``amd-latest`` for AMD). It is not a separate user-facing image version selector. -Using ``lds llm runtime ...`` persists the explicit runtime choice and matching derived -tag for compatibility/inspection. +Using ``lds llm runtime ...`` persists the explicit runtime choice, matching derived +tag and the automatic ``LDS_AI_IGPU_ENABLE`` value. On an AMD CPU with the AMD runtime, +the automatic value is ``1`` and is forwarded as ``OLLAMA_IGPU_ENABLE=1``; otherwise +it is ``0``. Use:: From 809d69036987daf0b93479c60a6d3a3b3fb76947 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:04:03 +0600 Subject: [PATCH 282/509] docs(ai): document AMD iGPU provider wiring --- docs/concepts/architecture.rst | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index 8ec2bd79..ce405762 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -161,7 +161,9 @@ No AI-specific Compose files are tracked. ``lds`` generates a temporary fragment ``docker/.runtime/`` only when NVIDIA GPU access, AMD device mappings, or loopback host-port exposure is required, then removes it after the Compose command. The service also forwards the configured ``LDS_AI_MODEL`` to the provider as ``LLM_SM_MODEL`` and -passes the documented provider safety/tuning settings from ``docker/.env``. +passes the documented provider safety/tuning settings from ``docker/.env``. For AMD +runtime on an AMD CPU, LocalDevStack persists ``LDS_AI_IGPU_ENABLE=1`` and forwards it +as ``OLLAMA_IGPU_ENABLE=1`` so Ollama admits the integrated Radeon GPU. Project Identity ---------------- From 68380a580cee32d40a8b4d0920f5786888a6a851 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:06:34 +0600 Subject: [PATCH 283/509] docs(cli): document AMD iGPU runtime state --- docs/reference/cli.rst | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index c0a694e5..c914b225 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -260,6 +260,10 @@ Runtime selection:: lds llm runtime lds llm runtime +The runtime command keeps the derived image tag in sync and also refreshes +``LDS_AI_IGPU_ENABLE``. AMD runtime on an AMD CPU uses ``1`` so Ollama admits the +integrated Radeon GPU; the other derived cases use ``0``. + Direct host-port control:: lds llm host-port status From af15a9095a0aaba3b69f3298d1b419a5bd1cb268 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:07:49 +0600 Subject: [PATCH 284/509] test(ai): align catalog default with qwen3 14b --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 16ba2b4c..0a4450af 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -51,7 +51,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$profile" == "ai" ]] || fail "AI profile must be ai" [[ "$service_key" == "llm-sm" ]] || fail "AI service key must be llm-sm" [[ "$runtime_modes" == "cpu,nvidia,amd" ]] || fail "AI runtime metadata drift" - [[ "$defaults" == *"LDS_AI_MODEL=qwen2.5:3b"* ]] || fail "AI model default drift" + [[ "$defaults" == *"LDS_AI_MODEL=qwen3:14b"* ]] || fail "AI model default drift" [[ "$version_env" == "LDS_LLM_ARCH" ]] || fail "AI image selector must be LDS_LLM_ARCH" [[ "$defaults" != *"LDS_AI_RUNTIME="* ]] || fail "AI profile wizard must not prompt for runtime" [[ "$defaults" == *"LDS_LLM_HOST_PORT=no"* ]] || fail "AI host-port default drift" From 233ed33c826b7c2ac259809bfbb920e45b0d0515 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:03 +0600 Subject: [PATCH 285/509] docs(ai): consume renamed llm-ollama image --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 35a25282..d601a7d2 100644 --- a/README.md +++ b/README.md @@ -166,8 +166,8 @@ Examples: - PostgreSQL: `postgres:alpine` - Tools / Runner / Nginx / Apache: published `:latest` - MySQL / MariaDB / MongoDB / Redis: their supported moving defaults -- local AI: `infocyph/llm-sm:latest` -- AMD local AI: `infocyph/llm-sm:amd-latest` +- local AI: `infocyph/llm-ollama:latest` +- AMD local AI: `infocyph/llm-ollama:amd-latest` Elasticsearch, Kibana, and Filebeat stay on one aligned Elastic version. @@ -239,7 +239,7 @@ LocalDevStack auto-detects the preferred AI runtime during setup: NVIDIA when `n The provider uses one image/tag contract: ```text -infocyph/llm-sm:${LDS_LLM_ARCH} +infocyph/llm-ollama:${LDS_LLM_ARCH} ``` CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. `LDS_LLM_ARCH` is derived by the LocalDevStack runtime selector; it is not a separate version choice users should maintain manually. `lds llm runtime ...` also refreshes `LDS_AI_IGPU_ENABLE` to match the selected runtime and host CPU. From 4b2d88c8f84e23d959247b1c339920b8537001a6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:07 +0600 Subject: [PATCH 286/509] feat(ai): consume infocyph/llm-ollama --- docker/compose/companion.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index dedf4fa4..d1408589 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -57,7 +57,7 @@ services: llm-sm: container_name: LLM_SM hostname: llm-sm - image: infocyph/llm-sm:${LDS_LLM_ARCH} + image: infocyph/llm-ollama:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] environment: From 9be5997f981797239c8bf18ee54875509e9ec874 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:16 +0600 Subject: [PATCH 287/509] chore(ai): report llm-ollama image name --- lib/ai.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/ai.sh b/lib/ai.sh index 563a9c45..ff4d6c40 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -137,7 +137,7 @@ cmd_llm() { update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" update_env "$ENV_DOCKER" LDS_LLM_ARCH "$arch" update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable" - ok "LLM runtime set to $normalized (infocyph/llm-sm:$arch, iGPU=$igpu_enable). Recreate llm-sm to apply the change." + ok "LLM runtime set to $normalized (infocyph/llm-ollama:$arch, iGPU=$igpu_enable). Recreate llm-sm to apply the change." ;; *) die "llm runtime " ;; esac From 7b1802dd8b70af4b87bea69b0c279a598b572d64 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:35 +0600 Subject: [PATCH 288/509] test(ai): expect llm-ollama provider image --- tests/ai-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 3a05e453..8d4b5b7c 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -54,7 +54,7 @@ pass "latest Tools reaches the separate provider contract" if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -quit | grep -q .; then fail "AI-specific Compose overlays must be generated ephemerally" fi -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_SM' From e23cad77fb6e12fbe196f85e71312bbda3b112c1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:38 +0600 Subject: [PATCH 289/509] test(ai): align compose image contract with llm-ollama --- tests/compose-contract.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index b9849450..8ab0b24d 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -140,7 +140,7 @@ python3 -c ' import json,sys d=json.load(sys.stdin) s=d["services"]["llm-sm"] -assert s["image"] == "infocyph/llm-sm:latest" +assert s["image"] == "infocyph/llm-ollama:latest" assert s["container_name"] == "LLM_SM" assert not s.get("ports") assert set(s["networks"]) == {"frontend","backend"} @@ -201,7 +201,7 @@ amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config sh python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] -assert s["image"] == "infocyph/llm-sm:amd-latest" +assert s["image"] == "infocyph/llm-ollama:amd-latest" devices=" ".join(str(x) for x in s.get("devices", [])) assert "/dev/kfd" in devices and "/dev/dri" in devices ' <<<"$amd_json" @@ -211,7 +211,7 @@ nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" --quiet con python3 -c ' import json,sys s=json.load(sys.stdin)["services"]["llm-sm"] -assert s["image"] == "infocyph/llm-sm:latest" +assert s["image"] == "infocyph/llm-ollama:latest" assert s.get("gpus") ' <<<"$nvidia_json" pass "NVIDIA AI runtime is generated dynamically" @@ -233,5 +233,5 @@ fi if [[ -d "$ROOT/docker/.runtime" ]] && find "$ROOT/docker/.runtime" -type f -print -quit | grep -q .; then fail "temporary AI Compose overrides were not cleaned up" fi -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' pass "single LLM service plus ephemeral hardware/port overrides" From f994dd3430f632d0c660925f93fa21f671268bf2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:45 +0600 Subject: [PATCH 290/509] test(images): track llm-ollama provider image --- tests/published-images.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index 4648a983..c0047605 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -35,7 +35,7 @@ for image in "$tools_image" "$runner_image"; do done pass "Tools and Runner publish healthchecks" -grep -Fq 'image: infocyph/llm-sm:${LDS_LLM_ARCH}' "$ROOT/docker/compose/companion.yaml" || +grep -Fq 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' "$ROOT/docker/compose/companion.yaml" || fail "LLM service must use the single LDS_LLM_ARCH selector" grep -Fq "amd) printf '%s' amd-latest" "$ROOT/lib/platform.sh" || fail "AMD runtime must map to amd-latest" From 1513e9095abf10b7ede347e4bb8deb447cd9feae Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:51 +0600 Subject: [PATCH 291/509] docs(ai): rename provider image to llm-ollama --- docs/guides/local-ai.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index fa7c9def..ca8bfb22 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -37,7 +37,7 @@ An AMD CPU alone does **not** select the AMD image. When the effective runtime i The single provider service uses:: - image: infocyph/llm-sm:${LDS_LLM_ARCH} + image: infocyph/llm-ollama:${LDS_LLM_ARCH} The derived image mapping is: From 4c270fc2a5fc7282df1f010be9b252c0bd6a0ee7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:55 +0600 Subject: [PATCH 292/509] docs(ai): rename provider image in architecture --- docs/concepts/architecture.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index ce405762..e3f3bbb5 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -153,7 +153,7 @@ The provider is one ``llm-sm`` service declared in ``docker/compose/companion.yaml`` and enabled only by the ``ai`` profile. The service keeps ``container_name: LLM_SM`` for current single-stack compatibility, while all internal routing continues to use the Compose service/hostname ``llm-sm``. Its image -is ``infocyph/llm-sm:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and +is ``infocyph/llm-ollama:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. ``LDS_LLM_ARCH`` is derived from the effective runtime rather than maintained as an independent version selector. From 0040db219d612fd61649711a18611f066e495d72 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:14:58 +0600 Subject: [PATCH 293/509] docs(ai): rename provider image in environment guide --- docs/concepts/profiles-and-env.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index e5a53994..8e8946f7 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -161,8 +161,8 @@ Examples: - PostgreSQL defaults to ``postgres:alpine``. - Tools, Runner, Nginx, and Apache use their published ``:latest`` aliases. -- Standard local AI uses ``infocyph/llm-sm:latest``. -- AMD local AI uses ``infocyph/llm-sm:amd-latest``. +- Standard local AI uses ``infocyph/llm-ollama:latest``. +- AMD local AI uses ``infocyph/llm-ollama:amd-latest``. - Elasticsearch, Kibana, and Filebeat stay on one aligned version. Run:: From aa495a72ef76a703292c4f26dae31dd5a1c2e8cd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:15:03 +0600 Subject: [PATCH 294/509] docs(plan): rename llm provider repository --- .../docker-ecosystem/00-master-bottom-up-plan.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md index 6a17147d..da276b72 100644 --- a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md +++ b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md @@ -13,7 +13,7 @@ Completed/published baseline as of 2026-09-18: - `infocyph/nginx:0.4.1` - `infocyph/apache:0.4.2` - `infocyph/tools:0.23.2` -- `infocyph/llm-sm:0.03` / `amd-0.03` +- `infocyph/llm-ollama:0.03` / `amd-0.03` The authoritative active implementation plan is now `07-localdevstack-integration-plan.md`. Earlier 01–06 files remain historical planning records for the completed lower layers. @@ -33,7 +33,7 @@ Bottom to top: 4. `infocyph/docker-nginx` — primary HTTP/TLS edge and local routing. 5. `infocyph/docker-apache` — optional Apache backend for PHP/vhost compatibility. 6. `infocyph/docker-tools` — LocalDevStack control plane: domain/TLS/config generation, service/profile helpers, monitoring, admin panel, environment/secrets tooling. -7. `infocyph/docker-llm-sm` — published optional local-AI capability; already follows the newer image publication model. +7. `infocyph/docker-llm-ollama` — published optional local-AI capability; already follows the newer image publication model. 8. `infocyph/LocalDevStack` — product/orchestrator: `lds`, Compose topology, PHP/Node runtime Dockerfiles, generated configuration, user-facing workflow, platform integration. ## Core Architectural Rules @@ -46,7 +46,7 @@ Published images remain independently versioned infrastructure: - `infocyph/nginx` - `infocyph/apache` - `infocyph/tools` -- `infocyph/llm-sm` +- `infocyph/llm-ollama` PHP and Node remain locally generated runtime images because they are customized by selected runtime version, host UID/GID, extensions/packages, and project-level needs: @@ -65,7 +65,7 @@ For every published `docker-*` repository: - source release and resulting image revision must be identifiable through OCI metadata/provenance; - Docker Hub and GHCR should receive the same built digest for the same variant. -`docker-llm-sm` is the reference implementation for this contract. +`docker-llm-ollama` is the reference implementation for this contract. ### Dependency reproducibility @@ -174,9 +174,9 @@ Primary goals: ### Phase 3 — Local AI capability — COMPLETE -Plan: `06-docker-llm-sm-plan.md` +Plan: `06-docker-llm-ollama-plan.md` -`docker-llm-sm` is published as `0.03` and its provider/runtime contract is complete for this program. +`docker-llm-ollama` is published as `0.03` and its provider/runtime contract is complete for this program. Primary goals: @@ -264,6 +264,6 @@ The ecosystem work is complete when: 5. Static IP dependence is removed unless a documented unavoidable case remains. 6. PHP and Node runtime generation remains version-flexible and user-customizable. 7. Local domains/TLS, mail, DB/admin clients, cron/supervisor, secrets, and diagnostics still work. -8. Optional local AI can be enabled through `infocyph/llm-sm` without contaminating the default stack. +8. Optional local AI can be enabled through `infocyph/llm-ollama` without contaminating the default stack. 9. Documentation matches the actual storage/network/release behavior. 10. A clean install on supported host categories can reach a working PHP or Node local domain with TLS using the documented workflow. From 3674cd48ee28bbc690a6f6b7d0ead9f9fa506b95 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:25:12 +0600 Subject: [PATCH 295/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- docker/compose/companion.yaml | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index d1408589..bf6c4093 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -19,7 +19,7 @@ services: - COMPOSE_PROFILES=${COMPOSE_PROFILES:-} - LDS_AI_ENABLED=${LDS_AI_ENABLED:-auto} - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-ollama} - - LDS_AI_URL=${LDS_AI_URL:-http://llm-sm:11434} + - LDS_AI_URL=${LDS_AI_URL:-http://llm-ollama:11434} - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen3:14b} - LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2} - LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5} @@ -54,23 +54,23 @@ services: - backend - datastore - llm-sm: - container_name: LLM_SM - hostname: llm-sm + llm-ollama: + container_name: LLM_OLLAMA + hostname: llm-ollama image: infocyph/llm-ollama:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] environment: - - LLM_SM_MODEL=${LDS_AI_MODEL:-qwen3:14b} - - LLM_SM_SYSTEM=${LLM_SM_SYSTEM:-} - - LLM_SM_INPUT_WARN_BYTES=${LLM_SM_INPUT_WARN_BYTES:-1048576} - - LLM_SM_INPUT_MAX_BYTES=${LLM_SM_INPUT_MAX_BYTES:-0} - - LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216} - - LLM_SM_ATTACHMENTS_MAX_BYTES=${LLM_SM_ATTACHMENTS_MAX_BYTES:-33554432} - - LLM_SM_ATTACHMENT_MAX_COUNT=${LLM_SM_ATTACHMENT_MAX_COUNT:-16} - - LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24} - - LLM_SM_PDF_DPI=${LLM_SM_PDF_DPI:-120} - - LLM_SM_ALLOW_LARGE_INPUT=${LLM_SM_ALLOW_LARGE_INPUT:-0} + - LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b} + - LLM_OLLAMA_SYSTEM=${LLM_OLLAMA_SYSTEM:-} + - LLM_OLLAMA_INPUT_WARN_BYTES=${LLM_OLLAMA_INPUT_WARN_BYTES:-1048576} + - LLM_OLLAMA_INPUT_MAX_BYTES=${LLM_OLLAMA_INPUT_MAX_BYTES:-0} + - LLM_OLLAMA_ATTACHMENT_MAX_BYTES=${LLM_OLLAMA_ATTACHMENT_MAX_BYTES:-16777216} + - LLM_OLLAMA_ATTACHMENTS_MAX_BYTES=${LLM_OLLAMA_ATTACHMENTS_MAX_BYTES:-33554432} + - LLM_OLLAMA_ATTACHMENT_MAX_COUNT=${LLM_OLLAMA_ATTACHMENT_MAX_COUNT:-16} + - LLM_OLLAMA_PDF_MAX_PAGES=${LLM_OLLAMA_PDF_MAX_PAGES:-24} + - LLM_OLLAMA_PDF_DPI=${LLM_OLLAMA_PDF_DPI:-120} + - LLM_OLLAMA_ALLOW_LARGE_INPUT=${LLM_OLLAMA_ALLOW_LARGE_INPUT:-0} - OLLAMA_IGPU_ENABLE=${LDS_AI_IGPU_ENABLE:-0} - OLLAMA_NUM_PARALLEL=${OLLAMA_NUM_PARALLEL:-1} - OLLAMA_MAX_LOADED_MODELS=${OLLAMA_MAX_LOADED_MODELS:-1} From f27ac3f940e952747332ae0338240a00e81c393a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:25:16 +0600 Subject: [PATCH 296/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index dd1ac952..57afa271 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.3|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm-sm|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b;LDS_LLM_HOST_PORT=no|Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd +AI|ai|Local AI|llm-ollama|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b;LDS_LLM_HOST_PORT=no|Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm-ollama.localhost|ai|1|0|cpu,nvidia,amd From ab3bc3f571e671466a84028b332acadbf9aadfe0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:25:28 +0600 Subject: [PATCH 297/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- lib/ai.sh | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index ff4d6c40..422845f3 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -31,18 +31,18 @@ cmd_ai() { _graphify_local_base_url() { local ctr published host_port - ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" + ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" [[ -n "$ctr" ]] || - die "llm-sm is not running. Enable the ai profile and start the stack first." + die "llm-ollama is not running. Enable the ai profile and start the stack first." docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || - die "llm-sm container exists but is not running." + die "llm-ollama container exists but is not running." published="$( docker inspect -f '{{with (index .NetworkSettings.Ports "11434/tcp")}}{{(index . 0).HostPort}}{{end}}' "$ctr" 2>/dev/null || true )" [[ "$published" =~ ^[0-9]+$ ]] || - die "Graphify needs the llm-sm loopback API. Run: lds llm host-port on && lds up -d llm-sm" + die "Graphify needs the llm-ollama loopback API. Run: lds llm host-port on && lds up -d llm-ollama" host_port="$published" printf 'http://127.0.0.1:%s/v1' "$host_port" @@ -107,14 +107,14 @@ cmd_graphify() { _llm_exec() { local ctr - ctr="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" - [[ -n "$ctr" ]] || die "llm-sm is not running. Enable the ai profile and start the stack first." + ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" + [[ -n "$ctr" ]] || die "llm-ollama is not running. Enable the ai profile and start the stack first." docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || - die "llm-sm container exists but is not running." + die "llm-ollama container exists but is not running." local -a exec_args=(exec) [[ -t 0 && -t 1 ]] || exec_args+=(-T) - docker_compose "${exec_args[@]}" llm-sm llm-sm "$@" + docker_compose "${exec_args[@]}" llm-ollama llm-ollama "$@" } cmd_llm() { @@ -137,7 +137,7 @@ cmd_llm() { update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" update_env "$ENV_DOCKER" LDS_LLM_ARCH "$arch" update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable" - ok "LLM runtime set to $normalized (infocyph/llm-ollama:$arch, iGPU=$igpu_enable). Recreate llm-sm to apply the change." + ok "LLM runtime set to $normalized (infocyph/llm-ollama:$arch, iGPU=$igpu_enable). Recreate llm-ollama to apply the change." ;; *) die "llm runtime " ;; esac @@ -148,11 +148,11 @@ cmd_llm() { status) printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" ;; on | enable | enabled | 1) update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 1 - ok "Direct LLM API enabled on loopback only. Recreate llm-sm to apply." + ok "Direct LLM API enabled on loopback only. Recreate llm-ollama to apply." ;; off | disable | disabled | 0) update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 0 - ok "Direct LLM host API disabled. Recreate llm-sm to apply." + ok "Direct LLM host API disabled. Recreate llm-ollama to apply." ;; *) die "llm host-port " ;; esac From 49adb5c6c4661a81d1a0d336c49d4c2bd6e3a580 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:25:36 +0600 Subject: [PATCH 298/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- lib/compose.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/compose.sh b/lib/compose.sh index fa3ec17f..df20edd3 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -64,9 +64,9 @@ docker_compose() { *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; esac - llm_port="$(compose_control_value LLM_SM_PORT 11434)" + llm_port="$(compose_control_value LLM_OLLAMA_PORT 11434)" [[ "$llm_port" =~ ^[0-9]+$ ]] && ((llm_port >= 1 && llm_port <= 65535)) || - die "Invalid LLM_SM_PORT: $llm_port (expected 1-65535)" + die "Invalid LLM_OLLAMA_PORT: $llm_port (expected 1-65535)" if [[ "$ai_runtime" != "cpu" || "$ai_host_port_enabled" == "1" ]]; then mkdir -p "$CFG/.runtime" @@ -74,7 +74,7 @@ docker_compose() { die "Unable to create temporary AI Compose override" { - printf '%s\n' 'services:' ' llm-sm:' + printf '%s\n' 'services:' ' llm-ollama:' case "$ai_runtime" in nvidia) printf '%s\n' ' gpus: all' From 9e1ae9da7686df11c3cc53f6fdf21f9572531542 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:25:44 +0600 Subject: [PATCH 299/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- README.md | 48 ++++++++++++++++++++++++------------------------ 1 file changed, 24 insertions(+), 24 deletions(-) diff --git a/README.md b/README.md index d601a7d2..bd4f772c 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ It is designed for **trusted local development infrastructure**. It is not a pro - Mailpit is part of the core stack and persists captured mail. - Runner provides cron, Supervisor, and log-rotation support. - Tools owns domains, certificates, the admin UI, developer helpers, secrets integration, diagnostics, monitoring, and AI-consumer features. -- Optional `llm-sm` provides a small local Ollama runtime with persistent models. +- Optional `llm-ollama` provides a small local Ollama runtime with persistent models. - Dynamic Docker networking through service DNS removes fixed subnet dependencies. - Linux, Windows/Git Bash, WSL, and Docker Desktop workflows are supported by the host CLI. @@ -34,13 +34,13 @@ Host │ ├─ apache │ ├─ optional databases/admin clients │ ├─ generated PHP/Node runtimes - │ └─ optional llm-sm + │ └─ optional llm-ollama │ ├─ configuration/ └─ logs/ ``` -LocalDevStack uses three logical Docker networks: `Frontend`, `Backend`, and `DataStore`. Docker assigns their address ranges dynamically. Internal communication uses service names such as `nginx`, `postgres`, `redis`, `server-tools`, and `llm-sm`. For single-stack compatibility the LLM service keeps the fixed container name `LLM_SM`; internal routing still uses the Compose service/hostname `llm-sm`, never the fixed container name. +LocalDevStack uses three logical Docker networks: `Frontend`, `Backend`, and `DataStore`. Docker assigns their address ranges dynamically. Internal communication uses service names such as `nginx`, `postgres`, `redis`, `server-tools`, and `llm-ollama`. For single-stack compatibility the LLM service keeps the fixed container name `LLM_OLLAMA`; internal routing still uses the Compose service/hostname `llm-ollama`, never the fixed container name. ## Prerequisites @@ -112,7 +112,7 @@ The exact optional endpoints shown by `lds urls` depend on enabled profiles. | RedisInsight | `https://ri.localhost` | | Mongo Express | `https://me.localhost` | | Kibana | `https://kibana.localhost` | -| Local AI | `https://llm.localhost` | +| Local AI | `https://llm-ollama.localhost` | Use `lds open admin`, `lds open mail`, `lds open db`, `lds open redis`, `lds open mongo`, `lds open kibana`, or `lds open ai` to open a known endpoint. @@ -228,8 +228,8 @@ See `docs/guides/databases-and-clients.rst` for the profile/client map. Enable the `ai` profile through `lds setup profile`. ```text -Tools consumer -> http://llm-sm:11434 -User HTTPS -> https://llm.localhost +Tools consumer -> http://llm-ollama:11434 +User HTTPS -> https://llm-ollama.localhost Default model -> qwen3:14b Model store -> LLMModels ``` @@ -272,15 +272,15 @@ lds llm ask "Explain dependency injection briefly" lds llm chat ... ``` -Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. Change that loopback port with `LLM_SM_PORT` in `docker/.env`. +Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. Change that loopback port with `LLM_OLLAMA_PORT` in `docker/.env`. -`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets the actual loopback-published `llm-sm` API. The command runs extraction with `--backend ollama --no-cluster`, then runs `cluster-only` for the same path if extraction succeeds. This avoids clustering twice. +`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets the actual loopback-published `llm-ollama` API. The command runs extraction with `--backend ollama --no-cluster`, then runs `cluster-only` for the same path if extraction succeeds. This avoids clustering twice. Before first use, publish the provider port and apply that Compose change: ```bash lds llm host-port on -lds up -d llm-sm +lds up -d llm-ollama lds graphify ``` @@ -293,7 +293,7 @@ The LocalDevStack LLM Compose layout is intentionally small: ```text docker/compose/main.yaml └─ includes docker/compose/companion.yaml - └─ llm-sm service (profile: ai) + └─ llm-ollama service (profile: ai) docker/.runtime/ai.* temporary per-command overlay generated by lds only when NVIDIA, AMD/ROCm, or host-port settings require it @@ -301,7 +301,7 @@ docker/.runtime/ai.* temporary per-command overlay generated by lds There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. AI hardware and host-port additions are generated ephemerally and removed after the Compose command. Files under `configuration/compose/` remain the normal LocalDevStack extras/runtime-fragment area; they are not built-in LLM variant files. -Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where applicable: +Provider configuration placed in `docker/.env` is forwarded to `llm-ollama` where applicable: | Setting | Default | Effect | |---|---:|---| @@ -309,16 +309,16 @@ Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where ap | `LDS_AI_RUNTIME` | auto-detected | `cpu`, `nvidia`, or `amd` runtime selection | | `LDS_AI_IGPU_ENABLE` | `1` for AMD CPU + AMD runtime, otherwise `0` | Forwarded to Ollama as `OLLAMA_IGPU_ENABLE` | | `LDS_LLM_HOST_PORT` | `0` | Enables/disables direct loopback API mapping | -| `LLM_SM_PORT` | `11434` | Host loopback port when direct mapping is enabled | -| `LLM_SM_SYSTEM` | empty | Default system instruction for provider prompts | -| `LLM_SM_INPUT_WARN_BYTES` | `1048576` | Text/diff warning threshold | -| `LLM_SM_INPUT_MAX_BYTES` | `0` | Text/diff hard ceiling; `0` disables it | -| `LLM_SM_ATTACHMENT_MAX_BYTES` | `16777216` | Per attachment/source-file ceiling | -| `LLM_SM_ATTACHMENTS_MAX_BYTES` | `33554432` | Aggregate attachment ceiling | -| `LLM_SM_ATTACHMENT_MAX_COUNT` | `16` | Source-attachment count ceiling | -| `LLM_SM_PDF_MAX_PAGES` | `24` | PDF-vision page ceiling | -| `LLM_SM_PDF_DPI` | `120` | PDF-vision render DPI | -| `LLM_SM_ALLOW_LARGE_INPUT` | `0` | Deliberate bypass for configured provider limits | +| `LLM_OLLAMA_PORT` | `11434` | Host loopback port when direct mapping is enabled | +| `LLM_OLLAMA_SYSTEM` | empty | Default system instruction for provider prompts | +| `LLM_OLLAMA_INPUT_WARN_BYTES` | `1048576` | Text/diff warning threshold | +| `LLM_OLLAMA_INPUT_MAX_BYTES` | `0` | Text/diff hard ceiling; `0` disables it | +| `LLM_OLLAMA_ATTACHMENT_MAX_BYTES` | `16777216` | Per attachment/source-file ceiling | +| `LLM_OLLAMA_ATTACHMENTS_MAX_BYTES` | `33554432` | Aggregate attachment ceiling | +| `LLM_OLLAMA_ATTACHMENT_MAX_COUNT` | `16` | Source-attachment count ceiling | +| `LLM_OLLAMA_PDF_MAX_PAGES` | `24` | PDF-vision page ceiling | +| `LLM_OLLAMA_PDF_DPI` | `120` | PDF-vision render DPI | +| `LLM_OLLAMA_ALLOW_LARGE_INPUT` | `0` | Deliberate bypass for configured provider limits | | `OLLAMA_NUM_PARALLEL` | `1` | Ollama request parallelism | | `OLLAMA_MAX_LOADED_MODELS` | `1` | Loaded-model limit | | `OLLAMA_KEEP_ALIVE` | `5m` | Model keep-alive | @@ -326,9 +326,9 @@ Provider configuration placed in `docker/.env` is forwarded to `llm-sm` where ap `LDS_AI_CONNECT_TIMEOUT=2` and `LDS_AI_PREFLIGHT_TIMEOUT=5` intentionally fail fast. `LDS_AI_TIMEOUT=1800` gives local model loading/inference up to 30 minutes and is also forwarded to the dedicated Nginx LLM route as `LLM_PROXY_TIMEOUT_SECONDS`. `LDS_AI_AVAILABILITY_TTL=5`, `LDS_AI_MAX_CONTEXT_BYTES=524288`, `LDS_AI_MAX_REQUEST_BYTES=1048576`, and `LDS_AI_MAX_RESPONSE_BYTES=2097152` remain Tools consumer controls rather than provider-container limits. -Use the `lds` wrapper for provider commands. LocalDevStack intentionally has no repository-root `compose.yml`; its effective Compose project is assembled from `docker/compose/main.yaml`, release/user env files, optional extras, and temporary runtime-specific overrides. Therefore a bare command such as `docker compose exec llm-sm ...` from the LocalDevStack root is not equivalent and fails before reaching the container. Use `lds llm ...` instead. +Use the `lds` wrapper for provider commands. LocalDevStack intentionally has no repository-root `compose.yml`; its effective Compose project is assembled from `docker/compose/main.yaml`, release/user env files, optional extras, and temporary runtime-specific overrides. Therefore a bare command such as `docker compose exec llm-ollama ...` from the LocalDevStack root is not equivalent and fails before reaching the container. Use `lds llm ...` instead. -The `llm-sm` container receives **no Docker socket and no project/repository bind mount** by default. Current published `llm-sm` images are `linux/amd64`; the rest of LocalDevStack can still run on arm64 with the AI profile disabled. +The `llm-ollama` container receives **no Docker socket and no project/repository bind mount** by default. Current published `llm-ollama` images are `linux/amd64`; the rest of LocalDevStack can still run on arm64 with the AI profile disabled. ## Storage and trust boundaries @@ -336,7 +336,7 @@ Important named volumes include `NginxHosts`, `ApacheHosts`, `SSLKeys`, `SSLRoot Generated Nginx/Apache vhosts are Docker-managed state. `lds domain ls`, support tracing, and support bundles read the persisted vhost state through the control plane. -`server-tools` and `runner` intentionally receive `/var/run/docker.sock`. Docker socket access is equivalent to powerful host Docker control. Ordinary databases, admin clients, Nginx/Apache, generated runtimes, and `llm-sm` do not receive it unless a user explicitly opts into a separate ad-hoc runner `--sock` flow. +`server-tools` and `runner` intentionally receive `/var/run/docker.sock`. Docker socket access is equivalent to powerful host Docker control. Ordinary databases, admin clients, Nginx/Apache, generated runtimes, and `llm-ollama` do not receive it unless a user explicitly opts into a separate ad-hoc runner `--sock` flow. ## Operations and support From 5dbf5229d15c6eb47be797071cf7e382aeab930e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:25:59 +0600 Subject: [PATCH 300/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- docs/guides/local-ai.rst | 68 ++++++++++++++++++++-------------------- 1 file changed, 34 insertions(+), 34 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index ca8bfb22..0484a91b 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -2,7 +2,7 @@ Local AI ======== LocalDevStack can run a local Ollama-compatible provider as an optional profile. -``llm-sm`` owns the provider/runtime; Tools remains the higher-level AI consumer and +``llm-ollama`` owns the provider/runtime; Tools remains the higher-level AI consumer and never embeds a second Ollama runtime. Enable AI @@ -18,7 +18,7 @@ The default consumer contract is:: LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama - LDS_AI_URL=http://llm-sm:11434 + LDS_AI_URL=http://llm-ollama:11434 LDS_AI_MODEL=qwen3:14b The explicit model default avoids ambiguity when multiple models are installed in the @@ -62,11 +62,11 @@ Access Internal provider endpoint:: - http://llm-sm:11434 + http://llm-ollama:11434 User-facing HTTPS endpoint:: - https://llm.localhost + https://llm-ollama.localhost Internal Docker consumers should use the service endpoint directly rather than routing through Nginx. @@ -84,7 +84,7 @@ Show/disable it with:: lds llm host-port status lds llm host-port off -The port can be changed through ``LLM_SM_PORT`` when direct host access is enabled. +The port can be changed through ``LLM_OLLAMA_PORT`` when direct host access is enabled. Compose Ownership ----------------- @@ -93,9 +93,9 @@ The tracked provider service exists only in:: docker/compose/companion.yaml -The Compose service key/hostname is ``llm-sm`` and the fixed container name remains -``LLM_SM`` for LocalDevStack single-stack compatibility. Internal consumers must continue -to route through ``llm-sm`` rather than depending on the container name. +The Compose service key/hostname is ``llm-ollama`` and the fixed container name remains +``LLM_OLLAMA`` for LocalDevStack single-stack compatibility. Internal consumers must continue +to route through ``llm-ollama`` rather than depending on the container name. and is included through:: @@ -109,7 +109,7 @@ under ``docker/.runtime/`` for the current command only: - NVIDIA -> ``gpus: all``; - AMD/ROCm -> ``/dev/kfd`` and ``/dev/dri``; -- direct host API -> ``127.0.0.1:${LLM_SM_PORT:-11434}:11434``. +- direct host API -> ``127.0.0.1:${LLM_OLLAMA_PORT:-11434}:11434``. The fragment is removed after the Compose command. ``configuration/compose/`` remains the normal extras/generated-runtime area and is not the location of built-in LLM @@ -154,8 +154,8 @@ the provider image. Always invoke provider commands through ``lds llm`` inside LocalDevStack. The repository does not expose a root ``compose.yml`` because ``lds`` assembles the effective Compose project from the tracked main file, env layers, optional extras, and runtime-specific -temporary overrides. A bare ``docker compose exec llm-sm ...`` from the LocalDevStack -repository root therefore fails at the host Compose layer before ``llm-sm`` runs. +temporary overrides. A bare ``docker compose exec llm-ollama ...`` from the LocalDevStack +repository root therefore fails at the host Compose layer before ``llm-ollama`` runs. Model Persistence ----------------- @@ -178,7 +178,7 @@ Pull a model explicitly:: Then set ``LDS_AI_MODEL`` in ``docker/.env`` if it should become the default for both Tools AI commands and ``lds llm`` provider commands. LocalDevStack forwards that value to -the provider as ``LLM_SM_MODEL``. A command-scoped shell value remains the +the provider as ``LLM_OLLAMA_MODEL``. A command-scoped shell value remains the highest-precedence LocalDevStack override. The provider CLI does not silently download a missing model for an unrelated command. @@ -187,17 +187,17 @@ Provider Settings ----------------- These provider settings can be placed in ``docker/.env`` and are forwarded to -``llm-sm``:: - - LLM_SM_SYSTEM= - LLM_SM_INPUT_WARN_BYTES=1048576 - LLM_SM_INPUT_MAX_BYTES=0 - LLM_SM_ATTACHMENT_MAX_BYTES=16777216 - LLM_SM_ATTACHMENTS_MAX_BYTES=33554432 - LLM_SM_ATTACHMENT_MAX_COUNT=16 - LLM_SM_PDF_MAX_PAGES=24 - LLM_SM_PDF_DPI=120 - LLM_SM_ALLOW_LARGE_INPUT=0 +``llm-ollama``:: + + LLM_OLLAMA_SYSTEM= + LLM_OLLAMA_INPUT_WARN_BYTES=1048576 + LLM_OLLAMA_INPUT_MAX_BYTES=0 + LLM_OLLAMA_ATTACHMENT_MAX_BYTES=16777216 + LLM_OLLAMA_ATTACHMENTS_MAX_BYTES=33554432 + LLM_OLLAMA_ATTACHMENT_MAX_COUNT=16 + LLM_OLLAMA_PDF_MAX_PAGES=24 + LLM_OLLAMA_PDF_DPI=120 + LLM_OLLAMA_ALLOW_LARGE_INPUT=0 LDS_AI_IGPU_ENABLE=1 # auto-persisted for AMD CPU + AMD runtime; otherwise 0 OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_LOADED_MODELS=1 @@ -206,9 +206,9 @@ These provider settings can be placed in ``docker/.env`` and are forwarded to ``LDS_AI_IGPU_ENABLE`` is forwarded as ``OLLAMA_IGPU_ENABLE``. The automatic value is ``1`` only for an AMD CPU with the AMD runtime selected; users may override the persisted value deliberately. -``LLM_SM_INPUT_MAX_BYTES=0`` disables only the hard text/diff ceiling. Attachment and +``LLM_OLLAMA_INPUT_MAX_BYTES=0`` disables only the hard text/diff ceiling. Attachment and PDF-vision limits remain active unless their own value is set to ``0``. -``LLM_SM_ALLOW_LARGE_INPUT=1`` is the explicit escape hatch for a deliberate request. +``LLM_OLLAMA_ALLOW_LARGE_INPUT=1`` is the explicit escape hatch for a deliberate request. Consumer settings are separate. The following values configure the Tools AI layer, not the provider runtime:: @@ -225,16 +225,16 @@ Connection and provider/model preflight remain deliberately short. Generation an analysis use a separate 30-minute default because first model load, CPU inference, and larger local prompts can legitimately take much longer. LocalDevStack also forwards ``LDS_AI_TIMEOUT`` to Nginx as ``LLM_PROXY_TIMEOUT_SECONDS`` for the dedicated -``llm.localhost`` route so the edge proxy does not terminate a valid generation earlier. +``llm-ollama.localhost`` route so the edge proxy does not terminate a valid generation earlier. Privacy and Trust Boundaries ---------------------------- -The base ``llm-sm`` service lives in ``docker/compose/companion.yaml`` and is gated by +The base ``llm-ollama`` service lives in ``docker/compose/companion.yaml`` and is gated by the ``ai`` profile. Runtime-specific hardware/port additions are ephemeral as described above; no separate tracked AI Compose variants exist. -By default, ``llm-sm`` receives: +By default, ``llm-ollama`` receives: - no Docker socket; - no project/repository bind mount; @@ -254,7 +254,7 @@ LocalDevStack does not: Repository Context ------------------ -LocalDevStack intentionally does not mount the project/repository into llm-sm by default. +LocalDevStack intentionally does not mount the project/repository into llm-ollama by default. Repository-aware analysis should normally use the Tools consumer layer:: @@ -263,7 +263,7 @@ Repository-aware analysis should normally use the Tools consumer layer:: lds ai graphify ... For direct provider commands, file/PDF/image paths must exist inside the provider -container. The upstream ``llm-sm`` CLI also supports stdin-based flows such as +container. The upstream ``llm-ollama`` CLI also supports stdin-based flows such as ``ai-commit --diff-stdin`` when explicitly invoked. Graphify @@ -283,7 +283,7 @@ model from ``LDS_AI_MODEL``, derives ``GRAPHIFY_API_TIMEOUT`` from Enable and apply the direct provider port before the default workflow:: lds llm host-port on - lds up -d llm-sm + lds up -d llm-ollama lds graphify Internally the workflow runs extraction with ``--backend ollama --no-cluster`` and, @@ -301,7 +301,7 @@ suppresses Graphify's Ollama-backend warning. An explicitly supplied ``OLLAMA_API_KEY`` is preserved. A one-off ``--model`` or ``--api-timeout`` option is propagated to the clustering -phase through the corresponding Graphify environment value. ``llm-sm`` remains only +phase through the corresponding Graphify environment value. ``llm-ollama`` remains only the model provider and still receives no repository bind mount. Diagnostics @@ -313,12 +313,12 @@ Diagnostics lds llm models lds doctor lds urls - lds logs llm-sm + lds logs llm-ollama Platform Availability --------------------- -The current published ``llm-sm`` standard and AMD images have a linux/amd64 only +The current published ``llm-ollama`` standard and AMD images have a linux/amd64 only platform contract. LocalDevStack itself can still be used on arm64 with the ``ai`` profile disabled. Native From 089f410bdad2078944c06655397947b84086de42 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:06 +0600 Subject: [PATCH 301/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- tests/ai-contract.sh | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 8d4b5b7c..4523d054 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -17,7 +17,7 @@ trap cleanup EXIT docker network create "$network" >/dev/null docker build -q -t "$image" "$ROOT/tests/fixtures/fake-ollama" >/dev/null -docker run -d --name "$container" --network "$network" --network-alias llm-sm "$image" >/dev/null +docker run -d --name "$container" --network "$network" --network-alias llm-ollama "$image" >/dev/null for _ in {1..20}; do if docker exec "$container" python -c 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=1).read()' >/dev/null 2>&1; then @@ -44,7 +44,7 @@ pass "fake Ollama tags/generate/OpenAI-compatible contracts" docker pull infocyph/tools:latest >/dev/null provider_status="$( - docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-sm:11434 -e LDS_AI_MODEL=qwen3:14b infocyph/tools:latest --status + docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-ollama:11434 -e LDS_AI_MODEL=qwen3:14b infocyph/tools:latest --status )" assert_contains "$provider_status" "available=1" assert_contains "$provider_status" "model=qwen3:14b" @@ -57,11 +57,11 @@ fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_SM' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_MODEL=${LDS_AI_MODEL:-qwen3:14b}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_OLLAMA' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'OLLAMA_IGPU_ENABLE=${LDS_AI_IGPU_ENABLE:-0}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_ATTACHMENT_MAX_BYTES=${LLM_SM_ATTACHMENT_MAX_BYTES:-16777216}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_SM_PDF_MAX_PAGES=${LLM_SM_PDF_MAX_PAGES:-24}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_ATTACHMENT_MAX_BYTES=${LLM_OLLAMA_ATTACHMENT_MAX_BYTES:-16777216}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_PDF_MAX_PAGES=${LLM_OLLAMA_PDF_MAX_PAGES:-24}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800}' From ba44caa251573c174d1de33fbd49d3fadf3acbda Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:12 +0600 Subject: [PATCH 302/509] refactor(ai): rename LocalDevStack Ollama runtime contract --- tests/compose-contract.sh | 44 +++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 22 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 8ab0b24d..b6fdd767 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -127,7 +127,7 @@ python3 -c ' import json,sys d=json.load(sys.stdin) assert "apache" in d.get("services", {}) -assert "llm-sm" not in d.get("services", {}) +assert "llm-ollama" not in d.get("services", {}) assert d["volumes"]["lds_tools_state"]["name"] == "ToolsState" tools=d["services"]["server-tools"] targets={v["target"] for v in tools["volumes"]} @@ -139,25 +139,25 @@ ai_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' import json,sys d=json.load(sys.stdin) -s=d["services"]["llm-sm"] +s=d["services"]["llm-ollama"] assert s["image"] == "infocyph/llm-ollama:latest" -assert s["container_name"] == "LLM_SM" +assert s["container_name"] == "LLM_OLLAMA" assert not s.get("ports") assert set(s["networks"]) == {"frontend","backend"} targets={v["target"] for v in s["volumes"]} assert targets == {"/root/.ollama"} assert d["volumes"]["lds_llm"]["name"] == "LLMModels" env=s["environment"] -assert env["LLM_SM_MODEL"] == "qwen3:14b" -assert env["LLM_SM_SYSTEM"] == "" -assert env["LLM_SM_INPUT_WARN_BYTES"] == "1048576" -assert env["LLM_SM_INPUT_MAX_BYTES"] == "0" -assert env["LLM_SM_ATTACHMENT_MAX_BYTES"] == "16777216" -assert env["LLM_SM_ATTACHMENTS_MAX_BYTES"] == "33554432" -assert env["LLM_SM_ATTACHMENT_MAX_COUNT"] == "16" -assert env["LLM_SM_PDF_MAX_PAGES"] == "24" -assert env["LLM_SM_PDF_DPI"] == "120" -assert env["LLM_SM_ALLOW_LARGE_INPUT"] == "0" +assert env["LLM_OLLAMA_MODEL"] == "qwen3:14b" +assert env["LLM_OLLAMA_SYSTEM"] == "" +assert env["LLM_OLLAMA_INPUT_WARN_BYTES"] == "1048576" +assert env["LLM_OLLAMA_INPUT_MAX_BYTES"] == "0" +assert env["LLM_OLLAMA_ATTACHMENT_MAX_BYTES"] == "16777216" +assert env["LLM_OLLAMA_ATTACHMENTS_MAX_BYTES"] == "33554432" +assert env["LLM_OLLAMA_ATTACHMENT_MAX_COUNT"] == "16" +assert env["LLM_OLLAMA_PDF_MAX_PAGES"] == "24" +assert env["LLM_OLLAMA_PDF_DPI"] == "120" +assert env["LLM_OLLAMA_ALLOW_LARGE_INPUT"] == "0" assert env["OLLAMA_NUM_PARALLEL"] == "1" assert env["OLLAMA_MAX_LOADED_MODELS"] == "1" assert env["OLLAMA_KEEP_ALIVE"] == "5m" @@ -166,7 +166,7 @@ assert env["OLLAMA_IGPU_ENABLE"] == "0" tools=d["services"]["server-tools"]["environment"] assert tools["LDS_AI_ENABLED"] == "auto" assert tools["LDS_AI_PROVIDER"] == "ollama" -assert tools["LDS_AI_URL"] == "http://llm-sm:11434" +assert tools["LDS_AI_URL"] == "http://llm-ollama:11434" assert tools["LDS_AI_MODEL"] == "qwen3:14b" assert tools["LDS_AI_CONNECT_TIMEOUT"] == "2" assert tools["LDS_AI_PREFLIGHT_TIMEOUT"] == "5" @@ -180,15 +180,15 @@ assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" -printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_SM_PDF_MAX_PAGES=12' 'LLM_SM_SYSTEM=Answer briefly.' 'LDS_AI_TIMEOUT=2400' 'LDS_AI_IGPU_ENABLE=1' >>"$user_env" +printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_OLLAMA_PDF_MAX_PAGES=12' 'LLM_OLLAMA_SYSTEM=Answer briefly.' 'LDS_AI_TIMEOUT=2400' 'LDS_AI_IGPU_ENABLE=1' >>"$user_env" ai_override_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' import json,sys d=json.load(sys.stdin) -llm=d["services"]["llm-sm"]["environment"] -assert llm["LLM_SM_MODEL"] == "qwen2.5:1.5b" -assert llm["LLM_SM_PDF_MAX_PAGES"] == "12" -assert llm["LLM_SM_SYSTEM"] == "Answer briefly." +llm=d["services"]["llm-ollama"]["environment"] +assert llm["LLM_OLLAMA_MODEL"] == "qwen2.5:1.5b" +assert llm["LLM_OLLAMA_PDF_MAX_PAGES"] == "12" +assert llm["LLM_OLLAMA_SYSTEM"] == "Answer briefly." assert llm["OLLAMA_IGPU_ENABLE"] == "1" tools=d["services"]["server-tools"]["environment"] assert tools["LDS_AI_TIMEOUT"] == "2400" @@ -200,7 +200,7 @@ pass "LocalDevStack forwards configured provider and generation-timeout options" amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys -s=json.load(sys.stdin)["services"]["llm-sm"] +s=json.load(sys.stdin)["services"]["llm-ollama"] assert s["image"] == "infocyph/llm-ollama:amd-latest" devices=" ".join(str(x) for x in s.get("devices", [])) assert "/dev/kfd" in devices and "/dev/dri" in devices @@ -210,7 +210,7 @@ pass "AMD AI runtime is generated dynamically" nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys -s=json.load(sys.stdin)["services"]["llm-sm"] +s=json.load(sys.stdin)["services"]["llm-ollama"] assert s["image"] == "infocyph/llm-ollama:latest" assert s.get("gpus") ' <<<"$nvidia_json" @@ -219,7 +219,7 @@ pass "NVIDIA AI runtime is generated dynamically" host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys -ports=json.load(sys.stdin)["services"]["llm-sm"]["ports"] +ports=json.load(sys.stdin)["services"]["llm-ollama"]["ports"] assert len(ports) == 1 p=ports[0] assert p["host_ip"] == "127.0.0.1" From a37117ced7b395176943dda363c8aac8328592f2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:24 +0600 Subject: [PATCH 303/509] docs(ai): align llm-ollama runtime and domain contract --- docs/concepts/architecture.rst | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index e3f3bbb5..e121b97b 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -101,7 +101,7 @@ Service-to-service traffic uses Docker DNS names such as:: mongodb redis elasticsearch - llm-sm + llm-ollama The historical ``lds vpn-fix`` command remains only as a deprecated compatibility message because LocalDevStack no longer owns fixed bridge subnets. @@ -143,16 +143,16 @@ AI Flow When the ``ai`` profile is enabled: -1. ``llm-sm`` provides the Ollama runtime and persistent model store; -2. Tools consumes ``http://llm-sm:11434`` internally; -3. Nginx exposes ``https://llm.localhost`` to the user; +1. ``llm-ollama`` provides the Ollama runtime and persistent model store; +2. Tools consumes ``http://llm-ollama:11434`` internally; +3. Nginx exposes ``https://llm-ollama.localhost`` to the user; 4. ``lds ai`` delegates higher-level/operational AI to Tools; -5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-sm`` CLI. +5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-ollama`` CLI. -The provider is one ``llm-sm`` service declared in +The provider is one ``llm-ollama`` service declared in ``docker/compose/companion.yaml`` and enabled only by the ``ai`` profile. The service -keeps ``container_name: LLM_SM`` for current single-stack compatibility, while all -internal routing continues to use the Compose service/hostname ``llm-sm``. Its image +keeps ``container_name: LLM_OLLAMA`` for current single-stack compatibility, while all +internal routing continues to use the Compose service/hostname ``llm-ollama``. Its image is ``infocyph/llm-ollama:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and AMD/ROCm resolves to ``amd-latest``. ``LDS_LLM_ARCH`` is derived from the effective runtime rather than maintained as an independent version selector. @@ -160,7 +160,7 @@ runtime rather than maintained as an independent version selector. No AI-specific Compose files are tracked. ``lds`` generates a temporary fragment under ``docker/.runtime/`` only when NVIDIA GPU access, AMD device mappings, or loopback host-port exposure is required, then removes it after the Compose command. The service -also forwards the configured ``LDS_AI_MODEL`` to the provider as ``LLM_SM_MODEL`` and +also forwards the configured ``LDS_AI_MODEL`` to the provider as ``LLM_OLLAMA_MODEL`` and passes the documented provider safety/tuning settings from ``docker/.env``. For AMD runtime on an AMD CPU, LocalDevStack persists ``LDS_AI_IGPU_ENABLE=1`` and forwards it as ``OLLAMA_IGPU_ENABLE=1`` so Ollama admits the integrated Radeon GPU. @@ -195,7 +195,7 @@ equivalent to powerful host Docker control. The Docker socket is not mounted into: -- ``llm-sm``; +- ``llm-ollama``; - databases; - database admin clients; - Nginx or Apache; @@ -204,7 +204,7 @@ The Docker socket is not mounted into: The separate ad-hoc ``lds run --sock`` option is an explicit opt-in and should be used only with trusted Dockerfiles/code. -``llm-sm`` also receives no project/repository mount by default. AI output is not +``llm-ollama`` also receives no project/repository mount by default. AI output is not automatically executed as shell, SQL, or code. Persistence From 9dd7e2c5a5a697e7b502a1c3028f3d57b1ae7bbc Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:31 +0600 Subject: [PATCH 304/509] docs(ai): align llm-ollama runtime and domain contract --- docs/concepts/profiles-and-env.rst | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index 8e8946f7..a487a1df 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -209,12 +209,12 @@ Important AI settings include:: LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama - LDS_AI_URL=http://llm-sm:11434 + LDS_AI_URL=http://llm-ollama:11434 LDS_AI_MODEL=qwen3:14b LDS_AI_RUNTIME= LDS_AI_IGPU_ENABLE= LDS_LLM_HOST_PORT=0 - LLM_SM_PORT=11434 + LLM_OLLAMA_PORT=11434 When ``LDS_AI_RUNTIME`` is not explicitly set, LocalDevStack detects the preferred runtime for the Compose invocation. NVIDIA is selected only when ``nvidia-smi`` is @@ -236,19 +236,19 @@ Use:: to override the detected runtime or host-port behavior. The selected ``LDS_AI_MODEL`` is also forwarded to the provider as -``LLM_SM_MODEL``, so Tools and ``lds llm`` share the same default model. +``LLM_OLLAMA_MODEL``, so Tools and ``lds llm`` share the same default model. Provider-side options accepted in ``docker/.env`` include:: - LLM_SM_SYSTEM= - LLM_SM_INPUT_WARN_BYTES=1048576 - LLM_SM_INPUT_MAX_BYTES=0 - LLM_SM_ATTACHMENT_MAX_BYTES=16777216 - LLM_SM_ATTACHMENTS_MAX_BYTES=33554432 - LLM_SM_ATTACHMENT_MAX_COUNT=16 - LLM_SM_PDF_MAX_PAGES=24 - LLM_SM_PDF_DPI=120 - LLM_SM_ALLOW_LARGE_INPUT=0 + LLM_OLLAMA_SYSTEM= + LLM_OLLAMA_INPUT_WARN_BYTES=1048576 + LLM_OLLAMA_INPUT_MAX_BYTES=0 + LLM_OLLAMA_ATTACHMENT_MAX_BYTES=16777216 + LLM_OLLAMA_ATTACHMENTS_MAX_BYTES=33554432 + LLM_OLLAMA_ATTACHMENT_MAX_COUNT=16 + LLM_OLLAMA_PDF_MAX_PAGES=24 + LLM_OLLAMA_PDF_DPI=120 + LLM_OLLAMA_ALLOW_LARGE_INPUT=0 OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_LOADED_MODELS=1 OLLAMA_KEEP_ALIVE=5m From 0e08cdfdc932acff7a6171c4e2dfb212e1a9b7f2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:36 +0600 Subject: [PATCH 305/509] docs(ai): align llm-ollama runtime and domain contract --- docs/reference/cli.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index c914b225..62bc163e 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -212,14 +212,14 @@ It performs ``extract --backend ollama --no-cluster`` followed by By default it derives: -- ``OLLAMA_BASE_URL`` from the actual loopback-published ``llm-sm`` port; +- ``OLLAMA_BASE_URL`` from the actual loopback-published ``llm-ollama`` port; - ``OLLAMA_MODEL`` from ``LDS_AI_MODEL``; - ``GRAPHIFY_API_TIMEOUT`` from ``LDS_AI_TIMEOUT``. The host port must already be applied with:: lds llm host-port on - lds up -d llm-sm + lds up -d llm-ollama An explicitly supplied ``OLLAMA_BASE_URL`` bypasses that host-port lookup. @@ -248,10 +248,10 @@ LLM Provider lds llm help These commands execute the bundled provider CLI through LocalDevStack's Compose wrapper. -Do not replace them with bare ``docker compose exec llm-sm ...`` from the repository +Do not replace them with bare ``docker compose exec llm-ollama ...`` from the repository root; LocalDevStack has no root ``compose.yml``. -``LDS_AI_MODEL`` is forwarded to the provider as ``LLM_SM_MODEL``, so +``LDS_AI_MODEL`` is forwarded to the provider as ``LLM_OLLAMA_MODEL``, so ``lds ai`` and ``lds llm`` share the configured default model. Provider input, attachment, PDF and Ollama runtime knobs are documented in :doc:`../guides/local-ai`. @@ -270,7 +270,7 @@ Direct host-port control:: lds llm host-port on lds llm host-port off -The loopback port defaults to 11434 and can be changed with ``LLM_SM_PORT``. +The loopback port defaults to 11434 and can be changed with ``LLM_OLLAMA_PORT``. Rebuild ------- From 824963a551bd7ea06b2739c0cbf97e7097ee70e7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:43 +0600 Subject: [PATCH 306/509] docs(ai): align llm-ollama runtime and domain contract --- docs/plans/docker-ecosystem/00-master-bottom-up-plan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md index da276b72..2f54aba4 100644 --- a/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md +++ b/docs/plans/docker-ecosystem/00-master-bottom-up-plan.md @@ -198,7 +198,7 @@ Only after lower layers have stable contracts: - migrate static networking to Docker DNS; - reconcile generated-state/storage documentation with actual named volumes/bind mounts; - consume versioned infrastructure images; -- integrate optional `llm-sm` cleanly; +- integrate optional `llm-ollama` cleanly; - review Docker socket exposure; - keep PHP/Node runtime generation flexible. From 15f5bfdde253e9f80d0c05017ddb931c64328108 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:46 +0600 Subject: [PATCH 307/509] docs(ai): align llm-ollama runtime and domain contract --- .../docker-ecosystem/06-docker-llm-sm-plan.md | 24 +++++++++---------- 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md index be16dc2a..546325c1 100644 --- a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md +++ b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md @@ -1,8 +1,8 @@ -# docker-llm-sm — File-by-File Compatibility + Integration Plan +# docker-llm-ollama — File-by-File Compatibility + Integration Plan ## Role -`infocyph/docker-llm-sm` is the optional local-AI service for the LocalDevStack ecosystem. It is already published and already uses the newer single-repository tag model: +`infocyph/docker-llm-ollama` is the optional local-AI service for the LocalDevStack ecosystem. It is already published and already uses the newer single-repository tag model: - standard CPU/NVIDIA: `latest`, `` - AMD ROCm: `amd-latest`, `amd-` @@ -16,7 +16,7 @@ This plan is intentionally conservative. The image should remain independently u - `/root/.ollama` must be persisted by the consumer/orchestrator. - CLI commands bundled in the image remain available; users do not install/remove the CLI separately. - CPU/NVIDIA and AMD ROCm remain separate tag families within the same registry repository. -- LocalDevStack must not rebuild `docker-llm-sm` locally. +- LocalDevStack must not rebuild `docker-llm-ollama` locally. - Graphify or other AI clients consume the Ollama endpoint; they are not baked into this image merely because they can use it. ## Existing Files @@ -80,7 +80,7 @@ Keep published-image/persistent-volume NVIDIA example and GPU request semantics. Keep `amd-latest`/ROCm example and `/dev/kfd` + `/dev/dri` device requirements. -### `scripts/llm-sm` +### `scripts/llm-ollama` Keep as fixed dispatcher installed in the image. @@ -165,24 +165,24 @@ This lower-layer plan is implemented through the authoritative LocalDevStack int plan in `07-localdevstack-integration-plan.md`. LocalDevStack consumes the published provider through the optional `ai` profile. It does -not rebuild `docker-llm-sm` locally and does not consume this repository's standalone +not rebuild `docker-llm-ollama` locally and does not consume this repository's standalone Compose files. Current LocalDevStack controls: - `LDS_AI_MODEL` selects the shared Tools/provider default model and is forwarded as - `LLM_SM_MODEL`; + `LLM_OLLAMA_MODEL`; - `LDS_AI_RUNTIME` selects `cpu`, `nvidia`, or `amd` when explicitly configured; - `LDS_LLM_ARCH` is derived from the effective runtime (`latest` for CPU/NVIDIA, `amd-latest` for AMD/ROCm); - `LDS_LLM_HOST_PORT` controls optional direct loopback exposure; -- `LLM_SM_PORT` controls that loopback host port; +- `LLM_OLLAMA_PORT` controls that loopback host port; - provider input/PDF/Ollama tuning values are forwarded from LocalDevStack `docker/.env`. Compose ownership: -- one tracked `llm-sm` service in `docker/compose/companion.yaml`; +- one tracked `llm-ollama` service in `docker/compose/companion.yaml`; - no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml`; - NVIDIA, AMD/ROCm, and host-port additions are generated temporarily under @@ -195,15 +195,15 @@ Persistence: Workspace: -- LocalDevStack intentionally does not mount a project/repository into `llm-sm` by +- LocalDevStack intentionally does not mount a project/repository into `llm-ollama` by default; - repository-aware analysis normally uses the Tools consumer layer; - provider stdin flows remain available without weakening the default trust boundary. Networking: -- internal consumers use `http://llm-sm:11434`; -- Nginx exposes `https://llm.localhost`; +- internal consumers use `http://llm-ollama:11434`; +- Nginx exposes `https://llm-ollama.localhost`; - direct host `11434`-style access is opt-in and loopback-only. User/provider commands are invoked through `lds llm ...`, not a bare @@ -211,7 +211,7 @@ User/provider commands are invoked through `lds llm ...`, not a bare ## Acceptance Criteria -1. Existing published `docker-llm-sm` behavior remains standalone and stable. +1. Existing published `docker-llm-ollama` behavior remains standalone and stable. 2. LocalDevStack can enable it without building locally. 3. Standard and AMD tags are selectable. 4. Named volume persists pulled models across container recreation. From 858810a95b73c81d1a99e63644b9a37116e397e6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:52 +0600 Subject: [PATCH 308/509] docs(ai): align llm-ollama runtime and domain contract --- .../07-localdevstack-integration-plan.md | 150 +++++++++--------- 1 file changed, 75 insertions(+), 75 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 6a513842..5d552a79 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -32,8 +32,8 @@ LocalDevStack implementation must begin against this tested ecosystem set: | Nginx | `infocyph/nginx:0.4.1` | | Apache | `infocyph/apache:0.4.2` | | Tools | `infocyph/tools:0.23.2` | -| LLM standard | `infocyph/llm-sm:latest` | -| LLM AMD | `infocyph/llm-sm:amd-latest` | +| LLM standard | `infocyph/llm-ollama:latest` | +| LLM AMD | `infocyph/llm-ollama:amd-latest` | The implemented image policy now follows the ecosystem moving aliases for LocalDevStack infrastructure. Standard LLM uses `latest`; AMD/ROCm uses `amd-latest`. Release @@ -87,7 +87,7 @@ Host │ ├── admin.localhost -> server-tools:9911 │ ├── webmail.localhost -> mailpit:8025 │ ├── db/ri/me/kibana convenience routes - │ └── llm.localhost -> llm-sm:11434 + │ └── llm-ollama.localhost -> llm-ollama:11434 │ ├── apache 0.4.2 (optional backend) ├── PHP runtimes (local builds) @@ -96,7 +96,7 @@ Host │ ├── host/domain/TLS/config control plane │ ├── monitoring/admin panel │ ├── askai / aiops / gitx AI consumer paths - │ └── http://llm-sm:11434 + │ └── http://llm-ollama:11434 │ ├── runner 0.5 │ └── Supervisor / cron / logrotate / sibling exec @@ -104,7 +104,7 @@ Host ├── mailpit ├── databases and admin clients │ - └── llm-sm current stable (optional) + └── llm-ollama current stable (optional) ├── qwen3:14b baked default ├── persistent /root/.ollama └── Ollama API :11434 @@ -113,13 +113,13 @@ Host Service-to-service AI traffic must use: ```text -http://llm-sm:11434 +http://llm-ollama:11434 ``` Host/user-facing AI traffic must use: ```text -https://llm.localhost +https://llm-ollama.localhost ``` Do not route Tools -> LLM traffic through Nginx. @@ -172,12 +172,12 @@ Deleting volumes must remain an explicit destructive action. - AI is optional. - `docker-tools` is an AI consumer, never an Ollama runtime. -- `docker-llm-sm` is the only LocalDevStack Ollama/model runtime. -- LocalDevStack must remain fully usable when `llm-sm` is absent. +- `docker-llm-ollama` is the only LocalDevStack Ollama/model runtime. +- LocalDevStack must remain fully usable when `llm-ollama` is absent. - No automatic execution of model-generated shell, SQL or code is introduced. - No external/cloud AI fallback is added by LocalDevStack. -- No Docker socket is mounted into `llm-sm`. -- No repository/workspace is mounted into `llm-sm` by default. +- No Docker socket is mounted into `llm-ollama`. +- No repository/workspace is mounted into `llm-ollama` by default. ## 3.6 Single-stack compatibility @@ -317,17 +317,17 @@ Validate: Do not download a 3B model on every LocalDevStack PR. -Use a lightweight fake Ollama-compatible service named `llm-sm` for the normal PR test. +Use a lightweight fake Ollama-compatible service named `llm-ollama` for the normal PR test. Validate: -- Tools `askai --status` reaches `http://llm-sm:11434`; +- Tools `askai --status` reaches `http://llm-ollama:11434`; - Tools `aiops provider` works; -- Nginx `llm.localhost` reaches the fake service; +- Nginx `llm-ollama.localhost` reaches the fake service; - streamed response is not buffered incorrectly; - AI absence leaves core services healthy. -A manual/release-gate job may optionally exercise the real published `infocyph/llm-sm:latest`, because that image already has its own model-bearing runtime gate. +A manual/release-gate job may optionally exercise the real published `infocyph/llm-ollama:latest`, because that image already has its own model-bearing runtime gate. ## 5.2 New `tests/` @@ -373,7 +373,7 @@ SCRIPTOMATIC_REF=main Tools, Runner, Nginx and Apache are declared directly as their moving `:latest` product images. The LLM service is declared directly as -`infocyph/llm-sm:${LDS_LLM_ARCH}`, where `LDS_LLM_ARCH` is derived from the effective +`infocyph/llm-ollama:${LDS_LLM_ARCH}`, where `LDS_LLM_ARCH` is derived from the effective AI runtime (`latest` for CPU/NVIDIA, `amd-latest` for AMD/ROCm). Do not add `LDS_TOOLS_IMAGE`, `LDS_RUNNER_IMAGE`, `LDS_NGINX_IMAGE`, @@ -408,7 +408,7 @@ image: ${LDS_NGINX_IMAGE:-infocyph/nginx:0.4.1} image: ${LDS_APACHE_IMAGE:-infocyph/apache:0.4.2} ``` -Do the equivalent for `llm-sm`. +Do the equivalent for `llm-ollama`. ## 6.3 Update command / future dependency bumps @@ -532,9 +532,9 @@ Validate all of these without fixed IPs: - Nginx -> DB UIs; - Nginx -> Apache; - Nginx -> Node app; -- Nginx -> llm-sm; +- Nginx -> llm-ollama; - Tools -> DB/service diagnostics; -- Tools -> llm-sm; +- Tools -> llm-ollama; - Runner -> PHP/Node sibling execution; - DB clients -> databases; - Filebeat -> Elasticsearch; @@ -546,15 +546,15 @@ Validate all of these without fixed IPs: AI should become a first-class optional LocalDevStack capability while remaining absent from the default stack. -## 8.1 Single `llm-sm` service in `docker/compose/companion.yaml` +## 8.1 Single `llm-ollama` service in `docker/compose/companion.yaml` Base service: ```yaml services: - llm-sm: - container_name: LLM_SM - image: infocyph/llm-sm:${LDS_LLM_ARCH} + llm-ollama: + container_name: LLM_OLLAMA + image: infocyph/llm-ollama:${LDS_LLM_ARCH} restart: unless-stopped profiles: [ai] volumes: @@ -566,9 +566,9 @@ services: Important rules: -- service key must be exactly `llm-sm`; -- preserve the existing fixed container name `LLM_SM` for single-stack compatibility; -- route internally by the Compose service/hostname `llm-sm`, not by the fixed container name; +- service key must be exactly `llm-ollama`; +- preserve the existing fixed container name `LLM_OLLAMA` for single-stack compatibility; +- route internally by the Compose service/hostname `llm-ollama`, not by the fixed container name; - do not set a fixed IP; - do not mount Docker socket; - do not expose `11434` to all interfaces; @@ -578,8 +578,8 @@ Important rules: Why both networks: -- Nginx must reach `llm-sm:11434` for `https://llm.localhost`; -- Tools must reach `llm-sm:11434` directly for AI consumer commands. +- Nginx must reach `llm-ollama:11434` for `https://llm-ollama.localhost`; +- Tools must reach `llm-ollama:11434` directly for AI consumer commands. ## 8.2 `docker/compose/main.yaml` volume @@ -641,7 +641,7 @@ selects the AMD/ROCm image. `lds llm runtime ...` remains the explicit override. Default LocalDevStack access is: ```text -https://llm.localhost +https://llm-ollama.localhost ``` Do not expose `11434` by default. @@ -651,7 +651,7 @@ same temporary `docker/.runtime/ai.*` fragment used for hardware augmentation, b only: ```text -127.0.0.1:${LLM_SM_PORT:-11434}:11434 +127.0.0.1:${LLM_OLLAMA_PORT:-11434}:11434 ``` Never default to `0.0.0.0:11434`. @@ -663,14 +663,14 @@ No new Nginx image changes are required. `nginx:0.4.1` already reserves: ```text -llm.localhost -> llm-sm:11434 +llm-ollama.localhost -> llm-ollama:11434 ``` and uses lazy Docker DNS resolution plus streaming proxy behavior. LocalDevStack must validate: -- certificate coverage for `llm.localhost`; +- certificate coverage for `llm-ollama.localhost`; - HTTP -> HTTPS redirect; - `/api/tags`; - `/api/generate`; @@ -686,7 +686,7 @@ Pass the published Tools AI contract into `server-tools`: ```text LDS_AI_ENABLED=auto LDS_AI_PROVIDER=ollama -LDS_AI_URL=http://llm-sm:11434 +LDS_AI_URL=http://llm-ollama:11434 LDS_AI_MODEL=qwen3:14b ``` @@ -704,31 +704,31 @@ Also pass through supported advanced limits only when the user sets them: Keep connection and preflight bounds short, but give generation/analysis a 30-minute default. Forward `LDS_AI_TIMEOUT` to the Nginx container as -`LLM_PROXY_TIMEOUT_SECONDS` so the dedicated `llm.localhost` route has the same +`LLM_PROXY_TIMEOUT_SECONDS` so the dedicated `llm-ollama.localhost` route has the same long-running request budget. Do not reintroduce a shorter independent UI/process timeout for Admin AI analysis. Why LocalDevStack should default `LDS_AI_MODEL=qwen3:14b`: -- `llm-sm:latest` ships that model; +- `llm-ollama:latest` ships that model; - the 14B Qwen3 default provides materially stronger instruction/structured-output behavior than the previous 3B model while remaining practical on modern 32 GB unified-memory developer hosts; - Tools intentionally reports ambiguity when multiple models are installed and no model is selected; - users may pull more models without breaking Tools AI workflows. Users can change `LDS_AI_MODEL` explicitly. LocalDevStack forwards it to the provider -as `LLM_SM_MODEL` so Tools and `lds llm` share the same default model. - -Provider settings accepted through `docker/.env` and forwarded to `llm-sm` are: - -- `LLM_SM_SYSTEM`; -- `LLM_SM_INPUT_WARN_BYTES`; -- `LLM_SM_INPUT_MAX_BYTES`; -- `LLM_SM_ATTACHMENT_MAX_BYTES`; -- `LLM_SM_ATTACHMENTS_MAX_BYTES`; -- `LLM_SM_ATTACHMENT_MAX_COUNT`; -- `LLM_SM_PDF_MAX_PAGES`; -- `LLM_SM_PDF_DPI`; -- `LLM_SM_ALLOW_LARGE_INPUT`; +as `LLM_OLLAMA_MODEL` so Tools and `lds llm` share the same default model. + +Provider settings accepted through `docker/.env` and forwarded to `llm-ollama` are: + +- `LLM_OLLAMA_SYSTEM`; +- `LLM_OLLAMA_INPUT_WARN_BYTES`; +- `LLM_OLLAMA_INPUT_MAX_BYTES`; +- `LLM_OLLAMA_ATTACHMENT_MAX_BYTES`; +- `LLM_OLLAMA_ATTACHMENTS_MAX_BYTES`; +- `LLM_OLLAMA_ATTACHMENT_MAX_COUNT`; +- `LLM_OLLAMA_PDF_MAX_PAGES`; +- `LLM_OLLAMA_PDF_DPI`; +- `LLM_OLLAMA_ALLOW_LARGE_INPUT`; - `OLLAMA_NUM_PARALLEL`; - `OLLAMA_MAX_LOADED_MODELS`; - `OLLAMA_KEEP_ALIVE`; @@ -778,17 +778,17 @@ lds llm json ... lds llm ai-commit ... ``` -These should delegate to the bundled `llm-sm` CLI inside the provider container. +These should delegate to the bundled `llm-ollama` CLI inside the provider container. LocalDevStack must not reimplement Ollama/model logic. ## 8.8 Workspace/repository access -Default: no project mount into `llm-sm`. +Default: no project mount into `llm-ollama`. Tools already has the LocalDevStack project mounted at `/app` and its AI layer applies size/sensitivity/redaction guards. -For direct `llm-sm` repository-aware commands: +For direct `llm-ollama` repository-aware commands: - prefer stdin/file transfer where practical; - optionally provide a separate read-only workspace override; @@ -799,22 +799,22 @@ Never automatically mount arbitrary host repositories. ## 8.9 Graphify -Do not install Graphify into `llm-sm` or Tools solely for this integration. +Do not install Graphify into `llm-ollama` or Tools solely for this integration. Supported patterns: - `lds graphify [path] [extract-options...]` -> host Graphify workflow against the loopback-published provider, using `LDS_AI_MODEL` and `LDS_AI_TIMEOUT` by default; -- explicit host Graphify may use `https://llm.localhost/v1` or an overridden +- explicit host Graphify may use `https://llm-ollama.localhost/v1` or an overridden `OLLAMA_BASE_URL`; -- container Graphify on a shared network -> `http://llm-sm:11434/v1`; +- container Graphify on a shared network -> `http://llm-ollama:11434/v1`; - Tools `aiops graphify --file ` -> analyze an explicitly supplied Graphify output file. The `lds graphify` workflow runs `extract --backend ollama --no-cluster` followed by `cluster-only --backend ollama` so the requested two-phase flow clusters once rather than re-clustering immediately after the extraction command's default clustering pass. Graphify remains a host/external consumer and is not installed into -`llm-sm` or Tools. +`llm-ollama` or Tools. ## 8.10 AI admin panel @@ -1111,7 +1111,7 @@ Only LocalDevStack orchestration/delegation: - profile enabled?; - compose override selection; - `lds ai` -> Tools; -- `lds llm` -> llm-sm; +- `lds llm` -> llm-ollama; - model/runtime status. No provider implementation. @@ -1211,7 +1211,7 @@ or equivalent internal helpers invoked by `lds`. Keep them thin. -They must not duplicate `askai`, `aiops` or `llm-sm` logic. +They must not duplicate `askai`, `aiops` or `llm-ollama` logic. --- @@ -1369,7 +1369,7 @@ Runner helpers and user scheduler definitions may execute into sibling container 1. inventory exact Docker commands used; 2. classify read/write/destructive operations; 3. retain required socket mounts; -4. ensure `llm-sm` never receives the socket; +4. ensure `llm-ollama` never receives the socket; 5. do not mount socket into ordinary databases/admin clients; 6. document that LocalDevStack is trusted local developer infrastructure. @@ -1602,7 +1602,7 @@ https://db.localhost https://ri.localhost https://me.localhost https://kibana.localhost -https://llm.localhost # when AI enabled +https://llm-ollama.localhost # when AI enabled ``` Only show profile-dependent URLs when relevant. @@ -1706,7 +1706,7 @@ Explain Docker DNS routing. ## `docs/guides/tls-and-certificates.rst` -Ensure `llm.localhost` and convenience-host certificate behavior is covered. +Ensure `llm-ollama.localhost` and convenience-host certificate behavior is covered. ## New `docs/guides/local-ai.rst` @@ -1714,7 +1714,7 @@ Cover: - enabling AI; - CPU/NVIDIA/AMD; -- `https://llm.localhost`; +- `https://llm-ollama.localhost`; - `lds ai`; - `lds llm`; - model persistence; @@ -1764,7 +1764,7 @@ Detect/rebuild only when relevant inputs changed. ## Existing users without AI -Their stack should not pull `llm-sm`, create the model volume or consume GPU resources unless AI is selected. +Their stack should not pull `llm-ollama`, create the model volume or consume GPU resources unless AI is selected. --- @@ -1831,11 +1831,11 @@ With fake provider on normal CI: With real provider on manual/release gate when feasible: -- `infocyph/llm-sm:latest`; +- `infocyph/llm-ollama:latest`; - baked `qwen3:14b`; - persistent model volume; - Tools generation; -- Nginx `llm.localhost`. +- Nginx `llm-ollama.localhost`. ## Platforms @@ -1895,8 +1895,8 @@ This LocalDevStack phase is complete when all of the following are true: 9. Mailpit remains persistent and TLS-capable. 10. Runner cron/Supervisor/logrotate workflows still work. 11. AI can be omitted completely with no degradation to the default product. -12. When AI is enabled, `llm-sm` persists models and is reachable internally at `http://llm-sm:11434`. -13. `https://llm.localhost` works through Nginx streaming proxy behavior. +12. When AI is enabled, `llm-ollama` persists models and is reachable internally at `http://llm-ollama:11434`. +13. `https://llm-ollama.localhost` works through Nginx streaming proxy behavior. 14. Tools `askai`, `aiops` and AI-enabled `gitx` use the separate LLM provider. 15. No LocalDevStack component embeds a second Ollama runtime. 16. No AI component auto-executes model-generated commands. @@ -2019,7 +2019,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - optional `ai` profile; - persistent `LLMModels`; - CPU/NVIDIA/AMD modes; - - `https://llm.localhost`; + - `https://llm-ollama.localhost`; - loopback-only optional direct host port; - Tools consumer wiring; - `lds ai` / `lds llm` separation; @@ -2068,7 +2068,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - Prefer a moving Alpine variant when an image family provides a suitable one; otherwise use its normal moving latest alias. - PostgreSQL defaults to `postgres:alpine`. - Tools, Runner, Nginx and Apache consume their published `:latest` aliases. -- One LLM service uses `infocyph/llm-sm:${LDS_LLM_ARCH}`; CPU/NVIDIA map to `latest`, AMD/ROCm maps to `amd-latest`. +- One LLM service uses `infocyph/llm-ollama:${LDS_LLM_ARCH}`; CPU/NVIDIA map to `latest`, AMD/ROCm maps to `amd-latest`. - Elasticsearch, Kibana and Filebeat remain version-aligned on the tested stable version because their required image contract does not provide a suitable moving `latest` alias. - PHP/Node runtime selection remains user-driven and version-specific. - Existing named volumes and container names remain intentionally stable for this release. @@ -2115,7 +2115,7 @@ No legacy LocalDevStack command/service/storage feature was removed: New runtime additions are additive: -- `llm-sm`; +- `llm-ollama`; - `LLMModels`; - `ToolsState`; - AI/QoL/diagnostic commands. @@ -2185,7 +2185,7 @@ This is a non-blocking follow-up, not a release defect. The current published LLM-SM image contract is **linux/amd64 only**. LocalDevStack remains usable on arm64 with the `ai` profile disabled. -LocalDevStack intentionally does not mount a repository/workspace into `llm-sm` by +LocalDevStack intentionally does not mount a repository/workspace into `llm-ollama` by default. Direct model/API/chat/stdin workflows are supported. Repository-aware analysis is supported through the Tools consumer layer (`lds ai review`, `repo-review`). @@ -2244,12 +2244,12 @@ docker/compose/companion.yaml with: ```yaml -llm-sm: - container_name: LLM_SM - image: infocyph/llm-sm:${LDS_LLM_ARCH} +llm-ollama: + container_name: LLM_OLLAMA + image: infocyph/llm-ollama:${LDS_LLM_ARCH} profiles: [ai] environment: - LLM_SM_MODEL: ${LDS_AI_MODEL:-qwen3:14b} + LLM_OLLAMA_MODEL: ${LDS_AI_MODEL:-qwen3:14b} OLLAMA_IGPU_ENABLE: ${LDS_AI_IGPU_ENABLE:-0} # provider input/PDF/Ollama tuning values are forwarded from docker/.env ``` @@ -2289,7 +2289,7 @@ Hardware and optional direct-port settings are generated by `lds` only for the c - NVIDIA -> `gpus: all`; - AMD -> `/dev/kfd` and `/dev/dri`; -- direct host API -> `127.0.0.1:${LLM_SM_PORT:-11434}:11434`. +- direct host API -> `127.0.0.1:${LLM_OLLAMA_PORT:-11434}:11434`. The generated fragment is temporary, is not part of `configuration/compose/`, and is removed after the Compose command completes. From d17b2d7f9fa977f67b4511d56444a0b93c9437f6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:26:56 +0600 Subject: [PATCH 309/509] docs(ai): align llm-ollama runtime and domain contract --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 0a4450af..136b1752 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -49,7 +49,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp ;; AI) [[ "$profile" == "ai" ]] || fail "AI profile must be ai" - [[ "$service_key" == "llm-sm" ]] || fail "AI service key must be llm-sm" + [[ "$service_key" == "llm-ollama" ]] || fail "AI service key must be llm-ollama" [[ "$runtime_modes" == "cpu,nvidia,amd" ]] || fail "AI runtime metadata drift" [[ "$defaults" == *"LDS_AI_MODEL=qwen3:14b"* ]] || fail "AI model default drift" [[ "$version_env" == "LDS_LLM_ARCH" ]] || fail "AI image selector must be LDS_LLM_ARCH" From 950fbf9fc4a7397bf7bdf522457219677955d113 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:27:00 +0600 Subject: [PATCH 310/509] docs(ai): align llm-ollama runtime and domain contract --- tests/docs-contract.sh | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 7bcae8fa..dd91d4f3 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -60,31 +60,31 @@ assert_file_contains "$storage" 'SSLKeys / SSLRootCA' assert_file_contains "$storage" 'ToolsState' pass "docs distinguish persistent runtime/control state and public TLS exports" -assert_file_contains "$ai" 'http://llm-sm:11434' -assert_file_contains "$ai" 'https://llm.localhost' +assert_file_contains "$ai" 'http://llm-ollama:11434' +assert_file_contains "$ai" 'https://llm-ollama.localhost' assert_file_contains "$ai" '127.0.0.1:11434' assert_file_contains "$ai" 'no Docker socket' assert_file_contains "$ai" 'no project/repository bind mount' assert_file_contains "$ai" 'automatically execute model-generated shell commands' assert_file_contains "$ai" 'linux/amd64 only' -assert_file_contains "$ai" 'does not mount the project/repository into llm-sm by default' -assert_file_contains "$ai" 'infocyph/llm-sm:${LDS_LLM_ARCH}' +assert_file_contains "$ai" 'does not mount the project/repository into llm-ollama by default' +assert_file_contains "$ai" 'infocyph/llm-ollama:${LDS_LLM_ARCH}' assert_file_contains "$ai" 'LDS_LLM_ARCH=amd-latest' assert_file_contains "$ai" 'LDS_AI_MODEL=qwen3:14b' -assert_file_contains "$ai" 'LLM_SM_ATTACHMENT_MAX_BYTES=16777216' -assert_file_contains "$ai" 'LLM_SM_PDF_MAX_PAGES=24' +assert_file_contains "$ai" 'LLM_OLLAMA_ATTACHMENT_MAX_BYTES=16777216' +assert_file_contains "$ai" 'LLM_OLLAMA_PDF_MAX_PAGES=24' assert_file_contains "$ai" 'docker/compose/companion.yaml' assert_file_contains "$ai" 'docker/.runtime/' assert_file_contains "$ai" 'There are no tracked ``ai.yaml``, ``ai-nvidia.yaml``, ``ai-amd.yaml`` or' assert_file_contains "$ai" 'Always invoke provider commands through ``lds llm``' -assert_file_contains "$ai" 'docker compose exec llm-sm' -assert_file_contains "$profiles" 'LLM_SM_ALLOW_LARGE_INPUT=0' +assert_file_contains "$ai" 'docker compose exec llm-ollama' +assert_file_contains "$profiles" 'LLM_OLLAMA_ALLOW_LARGE_INPUT=0' assert_file_contains "$profiles" 'LDS_LLM_ARCH' assert_file_contains "$readme" 'lds llm ask "Explain dependency injection briefly"' -assert_file_contains "$readme" 'LLM_SM_ATTACHMENTS_MAX_BYTES' +assert_file_contains "$readme" 'LLM_OLLAMA_ATTACHMENTS_MAX_BYTES' pass "local AI trust boundary, Compose ownership, provider options and invocation limits are documented" -for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-sm:0.03'; do +for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-ollama:0.03'; do if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then fail "user-facing docs contain stale compatibility reference: $stale" fi From a9f258e67467ffe116c043d654dc78ef5c5b74a1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:28:26 +0600 Subject: [PATCH 311/509] docs(ai): finish llm-ollama rename --- docs/concepts/storage-layout.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 305502ad..7e1e5b94 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -125,7 +125,7 @@ Docker Socket Boundary because their supported workflows need Docker control. -Persistent databases, admin clients, Nginx/Apache, and ``llm-sm`` do not receive the +Persistent databases, admin clients, Nginx/Apache, and ``llm-ollama`` do not receive the socket by default. ``lds run --sock`` is a separate explicit opt-in for an ad-hoc container. From 2353625d01a7687132838041d896bf5171e86182 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:28:39 +0600 Subject: [PATCH 312/509] docs(ai): finish llm-ollama rename --- docs/plans/docker-ecosystem/02-docker-runner-plan.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/plans/docker-ecosystem/02-docker-runner-plan.md b/docs/plans/docker-ecosystem/02-docker-runner-plan.md index 9250fbce..c7b908f2 100644 --- a/docs/plans/docker-ecosystem/02-docker-runner-plan.md +++ b/docs/plans/docker-ecosystem/02-docker-runner-plan.md @@ -18,7 +18,7 @@ Replace the legacy publication workflow with the current ecosystem contract: -- current GitHub Action majors matching `docker-llm-sm` where compatible; +- current GitHub Action majors matching `docker-llm-ollama` where compatible; - release event: publish immutable `` + `latest`; - scheduled event: resolve latest published release source but publish only `latest`; - never overwrite a release-version tag on schedule; From f24fa7a05d706017622a1b76220f643b2e0f6e8c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:28:43 +0600 Subject: [PATCH 313/509] docs(plan): rename Ollama provider plan file --- .../06-docker-llm-ollama-plan.md | 222 ++++++++++++++++++ 1 file changed, 222 insertions(+) create mode 100644 docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md diff --git a/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md b/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md new file mode 100644 index 00000000..546325c1 --- /dev/null +++ b/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md @@ -0,0 +1,222 @@ +# docker-llm-ollama — File-by-File Compatibility + Integration Plan + +## Role + +`infocyph/docker-llm-ollama` is the optional local-AI service for the LocalDevStack ecosystem. It is already published and already uses the newer single-repository tag model: + +- standard CPU/NVIDIA: `latest`, `` +- AMD ROCm: `amd-latest`, `amd-` + +This plan is intentionally conservative. The image should remain independently useful outside LocalDevStack. + +## Invariants + +- LocalDevStack integration must remain optional. +- The default model remains replaceable by the user. +- `/root/.ollama` must be persisted by the consumer/orchestrator. +- CLI commands bundled in the image remain available; users do not install/remove the CLI separately. +- CPU/NVIDIA and AMD ROCm remain separate tag families within the same registry repository. +- LocalDevStack must not rebuild `docker-llm-ollama` locally. +- Graphify or other AI clients consume the Ollama endpoint; they are not baked into this image merely because they can use it. + +## Existing Files + +### `.github/workflows/docker.publish.yml` + +Treat as the reference workflow for the older Docker repos. + +Plan: + +- keep single-repository standard/AMD tag families; +- keep release tags immutable; +- keep scheduled builds refreshing moving `latest`/`amd-latest` tags only; +- retain separate cache scopes/variant builds; +- retain Docker Hub + GHCR publication and attestations; +- only change if a later ecosystem-wide workflow improvement (SBOM/scanning/action major) is adopted consistently. + +### `.github/workflows/cli.check.yml` + +Keep as the CLI/Compose validation baseline. + +Potential additions only if needed by integration: + +- verify image-bundled CLI layout; +- verify persistent `/root/.ollama` contract; +- verify repo-aware command behavior against `/workspace` mount; +- verify no LocalDevStack-specific dependency enters the image. + +### `Dockerfile` + +Plan: + +- keep Ollama-based build and baked default model architecture; +- keep build-time model pull separated from runtime cloud-disable policy; +- preserve fixed in-image CLI installation; +- preserve healthcheck; +- do not add LocalDevStack-specific orchestration scripts; +- if ecosystem image metadata conventions are standardized, align labels without changing runtime behavior. + +### `.env.example` + +Keep standalone variables for image users. + +LocalDevStack should define its own integration variables rather than requiring users to copy this file. + +### `compose.yml` + +Keep as standalone CPU/default example. + +Do not alter it merely to match LocalDevStack’s profile layout. + +### `examples/compose/cpu.yml` + +Keep published-image/persistent-volume CPU example. + +### `examples/compose/nvidia.yml` + +Keep published-image/persistent-volume NVIDIA example and GPU request semantics. + +### `examples/compose/amd.yml` + +Keep `amd-latest`/ROCm example and `/dev/kfd` + `/dev/dri` device requirements. + +### `scripts/llm-ollama` + +Keep as fixed dispatcher installed in the image. + +Integration requirement: + +- commands invoked through `docker exec` must work regardless of LocalDevStack container name chosen by profile; +- no host installer/uninstaller lifecycle. + +### `scripts/lib/core.sh` + +- keep model/API/config helpers generic; +- support environment overrides LocalDevStack can pass; +- no knowledge of LocalDevStack networks/profile names. + +### `scripts/lib/ollama.sh` + +- preserve internal Ollama CLI/API helpers; +- ensure endpoint assumptions work inside the same container. + +### `scripts/lib/commit.sh` + +- preserve Git workspace/safe-directory handling for mounted repositories; +- keep staged-diff behavior self-contained. + +### Command files + +Files include: + +- `commands/ai-commit.sh` +- `api.sh` +- `ask.sh` +- `chat.sh` +- `code.sh` +- `help.sh` +- `json.sh` +- `logs.sh` +- `models.sh` +- `ollama.sh` +- `prompt.sh` +- `ps.sh` +- `pull.sh` +- `restart.sh` +- `review.sh` +- `rm.sh` +- `run.sh` +- `show.sh` +- `start.sh` +- `status.sh` +- `stop.sh` +- `unload.sh` +- `version.sh` + +Plan for all command files: + +- keep image-local responsibilities clear; +- commands that manage models/prompts/API remain first-class; +- any start/stop/restart/log/status command must reflect the actual in-container Ollama process model and not imply Docker host lifecycle control; +- preserve exit codes suitable for `docker exec`; +- keep `ai-commit` able to use a mounted `/workspace` Git repository; +- keep stdin fallback where already supported; +- do not add LocalDevStack wrappers into this repository. + +### `scripts/prompts/ai-commit.txt` + +- keep bundled/self-contained prompt; +- version prompt changes with image releases; +- LocalDevStack must not override it by default. + +### `README.md` + +Keep standalone documentation authoritative for direct image users. + +Add LocalDevStack mention only after integration exists, and only as a short interoperability note/link. + +### `.dockerignore`, `.gitignore`, `.gitattributes`, `LICENSE` + +No LocalDevStack-driven change expected. + +## LocalDevStack Integration Contract + +This lower-layer plan is implemented through the authoritative LocalDevStack integration +plan in `07-localdevstack-integration-plan.md`. + +LocalDevStack consumes the published provider through the optional `ai` profile. It does +not rebuild `docker-llm-ollama` locally and does not consume this repository's standalone +Compose files. + +Current LocalDevStack controls: + +- `LDS_AI_MODEL` selects the shared Tools/provider default model and is forwarded as + `LLM_OLLAMA_MODEL`; +- `LDS_AI_RUNTIME` selects `cpu`, `nvidia`, or `amd` when explicitly configured; +- `LDS_LLM_ARCH` is derived from the effective runtime (`latest` for CPU/NVIDIA, + `amd-latest` for AMD/ROCm); +- `LDS_LLM_HOST_PORT` controls optional direct loopback exposure; +- `LLM_OLLAMA_PORT` controls that loopback host port; +- provider input/PDF/Ollama tuning values are forwarded from LocalDevStack + `docker/.env`. + +Compose ownership: + +- one tracked `llm-ollama` service in `docker/compose/companion.yaml`; +- no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or + `ai-host-port.yaml`; +- NVIDIA, AMD/ROCm, and host-port additions are generated temporarily under + `docker/.runtime/` by `lds`. + +Persistence: + +- LocalDevStack's `LLMModels` named volume mounts at `/root/.ollama`; +- user-pulled models survive container recreation/upgrades. + +Workspace: + +- LocalDevStack intentionally does not mount a project/repository into `llm-ollama` by + default; +- repository-aware analysis normally uses the Tools consumer layer; +- provider stdin flows remain available without weakening the default trust boundary. + +Networking: + +- internal consumers use `http://llm-ollama:11434`; +- Nginx exposes `https://llm-ollama.localhost`; +- direct host `11434`-style access is opt-in and loopback-only. + +User/provider commands are invoked through `lds llm ...`, not a bare +`docker compose exec` from the LocalDevStack repository root. + +## Acceptance Criteria + +1. Existing published `docker-llm-ollama` behavior remains standalone and stable. +2. LocalDevStack can enable it without building locally. +3. Standard and AMD tags are selectable. +4. Named volume persists pulled models across container recreation. +5. Other LocalDevStack containers can reach Ollama by service DNS. +6. Host clients can reach it through an explicitly configured loopback port when enabled. +7. `lds llm ...` delegates to the bundled provider CLI through LocalDevStack's Compose wrapper. +8. No project/repository bind mount is required for normal provider operation. +9. No Graphify/LocalDevStack-specific package is added to the image solely for integration. From df7cdd5a55488dc888dc427e6fe9e348b0f3df68 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:28:47 +0600 Subject: [PATCH 314/509] docs(plan): remove old llm-sm plan filename --- .../docker-ecosystem/06-docker-llm-sm-plan.md | 222 ------------------ 1 file changed, 222 deletions(-) delete mode 100644 docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md diff --git a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md b/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md deleted file mode 100644 index 546325c1..00000000 --- a/docs/plans/docker-ecosystem/06-docker-llm-sm-plan.md +++ /dev/null @@ -1,222 +0,0 @@ -# docker-llm-ollama — File-by-File Compatibility + Integration Plan - -## Role - -`infocyph/docker-llm-ollama` is the optional local-AI service for the LocalDevStack ecosystem. It is already published and already uses the newer single-repository tag model: - -- standard CPU/NVIDIA: `latest`, `` -- AMD ROCm: `amd-latest`, `amd-` - -This plan is intentionally conservative. The image should remain independently useful outside LocalDevStack. - -## Invariants - -- LocalDevStack integration must remain optional. -- The default model remains replaceable by the user. -- `/root/.ollama` must be persisted by the consumer/orchestrator. -- CLI commands bundled in the image remain available; users do not install/remove the CLI separately. -- CPU/NVIDIA and AMD ROCm remain separate tag families within the same registry repository. -- LocalDevStack must not rebuild `docker-llm-ollama` locally. -- Graphify or other AI clients consume the Ollama endpoint; they are not baked into this image merely because they can use it. - -## Existing Files - -### `.github/workflows/docker.publish.yml` - -Treat as the reference workflow for the older Docker repos. - -Plan: - -- keep single-repository standard/AMD tag families; -- keep release tags immutable; -- keep scheduled builds refreshing moving `latest`/`amd-latest` tags only; -- retain separate cache scopes/variant builds; -- retain Docker Hub + GHCR publication and attestations; -- only change if a later ecosystem-wide workflow improvement (SBOM/scanning/action major) is adopted consistently. - -### `.github/workflows/cli.check.yml` - -Keep as the CLI/Compose validation baseline. - -Potential additions only if needed by integration: - -- verify image-bundled CLI layout; -- verify persistent `/root/.ollama` contract; -- verify repo-aware command behavior against `/workspace` mount; -- verify no LocalDevStack-specific dependency enters the image. - -### `Dockerfile` - -Plan: - -- keep Ollama-based build and baked default model architecture; -- keep build-time model pull separated from runtime cloud-disable policy; -- preserve fixed in-image CLI installation; -- preserve healthcheck; -- do not add LocalDevStack-specific orchestration scripts; -- if ecosystem image metadata conventions are standardized, align labels without changing runtime behavior. - -### `.env.example` - -Keep standalone variables for image users. - -LocalDevStack should define its own integration variables rather than requiring users to copy this file. - -### `compose.yml` - -Keep as standalone CPU/default example. - -Do not alter it merely to match LocalDevStack’s profile layout. - -### `examples/compose/cpu.yml` - -Keep published-image/persistent-volume CPU example. - -### `examples/compose/nvidia.yml` - -Keep published-image/persistent-volume NVIDIA example and GPU request semantics. - -### `examples/compose/amd.yml` - -Keep `amd-latest`/ROCm example and `/dev/kfd` + `/dev/dri` device requirements. - -### `scripts/llm-ollama` - -Keep as fixed dispatcher installed in the image. - -Integration requirement: - -- commands invoked through `docker exec` must work regardless of LocalDevStack container name chosen by profile; -- no host installer/uninstaller lifecycle. - -### `scripts/lib/core.sh` - -- keep model/API/config helpers generic; -- support environment overrides LocalDevStack can pass; -- no knowledge of LocalDevStack networks/profile names. - -### `scripts/lib/ollama.sh` - -- preserve internal Ollama CLI/API helpers; -- ensure endpoint assumptions work inside the same container. - -### `scripts/lib/commit.sh` - -- preserve Git workspace/safe-directory handling for mounted repositories; -- keep staged-diff behavior self-contained. - -### Command files - -Files include: - -- `commands/ai-commit.sh` -- `api.sh` -- `ask.sh` -- `chat.sh` -- `code.sh` -- `help.sh` -- `json.sh` -- `logs.sh` -- `models.sh` -- `ollama.sh` -- `prompt.sh` -- `ps.sh` -- `pull.sh` -- `restart.sh` -- `review.sh` -- `rm.sh` -- `run.sh` -- `show.sh` -- `start.sh` -- `status.sh` -- `stop.sh` -- `unload.sh` -- `version.sh` - -Plan for all command files: - -- keep image-local responsibilities clear; -- commands that manage models/prompts/API remain first-class; -- any start/stop/restart/log/status command must reflect the actual in-container Ollama process model and not imply Docker host lifecycle control; -- preserve exit codes suitable for `docker exec`; -- keep `ai-commit` able to use a mounted `/workspace` Git repository; -- keep stdin fallback where already supported; -- do not add LocalDevStack wrappers into this repository. - -### `scripts/prompts/ai-commit.txt` - -- keep bundled/self-contained prompt; -- version prompt changes with image releases; -- LocalDevStack must not override it by default. - -### `README.md` - -Keep standalone documentation authoritative for direct image users. - -Add LocalDevStack mention only after integration exists, and only as a short interoperability note/link. - -### `.dockerignore`, `.gitignore`, `.gitattributes`, `LICENSE` - -No LocalDevStack-driven change expected. - -## LocalDevStack Integration Contract - -This lower-layer plan is implemented through the authoritative LocalDevStack integration -plan in `07-localdevstack-integration-plan.md`. - -LocalDevStack consumes the published provider through the optional `ai` profile. It does -not rebuild `docker-llm-ollama` locally and does not consume this repository's standalone -Compose files. - -Current LocalDevStack controls: - -- `LDS_AI_MODEL` selects the shared Tools/provider default model and is forwarded as - `LLM_OLLAMA_MODEL`; -- `LDS_AI_RUNTIME` selects `cpu`, `nvidia`, or `amd` when explicitly configured; -- `LDS_LLM_ARCH` is derived from the effective runtime (`latest` for CPU/NVIDIA, - `amd-latest` for AMD/ROCm); -- `LDS_LLM_HOST_PORT` controls optional direct loopback exposure; -- `LLM_OLLAMA_PORT` controls that loopback host port; -- provider input/PDF/Ollama tuning values are forwarded from LocalDevStack - `docker/.env`. - -Compose ownership: - -- one tracked `llm-ollama` service in `docker/compose/companion.yaml`; -- no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or - `ai-host-port.yaml`; -- NVIDIA, AMD/ROCm, and host-port additions are generated temporarily under - `docker/.runtime/` by `lds`. - -Persistence: - -- LocalDevStack's `LLMModels` named volume mounts at `/root/.ollama`; -- user-pulled models survive container recreation/upgrades. - -Workspace: - -- LocalDevStack intentionally does not mount a project/repository into `llm-ollama` by - default; -- repository-aware analysis normally uses the Tools consumer layer; -- provider stdin flows remain available without weakening the default trust boundary. - -Networking: - -- internal consumers use `http://llm-ollama:11434`; -- Nginx exposes `https://llm-ollama.localhost`; -- direct host `11434`-style access is opt-in and loopback-only. - -User/provider commands are invoked through `lds llm ...`, not a bare -`docker compose exec` from the LocalDevStack repository root. - -## Acceptance Criteria - -1. Existing published `docker-llm-ollama` behavior remains standalone and stable. -2. LocalDevStack can enable it without building locally. -3. Standard and AMD tags are selectable. -4. Named volume persists pulled models across container recreation. -5. Other LocalDevStack containers can reach Ollama by service DNS. -6. Host clients can reach it through an explicitly configured loopback port when enabled. -7. `lds llm ...` delegates to the bundled provider CLI through LocalDevStack's Compose wrapper. -8. No project/repository bind mount is required for normal provider operation. -9. No Graphify/LocalDevStack-specific package is added to the image solely for integration. From 5121a7f2c024d2b6ac8dd312ef9c5e9c3db5c36c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:41:08 +0600 Subject: [PATCH 315/509] test(ai): finish llm-ollama rename coverage --- tests/qol-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index b1d9fe64..04d7b955 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -52,7 +52,7 @@ assert_contains "$urls" "https://admin.localhost" assert_contains "$urls" "https://webmail.localhost" assert_contains "$urls" "https://db.localhost" assert_contains "$urls" "https://ri.localhost" -assert_contains "$urls" "https://llm.localhost" +assert_contains "$urls" "https://llm-ollama.localhost" if grep -Fq "https://kibana.localhost" <<<"$urls"; then fail "urls must not show disabled Elasticsearch profile URL" fi From 5a32daff44671e229651bcc20aa2ef6e1640edd0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:41:11 +0600 Subject: [PATCH 316/509] test(ai): finish llm-ollama rename coverage --- tests/service-hardening-contract.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index ec4835de..d9797790 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -75,7 +75,7 @@ pass "all CA install paths use current export with legacy fallback" assert_file_contains "$companion" 'COMPOSE_PROFILES=${COMPOSE_PROFILES:-}' pass "Tools profile visibility follows LocalDevStack profile selection" -if awk '/^ llm-sm:/ { in_llm=1; next } in_llm && /^ [a-zA-Z0-9_-]+:/ { in_llm=0 } in_llm { print }' "$companion" | grep -Eq '/var/run/docker.sock|PROJECT_DIR|/app'; then - fail "llm-sm must not receive Docker socket or project mounts" +if awk '/^ llm-ollama:/ { in_llm=1; next } in_llm && /^ [a-zA-Z0-9_-]+:/ { in_llm=0 } in_llm { print }' "$companion" | grep -Eq '/var/run/docker.sock|PROJECT_DIR|/app'; then + fail "llm-ollama must not receive Docker socket or project mounts" fi -pass "companion-owned llm-sm keeps the AI trust boundary" +pass "companion-owned llm-ollama keeps the AI trust boundary" From 353bc48d33c38bd0249c610a0bbd37a30a5be880 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:49:28 +0600 Subject: [PATCH 317/509] feat(ai): publish native Ollama port through Nginx --- docker/compose/http.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/http.yaml b/docker/compose/http.yaml index 6960f8a1..dd8fd9bb 100644 --- a/docker/compose/http.yaml +++ b/docker/compose/http.yaml @@ -10,6 +10,7 @@ services: ports: - "${HTTP_PORT:-80}:80" - "${HTTPS_PORT:-443}:443" + - "127.0.0.1:11434:11434" volumes: - ${PROJECT_DIR:-./../../../application}:/app - lds_nginx_host:/etc/nginx/conf.d From f2bb2fccce775c0f5bf1516ad6edd2d78e6cc158 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:49:31 +0600 Subject: [PATCH 318/509] refactor(ai): remove direct provider host-port override --- lib/compose.sh | 20 ++------------------ 1 file changed, 2 insertions(+), 18 deletions(-) diff --git a/lib/compose.sh b/lib/compose.sh index df20edd3..efcfeb25 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -42,8 +42,7 @@ docker_compose() { local -a env_files=(--env-file "$ENV_RELEASE") [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - local ai_runtime ai_host_port llm_arch llm_port runtime_override="" - local ai_host_port_enabled=0 + local ai_runtime llm_arch runtime_override="" local -a runtime_f=() ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" @@ -57,18 +56,7 @@ docker_compose() { llm_arch="$(llm_arch_for_runtime "$ai_runtime")" || die "Cannot resolve LLM image tag for runtime: $ai_runtime" - ai_host_port="$(compose_control_value LDS_LLM_HOST_PORT 0)" - case "${ai_host_port,,}" in - "" | 0 | false | no | off) ;; - 1 | true | yes | on) ai_host_port_enabled=1 ;; - *) die "Invalid LDS_LLM_HOST_PORT: $ai_host_port (expected 0|1)" ;; - esac - - llm_port="$(compose_control_value LLM_OLLAMA_PORT 11434)" - [[ "$llm_port" =~ ^[0-9]+$ ]] && ((llm_port >= 1 && llm_port <= 65535)) || - die "Invalid LLM_OLLAMA_PORT: $llm_port (expected 1-65535)" - - if [[ "$ai_runtime" != "cpu" || "$ai_host_port_enabled" == "1" ]]; then + if [[ "$ai_runtime" != "cpu" ]]; then mkdir -p "$CFG/.runtime" runtime_override="$(mktemp "$CFG/.runtime/ai.XXXXXX")" || die "Unable to create temporary AI Compose override" @@ -83,10 +71,6 @@ docker_compose() { printf '%s\n' ' devices:' ' - /dev/kfd:/dev/kfd' ' - /dev/dri:/dev/dri' ;; esac - if ((ai_host_port_enabled)); then - printf '%s\n' ' ports:' - printf ' - "127.0.0.1:%s:11434"\n' "$llm_port" - fi } >"$runtime_override" runtime_f=(-f "$runtime_override") From a5673345fc3a94433ec0b0819a388891e7ceb9dd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:49:46 +0600 Subject: [PATCH 319/509] refactor(ai): route Graphify through Nginx native Ollama endpoint --- lib/ai.sh | 29 +++-------------------------- 1 file changed, 3 insertions(+), 26 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 422845f3..8643be6a 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -30,7 +30,7 @@ cmd_ai() { } _graphify_local_base_url() { - local ctr published host_port + local ctr ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" [[ -n "$ctr" ]] || die "llm-ollama is not running. Enable the ai profile and start the stack first." @@ -38,14 +38,7 @@ _graphify_local_base_url() { docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || die "llm-ollama container exists but is not running." - published="$( - docker inspect -f '{{with (index .NetworkSettings.Ports "11434/tcp")}}{{(index . 0).HostPort}}{{end}}' "$ctr" 2>/dev/null || true - )" - [[ "$published" =~ ^[0-9]+$ ]] || - die "Graphify needs the llm-ollama loopback API. Run: lds llm host-port on && lds up -d llm-ollama" - - host_port="$published" - printf 'http://127.0.0.1:%s/v1' "$host_port" + printf '%s' 'http://llm-ollama.localhost:11434/v1' } cmd_graphify() { @@ -142,30 +135,14 @@ cmd_llm() { *) die "llm runtime " ;; esac ;; - host-port) - local state="${1:-status}" - case "${state,,}" in - status) printf '%s\n' "$(compose_control_value LDS_LLM_HOST_PORT 0)" ;; - on | enable | enabled | 1) - update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 1 - ok "Direct LLM API enabled on loopback only. Recreate llm-ollama to apply." - ;; - off | disable | disabled | 0) - update_env "$ENV_DOCKER" LDS_LLM_HOST_PORT 0 - ok "Direct LLM host API disabled. Recreate llm-ollama to apply." - ;; - *) die "llm host-port " ;; - esac - ;; models | ps | show | pull | rm | unload | run | ask | chat | prompt | code | review | json | ai-commit | ollama | api | version) _llm_exec "${sub,,}" "$@" ;; help | -h | --help) printf '%s\n' "llm " printf '%s\n' "llm runtime " - printf '%s\n' "llm host-port " ;; - *) die "llm " ;; + *) die "llm " ;; esac } From dfa4d64e5605046bf18bc040f9fb7e0568955b02 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:50:13 +0600 Subject: [PATCH 320/509] docs(ai): make Nginx the sole native Ollama port owner --- README.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/README.md b/README.md index bd4f772c..acf7269b 100644 --- a/README.md +++ b/README.md @@ -272,14 +272,13 @@ lds llm ask "Explain dependency injection briefly" lds llm chat ... ``` -Direct host Ollama access is off by default. `lds llm host-port on` binds only to `127.0.0.1:11434` by default. Change that loopback port with `LLM_OLLAMA_PORT` in `docker/.env`. +LocalDevStack publishes the native Ollama API through Nginx on the fixed loopback-only endpoint `http://llm-ollama.localhost:11434`. The `llm-ollama` container itself remains internal and never owns a host port. -`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets the actual loopback-published `llm-ollama` API. The command runs extraction with `--backend ollama --no-cluster`, then runs `cluster-only` for the same path if extraction succeeds. This avoids clustering twice. +`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets `http://llm-ollama.localhost:11434/v1` through Nginx. The command runs extraction with `--backend ollama --no-cluster`, then runs `cluster-only` for the same path if extraction succeeds. This avoids clustering twice. -Before first use, publish the provider port and apply that Compose change: +Start the AI profile, then Graphify can use the native endpoint directly: ```bash -lds llm host-port on lds up -d llm-ollama lds graphify ``` @@ -296,10 +295,10 @@ docker/compose/main.yaml └─ llm-ollama service (profile: ai) docker/.runtime/ai.* temporary per-command overlay generated by lds - only when NVIDIA, AMD/ROCm, or host-port settings require it + only when NVIDIA or AMD/ROCm settings require it ``` -There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. AI hardware and host-port additions are generated ephemerally and removed after the Compose command. Files under `configuration/compose/` remain the normal LocalDevStack extras/runtime-fragment area; they are not built-in LLM variant files. +There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. AI hardware additions are generated ephemerally and removed after the Compose command. Nginx owns the fixed loopback-only native Ollama publication on port 11434. Files under `configuration/compose/` remain the normal LocalDevStack extras/runtime-fragment area; they are not built-in LLM variant files. Provider configuration placed in `docker/.env` is forwarded to `llm-ollama` where applicable: @@ -308,8 +307,6 @@ Provider configuration placed in `docker/.env` is forwarded to `llm-ollama` wher | `LDS_AI_MODEL` | `qwen3:14b` | Tools model and `lds llm` default model | | `LDS_AI_RUNTIME` | auto-detected | `cpu`, `nvidia`, or `amd` runtime selection | | `LDS_AI_IGPU_ENABLE` | `1` for AMD CPU + AMD runtime, otherwise `0` | Forwarded to Ollama as `OLLAMA_IGPU_ENABLE` | -| `LDS_LLM_HOST_PORT` | `0` | Enables/disables direct loopback API mapping | -| `LLM_OLLAMA_PORT` | `11434` | Host loopback port when direct mapping is enabled | | `LLM_OLLAMA_SYSTEM` | empty | Default system instruction for provider prompts | | `LLM_OLLAMA_INPUT_WARN_BYTES` | `1048576` | Text/diff warning threshold | | `LLM_OLLAMA_INPUT_MAX_BYTES` | `0` | Text/diff hard ceiling; `0` disables it | From aa1d06e4b8a96959a570bfdbc15a79b422cb43a3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:50:16 +0600 Subject: [PATCH 321/509] docs(ai): document fixed Nginx Ollama endpoint --- docs/guides/local-ai.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 0484a91b..a80ecca5 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -109,7 +109,7 @@ under ``docker/.runtime/`` for the current command only: - NVIDIA -> ``gpus: all``; - AMD/ROCm -> ``/dev/kfd`` and ``/dev/dri``; -- direct host API -> ``127.0.0.1:${LLM_OLLAMA_PORT:-11434}:11434``. +- native host API -> ``127.0.0.1:11434:11434`` on Nginx -> ``llm-ollama:11434``. The fragment is removed after the Compose command. ``configuration/compose/`` remains the normal extras/generated-runtime area and is not the location of built-in LLM @@ -282,7 +282,6 @@ model from ``LDS_AI_MODEL``, derives ``GRAPHIFY_API_TIMEOUT`` from Enable and apply the direct provider port before the default workflow:: - lds llm host-port on lds up -d llm-ollama lds graphify From 6504ae91af4f23c1f1d331e965d1a549c714a33f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:50:20 +0600 Subject: [PATCH 322/509] docs(ai): remove obsolete host-port settings --- docs/concepts/profiles-and-env.rst | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index a487a1df..03b0e349 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -213,8 +213,6 @@ Important AI settings include:: LDS_AI_MODEL=qwen3:14b LDS_AI_RUNTIME= LDS_AI_IGPU_ENABLE= - LDS_LLM_HOST_PORT=0 - LLM_OLLAMA_PORT=11434 When ``LDS_AI_RUNTIME`` is not explicitly set, LocalDevStack detects the preferred runtime for the Compose invocation. NVIDIA is selected only when ``nvidia-smi`` is @@ -231,9 +229,7 @@ it is ``0``. Use:: lds llm runtime - lds llm host-port - -to override the detected runtime or host-port behavior. +to override the detected runtime behavior. The selected ``LDS_AI_MODEL`` is also forwarded to the provider as ``LLM_OLLAMA_MODEL``, so Tools and ``lds llm`` share the same default model. @@ -270,7 +266,7 @@ checks stay fast. The same generation timeout is passed to Nginx as The LLM service itself is tracked only in ``docker/compose/companion.yaml``. There are no tracked AI runtime-variant YAML files; ``lds`` creates temporary fragments under -``docker/.runtime/`` only for NVIDIA, AMD/ROCm, or direct host-port augmentation. +``docker/.runtime/`` only for NVIDIA or AMD/ROCm augmentation. Compose Extras -------------- From 64542252e72bb145b8075e397ecf71845e143305 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:50:24 +0600 Subject: [PATCH 323/509] docs(cli): remove llm host-port command --- docs/reference/cli.rst | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index 62bc163e..1054edc1 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -221,7 +221,8 @@ The host port must already be applied with:: lds llm host-port on lds up -d llm-ollama -An explicitly supplied ``OLLAMA_BASE_URL`` bypasses that host-port lookup. +An explicitly supplied ``OLLAMA_BASE_URL`` overrides the default +``http://llm-ollama.localhost:11434/v1`` endpoint. LLM Provider ------------ @@ -264,13 +265,9 @@ The runtime command keeps the derived image tag in sync and also refreshes ``LDS_AI_IGPU_ENABLE``. AMD runtime on an AMD CPU uses ``1`` so Ollama admits the integrated Radeon GPU; the other derived cases use ``0``. -Direct host-port control:: - - lds llm host-port status - lds llm host-port on - lds llm host-port off - -The loopback port defaults to 11434 and can be changed with ``LLM_OLLAMA_PORT``. +Native Ollama access is always routed through Nginx at +``http://llm-ollama.localhost:11434``; the provider container itself is not +published directly. Rebuild ------- From 93e74c104d8131a7f59179b68f82327c30822645 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:53:13 +0600 Subject: [PATCH 324/509] refactor(ai): remove host-port setup option --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index 57afa271..feb2eaeb 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.3|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm-ollama|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b;LDS_LLM_HOST_PORT=no|Preferred model;Direct localhost API (yes/no)||LLMModels|https://llm-ollama.localhost|ai|1|0|cpu,nvidia,amd +AI|ai|Local AI|llm-ollama|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b|Preferred model||LLMModels|https://llm-ollama.localhost|ai|1|0|cpu,nvidia,amd From 7590ab4890fdd7259ca54da3493cb635912b0cd8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:53:16 +0600 Subject: [PATCH 325/509] test(ai): remove obsolete host-port catalog contract --- tests/catalog-contract.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 136b1752..c96efef5 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -54,7 +54,6 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$defaults" == *"LDS_AI_MODEL=qwen3:14b"* ]] || fail "AI model default drift" [[ "$version_env" == "LDS_LLM_ARCH" ]] || fail "AI image selector must be LDS_LLM_ARCH" [[ "$defaults" != *"LDS_AI_RUNTIME="* ]] || fail "AI profile wizard must not prompt for runtime" - [[ "$defaults" == *"LDS_LLM_HOST_PORT=no"* ]] || fail "AI host-port default drift" ;; ELASTICSEARCH) [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.3"* ]] || fail "Elastic stable default drift" From bc4bca90b18bc524c39b1e83f600dd8ade94879e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:53:20 +0600 Subject: [PATCH 326/509] test(ai): assert Nginx owns native Ollama port --- tests/compose-contract.sh | 21 +++++++-------------- 1 file changed, 7 insertions(+), 14 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index b6fdd767..b4a8a078 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -175,8 +175,12 @@ assert tools["LDS_AI_AVAILABILITY_TTL"] == "5" assert tools["LDS_AI_MAX_CONTEXT_BYTES"] == "524288" assert tools["LDS_AI_MAX_REQUEST_BYTES"] == "1048576" assert tools["LDS_AI_MAX_RESPONSE_BYTES"] == "2097152" -nginx=d["services"]["nginx"]["environment"] -assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" +nginx=d["services"]["nginx"] +assert nginx["environment"]["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" +native=[p for p in nginx.get("ports", []) if int(p["target"]) == 11434] +assert len(native) == 1 +assert native[0]["host_ip"] == "127.0.0.1" +assert int(native[0]["published"]) == 11434 ' <<<"$ai_json" pass "companion-owned AI profile is internal-only and deterministic" @@ -216,17 +220,6 @@ assert s.get("gpus") ' <<<"$nvidia_json" pass "NVIDIA AI runtime is generated dynamically" -host_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu LDS_LLM_HOST_PORT=1 "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" -python3 -c ' -import json,sys -ports=json.load(sys.stdin)["services"]["llm-ollama"]["ports"] -assert len(ports) == 1 -p=ports[0] -assert p["host_ip"] == "127.0.0.1" -assert int(p["target"]) == 11434 and int(p["published"]) == 11434 -' <<<"$host_json" -pass "direct Ollama port is generated dynamically and loopback-only" - if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -quit | grep -q .; then fail "AI-specific Compose files must not exist" fi @@ -234,4 +227,4 @@ if [[ -d "$ROOT/docker/.runtime" ]] && find "$ROOT/docker/.runtime" -type f -pri fail "temporary AI Compose overrides were not cleaned up" fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' -pass "single LLM service plus ephemeral hardware/port overrides" +pass "single LLM service plus ephemeral hardware overrides" From fd875f72e1e0062e2c2336a839841868549b1c00 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:53:25 +0600 Subject: [PATCH 327/509] test(docs): align native Ollama endpoint contract --- tests/docs-contract.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index dd91d4f3..f004606b 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -43,7 +43,6 @@ assert_file_contains "$ops" 'lds support bundle' assert_file_contains "$ops" 'lds clean --global --yes' assert_file_contains "$runner" 'lds run --sock' assert_file_contains "$notify" 'Windows/Git Bash' -assert_file_contains "$cli" 'lds llm host-port' pass "docs describe current architecture, operations, and complete user surfaces" assert_file_contains "$profiles" 'docker/release.env' @@ -62,7 +61,7 @@ pass "docs distinguish persistent runtime/control state and public TLS exports" assert_file_contains "$ai" 'http://llm-ollama:11434' assert_file_contains "$ai" 'https://llm-ollama.localhost' -assert_file_contains "$ai" '127.0.0.1:11434' +assert_file_contains "$ai" 'http://llm-ollama.localhost:11434' assert_file_contains "$ai" 'no Docker socket' assert_file_contains "$ai" 'no project/repository bind mount' assert_file_contains "$ai" 'automatically execute model-generated shell commands' From feeec2dfa5f15a88b2e887ac307f17ddacbc27ed Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:53:38 +0600 Subject: [PATCH 328/509] docs(cli): remove obsolete llm host-port help --- lds | 2 -- 1 file changed, 2 deletions(-) diff --git a/lds b/lds index 63c2598b..a94b5405 100755 --- a/lds +++ b/lds @@ -773,7 +773,6 @@ cmd_help() { ## LLM provider - `lds llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit|ollama|api|version ...` - `lds llm runtime [cpu|nvidia|amd]` -- `lds llm host-port ` ## Rebuild - `lds rebuild` @@ -850,7 +849,6 @@ ${CYAN}AI:${NC} graphify [path] [graphify-extract-options...] llm models|ps|show|pull|rm|unload|run|ask|chat|prompt|code|review|json|ai-commit|ollama|api|version llm runtime [cpu|nvidia|amd] - llm host-port ${CYAN}Maintenance:${NC} rebuild [all|] From 256fc34a3b44730e935977a46a5d2c3271f02171 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:53:47 +0600 Subject: [PATCH 329/509] test(ai): lock Graphify Nginx endpoint contract --- tests/cli-contract.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index 1c84fd00..29a1f163 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -50,17 +50,18 @@ SH chmod +x "$tmpbin/graphify" PATH="$tmpbin:$PATH" \ -OLLAMA_BASE_URL=http://127.0.0.1:11434/v1 \ +OLLAMA_BASE_URL=http://custom-ollama.test:11434/v1 \ LDS_AI_MODEL=qwen3:14b \ LDS_AI_TIMEOUT=1800 \ GRAPHIFY_TEST_LOG="$graphify_log" \ "$ROOT/lds" graphify . --model test-model --api-timeout 42 --mode deep >/dev/null -grep -Fq 'base=http://127.0.0.1:11434/v1 key=local model=test-model timeout=42 args=extract . --backend ollama --no-cluster --model test-model --api-timeout 42 --mode deep' "$graphify_log" || +grep -Fq 'base=http://custom-ollama.test:11434/v1 key=local model=test-model timeout=42 args=extract . --backend ollama --no-cluster --model test-model --api-timeout 42 --mode deep' "$graphify_log" || fail "Graphify extract wrapper contract failed" -grep -Fq 'base=http://127.0.0.1:11434/v1 key=local model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || +grep -Fq 'base=http://custom-ollama.test:11434/v1 key=local model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || fail "Graphify cluster wrapper contract failed" rm -f "$graphify_log" +assert_file_contains "$ROOT/lib/ai.sh" "http://llm-ollama.localhost:11434/v1" pass "Graphify host workflow wrapper" assert_file_contains "$ROOT/lds" 'exec "$DIR/bin/tool-runner" "$cmd" "$@"' From 3eeddd079af2de6c3f942550da070c9e20e9f498 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:54:26 +0600 Subject: [PATCH 330/509] docs(plan): make Nginx sole Ollama host-access owner --- .../07-localdevstack-integration-plan.md | 44 ++++++++++--------- 1 file changed, 23 insertions(+), 21 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 5d552a79..dcd2ec63 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -636,37 +636,37 @@ user-facing image-version choice. Detection rules are: usable `nvidia-smi` -> NV both `/dev/kfd` and `/dev/dri` -> AMD/ROCm; otherwise CPU. An AMD CPU alone never selects the AMD/ROCm image. `lds llm runtime ...` remains the explicit override. -## 8.4 Optional direct host API +## 8.4 Native host API -Default LocalDevStack access is: +LocalDevStack exposes Ollama only through Nginx. The provider container itself remains +internal and has no host port mapping. + +Nginx owns both user-facing paths: ```text https://llm-ollama.localhost +http://llm-ollama.localhost:11434 ``` -Do not expose `11434` by default. - -If developers explicitly need direct Ollama access, `lds` adds the port mapping to the -same temporary `docker/.runtime/ai.*` fragment used for hardware augmentation, binding -only: +The native port is fixed and loopback-only on the host: ```text -127.0.0.1:${LLM_OLLAMA_PORT:-11434}:11434 +127.0.0.1:11434 -> nginx:11434 -> llm-ollama:11434 ``` -Never default to `0.0.0.0:11434`. +Never bind the native Ollama route to `0.0.0.0:11434`. ## 8.5 Nginx integration -No new Nginx image changes are required. - -`nginx:0.4.1` already reserves: +The Nginx image owns the dedicated TLS and native Ollama proxy routes: ```text -llm-ollama.localhost -> llm-ollama:11434 +llm-ollama.localhost:443 -> llm-ollama:11434 +nginx:11434 -> llm-ollama:11434 ``` -and uses lazy Docker DNS resolution plus streaming proxy behavior. +Both routes use lazy Docker DNS resolution and streaming proxy behavior. LocalDevStack +publishes Nginx's native listener as `127.0.0.1:11434:11434`. LocalDevStack must validate: @@ -2020,7 +2020,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - persistent `LLMModels`; - CPU/NVIDIA/AMD modes; - `https://llm-ollama.localhost`; - - loopback-only optional direct host port; + - Nginx-owned loopback-only native Ollama port; - Tools consumer wiring; - `lds ai` / `lds llm` separation; - fake-provider integration test. @@ -2283,14 +2283,16 @@ effective runtime/host combination and must not become independent image-version selectors. A deliberately persisted iGPU override remains user-controlled until the runtime selector is invoked again. -## Ephemeral hardware/host-port Compose augmentation +## Ephemeral hardware Compose augmentation -Hardware and optional direct-port settings are generated by `lds` only for the current Compose invocation: +Hardware settings are generated by `lds` only for the current Compose invocation: - NVIDIA -> `gpus: all`; -- AMD -> `/dev/kfd` and `/dev/dri`; -- direct host API -> `127.0.0.1:${LLM_OLLAMA_PORT:-11434}:11434`. +- AMD -> `/dev/kfd` and `/dev/dri`. -The generated fragment is temporary, is not part of `configuration/compose/`, and is removed after the Compose command completes. +The generated fragment is temporary, is not part of `configuration/compose/`, and is +removed after the Compose command completes. Native Ollama host access is not part of +this fragment; Nginx owns the fixed loopback-only `127.0.0.1:11434` publication. -This keeps the repository at one LLM service definition while still avoiding invalid GPU/device declarations on unsupported hosts. +This keeps the repository at one LLM service definition while still avoiding invalid +GPU/device declarations on unsupported hosts. From 68b5cafbde07134d8d2fceb342056f8ff6505d86 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:54:31 +0600 Subject: [PATCH 331/509] docs(plan): remove provider host-port controls --- .../06-docker-llm-ollama-plan.md | 16 +++++++++------- 1 file changed, 9 insertions(+), 7 deletions(-) diff --git a/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md b/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md index 546325c1..4bc18d72 100644 --- a/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md +++ b/docs/plans/docker-ecosystem/06-docker-llm-ollama-plan.md @@ -175,18 +175,18 @@ Current LocalDevStack controls: - `LDS_AI_RUNTIME` selects `cpu`, `nvidia`, or `amd` when explicitly configured; - `LDS_LLM_ARCH` is derived from the effective runtime (`latest` for CPU/NVIDIA, `amd-latest` for AMD/ROCm); -- `LDS_LLM_HOST_PORT` controls optional direct loopback exposure; -- `LLM_OLLAMA_PORT` controls that loopback host port; - provider input/PDF/Ollama tuning values are forwarded from LocalDevStack - `docker/.env`. + `docker/.env`; +- host access is owned by LocalDevStack Nginx, not by the provider container. Compose ownership: - one tracked `llm-ollama` service in `docker/compose/companion.yaml`; - no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml`; -- NVIDIA, AMD/ROCm, and host-port additions are generated temporarily under - `docker/.runtime/` by `lds`. +- NVIDIA and AMD/ROCm additions are generated temporarily under + `docker/.runtime/` by `lds`; +- Nginx publishes the fixed loopback-only native endpoint on host port `11434`. Persistence: @@ -204,7 +204,9 @@ Networking: - internal consumers use `http://llm-ollama:11434`; - Nginx exposes `https://llm-ollama.localhost`; -- direct host `11434`-style access is opt-in and loopback-only. +- Nginx also exposes `http://llm-ollama.localhost:11434` through a fixed + loopback-only host bind; +- the provider container itself has no published host port. User/provider commands are invoked through `lds llm ...`, not a bare `docker compose exec` from the LocalDevStack repository root. @@ -216,7 +218,7 @@ User/provider commands are invoked through `lds llm ...`, not a bare 3. Standard and AMD tags are selectable. 4. Named volume persists pulled models across container recreation. 5. Other LocalDevStack containers can reach Ollama by service DNS. -6. Host clients can reach it through an explicitly configured loopback port when enabled. +6. Host clients can reach it through Nginx at the fixed loopback-only native Ollama endpoint. 7. `lds llm ...` delegates to the bundled provider CLI through LocalDevStack's Compose wrapper. 8. No project/repository bind mount is required for normal provider operation. 9. No Graphify/LocalDevStack-specific package is added to the image solely for integration. From a013e18a29ddad55baf6179329b097a5c6f47c3e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:56:35 +0600 Subject: [PATCH 332/509] docs(ai): remove retired direct-port workflow --- docs/guides/local-ai.rst | 31 +++++++++++++------------------ 1 file changed, 13 insertions(+), 18 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index a80ecca5..1ca0c118 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -71,20 +71,13 @@ User-facing HTTPS endpoint:: Internal Docker consumers should use the service endpoint directly rather than routing through Nginx. -Direct host Ollama access is disabled by default. Opt in with:: +Host-side native Ollama clients use the Nginx-owned endpoint:: - lds llm host-port on + http://llm-ollama.localhost:11434 -The default direct binding is loopback-only:: - - 127.0.0.1:11434 - -Show/disable it with:: - - lds llm host-port status - lds llm host-port off - -The port can be changed through ``LLM_OLLAMA_PORT`` when direct host access is enabled. +LocalDevStack binds Nginx's native listener to ``127.0.0.1:11434``. The +``llm-ollama`` provider container itself has no published host port, and there is no +host-port toggle or configurable provider port. Compose Ownership ----------------- @@ -108,8 +101,10 @@ When runtime-specific Compose data is required, ``lds`` creates a temporary frag under ``docker/.runtime/`` for the current command only: - NVIDIA -> ``gpus: all``; -- AMD/ROCm -> ``/dev/kfd`` and ``/dev/dri``; -- native host API -> ``127.0.0.1:11434:11434`` on Nginx -> ``llm-ollama:11434``. +- AMD/ROCm -> ``/dev/kfd`` and ``/dev/dri``. + +The native host API is not part of the temporary fragment. Nginx permanently owns +``127.0.0.1:11434:11434`` and proxies it to ``llm-ollama:11434``. The fragment is removed after the Compose command. ``configuration/compose/`` remains the normal extras/generated-runtime area and is not the location of built-in LLM @@ -277,12 +272,12 @@ AI layer. ``lds graphify [path] [extract-options...]`` runs the host Graphify CLI against the LocalDevStack Ollama provider. The default path is the current directory. It derives the model from ``LDS_AI_MODEL``, derives ``GRAPHIFY_API_TIMEOUT`` from -``LDS_AI_TIMEOUT``, and uses the actual loopback-published provider port unless -``OLLAMA_BASE_URL`` is already set. +``LDS_AI_TIMEOUT``, and uses ``http://llm-ollama.localhost:11434/v1`` through +Nginx unless ``OLLAMA_BASE_URL`` is already set. -Enable and apply the direct provider port before the default workflow:: +Start the stack with the AI profile, then run:: - lds up -d llm-ollama + lds up -d lds graphify Internally the workflow runs extraction with ``--backend ollama --no-cluster`` and, From b42b7543d979a4fd828858df755a24a49e1657c0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:56:40 +0600 Subject: [PATCH 333/509] docs(cli): point Graphify at fixed Nginx Ollama endpoint --- docs/reference/cli.rst | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index 1054edc1..ddd6fb0c 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -212,14 +212,11 @@ It performs ``extract --backend ollama --no-cluster`` followed by By default it derives: -- ``OLLAMA_BASE_URL`` from the actual loopback-published ``llm-ollama`` port; +- ``OLLAMA_BASE_URL=http://llm-ollama.localhost:11434/v1``, routed through Nginx; - ``OLLAMA_MODEL`` from ``LDS_AI_MODEL``; - ``GRAPHIFY_API_TIMEOUT`` from ``LDS_AI_TIMEOUT``. -The host port must already be applied with:: - - lds llm host-port on - lds up -d llm-ollama +The stack and AI profile must be running; there is no separate host-port setup step. An explicitly supplied ``OLLAMA_BASE_URL`` overrides the default ``http://llm-ollama.localhost:11434/v1`` endpoint. From cfa02add793a8c93b0e59a73e56261b31b6dd588 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:57:02 +0600 Subject: [PATCH 334/509] feat(graphify): preflight Nginx native Ollama endpoint --- lib/ai.sh | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 8643be6a..64b0cca7 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -30,14 +30,20 @@ cmd_ai() { } _graphify_local_base_url() { - local ctr - ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" - [[ -n "$ctr" ]] || - die "llm-ollama is not running. Enable the ai profile and start the stack first." + local provider_ctr nginx_ctr - docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || + provider_ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" + [[ -n "$provider_ctr" ]] || + die "llm-ollama is not running. Enable the ai profile and start the stack first." + docker inspect -f '{{.State.Running}}' "$provider_ctr" 2>/dev/null | grep -qx true || die "llm-ollama container exists but is not running." + nginx_ctr="$(docker_compose ps -q nginx 2>/dev/null | sed -n '1p' || true)" + [[ -n "$nginx_ctr" ]] || + die "nginx is not running. Start the LocalDevStack edge before using Graphify." + docker inspect -f '{{.State.Running}}' "$nginx_ctr" 2>/dev/null | grep -qx true || + die "nginx container exists but is not running." + printf '%s' 'http://llm-ollama.localhost:11434/v1' } From 00ba33676b8b5460a13786e6838b9fb8af08c68b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:57:39 +0600 Subject: [PATCH 335/509] test(ai): lock Nginx-owned native Ollama endpoint --- tests/ai-contract.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 4523d054..30e55b34 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -74,5 +74,6 @@ assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" assert_file_contains "$ROOT/lib/ai.sh" 'update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable"' -assert_file_contains "$ROOT/lib/compose.sh" '"127.0.0.1:%s:11434"' -pass "single companion AI service with ephemeral hardware/port augmentation" +assert_file_contains "$ROOT/docker/compose/http.yaml" '"127.0.0.1:11434:11434"' +assert_file_contains "$ROOT/lib/ai.sh" 'docker_compose ps -q nginx' +pass "single companion AI service with ephemeral hardware augmentation and Nginx-owned native port" From 9cd9672d694953c2557d135010f5781765725573 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:58:52 +0600 Subject: [PATCH 336/509] docs(plan): finish LLM-Ollama naming --- .../docker-ecosystem/07-localdevstack-integration-plan.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index dcd2ec63..c6467939 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -2098,7 +2098,7 @@ Compared LocalDevStack against the current related releases/main contracts: - Runner **0.5** - Nginx **0.4.1** - Apache **0.4.2** -- LLM-SM **current stable** +- LLM-Ollama **current stable** - Toolset **2.0** - Scriptomatic current `main` @@ -2180,9 +2180,9 @@ profile bridge would regress Admin Panel-created Apache hosts. This is a non-blocking follow-up, not a release defect. -## LLM-SM capability boundaries +## LLM-Ollama capability boundaries -The current published LLM-SM image contract is **linux/amd64 only**. LocalDevStack +The current published LLM-Ollama image contract is **linux/amd64 only**. LocalDevStack remains usable on arm64 with the `ai` profile disabled. LocalDevStack intentionally does not mount a repository/workspace into `llm-ollama` by From 59e61a5928603015d129c0f64c6e5272335dff15 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 20:59:30 +0600 Subject: [PATCH 337/509] test(env): remove retired LLM host-port contract --- tests/env-contract.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index 8353053e..6e936a0d 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -20,7 +20,6 @@ assert_file_contains "$ROOT/lds" 'compose_control_value()' assert_file_contains "$ROOT/lds" 'dotenv_value()' assert_file_contains "$ROOT/lib/compose.sh" 'compose_control_value LDS_AI_RUNTIME ""' assert_file_contains "$ROOT/lib/compose.sh" 'LDS_LLM_ARCH="$llm_arch"' -assert_file_contains "$ROOT/lib/compose.sh" 'LDS_LLM_HOST_PORT 0' pass "environment file and precedence wiring" git -C "$ROOT" check-ignore -q docker/.env || fail "docker/.env must remain ignored user state" From b4a2ebfe9c0cb1aaca52a569e128f95c31d92c4b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:00:15 +0600 Subject: [PATCH 338/509] docs: rename reserved Ollama domain --- docs/guides/domain-setup.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/domain-setup.rst b/docs/guides/domain-setup.rst index 6c1912bc..5faa7750 100644 --- a/docs/guides/domain-setup.rst +++ b/docs/guides/domain-setup.rst @@ -121,7 +121,7 @@ The certificate SAN set includes at least:: and Tools can include generated domain/service names. The wildcard covers built-in convenience endpoints such as ``admin.localhost``, -``webmail.localhost``, and ``llm.localhost``. +``webmail.localhost``, and ``llm-ollama.localhost``. Working in a Domain Container ----------------------------- From 298a2d4d8a23f8824a4700e3923a5949ba8f5d87 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:00:46 +0600 Subject: [PATCH 339/509] docs(ai): describe both Nginx Ollama endpoints --- docs/concepts/architecture.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index e121b97b..256b29e1 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -145,7 +145,7 @@ When the ``ai`` profile is enabled: 1. ``llm-ollama`` provides the Ollama runtime and persistent model store; 2. Tools consumes ``http://llm-ollama:11434`` internally; -3. Nginx exposes ``https://llm-ollama.localhost`` to the user; +3. Nginx exposes ``https://llm-ollama.localhost`` and the loopback-only native endpoint ``http://llm-ollama.localhost:11434`` to host clients; 4. ``lds ai`` delegates higher-level/operational AI to Tools; 5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-ollama`` CLI. From a3b0079741754b376069487073d1e9f57f2f6987 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:00:51 +0600 Subject: [PATCH 340/509] docs(tls): rename Ollama convenience host --- docs/guides/tls-and-certificates.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/tls-and-certificates.rst b/docs/guides/tls-and-certificates.rst index 35ce58c6..1265ecc7 100644 --- a/docs/guides/tls-and-certificates.rst +++ b/docs/guides/tls-and-certificates.rst @@ -60,7 +60,7 @@ The wildcard covers built-in endpoints such as: - ``ri.localhost``; - ``me.localhost``; - ``kibana.localhost``; -- ``llm.localhost``. +- ``llm-ollama.localhost``. Install the Root CA ------------------- From f0c56995b979b2505c83a22c5e68a2eb1cfa0d38 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:02:28 +0600 Subject: [PATCH 341/509] refactor(ai): align diagnostics with llm-ollama --- lib/diagnostics.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 81b34e7e..823acd8f 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -430,7 +430,7 @@ cmd_images() { ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" llm_arch="$(llm_arch_for_runtime "$ai_runtime")" - printf '%-16s %s\n' "LLM" "infocyph/llm-sm:$llm_arch" + printf '%-16s %s\n' "LLM" "infocyph/llm-ollama:$llm_arch" printf '%-16s %s\n' "LLM runtime" "$ai_runtime" printf '%-16s postgres:%s\n' "PostgreSQL" "$(compose_control_value POSTGRES_VERSION alpine)" printf '%-16s mysql:%s\n' "MySQL" "$(compose_control_value MYSQL_VERSION latest)" @@ -519,11 +519,11 @@ cmd_doctor() { if profile_enabled ai; then local llm - llm="$(docker_compose ps -q llm-sm 2>/dev/null | sed -n '1p' || true)" + llm="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" if [[ -n "$llm" ]] && docker inspect -f '{{.State.Running}}' "$llm" 2>/dev/null | grep -qx true; then _doctor_ok "AI provider container is running." else - _doctor_warn "AI profile is selected but llm-sm is not running." + _doctor_warn "AI profile is selected but llm-ollama is not running." warnings=$((warnings + 1)) fi fi From f2786d5760393a032e7bdd52ed180b194c9780f9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:02:33 +0600 Subject: [PATCH 342/509] refactor(ai): remove host-port setup handling --- lib/profiles.sh | 9 +-------- 1 file changed, 1 insertion(+), 8 deletions(-) diff --git a/lib/profiles.sh b/lib/profiles.sh index a5aea5b6..52dd878f 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -202,7 +202,7 @@ setup_service() { local detected_runtime detected_arch detected_runtime="$(compose_control_value LDS_AI_RUNTIME "$(detect_ai_runtime)")" detected_arch="$(llm_arch_for_runtime "$detected_runtime")" - printf "%bDetected local-AI runtime:%b %s (%s)\n" "$CYAN" "$NC" "$detected_runtime" "infocyph/llm-sm:$detected_arch" + printf "%bDetected local-AI runtime:%b %s (%s)\n" "$CYAN" "$NC" "$detected_runtime" "infocyph/llm-ollama:$detected_arch" fi local defaults="${PROFILE_ENV[$profile]:-}" @@ -230,13 +230,6 @@ setup_service() { *) die "AI runtime must be cpu, nvidia, or amd" ;; esac ;; - LDS_LLM_HOST_PORT) - case "${val,,}" in - 1 | y | yes | true | on) val=1 ;; - 0 | n | no | false | off) val=0 ;; - *) die "Direct localhost API must be yes or no" ;; - esac - ;; esac queue_env "$key=$val" From bb2542dd791d5484ddf29a1a324594096b1a51b3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:02:37 +0600 Subject: [PATCH 343/509] refactor(ai): rename open shortcut to llm-ollama --- lib/services.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/services.sh b/lib/services.sh index 6858193d..aad02e16 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -367,7 +367,7 @@ cmd_open() { redis | redisinsight | redis-insight | rds) url="https://ri.localhost" ;; mongo | me | mongoexpress | mongo-express) url="https://me.localhost" ;; kibana | kbn) url="https://kibana.localhost" ;; - ai | llm | llm-sm) url="https://llm.localhost" ;; + ai | llm | llm-ollama | ollama) url="https://llm-ollama.localhost" ;; *) url="https://${target}" ;; From 10f470deae8932b82b106158b688296e6fb90e38 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sat, 19 Sep 2026 21:12:49 +0600 Subject: [PATCH 344/509] docs(ai): fix runtime literal block formatting --- docs/concepts/profiles-and-env.rst | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index 03b0e349..d8698c97 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -229,6 +229,7 @@ it is ``0``. Use:: lds llm runtime + to override the detected runtime behavior. The selected ``LDS_AI_MODEL`` is also forwarded to the provider as From c265d0b4128f0df09fd2bd2b18ff3dfff4c7e71e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:01:57 +0600 Subject: [PATCH 345/509] fix(cli): preserve proxied stdin and bundle paths --- lds | 14 +++++++++++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/lds b/lds index a94b5405..9b7ef6bb 100755 --- a/lds +++ b/lds @@ -162,9 +162,10 @@ lds_tools_cmd() { return 127 fi - local -a flags=() - [[ -t 0 ]] && flags+=(-i) - [[ -t 1 ]] && flags+=(-t) + # Keep stdin attached even for pipes/here-strings. Only allocate a TTY when + # both sides are interactive; proxied jq/rg/etc. must behave like host tools. + local -a flags=(-i) + [[ -t 0 && -t 1 ]] && flags+=(-t) "$(bin_path docker)" exec "${flags[@]}" "$ctr" "$cmd" "$@" return $? fi @@ -598,6 +599,13 @@ cmd_bundle() { local base="lds_bundle_${project}_${ts}" [[ -n "$out" ]] || out="$PWD/${base}.zip" + # The archive is created from inside the temporary staging directory. Resolve + # explicit relative destinations before changing directory so the final ZIP + # survives staging cleanup. + if [[ "$out" != /* ]]; then + out="$PWD/${out#./}" + fi + { echo "project=$project" echo "dir=$DIR" From 886c403a5ccdf13fefd211c204a3252bbf6c23dc Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:02:43 +0600 Subject: [PATCH 346/509] fix(profiles): make reselection preserve user state --- lib/profiles.sh | 39 +++++++++++++++++++++++++++++++++++---- 1 file changed, 35 insertions(+), 4 deletions(-) diff --git a/lib/profiles.sh b/lib/profiles.sh index 52dd878f..09198b43 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -123,7 +123,7 @@ setup_menu_print() { } >&2 } -# Parse user selection into indices or ALL/NONE (prints one token per line) +# Parse user selection into indices or ALL/NONE/CANCEL (prints one token per line) setup_menu_parse() { local input="${1//[[:space:]]/}" [[ -n "$input" ]] || return 1 @@ -157,10 +157,15 @@ setup_choose_services() { continue fi - if grep -qx "NONE" <<<"$parsed"; then + if grep -qx "CANCEL" <<<"$parsed"; then return 1 fi + if grep -qx "NONE" <<<"$parsed"; then + printf '%s\n' "__NONE__" + return 0 + fi + if grep -qx "ALL" <<<"$parsed"; then printf "%s\n" "${SERVICE_ORDER[@]}" return 0 @@ -214,13 +219,30 @@ setup_service() { IFS=';' read -r -a pairs <<<"$defaults" IFS=';' read -r -a prompts <<<"${PROFILE_PROMPTS[$profile]:-}" - local i pair key def val prompt + local i pair key def val prompt current input for i in "${!pairs[@]}"; do pair="${pairs[$i]}" [[ -n "$pair" ]] || continue IFS='=' read -r key def <<<"$pair" prompt="${prompts[$i]:-$key}" - val="$(read_default "$prompt" "$def")" + current="$(dotenv_value "$ENV_DOCKER" "$key" 2>/dev/null || true)" + + # Re-running setup must not reset a user's selected versions or credentials. + # Secret-like values are preserved without printing their current/default value. + case "$key" in + *PASSWORD* | *SECRET* | *TOKEN* | *PRIVATE_KEY* | *API_KEY* | *ACCESS_KEY*) + if [[ -n "$current" ]]; then + tty_readline input "$(printf '%b%s [configured; Enter keeps current]:%b ' "$CYAN" "$prompt" "$NC")" || return 1 + val="${input:-$current}" + else + tty_readline input "$(printf '%b%s [Enter uses catalog default]:%b ' "$CYAN" "$prompt" "$NC")" || return 1 + val="${input:-$def}" + fi + ;; + *) + val="$(read_default "$prompt" "${current:-$def}")" + ;; + esac case "$key" in LDS_AI_RUNTIME) @@ -238,11 +260,20 @@ setup_service() { process_all() { local selected + PENDING_ENVS=() + PENDING_PROFILES=() + if ! selected="$(setup_choose_services)"; then printf "\n%bSetup cancelled.%b\n" "$YELLOW" "$NC" return 0 fi + if [[ "$selected" == "__NONE__" ]]; then + flush_profiles + printf "\n%b✅ Catalog-managed service profiles cleared; generated runtime/domain profiles preserved.%b\n" "$GREEN" "$NC" + return 0 + fi + printf "\n%bWill configure:%b\n" "$CYAN" "$NC" while IFS= read -r svc; do printf " - %s (%s)\n" "${SERVICE_DISPLAY[$svc]:-$svc}" "${SERVICES[$svc]}" From 3baead7a9cdf0065a0b5de674751d4fdbf135a59 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:02:47 +0600 Subject: [PATCH 347/509] fix(profiles): report and validate effective compose profiles --- lib/services.sh | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/lib/services.sh b/lib/services.sh index aad02e16..82a2d839 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -412,8 +412,13 @@ cmd_profiles() { " "$CYAN" "$NC" "${cur:-}" printf "%bAvailable profiles:%b " "$CYAN" "$NC" - printf ' - %s -' "${SERVICES[@]}" | LC_ALL=C sort -u + local available + available="$(docker_compose config --profiles 2>/dev/null || true)" + if [[ -n "$available" ]]; then + printf '%s\n' "$available" | sed '/^[[:space:]]*$/d' | LC_ALL=C sort -fu | sed 's/^/ - /' + else + printf ' %b%b\n' "$DIM" "$NC" + fi # warn if enabled profile has no mention in compose if [[ -n "$cur" ]]; then local p @@ -428,7 +433,9 @@ cmd_profiles() { ;; add) [[ $# -gt 0 ]] || die "profiles add " + local p for p in "$@"; do + _known_profile "$p" || die "Unknown profile: $p" modify_profiles add "$p" done ;; From 70027b84b6476ede60f8932f79032d93e1d60e98 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:03:21 +0600 Subject: [PATCH 348/509] fix(ai): fail fast when Graphify model is unavailable --- lib/ai.sh | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 64b0cca7..1240aaba 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -47,6 +47,15 @@ _graphify_local_base_url() { printf '%s' 'http://llm-ollama.localhost:11434/v1' } +_graphify_local_model_preflight() { + local model="${1:-}" + [[ -n "$model" ]] || return 1 + + if ! docker_compose exec -T llm-ollama /bin/ollama show "$model" >/dev/null 2>&1; then + die "Ollama model '$model' is not available in LocalDevStack. Run: lds llm pull $model" + fi +} + cmd_graphify() { need_bin graphify "install the Graphify CLI on the host first" @@ -54,8 +63,13 @@ cmd_graphify() { [[ $# -eq 0 ]] || shift [[ -e "$target" ]] || die "Graphify target does not exist: $target" - local base_url timeout model api_key graphify_bin arg next_is_model=0 next_is_timeout=0 - base_url="${OLLAMA_BASE_URL:-$(_graphify_local_base_url)}" + local base_url timeout model api_key graphify_bin arg next_is_model=0 next_is_timeout=0 local_provider=0 + if [[ -n "${OLLAMA_BASE_URL:-}" ]]; then + base_url="$OLLAMA_BASE_URL" + else + base_url="$(_graphify_local_base_url)" + local_provider=1 + fi timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen3:14b)}" api_key="${OLLAMA_API_KEY:-local}" @@ -91,6 +105,8 @@ cmd_graphify() { [[ "$timeout" =~ ^[0-9]+$ ]] && ((timeout >= 1)) || die "GRAPHIFY_API_TIMEOUT/--api-timeout must be a positive integer" + ((local_provider == 0)) || _graphify_local_model_preflight "$model" + graphify_bin="$(bin_path graphify)" ( From 01568b473322d5cbdb6b74a682e1be1b0f8a77e8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:03:26 +0600 Subject: [PATCH 349/509] test(profiles): cover NONE and configured-value preservation --- tests/env-contract.sh | 25 +++++++++++++++++++++++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index 6e936a0d..1dcacf90 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -40,7 +40,10 @@ pass "release env contains only genuinely variable release defaults" assert_file_contains "$ROOT/lib/profiles.sh" 'CATALOG_FILE="$CFG/catalog/services.psv"' assert_file_contains "$ROOT/lib/profiles.sh" 'load_service_catalog()' assert_file_contains "$ROOT/lib/profiles.sh" 'load_service_catalog' -pass "profile setup loads the tracked host catalog" +assert_file_contains "$ROOT/lib/profiles.sh" 'configured; Enter keeps current' +assert_file_contains "$ROOT/lib/services.sh" 'docker_compose config --profiles' +assert_file_contains "$ROOT/lib/services.sh" 'Unknown profile: $p' +pass "profile setup loads the catalog, preserves user state, and validates effective profiles" assert_file_contains "$ROOT/lib/ai.sh" 'cmd_ai()' assert_file_contains "$ROOT/lib/ai.sh" 'cmd_llm()' @@ -82,9 +85,27 @@ profile_tmp="$(mktemp -d)" actual="$(grep '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1)" [[ "$actual" == 'COMPOSE_PROFILES=postgresql,apache,php84' ]] || fail "profile reselection drifted: $actual" + + printf '%s\n' 'COMPOSE_PROFILES=mysql,redis,ai,apache,php84' >"$ENV_DOCKER" + PENDING_PROFILES=() + flush_profiles + actual="$(grep '^COMPOSE_PROFILES=' "$ENV_DOCKER" | tail -n1)" + [[ "$actual" == 'COMPOSE_PROFILES=apache,php84' ]] || + fail "NONE selection did not clear only catalog-managed profiles: $actual" + + [[ "$(setup_menu_parse n)" == "NONE" ]] || fail "profile menu NONE token drifted" + [[ "$(setup_menu_parse q)" == "CANCEL" ]] || fail "profile menu CANCEL token drifted" + + printf '%s\n' 'REDIS_VERSION=7.4-alpine' >>"$ENV_DOCKER" + read_default() { printf '%s' "$2"; } + PENDING_ENVS=() + PENDING_PROFILES=() + setup_service REDIS >/dev/null + [[ "${PENDING_ENVS[0]:-}" == 'REDIS_VERSION=7.4-alpine' ]] || + fail "profile setup did not preserve configured value" ) rm -rf "$profile_tmp" -pass "profile setup replaces managed selections and preserves generated profiles" +pass "profile setup replaces/clears managed selections while preserving generated profiles and configured values" assert_file_contains "$ROOT/lib/compose.sh" 'compose_control_value COMPOSE_PROJECT_NAME LocalDevStack' pass "CLI project identity follows the Compose project contract" From 7d6c3e6832d36a772cece52746a258fc8feb8f78 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:27 +0600 Subject: [PATCH 350/509] chore(images): move Elastic defaults to latest --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index feb2eaeb..53b7cc03 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -2,7 +2,7 @@ POSTGRESQL|postgresql|PostgreSQL|postgres|POSTGRES_VERSION|POSTGRES_VERSION=alpine;POSTGRES_USER=postgres;POSTGRES_PASSWORD=postgres;POSTGRES_DATABASE=postgres|Version;User;Password;Database|cloudbeaver|PostgresStore|https://db.localhost|database|1|0| MYSQL|mysql|MySQL|mysql|MYSQL_VERSION|MYSQL_VERSION=latest;MYSQL_ROOT_PASSWORD=12345;MYSQL_USER=infocyph;MYSQL_PASSWORD=12345;MYSQL_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MySQLStore|https://db.localhost|database|1|0| MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_ROOT_PASSWORD=12345;MARIADB_USER=infocyph;MARIADB_PASSWORD=12345;MARIADB_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MariaDBStore|https://db.localhost|database|1|0| -ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.3|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| +ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=latest|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| AI|ai|Local AI|llm-ollama|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b|Preferred model||LLMModels|https://llm-ollama.localhost|ai|1|0|cpu,nvidia,amd From bf3a696c0814555e85301d8e11e41f9aa835d756 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:30 +0600 Subject: [PATCH 351/509] chore(images): use latest Elasticsearch by default --- docker/compose/db.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index 1e9b0270..686944b7 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -113,7 +113,7 @@ services: <<: *db-service container_name: ELASTICSEARCH hostname: elasticsearch - image: elasticsearch:${ELASTICSEARCH_VERSION:-9.5.3} + image: elasticsearch:${ELASTICSEARCH_VERSION:-latest} profiles: [elasticsearch] environment: - TZ=${TZ:-} From 71a6cde5e3fa7004ccd51f55f62c5da01fb36ec5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:33 +0600 Subject: [PATCH 352/509] chore(images): use latest Elastic clients by default --- docker/compose/db-client.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index da50d33c..7ff06f12 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -59,7 +59,7 @@ services: <<: *db-client-service container_name: KIBANA hostname: kibana - image: kibana:${ELASTICSEARCH_VERSION:-9.5.3} + image: kibana:${ELASTICSEARCH_VERSION:-latest} profiles: [elasticsearch] depends_on: elasticsearch: @@ -72,7 +72,7 @@ services: - ../../logs/kibana:/usr/share/kibana/logs filebeat: - image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.5.3} + image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-latest} container_name: FILEBEAT restart: unless-stopped user: root From e1f87623f5486ec3908a43f667278122ce1b8340 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:37 +0600 Subject: [PATCH 353/509] chore(images): report latest Elastic default --- lib/diagnostics.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 823acd8f..fdbc58c3 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -420,7 +420,7 @@ cmd_config() { cmd_images() { local elastic - elastic="$(compose_control_value ELASTICSEARCH_VERSION 9.5.3)" + elastic="$(compose_control_value ELASTICSEARCH_VERSION latest)" printf '%-16s %s\n' "Tools" "infocyph/tools:latest" printf '%-16s %s\n' "Runner" "infocyph/runner:latest" From 9d233e08cad249d8f0b3a6b109498ec8acae9061 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:40 +0600 Subject: [PATCH 354/509] test(images): require latest Elastic catalog default --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index c96efef5..5e90ec39 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -56,7 +56,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$defaults" != *"LDS_AI_RUNTIME="* ]] || fail "AI profile wizard must not prompt for runtime" ;; ELASTICSEARCH) - [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.3"* ]] || fail "Elastic stable default drift" + [[ "$defaults" == *"ELASTICSEARCH_VERSION=latest"* ]] || fail "Elastic latest default drift" ;; esac done <"$catalog" From 5d28483eb1dbac671765957b66b9cd9bcc7b0b6b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:43 +0600 Subject: [PATCH 355/509] test(images): require latest Elastic compose images --- tests/compose-contract.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index b4a8a078..77a94b95 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -108,9 +108,9 @@ elastic_json="$("${compose[@]}" --profile elasticsearch --profile filebeat confi python3 -c ' import json,sys d=json.load(sys.stdin) -assert d["services"]["elasticsearch"]["image"] == "elasticsearch:9.5.3" -assert d["services"]["kibana"]["image"] == "kibana:9.5.3" -assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9.5.3" +assert d["services"]["elasticsearch"]["image"] == "elasticsearch:latest" +assert d["services"]["kibana"]["image"] == "kibana:latest" +assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:latest" ' <<<"$elastic_json" pass "Elastic stack uses aligned current-stable tags because latest is unsupported" From 693b0d9ac1a2d01b5d94d96f4e36ffb6530d812b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:04:46 +0600 Subject: [PATCH 356/509] test(images): align image output with latest Elastic --- tests/qol-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 04d7b955..23ecb37a 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -43,7 +43,7 @@ assert_contains "$images" "infocyph/runner:latest" assert_contains "$images" "infocyph/nginx:latest" assert_contains "$images" "infocyph/apache:latest" assert_contains "$images" "postgres:alpine" -assert_contains "$images" "elasticsearch:9.5.3" +assert_contains "$images" "elasticsearch:latest" assert_contains "$images" "localdevstack-php: (Alpine)" pass "images is offline-safe and reports fixed plus configurable image selections" From 6ca332710bb4e3590e9cc830683d1533689b40aa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:05:04 +0600 Subject: [PATCH 357/509] test(bundle): cover relative archive destinations --- tests/qol-contract.sh | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 23ecb37a..a95f7360 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -98,6 +98,15 @@ default_bundle="$(find "$default_bundle_dir" -maxdepth 1 -type f -name 'lds_bund rm -rf "$default_bundle_dir" pass "support bundle option-only invocation uses a generated filename" +relative_bundle_dir="$(mktemp -d)" +( + cd "$relative_bundle_dir" + "$ROOT/lds" support bundle --redact relative-support.zip >/dev/null + [[ -s relative-support.zip ]] || fail "relative support bundle path was lost with staging cleanup" +) +rm -rf "$relative_bundle_dir" +pass "support bundle preserves explicit relative output paths" + help="$("$ROOT/lds" help)" assert_contains "$help" "doctor" assert_contains "$help" "images" From 0539766bb91ba409f6ba638c75e7889f993670e2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:05:07 +0600 Subject: [PATCH 358/509] test(cli): lock proxied stdin behavior --- tests/cli-contract.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index 29a1f163..3652741e 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -66,3 +66,7 @@ pass "Graphify host workflow wrapper" assert_file_contains "$ROOT/lds" 'exec "$DIR/bin/tool-runner" "$cmd" "$@"' pass "unknown command fallback remains delegated to tool-runner" + +assert_file_contains "$ROOT/lds" 'local -a flags=(-i)' +assert_file_contains "$ROOT/lds" '[[ -t 0 && -t 1 ]] && flags+=(-t)' +pass "proxied host tools preserve piped stdin without forcing a TTY" From 7a3e8be31c7e63638fc8333d887d2fe18049f521 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:05:10 +0600 Subject: [PATCH 359/509] test(ai): cover Graphify model preflight --- tests/ai-contract.sh | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 30e55b34..a2a9ece2 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -67,6 +67,8 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PREFLIGHT_TIM assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800}' assert_file_contains "$ROOT/docker/compose/http.yaml" 'LLM_PROXY_TIMEOUT_SECONDS=${LDS_AI_TIMEOUT:-1800}' assert_file_contains "$ROOT/lib/ai.sh" 'cmd_graphify()' +assert_file_contains "$ROOT/lib/ai.sh" '_graphify_local_model_preflight()' +assert_file_contains "$ROOT/lib/ai.sh" "Run: lds llm pull \$model" assert_file_contains "$ROOT/lib/ai.sh" 'OLLAMA_API_KEY="$api_key"' assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" extract "$target" --backend ollama --no-cluster' assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" cluster-only "$target" --backend ollama' @@ -77,3 +79,19 @@ assert_file_contains "$ROOT/lib/ai.sh" 'update_env "$ENV_DOCKER" LDS_AI_IGPU_ENA assert_file_contains "$ROOT/docker/compose/http.yaml" '"127.0.0.1:11434:11434"' assert_file_contains "$ROOT/lib/ai.sh" 'docker_compose ps -q nginx' pass "single companion AI service with ephemeral hardware augmentation and Nginx-owned native port" + +( + set -euo pipefail + die() { return 1; } + docker_compose() { + [[ "$*" == "exec -T llm-ollama /bin/ollama show qwen3:14b" ]] + } + # shellcheck source=lib/ai.sh + source "$ROOT/lib/ai.sh" + _graphify_local_model_preflight qwen3:14b || + fail "Graphify local-model preflight rejected an available model" + if _graphify_local_model_preflight missing-model; then + fail "Graphify local-model preflight accepted a missing model" + fi +) +pass "Graphify validates the selected local model before extraction" From c8187b55d569853d65a26617e13c72aa2aa38a5f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:05:38 +0600 Subject: [PATCH 360/509] docs: align profiles Graphify and latest image policy --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index acf7269b..65dd24b5 100644 --- a/README.md +++ b/README.md @@ -126,7 +126,7 @@ lds setup profile Catalog-managed optional profiles are `postgresql`, `mysql`, `mariadb`, `mongodb`, `redis`, `elasticsearch`, and `ai`. -Re-running the wizard **replaces the catalog-managed selection** while preserving generated domain/runtime profiles. +Re-running the wizard **replaces the catalog-managed selection** while preserving generated domain/runtime profiles and previously configured values. Choose `NONE` to clear catalog-managed profiles; `CANCEL / Back` leaves the current selection unchanged. Manual profile operations remain available: @@ -169,7 +169,7 @@ Examples: - local AI: `infocyph/llm-ollama:latest` - AMD local AI: `infocyph/llm-ollama:amd-latest` -Elasticsearch, Kibana, and Filebeat stay on one aligned Elastic version. +Elasticsearch, Kibana, and Filebeat share the same `ELASTICSEARCH_VERSION` selector and default to the moving `latest` tag. Inspect the effective defaults with `lds images`. @@ -274,7 +274,7 @@ lds llm chat ... LocalDevStack publishes the native Ollama API through Nginx on the fixed loopback-only endpoint `http://llm-ollama.localhost:11434`. The `llm-ollama` container itself remains internal and never owns a host port. -`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets `http://llm-ollama.localhost:11434/v1` through Nginx. The command runs extraction with `--backend ollama --no-cluster`, then runs `cluster-only` for the same path if extraction succeeds. This avoids clustering twice. +`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets `http://llm-ollama.localhost:11434/v1` through Nginx. For the LocalDevStack provider, the command first verifies that the selected model exists and fails immediately with an `lds llm pull ` hint when it does not. It then runs extraction with `--backend ollama --no-cluster`, followed by `cluster-only` for the same path if extraction succeeds. Start the AI profile, then Graphify can use the native endpoint directly: From 536ce8b7843e48bcf45080ee7b9266ec9f1e2950 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:05:56 +0600 Subject: [PATCH 361/509] docs: align profiles Graphify and latest image policy From a68e56693a32fe72aca2e864763ec75796830c52 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:05:59 +0600 Subject: [PATCH 362/509] docs(profiles): document replacement and current-value semantics --- docs/concepts/profiles-and-env.rst | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index d8698c97..f99c54e8 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -47,6 +47,11 @@ the generated runtime/server profiles:: while removing the previously selected catalog services. +Choosing ``NONE`` clears all catalog-managed service profiles while preserving generated +domain/runtime profiles. Choosing ``CANCEL / Back`` leaves the existing selection +unchanged. When a selected service is configured again, existing user values are reused +as prompt defaults; secret-like values are preserved without printing them. + Manual profile operations remain available:: lds profiles list @@ -163,7 +168,7 @@ Examples: - Tools, Runner, Nginx, and Apache use their published ``:latest`` aliases. - Standard local AI uses ``infocyph/llm-ollama:latest``. - AMD local AI uses ``infocyph/llm-ollama:amd-latest``. -- Elasticsearch, Kibana, and Filebeat stay on one aligned version. +- Elasticsearch, Kibana, and Filebeat share ``ELASTICSEARCH_VERSION`` and default to ``latest``. Run:: From 102abb1c5a67ab0db305ea96bd4d8a938f9c2182 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:06:02 +0600 Subject: [PATCH 363/509] docs(ai): document Graphify model preflight --- docs/guides/local-ai.rst | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 1ca0c118..d1b705af 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -280,6 +280,12 @@ Start the stack with the AI profile, then run:: lds up -d lds graphify +When LocalDevStack owns the provider endpoint, ``lds graphify`` first verifies the +selected model in the running Ollama provider. A missing model fails before repository +extraction starts and reports the matching ``lds llm pull `` command. An explicit +external ``OLLAMA_BASE_URL`` remains caller-controlled and skips this local-provider +preflight. + Internally the workflow runs extraction with ``--backend ollama --no-cluster`` and, only after a successful extraction, runs ``cluster-only`` against the same path. This preserves the requested two-phase flow without performing the default extraction From ee76822f7d83cd5d96106ac66694cf5c19a49334 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:06:05 +0600 Subject: [PATCH 364/509] docs(plan): adopt latest Elastic moving-tag policy --- .../07-localdevstack-integration-plan.md | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index c6467939..34f1492d 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1945,17 +1945,12 @@ LDS_LLM_ARCH=latest Other runtime defaults follow the same rule. PostgreSQL uses `postgres:alpine`; MySQL, MariaDB, MongoDB, Redis Stack/Redis Insight, CloudBeaver, Mongo Express and Mailpit use their normal moving latest tags because the selected image family does not provide a suitable moving Alpine alias for this stack. -## Elastic exception +## Elastic moving-tag policy -Elasticsearch and Kibana explicitly do not support a `latest` tag, and the Elastic Filebeat registry publishes versioned releases rather than a supported moving `latest` contract. - -Keep Elasticsearch, Kibana and Filebeat on one aligned current-stable version. At implementation time this is: - -```text -9.5.3 -``` - -When this Elastic exception is advanced, update all three together and update the LocalDevStack catalog/contract tests in the same change. +Elasticsearch, Kibana and Filebeat use the same `ELASTICSEARCH_VERSION` selector and +default to `latest`. This follows the product-wide moving-tag policy instead of pinning +LocalDevStack to an older tested release. The three services must continue to use the +same selector so a user override advances or pins them together. ## Override precedence @@ -2069,7 +2064,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - PostgreSQL defaults to `postgres:alpine`. - Tools, Runner, Nginx and Apache consume their published `:latest` aliases. - One LLM service uses `infocyph/llm-ollama:${LDS_LLM_ARCH}`; CPU/NVIDIA map to `latest`, AMD/ROCm maps to `amd-latest`. -- Elasticsearch, Kibana and Filebeat remain version-aligned on the tested stable version because their required image contract does not provide a suitable moving `latest` alias. +- Elasticsearch, Kibana and Filebeat share the `ELASTICSEARCH_VERSION` selector and default to the moving `latest` tag. - PHP/Node runtime selection remains user-driven and version-specific. - Existing named volumes and container names remain intentionally stable for this release. From 67a0a80b6d18c64d7ef8586c9c8bb814f1a46cde Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:06:13 +0600 Subject: [PATCH 365/509] test(images): verify every moving latest alias --- tests/published-images.sh | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/tests/published-images.sh b/tests/published-images.sh index c0047605..bf8e886c 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -29,6 +29,29 @@ for image in "${images[@]}"; do done pass "published infrastructure compatibility images exist" +moving_images=( + "postgres:alpine" + "mysql:latest" + "mariadb:latest" + "mongo:latest" + "redis/redis-stack-server:latest" + "redis/redisinsight:latest" + "dbeaver/cloudbeaver:latest" + "mongo-express:latest" + "axllent/mailpit:latest" + "elasticsearch:latest" + "kibana:latest" + "docker.elastic.co/beats/filebeat:latest" + "infocyph/llm-ollama:latest" + "infocyph/llm-ollama:amd-latest" +) + +for image in "${moving_images[@]}"; do + printf 'Checking moving image alias %s\n' "$image" + docker manifest inspect "$image" >/dev/null +done +pass "all configured moving image aliases resolve" + for image in "$tools_image" "$runner_image"; do health="$(docker image inspect "$image" --format '{{json .Config.Healthcheck}}')" [[ -n "$health" && "$health" != "null" ]] || fail "$image must publish a healthcheck" From 1ee59c96721fb9a4837c42da13559ec673e745c8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:06:45 +0600 Subject: [PATCH 366/509] fix(profiles): align menu labels with NONE and cancel semantics --- lib/profiles.sh | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/profiles.sh b/lib/profiles.sh index 09198b43..dd0b7e78 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -119,7 +119,8 @@ setup_menu_print() { i=$((i + 1)) done printf "\n a) ALL\n" - printf " n) NONE / Back\n\n" + printf " n) NONE\n" + printf " q) CANCEL / Back\n\n" } >&2 } From 9b50163ed9d29c1ca39f1f6d8be56ce5eb49cb26 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:07:46 +0600 Subject: [PATCH 367/509] fix(profiles): parse cancel and back selectors --- lib/profiles.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/profiles.sh b/lib/profiles.sh index dd0b7e78..7b6b87f2 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -141,6 +141,7 @@ setup_menu_parse() { /^[0-9]+$/ { print $0; next } /^[aA]$/ { print "ALL"; next } /^[nN]$/ { print "NONE"; next } + /^[qQbB]$/ { print "CANCEL"; next } { ok=0 } END { if (!ok) exit 2 } ' From 59ecf3fd96f86e9ae2595b28048539d17a2d1cfc Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:25 +0600 Subject: [PATCH 368/509] chore(images): use current Elastic 9.5.4 --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index 53b7cc03..b2f8e944 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -2,7 +2,7 @@ POSTGRESQL|postgresql|PostgreSQL|postgres|POSTGRES_VERSION|POSTGRES_VERSION=alpine;POSTGRES_USER=postgres;POSTGRES_PASSWORD=postgres;POSTGRES_DATABASE=postgres|Version;User;Password;Database|cloudbeaver|PostgresStore|https://db.localhost|database|1|0| MYSQL|mysql|MySQL|mysql|MYSQL_VERSION|MYSQL_VERSION=latest;MYSQL_ROOT_PASSWORD=12345;MYSQL_USER=infocyph;MYSQL_PASSWORD=12345;MYSQL_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MySQLStore|https://db.localhost|database|1|0| MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_ROOT_PASSWORD=12345;MARIADB_USER=infocyph;MARIADB_PASSWORD=12345;MARIADB_DATABASE=localdb|Version;Root password;User;Password;Database|cloudbeaver|MariaDBStore|https://db.localhost|database|1|0| -ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=latest|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| +ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.4|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| AI|ai|Local AI|llm-ollama|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b|Preferred model||LLMModels|https://llm-ollama.localhost|ai|1|0|cpu,nvidia,amd From a2fd7c775cfec5e431985afcfd8fbdfd61077fbe Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:30 +0600 Subject: [PATCH 369/509] chore(images): use current Elasticsearch release --- docker/compose/db.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index 686944b7..da1a99cf 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -113,7 +113,7 @@ services: <<: *db-service container_name: ELASTICSEARCH hostname: elasticsearch - image: elasticsearch:${ELASTICSEARCH_VERSION:-latest} + image: elasticsearch:${ELASTICSEARCH_VERSION:-9.5.4} profiles: [elasticsearch] environment: - TZ=${TZ:-} From abd9e6afa9dc15b218742055f856fb18ea2e6f6a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:34 +0600 Subject: [PATCH 370/509] chore(images): align Kibana and Filebeat 9.5.4 --- docker/compose/db-client.yaml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index 7ff06f12..140f286b 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -59,7 +59,7 @@ services: <<: *db-client-service container_name: KIBANA hostname: kibana - image: kibana:${ELASTICSEARCH_VERSION:-latest} + image: kibana:${ELASTICSEARCH_VERSION:-9.5.4} profiles: [elasticsearch] depends_on: elasticsearch: @@ -72,7 +72,7 @@ services: - ../../logs/kibana:/usr/share/kibana/logs filebeat: - image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-latest} + image: docker.elastic.co/beats/filebeat:${ELASTICSEARCH_VERSION:-9.5.4} container_name: FILEBEAT restart: unless-stopped user: root From 7a5533b412e4d10dde3297ff7d72acb5ade68267 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:37 +0600 Subject: [PATCH 371/509] chore(images): report current Elastic release --- lib/diagnostics.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index fdbc58c3..60d357d7 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -420,7 +420,7 @@ cmd_config() { cmd_images() { local elastic - elastic="$(compose_control_value ELASTICSEARCH_VERSION latest)" + elastic="$(compose_control_value ELASTICSEARCH_VERSION 9.5.4)" printf '%-16s %s\n' "Tools" "infocyph/tools:latest" printf '%-16s %s\n' "Runner" "infocyph/runner:latest" From 57d1ade77a59957317d2dc233e033a2ed4aea789 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:41 +0600 Subject: [PATCH 372/509] test(images): require current Elastic release --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 5e90ec39..734365a3 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -56,7 +56,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$defaults" != *"LDS_AI_RUNTIME="* ]] || fail "AI profile wizard must not prompt for runtime" ;; ELASTICSEARCH) - [[ "$defaults" == *"ELASTICSEARCH_VERSION=latest"* ]] || fail "Elastic latest default drift" + [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.4"* ]] || fail "Elastic current default drift" ;; esac done <"$catalog" From 5671316f11013089f71135e4513c08130be2be2c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:46 +0600 Subject: [PATCH 373/509] test(images): align Elastic 9.5.4 contracts --- tests/compose-contract.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 77a94b95..a9d42712 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -108,9 +108,9 @@ elastic_json="$("${compose[@]}" --profile elasticsearch --profile filebeat confi python3 -c ' import json,sys d=json.load(sys.stdin) -assert d["services"]["elasticsearch"]["image"] == "elasticsearch:latest" -assert d["services"]["kibana"]["image"] == "kibana:latest" -assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:latest" +assert d["services"]["elasticsearch"]["image"] == "elasticsearch:9.5.4" +assert d["services"]["kibana"]["image"] == "kibana:9.5.4" +assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9.5.4" ' <<<"$elastic_json" pass "Elastic stack uses aligned current-stable tags because latest is unsupported" From 21c8841653a6cbbcf767c7d959b53fd05f3efefe Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:49 +0600 Subject: [PATCH 374/509] test(images): report current Elastic release --- tests/qol-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index a95f7360..9f1b7723 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -43,7 +43,7 @@ assert_contains "$images" "infocyph/runner:latest" assert_contains "$images" "infocyph/nginx:latest" assert_contains "$images" "infocyph/apache:latest" assert_contains "$images" "postgres:alpine" -assert_contains "$images" "elasticsearch:latest" +assert_contains "$images" "elasticsearch:9.5.4" assert_contains "$images" "localdevstack-php: (Alpine)" pass "images is offline-safe and reports fixed plus configurable image selections" From f3c0ebbd3cde8ae16bd71f4346244dd5729669c3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:08:54 +0600 Subject: [PATCH 375/509] test(images): verify current Elastic release references --- tests/published-images.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index bf8e886c..535dfb03 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -50,7 +50,7 @@ for image in "${moving_images[@]}"; do printf 'Checking moving image alias %s\n' "$image" docker manifest inspect "$image" >/dev/null done -pass "all configured moving image aliases resolve" +pass "all configured current image references resolve" for image in "$tools_image" "$runner_image"; do health="$(docker image inspect "$image" --format '{{json .Config.Healthcheck}}')" From c69b4c155530596cf28d79f13405b78cc4c21e45 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:10:07 +0600 Subject: [PATCH 376/509] test(images): use resolvable current Elastic release --- tests/published-images.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index 535dfb03..4b940309 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -39,15 +39,15 @@ moving_images=( "dbeaver/cloudbeaver:latest" "mongo-express:latest" "axllent/mailpit:latest" - "elasticsearch:latest" - "kibana:latest" - "docker.elastic.co/beats/filebeat:latest" + "elasticsearch:9.5.4" + "kibana:9.5.4" + "docker.elastic.co/beats/filebeat:9.5.4" "infocyph/llm-ollama:latest" "infocyph/llm-ollama:amd-latest" ) for image in "${moving_images[@]}"; do - printf 'Checking moving image alias %s\n' "$image" + printf 'Checking configured image reference %s\n' "$image" docker manifest inspect "$image" >/dev/null done pass "all configured current image references resolve" From ac86cd01ebbdb2b85d0de528a07d22a275a6a48c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:10:11 +0600 Subject: [PATCH 377/509] docs(images): document current Elastic release exception --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 65dd24b5..164f2b04 100644 --- a/README.md +++ b/README.md @@ -169,7 +169,7 @@ Examples: - local AI: `infocyph/llm-ollama:latest` - AMD local AI: `infocyph/llm-ollama:amd-latest` -Elasticsearch, Kibana, and Filebeat share the same `ELASTICSEARCH_VERSION` selector and default to the moving `latest` tag. +Elastic does not provide a usable moving `latest` alias for this stack, so Elasticsearch, Kibana, and Filebeat share `ELASTICSEARCH_VERSION` and default to the current stable `9.5.4`; overriding that one value advances or pins all three together. Inspect the effective defaults with `lds images`. From 9df3a1c9186bd8be3a40a70ccd60ec861abcb471 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:10:14 +0600 Subject: [PATCH 378/509] docs(images): align Elastic exception with 9.5.4 --- docs/concepts/profiles-and-env.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index f99c54e8..adb932d9 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -168,7 +168,7 @@ Examples: - Tools, Runner, Nginx, and Apache use their published ``:latest`` aliases. - Standard local AI uses ``infocyph/llm-ollama:latest``. - AMD local AI uses ``infocyph/llm-ollama:amd-latest``. -- Elasticsearch, Kibana, and Filebeat share ``ELASTICSEARCH_VERSION`` and default to ``latest``. +- Elasticsearch, Kibana, and Filebeat share ``ELASTICSEARCH_VERSION`` and default to the current stable ``9.5.4`` because this Elastic image set does not expose a usable moving ``latest`` alias. Run:: From 7e574c8bfcca66129a187e536e54e39c89ba9260 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:10:19 +0600 Subject: [PATCH 379/509] docs(plan): restore verified Elastic current-release exception --- .../07-localdevstack-integration-plan.md | 21 +++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 34f1492d..588f48f0 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1945,12 +1945,21 @@ LDS_LLM_ARCH=latest Other runtime defaults follow the same rule. PostgreSQL uses `postgres:alpine`; MySQL, MariaDB, MongoDB, Redis Stack/Redis Insight, CloudBeaver, Mongo Express and Mailpit use their normal moving latest tags because the selected image family does not provide a suitable moving Alpine alias for this stack. -## Elastic moving-tag policy +## Elastic current-release exception -Elasticsearch, Kibana and Filebeat use the same `ELASTICSEARCH_VERSION` selector and -default to `latest`. This follows the product-wide moving-tag policy instead of pinning -LocalDevStack to an older tested release. The three services must continue to use the -same selector so a user override advances or pins them together. +The Elastic image set used by LocalDevStack does not expose a usable moving `latest` +alias. Elasticsearch, Kibana and Filebeat therefore share one +`ELASTICSEARCH_VERSION` selector and default to the newest stable release verified by +the compatibility gate. + +Current default: + +```text +9.5.4 +``` + +Advance this single selector when Elastic publishes a newer stable release; all three +services and their contract tests must move together. ## Override precedence @@ -2064,7 +2073,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - PostgreSQL defaults to `postgres:alpine`. - Tools, Runner, Nginx and Apache consume their published `:latest` aliases. - One LLM service uses `infocyph/llm-ollama:${LDS_LLM_ARCH}`; CPU/NVIDIA map to `latest`, AMD/ROCm maps to `amd-latest`. -- Elasticsearch, Kibana and Filebeat share the `ELASTICSEARCH_VERSION` selector and default to the moving `latest` tag. +- Elasticsearch, Kibana and Filebeat share the `ELASTICSEARCH_VERSION` selector and currently default to stable `9.5.4`, because the required Elastic image set has no usable moving `latest` alias. - PHP/Node runtime selection remains user-driven and version-specific. - Existing named volumes and container names remain intentionally stable for this release. From 33664313950eff53bcb92ada64fc978129b9ad58 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:11:50 +0600 Subject: [PATCH 380/509] fix(images): use official Elastic registry for Elasticsearch --- docker/compose/db.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/db.yaml b/docker/compose/db.yaml index da1a99cf..48a7cbba 100644 --- a/docker/compose/db.yaml +++ b/docker/compose/db.yaml @@ -113,7 +113,7 @@ services: <<: *db-service container_name: ELASTICSEARCH hostname: elasticsearch - image: elasticsearch:${ELASTICSEARCH_VERSION:-9.5.4} + image: docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION:-9.5.4} profiles: [elasticsearch] environment: - TZ=${TZ:-} From 9d0ab7c83ad0307a65c369fd30f47863f81fe4ca Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:11:54 +0600 Subject: [PATCH 381/509] fix(images): use official Elastic registry for Kibana --- docker/compose/db-client.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/db-client.yaml b/docker/compose/db-client.yaml index 140f286b..25662fc7 100644 --- a/docker/compose/db-client.yaml +++ b/docker/compose/db-client.yaml @@ -59,7 +59,7 @@ services: <<: *db-client-service container_name: KIBANA hostname: kibana - image: kibana:${ELASTICSEARCH_VERSION:-9.5.4} + image: docker.elastic.co/kibana/kibana:${ELASTICSEARCH_VERSION:-9.5.4} profiles: [elasticsearch] depends_on: elasticsearch: From e241750e6f0dd4fbd6e09dbd26c1b8c1316b664d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:11:57 +0600 Subject: [PATCH 382/509] test(images): validate official Elastic registry references --- tests/compose-contract.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index a9d42712..7ab1d599 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -108,8 +108,8 @@ elastic_json="$("${compose[@]}" --profile elasticsearch --profile filebeat confi python3 -c ' import json,sys d=json.load(sys.stdin) -assert d["services"]["elasticsearch"]["image"] == "elasticsearch:9.5.4" -assert d["services"]["kibana"]["image"] == "kibana:9.5.4" +assert d["services"]["elasticsearch"]["image"] == "docker.elastic.co/elasticsearch/elasticsearch:9.5.4" +assert d["services"]["kibana"]["image"] == "docker.elastic.co/kibana/kibana:9.5.4" assert d["services"]["filebeat"]["image"] == "docker.elastic.co/beats/filebeat:9.5.4" ' <<<"$elastic_json" pass "Elastic stack uses aligned current-stable tags because latest is unsupported" From 6785d5006bfbfe4a56418c0dda845785940956a3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:12:01 +0600 Subject: [PATCH 383/509] test(images): report official Elasticsearch reference --- tests/qol-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index 9f1b7723..a03094ac 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -43,7 +43,7 @@ assert_contains "$images" "infocyph/runner:latest" assert_contains "$images" "infocyph/nginx:latest" assert_contains "$images" "infocyph/apache:latest" assert_contains "$images" "postgres:alpine" -assert_contains "$images" "elasticsearch:9.5.4" +assert_contains "$images" "docker.elastic.co/elasticsearch/elasticsearch:9.5.4" assert_contains "$images" "localdevstack-php: (Alpine)" pass "images is offline-safe and reports fixed plus configurable image selections" From 18a9ee856a2d7159747a0180cb27cb9aeef936b5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:12:05 +0600 Subject: [PATCH 384/509] test(images): resolve official Elastic registry images --- tests/published-images.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index 4b940309..dbf1931b 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -39,8 +39,8 @@ moving_images=( "dbeaver/cloudbeaver:latest" "mongo-express:latest" "axllent/mailpit:latest" - "elasticsearch:9.5.4" - "kibana:9.5.4" + "docker.elastic.co/elasticsearch/elasticsearch:9.5.4" + "docker.elastic.co/kibana/kibana:9.5.4" "docker.elastic.co/beats/filebeat:9.5.4" "infocyph/llm-ollama:latest" "infocyph/llm-ollama:amd-latest" From 06dbf2466471ec30000cd6ee95cfc36bcc4aca30 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:12:08 +0600 Subject: [PATCH 385/509] fix(images): report official Elastic image references --- lib/diagnostics.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 60d357d7..128c68e2 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -437,8 +437,8 @@ cmd_images() { printf '%-16s mariadb:%s\n' "MariaDB" "$(compose_control_value MARIADB_VERSION latest)" printf '%-16s mongo:%s\n' "MongoDB" "$(compose_control_value MONGODB_VERSION latest)" printf '%-16s redis/redis-stack-server:%s\n' "Redis" "$(compose_control_value REDIS_VERSION latest)" - printf '%-16s elasticsearch:%s\n' "Elasticsearch" "$elastic" - printf '%-16s kibana:%s\n' "Kibana" "$elastic" + printf '%-16s docker.elastic.co/elasticsearch/elasticsearch:%s\n' "Elasticsearch" "$elastic" + printf '%-16s docker.elastic.co/kibana/kibana:%s\n' "Kibana" "$elastic" printf '%-16s docker.elastic.co/beats/filebeat:%s\n' "Filebeat" "$elastic" printf '%-16s %s\n' "PHP runtimes" "localdevstack-php: (Alpine)" printf '%-16s %s\n' "Node runtimes" "localdevstack-node: (Alpine)" From c9d7c92b050cac2e58e4b6b2b658c624fe67da3f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 08:13:33 +0600 Subject: [PATCH 386/509] test(images): use buildx registry inspection for current references --- tests/published-images.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index dbf1931b..a65c935f 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -48,7 +48,7 @@ moving_images=( for image in "${moving_images[@]}"; do printf 'Checking configured image reference %s\n' "$image" - docker manifest inspect "$image" >/dev/null + docker buildx imagetools inspect "$image" >/dev/null done pass "all configured current image references resolve" From ca996e60c30149126de1e987dd20e5f195cf7ccf Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 10:25:50 +0600 Subject: [PATCH 387/509] feat(ai): prefer supported XDNA2 NPU runtime --- lib/platform.sh | 76 ++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 72 insertions(+), 4 deletions(-) diff --git a/lib/platform.sh b/lib/platform.sh index dce320a9..7b96d0d7 100644 --- a/lib/platform.sh +++ b/lib/platform.sh @@ -1,9 +1,35 @@ # shellcheck shell=bash -# Detect the preferred local-AI runtime from host GPU capability. -# NVIDIA is preferred on hybrid hosts because the standard image supports -# CPU/NVIDIA, while the AMD image is specifically the ROCm variant. +# Detect a FastFlow-supported AMD XDNA2 NPU. +# FastFlowLM currently identifies XDNA2 as AMD PCI vendor/device 1022:17f0. +# The paths are injectable so the contract can be exercised without NPU hardware. +fastflow_npu_supported() { + local accel="${LDS_AI_ACCEL_DEVICE:-/dev/accel/accel0}" + local sysfs="${LDS_AI_ACCEL_SYSFS:-/sys/class/accel/accel0/device}" + local vendor="" device="" driver="" + + [[ -e "$accel" ]] || return 1 + [[ -r "$sysfs/vendor" && -r "$sysfs/device" ]] || return 1 + + vendor="$(tr '[:upper:]' '[:lower:]' <"$sysfs/vendor" | tr -d '[:space:]')" + device="$(tr '[:upper:]' '[:lower:]' <"$sysfs/device" | tr -d '[:space:]')" + if [[ -L "$sysfs/driver" ]]; then + driver="$(basename "$(readlink -f "$sysfs/driver" 2>/dev/null || true)")" + fi + + [[ "$vendor" == "0x1022" && "$device" == "0x17f0" ]] || return 1 + [[ -z "$driver" || "$driver" == "amdxdna" ]] || return 1 +} + +# Detect the preferred local-AI runtime from host accelerator capability. +# A supported XDNA2 NPU wins because FastFlowLM can use it directly. NVIDIA is +# next, then ROCm-capable AMD GPU, with CPU as the portable fallback. detect_ai_runtime() { + if fastflow_npu_supported; then + printf '%s' npu + return 0 + fi + if has_cmd nvidia-smi && nvidia-smi -L >/dev/null 2>&1; then printf '%s' nvidia return 0 @@ -23,6 +49,48 @@ detect_ai_runtime() { printf '%s' cpu } +ai_provider_for_runtime() { + case "${1,,}" in + npu) printf '%s' fastflow ;; + "" | cpu | nvidia | amd) printf '%s' ollama ;; + *) return 1 ;; + esac +} + +ai_service_for_runtime() { + case "$(ai_provider_for_runtime "${1:-}")" in + fastflow) printf '%s' llm-fastflow ;; + ollama) printf '%s' llm-ollama ;; + *) return 1 ;; + esac +} + +ai_model_default_for_runtime() { + case "${1,,}" in + npu) printf '%s' 'qwen3.5:9b' ;; + "" | cpu | nvidia | amd) printf '%s' 'qwen3:14b' ;; + *) return 1 ;; + esac +} + +effective_ai_runtime() { + local runtime + runtime="$(compose_control_value LDS_AI_RUNTIME "")" + [[ -n "$runtime" ]] || runtime="$(detect_ai_runtime)" + printf '%s' "${runtime,,}" +} + +effective_ai_model() { + local runtime="${1:-}" configured + [[ -n "$runtime" ]] || runtime="$(effective_ai_runtime)" + configured="$(compose_control_value LDS_AI_MODEL "")" + if [[ -n "$configured" ]]; then + printf '%s' "$configured" + else + ai_model_default_for_runtime "$runtime" + fi +} + host_cpu_is_amd() { if [[ -r /proc/cpuinfo ]] && grep -qiE '^[[:space:]]*vendor_id[[:space:]]*:[[:space:]]*AuthenticAMD([[:space:]]|$)' /proc/cpuinfo; then @@ -48,7 +116,7 @@ ai_igpu_default_for_runtime() { llm_arch_for_runtime() { case "${1,,}" in amd) printf '%s' amd-latest ;; - "" | cpu | nvidia) printf '%s' latest ;; + "" | cpu | nvidia | npu) printf '%s' latest ;; *) return 1 ;; esac } From f3681c7de410234f91e619210af6a88e9f5a2774 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 10:26:15 +0600 Subject: [PATCH 388/509] feat(ai): add mutually selected Ollama and FastFlow services --- docker/compose/companion.yaml | 47 +++++++++++++++++++++++++++++++---- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index bf6c4093..69c6c4d8 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -18,8 +18,8 @@ services: - SOPS_REPO_DIR=/etc/share/vhosts/sops - COMPOSE_PROFILES=${COMPOSE_PROFILES:-} - LDS_AI_ENABLED=${LDS_AI_ENABLED:-auto} - - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-ollama} - - LDS_AI_URL=${LDS_AI_URL:-http://llm-ollama:11434} + - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-llm} + - LDS_AI_URL=${LDS_AI_URL:-http://llm:11434} - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen3:14b} - LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2} - LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5} @@ -59,7 +59,7 @@ services: hostname: llm-ollama image: infocyph/llm-ollama:${LDS_LLM_ARCH} restart: unless-stopped - profiles: [ai] + profiles: ["${LDS_AI_OLLAMA_PROFILE:-__lds-ai-disabled-ollama}"] environment: - LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b} - LLM_OLLAMA_SYSTEM=${LLM_OLLAMA_SYSTEM:-} @@ -79,8 +79,45 @@ services: volumes: - lds_llm:/root/.ollama networks: - - frontend - - backend + frontend: + aliases: [llm] + backend: + aliases: [llm] + + llm-fastflow: + container_name: LLM_FASTFLOW + hostname: llm-fastflow + image: infocyph/llm-fastflow:latest + restart: unless-stopped + profiles: ["${LDS_AI_FASTFLOW_PROFILE:-__lds-ai-disabled-fastflow}"] + environment: + - LLM_FASTFLOW_MODEL=${LDS_AI_MODEL:-qwen3.5:9b} + - FLM_MODEL_PATH=/models + - FLM_SERVE_PORT=11434 + - FLM_HOST=0.0.0.0 + - FLM_CORS=0 + - FLM_DISABLE_UPDATE_CHECK=1 + - LLM_FASTFLOW_INPUT_WARN_BYTES=${LLM_FASTFLOW_INPUT_WARN_BYTES:-1048576} + - LLM_FASTFLOW_INPUT_MAX_BYTES=${LLM_FASTFLOW_INPUT_MAX_BYTES:-0} + - LLM_FASTFLOW_ATTACHMENT_MAX_BYTES=${LLM_FASTFLOW_ATTACHMENT_MAX_BYTES:-16777216} + - LLM_FASTFLOW_ATTACHMENTS_MAX_BYTES=${LLM_FASTFLOW_ATTACHMENTS_MAX_BYTES:-33554432} + - LLM_FASTFLOW_ATTACHMENT_MAX_COUNT=${LLM_FASTFLOW_ATTACHMENT_MAX_COUNT:-16} + - LLM_FASTFLOW_PDF_MAX_PAGES=${LLM_FASTFLOW_PDF_MAX_PAGES:-24} + - LLM_FASTFLOW_PDF_DPI=${LLM_FASTFLOW_PDF_DPI:-120} + - LLM_FASTFLOW_ALLOW_LARGE_INPUT=${LLM_FASTFLOW_ALLOW_LARGE_INPUT:-0} + devices: + - "${LDS_AI_ACCEL_DEVICE:-/dev/accel/accel0}:/dev/accel/accel0" + ulimits: + memlock: + soft: -1 + hard: -1 + volumes: + - lds_llm_fastflow:/models + networks: + frontend: + aliases: [llm] + backend: + aliases: [llm] runner: container_name: RUNNER From e96e5de2ff41e99c67f9dd134055ce735113f25c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 10:26:20 +0600 Subject: [PATCH 389/509] feat(ai): add persistent FastFlow model volume --- docker/compose/main.yaml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/docker/compose/main.yaml b/docker/compose/main.yaml index 7cfac84e..9a70e178 100644 --- a/docker/compose/main.yaml +++ b/docker/compose/main.yaml @@ -144,7 +144,13 @@ volumes: labels: com.infocyph.lds: "1" com.infocyph.stack: "LocalDevStack" - com.infocyph.purpose: "LLM Model Store" + com.infocyph.purpose: "Ollama Model Store" + lds_llm_fastflow: + name: LLMFastFlowModels + labels: + com.infocyph.lds: "1" + com.infocyph.stack: "LocalDevStack" + com.infocyph.purpose: "FastFlow Model Store" lds_tools_state: name: ToolsState labels: From d28d4c51b19cbb042b70272440aed0878ed55bcd Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 10:26:59 +0600 Subject: [PATCH 390/509] feat(ai): select one provider per runtime in Compose --- lib/compose.sh | 33 +++++++++++++++++++++++++++------ 1 file changed, 27 insertions(+), 6 deletions(-) diff --git a/lib/compose.sh b/lib/compose.sh index efcfeb25..155e2d19 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -42,21 +42,35 @@ docker_compose() { local -a env_files=(--env-file "$ENV_RELEASE") [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - local ai_runtime llm_arch runtime_override="" + local ai_runtime ai_provider ai_model llm_arch ollama_profile fastflow_profile runtime_override="" local -a runtime_f=() ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" case "${ai_runtime,,}" in - cpu | nvidia | amd) ;; - *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd)" ;; + cpu | nvidia | amd | npu) ;; + *) die "Invalid LDS_AI_RUNTIME: $ai_runtime (expected cpu|nvidia|amd|npu)" ;; esac ai_runtime="${ai_runtime,,}" + ai_provider="$(ai_provider_for_runtime "$ai_runtime")" || + die "Cannot resolve AI provider for runtime: $ai_runtime" + ai_model="$(effective_ai_model "$ai_runtime")" || + die "Cannot resolve AI model for runtime: $ai_runtime" llm_arch="$(llm_arch_for_runtime "$ai_runtime")" || die "Cannot resolve LLM image tag for runtime: $ai_runtime" - if [[ "$ai_runtime" != "cpu" ]]; then + if [[ "$ai_provider" == "fastflow" ]]; then + ollama_profile=__lds-ai-disabled-ollama + fastflow_profile=ai + else + ollama_profile=ai + fastflow_profile=__lds-ai-disabled-fastflow + fi + + # Only Ollama GPU modes need generated hardware overrides. FastFlow's XDNA2 + # device + memlock contract is part of its tracked service definition. + if [[ "$ai_runtime" == "nvidia" || "$ai_runtime" == "amd" ]]; then mkdir -p "$CFG/.runtime" runtime_override="$(mktemp "$CFG/.runtime/ai.XXXXXX")" || die "Unable to create temporary AI Compose override" @@ -68,7 +82,7 @@ docker_compose() { printf '%s\n' ' gpus: all' ;; amd) - printf '%s\n' ' devices:' ' - /dev/kfd:/dev/kfd' ' - /dev/dri:/dev/dri' + printf '%s\n' ' devices:' ' - /dev/kfd:/dev/kfd' ' - /dev/dri:/dev/dri' ;; esac } >"$runtime_override" @@ -88,7 +102,14 @@ docker_compose() { local host_os="${HOST_OS:-$(detect_host_os)}" local rc=0 - HOST_OS="$host_os" LDS_LLM_ARCH="$llm_arch" "${__LDS_DC_BIN[@]}" \ + HOST_OS="$host_os" \ + LDS_LLM_ARCH="$llm_arch" \ + LDS_AI_PROVIDER=llm \ + LDS_AI_URL=http://llm:11434 \ + LDS_AI_MODEL="$ai_model" \ + LDS_AI_OLLAMA_PROFILE="$ollama_profile" \ + LDS_AI_FASTFLOW_PROFILE="$fastflow_profile" \ + "${__LDS_DC_BIN[@]}" \ --project-directory "$DIR" \ -f "$COMPOSE_FILE" \ "${runtime_f[@]}" \ From a384f4fa311df45c87bb2d7b141a3ee33662df04 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 10:27:05 +0600 Subject: [PATCH 391/509] compat(ai): keep Ollama as bare-compose profile fallback --- docker/compose/companion.yaml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 69c6c4d8..19d858b4 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -59,7 +59,7 @@ services: hostname: llm-ollama image: infocyph/llm-ollama:${LDS_LLM_ARCH} restart: unless-stopped - profiles: ["${LDS_AI_OLLAMA_PROFILE:-__lds-ai-disabled-ollama}"] + profiles: ["${LDS_AI_OLLAMA_PROFILE:-ai}"] environment: - LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b} - LLM_OLLAMA_SYSTEM=${LLM_OLLAMA_SYSTEM:-} From 8e914e346b3214eba75a1284f5b306e3e734a6c5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 10:27:38 +0600 Subject: [PATCH 392/509] feat(ai): dispatch LLM CLI and Graphify through active provider --- lib/ai.sh | 124 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 99 insertions(+), 25 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 1240aaba..788b1a32 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -29,14 +29,35 @@ cmd_ai() { esac } +_active_llm_runtime() { + effective_ai_runtime +} + +_active_llm_service() { + ai_service_for_runtime "$(_active_llm_runtime)" +} + +_active_llm_provider() { + ai_provider_for_runtime "$(_active_llm_runtime)" +} + +_active_llm_cli() { + case "$(_active_llm_provider)" in + fastflow) printf '%s' llm-fastflow ;; + ollama) printf '%s' llm-ollama ;; + *) return 1 ;; + esac +} + _graphify_local_base_url() { - local provider_ctr nginx_ctr + local service provider_ctr nginx_ctr + service="$(_active_llm_service)" - provider_ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" + provider_ctr="$(docker_compose ps -q "$service" 2>/dev/null | sed -n '1p' || true)" [[ -n "$provider_ctr" ]] || - die "llm-ollama is not running. Enable the ai profile and start the stack first." + die "$service is not running. Enable the ai profile and start the stack first." docker inspect -f '{{.State.Running}}' "$provider_ctr" 2>/dev/null | grep -qx true || - die "llm-ollama container exists but is not running." + die "$service container exists but is not running." nginx_ctr="$(docker_compose ps -q nginx 2>/dev/null | sed -n '1p' || true)" [[ -n "$nginx_ctr" ]] || @@ -44,16 +65,19 @@ _graphify_local_base_url() { docker inspect -f '{{.State.Running}}' "$nginx_ctr" 2>/dev/null | grep -qx true || die "nginx container exists but is not running." - printf '%s' 'http://llm-ollama.localhost:11434/v1' + printf '%s' 'http://llm.localhost:11434/v1' } _graphify_local_model_preflight() { - local model="${1:-}" + local model="${1:-}" response [[ -n "$model" ]] || return 1 + need_bin curl "install curl to validate the selected LocalDevStack model" - if ! docker_compose exec -T llm-ollama /bin/ollama show "$model" >/dev/null 2>&1; then - die "Ollama model '$model' is not available in LocalDevStack. Run: lds llm pull $model" - fi + response="$(curl --connect-timeout 3 --max-time 10 -fsS 'http://llm.localhost:11434/v1/models' 2>/dev/null)" || + die "The selected LocalDevStack LLM endpoint is unavailable. Start the ai profile first." + + printf '%s' "$response" | grep -Fq ""$model"" || + die "Model '$model' is not available from the active LocalDevStack provider. Run: lds llm pull $model" } cmd_graphify() { @@ -71,7 +95,7 @@ cmd_graphify() { local_provider=1 fi timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" - model="${OLLAMA_MODEL:-$(compose_control_value LDS_AI_MODEL qwen3:14b)}" + model="${OLLAMA_MODEL:-$(effective_ai_model "$(_active_llm_runtime)")}" api_key="${OLLAMA_API_KEY:-local}" # Keep explicit model/timeout overrides consistent across extraction and clustering. @@ -121,15 +145,25 @@ cmd_graphify() { } _llm_exec() { - local ctr - ctr="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" - [[ -n "$ctr" ]] || die "llm-ollama is not running. Enable the ai profile and start the stack first." + local service cli ctr + service="$(_active_llm_service)" + cli="$(_active_llm_cli)" + + ctr="$(docker_compose ps -q "$service" 2>/dev/null | sed -n '1p' || true)" + [[ -n "$ctr" ]] || die "$service is not running. Enable the ai profile and start the stack first." docker inspect -f '{{.State.Running}}' "$ctr" 2>/dev/null | grep -qx true || - die "llm-ollama container exists but is not running." + die "$service container exists but is not running." local -a exec_args=(exec) [[ -t 0 && -t 1 ]] || exec_args+=(-T) - docker_compose "${exec_args[@]}" llm-ollama llm-ollama "$@" + docker_compose "${exec_args[@]}" "$service" "$cli" "$@" +} + +_llm_require_provider() { + local expected="$1" command="$2" active + active="$(_active_llm_provider)" + [[ "$active" == "$expected" ]] || + die "llm $command is available only with the $expected provider; active provider is $active." } cmd_llm() { @@ -140,31 +174,71 @@ cmd_llm() { runtime) local mode="${1:-}" if [[ -z "$mode" ]]; then - printf '%s\n' "$(compose_control_value LDS_AI_RUNTIME "$(detect_ai_runtime)")" + printf '%s\n' "$(_active_llm_runtime)" return 0 fi + case "${mode,,}" in - cpu | nvidia | amd) - local normalized arch igpu_enable + auto) + update_env "$ENV_DOCKER" LDS_AI_RUNTIME "" + update_env "$ENV_DOCKER" LDS_LLM_ARCH "" + update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "" + local detected + detected="$(detect_ai_runtime)" + ok "LLM runtime set to auto; detected $detected ($(ai_provider_for_runtime "$detected")). Recreate the AI service to apply it." + ;; + cpu | nvidia | amd | npu) + local normalized arch igpu_enable provider image normalized="${mode,,}" arch="$(llm_arch_for_runtime "$normalized")" igpu_enable="$(ai_igpu_default_for_runtime "$normalized")" + provider="$(ai_provider_for_runtime "$normalized")" + + if [[ "$normalized" == "npu" ]] && ! fastflow_npu_supported; then + warn "No FastFlow-supported XDNA2 NPU is currently detected; the explicit npu runtime will still be persisted." + fi + update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" update_env "$ENV_DOCKER" LDS_LLM_ARCH "$arch" update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable" - ok "LLM runtime set to $normalized (infocyph/llm-ollama:$arch, iGPU=$igpu_enable). Recreate llm-ollama to apply the change." + + if [[ "$provider" == "fastflow" ]]; then + image="infocyph/llm-fastflow:latest" + else + image="infocyph/llm-ollama:$arch" + fi + ok "LLM runtime set to $normalized ($provider, $image). Recreate the AI service to apply the change." ;; - *) die "llm runtime " ;; + *) die "llm runtime " ;; esac ;; - models | ps | show | pull | rm | unload | run | ask | chat | prompt | code | review | json | ai-commit | ollama | api | version) + + models | list | pull | rm | remove | run | ask | chat | prompt | code | review | json | ai-commit | api | version) + _llm_exec "${sub,,}" "$@" + ;; + + ps | show | unload | ollama) + _llm_require_provider ollama "${sub,,}" + _llm_exec "${sub,,}" "$@" + ;; + + validate | check | flm) + _llm_require_provider fastflow "${sub,,}" _llm_exec "${sub,,}" "$@" ;; + + provider) + printf '%s\n' "$(_active_llm_provider)" + ;; + help | -h | --help) - printf '%s\n' "llm " - printf '%s\n' "llm runtime " + printf '%s\n' "llm " + printf '%s\n' "llm provider" + printf '%s\n' "llm runtime " + printf '%s\n' "Ollama-only: llm " + printf '%s\n' "FastFlow-only: llm " ;; - *) die "llm " ;; + + *) die "llm " ;; esac } - From d6c7fa8175714757be9f22daf65eb055eb4706c1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:46:36 +0600 Subject: [PATCH 393/509] fix(ai): make service catalog provider-neutral --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index b2f8e944..d7d76edf 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.4|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm-ollama|LDS_LLM_ARCH|LDS_AI_MODEL=qwen3:14b|Preferred model||LLMModels|https://llm-ollama.localhost|ai|1|0|cpu,nvidia,amd +AI|ai|Local AI|llm|LDS_AI_RUNTIME|LDS_AI_MODEL=|Model override (blank = provider default)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd,npu From c005e44d7ca065f568e5fc523842d3b9730bc6e0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:46:40 +0600 Subject: [PATCH 394/509] fix(ai): report and validate provider-aware setup --- lib/profiles.sh | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-) diff --git a/lib/profiles.sh b/lib/profiles.sh index 7b6b87f2..8211f859 100644 --- a/lib/profiles.sh +++ b/lib/profiles.sh @@ -206,10 +206,18 @@ setup_service() { queue_profile "$profile" if [[ "$service" == "AI" ]]; then - local detected_runtime detected_arch + local detected_runtime detected_arch detected_provider detected_model detected_image detected_runtime="$(compose_control_value LDS_AI_RUNTIME "$(detect_ai_runtime)")" - detected_arch="$(llm_arch_for_runtime "$detected_runtime")" - printf "%bDetected local-AI runtime:%b %s (%s)\n" "$CYAN" "$NC" "$detected_runtime" "infocyph/llm-ollama:$detected_arch" + detected_provider="$(ai_provider_for_runtime "$detected_runtime")" + detected_model="$(effective_ai_model "$detected_runtime")" + if [[ "$detected_provider" == "fastflow" ]]; then + detected_image="infocyph/llm-fastflow:latest" + else + detected_arch="$(llm_arch_for_runtime "$detected_runtime")" + detected_image="infocyph/llm-ollama:$detected_arch" + fi + printf "%bDetected local-AI runtime:%b %s (%s, %s, model %s)\n" \ + "$CYAN" "$NC" "$detected_runtime" "$detected_provider" "$detected_image" "$detected_model" fi local defaults="${PROFILE_ENV[$profile]:-}" @@ -250,9 +258,10 @@ setup_service() { LDS_AI_RUNTIME) val="${val,,}" case "$val" in - cpu | nvidia | amd) ;; - *) die "AI runtime must be cpu, nvidia, or amd" ;; + "" | auto | cpu | nvidia | amd | npu) ;; + *) die "AI runtime must be auto, cpu, nvidia, amd, or npu" ;; esac + [[ "$val" == "auto" ]] && val="" ;; esac From f849b0212e9ccfa70ca6ae2abd19c280fe972f75 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:46:44 +0600 Subject: [PATCH 395/509] fix(ai): route user-facing open command through common llm --- lib/services.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/lib/services.sh b/lib/services.sh index 82a2d839..640bc8a7 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -367,7 +367,9 @@ cmd_open() { redis | redisinsight | redis-insight | rds) url="https://ri.localhost" ;; mongo | me | mongoexpress | mongo-express) url="https://me.localhost" ;; kibana | kbn) url="https://kibana.localhost" ;; - ai | llm | llm-ollama | ollama) url="https://llm-ollama.localhost" ;; + ai | llm) url="https://llm.localhost" ;; + llm-ollama | ollama) url="https://llm-ollama.localhost" ;; + llm-fastflow | fastflow) url="https://llm-fastflow.localhost" ;; *) url="https://${target}" ;; From a2ac71f4f4bef6685988616685b1a6ec406961fb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:47:02 +0600 Subject: [PATCH 396/509] fix(ai): make diagnostics provider-aware --- lib/diagnostics.sh | 23 ++++++++++++++++------- 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 128c68e2..69afeba8 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -426,11 +426,18 @@ cmd_images() { printf '%-16s %s\n' "Runner" "infocyph/runner:latest" printf '%-16s %s\n' "Nginx" "infocyph/nginx:latest" printf '%-16s %s\n' "Apache" "infocyph/apache:latest" - local ai_runtime llm_arch + local ai_runtime ai_provider llm_arch llm_image ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" - llm_arch="$(llm_arch_for_runtime "$ai_runtime")" - printf '%-16s %s\n' "LLM" "infocyph/llm-ollama:$llm_arch" + ai_provider="$(ai_provider_for_runtime "$ai_runtime")" + if [[ "$ai_provider" == "fastflow" ]]; then + llm_image="infocyph/llm-fastflow:latest" + else + llm_arch="$(llm_arch_for_runtime "$ai_runtime")" + llm_image="infocyph/llm-ollama:$llm_arch" + fi + printf '%-16s %s\n' "LLM" "$llm_image" + printf '%-16s %s\n' "LLM provider" "$ai_provider" printf '%-16s %s\n' "LLM runtime" "$ai_runtime" printf '%-16s postgres:%s\n' "PostgreSQL" "$(compose_control_value POSTGRES_VERSION alpine)" printf '%-16s mysql:%s\n' "MySQL" "$(compose_control_value MYSQL_VERSION latest)" @@ -518,12 +525,14 @@ cmd_doctor() { ) if profile_enabled ai; then - local llm - llm="$(docker_compose ps -q llm-ollama 2>/dev/null | sed -n '1p' || true)" + local llm llm_service llm_provider + llm_service="$(ai_service_for_runtime "$(effective_ai_runtime)")" + llm_provider="$(ai_provider_for_runtime "$(effective_ai_runtime)")" + llm="$(docker_compose ps -q "$llm_service" 2>/dev/null | sed -n '1p' || true)" if [[ -n "$llm" ]] && docker inspect -f '{{.State.Running}}' "$llm" 2>/dev/null | grep -qx true; then - _doctor_ok "AI provider container is running." + _doctor_ok "AI provider is running: $llm_provider ($llm_service)." else - _doctor_warn "AI profile is selected but llm-ollama is not running." + _doctor_warn "AI profile is selected but $llm_provider provider service '$llm_service' is not running." warnings=$((warnings + 1)) fi fi From 528cae871fa21493399ac6df9fd2950ffd7a89d2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:47:49 +0600 Subject: [PATCH 397/509] test(ai): make catalog contract provider-neutral --- tests/catalog-contract.sh | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 734365a3..3c4b7042 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -49,11 +49,11 @@ while IFS='|' read -r key profile display service_key version_env defaults promp ;; AI) [[ "$profile" == "ai" ]] || fail "AI profile must be ai" - [[ "$service_key" == "llm-ollama" ]] || fail "AI service key must be llm-ollama" - [[ "$runtime_modes" == "cpu,nvidia,amd" ]] || fail "AI runtime metadata drift" - [[ "$defaults" == *"LDS_AI_MODEL=qwen3:14b"* ]] || fail "AI model default drift" - [[ "$version_env" == "LDS_LLM_ARCH" ]] || fail "AI image selector must be LDS_LLM_ARCH" - [[ "$defaults" != *"LDS_AI_RUNTIME="* ]] || fail "AI profile wizard must not prompt for runtime" + [[ "$service_key" == "llm" ]] || fail "AI service identity must be provider-neutral llm" + [[ "$runtime_modes" == "cpu,nvidia,amd,npu" ]] || fail "AI runtime metadata drift" + [[ "$defaults" == "LDS_AI_MODEL=" ]] || fail "AI setup must leave provider model default automatic" + [[ "$version_env" == "LDS_AI_RUNTIME" ]] || fail "AI selector metadata must be LDS_AI_RUNTIME" + [[ "$url" == "https://llm.localhost" ]] || fail "AI convenience URL must use common llm identity" ;; ELASTICSEARCH) [[ "$defaults" == *"ELASTICSEARCH_VERSION=9.5.4"* ]] || fail "Elastic current default drift" From e50914028ed72b516e31b4d462c6715062fa34a1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:47:53 +0600 Subject: [PATCH 398/509] test(ai): expect common Graphify llm route --- tests/cli-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index 3652741e..72fe785f 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -61,7 +61,7 @@ grep -Fq 'base=http://custom-ollama.test:11434/v1 key=local model=test-model tim grep -Fq 'base=http://custom-ollama.test:11434/v1 key=local model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || fail "Graphify cluster wrapper contract failed" rm -f "$graphify_log" -assert_file_contains "$ROOT/lib/ai.sh" "http://llm-ollama.localhost:11434/v1" +assert_file_contains "$ROOT/lib/ai.sh" "http://llm.localhost:11434/v1" pass "Graphify host workflow wrapper" assert_file_contains "$ROOT/lds" 'exec "$DIR/bin/tool-runner" "$cmd" "$@"' From 602e2c6eedbecb18e19f124ecf83fde6b1b15f6f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:47:58 +0600 Subject: [PATCH 399/509] test(ai): make QoL output follow common llm identity --- tests/qol-contract.sh | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/tests/qol-contract.sh b/tests/qol-contract.sh index a03094ac..13c183da 100644 --- a/tests/qol-contract.sh +++ b/tests/qol-contract.sh @@ -25,6 +25,7 @@ trap cleanup EXIT cat >"$user_env" < (Alpine)" pass "images is offline-safe and reports fixed plus configurable image selections" @@ -52,7 +56,7 @@ assert_contains "$urls" "https://admin.localhost" assert_contains "$urls" "https://webmail.localhost" assert_contains "$urls" "https://db.localhost" assert_contains "$urls" "https://ri.localhost" -assert_contains "$urls" "https://llm-ollama.localhost" +assert_contains "$urls" "https://llm.localhost" if grep -Fq "https://kibana.localhost" <<<"$urls"; then fail "urls must not show disabled Elasticsearch profile URL" fi From 1c3f830060ccdada0ce66191c5ce81934630cb24 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:48:02 +0600 Subject: [PATCH 400/509] test(images): include published FastFlow provider --- tests/published-images.sh | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index a65c935f..ab64f925 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -44,6 +44,7 @@ moving_images=( "docker.elastic.co/beats/filebeat:9.5.4" "infocyph/llm-ollama:latest" "infocyph/llm-ollama:amd-latest" + "infocyph/llm-fastflow:latest" ) for image in "${moving_images[@]}"; do @@ -59,10 +60,14 @@ done pass "Tools and Runner publish healthchecks" grep -Fq 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' "$ROOT/docker/compose/companion.yaml" || - fail "LLM service must use the single LDS_LLM_ARCH selector" + fail "Ollama service must use the latest/amd-latest selector" +grep -Fq 'image: infocyph/llm-fastflow:latest' "$ROOT/docker/compose/companion.yaml" || + fail "FastFlow service must use its published latest image" grep -Fq "amd) printf '%s' amd-latest" "$ROOT/lib/platform.sh" || - fail "AMD runtime must map to amd-latest" -pass "LLM image selection follows the single latest/amd-latest tag contract" + fail "AMD runtime must map to Ollama amd-latest" +grep -Fq "npu) printf '%s' fastflow" "$ROOT/lib/platform.sh" || + fail "NPU runtime must map to FastFlow" +pass "LLM image selection follows mutually exclusive provider contracts" tools_profile_chooser="$( docker run --rm --entrypoint cat "$tools_image" /usr/local/bin/profile-chooser From c428a6ead027c5a6783bd38ce2a6cb5ed7944608 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:48:16 +0600 Subject: [PATCH 401/509] test(ai): lock NPU provider and model mappings --- tests/env-contract.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index 1dcacf90..05a55bab 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -118,6 +118,15 @@ pass "CLI project identity follows the Compose project contract" [[ "$(llm_arch_for_runtime cpu)" == "latest" ]] || fail "CPU LLM tag drift" [[ "$(llm_arch_for_runtime nvidia)" == "latest" ]] || fail "NVIDIA LLM tag drift" [[ "$(llm_arch_for_runtime amd)" == "amd-latest" ]] || fail "AMD LLM tag drift" + [[ "$(llm_arch_for_runtime npu)" == "latest" ]] || fail "NPU compatibility tag drift" + [[ "$(ai_provider_for_runtime cpu)" == "ollama" ]] || fail "CPU provider drift" + [[ "$(ai_provider_for_runtime nvidia)" == "ollama" ]] || fail "NVIDIA provider drift" + [[ "$(ai_provider_for_runtime amd)" == "ollama" ]] || fail "AMD provider drift" + [[ "$(ai_provider_for_runtime npu)" == "fastflow" ]] || fail "NPU provider drift" + [[ "$(ai_service_for_runtime npu)" == "llm-fastflow" ]] || fail "NPU service drift" + [[ "$(ai_service_for_runtime cpu)" == "llm-ollama" ]] || fail "Ollama service drift" + [[ "$(ai_model_default_for_runtime npu)" == "qwen3.5:9b" ]] || fail "FastFlow model default drift" + [[ "$(ai_model_default_for_runtime cpu)" == "qwen3:14b" ]] || fail "Ollama model default drift" host_cpu_is_amd() { return 0; } [[ "$(ai_igpu_default_for_runtime amd)" == "1" ]] || fail "AMD CPU + AMD runtime must enable iGPU" [[ "$(ai_igpu_default_for_runtime cpu)" == "0" ]] || fail "CPU runtime must not enable iGPU" @@ -125,7 +134,7 @@ pass "CLI project identity follows the Compose project contract" host_cpu_is_amd() { return 1; } [[ "$(ai_igpu_default_for_runtime amd)" == "0" ]] || fail "non-AMD CPU must not auto-enable iGPU" ) -pass "LLM runtime maps image tags and AMD CPU iGPU defaults deterministically" +pass "LLM runtime maps mutually exclusive providers, models, tags and AMD iGPU defaults deterministically" ai_env_tmp="$(mktemp -d)" ( From b2f2a0755266c4648759d5ca0093894136d66862 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:49:11 +0600 Subject: [PATCH 402/509] test(ai): validate mutually exclusive provider Compose states --- tests/compose-contract.sh | 107 ++++++++++++++++++++++++++------------ 1 file changed, 73 insertions(+), 34 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index 7ab1d599..d263a8e5 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -139,50 +139,32 @@ ai_json="$("${compose[@]}" --profile ai config --format json)" python3 -c ' import json,sys d=json.load(sys.stdin) -s=d["services"]["llm-ollama"] +services=d["services"] +assert "llm-ollama" in services +assert "llm-fastflow" not in services +s=services["llm-ollama"] assert s["image"] == "infocyph/llm-ollama:latest" assert s["container_name"] == "LLM_OLLAMA" assert not s.get("ports") assert set(s["networks"]) == {"frontend","backend"} -targets={v["target"] for v in s["volumes"]} -assert targets == {"/root/.ollama"} +assert {v["target"] for v in s["volumes"]} == {"/root/.ollama"} assert d["volumes"]["lds_llm"]["name"] == "LLMModels" env=s["environment"] assert env["LLM_OLLAMA_MODEL"] == "qwen3:14b" -assert env["LLM_OLLAMA_SYSTEM"] == "" -assert env["LLM_OLLAMA_INPUT_WARN_BYTES"] == "1048576" -assert env["LLM_OLLAMA_INPUT_MAX_BYTES"] == "0" -assert env["LLM_OLLAMA_ATTACHMENT_MAX_BYTES"] == "16777216" -assert env["LLM_OLLAMA_ATTACHMENTS_MAX_BYTES"] == "33554432" -assert env["LLM_OLLAMA_ATTACHMENT_MAX_COUNT"] == "16" -assert env["LLM_OLLAMA_PDF_MAX_PAGES"] == "24" -assert env["LLM_OLLAMA_PDF_DPI"] == "120" -assert env["LLM_OLLAMA_ALLOW_LARGE_INPUT"] == "0" -assert env["OLLAMA_NUM_PARALLEL"] == "1" -assert env["OLLAMA_MAX_LOADED_MODELS"] == "1" -assert env["OLLAMA_KEEP_ALIVE"] == "5m" assert env["OLLAMA_NO_CLOUD"] == "1" -assert env["OLLAMA_IGPU_ENABLE"] == "0" -tools=d["services"]["server-tools"]["environment"] +tools=services["server-tools"]["environment"] assert tools["LDS_AI_ENABLED"] == "auto" -assert tools["LDS_AI_PROVIDER"] == "ollama" -assert tools["LDS_AI_URL"] == "http://llm-ollama:11434" +assert tools["LDS_AI_PROVIDER"] == "llm" +assert tools["LDS_AI_URL"] == "http://llm:11434" assert tools["LDS_AI_MODEL"] == "qwen3:14b" -assert tools["LDS_AI_CONNECT_TIMEOUT"] == "2" -assert tools["LDS_AI_PREFLIGHT_TIMEOUT"] == "5" -assert tools["LDS_AI_TIMEOUT"] == "1800" -assert tools["LDS_AI_AVAILABILITY_TTL"] == "5" -assert tools["LDS_AI_MAX_CONTEXT_BYTES"] == "524288" -assert tools["LDS_AI_MAX_REQUEST_BYTES"] == "1048576" -assert tools["LDS_AI_MAX_RESPONSE_BYTES"] == "2097152" -nginx=d["services"]["nginx"] +nginx=services["nginx"] assert nginx["environment"]["LLM_PROXY_TIMEOUT_SECONDS"] == "1800" native=[p for p in nginx.get("ports", []) if int(p["target"]) == 11434] assert len(native) == 1 assert native[0]["host_ip"] == "127.0.0.1" assert int(native[0]["published"]) == 11434 ' <<<"$ai_json" -pass "companion-owned AI profile is internal-only and deterministic" +pass "bare Compose AI profile keeps Ollama compatibility fallback behind common llm identity" printf '%s\n' 'LDS_AI_MODEL=qwen2.5:1.5b' 'LLM_OLLAMA_PDF_MAX_PAGES=12' 'LLM_OLLAMA_SYSTEM=Answer briefly.' 'LDS_AI_TIMEOUT=2400' 'LDS_AI_IGPU_ENABLE=1' >>"$user_env" ai_override_json="$("${compose[@]}" --profile ai config --format json)" @@ -195,30 +177,85 @@ assert llm["LLM_OLLAMA_PDF_MAX_PAGES"] == "12" assert llm["LLM_OLLAMA_SYSTEM"] == "Answer briefly." assert llm["OLLAMA_IGPU_ENABLE"] == "1" tools=d["services"]["server-tools"]["environment"] +assert tools["LDS_AI_PROVIDER"] == "llm" +assert tools["LDS_AI_URL"] == "http://llm:11434" assert tools["LDS_AI_TIMEOUT"] == "2400" nginx=d["services"]["nginx"]["environment"] assert nginx["LLM_PROXY_TIMEOUT_SECONDS"] == "2400" ' <<<"$ai_override_json" -pass "LocalDevStack forwards configured provider and generation-timeout options" +pass "LocalDevStack forwards common Tools routing and Ollama-specific generation options" + +grep -v '^LDS_AI_MODEL=' "$user_env" >"$user_env.tmp" +mv "$user_env.tmp" "$user_env" + +npu_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=npu "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +services=d["services"] +assert "llm-fastflow" in services +assert "llm-ollama" not in services +s=services["llm-fastflow"] +assert s["image"] == "infocyph/llm-fastflow:latest" +assert s["container_name"] == "LLM_FASTFLOW" +assert not s.get("ports") +assert set(s["networks"]) == {"frontend","backend"} +assert s["environment"]["LLM_FASTFLOW_MODEL"] == "qwen3.5:9b" +assert s["environment"]["FLM_MODEL_PATH"] == "/models" +assert s["environment"]["FLM_SERVE_PORT"] == "11434" +assert s["environment"]["FLM_HOST"] == "0.0.0.0" +assert s["environment"]["FLM_CORS"] == "0" +assert "/dev/accel/accel0" in " ".join(str(x) for x in s.get("devices", [])) +assert s["ulimits"]["memlock"]["soft"] == -1 +assert s["ulimits"]["memlock"]["hard"] == -1 +assert {v["target"] for v in s["volumes"]} == {"/models"} +assert d["volumes"]["lds_llm_fastflow"]["name"] == "LLMFastFlowModels" +tools=services["server-tools"]["environment"] +assert tools["LDS_AI_PROVIDER"] == "llm" +assert tools["LDS_AI_URL"] == "http://llm:11434" +assert tools["LDS_AI_MODEL"] == "qwen3.5:9b" +' <<<"$npu_json" +pass "NPU runtime selects only FastFlow with its provider default model" amd_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=amd "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys -s=json.load(sys.stdin)["services"]["llm-ollama"] +d=json.load(sys.stdin) +services=d["services"] +assert "llm-ollama" in services +assert "llm-fastflow" not in services +s=services["llm-ollama"] assert s["image"] == "infocyph/llm-ollama:amd-latest" devices=" ".join(str(x) for x in s.get("devices", [])) assert "/dev/kfd" in devices and "/dev/dri" in devices +assert s["environment"]["LLM_OLLAMA_MODEL"] == "qwen3:14b" ' <<<"$amd_json" -pass "AMD AI runtime is generated dynamically" +pass "AMD runtime selects only Ollama with generated ROCm devices" nvidia_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=nvidia "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" python3 -c ' import json,sys -s=json.load(sys.stdin)["services"]["llm-ollama"] +d=json.load(sys.stdin) +services=d["services"] +assert "llm-ollama" in services +assert "llm-fastflow" not in services +s=services["llm-ollama"] assert s["image"] == "infocyph/llm-ollama:latest" assert s.get("gpus") +assert s["environment"]["LLM_OLLAMA_MODEL"] == "qwen3:14b" ' <<<"$nvidia_json" -pass "NVIDIA AI runtime is generated dynamically" +pass "NVIDIA runtime selects only Ollama with GPU augmentation" + +cpu_json="$(COMPOSE_PROFILES=ai LDS_AI_RUNTIME=cpu "$ROOT/lds" --quiet config show --json --raw 2>/dev/null | sed -n '/^[[:space:]]*{/,$p')" +python3 -c ' +import json,sys +d=json.load(sys.stdin) +services=d["services"] +assert "llm-ollama" in services +assert "llm-fastflow" not in services +assert services["llm-ollama"]["environment"]["LLM_OLLAMA_MODEL"] == "qwen3:14b" +' <<<"$cpu_json" +pass "CPU runtime selects only Ollama" if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -quit | grep -q .; then fail "AI-specific Compose files must not exist" @@ -227,4 +264,6 @@ if [[ -d "$ROOT/docker/.runtime" ]] && find "$ROOT/docker/.runtime" -type f -pri fail "temporary AI Compose overrides were not cleaned up" fi assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' -pass "single LLM service plus ephemeral hardware overrides" +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-fastflow:latest' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'aliases: [llm]' +pass "mutually exclusive LLM providers share one common llm network identity" From 445153bc54ba111aafbcf64e84f424a43b69bbd7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:49:27 +0600 Subject: [PATCH 403/509] fix(ai): validate Graphify model through common OpenAI catalog --- lib/ai.sh | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 788b1a32..18200435 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -72,11 +72,13 @@ _graphify_local_model_preflight() { local model="${1:-}" response [[ -n "$model" ]] || return 1 need_bin curl "install curl to validate the selected LocalDevStack model" + need_bin jq "install jq to validate the selected LocalDevStack model" - response="$(curl --connect-timeout 3 --max-time 10 -fsS 'http://llm.localhost:11434/v1/models' 2>/dev/null)" || + response="$(curl --connect-timeout 3 --max-time 10 -fsS \ + 'http://llm.localhost:11434/v1/models' 2>/dev/null)" || die "The selected LocalDevStack LLM endpoint is unavailable. Start the ai profile first." - printf '%s' "$response" | grep -Fq ""$model"" || + jq -e --arg model "$model" '[.data[]?.id // empty] | index($model) != null' <<<"$response" >/dev/null 2>&1 || die "Model '$model' is not available from the active LocalDevStack provider. Run: lds llm pull $model" } From f3566f1c6bab5255ab19c0981a0685cad3c5e1d8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:49:58 +0600 Subject: [PATCH 404/509] test(ai): replace Ollama-only contract with common llm contract --- tests/ai-contract.sh | 91 +++++++++++++++++++++++++------------------- 1 file changed, 51 insertions(+), 40 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index a2a9ece2..b890e32a 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -5,8 +5,8 @@ ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" # shellcheck source=tests/lib/assertions.sh source "$ROOT/tests/lib/assertions.sh" -image="lds-fake-ollama:ci" -container="lds-fake-ollama-ci" +image="lds-fake-llm:ci" +container="lds-fake-llm-ci" network="lds-ai-ci" cleanup() { @@ -17,81 +17,92 @@ trap cleanup EXIT docker network create "$network" >/dev/null docker build -q -t "$image" "$ROOT/tests/fixtures/fake-ollama" >/dev/null -docker run -d --name "$container" --network "$network" --network-alias llm-ollama "$image" >/dev/null +docker run -d --name "$container" --network "$network" --network-alias llm --network-alias llm-ollama "$image" >/dev/null for _ in {1..20}; do - if docker exec "$container" python -c 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=1).read()' >/dev/null 2>&1; then + if docker exec "$container" python -c 'import urllib.request; urllib.request.urlopen("http://127.0.0.1:11434/v1/models", timeout=1).read()' >/dev/null 2>&1; then break fi sleep 1 done -tags="$( - docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=2).read().decode())' -)" -assert_contains "$tags" "qwen3:14b" - -generate="$( - docker exec "$container" python -c 'import urllib.request; r=urllib.request.Request("http://127.0.0.1:11434/api/generate", data=b"{}", headers={"Content-Type":"application/json"}); print(urllib.request.urlopen(r, timeout=2).read().decode())' -)" -assert_contains "$generate" "LocalDevStack CI" - models="$( docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/v1/models", timeout=2).read().decode())' )" assert_contains "$models" "qwen3:14b" -pass "fake Ollama tags/generate/OpenAI-compatible contracts" + +tags="$( + docker exec "$container" python -c 'import urllib.request; print(urllib.request.urlopen("http://127.0.0.1:11434/api/tags", timeout=2).read().decode())' +)" +assert_contains "$tags" "qwen3:14b" +pass "fake provider exposes common OpenAI API plus Ollama-native compatibility" docker pull infocyph/tools:latest >/dev/null provider_status="$( - docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=ollama -e LDS_AI_URL=http://llm-ollama:11434 -e LDS_AI_MODEL=qwen3:14b infocyph/tools:latest --status + docker run --rm --network "$network" --entrypoint askai -e LDS_AI_ENABLED=1 -e LDS_AI_PROVIDER=llm -e LDS_AI_URL=http://llm:11434 -e LDS_AI_MODEL=qwen3:14b infocyph/tools:latest --status )" +assert_contains "$provider_status" "provider=llm" assert_contains "$provider_status" "available=1" assert_contains "$provider_status" "model=qwen3:14b" -pass "latest Tools reaches the separate provider contract" +pass "latest Tools reaches the common LocalDevStack llm contract" [[ ! -e "$ROOT/docker/compose/ai.yaml" ]] || fail "base AI service must remain consolidated into companion.yaml" if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -quit | grep -q .; then fail "AI-specific Compose overlays must be generated ephemerally" fi + assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: [ai]' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-fastflow:latest' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: ["${LDS_AI_OLLAMA_PROFILE:-ai}"]' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: ["${LDS_AI_FASTFLOW_PROFILE:-__lds-ai-disabled-fastflow}"]' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'aliases: [llm]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollama' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'container_name: LLM_OLLAMA' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm_fastflow:/models' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'OLLAMA_IGPU_ENABLE=${LDS_AI_IGPU_ENABLE:-0}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_ATTACHMENT_MAX_BYTES=${LLM_OLLAMA_ATTACHMENT_MAX_BYTES:-16777216}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_PDF_MAX_PAGES=${LLM_OLLAMA_PDF_MAX_PAGES:-24}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5}' -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_FASTFLOW_MODEL=${LDS_AI_MODEL:-qwen3.5:9b}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'FLM_SERVE_PORT=11434' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-llm}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_URL=${LDS_AI_URL:-http://llm:11434}' assert_file_contains "$ROOT/docker/compose/http.yaml" 'LLM_PROXY_TIMEOUT_SECONDS=${LDS_AI_TIMEOUT:-1800}' -assert_file_contains "$ROOT/lib/ai.sh" 'cmd_graphify()' -assert_file_contains "$ROOT/lib/ai.sh" '_graphify_local_model_preflight()' -assert_file_contains "$ROOT/lib/ai.sh" "Run: lds llm pull \$model" -assert_file_contains "$ROOT/lib/ai.sh" 'OLLAMA_API_KEY="$api_key"' -assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" extract "$target" --backend ollama --no-cluster' -assert_file_contains "$ROOT/lib/ai.sh" 'graphify_bin" cluster-only "$target" --backend ollama' +assert_file_contains "$ROOT/docker/compose/http.yaml" '"127.0.0.1:11434:11434"' +pass "companion defines mutually exclusive provider services behind common llm alias" + +assert_file_contains "$ROOT/lib/compose.sh" 'LDS_AI_PROVIDER=llm' +assert_file_contains "$ROOT/lib/compose.sh" 'LDS_AI_URL=http://llm:11434' +assert_file_contains "$ROOT/lib/compose.sh" 'ollama_profile=__lds-ai-disabled-ollama' +assert_file_contains "$ROOT/lib/compose.sh" 'fastflow_profile=ai' +assert_file_contains "$ROOT/lib/compose.sh" 'ollama_profile=ai' +assert_file_contains "$ROOT/lib/compose.sh" 'fastflow_profile=__lds-ai-disabled-fastflow' assert_file_contains "$ROOT/lib/compose.sh" "' gpus: all'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" -assert_file_contains "$ROOT/lib/ai.sh" 'update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable"' -assert_file_contains "$ROOT/docker/compose/http.yaml" '"127.0.0.1:11434:11434"' -assert_file_contains "$ROOT/lib/ai.sh" 'docker_compose ps -q nginx' -pass "single companion AI service with ephemeral hardware augmentation and Nginx-owned native port" +pass "Compose wrapper selects exactly one provider and augments only Ollama GPU modes" + +assert_file_contains "$ROOT/lib/ai.sh" 'http://llm.localhost:11434/v1' +assert_file_contains "$ROOT/lib/ai.sh" '/v1/models' +assert_file_contains "$ROOT/lib/ai.sh" 'index($model) != null' +assert_file_contains "$ROOT/lib/ai.sh" 'Run: lds llm pull $model' +assert_file_contains "$ROOT/lib/ai.sh" 'ai_service_for_runtime' +assert_file_contains "$ROOT/lib/ai.sh" 'llm-fastflow' +assert_file_contains "$ROOT/lib/ai.sh" 'llm-ollama' +pass "LLM CLI and Graphify resolve through active provider/common endpoint" ( set -euo pipefail + need_bin() { :; } die() { return 1; } - docker_compose() { - [[ "$*" == "exec -T llm-ollama /bin/ollama show qwen3:14b" ]] + curl() { + printf '%s\n' '{"object":"list","data":[{"id":"qwen3:14b"},{"id":"qwen3.5:9b"}]}' } # shellcheck source=lib/ai.sh source "$ROOT/lib/ai.sh" + _graphify_local_model_preflight qwen3:14b || - fail "Graphify local-model preflight rejected an available model" + fail "Graphify rejected the Ollama default through common model catalog" + _graphify_local_model_preflight qwen3.5:9b || + fail "Graphify rejected the FastFlow default through common model catalog" if _graphify_local_model_preflight missing-model; then - fail "Graphify local-model preflight accepted a missing model" + fail "Graphify accepted a model absent from the common model catalog" fi ) -pass "Graphify validates the selected local model before extraction" +pass "Graphify validates either provider model through /v1/models" From 81d76c28042e243350c7227b5f28e93db7b9d7c7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:50:17 +0600 Subject: [PATCH 405/509] ci(ai): rename provider-neutral LLM contract --- .github/workflows/check.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index ebcf9c8b..88c8c5e9 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -86,15 +86,15 @@ jobs: - name: Render supported Compose matrices run: tests/compose-contract.sh - fake-ollama: - name: Fake Ollama contract + fake-llm: + name: Common LLM contract runs-on: ubuntu-latest timeout-minutes: 10 steps: - name: Checkout uses: actions/checkout@v7 - - name: Validate lightweight Ollama-compatible fixture + - name: Validate common OpenAI-compatible LLM fixture run: tests/ai-contract.sh published-images: From 3406da507f52f4d5743855e3b12e7cd178859590 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:50:21 +0600 Subject: [PATCH 406/509] docs(ai): describe common mutually exclusive provider flow --- README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 164f2b04..dfb30337 100644 --- a/README.md +++ b/README.md @@ -112,7 +112,7 @@ The exact optional endpoints shown by `lds urls` depend on enabled profiles. | RedisInsight | `https://ri.localhost` | | Mongo Express | `https://me.localhost` | | Kibana | `https://kibana.localhost` | -| Local AI | `https://llm-ollama.localhost` | +| Local AI | `https://llm.localhost` | Use `lds open admin`, `lds open mail`, `lds open db`, `lds open redis`, `lds open mongo`, `lds open kibana`, or `lds open ai` to open a known endpoint. @@ -228,8 +228,8 @@ See `docs/guides/databases-and-clients.rst` for the profile/client map. Enable the `ai` profile through `lds setup profile`. ```text -Tools consumer -> http://llm-ollama:11434 -User HTTPS -> https://llm-ollama.localhost +Tools consumer -> http://llm:11434 +User HTTPS -> https://llm.localhost Default model -> qwen3:14b Model store -> LLMModels ``` From bc4c14a119e9b47a0d7a31e3e13c164ba07564c8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:55:49 +0600 Subject: [PATCH 407/509] docs(ai): align README with mutually exclusive providers --- README.md | 91 +++++++++++++++++-------------------------------------- 1 file changed, 27 insertions(+), 64 deletions(-) diff --git a/README.md b/README.md index dfb30337..80fc004a 100644 --- a/README.md +++ b/README.md @@ -227,31 +227,37 @@ See `docs/guides/databases-and-clients.rst` for the profile/client map. Enable the `ai` profile through `lds setup profile`. +LocalDevStack runs exactly one provider at a time: + ```text -Tools consumer -> http://llm:11434 -User HTTPS -> https://llm.localhost -Default model -> qwen3:14b -Model store -> LLMModels +supported AMD XDNA2 NPU -> infocyph/llm-fastflow:latest +NVIDIA GPU -> infocyph/llm-ollama:latest +AMD ROCm GPU -> infocyph/llm-ollama:amd-latest +otherwise -> infocyph/llm-ollama:latest ``` -LocalDevStack auto-detects the preferred AI runtime during setup: NVIDIA when `nvidia-smi` is usable, AMD only when the Linux ROCm device nodes `/dev/kfd` and `/dev/dri` are present, otherwise CPU. An AMD CPU by itself does not select the AMD image. When the effective runtime is AMD and the host CPU vendor is AMD, setup persists `LDS_AI_IGPU_ENABLE=1`; the provider forwards it as `OLLAMA_IGPU_ENABLE=1` so Ollama does not discard the integrated Radeon GPU. - -The provider uses one image/tag contract: +`llm-fastflow` and `llm-ollama` are mutually exclusive. The selected service owns the common Docker alias `llm` on port `11434`, so provider-neutral consumers use: ```text -infocyph/llm-ollama:${LDS_LLM_ARCH} +Tools consumer -> http://llm:11434 +User HTTPS -> https://llm.localhost +Host loopback -> http://127.0.0.1:11434 ``` -CPU/NVIDIA map to `LDS_LLM_ARCH=latest`; AMD/ROCm maps to `LDS_LLM_ARCH=amd-latest`. `LDS_LLM_ARCH` is derived by the LocalDevStack runtime selector; it is not a separate version choice users should maintain manually. `lds llm runtime ...` also refreshes `LDS_AI_IGPU_ENABLE` to match the selected runtime and host CPU. +Provider-specific `https://llm-ollama.localhost` and `https://llm-fastflow.localhost` remain available for diagnostics/native operations. -Override detection explicitly when needed: +Automatic runtime selection prefers a supported XDNA2 NPU, then NVIDIA, then AMD ROCm device nodes, then CPU. Override it explicitly when needed: ```bash -lds llm runtime cpu +lds llm runtime auto +lds llm runtime npu lds llm runtime nvidia lds llm runtime amd +lds llm runtime cpu ``` +Provider defaults are intentionally different: FastFlow/NPU uses `qwen3.5:9b`; Ollama uses `qwen3:14b`. New setup leaves `LDS_AI_MODEL` blank so the selected provider default can apply. An explicit `LDS_AI_MODEL` overrides whichever provider is active. + Common commands: ```bash @@ -261,72 +267,29 @@ lds ai troubleshoot ... lds ai review ... lds ai repo-review ... -# Host Graphify + LocalDevStack Ollama -lds graphify -lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 - +lds llm provider lds llm models lds llm pull -lds llm show +lds llm run lds llm ask "Explain dependency injection briefly" lds llm chat ... -``` -LocalDevStack publishes the native Ollama API through Nginx on the fixed loopback-only endpoint `http://llm-ollama.localhost:11434`. The `llm-ollama` container itself remains internal and never owns a host port. - -`lds graphify [path] [extract-options...]` is a host-side Graphify workflow. It requires the host `graphify` CLI, uses the configured LocalDevStack model and `LDS_AI_TIMEOUT`, and targets `http://llm-ollama.localhost:11434/v1` through Nginx. For the LocalDevStack provider, the command first verifies that the selected model exists and fails immediately with an `lds llm pull ` hint when it does not. It then runs extraction with `--backend ollama --no-cluster`, followed by `cluster-only` for the same path if extraction succeeds. - -Start the AI profile, then Graphify can use the native endpoint directly: - -```bash -lds up -d llm-ollama +# Host Graphify through the common LocalDevStack /v1 endpoint lds graphify +lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 ``` -`lds graphify` also sets `OLLAMA_API_KEY=local` when no key is supplied. Ollama does not require authentication for this local loopback endpoint; the non-empty placeholder only satisfies Graphify's Ollama-backend warning check. An explicitly supplied `OLLAMA_API_KEY` is preserved. - -You may override `OLLAMA_BASE_URL`, `OLLAMA_API_KEY`, `OLLAMA_MODEL`, or `GRAPHIFY_API_TIMEOUT` for a one-off run. A CLI `--model` or `--api-timeout` override is kept consistent across both Graphify phases. - -The LocalDevStack LLM Compose layout is intentionally small: - -```text -docker/compose/main.yaml - └─ includes docker/compose/companion.yaml - └─ llm-ollama service (profile: ai) - -docker/.runtime/ai.* temporary per-command overlay generated by lds - only when NVIDIA or AMD/ROCm settings require it -``` - -There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or `ai-host-port.yaml` files. AI hardware additions are generated ephemerally and removed after the Compose command. Nginx owns the fixed loopback-only native Ollama publication on port 11434. Files under `configuration/compose/` remain the normal LocalDevStack extras/runtime-fragment area; they are not built-in LLM variant files. - -Provider configuration placed in `docker/.env` is forwarded to `llm-ollama` where applicable: +`lds llm` dispatches to the active provider. Ollama-only low-level commands (`ps`, `show`, `unload`, `ollama`) and FastFlow-only commands (`validate`, `check`, `flm`) are guarded and rejected when the other provider is active. -| Setting | Default | Effect | -|---|---:|---| -| `LDS_AI_MODEL` | `qwen3:14b` | Tools model and `lds llm` default model | -| `LDS_AI_RUNTIME` | auto-detected | `cpu`, `nvidia`, or `amd` runtime selection | -| `LDS_AI_IGPU_ENABLE` | `1` for AMD CPU + AMD runtime, otherwise `0` | Forwarded to Ollama as `OLLAMA_IGPU_ENABLE` | -| `LLM_OLLAMA_SYSTEM` | empty | Default system instruction for provider prompts | -| `LLM_OLLAMA_INPUT_WARN_BYTES` | `1048576` | Text/diff warning threshold | -| `LLM_OLLAMA_INPUT_MAX_BYTES` | `0` | Text/diff hard ceiling; `0` disables it | -| `LLM_OLLAMA_ATTACHMENT_MAX_BYTES` | `16777216` | Per attachment/source-file ceiling | -| `LLM_OLLAMA_ATTACHMENTS_MAX_BYTES` | `33554432` | Aggregate attachment ceiling | -| `LLM_OLLAMA_ATTACHMENT_MAX_COUNT` | `16` | Source-attachment count ceiling | -| `LLM_OLLAMA_PDF_MAX_PAGES` | `24` | PDF-vision page ceiling | -| `LLM_OLLAMA_PDF_DPI` | `120` | PDF-vision render DPI | -| `LLM_OLLAMA_ALLOW_LARGE_INPUT` | `0` | Deliberate bypass for configured provider limits | -| `OLLAMA_NUM_PARALLEL` | `1` | Ollama request parallelism | -| `OLLAMA_MAX_LOADED_MODELS` | `1` | Loaded-model limit | -| `OLLAMA_KEEP_ALIVE` | `5m` | Model keep-alive | -| `OLLAMA_NO_CLOUD` | `1` | Keep cloud integration disabled | +Nginx owns the loopback-only native route `127.0.0.1:11434 -> nginx:11434 -> llm:11434`. Provider containers do not publish host ports. -`LDS_AI_CONNECT_TIMEOUT=2` and `LDS_AI_PREFLIGHT_TIMEOUT=5` intentionally fail fast. `LDS_AI_TIMEOUT=1800` gives local model loading/inference up to 30 minutes and is also forwarded to the dedicated Nginx LLM route as `LLM_PROXY_TIMEOUT_SECONDS`. `LDS_AI_AVAILABILITY_TTL=5`, `LDS_AI_MAX_CONTEXT_BYTES=524288`, `LDS_AI_MAX_REQUEST_BYTES=1048576`, and `LDS_AI_MAX_RESPONSE_BYTES=2097152` remain Tools consumer controls rather than provider-container limits. +`lds graphify` uses `http://llm.localhost:11434/v1`, validates the selected model through `/v1/models`, and then invokes Graphify's currently named `ollama` backend. The backend name is Graphify terminology; the LocalDevStack endpoint itself is provider-neutral and OpenAI-compatible. -Use the `lds` wrapper for provider commands. LocalDevStack intentionally has no repository-root `compose.yml`; its effective Compose project is assembled from `docker/compose/main.yaml`, release/user env files, optional extras, and temporary runtime-specific overrides. Therefore a bare command such as `docker compose exec llm-ollama ...` from the LocalDevStack root is not equivalent and fails before reaching the container. Use `lds llm ...` instead. +The built-in Compose layout keeps both provider definitions in `docker/compose/companion.yaml`, but runtime-generated profile selectors enable exactly one. NVIDIA/ROCm hardware augmentation is generated ephemerally under `docker/.runtime/`; FastFlow's `/dev/accel/accel0` + memlock contract lives in its tracked service definition. -The `llm-ollama` container receives **no Docker socket and no project/repository bind mount** by default. Current published `llm-ollama` images are `linux/amd64`; the rest of LocalDevStack can still run on arm64 with the AI profile disabled. +Ollama model state persists in `LLMModels` (`/root/.ollama`). FastFlow model state persists in `LLMFastFlowModels` (`/models`). Both providers receive no Docker socket and no project/repository bind mount by default. +See `docs/guides/local-ai.rst` for the full runtime, model, Graphify, and trust-boundary contract. ## Storage and trust boundaries Important named volumes include `NginxHosts`, `ApacheHosts`, `SSLKeys`, `SSLRootCA`, `FPMPools`, `FPMSocks`, `ComposerGlobal`, `GitConfig`, `ToolsState`, database/admin stores, `EmailStore`, and `LLMModels`. From 762a0bdc5ffb5ced4e7ed5da341db3a56e938046 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:56:20 +0600 Subject: [PATCH 408/509] docs(ai): rewrite guide for common mutually exclusive providers --- docs/guides/local-ai.rst | 391 +++++++++++++++++---------------------- 1 file changed, 174 insertions(+), 217 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index d1b705af..07478ca0 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -1,188 +1,185 @@ Local AI ======== -LocalDevStack can run a local Ollama-compatible provider as an optional profile. -``llm-ollama`` owns the provider/runtime; Tools remains the higher-level AI consumer and -never embeds a second Ollama runtime. +LocalDevStack can enable one local LLM provider through the optional ``ai`` profile. -Enable AI ---------- +Provider Selection +------------------ -Use the guided profile setup:: +Exactly one provider service is active for a LocalDevStack runtime: - lds setup profile +.. code-block:: text -Select Local AI / the ``ai`` profile. + supported AMD XDNA2 NPU -> llm-fastflow -> infocyph/llm-fastflow:latest + NVIDIA GPU -> llm-ollama -> infocyph/llm-ollama:latest + AMD ROCm GPU -> llm-ollama -> infocyph/llm-ollama:amd-latest + otherwise -> llm-ollama -> infocyph/llm-ollama:latest -The default consumer contract is:: +``llm-fastflow`` and ``llm-ollama`` are mutually exclusive. They are not designed to run +simultaneously for one stack. The selected service owns the common Docker-network alias +``llm`` and the normalized internal LLM port ``11434``. - LDS_AI_ENABLED=auto - LDS_AI_PROVIDER=ollama - LDS_AI_URL=http://llm-ollama:11434 - LDS_AI_MODEL=qwen3:14b +Enable and Inspect AI +--------------------- -The explicit model default avoids ambiguity when multiple models are installed in the -persistent Ollama store. +.. code-block:: bash -Runtime Modes -------------- + lds profiles add ai + lds start + lds llm runtime + lds llm provider + lds images + lds doctor -LocalDevStack detects the preferred runtime during environment setup. Detection is intentionally capability-based rather than CPU-vendor based: +Provider-neutral clients should use: -- NVIDIA when ``nvidia-smi`` is usable; -- AMD only when both ``/dev/kfd`` and ``/dev/dri`` are present for ROCm; -- CPU otherwise. +.. code-block:: text -An AMD CPU alone does **not** select the AMD image. When the effective runtime is ``amd`` and the host CPU vendor is AMD, LocalDevStack persists ``LDS_AI_IGPU_ENABLE=1`` and forwards it to Ollama as ``OLLAMA_IGPU_ENABLE=1`` so integrated Radeon GPUs are not discarded by Ollama's default iGPU filter. + Docker network: http://llm:11434/v1 + HTTPS: https://llm.localhost/v1 + host loopback: http://127.0.0.1:11434/v1 -The single provider service uses:: +Provider-specific diagnostic/native routes remain available: - image: infocyph/llm-ollama:${LDS_LLM_ARCH} +.. code-block:: text -The derived image mapping is: + https://llm-ollama.localhost + https://llm-fastflow.localhost -- ``cpu`` -> ``LDS_LLM_ARCH=latest``; -- ``nvidia`` -> ``LDS_LLM_ARCH=latest`` plus the NVIDIA GPU overlay; -- ``amd`` -> ``LDS_LLM_ARCH=amd-latest`` plus the AMD device overlay. +The inactive provider-specific route normally returns ``502`` because its service is absent. -``LDS_LLM_ARCH`` is derived from the selected runtime for each Compose invocation. It is -not an independent image-version selector. ``lds llm runtime ...`` persists the explicit -runtime choice and its matching derived tag for compatibility/inspection. +Runtime Detection +----------------- -Override detection explicitly when required:: +Automatic detection prefers: - lds llm runtime cpu - lds llm runtime nvidia - lds llm runtime amd +1. a FastFlow-supported XDNA2 NPU; +2. usable NVIDIA via ``nvidia-smi``; +3. AMD ROCm when both ``/dev/kfd`` and ``/dev/dri`` exist; +4. CPU. -Changing runtime mode updates ``LDS_AI_RUNTIME``, ``LDS_LLM_ARCH`` and the derived ``LDS_AI_IGPU_ENABLE`` value in ``docker/.env``. Recreate/start the service afterward so the Compose override and provider environment changes take effect. +Override detection explicitly when needed: -Access ------- +.. code-block:: bash -Internal provider endpoint:: + lds llm runtime auto + lds llm runtime npu + lds llm runtime nvidia + lds llm runtime amd + lds llm runtime cpu - http://llm-ollama:11434 +``auto`` clears the explicit runtime and returns to host detection. -User-facing HTTPS endpoint:: +FastFlow / NPU +-------------- - https://llm-ollama.localhost +FastFlow is selected only for ``npu``. The host must provide a supported XDNA2 NPU at +``/dev/accel/accel0`` with the compatible ``amdxdna`` host-driver/firmware contract. -Internal Docker consumers should use the service endpoint directly rather than routing -through Nginx. +LocalDevStack configures: -Host-side native Ollama clients use the Nginx-owned endpoint:: +.. code-block:: text - http://llm-ollama.localhost:11434 + image: infocyph/llm-fastflow:latest + LLM_FASTFLOW_MODEL= + FLM_MODEL_PATH=/models + FLM_SERVE_PORT=11434 + FLM_HOST=0.0.0.0 + FLM_CORS=0 + FLM_DISABLE_UPDATE_CHECK=1 -LocalDevStack binds Nginx's native listener to ``127.0.0.1:11434``. The -``llm-ollama`` provider container itself has no published host port, and there is no -host-port toggle or configurable provider port. +The container receives the NPU device plus unlimited memlock. It does not install the +kernel driver, require privileged mode, or receive the Docker socket. -Compose Ownership ------------------ +FastFlow model data persists in ``LLMFastFlowModels`` mounted at ``/models``. -The tracked provider service exists only in:: +Ollama +------ - docker/compose/companion.yaml +Ollama owns CPU, NVIDIA and AMD ROCm paths: -The Compose service key/hostname is ``llm-ollama`` and the fixed container name remains -``LLM_OLLAMA`` for LocalDevStack single-stack compatibility. Internal consumers must continue -to route through ``llm-ollama`` rather than depending on the container name. +.. code-block:: text -and is included through:: + cpu -> infocyph/llm-ollama:latest + nvidia -> infocyph/llm-ollama:latest + amd -> infocyph/llm-ollama:amd-latest - docker/compose/main.yaml +NVIDIA receives an ephemeral ``gpus: all`` Compose augmentation. AMD receives ephemeral +``/dev/kfd`` and ``/dev/dri`` mappings. On an AMD CPU with the AMD runtime, the derived +``LDS_AI_IGPU_ENABLE`` value may be ``1`` and is forwarded as ``OLLAMA_IGPU_ENABLE``. -There are no tracked ``ai.yaml``, ``ai-nvidia.yaml``, ``ai-amd.yaml`` or -``ai-host-port.yaml`` files. +Ollama state persists in ``LLMModels`` mounted at ``/root/.ollama``. -When runtime-specific Compose data is required, ``lds`` creates a temporary fragment -under ``docker/.runtime/`` for the current command only: +Model Defaults +-------------- -- NVIDIA -> ``gpus: all``; -- AMD/ROCm -> ``/dev/kfd`` and ``/dev/dri``. +Provider defaults are different: -The native host API is not part of the temporary fragment. Nginx permanently owns -``127.0.0.1:11434:11434`` and proxies it to ``llm-ollama:11434``. +.. code-block:: text -The fragment is removed after the Compose command. ``configuration/compose/`` remains -the normal extras/generated-runtime area and is not the location of built-in LLM -runtime variants. + FastFlow / NPU -> qwen3.5:9b + Ollama -> qwen3:14b -AI vs LLM Commands ------------------- +New setup leaves ``LDS_AI_MODEL`` blank so the selected provider default can apply. +An explicit ``LDS_AI_MODEL`` in ``docker/.env`` overrides whichever provider is active. +Because the providers may publish different model names, keep this blank when relying on +automatic runtime switching unless the override exists in both providers. -``lds ai`` is for Tools-owned operational/developer intelligence:: +Provider CLI +------------ - lds ai status - lds ai ask "explain this error" - lds ai explain ... - lds ai troubleshoot ... - lds ai review ... - lds ai repo-review ... - lds ai graphify ... +Always invoke provider commands through ``lds llm``: -``lds llm`` is for the provider/model CLI:: +.. code-block:: bash lds llm models - lds llm ps - lds llm show lds llm pull - lds llm rm - lds llm unload lds llm run lds llm ask "Explain dependency injection briefly" - lds llm chat ... - lds llm prompt ... - lds llm code ... - lds llm review ... - lds llm json ... - lds llm ai-commit ... - lds llm ollama ... - lds llm api ... + lds llm chat + lds llm prompt "Summarize this" + lds llm code "Implement this function" + lds llm review file.php + lds llm json "Return one JSON object" + lds llm api /v1/models lds llm version -This separation keeps application/operational AI in Tools and model/runtime behavior in -the provider image. +Provider-specific low-level commands are guarded: -Always invoke provider commands through ``lds llm`` inside LocalDevStack. The repository -does not expose a root ``compose.yml`` because ``lds`` assembles the effective Compose -project from the tracked main file, env layers, optional extras, and runtime-specific -temporary overrides. A bare ``docker compose exec llm-ollama ...`` from the LocalDevStack -repository root therefore fails at the host Compose layer before ``llm-ollama`` runs. +.. code-block:: text -Model Persistence ------------------ + Ollama-only: ps, show, unload, ollama + FastFlow-only: validate, check, flm -Ollama state persists in the ``LLMModels`` named volume mounted at -``/root/.ollama``. +LocalDevStack refuses those commands when the other provider is active. -Container recreation/image replacement does not delete user-pulled models while the -volume is retained. +Tools Consumer +-------------- -Do not use ``lds down --volumes --yes`` or destructive volume cleanup during a normal -upgrade. +``server-tools`` uses only the common provider-neutral contract: -Selecting a Different Model ---------------------------- +.. code-block:: text -Pull a model explicitly:: + LDS_AI_PROVIDER=llm + LDS_AI_URL=http://llm:11434 + LDS_AI_MODEL= - lds llm pull +Therefore ``lds ai`` commands do not need to know which provider owns ``llm``: -Then set ``LDS_AI_MODEL`` in ``docker/.env`` if it should become the default for both -Tools AI commands and ``lds llm`` provider commands. LocalDevStack forwards that value to -the provider as ``LLM_OLLAMA_MODEL``. A command-scoped shell value remains the -highest-precedence LocalDevStack override. +.. code-block:: bash -The provider CLI does not silently download a missing model for an unrelated command. + lds ai status + lds ai ask "Explain this error" + lds ai troubleshoot + lds ai review + lds ai repo-review -Provider Settings ------------------ +Provider Options +---------------- + +Ollama-specific settings remain available through ``docker/.env``: -These provider settings can be placed in ``docker/.env`` and are forwarded to -``llm-ollama``:: +.. code-block:: text LLM_OLLAMA_SYSTEM= LLM_OLLAMA_INPUT_WARN_BYTES=1048576 @@ -193,134 +190,94 @@ These provider settings can be placed in ``docker/.env`` and are forwarded to LLM_OLLAMA_PDF_MAX_PAGES=24 LLM_OLLAMA_PDF_DPI=120 LLM_OLLAMA_ALLOW_LARGE_INPUT=0 - LDS_AI_IGPU_ENABLE=1 # auto-persisted for AMD CPU + AMD runtime; otherwise 0 OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_LOADED_MODELS=1 OLLAMA_KEEP_ALIVE=5m OLLAMA_NO_CLOUD=1 -``LDS_AI_IGPU_ENABLE`` is forwarded as ``OLLAMA_IGPU_ENABLE``. The automatic value is ``1`` only for an AMD CPU with the AMD runtime selected; users may override the persisted value deliberately. - -``LLM_OLLAMA_INPUT_MAX_BYTES=0`` disables only the hard text/diff ceiling. Attachment and -PDF-vision limits remain active unless their own value is set to ``0``. -``LLM_OLLAMA_ALLOW_LARGE_INPUT=1`` is the explicit escape hatch for a deliberate request. - -Consumer settings are separate. The following values configure the Tools AI layer, not -the provider runtime:: - - LDS_AI_CONNECT_TIMEOUT=2 - LDS_AI_PREFLIGHT_TIMEOUT=5 - LDS_AI_TIMEOUT=1800 - LDS_AI_AVAILABILITY_TTL=5 - LDS_AI_MAX_CONTEXT_BYTES=524288 - LDS_AI_MAX_REQUEST_BYTES=1048576 - LDS_AI_MAX_RESPONSE_BYTES=2097152 +FastFlow developer-input limits may also be overridden: -Connection and provider/model preflight remain deliberately short. Generation and -analysis use a separate 30-minute default because first model load, CPU inference, and -larger local prompts can legitimately take much longer. LocalDevStack also forwards -``LDS_AI_TIMEOUT`` to Nginx as ``LLM_PROXY_TIMEOUT_SECONDS`` for the dedicated -``llm-ollama.localhost`` route so the edge proxy does not terminate a valid generation earlier. +.. code-block:: text -Privacy and Trust Boundaries ----------------------------- + LLM_FASTFLOW_INPUT_WARN_BYTES=1048576 + LLM_FASTFLOW_INPUT_MAX_BYTES=0 + LLM_FASTFLOW_ATTACHMENT_MAX_BYTES=16777216 + LLM_FASTFLOW_ATTACHMENTS_MAX_BYTES=33554432 + LLM_FASTFLOW_ATTACHMENT_MAX_COUNT=16 + LLM_FASTFLOW_PDF_MAX_PAGES=24 + LLM_FASTFLOW_PDF_DPI=120 + LLM_FASTFLOW_ALLOW_LARGE_INPUT=0 -The base ``llm-ollama`` service lives in ``docker/compose/companion.yaml`` and is gated by -the ``ai`` profile. Runtime-specific hardware/port additions are ephemeral as described -above; no separate tracked AI Compose variants exist. +Nginx +----- -By default, ``llm-ollama`` receives: +Nginx owns the fixed loopback publication: -- no Docker socket; -- no project/repository bind mount; -- no host port; -- only the model volume and LocalDevStack networks needed for provider access. +.. code-block:: text -Tools may send bounded/sanitized context to the local provider when the user invokes an -AI feature. Deterministic monitoring/system checks remain the source of truth. + 127.0.0.1:11434 -> nginx:11434 -> llm:11434 -LocalDevStack does not: +The provider containers themselves do not publish host ports. ``LDS_AI_TIMEOUT`` defaults +to ``1800`` seconds and is forwarded as ``LLM_PROXY_TIMEOUT_SECONDS``. -- silently fall back to a cloud AI provider; -- automatically execute model-generated shell commands; -- automatically execute model-generated SQL; -- automatically execute generated code. - -Repository Context ------------------- - -LocalDevStack intentionally does not mount the project/repository into llm-ollama by default. - -Repository-aware analysis should normally use the Tools consumer layer:: +Compose Ownership +----------------- - lds ai review ... - lds ai repo-review ... - lds ai graphify ... +Both provider definitions live in ``docker/compose/companion.yaml``. Runtime-generated +profile selectors enable exactly one provider. There are no tracked ``ai.yaml``, +``ai-nvidia.yaml``, ``ai-amd.yaml`` or ``ai-host-port.yaml`` variants. -For direct provider commands, file/PDF/image paths must exist inside the provider -container. The upstream ``llm-ollama`` CLI also supports stdin-based flows such as -``ai-commit --diff-stdin`` when explicitly invoked. +Only Ollama NVIDIA/ROCm hardware augmentation is generated temporarily under +``docker/.runtime/``. FastFlow's XDNA2 device/memlock contract is part of its tracked +service definition. Graphify -------- -Two Graphify flows are intentionally separate. - -``lds ai graphify`` analyzes an already-produced Graphify output file through the Tools -AI layer. +``lds graphify [path]`` uses the common LocalDevStack endpoint: -``lds graphify [path] [extract-options...]`` runs the host Graphify CLI against the -LocalDevStack Ollama provider. The default path is the current directory. It derives the -model from ``LDS_AI_MODEL``, derives ``GRAPHIFY_API_TIMEOUT`` from -``LDS_AI_TIMEOUT``, and uses ``http://llm-ollama.localhost:11434/v1`` through -Nginx unless ``OLLAMA_BASE_URL`` is already set. +.. code-block:: text -Start the stack with the AI profile, then run:: + OLLAMA_BASE_URL=http://llm.localhost:11434/v1 + OLLAMA_MODEL= + OLLAMA_API_KEY=local + GRAPHIFY_API_TIMEOUT= - lds up -d - lds graphify +Graphify currently calls this backend ``ollama``. That is Graphify terminology; the +LocalDevStack route is provider-neutral and OpenAI-compatible. Before extraction, +LocalDevStack checks ``/v1/models`` and fails fast when the selected model is absent. -When LocalDevStack owns the provider endpoint, ``lds graphify`` first verifies the -selected model in the running Ollama provider. A missing model fails before repository -extraction starts and reports the matching ``lds llm pull `` command. An explicit -external ``OLLAMA_BASE_URL`` remains caller-controlled and skips this local-provider -preflight. +An explicit ``OLLAMA_BASE_URL`` remains caller-controlled and bypasses the local-provider +preflight. Extraction still uses ``--backend ollama --no-cluster`` followed by +``cluster-only`` so clustering occurs once. -Internally the workflow runs extraction with ``--backend ollama --no-cluster`` and, -only after a successful extraction, runs ``cluster-only`` against the same path. This -preserves the requested two-phase flow without performing the default extraction -clustering and then immediately clustering a second time. +Trust Boundary +-------------- -Example:: +Neither provider receives the Docker socket or a project/repository bind mount by default. +LocalDevStack does not automatically execute model-generated shell commands, SQL or code, +and the common Tools AI client does not silently fall back to cloud AI. - lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 +Repository-aware analysis should normally flow through Tools or an explicit provider +workspace override. -``lds graphify`` sets ``OLLAMA_API_KEY=local`` when no key is supplied. Ollama does -not require authentication on this loopback endpoint; the non-empty placeholder only -suppresses Graphify's Ollama-backend warning. An explicitly supplied -``OLLAMA_API_KEY`` is preserved. +Platform Availability +--------------------- -A one-off ``--model`` or ``--api-timeout`` option is propagated to the clustering -phase through the corresponding Graphify environment value. ``llm-ollama`` remains only -the model provider and still receives no repository bind mount. +FastFlow is currently a ``linux/amd64`` XDNA2 runtime. LocalDevStack remains usable with +the ``ai`` profile disabled on unsupported platforms. -Diagnostics ------------ +Troubleshooting +--------------- -:: +.. code-block:: bash - lds ai status - lds llm models + lds llm runtime + lds llm provider lds doctor - lds urls + lds logs llm-fastflow lds logs llm-ollama + curl -fsS http://127.0.0.1:11434/v1/models | jq -Platform Availability ---------------------- - -The current published ``llm-ollama`` standard and AMD images have a linux/amd64 only -platform contract. - -LocalDevStack itself can still be used on arm64 with the ``ai`` profile disabled. Native -arm64 local-AI support should only be advertised after the provider publishes and -validates that platform. +If FastFlow was expected but not selected, verify ``/dev/accel/accel0`` and the host +``amdxdna`` driver before forcing ``lds llm runtime npu``. From 4f2afed904dc2129481b65a5580c7c83302ffb78 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:56:59 +0600 Subject: [PATCH 409/509] docs(ai): align profile and environment reference --- docs/concepts/profiles-and-env.rst | 83 +++++++++++++++++++----------- 1 file changed, 54 insertions(+), 29 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index adb932d9..6d700c64 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -166,8 +166,9 @@ Examples: - PostgreSQL defaults to ``postgres:alpine``. - Tools, Runner, Nginx, and Apache use their published ``:latest`` aliases. -- Standard local AI uses ``infocyph/llm-ollama:latest``. -- AMD local AI uses ``infocyph/llm-ollama:amd-latest``. +- XDNA2 NPU local AI uses ``infocyph/llm-fastflow:latest``. +- CPU/NVIDIA local AI uses ``infocyph/llm-ollama:latest``. +- AMD ROCm local AI uses ``infocyph/llm-ollama:amd-latest``. - Elasticsearch, Kibana, and Filebeat share ``ELASTICSEARCH_VERSION`` and default to the current stable ``9.5.4`` because this Elastic image set does not expose a usable moving ``latest`` alias. Run:: @@ -210,37 +211,53 @@ builds. Other arbitrary refs are rejected. AI Settings ----------- -Important AI settings include:: +Important LocalDevStack AI settings include:: LDS_AI_ENABLED=auto - LDS_AI_PROVIDER=ollama - LDS_AI_URL=http://llm-ollama:11434 - LDS_AI_MODEL=qwen3:14b - LDS_AI_RUNTIME= + LDS_AI_PROVIDER=llm + LDS_AI_URL=http://llm:11434 + LDS_AI_MODEL= + LDS_AI_RUNTIME= LDS_AI_IGPU_ENABLE= +``LDS_AI_PROVIDER`` and ``LDS_AI_URL`` describe the common consumer contract. They do +not identify which backend implementation is active. + When ``LDS_AI_RUNTIME`` is not explicitly set, LocalDevStack detects the preferred -runtime for the Compose invocation. NVIDIA is selected only when ``nvidia-smi`` is -usable; AMD is selected only when the ROCm Linux device nodes ``/dev/kfd`` and -``/dev/dri`` are present; otherwise CPU is selected. +runtime in this order: supported XDNA2 NPU, NVIDIA, AMD ROCm, CPU. + +Provider mapping is:: -``LDS_LLM_ARCH`` is derived from that runtime (``latest`` for CPU/NVIDIA, -``amd-latest`` for AMD). It is not a separate user-facing image version selector. -Using ``lds llm runtime ...`` persists the explicit runtime choice, matching derived -tag and the automatic ``LDS_AI_IGPU_ENABLE`` value. On an AMD CPU with the AMD runtime, -the automatic value is ``1`` and is forwarded as ``OLLAMA_IGPU_ENABLE=1``; otherwise -it is ``0``. + npu -> FastFlow / infocyph/llm-fastflow:latest / qwen3.5:9b + nvidia -> Ollama / infocyph/llm-ollama:latest / qwen3:14b + amd -> Ollama / infocyph/llm-ollama:amd-latest / qwen3:14b + cpu -> Ollama / infocyph/llm-ollama:latest / qwen3:14b + +Only one provider service is enabled. The selected service owns the ``llm`` Docker alias +on port ``11434``. Use:: - lds llm runtime + lds llm runtime auto + lds llm runtime npu + lds llm runtime nvidia + lds llm runtime amd + lds llm runtime cpu + +to control the runtime explicitly. ``auto`` clears the explicit override. -to override the detected runtime behavior. +``LDS_LLM_ARCH`` remains a derived Ollama compatibility tag (``latest`` or +``amd-latest``); it is not the FastFlow selector and should not be managed as a generic +LLM version field. -The selected ``LDS_AI_MODEL`` is also forwarded to the provider as -``LLM_OLLAMA_MODEL``, so Tools and ``lds llm`` share the same default model. +New setup leaves ``LDS_AI_MODEL`` blank so the active provider default applies. An +explicit model override is forwarded to both Tools and the active provider. Keep it blank +when switching providers automatically unless the same model exists in both runtimes. -Provider-side options accepted in ``docker/.env`` include:: +On an AMD CPU with the AMD runtime, the automatic ``LDS_AI_IGPU_ENABLE`` value is ``1`` +and is forwarded as ``OLLAMA_IGPU_ENABLE=1``; otherwise the derived value is ``0``. + +Ollama-side options include:: LLM_OLLAMA_SYSTEM= LLM_OLLAMA_INPUT_WARN_BYTES=1048576 @@ -256,7 +273,18 @@ Provider-side options accepted in ``docker/.env`` include:: OLLAMA_KEEP_ALIVE=5m OLLAMA_NO_CLOUD=1 -Tools consumer settings remain separate:: +FastFlow-side input options include:: + + LLM_FASTFLOW_INPUT_WARN_BYTES=1048576 + LLM_FASTFLOW_INPUT_MAX_BYTES=0 + LLM_FASTFLOW_ATTACHMENT_MAX_BYTES=16777216 + LLM_FASTFLOW_ATTACHMENTS_MAX_BYTES=33554432 + LLM_FASTFLOW_ATTACHMENT_MAX_COUNT=16 + LLM_FASTFLOW_PDF_MAX_PAGES=24 + LLM_FASTFLOW_PDF_DPI=120 + LLM_FASTFLOW_ALLOW_LARGE_INPUT=0 + +Tools consumer controls remain separate:: LDS_AI_CONNECT_TIMEOUT=2 LDS_AI_PREFLIGHT_TIMEOUT=5 @@ -266,14 +294,11 @@ Tools consumer settings remain separate:: LDS_AI_MAX_REQUEST_BYTES=1048576 LDS_AI_MAX_RESPONSE_BYTES=2097152 -Only generation/analysis receives the long 1800-second default. Connect and preflight -checks stay fast. The same generation timeout is passed to Nginx as -``LLM_PROXY_TIMEOUT_SECONDS`` for the dedicated LLM proxy path. - -The LLM service itself is tracked only in ``docker/compose/companion.yaml``. There are -no tracked AI runtime-variant YAML files; ``lds`` creates temporary fragments under -``docker/.runtime/`` only for NVIDIA or AMD/ROCm augmentation. +The same generation timeout is passed to Nginx as ``LLM_PROXY_TIMEOUT_SECONDS``. +Both provider definitions live in ``docker/compose/companion.yaml``. No tracked AI +runtime-variant YAML files exist; ``lds`` creates temporary fragments under +``docker/.runtime/`` only for NVIDIA/ROCm augmentation. Compose Extras -------------- From 4bf3eb0f28e7c39069a1b239da2c39eefcb84d73 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:57:13 +0600 Subject: [PATCH 410/509] docs(ai): align CLI reference with common provider flow --- docs/reference/cli.rst | 56 ++++++++++++++++++++++-------------------- 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index ddd6fb0c..5ab4e33a 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -206,32 +206,32 @@ Host Graphify Workflow lds graphify ./your-project lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 -This command runs the host ``graphify`` CLI against LocalDevStack's Ollama provider. +This command runs the host ``graphify`` CLI against the common LocalDevStack LLM route. It performs ``extract --backend ollama --no-cluster`` followed by -``cluster-only --backend ollama``, so clustering happens once. +``cluster-only --backend ollama`` so clustering happens once. Graphify's +backend is currently named ``ollama`` even though LocalDevStack presents a provider-neutral +OpenAI-compatible ``/v1`` endpoint. By default it derives: -- ``OLLAMA_BASE_URL=http://llm-ollama.localhost:11434/v1``, routed through Nginx; -- ``OLLAMA_MODEL`` from ``LDS_AI_MODEL``; +- ``OLLAMA_BASE_URL=http://llm.localhost:11434/v1`` through Nginx; +- ``OLLAMA_MODEL`` from the effective provider model; - ``GRAPHIFY_API_TIMEOUT`` from ``LDS_AI_TIMEOUT``. -The stack and AI profile must be running; there is no separate host-port setup step. - -An explicitly supplied ``OLLAMA_BASE_URL`` overrides the default -``http://llm-ollama.localhost:11434/v1`` endpoint. +Before extraction, LocalDevStack checks ``/v1/models`` and fails immediately when the +selected model is unavailable. An explicitly supplied ``OLLAMA_BASE_URL`` bypasses that +local-provider preflight. LLM Provider ------------ :: + lds llm provider + lds llm runtime lds llm models - lds llm ps - lds llm show ... lds llm pull ... lds llm rm ... - lds llm unload ... lds llm run ... lds llm ask ... lds llm chat ... @@ -240,32 +240,34 @@ LLM Provider lds llm review ... lds llm json ... lds llm ai-commit ... - lds llm ollama ... lds llm api ... lds llm version lds llm help -These commands execute the bundled provider CLI through LocalDevStack's Compose wrapper. -Do not replace them with bare ``docker compose exec llm-ollama ...`` from the repository -root; LocalDevStack has no root ``compose.yml``. +These commands execute the active provider CLI through LocalDevStack's Compose wrapper. +Exactly one provider is active: FastFlow for XDNA2 NPU, otherwise Ollama. -``LDS_AI_MODEL`` is forwarded to the provider as ``LLM_OLLAMA_MODEL``, so -``lds ai`` and ``lds llm`` share the configured default model. Provider input, -attachment, PDF and Ollama runtime knobs are documented in :doc:`../guides/local-ai`. +Provider-specific low-level commands are guarded: -Runtime selection:: +.. code-block:: text - lds llm runtime - lds llm runtime + Ollama-only: ps, show, unload, ollama + FastFlow-only: validate, check, flm + +Runtime selection:: -The runtime command keeps the derived image tag in sync and also refreshes -``LDS_AI_IGPU_ENABLE``. AMD runtime on an AMD CPU uses ``1`` so Ollama admits the -integrated Radeon GPU; the other derived cases use ``0``. + lds llm runtime auto + lds llm runtime npu + lds llm runtime nvidia + lds llm runtime amd + lds llm runtime cpu -Native Ollama access is always routed through Nginx at -``http://llm-ollama.localhost:11434``; the provider container itself is not -published directly. +The common Docker/API identity is ``llm:11434`` and the user-facing route is +``https://llm.localhost``. Nginx publishes the common native API loopback-only at +``http://127.0.0.1:11434``. +Provider defaults are ``qwen3.5:9b`` for FastFlow/NPU and ``qwen3:14b`` for Ollama. +Leaving ``LDS_AI_MODEL`` blank allows the runtime-specific default to apply. Rebuild ------- From 54eb1b4000d05f2939f2b6aae69bbf49acee2fda Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:57:37 +0600 Subject: [PATCH 411/509] test(docs): lock common LLM architecture documentation --- tests/docs-contract.sh | 41 ++++++++++++++++++++++++----------------- 1 file changed, 24 insertions(+), 17 deletions(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index f004606b..75b899ef 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -59,29 +59,36 @@ assert_file_contains "$storage" 'SSLKeys / SSLRootCA' assert_file_contains "$storage" 'ToolsState' pass "docs distinguish persistent runtime/control state and public TLS exports" -assert_file_contains "$ai" 'http://llm-ollama:11434' -assert_file_contains "$ai" 'https://llm-ollama.localhost' -assert_file_contains "$ai" 'http://llm-ollama.localhost:11434' +assert_file_contains "$ai" 'http://llm:11434/v1' +assert_file_contains "$ai" 'https://llm.localhost/v1' +assert_file_contains "$ai" 'http://127.0.0.1:11434/v1' +assert_file_contains "$ai" 'llm-fastflow' +assert_file_contains "$ai" 'llm-ollama' +assert_file_contains "$ai" 'mutually exclusive' +assert_file_contains "$ai" 'qwen3.5:9b' +assert_file_contains "$ai" 'qwen3:14b' +assert_file_contains "$ai" 'LDS_AI_PROVIDER=llm' +assert_file_contains "$ai" 'LDS_AI_URL=http://llm:11434' +assert_file_contains "$ai" 'FLM_SERVE_PORT=11434' +assert_file_contains "$ai" '/dev/accel/accel0' +assert_file_contains "$ai" 'LLMFastFlowModels' +assert_file_contains "$ai" 'LLMModels' assert_file_contains "$ai" 'no Docker socket' assert_file_contains "$ai" 'no project/repository bind mount' -assert_file_contains "$ai" 'automatically execute model-generated shell commands' -assert_file_contains "$ai" 'linux/amd64 only' -assert_file_contains "$ai" 'does not mount the project/repository into llm-ollama by default' -assert_file_contains "$ai" 'infocyph/llm-ollama:${LDS_LLM_ARCH}' -assert_file_contains "$ai" 'LDS_LLM_ARCH=amd-latest' -assert_file_contains "$ai" 'LDS_AI_MODEL=qwen3:14b' -assert_file_contains "$ai" 'LLM_OLLAMA_ATTACHMENT_MAX_BYTES=16777216' -assert_file_contains "$ai" 'LLM_OLLAMA_PDF_MAX_PAGES=24' +assert_file_contains "$ai" 'model-generated shell commands' assert_file_contains "$ai" 'docker/compose/companion.yaml' assert_file_contains "$ai" 'docker/.runtime/' -assert_file_contains "$ai" 'There are no tracked ``ai.yaml``, ``ai-nvidia.yaml``, ``ai-amd.yaml`` or' -assert_file_contains "$ai" 'Always invoke provider commands through ``lds llm``' -assert_file_contains "$ai" 'docker compose exec llm-ollama' +assert_file_contains "$ai" 'lds llm runtime npu' +assert_file_contains "$ai" 'OLLAMA_BASE_URL=http://llm.localhost:11434/v1' +assert_file_contains "$profiles" 'LDS_AI_PROVIDER=llm' +assert_file_contains "$profiles" 'LDS_AI_URL=http://llm:11434' +assert_file_contains "$profiles" 'infocyph/llm-fastflow:latest' assert_file_contains "$profiles" 'LLM_OLLAMA_ALLOW_LARGE_INPUT=0' -assert_file_contains "$profiles" 'LDS_LLM_ARCH' +assert_file_contains "$profiles" 'LLM_FASTFLOW_ALLOW_LARGE_INPUT=0' assert_file_contains "$readme" 'lds llm ask "Explain dependency injection briefly"' -assert_file_contains "$readme" 'LLM_OLLAMA_ATTACHMENTS_MAX_BYTES' -pass "local AI trust boundary, Compose ownership, provider options and invocation limits are documented" +assert_file_contains "$readme" 'infocyph/llm-fastflow:latest' +assert_file_contains "$readme" 'https://llm.localhost' +pass "local AI common identity, mutually exclusive providers, trust boundary, and runtime defaults are documented" for stale in 'Scriptomatic/master' 'infocyph/tools:0.23.2' 'infocyph/runner:0.5' 'infocyph/nginx:0.4.1' 'infocyph/apache:0.4.2' 'infocyph/llm-ollama:0.03'; do if grep -RqsF "$stale" "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then From a35bc976930a72934a21553b1628c5c94483c3f7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:58:34 +0600 Subject: [PATCH 412/509] docs(ai): align architecture with selected common provider --- docs/concepts/architecture.rst | 52 ++++++++++++++++++---------------- 1 file changed, 28 insertions(+), 24 deletions(-) diff --git a/docs/concepts/architecture.rst b/docs/concepts/architecture.rst index 256b29e1..4aee0389 100644 --- a/docs/concepts/architecture.rst +++ b/docs/concepts/architecture.rst @@ -101,7 +101,9 @@ Service-to-service traffic uses Docker DNS names such as:: mongodb redis elasticsearch + llm llm-ollama + llm-fastflow The historical ``lds vpn-fix`` command remains only as a deprecated compatibility message because LocalDevStack no longer owns fixed bridge subnets. @@ -141,30 +143,31 @@ profiles before recreating the stack. AI Flow ------- -When the ``ai`` profile is enabled: - -1. ``llm-ollama`` provides the Ollama runtime and persistent model store; -2. Tools consumes ``http://llm-ollama:11434`` internally; -3. Nginx exposes ``https://llm-ollama.localhost`` and the loopback-only native endpoint ``http://llm-ollama.localhost:11434`` to host clients; -4. ``lds ai`` delegates higher-level/operational AI to Tools; -5. ``lds llm`` delegates model/runtime operations to the bundled ``llm-ollama`` CLI. - -The provider is one ``llm-ollama`` service declared in -``docker/compose/companion.yaml`` and enabled only by the ``ai`` profile. The service -keeps ``container_name: LLM_OLLAMA`` for current single-stack compatibility, while all -internal routing continues to use the Compose service/hostname ``llm-ollama``. Its image -is ``infocyph/llm-ollama:${LDS_LLM_ARCH}``: CPU/NVIDIA resolve to ``latest`` and -AMD/ROCm resolves to ``amd-latest``. ``LDS_LLM_ARCH`` is derived from the effective -runtime rather than maintained as an independent version selector. - -No AI-specific Compose files are tracked. ``lds`` generates a temporary fragment under -``docker/.runtime/`` only when NVIDIA GPU access, AMD device mappings, or loopback -host-port exposure is required, then removes it after the Compose command. The service -also forwards the configured ``LDS_AI_MODEL`` to the provider as ``LLM_OLLAMA_MODEL`` and -passes the documented provider safety/tuning settings from ``docker/.env``. For AMD -runtime on an AMD CPU, LocalDevStack persists ``LDS_AI_IGPU_ENABLE=1`` and forwards it -as ``OLLAMA_IGPU_ENABLE=1`` so Ollama admits the integrated Radeon GPU. +When the ``ai`` profile is enabled, LocalDevStack selects exactly one provider: +.. code-block:: text + + supported XDNA2 NPU -> llm-fastflow + NVIDIA/ROCm/CPU -> llm-ollama + +The selected service owns the common Docker DNS alias ``llm`` on internal port ``11434``. +Tools consumes ``http://llm:11434`` and Nginx exposes ``https://llm.localhost`` plus the +loopback-only native route ``http://127.0.0.1:11434``. Provider-specific Nginx routes +remain available for diagnostics/native operations. + +``lds ai`` delegates higher-level operational/developer AI to Tools. ``lds llm`` resolves +the active provider and dispatches provider/model operations to ``llm-fastflow`` or +``llm-ollama`` as appropriate. + +Both provider definitions live in ``docker/compose/companion.yaml`` but dynamic profile +selection ensures they are mutually exclusive. FastFlow uses ``infocyph/llm-fastflow:latest`` +for ``npu``. Ollama uses ``latest`` for CPU/NVIDIA and ``amd-latest`` for AMD/ROCm. + +Provider defaults also differ: FastFlow uses ``qwen3.5:9b`` and Ollama uses +``qwen3:14b`` unless ``LDS_AI_MODEL`` is explicitly set. + +NVIDIA/ROCm hardware augmentation is generated temporarily under ``docker/.runtime/``. +FastFlow's XDNA2 device and memlock contract is tracked directly in its service definition. Project Identity ---------------- @@ -196,6 +199,7 @@ equivalent to powerful host Docker control. The Docker socket is not mounted into: - ``llm-ollama``; +- ``llm-fastflow``; - databases; - database admin clients; - Nginx or Apache; @@ -204,7 +208,7 @@ The Docker socket is not mounted into: The separate ad-hoc ``lds run --sock`` option is an explicit opt-in and should be used only with trusted Dockerfiles/code. -``llm-ollama`` also receives no project/repository mount by default. AI output is not +Neither LLM provider receives a project/repository mount by default. AI output is not automatically executed as shell, SQL, or code. Persistence From 1a11d638a03557584ad33b03bd54e66a8efa4902 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:58:49 +0600 Subject: [PATCH 413/509] docs(ai): document both provider model stores --- docs/concepts/storage-layout.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/concepts/storage-layout.rst b/docs/concepts/storage-layout.rst index 7e1e5b94..0ee15a02 100644 --- a/docs/concepts/storage-layout.rst +++ b/docs/concepts/storage-layout.rst @@ -36,7 +36,10 @@ Admin/observability volumes Mailpit persistence. ``LLMModels`` - Ollama/local-model persistence when the ``ai`` profile is enabled. + Ollama model persistence when the selected AI provider is Ollama. + +``LLMFastFlowModels`` + FastFlow model persistence under ``/models`` when the selected AI provider is FastFlow. These named volumes are intentionally stable so developer data can survive container and image replacement. @@ -125,7 +128,7 @@ Docker Socket Boundary because their supported workflows need Docker control. -Persistent databases, admin clients, Nginx/Apache, and ``llm-ollama`` do not receive the +Persistent databases, admin clients, Nginx/Apache, ``llm-ollama``, and ``llm-fastflow`` do not receive the socket by default. ``lds run --sock`` is a separate explicit opt-in for an ad-hoc container. From c82890d47303891855f8e302e9155b732f61af10 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:59:12 +0600 Subject: [PATCH 414/509] docs(ai): align README image policy --- README.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 80fc004a..37e842f2 100644 --- a/README.md +++ b/README.md @@ -166,8 +166,9 @@ Examples: - PostgreSQL: `postgres:alpine` - Tools / Runner / Nginx / Apache: published `:latest` - MySQL / MariaDB / MongoDB / Redis: their supported moving defaults -- local AI: `infocyph/llm-ollama:latest` -- AMD local AI: `infocyph/llm-ollama:amd-latest` +- XDNA2 NPU local AI: `infocyph/llm-fastflow:latest` +- CPU/NVIDIA local AI: `infocyph/llm-ollama:latest` +- AMD ROCm local AI: `infocyph/llm-ollama:amd-latest` Elastic does not provide a usable moving `latest` alias for this stack, so Elasticsearch, Kibana, and Filebeat share `ELASTICSEARCH_VERSION` and default to the current stable `9.5.4`; overriding that one value advances or pins all three together. From 0a5374ce2d1c49d7d423dd62163763e42ab5df7b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 12:59:19 +0600 Subject: [PATCH 415/509] docs(plan): finalize mutually exclusive LLM provider architecture --- .../07-localdevstack-integration-plan.md | 82 +++++++++++++++++++ 1 file changed, 82 insertions(+) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 588f48f0..9f32d8fc 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -2300,3 +2300,85 @@ this fragment; Nginx owns the fixed loopback-only `127.0.0.1:11434` publication. This keeps the repository at one LLM service definition while still avoiding invalid GPU/device declarations on unsupported hosts. + + +--- + +# Final AI provider architecture override — 2026-09-20 + +This section supersedes every earlier Ollama-only or single-provider-service statement +in this plan where they conflict with the final LocalDevStack implementation. + +## Provider selection + +LocalDevStack tracks two provider service definitions in `docker/compose/companion.yaml`, +but enables exactly one of them for the `ai` profile: + +```text +supported XDNA2 NPU -> llm-fastflow -> infocyph/llm-fastflow:latest +NVIDIA GPU -> llm-ollama -> infocyph/llm-ollama:latest +AMD ROCm GPU -> llm-ollama -> infocyph/llm-ollama:amd-latest +CPU fallback -> llm-ollama -> infocyph/llm-ollama:latest +``` + +`llm-fastflow` and `llm-ollama` are mutually exclusive. They must not be active at the +same time for one LocalDevStack runtime. + +## Common LLM identity + +The selected provider owns the common Docker network alias and normalized internal port: + +```text +llm:11434 +``` + +Provider-neutral consumers use: + +```text +LDS_AI_PROVIDER=llm +LDS_AI_URL=http://llm:11434 +https://llm.localhost/v1 +http://127.0.0.1:11434/v1 +``` + +Provider-specific `llm-ollama.localhost` and `llm-fastflow.localhost` routes are +diagnostic/native identities only. Nginx owns the loopback publication and proxies it +to the common `llm` alias. + +## Model defaults + +The default model follows the runtime: + +```text +FastFlow / NPU -> qwen3.5:9b +Ollama -> qwen3:14b +``` + +Setup leaves `LDS_AI_MODEL` blank by default so the correct provider default can apply. +An explicit user value remains authoritative for the active provider. + +## Runtime selection + +Automatic detection order is XDNA2 NPU, NVIDIA, AMD ROCm, CPU. The explicit selector is: + +```text +lds llm runtime auto|npu|nvidia|amd|cpu +``` + +FastFlow's XDNA2 device/memlock contract is tracked in its service definition. NVIDIA +and ROCm Ollama hardware settings remain temporary Compose augmentation. + +## Persistence + +```text +LLMModels -> Ollama /root/.ollama +LLMFastFlowModels -> FastFlow /models +``` + +Neither provider receives a Docker socket or project/repository bind mount by default. + +## Completion impact + +The release-readiness AI matrix must validate both provider selections, the common +OpenAI-compatible `/v1` route, published `llm-fastflow:latest`, published Ollama images, +and the rule that only one provider service is present in the effective Compose graph. From dca95c2ca4dca746ac75e683d5b911db0b2d2989 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:00:35 +0600 Subject: [PATCH 416/509] docs(ai): state provider trust boundary explicitly --- docs/guides/local-ai.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 07478ca0..058bca6e 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -254,9 +254,12 @@ preflight. Extraction still uses ``--backend ollama --no-cluster`` followed by Trust Boundary -------------- -Neither provider receives the Docker socket or a project/repository bind mount by default. -LocalDevStack does not automatically execute model-generated shell commands, SQL or code, -and the common Tools AI client does not silently fall back to cloud AI. +Provider images have: + +- no Docker socket; +- no project/repository bind mount by default; +- no automatic execution of model-generated shell commands, SQL or code; +- no silent cloud fallback in the common Tools AI client. Repository-aware analysis should normally flow through Tools or an explicit provider workspace override. From 5540b5e76df8cc82b0b4c78b0d4016eb8b6fdc03 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:00:41 +0600 Subject: [PATCH 417/509] test(ai): name provider-neutral trust boundary --- tests/service-hardening-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/service-hardening-contract.sh b/tests/service-hardening-contract.sh index d9797790..b79c7e11 100644 --- a/tests/service-hardening-contract.sh +++ b/tests/service-hardening-contract.sh @@ -78,4 +78,4 @@ pass "Tools profile visibility follows LocalDevStack profile selection" if awk '/^ llm-ollama:/ { in_llm=1; next } in_llm && /^ [a-zA-Z0-9_-]+:/ { in_llm=0 } in_llm { print }' "$companion" | grep -Eq '/var/run/docker.sock|PROJECT_DIR|/app'; then fail "llm-ollama must not receive Docker socket or project mounts" fi -pass "companion-owned llm-ollama keeps the AI trust boundary" +pass "companion-owned mutually exclusive LLM providers keep the AI trust boundary" From ac5b911b08dda78f80e14b7188a3187e4796bb34 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:02:03 +0600 Subject: [PATCH 418/509] test(docs): align final AI Compose wording --- tests/docs-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 75b899ef..4dc9161f 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -132,5 +132,5 @@ for stale in LDS_TOOLS_IMAGE LDS_RUNNER_IMAGE LDS_NGINX_IMAGE LDS_APACHE_IMAGE; fail "user-facing docs expose obsolete fixed-image variable: $stale" fi done -assert_file_contains "$ai" 'no separate tracked AI Compose variants exist' +assert_file_contains "$ai" 'Both provider definitions live in ``docker/compose/companion.yaml``' pass "docs reflect fixed infrastructure images and ephemeral AI overrides" From 646839d22d807e2f8c750cf99c4d79fc32c1ed04 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:18:05 +0600 Subject: [PATCH 419/509] feat(ai): resolve common llm service alias operationally --- lib/services.sh | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/lib/services.sh b/lib/services.sh index 640bc8a7..c5a4de5c 100644 --- a/lib/services.sh +++ b/lib/services.sh @@ -50,6 +50,16 @@ resolve_service() { return 0 } + # "llm" is the stable operational alias for whichever mutually-exclusive + # provider is active for the effective runtime. + if [[ "${raw,,}" == "llm" ]]; then + norm="$(ai_service_for_runtime "$(effective_ai_runtime)")" || + die "Unable to resolve the active LLM provider service" + compose_service_exists "$norm" || die "Active LLM provider service is unavailable: $norm" + printf '%s' "$norm" + return 0 + fi + norm="$(normalize_service "$raw")" compose_service_exists "$norm" && { printf '%s' "$norm" From cf7d94db8a256fbac5ec5389b8882e87178d0a0c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:18:09 +0600 Subject: [PATCH 420/509] test(ai): cover common llm operational alias --- tests/ai-contract.sh | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index b890e32a..d7f27b8b 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -106,3 +106,35 @@ pass "LLM CLI and Graphify resolve through active provider/common endpoint" fi ) pass "Graphify validates either provider model through /v1/models" + + +( + set -euo pipefail + # shellcheck source=lib/services.sh + source "$ROOT/lib/services.sh" + + normalize_service() { printf '%s' "${1,,}"; } + docker() { return 1; } + + __test_runtime=npu + effective_ai_runtime() { printf '%s' "$__test_runtime"; } + ai_service_for_runtime() { + case "${1,,}" in + npu) printf '%s' llm-fastflow ;; + *) printf '%s' llm-ollama ;; + esac + } + compose_service_exists() { + case "$1" in + llm-fastflow|llm-ollama) return 0 ;; + *) return 1 ;; + esac + } + + [[ "$(resolve_service llm)" == "llm-fastflow" ]] || + fail "common llm alias did not resolve FastFlow for NPU runtime" + __test_runtime=cpu + [[ "$(resolve_service llm)" == "llm-ollama" ]] || + fail "common llm alias did not resolve Ollama for CPU runtime" +) +pass "generic service commands resolve llm to the active provider" From 558644edc0f3af69e66c58adcd7122830b008e1b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:18:39 +0600 Subject: [PATCH 421/509] docs(ai): document common llm operational alias --- docs/reference/cli.rst | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/reference/cli.rst b/docs/reference/cli.rst index 5ab4e33a..efbf1ff0 100644 --- a/docs/reference/cli.rst +++ b/docs/reference/cli.rst @@ -266,6 +266,13 @@ The common Docker/API identity is ``llm:11434`` and the user-facing route is ``https://llm.localhost``. Nginx publishes the common native API loopback-only at ``http://127.0.0.1:11434``. +Generic service operations also accept ``llm`` and resolve it to the active provider:: + + lds logs llm + lds restart llm + lds exec llm ... + lds rebuild llm + Provider defaults are ``qwen3.5:9b`` for FastFlow/NPU and ``qwen3:14b`` for Ollama. Leaving ``LDS_AI_MODEL`` blank allows the runtime-specific default to apply. Rebuild From 3a387a3f9d0840fba7c90498d9adbe4a24adeeb4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:18:43 +0600 Subject: [PATCH 422/509] docs(ai): prefer common llm alias for operations --- docs/guides/local-ai.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 058bca6e..6e07bfa1 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -278,9 +278,11 @@ Troubleshooting lds llm runtime lds llm provider lds doctor - lds logs llm-fastflow - lds logs llm-ollama + lds logs llm curl -fsS http://127.0.0.1:11434/v1/models | jq +Use ``lds logs llm-fastflow`` or ``lds logs llm-ollama`` only when you explicitly +want the provider-specific service identity. + If FastFlow was expected but not selected, verify ``/dev/accel/accel0`` and the host ``amdxdna`` driver before forcing ``lds llm runtime npu``. From 008ef9c2120b6c875a62bf59ffe7f86b2645648c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:18:46 +0600 Subject: [PATCH 423/509] test(docs): lock common llm operational alias --- tests/docs-contract.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 4dc9161f..2a909ebf 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -79,6 +79,10 @@ assert_file_contains "$ai" 'model-generated shell commands' assert_file_contains "$ai" 'docker/compose/companion.yaml' assert_file_contains "$ai" 'docker/.runtime/' assert_file_contains "$ai" 'lds llm runtime npu' +assert_file_contains "$ai" 'lds logs llm' +assert_file_contains "$cli" 'lds restart llm' +assert_file_contains "$cli" 'lds exec llm' +assert_file_contains "$cli" 'lds rebuild llm' assert_file_contains "$ai" 'OLLAMA_BASE_URL=http://llm.localhost:11434/v1' assert_file_contains "$profiles" 'LDS_AI_PROVIDER=llm' assert_file_contains "$profiles" 'LDS_AI_URL=http://llm:11434' From bbd5a8aebefec2009ac76437bd925b7218206870 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:19:16 +0600 Subject: [PATCH 424/509] docs(plan): remove stale Ollama-only completion wording --- .../07-localdevstack-integration-plan.md | 146 +++++++----------- 1 file changed, 55 insertions(+), 91 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 9f32d8fc..e162abf3 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -1895,10 +1895,10 @@ This LocalDevStack phase is complete when all of the following are true: 9. Mailpit remains persistent and TLS-capable. 10. Runner cron/Supervisor/logrotate workflows still work. 11. AI can be omitted completely with no degradation to the default product. -12. When AI is enabled, `llm-ollama` persists models and is reachable internally at `http://llm-ollama:11434`. -13. `https://llm-ollama.localhost` works through Nginx streaming proxy behavior. -14. Tools `askai`, `aiops` and AI-enabled `gitx` use the separate LLM provider. -15. No LocalDevStack component embeds a second Ollama runtime. +12. When AI is enabled, exactly one provider is active: FastFlow for supported XDNA2 NPU, otherwise Ollama. +13. The active provider persists its own model store and owns the common `llm:11434` identity; `https://llm.localhost` and loopback-only `http://127.0.0.1:11434` work through Nginx. +14. Tools `askai` / `aiops`, Graphify, and the provider CLIs use the common OpenAI-compatible LLM route; provider-specific native routes remain diagnostic/low-level only. +15. No LocalDevStack component embeds a second model runtime or starts both `llm-fastflow` and `llm-ollama` for one stack. 16. No AI component auto-executes model-generated commands. 17. Existing user volumes and env overrides survive upgrade. 18. Docker socket exposure is documented and limited to components that actually require it. @@ -2021,13 +2021,14 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do 4. **Batch 4 — optional AI integration** - optional `ai` profile; - - persistent `LLMModels`; - - CPU/NVIDIA/AMD modes; - - `https://llm-ollama.localhost`; - - Nginx-owned loopback-only native Ollama port; - - Tools consumer wiring; + - mutually-exclusive FastFlow/Ollama provider services behind common `llm:11434`; + - persistent `LLMModels` and `LLMFastFlowModels`; + - XDNA2 NPU / NVIDIA / AMD ROCm / CPU runtime modes with automatic detection; + - `https://llm.localhost` plus provider-specific diagnostic routes; + - Nginx-owned loopback-only common native API port; + - provider-neutral Tools consumer wiring; - `lds ai` / `lds llm` separation; - - fake-provider integration test. + - common OpenAI-compatible fake-provider integration test. 5. **Batch 5 — profiles/catalog/runtime defaults** - canonical host service catalog; @@ -2072,7 +2073,7 @@ All planned LocalDevStack integration batches are implemented on branch `plan/do - Prefer a moving Alpine variant when an image family provides a suitable one; otherwise use its normal moving latest alias. - PostgreSQL defaults to `postgres:alpine`. - Tools, Runner, Nginx and Apache consume their published `:latest` aliases. -- One LLM service uses `infocyph/llm-ollama:${LDS_LLM_ARCH}`; CPU/NVIDIA map to `latest`, AMD/ROCm maps to `amd-latest`. +- Exactly one LLM provider is active: XDNA2 NPU uses `infocyph/llm-fastflow:latest`; CPU/NVIDIA use `infocyph/llm-ollama:latest`; AMD/ROCm uses `infocyph/llm-ollama:amd-latest`. - Elasticsearch, Kibana and Filebeat share the `ELASTICSEARCH_VERSION` selector and currently default to stable `9.5.4`, because the required Elastic image set has no usable moving `latest` alias. - PHP/Node runtime selection remains user-driven and version-specific. - Existing named volumes and container names remain intentionally stable for this release. @@ -2084,7 +2085,6 @@ These remain future work rather than release blockers: - multi-instance container/volume namespacing; - automatic dependency-update PRs; - Docker socket proxy if it can reduce privilege without breaking supported workflows; -- automatic GPU detection; - Graphify installation/management; - additional browser AI UI; - automatic model downloads beyond the provider defaults. @@ -2098,11 +2098,12 @@ Audit window: approximately **2026-09-16 13:53 Asia/Dhaka through 2026-09-18**. Compared LocalDevStack against the current related releases/main contracts: -- Tools **0.23.2** +- Tools **0.25** - Runner **0.5** -- Nginx **0.4.1** +- Nginx **0.6** - Apache **0.4.2** -- LLM-Ollama **current stable** +- LLM-FastFlow **0.01.2** +- LLM-Ollama **0.05** - Toolset **2.0** - Scriptomatic current `main` @@ -2114,13 +2115,13 @@ No legacy LocalDevStack command/service/storage feature was removed: - all **60** old public/support `lds` functions still exist after modularization; - all **9** old `bin/*` wrappers remain; -- all **16** old Compose services remain; -- all **19** old named volumes remain. +- all **16** old Compose services remain; the current graph has **18** services after adding the two mutually-exclusive LLM provider definitions; +- all **19** old named volumes remain; the current graph has **22** named volumes after adding the two provider stores and Tools durable state. New runtime additions are additive: -- `llm-ollama`; -- `LLMModels`; +- `llm-ollama` and `llm-fastflow` provider definitions (mutually exclusive at runtime); +- `LLMModels` and `LLMFastFlowModels`; - `ToolsState`; - AI/QoL/diagnostic commands. @@ -2184,18 +2185,20 @@ profile bridge would regress Admin Panel-created Apache hosts. This is a non-blocking follow-up, not a release defect. -## LLM-Ollama capability boundaries +## LLM provider capability boundaries -The current published LLM-Ollama image contract is **linux/amd64 only**. LocalDevStack -remains usable on arm64 with the `ai` profile disabled. +The published FastFlow and Ollama provider contracts used by this integration are +**linux/amd64** AI runtimes. LocalDevStack remains usable with the `ai` profile disabled +on unsupported platforms. -LocalDevStack intentionally does not mount a repository/workspace into `llm-ollama` by -default. Direct model/API/chat/stdin workflows are supported. Repository-aware analysis -is supported through the Tools consumer layer (`lds ai review`, `repo-review`). +LocalDevStack intentionally does not mount a repository/workspace into either provider +by default. Direct model/API/chat/stdin workflows are supported through the active +provider, while repository-aware analysis remains available through the Tools consumer +layer (`lds ai review`, `repo-review`) and explicit provider workspace features when a +user intentionally enables them. -The upstream optional workspace override is not automatically enabled because doing so -would weaken the no-repository-ingestion default and still requires explicit Git -safe-directory/identity decisions for writable repository operations. +Exactly one provider runs for a stack. FastFlow owns supported XDNA2 NPU execution; +Ollama owns NVIDIA, AMD ROCm and CPU execution. ## Lower-layer compatibility notes @@ -2213,16 +2216,20 @@ safe-directory/identity decisions for writable repository operations. ## Readiness conclusion -After the corrections above, there is no identified legacy feature loss or current -cross-image release blocker. The remaining items are explicit optional/future capability -work rather than regressions. +After the corrections above and the final 2026-09-20 provider integration, there is no +identified legacy feature loss or current cross-image release blocker. Tools 0.25, +Nginx 0.6, FastFlow 0.01.2 and Ollama 0.05 are validated by the current compatibility +gate. The remaining items are explicit optional/future capability work rather than +regressions. --- -# Final implementation simplification — single LLM service and fixed infrastructure images +# Historical interim simplification — fixed infrastructure images -This section supersedes earlier planning text in this file wherever it conflicts with the final implementation. +This section records the pre-FastFlow simplification that established fixed infrastructure +images and ephemeral GPU augmentation. Its Ollama-only provider statements are superseded +by the **Final AI provider architecture override — 2026-09-20** below. ## Fixed infrastructure images @@ -2237,70 +2244,27 @@ infocyph/apache:latest Do not add `LDS_TOOLS_IMAGE`, `LDS_RUNNER_IMAGE`, `LDS_NGINX_IMAGE`, or `LDS_APACHE_IMAGE` indirection. `docker/release.env` is reserved for release/build defaults that genuinely vary, currently including `SCRIPTOMATIC_REF`. -## Single LLM service +## Provider-service evolution -The only tracked LLM service definition lives in: +The earlier implementation temporarily tracked only `llm-ollama` in +`docker/compose/companion.yaml`, with CPU/NVIDIA mapped to `latest` and AMD/ROCm to +`amd-latest`. That intermediate state established the generated NVIDIA/ROCm hardware +augmentation and Nginx-owned loopback publication. -```text -docker/compose/companion.yaml -``` - -with: - -```yaml -llm-ollama: - container_name: LLM_OLLAMA - image: infocyph/llm-ollama:${LDS_LLM_ARCH} - profiles: [ai] - environment: - LLM_OLLAMA_MODEL: ${LDS_AI_MODEL:-qwen3:14b} - OLLAMA_IGPU_ENABLE: ${LDS_AI_IGPU_ENABLE:-0} - # provider input/PDF/Ollama tuning values are forwarded from docker/.env -``` - -There are no tracked `ai.yaml`, `ai-nvidia.yaml`, `ai-amd.yaml`, or -`ai-host-port.yaml` files. Do not reintroduce them. - -Runtime/tag mapping is: +The final implementation now tracks both provider definitions in the same companion file +and enables exactly one through generated profile selection: ```text -cpu -> LDS_LLM_ARCH=latest -nvidia -> LDS_LLM_ARCH=latest -amd -> LDS_LLM_ARCH=amd-latest +npu -> llm-fastflow / infocyph/llm-fastflow:latest +nvidia -> llm-ollama / infocyph/llm-ollama:latest +amd -> llm-ollama / infocyph/llm-ollama:amd-latest +cpu -> llm-ollama / infocyph/llm-ollama:latest ``` -Initial runtime detection is conservative: - -- usable `nvidia-smi` / `nvidia-smi.exe` -> `nvidia`; -- both `/dev/kfd` and `/dev/dri` present -> `amd`; -- otherwise -> `cpu`. - -An AMD CPU alone does not select the ROCm image. When the effective runtime is `amd` -and the host CPU vendor is AMD, persist `LDS_AI_IGPU_ENABLE=1` and forward it to the -provider as `OLLAMA_IGPU_ENABLE=1`; otherwise the derived default is `0`. This avoids -Ollama's integrated-GPU filter silently dropping Ryzen integrated Radeon devices that are -otherwise available through the ROCm device nodes. - -Explicit `LDS_AI_RUNTIME` / `lds llm runtime ...` selection remains authoritative. -`LDS_LLM_ARCH` and the automatic `LDS_AI_IGPU_ENABLE` value are derived from that -effective runtime/host combination and must not become independent image-version -selectors. A deliberately persisted iGPU override remains user-controlled until the -runtime selector is invoked again. - -## Ephemeral hardware Compose augmentation - -Hardware settings are generated by `lds` only for the current Compose invocation: - -- NVIDIA -> `gpus: all`; -- AMD -> `/dev/kfd` and `/dev/dri`. - -The generated fragment is temporary, is not part of `configuration/compose/`, and is -removed after the Compose command completes. Native Ollama host access is not part of -this fragment; Nginx owns the fixed loopback-only `127.0.0.1:11434` publication. - -This keeps the repository at one LLM service definition while still avoiding invalid -GPU/device declarations on unsupported hosts. - +NVIDIA/ROCm hardware augmentation remains ephemeral; FastFlow's XDNA2 +`/dev/accel/accel0` + memlock contract is part of its tracked service definition. +Nginx continues to own loopback-only `127.0.0.1:11434`, now proxying the common +`llm:11434` alias. --- From 1971985dbcab7b1c912634c6c3ce3bb68b78ea1c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 13:19:33 +0600 Subject: [PATCH 425/509] test(docs): lock final provider architecture plan --- tests/docs-contract.sh | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 2a909ebf..3ff98d87 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -20,8 +20,9 @@ runner="$ROOT/docs/guides/ad-hoc-runner.rst" notify="$ROOT/docs/guides/notifications.rst" secrets="$ROOT/docs/guides/secrets-sops-age.rst" cli="$ROOT/docs/reference/cli.rst" +plan="$ROOT/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md" -for file in "$index" "$readme" "$quick" "$arch" "$profiles" "$storage" "$domain" "$tls" "$ai" "$databases" "$ops" "$runner" "$notify" "$secrets" "$cli"; do +for file in "$index" "$readme" "$quick" "$arch" "$profiles" "$storage" "$domain" "$tls" "$ai" "$databases" "$ops" "$runner" "$notify" "$secrets" "$cli" "$plan"; do assert_file "$file" done @@ -138,3 +139,21 @@ for stale in LDS_TOOLS_IMAGE LDS_RUNNER_IMAGE LDS_NGINX_IMAGE LDS_APACHE_IMAGE; done assert_file_contains "$ai" 'Both provider definitions live in ``docker/compose/companion.yaml``' pass "docs reflect fixed infrastructure images and ephemeral AI overrides" + +assert_file_contains "$plan" 'Tools **0.25**' +assert_file_contains "$plan" 'Nginx **0.6**' +assert_file_contains "$plan" 'LLM-FastFlow **0.01.2**' +assert_file_contains "$plan" 'LLM-Ollama **0.05**' +assert_file_contains "$plan" 'current graph has **18** services' +assert_file_contains "$plan" 'current graph has **22** named volumes' +assert_file_contains "$plan" 'mutually exclusive' +assert_file_contains "$plan" 'llm:11434' +for stale in \ + '# Final implementation simplification — single LLM service' \ + '- automatic GPU detection;' \ + 'When AI is enabled, `llm-ollama` persists models'; do + if grep -Fq "$stale" "$plan"; then + fail "implementation plan contains superseded AI wording: $stale" + fi +done +pass "implementation plan matches the final provider architecture and current published baselines" From 0ca122ebaed46c7dcf35fb8d526f1bc362a600fa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 14:24:04 +0600 Subject: [PATCH 426/509] fix(ai): use Graphify OpenAI backend for FastFlow --- lib/ai.sh | 137 +++++++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 121 insertions(+), 16 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 18200435..11e71b71 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -82,6 +82,14 @@ _graphify_local_model_preflight() { die "Model '$model' is not available from the active LocalDevStack provider. Run: lds llm pull $model" } +_graphify_backend_for_provider() { + case "${1,,}" in + fastflow) printf '%s' openai ;; + ollama) printf '%s' ollama ;; + *) return 1 ;; + esac +} + cmd_graphify() { need_bin graphify "install the Graphify CLI on the host first" @@ -89,18 +97,45 @@ cmd_graphify() { [[ $# -eq 0 ]] || shift [[ -e "$target" ]] || die "Graphify target does not exist: $target" - local base_url timeout model api_key graphify_bin arg next_is_model=0 next_is_timeout=0 local_provider=0 - if [[ -n "${OLLAMA_BASE_URL:-}" ]]; then - base_url="$OLLAMA_BASE_URL" - else - base_url="$(_graphify_local_base_url)" - local_provider=1 - fi + local runtime provider backend base_url timeout model api_key graphify_bin arg + local local_provider=0 + local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 + local has_token_budget=0 has_max_concurrency=0 + local token_budget_value="" max_concurrency_value="" + local -a graphify_defaults=() + + runtime="$(_active_llm_runtime)" + provider="$(_active_llm_provider)" + backend="$(_graphify_backend_for_provider "$provider")" || + die "Unsupported active LLM provider for Graphify: $provider" + + case "$backend" in + openai) + if [[ -n "${OPENAI_BASE_URL:-}" ]]; then + base_url="$OPENAI_BASE_URL" + else + base_url="$(_graphify_local_base_url)" + local_provider=1 + fi + model="${OPENAI_MODEL:-$(effective_ai_model "$runtime")}" + api_key="${OPENAI_API_KEY:-local}" + ;; + ollama) + if [[ -n "${OLLAMA_BASE_URL:-}" ]]; then + base_url="$OLLAMA_BASE_URL" + else + base_url="$(_graphify_local_base_url)" + local_provider=1 + fi + model="${OLLAMA_MODEL:-$(effective_ai_model "$runtime")}" + api_key="${OLLAMA_API_KEY:-local}" + ;; + esac + timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" - model="${OLLAMA_MODEL:-$(effective_ai_model "$(_active_llm_runtime)")}" - api_key="${OLLAMA_API_KEY:-local}" - # Keep explicit model/timeout overrides consistent across extraction and clustering. + # Keep explicit model/timeout/resource overrides consistent while LocalDevStack + # retains ownership of backend selection and the two-stage extract/cluster flow. for arg in "$@"; do if ((next_is_model)); then model="$arg" @@ -112,37 +147,107 @@ cmd_graphify() { next_is_timeout=0 continue fi + if ((next_is_token_budget)); then + token_budget_value="$arg" + next_is_token_budget=0 + continue + fi + if ((next_is_max_concurrency)); then + max_concurrency_value="$arg" + next_is_max_concurrency=0 + continue + fi + case "$arg" in --model) next_is_model=1 ;; --model=*) model="${arg#--model=}" ;; --api-timeout) next_is_timeout=1 ;; --api-timeout=*) timeout="${arg#--api-timeout=}" ;; + --token-budget) + has_token_budget=1 + next_is_token_budget=1 + ;; + --token-budget=*) + has_token_budget=1 + token_budget_value="${arg#--token-budget=}" + ;; + --max-concurrency) + has_max_concurrency=1 + next_is_max_concurrency=1 + ;; + --max-concurrency=*) + has_max_concurrency=1 + max_concurrency_value="${arg#--max-concurrency=}" + ;; --backend | --backend=*) - die "lds graphify owns --backend=ollama; do not pass --backend" + die "lds graphify selects the Graphify backend from the active LLM provider; do not pass --backend" ;; --no-cluster) die "lds graphify already separates extraction and clustering; do not pass --no-cluster" ;; esac done + ((next_is_model == 0)) || die "--model requires a value" ((next_is_timeout == 0)) || die "--api-timeout requires a value" + ((next_is_token_budget == 0)) || die "--token-budget requires a value" + ((next_is_max_concurrency == 0)) || die "--max-concurrency requires a value" + [[ -n "$model" ]] || die "Graphify model cannot be empty" [[ "$timeout" =~ ^[0-9]+$ ]] && ((timeout >= 1)) || die "GRAPHIFY_API_TIMEOUT/--api-timeout must be a positive integer" + if ((has_token_budget)); then + [[ "$token_budget_value" =~ ^[0-9]+$ ]] && ((token_budget_value >= 1)) || + die "--token-budget must be a positive integer" + fi + if ((has_max_concurrency)); then + [[ "$max_concurrency_value" =~ ^[0-9]+$ ]] && ((max_concurrency_value >= 1)) || + die "--max-concurrency must be a positive integer" + fi + + # FastFlow's Qwen3.5 9B path is OpenAI-compatible but has a smaller practical + # context budget than the old Ollama default. Keep semantic chunks conservative + # unless the caller explicitly chooses different Graphify limits. + if [[ "$provider" == fastflow ]]; then + if ((has_token_budget == 0)); then + token_budget_value="${LDS_GRAPHIFY_TOKEN_BUDGET:-4000}" + [[ "$token_budget_value" =~ ^[0-9]+$ ]] && ((token_budget_value >= 1)) || + die "LDS_GRAPHIFY_TOKEN_BUDGET must be a positive integer" + graphify_defaults+=(--token-budget "$token_budget_value") + fi + if ((has_max_concurrency == 0)); then + max_concurrency_value="${LDS_GRAPHIFY_MAX_CONCURRENCY:-1}" + [[ "$max_concurrency_value" =~ ^[0-9]+$ ]] && ((max_concurrency_value >= 1)) || + die "LDS_GRAPHIFY_MAX_CONCURRENCY must be a positive integer" + graphify_defaults+=(--max-concurrency "$max_concurrency_value") + fi + fi + ((local_provider == 0)) || _graphify_local_model_preflight "$model" graphify_bin="$(bin_path graphify)" ( - export OLLAMA_BASE_URL="$base_url" - export OLLAMA_API_KEY="$api_key" - export OLLAMA_MODEL="$model" export GRAPHIFY_API_TIMEOUT="$timeout" - "$graphify_bin" extract "$target" --backend ollama --no-cluster "$@" && - "$graphify_bin" cluster-only "$target" --backend ollama + case "$backend" in + openai) + unset OLLAMA_BASE_URL OLLAMA_API_KEY OLLAMA_MODEL + export OPENAI_BASE_URL="$base_url" + export OPENAI_API_KEY="$api_key" + export OPENAI_MODEL="$model" + ;; + ollama) + unset OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL + export OLLAMA_BASE_URL="$base_url" + export OLLAMA_API_KEY="$api_key" + export OLLAMA_MODEL="$model" + ;; + esac + + "$graphify_bin" extract "$target" --backend "$backend" --no-cluster "${graphify_defaults[@]}" "$@" && + "$graphify_bin" cluster-only "$target" --backend "$backend" ) } From f437c931588abfb92812bdd96b619b27941ec33a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 14:24:45 +0600 Subject: [PATCH 427/509] test(ai): cover provider-aware Graphify backends --- tests/cli-contract.sh | 52 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/tests/cli-contract.sh b/tests/cli-contract.sh index 72fe785f..ef454afa 100755 --- a/tests/cli-contract.sh +++ b/tests/cli-contract.sh @@ -45,24 +45,60 @@ pass "Graphify workflow help" graphify_log="$(mktemp)" cat >"$tmpbin/graphify" <<'SH' #!/usr/bin/env sh -printf 'base=%s key=%s model=%s timeout=%s args=%s\n' "$OLLAMA_BASE_URL" "$OLLAMA_API_KEY" "$OLLAMA_MODEL" "$GRAPHIFY_API_TIMEOUT" "$*" >>"$GRAPHIFY_TEST_LOG" +printf 'ollama_base=%s ollama_model=%s openai_base=%s openai_model=%s timeout=%s args=%s\n' \ + "${OLLAMA_BASE_URL-}" "${OLLAMA_MODEL-}" "${OPENAI_BASE_URL-}" "${OPENAI_MODEL-}" \ + "${GRAPHIFY_API_TIMEOUT-}" "$*" >>"$GRAPHIFY_TEST_LOG" SH chmod +x "$tmpbin/graphify" +# Ollama runtime keeps Graphify's native Ollama backend. PATH="$tmpbin:$PATH" \ +LDS_AI_RUNTIME=cpu \ OLLAMA_BASE_URL=http://custom-ollama.test:11434/v1 \ -LDS_AI_MODEL=qwen3:14b \ +OLLAMA_MODEL=test-ollama \ LDS_AI_TIMEOUT=1800 \ GRAPHIFY_TEST_LOG="$graphify_log" \ - "$ROOT/lds" graphify . --model test-model --api-timeout 42 --mode deep >/dev/null + "$ROOT/lds" graphify . --api-timeout 42 --mode deep >/dev/null + +grep -Fq 'ollama_base=http://custom-ollama.test:11434/v1 ollama_model=test-ollama openai_base= openai_model= timeout=42 args=extract . --backend ollama --no-cluster --api-timeout 42 --mode deep' "$graphify_log" || + fail "Graphify Ollama extract wrapper contract failed" +grep -Fq 'args=cluster-only . --backend ollama' "$graphify_log" || + fail "Graphify Ollama cluster wrapper contract failed" + +# FastFlow runtime uses Graphify's generic OpenAI backend and conservative local +# chunking defaults so qwen3.5:9b does not receive oversized semantic requests. +: >"$graphify_log" +PATH="$tmpbin:$PATH" \ +LDS_AI_RUNTIME=npu \ +OPENAI_BASE_URL=http://custom-fastflow.test:11434/v1 \ +OPENAI_MODEL=test-fastflow \ +LDS_AI_TIMEOUT=1800 \ +GRAPHIFY_TEST_LOG="$graphify_log" \ + "$ROOT/lds" graphify . --mode deep >/dev/null + +grep -Fq 'ollama_base= ollama_model= openai_base=http://custom-fastflow.test:11434/v1 openai_model=test-fastflow timeout=1800 args=extract . --backend openai --no-cluster --token-budget 4000 --max-concurrency 1 --mode deep' "$graphify_log" || + fail "Graphify FastFlow extract wrapper contract failed" +grep -Fq 'args=cluster-only . --backend openai' "$graphify_log" || + fail "Graphify FastFlow cluster wrapper contract failed" + +# Explicit Graphify resource controls always win over LocalDevStack defaults. +: >"$graphify_log" +PATH="$tmpbin:$PATH" \ +LDS_AI_RUNTIME=npu \ +OPENAI_BASE_URL=http://custom-fastflow.test:11434/v1 \ +OPENAI_MODEL=test-fastflow \ +GRAPHIFY_TEST_LOG="$graphify_log" \ + "$ROOT/lds" graphify . --token-budget 6000 --max-concurrency 2 >/dev/null + +grep -Fq 'args=extract . --backend openai --no-cluster --token-budget 6000 --max-concurrency 2' "$graphify_log" || + fail "Graphify FastFlow explicit resource overrides were not preserved" -grep -Fq 'base=http://custom-ollama.test:11434/v1 key=local model=test-model timeout=42 args=extract . --backend ollama --no-cluster --model test-model --api-timeout 42 --mode deep' "$graphify_log" || - fail "Graphify extract wrapper contract failed" -grep -Fq 'base=http://custom-ollama.test:11434/v1 key=local model=test-model timeout=42 args=cluster-only . --backend ollama' "$graphify_log" || - fail "Graphify cluster wrapper contract failed" rm -f "$graphify_log" assert_file_contains "$ROOT/lib/ai.sh" "http://llm.localhost:11434/v1" -pass "Graphify host workflow wrapper" +assert_file_contains "$ROOT/lib/ai.sh" 'fastflow) printf '\''%s'\'' openai' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_TOKEN_BUDGET:-4000' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_MAX_CONCURRENCY:-1' +pass "Graphify provider-aware host workflow wrapper" assert_file_contains "$ROOT/lds" 'exec "$DIR/bin/tool-runner" "$cmd" "$@"' pass "unknown command fallback remains delegated to tool-runner" From 5680d639fb9a1ee9780001ecbcd8efcf1b42892e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 14:24:59 +0600 Subject: [PATCH 428/509] test(ai): lock Graphify provider backend mapping --- tests/ai-contract.sh | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index d7f27b8b..0e696ee3 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -85,7 +85,11 @@ assert_file_contains "$ROOT/lib/ai.sh" 'Run: lds llm pull $model' assert_file_contains "$ROOT/lib/ai.sh" 'ai_service_for_runtime' assert_file_contains "$ROOT/lib/ai.sh" 'llm-fastflow' assert_file_contains "$ROOT/lib/ai.sh" 'llm-ollama' -pass "LLM CLI and Graphify resolve through active provider/common endpoint" +assert_file_contains "$ROOT/lib/ai.sh" 'fastflow) printf '\''%s'\'' openai' +assert_file_contains "$ROOT/lib/ai.sh" 'ollama) printf '\''%s'\'' ollama' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_TOKEN_BUDGET:-4000' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_MAX_CONCURRENCY:-1' +pass "LLM CLI and Graphify resolve through provider-aware common endpoint" ( set -euo pipefail @@ -107,6 +111,21 @@ pass "LLM CLI and Graphify resolve through active provider/common endpoint" ) pass "Graphify validates either provider model through /v1/models" +( + set -euo pipefail + # shellcheck source=lib/ai.sh + source "$ROOT/lib/ai.sh" + + [[ "$(_graphify_backend_for_provider fastflow)" == openai ]] || + fail "FastFlow Graphify backend did not resolve to openai" + [[ "$(_graphify_backend_for_provider ollama)" == ollama ]] || + fail "Ollama Graphify backend did not resolve to ollama" + if _graphify_backend_for_provider unknown >/dev/null 2>&1; then + fail "Unknown LLM provider unexpectedly resolved a Graphify backend" + fi +) +pass "Graphify backend selection follows active LLM provider" + ( set -euo pipefail From 41a03ba1f13acc1efac35e2190e8df0e916f0ac0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 14:25:55 +0600 Subject: [PATCH 429/509] docs(ai): describe provider-aware Graphify backend --- README.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 37e842f2..5a344f51 100644 --- a/README.md +++ b/README.md @@ -277,14 +277,16 @@ lds llm chat ... # Host Graphify through the common LocalDevStack /v1 endpoint lds graphify -lds graphify ./your-project --mode deep --token-budget 4000 --max-concurrency 1 +lds graphify ./your-project --mode deep +# FastFlow already defaults to --token-budget 4000 --max-concurrency 1; +# pass explicit values only when you want to override them. ``` `lds llm` dispatches to the active provider. Ollama-only low-level commands (`ps`, `show`, `unload`, `ollama`) and FastFlow-only commands (`validate`, `check`, `flm`) are guarded and rejected when the other provider is active. Nginx owns the loopback-only native route `127.0.0.1:11434 -> nginx:11434 -> llm:11434`. Provider containers do not publish host ports. -`lds graphify` uses `http://llm.localhost:11434/v1`, validates the selected model through `/v1/models`, and then invokes Graphify's currently named `ollama` backend. The backend name is Graphify terminology; the LocalDevStack endpoint itself is provider-neutral and OpenAI-compatible. +`lds graphify` uses `http://llm.localhost:11434/v1` and validates the selected model through `/v1/models`. Backend selection follows the active provider automatically: FastFlow uses Graphify's generic `openai` backend, while Ollama uses Graphify's native `ollama` backend. For FastFlow, LocalDevStack also defaults to `--token-budget 4000 --max-concurrency 1` unless explicitly overridden; this keeps Qwen3.5 9B semantic chunks inside a safe local context budget. The built-in Compose layout keeps both provider definitions in `docker/compose/companion.yaml`, but runtime-generated profile selectors enable exactly one. NVIDIA/ROCm hardware augmentation is generated ephemerally under `docker/.runtime/`; FastFlow's `/dev/accel/accel0` + memlock contract lives in its tracked service definition. From 81b4044a8685b4109b14c42cc2c56dc17d5f3f59 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 14:25:59 +0600 Subject: [PATCH 430/509] docs(ai): document FastFlow Graphify OpenAI path --- docs/guides/local-ai.rst | 42 ++++++++++++++++++++++++++++++++-------- 1 file changed, 34 insertions(+), 8 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 6e07bfa1..bbe21023 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -234,22 +234,48 @@ service definition. Graphify -------- -``lds graphify [path]`` uses the common LocalDevStack endpoint: +``lds graphify [path]`` uses the common LocalDevStack endpoint and selects the +Graphify backend from the active provider. + +FastFlow / NPU: + +.. code-block:: text + + OPENAI_BASE_URL=http://llm.localhost:11434/v1 + OPENAI_MODEL= + OPENAI_API_KEY=local + GRAPHIFY_API_TIMEOUT= + backend=openai + default token budget=4000 + default max concurrency=1 + +Ollama / CPU, NVIDIA or ROCm: .. code-block:: text OLLAMA_BASE_URL=http://llm.localhost:11434/v1 - OLLAMA_MODEL= + OLLAMA_MODEL= OLLAMA_API_KEY=local GRAPHIFY_API_TIMEOUT= + backend=ollama + +Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the +selected model is absent. FastFlow uses Graphify's generic OpenAI-compatible backend; +Ollama continues to use Graphify's native Ollama backend. + +For FastFlow, LocalDevStack adds ``--token-budget 4000 --max-concurrency 1`` unless +the caller already supplied those flags. Override the defaults with explicit Graphify +flags, or set ``LDS_GRAPHIFY_TOKEN_BUDGET`` / +``LDS_GRAPHIFY_MAX_CONCURRENCY``. If FastFlow reports ``Max length reached!``, +reduce the token budget further, for example: + +.. code-block:: bash -Graphify currently calls this backend ``ollama``. That is Graphify terminology; the -LocalDevStack route is provider-neutral and OpenAI-compatible. Before extraction, -LocalDevStack checks ``/v1/models`` and fails fast when the selected model is absent. + lds graphify . --token-budget 3000 --max-concurrency 1 -An explicit ``OLLAMA_BASE_URL`` remains caller-controlled and bypasses the local-provider -preflight. Extraction still uses ``--backend ollama --no-cluster`` followed by -``cluster-only`` so clustering occurs once. +Explicit ``OPENAI_BASE_URL`` (FastFlow path) or ``OLLAMA_BASE_URL`` (Ollama path) +remains caller-controlled and bypasses the local-provider preflight. Extraction still +uses ``--no-cluster`` followed by ``cluster-only`` so clustering occurs once. Trust Boundary -------------- From 89fc7e4d677a6bcd2606dbb3caa80813619ab3b9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 14:26:04 +0600 Subject: [PATCH 431/509] docs(plan): finalize provider-aware Graphify flow --- .../07-localdevstack-integration-plan.md | 29 ++++++++++++------- 1 file changed, 19 insertions(+), 10 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index e162abf3..06e8765c 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -799,22 +799,31 @@ Never automatically mount arbitrary host repositories. ## 8.9 Graphify -Do not install Graphify into `llm-ollama` or Tools solely for this integration. +Do not install Graphify into either provider image or Tools solely for this integration. Supported patterns: - `lds graphify [path] [extract-options...]` -> host Graphify workflow against the - loopback-published provider, using `LDS_AI_MODEL` and `LDS_AI_TIMEOUT` by default; -- explicit host Graphify may use `https://llm-ollama.localhost/v1` or an overridden - `OLLAMA_BASE_URL`; -- container Graphify on a shared network -> `http://llm-ollama:11434/v1`; + common loopback-published `llm` endpoint, using the effective active-provider model + and `LDS_AI_TIMEOUT` by default; +- FastFlow/NPU -> Graphify `openai` backend with `OPENAI_BASE_URL` / + `OPENAI_MODEL` / `OPENAI_API_KEY`; +- Ollama/CPU/NVIDIA/ROCm -> Graphify `ollama` backend with `OLLAMA_BASE_URL` / + `OLLAMA_MODEL` / `OLLAMA_API_KEY`; +- explicit external provider URLs remain caller-controlled through the matching + backend-specific environment variables; - Tools `aiops graphify --file ` -> analyze an explicitly supplied Graphify output file. -The `lds graphify` workflow runs `extract --backend ollama --no-cluster` followed by -`cluster-only --backend ollama` so the requested two-phase flow clusters -once rather than re-clustering immediately after the extraction command's default -clustering pass. Graphify remains a host/external consumer and is not installed into -`llm-ollama` or Tools. +FastFlow defaults to `--token-budget 4000 --max-concurrency 1` unless the caller +supplies explicit values. These defaults keep Qwen3.5 9B semantic extraction below the +practical local context ceiling that can otherwise produce `Max length reached!`. +`LDS_GRAPHIFY_TOKEN_BUDGET` and `LDS_GRAPHIFY_MAX_CONCURRENCY` override the +LocalDevStack defaults. + +The `lds graphify` workflow always runs `extract --no-cluster` followed by +`cluster-only ` with the same provider-appropriate Graphify backend, so the +requested two-phase flow clusters once rather than re-clustering immediately after the +extraction command's default clustering pass. Graphify remains a host/external consumer. ## 8.10 AI admin panel From 143b03edb6503b54626d389cefd7445102a12417 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:19:03 +0600 Subject: [PATCH 432/509] refactor(ai): hardcode latest tags and common think switch --- docker/compose/companion.yaml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 19d858b4..41d3aa63 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -57,11 +57,12 @@ services: llm-ollama: container_name: LLM_OLLAMA hostname: llm-ollama - image: infocyph/llm-ollama:${LDS_LLM_ARCH} + image: infocyph/llm-ollama:latest restart: unless-stopped profiles: ["${LDS_AI_OLLAMA_PROFILE:-ai}"] environment: - LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b} + - LLM_THINK=${LDS_AI_THINK:-} - LLM_OLLAMA_SYSTEM=${LLM_OLLAMA_SYSTEM:-} - LLM_OLLAMA_INPUT_WARN_BYTES=${LLM_OLLAMA_INPUT_WARN_BYTES:-1048576} - LLM_OLLAMA_INPUT_MAX_BYTES=${LLM_OLLAMA_INPUT_MAX_BYTES:-0} @@ -92,6 +93,7 @@ services: profiles: ["${LDS_AI_FASTFLOW_PROFILE:-__lds-ai-disabled-fastflow}"] environment: - LLM_FASTFLOW_MODEL=${LDS_AI_MODEL:-qwen3.5:9b} + - LLM_THINK=${LDS_AI_THINK:-} - FLM_MODEL_PATH=/models - FLM_SERVE_PORT=11434 - FLM_HOST=0.0.0.0 From c69e5091ee4a02fa4e080fe485f9378f089c2ada Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:19:06 +0600 Subject: [PATCH 433/509] refactor(ai): remove obsolete LLM image architecture selector --- lib/platform.sh | 8 -------- 1 file changed, 8 deletions(-) diff --git a/lib/platform.sh b/lib/platform.sh index 7b96d0d7..13e0ddae 100644 --- a/lib/platform.sh +++ b/lib/platform.sh @@ -113,14 +113,6 @@ ai_igpu_default_for_runtime() { fi } -llm_arch_for_runtime() { - case "${1,,}" in - amd) printf '%s' amd-latest ;; - "" | cpu | nvidia | npu) printf '%s' latest ;; - *) return 1 ;; - esac -} - ############################################################################### # 2. INSTALL / PERMISSIONS (HOST) ############################################################################### From a6e30897728a9430fe6d41bdb3433f5d9ae6a41a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:19:09 +0600 Subject: [PATCH 434/509] refactor(ai): select Ollama tags without LDS_LLM_ARCH --- lib/compose.sh | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/lib/compose.sh b/lib/compose.sh index 155e2d19..7f62a05f 100644 --- a/lib/compose.sh +++ b/lib/compose.sh @@ -42,7 +42,7 @@ docker_compose() { local -a env_files=(--env-file "$ENV_RELEASE") [[ -r "$ENV_DOCKER" ]] && env_files+=(--env-file "$ENV_DOCKER") - local ai_runtime ai_provider ai_model llm_arch ollama_profile fastflow_profile runtime_override="" + local ai_runtime ai_provider ai_model ollama_profile fastflow_profile runtime_override="" local -a runtime_f=() ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" @@ -57,9 +57,6 @@ docker_compose() { die "Cannot resolve AI provider for runtime: $ai_runtime" ai_model="$(effective_ai_model "$ai_runtime")" || die "Cannot resolve AI model for runtime: $ai_runtime" - llm_arch="$(llm_arch_for_runtime "$ai_runtime")" || - die "Cannot resolve LLM image tag for runtime: $ai_runtime" - if [[ "$ai_provider" == "fastflow" ]]; then ollama_profile=__lds-ai-disabled-ollama fastflow_profile=ai @@ -82,7 +79,7 @@ docker_compose() { printf '%s\n' ' gpus: all' ;; amd) - printf '%s\n' ' devices:' ' - /dev/kfd:/dev/kfd' ' - /dev/dri:/dev/dri' + printf '%s\n' ' image: infocyph/llm-ollama:amd-latest' ' devices:' ' - /dev/kfd:/dev/kfd' ' - /dev/dri:/dev/dri' ;; esac } >"$runtime_override" @@ -103,7 +100,6 @@ docker_compose() { local rc=0 HOST_OS="$host_os" \ - LDS_LLM_ARCH="$llm_arch" \ LDS_AI_PROVIDER=llm \ LDS_AI_URL=http://llm:11434 \ LDS_AI_MODEL="$ai_model" \ From 24b4b770a21ea6d0681892afe91f59d07f7f0563 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:20:26 +0600 Subject: [PATCH 435/509] feat(env): add dotenv key removal helper --- lib/env.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/lib/env.sh b/lib/env.sh index 93d56339..aae6fb9c 100644 --- a/lib/env.sh +++ b/lib/env.sh @@ -97,3 +97,10 @@ update_env() { fi } + +remove_env() { + local file="$1" var="$2" + [[ -f "$file" ]] || return 0 + var="$(printf '%s' "$var" | sed 's/[]\/$*.^|[]/\\&/g')" + sed -Ei "/^[# ]*$var=/d" "$file" +} From 06f95572fdb5fd9820990ea818de51e6cde7c9ea Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:20:29 +0600 Subject: [PATCH 436/509] fix(ai): unify thinking controls and harden local Graphify --- lib/ai.sh | 128 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 102 insertions(+), 26 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 11e71b71..748499ad 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -90,6 +90,64 @@ _graphify_backend_for_provider() { esac } +_graphify_local_backend_for_provider() { + case "${1,,}" in + fastflow) printf '%s' lds-fastflow ;; + ollama) printf '%s' lds-ollama ;; + *) return 1 ;; + esac +} + +_graphify_write_local_provider() { + local dir="$1" provider="$2" base_url="$3" model="$4" token_budget="$5" + local backend num_ctx + backend="$(_graphify_local_backend_for_provider "$provider")" || return 1 + + mkdir -p "$dir/.graphify" + case "$provider" in + fastflow) + jq -n \ + --arg backend "$backend" \ + --arg base_url "$base_url" \ + --arg model "$model" \ + '{ + ($backend): { + base_url: $base_url, + default_model: $model, + env_key: "LDS_GRAPHIFY_API_KEY", + extra_body: {think: false} + } + }' >"$dir/.graphify/providers.json" + ;; + ollama) + num_ctx=$((token_budget + 8192 + 2400)) + ((num_ctx < 8192)) && num_ctx=8192 + ((num_ctx > 131072)) && num_ctx=131072 + num_ctx=$((((num_ctx + 1023) / 1024) * 1024)) + jq -n \ + --arg backend "$backend" \ + --arg base_url "$base_url" \ + --arg model "$model" \ + --argjson num_ctx "$num_ctx" \ + '{ + ($backend): { + base_url: $base_url, + default_model: $model, + env_key: "LDS_GRAPHIFY_API_KEY", + reasoning_effort: "none", + extra_body: { + options: {num_ctx: $num_ctx}, + keep_alive: "30m" + } + } + }' >"$dir/.graphify/providers.json" + ;; + *) return 1 ;; + esac + + printf '%s' "$backend" +} + cmd_graphify() { need_bin graphify "install the Graphify CLI on the host first" @@ -97,7 +155,7 @@ cmd_graphify() { [[ $# -eq 0 ]] || shift [[ -e "$target" ]] || die "Graphify target does not exist: $target" - local runtime provider backend base_url timeout model api_key graphify_bin arg + local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs local local_provider=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -206,10 +264,9 @@ cmd_graphify() { die "--max-concurrency must be a positive integer" fi - # FastFlow's Qwen3.5 9B path is OpenAI-compatible but has a smaller practical - # context budget than the old Ollama default. Keep semantic chunks conservative - # unless the caller explicitly chooses different Graphify limits. - if [[ "$provider" == fastflow ]]; then + # Local models are more reliable with conservative semantic chunking and + # serialized requests. Callers can still override both limits explicitly. + if ((local_provider)); then if ((has_token_budget == 0)); then token_budget_value="${LDS_GRAPHIFY_TOKEN_BUDGET:-4000}" [[ "$token_budget_value" =~ ^[0-9]+$ ]] && ((token_budget_value >= 1)) || @@ -227,27 +284,45 @@ cmd_graphify() { ((local_provider == 0)) || _graphify_local_model_preflight "$model" graphify_bin="$(bin_path graphify)" + target_abs="$(_realpath "$target")" + provider_dir="" + + if ((local_provider)); then + provider_dir="$(mktemp -d)" || die "Unable to create temporary Graphify provider directory" + backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$base_url" "$model" "$token_budget_value")" || { + rm -rf "$provider_dir" + die "Unable to build LocalDevStack Graphify provider configuration" + } + fi ( + [[ -z "$provider_dir" ]] || trap 'rm -rf "$provider_dir"' EXIT export GRAPHIFY_API_TIMEOUT="$timeout" - case "$backend" in - openai) - unset OLLAMA_BASE_URL OLLAMA_API_KEY OLLAMA_MODEL - export OPENAI_BASE_URL="$base_url" - export OPENAI_API_KEY="$api_key" - export OPENAI_MODEL="$model" - ;; - ollama) - unset OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL - export OLLAMA_BASE_URL="$base_url" - export OLLAMA_API_KEY="$api_key" - export OLLAMA_MODEL="$model" - ;; - esac + if ((local_provider)); then + unset OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL OLLAMA_BASE_URL OLLAMA_API_KEY OLLAMA_MODEL + export LDS_GRAPHIFY_API_KEY=local + export GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1 + cd "$provider_dir" + else + case "$backend" in + openai) + unset OLLAMA_BASE_URL OLLAMA_API_KEY OLLAMA_MODEL + export OPENAI_BASE_URL="$base_url" + export OPENAI_API_KEY="$api_key" + export OPENAI_MODEL="$model" + ;; + ollama) + unset OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL + export OLLAMA_BASE_URL="$base_url" + export OLLAMA_API_KEY="$api_key" + export OLLAMA_MODEL="$model" + ;; + esac + fi - "$graphify_bin" extract "$target" --backend "$backend" --no-cluster "${graphify_defaults[@]}" "$@" && - "$graphify_bin" cluster-only "$target" --backend "$backend" + "$graphify_bin" extract "$target_abs" --backend "$backend" --model "$model" --no-cluster "${graphify_defaults[@]}" "$@" && + "$graphify_bin" cluster-only "$target_abs" --backend "$backend" --model "$model" ) } @@ -288,16 +363,15 @@ cmd_llm() { case "${mode,,}" in auto) update_env "$ENV_DOCKER" LDS_AI_RUNTIME "" - update_env "$ENV_DOCKER" LDS_LLM_ARCH "" + remove_env "$ENV_DOCKER" LDS_LLM_ARCH update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "" local detected detected="$(detect_ai_runtime)" ok "LLM runtime set to auto; detected $detected ($(ai_provider_for_runtime "$detected")). Recreate the AI service to apply it." ;; cpu | nvidia | amd | npu) - local normalized arch igpu_enable provider image + local normalized igpu_enable provider image normalized="${mode,,}" - arch="$(llm_arch_for_runtime "$normalized")" igpu_enable="$(ai_igpu_default_for_runtime "$normalized")" provider="$(ai_provider_for_runtime "$normalized")" @@ -306,13 +380,15 @@ cmd_llm() { fi update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" - update_env "$ENV_DOCKER" LDS_LLM_ARCH "$arch" + remove_env "$ENV_DOCKER" LDS_LLM_ARCH update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable" if [[ "$provider" == "fastflow" ]]; then image="infocyph/llm-fastflow:latest" + elif [[ "$normalized" == "amd" ]]; then + image="infocyph/llm-ollama:amd-latest" else - image="infocyph/llm-ollama:$arch" + image="infocyph/llm-ollama:latest" fi ok "LLM runtime set to $normalized ($provider, $image). Recreate the AI service to apply the change." ;; From ec1da995c44f977c47333b4051f090ffca4c8696 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:21:29 +0600 Subject: [PATCH 437/509] feat(ai): add provider-neutral thinking switch --- lib/ai.sh | 33 ++++++++++++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 748499ad..4981c6cf 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -321,8 +321,8 @@ cmd_graphify() { esac fi - "$graphify_bin" extract "$target_abs" --backend "$backend" --model "$model" --no-cluster "${graphify_defaults[@]}" "$@" && - "$graphify_bin" cluster-only "$target_abs" --backend "$backend" --model "$model" + "$graphify_bin" extract "$target_abs" --backend "$backend" --no-cluster "${graphify_defaults[@]}" "$@" && + "$graphify_bin" cluster-only "$target_abs" --backend "$backend" ) } @@ -410,6 +410,32 @@ cmd_llm() { _llm_exec "${sub,,}" "$@" ;; + think) + local think_mode="${1:-}" + if [[ -z "$think_mode" ]]; then + local configured + configured="$(compose_control_value LDS_AI_THINK "")" + printf '%s\n' "${configured:-auto}" + return 0 + fi + [[ $# -eq 1 ]] || die "llm think " + case "${think_mode,,}" in + auto | default) + remove_env "$ENV_DOCKER" LDS_AI_THINK + ok "LLM thinking override cleared; provider/model default will be used after the AI service is recreated." + ;; + on | true | 1) + update_env "$ENV_DOCKER" LDS_AI_THINK true + ok "LLM thinking forced on. Recreate the AI service to apply the change." + ;; + off | false | 0) + update_env "$ENV_DOCKER" LDS_AI_THINK false + ok "LLM thinking forced off. Recreate the AI service to apply the change." + ;; + *) die "llm think " ;; + esac + ;; + provider) printf '%s\n' "$(_active_llm_provider)" ;; @@ -418,10 +444,11 @@ cmd_llm() { printf '%s\n' "llm " printf '%s\n' "llm provider" printf '%s\n' "llm runtime " + printf '%s\n' "llm think " printf '%s\n' "Ollama-only: llm " printf '%s\n' "FastFlow-only: llm " ;; - *) die "llm " ;; + *) die "llm " ;; esac } From 0e6773e8b65266784cf740d12626fe9fc3220ba0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:22:08 +0600 Subject: [PATCH 438/509] refactor(ai): drop LDS_LLM_ARCH migration references --- lib/ai.sh | 2 -- 1 file changed, 2 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 4981c6cf..245a0121 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -363,7 +363,6 @@ cmd_llm() { case "${mode,,}" in auto) update_env "$ENV_DOCKER" LDS_AI_RUNTIME "" - remove_env "$ENV_DOCKER" LDS_LLM_ARCH update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "" local detected detected="$(detect_ai_runtime)" @@ -380,7 +379,6 @@ cmd_llm() { fi update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$normalized" - remove_env "$ENV_DOCKER" LDS_LLM_ARCH update_env "$ENV_DOCKER" LDS_AI_IGPU_ENABLE "$igpu_enable" if [[ "$provider" == "fastflow" ]]; then From 3e3f389c2f380268fcc13ba87a5ec23e18203d85 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:22:11 +0600 Subject: [PATCH 439/509] feat(ai): expose provider-neutral thinking setting to consumers --- docker/compose/companion.yaml | 1 + 1 file changed, 1 insertion(+) diff --git a/docker/compose/companion.yaml b/docker/compose/companion.yaml index 41d3aa63..b442acc0 100644 --- a/docker/compose/companion.yaml +++ b/docker/compose/companion.yaml @@ -21,6 +21,7 @@ services: - LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-llm} - LDS_AI_URL=${LDS_AI_URL:-http://llm:11434} - LDS_AI_MODEL=${LDS_AI_MODEL:-qwen3:14b} + - LDS_AI_THINK=${LDS_AI_THINK:-} - LDS_AI_CONNECT_TIMEOUT=${LDS_AI_CONNECT_TIMEOUT:-2} - LDS_AI_PREFLIGHT_TIMEOUT=${LDS_AI_PREFLIGHT_TIMEOUT:-5} - LDS_AI_TIMEOUT=${LDS_AI_TIMEOUT:-1800} From d6517b64ad4ae785222c9a44a9f9d100f9df2cf1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:22:13 +0600 Subject: [PATCH 440/509] feat(ai): seed common thinking override --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index d7d76edf..84e94a8b 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.4|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm|LDS_AI_RUNTIME|LDS_AI_MODEL=|Model override (blank = provider default)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd,npu +AI|ai|Local AI|llm|LDS_AI_RUNTIME|LDS_AI_MODEL=;LDS_AI_THINK=|Model override (blank = provider default)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd,npu From 3fe0d2baea2b453afbb95c22cac6ba7a0f142789 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:22:33 +0600 Subject: [PATCH 441/509] test(ai): lock common thinking and direct image tag contracts --- tests/ai-contract.sh | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 0e696ee3..5312630d 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -51,7 +51,8 @@ if find "$ROOT/docker/compose" -maxdepth 1 -type f -name 'ai-*.yaml' -print -qui fail "AI-specific Compose overlays must be generated ephemerally" fi -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:latest' +assert_file_contains "$ROOT/lib/compose.sh" 'image: infocyph/llm-ollama:amd-latest' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-fastflow:latest' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: ["${LDS_AI_OLLAMA_PROFILE:-ai}"]' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'profiles: ["${LDS_AI_FASTFLOW_PROFILE:-__lds-ai-disabled-fastflow}"]' @@ -60,6 +61,8 @@ assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm:/root/.ollam assert_file_contains "$ROOT/docker/compose/companion.yaml" 'lds_llm_fastflow:/models' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_OLLAMA_MODEL=${LDS_AI_MODEL:-qwen3:14b}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_FASTFLOW_MODEL=${LDS_AI_MODEL:-qwen3.5:9b}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LLM_THINK=${LDS_AI_THINK:-}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_THINK=${LDS_AI_THINK:-}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'FLM_SERVE_PORT=11434' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_PROVIDER=${LDS_AI_PROVIDER:-llm}' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'LDS_AI_URL=${LDS_AI_URL:-http://llm:11434}' @@ -78,6 +81,11 @@ assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/kfd:/dev/kfd'" assert_file_contains "$ROOT/lib/compose.sh" "' - /dev/dri:/dev/dri'" pass "Compose wrapper selects exactly one provider and augments only Ollama GPU modes" +if grep -R -nF 'LDS_LLM_ARCH' "$ROOT/lib" "$ROOT/docker/compose" "$ROOT/docker/catalog"; then + fail "obsolete LDS_LLM_ARCH must not remain in active LocalDevStack code or catalog" +fi +pass "LLM image tags are direct and LDS_LLM_ARCH is removed" + assert_file_contains "$ROOT/lib/ai.sh" 'http://llm.localhost:11434/v1' assert_file_contains "$ROOT/lib/ai.sh" '/v1/models' assert_file_contains "$ROOT/lib/ai.sh" 'index($model) != null' @@ -89,6 +97,11 @@ assert_file_contains "$ROOT/lib/ai.sh" 'fastflow) printf '\''%s'\'' openai' assert_file_contains "$ROOT/lib/ai.sh" 'ollama) printf '\''%s'\'' ollama' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_TOKEN_BUDGET:-4000' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_MAX_CONCURRENCY:-1' +assert_file_contains "$ROOT/lib/ai.sh" 'lds-fastflow' +assert_file_contains "$ROOT/lib/ai.sh" 'lds-ollama' +assert_file_contains "$ROOT/lib/ai.sh" 'extra_body: {think: false}' +assert_file_contains "$ROOT/lib/ai.sh" 'reasoning_effort: "none"' +assert_file_contains "$ROOT/lib/ai.sh" 'llm think ' pass "LLM CLI and Graphify resolve through provider-aware common endpoint" ( @@ -126,6 +139,26 @@ pass "Graphify validates either provider model through /v1/models" ) pass "Graphify backend selection follows active LLM provider" +( + set -euo pipefail + # shellcheck source=lib/ai.sh + source "$ROOT/lib/ai.sh" + + tmp="$(mktemp -d)" + trap 'rm -rf "$tmp"' EXIT + + [[ "$(_graphify_write_local_provider "$tmp" fastflow http://llm.localhost:11434/v1 qwen3.5:9b 4000)" == lds-fastflow ]] || + fail "FastFlow local Graphify provider name drifted" + jq -e '."lds-fastflow".extra_body.think == false' "$tmp/.graphify/providers.json" >/dev/null || + fail "FastFlow local Graphify provider must disable thinking" + + [[ "$(_graphify_write_local_provider "$tmp" ollama http://llm.localhost:11434/v1 qwen3:14b 4000)" == lds-ollama ]] || + fail "Ollama local Graphify provider name drifted" + jq -e '."lds-ollama".reasoning_effort == "none" and ."lds-ollama".extra_body.options.num_ctx >= 8192' "$tmp/.graphify/providers.json" >/dev/null || + fail "Ollama local Graphify provider must disable thinking and retain context headroom" +) +pass "Graphify local providers enforce structured no-thinking contracts" + ( set -euo pipefail From 80bd8ce60e4e0377915bd0ce9c4e244a5822649e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:22:53 +0600 Subject: [PATCH 442/509] docs(ai): document common thinking and direct image tags --- docs/guides/local-ai.rst | 63 +++++++++++++++++++++++++++------------- 1 file changed, 43 insertions(+), 20 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index bbe21023..52126ef8 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -125,6 +125,24 @@ An explicit ``LDS_AI_MODEL`` in ``docker/.env`` overrides whichever provider is Because the providers may publish different model names, keep this blank when relying on automatic runtime switching unless the override exists in both providers. +Thinking Control +---------------- + +Thinking uses one provider-neutral LocalDevStack switch: + +.. code-block:: bash + + lds llm think # prints auto/on/off + lds llm think auto # provider/model default + lds llm think on + lds llm think off + +The persisted setting is ``LDS_AI_THINK``, which LocalDevStack maps to the common +provider variable ``LLM_THINK`` for both Ollama and FastFlow. Normal developer +commands inherit that setting. Strict structured-output paths such as ``json`` and +``lds graphify`` force thinking off so reasoning cannot displace the required JSON +payload. + Provider CLI ------------ @@ -228,8 +246,10 @@ profile selectors enable exactly one provider. There are no tracked ``ai.yaml``, ``ai-nvidia.yaml``, ``ai-amd.yaml`` or ``ai-host-port.yaml`` variants. Only Ollama NVIDIA/ROCm hardware augmentation is generated temporarily under -``docker/.runtime/``. FastFlow's XDNA2 device/memlock contract is part of its tracked -service definition. +``docker/.runtime/``. The tracked Ollama image is +``infocyph/llm-ollama:latest``; the AMD override directly selects +``infocyph/llm-ollama:amd-latest``. There is no ``LDS_LLM_ARCH`` setting. +FastFlow's XDNA2 device/memlock contract is part of its tracked service definition. Graphify -------- @@ -237,36 +257,39 @@ Graphify ``lds graphify [path]`` uses the common LocalDevStack endpoint and selects the Graphify backend from the active provider. +For the built-in LocalDevStack endpoint, Graphify runs through an ephemeral +provider definition created only for that invocation. Nothing is written into the +target repository or ``~/.graphify/providers.json``. + FastFlow / NPU: .. code-block:: text - OPENAI_BASE_URL=http://llm.localhost:11434/v1 - OPENAI_MODEL= - OPENAI_API_KEY=local - GRAPHIFY_API_TIMEOUT= - backend=openai - default token budget=4000 - default max concurrency=1 + base_url=http://llm.localhost:11434/v1 + backend=lds-fastflow + model= + extra_body={"think": false} Ollama / CPU, NVIDIA or ROCm: .. code-block:: text - OLLAMA_BASE_URL=http://llm.localhost:11434/v1 - OLLAMA_MODEL= - OLLAMA_API_KEY=local - GRAPHIFY_API_TIMEOUT= - backend=ollama + base_url=http://llm.localhost:11434/v1 + backend=lds-ollama + model= + reasoning_effort=none -Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the -selected model is absent. FastFlow uses Graphify's generic OpenAI-compatible backend; -Ollama continues to use Graphify's native Ollama backend. +The Ollama provider definition also keeps explicit context headroom for Graphify's +local chunks. Both local providers default to ``--token-budget 4000 +--max-concurrency 1`` unless the caller supplied those flags. These limits and the +no-thinking request are separate protections: the former prevents local context/resource +pressure, while the latter prevents reasoning output from producing hollow structured +responses. -For FastFlow, LocalDevStack adds ``--token-budget 4000 --max-concurrency 1`` unless -the caller already supplied those flags. Override the defaults with explicit Graphify +Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the +selected model is absent. Override the local chunk defaults with explicit Graphify flags, or set ``LDS_GRAPHIFY_TOKEN_BUDGET`` / -``LDS_GRAPHIFY_MAX_CONCURRENCY``. If FastFlow reports ``Max length reached!``, +``LDS_GRAPHIFY_MAX_CONCURRENCY``. If a local model reports ``Max length reached!``, reduce the token budget further, for example: .. code-block:: bash From 19ad7cb5f54adaffc743098e70db519d793114a8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:25:13 +0600 Subject: [PATCH 443/509] refactor(setup): remove obsolete LLM architecture env --- lib/certificates.sh | 1 - 1 file changed, 1 deletion(-) diff --git a/lib/certificates.sh b/lib/certificates.sh index 743ee0b3..cd2fd006 100644 --- a/lib/certificates.sh +++ b/lib/certificates.sh @@ -439,7 +439,6 @@ add_required_env() { ai_runtime="$(detect_ai_runtime)" update_env "$ENV_DOCKER" LDS_AI_RUNTIME "$ai_runtime" fi - update_env "$ENV_DOCKER" LDS_LLM_ARCH "$(llm_arch_for_runtime "$ai_runtime")" ai_igpu_enable="$(compose_control_value LDS_AI_IGPU_ENABLE "")" if [[ -z "$ai_igpu_enable" ]]; then From 575c6f149adcffb1253f40ac966bb21b86974546 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:25:16 +0600 Subject: [PATCH 444/509] fix(graphify): preserve external target semantics --- lib/ai.sh | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 245a0121..4a1403b5 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -266,7 +266,7 @@ cmd_graphify() { # Local models are more reliable with conservative semantic chunking and # serialized requests. Callers can still override both limits explicitly. - if ((local_provider)); then + if [[ "$provider" == fastflow ]] || ((local_provider)); then if ((has_token_budget == 0)); then token_budget_value="${LDS_GRAPHIFY_TOKEN_BUDGET:-4000}" [[ "$token_budget_value" =~ ^[0-9]+$ ]] && ((token_budget_value >= 1)) || @@ -284,10 +284,11 @@ cmd_graphify() { ((local_provider == 0)) || _graphify_local_model_preflight "$model" graphify_bin="$(bin_path graphify)" - target_abs="$(_realpath "$target")" + target_abs="$target" provider_dir="" if ((local_provider)); then + target_abs="$(_realpath "$target")" provider_dir="$(mktemp -d)" || die "Unable to create temporary Graphify provider directory" backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$base_url" "$model" "$token_budget_value")" || { rm -rf "$provider_dir" From 0f5df7eb83f75dcffa26b66f7b449058581109a1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:25:19 +0600 Subject: [PATCH 445/509] test(compose): align direct Ollama image tag contract --- tests/compose-contract.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/tests/compose-contract.sh b/tests/compose-contract.sh index d263a8e5..4e51f9c9 100755 --- a/tests/compose-contract.sh +++ b/tests/compose-contract.sh @@ -53,7 +53,6 @@ USER=$(id -un) UID=$(id -u) GID=$(id -g) PROJECT_DIR=$ROOT -LDS_LLM_ARCH=latest EOF compose=(docker compose @@ -263,7 +262,8 @@ fi if [[ -d "$ROOT/docker/.runtime" ]] && find "$ROOT/docker/.runtime" -type f -print -quit | grep -q .; then fail "temporary AI Compose overrides were not cleaned up" fi -assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' +assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-ollama:latest' +assert_file_contains "$ROOT/lib/compose.sh" 'image: infocyph/llm-ollama:amd-latest' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'image: infocyph/llm-fastflow:latest' assert_file_contains "$ROOT/docker/compose/companion.yaml" 'aliases: [llm]' pass "mutually exclusive LLM providers share one common llm network identity" From 98abe4ca63c12c7fff6afa7ac449282faf675005 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:25:43 +0600 Subject: [PATCH 446/509] test(images): align direct Ollama tag selection --- tests/published-images.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/tests/published-images.sh b/tests/published-images.sh index ab64f925..6779764e 100755 --- a/tests/published-images.sh +++ b/tests/published-images.sh @@ -59,12 +59,12 @@ for image in "$tools_image" "$runner_image"; do done pass "Tools and Runner publish healthchecks" -grep -Fq 'image: infocyph/llm-ollama:${LDS_LLM_ARCH}' "$ROOT/docker/compose/companion.yaml" || - fail "Ollama service must use the latest/amd-latest selector" +grep -Fq 'image: infocyph/llm-ollama:latest' "$ROOT/docker/compose/companion.yaml" || + fail "Ollama base service must use latest directly" +grep -Fq 'image: infocyph/llm-ollama:amd-latest' "$ROOT/lib/compose.sh" || + fail "AMD runtime override must select amd-latest directly" grep -Fq 'image: infocyph/llm-fastflow:latest' "$ROOT/docker/compose/companion.yaml" || fail "FastFlow service must use its published latest image" -grep -Fq "amd) printf '%s' amd-latest" "$ROOT/lib/platform.sh" || - fail "AMD runtime must map to Ollama amd-latest" grep -Fq "npu) printf '%s' fastflow" "$ROOT/lib/platform.sh" || fail "NPU runtime must map to FastFlow" pass "LLM image selection follows mutually exclusive provider contracts" From 110a16d80b3ad2ea07ff16dbf49c269494b0491d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:27:55 +0600 Subject: [PATCH 447/509] docs(plan): remove obsolete LLM architecture selector --- .../07-localdevstack-integration-plan.md | 19 +++++++++---------- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md index 06e8765c..0deaa487 100644 --- a/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md +++ b/docs/plans/docker-ecosystem/07-localdevstack-integration-plan.md @@ -372,9 +372,9 @@ SCRIPTOMATIC_REF=main ``` Tools, Runner, Nginx and Apache are declared directly as their moving `:latest` product -images. The LLM service is declared directly as -`infocyph/llm-ollama:${LDS_LLM_ARCH}`, where `LDS_LLM_ARCH` is derived from the effective -AI runtime (`latest` for CPU/NVIDIA, `amd-latest` for AMD/ROCm). +images. The tracked Ollama service is declared directly as +`infocyph/llm-ollama:latest`. AMD/ROCm uses a temporary runtime override that directly +selects `infocyph/llm-ollama:amd-latest`; there is no separate LLM image-tag selector. Do not add `LDS_TOOLS_IMAGE`, `LDS_RUNNER_IMAGE`, `LDS_NGINX_IMAGE`, `LDS_APACHE_IMAGE`, `LDS_LLM_IMAGE`, or `LDS_LLM_AMD_IMAGE` indirection. @@ -554,7 +554,7 @@ Base service: services: llm-ollama: container_name: LLM_OLLAMA - image: infocyph/llm-ollama:${LDS_LLM_ARCH} + image: infocyph/llm-ollama:latest restart: unless-stopped profiles: [ai] volumes: @@ -622,17 +622,17 @@ Do not create tracked runtime-variant Compose files. Keep one service in NVIDIA: -- derive `LDS_LLM_ARCH=latest`; +- keep the tracked `infocyph/llm-ollama:latest` image; - add `gpus: all` in the temporary fragment. AMD: -- derive `LDS_LLM_ARCH=amd-latest`; +- set `image: infocyph/llm-ollama:amd-latest` directly in the temporary fragment; - expose `/dev/kfd` and `/dev/dri` in the temporary fragment. -`LDS_AI_RUNTIME` is the explicit runtime selector when configured. `LDS_LLM_ARCH` -is derived from the effective runtime for Compose interpolation and is not a separate -user-facing image-version choice. Detection rules are: usable `nvidia-smi` -> NVIDIA; +`LDS_AI_RUNTIME` is the explicit runtime selector when configured. There is no +`LDS_LLM_ARCH` setting or user-facing image-version choice. Detection rules are: +usable `nvidia-smi` -> NVIDIA; both `/dev/kfd` and `/dev/dri` -> AMD/ROCm; otherwise CPU. An AMD CPU alone never selects the AMD/ROCm image. `lds llm runtime ...` remains the explicit override. @@ -1949,7 +1949,6 @@ LDS_TOOLS_IMAGE=infocyph/tools:latest LDS_RUNNER_IMAGE=infocyph/runner:latest LDS_NGINX_IMAGE=infocyph/nginx:latest LDS_APACHE_IMAGE=infocyph/apache:latest -LDS_LLM_ARCH=latest ``` Other runtime defaults follow the same rule. PostgreSQL uses `postgres:alpine`; MySQL, MariaDB, MongoDB, Redis Stack/Redis Insight, CloudBeaver, Mongo Express and Mailpit use their normal moving latest tags because the selected image family does not provide a suitable moving Alpine alias for this stack. From 70f267d1cb333c600643d936f3b68ced2c8e06ce Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:30:05 +0600 Subject: [PATCH 448/509] test(env): remove obsolete LLM architecture contract --- tests/env-contract.sh | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-) diff --git a/tests/env-contract.sh b/tests/env-contract.sh index 05a55bab..8d7dbdd9 100755 --- a/tests/env-contract.sh +++ b/tests/env-contract.sh @@ -19,7 +19,9 @@ assert_file_contains "$ROOT/lib/hosts.sh" 'var=COMPOSE_PROFILES' assert_file_contains "$ROOT/lds" 'compose_control_value()' assert_file_contains "$ROOT/lds" 'dotenv_value()' assert_file_contains "$ROOT/lib/compose.sh" 'compose_control_value LDS_AI_RUNTIME ""' -assert_file_contains "$ROOT/lib/compose.sh" 'LDS_LLM_ARCH="$llm_arch"' +if grep -RqsF 'LDS_LLM_ARCH' "$ROOT/lib" "$ROOT/docker/compose" "$ROOT/docker/catalog"; then + fail "obsolete LDS_LLM_ARCH remains in active environment wiring" +fi pass "environment file and precedence wiring" git -C "$ROOT" check-ignore -q docker/.env || fail "docker/.env must remain ignored user state" @@ -115,10 +117,6 @@ pass "CLI project identity follows the Compose project contract" set -euo pipefail has_cmd() { return 1; } source "$ROOT/lib/platform.sh" - [[ "$(llm_arch_for_runtime cpu)" == "latest" ]] || fail "CPU LLM tag drift" - [[ "$(llm_arch_for_runtime nvidia)" == "latest" ]] || fail "NVIDIA LLM tag drift" - [[ "$(llm_arch_for_runtime amd)" == "amd-latest" ]] || fail "AMD LLM tag drift" - [[ "$(llm_arch_for_runtime npu)" == "latest" ]] || fail "NPU compatibility tag drift" [[ "$(ai_provider_for_runtime cpu)" == "ollama" ]] || fail "CPU provider drift" [[ "$(ai_provider_for_runtime nvidia)" == "ollama" ]] || fail "NVIDIA provider drift" [[ "$(ai_provider_for_runtime amd)" == "ollama" ]] || fail "AMD provider drift" @@ -134,7 +132,7 @@ pass "CLI project identity follows the Compose project contract" host_cpu_is_amd() { return 1; } [[ "$(ai_igpu_default_for_runtime amd)" == "0" ]] || fail "non-AMD CPU must not auto-enable iGPU" ) -pass "LLM runtime maps mutually exclusive providers, models, tags and AMD iGPU defaults deterministically" +pass "LLM runtime maps mutually exclusive providers, models and AMD iGPU defaults deterministically" ai_env_tmp="$(mktemp -d)" ( @@ -168,7 +166,6 @@ ai_env_tmp="$(mktemp -d)" host_cpu_is_amd() { return 0; } add_required_env grep -Fxq "LDS_AI_RUNTIME=amd" "$ENV_DOCKER" || fail "detected AI runtime was not persisted" - grep -Fxq "LDS_LLM_ARCH=amd-latest" "$ENV_DOCKER" || fail "detected AMD tag was not persisted" grep -Fxq "LDS_AI_IGPU_ENABLE=1" "$ENV_DOCKER" || fail "AMD CPU iGPU preference was not persisted" update_env "$ENV_DOCKER" LDS_AI_RUNTIME nvidia @@ -176,7 +173,6 @@ ai_env_tmp="$(mktemp -d)" detect_ai_runtime() { printf "%s" amd; } add_required_env grep -Fxq "LDS_AI_RUNTIME=nvidia" "$ENV_DOCKER" || fail "explicit runtime must win over detection" - grep -Fxq "LDS_LLM_ARCH=latest" "$ENV_DOCKER" || fail "NVIDIA runtime must use standard latest tag" grep -Fxq "LDS_AI_IGPU_ENABLE=0" "$ENV_DOCKER" || fail "explicit iGPU preference must be preserved" ) rm -rf "$ai_env_tmp" From 752f6517552b1f0237ad9b4b3129d3bde56d3081 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:31:08 +0600 Subject: [PATCH 449/509] fix(catalog): pair common thinking default with setup prompt --- docker/catalog/services.psv | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docker/catalog/services.psv b/docker/catalog/services.psv index 84e94a8b..fc6ec8d6 100644 --- a/docker/catalog/services.psv +++ b/docker/catalog/services.psv @@ -5,4 +5,4 @@ MARIADB|mariadb|MariaDB|mariadb|MARIADB_VERSION|MARIADB_VERSION=latest;MARIADB_R ELASTICSEARCH|elasticsearch|Elasticsearch|elasticsearch|ELASTICSEARCH_VERSION|ELASTICSEARCH_VERSION=9.5.4|Version|kibana|ElasticSearchStore|https://kibana.localhost|search|1|0| MONGODB|mongodb|MongoDB|mongodb|MONGODB_VERSION|MONGODB_VERSION=latest;MONGODB_ROOT_USERNAME=root;MONGODB_ROOT_PASSWORD=12345|Version;Root username;Root password|mongo-express|MongoDBStore|https://me.localhost|database|1|0| REDIS|redis|Redis|redis|REDIS_VERSION|REDIS_VERSION=latest|Version|redis-insight|RedisStore|https://ri.localhost|cache|1|0| -AI|ai|Local AI|llm|LDS_AI_RUNTIME|LDS_AI_MODEL=;LDS_AI_THINK=|Model override (blank = provider default)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd,npu +AI|ai|Local AI|llm|LDS_AI_RUNTIME|LDS_AI_MODEL=;LDS_AI_THINK=|Model override (blank = provider default);Thinking override (blank = provider default, true/false)||LLMModels|https://llm.localhost|ai|1|0|cpu,nvidia,amd,npu From 8e1086304be99b97a6b124c066b66daa66169bb8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 15:58:54 +0600 Subject: [PATCH 450/509] test(catalog): allow automatic AI thinking default --- tests/catalog-contract.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/tests/catalog-contract.sh b/tests/catalog-contract.sh index 3c4b7042..9d1aceb3 100644 --- a/tests/catalog-contract.sh +++ b/tests/catalog-contract.sh @@ -51,7 +51,7 @@ while IFS='|' read -r key profile display service_key version_env defaults promp [[ "$profile" == "ai" ]] || fail "AI profile must be ai" [[ "$service_key" == "llm" ]] || fail "AI service identity must be provider-neutral llm" [[ "$runtime_modes" == "cpu,nvidia,amd,npu" ]] || fail "AI runtime metadata drift" - [[ "$defaults" == "LDS_AI_MODEL=" ]] || fail "AI setup must leave provider model default automatic" + [[ "$defaults" == "LDS_AI_MODEL=;LDS_AI_THINK=" ]] || fail "AI setup must leave provider model/thinking defaults automatic" [[ "$version_env" == "LDS_AI_RUNTIME" ]] || fail "AI selector metadata must be LDS_AI_RUNTIME" [[ "$url" == "https://llm.localhost" ]] || fail "AI convenience URL must use common llm identity" ;; From 8a18bea8ccb89fa51724af53ca70bd1525dfdb7c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:00:50 +0600 Subject: [PATCH 451/509] fix(images): use direct LLM image tags --- lib/diagnostics.sh | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/lib/diagnostics.sh b/lib/diagnostics.sh index 69afeba8..8cd81a67 100644 --- a/lib/diagnostics.sh +++ b/lib/diagnostics.sh @@ -426,15 +426,16 @@ cmd_images() { printf '%-16s %s\n' "Runner" "infocyph/runner:latest" printf '%-16s %s\n' "Nginx" "infocyph/nginx:latest" printf '%-16s %s\n' "Apache" "infocyph/apache:latest" - local ai_runtime ai_provider llm_arch llm_image + local ai_runtime ai_provider llm_image ai_runtime="$(compose_control_value LDS_AI_RUNTIME "")" [[ -n "$ai_runtime" ]] || ai_runtime="$(detect_ai_runtime)" ai_provider="$(ai_provider_for_runtime "$ai_runtime")" if [[ "$ai_provider" == "fastflow" ]]; then llm_image="infocyph/llm-fastflow:latest" + elif [[ "$ai_runtime" == "amd" ]]; then + llm_image="infocyph/llm-ollama:amd-latest" else - llm_arch="$(llm_arch_for_runtime "$ai_runtime")" - llm_image="infocyph/llm-ollama:$llm_arch" + llm_image="infocyph/llm-ollama:latest" fi printf '%-16s %s\n' "LLM" "$llm_image" printf '%-16s %s\n' "LLM provider" "$ai_provider" From 5f0ecb1205b961147cc730ce907b321daf205530 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:02:14 +0600 Subject: [PATCH 452/509] test(docs): assert provider-aware no-thinking Graphify contract --- tests/docs-contract.sh | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 3ff98d87..5d8cd128 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -84,7 +84,15 @@ assert_file_contains "$ai" 'lds logs llm' assert_file_contains "$cli" 'lds restart llm' assert_file_contains "$cli" 'lds exec llm' assert_file_contains "$cli" 'lds rebuild llm' -assert_file_contains "$ai" 'OLLAMA_BASE_URL=http://llm.localhost:11434/v1' +assert_file_contains "$ai" 'backend=lds-fastflow' +assert_file_contains "$ai" 'backend=lds-ollama' +assert_file_contains "$ai" 'extra_body={"think": false}' +assert_file_contains "$ai" 'reasoning_effort=none' +assert_file_contains "$ai" 'lds llm think' +assert_file_contains "$ai" 'LDS_AI_THINK' +if grep -RqsF 'LDS_LLM_ARCH' "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then + fail "user-facing docs expose removed LDS_LLM_ARCH setting" +fi assert_file_contains "$profiles" 'LDS_AI_PROVIDER=llm' assert_file_contains "$profiles" 'LDS_AI_URL=http://llm:11434' assert_file_contains "$profiles" 'infocyph/llm-fastflow:latest' From f100ef272b3179026f64904aa62df728f5d60bef Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:03:44 +0600 Subject: [PATCH 453/509] docs(ai): remove obsolete LLM tag variable wording --- docs/concepts/profiles-and-env.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/concepts/profiles-and-env.rst b/docs/concepts/profiles-and-env.rst index 6d700c64..5c24e4a6 100644 --- a/docs/concepts/profiles-and-env.rst +++ b/docs/concepts/profiles-and-env.rst @@ -246,9 +246,9 @@ Use:: to control the runtime explicitly. ``auto`` clears the explicit override. -``LDS_LLM_ARCH`` remains a derived Ollama compatibility tag (``latest`` or -``amd-latest``); it is not the FastFlow selector and should not be managed as a generic -LLM version field. +Ollama image tags are selected directly from the effective runtime: CPU/NVIDIA use +``infocyph/llm-ollama:latest``, while AMD/ROCm uses +``infocyph/llm-ollama:amd-latest``. New setup leaves ``LDS_AI_MODEL`` blank so the active provider default applies. An explicit model override is forwarded to both Tools and the active provider. Keep it blank From c71a210de15745edfb5b1bddbef595ef3ee85b2e Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:03:47 +0600 Subject: [PATCH 454/509] docs(ai): remove obsolete LLM tag variable reference --- docs/guides/local-ai.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 52126ef8..d3cecda9 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -248,7 +248,7 @@ profile selectors enable exactly one provider. There are no tracked ``ai.yaml``, Only Ollama NVIDIA/ROCm hardware augmentation is generated temporarily under ``docker/.runtime/``. The tracked Ollama image is ``infocyph/llm-ollama:latest``; the AMD override directly selects -``infocyph/llm-ollama:amd-latest``. There is no ``LDS_LLM_ARCH`` setting. +``infocyph/llm-ollama:amd-latest``. FastFlow's XDNA2 device/memlock contract is part of its tracked service definition. Graphify From 997c4748695fc908e8d3c8d7599aa9a0c8ecce39 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:49:59 +0600 Subject: [PATCH 455/509] feat(graphify): add suspect-response diagnostic proxy --- scripts/graphify-diagnostic-proxy.py | 294 +++++++++++++++++++++++++++ 1 file changed, 294 insertions(+) create mode 100644 scripts/graphify-diagnostic-proxy.py diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py new file mode 100644 index 00000000..43788163 --- /dev/null +++ b/scripts/graphify-diagnostic-proxy.py @@ -0,0 +1,294 @@ +#!/usr/bin/env python3 +"""Local Graphify diagnostic reverse proxy. + +Forwards OpenAI-compatible Graphify traffic to the active LocalDevStack LLM +without modifying request bodies. Only suspect chat-completion responses are +reported: empty content, malformed/non-graph JSON, or graph arrays containing +no usable object entries. Request prompts/source content are never logged. +""" +from __future__ import annotations + +import argparse +import json +import re +import sys +import urllib.error +import urllib.request +from datetime import datetime, timezone +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any + +_GRAPH_KEYS = ("nodes", "edges", "hyperedges") +_FENCE_RE = re.compile(r"~~~[ \t]*([A-Za-z0-9_+-]*)[ \t]*\r?\n(.*?)~~~", re.S) + + +def _balanced_object(text: str, start: int) -> str | None: + depth = 0 + in_string = False + escape = False + for index in range(start, len(text)): + char = text[index] + if escape: + escape = False + continue + if char == "\\": + escape = True + continue + if char == '"': + in_string = not in_string + continue + if in_string: + continue + if char == "{": + depth += 1 + elif char == "}": + depth -= 1 + if depth == 0: + return text[start:index + 1] + return None + + +def _json_candidates(content: str): + stripped = content.strip() + if stripped: + yield stripped + # Avoid embedding Markdown fence delimiters in this helper's own source + # contract; normalize them before matching. + fenced = content.replace(chr(96) * 3, "~~~") + for _lang, body in _FENCE_RE.findall(fenced): + body = body.strip() + if body: + yield body + start = content.find("{") + seen = 0 + while start != -1 and seen < 128: + candidate = _balanced_object(content, start) + if candidate is not None: + yield candidate + seen += 1 + start = content.find("{", start + 1) + + +def classify_graph_content(content: str | None) -> tuple[bool, str]: + """Return (suspect, reason), mirroring Graphify's hollow decision closely.""" + if content is None or not content.strip(): + return True, "empty assistant content" + + saw_json = False + saw_graph_shape = False + saw_wrong_shape = False + + for candidate in _json_candidates(content): + try: + parsed = json.loads(candidate) + except (json.JSONDecodeError, TypeError): + continue + if not isinstance(parsed, dict): + continue + + saw_json = True + if not any(key in parsed for key in _GRAPH_KEYS): + continue + + saw_graph_shape = True + usable = False + wrong_shape = False + for key in _GRAPH_KEYS: + value = parsed.get(key) + if value is None: + continue + if not isinstance(value, list): + wrong_shape = True + continue + if any(isinstance(entry, dict) for entry in value): + usable = True + if value and not any(isinstance(entry, dict) for entry in value): + wrong_shape = True + + if usable: + return False, "usable graph fragment" + saw_wrong_shape = saw_wrong_shape or wrong_shape + + if saw_wrong_shape: + return True, "graph arrays contain no usable object entries" + if saw_graph_shape: + return True, "valid but empty graph fragment" + if saw_json: + return True, "JSON response has no graph fragment keys" + return True, "response is not parseable as a graph JSON object" + + +def _request_metadata(body: bytes) -> dict[str, Any]: + try: + request = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return {} + if not isinstance(request, dict): + return {} + return { + "model": request.get("model"), + "think": request.get("think", ""), + "reasoning_effort": request.get("reasoning_effort", ""), + "max_completion_tokens": request.get("max_completion_tokens", request.get("max_tokens")), + "stream": request.get("stream"), + } + + +def _response_metadata(body: bytes) -> tuple[dict[str, Any], str | None]: + try: + response = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return {}, None + if not isinstance(response, dict): + return {}, None + + choices = response.get("choices") + choice = choices[0] if isinstance(choices, list) and choices and isinstance(choices[0], dict) else {} + message = choice.get("message") if isinstance(choice.get("message"), dict) else {} + usage = response.get("usage") if isinstance(response.get("usage"), dict) else {} + + return { + "finish_reason": choice.get("finish_reason"), + "prompt_tokens": usage.get("prompt_tokens"), + "completion_tokens": usage.get("completion_tokens"), + "total_tokens": usage.get("total_tokens"), + }, message.get("content") if isinstance(message, dict) else None + + +class DiagnosticHandler(BaseHTTPRequestHandler): + protocol_version = "HTTP/1.1" + upstream: str = "" + log_file: Path + preview_chars: int = 4096 + + def log_message(self, _format: str, *_args: object) -> None: + return + + def _forward(self) -> None: + length = int(self.headers.get("Content-Length", "0") or "0") + body = self.rfile.read(length) if length else b"" + + request_headers = {} + for name in ("Authorization", "Content-Type", "Accept", "User-Agent"): + value = self.headers.get(name) + if value: + request_headers[name] = value + + upstream_url = self.upstream.rstrip("/") + self.path + request = urllib.request.Request( + upstream_url, + data=body if self.command not in ("GET", "HEAD") else None, + headers=request_headers, + method=self.command, + ) + + status = 502 + response_headers: dict[str, str] = {"Content-Type": "application/json"} + response_body = b'{"error":{"message":"LocalDevStack Graphify diagnostic proxy upstream failure"}}' + + try: + with urllib.request.urlopen(request, timeout=1900) as upstream_response: + status = upstream_response.status + response_body = upstream_response.read() + content_type = upstream_response.headers.get("Content-Type") + if content_type: + response_headers["Content-Type"] = content_type + except urllib.error.HTTPError as exc: + status = exc.code + response_body = exc.read() + content_type = exc.headers.get("Content-Type") if exc.headers else None + if content_type: + response_headers["Content-Type"] = content_type + except Exception as exc: + response_body = json.dumps( + {"error": {"message": f"Graphify diagnostic proxy upstream error: {exc}"}} + ).encode("utf-8") + + if self.command == "POST" and self.path.rstrip("/").endswith("/v1/chat/completions"): + self._inspect_chat_response(body, response_body, status) + + self.send_response(status) + for name, value in response_headers.items(): + self.send_header(name, value) + self.send_header("Content-Length", str(len(response_body))) + self.send_header("Connection", "close") + self.end_headers() + if self.command != "HEAD": + self.wfile.write(response_body) + + def _inspect_chat_response(self, request_body: bytes, response_body: bytes, status: int) -> None: + req = _request_metadata(request_body) + resp, content = _response_metadata(response_body) + if status < 200 or status >= 300: + return + + suspect, reason = classify_graph_content(content) + if not suspect: + return + + record = { + "timestamp": datetime.now(timezone.utc).isoformat(), + "status": status, + "reason": reason, + **req, + **resp, + "assistant_content": content, + } + self.log_file.parent.mkdir(parents=True, exist_ok=True) + with self.log_file.open("a", encoding="utf-8") as handle: + handle.write(json.dumps(record, ensure_ascii=False) + "\n") + + preview = (content or "")[: self.preview_chars] + print( + "[lds graphify diagnostic] suspect LLM response: " + f"reason={reason}; model={req.get('model')}; " + f"think={req.get('think')}; reasoning_effort={req.get('reasoning_effort')}; " + f"finish_reason={resp.get('finish_reason')}; " + f"prompt_tokens={resp.get('prompt_tokens')}; " + f"completion_tokens={resp.get('completion_tokens')}", + file=sys.stderr, + flush=True, + ) + print( + f"[lds graphify diagnostic] assistant content preview ({len(preview)}/{len(content or '')} chars): " + f"{preview!r}", + file=sys.stderr, + flush=True, + ) + print( + f"[lds graphify diagnostic] full suspect response logged to {self.log_file}", + file=sys.stderr, + flush=True, + ) + + do_GET = _forward + do_HEAD = _forward + do_POST = _forward + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--upstream", required=True) + parser.add_argument("--ready-file", required=True) + parser.add_argument("--log-file", required=True) + parser.add_argument("--preview-chars", type=int, default=4096) + args = parser.parse_args() + + DiagnosticHandler.upstream = args.upstream + DiagnosticHandler.log_file = Path(args.log_file) + DiagnosticHandler.preview_chars = max(256, args.preview_chars) + + server = ThreadingHTTPServer(("127.0.0.1", 0), DiagnosticHandler) + ready = Path(args.ready_file) + ready.parent.mkdir(parents=True, exist_ok=True) + ready.write_text(str(server.server_address[1]), encoding="utf-8") + try: + server.serve_forever(poll_interval=0.2) + finally: + server.server_close() + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) From 4f86d350f7d4a838860ec7317659ff0aae31e3e3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:50:38 +0600 Subject: [PATCH 456/509] fix(graphify): inspect extraction responses only --- scripts/graphify-diagnostic-proxy.py | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 43788163..c7eb6cf9 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -126,7 +126,20 @@ def _request_metadata(body: bytes) -> dict[str, Any]: return {} if not isinstance(request, dict): return {} + + messages = request.get("messages") + extraction_request = False + if isinstance(messages, list): + for message in messages: + if not isinstance(message, dict) or message.get("role") != "system": + continue + content = message.get("content") + if isinstance(content, str) and "graphify semantic extraction agent" in content: + extraction_request = True + break + return { + "_extraction_request": extraction_request, "model": request.get("model"), "think": request.get("think", ""), "reasoning_effort": request.get("reasoning_effort", ""), @@ -220,7 +233,7 @@ def _forward(self) -> None: def _inspect_chat_response(self, request_body: bytes, response_body: bytes, status: int) -> None: req = _request_metadata(request_body) resp, content = _response_metadata(response_body) - if status < 200 or status >= 300: + if status < 200 or status >= 300 or not req.pop("_extraction_request", False): return suspect, reason = classify_graph_content(content) From 310fa8fc8b71de30277f6b2e0d44eabc001c6d00 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:51:02 +0600 Subject: [PATCH 457/509] feat(graphify): capture suspect local LLM responses --- lib/ai.sh | 95 ++++++++++++++++++++++++++++++++++++++++++++++++++++--- 1 file changed, 90 insertions(+), 5 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 4a1403b5..171e87f4 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -148,6 +148,38 @@ _graphify_write_local_provider() { printf '%s' "$backend" } +_graphify_python_bin() { + local graphify_bin="${1:-}" first_line="" candidate="" + + if [[ -n "$graphify_bin" && -f "$graphify_bin" ]]; then + IFS= read -r first_line <"$graphify_bin" || true + if [[ "$first_line" == '#!'* ]]; then + candidate="${first_line#\#!}" + candidate="${candidate%% *}" + if [[ -x "$candidate" ]]; then + printf '%s' "$candidate" + return 0 + fi + fi + fi + + for candidate in python3 python; do + if type -P -- "$candidate" >/dev/null 2>&1; then + type -P -- "$candidate" + return 0 + fi + done + return 1 +} + +_graphify_diagnostics_enabled() { + case "${LDS_GRAPHIFY_DIAGNOSTICS:-1}" in + 1 | true | TRUE | yes | YES | on | ON) return 0 ;; + 0 | false | FALSE | no | NO | off | OFF) return 1 ;; + *) die "LDS_GRAPHIFY_DIAGNOSTICS must be true/false" ;; + esac +} + cmd_graphify() { need_bin graphify "install the Graphify CLI on the host first" @@ -156,6 +188,7 @@ cmd_graphify() { [[ -e "$target" ]] || die "Graphify target does not exist: $target" local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs + local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="" local local_provider=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -290,17 +323,69 @@ cmd_graphify() { if ((local_provider)); then target_abs="$(_realpath "$target")" provider_dir="$(mktemp -d)" || die "Unable to create temporary Graphify provider directory" - backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$base_url" "$model" "$token_budget_value")" || { - rm -rf "$provider_dir" - die "Unable to build LocalDevStack Graphify provider configuration" - } + + if _graphify_diagnostics_enabled; then + graphify_python="$(_graphify_python_bin "$graphify_bin")" || + die "Unable to find the Python interpreter required for Graphify diagnostics" + if [[ -d "$target_abs" ]]; then + diagnostic_root="$target_abs" + else + diagnostic_root="$(dirname "$target_abs")" + fi + diagnostic_log="${LDS_GRAPHIFY_DIAGNOSTIC_LOG:-$diagnostic_root/graphify-out/lds-graphify-diagnostics.jsonl}" + diagnostic_preview="${LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW:-4096}" + [[ "$diagnostic_preview" =~ ^[0-9]+$ ]] && ((diagnostic_preview >= 256)) || + die "LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW must be an integer >= 256" + fi fi ( - [[ -z "$provider_dir" ]] || trap 'rm -rf "$provider_dir"' EXIT + proxy_pid="" + cleanup_graphify_local() { + if [[ -n "${proxy_pid:-}" ]]; then + kill "$proxy_pid" >/dev/null 2>&1 || true + wait "$proxy_pid" >/dev/null 2>&1 || true + fi + [[ -z "$provider_dir" ]] || rm -rf "$provider_dir" + } + [[ -z "$provider_dir" ]] || trap cleanup_graphify_local EXIT export GRAPHIFY_API_TIMEOUT="$timeout" if ((local_provider)); then + local_provider_base_url="$base_url" + + if [[ -n "$graphify_python" ]]; then + ready_file="$provider_dir/diagnostic-proxy.port" + mkdir -p "$(dirname "$diagnostic_log")" + : >"$diagnostic_log" + + "$graphify_python" "$DIR/scripts/graphify-diagnostic-proxy.py" \ + --upstream "${base_url%/v1}" \ + --ready-file "$ready_file" \ + --log-file "$diagnostic_log" \ + --preview-chars "$diagnostic_preview" & + proxy_pid=$! + + proxy_port="" + for _ in {1..100}; do + if [[ -s "$ready_file" ]]; then + proxy_port="$(cat "$ready_file")" + break + fi + kill -0 "$proxy_pid" >/dev/null 2>&1 || + die "Local Graphify diagnostic proxy exited before becoming ready" + sleep 0.05 + done + [[ "$proxy_port" =~ ^[0-9]+$ ]] || + die "Local Graphify diagnostic proxy did not become ready" + + local_provider_base_url="http://127.0.0.1:${proxy_port}/v1" + printf '%s\n' "[lds graphify] suspect-response diagnostics enabled: $diagnostic_log" >&2 + fi + + backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$local_provider_base_url" "$model" "$token_budget_value")" || + die "Unable to build LocalDevStack Graphify provider configuration" + unset OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL OLLAMA_BASE_URL OLLAMA_API_KEY OLLAMA_MODEL export LDS_GRAPHIFY_API_KEY=local export GRAPHIFY_ALLOW_LOCAL_PROVIDERS=1 From 39db7d1fecefe86765c301f1f0dada93858c3bfb Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:51:26 +0600 Subject: [PATCH 458/509] test(graphify): cover suspect-response diagnostics --- tests/ai-contract.sh | 53 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 53 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 5312630d..a13a1e99 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -101,6 +101,10 @@ assert_file_contains "$ROOT/lib/ai.sh" 'lds-fastflow' assert_file_contains "$ROOT/lib/ai.sh" 'lds-ollama' assert_file_contains "$ROOT/lib/ai.sh" 'extra_body: {think: false}' assert_file_contains "$ROOT/lib/ai.sh" 'reasoning_effort: "none"' +assert_file_contains "$ROOT/lib/ai.sh" 'graphify-diagnostic-proxy.py' +assert_file_contains "$ROOT/lib/ai.sh" 'http://127.0.0.1:${proxy_port}/v1' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-1' +assert_file_contains "$ROOT/lib/ai.sh" 'lds-graphify-diagnostics.jsonl' assert_file_contains "$ROOT/lib/ai.sh" 'llm think ' pass "LLM CLI and Graphify resolve through provider-aware common endpoint" @@ -159,6 +163,55 @@ pass "Graphify backend selection follows active LLM provider" ) pass "Graphify local providers enforce structured no-thinking contracts" +python3 -m py_compile "$ROOT/scripts/graphify-diagnostic-proxy.py" +python3 - "$ROOT/scripts/graphify-diagnostic-proxy.py" <<'PY' +import importlib.util +import json +import sys + +path = sys.argv[1] +spec = importlib.util.spec_from_file_location("lds_graphify_diagnostic_proxy", path) +module = importlib.util.module_from_spec(spec) +assert spec and spec.loader +spec.loader.exec_module(module) + +suspect, reason = module.classify_graph_content('{"nodes":[],"edges":[],"hyperedges":[]}') +assert suspect and reason == "valid but empty graph fragment" + +suspect, reason = module.classify_graph_content('{"nodes":["A","B"],"edges":[]}') +assert suspect and "no usable object entries" in reason + +suspect, _ = module.classify_graph_content( + '{"nodes":[{"id":"a","label":"A"}],"edges":[],"hyperedges":[]}' +) +assert not suspect + +fence = chr(96) * 3 +suspect, _ = module.classify_graph_content( + "answer follows\n" + fence + "json\n" + + '{"nodes":[{"id":"a"}],"edges":[]}' + "\n" + fence +) +assert not suspect + +suspect, reason = module.classify_graph_content("I found nothing useful in these documents.") +assert suspect and "not parseable" in reason + +body = json.dumps({ + "model": "qwen3.5:9b", + "messages": [ + {"role": "system", "content": "You are a graphify semantic extraction agent."}, + {"role": "user", "content": "private corpus content"} + ], + "think": False, + "stream": False +}).encode() +metadata = module._request_metadata(body) +assert metadata["_extraction_request"] is True +assert metadata["think"] is False +assert "private corpus content" not in json.dumps(metadata) +PY +pass "Graphify diagnostic proxy identifies suspect responses without logging prompts" + ( set -euo pipefail From 04b12148d1a8f0858ee3165151b3640889d19c4d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:52:17 +0600 Subject: [PATCH 459/509] fix(graphify): bound non-text diagnostic responses --- scripts/graphify-diagnostic-proxy.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index c7eb6cf9..bee8fd20 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -161,12 +161,16 @@ def _response_metadata(body: bytes) -> tuple[dict[str, Any], str | None]: message = choice.get("message") if isinstance(choice.get("message"), dict) else {} usage = response.get("usage") if isinstance(response.get("usage"), dict) else {} + content = message.get("content") if isinstance(message, dict) else None + if not isinstance(content, str): + content = None + return { "finish_reason": choice.get("finish_reason"), "prompt_tokens": usage.get("prompt_tokens"), "completion_tokens": usage.get("completion_tokens"), "total_tokens": usage.get("total_tokens"), - }, message.get("content") if isinstance(message, dict) else None + }, content class DiagnosticHandler(BaseHTTPRequestHandler): From efcdc880090fa8a8df2ef7fe79b2013415096de5 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:52:20 +0600 Subject: [PATCH 460/509] fix(graphify): resolve diagnostic Python safely --- lib/ai.sh | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 171e87f4..8421dcf2 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -151,24 +151,24 @@ _graphify_write_local_provider() { _graphify_python_bin() { local graphify_bin="${1:-}" first_line="" candidate="" + for candidate in python3 python; do + if type -P -- "$candidate" >/dev/null 2>&1; then + type -P -- "$candidate" + return 0 + fi + done + if [[ -n "$graphify_bin" && -f "$graphify_bin" ]]; then IFS= read -r first_line <"$graphify_bin" || true if [[ "$first_line" == '#!'* ]]; then candidate="${first_line#\#!}" candidate="${candidate%% *}" - if [[ -x "$candidate" ]]; then + if [[ -x "$candidate" && "${candidate##*/}" == python* ]]; then printf '%s' "$candidate" return 0 fi fi fi - - for candidate in python3 python; do - if type -P -- "$candidate" >/dev/null 2>&1; then - type -P -- "$candidate" - return 0 - fi - done return 1 } From 33a3edbf98f947b2f70227b21c6390f2130ffe6d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:52:37 +0600 Subject: [PATCH 461/509] docs(graphify): explain suspect-response diagnostics --- docs/guides/local-ai.rst | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index d3cecda9..45da6680 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -258,14 +258,18 @@ Graphify Graphify backend from the active provider. For the built-in LocalDevStack endpoint, Graphify runs through an ephemeral -provider definition created only for that invocation. Nothing is written into the -target repository or ``~/.graphify/providers.json``. +provider definition created only for that invocation. The provider definition itself +is never written into the target repository or ``~/.graphify/providers.json``. + +Local Graphify runs enable a localhost diagnostic proxy by default. The temporary +provider points to ``127.0.0.1:/v1``, and the proxy forwards the request +body unchanged to ``http://llm.localhost:11434/v1``. It inspects semantic-extraction +responses only; community-label requests are ignored. FastFlow / NPU: .. code-block:: text - base_url=http://llm.localhost:11434/v1 backend=lds-fastflow model= extra_body={"think": false} @@ -274,7 +278,6 @@ Ollama / CPU, NVIDIA or ROCm: .. code-block:: text - base_url=http://llm.localhost:11434/v1 backend=lds-ollama model= reasoning_effort=none @@ -283,8 +286,22 @@ The Ollama provider definition also keeps explicit context headroom for Graphify local chunks. Both local providers default to ``--token-budget 4000 --max-concurrency 1`` unless the caller supplied those flags. These limits and the no-thinking request are separate protections: the former prevents local context/resource -pressure, while the latter prevents reasoning output from producing hollow structured -responses. +pressure, while the latter keeps reasoning out of the structured response channel. + +When a semantic response itself looks suspect (empty content, valid-but-empty graph +JSON, malformed/non-graph JSON, or graph arrays with no object entries), LocalDevStack +prints a bounded assistant-content preview and stores the full suspect assistant +response as JSON Lines in: + +.. code-block:: text + + /graphify-out/lds-graphify-diagnostics.jsonl + +The diagnostic record contains request controls such as model, ``think``, +``reasoning_effort``, finish reason, and token usage, but never stores the Graphify +prompt or source corpus. Set ``LDS_GRAPHIFY_DIAGNOSTICS=0`` to bypass the proxy. +``LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW`` controls the terminal preview size (minimum 256, +default 4096). ``LDS_GRAPHIFY_DIAGNOSTIC_LOG`` overrides the JSONL path. Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the selected model is absent. Override the local chunk defaults with explicit Graphify From 9a83b562fe78b15207c46c0438399cb5c4ea4627 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 16:52:52 +0600 Subject: [PATCH 462/509] test(docs): lock Graphify diagnostic guidance --- tests/docs-contract.sh | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index 5d8cd128..dd7f6637 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -90,6 +90,10 @@ assert_file_contains "$ai" 'extra_body={"think": false}' assert_file_contains "$ai" 'reasoning_effort=none' assert_file_contains "$ai" 'lds llm think' assert_file_contains "$ai" 'LDS_AI_THINK' +assert_file_contains "$ai" 'LDS_GRAPHIFY_DIAGNOSTICS=0' +assert_file_contains "$ai" 'lds-graphify-diagnostics.jsonl' +assert_file_contains "$ai" 'think' +assert_file_contains "$ai" 'reasoning_effort' if grep -RqsF 'LDS_LLM_ARCH' "$ROOT/README.md" "$ROOT/docs" --exclude-dir=plans; then fail "user-facing docs expose removed LDS_LLM_ARCH setting" fi From d94d479949459f5b77d675011045bfb662c6bb5a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:08:44 +0600 Subject: [PATCH 463/509] fix(graphify): classify malformed graph JSON explicitly --- scripts/graphify-diagnostic-proxy.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index bee8fd20..a3959ff0 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -114,6 +114,8 @@ def classify_graph_content(content: str | None) -> tuple[bool, str]: return True, "graph arrays contain no usable object entries" if saw_graph_shape: return True, "valid but empty graph fragment" + if any(f'"{key}"' in content for key in _GRAPH_KEYS): + return True, "malformed graph JSON" if saw_json: return True, "JSON response has no graph fragment keys" return True, "response is not parseable as a graph JSON object" From 82ba0ac16e7726aab31475b23bfae2eb618f25fe Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:08:48 +0600 Subject: [PATCH 464/509] test(graphify): identify malformed graph JSON --- tests/ai-contract.sh | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index a13a1e99..3f483196 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -196,6 +196,11 @@ assert not suspect suspect, reason = module.classify_graph_content("I found nothing useful in these documents.") assert suspect and "not parseable" in reason +suspect, reason = module.classify_graph_content( + '{"nodes":[{"id":"a","source_file:".github/x.md"}],"edges":[]}' +) +assert suspect and reason == "malformed graph JSON" + body = json.dumps({ "model": "qwen3.5:9b", "messages": [ From 08ae9a39f9d0b9358504d86114c7112d30a5681d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:09:14 +0600 Subject: [PATCH 465/509] feat(graphify): add thinking A/B override --- lib/ai.sh | 70 +++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 55 insertions(+), 15 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 8421dcf2..eb3461ab 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -99,25 +99,58 @@ _graphify_local_backend_for_provider() { } _graphify_write_local_provider() { - local dir="$1" provider="$2" base_url="$3" model="$4" token_budget="$5" + local dir="$1" provider="$2" base_url="$3" model="$4" token_budget="$5" think_mode="${6:-off}" local backend num_ctx backend="$(_graphify_local_backend_for_provider "$provider")" || return 1 mkdir -p "$dir/.graphify" case "$provider" in fastflow) - jq -n \ - --arg backend "$backend" \ - --arg base_url "$base_url" \ - --arg model "$model" \ - '{ - ($backend): { - base_url: $base_url, - default_model: $model, - env_key: "LDS_GRAPHIFY_API_KEY", - extra_body: {think: false} - } - }' >"$dir/.graphify/providers.json" + case "$think_mode" in + off) + jq -n \ + --arg backend "$backend" \ + --arg base_url "$base_url" \ + --arg model "$model" \ + '{ + ($backend): { + base_url: $base_url, + default_model: $model, + env_key: "LDS_GRAPHIFY_API_KEY", + extra_body: {think: false} + } + }' >"$dir/.graphify/providers.json" + ;; + on) + jq -n \ + --arg backend "$backend" \ + --arg base_url "$base_url" \ + --arg model "$model" \ + '{ + ($backend): { + base_url: $base_url, + default_model: $model, + env_key: "LDS_GRAPHIFY_API_KEY", + reasoning_effort: "high", + extra_body: {think: true} + } + }' >"$dir/.graphify/providers.json" + ;; + auto) + jq -n \ + --arg backend "$backend" \ + --arg base_url "$base_url" \ + --arg model "$model" \ + '{ + ($backend): { + base_url: $base_url, + default_model: $model, + env_key: "LDS_GRAPHIFY_API_KEY" + } + }' >"$dir/.graphify/providers.json" + ;; + *) return 1 ;; + esac ;; ollama) num_ctx=$((token_budget + 8192 + 2400)) @@ -188,7 +221,7 @@ cmd_graphify() { [[ -e "$target" ]] || die "Graphify target does not exist: $target" local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs - local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="" + local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="" graphify_think="off" local local_provider=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -225,6 +258,13 @@ cmd_graphify() { timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" + case "${LDS_GRAPHIFY_THINK:-off}" in + off | false | 0) graphify_think=off ;; + on | true | 1) graphify_think=on ;; + auto | default) graphify_think=auto ;; + *) die "LDS_GRAPHIFY_THINK must be off/on/auto" ;; + esac + # Keep explicit model/timeout/resource overrides consistent while LocalDevStack # retains ownership of backend selection and the two-stage extract/cluster flow. for arg in "$@"; do @@ -383,7 +423,7 @@ cmd_graphify() { printf '%s\n' "[lds graphify] suspect-response diagnostics enabled: $diagnostic_log" >&2 fi - backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$local_provider_base_url" "$model" "$token_budget_value")" || + backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$local_provider_base_url" "$model" "$token_budget_value" "$graphify_think")" || die "Unable to build LocalDevStack Graphify provider configuration" unset OPENAI_BASE_URL OPENAI_API_KEY OPENAI_MODEL OLLAMA_BASE_URL OLLAMA_API_KEY OLLAMA_MODEL From ab58b6fd7343d33d6ee4cd4b48b71dc4b2ad9831 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:09:29 +0600 Subject: [PATCH 466/509] test(graphify): cover thinking A/B override --- tests/ai-contract.sh | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 3f483196..c81130d4 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -104,6 +104,7 @@ assert_file_contains "$ROOT/lib/ai.sh" 'reasoning_effort: "none"' assert_file_contains "$ROOT/lib/ai.sh" 'graphify-diagnostic-proxy.py' assert_file_contains "$ROOT/lib/ai.sh" 'http://127.0.0.1:${proxy_port}/v1' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-1' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_THINK:-off' assert_file_contains "$ROOT/lib/ai.sh" 'lds-graphify-diagnostics.jsonl' assert_file_contains "$ROOT/lib/ai.sh" 'llm think ' pass "LLM CLI and Graphify resolve through provider-aware common endpoint" @@ -151,10 +152,20 @@ pass "Graphify backend selection follows active LLM provider" tmp="$(mktemp -d)" trap 'rm -rf "$tmp"' EXIT - [[ "$(_graphify_write_local_provider "$tmp" fastflow http://llm.localhost:11434/v1 qwen3.5:9b 4000)" == lds-fastflow ]] || + [[ "$(_graphify_write_local_provider "$tmp" fastflow http://llm.localhost:11434/v1 qwen3.5:9b 4000 off)" == lds-fastflow ]] || fail "FastFlow local Graphify provider name drifted" - jq -e '."lds-fastflow".extra_body.think == false' "$tmp/.graphify/providers.json" >/dev/null || - fail "FastFlow local Graphify provider must disable thinking" + jq -e '."lds-fastflow".extra_body.think == false and (."lds-fastflow" | has("reasoning_effort") | not)' "$tmp/.graphify/providers.json" >/dev/null || + fail "FastFlow local Graphify provider must default to no-thinking" + + [[ "$(_graphify_write_local_provider "$tmp" fastflow http://llm.localhost:11434/v1 qwen3.5:9b 4000 on)" == lds-fastflow ]] || + fail "FastFlow local Graphify thinking-on provider name drifted" + jq -e '."lds-fastflow".extra_body.think == true and ."lds-fastflow".reasoning_effort == "high"' "$tmp/.graphify/providers.json" >/dev/null || + fail "FastFlow Graphify thinking-on override did not map to think=true/high" + + [[ "$(_graphify_write_local_provider "$tmp" fastflow http://llm.localhost:11434/v1 qwen3.5:9b 4000 auto)" == lds-fastflow ]] || + fail "FastFlow local Graphify auto provider name drifted" + jq -e '(."lds-fastflow" | has("extra_body") | not) and (."lds-fastflow" | has("reasoning_effort") | not)' "$tmp/.graphify/providers.json" >/dev/null || + fail "FastFlow Graphify auto override must omit thinking controls" [[ "$(_graphify_write_local_provider "$tmp" ollama http://llm.localhost:11434/v1 qwen3:14b 4000)" == lds-ollama ]] || fail "Ollama local Graphify provider name drifted" From 438e688ae15459e8110d072d58e82efe96cf43e7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:09:43 +0600 Subject: [PATCH 467/509] docs(graphify): document thinking A/B test --- docs/guides/local-ai.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 45da6680..daaf95f5 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -303,6 +303,21 @@ prompt or source corpus. Set ``LDS_GRAPHIFY_DIAGNOSTICS=0`` to bypass the proxy. ``LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW`` controls the terminal preview size (minimum 256, default 4096). ``LDS_GRAPHIFY_DIAGNOSTIC_LOG`` overrides the JSONL path. +FastFlow Graphify defaults to ``LDS_GRAPHIFY_THINK=off``. For a controlled +instruction-following A/B test, run the same corpus once with thinking enabled: + +.. code-block:: bash + + LDS_GRAPHIFY_THINK=on lds graphify . + +The ``on`` mode sends both ``think=true`` and +``reasoning_effort=high``. FastFlow keeps reasoning in +``reasoning_content`` and the final answer in ``message.content``, so the +Graphify parser still receives only the final graph JSON. Use +``LDS_GRAPHIFY_THINK=auto`` to omit both controls and return to the model/provider +default. This switch is intended for diagnosis until the better-performing mode is +confirmed on the real corpus. + Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the selected model is absent. Override the local chunk defaults with explicit Graphify flags, or set ``LDS_GRAPHIFY_TOKEN_BUDGET`` / From 854537936c77a0388f6ce641e4d9cf2077cd75b6 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:18:14 +0600 Subject: [PATCH 468/509] feat(graphify): recover malformed FastFlow graphs via tool calling --- scripts/graphify-diagnostic-proxy.py | 222 ++++++++++++++++++++++++++- 1 file changed, 216 insertions(+), 6 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index a3959ff0..44fe61ad 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -121,6 +121,162 @@ def classify_graph_content(content: str | None) -> tuple[bool, str]: return True, "response is not parseable as a graph JSON object" + +_GRAPH_TOOL = { + "type": "function", + "function": { + "name": "submit_graph", + "description": "Submit the extracted Graphify knowledge-graph fragment. Call exactly once.", + "parameters": { + "type": "object", + "required": ["nodes", "edges", "hyperedges"], + "properties": { + "nodes": {"type": "array", "items": {"type": "object"}}, + "edges": {"type": "array", "items": {"type": "object"}}, + "hyperedges": {"type": "array", "items": {"type": "object"}}, + }, + }, + }, +} + + +def _build_tool_recovery_request(body: bytes) -> bytes | None: + try: + request = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return None + if not isinstance(request, dict): + return None + + messages = request.get("messages") + if not isinstance(messages, list): + return None + + request = dict(request) + request["tools"] = [_GRAPH_TOOL] + # FastFlow v1.0.6 treats unsupported tool_choice modes as auto, so the + # explicit instruction is the enforcement mechanism for Qwen3.5. + request["tool_choice"] = "auto" + request["stream"] = False + + amended = [] + recovery_suffix = ( + "\n\nSTRUCTURED RECOVERY: Do not emit the graph as assistant text. " + "Call the submit_graph tool exactly once. Put the complete extraction " + "fragment into its nodes, edges, and hyperedges arguments. Do not add " + "new facts; follow the original Graphify schema and source_file rules." + ) + injected = False + for message in messages: + if not isinstance(message, dict): + amended.append(message) + continue + copied = dict(message) + if ( + not injected + and copied.get("role") == "system" + and isinstance(copied.get("content"), str) + and "graphify semantic extraction agent" in copied["content"] + ): + copied["content"] += recovery_suffix + injected = True + amended.append(copied) + if not injected: + amended.insert(0, {"role": "system", "content": recovery_suffix.strip()}) + request["messages"] = amended + return json.dumps(request, ensure_ascii=False).encode("utf-8") + + +def _coerce_graph_array(value: Any) -> list[dict[str, Any]] | None: + if isinstance(value, str): + try: + value = json.loads(value) + except json.JSONDecodeError: + return None + if not isinstance(value, list): + return None + if any(not isinstance(entry, dict) for entry in value): + return None + return value + + +def _extract_graph_tool_result(body: bytes) -> dict[str, Any] | None: + try: + response = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return None + if not isinstance(response, dict): + return None + + choices = response.get("choices") + if not isinstance(choices, list) or not choices or not isinstance(choices[0], dict): + return None + message = choices[0].get("message") + if not isinstance(message, dict): + return None + tool_calls = message.get("tool_calls") + if not isinstance(tool_calls, list): + return None + + for call in tool_calls: + if not isinstance(call, dict): + continue + function = call.get("function") + if not isinstance(function, dict) or function.get("name") != "submit_graph": + continue + arguments = function.get("arguments") + if isinstance(arguments, str): + try: + arguments = json.loads(arguments) + except json.JSONDecodeError: + continue + if not isinstance(arguments, dict): + continue + + graph: dict[str, Any] = {} + for key in _GRAPH_KEYS: + value = _coerce_graph_array(arguments.get(key)) + if value is None: + break + graph[key] = value + else: + if any(graph[key] for key in _GRAPH_KEYS): + return graph + return None + + +def _replace_response_content(body: bytes, graph: dict[str, Any]) -> bytes | None: + try: + response = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return None + if not isinstance(response, dict): + return None + choices = response.get("choices") + if not isinstance(choices, list) or not choices or not isinstance(choices[0], dict): + return None + + response = dict(response) + choices = [dict(choice) if isinstance(choice, dict) else choice for choice in choices] + first = choices[0] + message = first.get("message") + if not isinstance(message, dict): + message = {} + else: + message = dict(message) + message["role"] = "assistant" + message["content"] = json.dumps(graph, ensure_ascii=False, separators=(",", ":")) + message.pop("tool_calls", None) + message.pop("reasoning_content", None) + message.pop("reasoning", None) + message.pop("thinking", None) + first["message"] = message + first["finish_reason"] = "stop" + choices[0] = first + response["choices"] = choices + return json.dumps(response, ensure_ascii=False).encode("utf-8") + + def _request_metadata(body: bytes) -> dict[str, Any]: try: request = json.loads(body.decode("utf-8")) @@ -225,7 +381,7 @@ def _forward(self) -> None: ).encode("utf-8") if self.command == "POST" and self.path.rstrip("/").endswith("/v1/chat/completions"): - self._inspect_chat_response(body, response_body, status) + response_body = self._inspect_and_recover_chat_response(body, response_body, status) self.send_response(status) for name, value in response_headers.items(): @@ -236,16 +392,62 @@ def _forward(self) -> None: if self.command != "HEAD": self.wfile.write(response_body) - def _inspect_chat_response(self, request_body: bytes, response_body: bytes, status: int) -> None: + def _post_upstream(self, body: bytes) -> bytes | None: + request = urllib.request.Request( + self.upstream.rstrip("/") + "/v1/chat/completions", + data=body, + headers={ + "Content-Type": "application/json", + "Authorization": self.headers.get("Authorization", "Bearer local"), + }, + method="POST", + ) + try: + with urllib.request.urlopen(request, timeout=1900) as response: + if response.status < 200 or response.status >= 300: + return None + return response.read() + except Exception: + return None + + def _inspect_and_recover_chat_response( + self, request_body: bytes, response_body: bytes, status: int + ) -> bytes: req = _request_metadata(request_body) resp, content = _response_metadata(response_body) - if status < 200 or status >= 300 or not req.pop("_extraction_request", False): - return + extraction_request = req.pop("_extraction_request", False) + if status < 200 or status >= 300 or not extraction_request: + return response_body suspect, reason = classify_graph_content(content) if not suspect: - return - + return response_body + + original_reason = reason + recovery = _build_tool_recovery_request(request_body) + recovered_graph = None + recovered_response = None + if recovery is not None and req.get("think", "") != "": + recovered_response = self._post_upstream(recovery) + if recovered_response is not None: + recovered_graph = _extract_graph_tool_result(recovered_response) + + if recovered_graph is not None and recovered_response is not None: + replacement = _replace_response_content(recovered_response, recovered_graph) + if replacement is not None: + recovered_meta, _ = _response_metadata(replacement) + print( + "[lds graphify diagnostic] recovered malformed FastFlow graph via submit_graph tool: " + f"model={req.get('model')}; think={req.get('think')}; " + f"nodes={len(recovered_graph['nodes'])}; edges={len(recovered_graph['edges'])}; " + f"hyperedges={len(recovered_graph['hyperedges'])}; " + f"completion_tokens={recovered_meta.get('completion_tokens')}", + file=sys.stderr, + flush=True, + ) + return replacement + + reason = original_reason record = { "timestamp": datetime.now(timezone.utc).isoformat(), "status": status, @@ -280,6 +482,14 @@ def _inspect_chat_response(self, request_body: bytes, response_body: bytes, stat file=sys.stderr, flush=True, ) + if recovery is not None: + print( + "[lds graphify diagnostic] structured submit_graph recovery did not yield a usable graph; " + "returning the original response so Graphify can apply its normal retry policy", + file=sys.stderr, + flush=True, + ) + return response_body do_GET = _forward do_HEAD = _forward From 45a2e59510bc775adfccc3dd1c15bb31e0b0fc2c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:19:01 +0600 Subject: [PATCH 469/509] test(graphify): cover tool-call recovery --- tests/ai-contract.sh | 63 +++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 62 insertions(+), 1 deletion(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index c81130d4..d3a9f64a 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -225,8 +225,69 @@ metadata = module._request_metadata(body) assert metadata["_extraction_request"] is True assert metadata["think"] is False assert "private corpus content" not in json.dumps(metadata) + +recovery = module._build_tool_recovery_request(body) +assert recovery is not None +recovery_json = json.loads(recovery) +assert recovery_json["tools"][0]["function"]["name"] == "submit_graph" +assert recovery_json["tool_choice"] == "auto" +assert recovery_json["think"] is False +assert "STRUCTURED RECOVERY" in recovery_json["messages"][0]["content"] + +tool_response = { + "id": "chatcmpl-test", + "object": "chat.completion", + "choices": [{ + "index": 0, + "finish_reason": "tool_calls", + "message": { + "role": "assistant", + "reasoning_content": "private reasoning", + "content": "private reasoning", + "tool_calls": [{ + "id": "call_1", + "type": "function", + "function": { + "name": "submit_graph", + "arguments": json.dumps({ + "nodes": [{"id": "readme_pathwise", "label": "Pathwise"}], + "edges": [], + "hyperedges": [], + }), + }, + }], + }, + }], + "usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30}, +} +graph = module._extract_graph_tool_result(json.dumps(tool_response).encode()) +assert graph == { + "nodes": [{"id": "readme_pathwise", "label": "Pathwise"}], + "edges": [], + "hyperedges": [], +} +replacement = module._replace_response_content(json.dumps(tool_response).encode(), graph) +assert replacement is not None +replacement_json = json.loads(replacement) +message = replacement_json["choices"][0]["message"] +assert json.loads(message["content"]) == graph +assert "tool_calls" not in message +assert "reasoning_content" not in message +assert replacement_json["choices"][0]["finish_reason"] == "stop" + +string_array_response = json.loads(json.dumps(tool_response)) +string_array_response["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"] = json.dumps({ + "nodes": json.dumps([{"id": "a"}]), + "edges": "[]", + "hyperedges": "[]", +}) +assert module._extract_graph_tool_result(json.dumps(string_array_response).encode()) == { + "nodes": [{"id": "a"}], + "edges": [], + "hyperedges": [], +} PY -pass "Graphify diagnostic proxy identifies suspect responses without logging prompts" +pass "Graphify diagnostic proxy identifies and structurally recovers suspect responses" ( From 279c6457b113ab4e4d986de795f8396b1119158f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:19:13 +0600 Subject: [PATCH 470/509] docs(graphify): explain structured FastFlow recovery --- docs/guides/local-ai.rst | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index daaf95f5..acb34db4 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -297,6 +297,15 @@ response as JSON Lines in: /graphify-out/lds-graphify-diagnostics.jsonl +For FastFlow only, malformed semantic responses also get one provider-native +structured recovery attempt. LocalDevStack replays the same extraction request with +FastFlow's tool-calling channel and a ``submit_graph`` function, then converts valid +tool arguments back into the normal Graphify JSON response. The original prompt and +source corpus never leave the same local provider; the recovery only changes the +response channel from free-form assistant text to structured tool arguments. If the +tool call is absent or invalid, LocalDevStack returns the original response and +Graphify applies its normal retry policy. + The diagnostic record contains request controls such as model, ``think``, ``reasoning_effort``, finish reason, and token usage, but never stores the Graphify prompt or source corpus. Set ``LDS_GRAPHIFY_DIAGNOSTICS=0`` to bypass the proxy. From 3c767705e5a273e3ef0489ae63e48d3ca1965ac0 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:20:07 +0600 Subject: [PATCH 471/509] fix(graphify): constrain recovery tool to Graphify schema --- scripts/graphify-diagnostic-proxy.py | 80 ++++++++++++++++++++++++++-- 1 file changed, 77 insertions(+), 3 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 44fe61ad..ea1c98a4 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -131,9 +131,83 @@ def classify_graph_content(content: str | None) -> tuple[bool, str]: "type": "object", "required": ["nodes", "edges", "hyperedges"], "properties": { - "nodes": {"type": "array", "items": {"type": "object"}}, - "edges": {"type": "array", "items": {"type": "object"}}, - "hyperedges": {"type": "array", "items": {"type": "object"}}, + "nodes": { + "type": "array", + "items": { + "type": "object", + "required": ["id", "label", "file_type", "source_file"], + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"}, + "file_type": { + "type": "string", + "enum": ["code", "document", "paper", "image", "rationale", "concept"], + }, + "source_file": {"type": "string"}, + "source_location": {"type": ["string", "null"]}, + "source_url": {"type": ["string", "null"]}, + "captured_at": {"type": ["string", "null"]}, + "author": {"type": ["string", "null"]}, + "contributor": {"type": ["string", "null"]}, + "rationale": {"type": ["string", "null"]}, + }, + }, + }, + "edges": { + "type": "array", + "items": { + "type": "object", + "required": [ + "source", "target", "relation", "confidence", + "confidence_score", "source_file", "weight", + ], + "properties": { + "source": {"type": "string"}, + "target": {"type": "string"}, + "relation": { + "type": "string", + "enum": [ + "calls", "implements", "references", "cites", + "conceptually_related_to", "shares_data_with", + "semantically_similar_to", "rationale_for", + ], + }, + "confidence": { + "type": "string", + "enum": ["EXTRACTED", "INFERRED", "AMBIGUOUS"], + }, + "confidence_score": {"type": "number"}, + "source_file": {"type": "string"}, + "source_location": {"type": ["string", "null"]}, + "weight": {"type": "number"}, + }, + }, + }, + "hyperedges": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", "label", "nodes", "relation", "confidence", + "confidence_score", "source_file", + ], + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"}, + "nodes": {"type": "array", "items": {"type": "string"}}, + "relation": { + "type": "string", + "enum": ["participate_in", "implement", "form"], + }, + "confidence": { + "type": "string", + "enum": ["EXTRACTED", "INFERRED"], + }, + "confidence_score": {"type": "number"}, + "source_file": {"type": "string"}, + }, + }, + }, }, }, }, From 7f4b4605cc26a1707c1a1514c43525b6adb7d926 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:49:14 +0600 Subject: [PATCH 472/509] perf(graphify): make structured FastFlow extraction primary --- scripts/graphify-diagnostic-proxy.py | 49 ++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 3 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index ea1c98a4..cfb7dde7 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -424,10 +424,22 @@ def _forward(self) -> None: if value: request_headers[name] = value + is_chat = self.command == "POST" and self.path.rstrip("/").endswith("/v1/chat/completions") + metadata = _request_metadata(body) if is_chat else {} + extraction_request = bool(metadata.get("_extraction_request")) + + upstream_body = body + structured_primary = False + if extraction_request: + candidate = _build_tool_recovery_request(body) + if candidate is not None: + upstream_body = candidate + structured_primary = True + upstream_url = self.upstream.rstrip("/") + self.path request = urllib.request.Request( upstream_url, - data=body if self.command not in ("GET", "HEAD") else None, + data=upstream_body if self.command not in ("GET", "HEAD") else None, headers=request_headers, method=self.command, ) @@ -454,8 +466,27 @@ def _forward(self) -> None: {"error": {"message": f"Graphify diagnostic proxy upstream error: {exc}"}} ).encode("utf-8") - if self.command == "POST" and self.path.rstrip("/").endswith("/v1/chat/completions"): - response_body = self._inspect_and_recover_chat_response(body, response_body, status) + if is_chat and extraction_request: + if structured_primary and 200 <= status < 300: + graph = _extract_graph_tool_result(response_body) + if graph is not None: + replacement = _replace_response_content(response_body, graph) + if replacement is not None: + response_body = replacement + print( + "[lds graphify diagnostic] structured FastFlow graph via submit_graph tool: " + f"model={metadata.get('model')}; think={metadata.get('think')}; " + f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " + f"hyperedges={len(graph['hyperedges'])}", + file=sys.stderr, + flush=True, + ) + else: + response_body = self._fallback_freeform(body) + else: + response_body = self._fallback_freeform(body) + else: + response_body = self._inspect_and_recover_chat_response(body, response_body, status) self.send_response(status) for name, value in response_headers.items(): @@ -466,6 +497,18 @@ def _forward(self) -> None: if self.command != "HEAD": self.wfile.write(response_body) + def _fallback_freeform(self, original_body: bytes) -> bytes: + fallback = self._post_upstream(original_body) + if fallback is None: + return b'{"error":{"message":"Structured Graphify extraction failed and free-form fallback was unavailable"}}' + print( + "[lds graphify diagnostic] submit_graph primary extraction did not yield a usable graph; " + "falling back to the original free-form Graphify request", + file=sys.stderr, + flush=True, + ) + return self._inspect_and_recover_chat_response(original_body, fallback, 200) + def _post_upstream(self, body: bytes) -> bytes | None: request = urllib.request.Request( self.upstream.rstrip("/") + "/v1/chat/completions", From 0a675900049328733510d0e5b05dd24eab8d93aa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:49:44 +0600 Subject: [PATCH 473/509] fix(graphify): scope structured primary path to FastFlow --- scripts/graphify-diagnostic-proxy.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index cfb7dde7..81fdd651 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -427,10 +427,11 @@ def _forward(self) -> None: is_chat = self.command == "POST" and self.path.rstrip("/").endswith("/v1/chat/completions") metadata = _request_metadata(body) if is_chat else {} extraction_request = bool(metadata.get("_extraction_request")) + fastflow_request = metadata.get("think", "") != "" upstream_body = body structured_primary = False - if extraction_request: + if extraction_request and fastflow_request: candidate = _build_tool_recovery_request(body) if candidate is not None: upstream_body = candidate From a748baff30b9b3360e054962114e11a44a2d8c6a Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:50:44 +0600 Subject: [PATCH 474/509] feat(graphify): use native structured extraction for both providers --- scripts/graphify-diagnostic-proxy.py | 218 ++++++++++++++++----------- 1 file changed, 132 insertions(+), 86 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 81fdd651..af83d02b 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -1,3 +1,88 @@ + +_GRAPH_SCHEMA = { + "type": "object", + "required": ["nodes", "edges", "hyperedges"], + "properties": { + "nodes": { + "type": "array", + "items": { + "type": "object", + "required": ["id", "label", "file_type", "source_file"], + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"}, + "file_type": { + "type": "string", + "enum": ["code", "document", "paper", "image", "rationale", "concept"], + }, + "source_file": {"type": "string"}, + "source_location": {"type": ["string", "null"]}, + "source_url": {"type": ["string", "null"]}, + "captured_at": {"type": ["string", "null"]}, + "author": {"type": ["string", "null"]}, + "contributor": {"type": ["string", "null"]}, + "rationale": {"type": ["string", "null"]}, + }, + }, + }, + "edges": { + "type": "array", + "items": { + "type": "object", + "required": [ + "source", "target", "relation", "confidence", + "confidence_score", "source_file", "weight", + ], + "properties": { + "source": {"type": "string"}, + "target": {"type": "string"}, + "relation": { + "type": "string", + "enum": [ + "calls", "implements", "references", "cites", + "conceptually_related_to", "shares_data_with", + "semantically_similar_to", "rationale_for", + ], + }, + "confidence": { + "type": "string", + "enum": ["EXTRACTED", "INFERRED", "AMBIGUOUS"], + }, + "confidence_score": {"type": "number"}, + "source_file": {"type": "string"}, + "source_location": {"type": ["string", "null"]}, + "weight": {"type": "number"}, + }, + }, + }, + "hyperedges": { + "type": "array", + "items": { + "type": "object", + "required": [ + "id", "label", "nodes", "relation", "confidence", + "confidence_score", "source_file", + ], + "properties": { + "id": {"type": "string"}, + "label": {"type": "string"}, + "nodes": {"type": "array", "items": {"type": "string"}}, + "relation": { + "type": "string", + "enum": ["participate_in", "implement", "form"], + }, + "confidence": { + "type": "string", + "enum": ["EXTRACTED", "INFERRED"], + }, + "confidence_score": {"type": "number"}, + "source_file": {"type": "string"}, + }, + }, + }, + }, +} + #!/usr/bin/env python3 """Local Graphify diagnostic reverse proxy. @@ -127,93 +212,33 @@ def classify_graph_content(content: str | None) -> tuple[bool, str]: "function": { "name": "submit_graph", "description": "Submit the extracted Graphify knowledge-graph fragment. Call exactly once.", - "parameters": { - "type": "object", - "required": ["nodes", "edges", "hyperedges"], - "properties": { - "nodes": { - "type": "array", - "items": { - "type": "object", - "required": ["id", "label", "file_type", "source_file"], - "properties": { - "id": {"type": "string"}, - "label": {"type": "string"}, - "file_type": { - "type": "string", - "enum": ["code", "document", "paper", "image", "rationale", "concept"], - }, - "source_file": {"type": "string"}, - "source_location": {"type": ["string", "null"]}, - "source_url": {"type": ["string", "null"]}, - "captured_at": {"type": ["string", "null"]}, - "author": {"type": ["string", "null"]}, - "contributor": {"type": ["string", "null"]}, - "rationale": {"type": ["string", "null"]}, - }, - }, - }, - "edges": { - "type": "array", - "items": { - "type": "object", - "required": [ - "source", "target", "relation", "confidence", - "confidence_score", "source_file", "weight", - ], - "properties": { - "source": {"type": "string"}, - "target": {"type": "string"}, - "relation": { - "type": "string", - "enum": [ - "calls", "implements", "references", "cites", - "conceptually_related_to", "shares_data_with", - "semantically_similar_to", "rationale_for", - ], - }, - "confidence": { - "type": "string", - "enum": ["EXTRACTED", "INFERRED", "AMBIGUOUS"], - }, - "confidence_score": {"type": "number"}, - "source_file": {"type": "string"}, - "source_location": {"type": ["string", "null"]}, - "weight": {"type": "number"}, - }, - }, - }, - "hyperedges": { - "type": "array", - "items": { - "type": "object", - "required": [ - "id", "label", "nodes", "relation", "confidence", - "confidence_score", "source_file", - ], - "properties": { - "id": {"type": "string"}, - "label": {"type": "string"}, - "nodes": {"type": "array", "items": {"type": "string"}}, - "relation": { - "type": "string", - "enum": ["participate_in", "implement", "form"], - }, - "confidence": { - "type": "string", - "enum": ["EXTRACTED", "INFERRED"], - }, - "confidence_score": {"type": "number"}, - "source_file": {"type": "string"}, - }, - }, - }, - }, - }, + "parameters": _GRAPH_SCHEMA, }, } +def _build_ollama_schema_request(body: bytes) -> bytes | None: + try: + request = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return None + if not isinstance(request, dict): + return None + + request = dict(request) + request["stream"] = False + request["temperature"] = 0 + request["response_format"] = { + "type": "json_schema", + "json_schema": { + "name": "graphify_fragment", + "strict": True, + "schema": _GRAPH_SCHEMA, + }, + } + return json.dumps(request, ensure_ascii=False).encode("utf-8") + + def _build_tool_recovery_request(body: bytes) -> bytes | None: try: request = json.loads(body.decode("utf-8")) @@ -428,14 +453,23 @@ def _forward(self) -> None: metadata = _request_metadata(body) if is_chat else {} extraction_request = bool(metadata.get("_extraction_request")) fastflow_request = metadata.get("think", "") != "" + ollama_request = ( + not fastflow_request + and metadata.get("reasoning_effort", "") != "" + ) upstream_body = body - structured_primary = False + structured_primary = "" if extraction_request and fastflow_request: candidate = _build_tool_recovery_request(body) if candidate is not None: upstream_body = candidate - structured_primary = True + structured_primary = "fastflow-tool" + elif extraction_request and ollama_request: + candidate = _build_ollama_schema_request(body) + if candidate is not None: + upstream_body = candidate + structured_primary = "ollama-schema" upstream_url = self.upstream.rstrip("/") + self.path request = urllib.request.Request( @@ -468,7 +502,7 @@ def _forward(self) -> None: ).encode("utf-8") if is_chat and extraction_request: - if structured_primary and 200 <= status < 300: + if structured_primary == "fastflow-tool" and 200 <= status < 300: graph = _extract_graph_tool_result(response_body) if graph is not None: replacement = _replace_response_content(response_body, graph) @@ -486,6 +520,18 @@ def _forward(self) -> None: response_body = self._fallback_freeform(body) else: response_body = self._fallback_freeform(body) + elif structured_primary == "ollama-schema" and 200 <= status < 300: + _meta, content = _response_metadata(response_body) + suspect, _reason = classify_graph_content(content) + if suspect: + response_body = self._fallback_freeform(body) + else: + print( + "[lds graphify diagnostic] structured Ollama graph via response_format schema: " + f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}", + file=sys.stderr, + flush=True, + ) else: response_body = self._inspect_and_recover_chat_response(body, response_body, status) From b7b146b54a9f512226fdc7f1116497afeccd9a04 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:51:12 +0600 Subject: [PATCH 475/509] fix(graphify): keep schema below future import --- scripts/graphify-diagnostic-proxy.py | 43 ++++++++++++++-------------- 1 file changed, 21 insertions(+), 22 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index af83d02b..31fc133d 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -1,4 +1,25 @@ +#!/usr/bin/env python3 +"""Local Graphify diagnostic reverse proxy. + +Forwards OpenAI-compatible Graphify traffic to the active LocalDevStack LLM +without modifying request bodies. Only suspect chat-completion responses are +reported: empty content, malformed/non-graph JSON, or graph arrays containing +no usable object entries. Request prompts/source content are never logged. +""" +from __future__ import annotations + +import argparse +import json +import re +import sys +import urllib.error +import urllib.request +from datetime import datetime, timezone +from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer +from pathlib import Path +from typing import Any +_GRAPH_KEYS = ("nodes", "edges", "hyperedges") _GRAPH_SCHEMA = { "type": "object", "required": ["nodes", "edges", "hyperedges"], @@ -83,28 +104,6 @@ }, } -#!/usr/bin/env python3 -"""Local Graphify diagnostic reverse proxy. - -Forwards OpenAI-compatible Graphify traffic to the active LocalDevStack LLM -without modifying request bodies. Only suspect chat-completion responses are -reported: empty content, malformed/non-graph JSON, or graph arrays containing -no usable object entries. Request prompts/source content are never logged. -""" -from __future__ import annotations - -import argparse -import json -import re -import sys -import urllib.error -import urllib.request -from datetime import datetime, timezone -from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer -from pathlib import Path -from typing import Any - -_GRAPH_KEYS = ("nodes", "edges", "hyperedges") _FENCE_RE = re.compile(r"~~~[ \t]*([A-Za-z0-9_+-]*)[ \t]*\r?\n(.*?)~~~", re.S) From df0ad4c5706a0d25a9667e255672ff890ef78bbe Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:55:09 +0600 Subject: [PATCH 476/509] fix(graphify): make structured extraction provider-explicit --- scripts/graphify-diagnostic-proxy.py | 46 ++++++++++++++++++++-------- 1 file changed, 34 insertions(+), 12 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 31fc133d..2a50f0d1 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -154,6 +154,29 @@ def _json_candidates(content: str): start = content.find("{", start + 1) +def parse_graph_content(content: str | None) -> dict[str, list[dict[str, Any]]] | None: + """Return a structurally valid graph fragment, including an all-empty fragment.""" + if content is None or not content.strip(): + return None + for candidate in _json_candidates(content): + try: + parsed = json.loads(candidate) + except (json.JSONDecodeError, TypeError): + continue + if not isinstance(parsed, dict): + continue + + graph: dict[str, list[dict[str, Any]]] = {} + for key in _GRAPH_KEYS: + value = parsed.get(key) + if not isinstance(value, list) or any(not isinstance(entry, dict) for entry in value): + break + graph[key] = value + else: + return graph + return None + + def classify_graph_content(content: str | None) -> tuple[bool, str]: """Return (suspect, reason), mirroring Graphify's hollow decision closely.""" if content is None or not content.strip(): @@ -338,8 +361,7 @@ def _extract_graph_tool_result(body: bytes) -> dict[str, Any] | None: break graph[key] = value else: - if any(graph[key] for key in _GRAPH_KEYS): - return graph + return graph return None @@ -432,6 +454,7 @@ def _response_metadata(body: bytes) -> tuple[dict[str, Any], str | None]: class DiagnosticHandler(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" upstream: str = "" + provider: str = "" log_file: Path preview_chars: int = 4096 @@ -451,20 +474,15 @@ def _forward(self) -> None: is_chat = self.command == "POST" and self.path.rstrip("/").endswith("/v1/chat/completions") metadata = _request_metadata(body) if is_chat else {} extraction_request = bool(metadata.get("_extraction_request")) - fastflow_request = metadata.get("think", "") != "" - ollama_request = ( - not fastflow_request - and metadata.get("reasoning_effort", "") != "" - ) upstream_body = body structured_primary = "" - if extraction_request and fastflow_request: + if extraction_request and self.provider == "fastflow": candidate = _build_tool_recovery_request(body) if candidate is not None: upstream_body = candidate structured_primary = "fastflow-tool" - elif extraction_request and ollama_request: + elif extraction_request and self.provider == "ollama": candidate = _build_ollama_schema_request(body) if candidate is not None: upstream_body = candidate @@ -521,13 +539,15 @@ def _forward(self) -> None: response_body = self._fallback_freeform(body) elif structured_primary == "ollama-schema" and 200 <= status < 300: _meta, content = _response_metadata(response_body) - suspect, _reason = classify_graph_content(content) - if suspect: + graph = parse_graph_content(content) + if graph is None: response_body = self._fallback_freeform(body) else: print( "[lds graphify diagnostic] structured Ollama graph via response_format schema: " - f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}", + f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}; " + f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " + f"hyperedges={len(graph['hyperedges'])}", file=sys.stderr, flush=True, ) @@ -662,12 +682,14 @@ def _inspect_and_recover_chat_response( def main() -> int: parser = argparse.ArgumentParser() parser.add_argument("--upstream", required=True) + parser.add_argument("--provider", required=True, choices=("fastflow", "ollama")) parser.add_argument("--ready-file", required=True) parser.add_argument("--log-file", required=True) parser.add_argument("--preview-chars", type=int, default=4096) args = parser.parse_args() DiagnosticHandler.upstream = args.upstream + DiagnosticHandler.provider = args.provider DiagnosticHandler.log_file = Path(args.log_file) DiagnosticHandler.preview_chars = max(256, args.preview_chars) From 3f81e61c683cfe4e6eadce7681be76992a16e909 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:55:19 +0600 Subject: [PATCH 477/509] fix(graphify): pass active provider to structured proxy --- lib/ai.sh | 1 + 1 file changed, 1 insertion(+) diff --git a/lib/ai.sh b/lib/ai.sh index eb3461ab..fe3f5d0f 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -401,6 +401,7 @@ cmd_graphify() { "$graphify_python" "$DIR/scripts/graphify-diagnostic-proxy.py" \ --upstream "${base_url%/v1}" \ + --provider "$provider" \ --ready-file "$ready_file" \ --log-file "$diagnostic_log" \ --preview-chars "$diagnostic_preview" & From 7ae08f9e0706f96ced7515667c2d7e392089ced4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:55:36 +0600 Subject: [PATCH 478/509] test(graphify): cover provider-native structured extraction --- tests/ai-contract.sh | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index d3a9f64a..885e7a27 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -103,6 +103,7 @@ assert_file_contains "$ROOT/lib/ai.sh" 'extra_body: {think: false}' assert_file_contains "$ROOT/lib/ai.sh" 'reasoning_effort: "none"' assert_file_contains "$ROOT/lib/ai.sh" 'graphify-diagnostic-proxy.py' assert_file_contains "$ROOT/lib/ai.sh" 'http://127.0.0.1:${proxy_port}/v1' +assert_file_contains "$ROOT/lib/ai.sh" '--provider "$provider"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-1' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_THINK:-off' assert_file_contains "$ROOT/lib/ai.sh" 'lds-graphify-diagnostics.jsonl' @@ -212,6 +213,13 @@ suspect, reason = module.classify_graph_content( ) assert suspect and reason == "malformed graph JSON" +assert module.parse_graph_content('{"nodes":[],"edges":[],"hyperedges":[]}') == { + "nodes": [], + "edges": [], + "hyperedges": [], +} +assert module.parse_graph_content('{"nodes":["bad"],"edges":[],"hyperedges":[]}') is None + body = json.dumps({ "model": "qwen3.5:9b", "messages": [ @@ -234,6 +242,30 @@ assert recovery_json["tool_choice"] == "auto" assert recovery_json["think"] is False assert "STRUCTURED RECOVERY" in recovery_json["messages"][0]["content"] +assert recovery_json["tools"][0]["function"]["parameters"] == module._GRAPH_SCHEMA +assert "rationale_for" not in module._GRAPH_SCHEMA["properties"]["edges"]["items"]["properties"]["relation"]["enum"] + +ollama_body = json.dumps({ + "model": "qwen3:14b", + "messages": [ + {"role": "system", "content": "You are a graphify semantic extraction agent."}, + {"role": "user", "content": "private corpus content"} + ], + "reasoning_effort": "none", + "stream": False, + "temperature": 0, + "options": {"num_ctx": 8192}, +}).encode() +ollama_request = module._build_ollama_schema_request(ollama_body) +assert ollama_request is not None +ollama_json = json.loads(ollama_request) +assert ollama_json["reasoning_effort"] == "none" +assert ollama_json["options"]["num_ctx"] == 8192 +assert ollama_json["temperature"] == 0 +assert ollama_json["response_format"]["type"] == "json_schema" +assert ollama_json["response_format"]["json_schema"]["strict"] is True +assert ollama_json["response_format"]["json_schema"]["schema"] == module._GRAPH_SCHEMA + tool_response = { "id": "chatcmpl-test", "object": "chat.completion", @@ -286,6 +318,18 @@ assert module._extract_graph_tool_result(json.dumps(string_array_response).encod "edges": [], "hyperedges": [], } + +empty_tool_response = json.loads(json.dumps(tool_response)) +empty_tool_response["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"] = json.dumps({ + "nodes": [], + "edges": [], + "hyperedges": [], +}) +assert module._extract_graph_tool_result(json.dumps(empty_tool_response).encode()) == { + "nodes": [], + "edges": [], + "hyperedges": [], +} PY pass "Graphify diagnostic proxy identifies and structurally recovers suspect responses" From ecd0faacafc7a1350f3d89b2619601f9d3517540 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:56:49 +0600 Subject: [PATCH 479/509] refactor(graphify): decouple structured proxy from diagnostics --- scripts/graphify-diagnostic-proxy.py | 107 +++++++++++++++------------ 1 file changed, 60 insertions(+), 47 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 2a50f0d1..5243db85 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -455,6 +455,7 @@ class DiagnosticHandler(BaseHTTPRequestHandler): protocol_version = "HTTP/1.1" upstream: str = "" provider: str = "" + diagnostics: bool = False log_file: Path preview_chars: int = 4096 @@ -525,14 +526,15 @@ def _forward(self) -> None: replacement = _replace_response_content(response_body, graph) if replacement is not None: response_body = replacement - print( - "[lds graphify diagnostic] structured FastFlow graph via submit_graph tool: " - f"model={metadata.get('model')}; think={metadata.get('think')}; " - f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " - f"hyperedges={len(graph['hyperedges'])}", - file=sys.stderr, - flush=True, - ) + if self.diagnostics: + print( + "[lds graphify diagnostic] structured FastFlow graph via submit_graph tool: " + f"model={metadata.get('model')}; think={metadata.get('think')}; " + f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " + f"hyperedges={len(graph['hyperedges'])}", + file=sys.stderr, + flush=True, + ) else: response_body = self._fallback_freeform(body) else: @@ -543,14 +545,15 @@ def _forward(self) -> None: if graph is None: response_body = self._fallback_freeform(body) else: - print( - "[lds graphify diagnostic] structured Ollama graph via response_format schema: " - f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}; " - f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " - f"hyperedges={len(graph['hyperedges'])}", - file=sys.stderr, - flush=True, - ) + if self.diagnostics: + print( + "[lds graphify diagnostic] structured Ollama graph via response_format schema: " + f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}; " + f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " + f"hyperedges={len(graph['hyperedges'])}", + file=sys.stderr, + flush=True, + ) else: response_body = self._inspect_and_recover_chat_response(body, response_body, status) @@ -568,7 +571,7 @@ def _fallback_freeform(self, original_body: bytes) -> bytes: if fallback is None: return b'{"error":{"message":"Structured Graphify extraction failed and free-form fallback was unavailable"}}' print( - "[lds graphify diagnostic] submit_graph primary extraction did not yield a usable graph; " + "[lds graphify] structured extraction did not yield a usable graph; " "falling back to the original free-form Graphify request", file=sys.stderr, flush=True, @@ -631,40 +634,48 @@ def _inspect_and_recover_chat_response( return replacement reason = original_reason - record = { - "timestamp": datetime.now(timezone.utc).isoformat(), - "status": status, - "reason": reason, - **req, - **resp, - "assistant_content": content, - } - self.log_file.parent.mkdir(parents=True, exist_ok=True) - with self.log_file.open("a", encoding="utf-8") as handle: - handle.write(json.dumps(record, ensure_ascii=False) + "\n") - - preview = (content or "")[: self.preview_chars] - print( - "[lds graphify diagnostic] suspect LLM response: " - f"reason={reason}; model={req.get('model')}; " - f"think={req.get('think')}; reasoning_effort={req.get('reasoning_effort')}; " - f"finish_reason={resp.get('finish_reason')}; " - f"prompt_tokens={resp.get('prompt_tokens')}; " - f"completion_tokens={resp.get('completion_tokens')}", - file=sys.stderr, - flush=True, - ) print( - f"[lds graphify diagnostic] assistant content preview ({len(preview)}/{len(content or '')} chars): " - f"{preview!r}", - file=sys.stderr, - flush=True, - ) - print( - f"[lds graphify diagnostic] full suspect response logged to {self.log_file}", + "[lds graphify] provider returned a suspect extraction response: " + f"reason={reason}; model={req.get('model')}; finish_reason={resp.get('finish_reason')}", file=sys.stderr, flush=True, ) + + if self.diagnostics: + record = { + "timestamp": datetime.now(timezone.utc).isoformat(), + "status": status, + "reason": reason, + **req, + **resp, + "assistant_content": content, + } + self.log_file.parent.mkdir(parents=True, exist_ok=True) + with self.log_file.open("a", encoding="utf-8") as handle: + handle.write(json.dumps(record, ensure_ascii=False) + "\n") + + preview = (content or "")[: self.preview_chars] + print( + "[lds graphify diagnostic] suspect LLM response: " + f"reason={reason}; model={req.get('model')}; " + f"think={req.get('think')}; reasoning_effort={req.get('reasoning_effort')}; " + f"finish_reason={resp.get('finish_reason')}; " + f"prompt_tokens={resp.get('prompt_tokens')}; " + f"completion_tokens={resp.get('completion_tokens')}", + file=sys.stderr, + flush=True, + ) + print( + f"[lds graphify diagnostic] assistant content preview ({len(preview)}/{len(content or '')} chars): " + f"{preview!r}", + file=sys.stderr, + flush=True, + ) + print( + f"[lds graphify diagnostic] full suspect response logged to {self.log_file}", + file=sys.stderr, + flush=True, + ) if recovery is not None: print( "[lds graphify diagnostic] structured submit_graph recovery did not yield a usable graph; " @@ -683,6 +694,7 @@ def main() -> int: parser = argparse.ArgumentParser() parser.add_argument("--upstream", required=True) parser.add_argument("--provider", required=True, choices=("fastflow", "ollama")) + parser.add_argument("--diagnostics", choices=("on", "off"), default="off") parser.add_argument("--ready-file", required=True) parser.add_argument("--log-file", required=True) parser.add_argument("--preview-chars", type=int, default=4096) @@ -690,6 +702,7 @@ def main() -> int: DiagnosticHandler.upstream = args.upstream DiagnosticHandler.provider = args.provider + DiagnosticHandler.diagnostics = args.diagnostics == "on" DiagnosticHandler.log_file = Path(args.log_file) DiagnosticHandler.preview_chars = max(256, args.preview_chars) From ea8c1ce2f7e79d9c6c7d517aef5b0b0be92c18b4 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:57:09 +0600 Subject: [PATCH 480/509] refactor(graphify): keep structured proxy always enabled --- lib/ai.sh | 47 ++++++++++++++++++++++++++++------------------- 1 file changed, 28 insertions(+), 19 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index fe3f5d0f..be70c58b 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -206,7 +206,7 @@ _graphify_python_bin() { } _graphify_diagnostics_enabled() { - case "${LDS_GRAPHIFY_DIAGNOSTICS:-1}" in + case "${LDS_GRAPHIFY_DIAGNOSTICS:-0}" in 1 | true | TRUE | yes | YES | on | ON) return 0 ;; 0 | false | FALSE | no | NO | off | OFF) return 1 ;; *) die "LDS_GRAPHIFY_DIAGNOSTICS must be true/false" ;; @@ -221,7 +221,8 @@ cmd_graphify() { [[ -e "$target" ]] || die "Graphify target does not exist: $target" local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs - local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="" graphify_think="off" + local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" + local diagnostic_mode="off" local local_provider=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -364,18 +365,21 @@ cmd_graphify() { target_abs="$(_realpath "$target")" provider_dir="$(mktemp -d)" || die "Unable to create temporary Graphify provider directory" + graphify_python="$(_graphify_python_bin "$graphify_bin")" || + die "Unable to find the Python interpreter required for LocalDevStack Graphify structured output" + + if [[ -d "$target_abs" ]]; then + diagnostic_root="$target_abs" + else + diagnostic_root="$(dirname "$target_abs")" + fi + diagnostic_log="${LDS_GRAPHIFY_DIAGNOSTIC_LOG:-$diagnostic_root/graphify-out/lds-graphify-diagnostics.jsonl}" + diagnostic_preview="${LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW:-4096}" + [[ "$diagnostic_preview" =~ ^[0-9]+$ ]] && ((diagnostic_preview >= 256)) || + die "LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW must be an integer >= 256" + if _graphify_diagnostics_enabled; then - graphify_python="$(_graphify_python_bin "$graphify_bin")" || - die "Unable to find the Python interpreter required for Graphify diagnostics" - if [[ -d "$target_abs" ]]; then - diagnostic_root="$target_abs" - else - diagnostic_root="$(dirname "$target_abs")" - fi - diagnostic_log="${LDS_GRAPHIFY_DIAGNOSTIC_LOG:-$diagnostic_root/graphify-out/lds-graphify-diagnostics.jsonl}" - diagnostic_preview="${LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW:-4096}" - [[ "$diagnostic_preview" =~ ^[0-9]+$ ]] && ((diagnostic_preview >= 256)) || - die "LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW must be an integer >= 256" + diagnostic_mode=on fi fi @@ -395,13 +399,16 @@ cmd_graphify() { local_provider_base_url="$base_url" if [[ -n "$graphify_python" ]]; then - ready_file="$provider_dir/diagnostic-proxy.port" - mkdir -p "$(dirname "$diagnostic_log")" - : >"$diagnostic_log" + ready_file="$provider_dir/graphify-proxy.port" + if [[ "$diagnostic_mode" == on ]]; then + mkdir -p "$(dirname "$diagnostic_log")" + : >"$diagnostic_log" + fi "$graphify_python" "$DIR/scripts/graphify-diagnostic-proxy.py" \ --upstream "${base_url%/v1}" \ --provider "$provider" \ + --diagnostics "$diagnostic_mode" \ --ready-file "$ready_file" \ --log-file "$diagnostic_log" \ --preview-chars "$diagnostic_preview" & @@ -414,14 +421,16 @@ cmd_graphify() { break fi kill -0 "$proxy_pid" >/dev/null 2>&1 || - die "Local Graphify diagnostic proxy exited before becoming ready" + die "Local Graphify structured-output proxy exited before becoming ready" sleep 0.05 done [[ "$proxy_port" =~ ^[0-9]+$ ]] || - die "Local Graphify diagnostic proxy did not become ready" + die "Local Graphify structured-output proxy did not become ready" local_provider_base_url="http://127.0.0.1:${proxy_port}/v1" - printf '%s\n' "[lds graphify] suspect-response diagnostics enabled: $diagnostic_log" >&2 + if [[ "$diagnostic_mode" == on ]]; then + printf '%s\n' "[lds graphify] suspect-response diagnostics enabled: $diagnostic_log" >&2 + fi fi backend="$(_graphify_write_local_provider "$provider_dir" "$provider" "$local_provider_base_url" "$model" "$token_budget_value" "$graphify_think")" || From 8ccbfb0b4c34928b13fef46cfe17f35bdb421f82 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:57:20 +0600 Subject: [PATCH 481/509] test(graphify): lock always-on structured proxy contract --- tests/ai-contract.sh | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 885e7a27..1c789839 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -104,7 +104,8 @@ assert_file_contains "$ROOT/lib/ai.sh" 'reasoning_effort: "none"' assert_file_contains "$ROOT/lib/ai.sh" 'graphify-diagnostic-proxy.py' assert_file_contains "$ROOT/lib/ai.sh" 'http://127.0.0.1:${proxy_port}/v1' assert_file_contains "$ROOT/lib/ai.sh" '--provider "$provider"' -assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-1' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-0' +assert_file_contains "$ROOT/lib/ai.sh" '--diagnostics "$diagnostic_mode"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_THINK:-off' assert_file_contains "$ROOT/lib/ai.sh" 'lds-graphify-diagnostics.jsonl' assert_file_contains "$ROOT/lib/ai.sh" 'llm think ' @@ -220,6 +221,14 @@ assert module.parse_graph_content('{"nodes":[],"edges":[],"hyperedges":[]}') == } assert module.parse_graph_content('{"nodes":["bad"],"edges":[],"hyperedges":[]}') is None +assert module.parse_graph_content( + '{"nodes":[{"id":"a"}],"edges":[],"hyperedges":[]}' +) == { + "nodes": [{"id": "a"}], + "edges": [], + "hyperedges": [], +} + body = json.dumps({ "model": "qwen3.5:9b", "messages": [ From 817c7e49051e1918c3f9038565abb368543c551f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:57:44 +0600 Subject: [PATCH 482/509] docs(graphify): document provider-native structured extraction --- docs/guides/local-ai.rst | 74 ++++++++++++++++++++++------------------ 1 file changed, 40 insertions(+), 34 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index acb34db4..87e6772c 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -261,10 +261,11 @@ For the built-in LocalDevStack endpoint, Graphify runs through an ephemeral provider definition created only for that invocation. The provider definition itself is never written into the target repository or ``~/.graphify/providers.json``. -Local Graphify runs enable a localhost diagnostic proxy by default. The temporary -provider points to ``127.0.0.1:/v1``, and the proxy forwards the request -body unchanged to ``http://llm.localhost:11434/v1``. It inspects semantic-extraction -responses only; community-label requests are ignored. +Local Graphify runs always use a localhost structured-output compatibility proxy. +The temporary provider points to ``127.0.0.1:/v1``, and the proxy +forwards requests to ``http://llm.localhost:11434/v1``. Community-label requests +pass through unchanged; semantic-extraction requests use the active provider's native +structured-output mechanism. FastFlow / NPU: @@ -288,44 +289,49 @@ local chunks. Both local providers default to ``--token-budget 4000 no-thinking request are separate protections: the former prevents local context/resource pressure, while the latter keeps reasoning out of the structured response channel. -When a semantic response itself looks suspect (empty content, valid-but-empty graph -JSON, malformed/non-graph JSON, or graph arrays with no object entries), LocalDevStack -prints a bounded assistant-content preview and stores the full suspect assistant -response as JSON Lines in: +Structured extraction is provider-specific: -.. code-block:: text +* FastFlow / Qwen3.5 uses native tool calling with a single ``submit_graph`` + function whose arguments follow Graphify's node/edge/hyperedge schema. The proxy + converts valid tool arguments back into the normal assistant JSON content that + Graphify already understands. FastFlow currently ignores OpenAI + ``response_format`` on its chat-completions path, so tool calling is the + supported structured channel. +* Ollama uses its OpenAI-compatible ``response_format.type=json_schema`` path with + the same Graphify schema and ``temperature=0``. Ollama maps that schema to its + native structured-output ``format`` field. - /graphify-out/lds-graphify-diagnostics.jsonl +A structurally valid all-empty graph remains valid and is passed back to Graphify +unchanged; Graphify then decides whether to retry it as a hollow extraction. If a +provider's structured path itself fails or returns malformed data, LocalDevStack +falls back once to the original free-form Graphify request and leaves Graphify's +normal retry policy intact. -For FastFlow only, malformed semantic responses also get one provider-native -structured recovery attempt. LocalDevStack replays the same extraction request with -FastFlow's tool-calling channel and a ``submit_graph`` function, then converts valid -tool arguments back into the normal Graphify JSON response. The original prompt and -source corpus never leave the same local provider; the recovery only changes the -response channel from free-form assistant text to structured tool arguments. If the -tool call is absent or invalid, LocalDevStack returns the original response and -Graphify applies its normal retry policy. +Detailed suspect-response logging is optional and does not control the compatibility +proxy. Enable it with: -The diagnostic record contains request controls such as model, ``think``, -``reasoning_effort``, finish reason, and token usage, but never stores the Graphify -prompt or source corpus. Set ``LDS_GRAPHIFY_DIAGNOSTICS=0`` to bypass the proxy. -``LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW`` controls the terminal preview size (minimum 256, -default 4096). ``LDS_GRAPHIFY_DIAGNOSTIC_LOG`` overrides the JSONL path. +.. code-block:: bash -FastFlow Graphify defaults to ``LDS_GRAPHIFY_THINK=off``. For a controlled -instruction-following A/B test, run the same corpus once with thinking enabled: + LDS_GRAPHIFY_DIAGNOSTICS=1 lds graphify . -.. code-block:: bash +When enabled, a bounded assistant-content preview is printed and the full suspect +assistant response is stored as JSON Lines in: - LDS_GRAPHIFY_THINK=on lds graphify . +.. code-block:: text + + /graphify-out/lds-graphify-diagnostics.jsonl -The ``on`` mode sends both ``think=true`` and -``reasoning_effort=high``. FastFlow keeps reasoning in -``reasoning_content`` and the final answer in ``message.content``, so the -Graphify parser still receives only the final graph JSON. Use -``LDS_GRAPHIFY_THINK=auto`` to omit both controls and return to the model/provider -default. This switch is intended for diagnosis until the better-performing mode is -confirmed on the real corpus. +The diagnostic record contains request controls such as model, ``think``, +``reasoning_effort``, finish reason, and token usage, but never stores the Graphify +prompt or source corpus. ``LDS_GRAPHIFY_DIAGNOSTIC_PREVIEW`` controls the terminal +preview size (minimum 256, default 4096). ``LDS_GRAPHIFY_DIAGNOSTIC_LOG`` overrides +the JSONL path. + +FastFlow Graphify still defaults to ``LDS_GRAPHIFY_THINK=off``. The current +FastFlow Qwen3.5 non-stream parser can leave ``...`` text inside +``message.content``, so thinking is intentionally kept off for Graphify's +structured extraction path. ``LDS_GRAPHIFY_THINK=on`` and +``LDS_GRAPHIFY_THINK=auto`` remain diagnostic overrides, not recommended defaults. Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the selected model is absent. Override the local chunk defaults with explicit Graphify From d399cfe887cb13271c3b0866d5b2bad3dfadf0b3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 17:59:14 +0600 Subject: [PATCH 483/509] fix(graphify): align edge relations with Graphify schema --- scripts/graphify-diagnostic-proxy.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 5243db85..80f1fbf1 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -62,7 +62,7 @@ "enum": [ "calls", "implements", "references", "cites", "conceptually_related_to", "shares_data_with", - "semantically_similar_to", "rationale_for", + "semantically_similar_to", ], }, "confidence": { From f010d1f01ae067310978d26d376982fa99c2b400 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:00:23 +0600 Subject: [PATCH 484/509] test(docs): align Graphify structured-output guidance --- tests/docs-contract.sh | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index dd7f6637..d3b68f01 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -90,7 +90,9 @@ assert_file_contains "$ai" 'extra_body={"think": false}' assert_file_contains "$ai" 'reasoning_effort=none' assert_file_contains "$ai" 'lds llm think' assert_file_contains "$ai" 'LDS_AI_THINK' -assert_file_contains "$ai" 'LDS_GRAPHIFY_DIAGNOSTICS=0' +assert_file_contains "$ai" 'LDS_GRAPHIFY_DIAGNOSTICS=1' +assert_file_contains "$ai" 'submit_graph' +assert_file_contains "$ai" 'response_format.type=json_schema' assert_file_contains "$ai" 'lds-graphify-diagnostics.jsonl' assert_file_contains "$ai" 'think' assert_file_contains "$ai" 'reasoning_effort' From e773a333accee66eb9ff925afbf144084f5ce4e3 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:09:01 +0600 Subject: [PATCH 485/509] fix(graphify): bound structured FastFlow retries --- scripts/graphify-diagnostic-proxy.py | 115 +++++++++++++++++++++++---- 1 file changed, 98 insertions(+), 17 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 80f1fbf1..5fc77675 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -20,6 +20,8 @@ from typing import Any _GRAPH_KEYS = ("nodes", "edges", "hyperedges") +_STRUCTURED_MAX_TOKENS = 4096 + _GRAPH_SCHEMA = { "type": "object", "required": ["nodes", "edges", "hyperedges"], @@ -250,6 +252,8 @@ def _build_ollama_schema_request(body: bytes) -> bytes | None: request = dict(request) request["stream"] = False request["temperature"] = 0 + request.pop("max_tokens", None) + request["max_completion_tokens"] = _bounded_completion_cap(request) request["response_format"] = { "type": "json_schema", "json_schema": { @@ -261,7 +265,16 @@ def _build_ollama_schema_request(body: bytes) -> bytes | None: return json.dumps(request, ensure_ascii=False).encode("utf-8") -def _build_tool_recovery_request(body: bytes) -> bytes | None: +def _bounded_completion_cap(request: dict[str, Any]) -> int: + raw = request.get("max_completion_tokens", request.get("max_tokens", _STRUCTURED_MAX_TOKENS)) + try: + cap = int(raw) + except (TypeError, ValueError): + cap = _STRUCTURED_MAX_TOKENS + return max(1, min(cap, _STRUCTURED_MAX_TOKENS)) + + +def _build_tool_recovery_request(body: bytes, *, retry: bool = False) -> bytes | None: try: request = json.loads(body.decode("utf-8")) except (UnicodeDecodeError, json.JSONDecodeError): @@ -275,18 +288,27 @@ def _build_tool_recovery_request(body: bytes) -> bytes | None: request = dict(request) request["tools"] = [_GRAPH_TOOL] - # FastFlow v1.0.6 treats unsupported tool_choice modes as auto, so the - # explicit instruction is the enforcement mechanism for Qwen3.5. + # FastFlow currently supports tool_choice=auto|none only. The explicit + # system instruction is therefore what makes submit_graph mandatory. request["tool_choice"] = "auto" request["stream"] = False + request["temperature"] = 0 + request.pop("max_tokens", None) + request["max_completion_tokens"] = _bounded_completion_cap(request) amended = [] recovery_suffix = ( - "\n\nSTRUCTURED RECOVERY: Do not emit the graph as assistant text. " + "\n\nSTRUCTURED OUTPUT: Do not emit the graph as ordinary assistant text. " "Call the submit_graph tool exactly once. Put the complete extraction " "fragment into its nodes, edges, and hyperedges arguments. Do not add " "new facts; follow the original Graphify schema and source_file rules." ) + if retry: + recovery_suffix += ( + " A previous structured attempt did not produce a usable submit_graph call. " + "This retry MUST call submit_graph exactly once; do not answer with prose or raw JSON." + ) + injected = False for message in messages: if not isinstance(message, dict): @@ -365,6 +387,14 @@ def _extract_graph_tool_result(body: bytes) -> dict[str, Any] | None: return None +def _extract_fastflow_structured_graph(body: bytes) -> dict[str, Any] | None: + graph = _extract_graph_tool_result(body) + if graph is not None: + return graph + _meta, content = _response_metadata(body) + return parse_graph_content(content) + + def _replace_response_content(body: bytes, graph: dict[str, Any]) -> bytes | None: try: response = json.loads(body.decode("utf-8")) @@ -456,6 +486,8 @@ class DiagnosticHandler(BaseHTTPRequestHandler): upstream: str = "" provider: str = "" diagnostics: bool = False + upstream_timeout: int = 1800 + structured_timeout: int = 300 log_file: Path preview_chars: int = 4096 @@ -502,7 +534,8 @@ def _forward(self) -> None: response_body = b'{"error":{"message":"LocalDevStack Graphify diagnostic proxy upstream failure"}}' try: - with urllib.request.urlopen(request, timeout=1900) as upstream_response: + request_timeout = self.structured_timeout if structured_primary else self.upstream_timeout + with urllib.request.urlopen(request, timeout=request_timeout) as upstream_response: status = upstream_response.status response_body = upstream_response.read() content_type = upstream_response.headers.get("Content-Type") @@ -521,9 +554,29 @@ def _forward(self) -> None: if is_chat and extraction_request: if structured_primary == "fastflow-tool" and 200 <= status < 300: - graph = _extract_graph_tool_result(response_body) + graph = _extract_fastflow_structured_graph(response_body) + structured_response = response_body + + if graph is None: + retry_body = _build_tool_recovery_request(body, retry=True) + retry_response = ( + self._post_upstream(retry_body, timeout=self.structured_timeout) + if retry_body is not None + else None + ) + if retry_response is not None: + retry_graph = _extract_fastflow_structured_graph(retry_response) + if retry_graph is not None: + graph = retry_graph + structured_response = retry_response + print( + "[lds graphify] FastFlow structured extraction recovered on bounded retry", + file=sys.stderr, + flush=True, + ) + if graph is not None: - replacement = _replace_response_content(response_body, graph) + replacement = _replace_response_content(structured_response, graph) if replacement is not None: response_body = replacement if self.diagnostics: @@ -567,18 +620,33 @@ def _forward(self) -> None: self.wfile.write(response_body) def _fallback_freeform(self, original_body: bytes) -> bytes: - fallback = self._post_upstream(original_body) + try: + fallback_request = json.loads(original_body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + fallback_request = None + if isinstance(fallback_request, dict): + fallback_request = dict(fallback_request) + fallback_request.pop("max_tokens", None) + fallback_request["max_completion_tokens"] = _bounded_completion_cap(fallback_request) + fallback_request["temperature"] = 0 + fallback_body = json.dumps(fallback_request, ensure_ascii=False).encode("utf-8") + else: + fallback_body = original_body + + fallback = self._post_upstream(fallback_body, timeout=self.structured_timeout) if fallback is None: return b'{"error":{"message":"Structured Graphify extraction failed and free-form fallback was unavailable"}}' print( "[lds graphify] structured extraction did not yield a usable graph; " - "falling back to the original free-form Graphify request", + "falling back once to the bounded free-form Graphify request", file=sys.stderr, flush=True, ) - return self._inspect_and_recover_chat_response(original_body, fallback, 200) + return self._inspect_and_recover_chat_response( + original_body, fallback, 200, allow_structured_recovery=False + ) - def _post_upstream(self, body: bytes) -> bytes | None: + def _post_upstream(self, body: bytes, *, timeout: int | None = None) -> bytes | None: request = urllib.request.Request( self.upstream.rstrip("/") + "/v1/chat/completions", data=body, @@ -589,7 +657,7 @@ def _post_upstream(self, body: bytes) -> bytes | None: method="POST", ) try: - with urllib.request.urlopen(request, timeout=1900) as response: + with urllib.request.urlopen(request, timeout=timeout or self.upstream_timeout) as response: if response.status < 200 or response.status >= 300: return None return response.read() @@ -597,7 +665,12 @@ def _post_upstream(self, body: bytes) -> bytes | None: return None def _inspect_and_recover_chat_response( - self, request_body: bytes, response_body: bytes, status: int + self, + request_body: bytes, + response_body: bytes, + status: int, + *, + allow_structured_recovery: bool = True, ) -> bytes: req = _request_metadata(request_body) resp, content = _response_metadata(response_body) @@ -613,10 +686,14 @@ def _inspect_and_recover_chat_response( recovery = _build_tool_recovery_request(request_body) recovered_graph = None recovered_response = None - if recovery is not None and req.get("think", "") != "": - recovered_response = self._post_upstream(recovery) + if ( + allow_structured_recovery + and recovery is not None + and req.get("think", "") != "" + ): + recovered_response = self._post_upstream(recovery, timeout=self.structured_timeout) if recovered_response is not None: - recovered_graph = _extract_graph_tool_result(recovered_response) + recovered_graph = _extract_fastflow_structured_graph(recovered_response) if recovered_graph is not None and recovered_response is not None: replacement = _replace_response_content(recovered_response, recovered_graph) @@ -676,7 +753,7 @@ def _inspect_and_recover_chat_response( file=sys.stderr, flush=True, ) - if recovery is not None: + if allow_structured_recovery and recovery is not None: print( "[lds graphify diagnostic] structured submit_graph recovery did not yield a usable graph; " "returning the original response so Graphify can apply its normal retry policy", @@ -695,6 +772,8 @@ def main() -> int: parser.add_argument("--upstream", required=True) parser.add_argument("--provider", required=True, choices=("fastflow", "ollama")) parser.add_argument("--diagnostics", choices=("on", "off"), default="off") + parser.add_argument("--timeout", type=int, default=1800) + parser.add_argument("--structured-timeout", type=int, default=300) parser.add_argument("--ready-file", required=True) parser.add_argument("--log-file", required=True) parser.add_argument("--preview-chars", type=int, default=4096) @@ -703,6 +782,8 @@ def main() -> int: DiagnosticHandler.upstream = args.upstream DiagnosticHandler.provider = args.provider DiagnosticHandler.diagnostics = args.diagnostics == "on" + DiagnosticHandler.upstream_timeout = max(1, args.timeout) + DiagnosticHandler.structured_timeout = max(1, args.structured_timeout) DiagnosticHandler.log_file = Path(args.log_file) DiagnosticHandler.preview_chars = max(256, args.preview_chars) From d84beec9b54477359828e48af85071ea5ff0a176 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:09:15 +0600 Subject: [PATCH 486/509] feat(graphify): expose bounded structured timeout --- lib/ai.sh | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/lib/ai.sh b/lib/ai.sh index be70c58b..b06626d0 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -222,7 +222,7 @@ cmd_graphify() { local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" - local diagnostic_mode="off" + local diagnostic_mode="off" structured_timeout="" local local_provider=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -259,6 +259,10 @@ cmd_graphify() { timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" + structured_timeout="${LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-300}" + [[ "$structured_timeout" =~ ^[0-9]+$ ]] && ((structured_timeout >= 1)) || + die "LDS_GRAPHIFY_STRUCTURED_TIMEOUT must be a positive integer" + case "${LDS_GRAPHIFY_THINK:-off}" in off | false | 0) graphify_think=off ;; on | true | 1) graphify_think=on ;; @@ -409,6 +413,8 @@ cmd_graphify() { --upstream "${base_url%/v1}" \ --provider "$provider" \ --diagnostics "$diagnostic_mode" \ + --timeout "$timeout" \ + --structured-timeout "$structured_timeout" \ --ready-file "$ready_file" \ --log-file "$diagnostic_log" \ --preview-chars "$diagnostic_preview" & From 7afb7ae62d313ed3f2287829f348146621149e12 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:09:33 +0600 Subject: [PATCH 487/509] test(graphify): cover bounded structured retry --- tests/ai-contract.sh | 28 +++++++++++++++++++++++++++- 1 file changed, 27 insertions(+), 1 deletion(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 1c789839..3ec5cfa4 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -249,7 +249,9 @@ recovery_json = json.loads(recovery) assert recovery_json["tools"][0]["function"]["name"] == "submit_graph" assert recovery_json["tool_choice"] == "auto" assert recovery_json["think"] is False -assert "STRUCTURED RECOVERY" in recovery_json["messages"][0]["content"] +assert "STRUCTURED OUTPUT" in recovery_json["messages"][0]["content"] +assert recovery_json["temperature"] == 0 +assert recovery_json["max_completion_tokens"] == 4096 assert recovery_json["tools"][0]["function"]["parameters"] == module._GRAPH_SCHEMA assert "rationale_for" not in module._GRAPH_SCHEMA["properties"]["edges"]["items"]["properties"]["relation"]["enum"] @@ -271,6 +273,7 @@ ollama_json = json.loads(ollama_request) assert ollama_json["reasoning_effort"] == "none" assert ollama_json["options"]["num_ctx"] == 8192 assert ollama_json["temperature"] == 0 +assert ollama_json["max_completion_tokens"] == 4096 assert ollama_json["response_format"]["type"] == "json_schema" assert ollama_json["response_format"]["json_schema"]["strict"] is True assert ollama_json["response_format"]["json_schema"]["schema"] == module._GRAPH_SCHEMA @@ -307,6 +310,29 @@ assert graph == { "edges": [], "hyperedges": [], } + +assert module._extract_fastflow_structured_graph(json.dumps(tool_response).encode()) == graph + +content_only_response = { + "choices": [{ + "index": 0, + "finish_reason": "stop", + "message": { + "role": "assistant", + "content": json.dumps(graph), + }, + }], + "usage": {"prompt_tokens": 10, "completion_tokens": 20, "total_tokens": 30}, +} +assert module._extract_fastflow_structured_graph( + json.dumps(content_only_response).encode() +) == graph + +retry = module._build_tool_recovery_request(body, retry=True) +assert retry is not None +retry_json = json.loads(retry) +assert "previous structured attempt" in retry_json["messages"][0]["content"] +assert retry_json["max_completion_tokens"] == 4096 replacement = module._replace_response_content(json.dumps(tool_response).encode(), graph) assert replacement is not None replacement_json = json.loads(replacement) From 5bc906b3ef67bddd9a0c3c9e00dcf4ce43f61b2d Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:10:17 +0600 Subject: [PATCH 488/509] docs(graphify): document bounded structured extraction --- docs/guides/local-ai.rst | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 87e6772c..591de415 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -302,10 +302,21 @@ Structured extraction is provider-specific: native structured-output ``format`` field. A structurally valid all-empty graph remains valid and is passed back to Graphify -unchanged; Graphify then decides whether to retry it as a hollow extraction. If a -provider's structured path itself fails or returns malformed data, LocalDevStack -falls back once to the original free-form Graphify request and leaves Graphify's -normal retry policy intact. +unchanged; Graphify then decides whether to retry it as a hollow extraction. + +Structured generations are deliberately bounded independently of Graphify's larger +general output allowance. LocalDevStack caps a structured extraction at 4096 output +tokens and defaults each structured request to a 300-second timeout +(``LDS_GRAPHIFY_STRUCTURED_TIMEOUT``). This is important for FastFlow because its +Qwen3.5 non-stream tool parser recognizes ```` only after generation +finishes; without a smaller bound, a malformed/non-terminating tool response can run +toward Graphify's 16384-token completion cap for many minutes. + +FastFlow accepts one additional bounded structured retry when the first response has +neither a usable ``submit_graph`` call nor a valid graph in ``message.content``. +Only after those structured attempts fail does LocalDevStack make one bounded +free-form fallback request. Ollama uses the same output/timeout bounds on its native +JSON-schema request. Graphify's own retry policy remains the final fallback. Detailed suspect-response logging is optional and does not control the compatibility proxy. Enable it with: From dc57dbe8e59f8cd3679e7b8f26f485a8f262f00b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:10:28 +0600 Subject: [PATCH 489/509] test(graphify): lock structured timeout contract --- tests/ai-contract.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 3ec5cfa4..2a689928 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -106,6 +106,8 @@ assert_file_contains "$ROOT/lib/ai.sh" 'http://127.0.0.1:${proxy_port}/v1' assert_file_contains "$ROOT/lib/ai.sh" '--provider "$provider"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-0' assert_file_contains "$ROOT/lib/ai.sh" '--diagnostics "$diagnostic_mode"' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-300' +assert_file_contains "$ROOT/lib/ai.sh" '--structured-timeout "$structured_timeout"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_THINK:-off' assert_file_contains "$ROOT/lib/ai.sh" 'lds-graphify-diagnostics.jsonl' assert_file_contains "$ROOT/lib/ai.sh" 'llm think ' From d0d21e940b3aaef9cd3423cebc35864ff368f9fc Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:53:19 +0600 Subject: [PATCH 490/509] feat(graphify): surface incremental update mode --- lib/ai.sh | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index b06626d0..0288f013 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -223,7 +223,7 @@ cmd_graphify() { local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" local diagnostic_mode="off" structured_timeout="" - local local_provider=0 + local local_provider=0 existing_graph=0 force_rebuild=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 local token_budget_value="" max_concurrency_value="" @@ -315,6 +315,9 @@ cmd_graphify() { has_max_concurrency=1 max_concurrency_value="${arg#--max-concurrency=}" ;; + --force) + force_rebuild=1 + ;; --backend | --backend=*) die "lds graphify selects the Graphify backend from the active LLM provider; do not pass --backend" ;; @@ -362,11 +365,21 @@ cmd_graphify() { ((local_provider == 0)) || _graphify_local_model_preflight "$model" graphify_bin="$(bin_path graphify)" - target_abs="$target" + target_abs="$(_realpath "$target")" provider_dir="" + if [[ -d "$target_abs" && -f "$target_abs/graphify-out/graph.json" ]]; then + existing_graph=1 + if ((force_rebuild)); then + printf '%s\n' "[lds graphify] existing graph detected; --force requested, performing a full rebuild" >&2 + else + printf '%s\n' "[lds graphify] existing graph detected; using Graphify incremental update (changed files only)" >&2 + fi + else + printf '%s\n' "[lds graphify] no existing graph detected; performing initial full build" >&2 + fi + if ((local_provider)); then - target_abs="$(_realpath "$target")" provider_dir="$(mktemp -d)" || die "Unable to create temporary Graphify provider directory" graphify_python="$(_graphify_python_bin "$graphify_bin")" || From 1c47a602ffe07b5dcadd23a4ce6693a512d7eb04 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:53:31 +0600 Subject: [PATCH 491/509] fix(graphify): keep incremental mode detection minimal --- lib/ai.sh | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index 0288f013..c7273779 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -223,7 +223,7 @@ cmd_graphify() { local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" local diagnostic_mode="off" structured_timeout="" - local local_provider=0 existing_graph=0 force_rebuild=0 + local local_provider=0 force_rebuild=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 local token_budget_value="" max_concurrency_value="" @@ -369,7 +369,6 @@ cmd_graphify() { provider_dir="" if [[ -d "$target_abs" && -f "$target_abs/graphify-out/graph.json" ]]; then - existing_graph=1 if ((force_rebuild)); then printf '%s\n' "[lds graphify] existing graph detected; --force requested, performing a full rebuild" >&2 else From ac053a7e18fb34ac941ff5fae5806efbd13764a2 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:53:46 +0600 Subject: [PATCH 492/509] test(graphify): lock incremental update mode detection --- tests/ai-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 2a689928..72414b88 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -107,6 +107,9 @@ assert_file_contains "$ROOT/lib/ai.sh" '--provider "$provider"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-0' assert_file_contains "$ROOT/lib/ai.sh" '--diagnostics "$diagnostic_mode"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-300' +assert_file_contains "$ROOT/lib/ai.sh" 'existing graph detected; using Graphify incremental update (changed files only)' +assert_file_contains "$ROOT/lib/ai.sh" 'existing graph detected; --force requested, performing a full rebuild' +assert_file_contains "$ROOT/lib/ai.sh" 'no existing graph detected; performing initial full build' assert_file_contains "$ROOT/lib/ai.sh" '--structured-timeout "$structured_timeout"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_THINK:-off' assert_file_contains "$ROOT/lib/ai.sh" 'lds-graphify-diagnostics.jsonl' From 0226b15fd146c50935579d28901e0dcc20fdf06b Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:54:10 +0600 Subject: [PATCH 493/509] docs(graphify): clarify incremental existing-graph behavior --- docs/guides/local-ai.rst | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 591de415..04ac42ac 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -345,10 +345,20 @@ structured extraction path. ``LDS_GRAPHIFY_THINK=on`` and ``LDS_GRAPHIFY_THINK=auto`` remain diagnostic overrides, not recommended defaults. Before extraction, LocalDevStack checks ``/v1/models`` and fails fast when the -selected model is absent. Override the local chunk defaults with explicit Graphify -flags, or set ``LDS_GRAPHIFY_TOKEN_BUDGET`` / -``LDS_GRAPHIFY_MAX_CONCURRENCY``. If a local model reports ``Max length reached!``, -reduce the token budget further, for example: +selected model is absent. + +When ``/graphify-out/graph.json`` already exists, ``lds graphify`` keeps +using Graphify's lower-level ``extract`` pipeline, which automatically switches to +incremental mode: only changed code/docs/papers/images are re-extracted, deleted or +excluded sources are reconciled, and the result is merged into the existing graph. +This is intentionally preferred over the literal ``graphify update`` CLI command, +because current Graphify ``update`` refreshes code only and delegates semantic +document refreshes to the assistant update workflow. Pass ``--force`` only when a +full rebuild is intentionally required. + +Override the local chunk defaults with explicit Graphify flags, or set +``LDS_GRAPHIFY_TOKEN_BUDGET`` / ``LDS_GRAPHIFY_MAX_CONCURRENCY``. If a local +model reports ``Max length reached!``, reduce the token budget further, for example: .. code-block:: bash From 23ff9e88f425bdfac18c1f7af04acc520de3cf90 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 18:54:22 +0600 Subject: [PATCH 494/509] test(docs): lock incremental Graphify guidance --- tests/docs-contract.sh | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/docs-contract.sh b/tests/docs-contract.sh index d3b68f01..3a6394b7 100644 --- a/tests/docs-contract.sh +++ b/tests/docs-contract.sh @@ -93,6 +93,9 @@ assert_file_contains "$ai" 'LDS_AI_THINK' assert_file_contains "$ai" 'LDS_GRAPHIFY_DIAGNOSTICS=1' assert_file_contains "$ai" 'submit_graph' assert_file_contains "$ai" 'response_format.type=json_schema' +assert_file_contains "$ai" 'automatically switches to' +assert_file_contains "$ai" 'incremental mode' +assert_file_contains "$ai" 'Pass ``--force`` only when a' assert_file_contains "$ai" 'lds-graphify-diagnostics.jsonl' assert_file_contains "$ai" 'think' assert_file_contains "$ai" 'reasoning_effort' From 058c638ec7f54ae84c04bf06fb050fb23efdad09 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:08:59 +0600 Subject: [PATCH 495/509] fix(graphify): fail fast and delegate chunk retries --- scripts/graphify-diagnostic-proxy.py | 307 +++++++++++---------------- 1 file changed, 124 insertions(+), 183 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 5fc77675..163bf0e3 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -11,6 +11,7 @@ import argparse import json import re +import socket import sys import urllib.error import urllib.request @@ -20,7 +21,7 @@ from typing import Any _GRAPH_KEYS = ("nodes", "edges", "hyperedges") -_STRUCTURED_MAX_TOKENS = 4096 +_STRUCTURED_MAX_TOKENS = 2048 _GRAPH_SCHEMA = { "type": "object", @@ -274,7 +275,7 @@ def _bounded_completion_cap(request: dict[str, Any]) -> int: return max(1, min(cap, _STRUCTURED_MAX_TOKENS)) -def _build_tool_recovery_request(body: bytes, *, retry: bool = False) -> bytes | None: +def _build_tool_recovery_request(body: bytes) -> bytes | None: try: request = json.loads(body.decode("utf-8")) except (UnicodeDecodeError, json.JSONDecodeError): @@ -303,12 +304,6 @@ def _build_tool_recovery_request(body: bytes, *, retry: bool = False) -> bytes | "fragment into its nodes, edges, and hyperedges arguments. Do not add " "new facts; follow the original Graphify schema and source_file rules." ) - if retry: - recovery_suffix += ( - " A previous structured attempt did not produce a usable submit_graph call. " - "This retry MUST call submit_graph exactly once; do not answer with prose or raw JSON." - ) - injected = False for message in messages: if not isinstance(message, dict): @@ -395,6 +390,40 @@ def _extract_fastflow_structured_graph(body: bytes) -> dict[str, Any] | None: return parse_graph_content(content) +def _graphify_split_response(body: bytes, model: str | None) -> bytes: + """Return a standard completion that makes Graphify bisect the current chunk.""" + try: + response = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + response = {} + if not isinstance(response, dict): + response = {} + + out = dict(response) + out.setdefault("id", "chatcmpl-lds-graphify-retry") + out.setdefault("object", "chat.completion") + if model: + out["model"] = model + out["choices"] = [{ + "index": 0, + "finish_reason": "length", + "message": { + "role": "assistant", + "content": '{"nodes":[],"edges":[],"hyperedges":[]}', + }, + }] + return json.dumps(out, ensure_ascii=False).encode("utf-8") + + +def _is_timeout_error(exc: BaseException) -> bool: + if isinstance(exc, (TimeoutError, socket.timeout)): + return True + reason = getattr(exc, "reason", None) + if isinstance(reason, (TimeoutError, socket.timeout)): + return True + return "timed out" in str(exc).lower() or "timeout" in str(exc).lower() + + def _replace_response_content(body: bytes, graph: dict[str, Any]) -> bytes | None: try: response = json.loads(body.decode("utf-8")) @@ -487,7 +516,7 @@ class DiagnosticHandler(BaseHTTPRequestHandler): provider: str = "" diagnostics: bool = False upstream_timeout: int = 1800 - structured_timeout: int = 300 + structured_timeout: int = 120 log_file: Path preview_chars: int = 4096 @@ -548,35 +577,26 @@ def _forward(self) -> None: if content_type: response_headers["Content-Type"] = content_type except Exception as exc: - response_body = json.dumps( - {"error": {"message": f"Graphify diagnostic proxy upstream error: {exc}"}} - ).encode("utf-8") + if structured_primary and _is_timeout_error(exc): + status = 200 + response_body = _graphify_split_response(b"", metadata.get("model")) + print( + "[lds graphify] structured provider request timed out after " + f"{self.structured_timeout}s; asking Graphify to split the chunk", + file=sys.stderr, + flush=True, + ) + else: + status = 502 + response_body = json.dumps( + {"error": {"message": f"LocalDevStack Graphify proxy upstream error: {exc}"}} + ).encode("utf-8") - if is_chat and extraction_request: - if structured_primary == "fastflow-tool" and 200 <= status < 300: + if is_chat and extraction_request and 200 <= status < 300: + if structured_primary == "fastflow-tool": graph = _extract_fastflow_structured_graph(response_body) - structured_response = response_body - - if graph is None: - retry_body = _build_tool_recovery_request(body, retry=True) - retry_response = ( - self._post_upstream(retry_body, timeout=self.structured_timeout) - if retry_body is not None - else None - ) - if retry_response is not None: - retry_graph = _extract_fastflow_structured_graph(retry_response) - if retry_graph is not None: - graph = retry_graph - structured_response = retry_response - print( - "[lds graphify] FastFlow structured extraction recovered on bounded retry", - file=sys.stderr, - flush=True, - ) - if graph is not None: - replacement = _replace_response_content(structured_response, graph) + replacement = _replace_response_content(response_body, graph) if replacement is not None: response_body = replacement if self.diagnostics: @@ -588,27 +608,36 @@ def _forward(self) -> None: file=sys.stderr, flush=True, ) - else: - response_body = self._fallback_freeform(body) else: - response_body = self._fallback_freeform(body) - elif structured_primary == "ollama-schema" and 200 <= status < 300: + self._diagnose_suspect_response(body, response_body, status) + response_body = _graphify_split_response(response_body, metadata.get("model")) + print( + "[lds graphify] FastFlow structured response unusable; " + "asking Graphify to split the chunk", + file=sys.stderr, + flush=True, + ) + elif structured_primary == "ollama-schema": _meta, content = _response_metadata(response_body) graph = parse_graph_content(content) if graph is None: - response_body = self._fallback_freeform(body) - else: - if self.diagnostics: - print( - "[lds graphify diagnostic] structured Ollama graph via response_format schema: " - f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}; " - f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " - f"hyperedges={len(graph['hyperedges'])}", - file=sys.stderr, - flush=True, - ) - else: - response_body = self._inspect_and_recover_chat_response(body, response_body, status) + self._diagnose_suspect_response(body, response_body, status) + response_body = _graphify_split_response(response_body, metadata.get("model")) + print( + "[lds graphify] Ollama structured response unusable; " + "asking Graphify to split the chunk", + file=sys.stderr, + flush=True, + ) + elif self.diagnostics: + print( + "[lds graphify diagnostic] structured Ollama graph via response_format schema: " + f"model={metadata.get('model')}; reasoning_effort={metadata.get('reasoning_effort')}; " + f"nodes={len(graph['nodes'])}; edges={len(graph['edges'])}; " + f"hyperedges={len(graph['hyperedges'])}", + file=sys.stderr, + flush=True, + ) self.send_response(status) for name, value in response_headers.items(): @@ -619,98 +648,16 @@ def _forward(self) -> None: if self.command != "HEAD": self.wfile.write(response_body) - def _fallback_freeform(self, original_body: bytes) -> bytes: - try: - fallback_request = json.loads(original_body.decode("utf-8")) - except (UnicodeDecodeError, json.JSONDecodeError): - fallback_request = None - if isinstance(fallback_request, dict): - fallback_request = dict(fallback_request) - fallback_request.pop("max_tokens", None) - fallback_request["max_completion_tokens"] = _bounded_completion_cap(fallback_request) - fallback_request["temperature"] = 0 - fallback_body = json.dumps(fallback_request, ensure_ascii=False).encode("utf-8") - else: - fallback_body = original_body - - fallback = self._post_upstream(fallback_body, timeout=self.structured_timeout) - if fallback is None: - return b'{"error":{"message":"Structured Graphify extraction failed and free-form fallback was unavailable"}}' - print( - "[lds graphify] structured extraction did not yield a usable graph; " - "falling back once to the bounded free-form Graphify request", - file=sys.stderr, - flush=True, - ) - return self._inspect_and_recover_chat_response( - original_body, fallback, 200, allow_structured_recovery=False - ) - - def _post_upstream(self, body: bytes, *, timeout: int | None = None) -> bytes | None: - request = urllib.request.Request( - self.upstream.rstrip("/") + "/v1/chat/completions", - data=body, - headers={ - "Content-Type": "application/json", - "Authorization": self.headers.get("Authorization", "Bearer local"), - }, - method="POST", - ) - try: - with urllib.request.urlopen(request, timeout=timeout or self.upstream_timeout) as response: - if response.status < 200 or response.status >= 300: - return None - return response.read() - except Exception: - return None - - def _inspect_and_recover_chat_response( - self, - request_body: bytes, - response_body: bytes, - status: int, - *, - allow_structured_recovery: bool = True, - ) -> bytes: + def _diagnose_suspect_response( + self, request_body: bytes, response_body: bytes, status: int + ) -> None: req = _request_metadata(request_body) + req.pop("_extraction_request", None) resp, content = _response_metadata(response_body) - extraction_request = req.pop("_extraction_request", False) - if status < 200 or status >= 300 or not extraction_request: - return response_body - suspect, reason = classify_graph_content(content) if not suspect: - return response_body - - original_reason = reason - recovery = _build_tool_recovery_request(request_body) - recovered_graph = None - recovered_response = None - if ( - allow_structured_recovery - and recovery is not None - and req.get("think", "") != "" - ): - recovered_response = self._post_upstream(recovery, timeout=self.structured_timeout) - if recovered_response is not None: - recovered_graph = _extract_fastflow_structured_graph(recovered_response) - - if recovered_graph is not None and recovered_response is not None: - replacement = _replace_response_content(recovered_response, recovered_graph) - if replacement is not None: - recovered_meta, _ = _response_metadata(replacement) - print( - "[lds graphify diagnostic] recovered malformed FastFlow graph via submit_graph tool: " - f"model={req.get('model')}; think={req.get('think')}; " - f"nodes={len(recovered_graph['nodes'])}; edges={len(recovered_graph['edges'])}; " - f"hyperedges={len(recovered_graph['hyperedges'])}; " - f"completion_tokens={recovered_meta.get('completion_tokens')}", - file=sys.stderr, - flush=True, - ) - return replacement + reason = "structured response did not contain a usable provider-native graph" - reason = original_reason print( "[lds graphify] provider returned a suspect extraction response: " f"reason={reason}; model={req.get('model')}; finish_reason={resp.get('finish_reason')}", @@ -718,49 +665,43 @@ def _inspect_and_recover_chat_response( flush=True, ) - if self.diagnostics: - record = { - "timestamp": datetime.now(timezone.utc).isoformat(), - "status": status, - "reason": reason, - **req, - **resp, - "assistant_content": content, - } - self.log_file.parent.mkdir(parents=True, exist_ok=True) - with self.log_file.open("a", encoding="utf-8") as handle: - handle.write(json.dumps(record, ensure_ascii=False) + "\n") - - preview = (content or "")[: self.preview_chars] - print( - "[lds graphify diagnostic] suspect LLM response: " - f"reason={reason}; model={req.get('model')}; " - f"think={req.get('think')}; reasoning_effort={req.get('reasoning_effort')}; " - f"finish_reason={resp.get('finish_reason')}; " - f"prompt_tokens={resp.get('prompt_tokens')}; " - f"completion_tokens={resp.get('completion_tokens')}", - file=sys.stderr, - flush=True, - ) - print( - f"[lds graphify diagnostic] assistant content preview ({len(preview)}/{len(content or '')} chars): " - f"{preview!r}", - file=sys.stderr, - flush=True, - ) - print( - f"[lds graphify diagnostic] full suspect response logged to {self.log_file}", - file=sys.stderr, - flush=True, - ) - if allow_structured_recovery and recovery is not None: - print( - "[lds graphify diagnostic] structured submit_graph recovery did not yield a usable graph; " - "returning the original response so Graphify can apply its normal retry policy", - file=sys.stderr, - flush=True, - ) - return response_body + if not self.diagnostics: + return + + record = { + "timestamp": datetime.now(timezone.utc).isoformat(), + "status": status, + "reason": reason, + **req, + **resp, + "assistant_content": content, + } + self.log_file.parent.mkdir(parents=True, exist_ok=True) + with self.log_file.open("a", encoding="utf-8") as handle: + handle.write(json.dumps(record, ensure_ascii=False) + "\n") + + preview = (content or "")[: self.preview_chars] + print( + "[lds graphify diagnostic] suspect LLM response: " + f"reason={reason}; model={req.get('model')}; " + f"think={req.get('think')}; reasoning_effort={req.get('reasoning_effort')}; " + f"finish_reason={resp.get('finish_reason')}; " + f"prompt_tokens={resp.get('prompt_tokens')}; " + f"completion_tokens={resp.get('completion_tokens')}", + file=sys.stderr, + flush=True, + ) + print( + f"[lds graphify diagnostic] assistant content preview ({len(preview)}/{len(content or '')} chars): " + f"{preview!r}", + file=sys.stderr, + flush=True, + ) + print( + f"[lds graphify diagnostic] full suspect response logged to {self.log_file}", + file=sys.stderr, + flush=True, + ) do_GET = _forward do_HEAD = _forward @@ -773,7 +714,7 @@ def main() -> int: parser.add_argument("--provider", required=True, choices=("fastflow", "ollama")) parser.add_argument("--diagnostics", choices=("on", "off"), default="off") parser.add_argument("--timeout", type=int, default=1800) - parser.add_argument("--structured-timeout", type=int, default=300) + parser.add_argument("--structured-timeout", type=int, default=120) parser.add_argument("--ready-file", required=True) parser.add_argument("--log-file", required=True) parser.add_argument("--preview-chars", type=int, default=4096) From dacb72f0160967c7c7a7ad1cc3a37269a2b8cc45 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:09:26 +0600 Subject: [PATCH 496/509] fix(graphify): disable hidden SDK retry amplification --- lib/ai.sh | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index c7273779..5a659b93 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -222,7 +222,7 @@ cmd_graphify() { local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" - local diagnostic_mode="off" structured_timeout="" + local diagnostic_mode="off" structured_timeout="" graphify_sdk_retries="" local local_provider=0 force_rebuild=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -259,10 +259,14 @@ cmd_graphify() { timeout="${GRAPHIFY_API_TIMEOUT:-$(compose_control_value LDS_AI_TIMEOUT 1800)}" - structured_timeout="${LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-300}" + structured_timeout="${LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-120}" [[ "$structured_timeout" =~ ^[0-9]+$ ]] && ((structured_timeout >= 1)) || die "LDS_GRAPHIFY_STRUCTURED_TIMEOUT must be a positive integer" + graphify_sdk_retries="${GRAPHIFY_MAX_RETRIES:-${LDS_GRAPHIFY_SDK_RETRIES:-0}}" + [[ "$graphify_sdk_retries" =~ ^[0-9]+$ ]] || + die "GRAPHIFY_MAX_RETRIES/LDS_GRAPHIFY_SDK_RETRIES must be a non-negative integer" + case "${LDS_GRAPHIFY_THINK:-off}" in off | false | 0) graphify_think=off ;; on | true | 1) graphify_think=on ;; @@ -412,6 +416,7 @@ cmd_graphify() { export GRAPHIFY_API_TIMEOUT="$timeout" if ((local_provider)); then + export GRAPHIFY_MAX_RETRIES="$graphify_sdk_retries" local_provider_base_url="$base_url" if [[ -n "$graphify_python" ]]; then From db53a9e1226093726d2eaa68b508d0e55748a580 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:09:41 +0600 Subject: [PATCH 497/509] test(graphify): cover fail-fast structured retry delegation --- tests/ai-contract.sh | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 72414b88..10ebca1a 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -106,7 +106,9 @@ assert_file_contains "$ROOT/lib/ai.sh" 'http://127.0.0.1:${proxy_port}/v1' assert_file_contains "$ROOT/lib/ai.sh" '--provider "$provider"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_DIAGNOSTICS:-0' assert_file_contains "$ROOT/lib/ai.sh" '--diagnostics "$diagnostic_mode"' -assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-300' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-120' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_SDK_RETRIES:-0' +assert_file_contains "$ROOT/lib/ai.sh" 'export GRAPHIFY_MAX_RETRIES="$graphify_sdk_retries"' assert_file_contains "$ROOT/lib/ai.sh" 'existing graph detected; using Graphify incremental update (changed files only)' assert_file_contains "$ROOT/lib/ai.sh" 'existing graph detected; --force requested, performing a full rebuild' assert_file_contains "$ROOT/lib/ai.sh" 'no existing graph detected; performing initial full build' @@ -256,7 +258,7 @@ assert recovery_json["tool_choice"] == "auto" assert recovery_json["think"] is False assert "STRUCTURED OUTPUT" in recovery_json["messages"][0]["content"] assert recovery_json["temperature"] == 0 -assert recovery_json["max_completion_tokens"] == 4096 +assert recovery_json["max_completion_tokens"] == 2048 assert recovery_json["tools"][0]["function"]["parameters"] == module._GRAPH_SCHEMA assert "rationale_for" not in module._GRAPH_SCHEMA["properties"]["edges"]["items"]["properties"]["relation"]["enum"] @@ -278,7 +280,7 @@ ollama_json = json.loads(ollama_request) assert ollama_json["reasoning_effort"] == "none" assert ollama_json["options"]["num_ctx"] == 8192 assert ollama_json["temperature"] == 0 -assert ollama_json["max_completion_tokens"] == 4096 +assert ollama_json["max_completion_tokens"] == 2048 assert ollama_json["response_format"]["type"] == "json_schema" assert ollama_json["response_format"]["json_schema"]["strict"] is True assert ollama_json["response_format"]["json_schema"]["schema"] == module._GRAPH_SCHEMA @@ -333,11 +335,6 @@ assert module._extract_fastflow_structured_graph( json.dumps(content_only_response).encode() ) == graph -retry = module._build_tool_recovery_request(body, retry=True) -assert retry is not None -retry_json = json.loads(retry) -assert "previous structured attempt" in retry_json["messages"][0]["content"] -assert retry_json["max_completion_tokens"] == 4096 replacement = module._replace_response_content(json.dumps(tool_response).encode(), graph) assert replacement is not None replacement_json = json.loads(replacement) @@ -347,6 +344,16 @@ assert "tool_calls" not in message assert "reasoning_content" not in message assert replacement_json["choices"][0]["finish_reason"] == "stop" +split = module._graphify_split_response(json.dumps(tool_response).encode(), "qwen3.5:9b") +split_json = json.loads(split) +assert split_json["model"] == "qwen3.5:9b" +assert split_json["choices"][0]["finish_reason"] == "length" +assert json.loads(split_json["choices"][0]["message"]["content"]) == { + "nodes": [], + "edges": [], + "hyperedges": [], +} + string_array_response = json.loads(json.dumps(tool_response)) string_array_response["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"] = json.dumps({ "nodes": json.dumps([{"id": "a"}]), From 7ce29cfc8cb0235335d5ce9da88c81bb5becea6c Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:10:00 +0600 Subject: [PATCH 498/509] docs(graphify): document fail-fast structured extraction --- docs/guides/local-ai.rst | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 04ac42ac..a6fd48a2 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -305,18 +305,21 @@ A structurally valid all-empty graph remains valid and is passed back to Graphif unchanged; Graphify then decides whether to retry it as a hollow extraction. Structured generations are deliberately bounded independently of Graphify's larger -general output allowance. LocalDevStack caps a structured extraction at 4096 output -tokens and defaults each structured request to a 300-second timeout +general output allowance. LocalDevStack caps a structured extraction at 2048 output +tokens and defaults each structured request to a 120-second timeout (``LDS_GRAPHIFY_STRUCTURED_TIMEOUT``). This is important for FastFlow because its Qwen3.5 non-stream tool parser recognizes ```` only after generation finishes; without a smaller bound, a malformed/non-terminating tool response can run toward Graphify's 16384-token completion cap for many minutes. -FastFlow accepts one additional bounded structured retry when the first response has -neither a usable ``submit_graph`` call nor a valid graph in ``message.content``. -Only after those structured attempts fail does LocalDevStack make one bounded -free-form fallback request. Ollama uses the same output/timeout bounds on its native -JSON-schema request. Graphify's own retry policy remains the final fallback. +LocalDevStack deliberately performs exactly one provider-native structured request +per Graphify extraction attempt. It does not add its own structured retry or +free-form fallback chain. If the provider times out or returns an unusable structured +response, the proxy returns a bounded ``finish_reason=length`` signal so Graphify +can split the offending chunk through its existing adaptive-retry logic. For local +providers the OpenAI SDK retry layer also defaults to zero +(``LDS_GRAPHIFY_SDK_RETRIES=0``) to avoid hidden retry amplification; an explicit +``GRAPHIFY_MAX_RETRIES`` still wins. Detailed suspect-response logging is optional and does not control the compatibility proxy. Enable it with: From 670851a36098b04cc12c5b18f33056e274947c02 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:10:35 +0600 Subject: [PATCH 499/509] fix(graphify): bound local adaptive retry depth --- lib/ai.sh | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/lib/ai.sh b/lib/ai.sh index 5a659b93..bd730fac 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -222,7 +222,7 @@ cmd_graphify() { local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" - local diagnostic_mode="off" structured_timeout="" graphify_sdk_retries="" + local diagnostic_mode="off" structured_timeout="" graphify_sdk_retries="" graphify_retry_depth="" local local_provider=0 force_rebuild=0 local next_is_model=0 next_is_timeout=0 next_is_token_budget=0 next_is_max_concurrency=0 local has_token_budget=0 has_max_concurrency=0 @@ -267,6 +267,10 @@ cmd_graphify() { [[ "$graphify_sdk_retries" =~ ^[0-9]+$ ]] || die "GRAPHIFY_MAX_RETRIES/LDS_GRAPHIFY_SDK_RETRIES must be a non-negative integer" + graphify_retry_depth="${GRAPHIFY_MAX_RETRY_DEPTH:-${LDS_GRAPHIFY_MAX_RETRY_DEPTH:-1}}" + [[ "$graphify_retry_depth" =~ ^[0-9]+$ ]] || + die "GRAPHIFY_MAX_RETRY_DEPTH/LDS_GRAPHIFY_MAX_RETRY_DEPTH must be a non-negative integer" + case "${LDS_GRAPHIFY_THINK:-off}" in off | false | 0) graphify_think=off ;; on | true | 1) graphify_think=on ;; @@ -417,6 +421,7 @@ cmd_graphify() { if ((local_provider)); then export GRAPHIFY_MAX_RETRIES="$graphify_sdk_retries" + export GRAPHIFY_MAX_RETRY_DEPTH="$graphify_retry_depth" local_provider_base_url="$base_url" if [[ -n "$graphify_python" ]]; then From 2e20973819570a3f5bd6e34578bd0802881db5a9 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:10:45 +0600 Subject: [PATCH 500/509] test(graphify): lock bounded adaptive retry depth --- tests/ai-contract.sh | 2 ++ 1 file changed, 2 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index 10ebca1a..ce103e0a 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -109,6 +109,8 @@ assert_file_contains "$ROOT/lib/ai.sh" '--diagnostics "$diagnostic_mode"' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_STRUCTURED_TIMEOUT:-120' assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_SDK_RETRIES:-0' assert_file_contains "$ROOT/lib/ai.sh" 'export GRAPHIFY_MAX_RETRIES="$graphify_sdk_retries"' +assert_file_contains "$ROOT/lib/ai.sh" 'LDS_GRAPHIFY_MAX_RETRY_DEPTH:-1' +assert_file_contains "$ROOT/lib/ai.sh" 'export GRAPHIFY_MAX_RETRY_DEPTH="$graphify_retry_depth"' assert_file_contains "$ROOT/lib/ai.sh" 'existing graph detected; using Graphify incremental update (changed files only)' assert_file_contains "$ROOT/lib/ai.sh" 'existing graph detected; --force requested, performing a full rebuild' assert_file_contains "$ROOT/lib/ai.sh" 'no existing graph detected; performing initial full build' From bacee06b48755f74e9545cc4b445f378d9a89e6f Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:10:57 +0600 Subject: [PATCH 501/509] docs(graphify): document bounded retry depth --- docs/guides/local-ai.rst | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index a6fd48a2..875fc0cf 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -316,10 +316,13 @@ LocalDevStack deliberately performs exactly one provider-native structured reque per Graphify extraction attempt. It does not add its own structured retry or free-form fallback chain. If the provider times out or returns an unusable structured response, the proxy returns a bounded ``finish_reason=length`` signal so Graphify -can split the offending chunk through its existing adaptive-retry logic. For local -providers the OpenAI SDK retry layer also defaults to zero -(``LDS_GRAPHIFY_SDK_RETRIES=0``) to avoid hidden retry amplification; an explicit -``GRAPHIFY_MAX_RETRIES`` still wins. +can split the offending chunk through its existing adaptive-retry logic. + +For local providers, hidden retry amplification is bounded at both outer layers: +the OpenAI SDK retry count defaults to zero (``LDS_GRAPHIFY_SDK_RETRIES=0``) and +Graphify's adaptive retry depth defaults to one +(``LDS_GRAPHIFY_MAX_RETRY_DEPTH=1``). Explicit +``GRAPHIFY_MAX_RETRIES`` / ``GRAPHIFY_MAX_RETRY_DEPTH`` values still win. Detailed suspect-response logging is optional and does not control the compatibility proxy. Enable it with: From 7c656906a721a7c7ca7ea40795ece2603d4f63f1 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:13:37 +0600 Subject: [PATCH 502/509] fix(graphify): preserve caller target for explicit providers --- lib/ai.sh | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/lib/ai.sh b/lib/ai.sh index bd730fac..ebef391c 100644 --- a/lib/ai.sh +++ b/lib/ai.sh @@ -220,7 +220,7 @@ cmd_graphify() { [[ $# -eq 0 ]] || shift [[ -e "$target" ]] || die "Graphify target does not exist: $target" - local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs + local runtime provider backend base_url timeout model api_key graphify_bin arg provider_dir target_abs graphify_target local graphify_python="" diagnostic_root="" diagnostic_log="" diagnostic_preview="4096" graphify_think="off" local diagnostic_mode="off" structured_timeout="" graphify_sdk_retries="" graphify_retry_depth="" local local_provider=0 force_rebuild=0 @@ -374,6 +374,7 @@ cmd_graphify() { graphify_bin="$(bin_path graphify)" target_abs="$(_realpath "$target")" + graphify_target="$target" provider_dir="" if [[ -d "$target_abs" && -f "$target_abs/graphify-out/graph.json" ]]; then @@ -387,6 +388,7 @@ cmd_graphify() { fi if ((local_provider)); then + graphify_target="$target_abs" provider_dir="$(mktemp -d)" || die "Unable to create temporary Graphify provider directory" graphify_python="$(_graphify_python_bin "$graphify_bin")" || @@ -485,8 +487,8 @@ cmd_graphify() { esac fi - "$graphify_bin" extract "$target_abs" --backend "$backend" --no-cluster "${graphify_defaults[@]}" "$@" && - "$graphify_bin" cluster-only "$target_abs" --backend "$backend" + "$graphify_bin" extract "$graphify_target" --backend "$backend" --no-cluster "${graphify_defaults[@]}" "$@" && + "$graphify_bin" cluster-only "$graphify_target" --backend "$backend" ) } From 622026e34917d988aef4d621228f3d6efcd9de47 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:19:04 +0600 Subject: [PATCH 503/509] fix(graphify): tolerate FastFlow tool-call argument shapes --- scripts/graphify-diagnostic-proxy.py | 127 +++++++++++++++++++++------ 1 file changed, 100 insertions(+), 27 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 163bf0e3..d43c0ca2 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -325,12 +325,24 @@ def _build_tool_recovery_request(body: bytes) -> bytes | None: return json.dumps(request, ensure_ascii=False).encode("utf-8") -def _coerce_graph_array(value: Any) -> list[dict[str, Any]] | None: - if isinstance(value, str): +def _decode_jsonish(value: Any) -> Any: + """Decode nested JSON strings produced by OpenAI-compatible tool adapters.""" + current = value + for _ in range(3): + if not isinstance(current, str): + break + stripped = current.strip() + if not stripped: + break try: - value = json.loads(value) + current = json.loads(stripped) except json.JSONDecodeError: - return None + break + return current + + +def _coerce_graph_array(value: Any) -> list[dict[str, Any]] | None: + value = _decode_jsonish(value) if not isinstance(value, list): return None if any(not isinstance(entry, dict) for entry in value): @@ -338,6 +350,39 @@ def _coerce_graph_array(value: Any) -> list[dict[str, Any]] | None: return value +def _coerce_graph_object(value: Any) -> dict[str, Any] | None: + value = _decode_jsonish(value) + if not isinstance(value, dict): + return None + + # Some OpenAI-compatible servers wrap the function payload one level deeper. + for wrapper in ("arguments", "graph", "payload", "data"): + if wrapper in value and not all(key in value for key in _GRAPH_KEYS): + nested = _decode_jsonish(value.get(wrapper)) + if isinstance(nested, dict): + value = nested + break + + graph: dict[str, Any] = {} + for key in _GRAPH_KEYS: + array = _coerce_graph_array(value.get(key)) + if array is None: + return None + graph[key] = array + return graph + + +def _tool_call_candidates(message: dict[str, Any]) -> list[dict[str, Any]]: + calls = message.get("tool_calls") + if isinstance(calls, list): + return [call for call in calls if isinstance(call, dict)] + + single = message.get("tool_call") + if isinstance(single, dict): + return [single] + return [] + + def _extract_graph_tool_result(body: bytes) -> dict[str, Any] | None: try: response = json.loads(body.decode("utf-8")) @@ -352,33 +397,26 @@ def _extract_graph_tool_result(body: bytes) -> dict[str, Any] | None: message = choices[0].get("message") if not isinstance(message, dict): return None - tool_calls = message.get("tool_calls") - if not isinstance(tool_calls, list): - return None - for call in tool_calls: - if not isinstance(call, dict): - continue + for call in _tool_call_candidates(message): function = call.get("function") - if not isinstance(function, dict) or function.get("name") != "submit_graph": - continue - arguments = function.get("arguments") - if isinstance(arguments, str): - try: - arguments = json.loads(arguments) - except json.JSONDecodeError: - continue - if not isinstance(arguments, dict): + if isinstance(function, dict): + name = str(function.get("name", "")).strip() + arguments = function.get("arguments") + else: + # Tolerate flatter adapters that place name/arguments on the call. + name = str(call.get("name", "")).strip() + arguments = call.get("arguments") + + graph = _coerce_graph_object(arguments) + if graph is None: continue - graph: dict[str, Any] = {} - for key in _GRAPH_KEYS: - value = _coerce_graph_array(arguments.get(key)) - if value is None: - break - graph[key] = value - else: + # submit_graph is the only tool we provide. Accept a structurally valid + # graph even if FastFlow/Qwen adds harmless whitespace/name drift. + if not name or name == "submit_graph" or len(_tool_call_candidates(message)) == 1: return graph + return None @@ -485,6 +523,39 @@ def _request_metadata(body: bytes) -> dict[str, Any]: } +def _tool_call_summary(body: bytes) -> str: + try: + response = json.loads(body.decode("utf-8")) + except (UnicodeDecodeError, json.JSONDecodeError): + return "unparseable-response" + if not isinstance(response, dict): + return "non-object-response" + + choices = response.get("choices") + if not isinstance(choices, list) or not choices or not isinstance(choices[0], dict): + return "no-choice" + message = choices[0].get("message") + if not isinstance(message, dict): + return "no-message" + + summaries = [] + for call in _tool_call_candidates(message): + function = call.get("function") + if isinstance(function, dict): + name = str(function.get("name", "")).strip() or "" + args = _decode_jsonish(function.get("arguments")) + else: + name = str(call.get("name", "")).strip() or "" + args = _decode_jsonish(call.get("arguments")) + + if isinstance(args, dict): + summaries.append(f"{name}:keys={sorted(args.keys())}") + else: + summaries.append(f"{name}:args_type={type(args).__name__}") + + return "; ".join(summaries) if summaries else "no-tool-calls" + + def _response_metadata(body: bytes) -> tuple[dict[str, Any], str | None]: try: response = json.loads(body.decode("utf-8")) @@ -658,9 +729,11 @@ def _diagnose_suspect_response( if not suspect: reason = "structured response did not contain a usable provider-native graph" + tool_summary = _tool_call_summary(response_body) print( "[lds graphify] provider returned a suspect extraction response: " - f"reason={reason}; model={req.get('model')}; finish_reason={resp.get('finish_reason')}", + f"reason={reason}; model={req.get('model')}; finish_reason={resp.get('finish_reason')}; " + f"tool_calls={tool_summary}", file=sys.stderr, flush=True, ) From e63924e89a72b5036437734ed73dd4047a63aff7 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:19:27 +0600 Subject: [PATCH 504/509] test(graphify): cover FastFlow tool-call argument variants --- tests/ai-contract.sh | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index ce103e0a..ad3be902 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -368,6 +368,36 @@ assert module._extract_graph_tool_result(json.dumps(string_array_response).encod "hyperedges": [], } +nested_arguments_response = json.loads(json.dumps(tool_response)) +nested_arguments_response["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"] = json.dumps({ + "arguments": json.dumps({ + "nodes": [{"id": "nested"}], + "edges": [], + "hyperedges": [], + }) +}) +assert module._extract_graph_tool_result(json.dumps(nested_arguments_response).encode()) == { + "nodes": [{"id": "nested"}], + "edges": [], + "hyperedges": [], +} + +name_drift_response = json.loads(json.dumps(tool_response)) +name_drift_response["choices"][0]["message"]["tool_calls"][0]["function"]["name"] = " submit_graph " +assert module._extract_graph_tool_result(json.dumps(name_drift_response).encode()) == graph + +flat_call_response = json.loads(json.dumps(tool_response)) +call = flat_call_response["choices"][0]["message"]["tool_calls"][0] +flat_call_response["choices"][0]["message"]["tool_calls"][0] = { + "name": "submit_graph", + "arguments": call["function"]["arguments"], +} +assert module._extract_graph_tool_result(json.dumps(flat_call_response).encode()) == graph + +summary = module._tool_call_summary(json.dumps(tool_response).encode()) +assert "submit_graph:keys=" in summary +assert "nodes" in summary + empty_tool_response = json.loads(json.dumps(tool_response)) empty_tool_response["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"] = json.dumps({ "nodes": [], From 01b19155392895064715dbd056bbf6cfb18c9466 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:28:24 +0600 Subject: [PATCH 505/509] fix(graphify): preserve timeout split signal --- scripts/graphify-diagnostic-proxy.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index d43c0ca2..67c43c66 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -488,7 +488,11 @@ def _replace_response_content(body: bytes, graph: dict[str, Any]) -> bytes | Non message.pop("reasoning", None) message.pop("thinking", None) first["message"] = message - first["finish_reason"] = "stop" + # Preserve an upstream/synthetic length signal so Graphify's adaptive + # retry layer bisects the offending chunk. Successful tool-call responses + # are normalized to a regular completed assistant response. + if first.get("finish_reason") != "length": + first["finish_reason"] = "stop" choices[0] = first response["choices"] = choices return json.dumps(response, ensure_ascii=False).encode("utf-8") From 468dee71b7747781bde3c6d4efa5efe401c3ba02 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:28:36 +0600 Subject: [PATCH 506/509] test(graphify): preserve timeout length signal --- tests/ai-contract.sh | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index ad3be902..c0ced186 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -356,6 +356,13 @@ assert json.loads(split_json["choices"][0]["message"]["content"]) == { "hyperedges": [], } +split_graph = module._extract_fastflow_structured_graph(split) +assert split_graph == {"nodes": [], "edges": [], "hyperedges": []} +split_replacement = module._replace_response_content(split, split_graph) +assert split_replacement is not None +split_replacement_json = json.loads(split_replacement) +assert split_replacement_json["choices"][0]["finish_reason"] == "length" + string_array_response = json.loads(json.dumps(tool_response)) string_array_response["choices"][0]["message"]["tool_calls"][0]["function"]["arguments"] = json.dumps({ "nodes": json.dumps([{"id": "a"}]), From 4594fb5466e26427b9e5fc53e045182dc3aa5f12 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:36:38 +0600 Subject: [PATCH 507/509] fix(graphify): consume FastFlow tool calls from stream --- scripts/graphify-diagnostic-proxy.py | 114 +++++++++++++++++++++++++-- 1 file changed, 109 insertions(+), 5 deletions(-) diff --git a/scripts/graphify-diagnostic-proxy.py b/scripts/graphify-diagnostic-proxy.py index 67c43c66..7c9947a5 100644 --- a/scripts/graphify-diagnostic-proxy.py +++ b/scripts/graphify-diagnostic-proxy.py @@ -292,7 +292,10 @@ def _build_tool_recovery_request(body: bytes) -> bytes | None: # FastFlow currently supports tool_choice=auto|none only. The explicit # system instruction is therefore what makes submit_graph mandatory. request["tool_choice"] = "auto" - request["stream"] = False + # FastFlow's Qwen3.5 streaming parser emits a complete tool-call delta as + # soon as / is parsed. Using streaming here avoids + # waiting for model EOS on the non-stream path. + request["stream"] = True request["temperature"] = 0 request.pop("max_tokens", None) request["max_completion_tokens"] = _bounded_completion_cap(request) @@ -428,6 +431,98 @@ def _extract_fastflow_structured_graph(body: bytes) -> dict[str, Any] | None: return parse_graph_content(content) +def _fastflow_stream_completion(upstream_response, model: str | None) -> bytes: + """Collapse FastFlow SSE into one OpenAI completion, returning on a tool call.""" + content_parts: list[str] = [] + last_id = "chatcmpl-lds-fastflow" + finish_reason = "stop" + usage: dict[str, Any] = {} + + while True: + raw_line = upstream_response.readline() + if not raw_line: + break + try: + line = raw_line.decode("utf-8", errors="replace").strip() + except AttributeError: + line = str(raw_line).strip() + if not line.startswith("data:"): + continue + + payload = line[5:].strip() + if not payload: + continue + if payload == "[DONE]": + break + + try: + event = json.loads(payload) + except json.JSONDecodeError: + continue + if not isinstance(event, dict): + continue + + if isinstance(event.get("id"), str) and event["id"]: + last_id = event["id"] + if isinstance(event.get("usage"), dict): + usage = event["usage"] + + choices = event.get("choices") + if not isinstance(choices, list) or not choices or not isinstance(choices[0], dict): + continue + choice = choices[0] + if isinstance(choice.get("finish_reason"), str): + finish_reason = choice["finish_reason"] + + delta = choice.get("delta") + if not isinstance(delta, dict): + continue + + content = delta.get("content") + if isinstance(content, str) and content: + content_parts.append(content) + + calls = delta.get("tool_calls") + if isinstance(calls, list) and calls: + normalized_calls = [call for call in calls if isinstance(call, dict)] + if normalized_calls: + # FastFlow emits the complete function arguments in TOOL_DONE, + # not incremental argument fragments. Stop reading immediately + # so a model that fails to emit EOS cannot hold Graphify open. + response = { + "id": last_id, + "object": "chat.completion", + "model": model, + "choices": [{ + "index": 0, + "message": { + "role": "assistant", + "content": "".join(content_parts) or None, + "tool_calls": normalized_calls, + }, + "finish_reason": "tool_calls", + }], + "usage": usage, + } + return json.dumps(response, ensure_ascii=False).encode("utf-8") + + response = { + "id": last_id, + "object": "chat.completion", + "model": model, + "choices": [{ + "index": 0, + "message": { + "role": "assistant", + "content": "".join(content_parts), + }, + "finish_reason": finish_reason, + }], + "usage": usage, + } + return json.dumps(response, ensure_ascii=False).encode("utf-8") + + def _graphify_split_response(body: bytes, model: str | None) -> bytes: """Return a standard completion that makes Graphify bisect the current chunk.""" try: @@ -626,6 +721,9 @@ def _forward(self) -> None: structured_primary = "ollama-schema" upstream_url = self.upstream.rstrip("/") + self.path + if structured_primary == "fastflow-tool": + request_headers["Accept"] = "text/event-stream" + request = urllib.request.Request( upstream_url, data=upstream_body if self.command not in ("GET", "HEAD") else None, @@ -641,10 +739,16 @@ def _forward(self) -> None: request_timeout = self.structured_timeout if structured_primary else self.upstream_timeout with urllib.request.urlopen(request, timeout=request_timeout) as upstream_response: status = upstream_response.status - response_body = upstream_response.read() - content_type = upstream_response.headers.get("Content-Type") - if content_type: - response_headers["Content-Type"] = content_type + if structured_primary == "fastflow-tool": + response_body = _fastflow_stream_completion( + upstream_response, metadata.get("model") + ) + response_headers["Content-Type"] = "application/json" + else: + response_body = upstream_response.read() + content_type = upstream_response.headers.get("Content-Type") + if content_type: + response_headers["Content-Type"] = content_type except urllib.error.HTTPError as exc: status = exc.code response_body = exc.read() From 2c9eaa2e2412f64567f0bb1b02383bfac8a14dfa Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:36:56 +0600 Subject: [PATCH 508/509] test(graphify): cover streamed FastFlow tool completion --- tests/ai-contract.sh | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/tests/ai-contract.sh b/tests/ai-contract.sh index c0ced186..14922801 100755 --- a/tests/ai-contract.sh +++ b/tests/ai-contract.sh @@ -188,6 +188,7 @@ pass "Graphify local providers enforce structured no-thinking contracts" python3 -m py_compile "$ROOT/scripts/graphify-diagnostic-proxy.py" python3 - "$ROOT/scripts/graphify-diagnostic-proxy.py" <<'PY' import importlib.util +import io import json import sys @@ -257,6 +258,7 @@ assert recovery is not None recovery_json = json.loads(recovery) assert recovery_json["tools"][0]["function"]["name"] == "submit_graph" assert recovery_json["tool_choice"] == "auto" +assert recovery_json["stream"] is True assert recovery_json["think"] is False assert "STRUCTURED OUTPUT" in recovery_json["messages"][0]["content"] assert recovery_json["temperature"] == 0 @@ -322,6 +324,47 @@ assert graph == { assert module._extract_fastflow_structured_graph(json.dumps(tool_response).encode()) == graph +sse_tool_event = { + "id": "chatcmpl-stream-test", + "object": "chat.completion.chunk", + "model": "qwen3.5:9b", + "choices": [{ + "index": 0, + "delta": { + "tool_calls": [{ + "index": 0, + "id": "call_stream", + "type": "function", + "function": { + "name": "submit_graph", + "arguments": json.dumps(graph), + }, + }], + }, + "finish_reason": None, + }], +} +sse_tail = { + "id": "chatcmpl-stream-test", + "object": "chat.completion.chunk", + "model": "qwen3.5:9b", + "choices": [{ + "index": 0, + "delta": {"content": None}, + "finish_reason": "tool_calls", + }], + "usage": {"prompt_tokens": 100, "completion_tokens": 30, "total_tokens": 130}, +} +stream_bytes = ( + "data: " + json.dumps(sse_tool_event) + "\n\n" + + "data: " + json.dumps(sse_tail) + "\n\n" + + "data: [DONE]\n\n" +).encode() +collapsed = module._fastflow_stream_completion(io.BytesIO(stream_bytes), "qwen3.5:9b") +collapsed_json = json.loads(collapsed) +assert collapsed_json["choices"][0]["finish_reason"] == "tool_calls" +assert module._extract_fastflow_structured_graph(collapsed) == graph + content_only_response = { "choices": [{ "index": 0, From 77a66ebeaeeaeb5943c19591252db86d95d8ded8 Mon Sep 17 00:00:00 2001 From: "A. B. M. Mahmudul Hasan" Date: Sun, 20 Sep 2026 19:37:30 +0600 Subject: [PATCH 509/509] docs(graphify): document FastFlow streaming tool adapter --- docs/guides/local-ai.rst | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/docs/guides/local-ai.rst b/docs/guides/local-ai.rst index 875fc0cf..9c8c76c3 100644 --- a/docs/guides/local-ai.rst +++ b/docs/guides/local-ai.rst @@ -293,8 +293,10 @@ Structured extraction is provider-specific: * FastFlow / Qwen3.5 uses native tool calling with a single ``submit_graph`` function whose arguments follow Graphify's node/edge/hyperedge schema. The proxy - converts valid tool arguments back into the normal assistant JSON content that - Graphify already understands. FastFlow currently ignores OpenAI + requests FastFlow in streaming mode and stops reading as soon as FastFlow emits + the completed ``tool_calls`` delta, then converts that call into the normal + non-stream assistant JSON content Graphify expects. This avoids waiting for model + EOS on FastFlow's non-stream path. FastFlow currently ignores OpenAI ``response_format`` on its chat-completions path, so tool calling is the supported structured channel. * Ollama uses its OpenAI-compatible ``response_format.type=json_schema`` path with @@ -309,8 +311,9 @@ general output allowance. LocalDevStack caps a structured extraction at 2048 out tokens and defaults each structured request to a 120-second timeout (``LDS_GRAPHIFY_STRUCTURED_TIMEOUT``). This is important for FastFlow because its Qwen3.5 non-stream tool parser recognizes ```` only after generation -finishes; without a smaller bound, a malformed/non-terminating tool response can run -toward Graphify's 16384-token completion cap for many minutes. +finishes. The LDS FastFlow adapter therefore consumes the streaming parser instead; +the 2048-token/120-second bounds remain the safety ceiling if no complete tool call +arrives. LocalDevStack deliberately performs exactly one provider-native structured request per Graphify extraction attempt. It does not add its own structured retry or