diff --git a/.github/workflows/headless-smoke.yml b/.github/workflows/headless-smoke.yml index eb6785c4..fcc80d8c 100644 --- a/.github/workflows/headless-smoke.yml +++ b/.github/workflows/headless-smoke.yml @@ -2,13 +2,19 @@ name: Headless smoke (L2) # Builds the Linux bundle and runs the real app under Xvfb in portable mode with # scripts/dev/headless_app.sh: it has to start, show a window and stay up. The -# screenshot is kept as an artifact. Not part of ci.yml: a Linux build takes +# screenshot is kept as an artifact. Then the MCP path end to end +# (scripts/dev/headless_mcp_check.sh): the app with its MCP server on, a shared +# fixture connection, and a client reading it through the real querya-mcp. Not part of ci.yml: a Linux build takes # minutes, so this runs when the script or this file changes, weekly, and on # demand. on: pull_request: paths: - scripts/dev/headless_app.sh + - scripts/dev/headless_mcp_check.sh + - scripts/dev/mcp_stdio_check.py + - lib/core/mcp/** + - packages/querya_mcp_bridge/** - .github/workflows/headless-smoke.yml schedule: - cron: "0 4 * * 2" @@ -33,7 +39,7 @@ jobs: run: | sudo apt-get update sudo apt-get install -y clang cmake ninja-build pkg-config libgtk-3-dev \ - libsecret-1-dev xvfb xdotool imagemagick + libsecret-1-dev xvfb xdotool imagemagick sqlite3 - name: Setup Flutter uses: subosito/flutter-action@v2 @@ -57,6 +63,20 @@ jobs: --log build/headless/app.log \ --wait 90 --keep 5 + - name: Build querya-mcp + run: | + cd packages/querya_mcp_bridge + dart pub get + dart compile exe bin/querya_mcp.dart -o "$RUNNER_TEMP/querya-mcp" + + - name: MCP end to end under Xvfb + run: | + mkdir -p build/headless + scripts/dev/headless_mcp_check.sh \ + --mcp "$RUNNER_TEMP/querya-mcp" \ + --bundle build/linux/x64/release/bundle \ + --log build/headless/mcp-app.log + - name: Keep the screenshot and the log if: always() uses: actions/upload-artifact@v4 diff --git a/docs/dev-containers.md b/docs/dev-containers.md index 1863a220..56bbd796 100644 --- a/docs/dev-containers.md +++ b/docs/dev-containers.md @@ -130,6 +130,15 @@ command and each runnable in a container or on a CI runner | L0 | unit, widget, e2e and guard tests | `flutter test --exclude-tags=golden` | | L1 | an extension against a real database, no GUI | `querya-ext-tester ./ext --connect '{...}' --sandbox` (see *Testing an extension*) | | L2 | the real app under a virtual display, in portable mode | `scripts/dev/headless_app.sh` | +| L2 + MCP | the same, with the MCP server on and a client reading a fixture through `querya-mcp` | `scripts/dev/headless_mcp_check.sh --mcp PATH` | + +What each level needs from the database stack (start only that, by service name): + +| Level | Containers | +|-------|------------| +| L0 | none; the live MySQL test needs `docker compose --profile live-tests up -d --wait mysql-live` | +| L1 | the database the extension talks to, e.g. `docker compose up -d --wait postgres` (or `clickhouse`, `redis`, `mongo`) | +| L2, L2 + MCP | none: the fixture is a throw-away SQLite file | L1 needs a tester built from 0.4.19 or newer: older ones exited 0 whatever the result. @@ -157,9 +166,36 @@ The [Headless smoke (L2)](../.github/workflows/headless-smoke.yml) workflow buil the Linux bundle and runs the script (on changes to it, weekly, and on demand), keeping the screenshot and the log as an artifact. -Whether L2 should grow a real "headless Querya" (core services without a window, -so that `querya-mcp` does not need a running Desktop) is left open until this -exists. +Two more options let a script drive the running app: + +- `--seed-sql FILE` applies SQL to the profile database (`querya.db`). The app + starts once to create and migrate it, is stopped, `sqlite3` applies the file, + and the app starts again. Use the same `app_settings` keys and tables the UI + writes, so a test needs no code in the app. +- `--run COMMAND` runs a command once the window is up and keeps the app alive + meanwhile; its exit code becomes the script's. The command gets + `QUERYA_MCP_ENDPOINT` (this run's own endpoint file, so no other instance can + answer), `QUERYA_HEADLESS_DB` and `QUERYA_HEADLESS_DATA`. + +### L2 + MCP: `scripts/dev/headless_mcp_check.sh` + +```bash +dart compile exe packages/querya_mcp_bridge/bin/querya_mcp.dart -o build/querya-mcp +scripts/dev/headless_mcp_check.sh --mcp build/querya-mcp --bundle build/linux/x64/release/bundle +``` + +It creates a fixture SQLite database with two connections, turns the MCP server +on and shares only one of them (the same settings keys the Preferences page +writes), starts the app under Xvfb and runs `scripts/dev/mcp_stdio_check.py`: a +small MCP client that talks to the real `querya-mcp` and checks `initialize`, +`tools/list`, that only the shared connection is listed, the table, its columns +and data, that a `DELETE` is refused and changes nothing, and that the unshared +connection is refused. It needs `sqlite3` and `python3` besides what L2 needs. +The same step runs in the Headless smoke workflow. + +A window under Xvfb is all the MCP server needs, so there is no separate +"headless Querya" mode: `querya-mcp` still needs a running Desktop, and the +virtual display provides it. ## Run the app in the container @@ -245,11 +281,27 @@ See [extension-testing.md](extension-testing.md) for the options, reports and the GitHub Action. The exit code is 0 on success, 1 when a check failed and 2 on a usage error. +## For extension authors + +You do not need this repository to write a database-driver extension. Start from +[QueryaHub/extension-template](https://github.com/QueryaHub/extension-template) +(a Rust driver with a dev container, a test database and CI). The pieces it uses: + +- the **Dev Container Feature** `ghcr.io/queryahub/features/querya-ext-tester` + installs `querya-ext-tester` from a release, checked against `SHA256SUMS.txt` + (see [extension-testing.md](extension-testing.md#dev-container-feature)); +- **test database images** `ghcr.io/queryahub/querya-testdb-{postgres,mysql,mongo,clickhouse,redis}:` + are the `docker/` seeds baked into the official images, no bind mounts. + Built from `docker//Dockerfile.testdb`; a new `Dockerfile.testdb` adds a + database to the build by itself. + ## Known gaps -- **The extension-author container** (Dev Container Feature, test database - images, template repository, registry CI) is not done yet; it is tracked in - [#1487](https://github.com/QueryaHub/Querya-Desktop/issues/1487). +- **The extension registry** (registry CI and `StaticRegistryRepository`) and the + release hooks that publish the Feature and the images are not done yet; they + are tracked in [#1487](https://github.com/QueryaHub/Querya-Desktop/issues/1487). + Until then the images are published by hand with the *Test database images* + workflow. ## Troubleshooting diff --git a/scripts/dev/__pycache__/mcp_stdio_check.cpython-314.pyc b/scripts/dev/__pycache__/mcp_stdio_check.cpython-314.pyc new file mode 100644 index 00000000..18663043 Binary files /dev/null and b/scripts/dev/__pycache__/mcp_stdio_check.cpython-314.pyc differ diff --git a/scripts/dev/headless_app.sh b/scripts/dev/headless_app.sh index a9e7a58d..790fdfc7 100755 --- a/scripts/dev/headless_app.sh +++ b/scripts/dev/headless_app.sh @@ -12,10 +12,19 @@ # (default 3); catches crashes right after start # --log FILE copy the app's stdout / stderr there # --keep-data do not delete the temporary data directory +# --seed-sql FILE SQL applied to the profile database (querya.db) before +# the app under test starts: the app runs once to create +# and migrate the database, is stopped, FILE is applied +# with sqlite3, and the app starts again +# --run COMMAND shell command to run once the window is up (the app +# stays up meanwhile); the exit code is its exit code. +# Gets QUERYA_MCP_ENDPOINT (this run's own endpoint file), +# QUERYA_HEADLESS_DB and QUERYA_HEADLESS_DATA # -h, --help this text # -# Exit codes: 0 the app started, showed a window and stayed up; 1 it died, or no -# window appeared in time; 2 usage error, missing bundle or missing tool. +# Exit codes: 0 the app started, showed a window and stayed up (and --run +# succeeded); 1 it died, no window appeared in time, or --run failed; 2 usage +# error, missing bundle or missing tool. # # Needs: Xvfb; xdotool or xwininfo to see the window (without them the check is # "the process is still alive after --wait"); ImageMagick `import` for @@ -28,6 +37,8 @@ wait_s=60 keep_s=3 log_out="" keep_data=0 +seed_sql="" +run_cmd="" usage() { sed -n '2,/^set -u/p' "$0" | sed '$d' | sed 's/^# \{0,1\}//'; } die() { printf 'headless_app: %s\n' "$*" >&2; exit 2; } @@ -41,6 +52,8 @@ while [ $# -gt 0 ]; do --keep) keep_s="${2:-}"; shift 2 ;; --log) log_out="${2:-}"; shift 2 ;; --keep-data) keep_data=1; shift ;; + --seed-sql) seed_sql="${2:-}"; shift 2 ;; + --run) run_cmd="${2:-}"; shift 2 ;; -h | --help) usage; exit 0 ;; *) die "unknown option: $1 (see --help)" ;; esac @@ -54,6 +67,10 @@ fi [ -n "$bundle" ] || die "no bundle found; build one (flutter build linux) or pass --bundle DIR" [ -x "$bundle/querya_desktop" ] || die "$bundle/querya_desktop is missing or not executable" command -v Xvfb >/dev/null 2>&1 || die "Xvfb is not installed (apt-get install xvfb)" +if [ -n "$seed_sql" ]; then + [ -f "$seed_sql" ] || die "--seed-sql: $seed_sql is not a file" + command -v sqlite3 >/dev/null 2>&1 || die "--seed-sql needs the sqlite3 command (apt-get install sqlite3)" +fi if [ -n "$screenshot" ] && ! command -v import >/dev/null 2>&1; then die "--screenshot needs ImageMagick's 'import' (apt-get install imagemagick)" fi @@ -125,13 +142,32 @@ cp -a "$bundle" "$work/bundle" || die "could not copy $bundle" export QUERYA_PORTABLE=1 export NO_AT_BRIDGE=1 +# Own endpoint file, so that a second app on the machine (or a leftover one) +# can never be the one querya-mcp talks to. +export QUERYA_MCP_ENDPOINT="$work/mcp-endpoint.json" + runner=() if command -v dbus-run-session >/dev/null 2>&1; then runner=(dbus-run-session --) fi -${runner[@]+"${runner[@]}"} "$work/bundle/querya_desktop" >"$work/app.log" 2>&1 & -app_pid=$! -say "started querya_desktop (pid $app_pid)" + +start_app() { + ${runner[@]+"${runner[@]}"} "$work/bundle/querya_desktop" >>"$work/app.log" 2>&1 & + app_pid=$! + say "started querya_desktop (pid $app_pid)" +} + +stop_app() { + [ -n "$app_pid" ] || return 0 + kill -TERM "$app_pid" 2>/dev/null + for _ in $(seq 1 20); do + kill -0 "$app_pid" 2>/dev/null || break + sleep 0.5 + done + kill -KILL "$app_pid" 2>/dev/null + wait "$app_pid" 2>/dev/null + app_pid="" +} have_window_tool=0 if command -v xdotool >/dev/null 2>&1 || command -v xwininfo >/dev/null 2>&1; then @@ -147,21 +183,37 @@ window_up() { fi } -deadline=$((SECONDS + wait_s)) -shown=0 -while [ "$SECONDS" -lt "$deadline" ]; do - kill -0 "$app_pid" 2>/dev/null || fail_with_log "the app exited before showing a window" +wait_for_window() { + local deadline=$((SECONDS + wait_s)) shown=0 + while [ "$SECONDS" -lt "$deadline" ]; do + kill -0 "$app_pid" 2>/dev/null || fail_with_log "the app exited before showing a window" + if [ "$have_window_tool" -eq 1 ]; then + if window_up; then shown=1; break; fi + fi + sleep 1 + done if [ "$have_window_tool" -eq 1 ]; then - if window_up; then shown=1; break; fi + [ "$shown" -eq 1 ] || fail_with_log "no window after ${wait_s}s" + say "window is up" + else + say "no xdotool / xwininfo: only checking that the process stays alive" fi - sleep 1 -done +} -if [ "$have_window_tool" -eq 1 ]; then - [ "$shown" -eq 1 ] || fail_with_log "no window after ${wait_s}s" - say "window is up" -else - say "no xdotool / xwininfo: only checking that the process stays alive" +start_app +wait_for_window + +if [ -n "$seed_sql" ]; then + # The first start created and migrated the profile database. Let it settle, + # stop the app (it holds the file open), seed, and start again. + sleep 2 + stop_app + db="$(find "$work/bundle" -name querya.db -print -quit 2>/dev/null)" + [ -n "$db" ] || fail_with_log "the profile database querya.db was not created" + sqlite3 "$db" <"$seed_sql" || fail_with_log "could not apply $seed_sql to $db" + say "seeded $(basename "$seed_sql") into $db" + start_app + wait_for_window fi end=$((SECONDS + keep_s)) @@ -180,5 +232,16 @@ if [ -n "$screenshot" ]; then fi fi +if [ -n "$run_cmd" ]; then + export QUERYA_HEADLESS_DB + QUERYA_HEADLESS_DB="$(find "$work/bundle" -name querya.db -print -quit 2>/dev/null)" + export QUERYA_HEADLESS_DATA="$work/bundle/QueryaData" + say "running: $run_cmd" + if ! bash -c "$run_cmd"; then + fail_with_log "--run command failed" + fi + kill -0 "$app_pid" 2>/dev/null || fail_with_log "the app died while the command ran" +fi + say "ok" exit 0 diff --git a/scripts/dev/headless_mcp_check.sh b/scripts/dev/headless_mcp_check.sh new file mode 100755 index 00000000..1167e047 --- /dev/null +++ b/scripts/dev/headless_mcp_check.sh @@ -0,0 +1,58 @@ +#!/usr/bin/env bash +# L2 headless MCP end to end: the real app under Xvfb with its MCP server on, +# one fixture SQLite connection shared, and a client talking MCP to it through +# the real `querya-mcp` bridge. +# +# scripts/dev/headless_mcp_check.sh --mcp PATH [--bundle DIR] [--log FILE] +# +# --mcp PATH the querya-mcp executable (dart compile exe +# packages/querya_mcp_bridge/bin/querya_mcp.dart -o querya-mcp) +# --bundle DIR built bundle, as for headless_app.sh +# --log FILE the app's output, as for headless_app.sh +# +# Needs what headless_app.sh needs, plus sqlite3 and python3. Exit codes are +# headless_app.sh's: 0 all checks passed, 1 something failed, 2 usage / tools. +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +mcp="" +pass=() +while [ $# -gt 0 ]; do + case "$1" in + --mcp) mcp="${2:-}"; shift 2 ;; + --bundle | --log) pass+=("$1" "${2:-}"); shift 2 ;; + -h | --help) sed -n '2,/^set -euo/p' "$0" | sed '$d' | sed 's/^# \{0,1\}//'; exit 0 ;; + *) echo "headless_mcp_check: unknown option: $1" >&2; exit 2 ;; + esac +done +[ -x "$mcp" ] || { echo "headless_mcp_check: --mcp must point at the querya-mcp executable" >&2; exit 2; } +command -v python3 >/dev/null 2>&1 || { echo "headless_mcp_check: python3 is missing" >&2; exit 2; } +command -v sqlite3 >/dev/null 2>&1 || { echo "headless_mcp_check: sqlite3 is missing" >&2; exit 2; } + +tmp="$(mktemp -d "${TMPDIR:-/tmp}/querya-mcp-check.XXXXXX")" +trap 'rm -rf "$tmp"' EXIT + +# The database the model will query. +sqlite3 "$tmp/fixture.db" <<'SQL' +CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT NOT NULL, price REAL NOT NULL); +INSERT INTO items (name, price) VALUES ('alpha', 10), ('beta', 20), ('gamma', 30); +SQL + +# What the settings page would store: the connection, the server on, the +# connection shared. A second connection stays unshared, so the check can see +# that sharing is opt-in. +cat >"$tmp/seed.sql" <