Add API coverage and diagnose E2E target-access failures

.github/copilot-instructions.md

@@ -1,60 +1,52 @@
 # Copilot instructions — ecommerce-e2e-playwright
 
-## What this repo is
-End-to-end UI test suite for the demo shop **automationexercise.com**.
-Python + Playwright (sync API) + pytest, Page Object Model. 12 tests across
-auth, search, cart, checkout. ~365 LOC. Runs headless in CI on push and PR
-across Chromium/Firefox/WebKit.
+## Project
+
+Production-style Python + synchronous Playwright + pytest suite for the public training
+shop automationexercise.com: 12 UI, 15 API and 5 offline infrastructure checks.
 
 ## Layout
-- `pages/` — Page Object Model. One class per page; `base_page.py` holds shared
-  navigation (with 5xx retry) and the `parse_price` helper. `flows.py` holds
-  cross-page actions (e.g. `register_via_ui`) that several tests reuse.
-- `tests/` — one file per area (`test_auth`, `test_search`, `test_cart`,
-  `test_checkout`, `test_smoke`). Tests read as business scenarios; no selectors.
-- `utils/data_generator.py` — Faker-backed factories returning `TypedDict`s
-  (`User`, `PaymentCard`). Every test gets fresh, unique data.
-- `conftest.py` — fixtures: API-backed user setup/teardown, ad blocking via
-  network route, and a `base_url` fixture (sourced from `pytest.ini`).
-- `.github/workflows/tests.yml` — CI matrix.
-
-## How to build / run
+
+- `pages/`: selectors/actions; shared journeys in `flows.py`.
+- `tests/ui/`: UI scenarios; `tests/api/`: HTTP contracts without a browser.
+- `tests/unit/`: offline subprocess checks of real pytest fixture behavior.
+- `utils/data_generator.py`: typed Faker factories; `utils/api.py`: payload/contract helpers.
+- `conftest.py`: URL precedence, on-demand browser configuration, isolated account fixtures.
+- `scripts/check_target.py`: read-only CI preflight and JSON diagnostics.
+- `pyproject.toml`: pinned direct dependencies and pytest/Ruff/Pyright configuration.
+- `.github/workflows/tests.yml`: PR smoke; main/scheduled/manual regression.
+- `docs/`: verified failure analysis and an upstream bug reproduction.
+
+## Commands
+
 ```bash
-pip install -r requirements.txt   # versions are pinned, do not loosen
+python3 -m venv .venv
+source .venv/bin/activate
+python -m pip install -e '.[dev]'
 playwright install chromium
-pytest                            # full suite, chromium
-pytest -n 4                       # parallel (data isolation supports it)
-pytest --browser firefox          # other engines
+ruff check .
+ruff format --check .
+pyright
+pytest tests/unit
+pytest -m api
+pytest tests/ui -n 2
 ```
-Tests hit a live third-party site, so transient failures are expected; CI uses
-`--reruns 2`. A green local run is the bar before committing.
-
-## Conventions to follow in reviews and changes
-- **No `time.sleep()`.** Use Playwright auto-waiting and web-first `expect`.
-- **Selectors, in order of preference:** `data-qa` attributes, then stable CSS
-  classes, then `get_by_role`. Avoid matching on display text (breaks under
-  i18n) and never use XPath. Note: some buttons are `<a>` with no `href`, so
-  they have NO ARIA `link` role — `get_by_role("link", ...)` will not find them;
-  use the stable class (e.g. `a.check_out`).
-- **Page objects hold selectors and actions; tests hold assertions and flow.**
-  Don't put raw locators in test files. Reusable multi-page journeys go in
-  `pages/flows.py`, not duplicated across tests.
-- **Test data is typed.** Extend the `TypedDict`s in `utils/data_generator.py`
-  rather than passing loose dicts. The signup form and account API use different
-  field names (`first_name` vs `firstname`) — map explicitly, don't rename keys.
-- **Every test owns its data** and must be order-agnostic and xdist-safe (unique
-  emails via uuid). Don't introduce shared mutable state between tests.
-- **Pin dependencies.** `requirements.txt` uses `==`; bump deliberately, never
-  switch to `>=`.
-- **Don't hardcode `--browser` in `pytest.ini`** — it must stay overridable from
-  the CLI and the CI matrix.
-
-## What to flag in PR review
-- Any `time.sleep`, XPath, or text-based selector for a button.
-- New locators living in `tests/` instead of `pages/`.
-- Loosened dependency pins.
-- Tests that depend on execution order or another test's side effects.
-- New cross-page flows duplicated instead of added to `pages/flows.py`.
-
-## Trust these instructions
-Only search the codebase if something here is incomplete or proves wrong.
+
+## Conventions
+
+- Pin direct dependencies in `pyproject.toml`; `requirements.txt` only forwards to it.
+- Mark live tests `regression` and exactly one of `ui`/`api`. Add `smoke`/`critical`
+  deliberately. Offline tests use `unit` and run in the quality job.
+- Keep selectors in page objects and assertions in tests. Prefer `data-qa`, accessible
+  roles and stable CSS. An anchor without `href` may not have an ARIA link role.
+- No arbitrary sleeps or blanket reruns. Keep access challenges and outages visible.
+- Every created user needs fixture-owned cleanup, including when setup fails.
+- Keep user/card data typed; map UI/API field names explicitly.
+- API calls disable redirect following, check HTTP before parsing JSON, and then check
+  the target's application-level `responseCode`. Do not assert HTTP 201 for account creation.
+- Preserve CLI > environment > config URL precedence on xdist workers.
+- API/unit tests must work without installed browser binaries.
+- Keep browser CLI selection overridable. Limit live tests to two workers and serialize
+  browser engines in CI to avoid overloading the shared target.
+- Verify product requirements before reporting a search-relevance observation as a bug.
+- Run relevant local checks and report any target availability limitation honestly.

.github/workflows/tests.yml

@@ -1,48 +1,117 @@
 name: e2e-tests
 
 on:
-  push:
   pull_request:
+  push:
+    branches: [main]
+  schedule:
+    - cron: '23 5 * * 1-5'
   workflow_dispatch:
 
+permissions:
+  contents: read
+
+concurrency:
+  group: e2e-${{ github.event.pull_request.number || github.ref }}
+  cancel-in-progress: true
+
+env:
+  PYTHON_VERSION: '3.12'
+  TEST_SUITE: ${{ github.event_name == 'pull_request' && 'smoke' || 'regression' }}
+
 jobs:
-  test:
+  quality:
+    name: Ruff and Pyright
+    runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v7
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+          cache: pip
+          cache-dependency-path: pyproject.toml
+      - name: Install dependencies
+        run: python -m pip install -e '.[dev]'
+      - name: Lint
+        run: ruff check .
+      - name: Check formatting
+        run: ruff format --check .
+      - name: Check types
+        run: pyright
+      - name: Validate collection and markers
+        run: pytest --collect-only -q
+      - name: Check fixture behavior offline
+        run: pytest tests/unit
+
+  api:
+    name: API tests
+    needs: quality
     runs-on: ubuntu-latest
+    timeout-minutes: 10
+    steps:
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v7
+        with:
+          python-version: ${{ env.PYTHON_VERSION }}
+          cache: pip
+          cache-dependency-path: pyproject.toml
+      - name: Install dependencies (no browser binaries)
+        run: python -m pip install -e .
+      - name: Check target API availability
+        run: python scripts/check_target.py --output reports/preflight-api.json
+      - name: Run API suite once
+        run: >-
+          pytest tests/api -m "$TEST_SUITE"
+          --html=reports/api.html --self-contained-html
+          --junitxml=reports/api.xml
+      - name: Upload API diagnostics
+        uses: actions/upload-artifact@v7
+        if: always()
+        with:
+          name: api-report
+          path: reports/
+          retention-days: 14
+
+  ui:
+    name: UI (${{ matrix.browser }})
+    needs: api
+    runs-on: ubuntu-latest
+    timeout-minutes: 20
     strategy:
       fail-fast: false
+      # Serialize engines and cap workers: this is a shared public demo site.
+      max-parallel: 1
       matrix:
-        browser: [chromium, firefox, webkit]
+        browser: ${{ fromJSON(github.event_name == 'pull_request' && '["chromium"]' || '["chromium", "firefox", "webkit"]') }}
     steps:
-      - uses: actions/checkout@v4
-
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@v7
+      - uses: actions/setup-python@v7
         with:
-          python-version: '3.12'
+          python-version: ${{ env.PYTHON_VERSION }}
           cache: pip
-
+          cache-dependency-path: pyproject.toml
       - name: Install dependencies
-        run: pip install -r requirements.txt
-
-      - name: Install browser
+        run: python -m pip install -e .
+      - name: Install browser and system dependencies
         run: playwright install --with-deps ${{ matrix.browser }}
-
-      # -n 4 runs tests in parallel (data isolation in conftest.py supports it).
-      # Capped at 4 rather than `auto`: the demo site sheds load under heavy
-      # concurrency, so more workers means more flaky 503s, not more speed.
-      # --reruns guards against that intermittent load-shedding; genuine
-      # regressions still fail consistently and turn the badge red.
-      - name: Run e2e tests
-        run: >
-          pytest --browser ${{ matrix.browser }}
-          -n 4
-          --html=report.html --self-contained-html
-          --tracing retain-on-failure --reruns 2 --reruns-delay 30
-
-      - name: Upload report and failure artifacts
-        uses: actions/upload-artifact@v4
+      - name: Check target access from this browser
+        run: >-
+          python scripts/check_target.py --browser ${{ matrix.browser }}
+          --output reports/preflight-${{ matrix.browser }}.json
+      # No blanket reruns: site challenges and real assertion failures stay visible.
+      - name: Run UI suite
+        run: >-
+          pytest tests/ui -m "$TEST_SUITE" --browser ${{ matrix.browser }} -n 2
+          --html=reports/ui-${{ matrix.browser }}.html --self-contained-html
+          --junitxml=reports/ui-${{ matrix.browser }}.xml
+          --tracing retain-on-failure
+      - name: Upload UI report and failure artifacts
+        uses: actions/upload-artifact@v7
         if: always()
         with:
-          name: test-report-${{ matrix.browser }}
+          name: ui-report-${{ matrix.browser }}
           path: |
-            report.html
+            reports/
             test-results/
+          retention-days: 14

.gitignore

@@ -4,9 +4,14 @@ __pycache__/
 .venv/
 venv/
 .pytest_cache/
+.ruff_cache/
+*.egg-info/
+build/
+dist/
 
 # Test artifacts
 report.html
 test-results/
 playwright-report/
 trace.zip
+reports/