diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 24bcc19..3904a36 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -1,60 +1,52 @@ # Copilot instructions — ecommerce-e2e-playwright -## What this repo is -End-to-end UI test suite for the demo shop **automationexercise.com**. -Python + Playwright (sync API) + pytest, Page Object Model. 12 tests across -auth, search, cart, checkout. ~365 LOC. Runs headless in CI on push and PR -across Chromium/Firefox/WebKit. +## Project + +Production-style Python + synchronous Playwright + pytest suite for the public training +shop automationexercise.com: 12 UI, 15 API and 5 offline infrastructure checks. ## Layout -- `pages/` — Page Object Model. One class per page; `base_page.py` holds shared - navigation (with 5xx retry) and the `parse_price` helper. `flows.py` holds - cross-page actions (e.g. `register_via_ui`) that several tests reuse. -- `tests/` — one file per area (`test_auth`, `test_search`, `test_cart`, - `test_checkout`, `test_smoke`). Tests read as business scenarios; no selectors. -- `utils/data_generator.py` — Faker-backed factories returning `TypedDict`s - (`User`, `PaymentCard`). Every test gets fresh, unique data. -- `conftest.py` — fixtures: API-backed user setup/teardown, ad blocking via - network route, and a `base_url` fixture (sourced from `pytest.ini`). -- `.github/workflows/tests.yml` — CI matrix. - -## How to build / run + +- `pages/`: selectors/actions; shared journeys in `flows.py`. +- `tests/ui/`: UI scenarios; `tests/api/`: HTTP contracts without a browser. +- `tests/unit/`: offline subprocess checks of real pytest fixture behavior. +- `utils/data_generator.py`: typed Faker factories; `utils/api.py`: payload/contract helpers. +- `conftest.py`: URL precedence, on-demand browser configuration, isolated account fixtures. +- `scripts/check_target.py`: read-only CI preflight and JSON diagnostics. +- `pyproject.toml`: pinned direct dependencies and pytest/Ruff/Pyright configuration. +- `.github/workflows/tests.yml`: PR smoke; main/scheduled/manual regression. +- `docs/`: verified failure analysis and an upstream bug reproduction. + +## Commands + ```bash -pip install -r requirements.txt # versions are pinned, do not loosen +python3 -m venv .venv +source .venv/bin/activate +python -m pip install -e '.[dev]' playwright install chromium -pytest # full suite, chromium -pytest -n 4 # parallel (data isolation supports it) -pytest --browser firefox # other engines +ruff check . +ruff format --check . +pyright +pytest tests/unit +pytest -m api +pytest tests/ui -n 2 ``` -Tests hit a live third-party site, so transient failures are expected; CI uses -`--reruns 2`. A green local run is the bar before committing. - -## Conventions to follow in reviews and changes -- **No `time.sleep()`.** Use Playwright auto-waiting and web-first `expect`. -- **Selectors, in order of preference:** `data-qa` attributes, then stable CSS - classes, then `get_by_role`. Avoid matching on display text (breaks under - i18n) and never use XPath. Note: some buttons are `` with no `href`, so - they have NO ARIA `link` role — `get_by_role("link", ...)` will not find them; - use the stable class (e.g. `a.check_out`). -- **Page objects hold selectors and actions; tests hold assertions and flow.** - Don't put raw locators in test files. Reusable multi-page journeys go in - `pages/flows.py`, not duplicated across tests. -- **Test data is typed.** Extend the `TypedDict`s in `utils/data_generator.py` - rather than passing loose dicts. The signup form and account API use different - field names (`first_name` vs `firstname`) — map explicitly, don't rename keys. -- **Every test owns its data** and must be order-agnostic and xdist-safe (unique - emails via uuid). Don't introduce shared mutable state between tests. -- **Pin dependencies.** `requirements.txt` uses `==`; bump deliberately, never - switch to `>=`. -- **Don't hardcode `--browser` in `pytest.ini`** — it must stay overridable from - the CLI and the CI matrix. - -## What to flag in PR review -- Any `time.sleep`, XPath, or text-based selector for a button. -- New locators living in `tests/` instead of `pages/`. -- Loosened dependency pins. -- Tests that depend on execution order or another test's side effects. -- New cross-page flows duplicated instead of added to `pages/flows.py`. - -## Trust these instructions -Only search the codebase if something here is incomplete or proves wrong. + +## Conventions + +- Pin direct dependencies in `pyproject.toml`; `requirements.txt` only forwards to it. +- Mark live tests `regression` and exactly one of `ui`/`api`. Add `smoke`/`critical` + deliberately. Offline tests use `unit` and run in the quality job. +- Keep selectors in page objects and assertions in tests. Prefer `data-qa`, accessible + roles and stable CSS. An anchor without `href` may not have an ARIA link role. +- No arbitrary sleeps or blanket reruns. Keep access challenges and outages visible. +- Every created user needs fixture-owned cleanup, including when setup fails. +- Keep user/card data typed; map UI/API field names explicitly. +- API calls disable redirect following, check HTTP before parsing JSON, and then check + the target's application-level `responseCode`. Do not assert HTTP 201 for account creation. +- Preserve CLI > environment > config URL precedence on xdist workers. +- API/unit tests must work without installed browser binaries. +- Keep browser CLI selection overridable. Limit live tests to two workers and serialize + browser engines in CI to avoid overloading the shared target. +- Verify product requirements before reporting a search-relevance observation as a bug. +- Run relevant local checks and report any target availability limitation honestly. diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index dd1ec41..3ae24e7 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -1,48 +1,117 @@ name: e2e-tests on: - push: pull_request: + push: + branches: [main] + schedule: + - cron: '23 5 * * 1-5' workflow_dispatch: +permissions: + contents: read + +concurrency: + group: e2e-${{ github.event.pull_request.number || github.ref }} + cancel-in-progress: true + +env: + PYTHON_VERSION: '3.12' + TEST_SUITE: ${{ github.event_name == 'pull_request' && 'smoke' || 'regression' }} + jobs: - test: + quality: + name: Ruff and Pyright + runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-python@v7 + with: + python-version: ${{ env.PYTHON_VERSION }} + cache: pip + cache-dependency-path: pyproject.toml + - name: Install dependencies + run: python -m pip install -e '.[dev]' + - name: Lint + run: ruff check . + - name: Check formatting + run: ruff format --check . + - name: Check types + run: pyright + - name: Validate collection and markers + run: pytest --collect-only -q + - name: Check fixture behavior offline + run: pytest tests/unit + + api: + name: API tests + needs: quality runs-on: ubuntu-latest + timeout-minutes: 10 + steps: + - uses: actions/checkout@v7 + - uses: actions/setup-python@v7 + with: + python-version: ${{ env.PYTHON_VERSION }} + cache: pip + cache-dependency-path: pyproject.toml + - name: Install dependencies (no browser binaries) + run: python -m pip install -e . + - name: Check target API availability + run: python scripts/check_target.py --output reports/preflight-api.json + - name: Run API suite once + run: >- + pytest tests/api -m "$TEST_SUITE" + --html=reports/api.html --self-contained-html + --junitxml=reports/api.xml + - name: Upload API diagnostics + uses: actions/upload-artifact@v7 + if: always() + with: + name: api-report + path: reports/ + retention-days: 14 + + ui: + name: UI (${{ matrix.browser }}) + needs: api + runs-on: ubuntu-latest + timeout-minutes: 20 strategy: fail-fast: false + # Serialize engines and cap workers: this is a shared public demo site. + max-parallel: 1 matrix: - browser: [chromium, firefox, webkit] + browser: ${{ fromJSON(github.event_name == 'pull_request' && '["chromium"]' || '["chromium", "firefox", "webkit"]') }} steps: - - uses: actions/checkout@v4 - - - uses: actions/setup-python@v5 + - uses: actions/checkout@v7 + - uses: actions/setup-python@v7 with: - python-version: '3.12' + python-version: ${{ env.PYTHON_VERSION }} cache: pip - + cache-dependency-path: pyproject.toml - name: Install dependencies - run: pip install -r requirements.txt - - - name: Install browser + run: python -m pip install -e . + - name: Install browser and system dependencies run: playwright install --with-deps ${{ matrix.browser }} - - # -n 4 runs tests in parallel (data isolation in conftest.py supports it). - # Capped at 4 rather than `auto`: the demo site sheds load under heavy - # concurrency, so more workers means more flaky 503s, not more speed. - # --reruns guards against that intermittent load-shedding; genuine - # regressions still fail consistently and turn the badge red. - - name: Run e2e tests - run: > - pytest --browser ${{ matrix.browser }} - -n 4 - --html=report.html --self-contained-html - --tracing retain-on-failure --reruns 2 --reruns-delay 30 - - - name: Upload report and failure artifacts - uses: actions/upload-artifact@v4 + - name: Check target access from this browser + run: >- + python scripts/check_target.py --browser ${{ matrix.browser }} + --output reports/preflight-${{ matrix.browser }}.json + # No blanket reruns: site challenges and real assertion failures stay visible. + - name: Run UI suite + run: >- + pytest tests/ui -m "$TEST_SUITE" --browser ${{ matrix.browser }} -n 2 + --html=reports/ui-${{ matrix.browser }}.html --self-contained-html + --junitxml=reports/ui-${{ matrix.browser }}.xml + --tracing retain-on-failure + - name: Upload UI report and failure artifacts + uses: actions/upload-artifact@v7 if: always() with: - name: test-report-${{ matrix.browser }} + name: ui-report-${{ matrix.browser }} path: | - report.html + reports/ test-results/ + retention-days: 14 diff --git a/.gitignore b/.gitignore index d0747fb..fcdf28d 100644 --- a/.gitignore +++ b/.gitignore @@ -4,9 +4,14 @@ __pycache__/ .venv/ venv/ .pytest_cache/ +.ruff_cache/ +*.egg-info/ +build/ +dist/ # Test artifacts report.html test-results/ playwright-report/ trace.zip +reports/ diff --git a/README.md b/README.md index 0265d7a..e8a9722 100644 --- a/README.md +++ b/README.md @@ -1,88 +1,142 @@ -# E-commerce E2E Test Suite — Playwright + Python +# E-commerce UI + API Test Suite — Playwright + Python [![e2e-tests](https://github.com/larelgit/ecommerce-e2e-playwright/actions/workflows/tests.yml/badge.svg)](https://github.com/larelgit/ecommerce-e2e-playwright/actions/workflows/tests.yml) -This repo shows how I set up production-grade UI automation for a web shop: **12 end-to-end -tests** covering registration, login, search, cart and checkout, running headless in CI on -every push — with an HTML report and failure screenshots/traces attached to every run. +Production-style automation for a web shop: **12 UI tests and 15 API tests** covering +accounts, authentication, search, cart and checkout. Five offline infrastructure tests +check URL configuration and browser-free API fixtures. CI runs Ruff, Pyright and smoke +checks on pull requests, with full regression on `main` and a weekday schedule. -**Target application:** [automationexercise.com](https://automationexercise.com) — a full -e-commerce site (catalog, accounts, cart, payments) built as an automation target. +**Target application:** [Automation Exercise](https://automationexercise.com), a public +training shop. The live suites create temporary accounts and delete them afterwards. +They depend on the site's availability and its access policy for the runner's network. -## What's covered +## Coverage -| Suite | Tests | Highlights | -|---|---|---| -| `test_smoke.py` | 1 | App is up, home page renders | -| `test_auth.py` | 4 | Signup, login, **wrong password** and **taken email** negative paths | -| `test_search.py` | 2 | Matching results + **empty-result** negative path | -| `test_cart.py` | 3 | Add to cart, quantity × price totals, remove | -| `test_checkout.py` | 2 | Guest checkout gate + **full purchase flow**: registration → product → cart → checkout → payment → order confirmation | +| Suite | Tests | Main checks | +|---|---:|---| +| `tests/ui/test_smoke.py` | 1 | Home page title and featured products | +| `tests/ui/test_auth.py` | 4 | Registration, login, wrong password, existing email | +| `tests/ui/test_search.py` | 2 | Matching products and no results | +| `tests/ui/test_cart.py` | 3 | Add, quantity × price totals, remove | +| `tests/ui/test_checkout.py` | 2 | Guest gate and complete purchase journey | +| `tests/api/test_accounts.py` | 9 | Create/read back account, duplicate email, valid/invalid login, missing credentials, unsupported method, delete and reject subsequent login | +| `tests/api/test_products.py` | 6 | Product/brand structure, unique IDs, unsupported method, known-product search, no results, missing query | +| `tests/unit/test_fixtures.py` | 5 | URL precedence in real serial/xdist sessions and API fixture without a browser | -The showcase test is [`test_full_purchase_flow`](tests/test_checkout.py) — the complete -money path a real client pays to keep green. +The full purchase test covers registration → product → cart → delivery address → +payment → confirmation → account deletion. Card details are generated test data for +this training target. -## Stack +## Local setup -Playwright (Chromium · Firefox · WebKit) · pytest · Page Object Model · Faker · -pytest-html · pytest-xdist · GitHub Actions - -## Run it locally +Requires Python 3.12+. From the repository root: ```bash -pip install -r requirements.txt -playwright install -pytest --html=report.html --self-contained-html +# Creates a local environment and installs project dependencies into it. +python3 -m venv .venv +source .venv/bin/activate +python -m pip install -e '.[dev]' + +# Downloads Chromium into the user's Playwright cache. +playwright install chromium +pytest -m smoke --html=reports/smoke.html --self-contained-html +``` + +`python -m pip install -r requirements.txt` remains a compatibility entry point for +the same installation. Direct dependency pins, including Playwright itself, live in +[pyproject.toml](pyproject.toml), alongside pytest, Ruff and Pyright configuration. +Transitive dependencies are resolved by pip; this is not a complete lockfile. -# parallel (data isolation supports it) -pytest -n 4 +For all three engines, `playwright install chromium firefox webkit` downloads the +browser binaries. On Ubuntu, `playwright install --with-deps chromium firefox webkit` +**also installs system packages via apt and may require sudo**. Use it when the browser +reports missing shared libraries. -# a different engine -pytest --browser firefox +## Select a suite + +Run these commands inside the activated virtual environment: + +```bash +pytest -m smoke # 8 checks: 4 UI + 4 API +pytest -m regression # all 27 live UI/API checks +pytest -m api # 15 API checks; no browser install needed +pytest -m 'ui and smoke' # 4 quick UI checks +pytest -m critical # account, cart and full purchase journeys +pytest tests/unit # 5 offline infrastructure checks +pytest tests/ui --browser firefox -n 2 # full UI suite in another engine +pytest # all 32 checks, Chromium for UI + +ruff check . +ruff format --check . +pyright ``` +`smoke` and `critical` are subsets of `regression`. `ui` and `api` select a layer; +`unit` selects offline checks. Unknown markers and configuration keys fail collection. + +The same base URL is used for UI and API setup. Precedence is `--base-url`, then +`PYTEST_BASE_URL`, then the default in `pyproject.toml`; this also works with xdist. +Use an override only for a compatible, authorized instance of the target. + +## CI + +| Trigger | Quality | API | UI | +|---|---|---|---| +| Pull request | Ruff, formatting, Pyright, collection, unit tests | 4 smoke tests | 4 smoke tests, Chromium | +| Push to `main` | Same | All 15 | All 12 per engine: Chromium, Firefox, WebKit | +| Weekdays at 05:23 UTC / manual run | Same | All 15 | All 12 per engine | + +The [workflow](.github/workflows/tests.yml) checks API availability before the API +suite and browser access before each UI suite. It runs API tests once, then runs the +browser jobs one at a time with two workers each. Superseded runs are cancelled. +This limits traffic to the shared demo site. Persistent failures remain failures; +there are no blanket reruns or skips that turn an inaccessible target green. + +HTML reports, JUnit XML and preflight JSON are uploaded for 14 days. UI failures also +include screenshots and Playwright traces. Artifacts are uploaded even when preflight +fails; an HTML report exists only if pytest actually ran. + ## Design decisions -- **Page Object Model** — selectors and page flows live in [pages/](pages/), tests read as - business scenarios. -- **Every test owns its data** — accounts are generated with Faker and created/removed - through the site's [REST API](https://automationexercise.com/api_list) in fixtures - ([conftest.py](conftest.py)), so tests are independent and order-agnostic. -- **Resilient selectors** — `get_by_role` / `data-qa` attributes, no brittle XPath. - Where the markup fights ARIA (e.g. an `` with no `href`, which - therefore has no `link` role), a stable CSS class beats matching on display text. -- **Typed test data** — user/payment factories return `TypedDict`s ([utils/data_generator.py](utils/data_generator.py)), - so editors and `pyright` catch a missing or misspelled field before it reaches the form. -- **No `time.sleep()`** — Playwright auto-waiting and web-first `expect` assertions only. -- **Ad traffic is blocked at network level** — the target site serves ad iframes that can - cover buttons and steal clicks; a route filter in `conftest.py` keeps runs deterministic. -- **Cross-browser in CI** — the GitHub Actions matrix runs the full suite on Chromium, - Firefox and WebKit on every push ([.github/workflows/tests.yml](.github/workflows/tests.yml)). -- **Failure diagnostics** — screenshots on failure locally; in CI also Playwright traces, - uploaded with the HTML report as a per-browser workflow artifact. - -## Field notes from automating this target - -Real-world issues found and handled while building the suite: - -- Third-party ad scripts intermittently overlay call-to-action buttons and intercept - clicks — the kind of flakiness source you have to engineer around in production suites - (solved here with network-level request blocking rather than retries or sleeps). -- The shared demo site sheds load with `503 "queue full"` pages for minutes at a time. - Handled on two levels: page navigation retries on 5xx ([pages/base_page.py](pages/base_page.py)), - and test-level reruns in CI only — locally failures stay loud. Failure screenshots made - the diagnosis trivial: the "failing test" screenshot was just the 503 page. -- The signup form and the account API use different field names for the same data - (`first_name` vs `firstname`), which the data factory has to map explicitly. -- **Search relevance finding:** a strict assertion ("every result name contains the query") - failed honestly and exposed real behaviour — searching for `dress` also returns - *"Sleeves Top and Short - Blue & Pink"*. The engine matches fields that are not shown - on the result card (category/description), so users can't tell why an item matched. - On a client project this goes straight into the bug tracker as a UX/relevance issue; - this site is a training target without a public tracker, so it's documented here. - -## What I'd add next - -- Visual regression checks for key pages. -- An API test layer reusing the same data factories. -- Adopt an open-source target (e.g. RealWorld/Conduit) to file real bug reports upstream. +- **Page Object Model:** selectors and actions live in [pages/](pages/); tests express + scenarios. Shared registration steps live in `pages/flows.py`. +- **Independent data:** fixtures generate unique accounts and register cleanup before + creation. Cleanup accepts an already deleted account and reports other errors. +- **Browser-free API layer:** [tests/api/](tests/api/) uses Playwright request contexts. + Ad blocking is attached only when a UI test requests a browser context. +- **API contracts:** tests check transport, application status, messages and relevant + response data against the [published API scenarios](https://automationexercise.com/api_list). + This target returns HTTP 200 even for application errors; `responseCode` in JSON + carries codes such as 400 or 404. Unexpected redirects are exposed at the first hop. +- **Typed data:** `User` and `PaymentCard` describe factory outputs. Account payloads + explicitly map `first_name`/`last_name` to the API's `firstname`/`lastname`. +- **Readiness and diagnostics:** navigation waits for DOM readiness, checks HTTP errors + and waits briefly for known access-interstitial titles to disappear. It then reports + a target-access error if the challenge persists. Controls use Playwright auto-waiting. +- **Dependency checks:** Ruff and Pyright run in CI. Offline fixture checks protect the + CLI/environment URL precedence and ensure API tests cannot silently start a browser. + +## Findings and bug reporting + +The [CI failure analysis](docs/ci-troubleshooting.md) links to the actual failing run. +Its Chromium/Firefox jobs received an access challenge (`One moment, please...`), +while account API requests entered HTTP 302 loops. Changing selectors cannot resolve a +persistent site-side access restriction; the runner needs access to the training target. + +A [reproduced open-source bug report](docs/bugs/BUG-001-base-url-xdist.md) includes a +minimal offline example of an ini-configured URL becoming `None` on xdist workers. +This is a confirmation of an existing upstream issue, with the workaround covered by +our fixture tests. It is not presented as a newly discovered or newly submitted defect. + +**Search field note:** searching for `dress` has returned products whose displayed names +do not contain that term. The public scenarios do not define name-only matching. The +matching fields and expected relevance need owner confirmation, so this remains an +observation rather than a confirmed product defect. The API suite additionally checks +that a search using an actual catalog product name returns that exact product. + +## Possible next steps + +- Visual regression for key pages. +- A locally hosted open-source shop for deterministic CI independent of the public demo. +- A generated lockfile for transitive dependencies. diff --git a/conftest.py b/conftest.py index ec594d6..bb8a000 100644 --- a/conftest.py +++ b/conftest.py @@ -1,65 +1,55 @@ -"""Shared fixtures: browser tweaks and test users managed through the site's REST API.""" +"""Shared browser configuration and isolated accounts for UI and API tests.""" + import re +from collections.abc import Iterator import pytest -from playwright.sync_api import Playwright +from playwright.sync_api import APIRequestContext, BrowserContext, Playwright +from utils.api import account_payload, assert_api_response, delete_account from utils.data_generator import User, generate_user -# The target site is ad-supported; ad iframes sometimes cover buttons and -# intercept clicks, so all ad/analytics requests are dropped at network level. AD_HOSTS = re.compile( r"(googlesyndication|doubleclick|adservice|google-analytics|googletagmanager|fundingchoices)" ) -# pytest-playwright reads `base_url` from pytest.ini, but that value is NOT -# propagated to xdist workers (-n auto), so navigations there hit "/" with no -# host. Re-expose it as a fixture sourced from the ini so parallel runs work. + @pytest.fixture(scope="session") -def base_url(request) -> str: - return request.config.getini("base_url") +def base_url(request: pytest.FixtureRequest) -> str: + """Keep CLI/environment overrides and the ini fallback on xdist workers.""" + return request.config.getoption("base_url") or request.config.getini("base_url") -@pytest.fixture(autouse=True) -def block_ads(context): +@pytest.fixture +def context(context: BrowserContext) -> BrowserContext: + """Configure only requested browser contexts; API tests stay browser-free.""" context.route(AD_HOSTS, lambda route: route.abort()) + return context @pytest.fixture -def api(playwright: Playwright, base_url: str): - """API client for test-data setup/teardown (https://automationexercise.com/api_list).""" +def api(playwright: Playwright, base_url: str) -> Iterator[APIRequestContext]: client = playwright.request.new_context(base_url=base_url) - yield client - client.dispose() - - -def _delete_account(api, user: User) -> None: - # idempotent: a 404 response code for an already-deleted account is fine - api.delete( - "/api/deleteAccount", - form={"email": user["email"], "password": user["password"]}, - ) + try: + yield client + finally: + client.dispose() @pytest.fixture -def new_user(api): - """Fresh user data for UI-registration tests; the account is removed afterwards.""" +def new_user(api: APIRequestContext) -> Iterator[User]: + """Own cleanup before creation, including failed setup and UI registration.""" user = generate_user() - yield user - _delete_account(api, user) + try: + yield user + finally: + delete_account(api, user) @pytest.fixture -def registered_user(api): - """An existing account, created through the API so every test owns its data.""" - user = generate_user() - payload = { - **user, - "firstname": user["first_name"], # the API uses different field names - "lastname": user["last_name"], - } - response = api.post("/api/createAccount", form=payload) - body = response.json() - assert body.get("responseCode") == 201, f"user setup failed: {body}" - yield user - _delete_account(api, user) +def registered_user(api: APIRequestContext, new_user: User) -> User: + response = api.post( + "/api/createAccount", form=account_payload(new_user), max_redirects=0 + ) + assert_api_response(response, 201, "User created!") + return new_user diff --git a/docs/bugs/BUG-001-base-url-xdist.md b/docs/bugs/BUG-001-base-url-xdist.md new file mode 100644 index 0000000..a169d29 --- /dev/null +++ b/docs/bugs/BUG-001-base-url-xdist.md @@ -0,0 +1,81 @@ +# Bug Report + +**Title:** An ini-configured `base_url` becomes `None` on pytest-xdist workers + +**Environment:** Ubuntu 24.04, Python 3.12.3, pytest 9.1.0, +pytest-base-url 2.1.0, pytest-xdist 3.8.0. Verified on 2026-09-08. + +**Preconditions:** The three pytest packages are installed in the active virtual +environment. Run the reproduction in a temporary directory outside this repository, +because this repository's `conftest.py` contains a workaround. No browser, network +request, account or running application is required. + +**Steps to Reproduce:** + +1. Create an isolated directory and configuration. These commands create only + temporary files and use packages from the already activated environment: + + ```bash + QA_REPRO_DIR=$(mktemp -d /tmp/base-url-repro.XXXXXX) + cd "$QA_REPRO_DIR" + cat > pytest.ini <<'INI' + [pytest] + base_url = http://example.test + INI + cat > test_base_url.py <<'PY' + def test_base_url_from_ini(base_url): + assert base_url == "http://example.test" + PY + ``` + +2. Run the serial control, with inherited URL/pytest options removed: + + ```bash + env -u PYTEST_BASE_URL -u PYTEST_ADDOPTS -u VERIFY_BASE_URL python -m pytest -q + ``` + +3. Run the same test on xdist workers: + + ```bash + env -u PYTEST_BASE_URL -u PYTEST_ADDOPTS -u VERIFY_BASE_URL python -m pytest -q -n 2 + ``` + +4. Confirm the CLI workaround: + + ```bash + env -u PYTEST_BASE_URL -u PYTEST_ADDOPTS -u VERIFY_BASE_URL \ + python -m pytest -q -n 2 --base-url http://example.test + ``` + +**Actual Result:** Serial execution passes. Parallel execution fails with +`assert None == 'http://example.test'`. Parallel execution with `--base-url` passes. + +**Expected Result:** The configured URL is available through the `base_url` fixture +in both serial and parallel execution without requiring a CLI workaround. + +**Reproducibility:** 2/2 isolated parallel reproductions failed; the serial and +explicit-CLI controls passed in both reproductions. + +**Severity:** Medium — breaks parallel tests that rely on ini configuration; +serial execution and explicit CLI configuration remain available. + +**Priority:** Suggested P2; upstream maintainers determine their own priority. + +**Attachments / Evidence:** [Captured command output](evidence/base-url-xdist.txt). +The `.test` hostname is a reserved example value; the test only compares strings. + +**Additional Notes:** + +- This is a reproduced existing open-source defect, not a claim of first discovery. + Duplicate review found [pytest-xdist issue #800](https://github.com/pytest-dev/pytest-xdist/issues/800) + and the earlier [pytest-base-url issue #34](https://github.com/pytest-dev/pytest-base-url/issues/34). + No duplicate issue has been submitted from this project. +- The installed `pytest-base-url` plugin reads its fixture value from + `config.getoption("base_url")`. Its controller configuration hook resolves the ini + value but returns early on workers. This agrees with the observed missing value; + the upstream discussion covers the broader option propagation problem. +- Our fixture uses `getoption("base_url") or getini("base_url")`. Five offline + [infrastructure checks](../../tests/unit/test_fixtures.py) cover serial/parallel ini + configuration, environment/CLI precedence and browser-free API setup. +- This defect is separate from the historical CI access challenge. The old suite + already had an ini fallback, but that fallback ignored CLI/environment overrides. diff --git a/docs/bugs/evidence/base-url-xdist.txt b/docs/bugs/evidence/base-url-xdist.txt new file mode 100644 index 0000000..7ee9177 --- /dev/null +++ b/docs/bugs/evidence/base-url-xdist.txt @@ -0,0 +1,33 @@ +COMMAND: python -m pytest -q +EXIT: 0 +. [100%] +1 passed in 0.03s + +COMMAND: python -m pytest -q -n 2 +EXIT: 1 +bringing up nodes... +bringing up nodes... + +F [100%] +=================================== FAILURES =================================== +____________________________ test_base_url_from_ini ____________________________ +[gw0] linux -- Python 3.12.3 /.venv/bin/python + +base_url = None + + def test_base_url_from_ini(base_url): +> assert base_url == "http://example.test" +E AssertionError: assert None == 'http://example.test' + +test_base_url.py:2: AssertionError +=========================== short test summary info ============================ +FAILED test_base_url.py::test_base_url_from_ini - AssertionError: assert None... +1 failed in 0.86s + +COMMAND: python -m pytest -q -n 2 --base-url http://example.test +EXIT: 0 +bringing up nodes... +bringing up nodes... + +. [100%] +1 passed in 0.73s diff --git a/docs/ci-troubleshooting.md b/docs/ci-troubleshooting.md new file mode 100644 index 0000000..0594078 --- /dev/null +++ b/docs/ci-troubleshooting.md @@ -0,0 +1,93 @@ +# CI failure analysis + +The failed [main run 27691865635](https://github.com/larelgit/ecommerce-e2e-playwright/actions/runs/27691865635) +ran commit `9d42b04` on 2026-06-17. Dependency/browser installation succeeded. +Chromium and Firefox each finished with **9 failed, 5 errors, 24 reruns**; WebKit passed. + +## Evidence and diagnosis + +The [Chromium job](https://github.com/larelgit/ecommerce-e2e-playwright/actions/runs/27691865635/job/81904703950) +reported the following home-page assertion: + +```text +Page title expected to be 'Automation Exercise' +Actual value: One moment, please... +``` + +Both failed engines then timed out looking for normal shop controls. Account creation +and deletion failed with `Max redirect count exceeded`. Their request logs showed +HTTP 302 responses with `Location: /`, followed by repeated redirects at `/`. + +These observations show a target access challenge and redirect loop, rather than +nine independent selector regressions. The logs do not establish exactly which +site-side rule triggered the challenge. An IP, browser, traffic or regional rule is +possible, but unverified. WebKit's passing job does not prove the engine itself caused +the difference, because each job ran on a separate runner. + +## Changes in this repository + +- The preflight checks the catalog API and, for UI jobs, the selected browser's home + page. It writes `reports/preflight-*.json` and exits nonzero on failure. +- API requests disable redirects. HTTP status is checked before JSON, so the first + unexpected redirect is visible instead of a long redirect-loop traceback. +- Page navigation waits for DOM readiness, rejects HTTP errors, and waits up to + 15 seconds for recognized challenge titles to clear. Persistent challenges raise + `TargetUnavailableError` before tests start locating form controls. +- PRs run smoke checks; the full browser matrix runs on main/scheduled/manual runs. + Browser jobs are serialized and each uses two workers instead of four. API tests + run once in their own job. This reduces load; it cannot guarantee site access. +- Blanket test reruns and fixed navigation delays were removed. A failed environment + check still fails CI; it is never converted into a pass or a skip. +- The base URL fixture now preserves CLI/environment overrides on xdist workers. + Cleanup is registered before account creation and validates deletion responses. + +## Diagnose a new failure + +Inside the project's activated virtual environment: + +```bash +python scripts/check_target.py --browser chromium +pytest tests/ui/test_smoke.py --browser chromium --tracing retain-on-failure +``` + +Expected: preflight reports `"status": "passed"`; the home-page test passes. If a +compatible target is selected with pytest's `--base-url`, pass the same URL to +the preflight's `--base-url` option. Both commands also read `PYTEST_BASE_URL`. + +If the report shows 302, 403, 5xx or a challenge title, check access to the target +from the runner's network. A persistent site-side block requires a permitted runner +or a target instance you control. Increasing locator timeouts will not restore access. +If preflight passes but a UI assertion fails, inspect its screenshot and trace: + +```bash +playwright show-trace test-results//trace.zip +``` + +If the error instead names a missing shared library, the browser cannot start on +that machine. `playwright install --with-deps ` installs system packages on +Ubuntu and may require sudo. CI already includes this installation step. + +## Verification scope + +Local verification on 2026-09-08: + +| Check | Result | +|---|---| +| Full default suite, `pytest -n 2`, without reruns | 32 passed in 42.79 s: 15 API, 12 Chromium UI, 5 offline | +| Firefox UI, two workers | 12 passed in 40.05 s | +| WebKit UI, two workers with temporary library setup | 12 passed in 49.12 s | +| Ruff lint/format, Pyright, actionlint, pip dependency consistency | Passed | +| Preflight against local HTTP 302 loop, HTTP 503 and HTML challenge responses | Each exited 1 with diagnostics; redirects were not followed | +| Preflight against healthy local API, including environment URL override | Passed | + +The development machine lacked WebKit's GStreamer/AVIF libraries and passwordless +sudo. For verification only, the Ubuntu `libgstreamer-plugins-bad1.0-0`, `libavif16`, +`libgav1-1` and `libyuv0` packages were downloaded and extracted under +`/tmp/ecommerce-webkit-deps`. A temporary launcher added those library paths to the +same Playwright WebKit binary. No system packages were installed. Normal WebKit +execution on this PC still needs the documented browser system dependencies. +The standard Chromium and Firefox commands needed no such adjustment. + +Local results establish that the scenarios work from the development machine. +The updated workflow must still run on GitHub-hosted runners after publication; +local passes cannot establish that their network will be accepted by the live site. diff --git a/pages/base_page.py b/pages/base_page.py index 6d94d45..92820f6 100644 --- a/pages/base_page.py +++ b/pages/base_page.py @@ -1,4 +1,8 @@ -from playwright.sync_api import Locator, Page +import re + +from playwright.sync_api import Locator, Page, expect + +from utils.target import TargetUnavailableError class BasePage: @@ -6,24 +10,27 @@ class BasePage: path = "/" - # The demo site sheds load with 503 "queue full" pages under traffic; - # navigation retries keep a busy minute from failing the whole run. - RETRY_DELAYS_S = (5, 10, 20) - def __init__(self, page: Page): self.page = page def open(self) -> None: - # base_url is configured in pytest.ini, so paths stay relative - response = self.page.goto(self.path) - for delay in self.RETRY_DELAYS_S: - if response is None or response.status < 500: - return - self.page.wait_for_timeout(delay * 1000) - response = self.page.goto(self.path) - assert response is None or response.status < 500, ( - f"{self.path} kept returning HTTP {response.status} (site under heavy load)" - ) + # Waiting for DOM readiness avoids slow third-party load events. + response = self.page.goto(self.path, wait_until="domcontentloaded") + if response is None or response.status >= 400: + status = response.status if response else "no response" + raise TargetUnavailableError(f"GET {self.path}: HTTP {status}") + # Access challenges can be HTTP 200. Give a transient interstitial time + # to finish, then report the environment problem before locating controls. + try: + expect(self.page).not_to_have_title( + re.compile(r"one moment|just a moment|attention required", re.I), + timeout=15_000, + ) + except AssertionError as error: + raise TargetUnavailableError( + f"GET {self.path}: target access challenge ({self.page.title()!r}). " + "Check the runner's access to the demo site." + ) from error @staticmethod def parse_price(text: str) -> int: diff --git a/pages/flows.py b/pages/flows.py index a356d2a..e31c9d5 100644 --- a/pages/flows.py +++ b/pages/flows.py @@ -3,6 +3,7 @@ Lives outside the page objects because it spans pages (login -> signup), and outside conftest because it's a reusable action, not a fixture. """ + from playwright.sync_api import Page, expect from pages.login_page import LoginPage diff --git a/pyproject.toml b/pyproject.toml new file mode 100644 index 0000000..e6d4f2a --- /dev/null +++ b/pyproject.toml @@ -0,0 +1,51 @@ +[build-system] +requires = ["setuptools==84.0.0"] +build-backend = "setuptools.build_meta" + +[project] +name = "ecommerce-e2e-playwright" +version = "0.1.0" +description = "Production-style UI and API tests for Automation Exercise" +readme = "README.md" +requires-python = ">=3.12" +dependencies = [ + "playwright==1.62.0", + "pytest==9.1.0", + "pytest-playwright==0.8.0", + "pytest-html==4.2.0", + "pytest-xdist==3.8.0", + "faker==40.23.0", +] + +[project.optional-dependencies] +dev = ["ruff==0.16.6", "pyright==1.1.411"] + +[tool.setuptools.packages.find] +include = ["pages*", "utils*"] + +[tool.pytest.ini_options] +testpaths = ["tests"] +base_url = "https://automationexercise.com" +addopts = "--strict-markers --strict-config --screenshot only-on-failure --output test-results -ra" +markers = [ + "smoke: quick checks of the main UI and API journeys", + "regression: full UI and API coverage, including smoke tests", + "api: HTTP tests that do not launch a browser", + "ui: browser end-to-end tests", + "critical: account, cart, and purchase journeys", + "unit: offline checks of test infrastructure", +] + +[tool.ruff] +target-version = "py312" +line-length = 88 + +[tool.ruff.lint] +select = ["E4", "E7", "E9", "F", "I", "UP", "B"] + +[tool.pyright] +include = ["conftest.py", "pages", "utils", "tests", "scripts"] +exclude = [".venv", "test-results"] +pythonVersion = "3.12" +typeCheckingMode = "basic" +reportMissingTypeStubs = false diff --git a/pytest.ini b/pytest.ini deleted file mode 100644 index 7e2915d..0000000 --- a/pytest.ini +++ /dev/null @@ -1,6 +0,0 @@ -[pytest] -testpaths = tests -base_url = https://automationexercise.com -# Browser is NOT hardcoded here so `--browser firefox|webkit` works from the CLI -# and the CI matrix. Default (chromium) comes from pytest-playwright. -addopts = --screenshot only-on-failure --output test-results diff --git a/requirements.txt b/requirements.txt index a8a6739..b3e88a8 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,7 +1,2 @@ -# Pinned for reproducible CI runs. Bump deliberately, not by surprise. -pytest==9.1.0 -pytest-playwright==0.8.0 -pytest-html==4.2.0 -pytest-rerunfailures==16.3 -pytest-xdist==3.8.0 -faker==40.23.0 +# Compatibility entry point; dependency pins live in pyproject.toml. +-e .[dev] diff --git a/scripts/check_target.py b/scripts/check_target.py new file mode 100644 index 0000000..c479e79 --- /dev/null +++ b/scripts/check_target.py @@ -0,0 +1,61 @@ +"""Read-only CI preflight with an optional check from the selected browser.""" + +import argparse +import json +import os +import tomllib +from pathlib import Path + +from playwright.sync_api import Error, sync_playwright + +from pages.home_page import HomePage +from utils.api import assert_api_response +from utils.target import TargetUnavailableError + + +def main() -> int: + config = tomllib.loads( + (Path(__file__).resolve().parents[1] / "pyproject.toml").read_text() + ) + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument( + "--base-url", + default=os.getenv("PYTEST_BASE_URL") + or config["tool"]["pytest"]["ini_options"]["base_url"], + ) + parser.add_argument("--browser", choices=["chromium", "firefox", "webkit"]) + parser.add_argument("--output", type=Path, default=Path("reports/preflight.json")) + args = parser.parse_args() + report = {"base_url": args.base_url, "browser": args.browser, "status": "failed"} + try: + with sync_playwright() as playwright: + api = playwright.request.new_context(base_url=args.base_url) + try: + body = assert_api_response( + api.get("/api/productsList", max_redirects=0), 200 + ) + assert isinstance(body.get("products"), list) and body["products"], ( + "Catalog API returned no products" + ) + finally: + api.dispose() + if args.browser: + browser = getattr(playwright, args.browser).launch() + try: + page = browser.new_page(base_url=args.base_url) + home = HomePage(page) + home.open() + home.featured_items.first.wait_for(state="visible", timeout=15_000) + finally: + browser.close() + report["status"] = "passed" + except (AssertionError, Error, TargetUnavailableError) as error: + report["error"] = str(error).split("Call log:")[0].strip() + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps(report, indent=2)) + return 0 if report["status"] == "passed" else 1 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/tests/api/test_accounts.py b/tests/api/test_accounts.py new file mode 100644 index 0000000..f941802 --- /dev/null +++ b/tests/api/test_accounts.py @@ -0,0 +1,141 @@ +"""Account creation, authentication, deletion, and negative validation.""" + +import pytest +from playwright.sync_api import APIRequestContext + +from utils.api import account_payload, assert_api_response +from utils.data_generator import User, generate_user + +pytestmark = [pytest.mark.api, pytest.mark.regression] + + +@pytest.mark.smoke +@pytest.mark.critical +def test_create_account_persists_details( + api: APIRequestContext, new_user: User +) -> None: + assert_api_response( + api.post("/api/createAccount", form=account_payload(new_user), max_redirects=0), + 201, + "User created!", + ) + body = assert_api_response( + api.get( + "/api/getUserDetailByEmail", + params={"email": new_user["email"]}, + max_redirects=0, + ), + 200, + ) + saved = body["user"] + assert saved["id"] > 0 + for key in ("name", "email", "first_name", "last_name", "address1", "city"): + assert saved[key] == new_user[key] + + +def test_create_account_rejects_duplicate_email( + api: APIRequestContext, registered_user: User +) -> None: + assert_api_response( + api.post( + "/api/createAccount", + form=account_payload(registered_user), + max_redirects=0, + ), + 400, + "Email already exists!", + ) + + +@pytest.mark.smoke +@pytest.mark.critical +def test_login_with_valid_credentials( + api: APIRequestContext, registered_user: User +) -> None: + assert_api_response( + api.post( + "/api/verifyLogin", + form={ + "email": registered_user["email"], + "password": registered_user["password"], + }, + max_redirects=0, + ), + 200, + "User exists!", + ) + + +def test_login_rejects_wrong_password( + api: APIRequestContext, registered_user: User +) -> None: + assert_api_response( + api.post( + "/api/verifyLogin", + form={ + "email": registered_user["email"], + "password": "wrong-" + registered_user["password"], + }, + max_redirects=0, + ), + 404, + "User not found!", + ) + + +def test_login_rejects_unknown_account(api: APIRequestContext) -> None: + unknown_user = generate_user() + assert_api_response( + api.post( + "/api/verifyLogin", + form={"email": unknown_user["email"], "password": unknown_user["password"]}, + max_redirects=0, + ), + 404, + "User not found!", + ) + + +@pytest.mark.parametrize("missing_field", ["email", "password"]) +def test_login_requires_both_credentials( + api: APIRequestContext, missing_field: str +) -> None: + user = generate_user() + form: dict[str, str | float | bool] = { + "email": user["email"], + "password": user["password"], + } + del form[missing_field] + assert_api_response( + api.post("/api/verifyLogin", form=form, max_redirects=0), + 400, + "Bad request, email or password parameter is missing in POST request.", + ) + + +def test_login_rejects_delete_method(api: APIRequestContext) -> None: + assert_api_response( + api.delete("/api/verifyLogin", max_redirects=0), + 405, + "This request method is not supported.", + ) + + +@pytest.mark.critical +def test_deleted_account_can_no_longer_login( + api: APIRequestContext, registered_user: User +) -> None: + credentials: dict[str, str | float | bool] = { + "email": registered_user["email"], + "password": registered_user["password"], + } + assert_api_response( + api.delete("/api/deleteAccount", form=credentials, max_redirects=0), + 200, + "Account deleted!", + ) + assert_api_response( + api.post("/api/verifyLogin", form=credentials, max_redirects=0), + 404, + "User not found!", + ) diff --git a/tests/api/test_products.py b/tests/api/test_products.py new file mode 100644 index 0000000..16a48c2 --- /dev/null +++ b/tests/api/test_products.py @@ -0,0 +1,84 @@ +"""Catalog contracts, product search, and request validation.""" + +from uuid import uuid4 + +import pytest +from playwright.sync_api import APIRequestContext + +from utils.api import assert_api_response + +pytestmark = [pytest.mark.api, pytest.mark.regression] + + +@pytest.mark.smoke +def test_products_have_required_fields(api: APIRequestContext) -> None: + body = assert_api_response(api.get("/api/productsList", max_redirects=0), 200) + products = body["products"] + assert isinstance(products, list) and products + ids = [] + for product in products: + assert isinstance(product["id"], int) and product["id"] > 0 + ids.append(product["id"]) + for key in ("name", "brand"): + assert isinstance(product[key], str) and product[key].strip() + assert product["price"].startswith("Rs. ") + assert int(product["price"].removeprefix("Rs. ")) > 0 + assert product["category"]["category"] + assert product["category"]["usertype"]["usertype"] + assert len(ids) == len(set(ids)), "Product IDs must be unique" + + +def test_products_reject_post(api: APIRequestContext) -> None: + assert_api_response( + api.post("/api/productsList", max_redirects=0), + 405, + "This request method is not supported.", + ) + + +def test_brands_have_unique_ids_and_nonempty_names(api: APIRequestContext) -> None: + body = assert_api_response(api.get("/api/brandsList", max_redirects=0), 200) + brands = body["brands"] + assert isinstance(brands, list) and brands + ids = [] + for brand in brands: + assert isinstance(brand["id"], int) and brand["id"] > 0 + assert isinstance(brand["brand"], str) and brand["brand"].strip() + ids.append(brand["id"]) + assert len(ids) == len(set(ids)), "Brand IDs must be unique" + + +@pytest.mark.smoke +def test_search_returns_a_known_product(api: APIRequestContext) -> None: + catalog = assert_api_response(api.get("/api/productsList", max_redirects=0), 200) + known_product = catalog["products"][0] + result = assert_api_response( + api.post( + "/api/searchProduct", + form={"search_product": known_product["name"]}, + max_redirects=0, + ), + 200, + ) + assert isinstance(result["products"], list) + assert known_product in result["products"] + + +def test_search_with_no_matches_returns_empty_list(api: APIRequestContext) -> None: + body = assert_api_response( + api.post( + "/api/searchProduct", + form={"search_product": f"no-such-product-{uuid4().hex}"}, + max_redirects=0, + ), + 200, + ) + assert body["products"] == [] + + +def test_search_requires_search_parameter(api: APIRequestContext) -> None: + assert_api_response( + api.post("/api/searchProduct", form={}, max_redirects=0), + 400, + "Bad request, search_product parameter is missing in POST request.", + ) diff --git a/tests/test_auth.py b/tests/ui/test_auth.py similarity index 78% rename from tests/test_auth.py rename to tests/ui/test_auth.py index 4017f3e..f133ba3 100644 --- a/tests/test_auth.py +++ b/tests/ui/test_auth.py @@ -1,12 +1,17 @@ """Authentication: signup, login, and the negative paths real clients always ask about.""" + +import pytest from playwright.sync_api import Page, expect from pages.flows import register_via_ui from pages.login_page import LoginPage from utils.data_generator import User +pytestmark = [pytest.mark.ui, pytest.mark.regression] + -def test_register_new_user(page: Page, new_user: User): +@pytest.mark.critical +def test_register_new_user(page: Page, new_user: User) -> None: signup = register_via_ui(page, new_user) # removing the account through the UI doubles as a check of that flow @@ -14,7 +19,9 @@ def test_register_new_user(page: Page, new_user: User): expect(signup.account_deleted_message).to_be_visible() -def test_login_with_valid_credentials(page: Page, registered_user: User): +@pytest.mark.smoke +@pytest.mark.critical +def test_login_with_valid_credentials(page: Page, registered_user: User) -> None: login = LoginPage(page) login.open() expect(login.login_heading).to_be_visible() @@ -23,7 +30,9 @@ def test_login_with_valid_credentials(page: Page, registered_user: User): expect(login.logged_in_as(registered_user["name"])).to_be_visible() -def test_login_with_wrong_password_shows_error(page: Page, registered_user: User): +def test_login_with_wrong_password_shows_error( + page: Page, registered_user: User +) -> None: login = LoginPage(page) login.open() login.login(registered_user["email"], "wrong-" + registered_user["password"]) @@ -32,7 +41,7 @@ def test_login_with_wrong_password_shows_error(page: Page, registered_user: User expect(login.logged_in_as(registered_user["name"])).not_to_be_visible() -def test_signup_with_taken_email_shows_error(page: Page, registered_user: User): +def test_signup_with_taken_email_shows_error(page: Page, registered_user: User) -> None: login = LoginPage(page) login.open() login.start_signup("Another Person", registered_user["email"]) diff --git a/tests/test_cart.py b/tests/ui/test_cart.py similarity index 80% rename from tests/test_cart.py rename to tests/ui/test_cart.py index 7df9337..29763f7 100644 --- a/tests/test_cart.py +++ b/tests/ui/test_cart.py @@ -1,11 +1,17 @@ """Cart behaviour: adding, quantity/total maths, and removal.""" + +import pytest from playwright.sync_api import Page, expect from pages.cart_page import CartPage from pages.product_page import ProductPage +pytestmark = [pytest.mark.ui, pytest.mark.regression] + -def test_add_product_to_cart(page: Page): +@pytest.mark.smoke +@pytest.mark.critical +def test_add_product_to_cart(page: Page) -> None: products = ProductPage(page) products.open() added_name = products.add_to_cart(0) @@ -17,7 +23,7 @@ def test_add_product_to_cart(page: Page): assert cart.item_quantity(0) == 1 -def test_cart_total_reflects_quantity(page: Page): +def test_cart_total_reflects_quantity(page: Page) -> None: products = ProductPage(page) products.open() products.open_details(0) @@ -33,7 +39,7 @@ def test_cart_total_reflects_quantity(page: Page): assert cart.item_total(0) == unit_price * 3 -def test_remove_product_from_cart(page: Page): +def test_remove_product_from_cart(page: Page) -> None: products = ProductPage(page) products.open() products.add_to_cart(0) diff --git a/tests/test_checkout.py b/tests/ui/test_checkout.py similarity index 90% rename from tests/test_checkout.py rename to tests/ui/test_checkout.py index 277c011..920e5cc 100644 --- a/tests/test_checkout.py +++ b/tests/ui/test_checkout.py @@ -1,4 +1,6 @@ """Checkout: the guest gate and the full purchase journey (the suite's showcase).""" + +import pytest from playwright.sync_api import Page, expect from pages.cart_page import CartPage @@ -7,8 +9,10 @@ from pages.product_page import ProductPage from utils.data_generator import User, generate_payment_card +pytestmark = [pytest.mark.ui, pytest.mark.regression] + -def test_checkout_requires_login(page: Page): +def test_checkout_requires_login(page: Page) -> None: """Guests can fill a cart, but checkout must ask them to sign in.""" products = ProductPage(page) products.open() @@ -20,7 +24,8 @@ def test_checkout_requires_login(page: Page): expect(cart.checkout_login_prompt).to_be_visible() -def test_full_purchase_flow(page: Page, new_user: User): +@pytest.mark.critical +def test_full_purchase_flow(page: Page, new_user: User) -> None: """Registration -> product -> cart -> checkout -> payment -> confirmation.""" signup = register_via_ui(page, new_user) diff --git a/tests/test_search.py b/tests/ui/test_search.py similarity index 60% rename from tests/test_search.py rename to tests/ui/test_search.py index ff1b8c7..4e0d54f 100644 --- a/tests/test_search.py +++ b/tests/ui/test_search.py @@ -1,10 +1,15 @@ """Product search: the happy path and the empty-result case.""" + +import pytest from playwright.sync_api import Page, expect from pages.product_page import ProductPage +pytestmark = [pytest.mark.ui, pytest.mark.regression] + -def test_search_finds_matching_products(page: Page): +@pytest.mark.smoke +def test_search_finds_matching_products(page: Page) -> None: products = ProductPage(page) products.open() products.search("dress") @@ -12,13 +17,13 @@ def test_search_finds_matching_products(page: Page): expect(products.search_results_heading).to_be_visible() names = products.product_names() assert names, "search for a common term returned no products" - # A strict all() check here exposed a real quirk: the engine also matches - # fields hidden from the results card (category/description), e.g. "dress" - # returns "Sleeves Top and Short". Documented in README "Field notes". + # Search has returned names without "dress" (see README field notes). + # Name-only matching is not a documented contract; the matching fields + # and the relevance requirement still need confirmation from the owner. assert any("dress" in name.lower() for name in names), names -def test_search_with_no_matches_shows_empty_grid(page: Page): +def test_search_with_no_matches_shows_empty_grid(page: Page) -> None: products = ProductPage(page) products.open() products.search("definitely-not-a-product-9000") diff --git a/tests/test_smoke.py b/tests/ui/test_smoke.py similarity index 68% rename from tests/test_smoke.py rename to tests/ui/test_smoke.py index 67ab8b3..014b34c 100644 --- a/tests/test_smoke.py +++ b/tests/ui/test_smoke.py @@ -1,10 +1,15 @@ """Smoke check: the app is up and serves the home page.""" + +import pytest from playwright.sync_api import Page, expect from pages.home_page import HomePage +pytestmark = [pytest.mark.ui, pytest.mark.regression] + -def test_home_page_opens(page: Page): +@pytest.mark.smoke +def test_home_page_opens(page: Page) -> None: home = HomePage(page) home.open() expect(page).to_have_title("Automation Exercise") diff --git a/tests/unit/test_fixtures.py b/tests/unit/test_fixtures.py new file mode 100644 index 0000000..a8391f7 --- /dev/null +++ b/tests/unit/test_fixtures.py @@ -0,0 +1,81 @@ +"""Exercise actual pytest sessions so fixture regressions fail without the site.""" + +import os +import subprocess +import sys +from pathlib import Path + +import pytest + +pytestmark = pytest.mark.unit +ROOT = Path(__file__).resolve().parents[2] + + +def run_probe( + directory: Path, + source: str, + options: list[str], + environment_url: str | None = None, +) -> None: + (directory / "conftest.py").write_text((ROOT / "conftest.py").read_text()) + (directory / "pytest.ini").write_text( + "[pytest]\nbase_url = http://configured.example.test\n" + ) + (directory / "test_probe.py").write_text(source) + environment = os.environ.copy() + for key in ("PYTEST_BASE_URL", "PYTEST_ADDOPTS", "VERIFY_BASE_URL"): + environment.pop(key, None) + if environment_url: + environment["PYTEST_BASE_URL"] = environment_url + result = subprocess.run( + [sys.executable, "-m", "pytest", "-q", *options], + cwd=directory, + env=environment, + capture_output=True, + text=True, + timeout=30, + check=False, + ) + assert result.returncode == 0, result.stdout + result.stderr + + +@pytest.mark.parametrize( + ("options", "environment_url", "expected"), + [ + ([], None, "http://configured.example.test"), + (["-n", "2"], None, "http://configured.example.test"), + ( + ["-n", "2"], + "http://environment.example.test", + "http://environment.example.test", + ), + ( + ["-n", "2", "--base-url", "http://cli.example.test"], + "http://environment.example.test", + "http://cli.example.test", + ), + ], + ids=["serial-ini", "parallel-ini", "parallel-env", "cli-over-env"], +) +def test_base_url_precedence_in_real_pytest_sessions( + tmp_path: Path, options: list[str], environment_url: str | None, expected: str +) -> None: + run_probe( + tmp_path, + f"def test_url(base_url):\n assert base_url == {expected!r}\n", + options, + environment_url, + ) + + +def test_api_fixture_does_not_request_a_browser(tmp_path: Path) -> None: + run_probe( + tmp_path, + "import pytest\n" + "@pytest.fixture(scope='session')\n" + "def browser():\n" + " raise AssertionError('API test requested a browser')\n" + "def test_request_context(api):\n" + " assert api.storage_state() == {'cookies': [], 'origins': []}\n", + [], + ) diff --git a/utils/api.py b/utils/api.py new file mode 100644 index 0000000..88d6dca --- /dev/null +++ b/utils/api.py @@ -0,0 +1,76 @@ +"""Small contract helpers; tests use Playwright's HTTP client directly.""" + +from typing import Any + +from playwright.sync_api import APIRequestContext, APIResponse + +from utils.data_generator import User + + +def account_payload(user: User) -> dict[str, str | float | bool]: + """Send only the documented API fields, mapping the two UI field names.""" + return { + "name": user["name"], + "email": user["email"], + "password": user["password"], + "title": user["title"], + "birth_date": user["birth_date"], + "birth_month": user["birth_month"], + "birth_year": user["birth_year"], + "firstname": user["first_name"], + "lastname": user["last_name"], + "company": user["company"], + "address1": user["address1"], + "address2": user["address2"], + "country": user["country"], + "zipcode": user["zipcode"], + "state": user["state"], + "city": user["city"], + "mobile_number": user["mobile_number"], + } + + +def response_json(response: APIResponse) -> dict[str, Any]: + """Check HTTP before decoding: this API carries application codes in JSON.""" + assert response.status == 200, ( + f"{response.url}: HTTP {response.status}; " + f"Location={response.headers.get('location', '')}. " + "An unexpected redirect/HTML response can indicate a site access challenge." + ) + try: + body = response.json() + except ValueError: + raise AssertionError( + f"{response.url}: expected API JSON, received " + f"{response.headers.get('content-type', '')}; " + "check target availability/access challenges." + ) from None + assert isinstance(body, dict), f"{response.url}: expected a JSON object" + return body + + +def assert_api_response( + response: APIResponse, code: int, message: str | None = None +) -> dict[str, Any]: + body = response_json(response) + assert body.get("responseCode") == code, ( + f"{response.url}: expected responseCode={code}, " + f"got {body.get('responseCode')}; message={body.get('message')}" + ) + if message is not None: + assert body.get("message") == message + return body + + +def delete_account(api: APIRequestContext, user: User) -> None: + """Accept an already removed account, but report all other cleanup failures.""" + response = api.delete( + "/api/deleteAccount", + form={"email": user["email"], "password": user["password"]}, + max_redirects=0, + ) + body = response_json(response) + assert (body.get("responseCode"), body.get("message")) in { + (200, "Account deleted!"), + (404, "Account not found!"), + }, f"Unexpected account cleanup response: {body.get('message')}" diff --git a/utils/data_generator.py b/utils/data_generator.py index e9d1bd1..913c600 100644 --- a/utils/data_generator.py +++ b/utils/data_generator.py @@ -1,4 +1,5 @@ """Test data factories. Every test creates its own user, so tests stay independent.""" + import uuid from typing import TypedDict @@ -9,7 +10,7 @@ class User(TypedDict): """Shape of a generated user. Field names mirror the signup form; - the account API renames some of them (see conftest.registered_user).""" + the account API renames some of them (see utils.api.account_payload).""" title: str name: str diff --git a/utils/target.py b/utils/target.py new file mode 100644 index 0000000..17624f9 --- /dev/null +++ b/utils/target.py @@ -0,0 +1,5 @@ +"""A distinct error for target availability failures in navigation/preflight.""" + + +class TargetUnavailableError(RuntimeError): + """The target cannot serve the application to this test runner."""