From 2a65316c125ab769ae989bdbccf8fa1ed56b9f9c Mon Sep 17 00:00:00 2001 From: Vlada Dusek Date: Fri, 6 Nov 2026 09:37:00 +0100 Subject: [PATCH] docs: merge the test READMEs into `tests/README.md` --- CONTRIBUTING.md | 8 +- tests/README.md | 165 ++++++++++++++++++++++++++++++++++++ tests/e2e/README.md | 100 ---------------------- tests/integration/README.md | 35 -------- tests/unit/README.md | 19 ----- 5 files changed, 169 insertions(+), 158 deletions(-) create mode 100644 tests/README.md delete mode 100644 tests/e2e/README.md delete mode 100644 tests/integration/README.md delete mode 100644 tests/unit/README.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index d4e10e7ca..4822099f2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -30,11 +30,11 @@ We use [ruff](https://docs.astral.sh/ruff/) for linting and formatting, [ty](htt ## Tests -Each test level has a README describing its fixtures, isolation, and required tokens. +[`tests/README.md`](tests/README.md) describes each test level, with its fixtures, isolation, and required tokens. -- **Unit tests** (`tests/unit`) mock the Apify API and need no credentials. See [`tests/unit/README.md`](tests/unit/README.md). Run them with `uv run poe unit-tests`, or a single test with `uv run pytest tests/unit/.py::`. -- **Integration tests** (`tests/integration`) call the real Apify API without deploying Actors. See [`tests/integration/README.md`](tests/integration/README.md). Run them with `uv run poe integration-tests`. -- **E2E tests** (`tests/e2e`) build and run Actors on the Apify platform. See [`tests/e2e/README.md`](tests/e2e/README.md). Run them with `uv run poe e2e-tests`. +- **Unit tests** (`tests/unit`) mock the Apify API and need no credentials. Run them with `uv run poe unit-tests`, or a single test with `uv run pytest tests/unit/.py::`. +- **Integration tests** (`tests/integration`) call the real Apify API without deploying Actors. Run them with `uv run poe integration-tests`. +- **E2E tests** (`tests/e2e`) build and run Actors on the Apify platform. Run them with `uv run poe e2e-tests`. ## Code style diff --git a/tests/README.md b/tests/README.md new file mode 100644 index 000000000..66c43bc92 --- /dev/null +++ b/tests/README.md @@ -0,0 +1,165 @@ +# Tests + +The tests have three levels: + +- [Unit tests](#unit-tests) (`tests/unit`) mock the Apify API and need no credentials. +- [Integration tests](#integration-tests) (`tests/integration`) call the real Apify API without deploying Actors. +- [E2E tests](#e2e-tests) (`tests/e2e`) build and run Actors on the Apify platform. + +Use the lowest level that can test what you need. Integration tests run faster than E2E tests and you can debug them locally, so write an E2E test only when the test needs an Actor built and running on the platform. + +## Fixtures shared by several levels + +- `prepare_test_env` and its autouse wrapper `_isolate_test_environment` (all levels): reset the global state (Actor initialization, service locator, storages) and point `APIFY_LOCAL_STORAGE_DIR` to a temporary directory before each test. +- `apify_client_async` (integration and E2E): a session-scoped `ApifyClientAsync` configured with the test token and API URL. + +## Unit tests + +Unit tests check the SDK's logic in isolation. They mock every external dependency, such as the Apify API and other platform services, so they're fast and need no environment variables or credentials. + +### Running unit tests + +```bash +uv run poe unit-tests +``` + +### Unit test fixtures + +- `apify_client_async_patcher`: patches `ApifyClientAsync` methods to return fixed values or call replacement functions, and records the calls. +- `httpserver`: a local HTTP server (from `pytest-httpserver`) for testing HTTP interactions without real network calls. + +### Troubleshooting + +If a test fails with `SystemExit: 0` raised from `_actor.py` when `async with Actor:` exits, the virtual environment is missing the optional extras. Without Scrapy installed, the Actor exits the process on teardown. Run `uv sync --all-extras` or `uv run poe install-dev`. + +## Integration tests + +Integration tests make real requests to the Apify API, but they don't build or deploy Actors. + +### Running integration tests + +```bash +export APIFY_TEST_USER_API_TOKEN= +uv run poe integration-tests +``` + +To run them against a different environment, also set `APIFY_INTEGRATION_TESTS_API_URL`. + +### Integration test files + +| File | What it tests | +| --- | --- | +| `test_dataset.py` | Dataset operations | +| `test_key_value_store.py` | Key-value store operations | +| `test_request_queue.py` | Request queue operations | +| `test_storages.py` | Behavior across storages: aliases, unnamed default storages, explicit initialization | + +### Integration test fixtures + +- `request_queue_apify`: creates an Apify request queue on the platform, once for each access mode (`single` and `shared`), and drops it after the test. + +### Platform behavior to account for + +- Right after you add a request to a shared RQ, its head can still come back empty. Poll negative assertions, such as `is_finished()`, the same way as positive ones. +- In the response of `update_request`, `wasAlreadyHandled` describes the request's state before the update. +- RQ metadata counts (`handledRequestCount`, `pendingRequestCount`) lag a few seconds behind the operations that change them. + +## E2E tests + +E2E tests build Actors that use the SDK and run them on the Apify platform. Building and deploying the Actors makes them the slowest level. + +### Running E2E tests + +```bash +export APIFY_TEST_USER_API_TOKEN= +uv run poe e2e-tests +``` + +To run them against a different environment, also set `APIFY_INTEGRATION_TESTS_API_URL`. + +### E2E test fixtures + +- `make_actor`: creates a temporary Actor on the platform and builds it, retrying a failed build once. The Actor is deleted after the test. +- `run_actor`: starts an Actor run and waits up to 10 minutes for it to finish. + +### How test Actors are built + +Each test Actor gets the files from `tests/e2e/actor_source_base`, plus a wheel of the SDK built from the current code. The Dockerfile installs the wheel and the other dependencies from `requirements.txt` and runs the `src` package. The test fixture sets the base image to the Python version of the test run. + +### Writing E2E tests + +#### Creating an Actor from a Python function + +You can create an Actor straight from a Python function, which means the linter also checks the Actor's code. + +```python +async def test_something( + make_actor: MakeActorFunction, + run_actor: RunActorFunction, +) -> None: + async def main() -> None: + async with Actor: + print('Hello!') + + actor = await make_actor(label='something', main_func=main) + run_result = await run_actor(actor) + + assert run_result.status == 'SUCCEEDED' +``` + +`make_actor` writes the function to `src/main.py`, after `import asyncio` and `from apify import Actor`. Put any other imports inside the function body. + +#### Creating an Actor from source files + +For a single-file Actor, pass its code as `main_py`: + +```python +async def test_something( + make_actor: MakeActorFunction, + run_actor: RunActorFunction, +) -> None: + expected_output = f'ACTOR_OUTPUT_{crypto_random_object_id(5)}' + main_py_source = f""" + import asyncio + from apify import Actor + async def main(): + async with Actor: + await Actor.set_value('OUTPUT', '{expected_output}') + """ + + actor = await make_actor(label='something', main_py=main_py_source) + await run_actor(actor) + + output_record = await actor.last_run().key_value_store().get_record('OUTPUT') + assert output_record is not None + assert output_record['value'] == expected_output +``` + +For an Actor with several files, pass `source_files`: + +```python +actor_source_files = { + 'src/utils.py': """ + from datetime import datetime, timezone + def get_current_datetime(): + return datetime.now(timezone.utc) + """, + 'src/main.py': """ + import asyncio + from apify import Actor + from .utils import get_current_datetime + async def main(): + async with Actor: + print('Hello! It is ' + str(get_current_datetime())) + """, +} +actor = await make_actor(label='something', source_files=actor_source_files) +``` + +#### Assertions inside Actors + +Test Actors don't run under pytest, so a failed assertion inside one shows only a bare `AssertionError`, without the values involved. Always give the assertion a message: + +```python +assert is_finished is False, f'is_finished={is_finished}' +``` diff --git a/tests/e2e/README.md b/tests/e2e/README.md deleted file mode 100644 index 8f9d5ff1b..000000000 --- a/tests/e2e/README.md +++ /dev/null @@ -1,100 +0,0 @@ -# E2E tests - -These tests build and run Actors using the Python SDK on the Apify platform. They are slower than integration tests (see [`tests/integration/`](../integration/)) because they need to build and deploy Actors. - -When writing new tests, prefer integration tests if possible. Only write E2E tests when you need to test something that requires building and running an Actor on the platform. - -## Running - -```bash -export APIFY_TEST_USER_API_TOKEN= -uv run poe e2e-tests -``` - -To run against a different environment, also set `APIFY_INTEGRATION_TESTS_API_URL`. - -## Key fixtures - -- **`apify_client_async`** — A session-scoped `ApifyClientAsync` instance configured with the test token and API URL. -- **`prepare_test_env`** / **`_isolate_test_environment`** (autouse) — Resets global state and sets `APIFY_LOCAL_STORAGE_DIR` to a temporary directory before each test. -- **`make_actor`** — Factory for creating temporary Actors on the Apify platform (built, then auto-deleted after the test). -- **`run_actor`** — Starts an Actor run and waits for completion (10 min timeout). - -## How to write tests - -### Creating an Actor from a Python function - -You can create Actors straight from a Python function. This is great because the test Actor source code gets checked by the linter. - -```python -async def test_something( - make_actor: MakeActorFunction, - run_actor: RunActorFunction, -) -> None: - async def main() -> None: - async with Actor: - print('Hello!') - - actor = await make_actor(label='something', main_func=main) - run_result = await run_actor(actor) - - assert run_result.status == 'SUCCEEDED' -``` - -The `src/main.py` file will be set to the function definition, prepended with `import asyncio` and `from apify import Actor`. You can add extra imports directly inside the function body. - -### Creating an Actor from source files - -Pass the `main_py` argument for a single-file Actor: - -```python -async def test_something( - make_actor: MakeActorFunction, - run_actor: RunActorFunction, -) -> None: - expected_output = f'ACTOR_OUTPUT_{crypto_random_object_id(5)}' - main_py_source = f""" - import asyncio - from datetime import datetime - from apify import Actor - async def main(): - async with Actor: - await Actor.set_value('OUTPUT', '{expected_output}') - """ - - actor = await make_actor(label='something', main_py=main_py_source) - await run_actor(actor) - - output_record = await actor.last_run().key_value_store().get_record('OUTPUT') - assert output_record is not None - assert output_record['value'] == expected_output -``` - -Or pass `source_files` for multi-file Actors: - -```python -actor_source_files = { - 'src/utils.py': """ - from datetime import datetime, timezone - def get_current_datetime(): - return datetime.now(timezone.utc) - """, - 'src/main.py': """ - import asyncio - from apify import Actor - from .utils import get_current_datetime - async def main(): - async with Actor: - print('Hello! It is ' + str(get_current_datetime())) - """, -} -actor = await make_actor(label='something', source_files=actor_source_files) -``` - -### Assertions inside Actors - -Since test Actors are not executed as standard pytest tests, we don't get introspection of assertion expressions. In case of failure, only a bare `AssertionError` is shown. Always include explicit assertion messages: - -```python -assert is_finished is False, f'is_finished={is_finished}' -``` diff --git a/tests/integration/README.md b/tests/integration/README.md deleted file mode 100644 index ce49e1975..000000000 --- a/tests/integration/README.md +++ /dev/null @@ -1,35 +0,0 @@ -# Integration tests - -These tests make real requests to the Apify API, but do not build or deploy Actors on the platform. They are faster than E2E tests and can be fully debugged locally. - -When writing new tests, prefer this level if possible. Only write E2E tests (see [`tests/e2e/`](../e2e/)) when you need to test something that requires building and running an Actor on the platform. - -## Running - -```bash -export APIFY_TEST_USER_API_TOKEN= -uv run poe integration-tests -``` - -To run against a different environment, also set `APIFY_INTEGRATION_TESTS_API_URL`. - -## Structure - -| File | Description | -| --- | --- | -| `test_dataset.py` | Dataset operations | -| `test_key_value_store.py` | Key-value store operations | -| `test_request_queue.py` | Request queue operations | -| `test_storages.py` | Cross-storage tests (aliases, unnamed defaults, explicit init) | - -## Key fixtures - -- **`apify_client_async`** — A session-scoped `ApifyClientAsync` instance configured with the test token and API URL. -- **`request_queue_apify`** — Creates a parametrized (`single`/`shared` access mode) Apify request queue on the platform, yields it, and drops it after the test. Defined in `conftest.py`. -- **`prepare_test_env`** / **`_isolate_test_environment`** (autouse) — Resets global state and sets `APIFY_LOCAL_STORAGE_DIR` to a temporary directory before each test. - -## Platform behavior to account for - -- Right after a request is added to a shared RQ, the head can still come back empty. Poll negative assertions (such as `is_finished()`) as well as positive ones. -- In the response of `update_request`, `wasAlreadyHandled` reflects the request's state before the update. -- RQ metadata counts (`handledRequestCount`, `pendingRequestCount`) lag a few seconds behind the operations that change them. diff --git a/tests/unit/README.md b/tests/unit/README.md deleted file mode 100644 index fb8b8c903..000000000 --- a/tests/unit/README.md +++ /dev/null @@ -1,19 +0,0 @@ -# Unit tests - -These tests verify the SDK's internal logic in isolation, without making any real API calls. All external dependencies (Apify API, platform services) are mocked. They are fast and can be run without any environment variables or credentials. - -## Running - -```bash -uv run poe unit-tests -``` - -## Key fixtures - -- **`prepare_test_env`** / **`_isolate_test_environment`** (autouse) — Resets global state (Actor initialization, service locator, storage) and sets `APIFY_LOCAL_STORAGE_DIR` to a temporary directory before each test. -- **`apify_client_async_patcher`** — Helper for patching `ApifyClientAsync` methods to return fixed values or replacement functions, with automatic call tracking. -- **`httpserver`** — Local HTTP server (via `pytest-httpserver`) for testing HTTP interactions without real network calls. - -## Troubleshooting - -If tests fail with `SystemExit: 0` raised from `_actor.py` when `async with Actor:` exits, the virtual environment lacks the optional extras. Without Scrapy installed, the Actor exits the process on teardown. Run `uv sync --all-extras` (or `uv run poe install-dev`).