Single RQ re-processes requests the batch add reports as already handled #1185

In single-access mode, add_batch_of_requests commits every request the platform accepted to _requests_cache and _head_requests without looking at was_already_handled in the batch response (_request_queue_single_client.py#L189-L202). fetch_next_request then serves the cached copy, which has handled_at=None, and the crawler processes the request again. Nothing in crawlee filters it out on the way to the request handler.

The platform does report the state correctly: batch add on an existing request returns wasAlreadyPresent=true and wasAlreadyHandled=true when the stored request is handled.

It hits requests handled by an earlier client of the same RQ that are not among the 10,000 requests _init_caches prefetches (#L442-L464; the API caps the list at 10,000). In practice that's a run with more than 10,000 requests after a migration, reboot or resurrection: start URLs re-added by crawler.run() and links re-discovered by enqueue_links are crawled a second time, and their results are pushed again.

The JS single backend adds such requests to its handled set and keeps them out of the cache and the head (apify_request_queue_single_backend.ts#L99-L119).

Expected behavior

A request the batch response reports with was_already_handled=True goes to _requests_already_handled and never enters _requests_cache or _head_requests.

Implementation notes

• In the commit loop of add_batch_of_requests, map api_response.processed_requests by unique_key and, for a request reported as was_already_handled, add its id to _requests_already_handled and skip the cache and head. Keep everything else in the loop as it is (in-flight markers, committed_request_ids, metadata counting).
• The shared client caches request.was_already_handled from the caller's object at the same spot (_request_queue_shared_client.py#L186-L198). It doesn't re-process, because fetching goes through the platform head, but use the platform's flag there too for consistency.
• Add a unit test to tests/unit/storage_clients/test_apify_request_queue_client.py using the existing _make_single_client helper: list_requests returns nothing, batch_add_requests reports the request as present and handled, and fetch_next_request() returns None.

✍️ Drafted by Claude Code