Retries can duplicate side effects of non-idempotent requests #990

Summary

The HTTP client retries every request on transient failures, regardless of HTTP method. _is_retryable_error treats all impit.HTTPError subclasses as retryable, and _make_request retries any 429 or 5xx response (src/apify_client/http_clients/_impit.py, sync at ~L235-247, async at ~L496-508).

A POST can commit server-side and still be retried when the response never reaches the client — a dropped connection, or a gateway 5xx emitted after the write. The retry then repeats an operation that already happened.

Observed in this CI run: attempt = 2, method = 'POST', url = 'https://api.apify.com/v2/actors', followed by ConflictError: Some other Actor already has this name ("python-client-test-actor-0xjp5vwi") — on a name generated randomly moments earlier, so the only thing that could have taken it was the client's own first attempt.

Impact

Two flavours, depending on whether the endpoint has a uniqueness constraint.

1. Fails loudly, with a misleading error. Creates of named resources reject the retry:

The resource does exist, but the caller gets a conflict error and no way to distinguish it from a genuine name collision. This is the flake worked around test-side in #971 — CI is green, callers still hit it.

2. Silently duplicates the side effect. Endpoints without a uniqueness constraint accept the retry and no error surfaces:

• actor.start() / actor.call() / task.start() / task.call() — POST /v2/acts/{id}/runs has no idempotency mechanism, so the Actor starts twice. The extra run consumes compute and is billed.
• webhooks().create() — the API accepts an idempotencyKey, and create() exposes it, but the client never populates one. Callers who leave it unset get two webhooks for the same event, and every trigger fires twice.
• dataset.push_items() — items pushed twice.

Notes towards a fix

• Idempotency keys are the proper fix. There is already a precedent in this client: run.charge() generates an idempotency-key header when the caller doesn't supply one (_resource_clients/run.py). Extending the same pattern is cheap where the API supports it (webhooks), but it needs platform support for the endpoints that matter most (run start, resource creates).
• Retrying 429 is always safe — the request was rejected, not processed. Only 5xx and transport errors are ambiguous.
• Simply not retrying non-idempotent methods is a behaviour change, not a pure fix: transient failures that happen before the commit are currently rescued by the retry, and would start surfacing as errors.
• apify-client-js retries POST the same way (src/http_client.ts), so whatever we settle on should be aligned across clients.

✍️ Drafted by Claude Code