Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/02_concepts/02_error-handling.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ const client = new ApifyClient({
token: 'MY-APIFY-TOKEN',
maxRetries: 8,
minDelayBetweenRetriesMillis: 500, // 0.5s
timeoutSecs: 360, // 6 mins
});
```

A request that times out is retried the same way, with a longer timeout on each attempt. For how the timeouts are set, see [Timeouts](./06_timeouts.md).
66 changes: 66 additions & 0 deletions docs/02_concepts/06_timeouts.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
---
id: timeouts
title: Timeouts
sidebar_label: Timeouts
description: 'Configure the tiered timeouts that bound how long API requests made by the Apify API client for JavaScript can take.'
---

import ApiLink from '@theme/ApiLink';

The client gives every API request a timeout from one of three tiers, each with a default duration suited to the kind of request it covers. Methods that poll for a job to finish run without one:

| Tier | Default | Purpose |
| --- | --- | --- |
| `short` | 5 seconds | Metadata reads and writes (`get()`, `update()`, `delete()`) |
| `medium` | 30 seconds | Listing, batch and trigger operations (`list()`, `start()`, `batchAddRequests()`) |
| `long` | 360 seconds | Downloads, uploads and streaming (`listItems()`, `setRecord()`, `log().get()`) |
| `noTimeout` | none | Polling that waits for a job to finish (`call()`, `waitForFinish()`) |

Every client method is assigned the tier that matches the expected duration of its request. The reference of each method names its tier. You don't need to change the tiers unless you work with unusually large payloads or a slow network.

The client never aborts a request that runs without a timeout, so a connection that stalls is not retried either. <ApiLink to="class/ActorClient#call">`ActorClient.call()`</ApiLink> and <ApiLink to="class/RunClient#waitForFinish">`RunClient.waitForFinish()`</ApiLink> poll until the job ends; an explicit `timeoutSecs` bounds the requests they send, and has to leave room for the minute the API may hold each poll.

Methods such as <ApiLink to="class/ActorClient#start">`ActorClient.start()`</ApiLink> and <ApiLink to="class/RunClient#get">`RunClient.get()`</ApiLink> take a `waitForFinish` parameter, which asks the API to hold the response until the job finishes, for up to a minute. Such a request gets the requested wait on top of its tier, so the client doesn't abort it while the API is still holding it.

## Configuring the tiers

Set the duration of each tier on the <ApiLink to="class/ApifyClient">`ApifyClient`</ApiLink> constructor. The `timeoutMaxSecs` option caps the timeout of any single request attempt. It bounds the growth of the timeout across retries, and it caps tier and per-call timeouts alike, so raise it whenever you need a request timeout longer than the default 360 seconds. A tier configured above the cap is capped too, and the client logs a warning to make the cut-off visible.

```js
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({
token: 'MY-APIFY-TOKEN',
timeoutShortSecs: 10,
timeoutMediumSecs: 60,
timeoutLongSecs: 600,
timeoutMaxSecs: 600,
});
```

A single request queue client can be held to a shorter budget with `client.requestQueue(id, { timeoutSecs })`, which caps the default tier of every request that client sends. A per-call `timeoutSecs` is not capped by it.

## Per-call overrides

Every method that sends a request accepts a `timeoutSecs` option, which replaces the tier of the method for that call. Pass a number of seconds for an exact duration, a tier name to switch tiers, or `'noTimeout'` to let the request run for as long as it takes. For the type, see <ApiLink to="interface/TimeoutOptions">`TimeoutOptions`</ApiLink>.

```js
const datasetClient = client.dataset('my-dataset-id');

// An exact timeout for this call.
const { items } = await datasetClient.listItems({ timeoutSecs: 120 });

// Another tier.
const dataset = await datasetClient.get({ timeoutSecs: 'long' });

// No timeout at all.
await datasetClient.pushItems(items, { timeoutSecs: 'noTimeout' });
```

A number above `timeoutMaxSecs` is capped at it, and the client logs a warning. To let such a call use its full timeout, raise `timeoutMaxSecs` in the client constructor.

Methods that start an Actor run keep the API's run timeout apart from the request timeout. The `runTimeoutSecs` option of `start()`, `call()` and `resurrect()` bounds how long the run may execute on the platform, while `timeoutSecs` bounds the request that starts it.

## Interaction with retries

Timeouts work together with [retries](./02_error-handling.md#retries-with-exponential-backoff). A request that times out counts as a failed attempt and is retried, up to `maxRetries` times. The timeout applies to each attempt on its own, and doubles with every retry up to `timeoutMaxSecs`, so a request that timed out at 5 seconds gets 10 seconds on the second attempt and 20 on the third.
46 changes: 41 additions & 5 deletions docs/04_upgrading/upgrading_v3.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,8 @@ await client.actor('my-actor').update([{ name: 'my-actor' }]);
`ow` only checked the type, so `Infinity` passed as a number and an invalid `Date` passed as a date. Zod additionally requires a *finite* number and a *valid* date, so both now throw:

```js
// Now throws: Invalid input: expected a finite number at `timeout`, got `Infinity`
await client.actor('my-actor').call(undefined, { timeout: Infinity });
// Now throws: Invalid input: expected a finite number at `runTimeoutSecs`, got `Infinity`
await client.actor('my-actor').call(undefined, { runTimeoutSecs: Infinity });

// Now throws: Invalid input: expected a valid date at `startedBefore`
// Invalid input: expected string, received Date at `startedBefore`
Expand All @@ -80,7 +80,7 @@ await client.actor('my-actor').runs().list({ startedBefore: new Date('nonsense')

The second example reports a line per arm, because `startedBefore` accepts either a `Date` or a string.

This affects numeric options such as `waitSecs`, `timeout` and `memory`, and date options such as `startedBefore` / `startedAfter`. `KeyValueStoreClient.setRecord()` rejects `NaN` and `Infinity` as a record value too, since `JSON.stringify()` turns both into `null`.
This affects numeric options such as `waitSecs`, `runTimeoutSecs` and `memory`, and date options such as `startedBefore` / `startedAfter`. `KeyValueStoreClient.setRecord()` rejects `NaN` and `Infinity` as a record value too, since `JSON.stringify()` turns both into `null`.

### A few always-rejected options are gone from the types

Expand Down Expand Up @@ -268,9 +268,45 @@ A trailing slash appears only on a field the API returns without a path, so on `

A URL field whose value isn't a valid absolute URL now fails response validation and throws <ApiLink to="class/ResponseValidationError">`ResponseValidationError`</ApiLink>, the same as any other field that doesn't match the specification.

## `versions().list()` and `envVars().list()` take no options
## Timeouts are configured per tier

<ApiLink to="class/ActorVersionCollectionClient#list">`ActorVersionCollectionClient.list()`</ApiLink> and <ApiLink to="class/ActorEnvVarCollectionClient#list">`ActorEnvVarCollectionClient.list()`</ApiLink> now take no arguments. Neither endpoint reads `offset`, `limit` or `desc`, and both return every item in one response, so `chunkSize` had nothing to size either. The `ActorVersionCollectionListOptions` and `ActorEnvVarCollectionListOptions` types that declared those four options, deprecated since v2.21.0, are gone from the package. A call that passed an options object no longer compiles. Drop the argument and the call returns the same items as before.
The `timeoutSecs` option of the <ApiLink to="class/ApifyClient">`ApifyClient`</ApiLink> constructor is gone. Every method takes its timeout from one of three tiers, and the constructor sets the duration of each tier, along with a cap on any single request attempt:

```diff
- const client = new ApifyClient({ token: 'MY-APIFY-TOKEN', timeoutSecs: 360 }); // v2
+ const client = new ApifyClient({ // v3
+ token: 'MY-APIFY-TOKEN',
+ timeoutShortSecs: 5,
+ timeoutMediumSecs: 30,
+ timeoutLongSecs: 360,
+ timeoutMaxSecs: 360,
+ });
```

For what each tier covers and which one a method is assigned, see [Timeouts](../02_concepts/06_timeouts.md).

In v2 only the dataset, key-value store and request queue clients picked a timeout per method, and everything else ran with the global 360 seconds. A metadata call such as `actor.get()` now gets the 5 seconds of the `short` tier and a `list()` call the 30 of `medium`, so a call that used to wait out a slow API can fail sooner. <ApiLink to="class/RequestQueueClient#unlockRequests">`RequestQueueClient.unlockRequests()`</ApiLink> moves the other way, from 30 seconds to 360. Where you need a different duration, every method that sends a request takes a `timeoutSecs` option, which replaces the tier of the method for that call:

```js
await client.actor('my-actor').get({ timeoutSecs: 'long' });
```

`timeoutSecs` means the request timeout on every method, so the run timeout of <ApiLink to="class/ActorClient#start">`ActorClient.start()`</ApiLink> and <ApiLink to="class/ActorClient#call">`call()`</ApiLink>, <ApiLink to="class/TaskClient#start">`TaskClient.start()`</ApiLink> and <ApiLink to="class/TaskClient#call">`call()`</ApiLink>, and <ApiLink to="class/RunClient#resurrect">`RunClient.resurrect()`</ApiLink> is renamed:

```diff
- await client.actor('my-actor').call(input, { timeout: 300 }); // v2
+ await client.actor('my-actor').call(input, { runTimeoutSecs: 300 }); // v3
```

TypeScript reports a leftover `timeout` at compile time, and in JavaScript the option schemas reject it as an unrecognized key, so the call throws an `ArgumentValidationError` instead of silently dropping the run limit. `{ timeout: 0 }`, which asked for an unlimited run in v2, becomes `{ runTimeoutSecs: 0 }`.

The `timeoutSecs` option of <ApiLink to="class/KeyValueStoreClient#setRecord">`KeyValueStoreClient.setRecord()`</ApiLink> keeps its name and its meaning for a number of seconds, and now takes a tier name or `'noTimeout'` as well. The same goes for `client.requestQueue(id, { timeoutSecs })`, which caps the tier of every request that queue client sends rather than timing each one at the given value.

`client.httpClient.call()` takes `timeoutSecs` where it read the axios `timeout` in milliseconds. A direct call that passed `timeout: 30000` has to become `timeoutSecs: 30`.

## `versions().list()` and `envVars().list()` lose their pagination options

<ApiLink to="class/ActorVersionCollectionClient#list">`ActorVersionCollectionClient.list()`</ApiLink> and <ApiLink to="class/ActorEnvVarCollectionClient#list">`ActorEnvVarCollectionClient.list()`</ApiLink> accept only the `timeoutSecs` option. Neither endpoint reads `offset`, `limit` or `desc`, and both return every item in one response, so `chunkSize` had nothing to size either. The `ActorVersionCollectionListOptions` and `ActorEnvVarCollectionListOptions` types that declared those four options, deprecated since v2.21.0, are gone from the package. A call that passed any of them no longer compiles, and throws an `ArgumentValidationError` about an unrecognized key in JavaScript. Drop them and the call returns the same items as before.

## `limit` counts scanned rows when iterating `listItems()`

Expand Down
Loading
Loading