diff --git a/docs/02_concepts/02_error-handling.md b/docs/02_concepts/02_error-handling.md index 515de370d..415437f28 100644 --- a/docs/02_concepts/02_error-handling.md +++ b/docs/02_concepts/02_error-handling.md @@ -160,6 +160,7 @@ const client = new ApifyClient({ token: 'MY-APIFY-TOKEN', maxRetries: 8, minDelayBetweenRetriesMillis: 500, // 0.5s - timeoutSecs: 360, // 6 mins }); ``` + +A request that times out is retried the same way, with a longer timeout on each attempt. For how the timeouts are set, see [Timeouts](./06_timeouts.md). diff --git a/docs/02_concepts/06_timeouts.md b/docs/02_concepts/06_timeouts.md new file mode 100644 index 000000000..5f24e6042 --- /dev/null +++ b/docs/02_concepts/06_timeouts.md @@ -0,0 +1,66 @@ +--- +id: timeouts +title: Timeouts +sidebar_label: Timeouts +description: 'Configure the tiered timeouts that bound how long API requests made by the Apify API client for JavaScript can take.' +--- + +import ApiLink from '@theme/ApiLink'; + +The client gives every API request a timeout from one of three tiers, each with a default duration suited to the kind of request it covers. Methods that poll for a job to finish run without one: + +| Tier | Default | Purpose | +| --- | --- | --- | +| `short` | 5 seconds | Metadata reads and writes (`get()`, `update()`, `delete()`) | +| `medium` | 30 seconds | Listing, batch and trigger operations (`list()`, `start()`, `batchAddRequests()`) | +| `long` | 360 seconds | Downloads, uploads and streaming (`listItems()`, `setRecord()`, `log().get()`) | +| `noTimeout` | none | Polling that waits for a job to finish (`call()`, `waitForFinish()`) | + +Every client method is assigned the tier that matches the expected duration of its request. The reference of each method names its tier. You don't need to change the tiers unless you work with unusually large payloads or a slow network. + +The client never aborts a request that runs without a timeout, so a connection that stalls is not retried either. `ActorClient.call()` and `RunClient.waitForFinish()` poll until the job ends; an explicit `timeoutSecs` bounds the requests they send, and has to leave room for the minute the API may hold each poll. + +Methods such as `ActorClient.start()` and `RunClient.get()` take a `waitForFinish` parameter, which asks the API to hold the response until the job finishes, for up to a minute. Such a request gets the requested wait on top of its tier, so the client doesn't abort it while the API is still holding it. + +## Configuring the tiers + +Set the duration of each tier on the `ApifyClient` constructor. The `timeoutMaxSecs` option caps the timeout of any single request attempt. It bounds the growth of the timeout across retries, and it caps tier and per-call timeouts alike, so raise it whenever you need a request timeout longer than the default 360 seconds. A tier configured above the cap is capped too, and the client logs a warning to make the cut-off visible. + +```js +import { ApifyClient } from 'apify-client'; + +const client = new ApifyClient({ + token: 'MY-APIFY-TOKEN', + timeoutShortSecs: 10, + timeoutMediumSecs: 60, + timeoutLongSecs: 600, + timeoutMaxSecs: 600, +}); +``` + +A single request queue client can be held to a shorter budget with `client.requestQueue(id, { timeoutSecs })`, which caps the default tier of every request that client sends. A per-call `timeoutSecs` is not capped by it. + +## Per-call overrides + +Every method that sends a request accepts a `timeoutSecs` option, which replaces the tier of the method for that call. Pass a number of seconds for an exact duration, a tier name to switch tiers, or `'noTimeout'` to let the request run for as long as it takes. For the type, see `TimeoutOptions`. + +```js +const datasetClient = client.dataset('my-dataset-id'); + +// An exact timeout for this call. +const { items } = await datasetClient.listItems({ timeoutSecs: 120 }); + +// Another tier. +const dataset = await datasetClient.get({ timeoutSecs: 'long' }); + +// No timeout at all. +await datasetClient.pushItems(items, { timeoutSecs: 'noTimeout' }); +``` + +A number above `timeoutMaxSecs` is capped at it, and the client logs a warning. To let such a call use its full timeout, raise `timeoutMaxSecs` in the client constructor. + +Methods that start an Actor run keep the API's run timeout apart from the request timeout. The `runTimeoutSecs` option of `start()`, `call()` and `resurrect()` bounds how long the run may execute on the platform, while `timeoutSecs` bounds the request that starts it. + +## Interaction with retries + +Timeouts work together with [retries](./02_error-handling.md#retries-with-exponential-backoff). A request that times out counts as a failed attempt and is retried, up to `maxRetries` times. The timeout applies to each attempt on its own, and doubles with every retry up to `timeoutMaxSecs`, so a request that timed out at 5 seconds gets 10 seconds on the second attempt and 20 on the third. diff --git a/docs/04_upgrading/upgrading_v3.md b/docs/04_upgrading/upgrading_v3.md index f4ed5982b..fd68e85f8 100644 --- a/docs/04_upgrading/upgrading_v3.md +++ b/docs/04_upgrading/upgrading_v3.md @@ -70,8 +70,8 @@ await client.actor('my-actor').update([{ name: 'my-actor' }]); `ow` only checked the type, so `Infinity` passed as a number and an invalid `Date` passed as a date. Zod additionally requires a *finite* number and a *valid* date, so both now throw: ```js -// Now throws: Invalid input: expected a finite number at `timeout`, got `Infinity` -await client.actor('my-actor').call(undefined, { timeout: Infinity }); +// Now throws: Invalid input: expected a finite number at `runTimeoutSecs`, got `Infinity` +await client.actor('my-actor').call(undefined, { runTimeoutSecs: Infinity }); // Now throws: Invalid input: expected a valid date at `startedBefore` // Invalid input: expected string, received Date at `startedBefore` @@ -80,7 +80,7 @@ await client.actor('my-actor').runs().list({ startedBefore: new Date('nonsense') The second example reports a line per arm, because `startedBefore` accepts either a `Date` or a string. -This affects numeric options such as `waitSecs`, `timeout` and `memory`, and date options such as `startedBefore` / `startedAfter`. `KeyValueStoreClient.setRecord()` rejects `NaN` and `Infinity` as a record value too, since `JSON.stringify()` turns both into `null`. +This affects numeric options such as `waitSecs`, `runTimeoutSecs` and `memory`, and date options such as `startedBefore` / `startedAfter`. `KeyValueStoreClient.setRecord()` rejects `NaN` and `Infinity` as a record value too, since `JSON.stringify()` turns both into `null`. ### A few always-rejected options are gone from the types @@ -268,9 +268,45 @@ A trailing slash appears only on a field the API returns without a path, so on ` A URL field whose value isn't a valid absolute URL now fails response validation and throws `ResponseValidationError`, the same as any other field that doesn't match the specification. -## `versions().list()` and `envVars().list()` take no options +## Timeouts are configured per tier -`ActorVersionCollectionClient.list()` and `ActorEnvVarCollectionClient.list()` now take no arguments. Neither endpoint reads `offset`, `limit` or `desc`, and both return every item in one response, so `chunkSize` had nothing to size either. The `ActorVersionCollectionListOptions` and `ActorEnvVarCollectionListOptions` types that declared those four options, deprecated since v2.21.0, are gone from the package. A call that passed an options object no longer compiles. Drop the argument and the call returns the same items as before. +The `timeoutSecs` option of the `ApifyClient` constructor is gone. Every method takes its timeout from one of three tiers, and the constructor sets the duration of each tier, along with a cap on any single request attempt: + +```diff +- const client = new ApifyClient({ token: 'MY-APIFY-TOKEN', timeoutSecs: 360 }); // v2 ++ const client = new ApifyClient({ // v3 ++ token: 'MY-APIFY-TOKEN', ++ timeoutShortSecs: 5, ++ timeoutMediumSecs: 30, ++ timeoutLongSecs: 360, ++ timeoutMaxSecs: 360, ++ }); +``` + +For what each tier covers and which one a method is assigned, see [Timeouts](../02_concepts/06_timeouts.md). + +In v2 only the dataset, key-value store and request queue clients picked a timeout per method, and everything else ran with the global 360 seconds. A metadata call such as `actor.get()` now gets the 5 seconds of the `short` tier and a `list()` call the 30 of `medium`, so a call that used to wait out a slow API can fail sooner. `RequestQueueClient.unlockRequests()` moves the other way, from 30 seconds to 360. Where you need a different duration, every method that sends a request takes a `timeoutSecs` option, which replaces the tier of the method for that call: + +```js +await client.actor('my-actor').get({ timeoutSecs: 'long' }); +``` + +`timeoutSecs` means the request timeout on every method, so the run timeout of `ActorClient.start()` and `call()`, `TaskClient.start()` and `call()`, and `RunClient.resurrect()` is renamed: + +```diff +- await client.actor('my-actor').call(input, { timeout: 300 }); // v2 ++ await client.actor('my-actor').call(input, { runTimeoutSecs: 300 }); // v3 +``` + +TypeScript reports a leftover `timeout` at compile time, and in JavaScript the option schemas reject it as an unrecognized key, so the call throws an `ArgumentValidationError` instead of silently dropping the run limit. `{ timeout: 0 }`, which asked for an unlimited run in v2, becomes `{ runTimeoutSecs: 0 }`. + +The `timeoutSecs` option of `KeyValueStoreClient.setRecord()` keeps its name and its meaning for a number of seconds, and now takes a tier name or `'noTimeout'` as well. The same goes for `client.requestQueue(id, { timeoutSecs })`, which caps the tier of every request that queue client sends rather than timing each one at the given value. + +`client.httpClient.call()` takes `timeoutSecs` where it read the axios `timeout` in milliseconds. A direct call that passed `timeout: 30000` has to become `timeoutSecs: 30`. + +## `versions().list()` and `envVars().list()` lose their pagination options + +`ActorVersionCollectionClient.list()` and `ActorEnvVarCollectionClient.list()` accept only the `timeoutSecs` option. Neither endpoint reads `offset`, `limit` or `desc`, and both return every item in one response, so `chunkSize` had nothing to size either. The `ActorVersionCollectionListOptions` and `ActorEnvVarCollectionListOptions` types that declared those four options, deprecated since v2.21.0, are gone from the package. A call that passed any of them no longer compiles, and throws an `ArgumentValidationError` about an unrecognized key in JavaScript. Drop them and the call returns the same items as before. ## `limit` counts scanned rows when iterating `listItems()` diff --git a/docs/public-api/apify-client.api.md b/docs/public-api/apify-client.api.md index 5f9a7b356..e47be349d 100644 --- a/docs/public-api/apify-client.api.md +++ b/docs/public-api/apify-client.api.md @@ -52,7 +52,7 @@ export interface Actor extends Omit, Act } // @public -export interface ActorBuildOptions { +export interface ActorBuildOptions extends TimeoutOptions { // (undocumented) betaPackages?: boolean; // (undocumented) @@ -90,12 +90,12 @@ export class ActorClient extends ResourceClient { builds(): BuildCollectionClient; call(input?: ActorInput, options?: ActorCallOptions): Promise; defaultBuild(options?: BuildClientGetOptions): Promise; - delete(): Promise; - get(): Promise; + delete(options?: TimeoutOptions): Promise; + get(options?: TimeoutOptions): Promise; lastRun(options?: ActorLastRunOptions): RunClient; runs(): RunCollectionClient; start(input?: ActorInput, options?: ActorStartOptions): Promise; - update(newFields: ActorUpdateOptions): Promise; + update(newFields: ActorUpdateOptions, options?: TimeoutOptions): Promise; validateInput(input?: ActorInput, options?: ActorValidateInputOptions): Promise; version(versionNumber: string): ActorVersionClient; versions(): ActorVersionCollectionClient; @@ -105,7 +105,7 @@ export class ActorClient extends ResourceClient { // @public export class ActorCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - create(actor: ActorCollectionCreateOptions): Promise; + create(actor: ActorCollectionCreateOptions, options?: TimeoutOptions): Promise; list(options?: ActorCollectionListOptions): PaginatedIterator; } @@ -145,7 +145,7 @@ interface ActorCollectionListItemRePointed { } // @public (undocumented) -export interface ActorCollectionListOptions extends PaginationOptions { +export interface ActorCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; // (undocumented) @@ -184,16 +184,16 @@ export interface ActorEnvironmentVariable extends GeneratedEnvVar { // @public export class ActorEnvVarClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); - delete(): Promise; - get(): Promise; - update(actorEnvVar: ActorEnvironmentVariable): Promise; + delete(options?: TimeoutOptions): Promise; + get(options?: TimeoutOptions): Promise; + update(actorEnvVar: ActorEnvironmentVariable, options?: TimeoutOptions): Promise; } // @public export class ActorEnvVarCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - create(actorEnvVar: ActorEnvironmentVariable): Promise; - list(): Promise & AsyncIterable; + create(actorEnvVar: ActorEnvironmentVariable, options?: TimeoutOptions): Promise; + list(options?: TimeoutOptions): Promise & AsyncIterable; } // @public @@ -355,7 +355,7 @@ export interface ActorStandby extends GeneratedActorStandby { } // @public (undocumented) -export interface ActorStartOptions { +export interface ActorStartOptions extends TimeoutOptions { build?: string; contentType?: string; forcePermissionLevel?: ACTOR_PERMISSION_LEVEL; @@ -363,7 +363,7 @@ export interface ActorStartOptions { maxTotalChargeUsd?: number; memory?: number; restartOnError?: boolean; - timeout?: number; + runTimeoutSecs?: number; waitForFinish?: number; webhooks?: readonly WebhookUpdateData[]; } @@ -396,7 +396,7 @@ export type ActorTaggedBuilds = Record; export type ActorUpdateOptions = Partial>; // @public -export interface ActorValidateInputOptions { +export interface ActorValidateInputOptions extends TimeoutOptions { build?: string; contentType?: string; } @@ -407,11 +407,11 @@ export type ActorVersion = ActorVersionSourceFiles | ActorVersionGitRepo | Actor // @public export class ActorVersionClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); - delete(): Promise; + delete(options?: TimeoutOptions): Promise; envVar(envVarName: string): ActorEnvVarClient; envVars(): ActorEnvVarCollectionClient; - get(): Promise; - update(newFields: ActorVersionUpdateData): Promise; + get(options?: TimeoutOptions): Promise; + update(newFields: ActorVersionUpdateData, options?: TimeoutOptions): Promise; } // Not exported by the entry point; reachable only as a referenced type. @@ -424,8 +424,8 @@ interface ActorVersionClientNarrowings { // @public export class ActorVersionCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - create(actorVersion: ActorVersion): Promise; - list(): Promise & AsyncIterable; + create(actorVersion: ActorVersion, options?: TimeoutOptions): Promise; + list(options?: TimeoutOptions): Promise & AsyncIterable; } // @public @@ -493,29 +493,29 @@ abstract class ApiClient { // (undocumented) baseUrl: string; // (undocumented) + protected buildParams(endpointParams?: T): Record; + // (undocumented) + protected buildPublicUrl(path?: string | string[]): string; + // (undocumented) + protected buildUrl(path?: string | string[]): string; + // (undocumented) httpClient: HttpClient; // (undocumented) id?: string; - protected _listPaginatedFromCallback>(getPaginatedList: (options?: T) => Promise, options?: T): AsyncIterable & Promise; + protected listPaginatedFromCallback>(getPaginatedList: (options?: T) => Promise, options?: T): AsyncIterable & Promise; // (undocumented) params?: Record; - // (undocumented) - protected _params(endpointParams?: T): Record; publicBaseUrl: string; // (undocumented) - protected _publicUrl(path?: string | string[]): string; - // (undocumented) resourcePath: string; // (undocumented) safeId?: string; // (undocumented) - protected _subResourceOptions(moreOptions?: T): BaseOptions & T; + protected subResourceOptions(moreOptions?: T): BaseOptions & T; // (undocumented) - protected _toSafeId(id: string): string; + protected toSafeId(id: string): string; // (undocumented) url: string; - // (undocumented) - protected _url(path?: string | string[]): string; } // Not exported by the entry point; reachable only as a referenced type. @@ -615,7 +615,10 @@ export interface ApifyClientOptions { minDelayBetweenRetriesMillis?: number; publicBaseUrl?: string; requestInterceptors?: RequestInterceptorFunction[]; - timeoutSecs?: number; + timeoutLongSecs?: number; + timeoutMaxSecs?: number; + timeoutMediumSecs?: number; + timeoutShortSecs?: number; // (undocumented) token?: string; userAgentSuffix?: string | string[]; @@ -623,13 +626,14 @@ export interface ApifyClientOptions { // Not exported by the entry point; reachable only as a referenced type. // @public (undocumented) -interface ApifyRequestConfig extends AxiosRequestConfig { +interface ApifyRequestConfig extends Omit { // (undocumented) doNotRetryTimeouts?: boolean; // (undocumented) forceBuffer?: boolean; // (undocumented) stringifyFunctions?: boolean; + timeoutSecs?: Timeout; } // Not exported by the entry point; reachable only as a referenced type. @@ -669,22 +673,22 @@ export interface Build extends Omit, Bui // @public export class BuildClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); - abort(): Promise; - delete(): Promise; + abort(options?: TimeoutOptions): Promise; + delete(options?: TimeoutOptions): Promise; get(options?: BuildClientGetOptions): Promise; - getOpenApiDefinition(): Promise; + getOpenApiDefinition(options?: TimeoutOptions): Promise; log(): LogClient; waitForFinish(options?: BuildClientWaitForFinishOptions): Promise; } // @public -export interface BuildClientGetOptions { +export interface BuildClientGetOptions extends TimeoutOptions { // (undocumented) waitForFinish?: number; } // @public -export interface BuildClientWaitForFinishOptions { +export interface BuildClientWaitForFinishOptions extends TimeoutOptions { waitSecs?: number; } @@ -708,7 +712,7 @@ interface BuildCollectionClientListItemRePointed { } // @public (undocumented) -export interface BuildCollectionClientListOptions extends PaginationOptions { +export interface BuildCollectionClientListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; } @@ -2421,13 +2425,13 @@ export interface Dataset extends Omit = Record> extends ResourceClient { constructor(options: ApiClientSubResourceOptions); createItemsPublicUrl(options?: DatasetClientCreateItemsUrlOptions): Promise; - delete(): Promise; + delete(options?: TimeoutOptions): Promise; downloadItems(format: `${DownloadItemsFormat}`, options?: DatasetClientDownloadItemsOptions): Promise; - get(): Promise; - getStatistics(): Promise; + get(options?: TimeoutOptions): Promise; + getStatistics(options?: TimeoutOptions): Promise; listItems(options?: DatasetClientListItemOptions): PaginatedIterator; - pushItems(items: Data | Data[] | string | string[]): Promise; - update(newFields: DatasetClientUpdateOptions): Promise; + pushItems(items: Data | Data[] | string | string[], options?: TimeoutOptions): Promise; + update(newFields: DatasetClientUpdateOptions, options?: TimeoutOptions): Promise; } // @public @@ -2453,7 +2457,7 @@ export interface DatasetClientDownloadItemsOptions extends Omit; } // @public (undocumented) -export interface DatasetCollectionClientListOptions extends PaginationOptions { +export interface DatasetCollectionClientListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; ownership?: STORAGE_OWNERSHIP_FILTER; @@ -2838,7 +2842,7 @@ type GeneratedWebhookDispatchEventData = NonNullable; // (undocumented) userProvidedRequestInterceptors: RequestInterceptorFunction[]; // (undocumented) @@ -2887,7 +2891,13 @@ interface HttpClientOptions { // (undocumented) requestInterceptors: RequestInterceptorFunction[]; // (undocumented) - timeoutSecs: number; + timeoutLongSecs: number; + // (undocumented) + timeoutMaxSecs: number; + // (undocumented) + timeoutMediumSecs: number; + // (undocumented) + timeoutShortSecs: number; // (undocumented) token?: string; // (undocumented) @@ -2916,7 +2926,7 @@ export interface KeyValueClientCreateKeysUrlOptions extends Omit; - delete(): Promise; - deleteRecord(key: string): Promise; - get(): Promise; + delete(options?: TimeoutOptions): Promise; + deleteRecord(key: string, options?: TimeoutOptions): Promise; + get(options?: TimeoutOptions): Promise; getRecord(key: string): Promise | undefined>; // (undocumented) getRecord(key: string, options: Options): Promise> | undefined>; - getRecordPublicUrl(key: string): Promise; + getRecordPublicUrl(key: string, options?: TimeoutOptions): Promise; listKeys(options?: KeyValueClientListKeysOptions): Promise & AsyncIterable; - recordExists(key: string): Promise; + recordExists(key: string, options?: TimeoutOptions): Promise; setRecord(record: KeyValueStoreRecord, options?: KeyValueStoreRecordOptions): Promise; - update(newFields: KeyValueClientUpdateOptions): Promise; + update(newFields: KeyValueClientUpdateOptions, options?: TimeoutOptions): Promise; } // @public @@ -2988,13 +2998,13 @@ export class KeyValueStoreCollectionClient extends ResourceCollectionClient { } // @public -export interface KeyValueStoreCollectionClientGetOrCreateOptions { +export interface KeyValueStoreCollectionClientGetOrCreateOptions extends TimeoutOptions { // (undocumented) schema?: Record; } // @public (undocumented) -export interface KeyValueStoreCollectionClientListOptions extends PaginationOptions { +export interface KeyValueStoreCollectionClientListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; ownership?: STORAGE_OWNERSHIP_FILTER; @@ -3018,11 +3028,9 @@ export interface KeyValueStoreRecord { } // @public -export interface KeyValueStoreRecordOptions { +export interface KeyValueStoreRecordOptions extends TimeoutOptions { // (undocumented) doNotRetryTimeouts?: boolean; - // (undocumented) - timeoutSecs?: number; } // @public @@ -3074,7 +3082,7 @@ export class LoggerActorRedirect extends Logger { } // @public -export interface LogOptions { +export interface LogOptions extends TimeoutOptions { raw?: boolean; } @@ -3274,28 +3282,28 @@ export interface RequestQueue extends Omit; - batchAddRequests(requests: RequestQueueClientRequestToAdd[], options?: RequestQueueClientBatchAddRequestWithRetriesOptions): Promise; - protected _batchAddRequests(requests: RequestQueueClientRequestToAdd[], options?: RequestQueueClientAddRequestOptions): Promise; + protected addRequestBatch(requests: RequestQueueClientRequestToAdd[], options?: RequestQueueClientAddRequestOptions): Promise; // (undocumented) - protected _batchAddRequestsWithRetries(requests: RequestQueueClientRequestToAdd[], options?: RequestQueueClientBatchAddRequestWithRetriesOptions): Promise; - batchDeleteRequests(requests: RequestQueueClientRequestToDelete[]): Promise; - delete(): Promise; - deleteRequest(id: string): Promise; + protected addRequestBatchWithRetries(requests: RequestQueueClientRequestToAdd[], options?: RequestQueueClientBatchAddRequestWithRetriesOptions): Promise; + batchAddRequests(requests: RequestQueueClientRequestToAdd[], options?: RequestQueueClientBatchAddRequestWithRetriesOptions): Promise; + batchDeleteRequests(requests: RequestQueueClientRequestToDelete[], options?: TimeoutOptions): Promise; + delete(options?: TimeoutOptions): Promise; + deleteRequest(id: string, options?: TimeoutOptions): Promise; deleteRequestLock(id: string, options?: RequestQueueClientDeleteRequestLockOptions): Promise; - get(): Promise; - getRequest(id: string): Promise; + get(options?: TimeoutOptions): Promise; + getRequest(id: string, options?: TimeoutOptions): Promise; listAndLockHead(options: RequestQueueClientListAndLockHeadOptions): Promise; listHead(options?: RequestQueueClientListHeadOptions): Promise; listRequests(options?: RequestQueueClientListRequestsOptions): Promise & AsyncIterable; paginateRequests(options?: RequestQueueClientPaginateRequestsOptions): RequestQueueRequestsAsyncIterable; prolongRequestLock(id: string, options: RequestQueueClientProlongRequestLockOptions): Promise; - unlockRequests(): Promise; - update(newFields: RequestQueueClientUpdateOptions): Promise; + unlockRequests(options?: TimeoutOptions): Promise; + update(newFields: RequestQueueClientUpdateOptions, options?: TimeoutOptions): Promise; updateRequest(request: RequestQueueClientRequestToUpdate, options?: RequestQueueClientAddRequestOptions): Promise; } // @public (undocumented) -export interface RequestQueueClientAddRequestOptions { +export interface RequestQueueClientAddRequestOptions extends TimeoutOptions { // (undocumented) forefront?: boolean; } @@ -3305,7 +3313,7 @@ export interface RequestQueueClientAddRequestResult extends GeneratedRequestRegi } // @public -export interface RequestQueueClientBatchAddRequestWithRetriesOptions { +export interface RequestQueueClientBatchAddRequestWithRetriesOptions extends TimeoutOptions { // (undocumented) forefront?: boolean; // (undocumented) @@ -3324,7 +3332,7 @@ export interface RequestQueueClientBatchRequestsOperationResult extends Generate } // @public -export interface RequestQueueClientDeleteRequestLockOptions { +export interface RequestQueueClientDeleteRequestLockOptions extends TimeoutOptions { // (undocumented) forefront?: boolean; } @@ -3333,7 +3341,7 @@ export interface RequestQueueClientDeleteRequestLockOptions { export type RequestQueueClientGetRequestResult = RequestQueueClientRequestSchema; // @public -export interface RequestQueueClientListAndLockHeadOptions { +export interface RequestQueueClientListAndLockHeadOptions extends TimeoutOptions { // (undocumented) limit?: number; // (undocumented) @@ -3352,7 +3360,7 @@ interface RequestQueueClientListAndLockHeadResultRePointed { } // @public -export interface RequestQueueClientListHeadOptions { +export interface RequestQueueClientListHeadOptions extends TimeoutOptions { // (undocumented) limit?: number; } @@ -3373,7 +3381,7 @@ export interface RequestQueueClientListItem extends GeneratedHeadRequest { } // @public -export interface RequestQueueClientListRequestsOptions { +export interface RequestQueueClientListRequestsOptions extends TimeoutOptions { cursor?: string; filter?: readonly RequestQueueListRequestsFilter[]; // (undocumented) @@ -3396,7 +3404,7 @@ export interface RequestQueueClientLockedListItem extends GeneratedLockedHeadReq } // @public -export interface RequestQueueClientPaginateRequestsOptions { +export interface RequestQueueClientPaginateRequestsOptions extends TimeoutOptions { cursor?: string; filter?: readonly RequestQueueListRequestsFilter[]; // (undocumented) @@ -3406,7 +3414,7 @@ export interface RequestQueueClientPaginateRequestsOptions { } // @public -export interface RequestQueueClientProlongRequestLockOptions { +export interface RequestQueueClientProlongRequestLockOptions extends TimeoutOptions { // (undocumented) forefront?: boolean; // (undocumented) @@ -3447,12 +3455,12 @@ export interface RequestQueueClientUpdateOptions { // @public export class RequestQueueCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - getOrCreate(name?: string): Promise; + getOrCreate(name?: string, options?: TimeoutOptions): Promise; list(options?: RequestQueueCollectionListOptions): Promise & AsyncIterable; } // @public (undocumented) -export interface RequestQueueCollectionListOptions extends PaginationOptions { +export interface RequestQueueCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; ownership?: STORAGE_OWNERSHIP_FILTER; @@ -3512,11 +3520,12 @@ export interface RequestQueueUserOptions { // Not exported by the entry point; reachable only as a referenced type. // @public class ResourceClient extends ApiClient { - protected _delete(timeoutMillis?: number): Promise; - protected _get(schema: z.ZodType, options?: T, timeoutMillis?: number): Promise; + protected deleteResource(timeoutSecs: Timeout): Promise; + protected getResource(schema: z.ZodType, options: T, timeoutSecs: Timeout): Promise; + protected timeoutForWaitForFinish(timeoutSecs: Timeout | undefined, tier: TimeoutTier, waitForFinishSecs: number | undefined): Timeout; // (undocumented) - protected _update(schema: z.ZodType, newFields: T, timeoutMillis?: number): Promise; - protected _waitForFinish(schema: z.ZodType, newFields: T, timeoutSecs: Timeout): Promise; + protected waitForJobFinish(schema: z.ZodType, options?: WaitForFinishOptions): Promise; } @@ -3525,12 +3534,12 @@ class ResourceClient extends ApiClient { // @public class ResourceCollectionClient extends ApiClient { // (undocumented) - protected _create(schema: z.ZodType, resource: D): Promise; + protected createResource(schema: z.ZodType, resource: D, timeoutSecs: Timeout): Promise; // (undocumented) - protected _getOrCreate(schema: z.ZodType, name?: string, resource?: D): Promise; + protected getOrCreateResource(schema: z.ZodType, name: string | undefined, resource: D | undefined, timeoutSecs: Timeout): Promise; // (undocumented) - protected _list(schema: z.ZodType, options?: T): Promise; - protected _listPaginated>(schema: z.ZodType, options?: T): AsyncIterable & Promise; + protected listResources(schema: z.ZodType, options: T | undefined, timeoutSecs: Timeout): Promise; + protected listResourcesPaginated>(schema: z.ZodType, options: T, defaultTimeoutSecs: Timeout): AsyncIterable & Promise; } // @public @@ -3548,13 +3557,13 @@ export class ResponseValidationError extends Error { export type ReturnTypeFromOptions = Options['stream'] extends true ? Readable : Options['buffer'] extends true ? Buffer : JsonValue; // @public -export interface RunAbortOptions { +export interface RunAbortOptions extends TimeoutOptions { // (undocumented) gracefully?: boolean; } // @public -export interface RunChargeOptions { +export interface RunChargeOptions extends TimeoutOptions { count?: number; eventName: string; idempotencyKey?: string; @@ -3566,16 +3575,16 @@ export class RunClient extends ResourceClient { abort(options?: RunAbortOptions): Promise; charge(options: RunChargeOptions): Promise>>; dataset(): DatasetClient; - delete(): Promise; + delete(options?: TimeoutOptions): Promise; get(options?: RunGetOptions): Promise; getStreamedLog(options?: GetStreamedLogOptions): Promise; keyValueStore(): KeyValueStoreClient; log(): LogClient; metamorph(targetActorId: string, input?: ActorInput, options?: RunMetamorphOptions): Promise; - reboot(): Promise; + reboot(options?: TimeoutOptions): Promise; requestQueue(): RequestQueueClient; resurrect(options?: RunResurrectOptions): Promise; - update(newFields: RunUpdateOptions): Promise; + update(newFields: RunUpdateOptions, options?: TimeoutOptions): Promise; waitForFinish(options?: RunWaitForFinishOptions): Promise; } @@ -3586,7 +3595,7 @@ export class RunCollectionClient extends ResourceCollectionClient { } // @public (undocumented) -export interface RunCollectionListOptions extends PaginationOptions { +export interface RunCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; startedAfter?: Date | string; @@ -3596,20 +3605,20 @@ export interface RunCollectionListOptions extends PaginationOptions { } // @public -export interface RunGetOptions { +export interface RunGetOptions extends TimeoutOptions { // (undocumented) waitForFinish?: number; } // @public -export interface RunMetamorphOptions { +export interface RunMetamorphOptions extends TimeoutOptions { // (undocumented) build?: string; contentType?: string; } // @public -export interface RunResurrectOptions { +export interface RunResurrectOptions extends TimeoutOptions { // (undocumented) build?: string; maxItems?: number; @@ -3617,8 +3626,7 @@ export interface RunResurrectOptions { // (undocumented) memory?: number; restartOnError?: boolean; - // (undocumented) - timeout?: number; + runTimeoutSecs?: number; } // @public @@ -3630,7 +3638,7 @@ export interface RunUpdateOptions { } // @public -export interface RunWaitForFinishOptions { +export interface RunWaitForFinishOptions extends TimeoutOptions { waitSecs?: number; } @@ -3678,10 +3686,10 @@ export enum ScheduleActions { // @public export class ScheduleClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); - delete(): Promise; - get(): Promise; - getLog(): Promise; - update(newFields: ScheduleCreateOrUpdateData): Promise; + delete(options?: TimeoutOptions): Promise; + get(options?: TimeoutOptions): Promise; + getLog(options?: TimeoutOptions): Promise; + update(newFields: ScheduleCreateOrUpdateData, options?: TimeoutOptions): Promise; } // Not exported by the entry point; reachable only as a referenced type. @@ -3694,12 +3702,12 @@ interface ScheduleClientNarrowings { // @public export class ScheduleCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - create(schedule?: ScheduleCreateOrUpdateData): Promise; + create(schedule?: ScheduleCreateOrUpdateData, options?: TimeoutOptions): Promise; list(options?: ScheduleCollectionListOptions): PaginatedIterator; } // @public (undocumented) -export interface ScheduleCollectionListOptions extends PaginationOptions { +export interface ScheduleCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; } @@ -3759,7 +3767,7 @@ export class StoreCollectionClient extends ResourceCollectionClient { } // @public -export interface StoreCollectionListOptions extends PaginationOptions { +export interface StoreCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) category?: string; includeUnrunnableActors?: boolean; @@ -3801,28 +3809,28 @@ export interface TaskCallOptions extends Omit export class TaskClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); call(input?: Dictionary, options?: TaskCallOptions): Promise; - delete(): Promise; - get(): Promise; - getInput(): Promise; + delete(options?: TimeoutOptions): Promise; + get(options?: TimeoutOptions): Promise; + getInput(options?: TimeoutOptions): Promise; lastRun(options?: TaskLastRunOptions): RunClient; - publish(): Promise; + publish(options?: TimeoutOptions): Promise; runs(): RunCollectionClient; start(input?: Dictionary, options?: TaskStartOptions): Promise; - unpublish(): Promise; - update(newFields: TaskUpdateData): Promise; - updateInput(newFields: Dictionary | Dictionary[]): Promise; + unpublish(options?: TimeoutOptions): Promise; + update(newFields: TaskUpdateData, options?: TimeoutOptions): Promise; + updateInput(newFields: Dictionary | Dictionary[], options?: TimeoutOptions): Promise; webhooks(): WebhookCollectionClient; } // @public export class TaskCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - create(task: TaskCreateData): Promise; + create(task: TaskCreateData, options?: TimeoutOptions): Promise; list(options?: TaskCollectionListOptions): PaginatedIterator; } // @public (undocumented) -export interface TaskCollectionListOptions extends PaginationOptions { +export interface TaskCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; } @@ -3917,6 +3925,17 @@ export interface TieredPricingPerEvent { export interface TieredPricingPerEventEntry extends GeneratedTieredPricingPerEventEntry { } +// @public +export type Timeout = TimeoutTier | 'noTimeout' | number; + +// @public +export interface TimeoutOptions { + timeoutSecs?: Timeout; +} + +// @public +export type TimeoutTier = 'short' | 'medium' | 'long'; + // Not exported by the entry point; reachable only as a referenced type. // @public (undocumented) type Timezone = (typeof timezones)[number]; @@ -3951,10 +3970,10 @@ export interface User extends Omit; - limits(): Promise; - monthlyUsage(): Promise; - updateLimits(options: LimitsUpdateOptions): Promise; + get(options?: TimeoutOptions): Promise; + limits(options?: TimeoutOptions): Promise; + monthlyUsage(options?: TimeoutOptions): Promise; + updateLimits(newLimits: LimitsUpdateOptions, options?: TimeoutOptions): Promise; } // @public @@ -3998,7 +4017,7 @@ interface UserRePointed { // Not exported by the entry point; reachable only as a referenced type. // @public (undocumented) -interface WaitForFinishOptions { +interface WaitForFinishOptions extends TimeoutOptions { // (undocumented) waitSecs?: number; } @@ -4028,22 +4047,22 @@ export interface WebhookCertainRunCondition { // @public export class WebhookClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); - delete(): Promise; + delete(options?: TimeoutOptions): Promise; dispatches(): WebhookDispatchCollectionClient; - get(): Promise; - test(): Promise; - update(newFields: WebhookUpdateData): Promise; + get(options?: TimeoutOptions): Promise; + test(options?: TimeoutOptions): Promise; + update(newFields: WebhookUpdateData, options?: TimeoutOptions): Promise; } // @public export class WebhookCollectionClient extends ResourceCollectionClient { constructor(options: ApiClientSubResourceOptions); - create(webhook?: WebhookUpdateData): Promise; + create(webhook?: WebhookUpdateData, options?: TimeoutOptions): Promise; list(options?: WebhookCollectionListOptions): PaginatedIterator>; } // @public (undocumented) -export interface WebhookCollectionListOptions extends PaginationOptions { +export interface WebhookCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; } @@ -4062,7 +4081,7 @@ export interface WebhookDispatchCall extends GeneratedWebhookDispatchCall { // @public export class WebhookDispatchClient extends ResourceClient { constructor(options: ApiClientSubResourceOptions); - get(): Promise; + get(options?: TimeoutOptions): Promise; } // @public @@ -4072,7 +4091,7 @@ export class WebhookDispatchCollectionClient extends ResourceCollectionClient { } // @public (undocumented) -export interface WebhookDispatchCollectionListOptions extends PaginationOptions { +export interface WebhookDispatchCollectionListOptions extends PaginationOptions, TimeoutOptions { // (undocumented) desc?: boolean; } diff --git a/src/apify_api_error.ts b/src/apify_api_error.ts index e51c021ff..145717faa 100644 --- a/src/apify_api_error.ts +++ b/src/apify_api_error.ts @@ -8,15 +8,32 @@ import { isBuffer } from './utils.js'; export type { ApifyApiErrorType } from './models.js'; /** - * Examples of capturing groups for "...at ActorCollectionClient._list (/Users/..." - * 0: "at ActorCollectionClient._list (" + * Examples of capturing groups for "...at ActorCollectionClient.listResources (/Users/..." + * 0: "at ActorCollectionClient.listResources (" * 1: undefined * 2: "ActorCollectionClient" * 3: undefined - * 4: "list" + * 4: "listResources" * @private */ -const CLIENT_METHOD_REGEX = /at( async)? ([A-Za-z]+(Collection)?Client)\._?([A-Za-z]+) \(/; +const CLIENT_METHOD_REGEX = /at( async)? ([A-Za-z]+(Collection)?Client)\.([A-Za-z]+) \(/; + +/** + * A public method that returns the promise of a shared helper without awaiting it leaves only the helper on + * the stack, so the helper is reported under the method the caller invoked. + * @private + */ +const PUBLIC_METHOD_BY_HELPER: Record = { + getResource: 'get', + updateResource: 'update', + deleteResource: 'delete', + waitForJobFinish: 'waitForFinish', + listResources: 'list', + createResource: 'create', + getOrCreateResource: 'getOrCreate', + addRequestBatch: 'batchAddRequests', + addRequestBatchWithRetries: 'batchAddRequests', +}; /** * An `ApifyApiError` is thrown for successful HTTP requests that reach the API, @@ -152,8 +169,8 @@ export class ApifyApiError extends Error { private _extractClientAndMethodFromStack() { const match = this.stack!.match(CLIENT_METHOD_REGEX); - if (match) return `${match[2]}.${match[4]}`; - return 'unknown'; + if (!match) return 'unknown'; + return `${match[2]}.${PUBLIC_METHOD_BY_HELPER[match[4]] ?? match[4]}`; } /** diff --git a/src/apify_client.ts b/src/apify_client.ts index b9536bbaa..9b3240242 100644 --- a/src/apify_client.ts +++ b/src/apify_client.ts @@ -32,24 +32,31 @@ import { WebhookCollectionClient } from './resource_clients/webhook_collection.j import { WebhookDispatchClient } from './resource_clients/webhook_dispatch.js'; import { WebhookDispatchCollectionClient } from './resource_clients/webhook_dispatch_collection.js'; import { Statistics } from './statistics.js'; +import { + DEFAULT_TIMEOUT_LONG_SECS, + DEFAULT_TIMEOUT_MAX_SECS, + DEFAULT_TIMEOUT_MEDIUM_SECS, + DEFAULT_TIMEOUT_SHORT_SECS, +} from './timeouts.js'; import { parseArgument } from './utils.js'; -const DEFAULT_TIMEOUT_SECS = 360; - const clientOptionsSchema = z.strictObject({ baseUrl: z.string().default('https://api.apify.com'), publicBaseUrl: z.string().default('https://api.apify.com'), maxRetries: z.number().default(8), minDelayBetweenRetriesMillis: z.number().default(500), requestInterceptors: z.array(z.unknown()).default([]), - timeoutSecs: z.number().default(DEFAULT_TIMEOUT_SECS), + timeoutShortSecs: z.number().positive().default(DEFAULT_TIMEOUT_SHORT_SECS), + timeoutMediumSecs: z.number().positive().default(DEFAULT_TIMEOUT_MEDIUM_SECS), + timeoutLongSecs: z.number().positive().default(DEFAULT_TIMEOUT_LONG_SECS), + timeoutMaxSecs: z.number().positive().default(DEFAULT_TIMEOUT_MAX_SECS), token: z.string().optional(), userAgentSuffix: z.union([z.string(), z.array(z.string())]).optional(), }); const resourceIdSchema = z.string().min(1); const requestQueueOptionsSchema = z.strictObject({ clientKey: z.string().min(1).optional(), - timeoutSecs: z.number().optional(), + timeoutSecs: z.number().positive().optional(), }); /** @@ -98,7 +105,10 @@ export class ApifyClient { maxRetries, minDelayBetweenRetriesMillis, requestInterceptors, - timeoutSecs, + timeoutShortSecs, + timeoutMediumSecs, + timeoutLongSecs, + timeoutMaxSecs, token, } = parsed; @@ -116,14 +126,17 @@ export class ApifyClient { maxRetries, minDelayBetweenRetriesMillis, requestInterceptors, - timeoutSecs, + timeoutShortSecs, + timeoutMediumSecs, + timeoutLongSecs, + timeoutMaxSecs, logger: this.logger, token: this.token, userAgentSuffix: parsed.userAgentSuffix, }); } - private _options() { + private subClientOptions() { return { baseUrl: this.baseUrl, publicBaseUrl: this.publicBaseUrl, @@ -141,7 +154,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/acts-get */ actors(): ActorCollectionClient { - return new ActorCollectionClient(this._options()); + return new ActorCollectionClient(this.subClientOptions()); } /** @@ -165,7 +178,7 @@ export class ApifyClient { return new ActorClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -178,7 +191,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/actor-builds-get */ builds(): BuildCollectionClient { - return new BuildCollectionClient(this._options()); + return new BuildCollectionClient(this.subClientOptions()); } /** @@ -195,7 +208,7 @@ export class ApifyClient { return new BuildClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -208,7 +221,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/datasets-get */ datasets(): DatasetCollectionClient { - return new DatasetCollectionClient(this._options()); + return new DatasetCollectionClient(this.subClientOptions()); } /** @@ -241,7 +254,7 @@ export class ApifyClient { return new DatasetClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -254,7 +267,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/key-value-stores-get */ keyValueStores(): KeyValueStoreCollectionClient { - return new KeyValueStoreCollectionClient(this._options()); + return new KeyValueStoreCollectionClient(this.subClientOptions()); } /** @@ -281,7 +294,7 @@ export class ApifyClient { return new KeyValueStoreClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -297,7 +310,7 @@ export class ApifyClient { return new LogClient({ id: buildOrRunId, - ...this._options(), + ...this.subClientOptions(), }); } @@ -310,7 +323,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/request-queues-get */ requestQueues(): RequestQueueCollectionClient { - return new RequestQueueCollectionClient(this._options()); + return new RequestQueueCollectionClient(this.subClientOptions()); } /** @@ -340,7 +353,7 @@ export class ApifyClient { const apiClientOptions = { id, - ...this._options(), + ...this.subClientOptions(), }; return new RequestQueueClient(apiClientOptions, parsed); } @@ -355,7 +368,7 @@ export class ApifyClient { */ runs(): RunCollectionClient { return new RunCollectionClient({ - ...this._options(), + ...this.subClientOptions(), resourcePath: 'actor-runs', }); } @@ -384,7 +397,7 @@ export class ApifyClient { return new RunClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -397,7 +410,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/actor-tasks-get */ tasks(): TaskCollectionClient { - return new TaskCollectionClient(this._options()); + return new TaskCollectionClient(this.subClientOptions()); } /** @@ -420,7 +433,7 @@ export class ApifyClient { return new TaskClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -433,7 +446,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/schedules-get */ schedules(): ScheduleCollectionClient { - return new ScheduleCollectionClient(this._options()); + return new ScheduleCollectionClient(this.subClientOptions()); } /** @@ -450,7 +463,7 @@ export class ApifyClient { return new ScheduleClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -468,7 +481,7 @@ export class ApifyClient { return new UserClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -481,7 +494,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/webhooks-get */ webhooks(): WebhookCollectionClient { - return new WebhookCollectionClient(this._options()); + return new WebhookCollectionClient(this.subClientOptions()); } /** @@ -498,7 +511,7 @@ export class ApifyClient { return new WebhookClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -511,7 +524,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/webhook-dispatches-get */ webhookDispatches(): WebhookDispatchCollectionClient { - return new WebhookDispatchCollectionClient(this._options()); + return new WebhookDispatchCollectionClient(this.subClientOptions()); } /** @@ -526,7 +539,7 @@ export class ApifyClient { return new WebhookDispatchClient({ id, - ...this._options(), + ...this.subClientOptions(), }); } @@ -539,7 +552,7 @@ export class ApifyClient { * @see https://docs.apify.com/api/v2/store-get */ store(): StoreCollectionClient { - return new StoreCollectionClient(this._options()); + return new StoreCollectionClient(this.subClientOptions()); } /** @@ -582,8 +595,27 @@ export interface ApifyClientOptions { minDelayBetweenRetriesMillis?: number; /** @default [] */ requestInterceptors?: RequestInterceptorFunction[]; - /** @default 360 */ - timeoutSecs?: number; + /** + * Duration of the `short` timeout tier, in seconds: simple metadata reads and writes. + * @default 5 + */ + timeoutShortSecs?: number; + /** + * Duration of the `medium` timeout tier, in seconds: listing, batch and trigger operations. + * @default 30 + */ + timeoutMediumSecs?: number; + /** + * Duration of the `long` timeout tier, in seconds: downloads, uploads and streaming. + * @default 360 + */ + timeoutLongSecs?: number; + /** + * Cap on the timeout of a single request attempt, in seconds. It bounds the doubling of the timeout across + * retries, and caps tier and per-call timeouts alike, so raise it to allow a request timeout above 360 s. + * @default 360 + */ + timeoutMaxSecs?: number; token?: string; /** * @since Added in 2.10.0 diff --git a/src/base/api_client.ts b/src/base/api_client.ts index 190e9435b..01331a31e 100644 --- a/src/base/api_client.ts +++ b/src/base/api_client.ts @@ -47,7 +47,7 @@ export abstract class ApiClient { const { baseUrl, publicBaseUrl, apifyClient, httpClient, resourcePath, id, params = {} } = options; this.id = id; - this.safeId = id && this._toSafeId(id); + this.safeId = id && this.toSafeId(id); this.baseUrl = baseUrl; this.publicBaseUrl = publicBaseUrl; this.resourcePath = resourcePath; @@ -57,40 +57,40 @@ export abstract class ApiClient { this.params = params; } - protected _subResourceOptions(moreOptions?: T): BaseOptions & T { + protected subResourceOptions(moreOptions?: T): BaseOptions & T { const baseOptions: BaseOptions = { - baseUrl: this._url(), + baseUrl: this.buildUrl(), publicBaseUrl: this.publicBaseUrl, apifyClient: this.apifyClient, httpClient: this.httpClient, - params: this._params(), + params: this.buildParams(), }; return { ...baseOptions, ...moreOptions } as BaseOptions & T; } - protected _url(path?: string | string[]): string { + protected buildUrl(path?: string | string[]): string { return path ? `${this.url}/${toPath(path)}` : this.url; } - protected _publicUrl(path?: string | string[]): string { + protected buildPublicUrl(path?: string | string[]): string { const url = this.id ? `${this.publicBaseUrl}/${this.resourcePath}/${toPathSegment(this.safeId!)}` : `${this.publicBaseUrl}/${this.resourcePath}`; return path ? `${url}/${toPath(path)}` : url; } - protected _params(endpointParams?: T): Record { + protected buildParams(endpointParams?: T): Record { return { ...this.params, ...endpointParams }; } - protected _toSafeId(id: string): string { + protected toSafeId(id: string): string { return id.replaceAll('/', '~'); } /** * Returns async iterator to iterate through all items and Promise that can be awaited to get first page of results. */ - protected _listPaginatedFromCallback>( + protected listPaginatedFromCallback>( getPaginatedList: (options?: T) => Promise, options: T = {} as T, ): AsyncIterable & Promise { @@ -104,7 +104,8 @@ export abstract class ApiClient { }; // `chunkSize` only sizes this loop's requests; it is not an API parameter, so it must not reach - // `_params()` and the query string. + // `buildParams()` and the query string. The same goes for `timeoutSecs`, which callers take out before + // calling this, since it also picks the timeout of every page request. const { chunkSize, ...listOptions } = options; const paginatedListPromise = getPaginatedList({ diff --git a/src/base/resource_client.ts b/src/base/resource_client.ts index 7739d45a6..5ffe062f1 100644 --- a/src/base/resource_client.ts +++ b/src/base/resource_client.ts @@ -4,6 +4,7 @@ import type { z } from 'zod'; import type { ApifyApiError } from '../apify_api_error.js'; import type { ApifyRequestConfig } from '../http_client.js'; +import type { Timeout, TimeoutOptions, TimeoutTier } from '../timeouts.js'; import { catchNotFoundForResourceOrThrow, catchNotFoundOrThrow, parseResponse } from '../utils.js'; import { ApiClient } from './api_client.js'; @@ -14,29 +15,44 @@ import { ApiClient } from './api_client.js'; */ const MAX_WAIT_FOR_FINISH = 999999; -export const SMALL_TIMEOUT_MILLIS = 5 * 1000; // For fast and common actions. Suitable for idempotent actions. -export const MEDIUM_TIMEOUT_MILLIS = 30 * 1000; // For actions that may take longer. -export const DEFAULT_TIMEOUT_MILLIS = 360 * 1000; // 6 minutes +/** The API holds a `waitForFinish` response for at most a minute, however long the parameter asks for. */ +const MAX_WAIT_FOR_FINISH_HOLD_SECS = 60; /** * Resource client. * @private */ export class ResourceClient extends ApiClient { + /** + * Picks the timeout of a request that asks the API to hold its response with `waitForFinish`. The request + * gets the hold the caller asked for plus the round trip its tier allows, so the client does not abort a + * request while the API is still holding it - which for `start()` and `build()` would retry a call that + * creates a resource. An explicit per-call `timeoutSecs` is used as given. + */ + protected timeoutForWaitForFinish( + timeoutSecs: Timeout | undefined, + tier: TimeoutTier, + waitForFinishSecs: number | undefined, + ): Timeout { + if (timeoutSecs !== undefined) return timeoutSecs; + if (waitForFinishSecs === undefined) return tier; + + const holdSecs = Math.min(waitForFinishSecs, MAX_WAIT_FOR_FINISH_HOLD_SECS); + if (holdSecs <= 0) return tier; + + return holdSecs + this.httpClient.timeoutMillis[tier] / 1000; + } + /** * A 404 resolves to `undefined` only when the client names its resource by ID. A chained client without one, such * as `run.dataset()`, throws it instead (see `catchNotFoundForResourceOrThrow()`). */ - protected async _get( - schema: z.ZodType, - options: T = {} as T, - timeoutMillis?: number, - ): Promise { + protected async getResource(schema: z.ZodType, options: T, timeoutSecs: Timeout): Promise { const requestOpts: ApifyRequestConfig = { - url: this._url(), + url: this.buildUrl(), method: 'GET', - params: this._params(options), - timeout: timeoutMillis, + params: this.buildParams(options), + timeoutSecs, }; try { const response = await this.httpClient.call(requestOpts); @@ -48,13 +64,13 @@ export class ResourceClient extends ApiClient { return undefined; } - protected async _update(schema: z.ZodType, newFields: T, timeoutMillis?: number): Promise { + protected async updateResource(schema: z.ZodType, newFields: T, timeoutSecs: Timeout): Promise { const response = await this.httpClient.call({ - url: this._url(), + url: this.buildUrl(), method: 'PUT', - params: this._params(), + params: this.buildParams(), data: newFields, - timeout: timeoutMillis, + timeoutSecs, }); return parseResponse(response, schema); } @@ -63,13 +79,13 @@ export class ResourceClient extends ApiClient { * A 404 is swallowed, keeping the DELETE idempotent, only when the client names its resource by ID. A chained client * without one throws it instead (see `catchNotFoundForResourceOrThrow()`). */ - protected async _delete(timeoutMillis?: number): Promise { + protected async deleteResource(timeoutSecs: Timeout): Promise { try { await this.httpClient.call({ - url: this._url(), + url: this.buildUrl(), method: 'DELETE', - params: this._params(), - timeout: timeoutMillis, + params: this.buildParams(), + timeoutSecs, }); } catch (err) { catchNotFoundForResourceOrThrow(err as ApifyApiError, this.id); @@ -80,11 +96,11 @@ export class ResourceClient extends ApiClient { * This function is used in Build and Run endpoints so it's kept * here to stay DRY. */ - protected async _waitForFinish( + protected async waitForJobFinish( schema: z.ZodType, options: WaitForFinishOptions = {}, ): Promise { - const { waitSecs = MAX_WAIT_FOR_FINISH } = options; + const { waitSecs = MAX_WAIT_FOR_FINISH, timeoutSecs = 'noTimeout' } = options; const waitMillis = waitSecs * 1000; let job: R | undefined; @@ -103,9 +119,10 @@ export class ResourceClient extends ApiClient { const waitForFinish = Math.max(0, remainingWaitSeconds); const requestOpts: ApifyRequestConfig = { - url: this._url(), + url: this.buildUrl(), method: 'GET', - params: this._params({ waitForFinish }), + params: this.buildParams({ waitForFinish }), + timeoutSecs, }; try { const response = await this.httpClient.call(requestOpts); @@ -135,6 +152,6 @@ export class ResourceClient extends ApiClient { } } -export interface WaitForFinishOptions { +export interface WaitForFinishOptions extends TimeoutOptions { waitSecs?: number; } diff --git a/src/base/resource_collection_client.ts b/src/base/resource_collection_client.ts index e0d480abf..d4210ab8a 100644 --- a/src/base/resource_collection_client.ts +++ b/src/base/resource_collection_client.ts @@ -1,5 +1,6 @@ import type { z } from 'zod'; +import type { Timeout, TimeoutOptions } from '../timeouts.js'; import type { PaginatedResponse, PaginationOptions } from '../utils.js'; import { parseResponse } from '../utils.js'; import { ApiClient } from './api_client.js'; @@ -12,44 +13,57 @@ export class ResourceCollectionClient extends ApiClient { /** * @private */ - protected async _list(schema: z.ZodType, options: T = {} as T): Promise { + protected async listResources(schema: z.ZodType, options: T | undefined, timeoutSecs: Timeout): Promise { const response = await this.httpClient.call({ - url: this._url(), + url: this.buildUrl(), method: 'GET', - params: this._params(options), + params: this.buildParams(options), + timeoutSecs, }); return parseResponse(response, schema); } /** * Returns async iterator to iterate through all items and Promise that can be awaited to get first page of results. + * `defaultTimeoutSecs` applies to every page request unless `options.timeoutSecs` overrides it. */ - protected _listPaginated>( - schema: z.ZodType, - options: T = {} as T, - ): AsyncIterable & Promise { - return this._listPaginatedFromCallback( - async (listOptions?: T) => this._list(schema, listOptions), - options, + protected listResourcesPaginated< + T extends PaginationOptions & TimeoutOptions, + Data, + R extends PaginatedResponse, + >(schema: z.ZodType, options: T, defaultTimeoutSecs: Timeout): AsyncIterable & Promise { + // `timeoutSecs` only times the page requests; it is not an API parameter, so it must not reach the query string. + const { timeoutSecs = defaultTimeoutSecs, ...listOptions } = options; + + return this.listPaginatedFromCallback( + async (pageOptions?: T) => this.listResources(schema, pageOptions, timeoutSecs), + listOptions as T, ); } - protected async _create(schema: z.ZodType, resource: D): Promise { + protected async createResource(schema: z.ZodType, resource: D, timeoutSecs: Timeout): Promise { const response = await this.httpClient.call({ - url: this._url(), + url: this.buildUrl(), method: 'POST', - params: this._params(), + params: this.buildParams(), data: resource, + timeoutSecs, }); return parseResponse(response, schema); } - protected async _getOrCreate(schema: z.ZodType, name?: string, resource?: D): Promise { + protected async getOrCreateResource( + schema: z.ZodType, + name: string | undefined, + resource: D | undefined, + timeoutSecs: Timeout, + ): Promise { const response = await this.httpClient.call({ - url: this._url(), + url: this.buildUrl(), method: 'POST', - params: this._params({ name }), + params: this.buildParams({ name }), data: resource, + timeoutSecs, }); return parseResponse(response, schema); } diff --git a/src/http_client.ts b/src/http_client.ts index b76570993..b032744ed 100644 --- a/src/http_client.ts +++ b/src/http_client.ts @@ -14,6 +14,7 @@ import { ApifyApiError } from './apify_api_error.js'; import type { RequestInterceptorFunction } from './interceptors.js'; import { InvalidResponseBodyError, requestInterceptors, responseInterceptors } from './interceptors.js'; import type { Statistics } from './statistics.js'; +import type { Timeout, TimeoutTier } from './timeouts.js'; import { asArray, cast, getVersionData, isNode, isStream } from './utils.js'; const { version } = getVersionData(); @@ -31,7 +32,11 @@ export class HttpClient { logger: Log; - timeoutMillis: number; + /** Duration of each timeout tier, in milliseconds. */ + timeoutMillis: Record; + + /** Cap on the timeout of a single request attempt, in milliseconds. */ + timeoutMaxMillis: number; httpAgent?: http.Agent; @@ -51,11 +56,16 @@ export class HttpClient { this.maxRetries = options.maxRetries; this.minDelayBetweenRetriesMillis = options.minDelayBetweenRetriesMillis; this.userProvidedRequestInterceptors = options.requestInterceptors; - this.timeoutMillis = options.timeoutSecs * 1000; + this.timeoutMillis = { + short: options.timeoutShortSecs * 1000, + medium: options.timeoutMediumSecs * 1000, + long: options.timeoutLongSecs * 1000, + }; + this.timeoutMaxMillis = options.timeoutMaxSecs * 1000; this.logger = options.logger; this.workflowKey = options.workflowKey || process.env[APIFY_ENV_VARS.WORKFLOW_KEY]; this.userAgentSuffix = options.userAgentSuffix; - this._onRequestRetry = this._onRequestRetry.bind(this); + this.onRequestRetry = this.onRequestRetry.bind(this); this.axios = axios.create({ // Disable axios's built-in proxy handling since we're using custom agents @@ -78,7 +88,9 @@ export class HttpClient { transformRequest: undefined, transformResponse: undefined, responseType: 'arraybuffer', - timeout: this.timeoutMillis, + // Every request sets its own timeout in `createRequestHandler`, so the default only backs a raw + // `axios.request()` call. + timeout: this.timeoutMaxMillis, // maxBodyLength needs to be Infinity, because -1 falls back to a 10 MB default // from an axios subdependency - 'follow-redirects' maxBodyLength: Infinity, @@ -128,7 +140,7 @@ export class HttpClient { keepAlive: true, // Timeout for inactive sockets // Prevents socket leaks from idle connections - timeout: this.timeoutMillis, + timeout: this.timeoutMaxMillis, // Keep alive timeout for free sockets (15 seconds) // Node.js will close unused sockets after this period keepAliveMsecs: 15_000, @@ -171,16 +183,16 @@ export class HttpClient { async call(config: ApifyRequestConfig): Promise> { await this.ensureNodeInit(); this.stats.calls++; - const makeRequest = this._createRequestHandler(config); + const makeRequest = this.createRequestHandler(config); return retry(makeRequest, { retries: this.maxRetries, minTimeout: this.minDelayBetweenRetriesMillis, - onRetry: this._onRequestRetry, + onRetry: this.onRequestRetry, }); } - private _informAboutStreamNoRetry() { + private informAboutStreamNoRetry() { this.logger.warningOnce( 'Request body was a stream - retrying will not work, as part of it was already consumed.', ); @@ -194,7 +206,9 @@ export class HttpClient { * status codes are retried. See the following functions for the * retrying logic. */ - private _createRequestHandler(config: ApifyRequestConfig) { + private createRequestHandler(config: ApifyRequestConfig) { + const { timeoutSecs = 'medium', ...axiosConfig } = config; + const makeRequest: RetryFunction = async (stopTrying, attempt) => { this.stats.requests++; let response: ApifyResponse; @@ -205,19 +219,16 @@ export class HttpClient { // Handling redirects is not possible without buffering - part of the stream has already been sent and can't be recovered // when server sends the redirect. Therefore we need to override this in Axios config to prevent it from buffering the body. // see also axios/axios#1045 - config = { ...config, maxRedirects: 0 }; + axiosConfig.maxRedirects = 0; } - // Increase timeout with each attempt. Max timeout is bounded by the client timeout. - config.timeout = Math.min( - this.timeoutMillis, - (config.timeout ?? this.timeoutMillis) * 2 ** (attempt - 1), - ); - - response = await this.axios.request(config); - if (this._isStatusOk(response.status)) return response; + response = await this.axios.request({ + ...axiosConfig, + timeout: this.computeTimeoutMillis(timeoutSecs, attempt), + }); + if (this.isStatusOk(response.status)) return response; } catch (err) { - return cast(this._handleRequestError(err as AxiosError, config, stopTrying)); + return cast(this.handleRequestError(err as AxiosError, config, stopTrying)); } if (response.status === RATE_LIMIT_EXCEEDED_STATUS_CODE) { @@ -225,9 +236,9 @@ export class HttpClient { } const apiError = ApifyApiError.fromResponse(response, attempt); - if (this._isStatusCodeRetryable(response.status)) { + if (this.isStatusCodeRetryable(response.status)) { if (requestIsStream) { - this._informAboutStreamNoRetry(); + this.informAboutStreamNoRetry(); } else { // allow a retry throw apiError; @@ -241,22 +252,45 @@ export class HttpClient { return makeRequest; } - private _isStatusOk(statusCode: number) { + private isStatusOk(statusCode: number) { return statusCode < 300; } + /** + * Resolves `timeoutSecs` to the number of milliseconds the given attempt gets. A tier name resolves to + * its configured duration, a number is taken as seconds, and `'noTimeout'` becomes `0`, which axios reads + * as no timeout. The result doubles with each attempt and is capped at `timeoutMaxMillis`. A requested + * value above the cap is capped too, which warns once, since it does not take effect in full. + */ + private computeTimeoutMillis(timeoutSecs: Timeout, attempt: number): number { + if (timeoutSecs === 'noTimeout') return 0; + + const requestedMillis = typeof timeoutSecs === 'number' ? timeoutSecs * 1000 : this.timeoutMillis[timeoutSecs]; + + if (requestedMillis > this.timeoutMaxMillis) { + // `warningOnce` keys by message, so each requested value warns once. + this.logger.warningOnce( + `The requested timeout of ${requestedMillis / 1000}s exceeds timeoutMaxSecs ` + + `(${this.timeoutMaxMillis / 1000}s) and is capped at it. ` + + 'Raise timeoutMaxSecs on the client to allow longer request timeouts.', + ); + } + + return Math.min(requestedMillis * 2 ** (attempt - 1), this.timeoutMaxMillis); + } + /** * Handles all unexpected errors that can happen, but are not * Apify API typed errors. E.g. network errors, timeouts and so on. */ - private _handleRequestError(err: AxiosError, config: ApifyRequestConfig, stopTrying: (e: Error) => void) { - if (this._isTimeoutError(err) && config.doNotRetryTimeouts) { + private handleRequestError(err: AxiosError, config: ApifyRequestConfig, stopTrying: (e: Error) => void) { + if (this.isTimeoutError(err) && config.doNotRetryTimeouts) { return stopTrying(err); } - if (this._isRetryableError(err)) { + if (this.isRetryableError(err)) { if (isStream(config.data)) { - this._informAboutStreamNoRetry(); + this.informAboutStreamNoRetry(); } else { throw err; } @@ -268,7 +302,7 @@ export class HttpClient { * Axios calls req.abort() on timeouts so timeout errors will * have a code ECONNABORTED. */ - private _isTimeoutError(err: AxiosError) { + private isTimeoutError(err: AxiosError) { return err.code === 'ECONNABORTED'; } @@ -278,8 +312,8 @@ export class HttpClient { * @param {Error} err * @private */ - private _isRetryableError(err: AxiosError) { - return this._isNetworkError(err) || this._isResponseBodyInvalid(err); + private isRetryableError(err: AxiosError) { + return this.isNetworkError(err) || this.isResponseBodyInvalid(err); } /** @@ -287,7 +321,7 @@ export class HttpClient { * a response, the request often does not fail, but simply contains * an incomplete response. This can often be fixed by retrying. */ - private _isResponseBodyInvalid(err: Error): err is InvalidResponseBodyError { + private isResponseBodyInvalid(err: Error): err is InvalidResponseBodyError { return err instanceof InvalidResponseBodyError; } @@ -296,7 +330,7 @@ export class HttpClient { * it throws an AxiosError, which will have the request * and config (and other) properties. */ - private _isNetworkError(err: AxiosError) { + private isNetworkError(err: AxiosError) { const hasRequest = err.request && typeof err.request === 'object'; const hasConfig = err.config && typeof err.config === 'object'; return hasRequest && hasConfig; @@ -307,13 +341,13 @@ export class HttpClient { * For status codes 300-499 (except 429) we do not retry the request, * because it's probably caused by invalid url (redirect 3xx) or invalid user input (4xx). */ - private _isStatusCodeRetryable(statusCode: number) { + private isStatusCodeRetryable(statusCode: number) { const isRateLimitError = statusCode === RATE_LIMIT_EXCEEDED_STATUS_CODE; const isInternalError = statusCode >= 500; return isRateLimitError || isInternalError; } - private _onRequestRetry(error: Error, attempt: number) { + private onRequestRetry(error: Error, attempt: number) { if (attempt === Math.round(this.maxRetries / 2)) { this.logger.warning( `API request failed ${attempt} times. Max attempts: ${this.maxRetries + 1}.\nCause:${error.stack}`, @@ -322,10 +356,17 @@ export class HttpClient { } } -export interface ApifyRequestConfig extends AxiosRequestConfig { +export interface ApifyRequestConfig extends Omit { stringifyFunctions?: boolean; forceBuffer?: boolean; doNotRetryTimeouts?: boolean; + /** + * Timeout of the request: a tier name, a number of seconds, or `'noTimeout'`. The client resolves it to + * the axios `timeout` in milliseconds for each attempt, before axios runs its interceptors, so a request + * interceptor already sees a number of milliseconds. + * @default 'medium' + */ + timeoutSecs?: Timeout; } export interface ApifyResponse extends AxiosResponse { @@ -337,7 +378,10 @@ export interface HttpClientOptions { maxRetries: number; minDelayBetweenRetriesMillis: number; requestInterceptors: RequestInterceptorFunction[]; - timeoutSecs: number; + timeoutShortSecs: number; + timeoutMediumSecs: number; + timeoutLongSecs: number; + timeoutMaxSecs: number; logger: Log; token?: string; workflowKey?: string; diff --git a/src/index.ts b/src/index.ts index 19be88707..1377384e1 100644 --- a/src/index.ts +++ b/src/index.ts @@ -31,3 +31,4 @@ export { ArgumentValidationError } from '@apify/validations'; export * from './response_validation_error.js'; export { InvalidResponseBodyError } from './interceptors.js'; export type { PaginatedList, Dictionary } from './utils.js'; +export type { Timeout, TimeoutOptions, TimeoutTier } from './timeouts.js'; diff --git a/src/resource_clients/actor.ts b/src/resource_clients/actor.ts index b79a52a4e..b69472ffc 100644 --- a/src/resource_clients/actor.ts +++ b/src/resource_clients/actor.ts @@ -7,7 +7,9 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { ApifyRequestConfig } from '../http_client.js'; import type { Actor, ActorRun } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, parseArgument, parseResponse, stringifyWebhooksToBase64 } from '../utils.js'; import { ActorVersionClient } from './actor_version.js'; import { ActorVersionCollectionClient } from './actor_version_collection.js'; @@ -24,19 +26,20 @@ const startOptionsSchema = z.strictObject({ build: z.string().optional(), contentType: z.string().optional(), memory: z.number().optional(), - timeout: z.number().optional(), + runTimeoutSecs: z.number().min(0).optional(), waitForFinish: z.number().optional(), webhooks: z.array(anyObjectSchema).optional(), maxItems: z.number().min(0).optional(), maxTotalChargeUsd: z.number().min(0).optional(), restartOnError: z.boolean().optional(), forcePermissionLevel: z.enum(ACTOR_PERMISSION_LEVEL).optional(), + ...timeoutOptionsShape, }); const callOptionsSchema = z.strictObject({ build: z.string().optional(), contentType: z.string().optional(), memory: z.number().optional(), - timeout: z.number().min(0).optional(), + runTimeoutSecs: z.number().min(0).optional(), waitSecs: z.number().min(0).optional(), webhooks: z.array(anyObjectSchema).optional(), maxItems: z.number().min(0).optional(), @@ -44,10 +47,12 @@ const callOptionsSchema = z.strictObject({ log: z.union([z.null(), z.instanceof(Log), z.literal('default')]).optional(), restartOnError: z.boolean().optional(), forcePermissionLevel: z.enum(ACTOR_PERMISSION_LEVEL).optional(), + ...timeoutOptionsShape, }); const validateInputOptionsSchema = z.strictObject({ build: z.string().optional(), contentType: z.string().optional(), + ...timeoutOptionsShape, }); const versionNumberSchema = z.string().min(1); const buildOptionsSchema = z.strictObject({ @@ -55,7 +60,9 @@ const buildOptionsSchema = z.strictObject({ tag: z.string().optional(), useCache: z.boolean().optional(), waitForFinish: z.number().optional(), + ...timeoutOptionsShape, }); +const defaultBuildOptionsSchema = z.strictObject({ waitForFinish: z.number().optional(), ...timeoutOptionsShape }); const lastRunOptionsSchema = z.strictObject({ status: z.enum(ACTOR_JOB_STATUSES).optional(), origin: z.enum(META_ORIGINS).optional(), @@ -125,33 +132,44 @@ export class ActorClient extends ResourceClient { /** * Gets the Actor object from the Apify API. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The Actor object, or `undefined` if it does not exist * @see https://docs.apify.com/api/v2/act-get */ - async get(): Promise { - return this._get(schemas.Actor()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.Actor(), {}, timeoutSecs); } /** * Updates the Actor with specified fields. * * @param newFields - Fields to update in the Actor + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated Actor object * @see https://docs.apify.com/api/v2/act-put */ - async update(newFields: ActorUpdateOptions): Promise { + async update(newFields: ActorUpdateOptions, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.Actor(), newFields); + return this.updateResource(schemas.Actor(), newFields, timeoutSecs); } /** * Deletes the Actor. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/act-delete */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** @@ -165,11 +183,13 @@ export class ActorClient extends ResourceClient { * @param options - Run configuration options * @param options.build - Tag or number of the build to run (e.g., `'beta'` or `'1.2.345'`). If not provided, uses the default build. * @param options.memory - Memory in megabytes allocated for the run. If not provided, uses the Actor's default memory setting. - * @param options.timeout - Timeout for the run in seconds. Zero means no timeout. If not provided, uses the Actor's default timeout. + * @param options.runTimeoutSecs - Timeout for the run in seconds. Zero means no timeout. If not provided, uses the Actor's default timeout. * @param options.waitForFinish - Maximum time to wait (in seconds, max 60s) for the run to finish on the API side before returning. Default is 0 (returns immediately). * @param options.webhooks - Webhooks to trigger when the Actor run reaches a specific state (e.g., `SUCCEEDED`, `FAILED`). * @param options.maxItems - Maximum number of dataset items that will be charged (only for pay-per-result Actors). * @param options.maxTotalChargeUsd - Maximum cost in USD (only for pay-per-event Actors). + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`, extended to cover `waitForFinish` + * when the API is asked to hold the response. * @returns The Actor run object with status, usage, and storage IDs * @see https://docs.apify.com/api/v2/act-runs-post * @@ -182,7 +202,7 @@ export class ActorClient extends ResourceClient { * // Start Actor with specific build and memory * const run = await client.actor('my-actor').start( * { url: 'https://example.com' }, - * { build: '0.1.2', memory: 512, timeout: 300 } + * { build: '0.1.2', memory: 512, runTimeoutSecs: 300 } * ); * ``` */ @@ -191,18 +211,20 @@ export class ActorClient extends ResourceClient { const { waitForFinish, - timeout, + runTimeoutSecs, memory, build, maxItems, maxTotalChargeUsd, restartOnError, forcePermissionLevel, + timeoutSecs, } = parsed; + // The API's `timeout` parameter bounds the run, not the request. const params = { waitForFinish, - timeout, + timeout: runTimeoutSecs, memory, build, webhooks: stringifyWebhooksToBase64(parsed.webhooks), @@ -213,13 +235,14 @@ export class ActorClient extends ResourceClient { }; const request: ApifyRequestConfig = { - url: this._url('runs'), + url: this.buildUrl('runs'), method: 'POST', data: input, - params: this._params(params), + params: this.buildParams(params), // Apify internal property. Tells the request serialization interceptor // to stringify functions to JSON, instead of omitting them. stringifyFunctions: true, + timeoutSecs: this.timeoutForWaitForFinish(timeoutSecs, 'medium', waitForFinish), }; if (parsed.contentType) { request.headers = { @@ -244,7 +267,8 @@ export class ActorClient extends ResourceClient { * @param options.log - Log instance for streaming run logs. Use `'default'` for console output, `null` to disable logging, or provide a custom Log instance. * @param options.build - Tag or number of the build to run (e.g., `'beta'` or `'1.2.345'`). * @param options.memory - Memory in megabytes allocated for the run. - * @param options.timeout - Maximum run duration in seconds. + * @param options.runTimeoutSecs - Maximum run duration in seconds. + * @param options.timeoutSecs - Timeout for each API request, the start and every poll alike. Default is `'noTimeout'`. * @returns The finished Actor run object with final status (`SUCCEEDED`, `FAILED`, `ABORTED`, or `TIMED-OUT`) * @see https://docs.apify.com/api/v2/act-runs-post * @@ -270,8 +294,8 @@ export class ActorClient extends ResourceClient { async call(input?: ActorInput, options: ActorCallOptions = {}): Promise { const parsed = parseArgument(options, callOptionsSchema, 'ActorCallOptions'); - const { waitSecs, log, ...startOptions } = parsed; - const { id } = await this.start(input, startOptions); + const { waitSecs, log, timeoutSecs = 'noTimeout', ...startOptions } = parsed; + const { id } = await this.start(input, { ...startOptions, timeoutSecs }); // Calling root client because we need access to top level API. // Creating a new instance of RunClient here would only allow @@ -282,7 +306,7 @@ export class ActorClient extends ResourceClient { streamedLog?.start(); return this.apifyClient .run(id) - .waitForFinish({ waitSecs }) + .waitForFinish({ waitSecs, timeoutSecs }) .finally(async () => { await streamedLog?.stop(); }); @@ -300,6 +324,7 @@ export class ActorClient extends ResourceClient { * @param options - Validation options * @param options.build - Tag or number of the build whose input schema the input is validated against * (e.g., `'latest'` or `'1.2.345'`). If not provided, uses the default build. + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns `true` if the input is valid. Invalid input causes the underlying API call to throw an `ApifyApiError`. * @see https://docs.apify.com/api/v2/act-validate-input-post * @@ -320,13 +345,14 @@ export class ActorClient extends ResourceClient { const parsed = parseArgument(options, validateInputOptionsSchema, 'ActorValidateInputOptions'); const request: ApifyRequestConfig = { - url: this._url('validate-input'), + url: this.buildUrl('validate-input'), method: 'POST', data: input, - params: this._params({ build: parsed.build }), + params: this.buildParams({ build: parsed.build }), // Apify internal property. Tells the request serialization interceptor // to stringify functions to JSON, instead of omitting them. stringifyFunctions: true, + timeoutSecs: parsed.timeoutSecs ?? 'short', }; if (parsed.contentType) { request.headers = { @@ -350,6 +376,8 @@ export class ActorClient extends ResourceClient { * @param options.tag - Tag to be applied to the build (e.g., `'latest'`, `'beta'`). Existing tag with the same name will be replaced. * @param options.useCache - If `false`, Docker build cache will be ignored. Default is `true`. * @param options.waitForFinish - Maximum time to wait (in seconds, max 60s) for the build to finish on the API side before returning. Default is 0 (returns immediately). + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`, extended to cover `waitForFinish` + * when the API is asked to hold the response. * @returns The Build object with status and build details * @see https://docs.apify.com/api/v2/act-builds-post * @@ -369,15 +397,16 @@ export class ActorClient extends ResourceClient { */ async build(versionNumber: string, options: ActorBuildOptions = {}): Promise { parseArgument(versionNumber, versionNumberSchema); - const parsed = parseArgument(options, buildOptionsSchema, 'ActorBuildOptions'); + const { timeoutSecs, ...params } = parseArgument(options, buildOptionsSchema, 'ActorBuildOptions'); const response = await this.httpClient.call({ - url: this._url('builds'), + url: this.buildUrl('builds'), method: 'POST', - params: this._params({ + params: this.buildParams({ version: versionNumber, - ...parsed, + ...params, }), + timeoutSecs: this.timeoutForWaitForFinish(timeoutSecs, 'medium', params.waitForFinish), }); return parseResponse(response, schemas.Build()); @@ -392,6 +421,8 @@ export class ActorClient extends ResourceClient { * * @param options - Options for getting the default build * @param options.waitForFinish - Maximum time to wait (in seconds, max 60s) for the build to finish on the API side before returning. Default is 0 (returns immediately). + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`, extended to cover `waitForFinish` + * when the API is asked to hold the response. * @returns A client for the default build * @see https://docs.apify.com/api/v2/act-build-default-get * @@ -409,10 +440,13 @@ export class ActorClient extends ResourceClient { * @since Added in 2.12.2 */ async defaultBuild(options: BuildClientGetOptions = {}): Promise { + const { timeoutSecs, ...params } = parseArgument(options, defaultBuildOptionsSchema, 'BuildClientGetOptions'); + const response = await this.httpClient.call({ - url: this._url('builds/default'), + url: this.buildUrl('builds/default'), method: 'GET', - params: this._params(options), + params: this.buildParams(params), + timeoutSecs: this.timeoutForWaitForFinish(timeoutSecs, 'short', params.waitForFinish), }); const { id } = parseResponse(response, schemas.Build()); @@ -447,9 +481,9 @@ export class ActorClient extends ResourceClient { const parsed = parseArgument(options, lastRunOptionsSchema, 'ActorLastRunOptions'); return new RunClient( - this._subResourceOptions({ + this.subResourceOptions({ id: 'last', - params: this._params(parsed), + params: this.buildParams(parsed), resourcePath: 'runs', }), ); @@ -463,7 +497,7 @@ export class ActorClient extends ResourceClient { */ builds(): BuildCollectionClient { return new BuildCollectionClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'builds', }), ); @@ -477,7 +511,7 @@ export class ActorClient extends ResourceClient { */ runs(): RunCollectionClient { return new RunCollectionClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'runs', }), ); @@ -493,7 +527,7 @@ export class ActorClient extends ResourceClient { version(versionNumber: string): ActorVersionClient { parseArgument(versionNumber, versionNumberSchema); return new ActorVersionClient( - this._subResourceOptions({ + this.subResourceOptions({ id: versionNumber, }), ); @@ -506,7 +540,7 @@ export class ActorClient extends ResourceClient { * @see https://docs.apify.com/api/v2/act-versions-get */ versions(): ActorVersionCollectionClient { - return new ActorVersionCollectionClient(this._subResourceOptions()); + return new ActorVersionCollectionClient(this.subResourceOptions()); } /** @@ -516,7 +550,7 @@ export class ActorClient extends ResourceClient { * @see https://docs.apify.com/api/v2/act-webhooks-get */ webhooks(): WebhookCollectionClient { - return new WebhookCollectionClient(this._subResourceOptions()); + return new WebhookCollectionClient(this.subResourceOptions()); } } @@ -552,7 +586,7 @@ export type ActorUpdateOptions = Partial< */ export type ActorInput = object; -export interface ActorStartOptions { +export interface ActorStartOptions extends TimeoutOptions { /** * Tag or number of the Actor build to run (e.g. `beta` or `1.2.345`). * If not provided, the run uses build tag or number from the default Actor run configuration (typically `latest`). @@ -573,9 +607,9 @@ export interface ActorStartOptions { memory?: number; /** * Timeout for the Actor run in seconds. Zero value means there is no timeout. - * If not provided, the run uses timeout of the default Actor run configuration. + * If not provided, the run uses the timeout of the default Actor run configuration. */ - timeout?: number; + runTimeoutSecs?: number; /** * Maximum time to wait for the Actor run to finish, in seconds. @@ -652,7 +686,7 @@ export interface ActorCallOptions extends Omit { const parsed = parseArgument(options, listOptionsSchema, 'ActorCollectionListOptions'); - return this._listPaginated(schemas.ListOfActors(), parsed); + return this.listResourcesPaginated(schemas.ListOfActors(), parsed, 'medium'); } /** * Creates a new Actor. * * @param actor - The Actor data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The created Actor object. * @see https://docs.apify.com/api/v2/acts-post */ - async create(actor: ActorCollectionCreateOptions): Promise { + async create(actor: ActorCollectionCreateOptions, options: TimeoutOptions = {}): Promise { parseArgument(actor, actorCreateSchema); + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._create(schemas.Actor(), actor); + return this.createResource(schemas.Actor(), actor, timeoutSecs); } } @@ -105,9 +111,10 @@ const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), sortBy: z.enum(ActorListSortBy).optional(), + ...timeoutOptionsShape, }); -export interface ActorCollectionListOptions extends PaginationOptions { +export interface ActorCollectionListOptions extends PaginationOptions, TimeoutOptions { my?: boolean; desc?: boolean; /** diff --git a/src/resource_clients/actor_env_var.ts b/src/resource_clients/actor_env_var.ts index bdf12c5a7..1b7893a13 100644 --- a/src/resource_clients/actor_env_var.ts +++ b/src/resource_clients/actor_env_var.ts @@ -1,6 +1,8 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; import { anyObjectSchema, parseArgument } from '../utils.js'; import type { ActorEnvironmentVariable } from './actor_version.js'; @@ -41,31 +43,46 @@ export class ActorEnvVarClient extends ResourceClient { /** * Retrieves the environment variable. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The environment variable object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/act-version-env-var-get */ - async get(): Promise { - return this._get(schemas.EnvVar()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.EnvVar(), {}, timeoutSecs); } /** * Updates the environment variable. * * @param actorEnvVar - The updated environment variable data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated environment variable object. * @see https://docs.apify.com/api/v2/act-version-env-var-put */ - async update(actorEnvVar: ActorEnvironmentVariable): Promise { + async update( + actorEnvVar: ActorEnvironmentVariable, + options: TimeoutOptions = {}, + ): Promise { parseArgument(actorEnvVar, anyObjectSchema); - return this._update(schemas.EnvVar(), actorEnvVar); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.updateResource(schemas.EnvVar(), actorEnvVar, timeoutSecs); } /** * Deletes the environment variable. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/act-version-env-var-delete */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } } diff --git a/src/resource_clients/actor_env_var_collection.ts b/src/resource_clients/actor_env_var_collection.ts index f698bc853..46735c430 100644 --- a/src/resource_clients/actor_env_var_collection.ts +++ b/src/resource_clients/actor_env_var_collection.ts @@ -1,7 +1,9 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedList } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; import { anyObjectSchema, parseArgument } from '../utils.js'; import type { ActorEnvironmentVariable } from './actor_version.js'; @@ -60,23 +62,34 @@ export class ActorEnvVarCollectionClient extends ResourceCollectionClient { * for await (const singleItem of client.list()) {...} * ``` * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The environment variables, awaitable as a whole list or iterable one by one. * @see https://docs.apify.com/api/v2/act-version-env-vars-get */ - list(): Promise & AsyncIterable { - return this._listPaginated(schemas.ListOfEnvVars()); + list(options: TimeoutOptions = {}): Promise & AsyncIterable { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.listResourcesPaginated(schemas.ListOfEnvVars(), {}, timeoutSecs); } /** * Creates a new environment variable for this Actor version. * * @param actorEnvVar - The environment variable data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The created environment variable object. * @see https://docs.apify.com/api/v2/act-version-env-vars-post */ - async create(actorEnvVar: ActorEnvironmentVariable): Promise { + async create( + actorEnvVar: ActorEnvironmentVariable, + options: TimeoutOptions = {}, + ): Promise { parseArgument(actorEnvVar, actorEnvVarSchema); - return this._create(schemas.EnvVar(), actorEnvVar); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.createResource(schemas.EnvVar(), actorEnvVar, timeoutSecs); } } diff --git a/src/resource_clients/actor_version.ts b/src/resource_clients/actor_version.ts index d7bd902af..314a2e33f 100644 --- a/src/resource_clients/actor_version.ts +++ b/src/resource_clients/actor_version.ts @@ -3,7 +3,9 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { ActorVersion, FinalActorVersion } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; import { anyObjectSchema, parseArgument } from '../utils.js'; import { ActorEnvVarClient } from './actor_env_var.js'; import { ActorEnvVarCollectionClient } from './actor_env_var_collection.js'; @@ -60,33 +62,44 @@ export class ActorVersionClient extends ResourceClient { /** * Retrieves the Actor version. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The Actor version object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/act-version-get */ - async get(): Promise { - return this._get(schemas.Version()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.Version(), {}, timeoutSecs); } /** * Updates the Actor version with the specified fields. * * @param newFields - Fields to update. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated Actor version object. * @see https://docs.apify.com/api/v2/act-version-put */ - async update(newFields: ActorVersionUpdateData): Promise { + async update(newFields: ActorVersionUpdateData, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.Version(), newFields); + return this.updateResource(schemas.Version(), newFields, timeoutSecs); } /** * Deletes the Actor version. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/act-version-delete */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** @@ -100,7 +113,7 @@ export class ActorVersionClient extends ResourceClient { envVar(envVarName: string): ActorEnvVarClient { parseArgument(envVarName, envVarNameSchema); return new ActorEnvVarClient( - this._subResourceOptions({ + this.subResourceOptions({ id: envVarName, }), ); @@ -114,7 +127,7 @@ export class ActorVersionClient extends ResourceClient { * @since Added in 2.1.0 */ envVars(): ActorEnvVarCollectionClient { - return new ActorEnvVarCollectionClient(this._subResourceOptions()); + return new ActorEnvVarCollectionClient(this.subResourceOptions()); } } diff --git a/src/resource_clients/actor_version_collection.ts b/src/resource_clients/actor_version_collection.ts index e7f49365f..7b06e6ba4 100644 --- a/src/resource_clients/actor_version_collection.ts +++ b/src/resource_clients/actor_version_collection.ts @@ -1,7 +1,9 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedList } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; import { anyObjectSchema, parseArgument } from '../utils.js'; import type { ActorVersion, FinalActorVersion } from './actor_version.js'; @@ -59,24 +61,31 @@ export class ActorVersionCollectionClient extends ResourceCollectionClient { * for await (const singleItem of client.list()) {...} * ``` * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The Actor versions, awaitable as a whole list or iterable one by one. * @see https://docs.apify.com/api/v2/act-versions-get */ - list(): Promise & AsyncIterable { - return this._listPaginated(schemas.ListOfVersions()); + list(options: TimeoutOptions = {}): Promise & AsyncIterable { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.listResourcesPaginated(schemas.ListOfVersions(), {}, timeoutSecs); } /** * Creates a new Actor version. * * @param actorVersion - The Actor version data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The created Actor version object. * @see https://docs.apify.com/api/v2/act-versions-post */ - async create(actorVersion: ActorVersion): Promise { + async create(actorVersion: ActorVersion, options: TimeoutOptions = {}): Promise { parseArgument(actorVersion, actorVersionSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._create(schemas.Version(), actorVersion); + return this.createResource(schemas.Version(), actorVersion, timeoutSecs); } } diff --git a/src/resource_clients/build.ts b/src/resource_clients/build.ts index fdff0ff34..266f5539c 100644 --- a/src/resource_clients/build.ts +++ b/src/resource_clients/build.ts @@ -3,12 +3,14 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { Build } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { parseArgument, parseResponse } from '../utils.js'; import { LogClient } from './log.js'; -const getOptionsSchema = z.strictObject({ waitForFinish: z.number().optional() }); -const waitForFinishOptionsSchema = z.strictObject({ waitSecs: z.number().optional() }); +const getOptionsSchema = z.strictObject({ waitForFinish: z.number().optional(), ...timeoutOptionsShape }); +const waitForFinishOptionsSchema = z.strictObject({ waitSecs: z.number().optional(), ...timeoutOptionsShape }); export type { Build, BuildMeta, BuildOptions, BuildStats, BuildUsage } from '../models.js'; @@ -51,6 +53,8 @@ export class BuildClient extends ResourceClient { * * @param options - Get options * @param options.waitForFinish - Maximum time to wait (in seconds, max 60s) for the build to finish on the API side before returning. Default is 0 (returns immediately). + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`, extended to cover `waitForFinish` + * when the API is asked to hold the response. * @returns The Build object, or `undefined` if it does not exist * @see https://docs.apify.com/api/v2/actor-build-get * @@ -65,9 +69,13 @@ export class BuildClient extends ResourceClient { * ``` */ async get(options: BuildClientGetOptions = {}): Promise { - const parsed = parseArgument(options, getOptionsSchema, 'BuildClientGetOptions'); + const { timeoutSecs, ...params } = parseArgument(options, getOptionsSchema, 'BuildClientGetOptions'); - return this._get(schemas.Build(), parsed); + return this.getResource( + schemas.Build(), + params, + this.timeoutForWaitForFinish(timeoutSecs, 'short', params.waitForFinish), + ); } /** @@ -75,6 +83,8 @@ export class BuildClient extends ResourceClient { * * Stops the build process immediately. The build will have an `ABORTED` status. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated Build object with `ABORTED` status * @see https://docs.apify.com/api/v2/actor-build-abort-post * @@ -83,11 +93,14 @@ export class BuildClient extends ResourceClient { * await client.build('build-id').abort(); * ``` */ - async abort(): Promise { + async abort(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('abort'), + url: this.buildUrl('abort'), method: 'POST', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); return parseResponse(response, schemas.Build()); @@ -96,25 +109,34 @@ export class BuildClient extends ResourceClient { /** * Deletes the Actor build. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/actor-build-delete * @since Added in 2.8.1 */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** * Retrieves the OpenAPI definition for the Actor build. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The OpenAPI definition object. * @see https://docs.apify.com/api/v2/actor-build-openapi-json-get * @since Added in 2.11.2 */ - async getOpenApiDefinition(): Promise { + async getOpenApiDefinition(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('openapi.json'), + url: this.buildUrl('openapi.json'), method: 'GET', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); return response.data; @@ -135,7 +157,8 @@ export class BuildClient extends ResourceClient { * * @param options - Wait options * @param options.waitSecs - Maximum time to wait for the build to finish, in seconds. If omitted, waits indefinitely. - * @returns The Build object (finished or still building if timeout was reached) + * @param options.timeoutSecs - Timeout for each polling API request. Default is `'noTimeout'`. + * @returns The Build object (finished or still building if the timeout was reached) * * @example * ```javascript @@ -153,7 +176,7 @@ export class BuildClient extends ResourceClient { async waitForFinish(options: BuildClientWaitForFinishOptions = {}): Promise { const parsed = parseArgument(options, waitForFinishOptionsSchema, 'BuildClientWaitForFinishOptions'); - return this._waitForFinish(schemas.Build(), parsed); + return this.waitForJobFinish(schemas.Build(), parsed); } /** @@ -174,7 +197,7 @@ export class BuildClient extends ResourceClient { */ log(): LogClient { return new LogClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'log', }), ); @@ -184,14 +207,14 @@ export class BuildClient extends ResourceClient { /** * Options for getting a Build. */ -export interface BuildClientGetOptions { +export interface BuildClientGetOptions extends TimeoutOptions { waitForFinish?: number; } /** * Options for waiting for a Build to finish. */ -export interface BuildClientWaitForFinishOptions { +export interface BuildClientWaitForFinishOptions extends TimeoutOptions { /** * Maximum time to wait for the build to finish, in seconds. * If the limit is reached, the returned promise is resolved to a build object that will have diff --git a/src/resource_clients/build_collection.ts b/src/resource_clients/build_collection.ts index 93685ecc0..80e7e3a34 100644 --- a/src/resource_clients/build_collection.ts +++ b/src/resource_clients/build_collection.ts @@ -3,8 +3,10 @@ import { z } from 'zod'; import type { ApiClientOptionsWithOptionalResourcePath } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; import type { BuildCollectionClientListItem } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginatedList, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsShape } from '../timeouts.js'; import { paginationOptionsShape, parseArgument } from '../utils.js'; export type { BuildCollectionClientListItem } from '../models.js'; @@ -12,6 +14,7 @@ export type { BuildCollectionClientListItem } from '../models.js'; const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), + ...timeoutOptionsShape, }); /** @@ -63,17 +66,18 @@ export class BuildCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination and sorting options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of Actor builds. * @see https://docs.apify.com/api/v2/actor-builds-get */ list(options: BuildCollectionClientListOptions = {}): PaginatedIterator { const parsed = parseArgument(options, listOptionsSchema, 'BuildCollectionClientListOptions'); - return this._listPaginated(schemas.ListOfBuilds(), parsed); + return this.listResourcesPaginated(schemas.ListOfBuilds(), parsed, 'medium'); } } -export interface BuildCollectionClientListOptions extends PaginationOptions { +export interface BuildCollectionClientListOptions extends PaginationOptions, TimeoutOptions { desc?: boolean; } diff --git a/src/resource_clients/dataset.ts b/src/resource_clients/dataset.ts index ad2a03452..a38121178 100644 --- a/src/resource_clients/dataset.ts +++ b/src/resource_clients/dataset.ts @@ -4,16 +4,13 @@ import type { STORAGE_GENERAL_ACCESS } from '@apify/consts'; import { createStorageContentSignatureAsync } from '@apify/utilities'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; -import { - DEFAULT_TIMEOUT_MILLIS, - MEDIUM_TIMEOUT_MILLIS, - ResourceClient, - SMALL_TIMEOUT_MILLIS, -} from '../base/resource_client.js'; +import { ResourceClient } from '../base/resource_client.js'; import type { ApifyResponse } from '../http_client.js'; import type { Dataset, DatasetStatistics } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginatedList, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, applyQueryParamsToUrl, @@ -39,6 +36,7 @@ const listItemsOptionsSchema = z.strictObject({ unwind: z.union([z.string(), z.array(z.string())]).optional(), view: z.string().optional(), signature: z.string().optional(), + ...timeoutOptionsShape, }); // Spells out `limit` / `offset` instead of spreading `paginationOptionsShape`: a download is one // request, so `chunkSize` has nothing to size. The options type omits it to match. @@ -61,10 +59,12 @@ const downloadItemsOptionsSchema = z.strictObject({ xmlRoot: z.string().optional(), xmlRow: z.string().optional(), signature: z.string().optional(), + ...timeoutOptionsShape, }); const pushItemsSchema = z.union([itemSchema, z.string(), z.array(z.union([itemSchema, z.string()]))]); -// Every option becomes a query parameter of the generated URL, so `chunkSize` (client-side only) and -// `signature` (which this method produces) are left out. The options type omits both to match. +// Apart from `timeoutSecs` and `expiresInSecs`, every option becomes a query parameter of the generated URL, so +// `chunkSize` (client-side only) and `signature` (which this method produces) are left out. The options type +// omits both to match. const createItemsPublicUrlOptionsSchema = z.strictObject({ clean: z.boolean().optional(), desc: z.boolean().optional(), @@ -78,6 +78,7 @@ const createItemsPublicUrlOptionsSchema = z.strictObject({ unwind: z.union([z.string(), z.array(z.string())]).optional(), view: z.string().optional(), expiresInSecs: z.number().optional(), + ...timeoutOptionsShape, }); export type { Dataset, DatasetStatistics, DatasetStats, FieldStatistics } from '../models.js'; @@ -127,33 +128,44 @@ export class DatasetClient< /** * Gets the dataset object from the Apify API. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The Dataset object, or `undefined` if it does not exist * @see https://docs.apify.com/api/v2/dataset-get */ - async get(): Promise { - return this._get(schemas.Dataset(), {}, SMALL_TIMEOUT_MILLIS); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.Dataset(), {}, timeoutSecs); } /** * Updates the dataset with specified fields. * * @param newFields - Fields to update in the dataset + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated Dataset object * @see https://docs.apify.com/api/v2/dataset-put */ - async update(newFields: DatasetClientUpdateOptions): Promise { + async update(newFields: DatasetClientUpdateOptions, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.Dataset(), newFields, SMALL_TIMEOUT_MILLIS); + return this.updateResource(schemas.Dataset(), newFields, timeoutSecs); } /** * Deletes the dataset. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/dataset-delete */ - async delete(): Promise { - return this._delete(SMALL_TIMEOUT_MILLIS); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** @@ -176,6 +188,7 @@ export class DatasetClient< * @param options.flatten - Array of field names to flatten. Nested objects are converted to dot notation (e.g., `obj.field`). * @param options.unwind - Field name or array of field names to unwind. Each array value creates a separate item. * @param options.view - Name of a predefined view to use for field selection. + * @param options.timeoutSecs - Timeout for each API request. Default is `'long'`. * @returns A paginated list with `items`, `total` count, `offset`, `count`, and `limit` * @see https://docs.apify.com/api/v2/dataset-items-get * @@ -201,22 +214,26 @@ export class DatasetClient< * ``` */ listItems(options: DatasetClientListItemOptions = {}): PaginatedIterator { - const parsed = parseArgument(options, listItemsOptionsSchema, 'DatasetClientListItemOptions'); + const { timeoutSecs = 'long', ...listOptions } = parseArgument( + options, + listItemsOptionsSchema, + 'DatasetClientListItemOptions', + ); const fetchItems = async ( datasetListOptions: DatasetClientListItemOptions = {}, ): Promise> => { const response = await this.httpClient.call({ - url: this._url('items'), + url: this.buildUrl('items'), method: 'GET', - params: this._params(datasetListOptions), - timeout: DEFAULT_TIMEOUT_MILLIS, + params: this.buildParams(datasetListOptions), + timeoutSecs, }); - return this._createPaginationList(response, datasetListOptions.desc ?? false); + return this.createPaginationList(response, datasetListOptions.desc ?? false); }; - return this._listPaginatedFromCallback(fetchItems, parsed); + return this.listPaginatedFromCallback(fetchItems, listOptions); } /** @@ -236,6 +253,7 @@ export class DatasetClient< * @param options.xmlRow - Name of the XML element for each item. Default is `'item'`. * @param options.fields - Array of field names to include in the export. * @param options.omit - Array of field names to exclude from the export. + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @returns Buffer containing the serialized data in the specified format * @see https://docs.apify.com/api/v2/dataset-items-get * @@ -264,17 +282,21 @@ export class DatasetClient< options: DatasetClientDownloadItemsOptions = {}, ): Promise { parseArgument(format, itemFormatSchema); - const parsed = parseArgument(options, downloadItemsOptionsSchema, 'DatasetClientDownloadItemsOptions'); + const { timeoutSecs = 'long', ...query } = parseArgument( + options, + downloadItemsOptionsSchema, + 'DatasetClientDownloadItemsOptions', + ); const { data } = await this.httpClient.call({ - url: this._url('items'), + url: this.buildUrl('items'), method: 'GET', - params: this._params({ + params: this.buildParams({ format, - ...parsed, + ...query, }), forceBuffer: true, - timeout: DEFAULT_TIMEOUT_MILLIS, + timeoutSecs, }); return cast(data); @@ -290,6 +312,8 @@ export class DatasetClient< * * @param items - A single item (object or string) or an array of items to store. * Objects are automatically stringified to JSON. Strings are stored as-is. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @see https://docs.apify.com/api/v2/dataset-items-post * * @example @@ -312,19 +336,20 @@ export class DatasetClient< * await client.dataset('my-dataset').pushItems(['item1', 'item2', 'item3']); * ``` */ - async pushItems(items: Data | Data[] | string | string[]): Promise { + async pushItems(items: Data | Data[] | string | string[], options: TimeoutOptions = {}): Promise { parseArgument(items, pushItemsSchema); + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); await this.httpClient.call({ - url: this._url('items'), + url: this.buildUrl('items'), method: 'POST', headers: { 'content-type': 'application/json; charset=utf-8', }, data: items, - params: this._params(), - doNotRetryTimeouts: true, // see timeout handling in http-client - timeout: MEDIUM_TIMEOUT_MILLIS, + params: this.buildParams(), + doNotRetryTimeouts: true, // see timeoutSecs handling in http-client + timeoutSecs, }); } @@ -334,16 +359,20 @@ export class DatasetClient< * Returns statistics for each field in the dataset, including information about * data types, null counts, and value ranges. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns Dataset statistics * @see https://docs.apify.com/api/v2/dataset-statistics-get * @since Added in 2.11.2 */ - async getStatistics(): Promise { + async getStatistics(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('statistics'), + url: this.buildUrl('statistics'), method: 'GET', - params: this._params(), - timeout: SMALL_TIMEOUT_MILLIS, + params: this.buildParams(), + timeoutSecs, }); return parseResponse(response, schemas.DatasetStatistics()); } @@ -360,6 +389,7 @@ export class DatasetClient< * @param options.fields - Array of field names to include in the response. * @param options.limit - Maximum number of items to return. * @param options.offset - Number of items to skip. + * @param options.timeoutSecs - Timeout for the API request that fetches the dataset. Default is `'long'`. * @returns A public URL string for accessing the dataset items * * @example @@ -381,13 +411,15 @@ export class DatasetClient< * @since Added in 2.13.0 */ async createItemsPublicUrl(options: DatasetClientCreateItemsUrlOptions = {}): Promise { - const parsed = parseArgument(options, createItemsPublicUrlOptionsSchema, 'DatasetClientCreateItemsUrlOptions'); - - const dataset = await this.get(); + const { + timeoutSecs = 'long', + expiresInSecs, + ...queryOptions + } = parseArgument(options, createItemsPublicUrlOptionsSchema, 'DatasetClientCreateItemsUrlOptions'); - const { expiresInSecs, ...queryOptions } = parsed; + const dataset = await this.get({ timeoutSecs }); - let createdItemsPublicUrl = new URL(this._publicUrl('items')); + let createdItemsPublicUrl = new URL(this.buildPublicUrl('items')); if (dataset?.urlSigningSecretKey) { const signature = await createStorageContentSignatureAsync({ @@ -403,7 +435,7 @@ export class DatasetClient< return createdItemsPublicUrl.toString(); } - private _createPaginationList(response: ApifyResponse, userProvidedDesc: boolean): PaginatedList { + private createPaginationList(response: ApifyResponse, userProvidedDesc: boolean): PaginatedList { const page: PaginatedList = { items: response.data, total: Number(response.headers['x-apify-pagination-total']), @@ -444,7 +476,7 @@ export interface DatasetClientUpdateOptions { * Provides various filtering, pagination, and transformation options to customize * the output format and content of retrieved items. */ -export interface DatasetClientListItemOptions extends PaginationOptions { +export interface DatasetClientListItemOptions extends PaginationOptions, TimeoutOptions { clean?: boolean; desc?: boolean; /** diff --git a/src/resource_clients/dataset_collection.ts b/src/resource_clients/dataset_collection.ts index 525641428..b5a89462e 100644 --- a/src/resource_clients/dataset_collection.ts +++ b/src/resource_clients/dataset_collection.ts @@ -4,8 +4,10 @@ import { STORAGE_OWNERSHIP_FILTER } from '@apify/consts'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedList, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { optionalTimeoutSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, paginationOptionsShape, parseArgument } from '../utils.js'; import type { Dataset } from './dataset.js'; @@ -14,6 +16,7 @@ const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), ownership: z.enum(STORAGE_OWNERSHIP_FILTER).optional(), + ...timeoutOptionsShape, }); const nameSchema = z.string().optional(); const schemaSchema = anyObjectSchema.optional(); @@ -66,6 +69,7 @@ export class DatasetCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of Datasets. * @see https://docs.apify.com/api/v2/datasets-get */ @@ -74,7 +78,7 @@ export class DatasetCollectionClient extends ResourceCollectionClient { ): Promise & AsyncIterable { const parsed = parseArgument(options, listOptionsSchema, 'DatasetCollectionClientListOptions'); - return this._listPaginated(schemas.ListOfDatasets(), parsed); + return this.listResourcesPaginated(schemas.ListOfDatasets(), parsed, 'medium'); } /** @@ -82,18 +86,26 @@ export class DatasetCollectionClient extends ResourceCollectionClient { * * @param name - Name of the dataset. If not provided, a default dataset is used. * @param options - Additional options like schema. + * @param options.schema - Schema of the dataset. + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The dataset object. * @see https://docs.apify.com/api/v2/datasets-post */ async getOrCreate(name?: string, options?: DatasetCollectionClientGetOrCreateOptions): Promise { parseArgument(name, nameSchema); parseArgument(options?.schema, schemaSchema); // TODO: Add schema validation + parseArgument(options?.timeoutSecs, optionalTimeoutSchema); - return this._getOrCreate(schemas.Dataset(), name, options); + // `timeoutSecs` is not part of the resource, so the body carries only the rest, and stays absent when + // there is nothing else to send. + const { timeoutSecs = 'short', ...resource } = options ?? {}; + const hasResource = Object.keys(resource).length > 0; + + return this.getOrCreateResource(schemas.Dataset(), name, hasResource ? resource : undefined, timeoutSecs); } } -export interface DatasetCollectionClientListOptions extends PaginationOptions { +export interface DatasetCollectionClientListOptions extends PaginationOptions, TimeoutOptions { unnamed?: boolean; desc?: boolean; /** @@ -106,7 +118,7 @@ export interface DatasetCollectionClientListOptions extends PaginationOptions { /** * @since Added in 2.3.0 */ -export interface DatasetCollectionClientGetOrCreateOptions { +export interface DatasetCollectionClientGetOrCreateOptions extends TimeoutOptions { schema?: Record; } diff --git a/src/resource_clients/key_value_store.ts b/src/resource_clients/key_value_store.ts index 16b7e8aec..2cca0d770 100644 --- a/src/resource_clients/key_value_store.ts +++ b/src/resource_clients/key_value_store.ts @@ -9,15 +9,12 @@ import { createHmacSignatureAsync, createStorageContentSignatureAsync } from '@a import type { ApifyApiError } from '../apify_api_error.js'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; -import { - DEFAULT_TIMEOUT_MILLIS, - MEDIUM_TIMEOUT_MILLIS, - ResourceClient, - SMALL_TIMEOUT_MILLIS, -} from '../base/resource_client.js'; +import { ResourceClient } from '../base/resource_client.js'; import type { ApifyRequestConfig } from '../http_client.js'; import type { KeyValueClientListKeysResult, KeyValueListItem, KeyValueStore } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, applyQueryParamsToUrl, @@ -35,6 +32,7 @@ const listKeysOptionsSchema = z.strictObject({ collection: z.string().optional(), prefix: z.string().optional(), signature: z.string().optional(), + ...timeoutOptionsShape, }); const nonEmptyKeySchema = z.string().min(1); // `signature` is left out - this method produces one. The options type omits it to match. @@ -44,6 +42,7 @@ const createKeysPublicUrlOptionsSchema = z.strictObject({ collection: z.string().optional(), prefix: z.string().optional(), expiresInSecs: z.number().optional(), + ...timeoutOptionsShape, }); const keySchema = z.string(); const getRecordOptionsSchema = z.strictObject({ @@ -51,6 +50,7 @@ const getRecordOptionsSchema = z.strictObject({ stream: z.boolean().optional(), disableRedirect: z.boolean().optional(), signature: z.string().optional(), + ...timeoutOptionsShape, }); const recordSchema = z.strictObject({ key: z.string(), @@ -70,7 +70,7 @@ const recordSchema = z.strictObject({ contentType: z.string().min(1).optional(), }); const recordOptionsSchema = z.strictObject({ - timeoutSecs: z.number().optional(), + ...timeoutOptionsShape, doNotRetryTimeouts: z.boolean().optional(), }); @@ -118,11 +118,15 @@ export class KeyValueStoreClient extends ResourceClient { /** * Gets the key-value store object from the Apify API. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The KeyValueStore object, or `undefined` if it does not exist * @see https://docs.apify.com/api/v2/key-value-store-get */ - async get(): Promise { - return this._get(schemas.KeyValueStore(), {}, SMALL_TIMEOUT_MILLIS); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.KeyValueStore(), {}, timeoutSecs); } /** @@ -132,22 +136,29 @@ export class KeyValueStoreClient extends ResourceClient { * @param newFields.name - New name for the store * @param newFields.title - New title for the store * @param newFields.generalAccess - General resource access level ('FOLLOW_USER_SETTING', 'ANYONE_WITH_ID_CAN_READ' or 'RESTRICTED') + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @returns The updated KeyValueStore object * @see https://docs.apify.com/api/v2/key-value-store-put */ - async update(newFields: KeyValueClientUpdateOptions): Promise { + async update(newFields: KeyValueClientUpdateOptions, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'long' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.KeyValueStore(), newFields, DEFAULT_TIMEOUT_MILLIS); + return this.updateResource(schemas.KeyValueStore(), newFields, timeoutSecs); } /** * Deletes the key-value store. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/key-value-store-delete */ - async delete(): Promise { - return this._delete(SMALL_TIMEOUT_MILLIS); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** @@ -161,6 +172,7 @@ export class KeyValueStoreClient extends ResourceClient { * @param options.exclusiveStartKey - Key to start listing from (for pagination). The listing starts with the next key after this one. * @param options.collection - Filter keys by collection name. * @param options.prefix - Filter keys that start with this prefix. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns Object containing `items` array of key metadata, pagination info (`count`, `limit`, `isTruncated`, `nextExclusiveStartKey`) * @see https://docs.apify.com/api/v2/key-value-store-keys-get * @@ -188,16 +200,20 @@ export class KeyValueStoreClient extends ResourceClient { listKeys( options: KeyValueClientListKeysOptions = {}, ): Promise & AsyncIterable { - const parsed = parseArgument(options, listKeysOptionsSchema, 'KeyValueClientListKeysOptions'); + const { timeoutSecs = 'medium', ...parsed } = parseArgument( + options, + listKeysOptionsSchema, + 'KeyValueClientListKeysOptions', + ); const getPaginatedList = async ( kvsListOptions: KeyValueClientListKeysOptions = {}, ): Promise => { const response = await this.httpClient.call({ - url: this._url('keys'), + url: this.buildUrl('keys'), method: 'GET', - params: this._params(kvsListOptions), - timeout: MEDIUM_TIMEOUT_MILLIS, + params: this.buildParams(kvsListOptions), + timeoutSecs, }); return parseResponse(response, schemas.ListOfKeys()); @@ -243,6 +259,8 @@ export class KeyValueStoreClient extends ResourceClient { * requiring an API token. * * @param key - The record key + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request that fetches the store. Default is `'long'`. * @returns A public URL string for accessing the record * * @example @@ -253,12 +271,13 @@ export class KeyValueStoreClient extends ResourceClient { * ``` * @since Added in 2.14.0 */ - async getRecordPublicUrl(key: string): Promise { + async getRecordPublicUrl(key: string, options: TimeoutOptions = {}): Promise { parseArgument(key, nonEmptyKeySchema); + const { timeoutSecs = 'long' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - const store = await this.get(); + const store = await this.get({ timeoutSecs }); - const recordPublicUrl = new URL(this._publicUrl(['records', key])); + const recordPublicUrl = new URL(this.buildPublicUrl(['records', key])); if (store?.urlSigningSecretKey) { const signature = await createHmacSignatureAsync(store.urlSigningSecretKey, key); @@ -278,6 +297,7 @@ export class KeyValueStoreClient extends ResourceClient { * @param options.expiresInSecs - Number of seconds until the signed URL expires. If omitted, the URL never expires. * @param options.limit - Maximum number of keys to return. * @param options.prefix - Filter keys by prefix. + * @param options.timeoutSecs - Timeout for the API request that fetches the store. Default is `'long'`. * @returns A public URL string for accessing the keys list * * @example @@ -292,13 +312,15 @@ export class KeyValueStoreClient extends ResourceClient { * @since Added in 2.13.0 */ async createKeysPublicUrl(options: KeyValueClientCreateKeysUrlOptions = {}) { - const parsed = parseArgument(options, createKeysPublicUrlOptionsSchema, 'KeyValueClientCreateKeysUrlOptions'); + const { + timeoutSecs = 'long', + expiresInSecs, + ...queryOptions + } = parseArgument(options, createKeysPublicUrlOptionsSchema, 'KeyValueClientCreateKeysUrlOptions'); - const store = await this.get(); + const store = await this.get({ timeoutSecs }); - const { expiresInSecs, ...queryOptions } = parsed; - - let createdPublicKeysUrl = new URL(this._publicUrl('keys')); + let createdPublicKeysUrl = new URL(this.buildPublicUrl('keys')); if (store?.urlSigningSecretKey) { const signature = await createStorageContentSignatureAsync({ @@ -320,6 +342,8 @@ export class KeyValueStoreClient extends ResourceClient { * This is more efficient than {@link getRecord} when you only need to check for existence. * * @param key - The record key to check + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @returns `true` if the record exists, `false` if it does not * @see https://docs.apify.com/api/v2/key-value-store-record-get * @@ -332,11 +356,14 @@ export class KeyValueStoreClient extends ResourceClient { * ``` * @since Added in 2.9.0 */ - async recordExists(key: string): Promise { - const requestOpts: Record = { - url: this._url(['records', key]), + async recordExists(key: string, options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'long' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + const requestOpts: ApifyRequestConfig = { + url: this.buildUrl(['records', key]), method: 'HEAD', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }; try { @@ -357,6 +384,12 @@ export class KeyValueStoreClient extends ResourceClient { * When the record does not exist, the function resolves to `undefined`. It does * NOT resolve to a `KeyValueStore` record with an `undefined` value. * + * @param key - The record key + * @param options - Retrieval options + * @param options.buffer - If `true`, the value is returned as a Buffer (Node.js) or ArrayBuffer (browser). + * @param options.stream - If `true`, the value is returned as a Readable stream. Node.js only. + * @param options.signature - Signature of a public URL, for reading a record without a token. + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @see https://docs.apify.com/api/v2/key-value-store-record-get */ async getRecord(key: string): Promise | undefined>; @@ -387,11 +420,11 @@ export class KeyValueStoreClient extends ResourceClient { const queryParams: Record = { attachment: 'true' }; if (parsed.signature) queryParams.signature = parsed.signature; - const requestOpts: Record = { - url: this._url(['records', key]), + const requestOpts: ApifyRequestConfig = { + url: this.buildUrl(['records', key]), method: 'GET', - params: this._params(queryParams), - timeout: DEFAULT_TIMEOUT_MILLIS, + params: this.buildParams(queryParams), + timeoutSecs: parsed.timeoutSecs ?? 'long', }; if (parsed.buffer) requestOpts.forceBuffer = true; @@ -435,7 +468,7 @@ export class KeyValueStoreClient extends ResourceClient { * `'application/octet-stream'` fallback is treated as compressible, because * it could hold any binary data. * @param options - Storage options - * @param options.timeoutSecs - Timeout for the upload in seconds. Default varies by value size. + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @param options.doNotRetryTimeouts - If `true`, don't retry on timeout errors. Default is `false`. * @see https://docs.apify.com/api/v2/key-value-store-record-put * @@ -472,7 +505,7 @@ export class KeyValueStoreClient extends ResourceClient { const { key } = record; let { value, contentType } = record; - const { timeoutSecs, doNotRetryTimeouts } = parsed; + const { timeoutSecs = 'long', doNotRetryTimeouts } = parsed; const isValueStreamOrBuffer = isStream(value) || isBuffer(value); // To allow saving Objects to JSON without providing content type @@ -493,13 +526,13 @@ export class KeyValueStoreClient extends ResourceClient { } const uploadOpts: ApifyRequestConfig = { - url: this._url(['records', key]), + url: this.buildUrl(['records', key]), method: 'PUT', - params: this._params(), + params: this.buildParams(), data: value, headers: contentType ? { 'content-type': contentType } : undefined, doNotRetryTimeouts, - timeout: timeoutSecs !== undefined ? timeoutSecs * 1000 : DEFAULT_TIMEOUT_MILLIS, + timeoutSecs, }; await this.httpClient.call(uploadOpts); @@ -509,6 +542,8 @@ export class KeyValueStoreClient extends ResourceClient { * Deletes a record from the key-value store. * * @param key - The record key to delete + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/key-value-store-record-delete * * @example @@ -516,14 +551,15 @@ export class KeyValueStoreClient extends ResourceClient { * await client.keyValueStore('my-store').deleteRecord('temp-data'); * ``` */ - async deleteRecord(key: string): Promise { + async deleteRecord(key: string, options: TimeoutOptions = {}): Promise { parseArgument(key, keySchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); await this.httpClient.call({ - url: this._url(['records', key]), + url: this.buildUrl(['records', key]), method: 'DELETE', - params: this._params(), - timeout: SMALL_TIMEOUT_MILLIS, + params: this.buildParams(), + timeoutSecs, }); } } @@ -546,7 +582,7 @@ export interface KeyValueClientUpdateOptions { /** * Options for listing keys in a Key-Value Store. */ -export interface KeyValueClientListKeysOptions { +export interface KeyValueClientListKeysOptions extends TimeoutOptions { limit?: number; exclusiveStartKey?: string; collection?: string; @@ -574,7 +610,7 @@ export interface KeyValueClientCreateKeysUrlOptions extends Omit { * Options for storing a record in a Key-Value Store. * @since Added in 2.12.4 */ -export interface KeyValueStoreRecordOptions { - timeoutSecs?: number; +export interface KeyValueStoreRecordOptions extends TimeoutOptions { doNotRetryTimeouts?: boolean; } diff --git a/src/resource_clients/key_value_store_collection.ts b/src/resource_clients/key_value_store_collection.ts index fcb87672c..6522bf67b 100644 --- a/src/resource_clients/key_value_store_collection.ts +++ b/src/resource_clients/key_value_store_collection.ts @@ -4,8 +4,10 @@ import { STORAGE_OWNERSHIP_FILTER } from '@apify/consts'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedList, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { optionalTimeoutSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, paginationOptionsShape, parseArgument } from '../utils.js'; import type { KeyValueStore } from './key_value_store.js'; @@ -14,6 +16,7 @@ const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), ownership: z.enum(STORAGE_OWNERSHIP_FILTER).optional(), + ...timeoutOptionsShape, }); const nameSchema = z.string().optional(); const schemaSchema = anyObjectSchema.optional(); @@ -66,6 +69,7 @@ export class KeyValueStoreCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of Key-value stores. * @see https://docs.apify.com/api/v2/key-value-stores-get */ @@ -74,7 +78,7 @@ export class KeyValueStoreCollectionClient extends ResourceCollectionClient { ): Promise & AsyncIterable { const parsed = parseArgument(options, listOptionsSchema, 'KeyValueStoreCollectionClientListOptions'); - return this._listPaginated(schemas.ListOfKeyValueStores(), parsed); + return this.listResourcesPaginated(schemas.ListOfKeyValueStores(), parsed, 'medium'); } /** @@ -82,6 +86,8 @@ export class KeyValueStoreCollectionClient extends ResourceCollectionClient { * * @param name - Name of the key-value store. If not provided, a default store is used. * @param options - Additional options like schema. + * @param options.schema - Schema of the key-value store. + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The key-value store object. * @see https://docs.apify.com/api/v2/key-value-stores-post */ @@ -91,12 +97,18 @@ export class KeyValueStoreCollectionClient extends ResourceCollectionClient { ): Promise { parseArgument(name, nameSchema); parseArgument(options?.schema, schemaSchema); // TODO: Add schema validation + parseArgument(options?.timeoutSecs, optionalTimeoutSchema); - return this._getOrCreate(schemas.KeyValueStore(), name, options); + // `timeoutSecs` is not part of the resource, so the body carries only the rest, and stays absent when + // there is nothing else to send. + const { timeoutSecs = 'short', ...resource } = options ?? {}; + const hasResource = Object.keys(resource).length > 0; + + return this.getOrCreateResource(schemas.KeyValueStore(), name, hasResource ? resource : undefined, timeoutSecs); } } -export interface KeyValueStoreCollectionClientListOptions extends PaginationOptions { +export interface KeyValueStoreCollectionClientListOptions extends PaginationOptions, TimeoutOptions { unnamed?: boolean; desc?: boolean; /** @@ -109,7 +121,7 @@ export interface KeyValueStoreCollectionClientListOptions extends PaginationOpti /** * @since Added in 2.3.0 */ -export interface KeyValueStoreCollectionClientGetOrCreateOptions { +export interface KeyValueStoreCollectionClientGetOrCreateOptions extends TimeoutOptions { schema?: Record; } diff --git a/src/resource_clients/log.ts b/src/resource_clients/log.ts index f2a01f2e2..bb8d1b94a 100644 --- a/src/resource_clients/log.ts +++ b/src/resource_clients/log.ts @@ -2,6 +2,7 @@ import type { Readable } from 'node:stream'; import c from 'ansi-colors'; +import { z } from 'zod'; import type { Log } from '@apify/log'; import log, { Logger, LogLevel } from '@apify/log'; @@ -10,7 +11,11 @@ import type { ApifyApiError } from '../apify_api_error.js'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { ApifyRequestConfig } from '../http_client.js'; -import { cast, catchNotFoundForResourceOrThrow } from '../utils.js'; +import type { TimeoutOptions } from '../timeouts.js'; +import { timeoutOptionsShape } from '../timeouts.js'; +import { cast, catchNotFoundForResourceOrThrow, parseArgument } from '../utils.js'; + +const logOptionsSchema = z.strictObject({ raw: z.boolean().optional(), ...timeoutOptionsShape }); /** * Client for accessing Actor run or build logs. @@ -50,15 +55,19 @@ export class LogClient extends ResourceClient { * * @param options - Log retrieval options. * @param options.raw - If `true`, returns raw log content without any processing. Default is `false`. + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @returns The log content as a string, or `undefined` if it does not exist. A chained client such as * `run.log()` throws an `ApifyApiError` on a 404, since the run itself may be what is missing. * @see https://docs.apify.com/api/v2/log-get */ async get(options: LogOptions = {}): Promise { + const { timeoutSecs = 'long', ...params } = parseArgument(options, logOptionsSchema, 'LogOptions'); + const requestOpts: ApifyRequestConfig = { - url: this._url(), + url: this.buildUrl(), method: 'GET', - params: this._params(options), + params: this.buildParams(params), + timeoutSecs, }; try { @@ -76,21 +85,25 @@ export class LogClient extends ResourceClient { * * @param options - Log retrieval options. * @param options.raw - If `true`, returns raw log content without any processing. Default is `false`. + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @returns The log content as a Readable stream, or `undefined` if it does not exist. A chained client such as * `run.log()` throws an `ApifyApiError` on a 404, since the run itself may be what is missing. * @see https://docs.apify.com/api/v2/log-get */ async stream(options: LogOptions = {}): Promise { + const { timeoutSecs = 'long', raw } = parseArgument(options, logOptionsSchema, 'LogOptions'); + const params = { stream: true, - raw: options.raw, + raw, }; const requestOpts: ApifyRequestConfig = { - url: this._url(), + url: this.buildUrl(), method: 'GET', - params: this._params(params), + params: this.buildParams(params), responseType: 'stream', + timeoutSecs, }; try { @@ -107,7 +120,7 @@ export class LogClient extends ResourceClient { /** * @since Added in 2.20.0 */ -export interface LogOptions { +export interface LogOptions extends TimeoutOptions { /** @default false */ raw?: boolean; } diff --git a/src/resource_clients/request_queue.ts b/src/resource_clients/request_queue.ts index f6399c937..54e06ac69 100644 --- a/src/resource_clients/request_queue.ts +++ b/src/resource_clients/request_queue.ts @@ -7,7 +7,7 @@ import log from '@apify/log'; import type { ApifyApiError } from '../apify_api_error.js'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; -import { MEDIUM_TIMEOUT_MILLIS, ResourceClient, SMALL_TIMEOUT_MILLIS } from '../base/resource_client.js'; +import { ResourceClient } from '../base/resource_client.js'; import type { ApifyRequestConfig } from '../http_client.js'; import { ResponseValidationError } from '../response_validation_error.js'; import type { @@ -24,7 +24,9 @@ import type { RequestQueueClientRequestToUpdate, RequestQueueClientUnlockRequestsResult, } from '../models.js'; +import type { Timeout, TimeoutOptions, TimeoutTier } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { optionalTimeoutSchema, timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, cast, @@ -43,10 +45,11 @@ const DEFAULT_MIN_DELAY_BETWEEN_UNPROCESSED_REQUESTS_RETRIES_MILLIS = 500; const DEFAULT_REQUEST_QUEUE_REQUEST_PAGE_LIMIT = 1000; const SAFETY_BUFFER_PERCENT = 0.01 / 100; // 0.01% -const listHeadOptionsSchema = z.strictObject({ limit: z.number().min(0).optional() }); +const listHeadOptionsSchema = z.strictObject({ limit: z.number().min(0).optional(), ...timeoutOptionsShape }); const listAndLockHeadOptionsSchema = z.strictObject({ lockSecs: z.number(), limit: z.number().min(0).optional(), + ...timeoutOptionsShape, }); // Predicates, not `z.looseObject` arms: these run over a whole batch, and an object arm would copy // every key of every request. `id` is assigned by the API, so a new request must not carry one. @@ -54,7 +57,7 @@ const newRequestSchema = z.custom( (value) => isNonArrayObject(value) && value.id === undefined, 'Expected a request object without an `id`', ); -const forefrontOptionsSchema = z.strictObject({ forefront: z.boolean().optional() }); +const forefrontOptionsSchema = z.strictObject({ forefront: z.boolean().optional(), ...timeoutOptionsShape }); const batchAddRequestsSchema = z.array(newRequestSchema).min(1).max(REQUEST_QUEUE_MAX_REQUESTS_PER_BATCH_OPERATION); const batchAddRequestsWithRetriesSchema = z.array(newRequestSchema).min(1); const optionalBooleanSchema = z.boolean().optional(); @@ -75,18 +78,21 @@ const existingRequestSchema = z.custom( const prolongRequestLockOptionsSchema = z.strictObject({ lockSecs: z.number(), forefront: z.boolean().optional(), + ...timeoutOptionsShape, }); const requestFilterSchema = z.array(z.enum(['locked', 'pending'])).min(1); const listRequestsOptionsSchema = z.strictObject({ limit: z.number().min(0).optional(), cursor: z.string().optional(), filter: requestFilterSchema.optional(), + ...timeoutOptionsShape, }); const paginateRequestsOptionsSchema = z.strictObject({ limit: z.number().min(0).optional(), maxPageLimit: z.number().default(DEFAULT_REQUEST_QUEUE_REQUEST_PAGE_LIMIT), cursor: z.string().optional(), filter: requestFilterSchema.optional(), + ...timeoutOptionsShape, }); export type { @@ -141,7 +147,7 @@ export type { export class RequestQueueClient extends ResourceClient { private clientKey?: string; - private timeoutMillis?: number; + private timeoutSecs?: number; /** * @hidden @@ -153,39 +159,61 @@ export class RequestQueueClient extends ResourceClient { }); this.clientKey = userOptions.clientKey; - this.timeoutMillis = userOptions.timeoutSecs ? userOptions.timeoutSecs * 1e3 : undefined; + this.timeoutSecs = userOptions.timeoutSecs; + } + + /** + * Picks the timeout of a request: an explicit per-call `timeoutSecs` as is, otherwise the method's default + * tier, capped at the queue-wide `timeoutSecs` when one was given. + */ + private resolveTimeout(timeoutSecs: Timeout | undefined, defaultTier: TimeoutTier): Timeout { + if (timeoutSecs !== undefined) return timeoutSecs; + if (this.timeoutSecs === undefined) return defaultTier; + const tierSecs = this.httpClient.timeoutMillis[defaultTier] / 1000; + return tierSecs <= this.timeoutSecs ? defaultTier : this.timeoutSecs; } /** * Gets the Request queue object from the Apify API. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The RequestQueue object, or `undefined` if it does not exist * @see https://docs.apify.com/api/v2/request-queue-get */ - async get(): Promise { - return this._get(schemas.RequestQueue(), {}, SMALL_TIMEOUT_MILLIS); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.RequestQueue(), {}, this.resolveTimeout(timeoutSecs, 'short')); } /** * Updates the Request queue with specified fields. * * @param newFields - Fields to update in the Request queue + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated RequestQueue object * @see https://docs.apify.com/api/v2/request-queue-put */ - async update(newFields: RequestQueueClientUpdateOptions): Promise { + async update(newFields: RequestQueueClientUpdateOptions, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.RequestQueue(), newFields, SMALL_TIMEOUT_MILLIS); + return this.updateResource(schemas.RequestQueue(), newFields, this.resolveTimeout(timeoutSecs, 'short')); } /** * Deletes the Request queue. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/request-queue-delete */ - async delete(): Promise { - return this._delete(SMALL_TIMEOUT_MILLIS); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(this.resolveTimeout(timeoutSecs, 'short')); } /** @@ -195,6 +223,8 @@ export class RequestQueueClient extends ResourceClient { * inspecting what requests are waiting to be processed. * * @param options - Options for listing (e.g., limit) + * @param options.limit - Maximum number of requests to return. + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns List of requests from the queue head * @see https://docs.apify.com/api/v2/request-queue-head-get */ @@ -202,10 +232,10 @@ export class RequestQueueClient extends ResourceClient { const parsed = parseArgument(options, listHeadOptionsSchema, 'RequestQueueClientListHeadOptions'); const response = await this.httpClient.call({ - url: this._url('head'), + url: this.buildUrl('head'), method: 'GET', - timeout: Math.min(SMALL_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'short'), + params: this.buildParams({ limit: parsed.limit, clientKey: this.clientKey, }), @@ -226,6 +256,7 @@ export class RequestQueueClient extends ResourceClient { * @param options - Lock configuration * @param options.lockSecs - **Required.** Duration in seconds to lock the requests. After this time, the locks expire and requests can be retrieved by other clients. * @param options.limit - Maximum number of requests to return. Default is 25. + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns Object containing `items` (locked requests), `queueModifiedAt`, `hadMultipleClients`, and lock information * @see https://docs.apify.com/api/v2/request-queue-head-lock-post * @@ -253,10 +284,10 @@ export class RequestQueueClient extends ResourceClient { const parsed = parseArgument(options, listAndLockHeadOptionsSchema, 'RequestQueueClientListAndLockHeadOptions'); const response = await this.httpClient.call({ - url: this._url('head/lock'), + url: this.buildUrl('head/lock'), method: 'POST', - timeout: Math.min(MEDIUM_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'medium'), + params: this.buildParams({ limit: parsed.limit, lockSecs: parsed.lockSecs, clientKey: this.clientKey, @@ -282,6 +313,7 @@ export class RequestQueueClient extends ResourceClient { * @param request.payload - HTTP payload for POST/PUT requests (string). * @param options - Additional options * @param options.forefront - If `true`, adds the request to the beginning of the queue. Default is `false` (adds to the end). + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns Object with `requestId`, `wasAlreadyPresent`, and `wasAlreadyHandled` flags * @see https://docs.apify.com/api/v2/request-queue-requests-post * @@ -312,11 +344,11 @@ export class RequestQueueClient extends ResourceClient { const parsed = parseArgument(options, forefrontOptionsSchema, 'RequestQueueClientAddRequestOptions'); const response = await this.httpClient.call({ - url: this._url('requests'), + url: this.buildUrl('requests'), method: 'POST', - timeout: Math.min(SMALL_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'short'), data: request, - params: this._params({ + params: this.buildParams({ forefront: parsed.forefront, clientKey: this.clientKey, }), @@ -330,7 +362,7 @@ export class RequestQueueClient extends ResourceClient { * * @private */ - protected async _batchAddRequests( + protected async addRequestBatch( requests: RequestQueueClientRequestToAdd[], options: RequestQueueClientAddRequestOptions = {}, ): Promise { @@ -338,11 +370,11 @@ export class RequestQueueClient extends ResourceClient { const parsed = parseArgument(options, forefrontOptionsSchema, 'RequestQueueClientAddRequestOptions'); const response = await this.httpClient.call({ - url: this._url('requests/batch'), + url: this.buildUrl('requests/batch'), method: 'POST', - timeout: Math.min(MEDIUM_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'medium'), data: requests, - params: this._params({ + params: this.buildParams({ forefront: parsed.forefront, clientKey: this.clientKey, }), @@ -351,12 +383,13 @@ export class RequestQueueClient extends ResourceClient { return parseResponse(response, schemas.BatchAddResult()); } - protected async _batchAddRequestsWithRetries( + protected async addRequestBatchWithRetries( requests: RequestQueueClientRequestToAdd[], options: RequestQueueClientBatchAddRequestWithRetriesOptions = {}, ): Promise { const { forefront, + timeoutSecs, maxUnprocessedRequestsRetries = DEFAULT_UNPROCESSED_RETRIES_BATCH_ADD_REQUESTS, minDelayBetweenUnprocessedRequestsRetriesMillis = DEFAULT_MIN_DELAY_BETWEEN_UNPROCESSED_REQUESTS_RETRIES_MILLIS, } = options; @@ -369,8 +402,9 @@ export class RequestQueueClient extends ResourceClient { let unprocessedRequests: RequestQueueClientBatchRequestsOperationResult['unprocessedRequests'] = []; for (let i = 0; i < 1 + maxUnprocessedRequestsRetries; i++) { try { - const response = await this._batchAddRequests(remainingRequests, { + const response = await this.addRequestBatch(remainingRequests, { forefront, + timeoutSecs, }); processedRequests.push(...response.processedRequests); unprocessedRequests = response.unprocessedRequests; @@ -436,6 +470,7 @@ export class RequestQueueClient extends ResourceClient { * @param options.maxUnprocessedRequestsRetries - Maximum number of retry attempts for rate-limited requests. Default is 3. * @param options.maxParallel - Maximum number of parallel batch API calls. Default is 5. * @param options.minDelayBetweenUnprocessedRequestsRetriesMillis - Minimum delay before retrying rate-limited requests. Default is 500ms. + * @param options.timeoutSecs - Timeout for each batch API request. Default is `'medium'`. * @returns Object with `processedRequests` (successfully added) and `unprocessedRequests` (failed after all retries) * @see https://docs.apify.com/api/v2/request-queue-requests-batch-post * @@ -465,12 +500,14 @@ export class RequestQueueClient extends ResourceClient { ): Promise { const { forefront, + timeoutSecs, maxUnprocessedRequestsRetries = DEFAULT_UNPROCESSED_RETRIES_BATCH_ADD_REQUESTS, maxParallel = DEFAULT_PARALLEL_BATCH_ADD_REQUESTS, minDelayBetweenUnprocessedRequestsRetriesMillis = DEFAULT_MIN_DELAY_BETWEEN_UNPROCESSED_REQUESTS_RETRIES_MILLIS, } = options; parseArgument(requests, batchAddRequestsWithRetriesSchema); parseArgument(forefront, optionalBooleanSchema); + parseArgument(timeoutSecs, optionalTimeoutSchema); parseArgument(maxUnprocessedRequestsRetries, optionalNumberSchema); parseArgument(maxParallel, optionalNumberSchema); parseArgument(minDelayBetweenUnprocessedRequestsRetriesMillis, optionalNumberSchema); @@ -485,7 +522,7 @@ export class RequestQueueClient extends ResourceClient { while (i < requests.length) { const slicedRequests = requests.slice(i, i + REQUEST_QUEUE_MAX_REQUESTS_PER_BATCH_OPERATION); const requestsInBatch = sliceArrayByByteLength(slicedRequests, payloadSizeLimitBytes, i); - const requestPromise = this._batchAddRequestsWithRetries(requestsInBatch, options); + const requestPromise = this.addRequestBatchWithRetries(requestsInBatch, options); executingRequests.add(requestPromise); // A rejection reaches the caller through the awaits below; this bookkeeping chain only has to avoid // turning it into an unhandled one of its own. @@ -522,21 +559,25 @@ export class RequestQueueClient extends ResourceClient { * Requests can be identified by either their ID or unique key. * * @param requests - Array of requests to delete (by id or uniqueKey) + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns Result containing processed and unprocessed requests * @see https://docs.apify.com/api/v2/request-queue-requests-batch-delete * @since Added in 2.3.0 */ async batchDeleteRequests( requests: RequestQueueClientRequestToDelete[], + options: TimeoutOptions = {}, ): Promise { parseArgument(requests, batchDeleteRequestsSchema); + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); const response = await this.httpClient.call({ - url: this._url('requests/batch'), + url: this.buildUrl('requests/batch'), method: 'DELETE', - timeout: Math.min(SMALL_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), + timeoutSecs: this.resolveTimeout(timeoutSecs, 'short'), data: requests, - params: this._params({ + params: this.buildParams({ clientKey: this.clientKey, }), }); @@ -548,16 +589,23 @@ export class RequestQueueClient extends ResourceClient { * Gets a specific request from the queue by its ID. * * @param id - Request ID + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The request object, or `undefined` if not found * @see https://docs.apify.com/api/v2/request-queue-request-get */ - async getRequest(id: string): Promise { + async getRequest( + id: string, + options: TimeoutOptions = {}, + ): Promise { parseArgument(id, requestIdSchema); + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const requestOpts: ApifyRequestConfig = { - url: this._url(['requests', id]), + url: this.buildUrl(['requests', id]), method: 'GET', - timeout: Math.min(SMALL_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params(), + timeoutSecs: this.resolveTimeout(timeoutSecs, 'short'), + params: this.buildParams(), }; try { const response = await this.httpClient.call(requestOpts); @@ -574,6 +622,8 @@ export class RequestQueueClient extends ResourceClient { * * @param request - The updated request object (must include id) * @param options - Update options such as whether to move to front + * @param options.forefront - If `true`, moves the request to the beginning of the queue. Default is `false`. + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns Information about the updated request * @see https://docs.apify.com/api/v2/request-queue-request-put */ @@ -585,11 +635,11 @@ export class RequestQueueClient extends ResourceClient { const parsed = parseArgument(options, forefrontOptionsSchema, 'RequestQueueClientAddRequestOptions'); const response = await this.httpClient.call({ - url: this._url(['requests', request.id]), + url: this.buildUrl(['requests', request.id]), method: 'PUT', - timeout: Math.min(MEDIUM_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'medium'), data: request, - params: this._params({ + params: this.buildParams({ forefront: parsed.forefront, clientKey: this.clientKey, }), @@ -602,15 +652,18 @@ export class RequestQueueClient extends ResourceClient { * Deletes a specific request from the queue. * * @param id - Request ID + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. */ - async deleteRequest(id: string): Promise { + async deleteRequest(id: string, options: TimeoutOptions = {}): Promise { parseArgument(id, requestIdSchema); + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); await this.httpClient.call({ - url: this._url(['requests', id]), + url: this.buildUrl(['requests', id]), method: 'DELETE', - timeout: Math.min(SMALL_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: this.resolveTimeout(timeoutSecs, 'short'), + params: this.buildParams({ clientKey: this.clientKey, }), }); @@ -627,6 +680,7 @@ export class RequestQueueClient extends ResourceClient { * @param options - Lock extension options * @param options.lockSecs - **Required.** New lock duration in seconds from now. * @param options.forefront - If `true`, moves the request to the beginning of the queue when the lock expires. Default is `false`. + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns Object with new `lockExpiresAt` timestamp * @see https://docs.apify.com/api/v2/request-queue-request-lock-put * @@ -653,10 +707,10 @@ export class RequestQueueClient extends ResourceClient { ); const response = await this.httpClient.call({ - url: this._url(['requests', id, 'lock']), + url: this.buildUrl(['requests', id, 'lock']), method: 'PUT', - timeout: Math.min(MEDIUM_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'medium'), + params: this.buildParams({ forefront: parsed.forefront, lockSecs: parsed.lockSecs, clientKey: this.clientKey, @@ -674,6 +728,8 @@ export class RequestQueueClient extends ResourceClient { * * @param id - Request ID * @param options - Options such as whether to move to front + * @param options.forefront - If `true`, moves the request to the beginning of the queue. Default is `false`. + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/request-queue-request-lock-delete * @since Added in 2.4.1 */ @@ -682,10 +738,10 @@ export class RequestQueueClient extends ResourceClient { const parsed = parseArgument(options, forefrontOptionsSchema, 'RequestQueueClientDeleteRequestLockOptions'); await this.httpClient.call({ - url: this._url(['requests', id, 'lock']), + url: this.buildUrl(['requests', id, 'lock']), method: 'DELETE', - timeout: Math.min(SMALL_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: this.resolveTimeout(parsed.timeoutSecs, 'short'), + params: this.buildParams({ forefront: parsed.forefront, clientKey: this.clientKey, }), @@ -699,6 +755,7 @@ export class RequestQueueClient extends ResourceClient { * queue contents. * * @param options - Pagination options + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns List of requests with pagination information * @see https://docs.apify.com/api/v2/request-queue-requests-get * @since Added in 2.5.1 @@ -706,16 +763,22 @@ export class RequestQueueClient extends ResourceClient { listRequests( options: RequestQueueClientListRequestsOptions = {}, ): Promise & AsyncIterable { - const parsed = parseArgument(options, listRequestsOptionsSchema, 'RequestQueueClientListRequestsOptions'); + // `timeoutSecs` times every page request; it is not an API parameter, so it must not reach the query string. + const { timeoutSecs, ...parsed } = parseArgument( + options, + listRequestsOptionsSchema, + 'RequestQueueClientListRequestsOptions', + ); + const pageTimeout = this.resolveTimeout(timeoutSecs, 'medium'); const getPaginatedList = async ( rqListOptions: RequestQueueClientListRequestsOptions = {}, ): Promise => { const response = await this.httpClient.call({ - url: this._url('requests'), + url: this.buildUrl('requests'), method: 'GET', - timeout: Math.min(MEDIUM_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: pageTimeout, + params: this.buildParams({ ...rqListOptions, filter: rqListOptions.filter ? rqListOptions.filter.join(',') : undefined, clientKey: this.clientKey, @@ -763,16 +826,20 @@ export class RequestQueueClient extends ResourceClient { * This is useful for releasing all locks at once, for example when shutting down * a crawler gracefully. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'long'`. * @returns Number of requests that were unlocked * @see https://docs.apify.com/api/v2/request-queue-requests-unlock-post * @since Added in 2.12.5 */ - async unlockRequests(): Promise { + async unlockRequests(options: TimeoutOptions = {}): Promise { + const { timeoutSecs } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('requests/unlock'), + url: this.buildUrl('requests/unlock'), method: 'POST', - timeout: Math.min(MEDIUM_TIMEOUT_MILLIS, this.timeoutMillis ?? Infinity), - params: this._params({ + timeoutSecs: this.resolveTimeout(timeoutSecs, 'long'), + params: this.buildParams({ clientKey: this.clientKey, }), }); @@ -787,6 +854,7 @@ export class RequestQueueClient extends ResourceClient { * automatically handling pagination behind the scenes. * * @param options - Pagination options + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns An async iterable of request pages * @see https://docs.apify.com/api/v2/request-queue-requests-get * @@ -801,13 +869,13 @@ export class RequestQueueClient extends ResourceClient { paginateRequests( options: RequestQueueClientPaginateRequestsOptions = {}, ): RequestQueueRequestsAsyncIterable { - const { limit, cursor, filter, maxPageLimit } = parseArgument( + const { limit, cursor, filter, maxPageLimit, timeoutSecs } = parseArgument( options, paginateRequestsOptionsSchema, 'RequestQueueClientPaginateRequestsOptions', ); return new RequestQueuePaginationIterator({ - getPage: async (pageOptions) => this.listRequests({ ...pageOptions, filter }), + getPage: async (pageOptions) => this.listRequests({ ...pageOptions, filter, timeoutSecs }), limit, cursor, maxPageLimit, @@ -821,6 +889,8 @@ export class RequestQueueClient extends ResourceClient { export interface RequestQueueUserOptions { clientKey?: string; /** + * Cap, in seconds, on the default timeout tier of every request this client sends. An explicit per-call + * `timeoutSecs` is not capped. * @since Added in 2.2.0 */ timeoutSecs?: number; @@ -844,7 +914,7 @@ export interface RequestQueueClientUpdateOptions { /** * Options for listing requests from the queue head. */ -export interface RequestQueueClientListHeadOptions { +export interface RequestQueueClientListHeadOptions extends TimeoutOptions { limit?: number; } @@ -857,7 +927,7 @@ export type RequestQueueListRequestsFilter = 'locked' | 'pending'; * Options for listing all requests in the queue. * @since Added in 2.5.1 */ -export interface RequestQueueClientListRequestsOptions { +export interface RequestQueueClientListRequestsOptions extends TimeoutOptions { limit?: number; /** * @since Added in 2.23.2 @@ -873,7 +943,7 @@ export interface RequestQueueClientListRequestsOptions { * Options for paginating through requests in the queue. * @since Added in 2.5.1 */ -export interface RequestQueueClientPaginateRequestsOptions { +export interface RequestQueueClientPaginateRequestsOptions extends TimeoutOptions { limit?: number; maxPageLimit?: number; /** @@ -890,19 +960,19 @@ export interface RequestQueueClientPaginateRequestsOptions { * Options for listing and locking requests from the queue head. * @since Added in 2.4.1 */ -export interface RequestQueueClientListAndLockHeadOptions { +export interface RequestQueueClientListAndLockHeadOptions extends TimeoutOptions { lockSecs: number; limit?: number; } -export interface RequestQueueClientAddRequestOptions { +export interface RequestQueueClientAddRequestOptions extends TimeoutOptions { forefront?: boolean; } /** * @since Added in 2.4.1 */ -export interface RequestQueueClientProlongRequestLockOptions { +export interface RequestQueueClientProlongRequestLockOptions extends TimeoutOptions { forefront?: boolean; lockSecs: number; } @@ -910,14 +980,14 @@ export interface RequestQueueClientProlongRequestLockOptions { /** * @since Added in 2.4.1 */ -export interface RequestQueueClientDeleteRequestLockOptions { +export interface RequestQueueClientDeleteRequestLockOptions extends TimeoutOptions { forefront?: boolean; } /** * @since Added in 2.3.0 */ -export interface RequestQueueClientBatchAddRequestWithRetriesOptions { +export interface RequestQueueClientBatchAddRequestWithRetriesOptions extends TimeoutOptions { forefront?: boolean; maxUnprocessedRequestsRetries?: number; maxParallel?: number; diff --git a/src/resource_clients/request_queue_collection.ts b/src/resource_clients/request_queue_collection.ts index 8e16d57b9..c4340c8ae 100644 --- a/src/resource_clients/request_queue_collection.ts +++ b/src/resource_clients/request_queue_collection.ts @@ -4,8 +4,10 @@ import { STORAGE_OWNERSHIP_FILTER } from '@apify/consts'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedList, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { paginationOptionsShape, parseArgument } from '../utils.js'; import type { RequestQueue } from './request_queue.js'; @@ -14,6 +16,7 @@ const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), ownership: z.enum(STORAGE_OWNERSHIP_FILTER).optional(), + ...timeoutOptionsShape, }); const nameSchema = z.string().optional(); @@ -65,6 +68,7 @@ export class RequestQueueCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of Request queues. * @see https://docs.apify.com/api/v2/request-queues-get */ @@ -73,24 +77,27 @@ export class RequestQueueCollectionClient extends ResourceCollectionClient { ): Promise & AsyncIterable { const parsed = parseArgument(options, listOptionsSchema, 'RequestQueueCollectionListOptions'); - return this._listPaginated(schemas.ListOfRequestQueues(), parsed); + return this.listResourcesPaginated(schemas.ListOfRequestQueues(), parsed, 'medium'); } /** * Gets or creates a Request queue with the specified name. * * @param name - Name of the Request queue. If not provided, a default queue is used. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The Request queue object. * @see https://docs.apify.com/api/v2/request-queues-post */ - async getOrCreate(name?: string): Promise { + async getOrCreate(name?: string, options: TimeoutOptions = {}): Promise { parseArgument(name, nameSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._getOrCreate(schemas.RequestQueue(), name); + return this.getOrCreateResource(schemas.RequestQueue(), name, undefined, timeoutSecs); } } -export interface RequestQueueCollectionListOptions extends PaginationOptions { +export interface RequestQueueCollectionListOptions extends PaginationOptions, TimeoutOptions { unnamed?: boolean; desc?: boolean; /** diff --git a/src/resource_clients/run.ts b/src/resource_clients/run.ts index ebff18307..c707f193b 100644 --- a/src/resource_clients/run.ts +++ b/src/resource_clients/run.ts @@ -1,4 +1,3 @@ -import type { AxiosRequestConfig } from 'axios'; import { z } from 'zod'; import type { RUN_GENERAL_ACCESS } from '@apify/consts'; @@ -6,8 +5,10 @@ import { LEVELS, Log } from '@apify/log'; import type { ApiClientOptionsWithOptionalResourcePath } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; -import type { ApifyResponse } from '../http_client.js'; +import type { ApifyRequestConfig, ApifyResponse } from '../http_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { optionalTimeoutSchema, timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, isNode, parseArgument, parseResponse } from '../utils.js'; import type { ActorInput, ActorRun } from './actor.js'; import { DatasetClient } from './dataset.js'; @@ -17,27 +18,30 @@ import { RequestQueueClient } from './request_queue.js'; const RUN_CHARGE_IDEMPOTENCY_HEADER = 'idempotency-key'; -const getOptionsSchema = z.strictObject({ waitForFinish: z.number().optional() }); -const abortOptionsSchema = z.strictObject({ gracefully: z.boolean().optional() }); +const getOptionsSchema = z.strictObject({ waitForFinish: z.number().optional(), ...timeoutOptionsShape }); +const abortOptionsSchema = z.strictObject({ gracefully: z.boolean().optional(), ...timeoutOptionsShape }); const targetActorIdSchema = z.string(); const metamorphOptionsSchema = z.strictObject({ contentType: z.string().optional(), build: z.string().optional(), + ...timeoutOptionsShape, }); const resurrectOptionsSchema = z.strictObject({ build: z.string().optional(), memory: z.number().optional(), - timeout: z.number().optional(), + runTimeoutSecs: z.number().optional(), maxItems: z.number().optional(), maxTotalChargeUsd: z.number().optional(), restartOnError: z.boolean().optional(), + ...timeoutOptionsShape, }); const chargeOptionsSchema = z.strictObject({ eventName: z.string(), count: z.number().default(1), idempotencyKey: z.string().optional(), + ...timeoutOptionsShape, }); -const waitForFinishOptionsSchema = z.strictObject({ waitSecs: z.number().optional() }); +const waitForFinishOptionsSchema = z.strictObject({ waitSecs: z.number().optional(), ...timeoutOptionsShape }); /** * Client for managing a specific Actor run. @@ -78,6 +82,8 @@ export class RunClient extends ResourceClient { * * @param options - Get options * @param options.waitForFinish - Maximum time to wait (in seconds, max 60s) for the run to finish on the API side before returning. Default is 0 (returns immediately). + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`, extended to cover `waitForFinish` + * when the API is asked to hold the response. * @returns The ActorRun object, or `undefined` if it does not exist * @see https://docs.apify.com/api/v2/actor-run-get * @@ -92,9 +98,13 @@ export class RunClient extends ResourceClient { * ``` */ async get(options: RunGetOptions = {}): Promise { - const parsed = parseArgument(options, getOptionsSchema, 'RunGetOptions'); + const { timeoutSecs, ...params } = parseArgument(options, getOptionsSchema, 'RunGetOptions'); - return this._get(schemas.Run(), parsed); + return this.getResource( + schemas.Run(), + params, + this.timeoutForWaitForFinish(timeoutSecs, 'short', params.waitForFinish), + ); } /** @@ -102,6 +112,7 @@ export class RunClient extends ResourceClient { * * @param options - Abort options * @param options.gracefully - If `true`, the Actor run will abort gracefully - it can send status messages and perform cleanup. Default is `false` (immediate abort). + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The updated ActorRun object with `ABORTING` or `ABORTED` status * @see https://docs.apify.com/api/v2/actor-run-abort-post * @@ -115,12 +126,13 @@ export class RunClient extends ResourceClient { * ``` */ async abort(options: RunAbortOptions = {}): Promise { - const parsed = parseArgument(options, abortOptionsSchema, 'RunAbortOptions'); + const { timeoutSecs = 'medium', ...params } = parseArgument(options, abortOptionsSchema, 'RunAbortOptions'); const response = await this.httpClient.call({ - url: this._url('abort'), + url: this.buildUrl('abort'), method: 'POST', - params: this._params(parsed), + params: this.buildParams(params), + timeoutSecs, }); return parseResponse(response, schemas.Run()); @@ -129,11 +141,15 @@ export class RunClient extends ResourceClient { /** * Deletes the Actor run. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/actor-run-delete * @since Added in 2.8.1 */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** @@ -147,6 +163,7 @@ export class RunClient extends ResourceClient { * @param input - Input for the target Actor, serialized to JSON. Omit it to metamorph without input. * @param options - Metamorph options * @param options.build - Tag or number of the target Actor's build to run. Default is the target Actor's default build. + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The metamorphed ActorRun object (same ID, but now running the target Actor) * @see https://docs.apify.com/api/v2/actor-run-metamorph-post * @@ -164,23 +181,22 @@ export class RunClient extends ResourceClient { parseArgument(targetActorId, targetActorIdSchema); const parsed = parseArgument(options, metamorphOptionsSchema, 'RunMetamorphOptions'); - const safeTargetActorId = this._toSafeId(targetActorId); + const safeTargetActorId = this.toSafeId(targetActorId); const params = { targetActorId: safeTargetActorId, build: parsed.build, }; - const request: AxiosRequestConfig = { - url: this._url('metamorph'), + const request: ApifyRequestConfig = { + url: this.buildUrl('metamorph'), method: 'POST', data: input, - params: this._params(params), + params: this.buildParams(params), // Apify internal property. Tells the request serialization interceptor // to stringify functions to JSON, instead of omitting them. - // TODO: remove this ts-expect-error once we have defined custom Apify axios configs - // @ts-expect-error Custom Apify property stringifyFunctions: true, + timeoutSecs: parsed.timeoutSecs ?? 'medium', }; if (parsed.contentType) { @@ -200,6 +216,8 @@ export class RunClient extends ResourceClient { * This can be useful to recover from certain errors or to force the Actor to restart * with a fresh environment. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The updated ActorRun object * @see https://docs.apify.com/api/v2/actor-run-reboot-post * @@ -209,10 +227,13 @@ export class RunClient extends ResourceClient { * ``` * @since Added in 2.8.0 */ - async reboot(): Promise { - const request: AxiosRequestConfig = { - url: this._url('reboot'), + async reboot(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + const request: ApifyRequestConfig = { + url: this.buildUrl('reboot'), method: 'POST', + timeoutSecs, }; const response = await this.httpClient.call(request); @@ -226,6 +247,8 @@ export class RunClient extends ResourceClient { * @param newFields.statusMessage - Custom status message to display (e.g., "Processing page 10/100") * @param newFields.isStatusMessageTerminal - If `true`, the status message is final and won't be overwritten. Default is `false`. * @param newFields.generalAccess - General resource access level ('FOLLOW_USER_SETTING', 'ANYONE_WITH_ID_CAN_READ' or 'RESTRICTED') + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated ActorRun object * * @example @@ -237,10 +260,11 @@ export class RunClient extends ResourceClient { * ``` * @since Added in 2.6.0 */ - async update(newFields: RunUpdateOptions): Promise { + async update(newFields: RunUpdateOptions, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.Run(), newFields); + return this.updateResource(schemas.Run(), newFields, timeoutSecs); } /** @@ -252,10 +276,11 @@ export class RunClient extends ResourceClient { * @param options - Resurrection options (override original run settings) * @param options.build - Tag or number of the build to use. If not provided, uses the original run's build. * @param options.memory - Memory in megabytes. If not provided, uses the original run's memory. - * @param options.timeout - Timeout in seconds. If not provided, uses the original run's timeout. + * @param options.runTimeoutSecs - Timeout for the resurrected run in seconds. If not provided, uses the original run's timeout. * @param options.maxItems - Maximum number of dataset items (pay-per-result Actors). * @param options.maxTotalChargeUsd - Maximum cost in USD (pay-per-event Actors). * @param options.restartOnError - Whether to restart on error. + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The new (resurrected) ActorRun object * @see https://docs.apify.com/api/v2/post-resurrect-run * @@ -267,12 +292,18 @@ export class RunClient extends ResourceClient { * ``` */ async resurrect(options: RunResurrectOptions = {}): Promise { - const parsed = parseArgument(options, resurrectOptionsSchema, 'RunResurrectOptions'); + const { + timeoutSecs = 'medium', + runTimeoutSecs, + ...params + } = parseArgument(options, resurrectOptionsSchema, 'RunResurrectOptions'); const response = await this.httpClient.call({ - url: this._url('resurrect'), + url: this.buildUrl('resurrect'), method: 'POST', - params: this._params(parsed), + // The API's `timeout` parameter bounds the run, not the request. + params: this.buildParams({ ...params, timeout: runTimeoutSecs }), + timeoutSecs, }); return parseResponse(response, schemas.Run()); @@ -285,6 +316,7 @@ export class RunClient extends ResourceClient { * @param options.eventName - **Required.** Name of the event to charge for. * @param options.count - Number of times to charge the event. Default is 1. * @param options.idempotencyKey - Optional key to ensure the charge is not duplicated. If not provided, one is auto-generated. + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns Empty response object. * @see https://docs.apify.com/api/v2/post-charge-run * @since Added in 2.11.0 @@ -294,14 +326,15 @@ export class RunClient extends ResourceClient { eventName, count, idempotencyKey: providedIdempotencyKey, + timeoutSecs = 'short', } = parseArgument(options, chargeOptionsSchema, 'RunChargeOptions'); /** To avoid duplicates during the same milisecond, doesn't need to by crypto-secure. */ const randomSuffix = (Math.random() + 1).toString(36).slice(3, 8); const idempotencyKey = providedIdempotencyKey ?? `${this.id}-${eventName}-${Date.now()}-${randomSuffix}`; - const request: AxiosRequestConfig = { - url: this._url('charge'), + const request: ApifyRequestConfig = { + url: this.buildUrl('charge'), method: 'POST', data: { eventName, @@ -310,6 +343,7 @@ export class RunClient extends ResourceClient { headers: { [RUN_CHARGE_IDEMPOTENCY_HEADER]: idempotencyKey, }, + timeoutSecs, }; const response = await this.httpClient.call(request); return response; @@ -328,7 +362,8 @@ export class RunClient extends ResourceClient { * * @param options - Wait options * @param options.waitSecs - Maximum time to wait for the run to finish, in seconds. If the limit is reached, the returned promise resolves to a run object that will have status `READY` or `RUNNING`. If omitted, waits indefinitely. - * @returns The ActorRun object (finished or still running if timeout was reached) + * @param options.timeoutSecs - Timeout for each polling API request. Default is `'noTimeout'`. + * @returns The ActorRun object (finished or still running if the timeout was reached) * * @example * ```javascript @@ -346,7 +381,7 @@ export class RunClient extends ResourceClient { async waitForFinish(options: RunWaitForFinishOptions = {}): Promise { const parsed = parseArgument(options, waitForFinishOptionsSchema, 'RunWaitForFinishOptions'); - return this._waitForFinish(schemas.Run(), parsed); + return this.waitForJobFinish(schemas.Run(), parsed); } /** @@ -365,7 +400,7 @@ export class RunClient extends ResourceClient { */ dataset(): DatasetClient { return new DatasetClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'dataset', }), ); @@ -388,7 +423,7 @@ export class RunClient extends ResourceClient { */ keyValueStore(): KeyValueStoreClient { return new KeyValueStoreClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'key-value-store', }), ); @@ -411,7 +446,7 @@ export class RunClient extends ResourceClient { */ requestQueue(): RequestQueueClient { return new RequestQueueClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'request-queue', }), ); @@ -434,7 +469,7 @@ export class RunClient extends ResourceClient { */ log(): LogClient { return new LogClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'log', }), ); @@ -442,10 +477,17 @@ export class RunClient extends ResourceClient { /** * Get StreamedLog for convenient streaming of the run log and their redirection. + * + * @param options - Streaming options + * @param options.toLog - Log instance to redirect the run log to. Use `'default'` for a preconfigured one, or `null` to disable the redirection. + * @param options.fromStart - If `true`, the log is streamed from its beginning. Default is `true`. + * @param options.timeoutSecs - Timeout for the API requests that fetch the run and its Actor. Default is `'long'`. * @since Added in 2.20.0 */ async getStreamedLog(options: GetStreamedLogOptions = {}): Promise { - const { fromStart = true } = options; + parseArgument(options.timeoutSecs, optionalTimeoutSchema); + + const { fromStart = true, timeoutSecs = 'long' } = options; let { toLog } = options; if (toLog === null || !isNode()) { // Explicitly no logging or not in Node.js @@ -454,12 +496,12 @@ export class RunClient extends ResourceClient { if (toLog === undefined || toLog === 'default') { // Create default StreamedLog // Get actor name and run id - const runData = await this.get(); + const runData = await this.get({ timeoutSecs }); const runId = runData?.id ?? ''; const actorId = runData?.actId ?? ''; // `apifyClient.actor()` rejects an empty ID, which is what a run that could not be read leaves here. - const actorData = actorId ? await this.apifyClient.actor(actorId).get() : undefined; + const actorData = actorId ? await this.apifyClient.actor(actorId).get({ timeoutSecs }) : undefined; const actorName = actorData?.name ?? ''; const name = [actorName, `runId:${runId}`].filter(Boolean).join(' '); @@ -474,7 +516,7 @@ export class RunClient extends ResourceClient { * Options for getting a streamed log. * @since Added in 2.20.0 */ -export interface GetStreamedLogOptions { +export interface GetStreamedLogOptions extends TimeoutOptions { toLog?: Log | null | 'default'; fromStart?: boolean; } @@ -482,21 +524,21 @@ export interface GetStreamedLogOptions { /** * Options for getting a Run. */ -export interface RunGetOptions { +export interface RunGetOptions extends TimeoutOptions { waitForFinish?: number; } /** * Options for aborting a Run. */ -export interface RunAbortOptions { +export interface RunAbortOptions extends TimeoutOptions { gracefully?: boolean; } /** * Options for metamorphing a Run into another Actor. */ -export interface RunMetamorphOptions { +export interface RunMetamorphOptions extends TimeoutOptions { /** * Content type of the request body, which becomes the content type of the run's `INPUT` record. * Without it, an input is serialized to JSON and sent as `application/json`. Pairing an object @@ -525,10 +567,11 @@ export interface RunUpdateOptions { /** * Options for resurrecting a finished Run. */ -export interface RunResurrectOptions { +export interface RunResurrectOptions extends TimeoutOptions { build?: string; memory?: number; - timeout?: number; + /** Timeout for the resurrected run in seconds. If not provided, the run keeps its original timeout. */ + runTimeoutSecs?: number; /** * @since Added in 2.12.1 */ @@ -547,7 +590,7 @@ export interface RunResurrectOptions { * Options for charging events in a pay-per-event Actor run. * @since Added in 2.11.0 */ -export interface RunChargeOptions { +export interface RunChargeOptions extends TimeoutOptions { /** Name of the event to charge. Must be defined in the Actor's pricing info else the API will throw. */ eventName: string; /** Defaults to 1 */ @@ -559,7 +602,7 @@ export interface RunChargeOptions { /** * Options for waiting for a Run to finish. */ -export interface RunWaitForFinishOptions { +export interface RunWaitForFinishOptions extends TimeoutOptions { /** * Maximum time to wait for the run to finish, in seconds. * If the limit is reached, the returned promise is resolved to a run object that will have diff --git a/src/resource_clients/run_collection.ts b/src/resource_clients/run_collection.ts index 9b051e397..7dba75f77 100644 --- a/src/resource_clients/run_collection.ts +++ b/src/resource_clients/run_collection.ts @@ -4,8 +4,10 @@ import { ACTOR_JOB_STATUSES } from '@apify/consts'; import type { ApiClientOptionsWithOptionalResourcePath } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsShape } from '../timeouts.js'; import { paginationOptionsShape, parseArgument } from '../utils.js'; import type { ActorRunListItem } from './actor.js'; @@ -16,6 +18,7 @@ const listOptionsSchema = z.strictObject({ status: z.union([jobStatusSchema, z.array(jobStatusSchema)]).optional(), startedBefore: z.union([z.date(), z.string()]).optional(), startedAfter: z.union([z.date(), z.string()]).optional(), + ...timeoutOptionsShape, }); /** @@ -67,17 +70,18 @@ export class RunCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination and filtering options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of Actor runs. * @see https://docs.apify.com/api/v2/actor-runs-get */ list(options: RunCollectionListOptions = {}): PaginatedIterator { const parsed = parseArgument(options, listOptionsSchema, 'RunCollectionListOptions'); - return this._listPaginated(schemas.ListOfRuns(), parsed); + return this.listResourcesPaginated(schemas.ListOfRuns(), parsed, 'medium'); } } -export interface RunCollectionListOptions extends PaginationOptions { +export interface RunCollectionListOptions extends PaginationOptions, TimeoutOptions { desc?: boolean; status?: | (typeof ACTOR_JOB_STATUSES)[keyof typeof ACTOR_JOB_STATUSES] diff --git a/src/resource_clients/schedule.ts b/src/resource_clients/schedule.ts index 759a5f443..7a761bb4d 100644 --- a/src/resource_clients/schedule.ts +++ b/src/resource_clients/schedule.ts @@ -3,8 +3,10 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { Schedule, ScheduleAction, ScheduleInvoked } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { DistributiveOptional } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; import { anyObjectSchema, parseArgument, parseResponse } from '../utils.js'; export type { @@ -57,45 +59,62 @@ export class ScheduleClient extends ResourceClient { /** * Retrieves the schedule. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The schedule object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/schedule-get */ - async get(): Promise { - return this._get(schemas.Schedule()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.Schedule(), {}, timeoutSecs); } /** * Updates the schedule with the specified fields. * * @param newFields - Fields to update. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated schedule object. * @see https://docs.apify.com/api/v2/schedule-put */ - async update(newFields: ScheduleCreateOrUpdateData): Promise { + async update(newFields: ScheduleCreateOrUpdateData, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); - return this._update(schemas.Schedule(), newFields); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.updateResource(schemas.Schedule(), newFields, timeoutSecs); } /** * Deletes the schedule. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/schedule-delete */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** * Retrieves the schedule's log. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The schedule log, one entry per invocation. * @see https://docs.apify.com/api/v2/schedule-log-get */ - async getLog(): Promise { + async getLog(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('log'), + url: this.buildUrl('log'), method: 'GET', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); return parseResponse(response, scheduleLogSchema); } diff --git a/src/resource_clients/schedule_collection.ts b/src/resource_clients/schedule_collection.ts index 2c8c70d2c..819df081a 100644 --- a/src/resource_clients/schedule_collection.ts +++ b/src/resource_clients/schedule_collection.ts @@ -2,14 +2,17 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, paginationOptionsShape, parseArgument } from '../utils.js'; import type { Schedule, ScheduleCreateOrUpdateData } from './schedule.js'; const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), + ...timeoutOptionsShape, }); const scheduleCreateSchema = anyObjectSchema.optional(); @@ -65,29 +68,33 @@ export class ScheduleCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination and sorting options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of schedules. * @see https://docs.apify.com/api/v2/schedules-get */ list(options: ScheduleCollectionListOptions = {}): PaginatedIterator { const parsed = parseArgument(options, listOptionsSchema, 'ScheduleCollectionListOptions'); - return this._listPaginated(schemas.ListOfSchedules(), parsed); + return this.listResourcesPaginated(schemas.ListOfSchedules(), parsed, 'medium'); } /** * Creates a new schedule. * * @param schedule - The schedule data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The created schedule object. * @see https://docs.apify.com/api/v2/schedules-post */ - async create(schedule?: ScheduleCreateOrUpdateData): Promise { + async create(schedule?: ScheduleCreateOrUpdateData, options: TimeoutOptions = {}): Promise { parseArgument(schedule, scheduleCreateSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._create(schemas.Schedule(), schedule); + return this.createResource(schemas.Schedule(), schedule, timeoutSecs); } } -export interface ScheduleCollectionListOptions extends PaginationOptions { +export interface ScheduleCollectionListOptions extends PaginationOptions, TimeoutOptions { desc?: boolean; } diff --git a/src/resource_clients/store_collection.ts b/src/resource_clients/store_collection.ts index 9244f58b2..add97a8d7 100644 --- a/src/resource_clients/store_collection.ts +++ b/src/resource_clients/store_collection.ts @@ -3,8 +3,10 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; import type { ActorStoreList } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsShape } from '../timeouts.js'; import { paginationOptionsShape, parseArgument } from '../utils.js'; export type { ActorStoreList, PricingInfo } from '../models.js'; @@ -17,6 +19,7 @@ const listOptionsSchema = z.strictObject({ username: z.string().optional(), pricingModel: z.string().optional(), includeUnrunnableActors: z.boolean().optional(), + ...timeoutOptionsShape, }); /** @@ -68,20 +71,21 @@ export class StoreCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Search and pagination options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of store Actors. * @see https://docs.apify.com/api/v2/store-get */ list(options: StoreCollectionListOptions = {}): PaginatedIterator { const parsed = parseArgument(options, listOptionsSchema, 'StoreCollectionListOptions'); - return this._listPaginated(schemas.ListOfStoreActors(), parsed); + return this.listResourcesPaginated(schemas.ListOfStoreActors(), parsed, 'medium'); } } /** * @since Added in 2.7.2 */ -export interface StoreCollectionListOptions extends PaginationOptions { +export interface StoreCollectionListOptions extends PaginationOptions, TimeoutOptions { search?: string; sortBy?: string; category?: string; diff --git a/src/resource_clients/task.ts b/src/resource_clients/task.ts index 4ade81ba0..7ac828dba 100644 --- a/src/resource_clients/task.ts +++ b/src/resource_clients/task.ts @@ -6,8 +6,10 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { ApifyRequestConfig } from '../http_client.js'; import type { Task, TaskPublicConfig } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { Dictionary } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, cast, parseArgument, parseResponse, stringifyWebhooksToBase64 } from '../utils.js'; import type { ActorLastRunOptions, ActorRun, ActorStartOptions } from './actor.js'; import { RunClient } from './run.js'; @@ -18,22 +20,24 @@ const inputSchema = anyObjectSchema.optional(); const startOptionsSchema = z.strictObject({ build: z.string().optional(), memory: z.number().optional(), - timeout: z.number().optional(), + runTimeoutSecs: z.number().min(0).optional(), waitForFinish: z.number().optional(), webhooks: z.array(anyObjectSchema).optional(), maxItems: z.number().min(0).optional(), maxTotalChargeUsd: z.number().min(0).optional(), restartOnError: z.boolean().optional(), + ...timeoutOptionsShape, }); const callOptionsSchema = z.strictObject({ build: z.string().optional(), memory: z.number().optional(), - timeout: z.number().min(0).optional(), + runTimeoutSecs: z.number().min(0).optional(), waitSecs: z.number().min(0).optional(), webhooks: z.array(anyObjectSchema).optional(), maxItems: z.number().min(0).optional(), maxTotalChargeUsd: z.number().min(0).optional(), restartOnError: z.boolean().optional(), + ...timeoutOptionsShape, }); const lastRunOptionsSchema = z.strictObject({ status: z.enum(ACT_JOB_STATUSES).optional(), @@ -76,24 +80,31 @@ export class TaskClient extends ResourceClient { /** * Retrieves the Actor task. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The task object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/actor-task-get */ - async get(): Promise { - return this._get(schemas.Task()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.Task(), {}, timeoutSecs); } /** * Updates the task with the specified fields. * * @param newFields - Fields to update. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated task object. * @see https://docs.apify.com/api/v2/actor-task-put */ - async update(newFields: TaskUpdateData): Promise { + async update(newFields: TaskUpdateData, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.Task(), newFields); + return this.updateResource(schemas.Task(), newFields, timeoutSecs); } /** @@ -104,12 +115,14 @@ export class TaskClient extends ResourceClient { * (`publicConfig`) set up first. Requires write permission to both the task and its Actor. * Publishing an already published task does nothing. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The task object. * @see https://docs.apify.com/api/v2/actor-task-put * @since Added in 2.25.0 */ - async publish(): Promise { - return this.update({ isPublic: true }); + async publish(options: TimeoutOptions = {}): Promise { + return this.update({ isPublic: true }, options); } /** @@ -120,21 +133,27 @@ export class TaskClient extends ResourceClient { * published again without re-entering it. Requires write permission to both the task and its * Actor. Unpublishing a task that is not published does nothing. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The task object. * @see https://docs.apify.com/api/v2/actor-task-put * @since Added in 2.25.0 */ - async unpublish(): Promise { - return this.update({ isPublic: false }); + async unpublish(options: TimeoutOptions = {}): Promise { + return this.update({ isPublic: false }, options); } /** * Deletes the Task. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/actor-task-delete */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** @@ -144,12 +163,14 @@ export class TaskClient extends ResourceClient { * @param options - Run options. * @param options.build - Tag or number of the Actor build to run (e.g., `'beta'` or `'1.2.345'`). * @param options.memory - Memory in megabytes allocated for the run. - * @param options.timeout - Timeout for the run in seconds. Zero means no timeout. + * @param options.runTimeoutSecs - Timeout for the run in seconds. Zero means no timeout. * @param options.waitForFinish - Maximum time to wait (in seconds, max 60s) for the run to finish before returning. * @param options.webhooks - Webhooks to trigger for specific Actor run events. * @param options.maxItems - Maximum number of dataset items (for pay-per-result Actors). * @param options.maxTotalChargeUsd - Maximum cost in USD (for pay-per-event Actors). * @param options.restartOnError - Whether to restart the run on error. + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`, extended to cover `waitForFinish` + * when the API is asked to hold the response. * @returns The Actor Run object. * @see https://docs.apify.com/api/v2/actor-task-runs-post */ @@ -157,11 +178,21 @@ export class TaskClient extends ResourceClient { parseArgument(input, inputSchema); const parsed = parseArgument(options, startOptionsSchema, 'TaskStartOptions'); - const { waitForFinish, timeout, memory, build, maxItems, maxTotalChargeUsd, restartOnError } = parsed; + const { + waitForFinish, + runTimeoutSecs, + memory, + build, + maxItems, + maxTotalChargeUsd, + restartOnError, + timeoutSecs, + } = parsed; + // The API's `timeout` parameter bounds the run, not the request. const params = { waitForFinish, - timeout, + timeout: runTimeoutSecs, memory, build, webhooks: stringifyWebhooksToBase64(parsed.webhooks), @@ -171,16 +202,17 @@ export class TaskClient extends ResourceClient { }; const request: ApifyRequestConfig = { - url: this._url('runs'), + url: this.buildUrl('runs'), method: 'POST', data: input, - params: this._params(params), + params: this.buildParams(params), // Apify internal property. Tells the request serialization interceptor // to stringify functions to JSON, instead of omitting them. stringifyFunctions: true, headers: { 'Content-Type': 'application/json', }, + timeoutSecs: this.timeoutForWaitForFinish(timeoutSecs, 'medium', waitForFinish), }; const response = await this.httpClient.call(request); @@ -195,12 +227,13 @@ export class TaskClient extends ResourceClient { * @param options - Run and wait options. * @param options.build - Tag or number of the Actor build to run. * @param options.memory - Memory in megabytes allocated for the run. - * @param options.timeout - Timeout for the run in seconds. + * @param options.runTimeoutSecs - Timeout for the run in seconds. * @param options.waitSecs - Maximum time to wait for the run to finish, in seconds. If omitted, waits indefinitely. * @param options.webhooks - Webhooks to trigger for specific Actor run events. * @param options.maxItems - Maximum number of dataset items (for pay-per-result Actors). * @param options.maxTotalChargeUsd - Maximum cost in USD (for pay-per-event Actors). * @param options.restartOnError - Whether to restart the run on error. + * @param options.timeoutSecs - Timeout for each API request, the start and every poll alike. Default is `'noTimeout'`. * @returns The Actor run object. * @see https://docs.apify.com/api/v2/actor-task-runs-post */ @@ -208,27 +241,32 @@ export class TaskClient extends ResourceClient { parseArgument(input, inputSchema); const parsed = parseArgument(options, callOptionsSchema, 'TaskCallOptions'); - const { waitSecs, ...startOptions } = parsed; + const { waitSecs, timeoutSecs = 'noTimeout', ...startOptions } = parsed; - const { id } = await this.start(input, startOptions); + const { id } = await this.start(input, { ...startOptions, timeoutSecs }); // Calling root client because we need access to top level API. // Creating a new instance of RunClient here would only allow // setting it up as a nested route under task API. - return this.apifyClient.run(id).waitForFinish({ waitSecs }); + return this.apifyClient.run(id).waitForFinish({ waitSecs, timeoutSecs }); } /** * Retrieves the Actor task's input object. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The Task's input. * @see https://docs.apify.com/api/v2/actor-task-input-get */ - async getInput(): Promise { + async getInput(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('input'), + url: this.buildUrl('input'), method: 'GET', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); return cast(response.data); } @@ -237,15 +275,23 @@ export class TaskClient extends ResourceClient { * Updates the Actor task's input object. * * @param newFields - New input data for the task. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated task input. * @see https://docs.apify.com/api/v2/actor-task-input-put */ - async updateInput(newFields: Dictionary | Dictionary[]): Promise { + async updateInput( + newFields: Dictionary | Dictionary[], + options: TimeoutOptions = {}, + ): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('input'), + url: this.buildUrl('input'), method: 'PUT', - params: this._params(), + params: this.buildParams(), data: newFields, + timeoutSecs, }); return cast(response.data); @@ -264,9 +310,9 @@ export class TaskClient extends ResourceClient { const parsed = parseArgument(options, lastRunOptionsSchema, 'TaskLastRunOptions'); return new RunClient( - this._subResourceOptions({ + this.subResourceOptions({ id: 'last', - params: this._params(parsed), + params: this.buildParams(parsed), resourcePath: 'runs', }), ); @@ -280,7 +326,7 @@ export class TaskClient extends ResourceClient { */ runs(): RunCollectionClient { return new RunCollectionClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'runs', }), ); @@ -293,7 +339,7 @@ export class TaskClient extends ResourceClient { * @see https://docs.apify.com/api/v2/actor-task-webhooks-get */ webhooks(): WebhookCollectionClient { - return new WebhookCollectionClient(this._subResourceOptions()); + return new WebhookCollectionClient(this.subResourceOptions()); } } diff --git a/src/resource_clients/task_collection.ts b/src/resource_clients/task_collection.ts index cb4b2c9bf..069b7d9e8 100644 --- a/src/resource_clients/task_collection.ts +++ b/src/resource_clients/task_collection.ts @@ -3,14 +3,17 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; import type { TaskList } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, paginationOptionsShape, parseArgument } from '../utils.js'; import type { Task, TaskUpdateData } from './task.js'; const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), + ...timeoutOptionsShape, }); export type { TaskList } from '../models.js'; @@ -67,30 +70,34 @@ export class TaskCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination and sorting options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of tasks. * @see https://docs.apify.com/api/v2/actor-tasks-get */ list(options: TaskCollectionListOptions = {}): PaginatedIterator { const parsed = parseArgument(options, listOptionsSchema, 'TaskCollectionListOptions'); - return this._listPaginated(schemas.ListOfTasks(), parsed); + return this.listResourcesPaginated(schemas.ListOfTasks(), parsed, 'medium'); } /** * Creates a new task. * * @param task - The task data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The created task object. * @see https://docs.apify.com/api/v2/actor-tasks-post */ - async create(task: TaskCreateData): Promise { + async create(task: TaskCreateData, options: TimeoutOptions = {}): Promise { parseArgument(task, anyObjectSchema); + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._create(schemas.Task(), task); + return this.createResource(schemas.Task(), task, timeoutSecs); } } -export interface TaskCollectionListOptions extends PaginationOptions { +export interface TaskCollectionListOptions extends PaginationOptions, TimeoutOptions { desc?: boolean; } diff --git a/src/resource_clients/user.ts b/src/resource_clients/user.ts index 9d9fc0969..decf59e32 100644 --- a/src/resource_clients/user.ts +++ b/src/resource_clients/user.ts @@ -2,8 +2,10 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { ApifyRequestConfig } from '../http_client.js'; import type { AccountAndUsageLimits, MonthlyUsage, User } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; -import { parseResponse } from '../utils.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; +import { parseArgument, parseResponse } from '../utils.js'; export type { AccountAndUsageLimits, @@ -67,25 +69,34 @@ export class UserClient extends ResourceClient { * Depending on whether ApifyClient was created with a token, * the method will either return public or private user data. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The user object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/user-get */ - async get(): Promise { - return this._get(schemas.UserPrivateInfo()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.UserPrivateInfo(), {}, timeoutSecs); } /** * Retrieves the user's monthly usage data. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The monthly usage object. * @see https://docs.apify.com/api/v2/users-me-usage-monthly-get * @since Added in 2.9.2 */ - async monthlyUsage(): Promise { + async monthlyUsage(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('usage/monthly'), + url: this.buildUrl('usage/monthly'), method: 'GET', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); // `dailyServiceUsages[].date` does not end in `At`, so it has to be named for `parseDateFields`. return parseResponse(response, schemas.MonthlyUsage(), (key) => key === 'date'); @@ -94,15 +105,20 @@ export class UserClient extends ResourceClient { /** * Retrieves the user's account and usage limits. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The account and usage limits object. * @see https://docs.apify.com/api/v2/users-me-limits-get * @since Added in 2.9.2 */ - async limits(): Promise { + async limits(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('limits'), + url: this.buildUrl('limits'), method: 'GET', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); return parseResponse(response, schemas.AccountLimits()); } @@ -110,16 +126,21 @@ export class UserClient extends ResourceClient { /** * Updates the user's account and usage limits. * - * @param options - The new limits to set. + * @param newLimits - The new limits to set. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/users-me-limits-put * @since Added in 2.10.0 */ - async updateLimits(options: LimitsUpdateOptions): Promise { + async updateLimits(newLimits: LimitsUpdateOptions, options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const requestOpts: ApifyRequestConfig = { - url: this._url('limits'), + url: this.buildUrl('limits'), method: 'PUT', - params: this._params(), - data: options, + params: this.buildParams(), + data: newLimits, + timeoutSecs, }; await this.httpClient.call(requestOpts); } diff --git a/src/resource_clients/webhook.ts b/src/resource_clients/webhook.ts index 1a1b7940f..fd2e23539 100644 --- a/src/resource_clients/webhook.ts +++ b/src/resource_clients/webhook.ts @@ -1,7 +1,9 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { Webhook, WebhookEventType } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; import { anyObjectSchema, parseArgument, parseResponse } from '../utils.js'; import type { WebhookDispatch } from './webhook_dispatch.js'; import { WebhookDispatchCollectionClient } from './webhook_dispatch_collection.js'; @@ -59,46 +61,62 @@ export class WebhookClient extends ResourceClient { /** * Retrieves the webhook. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The webhook object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/webhook-get */ - async get(): Promise { - return this._get(schemas.Webhook()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.Webhook(), {}, timeoutSecs); } /** * Updates the webhook with the specified fields. * * @param newFields - Fields to update. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The updated webhook object. * @see https://docs.apify.com/api/v2/webhook-put */ - async update(newFields: WebhookUpdateData): Promise { + async update(newFields: WebhookUpdateData, options: TimeoutOptions = {}): Promise { parseArgument(newFields, anyObjectSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._update(schemas.Webhook(), newFields); + return this.updateResource(schemas.Webhook(), newFields, timeoutSecs); } /** * Deletes the webhook. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @see https://docs.apify.com/api/v2/webhook-delete */ - async delete(): Promise { - return this._delete(); + async delete(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.deleteResource(timeoutSecs); } /** * Tests the webhook by dispatching a test event. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'medium'`. * @returns The webhook dispatch object. * @see https://docs.apify.com/api/v2/webhook-test-post */ - async test(): Promise { + async test(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'medium' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + const response = await this.httpClient.call({ - url: this._url('test'), + url: this.buildUrl('test'), method: 'POST', - params: this._params(), + params: this.buildParams(), + timeoutSecs, }); return parseResponse(response, schemas.WebhookDispatch()); } @@ -111,7 +129,7 @@ export class WebhookClient extends ResourceClient { */ dispatches(): WebhookDispatchCollectionClient { return new WebhookDispatchCollectionClient( - this._subResourceOptions({ + this.subResourceOptions({ resourcePath: 'dispatches', }), ); diff --git a/src/resource_clients/webhook_collection.ts b/src/resource_clients/webhook_collection.ts index 00d6e27b9..24a9fd54d 100644 --- a/src/resource_clients/webhook_collection.ts +++ b/src/resource_clients/webhook_collection.ts @@ -2,14 +2,17 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema, timeoutOptionsShape } from '../timeouts.js'; import { anyObjectSchema, paginationOptionsShape, parseArgument } from '../utils.js'; import type { Webhook, WebhookUpdateData } from './webhook.js'; const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), + ...timeoutOptionsShape, }); const webhookCreateSchema = anyObjectSchema.optional(); @@ -64,6 +67,7 @@ export class WebhookCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination and sorting options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of webhooks. * @see https://docs.apify.com/api/v2/webhooks-get */ @@ -73,23 +77,26 @@ export class WebhookCollectionClient extends ResourceCollectionClient { ): PaginatedIterator> { const parsed = parseArgument(options, listOptionsSchema, 'WebhookCollectionListOptions'); - return this._listPaginated(schemas.ListOfWebhooks(), parsed); + return this.listResourcesPaginated(schemas.ListOfWebhooks(), parsed, 'medium'); } /** * Creates a new webhook. * * @param webhook - The webhook data. + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The created webhook object. * @see https://docs.apify.com/api/v2/webhooks-post */ - async create(webhook?: WebhookUpdateData): Promise { + async create(webhook?: WebhookUpdateData, options: TimeoutOptions = {}): Promise { parseArgument(webhook, webhookCreateSchema); + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); - return this._create(schemas.Webhook(), webhook); + return this.createResource(schemas.Webhook(), webhook, timeoutSecs); } } -export interface WebhookCollectionListOptions extends PaginationOptions { +export interface WebhookCollectionListOptions extends PaginationOptions, TimeoutOptions { desc?: boolean; } diff --git a/src/resource_clients/webhook_dispatch.ts b/src/resource_clients/webhook_dispatch.ts index 34339e29c..22cd11d54 100644 --- a/src/resource_clients/webhook_dispatch.ts +++ b/src/resource_clients/webhook_dispatch.ts @@ -1,7 +1,10 @@ import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceClient } from '../base/resource_client.js'; import type { WebhookDispatch } from '../models.js'; +import type { TimeoutOptions } from '../timeouts.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsSchema } from '../timeouts.js'; +import { parseArgument } from '../utils.js'; export type { WebhookDispatch, @@ -43,10 +46,14 @@ export class WebhookDispatchClient extends ResourceClient { /** * Retrieves the webhook dispatch. * + * @param options - Request options + * @param options.timeoutSecs - Timeout for the API request. Default is `'short'`. * @returns The webhook dispatch object, or `undefined` if it does not exist. * @see https://docs.apify.com/api/v2/webhook-dispatch-get */ - async get(): Promise { - return this._get(schemas.WebhookDispatch()); + async get(options: TimeoutOptions = {}): Promise { + const { timeoutSecs = 'short' } = parseArgument(options, timeoutOptionsSchema, 'TimeoutOptions'); + + return this.getResource(schemas.WebhookDispatch(), {}, timeoutSecs); } } diff --git a/src/resource_clients/webhook_dispatch_collection.ts b/src/resource_clients/webhook_dispatch_collection.ts index 6c9df8eca..b383ac21c 100644 --- a/src/resource_clients/webhook_dispatch_collection.ts +++ b/src/resource_clients/webhook_dispatch_collection.ts @@ -2,14 +2,17 @@ import { z } from 'zod'; import type { ApiClientSubResourceOptions } from '../base/api_client.js'; import { ResourceCollectionClient } from '../base/resource_collection_client.js'; +import type { TimeoutOptions } from '../timeouts.js'; import type { PaginatedIterator, PaginationOptions } from '../utils.js'; import * as schemas from '../schemas.js'; +import { timeoutOptionsShape } from '../timeouts.js'; import { paginationOptionsShape, parseArgument } from '../utils.js'; import type { WebhookDispatch } from './webhook_dispatch.js'; const listOptionsSchema = z.strictObject({ ...paginationOptionsShape, desc: z.boolean().optional(), + ...timeoutOptionsShape, }); /** @@ -58,16 +61,17 @@ export class WebhookDispatchCollectionClient extends ResourceCollectionClient { * ``` * * @param options - Pagination and sorting options. + * @param options.timeoutSecs - Timeout for each API request. Default is `'medium'`. * @returns A paginated iterator of webhook dispatches. * @see https://docs.apify.com/api/v2/webhook-dispatches-get */ list(options: WebhookDispatchCollectionListOptions = {}): PaginatedIterator { const parsed = parseArgument(options, listOptionsSchema, 'WebhookDispatchCollectionListOptions'); - return this._listPaginated(schemas.ListOfWebhookDispatches(), parsed); + return this.listResourcesPaginated(schemas.ListOfWebhookDispatches(), parsed, 'medium'); } } -export interface WebhookDispatchCollectionListOptions extends PaginationOptions { +export interface WebhookDispatchCollectionListOptions extends PaginationOptions, TimeoutOptions { desc?: boolean; } diff --git a/src/timeouts.ts b/src/timeouts.ts new file mode 100644 index 000000000..f1890743e --- /dev/null +++ b/src/timeouts.ts @@ -0,0 +1,70 @@ +import { z } from 'zod'; + +/** Default duration of the `short` timeout tier, in seconds. */ +export const DEFAULT_TIMEOUT_SHORT_SECS = 5; + +/** Default duration of the `medium` timeout tier, in seconds. */ +export const DEFAULT_TIMEOUT_MEDIUM_SECS = 30; + +/** Default duration of the `long` timeout tier, in seconds. */ +export const DEFAULT_TIMEOUT_LONG_SECS = 360; + +/** Default cap on the timeout of a single request attempt, in seconds. */ +export const DEFAULT_TIMEOUT_MAX_SECS = 360; + +/** + * Preconfigured timeout tiers. Every client method picks the tier that fits the expected duration of its + * request: `short` for simple metadata reads and writes, `medium` for listing, batch and trigger operations, + * `long` for downloads, uploads and streaming. The duration of each tier is set on the {@link ApifyClient} + * constructor. + */ +export type TimeoutTier = 'short' | 'medium' | 'long'; + +/** + * Timeout of a single API request. + * + * A tier name picks the duration configured for that tier on the {@link ApifyClient} constructor, a number is + * an exact duration in seconds, and `'noTimeout'` lets the request run for as long as it takes. A tier or an + * explicit duration above `timeoutMaxSecs` is capped at it, and the client logs a warning when that happens. + * The timeout applies to each attempt separately, and doubles with every retry up to the same cap. + * + * With `'noTimeout'` the client never aborts the request, so a connection that stalls is not retried either. + */ +export type Timeout = TimeoutTier | 'noTimeout' | number; + +/** + * Options shared by every method that sends a request to the API. + */ +export interface TimeoutOptions { + /** + * Timeout for the API request: a tier name (`'short'`, `'medium'`, `'long'`), a number of seconds, or + * `'noTimeout'`. Defaults to the tier the method is assigned, which its documentation names. + */ + timeoutSecs?: Timeout; +} + +/** + * Schema of {@link Timeout}. Zero is rejected on purpose - `'noTimeout'` is the explicit way to run without one. + */ +const timeoutSchema = z.union([z.enum(['short', 'medium', 'long', 'noTimeout']), z.number().positive()]); + +/** + * Schema of {@link Timeout}, optional, for the methods that validate `timeoutSecs` on its own. + * @internal + */ +export const optionalTimeoutSchema = timeoutSchema.optional(); + +/** + * Schema shape of {@link TimeoutOptions}, to spread into the option schema of every method that sends a + * request. One copy stops it drifting from the interface. + * @internal + */ +export const timeoutOptionsShape = { + timeoutSecs: optionalTimeoutSchema, +}; + +/** + * Schema of {@link TimeoutOptions}, for the methods that take no other option. + * @internal + */ +export const timeoutOptionsSchema = z.strictObject(timeoutOptionsShape); diff --git a/test/actors.test.ts b/test/actors.test.ts index 5f8e832fd..f93f6d37b 100644 --- a/test/actors.test.ts +++ b/test/actors.test.ts @@ -158,13 +158,11 @@ describe('Actor methods', () => { const actorId = 'some-id'; const input = { some: 'body' }; - const query = { - timeout: 120, - memory: 256, - build: '1.2.0', - }; + // The run timeout travels to the API as `timeout`, while the client option is `runTimeoutSecs`. + const options = { runTimeoutSecs: 120, memory: 256, build: '1.2.0' }; + const query = { timeout: 120, memory: 256, build: '1.2.0' }; - const res = await client.actor(actorId).start(input, query); + const res = await client.actor(actorId).start(input, options); expect(res.id).toEqual('run-actor'); validateRequest({ query, params: { actorId }, body: input }); @@ -172,7 +170,7 @@ describe('Actor methods', () => { (id, i, opts) => client.actor(id).start(i, opts), actorId, input, - query, + options, ); expect(browserRes).toEqual(asBrowserResult(res)); validateRequest({ query, params: { actorId }, body: input }); @@ -295,7 +293,7 @@ describe('Actor methods', () => { test('call() works', async () => { const actorId = 'some-id'; const input = { some: 'body' }; - const timeout = 120; + const runTimeoutSecs = 120; const memory = 256; const build = '1.2.0'; const runId = 'started-run-id'; @@ -306,7 +304,7 @@ describe('Actor methods', () => { mockServer.setResponse({ body }); const res = await client.actor(actorId).call(input, { memory, - timeout, + runTimeoutSecs, build, waitSecs, log: null, @@ -316,7 +314,7 @@ describe('Actor methods', () => { validateRequest({ query: { waitForFinish: waitSecs }, params: { runId } }); validateRequest({ query: { - timeout, + timeout: runTimeoutSecs, memory, build, }, @@ -330,7 +328,7 @@ describe('Actor methods', () => { input, { memory, - timeout, + runTimeoutSecs, build, waitSecs, }, @@ -339,7 +337,7 @@ describe('Actor methods', () => { validateRequest({ query: { waitForFinish: waitSecs }, params: { runId } }); validateRequest({ query: { - timeout, + timeout: runTimeoutSecs, memory, build, }, diff --git a/test/client_timeouts.test.ts b/test/client_timeouts.test.ts index f853e04e7..928379689 100644 --- a/test/client_timeouts.test.ts +++ b/test/client_timeouts.test.ts @@ -1,7 +1,8 @@ -import { ApifyClient, type Dictionary } from 'apify-client'; +import type { Timeout, TimeoutTier } from 'apify-client'; +import { ApifyClient, ArgumentValidationError } from 'apify-client'; import { beforeEach, describe, expect, test, vi } from 'vitest'; -import type { HttpClient } from '../src/http_client.js'; +import type { ApifyRequestConfig, HttpClient } from '../src/http_client.js'; import type * as utils from '../src/utils.js'; @@ -16,146 +17,382 @@ vi.mock('../src/utils', async (importOriginal) => { }; }); -// Mock for testing timeout functionality +// A terminal `status` keeps `call()` and `waitForFinish()` from polling forever. +const DEFAULT_RESPONSE_DATA = { id: 'test-id', status: 'SUCCEEDED' }; + +/** + * Stands in for `HttpClient`: records every request config and answers with a canned body, so the tests can + * read the `timeoutSecs` each method asks for without a server. + */ class MockHttpClient { - public timeoutSecs: number; - public callHistory: any[]; - public stats: { addRateLimitError: ReturnType }; - - constructor(timeoutSecs: number) { - this.timeoutSecs = timeoutSecs * 1000; // Convert to milliseconds - this.callHistory = []; - this.stats = { - addRateLimitError: vi.fn(), - }; + public timeoutMillis: Record; + public callHistory: ApifyRequestConfig[] = []; + public stats = { addRateLimitError: vi.fn() }; + public responseData: unknown = DEFAULT_RESPONSE_DATA; + + constructor(httpClient: HttpClient) { + this.timeoutMillis = httpClient.timeoutMillis; } - async call(config: Dictionary) { - // Track the call for assertions - this.callHistory.push({ - ...config, - timeoutUsed: config.timeout, - }); + async call(config: ApifyRequestConfig) { + this.callHistory.push(config); - // Simulate successful response return { - data: { data: { id: 'test-id' } }, + data: { data: this.responseData }, status: 200, headers: {}, }; } + + get lastCall() { + return this.callHistory[this.callHistory.length - 1]; + } } -// Test parameters: [ClientClass, methodName, expectedTimeoutMillis, methodArgs] -const timeoutTestParams = [ - // Dataset Client - ['DatasetClient', 'get', 5000, []], - ['DatasetClient', 'update', 5000, [{ name: 'new-name' }]], - ['DatasetClient', 'delete', 5000, []], - ['DatasetClient', 'listItems', 360000, []], - ['DatasetClient', 'downloadItems', 360000, ['json']], - ['DatasetClient', 'pushItems', 30000, [[{ test: 'data' }]]], - ['DatasetClient', 'getStatistics', 5000, []], - - // KeyValueStore Client - ['KeyValueStoreClient', 'get', 5000, []], - ['KeyValueStoreClient', 'update', 360000, [{}]], - ['KeyValueStoreClient', 'delete', 5000, []], - ['KeyValueStoreClient', 'listKeys', 30000, []], - ['KeyValueStoreClient', 'getRecord', 360000, ['test-key']], - ['KeyValueStoreClient', 'setRecord', 360000, [{ key: 'test-key', value: 'some-value' }]], - ['KeyValueStoreClient', 'deleteRecord', 5000, ['test-key']], - - // RequestQueue Client - ['RequestQueueClient', 'get', 5000, []], - ['RequestQueueClient', 'update', 5000, [{ name: 'new-name' }]], - ['RequestQueueClient', 'delete', 5000, []], - ['RequestQueueClient', 'listHead', 5000, []], - ['RequestQueueClient', 'listAndLockHead', 30000, [{ lockSecs: 10 }]], - ['RequestQueueClient', 'addRequest', 5000, [{ url: 'https://example.com', uniqueKey: 'test' }]], - ['RequestQueueClient', 'getRequest', 5000, ['request-id']], - [ - 'RequestQueueClient', - 'updateRequest', - 30000, - [{ id: 'request-id', url: 'https://example.com', uniqueKey: 'test' }], - ], - ['RequestQueueClient', 'deleteRequest', 5000, ['request-id']], - ['RequestQueueClient', 'prolongRequestLock', 30000, ['request-id', { lockSecs: 10 }]], - ['RequestQueueClient', 'deleteRequestLock', 5000, ['request-id']], - ['RequestQueueClient', 'batchAddRequests', 30000, [[{ uniqueKey: 'request-key' }], {}]], - ['RequestQueueClient', 'batchDeleteRequests', 5000, [[{ id: 'request-id' }]]], - ['RequestQueueClient', 'listRequests', 30000, []], -] as const; - -describe('Client Timeout Tests', () => { +type ClientFactory = (client: ApifyClient) => object; + +const CLIENTS: Record = { + ActorClient: (client) => client.actor('test-id'), + ActorCollectionClient: (client) => client.actors(), + ActorEnvVarClient: (client) => client.actor('test-id').version('0.0').envVar('VAR'), + ActorEnvVarCollectionClient: (client) => client.actor('test-id').version('0.0').envVars(), + ActorVersionClient: (client) => client.actor('test-id').version('0.0'), + ActorVersionCollectionClient: (client) => client.actor('test-id').versions(), + BuildClient: (client) => client.build('test-id'), + BuildCollectionClient: (client) => client.builds(), + DatasetClient: (client) => client.dataset('test-id'), + DatasetCollectionClient: (client) => client.datasets(), + KeyValueStoreClient: (client) => client.keyValueStore('test-id'), + KeyValueStoreCollectionClient: (client) => client.keyValueStores(), + LogClient: (client) => client.log('test-id'), + RequestQueueClient: (client) => client.requestQueue('test-id'), + RequestQueueCollectionClient: (client) => client.requestQueues(), + RunClient: (client) => client.run('test-id'), + RunCollectionClient: (client) => client.runs(), + ScheduleClient: (client) => client.schedule('test-id'), + ScheduleCollectionClient: (client) => client.schedules(), + StoreCollectionClient: (client) => client.store(), + TaskClient: (client) => client.task('test-id'), + TaskCollectionClient: (client) => client.tasks(), + UserClient: (client) => client.user('test-id'), + WebhookClient: (client) => client.webhook('test-id'), + WebhookCollectionClient: (client) => client.webhooks(), + WebhookDispatchClient: (client) => client.webhookDispatch('test-id'), + WebhookDispatchCollectionClient: (client) => client.webhookDispatches(), +}; + +interface TimeoutCase { + client: keyof typeof CLIENTS; + method: string; + /** The tier the method is assigned, and so the `timeoutSecs` it asks the HTTP client for. */ + tier: Timeout; + /** Arguments of the call ahead of its options object. */ + args?: unknown[]; + /** The options object the call needs, when the defaults do not do; a `timeoutSecs` override is merged into it. */ + options?: object; + /** A response body the method needs to complete, when the default one does not fit its shape. */ + responseData?: unknown; +} + +// One entry per method that sends a request, with the default tier each is assigned. The tiers mirror the +// Python client method for method. The effective timeout can still differ: only this client extends it to +// cover a `waitForFinish` hold. +const TIMEOUT_CASES: TimeoutCase[] = [ + { client: 'ActorClient', method: 'get', tier: 'short' }, + { client: 'ActorClient', method: 'update', tier: 'short', args: [{ name: 'x' }] }, + { client: 'ActorClient', method: 'delete', tier: 'short' }, + { client: 'ActorClient', method: 'start', tier: 'medium', args: [undefined] }, + { client: 'ActorClient', method: 'call', tier: 'noTimeout', args: [undefined], options: { log: null } }, + { client: 'ActorClient', method: 'build', tier: 'medium', args: ['0.0'] }, + { client: 'ActorClient', method: 'defaultBuild', tier: 'short' }, + { client: 'ActorClient', method: 'validateInput', tier: 'short', args: [{}] }, + { client: 'ActorCollectionClient', method: 'list', tier: 'medium' }, + { client: 'ActorCollectionClient', method: 'create', tier: 'medium', args: [{ name: 'x' }] }, + { client: 'ActorEnvVarClient', method: 'get', tier: 'short' }, + { client: 'ActorEnvVarClient', method: 'update', tier: 'short', args: [{ name: 'VAR', value: 'x' }] }, + { client: 'ActorEnvVarClient', method: 'delete', tier: 'short' }, + { client: 'ActorEnvVarCollectionClient', method: 'list', tier: 'short' }, + { client: 'ActorEnvVarCollectionClient', method: 'create', tier: 'short', args: [{ name: 'VAR', value: 'x' }] }, + { client: 'ActorVersionClient', method: 'get', tier: 'short' }, + { client: 'ActorVersionClient', method: 'update', tier: 'short', args: [{ versionNumber: '0.0' }] }, + { client: 'ActorVersionClient', method: 'delete', tier: 'short' }, + { client: 'ActorVersionCollectionClient', method: 'list', tier: 'short' }, + { client: 'ActorVersionCollectionClient', method: 'create', tier: 'short', args: [{ versionNumber: '0.0' }] }, + { client: 'BuildClient', method: 'get', tier: 'short' }, + { client: 'BuildClient', method: 'abort', tier: 'short' }, + { client: 'BuildClient', method: 'delete', tier: 'short' }, + { client: 'BuildClient', method: 'getOpenApiDefinition', tier: 'medium' }, + { client: 'BuildClient', method: 'waitForFinish', tier: 'noTimeout' }, + { client: 'BuildCollectionClient', method: 'list', tier: 'medium' }, + { client: 'DatasetClient', method: 'get', tier: 'short' }, + { client: 'DatasetClient', method: 'update', tier: 'short', args: [{ name: 'x' }] }, + { client: 'DatasetClient', method: 'delete', tier: 'short' }, + { client: 'DatasetClient', method: 'listItems', tier: 'long' }, + { client: 'DatasetClient', method: 'downloadItems', tier: 'long', args: ['json'] }, + { client: 'DatasetClient', method: 'pushItems', tier: 'medium', args: [[{ test: 'data' }]] }, + { client: 'DatasetClient', method: 'getStatistics', tier: 'short' }, + { client: 'DatasetClient', method: 'createItemsPublicUrl', tier: 'long' }, + { client: 'DatasetCollectionClient', method: 'list', tier: 'medium' }, + { client: 'DatasetCollectionClient', method: 'getOrCreate', tier: 'short', args: ['name'] }, + { client: 'KeyValueStoreClient', method: 'get', tier: 'short' }, + { client: 'KeyValueStoreClient', method: 'update', tier: 'long', args: [{ name: 'x' }] }, + { client: 'KeyValueStoreClient', method: 'delete', tier: 'short' }, + { client: 'KeyValueStoreClient', method: 'listKeys', tier: 'medium' }, + { client: 'KeyValueStoreClient', method: 'getRecordPublicUrl', tier: 'long', args: ['key'] }, + { client: 'KeyValueStoreClient', method: 'createKeysPublicUrl', tier: 'long' }, + { client: 'KeyValueStoreClient', method: 'recordExists', tier: 'long', args: ['key'] }, + { client: 'KeyValueStoreClient', method: 'getRecord', tier: 'long', args: ['key'] }, + { client: 'KeyValueStoreClient', method: 'setRecord', tier: 'long', args: [{ key: 'key', value: 'value' }] }, + { client: 'KeyValueStoreClient', method: 'deleteRecord', tier: 'short', args: ['key'] }, + { client: 'KeyValueStoreCollectionClient', method: 'list', tier: 'medium' }, + { client: 'KeyValueStoreCollectionClient', method: 'getOrCreate', tier: 'short', args: ['name'] }, + { client: 'LogClient', method: 'get', tier: 'long' }, + { client: 'LogClient', method: 'stream', tier: 'long' }, + { client: 'RequestQueueClient', method: 'get', tier: 'short' }, + { client: 'RequestQueueClient', method: 'update', tier: 'short', args: [{ name: 'x' }] }, + { client: 'RequestQueueClient', method: 'delete', tier: 'short' }, + { client: 'RequestQueueClient', method: 'listHead', tier: 'short' }, + { client: 'RequestQueueClient', method: 'listAndLockHead', tier: 'medium', options: { lockSecs: 10 } }, + { client: 'RequestQueueClient', method: 'addRequest', tier: 'short', args: [{ url: 'http://x', uniqueKey: 'x' }] }, + { + client: 'RequestQueueClient', + method: 'batchAddRequests', + tier: 'medium', + args: [[{ url: 'http://x', uniqueKey: 'x' }]], + responseData: { processedRequests: [{ uniqueKey: 'x' }], unprocessedRequests: [] }, + }, + { client: 'RequestQueueClient', method: 'batchDeleteRequests', tier: 'short', args: [[{ id: 'x' }]] }, + { client: 'RequestQueueClient', method: 'getRequest', tier: 'short', args: ['x'] }, + { client: 'RequestQueueClient', method: 'updateRequest', tier: 'medium', args: [{ id: 'x', url: 'http://x' }] }, + { client: 'RequestQueueClient', method: 'deleteRequest', tier: 'short', args: ['x'] }, + { + client: 'RequestQueueClient', + method: 'prolongRequestLock', + tier: 'medium', + args: ['x'], + options: { lockSecs: 10 }, + }, + { client: 'RequestQueueClient', method: 'deleteRequestLock', tier: 'short', args: ['x'] }, + { client: 'RequestQueueClient', method: 'listRequests', tier: 'medium' }, + { client: 'RequestQueueClient', method: 'unlockRequests', tier: 'long' }, + { client: 'RequestQueueCollectionClient', method: 'list', tier: 'medium' }, + { client: 'RequestQueueCollectionClient', method: 'getOrCreate', tier: 'short', args: ['name'] }, + { client: 'RunClient', method: 'get', tier: 'short' }, + { client: 'RunClient', method: 'abort', tier: 'medium' }, + { client: 'RunClient', method: 'delete', tier: 'short' }, + { client: 'RunClient', method: 'metamorph', tier: 'medium', args: ['target', {}] }, + { client: 'RunClient', method: 'reboot', tier: 'medium' }, + { client: 'RunClient', method: 'update', tier: 'short', args: [{ statusMessage: 'x' }] }, + { client: 'RunClient', method: 'resurrect', tier: 'medium' }, + { client: 'RunClient', method: 'charge', tier: 'short', options: { eventName: 'x' } }, + { client: 'RunClient', method: 'waitForFinish', tier: 'noTimeout' }, + { + client: 'RunClient', + method: 'getStreamedLog', + tier: 'long', + // The method reads the run to find its Actor, and that lookup is what the tier times. + responseData: { ...DEFAULT_RESPONSE_DATA, actId: 'actor-id' }, + }, + { client: 'RunCollectionClient', method: 'list', tier: 'medium' }, + { client: 'ScheduleClient', method: 'get', tier: 'short' }, + { client: 'ScheduleClient', method: 'update', tier: 'short', args: [{ name: 'x' }] }, + { client: 'ScheduleClient', method: 'delete', tier: 'short' }, + { client: 'ScheduleClient', method: 'getLog', tier: 'medium' }, + { client: 'ScheduleCollectionClient', method: 'list', tier: 'medium' }, + { client: 'ScheduleCollectionClient', method: 'create', tier: 'short', args: [{ name: 'x' }] }, + { client: 'StoreCollectionClient', method: 'list', tier: 'medium' }, + { client: 'TaskClient', method: 'get', tier: 'short' }, + { client: 'TaskClient', method: 'update', tier: 'short', args: [{ name: 'x' }] }, + { client: 'TaskClient', method: 'publish', tier: 'short' }, + { client: 'TaskClient', method: 'unpublish', tier: 'short' }, + { client: 'TaskClient', method: 'delete', tier: 'short' }, + { client: 'TaskClient', method: 'start', tier: 'medium', args: [undefined] }, + { client: 'TaskClient', method: 'call', tier: 'noTimeout', args: [undefined] }, + { client: 'TaskClient', method: 'getInput', tier: 'short' }, + { client: 'TaskClient', method: 'updateInput', tier: 'short', args: [{ x: 1 }] }, + { client: 'TaskCollectionClient', method: 'list', tier: 'medium' }, + { client: 'TaskCollectionClient', method: 'create', tier: 'medium', args: [{ actId: 'x', name: 'x' }] }, + { client: 'UserClient', method: 'get', tier: 'short' }, + { client: 'UserClient', method: 'monthlyUsage', tier: 'short' }, + { client: 'UserClient', method: 'limits', tier: 'short' }, + { client: 'UserClient', method: 'updateLimits', tier: 'short', args: [{ maxMonthlyUsageUsd: 1 }] }, + { client: 'WebhookClient', method: 'get', tier: 'short' }, + { client: 'WebhookClient', method: 'update', tier: 'short', args: [{ requestUrl: 'http://x' }] }, + { client: 'WebhookClient', method: 'delete', tier: 'short' }, + { client: 'WebhookClient', method: 'test', tier: 'medium' }, + { client: 'WebhookCollectionClient', method: 'list', tier: 'medium' }, + { client: 'WebhookCollectionClient', method: 'create', tier: 'short', args: [{ requestUrl: 'http://x' }] }, + { client: 'WebhookDispatchClient', method: 'get', tier: 'short' }, + { client: 'WebhookDispatchCollectionClient', method: 'list', tier: 'medium' }, +]; + +describe('Client timeouts', () => { let client: ApifyClient; let mockHttpClient: MockHttpClient; + /** Builds the resource client under test on top of the recording HTTP client. */ + const resourceClient = (name: keyof typeof CLIENTS) => { + const instance = CLIENTS[name](client) as Record; + instance.httpClient = mockHttpClient; + return instance as Record Promise>; + }; + beforeEach(() => { - mockHttpClient = new MockHttpClient(360); // 6 minutes default client = new ApifyClient(); - // Replace the http client with our mock - client.httpClient = mockHttpClient as any as HttpClient; + mockHttpClient = new MockHttpClient(client.httpClient); + client.httpClient = mockHttpClient as unknown as HttpClient; + }); + + describe.each(TIMEOUT_CASES)( + '$client $method()', + ({ client: clientName, method, tier, args = [], options, responseData }) => { + beforeEach(() => { + if (responseData) mockHttpClient.responseData = responseData; + }); + + test(`asks for the ${String(tier)} tier by default`, async () => { + await resourceClient(clientName)[method](...args, ...(options ? [options] : [])); + + expect(mockHttpClient.lastCall.timeoutSecs).toBe(tier); + }); + + test('sends an explicit per-call timeout instead', async () => { + await resourceClient(clientName)[method](...args, { ...options, timeoutSecs: 42 }); + + expect(mockHttpClient.lastCall.timeoutSecs).toBe(42); + }); + }, + ); + + test('the timeout never reaches the query string', async () => { + await resourceClient('DatasetClient').listItems({ limit: 5, timeoutSecs: 'short' }); + expect(mockHttpClient.lastCall.params).toEqual({ limit: 5 }); + + await resourceClient('RunCollectionClient').list({ desc: true, timeoutSecs: 'short' }); + expect(mockHttpClient.lastCall.params).toEqual({ desc: true }); + + await resourceClient('DatasetCollectionClient').getOrCreate('name', { schema: { a: 1 }, timeoutSecs: 'long' }); + expect(mockHttpClient.lastCall.data).toEqual({ schema: { a: 1 } }); + expect(mockHttpClient.lastCall.params).toEqual({ name: 'name' }); + }); + + test('call() passes its timeout to the start request and to every poll', async () => { + await resourceClient('ActorClient').call(undefined, { log: null, timeoutSecs: 7 }); + + expect(mockHttpClient.callHistory.map((config) => config.timeoutSecs)).toEqual([7, 7]); }); - describe('Dynamic Timeout with Exponential Backoff', () => { - test('timeout increases with each retry attempt', () => { - const initialTimeoutSecs = 5; - const clientTimeoutSecs = 60; + describe('a waitForFinish the API holds the response for', () => { + test.each([ + { client: 'ActorClient', method: 'start', args: [undefined], expected: 90 }, + { client: 'ActorClient', method: 'build', args: ['0.0'], expected: 90 }, + { client: 'ActorClient', method: 'defaultBuild', args: [], expected: 65 }, + { client: 'TaskClient', method: 'start', args: [undefined], expected: 90 }, + { client: 'RunClient', method: 'get', args: [], expected: 65 }, + { client: 'BuildClient', method: 'get', args: [], expected: 65 }, + ] satisfies { client: keyof typeof CLIENTS; method: string; args: unknown[]; expected: number }[])( + 'extends the timeout of $client $method() past its tier', + async ({ client: clientName, method, args, expected }) => { + await resourceClient(clientName)[method](...args, { waitForFinish: 60 }); - // Attempt 1: 5 seconds - const timeout1 = Math.min(clientTimeoutSecs, initialTimeoutSecs * 2 ** (1 - 1)); - expect(timeout1).toBe(5); + expect(mockHttpClient.lastCall.timeoutSecs).toBe(expected); + }, + ); - // Attempt 2: 10 seconds - const timeout2 = Math.min(clientTimeoutSecs, initialTimeoutSecs * 2 ** (2 - 1)); - expect(timeout2).toBe(10); + test('asks for no more than the minute the API holds', async () => { + await resourceClient('RunClient').get({ waitForFinish: 600 }); - // Attempt 3: 20 seconds - const timeout3 = Math.min(clientTimeoutSecs, initialTimeoutSecs * 2 ** (3 - 1)); - expect(timeout3).toBe(20); + expect(mockHttpClient.lastCall.timeoutSecs).toBe(65); + }); - // Attempt 4: 40 seconds - const timeout4 = Math.min(clientTimeoutSecs, initialTimeoutSecs * 2 ** (4 - 1)); - expect(timeout4).toBe(40); + test('leaves an explicit per-call timeout alone', async () => { + await resourceClient('ActorClient').start(undefined, { waitForFinish: 60, timeoutSecs: 'short' }); - // Attempt 5: Would be 80 seconds, but limited to client timeout (60) - const timeout5 = Math.min(clientTimeoutSecs, initialTimeoutSecs * 2 ** (5 - 1)); - expect(timeout5).toBe(60); + expect(mockHttpClient.lastCall.timeoutSecs).toBe('short'); }); }); - describe.each(timeoutTestParams)( - 'Specific timeouts for specific endpoints', - (clientType, methodName, expectedTimeoutMillis, methodArgs) => { - test(`${clientType}.${methodName}() uses ${expectedTimeoutMillis} millisecond timeout`, async () => { - let clientInstance: Record & { httpClient: HttpClient }; - // Create the appropriate client instance - switch (clientType) { - case 'DatasetClient': - clientInstance = client.dataset('test-id') as unknown as typeof clientInstance; - break; - case 'KeyValueStoreClient': - clientInstance = client.keyValueStore('test-id') as unknown as typeof clientInstance; - break; - case 'RequestQueueClient': - clientInstance = client.requestQueue('test-id') as unknown as typeof clientInstance; - break; - default: - throw new Error(`Unknown client type: ${clientType}`); - } - - // Replace with our mock - clientInstance.httpClient = mockHttpClient as unknown as HttpClient; - - // Call the method - await (clientInstance[methodName] as (...args: readonly unknown[]) => Promise)(...methodArgs); - - // Check the timeout was set correctly - const lastCall = mockHttpClient.callHistory[mockHttpClient.callHistory.length - 1]; - expect(lastCall.timeout).toBe(expectedTimeoutMillis); - }); - }, - ); + test('runTimeoutSecs is what the API receives as the run `timeout`', async () => { + await resourceClient('ActorClient').start(undefined, { runTimeoutSecs: 120, timeoutSecs: 'long' }); + expect(mockHttpClient.lastCall.params).toMatchObject({ timeout: 120 }); + expect(mockHttpClient.lastCall.timeoutSecs).toBe('long'); + + await resourceClient('RunClient').resurrect({ runTimeoutSecs: 60 }); + expect(mockHttpClient.lastCall.params).toMatchObject({ timeout: 60 }); + expect(mockHttpClient.lastCall.timeoutSecs).toBe('medium'); + }); + + test.each([ + { timeoutSecs: 0, reason: 'zero' }, + { timeoutSecs: -1, reason: 'a negative number' }, + { timeoutSecs: 'fast', reason: 'an unknown tier' }, + { timeoutSecs: null, reason: 'null' }, + ])('rejects $reason as a timeout', async ({ timeoutSecs }) => { + await expect(resourceClient('DatasetClient').get({ timeoutSecs })).rejects.toThrow(ArgumentValidationError); + }); + + test('the list methods whose other options the API ignores still validate the timeout', () => { + expect(() => resourceClient('ActorEnvVarCollectionClient').list({ timeoutSecs: 0 })).toThrow( + ArgumentValidationError, + ); + expect(() => resourceClient('ActorVersionCollectionClient').list({ timeoutSecs: 'fast' })).toThrow( + ArgumentValidationError, + ); + }); + + describe('RequestQueueClient with a queue-wide timeoutSecs', () => { + test('caps the default tier of every request', async () => { + const queue = client.requestQueue('test-id', { timeoutSecs: 2 }) as unknown as Record; + queue.httpClient = mockHttpClient; + const queueClient = queue as unknown as ReturnType; + + await queueClient.get(); + expect(mockHttpClient.lastCall.timeoutSecs).toBe(2); + + await queueClient.unlockRequests(); + expect(mockHttpClient.lastCall.timeoutSecs).toBe(2); + }); + + test('leaves a tier below the cap alone', async () => { + const queue = client.requestQueue('test-id', { timeoutSecs: 60 }) as unknown as Record; + queue.httpClient = mockHttpClient; + const queueClient = queue as unknown as ReturnType; + + await queueClient.get(); + expect(mockHttpClient.lastCall.timeoutSecs).toBe('short'); + + await queueClient.unlockRequests(); + expect(mockHttpClient.lastCall.timeoutSecs).toBe(60); + }); + + test('does not cap an explicit per-call timeout', async () => { + const queue = client.requestQueue('test-id', { timeoutSecs: 2 }) as unknown as Record; + queue.httpClient = mockHttpClient; + const queueClient = queue as unknown as ReturnType; + + await queueClient.get({ timeoutSecs: 'long' }); + expect(mockHttpClient.lastCall.timeoutSecs).toBe('long'); + + await queueClient.getRequest('x', { timeoutSecs: 30 }); + expect(mockHttpClient.lastCall.timeoutSecs).toBe(30); + }); + + test('compares against the tier durations configured on the client', async () => { + client = new ApifyClient({ timeoutShortSecs: 1 }); + mockHttpClient = new MockHttpClient(client.httpClient); + const queue = client.requestQueue('test-id', { timeoutSecs: 2 }) as unknown as Record; + queue.httpClient = mockHttpClient; + const queueClient = queue as unknown as ReturnType; + + await queueClient.get(); + expect(mockHttpClient.lastCall.timeoutSecs).toBe('short'); + }); + + test.each([ + { timeoutSecs: 0, reason: 'zero' }, + { timeoutSecs: -1, reason: 'a negative number' }, + ])('rejects $reason as the cap', ({ timeoutSecs }) => { + expect(() => client.requestQueue('test-id', { timeoutSecs })).toThrow(ArgumentValidationError); + }); + }); }); diff --git a/test/http_client.test.ts b/test/http_client.test.ts index 6c0b668ef..47ebd6909 100644 --- a/test/http_client.test.ts +++ b/test/http_client.test.ts @@ -5,6 +5,7 @@ import os from 'node:os'; import { gzipSync } from 'node:zlib'; import { ApifyClient } from 'apify-client'; +import type { InternalAxiosRequestConfig } from 'axios'; import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, test, vi } from 'vitest'; import { Browser } from './_helper.js'; @@ -24,11 +25,14 @@ describe('HttpClient', () => { await Promise.all([mockServer.close(), browser.cleanUpBrowser()]); }); + /** A resource id the mock server reads as "answer after this many milliseconds". */ + const delayedResourceId = (delayMillis: number) => Buffer.from(JSON.stringify({ delayMillis })).toString('hex'); + let client: ApifyClient; beforeEach(() => { client = new ApifyClient({ baseUrl, - timeoutSecs: 1, + timeoutShortSecs: 1, maxRetries: 0, userAgentSuffix: ['SDK/3.1.1', 'Crawlee/3.11.5'], }); @@ -36,17 +40,17 @@ describe('HttpClient', () => { afterEach(() => { client = null as unknown as ApifyClient; }); - test('requests timeout after timeoutSecs', async () => { - const context = { delayMillis: 3000 }; - const resourceId = Buffer.from(JSON.stringify(context)).toString('hex'); + test('requests time out after the duration configured for their tier', async () => { + const resourceId = delayedResourceId(3000); + // `get()` sits in the short tier, which the client above shortens to one second. await expect(client.actor(resourceId).get()).rejects.toThrow('timeout of 1000ms exceeded'); const ua = mockServer.getLastRequest()?.headers['user-agent']; expect(ua).toMatch(/ApifyClient\/\d+\.\d+\.\d+/); expect(ua).toMatch(`(${os.platform()}; Node/${process.version})`); expect(ua).toMatch('isAtHome/false; SDK/3.1.1; Crawlee/3.11.5'); - const page = await browser.getInjectedPage(baseUrl, { timeoutSecs: 1 }); + const page = await browser.getInjectedPage(baseUrl, { timeoutShortSecs: 1 }); try { await expect(page.evaluate((rId) => client.task(rId).get(), resourceId)).rejects.toThrow(); // this is failing after axios upgrade, the error is returned with a wrong name and message @@ -56,6 +60,38 @@ describe('HttpClient', () => { } }); + test('a per-call timeout replaces the tier of the method', async () => { + const resourceId = delayedResourceId(1500); + + // Another tier, with its own duration, and an explicit number of seconds. + await expect(client.actor(resourceId).get({ timeoutSecs: 'medium' })).resolves.toBeDefined(); + await expect(client.actor(resourceId).get({ timeoutSecs: 0.5 })).rejects.toThrow('timeout of 500ms exceeded'); + }); + + test("'noTimeout' lets a request run past every tier", async () => { + const resourceId = delayedResourceId(1500); + + await expect(client.actor(resourceId).get({ timeoutSecs: 'noTimeout' })).resolves.toBeDefined(); + }); + + test('a timeout above timeoutMaxSecs is capped at it, with a warning', async () => { + const cappedClient = new ApifyClient({ baseUrl, timeoutLongSecs: 10, timeoutMaxSecs: 1, maxRetries: 0 }); + const warningOnce = vi.spyOn(cappedClient.logger, 'warningOnce'); + const resourceId = delayedResourceId(1500); + + await expect(cappedClient.actor(resourceId).get({ timeoutSecs: 'long' })).rejects.toThrow( + 'timeout of 1000ms exceeded', + ); + await expect(cappedClient.actor(resourceId).get({ timeoutSecs: 5 })).rejects.toThrow( + 'timeout of 1000ms exceeded', + ); + + expect(warningOnce.mock.calls.map(([message]) => message)).toEqual([ + 'The requested timeout of 10s exceeds timeoutMaxSecs (1s) and is capped at it. Raise timeoutMaxSecs on the client to allow longer request timeouts.', + 'The requested timeout of 5s exceeds timeoutMaxSecs (1s) and is capped at it. Raise timeoutMaxSecs on the client to allow longer request timeouts.', + ]); + }); + test('requests go through the proxy named in HTTP_PROXY, and skip it for hosts in NO_PROXY', async () => { const proxied: string[] = []; // A forward proxy: the agent puts the absolute target URL on the request line, so relay it as it is. @@ -83,7 +119,7 @@ describe('HttpClient', () => { // A fresh client for the exempted host, because the first one keeps its socket to the proxy alive // and would reuse it without consulting the environment again. for (const name of ['NO_PROXY', 'no_proxy']) vi.stubEnv(name, 'localhost'); - const direct = await new ApifyClient({ baseUrl, timeoutSecs: 1, maxRetries: 0 }).user('me').get(); + const direct = await new ApifyClient({ baseUrl, timeoutShortSecs: 1, maxRetries: 0 }).user('me').get(); expect(direct?.id).toBe('get-user'); expect(proxied).toHaveLength(1); } finally { @@ -107,7 +143,7 @@ describe('HttpClient', () => { for (const name of ['NO_PROXY', 'no_proxy']) vi.stubEnv(name, ''); try { - const secure = new ApifyClient({ baseUrl: 'https://api.apify.test', timeoutSecs: 1, maxRetries: 0 }); + const secure = new ApifyClient({ baseUrl: 'https://api.apify.test', timeoutShortSecs: 1, maxRetries: 0 }); await expect(secure.user('me').get()).rejects.toThrow(); expect(tunneled).toEqual(['api.apify.test:443']); } finally { @@ -159,4 +195,61 @@ describe('HttpClient', () => { expect(request?.headers['content-encoding']).toBeUndefined(); expect(request?.headers['content-length']).toBe(String(payload.length)); }); + + describe('timeout across retries', () => { + /** + * Routes the client's axios instance to an in-process adapter that fails `failures` times with a 500 and + * then succeeds, recording the timeout of every attempt. No network, so no timing to get right. + */ + const recordAttemptTimeouts = (retryingClient: ApifyClient, failures: number) => { + const timeouts: number[] = []; + retryingClient.httpClient.axios.defaults.adapter = async (config: InternalAxiosRequestConfig) => { + timeouts.push(config.timeout!); + const failed = timeouts.length <= failures; + const body = failed + ? { error: { type: 'internal-server-error', message: 'boom' } } + : { data: { id: 'test-id' } }; + return { + status: failed ? 500 : 200, + statusText: failed ? 'Internal Server Error' : 'OK', + headers: { 'content-type': 'application/json; charset=utf-8' }, + data: Buffer.from(JSON.stringify(body)), + config, + }; + }; + return timeouts; + }; + + test('doubles from the requested value on each attempt, up to timeoutMaxSecs', async () => { + const retryingClient = new ApifyClient({ + baseUrl, + timeoutMaxSecs: 12, + maxRetries: 3, + minDelayBetweenRetriesMillis: 1, + }); + const timeouts = recordAttemptTimeouts(retryingClient, 3); + + await retryingClient.httpClient.call({ url: `${baseUrl}/v2/x`, method: 'GET', timeoutSecs: 'short' }); + + expect(timeouts).toEqual([5000, 10000, 12000, 12000]); + }); + + test('an explicit number of seconds doubles the same way', async () => { + const retryingClient = new ApifyClient({ baseUrl, maxRetries: 2, minDelayBetweenRetriesMillis: 1 }); + const timeouts = recordAttemptTimeouts(retryingClient, 2); + + await retryingClient.httpClient.call({ url: `${baseUrl}/v2/x`, method: 'GET', timeoutSecs: 2 }); + + expect(timeouts).toEqual([2000, 4000, 8000]); + }); + + test("'noTimeout' stays off on every attempt", async () => { + const retryingClient = new ApifyClient({ baseUrl, maxRetries: 2, minDelayBetweenRetriesMillis: 1 }); + const timeouts = recordAttemptTimeouts(retryingClient, 2); + + await retryingClient.httpClient.call({ url: `${baseUrl}/v2/x`, method: 'GET', timeoutSecs: 'noTimeout' }); + + expect(timeouts).toEqual([0, 0, 0]); + }); + }); }); diff --git a/test/integration/actor.test.ts b/test/integration/actor.test.ts index f54c04f9a..8272dd30e 100644 --- a/test/integration/actor.test.ts +++ b/test/integration/actor.test.ts @@ -207,7 +207,7 @@ test('validateInput() accepts an empty input for apify/hello-world', async () => test('start() applies the build, memory and timeout overrides it is given', async () => { const run = await client .actor(HELLO_WORLD_ACTOR) - .start(undefined, { build: 'latest', memory: 256, timeout: 120, waitForFinish: 60 }); + .start(undefined, { build: 'latest', memory: 256, runTimeoutSecs: 120, waitForFinish: 60 }); const runClient = client.run(run.id); try { diff --git a/test/integration/run.test.ts b/test/integration/run.test.ts index 0fed972f4..12e59d6c1 100644 --- a/test/integration/run.test.ts +++ b/test/integration/run.test.ts @@ -42,7 +42,7 @@ test('runs().list() filters by a single status and by a list of statuses', async // One run of each status the filter is about to ask for. const actorClient = client.actor(HELLO_WORLD_ACTOR); createdRunIds.push((await actorClient.call(undefined, NO_LOG_REDIRECT)).id); - createdRunIds.push((await actorClient.call(undefined, { timeout: 1, ...NO_LOG_REDIRECT })).id); + createdRunIds.push((await actorClient.call(undefined, { runTimeoutSecs: 1, ...NO_LOG_REDIRECT })).id); const runCollection = actorClient.runs(); diff --git a/test/integration/task.test.ts b/test/integration/task.test.ts index d840f1c1f..db163ce0d 100644 --- a/test/integration/task.test.ts +++ b/test/integration/task.test.ts @@ -179,7 +179,7 @@ test('call() applies the build and memory overrides it is given', async () => { const taskClient = client.task(createdTask.id); try { - const run = await taskClient.call(undefined, { build: 'latest', memory: 256, timeout: 120 }); + const run = await taskClient.call(undefined, { build: 'latest', memory: 256, runTimeoutSecs: 120 }); expect(run.status).toBe('SUCCEEDED'); expect(run.options.memoryMbytes).toBe(256); diff --git a/test/pagination.test.ts b/test/pagination.test.ts index 5af1226cd..32f986bcc 100644 --- a/test/pagination.test.ts +++ b/test/pagination.test.ts @@ -1,3 +1,4 @@ +import type { TimeoutOptions } from 'apify-client'; import { ApifyClient } from 'apify-client'; import { describe, expect, test, vi } from 'vitest'; @@ -79,7 +80,7 @@ const descPaginationOptions = [ }, ]; -interface ListOptions { +interface ListOptions extends TimeoutOptions { offset?: number; limit?: number; desc?: boolean; diff --git a/test/runs.test.ts b/test/runs.test.ts index 2c0d8736c..74d7f445b 100644 --- a/test/runs.test.ts +++ b/test/runs.test.ts @@ -158,19 +158,17 @@ describe('Run methods', () => { test('resurrect() works', async () => { const runId = 'some-run-id'; - const options = { - build: 'some-build', - memory: 1024, - timeout: 400, - }; + // The run timeout travels to the API as `timeout`, while the client option is `runTimeoutSecs`. + const options = { build: 'some-build', memory: 1024, runTimeoutSecs: 400 }; + const query = { build: 'some-build', memory: 1024, timeout: 400 }; const res = await client.run(runId).resurrect(options); expect(res.id).toEqual('resurrect-run'); - validateRequest({ query: options, params: { runId } }); + validateRequest({ query, params: { runId } }); const browserRes = await page.evaluate((rId, opts) => client.run(rId).resurrect(opts), runId, options); expect(browserRes).toEqual(asBrowserResult(res)); - validateRequest({ query: options, params: { runId } }); + validateRequest({ query, params: { runId } }); }); test('metamorph() works', async () => { diff --git a/test/tasks.test.ts b/test/tasks.test.ts index 73275fee7..fe8e103bb 100644 --- a/test/tasks.test.ts +++ b/test/tasks.test.ts @@ -281,7 +281,7 @@ describe('Task methods', () => { test('call() works', async () => { const taskId = 'some-task-id'; const input = { some: 'body' }; - const timeout = 120; + const runTimeoutSecs = 120; const memory = 256; const build = '1.2.0'; const actId = 'started-actor-id'; @@ -290,8 +290,9 @@ describe('Task methods', () => { const body = { data }; const waitSecs = 1; + // The run timeout travels to the API as `timeout`, while the client option is `runTimeoutSecs`. const query = { - timeout, + timeout: runTimeoutSecs, memory, build, }; @@ -299,7 +300,7 @@ describe('Task methods', () => { mockServer.setResponse({ body }); const res = await client.task(taskId).call(input, { memory, - timeout, + runTimeoutSecs, build, waitSecs, }); @@ -309,21 +310,13 @@ describe('Task methods', () => { const callBrowserRes = await page.evaluate((id, i, opts) => client.task(id).call(i, opts), taskId, input, { memory, - timeout, + runTimeoutSecs, build, waitSecs, }); expect(callBrowserRes).toEqual(asBrowserResult(res)); validateRequest({ query: { waitForFinish: waitSecs }, params: { runId } }); - validateRequest({ - query: { - timeout, - memory, - build, - }, - params: { taskId }, - body: { some: 'body' }, - }); + validateRequest({ query, params: { taskId }, body: { some: 'body' } }); }); test('call() works with maxItems', async () => {