Pagination iterator mishandles server-side-filtered dataset pages #979

Description

DatasetClient.listItems() used as an async iterable returns wrong results with clean / skipEmpty / skipHidden.

The API applies offset/limit to the raw item stream and filters afterwards, so a page can scan up to limit rows while returning fewer (or zero) items. x-apify-pagination-count reports the scanned window; the iterator ignores it. Two bugs in _listPaginatedFromCallback:

1. Stops early — items.length > 0 (src/base/api_client.ts:118) ends iteration on a fully filtered page, even though later pages hold matching items.
2. Yields duplicates — currentOffset += currentPage.items.length (src/base/api_client.ts:128) advances by returned instead of scanned items, so the next page re-scans rows already yielded. The header is discarded at src/resource_clients/dataset.ts:400.

Cursor iterators (listKeys, listRequests, paginateRequests) are unaffected — the API never returns an empty page with a usable cursor.

Fixed in the Python client in apify/apify-client-python#964. A JS fix must read x-apify-pagination-count separately for bookkeeping, since count in PaginatedList is public and intentionally set to data.length.

API-side root cause: apify/apify-core#27324.

Description from #1032

paginateRequests() returns as soon as a page carries no items (src/utils.ts:329), before looking at nextCursor. With filter: 'pending' or 'locked' the API can filter out a whole page while the cursor still points at more data, so the iterator truncates silently. listKeys() has the same stop condition (src/resource_clients/key_value_store.ts:215). Termination should rely on the cursor alone.

#979 states cursor iterators are unaffected — that does not hold for a server-side filter.

Python: apify/apify-client-python#964.

✍️ Drafted by Claude Code