BulkImageScrape: some images are missed

[punamust]

I have a custom scraper which returns a tag in every imageByFragment scrape: one tag when the scraper succeeded and another when it failed (this makes it easier to debug where it didn't work).
I used BulkImageScrape with this scraper, expecting all images to be tagged at the end, but that wasn't the case: some images were somehow missed and had neither tag.

I could then add the two tags (the success and the failure one) to the skipped tags option of BulkImageScrape and re-run it. That way I can re-run it only on the missed ones.
I had to repeat this process ~15 times until all ~64k of my images ended up tagged.

Every time it reports the correct number of untagged images in the [Plugin / Bulk Image Scrape] Found X images log message, but when iterating over them in pages of 100, some are somehow missed.

Even though I've looked at the code recently to make some PRs, nothing stands out to me why this should be happening.
I'm not sure what's the point of it querying images in pages of 100 when it initially queries them all anyway, but perhaps it's a problem that some images are modified while simultaneously iterating over them.

[punamust]

I'm not sure what's the point of it querying images in pages of 100 when it initially queries them all anyway, but perhaps it's a problem that some images are modified while simultaneously iterating over them.

I think that might be the problem, but it's only revealed when using some of the filters (skipping tags, picking one performer). Because in that case the scraper, which adds tags or changes performer, causes the modified image to be excluded from the filter and the pagination will be off.

I don't think there's any other way to fix this than get all images at the beginning. It will be more error prone to try to account for it (by checking if the scraper just added some tags are included in the skipped filter, etc.)