[feature-request] Block data that is flagged by a CC "no-AI-training" license from being ingested into a DataLoader

[GarrettMerz]

🚀 The feature

Add support for Creative Commons No-AI-Training license flags

Motivation, pitch

Hello! Creative Commons is introducing "preference signal" licenses, an addition to CC licenses that indicates that a contributor does not wish for their data to be used in model training without attribution, or at all (https://github.com/creativecommons/cc-signals). Currently, they are indicated in the robots.txt and the http header.

From what I can tell, this mechanism can't be meaningfully enforced at the point of site-scraping (as there is no indication within a scraper that data will subsequently be passed to a model), but I am curious about whether the strictest of these are implementable at a technical level at the point of ingestion into the Pytorch Dataloader.

What features would need to be added to ensure that data that is explicitly flagged as do-not-train is not ingestible by a model (is this even doable technically)? If it is not doable, would this change if the license information was implemented in EXIF metadata or similar?

Alternatives

There may be other ways to implement this at other stages within training pipelines.

Additional context

I am not affiliated with Creative Commons! This just seemed like a good discussion to kick off.

[divyanshk]

Thanks for the question. By data do you mean actual training data or code from Github which can be also used as data ? If it the former, then I'm afraid PyTorch doesn't have access to or control over it. Not sure I am fully following.

[GarrettMerz]

What I'm proposing here is essentially an extended form of metadata support- which may not be the best way to address the problem. I'm referring to actual data that is scraped from the Internet, not code.

Currently, for data hosted on a webpage, the different CC licenses that indicate what you can do with it (i.e., train a model, use it without attribution, etc.) are visible to webscrapers within both the robots.txt and HTML header files in the page source. However, even if a page is explicitly flagged with a license containing a "do not train" signal in the robots/header files (following the example in the linked repo) there's no technical mechanism in place that I am aware of to prevent a bad actor from simply scraping the data anyway using a proprietary scraper that ignores this usage signal and then training a Pytorch model on it.

Thus, I'm mostly curious as to whether or not there's some way to implement a license or data-consent check of some kind on the PyTorch end (in a way that doesn't cause major headaches across the entire ecosystem) to make it harder to ignore explicit objections creators may have to being trained on. There are other scenarios (say, HIPAA-compliance) where I can imagine a similar data-consent check mechanism might be useful.

At the end of the day, it's entirely possible that this is largely a social-contract issue and that the only real protection against training on data without the consent of its creators is for those creators to simply not share their work in a manner that could potentially be scraped. However, given the current information ecosystem, this seems like it may also be an increasingly difficult fix to implement.