#dev 2023-06-02

2023-06-02 UTC

[tantek] vladimyr, except that many companies that are training AIs already support robots.txt / noindex in general (in their crawlers, apps etc.) e.g. Google supports it both for search and things like Google Calendar crawling/subscribing