[tantek]Very good points [snarfed]. Seems like there's an opportunity for an AP-specific search engine that starts spidering with mastodon.social and strictly obeys robots.txt (including rechecking maybe once a day/week or so to remove/hide sites that change their mind)
