[snarfed]"Blueskid is not an entry in the Satellite contest. First of all, Iβm sort of a @bluesky insider and the idea is to bring in ideas from the community. Second, Satellite is looking for something with a focus on decentralization and radical innovation. Blueskid uses public-key and ledger technologies that, in the software-technology context, are as old as dirt. I offer Blueskid as a low bar that the Satellite offerings really ought to
capjamesg[d]Maybe I need to have a "next crawl" value saved to a file and every day a cron job executes a script that reads that file and crawls any sites in it.
Murray[d]capjamesg[d] I have zero technical understanding of these types of challenges, so this may be completely wide of the mark, but would feed readers be a potential source of inspiration? It feels like they would need to solve this problem to an extent: when to refresh a feed/check for updates? As you're (mainly) indexing Indieweb sites, could you even have a feature where a site triggered its own recrawl? (With frequency limits to stop abuse)
petermolnarfor someone who lived through the web 2.0 craze and believed it will bring goodness, only to see the walled app world to emerge from it, I don't get good vibes from the term web 3.0.
[tantek]tbh even our attempts with the "lowercase semantic web", though more practical, less work for more immediate benefit, also suffered from being too plumbing focused rather than user (observable features) focused
[tantek]which eventually did manifest poorly as folks being more focused on marking things up than in enabling & building publishing/consuming ecosystems that provided real user benefits and value
[tantek]if there's a way to distinguish site vs post, even if it's "just" domain home page (path == "/") meaning site, and any other path meaning "post" or "page"