#dev 2024-02-20

2024-02-20 UTC
jacky and to2ds joined the channel
#
to2ds Trying to imagine a rabbit doom scrolling.
#
[KevinMarks] Using his Apple Watch
#
[KevinMarks] https://chat-uploads.indieweb.org/files/2024/02/dev-20240220-001355-il_570xn.1747526627_8zrl.jpg
#
to2ds 😄Perfect!
shoesNsocks, jacky, geoffo, burley, Xe, gRegorLove_, gRegorLove__, burley_ and [jeremycherfas] joined the channel
#
[jeremycherfas] Oh my paws and whiskers!
burley, johntheweb, tPoltergeist and gRegor joined the channel
#
[tantek] oops apparently I had been putting my likes in my Atom feed which was showing up as not particularly useful brief posts in my http://micro.blog
#
[tantek] I thought I'd removed those ages ago from my Atom feed but just realized they were still there and have now corrected them. will have to follow-up with folks on http://micro.blog to confirm
burley, [aciccarello], [benatwork], [Ana_R], IWSlackGateway, [Joe_Crawford], [tantek], [KevinMarks], [benji], [qubyte], [Murray], [jeremycherfas], aaronpk, tPoltergeist, gRegor, gRegorLove_, mahboubine, bacardi55[m], Guest6, jacky, [schmarty], bonzz and jeremycherfas joined the channel
#
jacky anyone working on anything interesting regarding Micropub?
jeremycherfas, box464 and jacky joined the channel
#
[schmarty] not actively. i am back to noodling on micropub-based tools tho. like making book-reading posts from a book's goodreads or http://bookshop.org page.
#
jacky ooh that sounds fun!
#
jacky def down to test it if you want someone to
#
[schmarty] "noodling" just meaning thinking about, at the moment. i've got two or three different workflows using other folks' tools and each has speedbumps. so i end up with a bunch of tabs open in my phone that are just goodreads / bookshop pages for books for me to remember to add to my site when i am at my laptop. (i never remember to do this)
#
[schmarty] i'mma blog post it rather than trying to infodump here!
#
aaronpk blogpost++
#
Loqi blogpost has 2 karma in this channel over the last year (3 in all channels)
#
jacky hah bet
burley, [Jo], risingspiral, jacky, mahboubine, [tantek], jonnybarnes, [keithjgrant], gRegorLove_ and gRegorLove__ joined the channel
#
gRegor Look forward to reading! Open to ideas for ibc
[qubyte] joined the channel
#
[qubyte] I log my Japanese language study sessions with an iOS shortcut and micropub. It’s sort of a menu driven thing (category like _reading, flashcards_ etc., minutes…) and it turns a number of minutes into a duration. All publishing to https://qubyte.codes/study-sessions/. It’s not _that_ interesting, but I’ve not encountered this use of h-event elsewhere. Might be a bit of a hack.
#
gRegor bookshop.org does have a metaformat "book:" like book:isbn
#
[qubyte] ↩️ Most importantly, it’s super low friction. I have more than two years of logs using it now.
#
gRegor book:author (URL), book:isbn, and book:release_date looks like. The full title and author name appear in the og:title. Hm.
#
gRegor Huh, had missed those are listed on https://ogp.me/. My first time seeing them in the wild.
#
gRegor That's cool, [qubyte]!
#
gRegor Do you generate aggregates from that, like time studied per month/year?
#
[qubyte] I do! The most recent example: https://qubyte.codes/blog/my-2023-japanese-language-study-habits
[manton] joined the channel
#
[manton] For the ActivityPub hackers out there, I just fixed a couple new issues in http://Micro.blog where things broke because of (I think) recent Mastodon security fixes. One non-obvious change is they appear to require WebFinger content-type to be jrd+json now. I also simplified a couple things like no longer redirecting WebFinger for custom domains.
#
[manton] Pretty sure Bridgy Fed was fine already, but anyway something to keep an eye on.
[snarfed] joined the channel
#
[snarfed] thanks [manton]! yup, BF returns webfinger as jrd+json. good heads up regardless though
#
[snarfed] unrelated, I just discovered that CloudFlare's CDN and caching products ignore the Vary header. wtf Cloudflare?!? https://developers.cloudflare.com/cache/concepts/cache-control/#other
#
[snarfed] they have a clunky exception for images, https://developers.cloudflare.com/cache/advanced-configuration/vary-for-images/ , but still
#
[snarfed] I don't even fully understand this. iirc at least some fediverse servers use Cloudflare, but AP requires conneg. so are requests to those servers regularly getting cached responses with the wrong content type, ie AP requests getting HTML responses and browser requests getting JSON?
#
[snarfed] confused
burley joined the channel
#
[manton] Hmm, interesting. Maybe Cloudflare has a separate cache for each set of Accept headers?
#
[snarfed] (discovered via https://github.com/Automattic/wordpress-activitypub/pull/683#issuecomment-1954432597 )
#
[snarfed] oh interesting, so maybe it's "automatically" `Vary: Accept`?
#
[manton] I have no idea, just hoping they’ve thought of this.
#
[snarfed] maybe! that's now how I read their docs, but maybe
#
[manton] I really try to avoid content negotiation but it’s impossible to avoid at least for actor URLs.
#
[snarfed] yuuup
#
[snarfed] oh yeah I'm sure they've thought about this deeply and have good reasons, cache bloat and duplication probably being a big one. I'm just still surprised and confused
#
gRegor whoa snarfed mentioned conneg but didn't conneg--
#
Loqi conneg has -15 karma in this channel over the last year (-19 in all channels)
#
gRegor gotchu
#
[snarfed] 🙏🙏
#
[snarfed] conneg-- squad
#
Loqi conneg has -16 karma in this channel over the last year (-20 in all channels)
#
aaronpk am i gonna have to finally update my activitypub code in my site?
#
[snarfed] don't do it it's a trap
#
aaronpk is this defederation? 🫴🦋
#
[tantek] is the Mastodon stampede something dev-centric we should document? because I can't see folks in general wanting to know about it or troubleshooting it or figuring out how to fix on their server
#
[tantek] phrase from jwz's post https://www.jwz.org/blog/2022/11/mastodon-stampede/ via [Joe_Crawford]
#
[tantek] related GH issue: https://github.com/mastodon/mastodon/issues/27266
bret_ joined the channel
#
[tantek] thoughts on naming? Mastodon stampede? Mastodon DDOS? Mastodon thundering herd? etc.? any of these have a larger existing usage than the others? Which seems the most user-understandable (not DDOS lol) ?
#
aaronpk "thundering herd" was used earlier to describe the same PubSubHubbub problem right?
[Joe_Crawford] joined the channel
#
[Joe_Crawford] I am struggling to find more recent documentation of the effect
#
[snarfed] yeah it's a special case of the more general slashdotting, thundering herd, etc problem
#
[snarfed] flash traffic, bursty traffic
#
[snarfed] 🤷
#
[snarfed] traffic spike
#
[tantek] not the same [snarfed]. people clicking on browsers is very different than automated distributed fetches.
#
aaronpk "mastodon stampede" has a nice ring to it
#
[snarfed] right, hence special case
#
[tantek] aaronpk, right, "thundering herd" was I believe first used to describe all RSS readers fetching on the same schedule. like on the exact hour or 30-min
#
[tantek] [snarfed] there's a big difference between manual and automated DDOS 🙂
#
aaronpk i thought it was also when you send out the pubsubhubbub ping and all the readers fetch the post
#
[tantek] aaronpk, I think that was a purely theoretical concern because I don't think there were ever enough PuSH consuming apps/sites to actually create that effect (all of them fetching)
#
[tantek] it was used as an excuse for a priori designing fat-pings into PuSH
#
aaronpk really??
burley joined the channel
#
aaronpk well there is a mention of "thundering herd" in the spec https://pubsubhubbub.github.io/PubSubHubbub/pubsubhubbub-core-0.3.html#anchor4
#
[tantek] something along the lines of "we can't only do thin pings because then all the clients retrieving the content simultaneously will cause a thundering herd problem" (which never happened AFAIK)
#
[tantek] what did happen is polling RSS readers all hammering a site exactly on the hour or half hour or whatever their frequency was because of course they couldn't be bothered to randomize a time offset of their frequency
#
aaronpk oh wow and a 2009 techcrunch article https://techcrunch.com/2009/09/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/
#
[tantek] this was back when it was presumed that Google would implement PuSH (sending, consuming) across the board, and then so would others, and thus PuSH would have to scale accordingly, which wasn't a bad presumption since PuSH co-authors were at Google
#
[tantek] also makes me wonder if we need to solve this at a protocol level, e.g. what would it take for an HTTP HEAD request to provide sufficient information for a link-preview. I mean if people are going to put their link preview info in hidden meta tag crap anyway, why not further meta it into the HTTP header?
#
aaronpk interesting point
#
[tantek] also the full irony here is that AP itself delivers "fat pings" to inboxes
#
[tantek] so yeah it's a fat ping system (Mastodon) that is still hammering on other sites 🙄
#
[snarfed] I wonder about the value of HTTP HEADs though, esp for documents. much of the burden on the server is fetching and compiling the response data, often more than the network bandwidth to return the document
#
[snarfed] and you still have to do most/all that to return even just this kind of metadata in HTTP headers
#
aaronpk i think you could easily do a scaled down version just for the data
#
[snarfed] true. I guess omitting the full response is some savings, and doesn't hurt. just seems like a smaller win instead of a full fledged solution to anything
#
aaronpk for example if you show comments on the page, you need to fetch all the comments for a full page view, but would skip it for the HEAD
#
[snarfed] true
#
[tantek] [snarfed] or people will stuff generic metacrap anyway like "Xitter is the best site ever for xitting your thoughts!"
#
[snarfed] as long as the requester doesn't care about other header metadata like eg Content-Length
#
aaronpk (i have some missing index somewhere that causes my posts with a ton of comments to load slow, so on some of my most popular pages i just turned off showing comments entirely)
#
[tantek] no need for a database request for that
#
aaronpk i mean in practice right now the goal is to fetch enough to show a link preview, which is the title, a snippet of the page, and an image URL
#
[tantek] aaronpk for that same reason I'm (still) considering putting comments (and reactions) behind a JS embed so even curls of my permalinks don't cause such external things to be retrieved
#
aaronpk yeah not a bad idea
#
[tantek] though an iframe embed for comments could also work for that
#
[tantek] except all that breaks if you want to implement Salmentions
#
aaronpk yeah, but maybe there's a better solution for that
#
[snarfed] unrelated, it still amuses me that salmentions, maybe one of the most useful OStatus features but also one of the most confusing and complicated to implement, effectively survived into ActivityPub as inbox forwarding, https://www.w3.org/TR/activitypub/#inbox-forwarding
#
aaronpk heh
burley joined the channel
#
[snarfed] holy crap, GitLab's blog's Atom feed file is currently 2.7MB, https://about.gitlab.com/atom.xml
#
[snarfed] looks like they stuff the each blog web page's entire HTML into its entry
#
aaronpk 😮
#
[aciccarello] I have my feed file only include full content for the latest few posts
#
[tantek] lol. /me checks size of my atom feed file
#
[tantek] hah 24k. 1/100th 😛
#
superkuh I have 2 feeds for blog posts. One with the recent posts (default) and one with all posts.
#
[qubyte] On that, has anyone explored paginated/archived feeds? I’ve been meaning to look into it, but I’ve no idea what support is like amongst readers: https://www.rfc-editor.org/rfc/rfc5005.html
#
[snarfed] yes! sigh I've been deprioritizing https://github.com/snarfed/granary/issues/152 for over 4y now
#
[tantek] feeds--
#
Loqi feeds has -1 karma over the last year
#
[snarfed] hmm advanced paging functionality applies to h-feeds too, right? ie it's not sidefile specific
#
[tantek] [snarfed] except HTML already has rel=next and rel=prev
#
[tantek] HTML has pagination built in for anything that wants to use it. We don't need format specific paging, unlike RSS/Atom and all XML formats which have to reinvent all this kind of baseline stuff from scratch
#
[tantek] XMLformats--
#
Loqi XMLformats has -1 karma over the last year
#
[snarfed] aha sure!
#
[tantek] layering++
#
Loqi layering has 1 karma over the last year
#
[tantek] modularity++
#
Loqi modularity has 1 karma over the last year
#
[snarfed] agreed!
#
[snarfed] my understanding of that issue and RFC5005 is that they have more sophisticated use cases, but I haven't really dug into them
#
[qubyte] What I want from it is to build a yearly archive or something, so the top-most feed has only those entries in this calendar year. Having said that, lots of my functionality is built around the atom feed (webmention dispatch for example), so I’d probably still maintain a single-page feed too. Mostly what I worry about is the feed growing forever when each item is a complete post (not just a snippet).
to2ds joined the channel
#
to2ds After entering the conversation too late: sudo apt-get install fat-ping
burley, AramZS and [KevinMarks] joined the channel
#
[KevinMarks] The point iirc was to consistently handle next/prev when there were potentially more posts generated inbetween, so naïve start=10 type pagination didn't give you dupes. This ties in with websub too.
#
capjamesg [tantek] In the ~1 minute after posting https://indieweb.social/@capjamesg/111966044917737379, I had 180 requests to the page from Mastodon.
#
capjamesg Yep, 180.
#
[tantek] like averaging 3/s?
#
capjamesg There was a bigger burst then decay.
#
capjamesg I was using the unscientific `sudo tail -n 1000 /var/log/nginx/access.log | grep "Mastodon" | wc -l` method of measurement 😄
#
[tantek] clearly you need a sparkline showing your Mastobursts on your posts
to2ds joined the channel
#
to2ds Maybe it should be called MastoDDOS?
lazcorp and burley joined the channel
#
[tantek] lol to2ds
#
[tantek] [jamietanna] I have error messages in my Atom markup for non-compliant feed readers 😄
#
[snarfed] "malicious compliance"
#
[snarfed] 😎
to2ds joined the channel
#
[tantek] prickly compliance?