#dev 2024-05-02

2024-05-02 UTC
nertzy, jeremycherfas, barnaby, geoffo, jonnybarnes, gruetzhaxe, superkuh, stefen, [Jan_Tuomi] and [KevinMarks] joined the channel
geoffo, barnaby, gruetzhaxe, superkuh, [tantek], [aciccarello], [Ros] and [qubyte] joined the channel
#
[tantek]
except that first row of emoji should have been 🐘 :mammoth: 🐘 :mammoth: 🐘 :mammoth:
#
[tantek]
I don't think it's right about "only once per instance". plenty of sites (including http://indieweb.org) would be able to handle that just fine
#
[tantek]
I think what's *actually* happening is people's individual Mastodon native mobile clients are requesting & generating the previews and THAT is the cause of the DDOS
#
[tantek]
I'll add it anyway
#
[aciccarello]
there are also lots of people with individual (1 person) instances
#
sknebel
I mean, indieweb.org is handling being linked on mastodon just fine
#
sknebel
or did I miss an occurence where it was a problem?
#
sknebel
because people link to our pages regularly
#
[aciccarello]
I assume http://indieweb.org is cached well so less of a problem for that site. But there were a couple articles this week about the problem for other sites.
#
aaronpk
microcache++
#
Loqi
microcache has 1 karma over the last year
#
sknebel
yeah, see -chat for my opinion on this weeks article ...
#
[tantek]
to-do << ^ re: [[thundering herd]] / Mastodon in particular: https://gist.github.com/renchap/3ae0df45b7b4534f98a8055d91d52186 (though seems to omit that what may actually be happening is people's individual Mastodon native mobile clients are requesting & generating the previews and THAT (millions of clients) is the cause of the DDOS, not "1000 instances")
#
Loqi
ok, I added "^ re: [[thundering herd]] / Mastodon in particular: https://gist.github.com/renchap/3ae0df45b7b4534f98a8055d91d52186 (though seems to omit that what may actually be happening is people's individual Mastodon native mobile clients are requesting & generating the previews and THAT (millions of clients) is the cause of the DDOS, not "1000 instances")" to the "See Also" section of /to-do https://indieweb.org/wiki/index.php?diff=94841&oldid=94838
gRegor joined the channel
#
sknebel
[tantek]: I've seen nothing to suggest that mobile clients are the issue and are not using the preview data from the instance
#
aaronpk
yea pretty sure this is well documented that the herd is the mastodon servers themselves
#
[snarfed]
right, confirmed
barnaby joined the channel
#
aaronpk
even on the mastodon web app the image is served from the instance, not hotlinked to the original site
#
sknebel
yeah, hotlinking would be bad, but nothing in the ecosystem is doing it
#
sknebel
(would be bad both for resource use and for leaking info about users to the site)
#
sknebel
and the model generally is that your local instance caches things for you and your clients *only* talk to it
#
sknebel
for me this is like yeah it'd be nice if they find a good model to mitigate it, but if your site falls over from it any scraper going wild or a bored teenager with a laptop can take your site down too
#
sknebel
and the mastodon-traffic is easy to identify identical requests, that's pretty much nicest traffic spike you can ask for
#
aaronpk
☝️
#
sknebel
(and talk about "request amplification factors" are IMHO misplaced because it's actually work to setup to a point where you can make use of that and then the fediverse is just to small to make a dent compared to actual (D)DoS attacks
#
[tantek]
Yeah that makes more sense, especially from a privacy perspective
#
[tantek]
That clients would only communicate with your direct account instance by default
#
[tantek]
And then defer actual access to other sites to clicking on a link that takes you to a browser that already has lots more privacy precautions built in
#
sknebel
right, and only once youve chosen to interact, not just seeing the preview
#
[tantek]
What's ironic about that gist is that it's ignoring all the existing properties (in AS2 etc) that are already part of the "activity" that serve the role of "preview data"
#
[tantek]
like "summary"
#
[tantek]
So you don't actually need og:description or whatever
#
[tantek]
But folks are so cargoculting with OGP = link preview that they're not bothering to step back and ask what is they're actually trying to implement for the user
#
aaronpk
well the classic debate is whether you want to trust the information from the server where the post is coming from or the actual website that is being linked to
#
aaronpk
wasn't there an incident a while ago where people were hacking the link preview?
#
[tantek]
and AS2 has thumbnail or something like it also right?
#
sknebel
right, the fetching per-instance is the decision to not trust the source to provide accurate previews
#
[tantek]
Beyond [KevinMarks] link preview hacks?
#
sknebel
(source=instance the post containing the link was made on)
#
sknebel
and I think one of the things discussed has been "ok, but we can include that and other instances can decide whos previews they trust maybe"
#
[tantek]
That whole "not trust the source" reasoning is dumb because "the source" is what "pushed" the whole activity into your inbox in the first place 🤦‍♂️
#
sknebel
not necessarily
#
[tantek]
So like either you trust https + DNS or you don't
#
aaronpk
i swear this was documented
#
[tantek]
Half trusting it makes no sense
#
aaronpk
no it's a different question
#
[tantek]
aaronpk there was a link text hack
#
[tantek]
Nothing to do with link previews
#
sknebel
you trust it to say "my user made this post"
#
[tantek]
where the text of the link would look like one URL and the actual hyperlink was something else
#
aaronpk
oh it wasn't mastodon, it was in bluesky i think
#
sknebel
but not "and the cnn.com link they include says 'US announced to accept King Charles as president'"
#
sknebel
because people associate the preview with the site linked, not the poster
#
sknebel
because thats how it works "everywhere"
#
sknebel
that the poster of the link cant really fake the link preview
#
sknebel
(some allow the poster to decide if a preview is included at all, but not what it says)
#
sknebel
and thus the UX expectation is "whats shown in the preview box in the UI is actually from the source"
#
aaronpk
right, this isn't even an #indieweb-dev conversation, it's all about user expectations
#
sknebel
which in centralized systems is based on "I trust the platform to not randomly fake link previews", and on mastodon is then translated to "I trust *my* instance to not fake link previews"
#
[KevinMarks]
Facebook historically let the poster edit the link previews, and switched to trusting the link's OGP+secret sauce
#
sknebel
(now you could argue that not faking previews should just merely be a social code a la "if we catch your instance lying about a preview we'll defederate it", but that's also messy so it makes sense to instead have the instances fetch independently)
#
[KevinMarks]
Isn't this the reason oEmbed was invented in the first place?
#
sknebel
oEmbed still means fetching things
#
aaronpk
bahaha jort link is a terrible idea
#
sknebel
and so you again have the question of "who fetches, what privacy implications has that, how many are they"
#
sknebel
and oembed endpoints get discovered just like ogp - by looking at the page header
#
[KevinMarks]
They also have a collected list at https://oembed.com/providers.json and can be served as Head links
#
sknebel
neither helps for these random websites
#
sknebel
which "strongly encourages" to not use the registry, and a random small blog isnt going to be in it anyways
#
sknebel
(and link headers also doesnt help, even if the random blog happens to use them)
#
sknebel
(mastodon also uses oembed for video embeds I'm fairly sure)
#
sknebel
(checked source code, yes it does)
[snarfed] joined the channel
#
[snarfed]
yup. lots of these points and discussion were in the github issue(s), aaronpk maybe that's what you're thinking of. https://github.com/mastodon/mastodon/issues/4486
#
[snarfed]
part of the original problem wasn't just the volume of requests, it was the spike of flash traffic, ie that volume in a very short time
#
[snarfed]
sure we can always say git good, provision your server better, etc...but the fact remains that getting linked from a popular fediverse user still often did hurt linked web sites, and that's a real problem, _especially_ if it stays a problem for years and makes publishers end up saying "don't share links on the fediverse" loudly
#
sknebel
I dont think anyone is saying that it shouldnt be improved where possible
#
sknebel
but as I said in chat, I have a hard time taking this specific publisher seriously with that
[Joe_Crawford] joined the channel
#
[Joe_Crawford]
distributed simultaneity
#
[snarfed]
a different angle is, this specific problem doesn't happen with feed readers, centralized social networks, Bluesky, etc. it's a relatively fediverse-specific drawback. which isn't great
#
sknebel
actually how does bluesky approach it?
#
sknebel
because it has the same tradeoff to make in a way
ttybitnik joined the channel
#
sknebel
it would happen with feed readers if a large number (vs a small number of hosted services) of websub enabled readers were a thing I guess?
#
sknebel
but yes, centralized things generally dont
#
sknebel
some chat systems do, where there is no central support for it and some clients choose to implement it
#
sknebel
(including some IRC clients)
nertzy joined the channel
#
[snarfed]
bluesky generates link previews client side, evidently they don't worry so much about malicious previews
geoffo joined the channel
#
aaronpk
lol at the wiki edit showing up in the issue history for that
#
[tantek]
oh oops I forgot that was a thing
#
[tantek]
ladies & gentlemen & other gentlegenders, this is why we use #indieweb-chat to make wiki edits with < < that include GH issue/PR links. don't be like me, be better
#
gRegor
haha, think of it as indieweb marketing
#
[tantek]
I feel a little self-conscious about introducing bot-noise into someone's issues/PRs
#
[tantek]
if we're having an actual on topic conversation about a GH issue in a channel, then yes, that's a useful cross-linkage for discovery
#
[tantek]
I believe a few times people have shown up as a result and been like hey what's up I saw y'all discussing this
win0err joined the channel
#
aaronpk
wonders what the breadcrumb trail is from seeing that in github to joining the chat
#
[tantek]
ah, TIL: linking to a gist here does NOT add our logs commit link to its comment stream
#
[tantek]
only issues & PRs