#dev 2022-12-20

2022-12-20 UTC
#
[snarfed]2
hah, I imagine
#
aaronpk
still no verification on the three instances i checked on
sp1ff`` joined the channel
#
[KevinMarks]
not sure what triggers the verification - it looks like it is asking for text/html though https://github.com/mastodon/mastodon/blob/e98833748e80275a88560155a0b912667dd2d70b/app/services/verify_link_service.rb
#
aaronpk
which side is that tho?
#
aaronpk
"Mastodon checks if that link resolves to a web page that links back to your Mastodon profile" what does mastodon consider to be your mastodon profile URL?
#
[KevinMarks]
it's an async service in the SideKiq queue kicked off by a worker from various other points, but tracing beyond that is tricky - maybe from here?https://github.com/mastodon/mastodon/blob/e98833748e80275a88560155a0b912667dd2d70b/app/controllers/settings/profiles_controller.rb#L11
#
aaronpk
the problem is there are two separate things here. one is your own mastodon server doing the verification when you add a new link to your profile in the mastodon software.
#
aaronpk
the other is how mastodon decides to show the checkmark for profiles on *other* mastodon servers
#
[KevinMarks]
right, but there does seem to be just one code path for it. I assume it is using https://aaronparecki.com/aaronpk as your profile url, as that is what it gives as 'original url'
#
Loqi
[preview] Aaron Parecki
#
aaronpk
that URL is what's in my activitypub actor JSON data so that's what i would expect
#
aaronpk
and both the HTML and JSON version of that page have a rel=me link back to my home page, full URL, not relative /
#
[KevinMarks]
feels like you need a running instance and to look at the logs
#
aaronpk
assuming it logs anything
#
[snarfed]2
it does
#
aaronpk
well i do have an instance running... guess it's time to bust out some debugging
#
[snarfed]2
ugh, gl, deb(h)ugops
#
aaronpk
oh good i already have a way to trigger sending a profile update activity to a specific server
#
[KevinMarks]
This is the kind of thing that honeycomb.io is useful for. It can auto instrument a Ruby app and give you back traces.
#
aaronpk
ok so i added a link to my profile, sent an Update, saw that come in, my profile is updated on that server. but nowhere in that process did the server try to fetch the new link to look for the rel=me back
starrwulfe[m] joined the channel
#
aaronpk
i should have seen a request to that server
#
Loqi
totally
#
aaronpk
Loqi knows what's up
#
[KevinMarks]
It queues a sidekiq task to do it
#
aaronpk
any clue as to what the schedule is?
#
aaronpk
i can't even figure out how to view the sidekiq dashboard
#
[manton]
It’ll just run the task as soon as it can. I know when some Mastodon servers were overloaded queues were taking hours, but I expect usually it’s seconds or minutes.
#
aaronpk
yeah that was my thought
#
[KevinMarks]
it's on the default queue
#
[KevinMarks]
my mastodon tab has 2.9GB of RAM at the moment
#
aaronpk
according to this there should be a link to the sidekiq admin panel inside mastodon but i can't find it "If you log into the admin panel, you can find a link to monitor its state." https://www.zerotohero.dev/mastodon-101/
#
aaronpk
in any case, this server has basically nothing to do all the time so any sidekiq jobs should be running nearly instantly
#
[manton]
I’m not running Sidekiq, but skimming the source I think the dashboard is probably just at /sidekiq.
#
[manton]
Er, I’m not running _Mastodon_. I do use Sidekiq in Micro.blog.
#
aaronpk
it is sadly not there
#
aaronpk
hmm behind some sort of devops user role
#
[manton]
Where it adds the Sidekiq dashboard, if it helps. Dunno exactly how things are set up.
#
aaronpk
there is only one user on this instance, the admin user, so i'm confused why this user doesn't have devops privileges
#
aaronpk
supposedly the admin user: "Users with this permission will bypass every permission"
#
aaronpk
aha! turns out the devops user is actually higher than the admin user! i forgot there were actually two users on this, the one i was logged in to was only an admin
#
aaronpk
not that this gets me any closer to anything
#
aaronpk
yeah i give up i don't know what else to look at
#
Loqi
[preview] [donkoch] #22449 If you follow someone before they verify a link from a different server, it doesn't show up as verified.
#
starrwulfe[m]
err. unless you're planning on rolling your instance out to a few hundred friends, maybe give pleroma/soapbox a try? I hear it's better on admin and resources for smaller installs.
#
aaronpk
deleting and re-adding it didn't work just now tho
#
aaronpk
starrwulfe[m]: i'm trying to get my own website to show up as verified on mastodon. i'm not trying to actually run mastodon
#
[KevinMarks]
if you want to try the honeycomb path, there is https://github.com/mastodon/mastodon/pull/20338
#
Loqi
[preview] [robbkidd] #20338 Add OpenTelemetry instrumentation (opt-in)
geoffo joined the channel
#
starrwulfe[m]
aaronpk: Ohh, OK carry on.
#
starrwulfe[m]
*scurries back to sidelines*
#
aaronpk
tried with and without trailing slash and that still didn't work https://github.com/mastodon/mastodon/issues/20459
#
Loqi
[preview] [wkfry] #20459 Verified URL's don't seem to work if you have a trailing "/" character
#
aaronpk
can't find any other relevant issues
#
aaronpk
i give up again
#
[manton]
It is surprising the link verifying code just does a simple `==` compare on the URLs.
#
[manton]
Oh sorry, maybe that was on the actor URL.
bterry1 joined the channel
#
GWG
Need some thoughts on POSSE interfaces, if anyone has experiences to share
#
GWG
My internal interface was designed to match a basic Micropub interface
#
GWG
Simple checkboxes
#
GWG
The question is, if someone retrieves the post in the editor after pubishing, should I disable things that have already been successfully sent? Being as none of the services I'm using right now support updates?
gerben joined the channel
#
GWG
[snarfed]2: If I want a backlink to be generated in a Micropub post to Bridgy, would I add a url property or is there another way? I tried putting it in content but it doesn't seem to be coming back in my test account.
[iambismark] joined the channel
#
[snarfed]2
GWG what's the post or tweet?
#
GWG
I told it to, if the length of content is greater than X, to go with a backlink and the name in the content property.
jjuran and mro joined the channel
#
[snarfed]2
thanks for the report! I think this is a Bridgy Micropub bug, I'll look more
tiim, gRegor, mro, jjuran, [pfefferle] and rupl joined the channel
#
IWDiscordRelay
<c​apjamesg#4492> I made a thing: https://novacast.dev/u/jgbr?query=Adventures+on+the+IndieWeb
#
IWDiscordRelay
<c​apjamesg#4492> (Almost, give or take a few PDFs and other non-HTML assets), all of my bookmarks now show up on that site. The goal is to provide a more exploratory interface through which you can navigate my bookmarks.
#
capjamesg
(Loading can take a bit of time. I need to work on that.)
#
Loqi
[preview] Showing content related to perl
mro joined the channel
#
capjamesg
[snarfed] Does granary show images in RSS? I can't see my featured photos https://granary.io/url?input=html&output=rss&url=https%3A%2F%2Fjamesg.blog.
#
IWDiscordRelay
<c​apjamesg#4492> Would anyone be willing to provide me with a big RSS feed of their site to create a content recommendation engine for you to try?
mro joined the channel
#
[KevinMarks]
Alex Russell has been measuring desktop performance on Edge, and the story is not great https://infrequently.org/2022/12/performance-baseline-2023/
barnaby joined the channel
#
[jacky]
prepares chat with a potentially controversial topic about key management and IndieAuth
#
[jacky]
it's not technically that bad
#
[jacky]
but how about private messaging with our own website's certs?
#
aaronpk
Like with the tls cert key?
#
[jacky]
I'm looking at things like indiecert and wondering if it's _somehow_ possible to extend that to specifically to messaging
#
[jacky]
probably not possible (nor do I have the time to spike this out) but I do want to explore something like this
#
aaronpk
Probably not a good idea since a lot of times you do t even have access to it, like on managed hosting or if you're behind cloudflare and such
#
[jacky]
True yeah
#
[jacky]
keys--
#
Loqi
keys has -1 karma over the last year
#
[jacky]
okay more legit question
#
[jacky]
as I'm working on Sele, https://git.jacky.wtf/indieweb/sele/issues/27#issuecomment-2221 is something I'm running into
#
[jacky]
I'm working on the gating of access to people without an authenticated session (so things like approving requests can be done)
#
[jacky]
but I'm stuck b/c currently Sele can confirm a local account's identity using either e-mail or a password for now and I plan to add support for PGP, AGE and _potentially_ RelMeAuth once I figure out how to do it while not needing to bring in any new libraries (aka can I craft these OAuth requests using a library? they're not hard!)
#
[jacky]
do people have ideas/examples of a sign in screen with options beyond the SSO + email/password approach?
#
[jacky]
there was a site that immediately prompted WebAuthn once I put in my email but also had a fallback to password when it failed (yubikey wasn't nearby)
#
sknebel
sebsel afaik has a field you can put various kinds of urls or an email address in and chooses based on that
#
sknebel
accepted oauth-source -> do that. email -> email flow. other url -> indieauth. I guess you could do "thing that looks like a username" -> password flow?
#
[jacky]
the password's only prompted if it's a local account with one stored (so that can work!)
#
sknebel
ah, I misrembered the email part I guess
#
[jacky]
okay I'll munge with this
#
[jacky]
nah tbh it's a bit weird
#
sknebel
for sebs example
#
[jacky]
it's essentially like a magic link signin flow
#
[jacky]
AH okay
#
sknebel
(on a train that's connected by mouse coughing in a string can, so sorry if those messages appear delayed or out of order)
geoffo joined the channel
#
[jacky]
that's one heck of a metaphor for train wifi lol
#
[jacky]
all good
#
barnaby
very accurate for (presumably) german train wifi in my experience
#
sknebel
my IRC client shows the lag between it and my bouncer. it is currently showing data in seconds
#
sknebel
[jacky]: tbh the flow with only 1 initial field being shown is kinda common nowadays
#
sknebel
e.g. google or MS logins do it too, because they then can tell if they need to redirect you to a corporate login or auth you through your phone or ...
#
sknebel
and the UI for that second stage thus looks differently
#
[jacky]
that makes sense!
#
[jacky]
I think I'm stuck on that second flow
#
sknebel
so you know which user, but the user has a bunch of options to choose from?
#
[jacky]
precisely!
#
sknebel
feel like a "default" and a link to open a list of the rest is ok for that
#
[jacky]
I was just going to type that so it's validating to see that 🙂
#
[jacky]
okay I'll do that
#
[jacky]
thanks!
#
sknebel
like if someone signs up to your site with a dozen methods and that gets unwiedly it is really their own fault :D
#
sknebel
e.g. for a customer login I need a lot their flow has "please confirm in your authenticator app. if you dont have your phone at hand, click here: ..."
#
[jacky]
yeah that's something I'd want to avoid too
#
[jacky]
sknebel++ for the random brainstorm 🙂
#
Loqi
sknebel has 21 karma in this channel over the last year (42 in all channels)
geoffo joined the channel
#
[jacky]
this spec looks like it could help even bring something like OCAP over OAuth tbh https://www.ietf.org/archive/id/draft-ietf-oauth-rar-21.html#name-request-parameter-authoriza
#
[jacky]
yeah I like this quite a bit but that `type` field is doing a lot of the heavy lifting here
#
[jacky]
everything goes back to schemas (lol)
#
@cagrimmett
I like the idea of @akirk's Friends plugin for #WordPress: Part feed reader, part private social network, part indieauth. https://wpfriends.at/
(twitter.com/_/status/1605235393422856192)
geoffo, gRegor and [Michelle_Moore] joined the channel
#
[snarfed]2
capjamesg hmm, re granary and featured images in granary, I see all of the five u-featured imgs on https://jamesg.blog/ showing up in <img> tags in the item.description elements in https://granary.io/url?input=html&output=rss&url=https://jamesg.blog/
#
[snarfed]2
(fwiw, all five of those images on your site have both u-photo and u-featured)
#
[tantek]
GWG, re: "if someone retrieves the post in the editor after publishing, should I disable things that have already been successfully sent?" really depends on the specific destination. I wrote down some thoughts about how to handle this for Twitter in particular, but it's not clear how much it's worth your time to code anything Twitter-specific at this point 😕
#
[tantek]
[snarfed] that problem of Bridgy Fed taking too long to respond to Webmentions finally hit the logic on my site hard enough that I need to go rewrite some code to put the BridgyFed Webmention *last* in my publishing code, and maybe even figure out a way to do it asynchronously without waiting for an error response.
[schmarty] joined the channel
#
[schmarty]
it has me thinking about more thoughtful webmention sending for my site. making them actual jobs, catching errors and handling retries, ...
#
[schmarty]
currently i do like at-most-once sending which is design to fail towards "don't bother if anything goes weird."
#
[snarfed]2
[tantek]++ sorry for the trouble!
#
Loqi
[tantek] has 21 karma in this channel over the last year (74 in all channels)
#
[tantek]
This is an FYI, no need to change the prioritization. I'll try to document what I do (coding-wise) to mitigate the issue from my end, in the hopes that perhaps I can contribute some guidance to the BridgyFed docs accordingly
#
[tantek]
IMO BridgyFed is still very much a "developer" thing so details about how you should write your code to "call" BridgyFed feel in-scope for the docs. Correct me if I'm wrong [snarfed]
#
[tantek]
[schmarty] yes that's a good general way of thinking about the problem
#
[tantek]
might be worth starting a new Brainstorming subsection in /Webmention-developer if there isn't one already on that
#
[tantek]
there's a bunch of "non-obvious" stuff about that especially hits you when your site starts to slowly "scale" (e.g. from 1/10/100/1000 users/readers/followers/links/mentions
#
barnaby
yeah, I’ve even run into interesting scaling issues just from having an extremely link-heavy post. my not-yet-asynchronous webmention sending took ages to send webmentions for hundreds of links
#
barnaby
as much as I like the follow-your-nose discovery for webmention endpoints, one advantage of something like .well-known is that you can check once per domain rather theoretically having to check once per resource. I vaguely remember [snarfed]2 mentioning something about bridgy aggressively caching webmention endpoints, was that caching done per-domain?
#
[snarfed]2
tantek yup, BF is at least a technical power user thing
#
[snarfed]2
the docs don't assume users write their own code to send wms, but they do assume technical understanding of those building blocks
#
[snarfed]2
barnaby yup, per domain and scheme
#
[snarfed]2
2h expiration
#
barnaby
[snarfed]2: do you also cache negative results, or will bridgy keep trying to find webmention endpoints if one resource at a particular domain doesn’t have one?
#
[snarfed]2
negative results are also cached. home page vs other pages are separate, otherwise domain-wide
#
barnaby
in my particular case with the link-heavy post, caching a negative result would have been necessary as the post contained hundreds of links to different pages on a domain which didn’t support webmention
#
[schmarty]
i forget, does bridgy have a blocklist of domains that it never checks?
#
[schmarty]
ooh holding space for future .onion integration :star-struck:
#
barnaby
wow that list is fascinating. I love the long list of domains which accidentally mention @t xD
#
barnaby
or rather, which are often associated with @t mentions
#
[schmarty]
yes this list is very good snarfed++
#
Loqi
snarfed has 53 karma in this channel over the last year (98 in all channels)
#
[snarfed]2
barnaby oh I think that was just that one, pet.t-com.ne.jp
#
barnaby
ah I see, I thought the comment applied to more of the following domains
#
[tantek]
what is follow your nose
#
Loqi
It looks like we don't have a page for "follow your nose" yet. Would you like to create it? (Or just say "follow your nose is ____", a sentence describing the term)
#
[tantek]
follow your nose is an intentional local and web-centric methodology for designing [[discovery algorithms]] that start with a specific URL that you want to discover things about, and then inspect its contents, often for specific hyperlink [[rel]] values to predictably discover information about and resources for that URL, in contrast to the ”[[well-known]]” approach of instead checking outside the URL, like the domain of a URL wi
#
[tantek]
hard-coded path, for information.
#
[snarfed]2
remind me of the use case for advertising different wm endpoints (or none) per page? webmentions already include the target URL, which lets you change behavior per page in the same way
#
[tantek]
barnaby, if you'd like to capture your experience with "follow your nose" style discovery / contrasts, please add to a new Brainstorming or Thoughts section on /follow_your_nose
#
barnaby
the only reason my webmention endpoints are ever different across my domain is that they contain a token parameter and expire after a certain amount of time
#
aaronpk
oh i used to do that, but i shut it down beceause it wasn't providing any benefit
#
[tantek]
it was about time we documented this because we use it a lot
#
[tantek]
will add a few links here in chat inline because they add more perspectives / are interesting to us devs who work on this
#
Loqi
ok, I added "https://www.w3.org/wiki/FollowYourNose" to the "See Also" section of /follow_your_nose https://indieweb.org/wiki/index.php?diff=85185&oldid=85184
#
barnaby
aaronpk: yeah, I remember it being a vague community recommendation for spam prevention a while ago, but it looks like it’s not as relevant as we thought it would be
#
Loqi
[preview] [Aaron Parecki] Results from 6 Months of Expiring Webmention Endpoints
#
barnaby
oh cool, didn’t know you had written up data about it! aaronpk++
#
Loqi
aaronpk has 34 karma in this channel over the last year (107 in all channels)
#
[tantek]
follow your nose << 2002 possible first use of concept/phrase: https://lists.w3.org/Archives/Public/www-webont-wg/2002Oct/0162.html (CW: [[Semantic Web]])
#
barnaby
I don’t recall any discussion of concrete use cases for having different webmention endpoints per resource. I think it’s mostly a technical consequence of the follow-your-nose discovery approach, coupled with no explicit way of saying “use this webmention endpoint for all pages on this domain”
#
[tantek]
barnaby, yes exactly this: "technical consequence of the follow-your-nose discovery approach", please expand on what you mean by that in the /follow_your_nose article!
#
barnaby
[tantek]: sure, wanted to discuss it a little further here in case I’m missing something
#
[schmarty]
i think of publishing a wm endpoint per-page as a way of indicating whether the page/post accepts webmentions or not.
#
[tantek]
barnaby, totally ok to capture such "half-formed" thoughts/questions in a "Brainstorming" section with such caveats
#
barnaby
it’d be interesting to look at the indiemap data to see how many (if any) domains use different webmention endpoints for different pages, and if so, what they look like
#
barnaby
because it seems like there are clear patterns in webmention endpoint usage, and potentially some performance benefits to be had by either making them explicit, or recommending them
#
barnaby
and potentially some algorithms which could be proposed, e.g. “cache webmention endpoints per-domain, attempt to use that endpoint for all webmention requests for that domain, if sending a webmention for a particular resource throws an error then refetch a webmention endpoint from that page and retry”
#
[snarfed]2
true!
#
barnaby
all of the recent discussion around mastodon and the thundering herd gives us the opportunity to try to mitigate those problems in the protocols/projects we maintain
#
[tantek]
follow your nose << Advantage: more predictable portability. When you move a document that is on a one site to another, everything ”about” that document moves with it, there’s no need to check ”other places” (whether at the root of the domain or somewhere else) for ”other things that need to move with it”.
#
Loqi
ok, I added "Advantage: more predictable portability. When you move a document that is on a one site to another, everything ”about” that document moves with it, there’s no need to check ”other places” (whether at the root of the domain or somewhere else) for ”other things that need to move with it”." to the "See Also" section of /follow_your_nose https://indieweb.org/wiki/index.php?diff=85189&oldid=85188
#
GWG
[tantek]: I was using Twitter for testing because I had an old test account... didn't create a test Mastodon or such yet
#
barnaby
[tantek]: I feel like that advantage is a double-edged sword, at least for webmention endpoint discovery. If you’re migrating websites, there’s a good chance that your webmention endpoint would change, in which case it’s easier to update it in one place (a la .well-known) than it is for every single document. It certainly applies for rels which are somehow specific to that document, so maybe it’s worth categorising rels into document-specific
#
barnaby
and domain-specific somehow?
#
barnaby
not sure if there’s an existing mechanism for doing that
#
barnaby
but it’s something which fuzzy algorithms like the one I mentioned earlier could use to keep the advantages of both
#
barnaby
e.g. for certain rel values, assume that they’re domain-scoped until they stop working, at which point parse them from individual documents, to fall back to potentially different local rels
#
GWG
[tantek]: I am going to have to find more places to posse to
#
barnaby
in some cases we already deprecated page-scoped rels in favour of mf2 properties (e.g. in-reply-to) so maybe that’s a pattern which could be built upon
#
[snarfed]2
also those benefits and drawbacks both assume static sites with html pages that aren't auto-generated, which are less and less common these days
#
[tantek]
barnaby, whether it is easier to move or not, the larger point is that with "follow your nose" discovery, you at least know ALL the places that need updating very clearly there in the doc
#
capjamesg
[snarfed] Are the images supposed to show up in the item.description element?
#
aaronpk
just had an interesting feature request come up for indielogin.com. wanting to use an IndieAuth provider as one of the rel=me providers
#
[tantek]
in contrast to, who knows what random well-known hardcoded paths will be invented in the future
#
[tantek]
you literally have no idea when you move a doc that depends on some random number of "well-known" rando strings
#
[tantek]
you have no idea when you’re "done" with moving a doc
#
[tantek]
that's the problem. fragility
#
barnaby
sure, I’m not in favour of .well-known, just trying to figure out possible optimisations for follow-your-nose
#
[tantek]
optimizations++ especially when based on experience / measuring first!
#
Loqi
optimizations has 1 karma over the last year
#
[tantek]
I say the advantage stands
#
barnaby
because it does end up requiring a lot more network traffic, and as we’ve seen from the thundering herd of mastodons, people tend not to enjoy being on the receiving end of that!
#
[tantek]
follow your nose << Advantage: is anti-fragile, that is, when a document only depends on follow your nose discovery methods, you can fully inspect it for any and all dependencies, whereas in contrast, for anything that depends on any hardcoded domain-based paths (like well-known), you have no idea when you are ”done” moving something or what new dependencies may have accrued by new hardcoded paths.
#
Loqi
ok, I added "Advantage: is anti-fragile, that is, when a document only depends on follow your nose discovery methods, you can fully inspect it for any and all dependencies, whereas in contrast, for anything that depends on any hardcoded domain-based paths (like well-known), you have no idea when you are ”done” moving something or what new dependencies may have accrued by new hardcoded paths." to the "See Also" section of /follow_your_nose https://indieweb.org/wiki/index.php?diff=85191&oldid=85190
#
[tantek]
what is thundering herd
#
Loqi
It looks like we don't have a page for "thundering herd" yet. Would you like to create it? (Or just say "thundering herd is ____", a sentence describing the term)
#
[tantek]
^ your turn barnaby 🙂
#
barnaby
ha ha I’m working on it!
#
barnaby
and while I can appreciate the motivation, I prefer to be able to compose my wiki updates in a stress-free environment 😁
#
[tantek]
I feel that, I mean updates are one thing but creating a whole new page! The pressure! 😂
#
[KevinMarks]
Now I want to create /.poorly-known for deprecated endpoints
#
[snarfed]2
[KevinMarks]++
#
Loqi
[KevinMarks] has 13 karma in this channel over the last year (51 in all channels)
#
[snarfed]2
capjamesg yes! they're there now, right?
#
[tantek]
[KevinMarks] make that /.well-abandoned and I'm in 🙂
#
[tantek]
tempted to redirect /.well-actually to indieweb.org/follow_your_nose
#
[tantek]
follow your nose << Advantage: can work even when documents are places without domains or root paths that the document owner can control, like local multi-user file systems, or alternative non-DNS systems like [[IPFS]], in contrast to [[well-known]] which makes assumptions about domains and root control.
#
Loqi
ok, I added "Advantage: can work even when documents are places without domains or root paths that the document owner can control, like local multi-user file systems, or alternative non-DNS systems like [[IPFS]], in contrast to [[well-known]] which makes assumptions about domains and root control." to the "See Also" section of /follow_your_nose https://indieweb.org/wiki/index.php?diff=85193&oldid=85191
#
capjamesg
[snarfed] Is there any reason the image isn't an RSS property?
#
[KevinMarks]
RSS does not define one (except as enclosure, which would not do what you want)
#
[tantek]
[snarfed] in the interim, if there’s an easy/simple recipe for using curl asynchronously to send a webmention, might be worth adding a subsection / short example such "async curl" here: https://indieweb.org/Webmention-developer#How_to_send_webmentions_with_cURL
#
[tantek]
and then pointing Bridgy Fed docs to it as a heads-up for folks
[chrisaldrich] joined the channel
#
barnaby
the one case I’m not so sure about in that mitigation is whether to cache negative cases, and if so for how long. There could definitely be cases where one page on a site doesn’t have a webmention endpoint, but others do, and if the first one is fetched and the negative result cached, results in missed webmention sends
#
barnaby
some potential solutions could involve either caching negative results for a shorter time, or to only cache negative results if they’re from pages with mf2 markup on
Jan-LukasElse[m] left the channel
#
barnaby
that way no webmention endpoint being found on a “plain” page with no mf2 wouldn’t prevent checking for webmention endpoints in future pages from the same URL
#
[tantek]
definitely an area worthy of experimentation and if a consensus best practice emerges, documenting as a SHOULD in a spec update
#
[tantek]
good point
Nuve, bterry2 and tiim joined the channel
#
[snarfed]2
afaik the spec is a few steps behind any of this, it explicitly doesn't allow caching endpoint discovery at all, full stop
#
[tantek]
hmm, that's probably worth at least a change from MUST to SHOULD to at least allow caching endpoint discovery
#
[snarfed]2
yes!
#
[tantek]
[snarfed] since you’ve implemented this for good reason, want to file an issue on https://webmention.net/draft/#sender-discovers-receiver-webmention-endpoint, like s/The sender MUST fetch the target URL /The sender _SHOULD_ fetch the target URL/
#
gRegor
Similar to schmarty, I interpreted the spec as if a page doesn't advertise a wm endpoint, it doesn't accept them, so not sure about the domain-scoped. "In order to receive a Webmention, there must be a URL that advertises the Receiver's Webmention endpoint."
#
[tantek]
and perhaps a subsequent paragraph on something like: The sender MAY re-use a cached endpoint from a previous successful execution of this endpoint discovery algorithm on the same site. [insert dfn of "same site" for these purposes]
#
barnaby
gRegor: the optimisation I propose is to improve the case where you’re sending webmentions to many URLs on a domain which you haven’t necessarily fetched them all already, e.g. in a long blog post
#
barnaby
and with the negative cases, potentially also reducing fetches to pages which are unlikely to declare a webmention endpoint based on prior experience
#
gRegor
Understood, just saying the spec seems to indicate you can only know if you can send a webmention for a URL by checking if it has an endpoint listed, so not sure the domain-scoped caching fits
#
gRegor
I thought I had some pages on my site that don't accept wms, trying to find them now as an example
#
barnaby
in many real-world cases, assuming a domain-scoped webmention endpoint works, and ultimately its up to the webmention endpoint to return errors if it recieves requests for unsupported resources
#
gRegor
True, worst case is a 4xx error I guess
#
[schmarty]
In the case that a particular post has no WM endpoint (but the site does in general) this kind of caching is trading a "wasted" fetch for a "wasted" WM POST.
#
gRegor
I do think that line I quoted would need to be changed in the spec though if it's expanded to allow that via "should"
#
barnaby
yep, and generally speaking a wasteful wm POST is likely lighter on resources both for the render and receiver
#
[schmarty]
I'm not sure I agree with that!
#
[tantek]
barnaby, seems like there may be a use-case of sites wanting to clearly advertise that here is a link to my webmention endpoint for this page *and my site* so you can cache it accordingly without having to do re-discovery
#
barnaby
[schmarty]: because post pages are more likely to be served from a cache than requests to a WM endpoint?
#
aaronpk
i would agree with that. a GET on the page might require one or more DB queries to render the page. rejecting a webmention request for an unsupported URL likely doesn't require any lookups
#
[tantek]
thinking out loud, I wonder if there would be some way to combine rel values to express that semantic, e.g. rel="webmention site"
#
aaronpk
if you have a good caching strategy on your site that would of course not be true
#
barnaby
[tantek]: yep, a way of explicitly declaring a domain-scoped endpoint would solve this without additional heuristics, provided it was widely supported
#
[schmarty]
whether it's "lighter" depends on how the WM receiver is implemented. for example, webmention.io will process the entire webmention and store it. my site would then have that data and then later need to filter it out.
#
[tantek]
for security reasons you have to loop in the root
#
gRegor
Mine would require a db hit either way. No strong opinion on which is more resources intensive though
#
barnaby
although allowing individual pages to declare a domain-scoped rel could lead to security issues
#
[tantek]
and then the protocol would be for the Sender (code doing the discovering) to then do webmention discovery on the "site" (e.g. root) and IF that discovery returns the same endpoint, then re-use it / cache it for all other paths on that domain
#
barnaby
and verification would essentially involve re-inventing .well-known 🙈
#
[tantek]
that way a directory inside a site couldn't takeover webmentions for the whole site
#
barnaby
but at least looking up the root is looking at a real webpage rather than some weird hidden URL
#
[schmarty]
what tantek said: fetch for the page, confirm on the root, cache if it looks good.
#
[tantek]
barnaby, if you prefer, we combine this with rel-home discovery 😛
#
barnaby
and there’s a precedent for using the root for domain-scoped endpoints at the root from indieauth and micropub/sub
#
[schmarty]
imo that can probably be done without a special signal, if you are being aggressive about endpoint caching.
#
[tantek]
as in, use rel-home discovery to find the "site" root for a page
[jamietanna] joined the channel
#
[jamietanna]
How does one unfollow with Bridgy Fed?
#
gRegor
I like the add'l rel for site-wide, similar to how rel-me lets us specify 'authn' for select ones
#
[tantek]
no weird hidden directories needed
#
[tantek]
gRegor precisely
#
[snarfed]2
[jamietanna] first we have to describe how one unfollows in the indieweb 😎 https://github.com/snarfed/bridgy-fed/issues/320
#
[snarfed]2
er, decide
#
Loqi
[preview] [datakurre] #320 Unfollow with webmention
#
[tantek]
what is unfollow
#
Loqi
Unfollow is the opposite action of following someone, typically on a silo, commonly implemented on asymmetric follow silos such as Instagram, Google+, initially pioneered by delicious, Flickr, and Upcoming, but most greatly popularized by Twitter in contrast to the mutual friending model from explicitly "social" silos from Friendster to Facebook https://indieweb.org/unfollow
#
[tantek]
well that needs some gardening
#
[snarfed]2
This has a bit on negative caching, but pretty narrow: https://github.com/w3c/webmention/issues/48
#
Loqi
[preview] [aaronpk] #48 Recommend a backoff strategy for discovering webmention endpoints
#
[tantek]
yeah that's a bit different
#
capjamesg
TFW you crawl 10k pages and you realize you forgot to save the content...
#
[snarfed]2
Oof, hugops
#
capjamesg
I have the vectors.
#
capjamesg
But not the text and metadata, which is to say that the 40MB of vectors cannot be used.
#
capjamesg
The goal is for me to have a reliable site-based content recommendation system.
nedzadhrnjica joined the channel
#
capjamesg
Search infra is hard.
#
Loqi
[preview] Showing content related to Building a search engine for my blog
#
capjamesg
I have two indexes done so far.
#
[snarfed]2
capjamesg++ on launching novacast!
#
Loqi
capjamesg has 32 karma in this channel over the last year (83 in all channels)