#dev 2024-09-23

2024-09-23 UTC
AramZS and Dryusdan joined the channel
#
[tantek] capjamesg[d], Technorati was known for *real time search* which literally no one else did, except Google prototyped/shipped an "80%" solution, enough to seem like "competition" to Technorati, then a few years after Technorati folded (for many reasons, can share more informally), Google shut off their real time Blog Search.
#
[tantek] So today, literally no one does cross-site real time search
#
[tantek] sure you can search Twitter using Twitter in near realtime. and you can use Google Alerts for some tiny fraction of the web (various news publications), but it's something that doesn't exist, anywhere
#
aaronpk didn't google even drop their realtime twitter search they did a while ago?
#
[tantek] yes that too
#
[tantek] Google Search is pretty much C grade right now, barely usable 70% of the time.
#
[tantek] other web searches are of even lower quality
#
[tantek] so "technorati for source code repos" would probably mean some way to index changes to source code repos in real time, i.e. show search results within seconds of when a merge request was landed
[tw2113] joined the channel
#
[tw2113] ugh, i dread any time i need to perform a search
#
catgirlin.space i find kagi to be pretty good i think,,,,,
#
catgirlin.space am confused what you mean by real time search exactly [tantek]
#
catgirlin.space like, doing a search on funny search engine would go and do a site search on every website it knows about? orrrr
#
[tantek] no it already has it all indexed in real time
#
[tantek] so doing a search checks that index and returns you a result
#
catgirlin.space ooo
#
[tantek] this is what Technorati did for (nearly?) all blogs back in the mid-2000s. millions of blogs
#
catgirlin.space is that uh, kinda similar-ish to what indexnow does? telling search engines that something changed and then they can crawl it again if they want to,,,,
#
[tantek] you could blog something and minutes later, eventually seconds, someone could search for it and find your blog post
#
[tantek] nope, not at all because no search engines actually crawl things in real time
#
[tantek] or "promptly" in response to being told "something changed"
#
[tantek] mostly those signals are ignored I find
#
catgirlin.space wait was technorati just, constantly polling feeds,,,, for like, every single blog then? [woozycat](https://cdn.discordapp.com/emojis/1183246214595092561.webp?size=48&quality=lossless&name=woozycat)
#
catgirlin.space [edit] wait was technorati just, constantly polling feeds,,,, for like, every single blog then? [woozycat](https://cdn.discordapp.com/emojis/1183246214595092561.webp?size=48&quality=lossless&name=woozycat)
#
[tantek] no it was not polling either
#
[tantek] catgirlinspace you can read the http://enwp.org/Technorati article for some more background if you're curious
#
catgirlin.space huh
#
[tantek] what is Technorati
#
Loqi Technorati was a real-time blog search engine that provided date-ordered results for text phrases or links, typically within seconds of when people published on their blogs https://indieweb.org/Technorati
#
[tantek] and that
#
[tantek] so no, don't try to imagine what "was technorati just" because you're very unlikely to figure it out from first principles in a matter of seconds — it was built by a small handful of very clever engineers over months and improved over a few years
#
catgirlin.space so confused how it knew about new posts then. skimming the wikipedia article it doesnt seem to detail that?
#
catgirlin.space > Tantek Çelik was the site's Chief Technologist.
#
catgirlin.space omg that's you, that's so cool,,,,,
#
[tantek] !tell [snarfed] ah I see my posts are gone from lots of tag searches now. that's really too bad / sad as that's one place people do (re-)discover posts, when they themselves blog about a topic, and then go see what others have said on that topic previously. re: "still looking at exactly what/how I can repair" - here is a suggestion: if I resend a webmention to BF for an old post of mine, BF should go deliver it to everyone. if BF thinks it
#
Loqi Ok, I'll tell them that when I see them next
#
[tantek] already delivered it, then BF should send an UPDATE. then if it still does not show up in Masto profiles/tag-searches, then those are Masto bugs and you can help with filing them
#
[tantek] I realize that still requires manual work on my part to re-webmention BF for a bunch of my posts (all of them since Oct 2022 lol?) however, at least that will shake out a bunch of bugs in BF/Masto interactions and then we can file bugs and advocate for Mastodon folks to fix the Mastodon problems with properly handling UPDATEs
#
[snarfed] yes! BF already does all that, including updates, and it also compares what it fetches from your site to what it last delivered. in this case, they'd be the same, so it wouldn't send any updtes
#
[tantek] So I'd have alter blank space or something before re-webmentioning?
#
[tantek] like add a blank space at the end of a line?
#
[tantek] presumably BF does not do any "clean-up" before it "compares what it fetches from your site to what it last delivered"
#
[snarfed] right
#
[tantek] so lets file those bugs
#
[snarfed] but I'm also looking at doing this manually on my end, which will be easier and more efficient than you modifying a bunch of posts and sending wms
#
[tantek] because that has been frustrating for too long (Mastodon ignoring new tags on UPDATEs etc.)
#
[tantek] ah that of course would be better
#
[tantek] LMK how I can help!
#
[snarfed] alternatively since the posts already exist, we can find and construct the POSTs for Mastodon's search for each post URL, ie https://fed.brid.gy/r/https://tantek.com/... , since searching for those makes the instance re-fetch the post
#
Loqi [preview] Tantek Çelik
#
[tantek] ah yes that makes A LOT more sense
[morganm] joined the channel
#
GWG I was appreciating the whole..."That's you...that's so cool."
#
[tantek] [snarfed] looks like reply-threading was broken as well when posts got deleted 😞 e.g. this post was a reply to my three phase pomodoro post, used to display it as a reply-context but now looks oddly isolated, and the @-@ for my name also doesn't link. Not great 😕 https://aus.social/@subtl/113105670128002197
#
GWG I'm so used to [tantek] and other people here I forget some times their contributions
#
Loqi [preview] [Stephen] @tantek.com@tantek.com if you don't mind me asking, what kinds of things go in the different categories? Does that mean you've only got 20 minutes per hour to do "computer stuff"?
#
[tantek] GWG, it's ok to forget long past contributions. The internet keeps reasonably good records. Focus on the now
#
[tantek] !tell [snarfed] how about broken reply contexts due to posts being deleted? e.g. https://aus.social/@subtl/113105670128002197 used to show my three phase pomodoro post above it as a reply context but no longer does and the @-@ for my name was also unlinked which maybe is a different BF bug?
#
Loqi Ok, I'll tell them that when I see them next
#
Loqi [preview] [Stephen] @tantek.com@tantek.com if you don't mind me asking, what kinds of things go in the different categories? Does that mean you've only got 20 minutes per hour to do "computer stuff"?
jjuran, Jean, [Jo], ttybitnik and Jon2 joined the channel
#
[snarfed] [tantek] agreed, definitely not great. apologies again
#
[snarfed] your handle being unlinked on aus.social looks like it's because your profile isn't cached on that instance any more, maybe because it doesn't have any activities from you any more? not sure
#
[tantek] [snarfed] do replies get reconnected when the messages are resent as UPDATES or ... ? (they should right?)
#
[snarfed] I sent it a `Create`, your profile is now there again
#
[snarfed] if you mean the @-mention, I don't know, "reconnected" is UI-specific, off-protocol
#
[tantek] no I mean the in-reply-to -> then showing up as a reply-context
#
[snarfed] ah ok. maybe? same point
#
[tantek] worth experimenting with and seeing if it doesn't then filing a Mastodon bug accordingly. if it's literally the same permalink, it should "reconnect" like that
#
[tantek] resiliency, forgiveness etc.
#
[snarfed] yup!
[schmarty] and AramZS joined the channel
#
[mattl] What are people using for sending email to users these days?
#
Loqi It looks like we don't have a page for "people using for sending email to users these days" yet. Would you like to create it? (Or just say "people using for sending email to users these days is ____", a sentence describing the term)
#
capjamesg[d] [mattl] I haven't used it but https://postmarkapp.com/ looks pretty cool.
#
capjamesg[d] [edit] [mattl] I haven't used it but https://postmarkapp.com/ looks pretty cool.
#
jimw I either just use the free trial level of Postmark (https://postmarkapp.com), or through the SMTP server for my Fastmail account, but I send very few emails.
#
capjamesg[d] The developer experience looks much better than anything else I have used.
#
jimw I have used Postmark as a paid service before (for transactional emails from an ecommerce site) and it was great.
#
pcarrier[d] [mattl] if you run your own server, and you only want to send, and your ISP doesn't block SMTP unconditionally, it's fairly easy to set up a service those days
#
[mattl] those emails always seem to wind up in people's spam folders, and I don't want to run a mail server in 2024.
#
jimw I used Sparkpost (now called Bird?) before Postmark, and it was fine, but they seemed to have more issues with deliverability at the time.
#
pcarrier[d] [mattl] did you set up SPF? the mail-over-https companies don't have any magic tricks (well DKIM too but doesn't help delivery all that much in my experience)
#
[mattl] I think I did. Going to try Amazon SES.
#
pcarrier[d] (I really don't want to deal with mailservers either, except for outbound-only opensmtpd which has made my life easier than the vendors I tried. granted, I'm very comfortable around DNS, including SPF and PTR records, and Unix-like systems)
AramZS joined the channel
#
[mattl] I'm comfortable with that stuff but it used to be my full time job and I don't want to do that stuff anymore.
#
catgirlin.space postmark has been great for me
gRegor joined the channel
#
[snarfed] yeah owning your email deliverability is maybe the worst kind of owning anything
#
aaronpk +1
#
Zegnat Just to throw another name on the pile, I was looking at Scaleway, who are the only European provider I know for transactional email. https://www.scaleway.com/en/transactional-email-tem/
#
[mattl] went down the SES rabbit hole. they're now reviewing my account.
#
[mattl] i was using sendgrid for years but they make it very hard to contact them
[Joe_Crawford] joined the channel
#
[Joe_Crawford] There was that kickstarter for a book that purported to teach you to do the ugly work of really running your own mailserver. It looks like it's in the gray area after the Kickstarter and before publication so it seems not obviously available. https://www.kickstarter.com/projects/mwlucas/run-your-own-mail-server
#
[snarfed] "fun"
#
[Joe_Crawford] The book's cover strikes the balance of cartoon fun in the context of World War I-style carnage and devastation and pain.
#
aaronpk wow
#
pcarrier_com[d] thankfully for me, local outbound is a tiny part of the problem space, and we have opensmtpd since 2013. never am I ever touching exim, postfix, or imap services ever again
#
[mattl] i'd look at that if/when I move away from exim4
#
[snarfed] ok [tantek] I have a bit of good news. I forgot that BF's webmention handler supports a force=true param. I sent it a wm for https://tantek.com/2024/264/t1/cc-nt-for-no-training-llm-genai with force=true, and it recreated that post in all downstream instances. I also confirmed that those instances "reconnect" reposts and likes
#
[snarfed] I'll do that for your past posts
#
Loqi [preview] [Tantek Çelik] Dear Creative Commons (@creativecommons.org @creativecommons@mastodon.social @creativecommons@x.com), Can we have CC-NT licenses for no-training (ML/LLM, GenAI in general), just like we have CC-NC for non-commercial? My previous post¹ reminded me...
AramZS joined the channel
#
capjamesg[d] [snarfed] How do you manage larger projects like Brid.gy?
#
capjamesg[d] I am hesitant to make bigger things in case I take something on and can't keep it up.
#
[snarfed] hmm, what do you mean specifically by "manage"?
#
[snarfed] I mean, one day at a time is the cliched but true answer
#
[mattl] http://Libre.fm has 372,704 registered users 😕
#
[snarfed] discovered [tantek]'s semi-hidden b* and f* posts just now
#
[snarfed] eg https://tantek.com/2023/100/b1/ , https://tantek.com/2024/050/f10
#
Loqi [preview] [Tantek Çelik] Warn if event or Etherpad link already archived
#
capjamesg[d] [snarfed] I guess what happens if there is an anomaly or a brings something down?
#
capjamesg[d] The prospect of hosting projects for others stresses me out a bit.
#
[snarfed] yup. a little of that stress is probably ok. you want to take it seriously! or at least, as seriously as you choose to
#
[snarfed] you can communicate your level of maintenance to users loudly up front, what they should expect, etc
#
[snarfed] also nothing is big at the beginning, things get big gradually over time
#
[mattl] users will demand 100% of your time if they can
#
[snarfed] yup, which is their prerogative. but it's your prerogative to choose (hopefully proactively) how much you put into it
#
[mattl] I just closed registration on http://Libre.fm for two years because one user was such a needy jerk to me.
#
[snarfed] and if they're not paying or donating to you, you have much less obligation
#
[snarfed] [mattl] whoa! intense
#
[mattl] They went WAY too far. Started sending emails to my job, etc.
#
[snarfed] wowww yeah not ok
#
[mattl] Because they operate their own mail server and couldn't get a registration email from me.
#
[snarfed] phew, they needed to touch grass
#
[mattl] they need to be mailed a bobcat in a box
#
[snarfed] lol
#
[mattl] https://xkcd.com/325/
#
Loqi [preview] [XKCD] A-Minus-Minus https://imgs.xkcd.com/comics/a-minus-minus.png
#
[Joe_Crawford] "also nothing is big at the beginning, things get big gradually over time" snarfed++
#
Loqi snarfed has 51 karma in this channel over the last year (102 in all channels)
barnaby, AramZS and ttybitnik joined the channel
#
[tantek] [snarfed] lol re: "semi-hidden b* and f* posts"
#
[tantek] they're not hidden! they're part of my home page and my sequential navigation!
#
[tantek] I've only been pinging BF for posts that are top-level or responses to Mastodon posts
[aciccarello] joined the channel
#
[snarfed] oh they show up on your homepage! even the blank-looking likes? like https://tantek.com/2024/050/f10 ?
#
[tantek] e.g. if I file a GitHub issue, in-reply-to a repo or issues URL, presumably BF won't do anything with it because it's a reply not to any fedi-capable URL(s)
#
[snarfed] oh nm it's not blank, just small font
#
Loqi [preview] [Tantek Çelik] likes @beep@follow.ethanmarcotte.com reply
#
[snarfed] right
#
[tantek] [snarfed] yeah my permalink display of likes needs a bunch of improvement
#
[snarfed] np, everything is a work in progress
#
[tantek] to answer yes my likes show up on my home page. search for "likes " and I see on there now
#
[snarfed] 👍
#
[tantek] see *one there
__DuBPiRaTe__, AramZS and strugee joined the channel