#dev 2024-05-28

2024-05-28 UTC
amyiscoolz, PuercoPop, geoffo, gerben, peterkaminski, box464, jonnybarnes, GuestZero and sebbu joined the channel
#
capjamesg[d]
My new static site generator can render 4000 markdown files in 1.3-1.6s. 11ty reports 1.93s for generating 4000 markdown files.
#
capjamesg[d]
(Not apples for apples because I'm not sure what contents were in the 11ty test files.)
GuestZero joined the channel
#
[Jo]
thats crazy!
#
[KevinMarks]
Hugo is one to benchmark against ime
#
capjamesg[d]
I'm not as fast as Hugo, and I'm probably not going to get it that fast.
#
capjamesg[d]
I'm using jinja2 for templating right now. That and the `frontmatter` library, which depends on `yaml`, constitute most of the generation time.
#
capjamesg[d]
[KevinMarks] You wouldn't happen to know ways to speed up jinja2? 😄
#
[KevinMarks]
I haven't looked at in detail, no. You could use the python profiler and see where it's spending time, and check that the template compiler and bytecode cache are behaving the way you want.
#
[KevinMarks]
the other way that might be fun would be to use http://honeycomb.io to instrument the app and look at the traces
#
capjamesg[d]
I have been using cProfile and snakeviz thus far.
#
[KevinMarks]
reading the docs, it looks like Jinja2 is designed to cache and reuse templates, so it should be mostly working in memory while building your site
AramZS joined the channel
#
capjamesg[d]
Yeah. It looks like < 0.2s is from jinja now I'm looking further. 0.3s is from python-frontmatter, which I use to process yaml.
#
capjamesg[d]
In the `aurora/graph.py` file.
#
Loqi
[preview] [capjamesg] aurora: A static site generator implemented in Python.
#
[Paul_Robert_Ll]
I wonder if using a simpler (yet more constraining) template language like Moustache or Handlebars might be quicker than Jinja2?
#
capjamesg[d]
The biggest issues now are file management (0.6s for 4k files).
#
superkuh
If you make everything HTML instead of mark-up the combining the HTML fragments is really fast.
[tantek] joined the channel
#
Zegnat
Sometimes a good thing for iops to be your bottleneck, not a lot left to do. On the other hand, depending on what you develop in, it can sure be a pain to try and gain any wins there, haha
#
capjamesg[d]
Zegnat Any Python file IO performance tips? Stack Overflow and ChatGPT have thus far been mostly unhelpful.
#
pcarrier[d]
capjamesg[d] spawning threads
#
capjamesg[d]
Threading adds too much overhead.
#
superkuh
Could just do like science.org/aaas does and only allow "feed readers" if they're incorporated (ie, feedly).
AramZS joined the channel
#
[snarfed]
Sonit moving AP protocol details here
#
[snarfed]
er, Soni
#
[snarfed]
...Mastodon renders notes as full text, but articles as only title + link, so even if the AP actor account was set to discoverable, I don't know if Mastodon full text indexes articles
#
[snarfed]
worth testing
#
Soni
y'know, we hate how "move things you want ppl to set up to -dev"
#
Soni
it really helps to kill the project
#
Soni
like yeah we get it don't use jargon, but the solution is not to move it here it's to not use jargon
#
[Joe_Crawford]
It's about consideration of the time of others in the community
#
Soni
(btw you should absolutely change the Loqi message to remind ppl not to use jargon instead of just moving everything over to here)
#
[Joe_Crawford]
So picking up where the search indexing conversation was...
#
[Joe_Crawford]
what is bridgyfed
#
Loqi
Bridgy Fed connects IndieWeb sites with Mastodon & other fediverse instances, allowing users on those networks to follow your IndieWeb site, including receiving notifications, reading, and replying to your posts https://indieweb.org/BridgyFed
#
capjamesg[d]
Soni, #indieweb is intended to be accomodating for everyone, irrespective of their knowledge of programming or development. Any technical discussions should happen here.
#
Soni
well we don't want the technical discussions, we want something we can install and solve our problem with
#
Soni
but if you want the techncial discussions: if universal web search is colonialism and nobody's gonna bother to make a local web search (index sharing) protocol then the easiest solution is to piggyback off of mastodon and shove your entire corpus into it and let it build its own index (as inefficient as that may be)
#
Soni
if we can't make it good, then we should at least make it work
#
[Joe_Crawford]
BridgyFed strikes me as something to get you closer to the goal of visibility in Fediverse/Mastodon
#
Soni
we don't want to deal with webmentions or replies or whatever
#
Soni
and relying on a third-party is unideal
#
[Joe_Crawford]
If you’re unwilling to reckon with the details I’m not sure there’s a plausible way to satisfy you
#
Soni
didn't someone come up with a way to make mastodon posts using a static webserver?
#
[Joe_Crawford]
I do think I’ve seen simple implementations of Fedivdrse protocols
#
[snarfed]
[Joe_Crawford] is right. Soni I agree, getting your blog posts to show up in Mastodon search results would be awesome! but there may not be something you can install and solve your problem with right now
#
[Joe_Crawford]
Further while you don’t need webmention, replies may be part of the bargain of being in Fediverse. But whether they propagate to your site would be up to you. I think.
#
[snarfed]
hence us talking about technical details
#
Soni
so for the static webserver approach we'd just have 2 web roots based on the Accept header
#
[snarfed]
the first problem is that there's no "one" Mastodon to "shove a corpus" into. there are 25-30k different instances across the fediverse, each with their own separate search index. so you'd have to determine which of those you want your posts to show up in
#
Soni
and then we'd have a tool to handle pushing it out to instances
#
[snarfed]
the usual answer is, you send to the instances your followers are on
#
Soni
okay, so we guess we do need to handle follow requests somehow...
#
[snarfed]
if you want something self hosted, there are definitely "minimal activitypub" projects for adding support to web sites! pcarrier was asking about the same thing the other day. https://github.com/aaronpk/nautilus , https://humungus.tedunangst.com/r/honk , https://codeberg.org/seppo/seppo#seppo , etc
#
Soni
we don't need a proper fedi server for that, we can just store them (with limits) and go through them manually
#
aaronpk
i'm not sure you can do that manually, the follow request is a bit of a back and forth exchange
#
aaronpk
but yes generally you need to implement a base level of activitypub functionality for anything to work with mastodon. it's not as simple as just publishing a feed or handing a post request here and there
#
Soni
a follow request is just a POST message tho, isn't it?
#
[snarfed]
you need to serve both webfinger and an AP actor, with a keypair, before they'll send you that request
#
[snarfed]
and your accept request back will need to be signed with the private key
#
aaronpk
right, you also need to send a signed request back to acknowledge the follow request
#
Soni
yeah but that's the easy part, and the signed request can be sent at any later date
#
[snarfed]
aaronpk is right, it's doable, but it does take some setup, it's not as simple as isolated POST requests and data in HTML like webmention + microformats
#
aaronpk
yes you can delay the accept
#
Soni
we can just store the request.json and use a different script (not part of the webserver) to send the accept
#
Soni
(this different script would also handle pushing out the posts)
GuestZero joined the channel
#
Soni
(and this different script probably doesn't need to run on the actual server?)
#
aaronpk
sure, definitely doable. you might want to look at some of the guides out there about how to do this kind of manual thing
#
aaronpk
pretty sure i've seen one from darius kazemi?
#
aaronpk
but in general, if you want something easy to install that "just works" for people, that is the opposite of going down this path
#
[snarfed]
aaronpk++
#
Loqi
aaronpk has 50 karma in this channel over the last year (139 in all channels)
#
Soni
(why does it use fucking rsa in 2024; well anyway...)
#
Soni
so basically, yes, this should be a webserver plugin, mod_fedi or something
#
Soni
(hmm, we don't know how to make those...)
GuestZero joined the channel
#
[Joe_Crawford]
Thanks [aaronpk] I feel like I learned a lot about the pieces of this real fast. Not something I'm looking to do but knowing the pieces super helpful. And [snarfed] thanks too.
#
[snarfed]
[Joe_Crawford]++
#
Loqi
[Joe_Crawford] has 6 karma in this channel over the last year (49 in all channels)
#
[Joe_Crawford]
Aside: `text-wrap: balance` has made so many pages on my site much nicer to scan. Trying not to add it everywhere because the docs allude to a performance hit but so far I like it a lot. For headlines and short captions it's great. https://developer.mozilla.org/en-US/docs/Web/CSS/text-wrap
#
[KevinMarks]
did you try text-wrap:pretty too?
#
[Joe_Crawford]
Have not tried. Putting into my quiver for next time I'm looking at body copy! Thanks [KevinMarks]++
#
Loqi
[KevinMarks] has 18 karma in this channel over the last year (32 in all channels)
GuestZero, AramZS, gRegor and gRegorLove_ joined the channel
#
[tantek]
This: "But I really want to streamline the CMS I’m using, without having to go to a full static site build." (from https://chronosaur.us/2024/05/28/never-enough/ shared on main)
GuestZero and [aciccarello] joined the channel
#
pcarrier[d]
if someone has ideas for what/how to build a minimally dynamic hosting provider
#
pcarrier[d]
something that would allow for "CMSes" without eg arbitrary PHP for all visitors
#
pcarrier[d]
I'd be down to build experiments
#
aaronpk
could look into templating languages used in things like shopify for that
#
[snarfed]
also sounds like http://micro.blog's Hugo themes support
#
Soni
mastodon is a mistake tbh
#
amyiscoolz
it's the bestest and the only fediverse software wdym /sarcasm
#
Soni
it doesn't even have *audits*
#
Soni
(the feature)
#
starrwulfe
to add to the minimal activitypub instances, here’s a few I’ve played with previously:
#
starrwulfe
grunfink/snac2: A simple, minimalistic ActivityPub instance
#
IWDiscord
<s​tarrwulfe>
#
IWDiscord
<s​tarrwulfe>
#
starrwulfe
Single user activityPub based blog engine.
#
IWDiscord
<s​tarrwulfe>
#
starrwulfe
jlelse/GoBlog: Simple blogging system written in Go
#
Loqi
[preview] [jlelse] GoBlog: Simple blogging system written in Go
#
Loqi
[preview] @dev@microblog.pub
#
amyiscoolz
GoToSocial is written in go as well, but it's in alpha state rn afaik
#
[Joe_Crawford]
I keep looking at my various huge stashes of links over the years and systems like these and wonder if there's a match - if I maybe want to syndicate across services. I keep landing on the fact that what I need is a tool to manage my own links for myself. I am loathe to use custom post types to import individual links and link groups into WordPress even though that's where all the other stuff on my site keeps getting absorbed into.
#
[Joe_Crawford]
I used pinboard to absorb links I tweeted out so I have... a lot. 😕
[James_Van_Dyne] and jeremycherfas joined the channel
#
[KevinMarks]
Yeah me too. I need to make something to look them up for myself now Maciej has given up.
bret joined the channel
#
[Joe_Crawford]
I know I've mentioned a vague "let's have a music/scrobbling popup" but a "link management" showcase/discussion might be more relevant.
#
[Joe_Crawford]
or more acute, for my own selfish reasons. 🙂
#
capjamesg[d]
I would love this!
#
[Joe_Crawford]
I'll be in EU meetup (planning to) let's talk scope in there tomorrow!
#
[Joe_Crawford]
Or heck, if you have a notion James, I'm interested. 🙂
#
[Joe_Crawford]
or - sorry, not EU. UK. 😐
#
[Jo]
I like the idea of a music popup
#
gRegor
what is pinboard
#
Loqi
Pinboard is a silo for hosting bookmarks that you may POSSE to using the WordPress Syndication links plugin https://indieweb.org/Pinboard
#
gRegor
Wait, given up on Pinboard? Is it dying?
#
gRegor
Looks like it's still going, though last blog post was 2020
#
gRegor
Signing in for the first time in ages. I'm sure I have a lot of those Twitter links in there too :/
#
[Joe_Crawford]
I can't tell myself gRegor. I don't see shutdown notices in the pinboard X dot com account
#
gRegor
Yeah I thought he was pretty committed to longevity of it
#
gRegor
[KevinMarks], what did you mean by "has given up"?