#dev 2026-06-02
2026-06-02 UTC
kenichimiyake, dabeglavins, cotsuka, ax562, dabeglavins5, grufwub, NaomiAmethyst, aaronpk, [jeremycherfas] and calciume joined the channel
# 
[capjamesg] Does anyone know why a Sqlite3 database would be slow on running a query for the first time, but quicker on subsequent runs?
# [capjamesg] I have set up indexes for my query but the first run is slow.
# [capjamesg] I have been trying to figure this out for weeks but have had no luck 🤦
blacklight joined the channel
# 
[pcarrier] Page cache
# [pcarrier] Try mlocking the whole file in eg pcma to confirm
# [capjamesg] The database file?
# [capjamesg] Are there any side effects of mlocking it?
cotsuka joined the channel
# [pcarrier] yes. sure, forces it to be in RAM at all times
# [pcarrier] so your kernel doesn't get to prioritize other stuff over that memory region
# [pcarrier] but I wrote pcma because well, sometimes, it just makes sense (it was for in-memory DB servers at spotify)
# [pcarrier] at least confirm that's what the slowness is from. you can empty the page cache with /proc/sys/vm/drop_caches and/or force it to stay there with pcma or equivalent
# [pcarrier] but "slow the first time, fast shortly after" is pretty much tell tale
# [capjamesg] Sounds good! I will try this. [pcarrier]++
# [capjamesg] Wait if it puts the whole DB in memory that will not work. The file is ~20 GB right now and I only have 8 GB of RAM on the server.
# [pcarrier] yup, not an option
# [pcarrier] at the very least you can confirm dropping the cache reliably brings you back to the perf of the first query
# [pcarrier] and of course to speed that one up, you'll want either faster storage, or a warmup
# [pcarrier] but if it has to read a bunch from (relatively) slow storage, not much else you can do I'm afraid
lavandula joined the channel
# [capjamesg] I am using a mounted Hetzner Volume right now for storage.
calciume joined the channel
# 
[KevinMarks] Also the first query will usually read the indexes into RAM, so subsequent ones will not hit the disc for those, and if you're joining smaller tables they may end up, cached in RAM too.
# [KevinMarks] When the indexes no longer fit in RAM you see another order of magnitude slow down
# [capjamesg] Interesting.
# [capjamesg] There is about a 10x difference between first and second runs.
# [capjamesg] ~1s -> ~10s
duanin2, ZodiacalComet, oxtyped, cotsuka, barnaby, jorisvandijk, blacklight, deathcomestotime and lwindolf joined the channel; brn left the channel
# lwindolf Hello! I'm currently trying to decide whether to add microsub support to GTK feed reader liferea. Technically (aside from the need to support OIDC) it seems easy. I worried though that the microsub spec is still a draft with man open issues in Github. What to expect there?
gmb_ joined the channel
# lwindolf One thing I noted from a desktop perspective is that drag&drop of feeds between folders is usually a part of functionality which is for now missing in microsub. But it is easy to prevent.
# 
aaronpk hm there isn't the concept of folers in microsub yet, but you could approximate it in a client with a naming convention of using slashes in the names, then update would work https://indieweb.org/Microsub-spec#Update_a_Channel
cotsuka, btrem and blacklight joined the channel
# 
Loqi A post or posts may refer to individual pieces of content published on an indieweb site such as notes, articles, responses, or the act of creating the aforementioned content (present tense), or Posts about the IndieWeb https://indieweb.org/post_type
# Loqi Post Type Discovery specifies an algorithm for consuming code to determine the type of a post by its content properties and their values rather than an explicit “post type” property, thus better matched to modern post creation UIs that allow combining text, media, etc in a variety of ways without burdening users with any notion of what kind of post they are creating https://indieweb.org/Post_Type_Discovery
Sk joined the channel
# 
jak2k I get the reasoning. On my website, I don't want to mix articles, posts, bookmarks & so on and want to use totally different templates. This means I would have to filter in each database query (which is more expensive than I'd wish) or I would have to normalize into my data model before storing. But that would throw away some data and would make the endpoint logic even more complex. And clients such as
[Sophia_wood] and rossabaker joined the channel
# [snarfed] would that go onto https://microformats.org/wiki/microformats2-faq ?
[Sophia_wood] and [fnhipster] joined the channel
cotsuka joined the channel
# aaronpk i don't think HTML ever specified json encoding for HTML forms, looks like this never went anywhere https://www.w3.org/TR/html-json-forms/
ax562 joined the channel
cotsuka and barnaby joined the channel
# 
ZodiacalComet I also thought of using a hidden field for my own micropub server/client, even considered allowing using a logged-in session for auth. But I pivoted into trying to make a social reader while I consider what to do.
# ZodiacalComet There was also the problem that I tried to place everything into a single server, which I'm starting to rethink
# ZodiacalComet As much as I wanted to use Rust, I cannot handle the compile times. So I'm using Python
# ZodiacalComet And that thing is that I'm thinking of doing too. A single server that only does authentication and everything else delegates to it. Granted, it's more to have in my LAN instead of having it into the open internet.
# ZodiacalComet *is what
# ZodiacalComet Right, might be stretching it too much there on my part. It just does the auth dance that I read on the spec
blacklight joined the channel
# [snarfed] [tantek] re mf2 multiply valued faq, maybe? not on that page afaict, or on https://microformats.org/wiki/microformats2-parsing-faq . https://microformats.org/wiki/microformats2#microformats2_design is the closest I've found so far, but it's not great. "potentially" should probably be removed, and it's very terse and doesn't explain _why_
# 
[tantek] Good catch [snarfed]. I can see how the meaning of "potential" there could be ambiguous. I have amended it for clarity to "multivalue-capable". See update: https://microformats.org/wiki/microformats2#microformats2_design
cotsuka joined the channel
# [snarfed] ok! if you do, we can maybe add links to it from https://microformats.org/wiki/microformats2-parsing-faq and https://microformats.org/wiki/microformats2-faq
cotsuka joined the channel
[Leo] joined the channel
# ZodiacalComet ^x2
[Leo] joined the channel
# ZodiacalComet thought of "multivariable," but that also is weird
# ZodiacalComet sounds better
btrem joined the channel
rozodru joined the channel
cotsuka joined the channel
# ZodiacalComet You are the one that asked to check for extracting the name before?
# ZodiacalComet Because I tried to debug it myself, but didn't say anything because I thought it would be a difference in version. For `Fetch` from `@11ty/eleventy-fetch` to return a string `
{ type: "text" }` is required, otherwise it returns something else# ZodiacalComet At least that is what I remember when I tried it, will check it again
# ZodiacalComet Ah, it returns a buffer. Gave up because I didn't want to check for how to convert it into a string
# ZodiacalComet Okay, this is weird.
# ZodiacalComet The first time it returns an object. You need to use the attribute `body` on the result to get the response content
# ZodiacalComet It is a string when it is a cache miss, otherwise is a buffer of bytes
# [capjamesg] aaronpk Can you take a look at the scopes that indielogin requires for auth? Someone at HWC Edinburgh this evening sent me this screenshot from their trying to log into the wiki with Codeberg via indielogin:
# [capjamesg] The “including private repos and organisations” part here feels inappropriate.
# ZodiacalComet btrem: the fetcher has a cache to store the responses it gets. So the next time you try to request the same page it uses that instead of hitting the page again. This is what happens when I do it twice on a clean cache https://i.ibb.co/Xf5KjqQB/mf2-test.png
# [capjamesg] I have no idea.
# ZodiacalComet That is also why I didn't respond to you back then, thought it was just deno being different
# ZodiacalComet btrem: Yeh, but something weird happens there. If you are able, throw in a `console.debug(body)` to check if it is a Buffer of bytes.
cotsuka joined the channel
# ZodiacalComet btrem: you can do something like `typeof body == 'string'` or the `response.cache` equals 'hit' (from cache) or 'miss' (fetched).
# ZodiacalComet Yeah, that is exactly what is happening on my side with deno
# ZodiacalComet The only thing if converting the buffer into a string which I didn't look into
# ZodiacalComet *is
# ZodiacalComet aaronpk: it actually executed that?
# [capjamesg] Oof what?
# [capjamesg] Is there a way to make it not request private repos?
# [capjamesg] It’s crazy that you can type in random scopes and it still says that.
# [capjamesg] 😮
# [capjamesg] jak2k++ for that piece of information.
# [capjamesg] But codeberg— for that.
# aaronpk wait, i see an issue about forgejo as an OIDC provider so theoretically this should be possible https://codeberg.org/forgejo/forgejo/issues/11430
# jak2k It's tracked in this issue: https://codeberg.org/forgejo/forgejo/issues/4052
cotsuka joined the channel
# ZodiacalComet btrem++
[pcarrier], ax5623 and cotsuka joined the channel
# btrem Also, since I asked what algorithm I should use to get the title, I'll update what I've learned. Nothing new for veterans, but for someone just coming to it: my first thought was [[ if microformats.items == 1, return items[0]["p-name"]; if no p-name, return querySelector('h1').textContent; if no <h1>, return querySelector('head>title').textContent ]]