#dev 2021-10-30

2021-10-30 UTC
edgeduchess[d], hepphepp[d], grantcodes[d], edburns[d], marksuth[d], Murray[d], rhiaro_, lanodan, rockorager, sp1ff, mikeputnam, Seirdy, jaylamo, Hy, j12t1, [jacky]1, akevinhuang, P1000[d], jeremycherfas, hendursaga and Terry_Davis[d] joined the channel
#
doosboox
good morning!
#
Loqi
good morning!
#
doosboox
I'm re-imaging my RPi 2, hoping to fix the network issues I've had with it
hendursa1 and kimberlyhirsh[d] joined the channel
#
capjamesg[d]
What are you planning to do with the Pi?
lanodan, kogepan, hendursa1 and tetov-irc joined the channel
#
doosboox
capjamesg[d]: I'm using my 3 as server right now, but it's kinnda borked. I can't upgrade b/c dependency issues that can't be solved without upgrading... I decided to configure the 2 to take over. A fresh start with less crap (i.e. no damn Rust) and config based on what I've learned from my usage.
#
doosboox
the problem is that I start with a fresh image running Debian Buster, upgrade to Bullseye, and then after reboot it can't regain network connection. I have no idea why
#
doosboox
but I've replicated that error today, meaning I know there's something in Bullseye that breaks it. Going to try with a different OS this evening. Probably Ubuntu Core
pstuifzand and kogepan joined the channel
#
capjamesg[d]
That sounds frustrating.
#
capjamesg[d]
[tantek]++
#
Loqi
[tantek] has 21 karma in this channel over the last year (70 in all channels)
#
capjamesg[d]
Now I have to tackle relevance. I sometimes have great searches where my intent is met but often that is not the case.
#
capjamesg[d]
I use this formula for calculating relevance: return _score + Math.log((1 + (doc['incoming_links'].value)) * 3.5) + Math.log((1 + (doc['word_count'].value)));
#
capjamesg[d]
Where _score is equal to the actual keyword relevance defined by various elasticsearch factors.
#
GWG
I think today I'll update my PR for IndieAuth server metadata
hs0ucy and akevinhuang joined the channel
#
GWG
Okay, I think https://github.com/indieweb/indieauth/pull/102 is ready for an editor review.
#
Loqi
[dshanske] #102 Introduce Server Metadata and Iss Parameter
[sebsel] and jaylamo joined the channel
#
jaylamo
doosboox: if you're configuring from scratch anyway give pyinfra a try. All my pis are disposable now. It's like ansible but easier to get into. Supports idempotency etc
hs0ucy, nertzy, chenghiz_, akevinhuang2 and kogepan joined the channel
#
GWG
I just updated my parsing code to add a last updated and last published property to any returned feed derived by sorting the original posts.
#
GWG
Not enough people are returning etag or last modified headers
#
capjamesg[d]
Yeah, support for those is more limited 😦
#
capjamesg[d]
Despite adding all of the features we talked about yesterday -- aside from having an algorithm to determine how often to poll something -- there are still feeds that I have to download and parse every hour.
#
capjamesg[d]
I have feeling Netlify sets an ETag for every URL.
#
capjamesg[d]
That's interesting.
#
capjamesg[d]
Not just feeds.
#
capjamesg[d]
I wouldn't add support for that behavior in IndieWeb Search but it's still interesting.
#
GWG
I just figured I'd have to make some compromises
Murray[d] joined the channel
#
[KevinMarks]
People are neglecting that now? You could also use a sha256 of the content as a proxy etag, though that only really helps after you fetch it again.
#
GWG
[KevinMarks]: That's why I'm sorting published and updated fields to find the last one referenced for feeds.
#
GWG
[KevinMarks]: But you can poll some indieweb sites and see.
#
GWG
You have one, for example.
#
GWG
[tantek] doesn't, aaronpk doesn't....
#
GWG
gRegor doesn't.
#
GWG
I forget who else I checked earlier
#
GWG
I don't, but I may rig up a plugin for it.
#
GWG
Then I have to support 304
#
Zegnat
I do not think I have them either. Probably because the server in front of PHP doesn’t keep track. It would have to be my PHP code regenerating the page and then sending an etag. At that point the server has already done all the work to create the document though, and default setup is to just send it back. It wastes the bandwidth, but does not really waste CPU as the document had to be created anyway.
#
Zegnat
I guess you would need some sort of cache layer first to be able to do accurate checks of etags?
#
Zegnat
has not actually looked into it much
#
GWG
Zegnat: I have some FastCGI Caching going on with Nginx. I think it supports using the If-None-Match and If-Modified-Since headers
#
GWG
For now, based on limited support, I'm going to calculated the last modified date of a feed after parsing it, and if it isn't updated since the stored date, I will just stop, saving the time of updating all feed items.
hendursaga, j12t, [snarfed] and [jeremycherfas] joined the channel
#
petermolnar
fastcgi caching is nice but if you use that the request never hits php
#
petermolnar
that's the point
#
petermolnar
you can, however, invalidate it through an api(ish) call
#
petermolnar
so theorecitally if the content changes in the background and that process is unrelated to a request - like a cron job - then it can invalidate the fastcgi cache
akevinhuang, tetov-irc, Seirdy, akevinhuang2 and hs0ucy joined the channel