#dev 2025-12-14

2025-12-14 UTC
CrimeWave, gRegor, kepeken, grufwub, NatePro, DanQ, opencircuit, proofreed and [KevinMarks] joined the channel
#
[KevinMarks] [Al_Abut] temporary location sharing is quite doable as a self-hosted page, as the location api fits well with leaflet as a mapping api
thegreekgeek joined the channel
#
[Al_Abut] [KevinMarks] interesting, I'll have to dig into what you mean and look up those terms. E.g. what is leaflet?
#
[Al_Abut] Clicking on the link to fire up gmaps with pin seemed to work well, having the map embedded was a reach goal that may not have been as good a UI in hindsight for mobile.
#
[KevinMarks] Leaflet.js is the very easy mapping ui
#
[KevinMarks] I think this example still works https://github.com/kevinmarks/maptime/blob/master/index.html
#
[Al_Abut] I'll check it out, thanks!
#
[Al_Abut] I'm going to put the location stuff on a shelf for a bit, possibly until I host another cycling group or other in-person event.
#
[Al_Abut] Today I'll be noodling on microformats for photos with [gRegorLove] and micropub with [aciccarello] using [benji]‘s Sparkle, then a reach goal after that is to wrap my head around microsub enough to set up Monocle eventually.
#
[KevinMarks] That should make a map, ask you for location permissions and zoom to your location
#
[KevinMarks] Obviously you'd need to set up an event to feed your location to the server page
#
[morgan] [Mike_Mai] Hey you said you were interested in this CSS animation article. here it is, for everyone else also 🙂. https://calendar.perfplanet.com/2025/a-tale-of-two-animations-the-compositor-in-the-skies/
gRegor joined the channel
#
GWG gRegor said this question was -dev...
gRegor joined the channel
#
GWG Wondering if anyone had researched offloading their images from their site to a place that offered better storage rates, e.g. S3, B2, R2....or any other letter plus number combo
#
capjamesg [KevinMarks] / anyone else Do you have any opinions on using a separate sqlite3 database for each user of an application?
gRegor joined the channel
#
capjamesg My feed reader is really struggling to scale right now. Users are getting errors all the time. The disk where the database is stored is slow, and there is a lot of data now. There are a lot of old indexes that are taking up lots of space, too.
#
capjamesg It does mean that to make an update involving the DB (i.e. adding a new field) I would need to roll out schema changes to every individual database.
#
capjamesg I'm looking for a solution that means I will not need to fix everything again in two months 😅
#
[tantek] capjamesg welcome to the scaling problem!
#
[artlung] "sharding" is the keyword that may point you in the right direction capjamesg
#
[artlung] I have seen places with 2 databases, literally one to handle customers with odd numbers in an identifier (ssn last 4, postal code, or even just an autoincrement field that was always odd, and one, even).
#
capjamesg Yeah. I have had at least 4-5 individual user reports in the last week of errors that I think are database-related, and generally the service has been slow for various user actions for a while.
#
[artlung] and yes, migrations will then be something you'll need to handle in an organized way
gRegor joined the channel
#
[artlung] you are now on postgres? I thought I remembered you talking about that
#
capjamesg I had so much data I ran out of space during the migration 😭
#
capjamesg And it was going to copy over indexes I'm not 100% sure I even need.
#
capjamesg (Pro tip: remove indexes you made as a trial _before_ your service ends up with 100s of GBs of data.)
Pixi joined the channel
#
capjamesg If I give each user their own DB, the code change required would be relatively simple, and saving feed results would no longer lock the database for a couple dozen seconds every hour.
#
GWG That sounds complicated though
#
capjamesg It does mean I'd have to store posts more than once, but I _think_ that's okay. At the very least I'd have visibility into how much data each user takes, and cap out everyone's storage in GB rather than offering unlimited storage as I do now.
gRegor joined the channel
#
[Al_Abut] @gwg that's my setup, I serve my images from S3/Cloudfront
#
[Al_Abut] I've offloaded my images and other media separately from my hosting for like a decade plus, first because hosting was so expensive and needed to be reliable for bursty traffic, these days because I'm a cheapo that likes the free tier on Netlify.
gRegor joined the channel
#
[Al_Abut] I figure 33 cents a month for media storage and hosting is better than however many dollars a month for paid hosting
#
[KevinMarks] The core sqlite use case is single writer, multiple readers - does that map to your use case? Single writer, single reader is a subset of that.
#
[KevinMarks] The hard part of database stuff is locking and rollback. If you can avoid locking by having a separate db per user, then that's a good simplification.
gRegor joined the channel
#
GWG [Al_Abut]: I am thinking of doing it, but need to decide on which, Cloudflare R2 looks to be a good combo of price plus feature
#
[KevinMarks] At a lower level, you hit file system contention. Again that is mostly optimised for one writer. So a sqlite db per user delegates that to a file per user at the filesystem layer.
gRegor joined the channel
#
capjamesg The scenario when I would have two writers is when the user tries to write at exactly the same time the feed poll results are being saved.
#
aaronpk omg the 10 year root cert I made for local testing finally expired
gRegor, chrisaldrich and DanQ joined the channel
#
[morgan] lol. You survived.
#
[morgan] aaronpk++
#
Loqi aaronpk has 53 karma in this channel over the last year (135 in all channels)
#
[morgan] rootcert--
#
Loqi rootcert has -1 karma over the last year
#
aaronpk time to start another 10 year countdown
#
aaronpk 9 years 11 months until 1 month until aaronpk's 10-year development root cert expires again
#
Loqi I added a countdown scheduled for 2035-11-07 12:31pm PST (#7229)
#
[artlung] The Long Now: _SSL Cert Edition_
#
[tantek] that's like the opposite of Let's Encrypt short term certs!
#
aaronpk well it's the CA's root cert, so not _that_ different, theirs also look like they are 10 years
#
aaronpk browsers are going to start rejecting long lifetime certs tho so i have to start issuing my own certs from this for individual websites more often than 10 years soon
#
[morgan] Well I just have to say I'm really happy about my little article I shared above about css animations
#
[morgan] Made my day
#
capjamesg I finally fixed my web reader!!!
#
capjamesg I didn't need to create separate DBs at all.
#
capjamesg 290 GB of storage space was being taken up by indexes for a feature I never launched.
#
capjamesg I moved all the data to a new DB and now everything is snappy again.
#
capjamesg Consider this my Create Day ship ^^
#
osteophage capjamesg[d]: 290 GB. Wow.
jack7087 left the channel
#
capjamesg Yeah...
#
capjamesg I wanted to offer the ability to read the plain text contents of posts, and also had search indexes on top of that for full text search, and wrappers I never used.
adamsaunders and troojg joined the channel
#
[tantek] capjamesg++ way to buy yourself more time before refactoring!
#
Loqi capjamesg has 53 karma in this channel over the last year (212 in all channels)
#
[tantek] Do you have a growth chart so you can estimate when you will need to?
#
capjamesg No but I should make one! Great idea!
chrisaldrich, barnaby, girlonthemoon, oodani, gRegor, DanQ, CrimeWave and [aciccarello] joined the channel
#
[aciccarello] [benji] I was helping Al with getting the serverless micropub endpoint. We got fairly far but ran into an issue when trying to publish. Do you know how to debug this? https://github.com/benjifs/micropub/issues/16
#
[aciccarello] I think it's an issue with the request to github