#dev 2024-08-28

2024-08-28 UTC
aaaa, sp1ff, superkuh, IWSlackGateway1, [schmarty]1, [tantek]1 and [jeremycherfas]1 joined the channel
#
capjamesg[d] [snarfed] Yeah. I'm not really sure how to best save to disk periodically. Any ideas?
[Joe_Crawford]1, thegreekgeek, [snarfed]1, lockywolf_, wiktor1, CRISPR, GuestZero, aaaa, ancarda, eb, nnrx, srushe, suki, okCiel, capjamesg, roxwize, vikanezrimaya, ttybitnik, AramZS, freecrypto and CatalystND joined the channel
#
[snarfed] capjames durability is hard to tack on after the initial design. I guess I'd start with, what are your design goals/req'ts? is it a system of record for data? if not, will it be colocated with the primary datastore? what kind of availability, consistency, correctness, etc do you need? etc
#
[snarfed] those will drive your storage decisions, if any
Lars-Christian joined the channel
#
aaronpk https://www.mnot.net/blog/2024/08/25/feeds 👀
#
superkuh Strongly disagree with fragmenting RSS/Atom feeds now by adding "communication" in the style of the HTTP "upgrades" after 1.1. Those upgrades mostly were bad for human persons and devoted entirely to the use case of corporations and institutions. The biggest thing I'd like to see with feeds is large organizations like those not setting default cloudflare up and blocking feed readers from accessing feeds.
#
superkuh Technically it's perfect as is, like IRC.
#
superkuh The idea of embedding video in feeds and being worried if they won't display is a different angle than I'm used to. For me a feed reader is used to open links in a browser, not to try to render everything there.
#
superkuh That just leaves you with a constantly growing whateverlibyouchoseforvideorenderingstuffkit tech debt.
#
superkuh Wow. Every paragraph I just disgree more. I'm trying to find something I can agree with here but... yike.
#
superkuh KISS, imo.
#
sebbu hum
#
sebbu my nostr and mastodon accounts are following each others
#
superkuh One positive aspect of the article is the link to the IETF web feeds list.
#
catgirlin.space superkuh how was http 1.1 bad? /gen
#
superkuh That is my point. http/1.1 is great.
#
superkuh http/2 and http/3 with their limited a no ability to connect to a remote webserver without that server first getting a short term permission certification from a corporation is bad.
#
superkuh "limited" and "no"
#
[KevinMarks] https://thenewstack.io/developers-rail-against-javascript-merchants-of-complexity/
#
aaronpk “Migrating my main site off PHP in 2010 was one of my worst career mistakes,” 👀
#
[KevinMarks] I thought you'd like that one
#
aaronpk :D
#
capjamesg[d] [snarfed] How would I design a durable system?
#
capjamesg[d] (I know that's a loaded question. I need a map to help me refine it.)
#
capjamesg[d] Re: availability / consistency: I thought about building in replication but it's not something I need.
#
capjamesg[d] (yet)
#
aaronpk when I took over oauth.net, I kept it using the same simple php architecture and now 14 years later it's still working just fine
#
capjamesg[d] I guess I could use an append-only log that saves records as they come in, then load from that to reconstitute the database?
#
aaronpk not quite at the million pageview per month mark, but it is around 150k/mo
#
capjamesg[d] That would allow for the records themselves to be stored in real time, since saving a single record to a file on an add() operation is inexpensive.
sp1ff` joined the channel
#
capjamesg[d] But as for the indexes themselves, it isn't as easy. I guess it would be okay to recompute those since that happens on an add() call anyway.
#
[snarfed] aaronpk++
#
Loqi aaronpk has 46 karma in this channel over the last year (137 in all channels)
#
[snarfed] capjamesg yeah this is a bigger question than I can reasonably answer in a brief chat 😆 hopefully some searches for "database storage engine" will be a good start! esp if you break them down into filesystem vs block storage
#
[snarfed] but also totally fine to keep this as an in memory only and/or indexing only system!
#
[snarfed] but yes a write-ahead log (ie journal) is a great building block. to oversimplify, the only two real building blocks we know of for providing transactionality are WALs and locks
#
[snarfed] good on WALs: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying
#
capjamesg[d] Thank you so much!!!
#
capjamesg[d] [snarfed]++
#
Loqi [snarfed] has 51 karma in this channel over the last year (102 in all channels)
#
[tantek] what is durable
#
Loqi It looks like we don't have a page for "durable" yet. Would you like to create it? (Or just say "durable is ____", a sentence describing the term)
#
[tantek] what is resilient
#
Loqi It looks like we don't have a page for "resilient" yet. Would you like to create it? (Or just say "resilient is ____", a sentence describing the term)
#
[tantek] what is robust
#
Loqi It looks like we don't have a page for "robust" yet. Would you like to create it? (Or just say "robust is ____", a sentence describing the term)
#
[tantek] 🤔
#
[tantek] durable is /longevity
#
Loqi ok
#
[tantek] resilient is /longevity
#
Loqi ok
hasanhaja joined the channel
#
[snarfed] oh no not quite, I only meant durable as in persistent, stored to disk
#
[snarfed] vs how in memory only is transient
#
[snarfed] that's generally what "durability" means in systems and storage eng
#
[snarfed] (it's the D in ACID 😁)
#
[snarfed] will wikify
#
[snarfed] ...although I guess that's not too far off from resilient/longevity 🤷
#
[snarfed] https://en.wikipedia.org/wiki/Durability_(database_systems) is fine, we could just link to that
#
[tantek] yes "durable" from a user perspective is more about /longevity
#
[tantek] if you wanted to wikify "durable storage" as a dev thing, that would make sense
#
[tantek] ah yes, pre-cloud tech jargon. what memories
#
[tantek] !tell [Joe_Crawford] perhaps that article on "merchants of complexity" is worth queuing up for the next FrESH?
#
Loqi Ok, I'll tell them that when I see them next
btrem joined the channel
#
btrem Vent: I'm trying to decide what I'm more angry with: json or vs code. >:-{
#
[tantek] Loqi, angry at json or angry at code?
#
Loqi angry at json
#
btrem :-D
#
btrem Thanks Loqi. Always there with an answer.
#
Loqi you're welcome, btrem
#
btrem I'm literally lol right now.
#
btrem And again, I must confess I had no idea one could ask Loqi such a question.
#
btrem Anyone here familiar with json schema? If so, a question: I can apparently require an enum comply with a json schema in an external file, by importing it. Can I require that an enum value exist in a plain json file? It seems no, but figured I'd ask.
cuibonobo joined the channel
#
btrem Loqi++
#
Loqi Loqi has 3 karma in this channel over the last year (15 in all channels)
#
btrem json--
#
Loqi json has -1 karma over the last year
#
btrem vs-code--
#
Loqi vs-code has -1 karma over the last year
benatkin joined the channel