#snarfedhey aaronpk, for indieauth's auth flow, if i don't include a state param when i redirect, but it gives me a state param value back in the callback URL, is that expected?
#cmalso if anyone has pointers (pun intended) regarding a filesystem or just about anything that could allow me to inject binary data at a certain point without overwriting the data already present, and without requiring to read/copy the entire file content… :-/
#cmal(full-disclosure: I was just trying to implement some binary search to find entries (works fine) and insert entries (doesn't) in huge ordered collections
#cmalor I could just give up on the time-ordering but then, if I'm adding an article to a collection (say, a tag) it will appear at the top of the collection (which is expected in some collections for reshare purposes, but I would not expect this behaviours in streams (as defined by ActivityPub)
#cmalthe closest I've found so far would be FALLOC_FL_INSERT_RANGE (supported by ext4 and xfs), but the practical limitation is that the inserted data must be a multiplication of the logical block size (to keep block-alignment)
#cmalso that would mean inserting data in the middle of the file would require to allocate N blocks and rewrite there data from the next blocks until we reach alignment, which is probably less horrible than rewriting all the data until EOF, but is tricky when the size of data doesn't really fit in the block (for example, 200 bytes blocks inserting 199 bytes data would be terrible)
mindB joined the channel
#cmalor my collections could just be comprised of symlinks to smaller indexes which could then be rewritten as needed, I guess?
#sknebelcmal: maybe there is something tricky you can do with sparse files, at least for a while.
#sknebelcmal: also, look into how real databases structure their data on disk for inspiration
#sknebelan external index strukture doesn't have to be so bad
#cmalyeah I'm not really convinced by sparse files, mostly because the program running will just see a bunch of empty bytes (which somewhat mitigates my attempt to move throughout the file using fseek)
#sknebelyeah, you'd need to store offsets somewhere. and then you could just have random order and keep the data-offsets in an external file...
#cmalnot sure I need them at all, I could just implement binary search on top of these chunks, first looking into the middle of the middle-chunk, etc…
#cmalor I could indeed store the offset along with the chunk reference in the main index and that would make finding the right timespan even easier I guess
doesntgolf joined the channel
#cmalyeah I don't know, I'll do some thinking, some digging around and some experiments and we'll see :D
doesntgolf, KevinMarks, ChrisAldrich and KevinMarks_ joined the channel
#KevinMarks_If you're doing your own block management you're recreating the file system, but without the hardware coupling. You're probably better off just rewriting the file
#aaronpkyeah that sounds like a path i never want to go down :)
#aaronpkin my QuartzDB i only ever append to the file. i made a "maintenance task" that I can run periodically to re-sort the file if I'm worried that some data was inserted out of order.
#aaronpkthat task really just re-creates the file from scratch
#KevinMarks_Right, which is pretty much what databases do too
#aaronpkwell i want the file in sort order so it's easier to seek through in my code
#aaronpki just do a simple iteration over each line
ChrisAldrich joined the channel
#cmalI'm actually running down the path of a collection comprised of different chunks (each one in a file) and I think it's not such a bad way to deal with the issue at all
#cmalI mean, let's talk about it again when I'm done implementing and I have actual code to show :)
#cmal(hopefully just a few hours/days of messing around ^^)
#sknebelcmal: since I don't think I mentioned it: my solution for time-ordered posts is a directory full of symlinks, named after the timestamp ;)
#cmalI've thought about this, but then depending on your filesystem your folder may end up full
#cmalalso I'm curious, how do you retrieve the latest articles?
#aaronpkthe main limitation i've found with working with filesystems is having a single folder filled with tons of files
#aaronpkso i always break mine up, usually by year/month/day
#sknebelcmal: right now I don't have that many posts, once that becomes an issue I'll probably add directory layers like aaronpk said. Or replace the directory with sqlite, or ...
#sknebelI can always get the timestamp from the post metadata, this is just for easy lookup
#sknebel(and yes, listing all files in a directory is a relatively slow operation, but I'm sure there are a lot of relatively slow operations in my setup ;))
#cmalas I said, it's more the goals and means that are interesting more than the implementation itself (at least for the moment, we'll see what it turns out to be a year from now)
#voxpellirisk assessments etc makes it a lot harder to include AGPL code than to include MIT/BSD one. I barely know where to start to get an approval for that kind of code inclusion
#voxpelliI much rather have everyone be able to use my code and to contribute to it, no matter if their legal or business teams find the risk of AGPL to be acceptable or not
#cmalalthough, I'd say a freedom-enforcing mechanism like copyleft, which only practices constraints on those trying to privatize stuff, is actually the closest you get to "freedom to be free" in the context of systemic oppressions (like standard copyright when it comes to code)
#Loqi[Will Norris] imageproxy is a caching image proxy server written in go. It features:
basic image adjustments like resizing, cropping, and rotation
access control using host whitelists or request signing (HMAC-SHA256)
support for jpeg, png, and gif image f...
#voxpellicmal: the classic case of MIT/BSD vs GPL :) And funnily enough those using AGPL the most is those who wants to keep their open stuff as private as possible ;)
#voxpelliI've only seen AGPL used when one also sells a proprietary license for the same software as well, like Neo4j and this Imageflow hing
#miklbwait until you throw Apache Software License into the mix ;-)
#cmalanyway I'll settle for saying this is just legalese bullshit anyway and once we've abolished the State we won't have to talk about such boring lawyering anymore, deal?
#voxpellimiklb: that one actually does some good stuff ;) It handles patents
#miklbyes. I don't disagree with its use. Just adds another layer to the discussion.
#voxpelliIndeed it does :P Happy that software patents don't exist here in Sweden
#miklbpartly because we thought about getting into the Apache Incubator, but mostly for the freedoms it affords
#voxpelliI think Google picks ASL and I guess that can have its benefits over eg Facebook that picked a standard BSD for React, but with a custom patents clause that has caused some drama
#miklbtruthfully my eyes glaze over license discussions after a bit :-)
#sknebelthat Imgix stuff always confused me (MacOS APIs can't be *that* superior to make that worth it, can they?), but that's the kind of thing GPUs are really good at, so if you have to do it a lot...