2013-05-20 UTC
# 00:02 tantek aaronpk - leaving notes for you to read is a subset of potential edits
# 00:03 tantek you could also choose to make only the question sections editable
# 00:06 aaronpk yes, that would make sense! I wonder what the easiest path to that is for me
tantek joined the channel
# 00:29 tantek aaronpk: Remind me what was the use case for displaying "view this post on twitter" etc links?
# 00:30 tantek Trying to inform my presentation design accordingly.
# 00:31 tantek pdurbin: Ah perhaps that is a good short term usecase until I have implemented a "reply" webaction/button that does that for them.
# 00:32 aaronpk yes, to be able to reply to the tweet, as well as to see the conversation happening on twitter
# 00:32 tantek Ooh: "view this conversation on twitter" is more compelling than "view this post on twitter"
# 00:33 tantek I think we should document the use cases for anything we add to our indieweb UI designs
# 00:33 pdurbin tantek: they'd have to be logged in to the twitter web interface for that web action to work, right? or could it open the twitter app on android, for example?
# 00:34 tantek pdurbin: By using tweet action endpoint URLs (what they errantly call Twitter Web Intents) it takes care of login flow for them.
# 00:35 aaronpk i have no idea what those do on mobile... are the clever enough to launch teh native app, or do they just do the same thing as on the web?
# 00:35 pdurbin tantek: yeah but I never log in to stuff like twitter via a browser on mobile. just apps
# 00:36 aaronpk the reason i'm logged in to twitter in my phone's browser is because of indieauth :)
# 00:38 pdurbin huh. maybe I'm just weird but I don't log into anything from my phone's browser. not google and I don't use facebook
# 00:38 aaronpk that's why :) I find sites tend to behave worse on android than on iphone
# 00:41 pdurbin sure. I never worry about clearing all cookies. I don't try to keep them around on mobile
# 00:45 tantek And yes I think on mobile it might provide the option to use a native app somehow.
# 00:46 pdurbin when I click a Twitter link from my phone I'm asked if I want to open it in the app, which I typicaly do. and reply from the app. or favorite or whatever
# 00:51 pdurbin bottom line for me is that links to twitter seems like a good idea
# 00:56 tantek Yeah, I felt the same way, but just wanted to get a better understanding of why so I could work that into the design if where he links go on posts (and the respective microcopy as well)
# 00:57 tantek pdurbin: I agree re: all the tracking just weirds me out
# 00:58 tantek Have you tried out the Collision add-on from Mozilla that shows a visualization of how sites are colluding to track you via 3rd party cookies?
# 01:06 pdurbin with Google I finally gave up and just stay logged in all the time. to multiple accounts even. so I can easily get to mail/calendar/plus
# 01:07 pdurbin for a long time I had separate profiles in Firefox. the normal non-logged in one, and then different profiles that are logged in to various services
# 01:08 pdurbin and I would spend the bulk of my time in the non-logged in profile
# 01:11 tantek One advantage of doing everything from your own site is that ideally your site can handle logging in (if necessary) on the server side and then you should never have to login client side (except maybe to do account maintenance).
# 01:11 tantek An indieweb setup would be an excellent step towards using sites without being tracked across the web.
# 01:12 pdurbin dunno if indieweb can get me off Google Calendar, which I find so useful :)
duckbillp joined the channel
# 01:14 aaronpk interesting change in the facebook API: "Publishing a Checkin object is deprecated in favor of creating an Open Graph story with a location attached. You can also create a Post with a location attached using the 'publish_steam' extended permission."
# 01:21 tantek Wow that is fascinating aaronpk ! Especially per the conversation we had at that previous party at Jillian's! (Which party was that? When was that? I think we never wikified the combo)
xtof and duckbillp joined the channel
# 05:02 bret Anyone here a JS guru? trying to decipher some JS in order to create a pingback.me parser
# 05:03 aaronpk I would not consider myself a JS guru, although I have been known to write javascript
# 05:03 bret but I am not seeing where exactly where it pulls in the data
# 05:04 bret all I can figure out pulling in data to my JS via XMLHttpRequest
# 05:04 bret oh, cause its not in vogue to write php?
# 05:05 aaronpk all the cool kids use javascript or ruby or clojure or something
# 05:05 aaronpk i'm getting old, i'm totally going to turn into one of those guys that says "back in my day..."
# 05:51 bret In order to associate with a websites search results, google requires a G+ account
# 05:51 bret also, it turns out you need G+ if you want to use google as a genaric openid provider for your domain
# 05:51 aaronpk for the google search result thing it kind of makes sense though
# 05:52 bret Why cant I associate it with my non G+ google account?
# 05:52 aaronpk cause there is no web presence for your gmail account
# 05:52 bret i have a google account, with a non-active G+
# 05:53 aaronpk there should be a minimal G+ account option where you upload a pic, set your name, and set your URL to your own domain
# 05:53 bret i'm not okay with people trying to add me and stuff like that
# 05:54 aaronpk oh yea I recently accidentally turned aaron@parecki.com into a G+ account, now people are adding me all the time
# 05:55 bret its like, even having a facebook account that yo dont use, people start talking to you on it and doing things on your behalf
# 05:55 bret the one protest, as far as I can see, against those services, is not to use them
# 05:57 bret i can see that true for many, but myself its been opposite since quitting
# 05:57 bret its been great to get off of facebook
# 05:57 aaronpk huh... I've recently gotten back onto facebook since so freaking many people are there
# 05:58 bret i have a theory its a generational wave thing... college and high school kids started out as the primary user base and now its going up in use with older people
# 05:59 bret my facebook went nearly silent around 09... very few people were using it that I actually cared about hearing from
# 06:00 bret im a weird grad student though, we tend to stay in the same building anyway and send way to many emails to each other
# 06:00 aaronpk and because I stopped using it, increasingly the people I met in real life I would not friend on facebook, contributing further to the lack of interest of the people in my facbeook network
# 06:01 bret plus it turned out to be a huge tupperware party
# 06:02 bret you know, the whole Douglas rushkoff analogy that facebook exploits our social interactions the same way a tupperware party does?
# 06:03 bret its opportunistic, and creates illusions in order to ecourage sharing and liking for the benafit of those who are there to do buisness with the data
# 06:04 bret right? I mean, thats why we are here, to find ways around that whole situation
tantek joined the channel
# 06:07 bret or at least a way I can email friends pictures or have them posted if they are interested and remember I exist :p
duckbillp, seyz, friedcell, xtof, peck_lx, adactio and andreypopp joined the channel
# 10:06 pdurbin cynical is the word I often think of when I think of facebook
xtof, eschnou, barnabywalters, duckbillp, mxuribe and seyz joined the channel
josephboyle, singpolyma, barnabywalters and hmans joined the channel
# 14:57 tantek aaronpk for indicating the language of a post (or anything marked up with HTML), use the 'lang' attribute
josephboyle and jzb joined the channel
# 15:16 tantek really getting tired of venue griefers/abusers/overmergers on Foursquare
# 15:16 tantek indie venues and indie checkin is getting to be more and more of an itch
# 15:19 tantek barnabywalters it's where people take two distinct venues and merge them into one because they have some mistaken idea of what a "venue" is
# 15:20 tantek which ends up redirect both previous unique URLs to one of them
# 15:20 tantek and it effectively destroys your history at checking into either
# 15:20 tantek because after that, you don't know which it was
# 15:20 tantek or they move your venue to the middle of the pacific ocean
# 15:21 tantek it's like wiki vandalism but worse because there is no history and no accountability
# 15:21 tantek you have to file support requests to undo such actions
# 15:21 barnabywalters being able to create indieweb venues which safely share information would be a real achievement
# 15:21 tantek well sharing such info would be based on social trust
# 15:22 tantek I'd only share venues with people who I knew had similar ideas of how venues should work
# 15:22 tantek and if other people wanted to do other things with venues, they could
josephboyle and xtof joined the channel
# 16:05 aaronpk yea seriously indieweb checkins getting more and more itchy for me too
josephboyle and spinnerin joined the channel
# 16:07 aaronpk that reminds me, I really need to download my foursquare data again before more venues get merged/renamed/moved
# 16:27 tantek aaronpk, citation? (that's pretty hilarious, and inevitably Tumblr gif-meme-worthy)
# 16:27 tantek aaronpk, re: indieweb checkins getting more itchy - I know right? If it weren't for being distracted with this pesky reply / federated comments thing ;)
# 16:28 tantek (it's not like we're fixing 10 year-old neglected distributed social web problems or anything ;) )
# 16:28 aaronpk ironically if you google that phrase you get a zillion articles that came out an hour ago
# 16:36 aaronpk yes... need to finish my comment thingy then can switch to checkins
# 16:36 aaronpk I made some progress yesterday on displaying comments below my notes
# 16:37 tantek I'm still working on displaying rel syndication/in-reply-to links, and then reply context
# 16:38 tantek I feel like receiving webmentions and displaying comments is going to be more work
# 16:38 tantek if I can get to the point of *sending* webmentions I may park my reply/comments efforts for now and move onto another post type (like checkins)
# 16:38 aaronpk ah! you're taking the same implementation path as me!
# 16:39 tantek aaronpk - indeed! except I may procastinate on receiving webmentions / displaying others' comments until indiewebcamp 2013
# 16:39 tantek I feel like I may need a bunch of in-person help on that
# 16:39 barnabywalters I’m revamping my persistence, then can get on with doing exciting stuff like displaying comments and self-hosting venues
# 16:39 tantek aaronpk - btw now that you have code to only show some posts to folks that are indieauth logged in, should be easy to expand that to show checkins to only those (or a subset of those) as well.
# 16:40 tantek that's likely what I'm going to do (only show checkins to an indieauth whitelist)
# 16:40 aaronpk I'm still not receiving webmentions yet, so as soon as I do I will be able to help people through it :)
# 16:40 aaronpk showing checkins to a whitelist is probably a good idea
# 16:40 tantek aaronpk - I thought you received webmentions and put them in a queue for looking at later?
# 16:41 tantek my primary reason for implementing checkins on my own site is personal data fidelity, but POSSE to Foursquare is a must as well
# 16:44 aaronpk so, I wrote it in Ruby for pingback.me, now have to do it in PHP
# 16:44 aaronpk although IndieNews is written in PHP so I've already done most of the work in that codebase
# 16:52 aaronpk good idea, was just thinking about that yesterday!
# 16:55 aaronpk done! there is also a "related changes" link which is interesting
# 16:56 tantek I never understand the utility of related changes
# 16:56 tantek maybe tommorris has a clue as to how to use the "related changes" link
# 16:56 aaronpk if it shows me what pages have changed that link to/from this page, that might be cool
# 17:02 tantek.com edited /Falcon (+1491) "/* Working On */ add Itching section of things bugging me but I'm not yet explicitly working on implementing, to at least collect them, in rough order of annoyance (most first)" (
view diff )
# 17:03 tantek somehow itches seem less annoying when a) shared publicly, and then b) documented so you know you can refind them and won't forget about them.
# 17:03 tantek I'm actually really curious what others' itches are (in terms of annoyances that they want to address by adding features to their own indieweb content posting/hosting solutions)
tantek joined the channel
# 17:29 aaronpk barnabywalters: I included a screenshot of your site there too
# 17:32 Loqi you're not using @wordpress (.org) #ownyourdata #rememberposterous
andreypopp joined the channel
# 17:35 tantek Also - that's the first I've seen of one of my notes showing up as reply context on another's reply post!
# 17:36 tantek Very cool barnabywalters. Was there anything special you had to do to get the info about my post or did it "just work"?
# 17:38 tantek Do you have any kind of a post-info preview tool that will show you what taproot will find/show about a post you're replying to?
# 17:39 barnabywalters tantek: you mean for me before I post or for others as a “what will my content look like if barnaby replies to it”?
# 17:39 barnabywalters that would be a much more user friendly and useful microformats validator than php-mf2
# 17:40 aaronpk p3k just has a "reply url" field, and I don't see what php-mf2 was able to pull out from the site until after it's published
b0bg0d joined the channel
# 17:51 barnabywalters tantek: I don’t have either yet, will probably build public one first then maybe integrate it into my posting UI
# 17:51 tantek The latter would be especially useful for encouraging people to add h-entry + h-card to their blog templates.
# 17:51 aaronpk a public preview where you can go put in a URL and see how it will look on my site?
# 17:52 tantek It answers the "how do I benefit from this work?" question about adding h-entry
tantek_ joined the channel
# 18:15 tantek.com edited /p3k (+734) "/* Public preview of h-entry parsing */ equivalent to blog specific preview in traditional comment UIs" (
view diff )
hmans joined the channel
# 18:29 barnabywalters u-syndication is the generally accepted µf2 classname for syndicated copies, right?
# 18:32 barnabywalters I’m just importing all my notes into the new system, and taking the opportunity to refine my schema and clear up some of the cruft that’s accumulated over the last year or so
# 18:36 tantek u-syndication is for a link to a syndicated copy
# 18:37 tantek delighted-announce-reached-agreement-acquire-tumblr would have been better
# 18:37 tantek simple stop-word filtering would fix some of that (drop the "im" "to" "that" "weve" "an")
# 18:38 tantek this is interesting: "We will operate Tumblr independently"
# 18:38 tantek almost the same wording used when Y! acquired Flickr
# 18:38 tantek also interesting: "… an amazing place to follow the world’s creators"
# 18:39 tantek Tumblr is ripe for recruiting people over to the indieweb.
# 18:39 Loqi tantek meant to say: Tumblr is ripe for recruiting creators over to the indieweb.
# 18:39 tantek Perhaps it's time to take a deeper dive on studying Tumblr's UI niceties and seeing waht we can incorporate into our indieweb solutions.
# 18:39 Loqi tantek meant to say: Perhaps it's time to take a deeper dive on studying Tumblr's UI niceties and seeing what we can incorporate into our indieweb solutions.
brianloveswords joined the channel
# 18:47 aaronpk simple posting UI, "reblogging" seems to be popular and made very easy,
# 18:48 aaronpk a lot of what people like is things around the community
# 18:48 barnabywalters question: I’m wanting to move my geo information (both lat/long and address info) out of machine tags and into the note structure. I want to maintain microformats semantics, but I’m not sure where to put it all
# 18:52 barnabywalters other thing to consider is where to put venue information when I have first-class venues
# 18:52 aaronpk agreed that lat/lng aren't always part of an address. it's entirely possible to have a lat/lng and no address info
# 18:53 barnabywalters currently everything’s under the geo namespace in my machine tags, so I’m tempted just to keep that
# 18:55 barnabywalters I think I’ll go everything flat under note.geo, then worry about venue when I actually implement it
fmarier, morrocco_mole, jedahan, eschnou and spinnerin joined the channel
# 19:14 tantek barnabywalters - venues are h-cards, pure and simple, and they can have address components and lat/long/altitude at the top level
# 19:14 tantek and for that I'm leaning towards generalizing the h-event p-location property to h-entry as well
# 19:14 tantek which allows a flat string, which you can then further substructure with an embedded h-adr or h-geo (or both)
# 19:15 tantek is having a hard time finding stuff in IRC logs that he didn't have time to wikify at the time that he wants to wikify now, like discussion of use-cases for rel=syndication links
# 19:15 tantek barnabywalters - yeah, we discussed generalizing p-location in IRC previously
# 19:15 tantek but as with above complaint, I have no way of easily finding it
# 19:16 barnabywalters sure, I’ll be marking up venues as h-cards, right now more concerned about how to represent that information in the nodes
# 19:16 Loqi barnabywalters meant to say: sure, I’ll be marking up venues as h-cards, right now more concerned about how to represent that information in the notes
# 19:16 tantek huh, maybe we discussed rel syndication use cases when Loqi was gone?
# 19:16 tantek barnabywalters - start with how you want to present the information, and we can figureout the markup after that
# 19:16 aaronpk I've been prety good about back-importing logs when loqi leaves
# 19:17 aaronpk maybe I should build an actual search instead of just relying on google
# 19:17 tantek oh and colloquy's find/search is totally broken
# 19:17 aaronpk if colloquy saves logs as text files you can just use grep
# 19:17 barnabywalters aaronpk: how about something like !wikify which adds a marker to the log entry?
# 19:18 barnabywalters so when talking we can just go !wikify, then immediately see which days conversations need wikifying on the IRC log list
eschnou joined the channel
# 19:23 tantek I think there's a lot more potential to "preview my reply" than just h-entry validation
# 19:23 tantek I've added some thoughts about it here (which is where the info was so far)
# 19:24 tantek I'm especially interested in everyone's opinion on a possible "cross-blog reply preview protocol"
eschnou joined the channel
# 19:26 tantek while writing a reply (on my own site), when I'm done writing it (or maybe even during), I want to see a preview of *BOTH* what it will look like on my site (WYSIWYG) *and* what it will look like on the original post (that I'm replying to), once my site sends a webmention to it
# 19:27 tantek for example, the comment syndication destination may decide to shorten my comment (for editorial / space reasons) and so I may want to craft my reply accordingly.
# 19:27 barnabywalters hm, seems like a good plan. Easy to implement too, just a <link> with a @rel and an agreed-on placeholder
# 19:27 tantek seeing a preview of that abbreviation would be useful
# 19:27 tantek well I'm not sure it's that easy though it might be
# 19:27 aaronpk very true! considering I already do that with Twitter since they publish their rules/limits
# 19:28 tantek i.e. and endpoint for submitting a URL and getting back preview HTML (to either embed/nest, or iframe maybe)
tantek-ipod and tantek joined the channel
josephboyle, andreypopp and tantek joined the channel
# 20:08 tantek I'm thinking of syndication links as a short term substitute for actual web action buttons.
# 20:08 tantek In particular just a twitter icon and "View on Twitter" for notes
# 20:09 aaronpk especially because of the challenge of mobile -- anything we do with web action buttons most likely will have to be re-considered for mobile
# 20:09 tantek or "View conversation on Twitter" for replies
# 20:09 tantek Thought twitter's tweet action endpoints seem to work pretty well on mobile
# 20:11 tantek Seems like a good short term step before I figure out actual webaction buttons below my notes
# 20:11 aaronpk agreed, and also even after we have webaction buttons I will probably keep the link to the twitter.com version there
# 20:12 aaronpk because "view conversation on twitter" will still be relevant
josephboyle, andreypopp, tantek and eschnou joined the channel
spinnerin and b0bg0d joined the channel
# 22:04 reidab overlays on everything!
tantek and sandeepshetty joined the channel
# 22:51 reidab SQLite is surprisingly capable
# 22:51 reidab I think we're still using it for calagator in production :P
# 22:51 reidab it's terrible for write-heavy applications though
# 22:52 tantek databases are all a pain to maintain, and more fragile than the file system
# 22:53 tantek so for any content you care about it, don't put the primary copy in a database
# 22:54 sandeepshetty tantek: could you explain your concept of dba tax? I've tried searching but haven't found any detials
# 22:54 tantek yeah - the hassle of setting up a database, doing tables, altering tables, exporting, importing etc.
# 22:54 tantek all *extra crap* (thus tax) compared to "just" using the file system
# 22:54 tantek nevermind all the username/login crap that is database specific
# 22:55 sandeepshetty you have none of that in SQLite and the ones you so have are common with file based approaches
# 22:55 tantek doesn't matter if they're "in common" - it's yet another space of stuff
# 22:55 tantek one space of stuff is easier to maintain / keep track of than two spaces of stuff
# 22:55 reidab sandeepshetty: you still have the table-altering complication, i suppose
# 22:56 tantek people lose things (and have them corrupted) in databases all the time
# 22:56 sandeepshetty tantek: SQLite is one file that can be in the same space as the rest of your site
# 22:56 tantek because databases always require extra "magic" for backups etc.
# 22:56 tantek as opposed to simple HTML you can always look at in a browser
# 22:57 tantek now if you want to try building a database-like API on top of an HTML file, that could be interesting
# 22:57 tantek and frankly, with HTML+microformats + a microformats parser, we already *have* a database like API that gives you data back from HTML
# 22:58 sandeepshetty Not really when you think about things like listing, pagination, doing aggregations (counts)
# 22:58 aaronpk i've been making interesting progress on filesystem as an API for p3k
# 22:58 tantek sandeepshetty , none of those have anything to do with storage format (HTML vs. magic database format), that statement is totally wrong.
# 22:58 aaronpk `ls -1 content/articles/*/*/*.md` goes a long way
# 22:59 reidab you can do all of those databasey things over files, using something like map/reduce
# 22:59 reidab but it's not there by default
# 22:59 tantek databases are essentially dead for anything but caching/performance needs for high volume sites.
# 23:00 aaronpk and then yea a map/reduce thing build my home page and tag listings, and is easily extendible to other collections
# 23:00 sandeepshetty tantek: The database API on top of regular html files will need to provide for stuff like that..
# 23:00 tantek sandeepshetty - so build that instead of using sqlite with a magic file format
# 23:01 reidab all file formats are magic
# 23:01 tantek reidab - no, anything based on plain text is inspectable
# 23:01 reidab you still need tools to inspect it
# 23:01 tantek right, it's not like looking at in raw binary sure
# 23:02 reidab overall, I agree with you
# 23:02 tantek but the fact that it's a well documented format that supported by *numerous* tools (1:1 like text editors, and higher level tools) makes it much more robust/reliable
# 23:02 tantek as opposed to magic database formats which typically *only* work with one implementation
# 23:02 tantek same problem with proprietary binary formats in general
# 23:03 tantek anyway - I'll leave it as an exercise to the reader to lose enough data in random/binary file formats until it's painful enough that you decide to stop using such things for anything you care about
# 23:03 pdurbin dunno that I would call the binary format sqlite uses "random"
# 23:03 aaronpk sandeepshetty: i'm doing map reduce in php on text files, yes it is
# 23:04 pdurbin dunno if I'd say databases are essentially dead either
# 23:04 tantek sandeepshetty - go ahead and try to make it work if you like, and report back on how well your data is doing in a few years.
# 23:04 aaronpk couple seconds, but i haven't optimized it, no need to yet since it only needs to run when I add new content
# 23:05 tantek pdurbin it's random over time - the implementation changes, the format changes typically in lockstep. if you're lucky you may be able to import/export across versions. or different versions of the implementation may corrupt different versions of the file format.
# 23:05 tantek hence both semi-random (unpredictable future), and frankly, randomizing
# 23:06 tantek also - most such databases tend to be biased/tied to a particular programming language as well - more unnecessary constraints
# 23:06 sandeepshetty I've been doing some work on moving my blog from Mysql to json files and came up with an interesting doubly linked list type approach that does most of what I need except aggregations.
# 23:06 tantek whereas every language / OS has flat file APIs
# 23:07 tantek so any design that depends on databases is optimizing prematurely - before you have any idea what the performance characteristics of the system are
# 23:07 sandeepshetty Jsut feels like I'm wasting time reinventing the wheel when I could be adding useful features
# 23:07 tantek sandeepshetty - depends on how much you care about your data
# 23:08 tantek I mean, why not use Tumblr as your backing store?
# 23:08 pdurbin tantek: what about nosql databases? essentially dead too?
# 23:08 reidab there are always ways to make sure your data is safe, regardless of where your "primary" storage is
# 23:08 tantek you'll waste more time (over the long run) maintaining your databases (special backups etc.)
# 23:08 reidab you just have to actually care enough to put those measures into place
# 23:09 sandeepshetty tantek: Because I can't trivially backup or replicate my data from tumblr
# 23:09 tantek pdurbin - if it uses a random/magic/binary/implementation-specific file format - it's not safe
# 23:09 reidab one-time export script vs. re-writing a bunch of things
# 23:09 reidab the one-time script seems less taxy
# 23:09 reidab it is if my software writes to file at the same time it writes to the database
# 23:09 tantek just search the web for people complaining about having to backup their wordpress databases before upgrading
# 23:10 tantek if you don't think it's a pain, the only way you'll learn it is is by doing it for years
# 23:10 tantek so please go ahead - because nothing else will likely convince you
# 23:10 tantek this is a pretty long term perspective (having kept data for 10+ years)
# 23:10 pdurbin tantek: nah. databases still have their uses. file systems don't deal well with zillions of tiny files
# 23:10 tantek who said anything about zillions of tiny files? (strawman)
# 23:11 tantek pdurbin - only pragmatic use databases have is caching
# 23:11 tantek for perf reasons - which means only introduce them when you have measurable perf you need to improve
# 23:12 pdurbin I guess this is #indiewebcamp so the main use cases imagined are personal websites
# 23:12 tantek well of course - what were you thinking the use cases were?
# 23:12 reidab I buy the "text files for data safety" argument, but I don't buy the "less work" argument
# 23:12 aaronpk I'm seeing how far I can push my site before I introduce a database
# 23:12 tantek seriously - this is why nearly every indieweb implementation has left databases behind.
# 23:12 tantek = ongoing work added on top of everything you do on a regular (time period) basis
# 23:13 aaronpk two uses I have for databases right now: 1) full-text search and 2) geo-search after I launch checkins. In both cases the DBs will be a cache/query store, the data will still be in my .md files
# 23:13 tantek it's likely that eventually we'll come up with something resembling a database / storage API on top of an HTML+microformats file
# 23:13 tantek and then that "less work" argument will be even more true
brianloveswords joined the channel
# 23:16 pdurbin aaronpk: I'm curious about your full text search. a database and not something like solr or elasticsearch?
# 23:16 pdurbin aaronpk: you have a broad definition of databases :)
# 23:17 aaronpk when I say "database" I don't necessarily mean RDBMS
# 23:19 tantek sandeepshetty - something where you can get back structured data and potentially write structured data
b0bg0d joined the channel
# 23:21 tantek sandeepshetty - the phpmf2 api is an example of getting back structured data from an HTML+microformats file
andreypopp joined the channel
# 23:24 tantek sandeepshetty - data from structured data - base from multiple instances of similar data - both returned from the API
# 23:28 sandeepshetty On my file based implementation I started with the same approach to get the most recent posts..
# 23:28 sandeepshetty but then moved to a double linked list approach where each post post maintains a next & prev (like the link rel) and I have a head..
# 23:29 aaronpk interesting, I get my next/prev links because my filenames are sorted in post order, so I just go up and down the array of files
# 23:31 tantek I prefer not to depend on a linked list structure
# 23:31 aaronpk are you thinking the directory listing is expensive?
# 23:32 sandeepshetty aaronpk: yeah it doesn't feel right especially when there are no post in the current day, month year and you have to keep searching for the head
# 23:32 aaronpk tantek: how do you know what posts are on your home page? and how do you know what is the next/prev post given a single post page?
# 23:32 tantek aaronpk - I start from today and read the most recent N posts
# 23:32 aaronpk how do you do that without a directory listing of files?
# 23:33 tantek when I "read" a post on a page I read the entire bim that it is in so looking at the previous element and next element is trivial. edges are handled by loading the previous/next bim respectively.
# 23:33 tantek because everything is stored computationally indexed by date
# 23:34 aaronpk sandeepshetty: `ls -1 content/articles/*/*/*.md` is basically what my code does to build the list of all posts
# 23:34 sandeepshetty tantek: sorry wrong term.. how do you find the post to show on your homepage..
# 23:35 tantek sandeepshetty see above - I start from today and read the most recent N posts
# 23:35 tantek all my access to find the file to open is pure math
# 23:35 aaronpk that relies on you storing multiple posts in a single file
# 23:35 tantek aaronpk - indeed - I picked a manageable time period for an indieweb site
# 23:36 tantek I did have to invent my own calendar, but hey, while you're at it, you know...
# 23:36 tantek it's not my fault months are historically screwed up mathematically
# 23:38 sandeepshetty Which is essentially a seek right? loading all the data (similar to ls -l) and then picking what you need
# 23:40 tantek sandeepshetty - no - you can't say "is essentially" about anything like that without understanding exactly what the pieces are doing from a CS perspective.
# 23:40 tantek you could say XPath is a sort of standard query API
# 23:41 tantek what it "is essentially" doing you can't say unless you want to go dive into the specific implementation code and figure it out
# 23:42 sandeepshetty Just saying that it has to go over the bim file to get what it needs and trying to figure out how that is o(1)
# 23:50 tantek it doesn't have to "go over" the bim file other than loading/parsing it
# 23:51 tantek typical DOM parsers index all the IDs for immediate access
# 23:55 tantek no because the size / count is effectively limited by the size of a bim - 61 days
# 23:55 tantek and typical posting behavior is no more than a few (even one?) post a day.
# 23:56 tantek so finding the right bim is O(1) (purely computational), and then there's always a small limited number (k) of things in any particular bim, thus once again, O(1).