#indiewebcamp 2013-05-18

2013-05-18 UTC
andreypopp, duckbillp, josephboyle, xtof, tilgovi, fmarier, b0bg0d and legastero joined the channel
#
aaronpk heh, just googled "html is my api" and got this as the top result http://tantek.com/2013/030/t1/html-my-api-xml-died-microformats-live-give-json
duckbillp joined the channel
#
christopheducamp.com created /design-fr (+1103) "[fr: sync'd translation]" (view diff)
#
christopheducamp.com edited /design-fr (-34) "<cite class="h-cite">Cette page a démarré sur [[design]]</cite>" (view diff)
duckbillp, andreypopp and xtof joined the channel
#
christopheducamp.com created /home_page-fr (+1634) "[fr:.]" (view diff)
duckbillp, b0bg0d, cweiske, tommorris and xtof joined the channel
#
cweiske http://scott.yang.id.au/code/trackback-em-all/ seemds to be a tool that does what I want (take my blog's atom feed and send pingbacks)
duckbillp joined the channel
#
aaronpk ah cool! that's what I want to add next to pingback.me
#
aaronpk one thing I encountered on my site is I don't want to send pingbacks to *all* links on the page, just the ones inside the h-entry
#
cweiske do you have more content in your feed than the h-entry?
andreypopp joined the channel
#
cweiske trackback em all does not work for my feed :/
xtof, barnabywalters and andreypopp joined the channel
#
pdurbin I think people in this channel would enjoy the last five or ten minutes of http://javaposse.com/java-posse-419-roundup-13-big-data
#
pdurbin lots of talk of privacy etc.
#
pdurbin facebook
brianloveswords, xtof and pdurbin_m joined the channel
#
pdurbin_m if ubuntu is invading privacy
#
pdurbin_m sending data about you into their cloud
#
pdurbin_m http://www.sparsebundle.net/posts/the-pied-piper-of-silicon-valley/
#
pdurbin_m via https://twitter.com/hnzz/status/335764744067772416
#
@hnzz Yep : Google uses open protocols to lure users, then locks them in http://t.co/bs5I3BAm6e
xtof, seyz, spinnerin and peck_lx joined the channel
#
aaronpk cweiske: yea, like my footer, sidebar, etc
scor joined the channel
#
aaronpk cweiske: oh you meant my atom feed. I consider that deprecated at this point :)
#
cweiske why would you deprecate your feed?
#
aaronpk I want to encourage more use of microformats2, so anything new I develop is going to be based around my h-entry feed, not my atom feed
#
cweiske what tf is a h-entry feed?
#
aaronpk http://aaronparecki.com/articles.json
#
aaronpk that is also what you'd get if you parsed http://aaronparecki.com/articles with a microformats-2 parser
xtof joined the channel
#
cweiske so you really ditch an established, wide-supported feed technology for a json file?
#
cweiske that's crazy, and not in a good way
#
aaronpk not a json file, microformats markup. I just happen to provide an already-parsed version of it at a URL
#
cweiske do you expect feed readers to support mf2 in near future?
#
cweiske s/near//
#
Loqi cweiske meant to say: do you expect feed readers to support mf2 in  future?
#
gregorlove.com edited /projects (+561) "/* other */ added Shaarli" (view diff)
#
gregorlove.com edited /projects (+23) "/* Shaarli */" (view diff)
#
aaronpk i expect whatever replaces feed readers to support mf2. I think we're already seeing an extreme decline in the use of feed readers, especially now that google killed Reader
#
aaronpk facebook and twitter have proven that their version of a feed is liked more by people than traditional RSS readers
scor joined the channel
#
gregorlove.com edited /projects (-1) "/* explorations */ fixed No More Sharecropping link, added published date." (view diff)
gRegorLove joined the channel
#
barnabywalters aaronpk: thanks for the heads up :)
#
aaronpk everything is offline!
#
aaronpk thought about trying to do a HA setup with it, but it's too much work
#
tantek HA?
#
aaronpk high availability
#
tantek those of you showing original context on your reply posts - how are you storing original context in your storage?
#
tantek barnabywalters and aaronpk - you might be the only ones showing context on your replies
#
aaronpk tantek: I switched my storage to store both the raw HTML as well as the parsed JSON
#
tantek is anyone else?
#
tantek the raw HTML of the h-entry of the original?
#
aaronpk raw HTML of the whole page
#
tantek oh wow
#
barnabywalters tantek: I’m flattening the microformats and dumping it as JSON into storage. Soon my mentions will store both source HTML and flattened µf
#
tantek right that is what we discussed last time aaronpk
#
aaronpk yep. i'm not doing any parsing other than what php-mf2 does
#
tantek so entire source of the original? no filtering of scripts or other potential nasties?
#
aaronpk i'm not displaying it anywhere, so yes
#
tantek just treating it like a webcache snapshot - that makes sense
#
tantek so it would purely be for reparsing later as we discussed
#
aaronpk yep
#
tantek welcome back Loqi
#
aaronpk oh hey that's nice that he starts up automatically when this machine reboots
#
tantek I may follow a similar path
#
tantek first storing only the pieces of original context I need to create the original context display
#
tantek and then later caching entire pages
barnabywalters joined the channel
#
aaronpk I'd suggest just storing both from the beginning
#
aaronpk decide what to do with the original later, but keep it for now. it's not much extra code to do that since you already have to fetch the page to parse it
#
tantek are you storing HTTP headers returned as well?
#
tantek like a warc?
#
aaronpk heh, I am not
#
tantek might be useful for retrying pingbacks/webmentions etc.
#
aaronpk good point, lol
#
aaronpk although those are often also in the html header
#
tantek and if we can/do store something warc-compatible, then we start archiving interesting (worth replying to) bits of the web in a very distributed manner
tantek joined the channel
#
tantek aaronpk - indeed
#
tantek cweiske - when you say "do you expect feed readers to support mf2 in near future?" which specific feed readers are you thinking of? (since they seem to be dying off themselves)
#
tantek.com edited /Facebook (+1065) "Comments Embed" (view diff)
#
cweiske tantek, no specific ones
#
cweiske just feed readers in general
#
cweiske I personally use tt-rss
#
cweiske bb
#
tantek well without a specific example, it's perhaps not worrying about
#
aaronpk so how do we create more incentives for the large players like google/facebook/twitter to parse microformats-2 data?
#
tantek aaronpk, for larger players it's generally a simple formula of amount of deployment compared to cost of implementation
#
aaronpk do you think they are likely to take an existing open source parsing library into account in the cost of implementation? or would they rewrite a parser? or is that not significant?
#
tantek some might
#
tantek but rather than attempting to directly influence larger players (beyond casual suggestions to standards-friendly advocates inside their walls), it's likely to be more effective (short and long term) to influence up the long tail of sites, from smallest to largest.
#
tantek as we build a critical mass of interoperable sites and implementations, it naturally becomes more and more compelling to larger and larger players
#
aaronpk so then the challenge is showing the long-tail the benefits even when the major players don't support it
#
tantek benefits over the costs
#
barnabywalters you guys seriously how can you not be watching eurovision right now I dont even
#
tantek hahaha
#
tantek barnabywalters is that something on youtube?
#
tantek some form of video silo?
#
tantek ;)
#
barnabywalters tantek: there’s a live feed, in flash unfortunately :(
#
tantek like on Hulu?
#
barnabywalters http://www.eurovision.tv/page/webtv?program=83843
#
tommorris tantek: no, it's basically a grand unifying European-wide movement that brings together white-van-man racists, cynical Twitterers and very camp fashionistas to share their mutual distaste for the French.
#
tommorris through song.
#
tantek.com edited /comment (+295) "Comments Embeds" (view diff)
#
tantek tommorris - sounds like a wikipedia summary
#
barnabywalters tommorris: wow, you managed to sum up eurovision in a paragraph. it took me ages to explain it to brennan
#
tommorris I should start my own cynical version of Wikipedia.
#
aaronpk Cynipedia
#
Loqi agreed.
#
tommorris pedia.tommorris.org ;)
#
barnabywalters tommorris: do it ;)
#
tantek.com created /Disqus (+1187) "stub with description, issues" (view diff)
#
barnabywalters disqus doesn’t work in XHTML? weirdness
sandeepshetty joined the channel
#
barnabywalters good evening sandeepshetty
#
sandeepshetty Hey
#
sandeepshetty barnabywalters: remember reading somewhere that you were moving to a file-system based store for Taproot?
#
barnabywalters sandeepshetty: that’s right, file based store with SQL indexes
#
barnabywalters designed from the bottom up to be really easy to maintain
#
barnabywalters it’s almost ready to go
#
sandeepshetty SQL indexes?
#
barnabywalters code here: https://github.com/Taproot/librarian
#
barnabywalters yeah, for flexible querying
#
barnabywalters so I store data in files with an ID, then store things about that file I want to query (e.g. tags) in the DB
tantek joined the channel
#
barnabywalters see https://github.com/Taproot/librarian/blob/master/librarianApiNoodling.php for approximately what the API looks like
#
sandeepshetty Have you looked at dropping the db completely?
#
sandeepshetty I'm currently toying with the idea...
#
barnabywalters sandeepshetty: not really — I like being able to do complex queries over my data, so I decided to keep the DB in there
#
barnabywalters sandeepshetty: I think aaronpk is doing file-based indexes
#
barnabywalters for tagging
#
aaronpk yea, just for the tags
#
aaronpk I don't have any way to do complex queries right now. my site search goes to google
#
barnabywalters yeah, I’m leaving generic search to google/DDG too
#
barnabywalters has anyone here tried elasticsearch?
#
sandeepshetty aaronpk: so you have post organized by time and then a parallel hierarchy for tags?
#
sandeepshetty *posts
#
aaronpk yes, I do a scandir and sort by filename (my files are named by date) to get the lists on /notes and /articles
#
aaronpk if you click a tag, it loads the tags.yml file and grabs all the filenames from there. I rebuild the tags file when I make a new post
#
Loqi GRABS ALL THE FILENAMES http://loqi.me/7HU
#
sandeepshetty How many post do you have and how is the performance?
#
tantek oh hey - welcome back sandeepshetty, nice to see you here
#
tantek aaronpk - I'm getting a 504 from http://indiewebcamp.com/irc/2013-05-18
#
aaronpk everything is down right now
#
aaronpk doing some migrations
#
sandeepshetty tantek: trying to figure out a good way to avoid db tax on converspace (https://github.com/converspace/converspace)
#
tantek aaronpk - ah ok
#
tantek didn't see anything in IRC for about 20 minutes after "how would we fit Disqus on the map?"
#
barnabywalters oh yeah I forgot to ask about the map — what is that?
#
Loqi that is also what you
#
barnabywalters Loqi: oh, okay. thanks
#
Loqi yeah!
#
tantek ??
#
barnabywalters aaronpk: Loqi seems a little more talkative recently — have you been working on him?
#
aaronpk no
#
aaronpk hasn't changed much actually
#
aaronpk although his talkativity level does depend on the amount of traffic in other channels
#
barnabywalters you mean… sentience?! ;)
#
barnabywalters ah, interesting
#
barnabywalters tantek: the social network map thing I saw mentioned in the logs
#
tantek barnabywalters - can't reference it while the server is down
#
tantek see /map when it's back up
#
aaronpk ok it's back
#
tantek barnabywalters, see: http://indiewebcamp.com/social_web_map for background, context
#
barnabywalters ha ha ha awesome
#
aaronpk WHOA images
#
aaronpk uplodaed those from my phone...haven't seen that page since
#
barnabywalters yeah, they could probably do with max-width: 100%
#
tantek aaronpk - yeah - your uploads don't seem to scale well in the current mediawiki CSS
#
barnabywalters was just changing the widths
#
aaronpk oops! one is upside down
#
tantek and out of order!
#
waterpigs.co.uk edited /social_web_map (+12) "/* Sketches */ Scaled photos to be a little more viewable" (view diff)
#
barnabywalters hm, an indieweb + silo islands map would be a really cool thing to generate automatically
#
aaronpk heh
#
barnabywalters and would be a fascinating and unique discovery channel
#
barnabywalters I especially like the elevation = indiemark idea
#
aaronpk would be interesting to automate the indiemark score check
#
tantek aaronpk - I'd rather indiemark be meaningful than automated
#
tantek automation typically results in superficial gaming
#
aaronpk heh
#
aaronpk but there's a couple things that would be easy to check for
#
@Johannes_Ernst Introductions: Indieweb, meet Personal Cloud. #pcloud, meet #indieweb http://indiewebcamp.com/Main_Page http://personal-clouds.org/wiki/Main_Page #fb
#
barnabywalters I’m thinking /indiemark-checklist is probably a good thing to make
#
aaronpk fyi I'm migrating more servers, stuff will be down for about 30 minutes
#
aaronpk including Loqi this time
#
Loqi yeah!
#
aaronpk loqi gets to take a nap
#
tantek potentially sharing such warcs with ArchiveTeam or the Internet Archive if/when such sites go down
#
tantek is tempted to store a copy of any page I link to for that reason.
#
aaronpk that's not a bad idea. I haven't done anything with warc files before... any good getting started guides?
#
tantek great question
#
aaronpk indiewebcamp wiki is back online
#
tantek about to run out of power here
#
aaronpk and should be a bit faster now too :)
#
tantek http://en.wikipedia.org/wiki/Web_ARChive
#
tantek dammit: "WARC file format is available from ISO for purchase." http://www.digitalpreservation.gov/formats/fdd/fdd000236.shtml
#
waterpigs.co.uk created /indiemark-checklist (+1209) "dumped some of the list from indiemark along with some of my own additions in checklist form" (view diff)
#
aaronpk wtf
#
barnabywalters wut?
#
tantek ok well if anyone finds non-ISO documentation of how to construct a WARC file
#
aaronpk here's something http://archive-access.sourceforge.net/warc/warc_file_format.txt
#
barnabywalters good grief, they don’t make it easy
#
barnabywalters there are what look like some examples here: http://bibnum.bnf.fr/warc/WARC_ISO_28500_version1_latestdraft.pdf#page=27
#
barnabywalters goodnight all
#
Loqi buenas noches
#
aaronpk here's another option: http://www.archiveteam.org/index.php?title=Wget_with_WARC_output
gRegorLove and xtof joined the channel