#indiewebcamp 2016-03-25

2016-03-25 UTC
snarfed1 joined the channel
#
@manton2 @dteare @genegoykhman Ha! Yes. I want to support Webmention but have to write some code first. So for now it’s just thoughts. :-) (twitter.com/_/status/713163605101793280)
snarfed joined the channel
#
Loqi [indieweb] "Why we built Known" by Ben Werdmuller http://stream.withknown.com/2015/why-we-built-known
#
Loqi [indieweb] "A short note about web standards from your friends at Known" by Ben Werdmuller http://stream.withknown.com/2015/a-short-note-about-web-standards-from-your-friends-at
#
Loqi [indieweb] "Self-hosted Known 0.8.5 has left the building - with better indieweb, micropub and AMP support." by Ben Werdmuller http://stream.withknown.com/2015/self-hosted-known-085-has-left-the-building---with-better
#
aaronpk snarfed: I've started using the value that comes back in from the token endpoint as the ID which lets you type stuff like "xyz.withknown.com" in the web sign in box and have the server return an ID of xyz.withknown.com/username
#
snarfed aaronpk: thanks, true!
#
aaronpk that also means as long as the site returns the http/https as the identity consistently, it doesn't matter which they type in when they log in
#
snarfed the "web sign in box"...meaning we'd redirect to indieauth first, and they'd type their URL/domain there instead of on our site?
#
snarfed or on our site?
#
aaronpk On yours
#
snarfed ok
#
snarfed so if client_id is missing scheme, indieauth defaults to...http...and follows redirects if necessary?
#
aaronpk client id?
#
aaronpk man I wish I had called indieauth.com something else. Makes these conversations very confusing
#
snarfed heh
#
snarfed oh sorry, 'me' not 'client_id'
#
snarfed gtg, back later
#
aaronpk me too
#
aaronpk and yes
yakker, M-RyanRix, begriffs and tantek joined the channel
#
@ChrisAldrich Where does my website go when I die? #indieweb @michaelhoffman https://twitter.com/michaelhoffman/status/713010027154550784 (twitter.com/_/status/713193433435742208)
#
Loqi [indieweb] "Where does my website go when I die? #indieweb @michaelhoffman https://twitter.com/michaelhoffman/status/713010027154550784" by Chris Aldrich http://stream.boffosocko.com/2016/where-does-my-website-go-when-i-die-indieweb-michaelhoffman
#
KevinMarks hm, this is sketchy
#
KevinMarks https://twitter.com/brosandprose/status/712757373631389696
#
@brosandprose I'm really uncomfortable with the trend of people (esp cis, straight, white people) annotating blog posts and articles using Genius. (twitter.com/_/status/712757373631389696)
#
KevinMarks https://twitter.com/brosandprose/status/713069540003983360
#
@brosandprose .@Genius @Web_Annotator I have not installed the extension on my site. How can I prevent other people from using the URL on my site? (twitter.com/_/status/713069540003983360)
#
KevinMarks https://twitter.com/Genius/status/713068675264282624
#
@Genius @brosandprose hi Ella, you can opt out by not using the http://genius.it//genius.com/ URL or @Web_Annotator extensions. http://genius.com/web-annotator (twitter.com/_/status/713068675264282624)
#
@bhansmeyer Agreed! I hope Webmentions continue to evolve & catch on. https://twitter.com/manton2/status/713073075286216705 (twitter.com/_/status/713198177541697537)
#
KartikPrabhu KevinMarks: I am confused. Is the problem that someones else is annotating their site on a third party extension and they don't want it?
#
KevinMarks yes, which has implications for webmentions etc. But the thing is that they are pulling her site and oevrlaying on it
#
KartikPrabhu hard to stop that
#
KartikPrabhu on the Web anyone can pull any site
#
KevinMarks well, she should be able to block their scraper with robots.txt
#
KartikPrabhu does have potential for abuse and misinformation
#
KevinMarks so her post ends up lookign liek this: https://genius.it/ellacydawson.wordpress.com/2016/03/14/i-am-not-suffering-from-herpes/
begriffs joined the channel
#
KevinMarks if genius sent webmentions, that would be intereatsing
KartikPrabhu joined the channel
#
KartikPrabhu KevinMarks: yeah that does look really really bad
singpolyma joined the channel
#
aaronpk that is tough, anyone can do anything with your content once it's online. but i feel like it's disingenuous to copy not just the quoted snippets but the full article *plus* the whole layout of the site, making it look like something she supports
#
KartikPrabhu yeah that is part that is bothering me too
#
KevinMarks well, they present ti as a browser plugin adn WP plugin too
#
KevinMarks quicktopic did this over a decade ago
#
KevinMarks though theirs was per paragraph comments not highlights
#
aaronpk i feel like ive heard of others too
#
KevinMarks there were a whole series of 'annotate the web' startups that died
#
KartikPrabhu isn't hypothesis like that too?
#
KevinMarks Third Voice, SideWiki
#
KevinMarks Annotea
#
KevinMarks and yes hypothesis
#
KevinMarks https://www.w3.org/2001/Annotea/
mlncn joined the channel
#
KartikPrabhu yup here it is: https://via.hypothes.is/https://kartikprabhu.com/articles/marginalia
#
KartikPrabhu and it does not load icons and avatars for some reason
#
aaronpk also what? genius.it redirects to genius.com which is "Genius is the world’s biggest collection of song lyrics and crowdsourced musical knowledge."
#
KartikPrabhu strange indeed
#
KevinMarks should redirect to http://genius.com/web-annotator
#
KevinMarks https://twitter.com/kevinmarks/status/713194010135769088
#
@kevinmarks @Genius @Web_Annotator that's not what @brosandprose asked. Do you respect robots.txt ? How should Ella edit that to block your crawler? (twitter.com/_/status/713194010135769088)
#
KevinMarks I din't see an obvious fetch from them in the logs when I tried it on one of my sites
#
KartikPrabhu interesting
#
KevinMarks maybe they're just using a browser string
#
KevinMarks or maybe they are loading the page from the browser
#
KartikPrabhu yeah I think they are
#
KevinMarks so are they not running client js?
#
KevinMarks yes, if I do http://genius.it/kevinmarks.com my webmentions don't show
#
KartikPrabhu yeah same-ish. It runs my JS but again does not dowload icons and avatars
snarfed joined the channel
#
KevinMarks ah, not the webmentiosn are failing because the url being sent is genius.it/kevinmarks.com
yakker, wolftune and j12t joined the channel
#
@cdf1982 RT @bhansmeyer Agreed! I hope Webmentions continue to evolve & catch on. https://twitter.com/manton2/status/713073075286216705 (twitter.com/_/status/713257042362941440)
j12t, yakker, strugee, friedcell and loic_m joined the channel
#
KevinMarks http://www.confluent.io/blog/turning-the-database-inside-out-with-apache-samza/
j12t, glennjones, loic_m, dietrich, tantek, loic_m_, Tristitia, friedcell, jrenslin, mlncn, j4y_funabashi, sivoais, Pierre-O, finchd, hs0ucy, snarfed, nitot, shiflett and Lancey joined the channel
#
tantek aaronpk congrats on sending salmentions!!!
#
tantek does that mean we have 3+ implementations of sending & receiving salmentions?
#
tantek what is salmention?
#
Loqi Salmentions are a protocol extension to Webmention to propagate comments and other interactions upstream by sending a webmention from a response to the original post when the response itself receives a response (comment, like, etc.) https://indiewebcamp.com/salmention
#
aaronpk checks /Salmention and adds himself
#
tantek :D
#
aaronpk ooh i'd better double check I still receive them after the rewrite
#
aaronparecki.com edited /Salmentions (+42) "/* Aaron Parecki */" (view diff)
#
tantek indeed!
#
tantek is there a service for testing that?
#
tantek like an indiewebify.me feature? or feature request?
#
aaronpk that would be a good idea
wolftune joined the channel
#
KartikPrabhu are salmentions sent only to the in-reply-to post or all links linked to in the post?
#
KartikPrabhu afaik when you update a post you should resend webmentions to all links
j12t joined the channel
#
aaronpk all links is what /salmention says
#
KartikPrabhu hmm
#
aaronpk in fact it says all previous webmentions sent, which means it also includes webmentions that were previously sent that may now be removed
[kevinmarks] joined the channel
#
[kevinmarks] Well, we lost an implementation when Kyle switched to known from redwind
#
@Swissendo @herrfabu Nutzen doch nun eh alle Webmention!!!!1elf (twitter.com/_/status/713396421882429440)
#
kylewm and I'm a bit hesitant to add salmentions to Known since everything is synchronous
#
aaronpk sounds like it might be time to add something like wp-cron to known?
begriffs, KartikPrabhu, snarfed and shiflett_ joined the channel
#
GWG Greetings Indieweb friends
snarfed1, shiflett_, shiflett and snarfed joined the channel
#
snarfed switching all bridgy instagram users to scraping in a minute
#
snarfed "hold onto your butts!"
#
aaronpk wow
j4y_funabashi joined the channel
#
snarfed unrelated, aaronpk, just fyi https://indiewebcat.com/ 's footer has a broken image
#
aaronpk oops thanks
#
aaronpk i think i have a bunch of updates to launch there actually
begriffs, gRegorLove and quails joined the channel
#
gRegorLove Man, that Genius Web Annotator stuff is messed up, particularly their responses in the thread "if you don't like it, you don't have to read it" basically. https://indiewebcamp.com/irc/2016-03-24#t1458873924698
#
snarfed switching bridgy instagram back to API for now. found scraping bugs to fix!
#
gRegorLove And https://twitter.com/jamescrugnale/status/712766490374770692
#
@jamescrugnale @brosandprose have you reached out to the moderators? The site has an abuse policy http://genius.com/Genius-become-a-genius-annotated (twitter.com/_/status/712766490374770692)
#
gRegorLove https://twitter.com/brosandprose/status/712766615599886337
#
@brosandprose @jamescrugnale the editor of the news division is the person annotating my blog. (twitter.com/_/status/712766615599886337)
#
gRegorLove https://ellacydawson.wordpress.com/2016/03/25/how-news-genius-silences-writers/
#
kylewm re; time for Known to get a queue, is this a use case for Amazon SQS? like a hosted service where I can send events, and it would dequeue them and send them back to my site as POSTs to a given endpoint
#
snarfed kylewm: context?
#
aaronpk possibly!
#
kylewm snarfed: minimal context http://indiewebcamp.com/irc/2016-03-25/line/1458922019344
#
Loqi [kylewm] and I'm a bit hesitant to add salmentions to Known since everything is synchronous
#
snarfed thx
#
snarfed SQS would definitely work but maybe a bit heavy, esp since it would require an AWS account, either known's or the users' own.
#
snarfed some lightweight PHP package would maybe be ideal
#
snarfed reaches the end of his knowledge and stops talking.
#
aaronpk there are queue mechanisms that use a swappable backend
#
kylewm like, a lightweight php package that people could self-host?
#
aaronpk that way withknown.com could run a production one on SQS, but self-hosted ones could use a built-in one by default
#
snarfed right
#
sknebel in amazon-land you probably want SNS, not SQS
#
snarfed kylewm: yeah, but not even self host, just link in directly like the rest of the third party libs known uses
#
aaronpk i thought SNS was more for broadcast-like things
#
sknebel but it can do POST-callbacks
#
KevinMarks the appengine queue thing is good
#
sknebel I thought SQS was amazon-internal only
#
snarfed sknebel: nope
#
sknebel as in AWS internal
#
snarfed oh maybe
#
sknebel can connect to all kinds of AWS services, but not talk to the outside world
#
ben_thatmustbeme huh.... looking at how slack integrations work... i wonder if i could just use a personal slack account as my interface to my site
#
ben_thatmustbeme when i get a webmention bring it in to slack channel, and my responses generate a post on my site
#
aaronpk definitely
#
aaronpk that's basically how my new Monocle is going to work
#
ben_thatmustbeme nice
#
kylewm snarfed: you'd still need to run a separate process somehow right? (re: link in directly like the rest of the third party libs)
#
aaronpk yes or do it like wp-cron
#
snarfed kylewm: depends. not if it uses the filesystem as the backing store, or if you just say durability is best effort
#
aaronpk you can use the DB as the backing store too. it's just a question of when the task actually runs
#
snarfed oh you mean for execution, not storage. yeah, or wp-cron style, or who knows
#
KevinMarks can PHP return the page and then continue executing?
#
@dkreuz @benwerd I had a similar idea: my students work on #indieweb during their projects to show what is possible, both technical and social (twitter.com/_/status/713420438752333824)
#
aaronpk KevinMarks: kinda, but most of the time the web server has a limit on how long a script can run anyway
#
kylewm KevinMarks: yeah, Known does stuff like that in some places
#
@dkreuz RT @benwerd Last night I dreamt of an #indieweb studio, hosted at a university, acting as a hub for the movement. (twitter.com/_/status/713420938671431684)
#
kylewm it's a PITA because apache and nginx handle it differently
#
kylewm so like when I try to do teh export, I always get a Gateway Timeout
#
aaronpk that's not actually going to be fixed by using wp-cron or another method that runs the queue via POST request, since you'll still be limited by that timeout
#
aaronpk it's more like the super long-running jobs need to be rewritten to perform their tasks incrementally and in a way that can be resumed part way through
#
gRegorLove Yes, PHP can keep executing after the page is delivered. That's how the "lazy cron" works in ProcessWire and the PW Webmention plugin uses it for async
#
KevinMarks what I was thinking is that you could have a queue endpoint that you post to, it returns and keeps executing, and you then exit
#
aaronpk that only works if the task takes less time than the nginx/apache timeout
#
KevinMarks so you can hand over to it
#
sknebel it's a useful part of splitting up jobs though
#
gRegorLove Is set_time_limit(0) not an option?
#
KevinMarks right, but once you have that you can do what you said and split up the long list of things into a chain of them
#
aaronpk sometimes the server overrides it
#
aaronpk and in the case of nginx with php-fpm, it's actually nginx that's hanging up, not php quitting on its own
#
KevinMarks I'm assuming PHP yields when waiting on network etc here, whcich may not be right
#
aaronpk "gateway timeout" means nginx didn't get a response from the backend in time, so even if php is letting itself run forever, nginx will give up
#
kylewm aaronpk: really interesting point about wp-cron, i hadn't thought of that
#
aaronpk but yes step 1 is going to be rewriting anything that runs >30 seconds to be able to run in smaller pieces on multiple requests
#
aaronpk odds are async webmention verification won't take 30 seconds, so you can get away with the lazy cron approach for that
#
kylewm and if you hit the cron every five minutes or something, then a big job could take a really long time
#
aaronpk yeah but if you run it on every request like wp-cron it would go faster
#
KevinMarks when testing salmentions with acegiak I had to turn up appengine's timeout from 5 seconds as sometimes fetching her posts was blocked by the feedreader task running
#
KevinMarks wp-cron assumes you have a lot of requests
#
KevinMarks though I suppose if you POST and then load the posted page you're guaranteed at least one
#
kylewm aaronpk: does nginx actually kill the thread before it's done executing, or just stops waiting for it to return a response?
#
aaronpk i'm not actually sure
Pierre-O joined the channel
#
@lightcoin @larrysalibra I pushed 4 indieweb POSSE strategy, sadly Guy didn't have time to build a blog. Up to @BlockstackOrg community to take it on! (twitter.com/_/status/713434752225640450)
#
boffosocko.com edited /User:Boffosocko.com (+226) "/* Itches */ added a few itches, recategorized based on platform" (view diff)
tvn joined the channel
#
boffosocko.com edited /User:Boffosocko.com (+716) "/* WordPress */ elaborated on SNAP experiments a tad" (view diff)
begriffs and wolftune joined the channel
#
@lightcoin @muneeb @larrysalibra nothing wrong with syndicating to Medium (as I and others do), but publishing to own domain first is good for indieweb (twitter.com/_/status/713450742112133120)
#
@lightcoin @muneeb @larrysalibra see https://indiewebcamp.com/POSSE (twitter.com/_/status/713451107662434304)
mlncn joined the channel
#
bear nginx will close the socket which should cause php-fpm to kill the process and recycle it
#
bear to make sure your php has time I would use php-fpm pools and have one pool with a longer request_terminate_timeout and then configure nginx to set the pool for a location to the higher threshold one
#
bear nginx is acting as a normal proxy so it will always just close the socket for a send/read timeout
hober joined the channel
#
boffosocko.com edited /User:Boffosocko.com (+252) "Reconfigured Finished section" (view diff)
#
kylewm i'm thinking maybe the way forward with Known is to continue to do things like POSSE and PuSH and webmentions synchronously by default but add hooks so that a plugin could offload those tasks to a queue instead
#
bear IMO known should always allow for sync processing because the average person will not want to work with a job queue
#
kylewm cool
#
snarfed kylewm++
#
snarfed bear++
#
Loqi kylewm has 310 karma
#
snarfed for starting simple
#
Loqi bear has 112 karma
#
kylewm i just want to have a way to experiment first ya know
#
bear and if the code for doing a job is exactly the same but called from different entry-point helpers... then it's a total win IMO
#
kylewm bear: do you think background jobs should be triggered by a long-running background process totally outside of the request/response world, or triggered by a POST?
#
bear POST as that is what web folks expect
#
kylewm that's the fundamental confusion i have right now
#
bear and you don't have the pain of maintaining a daemon
#
bear using a POST means you can isolate the timeout settings in php-fpm
#
kylewm ok so like google appengine does, rather than like celery
#
bear if your location is /jobs/* then you can have that handled by a different php-fpm pool
#
aaronpk i had never even considered that
#
aaronpk that's very clever
#
kylewm oh awesome
#
bear production php ops (I have tons of scars and hard fought lessons)
#
aaronpk oh for the record, AWS SQS totally does work with non-amazon services
#
aaronpk it's one of the queue providers supported by Laravel https://laravel.com/docs/5.2/queues
#
kylewm awesome
#
kylewm so if i do my roll-your-own thing that way then it should be possible to abstract to SQS for the hosted service if they ever want to
#
bear I don't see why not - it would just be another plugin like thing to define the helper entry point
#
kylewm wonders how much of my total time on iwc irc has been me trying to understand task queues
#
kylewm bear++
#
bear sync, sync+post, async+sqs, async+postgres
#
Loqi bear has 113 karma
#
kylewm aaronpk++ thanks both of yall
#
Loqi aaronpk has 100 karma
#
kylewm bear: it kind of seems like you are doing a Will Shortz puzzle there
#
bear haha - yea, it does
#
bear WE Sunday is one of my all time fav shows
#
aaronpk now i'm considering writing a queue adapter for Laravel which does what bear is suggesting
#
aaronpk queues the job in the DB like normal, but then makes a POST to some /jobs/* URL to actually process it on a different fpm pool
#
aaronpk that does sound easier to maintain than a separate background process
#
bear much
#
bear and also runnable in environments where a lot of php folk find themselves in
#
bear php + db + a bit of apache/nginx config
#
aaronpk even if it runs on the same fpm pool it would be not terrible
#
aaronpk most of my tasks don't actually last >30 seconds anyway
#
bear yea, worse case is you have one long timeout
#
bear heck, even if you do - break the tasks up into 25 second chunks and chain them
#
aaronpk i'm thinking something like, client POSTS to /webmention, which then queues a job in the DB, then POSTS internally to /jobs/>{id} which closes the HTTP connection and keeps processing
#
bear that sounds very sane
#
bear the public facing endpoint never changes
#
aaronpk right
#
aaronpk and if i make it just another Laravel queue adapter, I can always run a real background task if I want to
#
bear you can even add some protection to the /jobs/ endpoint
snarfed joined the channel
#
aaronpk hmm Loqi forgot to post about this-week
#
Loqi yo
#
aaronpk yeah you what's up loqi
#
Loqi yeah!
#
kylewm Huh you could end up running some jobs in parallel that way
[shaners] joined the channel
#
[shaners] Hello, all.
#
Loqi [shaners]: gRegorLove left you a message on 3/22 at 3:34pm: http://veganstraightedge.com/articles/2010/11/7/1/no-more-sharecropping appears to be 404 and I can't seem to find it in your site search. Is there a current URL for that? http://indiewebcamp.com/irc/2016-03-22/line/1458686056394
#
Loqi [shaners]: gRegorLove left you a message on 3/22 at 3:36pm: Nvm, found it: http://veganstraightedge.com/articles/2010/11/07/no-more-sharecropping http://indiewebcamp.com/irc/2016-03-22/line/1458686201996
#
[shaners] !tell gregorlove: Yeah. I need to build up the redirects feature in Dark Matter better to handle the new URL scheme. (I’ve dropped the nth of day bit from my paths.)
#
Loqi Ok, I'll tell them that when I see them next
#
[shaners] I’ve been working out of the Pivotal office in Santa Monica, CA for the past couple few weeks.
snarfed joined the channel
#
[shaners] Today I signed up to do a lightning talk about #indieweb & Dark Matter. (on April 26)
#
[shaners] And I’ll prolly setup an evening session and if it has interest, a recurring evening session (weekly/monthly).
snarfed joined the channel
#
Loqi Just generated this week's newsletter! You still have a few minutes to make changes, and I'll re-generate it 10 minutes before it gets sent out at 2pm. http://indiewebcamp.com/this-week/2016-03-25.html
#
aaronpk ohh timezones
#
aaronpk sigh
#
bear is the server on UTC or a EU timezone that hasn't shifted yet?
#
aaronpk server is UTC
KartikPrabhu joined the channel
#
aaronpk and cron isn't smart enough to support different timezones per job
#
bear yea, it's all one TZ - if you want fancy - use fcron
begriffs joined the channel
#
@indiewebcamp This week in the #indieweb https://indiewebcamp.com/this-week/2016-03-25.html (twitter.com/_/status/713485621340336129)
yakker joined the channel
#
sknebel hm, why is IWC Nuremberg only as text, but not linked in there?
#
aaronpk Must be missing u-url markup on the event page
#
aaronpk the newsletter is generated from the microformats on the event page
#
sknebel hm, I wanted to fix, bit it looks identical to the düsseldorf one to me?
begriffs joined the channel
#
KevinMarks following the annotation thing from yesterday http://glog.glennf.com/blog/2016/3/25/citation-appropriation-and-fair-use
#
bear wouldn't the no-iframes content header prevent that kind of appropriation?
#
KevinMarks it isn't iframe as far as we can tell; I think they fetch it from the browser
#
bear ah, that's sketchy
#
KevinMarks when I tried it on one of my sites I didn't see a robots.txt probe or a non-browser useragent
#
bear not honouring robots.txt is a cardinal sin -- one of the many small reasons I wish the ops world would adopt a code-of-conduct
#
KevinMarks hm, hang on
#
KevinMarks looks like they fetch from their server using my browser sig
[shaners] joined the channel
#
[shaners] !tell tantek: Just noticed that the Microformat page on Wikipedia could prolly use an updated re-write if you feel up to it sometime. https://en.wikipedia.org/wiki/Microformat
#
Loqi Ok, I'll tell him that when I see him next
#
bear oh, so they have the server act as a retrieval proxy
#
KevinMarks right
#
KevinMarks then they inject their JS into it and have my browser fetch the rest
#
bear that's borderline sketchy - slippery slope stuff to not think of honouring robots
#
bear that's like an ISP allowing pizzahut to put overlays of coupons to every visitor of dominos
#
KevinMarks right, though it is also like the archive.org header
#
bear archival viewing is still different than current viewing with injected content IMO
#
sknebel yeah, they load all the assets directly, but the main page html comes from their server
tantek joined the channel
#
sknebel that's why scripts and co partly break, it's suddenly a cross origin request and a security violation the browser stops
snarfed and yakker joined the channel