#indiewebcamp 2014-02-15

2014-02-15 UTC
#
Jeena hm or is it some extencion of mine? I have investigated that once already but I don't think I was able to find a solution.
#
snarfed might be your setup
#
snarfed the https facebook and twitter avatars here render for me in firefox: https://snarfed.org/2014-01-16_coffee-the-third-most-popular-stimulant-in-the-neighborhood
#
Jeena just restarted my firefox without addons and they show up here too now
#
snarfed aha
#
Jeena ah it was the Disconnect addon, which kind of does exactly what it was installed to do, hehe damn
#
snarfed lol
tilgovi, hallettj, CheckDavid, paulcp, pasevin, gRegor`, nloadholtes, snarfed, bnvk, jedahan and skinny joined the channel
#
KartikPrabhu tommorris: was lookinf through the mf2py code in hopes of contributing to it. Any reason you use html5lib instead of beautiful soup? The latter already has some features that you have sough to implement like parsing multiple attribute values like classnames
bnvk, snarfed, squeakytoy, jedahan and squeakytoy2 joined the channel
#
tommorris KartikPrabhu: I’m not sure BeautifulSoup has good HTML5 support.
#
tommorris but if BeautifulSoup does the job, I’m happy to switch
#
KartikPrabhu interesting! I just finished adding a lot of comments while reading the parsing code. Is it possible that you (or I) put this on github? (I don't quite have the hang of mercurial yet! :D )
eschnou, tantek, KartikPrabhu, indiewebcamp-vis, bnvk, ttepasse, glennjones, jedahan, Kopfstein, pasevin, melvster, brianloveswords and snarfed joined the channel
#
snarfed morning indiewebcamp
snarfed joined the channel
#
snarfed twitter's streaming api is getting to be more trouble than it's worth, at least for bridgy
#
snarfed it stared 401ing all of my connections a week ago. i've done a lot of digging, but haven't figured out why yet
#
snarfed https://github.com/snarfed/bridgy/issues/57
#
snarfed i only need it for favorites, and i'm about to switch to scraping to get those :/
#
bear your using the control_uri / user_uri streaming method?
#
snarfed user streams
#
bear are you cycling thru multiple IPs per oauth id?
#
bear they have maximums for IPs *and* connections
#
snarfed yeah, i've been looking at that. i believe i cycle through IPs, but only a few per day at most
#
snarfed and 53 users right now, which as far as i can tell is comfortably under their limit
#
bear the problem is that twitter has never said how often they flush the IP bucket
#
snarfed sigh
#
bear and 50 is one of those numbers i have never found in the docs but seems to be a magic user stream number
#
bear they really want folks to use site streams
#
bear ah - the appengine twitter lib (I was wondering which twitter lib you used)
#
snarfed i use tweepy
#
snarfed any chance you remember where you saw the 50 number?
#
bear in a post to twitter dev like a year ago
#
snarfed ok
#
snarfed right now it 401s for me when deployed even when i only try to connect one user, but that may be left over from triggering it with >50
#
bear ah tweepy - cool
#
bear <-- author (one of) of twitter-python
#
snarfed ah, cool!
#
snarfed glad you saw my messages :P
#
bear :)
#
snarfed scraping is probably the answer, then
#
snarfed i'd love to sign up for site streams, but i worry about attracting any human attention, since i suspect they'd shut it down
#
bear i'm watching the bridgy repo :)
#
snarfed thanks!
#
bear yea, it's a fine line that many have crossed once you sign up for site streams
#
snarfed yup. happened to me with another app a bit ago, even without site streams. https://snarfed.org/2013-07-09_twitter-atom-is-back
#
bear the worse problem for user streams i've found is that it's sampled output
#
snarfed happened around 1k users, which may be a threshold for human reviewing
#
snarfed really? i think i was seeing complete data with user streams on bridgy
#
bear it's fine as long as the users your tracking *and* the total output is low
#
snarfed got it
#
bear but once you get someone really popular then they start dropping data
#
snarfed do you think site streams is better?
#
bear it's much better - but the hoops you have to jump thru are tough
#
bear that's why so many stick to user streams and just have multiple workers
#
bear each with different oauth keys and IPs
#
snarfed yeah. i worked around getting banned by asking users to geneate their own app key
#
snarfed :/
#
snarfed awful UX, but oh well
#
bear yep
#
bear scraping is sometimes faster because they put so much data in the page response
#
snarfed ah, rather than a bunch of api calls
#
bear yep
#
snarfed it's so frustrating that you can't get favorites from the rest api
#
snarfed at least not favs of a tweet
#
snarfed the rest of the api is so complete, i don't get why that's not there
#
snarfed meh
#
bear a lot of the rest api stopped growing once they got streams and were taken over by business people
#
bear (IMO)
#
snarfed hmph
#
bear i've taken to archiving my twitter emails - because I get one everytime someone favorites my posts
#
bear to backfill what I miss from the user stream
#
snarfed oh wow, interesting
#
snarfed what's the app? or use case?
#
bear just my personal testing tool
#
snarfed ah ok
#
bear always making sure python-twitter is working
bnvk joined the channel
#
bear yea, the sad part is that the twitter web client does exactly what I want to do - return a list of people who fav'd a post
#
bear via GET https://twitter.com/i/tweet/html?id=434385579090788353&modal=favorited_activity
#
snarfed right!
#
snarfed or /i/activity/favorited_popup , which returns json
#
snarfed e.g. https://twitter.com/i/activity/favorited_popup?id=415371781264781312
#
bear nods
#
snarfed either way
snarfed joined the channel
#
bear snarfed - question, are you pulling out the event notifications from the stream as they happen?
#
snarfed bear: yes
#
bear ah - ok
#
snarfed https://github.com/snarfed/bridgy/blob/master/twitter_streaming.py#L93
#
bear yea, then you are doing everything I would do - just twitter isn't giving you the data flow it should be
#
snarfed you mean, when it 401s?
#
bear yes
#
snarfed thanks for looking
#
bear np
#
bear snarfed - yes, they shutdown all v1 API first week of february
barnabywalters joined the channel
#
snarfed ah, good to know
barnabywalters_, barnabywalters, saeidw, tilgovi, snarfed, eschnou and caseorganic joined the channel
#
aaronpk !tell barnabywalters there's something strange with the markup here. Check it out on the pin13 parser http://pin13.net/mf2/?url=http%3A%2F%2Fwaterpigs.co.uk%2Farticles%2Fhalsway-gurdy-weekend-2014%2F
#
Loqi Ok, I'll tell him that when I see him next
Acidnerd and eschnou joined the channel
#
caseorganic.com edited /site-deaths (+19) "/* Site Deaths */ Added link to site changes page under See Also" (view diff)
#
caseorganic.com created /site-changes (+1923) "Created page. Added sections, structure and content. Used the template of site-deaths" (view diff)
snarfed joined the channel
#
caseorganic.com edited /site-changes (+2359) "/* Site Changes */ Added stubs for new examples." (view diff)
#
caseorganic Yes! Site Changes! Have been wanting to make that page for a long time on cyborganthropology.com. Makes much more sense on IndieWebCamp wiki
#
snarfed nice!
#
caseorganic.com edited /why (+40) "/* Identity loss */ Added site changes" (view diff)
#
caseorganic.com edited /stable-pages (+19) "/* Stable pages */ Added site changes" (view diff)
#
snarfed hey bear, do you know of any better way to scrape twitter favorites than /i/activity/favorited_popup? it cuts off at 25. :/
#
snarfed (/i/tweet/html doesn't have favs at all)
#
caseorganic.com edited /stable-pages (+15) "/* Stable pages */ added site changes and timeline" (view diff)
#
caseorganic.com edited /silo (+19) "/* See also */ Added site changes" (view diff)
#
caseorganic.com edited /NeoCities (-2) "fixed grammer" (view diff)
#
aaronpk snarfed: you enabled twitter cards on your site!
#
snarfed aaronpk: yup! been there for a while
#
aaronpk nice
#
aaronpk just saw cause one of tantek's tweets embedded your site's card
#
snarfed i don't love them, since they somehow get the raw markdown, before it's rendered…but probably still a net win
#
snarfed yours look good!
#
aaronpk yeah it's definitely a hack. I'd love to embed other page's summaries by parsing the microformats on ppls' sites!
#
caseorganic.com edited /history (+19) "/* See Also */ Added site changes" (view diff)
#
aaronpk thanks! mine are ok except the default image is my face which is sometimes a bit much
#
snarfed :P
#
snarfed that's my single biggest motivation for not liking invisible metadata
#
snarfed i want my picture in my home page's h-card, but i *don't* want my picture there
#
snarfed er, sorry, motivation for *liking* it. or at least tolerating it
caseorganic joined the channel
#
@TassianaBeta RT @t: Two wonderful talks on #privacy #ownyourdata #indieweb by @caseorganic: http://www.youtube.com/ http://vimeo.com/86329113 (ttk.me t4UZ1) (twitter.com/_/status/434800047025553408)
#
@TassianaBeta RT @t: Sign-up page is up for the next week's Homebrew Website Club meetup @MozSF & @ESRIPDX: http://indiewebcamp.com/events/2014-02-12-homebrew-website-club (ttk.me t4US1) (twitter.com/_/status/434800062825517057)
#
@TassianaBeta RT @t: @benwerd has offered handmade art with a twist: write an #indieweb reply (like this) & send a webmention. I want one! (ttk.me t4UL1) (twitter.com/_/status/434800076297629696)
#
@TassianaBeta RT @t: much #IndieWeb Bridgy & #IndieAuth progress! Come to 6:30pm Homebrew Website Club in SF & Portland TONIGHT: http://tantek.com/2014/029/t1/indieweb-progress-bridgy-indieauth-homebrew-tonight (twitter.com/_/status/434800079036481536)
#
@TassianaBeta RT @t: going to @benwerd's Introduction to the IndieWeb talk late http://werd.io/2014/introduction-to-the-indieweb had another event before. (ttk.me t4UB2) (twitter.com/_/status/434800087274123264)
#
@TassianaBeta RT @t: Reminder: Homebrew Website Club meetup TONIGHT 18:30 @MozSF @ESRIPDX http://indiewebcamp.com/events/2014-01-15-homebrew-website-club and http://snarfed.org/2014-01-15_homebrew-website-club-tonight (ttk.me t4U55) (twitter.com/_/status/434800103841624064)
#
@TassianaBeta RT @t: @adactio @jkphl no inferred p-author. Even better, there's the page authorship building block: http://indiewebcamp.com/authorship (ttk.me t4U51) (twitter.com/_/status/434800114818105345)
#
@TassianaBeta RT @t: First Homebrew Website Club meetup this year is in one week 2014-015 18:30 both @MozSF @EsriPDX http://indiewebcamp.com/events/2014-01-15-homebrew-website-club (ttk.me t4Ty1) (twitter.com/_/status/434800132971053056)
#
@TassianaBeta RT @t: @ericnakagawa more validators welcome! Check out: http://microformats.org/wiki/validators Especially: http:///validate-h-entry/ #indieweb (ttk.me t4Tw1) (twitter.com/_/status/434800144480231424)
#
Loqi [@TassianaBeta] RT @t: 2013 #indieweb pieces: IndieAuth comments webmention events Bridgy
#
Loqi 2014 together invincible. ♫ http://vimeo.com/40111334 (ttk.me t4Tu1) (http://twtr.io/i8gGB78QPU)
#
aaronpk is suspicious of people who retweets all of tantek's recent tweets
#
caseorganic aaronpk: yes
#
aaronpk !spammer TassianaBeta
#
Loqi Got it! There are now 8 spammers blacklisted
lmjabreu joined the channel
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/idno (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/events/2014-02-26-homebrew-website-club (webmention)
#
aaronpk oh I turned off mention notifications here after that bridgy incident
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/idno (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/events/2014-02-26-homebrew-website-club (webmention)
#
aaronpk forgot about that
#
caseorganic.com created /events/2014-02-26-homebrew-website-club (+2910) "Created meeting page for the next Homebrew Website Club meetup!" (view diff)
#
caseorganic.com edited /events/2014-02-26-homebrew-website-club (+66) "/* Blog posts */" (view diff)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/idno (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/events/2014-02-26-homebrew-website-club (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/idno (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/events/2014-02-26-homebrew-website-club (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/idno (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/events/2014-02-26-homebrew-website-club (webmention)
#
snarfed aaronpk: apologies again for that flood
#
aaronpk hehe
#
aaronpk was fun
#
caseorganic.com edited /Main_Page (+1) "/* Homebrew Website Club */ Changed page to show next meeting date" (view diff)
#
snarfed i think we ended up deciding to keep invitee mentions though
#
snarfed so it could easily happen again with the next big fb event
#
snarfed i'm guessing it would take a bit of work to suppress only invite rsvp webmentions?
#
snarfed (in loqi)
#
aaronpk yeah i'm not doing too much parsing of the actual page content, but that's a reasonable thing to add
#
aaronpk https://github.com/aaronpk/webmention.io/issues
#
snarfed discussion in https://github.com/snarfed/bridgy/issues/47
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/idno (webmention)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 linked to http://indiewebcamp.com/events/2014-02-26-homebrew-website-club (webmention)
#
snarfed only send the first mention of a page? you mean, of a source?
#
snarfed good idea, but wouldn't prevent the flood specifically. each invite has its own bridgy url :/
#
@caseorganic Next Homebrew Website Club is 2014-02-26 in Portland+SF! cc @hertling http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 #indieweb, #hwc (http://caseorganic.com/notes/2014/02/15/2/indieweb-hwc) (twitter.com/_/status/434812537251442688)
snarfed joined the channel
#
Loqi [mention] http://caseorganic.com/articles/2014/02/16/1/an-illustrated-dictionary-of-cyborg-anthropology-now-on-amazon-and-kindle linked to http://indiewebcamp.com/ (webmention)
skinny joined the channel
#
aaronpk yes, but it would prevent the little flood above :)
#
Loqi [mention] http://caseorganic.com/articles/2014/02/16/1/an-illustrated-dictionary-of-cyborg-anthropology-now-on-amazon-and-kindle linked to http://indiewebcamp.com/ (webmention)
#
snarfed ah. sure!
#
Loqi [mention] http://caseorganic.com/articles/2014/02/16/1/an-illustrated-dictionary-of-cyborg-anthropology-now-on-amazon-and-kindle linked to http://indiewebcamp.com/ (webmention)
#
@kevinmarks RT @caseorganic: Next Homebrew Website Club is 2014-02-26 in Portland+SF! cc @hertling http://caseorganic.com/articles/2014/02/15/1/the-next-homebrew-website-club-meeting-is-wed-feb-26-2014 #indieweb, #hwc (http://t.co/… (twitter.com/_/status/434814990520811520)
caseorganic and ttepasse joined the channel
#
@SimonCoopey POSSE! (http://indiewebcamp.com/POSSE) [pZpY6] (twitter.com/_/status/434827519771283456)
#
Loqi [mention] https://brid-gy.appspot.com/comment/twitter/SimonCoopey/434827519771283456/434829804207366144 linked to http://indiewebcamp.com/POSSE (webmention)
snarfed joined the channel
#
aaronparecki.com created /principles-de (+4535) "Copy to German page for translation" (view diff)
caseorganic joined the channel
#
aaronparecki.com edited /principles-de () "(-700) Edits to remove "offensive language"" (view diff)