#dev 2018-07-31

2018-07-31 UTC
#
[cleverdevil] This is really weird. I have no idea why this is being slow.
#
snarfed [cleverdevil]: this isn't mf2py, right?
#
[cleverdevil] Ah hah!
#
[cleverdevil] I figured it out.
#
snarfed [cleverdevil]++ yay
#
Loqi cleverdevil has 40 karma in this channel (119 overall)
#
[cleverdevil] VERRRY odd, but I am using python-requests to make the HTTP calls.
#
[cleverdevil] When I fetch the JSON feed [manton], for some reason decoding the content is super slow unless I manually tell the requests library that I expect the content to be encoded as UTF-8.
#
[cleverdevil] No idea why this is the case, but its what is going on.
#
[cleverdevil] Now, everything is insanely fast on my local machine.
#
[cleverdevil] Gonna clean up a bit and then push to lambda.
#
snarfed nice sleuthing
#
snarfed [cleverdevil]: also, unrelated, and low priority, if you have any interest in adding the JF2 conversion code from https://github.com/cleverdevil/microformats2 to granary, i'd happily accept a PR!
#
Loqi [cleverdevil] microformats2: Validate Microformats2 JSON with JSON Schema, and perform Post Type Discovery
#
snarfed https://github.com/snarfed/granary/issues/109
#
Loqi [snarfed] #109 support JF2
#
[cleverdevil] Cool, I'll add it to my todo list.
#
aaronpk wow good find
#
[cleverdevil] [manton] http://cleverangel.org/on-this-day/
#
[cleverdevil] Look how fast now!
#
[cleverdevil] Sub-second.
#
[cleverdevil] (For a site with nearly 5k posts)
#
[cleverdevil] Compared to [manton]'s site - https://www.manton.org/on-this-day/
#
[cleverdevil] (Which doesn't yet have a JSON feed index)
#
tantek [cleverdevil]: that's amazing
#
tantek wait when did manton get on this day?
#
[cleverdevil] Indeed.
#
[cleverdevil] He got it from my code!
#
[cleverdevil] I built a way for people to add the feature to their Micro.blog site just by pasting in a few lines of markup.
#
tantek uh, does that mean every microblog user has it?
#
[cleverdevil] If they want it.
#
tantek cleverdevil++ amazing
#
Loqi cleverdevil has 41 karma in this channel (120 overall)
mblaney joined the channel
#
[cleverdevil] [tantek] - https://github.com/cleverdevil/micromemories
#
Loqi [cleverdevil] micromemories: Add an "On This Day" feature to a Micro.blog website
#
[cleverdevil] (This doesn't include the new code, which prefers JSON to parsing the gigantic archive HTML with mf2py)
#
[cleverdevil] [manton]++ for creating the JSON feed index for archive pages, making this *way* faster.
#
Loqi manton has 15 karma in this channel (77 overall)
#
tantek what is on this day
#
Loqi on this day is a feature on some silos and indieweb sites which show posts published on the current (typically Gregorian) day of the year in previous years; commons sites tend to show more general historical events that occurred on that same day https://indieweb.org/on_this_day
tantek__ joined the channel
#
tantek.com created /on-this-day (+25) "r" (view diff)
benwerd joined the channel
#
tantek [cleverdevil]: can you add those new examples to https://indieweb.org/on_this_day#IndieWeb_Examples
snarfed joined the channel
#
tantek and maybe your code too somehwere? maybe as a subsection here: https://indieweb.org/on_this_day#CMS_support_.2F_Plugins
#
[cleverdevil] I think my code is already in there under "see also"
#
tantek yes worth moving it to highlight it since it's kind of CMS support / plugin :)
#
[cleverdevil] So, this code also works on any site with an archive page in h-feed.
#
[cleverdevil] Theoretically, it could be used for *any* site.
#
[cleverdevil] All you need is either a JSON based index or an HTML h-feed that lists references to all pages.
#
cleverdevil.io edited /on_this_day (+285) "/* CMS support / Plugins */" (view diff)
#
tantek lists references - like minimal h-entry (h-cite?) with u-url dt-published ?
#
[cleverdevil] It just needs an h-feed with h-entries inside that contain a u-url and a dt-published.
#
[cleverdevil] It does all of the work after that.
#
[cleverdevil] Pushed up the changes in code.
#
[cleverdevil] I also shaved about a half second of time by using a super terrible regex to match dates rather than fully parsing them 😛
#
tantek__ subsecond for rendering that for 10 years of post is pretty amazing
#
tantek__ posts*
#
[cleverdevil] I agree!
#
[cleverdevil] Way faster than my own Known site, which queries a freakin' database.
#
tantek wait a minute. you're doing static file parsing and then an "on this day" query against that data faster than PHP+MySQL to do the same?
#
[cleverdevil] Yup.
#
[cleverdevil] Its not a fully fair comparison, but its still pretty awesome.
#
tantek.com created /alpo (+21) "for jmac" (view diff)
#
jmac 🐶👍
#
jmac.org edited /Web::Microformats2 (-22) (view diff)
renem and benwerd joined the channel
#
aaronpk xray now supports rel=alternate as well as activitystreams!
[snarfed] joined the channel
#
[snarfed] aaronpk++
#
Loqi aaronpk has 168 karma in this channel (1693 overall)
#
aaronpk wtf newlines lol
#
aaronpk I thought I fixed that
#
ludovicchabant hey there! I made a thing: https://ludovic.chabant.com/devblog/2018/07/30/announcing-silorider/
#
ludovicchabant tl;dr: a python CLI tool that parses MF2 markup from your website, and does POSSE for you. Twitter/Mastodon currently supported.
#
ludovicchabant I know there's a bunch of tools that do things like this already, more or less, but hey, it's not called the _Homebrew_ website community for nothing ;)
#
aaronpk Nice
#
aaronpk Add it to /POSSE !
eli_oat and [cleverdevil] joined the channel
#
[cleverdevil] [aaronpk] https://jnyexhmjv8.execute-api.us-east-1.amazonaws.com/production/validate?url=https://aaronparecki.com/2018/07/07/7/oauth-for-the-open-web.json
#
aaronpk Ooh
#
[cleverdevil] Its also got a better name, but DNS needs to propagate
#
aaronpk [cleverdevil]++ that was fast
#
Loqi cleverdevil has 42 karma in this channel (121 overall)
#
[cleverdevil] Super quick and dirty.
#
[cleverdevil] Won't handle exceptions well, FWIW 😛
#
[cleverdevil] https://jnyexhmjv8.execute-api.us-east-1.amazonaws.com/production/validate?url=https://cleverdevil.io/test/invalid.json
#
[cleverdevil] It also looks like [manton]'s site has been updated with the JSON index for his archive page, which means his /on-this-day page is now snappy too.
eli_oat, GWG-, wagle, rhiaro, klez and [manton] joined the channel
#
[manton] Just tested it. It is really fast now. Thanks for doing this!
#
[cleverdevil] Sure thing! Now that I have a Micro.blog hosted customer in the house, I'll likely be doing more 😉
#
[cleverdevil] Funny, my seven year old just asked for her own site, and then my four year old echoed the sentiment.
#
[cleverdevil] So, you may be getting two more customers 😛
#
KartikPrabhu [cleverdevil]: that is awesome!
gRegorLove and eli_oat joined the channel
#
@metbril Test webmention https://www.robertvanbregt.nl/2018/07/28/zelfgemaakte-recepten-voor-aeropress-bijhouden-op-je-iphone/ (twitter.com/_/status/1024157842742370304)
[matpacker] joined the channel
#
[matpacker] Has anyone used [cleverdevil]'s ditchbook to migrate their FB to a WordPress install?
[cleverdevil] joined the channel
#
[cleverdevil] Doubtful. It’s only ever been used on one person’s Facebook data.
#
[cleverdevil] But I’m happy to help if you run into problems!
#
[matpacker] [cleverdevil] just installed Python on my laptop, so I'll give it a try later on and let you know the results.
#
[cleverdevil] Cool. It’s highly likely that you’ll find an issue because it’s not been widely tested yet. But be persistent and share the results and I’ll do my best to fix anything you find.
eli_oat, KartikPrabhu, [xavierroy], [chrisaldrich], cweiske, eli_oat1, KevinMarks and [kevinmarks] joined the channel
#
[kevinmarks] What is the time difference between parsing mf2 and fetching json?
#
Loqi It looks like we don't have a page for "time difference between parsing mf2 and fetching json" yet. Would you like to create it? (Or just say "time difference between parsing mf2 and fetching json is ____", a sentence describing the term)
#
[kevinmarks] And could the utf8 thing be affecting the html decode as well?
KevinMarks joined the channel
#
@jgmac1106 So with all the webmention articles floating around I sent a gentle reminder to authors asking for more diverse names beyond: Alice, Bobm and Chuck. @chrisaldrich pointed me to do this: https://en.m.wikipedia.org/wiki/Alice_and_Bob I had no idea (but not surprised) bias was… http://jgregorymcverry.com/5332-2/ (twitter.com/_/status/1024243357013422080)
#
Zegnat jgmac1106: talk of bias in placeholders, look at the gender distribution of this list: https://en.wikipedia.org/wiki/List_of_terms_related_to_an_average_person#United_States
#
Loqi Zegnat: petermolnar left you a message 6 days, 23 hours ago: why u no webring? https://indieweb.org/indiewebring
[jgmac1106] joined the channel
#
Zegnat !tell petermolnar “why u no webring?” I no webring coz webring not likey my indieauth before ;)
#
Loqi Ok, I'll tell them that when I see them next
#
[jgmac1106] @zegnat thx for this as well...
#
[jgmac1106] Webring love Indieauth now
#
Zegnat Woops, just noticed that was also discussed on -meta
#
[jgmac1106] Not your fault... It's where the feed took you
#
@KennParmar Webmentions: Enabling Better Communication on the Internet http://alistapart.com/article/webmentions-enabling-better-communication-on-the-internet (twitter.com/_/status/1024249050202177536)
KevinMarks, [xavierroy], Toffee and [grantcodes] joined the channel
#
[grantcodes] Finally added 🕸 links. [jgmac1106] will be happy 😛
#
@JmacDotOrg My #Perl library for Microformats2 parsing is now listed on http://microformats.org/wiki/microformats2#Development_parsers; thanks for the add, @gRegorLove! Hope to have it listed as production-ready someday. That means getting my experimental Webmention-capable Plerd branch merged into master, and that is Not Today. (twitter.com/_/status/1024274266324459520)
#
jgmac1106 grantcodes++
#
Loqi grantcodes has 16 karma in this channel (26 overall)
jeremycherfas joined the channel
#
@JmacDotOrg ↩️ The Plerd instance running on http://Fogknife.com now has supported Webmention for months. It works great! But, currently there’s no way to moderate received webmentions outside of hand-editing JSON files. And… that’s not public-release-ready, no. (twitter.com/_/status/1024275102689583104)
#
@vhendetta Webmentions: Enabling Better Communication on the Internet #webmentions http://rviv.ly/1HJrxD (twitter.com/_/status/1024277590650089472)
[tantek] and snarfed joined the channel
#
@jgmac1106 Teaching with RSS and webmentions is so much fun #DoOO #indieweb #clmooc #ds106 #literacies #edtechchat (http://jgregorymcverry.com/5346-2/) (twitter.com/_/status/1024283519554596864)
#
sknebel gonna look at it in more detail later, but with cleverdevils big feed mf2py with lxml spends pretty much 1/3 of time on the initial html parsing
#
snarfed sknebel++ ideally that should be more like 90%
#
Loqi sknebel has 42 karma in this channel (122 overall)
#
sknebel the rest seems on first glance annoyingly evenly distributed
jeremycherfas joined the channel
#
sknebel but found a few candiates to start with
#
Zegnat Sounds like you’ve got your hack time filled for IWC SF today, sknebel :D
#
sknebel gonna spend time on open PRs first, but yes
#
sknebel although I really should spend some time on writing stuff for my own site one of these days :P
#
sknebel but other people have shiny bugs to distract me all the time
snarfed joined the channel
#
sknebel oh snarfed, you checked if you really had updated to the copy-free mf2py?
snarfed and [snarfed] joined the channel
#
[snarfed] sknebel yes
snarfed, [jgmac1106] and [tantek] joined the channel
#
@EchoChamberUncu Liked INTERTEXTrEVOLUTION (http://jgregorymcverry.com)Teaching with RSS and webmentions is so much fun #DoOO… | john https://educationechochamberuncut.wordpress.com/2018/07/31/liked-intertextrevolution-jgregorymcverry-comteaching-with-rss-and-webmentions-is-so-much-fun-dooo-john/ (twitter.com/_/status/1024310112893247488)
tantek__ and [kevinmarks] joined the channel
#
[kevinmarks] Is that Html5lib+lxml or just lxml?
[cleverdevil] joined the channel
#
[cleverdevil] [kevinmarks] yes, the content encoding issue also impacted the HTML version, but represented an overall much smaller percentage of the total processing time.
[snarfed] joined the channel
#
[snarfed] [kevinmarks] hmm good question. for me at least, i tell bs4 lxml explicitly, so i assume/hope just lxml
#
[cleverdevil] Parsing the MF2 takes around 1.9 seconds, while parsing the JSON Feed takes around 0.007 seconds 🙂
#
[kevinmarks] Thing is, lxml gets some html rules a bit off (better than html-parser)
#
tantek__ that's amazing
#
tantek__ cleverdevil are you using an HTML5 parser? I've heard anecdotal reports that all the error correction / backcompat causes HTML parsing to be 10x slower than if you just did "plain" fragile XML parsing
#
@JRDingwall "Webmention" http://www.downes.ca/cgi-bin/page.cgi?post=68359 via ⁦@oldaily⁩ (twitter.com/_/status/1024314064430755842)
#
[cleverdevil] Whatever mf2py does.
#
[cleverdevil] (I am specifying lxml)
tantek, singpolyma and eli_oat1 joined the channel
#
aaronpk apparently I need to make my image proxy in monocle also resize images, because any time one of grantcodes' albums show up, it kills my browser trying to load that many high-res images
#
eight45.net deleted /Template:noffle "I misunderstood how user pages work!"
#
[cleverdevil] Oh, [aaronpk] that service is now available at https://validator.cleverdevil.io/validate?url=https://aaronparecki.com/2018/07/07/7/oauth-for-the-open-web.json
#
aaronpk yay nice
#
[cleverdevil] I am *positive* that I have gaps in the JSON Schema.
#
Loqi giggles
#
[cleverdevil] So, consider this turbo-beta.
#
[cleverdevil] 😄
KevinMarks joined the channel
#
Zegnat You can always fall back to my schema, [cleverdevil]. That should allow 100% of all mf2. Just doesn’t split up different types of objects like yours does
#
aaronpk so for what i'm thinking about, I actually would love both
#
aaronpk one pass to validate just the syntax of the JSON, ensuring all values are arrays, ensuring every object has a "properties" key, etc
#
aaronpk then another pass that goes into vocab to check whether the objects actually make sense
#
Zegnat just spotted an error in his schema
[miklb] joined the channel
#
[miklb] +1
#
Zegnat I haven’t updated for vendor prefixes, looks like
jgmac1106_ joined the channel
#
sknebel I should try mf2py under PyPy, I feel like it's the kind of code that probably benefits quite a bit
#
Loqi I agree
#
sknebel not really a good answer for general performance complaints though ;)
#
sknebel found a few spots worth trying to optimize
#
sknebel but of course the parsing baseline stays
#
sknebel and re the questions above, yes, I was looking to at using lxml in beautifulSoup, so not involving html5lib
snarfed and jgmac1106 joined the channel
#
Zegnat [cleverdevil], is that validator something that should be added to microformats.org/wiki/microformats2-json?
jgmac1106 and jackjamieson joined the channel
#
[cleverdevil] (Sorry, was on a call)
#
[cleverdevil] Maybe?
#
[cleverdevil] I am delighted to make any changes needed! Feel free to open issues on https://github.com/cleverdevil/microformats2
#
Loqi [cleverdevil] microformats2: Validate Microformats2 JSON with JSON Schema, and perform Post Type Discovery
#
[cleverdevil] I can easily add more lax schemas for basic validation, etc.
#
Zegnat Ah, I think I am already linking to that repo :D
[metbril] and snarfed joined the channel
#
@SoundEvo @MercedesBenz #Aclass got @JLAudio CP106LG-W3V3 MicroSub slot-ported #enclosure #subwoofer + C2-650/ C3-650 #Speakers + XD600/6V2 6-channel #amplifier (twitter.com/_/status/1024334955407724546)
#
Zegnat aaronpk, new microsub implementation ^^^ ;)
#
aaronpk lol
Evo, snarfed, leg and tantek__ joined the channel
#
@jgmac1106 ↩️ In just one day I have been able to get folks online with WordPress and microblog and sending receiving webmentions . Magic or hand coding not needed. #IndieWeb (http://jgregorymcverry.com/5355-2/) (twitter.com/_/status/1024337809421860869)
[chrisaldrich], jgmac1106, singpolyma, chrisaldrich, ibnesayeed, dbryant, eli_oat, [snarfed] and 17SAAJIW3 joined the channel
#
@aaronpk ↩️ oh yeah for sure. To be clear, my only involvement in http://micro.blog is having written several of the protocols that they are built on. e.g. https://www.w3.org/TR/micropub/ https://www.w3.org/TR/webmention/ https://www.w3.org/TR/indieauth/ I'm a big fan of getting many implementations early. (twitter.com/_/status/1024353712808943617)
Evo and snarfed joined the channel
#
gregorlove.com created /scuttlebut (+32) "r" (view diff)
dbryant joined the channel
#
sknebel sigh, it's too warm for benchmarking, my laptops performance is going all over the place
#
aaronpk lol
#
sknebel (vidyo doesn't help either I guess)
#
aaronpk can you do it on a vps somewhere?
#
sknebel yeah, or later on my desktop
#
sknebel but right now it's still too hot upstairs, so I'm holed up in the kitchen with my lapto
#
sknebel p
#
aaronpk sometimes I wonder if I would enjoy using a desktop
#
sknebel having multiple computers is both useful and a hassle
#
aaronpk yeah plus I wouldn't have a desktop mac
#
aaronpk anyway -> #chat
snarfed joined the channel
#
tantek.com uploaded /Special:Log/upload "uploaded "[[File:iwc-sf-2018-grid.jpg]]" https://indieweb.org/File:iwc-sf-2018-grid.jpg"
eli_oat1 and snarfed joined the channel
#
tantek.com created /acquire (+25) "Redirected page to [[acquisition]]" (view diff)
eli_oat joined the channel
#
tantek.com edited /Special:Log/upload () "uploaded a new version of "[[File:iwc-sf-2018-grid.jpg]]"" (view diff)
snarfed and [kevinmarks] joined the channel
#
@jgmac1106 Here is a video describing the power of webmentions in #edu5222. My students are amazed that they do all their learning from their own domain but their replies show up on each other’s post and our rss feed magically slurps up everything they write. (http://jgregorymcverry.com/5364-2/) (twitter.com/_/status/1024379643951034368)
ibnesayeed, KevinMarks, jackjamieson, eli_oat, singpolyma, eli_oat1, [jgmac1106] and [pfefferle] joined the channel
#
GWG snarfed, re the thing earlier... I will write something either way
#
snarfed thank you GWG!
snarfed, [Vanessa], tantek, leg and [chrisaldrich] joined the channel
#
loqi.me created /MeWe (+121) "prompted by [chrisaldrich] and dfn added by [chrisaldrich]" (view diff)
#
kaja.sknebel.net edited /MeWe (+1) "linkify ('x is y. <url>.' pattern)" (view diff)
#
boffosocko.com created /mewe (+187) "prompted by [chrisaldrich]" (view diff)
chrisaldrich, dbryant, myravery, jackjamieson and KevinMarks joined the channel
#
GWG Out of blind curiosity, how do people log outgoing Webmentions?
#
snarfed i doubt many people do
#
aaronpk telegraph logs them
#
aaronpk but I am not "many people" :D
#
GWG Is there an advantage?
#
aaronpk I only look at it now and then, really just if I happen to notice my webmention isn't showing up, then I check telegraph and see what happened
#
tantek__ that's exactly the kind of scenario we're trying to debug
#
tantek__ we can't tell if a webmention got sent for an RSVP from a WordPress site
#
GWG So that's why I am brainstorming
#
gRegorLove The ProcessWire module does fairly verbose logging. source, target, webmention endpoint, response code, response body
#
dougbeal What is ProcessWire
#
aaronpk ultimately it is only marginally useful because all I can usually see is whether telegraph tried to send a webmention or whether it failed to find the webmention endpoint. since most webmention processing happens async, the response from sending the webmention doesn't usually have anything useful in it
#
Loqi ProcessWire is an open source PHP content management system/content management framework https://indieweb.org/ProcessWire
#
gRegorLove It has been handy in the past when sending a webmention to a WP site returned an error
#
gRegorLove search "webmentions-sent" in https://github.com/gRegorLove/ProcessWire-Webmention/blob/master/Webmention/Webmention.module
#
tantek__ aaronpk, still useful to see if it sending was even attempted
#
tantek__ and whether via a wm endpoint or pingback endpoint etc.
#
aaronpk yep, it's helpful
#
tantek__ pretty sure I had some brainstorming on this
eli_oat1 joined the channel
#
gRegorLove And particularly the HTTP response codes / WP REST API responses. I got some unexpected ones, like comments being closed.
#
@oldaily What the heck is a webmention? #oldaily https://www.downes.ca/post/68391 From the summary of this video: " Here is a video describing the power of webmentions in #edu5222. (twitter.com/_/status/1024428508792139776)
dougbeal|mb1 and KevinMarks joined the channel
#
GWG I just have to figure out how to store it. I have a hook that is designed for triggering a log entry
#
dougbeal GWG: is there some super logging plugin we could use?
#
GWG I haven't looked
#
GWG I could log to the server logs
#
GWG Or the WordPress post meta table
#
dougbeal I would love a admin feed widget of outgoing webmentions
#
GWG Will brainstorm
#
dougbeal Are there other things that would useful? incomming/outgoing webmentions, salmentions, vouches?
#
dougbeal Indieauth
[jgmac1106] and snarfed joined the channel
#
dougbeal GWG: should file an improvment request? To the indieweb plugin?
snarfed joined the channel
#
GWG Webmention plugin
snarfed and Evo joined the channel
#
gregorlove.com edited /WebRTC (+12) "/* Services */ +vidyo" (view diff)
tantek joined the channel
#
@nick_chater What the heck is a webmention? https://ift.tt/2AtvfyZ #EdChat (twitter.com/_/status/1024441870628409345)
snarfed and eli_oat1 joined the channel