#dev 2017-08-22

2017-08-22 UTC
#
aaronpk
hm i think i need to use shpub more
#
KartikPrabhu
what is shpub?
#
KartikPrabhu
what is shpub?
#
Loqi
shpub is a command line micropub client written by Christian Weiske https://indieweb.org/shpub
#
aaronpk
hm shpub didn't find my media endpoint
#
aaronpk
oh i forgot i had already connected a different micropub endpoint that doens't have one, oops
#
aaronpk
hm nope that wasn't it
#
ashersilberman.com
edited /IRC_People (+2) "/* Nicknames */"
(view diff)
[kevinmarks] and [manton] joined the channel
#
aaronpk
i really need a way to post videos to my site without instagram
#
aaronpk
generating the thumbnail frame is probably the biggest hurdle right now
[cleverdevil] joined the channel
#
[cleverdevil]
I am also interested in this aaronpk.
#
aaronpk
yeah, i could add an interface to quill where it lets you upload a video and thumbnail, but i'm not sure that would actually solve it for me
#
aaronpk
i like how instagram lets you choose the frame to use as a thumbnail, it makes it very easy
#
[cleverdevil]
I wrote this not so impressive script earlier today
#
aaronpk
i've been lazy and have only been posting mp4s
#
[cleverdevil]
I don’t blame you. Covers 98% of the audience.
#
Ruxton
aaronpk: are you looking for a conversion service or looking to write something?
mblaney joined the channel
#
aaronpk
Ruxton: looking for a UI solution before I make that decision
#
Ruxton
JS side thumbnailing for videos that are browser supported
#
mblaney
!tell jeremycherfas SimplePie can help with a lot of what you're after, it's written in PHP.
#
Loqi
Ok, I'll tell them that when I see them next
KartikPrabhu, loicm, [kevinmarks], cweiske, barpthewire, [pfefferle] and schmarty joined the channel
#
ignite.digitalignition.net
edited /SemPress (+69) "/* Child Themes */"
(view diff)
#
Zegnat
TIL: IE will add “helpful” LTR/RTL reading direction characters to date strings created with JavaScript’s toLocaleString. https://stackoverflow.com/a/25317146
#
Zegnat
Helpful until code feeds it into a PDF processor that then renders those glyphs as question marks… And then I spent a good 2 hours trying to figure out where those question marks came from.
#
sknebel
sounds like a bad PDF processor ;)
GWG and jeremycherfas joined the channel
#
jeremycherfas
Loqi messages?
#
Loqi
jeremycherfas: mblaney left you a message 5 hours, 9 minutes ago: SimplePie can help with a lot of what you're after, it's written in PHP.
#
jeremycherfas
Thanks mblaney I remember looking at simplepie a good long while ago. Currently seems like overkill for me, but will investigate.
KartikPrabhu and [kevinmarks] joined the channel
#
[kevinmarks]
You don't want to do your own feed parsing. So many dragons there. You want a library that has been through the pain.
j12t joined the channel
#
jeremycherfas
Even if I want to parse only one feed from one source?
#
jeremycherfas
And is there a difference between the guid I see in the RSS Atom feed and the UID that Ryan was referring to?
#
Zegnat
No, that’s the same thing in this case, jeremycherfas
#
jeremycherfas
Thanks zegnat
#
Zegnat
A lot of the problems with RSS parsing come when you want to support any and all RSS. Because people have been doing so many things with RSS over a long time, introducing more and more pitfalls.
#
jeremycherfas
My worry about using SimplePie, for example, is that it will introduce complexities that — for now at least — will make life more difficult.
#
Zegnat
If you are only parsing 1 feed, and you know the source of that feed, and how it is structured, you might as well treat it as “I am parsing an XML file with data” instead of “I want to parse RSS”
#
jeremycherfas
Maybe if I want to extend the little script I am writing, to post from MY RSS feed to MY Known instance, I’ll get into that
#
jeremycherfas
That’s exactly my approach. And I have step 1 - a datafile that contains the XML
#
Zegnat
For reading XML in PHP, I recommend just using the built in DOMDocument class.
#
jeremycherfas
That is exactly where I am right now. Is there a better source of instruction that php.net/manual?
#
Zegnat
Hmm, I don’t think so, but maybe someone will link something
[colinwalker] joined the channel
#
Zegnat
Actually, this chat might be a good source of instructions, jeremycherfas ;)
davidmead joined the channel
#
jeremycherfas
Do I really need domdocument? It seems to require me to define the structure of the Document, but that is already given by the RSS.
#
petermolnar
jeremycherfas simplepie is good for you
#
jeremycherfas
If you say so, petermolnar. I’m going to go a little further into this rabbit hole and then re-assess.
#
petermolnar
the unfortunate truth is that it's not an overkill, given all the things that can go wrong with xml parsing
#
jeremycherfas
OK. Time to reassess then.
#
jeremycherfas
But doesn’t it mean I have to download and use simplipie locally?
#
petermolnar
it does, yes
#
petermolnar
you have to have it as a lib at the place you're running the script
#
petermolnar
as an include
Zegnat, Kaja and sknebel_ joined the channel
#
Zegnat
You don’t need to define anything for DOMDocument, jeremycherfas. Not sure what you are looking at
#
Zegnat
Also, petermolnar, I believe most XML attacks like billion laughs are patched against in DOMDocument. No extra tools required for it.
#
jeremycherfas
I have the data in $xml and was trying to use $dom->loadXML($xml);
#
Zegnat
That sounds about right
#
jeremycherfas
I got this error Start tag expected, '<' not found
#
Zegnat
That sounds like something is wrong with the data then, if it can’t detect a < to start parsing
#
jeremycherfas
Ah, my mistake. I was trying to validate the xml, because I thought tht’s where the mistake was, and that wanted the DTD. But that is not where the mistake was.
#
jeremycherfas
Well, that’s good. I can now get the RSS and read each guid in it. That seems like a good place to stop for lunch.
#
jeremycherfas
Next I think I will try to see whather I can read the XML from the stored file
schmarty, plindner, barpthewire and j4y_funabashi joined the channel
#
cweiske
for reading XML in PHP I recommend PHP's built-in simplexml
#
j4y_funabashi
aaronpk: I implemented some of https://indieweb.org/Micropub-brainstorming#Query_for_post_list in my micropub server
#
Zegnat
cweiske, I have had a lot of headaches with simplexml and its tendency to make stuff array-accessible. I prefer the stricter design of DOMDocument.
#
j4y_funabashi
aaronpk: not got pagination in there yet, it just returns the latest 30 posts but it works great! have got the feed showing up in my micropub client, just need to add some action buttons
[kevinmarks] joined the channel
#
[kevinmarks]
if you're just parsing your own site, why use RSS and not mf2?
#
cweiske
1. I want to parse an rss feed
#
cweiske
2. But you need a library for that!
#
cweiske
3. ok, I'll use simplepie
#
cweiske
4. why don't you use mf2, now that you already comitted to using a lib?
#
sknebel
(for which you need a library :P)
#
[kevinmarks]
when he said it was just for his own site, that was when I was puzzled
#
[kevinmarks]
where did this RSS yak come from? What was the original goal again?
#
cweiske
push things from a feed into jeremy's known instance
#
[kevinmarks]
but what was the feed from?
#
cweiske
no idea
#
[kevinmarks]
is this a PESOS tool?
#
cweiske
I think yes, but don't know
j12t joined the channel
#
jeremycherfas
Cweiske: Good to know. I am using simplexml at the moment to turn it into JSON
#
jeremycherfas
Kevinmarks: I am not parsing my own site. I am parsing an RSS of my personal feed at reading.am with a view to automating PESOS from there.
#
j4y.co
edited /Micropub-brainstorming (+337) "/* Query for post list */"
(view diff)
#
Zegnat
Wow, reading.am lets you set-up your own webhooks, that’s pretty cool!
loicm joined the channel
#
Zegnat
jeremycherfas, have you thought about using their webhook to post to your site whenever you start reading something?
#
cweiske
that would need a reading.am-webhook2micropub-proxy
#
jeremycherfas
Er, no. I’m not sure I actually saw that they have webhooks. Except, perhaps subconsciously, because it shares to places like Pinboard
#
Zegnat
If you go to https://www.reading.am/settings/hooks and select “URL” on the dropdown. You can give it a URL for a file on your site.
#
Zegnat
It will then do a POST request to that file for every new thing you read.
#
Zegnat
Doesn’t say exactly what the POST contains though
#
jeremycherfas
Hang on. Doesn’t it need an authentication to POST to my site?
#
cweiske
that depends on the script that handles the POST
#
jeremycherfas
At my end?
#
cweiske
you might add a ?key=asdf to the POST URL, identifying reading.am via that key
#
cweiske
yes, on your end
#
jeremycherfas
God, this could save me tearing my hair out pretending to be a developer.
#
Zegnat
You can just have a “thisisalongrandomfilename.php” file that accepts POST data. reading.am will send off a POST request to that with whatever you are reading at the time. And your PHP script can do a thing with it, e.g. make a post on your own site.
#
jeremycherfas
Maybe I should wait until vHWC to understand this.
#
jeremycherfas
Nothing comes through sending it to /like/edit but maybe I should be sending it to micropub?
#
cweiske
it won't work
#
cweiske
because the structure of the data in the POST request is not understood by known at all
#
cweiske
you basically need a script that translates the POST data coming from reading.am into something that known understands
#
cweiske
i.e. micropub
#
jeremycherfas
I suppose I need to look at my logs to see what was sent.
#
cweiske
no, this is not in the logs at all
#
cweiske
the logs only contain URLs and GET parameters
#
Zegnat
jeremycherfas, you can have a PHP file like this on your server and give reading the link to it: https://gist.github.com/Zegnat/8b7af691e1ad6781bf35ab09a7397324
#
jeremycherfas
Nothing from POST, you’re right.
#
Zegnat
That would create a POST.log file with the contents of PHP’s $_POST array.
#
jeremycherfas
Zegnat: I put that in the root of https://stream.jeremycherfas.net and it would write the details of every incoming POST request to POST.log?
#
Zegnat
Only the latest POST request. Otherwise you need to add an append flag to the function so it will become a real log
#
Zegnat
Otherwise, yes
#
Zegnat
Also, not every POST request, sorry, only POST requests aimed at the file
#
Zegnat
I am not sure how you would go to log every POST request to a server … cweiske?
#
jeremycherfas
I honestly think I’ve got too many conversations going on at once here. Each of them sends me off in a different direction. I think maybe I need to keep plodding along using things I understand a bit better, like RSS and very limited PHP.
#
jeremycherfas
So far I am able to get the RSS file from reading.am. I have tried using DOMDocument to read specific nodes, but I had trouble writing them to file because they need to be strings.
#
jeremycherfas
And I can convert the XML to JSON and read the whole object.
#
Zegnat
“trouble writing them to file because they need to be strings” - what do you need to be a string?
#
jeremycherfas
Currently I am searching around for ways to convert $guid->nodeValue into a string so I can write it to file.
#
jeremycherfas
But I have a feeling I may be better off concentrating on simplexml and working with that, rather than with DOMDocument
#
jeremycherfas
And I need to stop and do some other work anyway.
#
jeremycherfas
But I’ll be lurking here.
#
jeremycherfas
BTW zegnat, did you get my RSVP?
#
Zegnat
I did! I was surprised by the URL though, the “-1” was something I didn’t expect.
#
jeremycherfas
All I said was Yes.
#
cweiske
Zegnat, one could use a postbin tool
singpolyma joined the channel
#
jeremycherfas
Well, that certainly looks handy.
#
jeremycherfas
bookmarks it in my super-cluttered iamnotadev folder
KartikPrabhu joined the channel
#
schmarty
good morning, dev
#
Zegnat
jeremycherfas, the permalink to your RSVP is -1: https://stream.jeremycherfas.net/2017/-1
#
Loqi
[Jeremy Cherfas] Yes:
#
jeremycherfas
That is very odd indeed. But Known can be very odd indeed.
snarfed joined the channel
#
Zegnat
I should maybe put some time into displaying RSVPs on event posts. I get the webmentions, but people like confirmation
#
Loqi
definitely
j12t joined the channel
#
aaronpk
j4y_funabashi: awesome!
[miklb] joined the channel
#
j4y_funabashi
aaronpk: I have altered the wiki page to reflect the current state of my implementation
#
j4y_funabashi
It is making me think that this extension opens up a new set of clients that render feeds/posts etc. So they can act as the front end to your website
#
aaronpk
what is microsub?
#
Loqi
Microsub is an early draft of a spec that provides a standardized way for clients to consume and interact with feeds collected by a server https://indieweb.org/microsub
#
aaronpk
that's kind of what i was thinking with https://indieweb.org/Microsub-spec
#
j4y_funabashi
yeah tis similar to that but for your 'private' posts
snarfed joined the channel
#
j4y_funabashi
eg I only really show photos on my site/in feeds but I post notes/bookmarks/likes etc
#
aaronpk
instead of just showing those to you on your site when you're logged in?
#
j4y_funabashi
I am kind of thinking that this client is going to be the frontend to my site so it would be the thing I login to see everything
#
aaronpk
interesting, and then your site is more just the public-facing part?
#
j4y_funabashi
currently I kind of lump in post/feed rendering with my micropub server.
#
j4y_funabashi
but it is making more sense to have rendering in this new client, that way when I add the ability to post new types then I can add rendering code in the same place
snarfed joined the channel
#
j4y_funabashi
because I am rendering my site with the same client I am posting with
#
j4y_funabashi
hope that makes sense
#
j4y_funabashi
tis basically decoupling micropub server from rendering
#
aaronpk
interesting
#
j4y_funabashi
which could also mean people could 'plug in' diff clients to their micropub server
#
j4y_funabashi
and have their site rendered differently
[kevinmarks] joined the channel
#
[kevinmarks]
A "turn a reading.am post into a micropub post sounds like an amazon lambda thingy
#
jeremycherfas
What is amazon lambda?
#
Loqi
It looks like we don't have a page for "amazon lambda" yet. Would you like to create it?
#
sknebel
or a php file living next to their other php files. no need to bring an entirely new stack into it
#
sknebel
if you wanted to provide it as a public service, *maaayyybee*. but not for a one-off
#
aaronpk
lambda is a great way to make a simple thing complicated
#
jeremycherfas
I was just going to say, not another rabbit hole to dive down.
#
aaronpk
don't bother. I seriously doubt your application would gain anything from the benefits of lambda
#
jeremycherfas
My “application” is currently a heap of mess, but cleaning up that mess will probably teach me more than starting a new mess.
#
voxpelli
Anyone good at Ruby that wants to attempt a PR for https://github.com/jcs/lobsters/issues/383#issuecomment-324051466 ?
#
Loqi
[jcs] I'm not opposed to it, feel free to send a PR for it.
#
voxpelli
aaronpk maybe? ;)
#
aaronpk
eeeenteresting
#
aaronpk
hm they don't have a queueing mechanism yet, they use cron jobs for things like emailing and posting to twitter
#
aaronpk
i wonder if sending webmentions on a cron job would be good enough
#
voxpelli
probably a good first step to keep PR small
#
aaronpk
and probably sending webmentions only for submitted stories, not links in comments just yet
#
voxpelli
they already have the entire mechanism for fetching meta data from the page, so detecting webmention support should probably be easy
#
aaronpk
oh good call, where's that?
#
voxpelli
that looks like what I found last time
#
aaronpk
so, look for webmention endpoint there, store it in the DB along with the story info. then keep track of whether a webmention was sent and send them on a cron job?
#
voxpelli
that sounds enough for a first version
#
aaronpk
oh, what about mf2
#
voxpelli
right, that would be good as well, perhaps as a separate commit to make it optional if they feel more hesitant about that?
eli_oat joined the channel
#
aaronpk
hm yeah that's probably best kept as a separate PR cause i can see how people would have strong opinions about it
#
voxpelli
perhaps add it in the same PR but just as a clearly separate commit so that one can see how easily it can be thrown out if disliked?
#
voxpelli
I would happily help with any non-ruby-specific parts :)
#
j4y_funabashi
Does anyone else return extra information from their media endpoint?
#
j4y_funabashi
I am currently returning EXIF date and lat/lng read from the photo after upload. Just wondering if other people do something similar?
#
voxpelli
(also: if someone needs a Lobster invite, I do have, so just ask)
[miklb] joined the channel
#
[miklb]
what is Lobster
#
Loqi
It looks like we don't have a page for "Lobster" yet. Would you like to create it?
#
sknebel
What is lobsters?
#
Loqi
Lobsters is an open source, technology-focused link-aggregation site similar to Hacker News https://indieweb.org/lobsters
#
sknebel
[miklb]: ^^^
#
[miklb]
thanks
snarfed joined the channel
#
www.svenknebel.de
moved /lobsters to /Lobsters "canonical name is capitalized"
#
snarfed
reviews when and why wm.io returns HTTP 303. https://chat.indieweb.org/dev/2017-03-22#t1490214293318000
#
Loqi
[aaronpk] ah it sends a 303 if you send an accept html header
#
aaronpk
Yeah for the browser webmention form
#
snarfed
i have to do conneg with rhiaro.co.uk to make it send me html, so i'm accidentally sending the same conneg header to wm.io for rhiaro's webmentions ??
[manton] joined the channel
#
aaronpk
Oh gosh
j12t joined the channel
#
snarfed
i actually wonder how many non-bridgy wms rhiaro gets at all, since you have to send her site conneg to even discover her endpoint
#
sknebel
snarfed: she has the endpoint in the headers always
#
snarfed
ah true good point
#
sknebel
verifying WMs from her site might fail because of conneg through
gRegorLove, j12t, j12t_, snarfed, j12t__, KartikPrabhu, [jeremycherfas] and [manton] joined the channel
#
Zegnat
To be fair, rhiaro is perfectly in the right, without sending an Accept header you can’t assume you will get HTML back.
[miklb], snarfed and davidmead joined the channel
#
rhiaro
Zegnat: I figure since browsers send Accept headers and html is for browsers..
#
rhiaro
Also my wm ep is in an http header
#
rhiaro
So discoverable without even fetching content
#
snarfed
rhiaro: yup! sknebel mentioned
#
snarfed
fwiw though, this isn't going to be the most receptive audience for "html is for browsers" :P
#
rhiaro
Oops i missed that
#
snarfed
also as i mentioned, lots of the webmention code i've seen checks for text/html and gives up otherwise, before it even starts looking for a wm endpoint, even in headers. so just so you know. you may miss lots of wms due to that
#
rhiaro
I'll live
eli_oat and leg joined the channel