#dev 2017-10-09

2017-10-09 UTC
snarfed and tantek joined the channel
#
tantek.com edited /Homebrew_Website_Club (-112) "/* Cities getting started */ Brussels first meetup" (view diff)
leg, renem, KartikPrabhu, j12t, snarfed and cweiske joined the channel
#
seblog.nl edited /bookmark (+72) "/* Silos */ add Pocket to the list" (view diff)
jeremycherfas joined the channel
#
seblog.nl edited /Special:Log/upload () "uploaded a new version of "[[File:2017-10-09-pocket-reading-list.jpg]]": added a fake mouse to make the hover more obvious" (view diff)
#
seblog.nl edited /bookmark (+1008) "/* Silos */ add Pocket UI documentation" (view diff)
#
@pfefferle @chrisbergr Schick! Ich bin gespannt! Sogar mit Webmentions und POSSE/Syndication! #indieweb (twitter.com/_/status/917314350469263360)
#
petermolnar what is warc?
#
Loqi WARC is a web archive (Web ARChive) format designed to collect all of the resources that make up a web page into a single item that does not require any external references when opened https://indieweb.org/WARC
j12t, jonnybarnes and John___ joined the channel
#
seblog.nl edited /recommend (+46) "Medium: make past tense, add reference to the clap" (view diff)
jeremycherfas, eli_oat, tantek, jeremych_, j12t, John___, [eddie], [kevinmarks], KartikPrabhu and ludovicchabant joined the channel
#
ludovicchabant Heya -- didn't know this channel was there @aaronpk
#
tantek welcome ludovicchabant!
#
ludovicchabant thanks :) so yeah I was wondering if there was library support for handling both url-encoded forms and multipart/form-data in a transparent way
#
aaronpk hi! yep we try to keep tech jargon in this channel
#
ludovicchabant ok
#
aaronpk like I was saying, it should be part of most standard web frameworks, since it's how HTML forms work
#
ludovicchabant looking back at my logs, it looks like Werkzeug is maybe not handling it correctly because the request is done with the `application/json` mime type and not `multipart/form-data` ?
#
ludovicchabant (the request is coming from Quill's HTML editor btw)
#
aaronpk ah yeah so Quill's HTML editor only sends JSON requests
#
aaronpk if you want a tool to try sending all combinations of requests to your endpoint, https://micropub.rocks will do the job
#
ludovicchabant ah thanks, I'll try that
#
ludovicchabant is there any reason clients don't just stick to the standard HTML forms then?
#
aaronpk some of them do
#
aaronpk but we decided that instead of continuing to overload the form-encoded syntax for more complicated things like nested objects, we'd just use the JSON syntax at that point
#
aaronpk HTML content is an example of that. in form-encoded syntax it would have been content[html]=<b>hello</b> but then even fewer backend frameworks support that out of the box
#
ludovicchabant ok, makes sense, but for example for Quill it looks like the JSON is more complicated than it could be? Pretty much all properties are lists when most could be just strings? Or is there any use-case for, say, multiple 'name' or 'mp-slug' values?
#
aaronpk yeah that's because micropub's JSON syntax is the parsed Microformats JSON, and in that case every value is always an array
#
aaronpk it's a tradeoff between making the spec vocabulary-aware vs generic
#
ludovicchabant ah ok -- I'll have to lookup the Microformats JSON spec then
#
ludovicchabant for the content[html] stuff, what's the pros/cons vs. just making another property, like "content_html" vs "content"?
#
ludovicchabant or is that basically a problem of remapping Microformats JSON syntax back into HTML form data?
#
aaronpk well tying it to microformats means micropub is defining one less thing on its own, which is usually a good thing for specs
#
ludovicchabant ok
#
aaronpk and trying to represent everything as form-encoded unfortunately means you have to make up something new at some point, since form-encoded syntax is very limited by default
#
ludovicchabant well I guess I'll check if there's a Microformats lib for python then :) otherwise I think if the spec guarantees that every property is a list, the server code might not be too complex
#
aaronpk yeah in the JSON micropub syntax, every value will be a list even if there is only one value
#
ludovicchabant cheers! (spoiler warning: you'll probably hear more from me in the next couple weeks :) )
#
aaronpk excellent :)
[eddie] joined the channel
#
[eddie] hey aaronpk: I fixed some errors on my compass server, but I’ve found a new one I’m not sure about: “No supported encrypter found. The cipher and / or key length are invalid.” Stack overflow said that is typically an OpenSSL issue or an .env error. I don’t see anything in the .env.example about cipher so do I need to install something open SSL separately?
#
[eddie] I’m not sure what all software composer takes care of
#
aaronpk wow interesting
#
aaronpk what PHP version?
#
[eddie] 7.0.22
#
aaronpk ooh i bet the key in .env is wrong
#
aaronpk try running php artisan key:generate and it should generate one
#
[eddie] Hmm maybe I’m missing something installed: “There are no commands defined in the "key" namespace.”
#
aaronpk ?
#
aaronpk k just make 32 random characters and set that in the APP_KEY property in .env
#
[eddie] Gotcha ?
jeremycherfas joined the channel
#
[eddie] (I am making notes so we can have a “troubleshooting” section of the README in case anyone runs into some of these in the future. Once I get this all working 100% I’ll send a PR with that section)
#
[eddie] Victory! Looks like I don’t have memcached installed ? that should be easy enough. Hopefully we’re close to the finish line
#
aaronpk hm i think memcache is optional
#
aaronpk yeah you can swap that with something else if you want, check .env
#
aaronpk this is the framework it uses https://lumen.laravel.com/docs/5.1
#
aaronpk actually i dont think it uses a cache at all
#
aaronpk try setting SESSION_DRIVER=file
davidmead joined the channel
#
[eddie] Ohhhh
#
[eddie] Wooo!!!! https://files.slack.com/files-pri/T03QR2B2T-F7EV58AGG/image_uploaded_from_ios.png?pub_secret=4ffb060f2a&name=Image uploaded from iOS.png
#
aaronpk awesome!
#
aaronpk now hopefully you can log in!
[cleverdevil] joined the channel
#
[cleverdevil] So, FYI, grantcodes has been making good progress on the prototype for Together - https://github.com/cleverdevil/together
#
Loqi [cleverdevil] together: The together project: an IndieWeb experience
#
aaronpk oh wow
#
[cleverdevil] http://share.cleverdevil.io/UDg7MT4im9.png
#
[eddie] Yeah it’s looking nice! Downloaded it the other day!
#
[cleverdevil] Good job, grantcodes ?
[grantcodes] joined the channel
#
[grantcodes] Yeah it's coming along
#
[grantcodes] Really need a microsub server to be able to build it out further
#
[cleverdevil] Indeed.
#
[eddie] ?? compass: “Fatal error: Call to undefined function IndieAuth\curl_init()”
#
[cleverdevil] That's an aspect I can potentially actually help with.
#
[cleverdevil] I am not so awesome with JavaScript though ?
#
aaronpk [eddie]: oh no! that sounds like php is missing curl functions
#
[cleverdevil] I may write a Python-based microsub server.
#
[cleverdevil] aaronpk do you have a prototype microsub server somewhere?
#
[cleverdevil] Or know of one?
#
aaronpk ii don't actually yet. it's all just brainstorming right now
#
[cleverdevil] Gotcha.
dougbeal|mb1 joined the channel
#
[eddie] aaronpk: huh that’s odd
#
[eddie] I’ll install that
#
[grantcodes] cleverdevil Yeah even some sort of mock server would be super helpful, doesn't need to actually work fully according to the spec, just return sample data
#
[grantcodes] Then there is also the cors issue unless the microsub spec is going to define something about that
#
aaronpk that's an interesting one
#
[grantcodes] same deal with micropub
#
[grantcodes] But it's kind of difficult to solve as well since you also need to scrape the persons site to find their endpoints, so their homepage would need to have the Access-Control-Allow-Origin header set up
#
aaronpk i should probably start experimenting with writing a Microsub server
#
Loqi agreed.
Kongaloosh joined the channel
#
[cleverdevil] Sometimes the best way to hash out details on a spec is to actually build something.
[sebsel] joined the channel
#
aaronpk definitely
#
aaronpk almost always in fact
[kevinmarks] joined the channel
#
[grantcodes] I'm almost tempted to have a go at building one as well. Would be kinda awesome to have 3+ implementations before the spec is even complete ?
#
[cleverdevil] Haha
#
Loqi rofl
#
[cleverdevil] As fun as that'd be, I'd rather collaborate on one ?
#
[grantcodes] Yeah that's fair
#
[grantcodes] In another topic should micropub endpoints support delete requests for files uploaded to the media endpoint? Just ran into that problem.
#
[cleverdevil] I don't see anything in the micropub.rocks tests for that.
#
[grantcodes] Yeah there's no documentation on it. All I can see is "Micropub servers should support deleting posts"
#
[grantcodes] But no reason you couldn't just pass the url to a file rather than a post
#
[kevinmarks] Having 3 implementations is really good for a spec
#
[grantcodes] But then that would be to the micropub endpoint not the media endpoint and they could be seperate
#
aaronpk if you're going to delete from the media endpoint I'd think that request would be made to the media endpoint
#
aaronpk the media endpoint isn't intended to be a full media collection manager tho, more like an in-transit thing
#
aaronpk the way mine works, my micropub endpoint ends up downloading a copy of the file at the media endpoint and storing it locally anyway. i could eventually write a cleanup script that would delete old stuff from the media endpoint even.
gRegorLove joined the channel
#
[grantcodes] Yeah, I thought it would need to be to the media endpoint because of the separation. Could the media endpoint not just support DELETE requests then? Actually no I just realised that wouldn't work, because the url of the file is not the same as the endpoint
#
[grantcodes] My use case is if I upload the wrong photo to an article or gallery or whatever and remove it - it gets removed from the micropub request but the wrong file is still sitting there
#
aaronpk yeah that's part of the "cleanup" that the media endpoint can do
#
aaronpk the twitter API docs (where the media endpoint idea came from) says that if a file isn't used in a tweet in some amount of time, it will be deleted
#
[grantcodes] Hmm yeah I suppose that's possible, bit more difficult to build though!
#
aaronpk easier for clients tho!
#
aaronpk since they don't have to do anything
#
[grantcodes] Actually how would the media endpoint know that the file isn't used if it is separate from the micropub endpoint?
#
aaronpk it can know whether the file has been downloaded
#
[grantcodes] But the file could potentially never be downloaded.
#
aaronpk right, then it's safe to delete
#
aaronpk pick some time window, like a week. if nothing has requested the file in a week, it was likely never actually used
#
[grantcodes] Not necessarily, I have mine on the same server so I just link to the file, I don't download it, so it would require someone to actually view the post that the file is attached to
#
aaronpk does your browser show you the post after it's been created?
#
[grantcodes] Depends on the micropub client.
#
aaronpk i guess my point is if both endpoints on an implementation are tightly coupled, then there are several mechanisms you can use to determine whether a file has been used in a post
#
[grantcodes] Yeah I agree, but if they are not then automatic deletion is a (quite unlikely) potential issue
#
aaronpk also disk space is cheap, so having an extra file or two around isn't the end of the world
#
[grantcodes] If you are dealing with full res photo galleries it can become an issue ? - I know I'm about the only person doing that at the moment though
#
aaronpk yea i suppose. I still think there are ways to deal with it when it becomes important, without requiring that clients add any additional logic
mdcox joined the channel
#
[grantcodes] Cool, fair enough. May be worth mentioning somewhere in the docs ?
#
aaronpk considers attempting to write a microsub endpoint right now
KartikPrabhu and [kevinmarks] joined the channel
#
aaronpk reviewing my AWS usage from last month, and discovered that people have downloaded 63gb of avatar icons off my site in september
#
KartikPrabhu wow!
#
aaronpk i think i might stick cloudflare in front of that
#
KartikPrabhu cloudfare in front of AWS infront of your site?!
tantek joined the channel
#
aaronpk just the avatars
#
KartikPrabhu that seems like a lot of moving parts though
#
aaronpk they're hotlinked directly to my S3 bucket they're hosted on
#
KartikPrabhu oh Isee, so your site is skipped in some sense
#
sknebel I think technically Cloudflare doesn't allow that kind of use, but it's low enough that it seems unlikely they care
#
sknebel (doesn't AWS have some CDN-thing as well? cloudfront?)
#
aaronpk yes but it's more expensive per GB than serving directly from S3
#
sknebel lol
#
aaronpk i mean that's fair, running a geographically distributed CDN is more work than serving from S3
#
sknebel ok, yes, depends on how involved it is
#
aaronpk this looks like what i want https://wsvincent.com/static-site-hosting-with-s3-and-cloudflare/
[eddie] joined the channel
#
aaronpk ah nuts, I need to rename the bucket for this to work
tantek joined the channel
#
aaronpk new bucket configured, new avatars will end up there. eventually i can migrate the old ones over, but in the mean time will see how this goes.
#
aaronpk !tell snarfed i think something is wrong with my instagram bridgy. in the last log https://brid.gy/log?start_time=1506251680&key=aglzfmJyaWQtZ3lyFgsSCUluc3RhZ3JhbSIHYWFyb25waww it says "Demoting original post links because user tag:instagram.com,2013:aaronpk doesn't match author tag:instagram.com,2013:1500881" and it says the last syndication link was found 2 weeks ago.
#
Loqi Ok, I'll tell them that when I see them next
#
[eddie] aaronpk: Back home. Compass is getting so much closer! But not quite there ? Mysql error this time. the users table doesn't seem to exist. Is that supposed to be auto-created at some step or is there a script I'm supposed to run?
#
aaronpk ah there's a DB init step
#
aaronpk php artisan db:migrate
#
aaronpk oops
#
aaronpk php artisan migrate
#
[eddie] Yay!! I have a user interface screen!
#
Loqi giggles
#
[eddie] Thanks loqi
#
Loqi you're welcome
#
[eddie] very appropriate
#
[eddie] lol blast you!
#
[eddie] Thank aaronpk! This is exciting, I think I might be there!
#
aaronpk woo!
KartikPrabhu joined the channel
#
aaronpk sorry it's been such a hassle! will definitely appreciate your notes on this
#
[eddie] No worries, it takes different environments to get this stuff worked out ? I'm just happy to help.
#
[eddie] Speaking off... looks like i got an internal server error (500) back to Overland. I'll check my logs on Compass real quick to see what is causing that. I created the database inside of Compass and copied the API endpoint string into the app.
#
aaronpk hooboy
#
loqi.me created /static_website (+24) "prompted by tantek and dfn added by tantek" (view diff)
#
[eddie] Hmm, I successfully can hit the /api/last with my read token and no 500 error
#
[eddie] so it's just the /api/input that presents the 500 error
#
loqi.me created /PieCrust (+124) "prompted by tantek and dfn added by tantek" (view diff)
#
kaja.sknebel.net edited /PieCrust (+1) "linkify ('… is <url>' pattern)" (view diff)
#
[eddie] Even stranger. If I send a GET to the /api/input I get the database name back
#
loqi.me edited /content_management_system (+15) "tantek added "[[PieCrust]]" to "See Also"" (view diff)