#dev 2016-10-05

2016-10-05 UTC
bret, KevinMarks, KevinMarks_, gRegorLove, tantek, cweiske, ChrisAldrich, kylewm and AngeloGladding joined the channel
#
vanderven.se martijn
edited /Main_Page (+21) "/* Homebrew Website Club */ Nürnberg has HWC today."
(view diff)
#
sebastiangreger.net
edited /Planning (-64) "/* Berlin */"
(view diff)
#
bear
grr, php-isms are all over the place
#
bear
am I going to have to just assume that any micropub item could possibly have an "array" variant and process all of them as arrays
#
bear
photo currently is what is vexing me
#
cweiske
because of $_FILES['photo']['name'] vs. $_FILES['photo'][0]['name']?
#
bear
I'm working in python
#
bear
and when I get a list of keys from the POST request I have to handle 'photo', 'photo[value]', 'photo[alt]' now
#
cweiske
it gets even better if someone uses photo[0][value]
#
cweiske
or photo[][value]
#
bear
my php-ism rant is about how php has leaked/polluted urls
#
bear
yea, well - that is my hope for this micropub.rocks test site - we can stamp out clients that send that kind of nonsense
#
cweiske
bear, that's actually part of the spec
#
bear
oh my word WHY
#
cweiske
and those examples are POST data, not URL parameters
#
cweiske
bear, because everything in micropub is an array
#
cweiske
if it isn't, you have to convert it
#
cweiske
.. because in microformats2 everything is an array and MP follows MF
#
bear
yea, see I'm not a web dev, so when I see the url contain those items I call them url params
#
cweiske
are you talking about POST or GET ?
#
bear
POST
#
bear
oh - i'm probably being confused by how they are displayed in micropub.rocks
#
cweiske
it's the HTTP request body
#
bear
'''h=entry&
#
bear
content=Micropub+test+of+creating+a+photo+referenced+by+URL&
#
bear
photo[value]=https%3A%2F%2Fmicropub.rocks%2Fmedia%2Fsunset.jpg&
#
bear
photo[alt]=Photo+of+a+sunset'''
#
cweiske
www-form encoded
#
bear
that looks like url data to me
#
cweiske
it's actually the body of the http request, form-encoded
#
bear
k, apologies for the wrong lingo
#
bear
my rant still applies tho :/
#
cweiske
about []?
#
bear
if we have photo[]=foo&photo[]=bar for two photo urls then why use photo[value]=foo&photo[value]=bar
#
bear
the alt would then become photo[alt]=foo&photo[alt]=bar
#
bear
also, is it valid to have: photo=foo&photo[]=bar&photo[]=baz ??
#
cweiske
what would python do instead?
#
bear
oh, I can handle this in python - the requests lib does a lot of work behind the scenes to deal with this
#
bear
I am just commenting about a spec that seems to have redudant ways to send the same information
#
cweiske
there are redundant ways, yes
#
cweiske
category=foo or category[]=foo
#
bear
if the reason for the array parts is "because MF2 is all arrays" then the standalone photo= should not be allowed
#
cweiske
category[0]=foo
#
cweiske
bear, that would ease implementation indeed
#
bear
yea, I'm going to have to add so many edge cases to this parsing code that it will increase the surface area for bugs mightily
#
bear
goes back to coding for the current method and will make a case to aaronpk later
tantek joined the channel
#
aaronpk
The form encoded version is meant to be a shortcut, easier to implement on both sides
#
aaronpk
the JSON version is strict, and always has to be a mf2 object, so everything will always be an array
#
aaronpk
if it's getting problematic with the form encoded version then I'm okay with stripping some of the edge cases from it and basically saying "if you want to do X then you need to make a JSON request"
#
aaronpk
Great example is the photo alt thing. It very well may be that checking whether photo is an array and whether that array has numeric keys or a key called "alt" is too complicated, and we just drop alt from the form encoded version and require people send that as JSON instead
#
aaronpk
But I will say that it was not hard to accept single value or arrays for normal string properties. PHP decodes the form encoded request so $_POST['category'] ends up as either a string or an array. My code looks at it and if it's not an array it wraps it in one.
#
aaronpk
I'd recommend doing something similar in other languages, basically adding a preprocessing step to normalize the params so your code doesn't have to deal with the literal [] in the post body param names
#
bear
aaronpk - yes, that is exactly what i'm having to do
#
bear
add all kinds of preprocessing for these edge cases that seem extremely redundant
#
bear
which are different than the processing for a json post (which is supposed to be strict but since json is freeform I will still have to validate it's values)
#
bear
case in point is category - I now have to handle category= category[]= and category[N]=
#
bear
oh, and i'll stop before I even get to the nested object example - how can I possibly handle that when I cannot safely evaluate the array other than storing it as a raw string which really chaffs my beard
#
aaronpk
it's really not that hard in PHP, so maybe you can do what PHP did and pre-process the raw request to turn it into a data structure before it hits your code?
#
bear
that's my point - your now asking me to run a eval against data i'm receiving from the outside
#
aaronpk
i would assume someone has made a library for this tho
#
aaronpk
this is not the first time form-encoded data has been sent to a python backend
#
cweiske
pip install arrayizeIncomingParams
#
bear
yes, the python requests lib handles it - but it returns one of the sample items as: key="x-weight[properties][num]" value=70.64
#
aaronpk
right, that's not doing full the decoding
#
aaronpk
the full
#
bear
let me see how that arraizeIncomingParams does things
#
cweiske
that was a joke
#
cweiske
I have no idea about python web development
#
bear
in order to decode that I would have to run the string "x-weight[properties][num]" thru a parser
#
cweiske
is the "-" the problem in there?
#
cweiske
does the requests lib fail with that?
#
aaronpk
no it's not recognizing the "[]"
#
Zegnat
Maybe sknebel (ping) has an idea? He was working on micropub in python, IIRC
#
bear
sorry if this feels like me whining - i'm just not enjoying taking things that are not even full HTTP spec'd items and having to parse them
#
bear
rfc3986 technically doesn't even allow for []
#
aaronpk
it's not a URI tho
#
bear
yes, but the form encoded is supposed to be the same data but in the request body and not the uri
singpolyma joined the channel
#
aaronpk
which rfc did i cite anyway?
#
sknebel
Zegnat: nope. I do very basic parsing and will probably wait with extending it until I find a client I absolutly want to use that requires it
#
bear
i.e. if you can't take what is in the request body and put it in a URI then it's not valid
#
aaronpk
has anyone spec'd out the array bit for form-encoding?
#
bear
that's my point - it's not even covered by the w3c
#
cweiske
bear, what would be the python way of encoding arrays in an URL parameter?
#
bear
I really want to give a pendantic answer and say you don't
#
bear
the real world says that "foo=bar&foo=baz" will translate to foo=[bar,baz]
#
bear
and the requests lib handles that
#
aaronpk
that's in the spec i believe
#
cweiske
and dicts?
#
bear
it's dicts that are the issue
#
bear
this is a very old "battle" between js, php and ruby, python
#
aaronpk
ruby (or at least rails) supports arrays/hashes too
#
bear
I haven't looked at ruby/rails in more than a decade
#
Zegnat
According to Wikipedia, "the CGI specification contains rules for how web servers decode data of [the application/x-www-form-urlencoded] type and make it available to applications". But I can't find it in the CGI RFC
#
singpolyma
foo[]=bar&foo[]=baz is more universally supported IME. YMMZ
#
cweiske
even the '&' to separate parameters was debated
#
cweiske
java used to use ; for that
#
cweiske
php even has a config option
#
bear
I know this is not going to end well for me because of how small the python usage is in this community
#
bear
but I can at least get some push back for an official thing like micropub.rocks to not suggest that nested arrays are used for form-encoded micropub
#
singpolyma
it's not a per-language difference, but per-framework. like even sinatra and rails (both ruby) support slightly different transformations
#
bear
yea, even in the js world the various ways of doing it have caused issues
#
aaronpk
heh even the HTML JSON form note uses [] for array notation https://www.w3.org/TR/html-json-forms/
#
Loqi
[Robin Berjon] World Wide Web Consortium HTML JSON form submission
#
bear
*which* is why I'm pushing back hard on this - this spec is supposed to be language agnostic and define service to service communication
#
cweiske
does that help?
#
Zegnat
If we can't point people to a spec for parsing application/x-www-form-urlencoded, we should really wonder if you should recommend data of that type at all... So probably very right of you to push back "hard", bear!
#
aaronpk
language agnostic, yes, but also it's supposed to be easy for people, and there are few things easier for web developers than plain HTML forms
#
sknebel
cweiske: no, that doesn't interpret the value name at all
#
aaronpk
so i'm in favor of simplifying the form-encoded requests to make sure they are easy to handle in all languages/frameworks
#
singpolyma
I mean, basic form-encoded will still support the a[] and a[b] key names, it's just your application may choose to represent the data structures differently. but the basic key => value mapping is always true
#
bear
cweiske - it looks like this: >>> urlparse.parse_qs('h=entry&summary=Weighed+70.64+kg&x-weight[type]=h-measure&x-weight[properties][num]=70.64&x-weight[properties][unit]=kg')
#
bear
{'h': ['entry'], 'x-weight[type]': ['h-measure'], 'x-weight[properties][unit]': ['kg'], 'x-weight[properties][num]': ['70.64'], 'summary': ['Weighed 70.64 kg']}
#
cweiske
that's like totally useless then
#
singpolyma
bear: yeah, that looks like what I would expect from a base-level parser
#
bear
aaronpk - yep, that's why i'm suggesting that simple arrays be part of the form-encoded spec but anything else be shunted to json
#
bear
with the caveat that category[0] and photo[value] DIAF ;)
#
cweiske
that would mean 1 category = form-encoded and two categories=json
#
bear
then i'm not being clear (sorry) - I am saying that "category=foo" and "category[]=foo&category[]=bar" are ok
#
cweiske
then you must forbid category[0]
#
bear
another reason for this is because if "photo[value]" is allowed then what about "photo[german-word-for-value]"
#
cweiske
photo[wert]
#
bear
thanks :)
#
bear
the somewhat frustrating part of all of this discussion is that now, after having ranted above, I have code that handles all of these edge cases for python :/
#
bear
which I should really add to a small lib - pass in a requests object and get back a json blob
#
bear
hopes the above conversation was not overly aggressive or hostile even
#
bear
hugs aaronpk
#
bear
\o/ micropub.rocks tests 101 - 106 all pass with kaku
KevinMarks and AngeloGladding joined the channel
#
aaronpk
nope all good. bear hugs! my favorite
#
bear
i'm going to definitely split the pre-processing code out into a helper library
#
aaronpk
i'm sure you are not the only one who would appreciate that
#
bear
yea, try and get everything to map to a mf2 flavoured json blob
#
aaronpk
hm, the "content[html]" thing wouldn't be possible in the simple form-encoding version then either
#
bear
it really should be html=
#
aaronpk
but you might also have summary[html]=
#
bear
sighs
#
aaronpk
content[html] matches the mf2
#
bear
but is it in the json 'content': { 'html': '' } or 'content[html]': ''
#
aaronpk
it's {"content":{"html":"..."}}
#
bear
right, so that is nested params then
#
aaronpk
Maybe it's okay for that to not be supported in form encoding
#
bear
well, i'm thinking that for some of the more well known single level items like content[html] or summary[html] it's ok
#
bear
as we can handle that by looking for "content[html]" in python land
#
aaronpk
i guess so
#
aaronpk
doesn't that make the spec more confusing though?
#
bear
so saying that they need to be in json makes more sense (I was just trying to be a little accomodating (sp?))
#
aaronpk
yeah, tho clients that are posting HTML most likely will be doing other fancier things like editing as well, so they'll need to support the JSON post anyway
#
bear
then that sets a nice firm "bar" for the simple vs complex use cases
cweiske joined the channel
#
cweiske
bear, this also forces everyone to have a media endpoint if they want to post html content
#
bear
that feels like a good thing, no?
#
cweiske
if you say so
#
bear
hahaha
#
Loqi
bear: lol
#
bear
but i'm not following exactly why it forces that (thinking about it more...)
#
bear
my micropub endpoint can accept json or form-encoded
#
cweiske
how do you upload files?
#
cweiske
in form-encoding it's easy
#
cweiske
with json you cannot upload files directly
#
bear
oh, right - I rarely post photos
#
jkphl.is
edited /Planning (-96) "/* Berlin */"
(view diff)
#
aaronpk
I still want to support multipart photo upload
#
aaronpk
it would just only work with one photo and no alt text in the upload
#
bear
riddle me this - if it's "content[html]" then why is the json: "content": [{"html": ""}]
#
bear
then shouldn't the non html version be: "content": [{"text": ""}]
#
bear
instead of: "content": ["this is text", "more text"]
#
KevinMarks
The [] thing was a workaround for php (and Ruby maybe) not handling duplicate values right. Url?foo=1&foo=2 means foo=["1", "2"] but php makes it foo="2"
#
sknebel
*facepalm*
#
KevinMarks
So the [] thing was made up, and now we're ratifying it
#
bear
I think that i've purged all of those older memories for my sanity
#
aaronpk
To be clear, we didn't make it up
#
aaronpk
its been around for decades
#
aaronpk
bear: the content[html] thing is because content is a single value that is an object with a key html
#
KevinMarks
I did not mean to imply we made it up, but I'm not sure if it is in a w3c spec anywhere
#
bear
that would mean: "content": {"html": ""}
#
bear
goes to add in more edge cases
#
bear
yep, the requests lib hides a lot of that - you just have to check for a "plain" argument and also a "lists" variant
#
sknebel
urllib parses that fine as well, as do flask and django do the same
#
sknebel
but none seem to have any special casing for [] or [1] or anything like that
#
aaronpk
in JSON it would always be an array {"content":[{"html":"<b>foo</b>"}]}
#
aaronpk
and the non-HTML version doesn't need to specify content[text] because you can just do "content" like in mf2
pfefferle joined the channel
#
bear
swears all web devs suffer from stockholm syndrome
#
aaronpk
nah, just not a fan of prioritizing theoretical purity over practical applications ;-)
#
bear
that smells like cargo culting to me ;)
#
aaronpk
okay i'm going to go do a bunch of edits to the spec to clean this up
#
bear
k, i'll table my work until after that set of edits - I need a break from this anyways
#
bear
the scenary going by the train window is very distracting
#
KevinMarks
Partly Stockholm syndrome, partly the fundamental attribution error, partly the Baader Meinhoff problem
#
bear
hugs KevinMarks for translating his loose hand-wavy thoughts into a coherant thought
#
KevinMarks
What the other one where you buy a poodle and suddenly see all the poodles everywhere?
#
bear
I thought that was what baader meinhoff was about
#
bear
also called "blue car syndrome"
#
aaronpk
I feel like this is worth having a section in the spec that talks specifically about form-encoding
#
KevinMarks
With examples of the [] syntaxes in more detail?
#
aaronpk
and describing the specific limitations of it, like calling out that you can't have a string name inside the []
#
aaronpk
this also means i'm dropping the form-encoded version of nested mf2 objects
#
KevinMarks
Oh. So no events with hCard locations?
#
aaronpk
not in form-encoded
#
aaronpk
besides you can always post the h-card first and then location=the-h-card-url
#
KevinMarks
Hm. Having a way to make more composite posts seems like a good thing.
#
KevinMarks
If the quill editor had ways of making other things inline than media for example
#
KevinMarks
As opposed to something like the moments editor where the pieces need to exist already
tantek, KevinMarks and gRegorLove joined the channel
#
ben_thatmustbeme
aaronpk: on micropub, I didn't see anything in the spec about content[html] when posting
#
Loqi
[Aaron Parecki] Micropub
#
aaronpk
but i'm going to remove that
#
aaronpk
see: the whole discussion this morning
#
ben_thatmustbeme
in this room, i'm guessing
#
ben_thatmustbeme
i'll read that back
#
Loqi
[Aaron Parecki] Micropub
#
ben_thatmustbeme
action - indicates whether this is a delete, or undelete
#
ben_thatmustbeme
it also indicates update
#
ben_thatmustbeme
or create (when missing)
#
aaronpk
oh hey guess what
#
aaronpk
that gets removed now because it's not a reserved property in form-encoded version
#
aaronpk
oh wait that's why i changed it to just delete and undelete
#
aaronpk
yeah it's correct as is
tantek joined the channel
#
aaronpk
bear: just pushed an update!
#
aaronpk
thanks for all the feedback, i hope this makes things easier
#
aaronpk
new section explicitly describing the use of form-encoding https://micropub.net/draft/#form-encoded-and-multipart-requests
#
Loqi
[Aaron Parecki] Micropub
#
sknebel
aaronpk: just to be 100% clear, it is REQUIRED to use that encoding when sending multiple values?
#
aaronpk
Yes, 'category=foo&category=bar' is not allowed
barryf joined the channel
#
daveredfern.com
edited /Homebrew_Website_Club (+160) "/* Meetings */"
(view diff)
#
daveredfern.com
edited /Homebrew_Website_Club (+141) "/* Meetings */"
(view diff)
#
daveredfern.com
edited /Homebrew_Website_Club (+0) "/* Meetings */"
(view diff)
#
aaronpk
now to go update the micropub.rocks tests accordingly
#
GWG
Howdy
#
julian.foad.me.uk
edited /events/2016-10-05-homebrew-website-club (+87) "/* Birmingham */ Add attendees present."
(view diff)
#
KevinMarks
Not allowed is a bit strong.
#
aaronpk
not allowed by the spec
#
aaronpk
right so it's fine for micropub to say it's not supported
#
KevinMarks
Yes, it's OK to say not supported, but it's not a bug if a python implementation supports it by default
#
aaronpk
well the good news is the 1xx series of tests got way shorter
#
gRegorLove
Interesting. Haven't seen people use etherpad for HWC before
tantek joined the channel
#
tantek.com
edited /read (+1017) "/* Tantek */ re-organize example for type, and flow (starting, finishing a chapter, finishing a book, series)"
(view diff)
#
tantek.com
edited /IndieWebCamps (+2) "fix heading level"
(view diff)
KartikPrabhu joined the channel
#
tantek.com
edited /IndieWebCamps (+174) "stub a Tips section"
(view diff)
#
@ChrisAldrich
@RikMende Hope it helps. Did webmentions on your site break? I'm surprised my reply didn't show up as a comment… http://stream.boffosocko.com/2016/rikmende-hope-it-helps-did-webmentions-on-your-site-break
(twitter.com/_/status/783748912167260160)
#
vanderven.se martijn
edited /IndieWebCamps (+1562) "/* Tips */ brain dump on teaching and breaking the ice for questions"
(view diff)
#
ben_thatmustbeme
aaronpk: confused why you said the 'action' thing is correct
#
aaronpk
the section of reserved property names is about form encoded requests
#
aaronpk
updates are not supported in form encoded requests
#
aaronpk
so action only can be "delete" and "undelete" in form encoded requests
KevinMarks_ joined the channel
#
ben_thatmustbeme
and i read back that confusing discussion this morning.... sooo still confused on content[html] when creating, you said you are taking that out from posting anyway (in which case ignore test 103?)
#
ben_thatmustbeme
makes even less sense with the updates having an array for content in the JSON format since the only time you would have multiple values would be if it has some sort of label to them "content":{"html": ... , "text": ... }
#
ben_thatmustbeme
okay, i see in the update, so posting with content[html] is just not allowed at all now, and so only unlabeled content format is supported
#
ben_thatmustbeme
and assumed plaintext
#
ben_thatmustbeme
and everything is an array in the json syntax, even if its an object, because apparently we want to go back to mf2's json format
#
ben_thatmustbeme
i'm starting to get annoyed that the spec seems to assume storage as mf2
#
KartikPrabhu
"The Micropub vocabulary is derived directly from the Microformats vocabulary. Micropub is meant to be a serialization of Microformats that can be submitted as an HTTP POST. The method for developing new Micropub vocabularies is to look at the Microformats representation and work backwards. " from https://indieweb.org/micropub#Background
#
ben_thatmustbeme
yeah, i know, its not just there though
#
ben_thatmustbeme
now we are taking functionality away from form encoding, and that means we push more towards JSON. as far as I know no one has really implemented it yet
#
ben_thatmustbeme
also the SHOULD store unrecognized values rather than what most others specs do of MUST ignore
#
ben_thatmustbeme
which is not something i would recommend
#
aaronpk
what's no there?
#
KartikPrabhu
SHOULD store is good since the original intent can be preserved if someone decides to upgrade to supporting un recognised values in the future
#
aaronpk
that's the idea yeah
#
aaronpk
also you're not going to be non-compliant if you really want to not store unrecognized values. that's what "SHOULD" means.
#
KartikPrabhu
yeah it is not "MUST store" for that reason I suppose
#
ben_thatmustbeme
the idea of most specs saying MUST ignore is so that you don't have conflicts with extensions you don't know about. if you always store then extension conflicts become an actual issue, whereas we get around that by just saying, well if its not specified it doesn't matter
#
aaronpk
but potentially every property in a micropub request is part of the post, which is why the form-encoded version reserves mp-*
#
aaronpk
so preventing extensions from using arbitrary names is actually a good thing
tantek joined the channel
#
ben_thatmustbeme
i'm talking extensions to mf2
#
ben_thatmustbeme
"This allows clients to post rich types of content to servers that recognize it, while also posting fallback content to servers that don't." this isn't true, because ignoring those values has the exact same effect
#
ben_thatmustbeme
SHOULD store is nice in the ideal sense, but only ever works for specific storage types and is a burden on those that don't use them
#
ben_thatmustbeme
as an implementor using mysql that feels like a big middle finger to everything i have done thus far
#
KartikPrabhu
how is this a problem about storage types?
#
KartikPrabhu
you can always store text
#
ben_thatmustbeme
text of what?
#
ben_thatmustbeme
the entire post in mf2
#
KartikPrabhu
unknown-property = whatever
#
KartikPrabhu
as a string
#
KartikPrabhu
again as aaronpk said you don't have to
#
KartikPrabhu
you can ignore the SHOULD if you like
#
aaronpk
if i was storing stuff in mysql i would just store the whole request in a TEXT blob and then pull out the specific things i am indexing into columns
#
ben_thatmustbeme
also the argument for it being a SHOULD is what exactly?
#
KartikPrabhu
here is an example. Suppose i have a very basic site and I haven't figured out "Tags" yet
#
ben_thatmustbeme
the only case is "you haven't finished it yet"
#
KartikPrabhu
I can still post tags using micropub clients but my site does not recognise them, and I store them anyway
KevinMarks joined the channel
#
ben_thatmustbeme
SHOULDs are usually for "its best for interop that you do this"
#
KartikPrabhu
then in the future when I implement tags I can go back and tag everything because I stored them
#
KartikPrabhu
yes, I just gave you an example of "interop"
#
KartikPrabhu
my site upgrade interops nicely with older posts posted through micropub
#
ben_thatmustbeme
no, once its internal, thats not interop at all
#
KartikPrabhu
ok fine maybe I don't know the dev-speak for interop
#
KartikPrabhu
but my example is still valid as to why storing is a good idea
#
KartikPrabhu
if you don't want to store that is fine since it is a SHOULD not a MUST
#
tantek
there's a lot of misconceptions about SHOULD vs MUST
#
Loqi
tantek: ChrisAldrich left you a message 1 day, 15 hours ago: Thanks for the checklist example, I've got a personal separate list, but should put up a public facing one for AngeloGladding and others to help out
#
tantek
so much so that I'm leaning towards the split of normative language of SHOULD vs MUST being a bad idea
#
tantek
and that implementation profiles (with different overlapping sets of MUSTs) are a better approach to the "only ever works for specific" etc. cases
#
KartikPrabhu
tantek: in usual language they are used interchangably, but any spec defines them so that should be fine
#
tantek
no. "any spec" typically defines them by referring to RFC 2119, which doesn't really help clarify things
#
tantek
and there's vastly different understandings of what SHOULD vs MUST mean even in W3C insider circles
#
KartikPrabhu
i thought w3c/wharwg specs defined them but I might be wrong
#
tantek
no they define them by reference to RFC 2119
#
tantek
using boilerplate which no one reads
#
KartikPrabhu
oh yeah " The key words MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL in this specification have the meaning defined in [RFC2119]."
#
KartikPrabhu
RFC2119 does have decent definitions though
#
KartikPrabhu
which seems consistent with how I was interpreting them in micropub context
#
ben_thatmustbeme
its really a question of SHOULD vs MAY
#
ben_thatmustbeme
in my opinion
#
KartikPrabhu
MAY is not defined anywhere is it?
#
ben_thatmustbeme
when there is only 1 reason to do it, future proofing for your own self, that to me, is MAY
#
KartikPrabhu
ben_thatmustbeme: see that rfc2119 link
#
KartikPrabhu
SHOULD: "[...]mean that there may exist valid reasons in particular circumstances to ignore a particular item, but the full implications must be understood and carefully weighed before choosing a different course.
#
KartikPrabhu
MUST: "[...]mean that the definition is an absolute requirement of the specification."
#
aaronpk
ah crap just hit $10 on google appengine budget again
#
tantek
no that definition for should is nearly totally useless
#
aaronpk
5 days into the month. that's rough.
#
tantek
it's basically - hey it's a judgment call
#
tantek
aaronpk is that for indiewebify.me?
#
tantek
oh dear
#
tantek
KevinMarks: ?!?
#
aaronpk
the browser extension KevinMarks made is hammering it
#
ben_thatmustbeme
KartikPrabhu: MAY is #5 in that list
#
tantek
does that mean people are using it?
#
aaronpk
at least a few
#
tantek
and thus they're hitting sites with rel=me?
#
KartikPrabhu
ben_thatmustbeme: oh opps, sorry
#
tantek
aaronpk, wait, that means indiewebify.me is gathering a TON of data on rel=me usage right?
#
tantek
I see the beginnings of an indieweb search engine
#
tantek
might be time to make some lemonade
#
tantek
a *user* browing a *page* that has rel=me on it is a very strong quality signal
#
tantek
(of a page to index for search)
#
aaronpk
"gathering" is not quite the right word since it doesn't store anything
#
tantek
server logs
#
tantek
of requests for rel=me checking
#
tantek
that sounds like a crawl queue to me
#
tantek
I don't know, KevinMarks, do we have anyone here who could take a crawl queue as input, and build a search index as output?
#
KartikPrabhu
ben_thatmustbeme: perhaps I am now agreeing with you that it should be "MAY store unrecognised values"
#
tantek
oh dear now I'm tempted to write a dissection of 2119 that demonstrates just how tautalogical it is
#
tantek
that would be bad right? if I came up with a proof of how the very definitions of terms that all standards depend on is bad?
#
KartikPrabhu
tantek: MAYbe you SHOULD propose better definitions
#
tantek
KartikPrabhu: I prefer to "just" drop the bad ones
#
KartikPrabhu
then you MUST write that up. which ones make sense and which should be dropped
#
tantek
modest proposal: just drop SHOULD (RECOMMEND), SHOULD NOT (NOT RECOMMENDED)
#
KartikPrabhu
yeah that was my first thought too
#
KartikPrabhu
MAY is better than SHOULD
#
tantek
I would define those to #ERROR; #EXIT_SPEC
#
KevinMarks
Rfc 6919 defines MAY and a bunch of others, but it is a satire
#
sknebel
does indiewebify.me keep logs?
#
tantek
searches for "SHOULD considered harmful"
#
tantek
finds a few posts on the general english usage of "should" but nothing regarding specs. hmmm...
#
KevinMarks
The plugin fires on every twitter, github, g+ profile page
#
KartikPrabhu
that's a lot
KevinMarks_ joined the channel
#
KevinMarks_
I am probably a lot of the traffic myself
KevinMarks joined the channel
#
sknebel
KevinMarks_: you really should make it clear in the store description of your extension that it automatically sends ALL urls with rel=me tags to a "third-party" service
#
sknebel
especially if people start talking about logging that and using it for further analysis
#
sknebel
the line to what I'd label "spyware" gets damn thin
tantek joined the channel
#
KevinMarks
That's fair
#
tantek
what sknebel said
#
tantek
the more transparent about that the better
#
KevinMarks
Or switch to Zegnat's idea of doing it locally
#
tantek
also probably worth noting on the indiewebify.me pages a /disclaimer noting if / how much time server logs are kept / rotated
#
sknebel
if I remember right his local extension can't do twitter verification in chrome, but other than that works fairly well
#
tantek
KevinMarks, you've filed various "Fetch" issues to be able to do it locally in the browser right?
#
tantek
I was watching some of your chatter on #whatwg re: that
#
KevinMarks
We didn't file issues yet, but we should.
#
tantek
yes you should :D
#
tantek
since you have an actual use-case
#
tantek
notes his local IRC client claims to have sent a few lines but didn't before he got disconnected
#
tantek
is the plugin firing post DOM loaded? like if a page adds rel=me via JS, but it's not in the curlable form
#
sknebel
think so, yes
#
tantek
then the logs are particularly valuable, as a crawl queue, because the crawler could determine which pages / sites were js;dr from a rel=me perspective
#
tantek
impressive snarfed. I think you've shared those before I just forgot
#
tantek
oops wrong channel
#
tantek.com
edited /watch (+215) "document snarfed example!"
(view diff)
#
aaronpk
i really would rather the extension not using indiewebify.me frankly
#
tantek
I think that's fair
#
tantek
hmm: No results found for "SHOULD considered harmful" RFC
KevinMarks joined the channel
#
loqi.me
created /tvtag (+146) "prompted by gRegorLove and dfn added by gRegorLove"
(view diff)
#
gregorlove.com
edited /tvtag (+69) "past tense, formerly GetGlue, site-deaths"
(view diff)
#
gregorlove.com
edited /tvtag (+12) "/* See Also */ watch"
(view diff)
#
tantek.com
edited /watch (+31) "no CA https on tantek.com yet"
(view diff)
#
gRegorLove
"Mayer and Yahoo General Counsel Ron Bell did not involve the company's security team in the process, instead asking Yahoo's email engineers to write a program to siphon off messages containing the character string the spies sought and store them for remote retrieval, according to the sources."
#
gRegorLove
"The sources said the program was discovered by Yahoo's security team in May 2015, within weeks of its installation. The security team initially thought hackers had broken in."
#
gRegorLove
"When Stamos found out that Mayer had authorized the program, he resigned as chief information security officer and told his subordinates that he had been left out of a decision that hurt users' security, the sources said."
#
www.svenknebel.de
edited /CloudFlare (+252) "/* Possible issues and disadvantages */ blocks wanted "crawlers" as well"
(view diff)
#
tantek
gRegorLove: who!
#
tantek
s/who/whoa
KevinMarks joined the channel
#
gregorlove.com
edited /Yahoo (+730) "/* Issues */ Privacy blockquote, link to Reuters"
(view diff)
#
tantek
ok wow: "They were also upset that Mayer and Yahoo General Counsel Ron Bell did not involve the company's security team in the process, instead asking Yahoo's email engineers to write a program to siphon off messages containing the character string the spies sought and store them for remote retrieval"
#
gRegorLove
Argh. MediaWiki doesn't auto add line-breaks inside blockquote?
#
gRegorLove
Yeah! Insane.
#
tantek
try style="whitespace:pre-wrap"
#
gregorlove.com
edited /Yahoo (+359) "/* Privacy */ white-space: pre-wrap. expand quotation"
(view diff)
KevinMarks joined the channel
#
tantek.com
edited /events/2016-10-05-homebrew-website-club (+43) "Brighton and Göteborg cancelled this week (apparently), though Birmingham posted notes!"
(view diff)
#
tantek.com
edited /versioning (+39) "see also mf version work"
(view diff)
KevinMarks and tantek joined the channel