#dev 2016-11-02

2016-11-02 UTC
#
tantek hey so I'm filling out the Webmention implementation report and I have a silly question
#
tantek what is a loopback address?
#
Loqi It looks like we don't have a page for "loopback address" yet. Would you like to create it?
#
aaronpk loopback address is an IP address that corresponds to the same machine that is making the request, also known as localhost. Requests made to this address bypass the network hardware, so are often used when testing websites while offline. The IPv4 space reserves all 127.*.*.* IP addresses as loopback addresses.
#
Loqi ok
#
loqi.me created /loopback_address (+343) "prompted by tantek and dfn added by aaronpk" (view diff)
#
tantek ok I think I'll submit without that and then add support
#
aaronpk yeah, i did the same. telegraph doesn't take that into account yet
#
@t @Falcon Webmention sending tests using https://webmention.rocks/ URLs (all 21 of them in a single post)... http://tantek.com/2016/306/t2 (twitter.com/_/status/793609096557568000)
#
KartikPrabhu bear: have you implemented the "loopback" avoidance in ronkyuu? https://www.w3.org/TR/2016/PR-webmention-20161101/#avoid-sending-webmentions-to-localhost
#
Loqi [Aaron Parecki] Webmention
#
aaronpk according to https://github.com/w3c/webmention/blob/master/implementation-reports/kaku.md yes
#
@t @Falcon Webmention sending tests using https://webmention.rocks/ URLs (all 21 of them in a single post)... http://tantek.com/2016/306/t2 (twitter.com/_/status/793611108984643585)
#
tantek all 21 discovery tests (still) pass! implementation report submitted!
#
aaronpk woo!
#
tantek next: to fix the loopback thing
KevinMarks_, KevinMarks and tantek joined the channel
#
tantek hmm - I'm wondering how early to reject loopback
#
tantek per https://www.w3.org/TR/2016/PR-webmention-20161101/#avoid-sending-webmentions-to-localhost "During the discovery step, if the sender discovers the endpoint is localhost or a loopback IP address (127.0.0.0/8), it SHOULD NOT send the Webmention."
#
Loqi [Aaron Parecki] Webmention
#
tantek I'm wondering if the intent there was ignore that endpoint, rather than not send
#
tantek i.e. if there's a second rel=webmention, why not use that?
#
tantek I want to interpret that test as: "During the discovery step, if the sender discovers the endpoint is localhost or a loopback IP address (127.0.0.0/8), it SHOULD NOT send the Webmention to that endpoint." - note the addition of "to that endpoint"
#
tantek aaronpk ^^^
#
tantek checks to see who else is ignoring loopback
#
KartikPrabhu is anyone following the loopback suggestion?
#
tantek well the implementation report says "The sender avoids sending a Webmention to a loopback address (SHOULD)"
#
tantek which means my interpretation of the spec would be consistent with what the implementation report is expecting
#
tantek KartikPrabhu: yes, according to https://webmention.net/implementation-reports/summary/ - 4 impls
#
tantek Dobrado, GNUSocial, Grav, Kaku
#
tantek I'm thinking loopback should be rejected for *any* rel value discovery
#
tantek not just webmention
#
KartikPrabhu oh dang! I don't even know how to detect loopback URLs in python. Might steal bear's kaku code
#
tantek what? how about host is an IP address that starts with 127. ?
KevinMarks joined the channel
#
KartikPrabhu yeah I don't know how to do that! :P I am really a n00b masquerading around here
#
KartikPrabhu i just search the Web for solutions and then implement them in my code
#
tantek anyway I'm considering rejecting any 127.* IP host as a rel= href
#
tantek so e.g. for pingbacks, XFN, tags, etc.
#
KartikPrabhu do you consume rel-tags? I thought those wre deprecated for u-category?
#
tantek I don't think it ever makes sense for a site to imply rel target that is 127.
#
tantek even a rel-tag, still makes no sense
#
tantek but yea, u-category is much better for "part of a microformat object" like in an h-card, h-event, h-entry
#
KartikPrabhu yes, that does make sense to reject 127.*
#
KartikPrabhu i wonder if websites actually refer to 127.* in rels
#
KartikPrabhu statistics on that would be good
#
tantek I'm not sure I've ever seen it - but it could happen accidentally as a bug, a default slipping through
#
tantek thanks KartikPrabhu, I'll file an editorial issue
#
KartikPrabhu I'll take it even though you did most of the sense-making! :P
#
KartikPrabhu :thumbs-up:
#
tantek KartikPrabhu: always good to have someone verify your analysis
#
tantek science!
KevinMarks joined the channel
#
KartikPrabhu careful of that bubble you might fall into ;)
#
tantek KartikPrabhu: it's served me quite well so far, applying more scientific rigor to web standards development
#
tantek issue filed: https://github.com/w3c/webmention/issues/71 - feel free to comment on that KartikPrabhu
#
KartikPrabhu if I am interpreting this correctly, you mean to suggest the language so that other discovered webmention endpoints ( which are not loopbacks) can still be used
#
tantek yes
#
KartikPrabhu yeah that sounds good
#
KartikPrabhu which brings me to my next question, do sites advertise more than one webmention endpoints?
#
tantek I think some of the discovery tests do
#
KartikPrabhu interesting
#
KartikPrabhu is there an actual use-case for that, other than tests?
#
KartikPrabhu also I guess the discovery tests should reject loopback addresses
cweiske joined the channel
#
KartikPrabhu and the micropub tests too ?
KevinMarks joined the channel
#
KevinMarks Hmm. Rel tag to localhost is ok if it is a relative url
#
KevinMarks Xfn too perhaps - if you're running locally
KevinMarks_ and ChrisAldrich joined the channel
#
tantek.com edited /Falcon (+446) "expand links to open source portions" (view diff)
#
tantek alright for now I'll limit the loopback filtering to webmention and pingback discovery
#
@JavierCasares Webmention is a W3C Proposed Recommendation https://www.w3.org/blog/news/archives/5916 (twitter.com/_/status/793720382368915457)
#
bear KartikPrabhu i'll check on that later - right now i'm heading to bed after a crazy evening where I had to save someone from themselves because they didn't realize they were having a stroke
#
bear but yea, I think I implemented it but let's be sure
#
@VinAndella Webmention is a W3C Proposed Recommendation #webmaster https://www.w3.org/blog/news/archives/5916?pk_campaign=feed&amp%3Bpk_kwd=webmention-is-a-w3c-proposed-recommendation (twitter.com/_/status/793724504551727106)
chrisaldrich1 joined the channel
#
tantek and just got loopback testing working on endpoints
tantek joined the channel
#
tantek !tell KartikPrabhu here is my loopback detection code - the new function "is_loopback" at the top of https://github.com/indieweb/link-rel-parser-php/blob/master/src/IndieWeb/get_rel_webmention.php - also live on Falcon and re-checked webmention tests to make sure they still all worked.
#
Loqi Ok, I'll tell them that when I see them next
#
tantek and updated Falcon implementation report for Webmention accordingly: https://github.com/w3c/webmention/pull/72
#
@t #Webmention is a @W3C Proposed Recommendation! I filed an implementation report for @Falcon. Add yours: http://tantek.com/2016/307/t1 (twitter.com/_/status/793741901312557057)
#
@voxpelli @t Is there a deadline of some kind to get these implementation reports in for Webmention and Micropub? Want to be sure to get mine in (twitter.com/_/status/793742473067651072)
cweiske joined the channel
#
tantek voxpelli: no explicit deadline per se, however, implicitly, the sooner the better, especially for Webmention, since now 500+ W3C members are looking at it and deciding on whether to vote for it to advance to Recommendation
#
voxpelli tantek: so higher priority with Webmentions? I should give KevinMarks_ PR a look then
#
tantek yes, higher priority for Webmentions at the moment
#
tantek plus if you know any W3C members, encourage them to vote YES to advance Webmention to Recommendation
#
@niksy First time hearing about Webmentions, pingbacks on steroids for decentralized social web https://www.w3.org/TR/2016/PR-webmention-20161101/ (twitter.com/_/status/793756128823894016)
#
unicyclic.com mal edited /loopback_address (+5) "fix IP address block" (view diff)
#
@dobradocms dobrado fully supports webmentions which are now a W3C Proposed Recommendation! https://www.w3.org/blog/news/archives/5916 (twitter.com/_/status/793773463383121920)
#
sebastianlasse.de edited /2016/Berlin (+42) "/* Participating */ car from Ruhr" (view diff)
mblaney joined the channel
#
mblaney tantek I just found out my landlord is a W3C member, probably pushing the relationship to bring it up though! ;-)
#
cweiske what the
#
mblaney btw reverted my change to the loopback defn... had /8 masking wrong in my head :-P
#
cweiske no ipv6 support here?
#
mblaney good point cweiske. shouldn't be too hard to add.
#
mblaney the only trick being that you also need to check for optional square brackets in urls, because using a character that delimits port numbers makes sense.
#
mblaney oh and :: can collapse an arbitrary number of sections.
#
mblaney besides those things, shouldn't be too hard.
#
sknebel note that you should also not accept hostnames that point in DNS to loopback
#
sknebel and that there are other ways of writing IPs than separated by dots
#
sknebel using a proxy or the system firewall IMHO is the best way of enforcing it, if you run on shared hosting it's mostly your hosters job to protect themselves and other users against it
#
martymcgui.re edited /User:Martymcgui.re (+426) "/* What Works */ add example urls for various post kinds. added support for events," (view diff)
loicm_, nebulon and tantek joined the channel
#
tantek hey so what's new with loopbacks?
gRegorLove joined the channel
#
aaronpk hm.. looking at this new function, it's only catching a specific case of loopbacks https://github.com/indieweb/link-rel-parser-php/blob/master/src/IndieWeb/get_rel_webmention.php#L12
#
tantek what is a loopback?
#
Loqi It looks like we don't have a page for "loopback" yet. Would you like to create it?
#
aaronpk that will only catch cases where the URL given is literally 127.*.*.*
#
aaronpk loopback is [[loopback address]]
#
Loqi ok
#
loqi.me created /loopback (+29) "prompted by tantek and dfn added by aaronpk" (view diff)
#
tantek no it catches all URLs with any 127.*.*.* hostname
#
aaronpk not according to https://github.com/tantek/cassis/blob/master/cassis.js#L857
#
aaronpk that's just doing string parsing
#
tantek right, that pulls out the host from the URL
#
aaronpk right, so it won't catch e.g. "http://localhost/"
#
tantek what is localhost?
#
Loqi It looks like we don't have a page for "localhost" yet. Would you like to create it?_
#
tantek that's a thing?
#
tantek ok
#
tantek clicks on that localhost link
#
aaronpk or even more sneaky, any other domain that happens to resolve to 120.0.0.1
#
tantek sees his own website. uhhhh
#
@iNavegar Webmention Rocks test http://werd.io/2016/webmention-rocks-test (twitter.com/_/status/793859541205151744)
#
tantek ooh
#
aaronpk localhost is a host name that means "this computer" and resolves to 127.0.0.1. See also: [[loopback address]]
#
loqi.me created /localhost (+141) "prompted by tantek and dfn added by aaronpk" (view diff)
#
Loqi ok
#
gRegorLove Don't forget ipv6 loopback, too
#
Loqi gRegorLove: tantek left you a message 1 day, 19 hours ago: mind updating the home page indieweb.org with latest / next HWC / IWC event infos? Thanks!
#
loqi.me created /mastodon.social (+21) "prompted by tantek and dfn added by KevinMarks_" (view diff)
#
tantek what is ipv6 loopback?
#
Loqi It looks like we don't have a page for "ipv6 loopback" yet. Would you like to create it?
#
gRegorLove ipv6 loopback is [[loopback address]]
#
loqi.me created /ipv6_loopback (+29) "prompted by tantek and dfn added by gRegorLove" (view diff)
#
Loqi ok
#
tantek.com uploaded /Special:Log/upload "uploaded "[[File:2016-11-02-mastodon-down-502.png]]""
#
aaronparecki.com edited /loopback_address (+781) "expand security section, see also" (view diff)
#
tantek.com edited /Mastodon (+258) "criticism, downtime, see also" (view diff)
#
tantek.com edited /Mastodon (+302) "IndieWeb Support" (view diff)
#
gregorlove.com edited /loopback_address (+96) "IPv6" (view diff)
#
tantek gRegorLove: since when can URLs take IPv6 loopback addresses?
#
tantek can you construct clickable links for those and add them to the wiki page?
#
tantek otherwise it looks quite technically theoretical
#
aaronpk depends on whether your client resolves ipv6 addresses
#
tantek no that's not a depends, what does the URL spec say?
#
gRegorLove I don't know IP stuff very well honestly
#
aaronpk it's not part of the URL spec
#
tantek is the syntax even allowed?
#
tantek that's my point
#
bear localhost also depends on the OS you use having a valid entry in /etc/hosts
#
aaronpk here's an IPv4 loopback URL http://localhost.webmention.rocks/
#
tantek bear and are there any that don't?
#
bear yes
#
tantek aaronpk, that's a neat trick
#
aaronpk in a few minutes an ipv6 one will be live too
#
bear osx does, windows doesn't, some versions of ubuntu server side dont
#
gRegorLove Yeah, a domain, or even http://localhost redirecting to an ipv6 loopback is what I was thinking when I updated my webmention code
#
tantek bear, might be worth noting that on /localhost
#
tantek gRegorLove: does that mean you have to pre-resolve the domain before calling cURL?
#
bear and then you also get localhost for ipv6 issues
#
tantek or can you tell cURL to not do localhost IPs? or?
#
bear any good tool simply takes the netaddr part of the URL and requests it to be resolved
#
bear they shouldn't care or know that it's a "localhost" or "loopback"
#
tantek what is netaddr?
#
Loqi It looks like we don't have a page for "netaddr" yet. Would you like to create it?
#
gRegorLove tantek: My code is using PHP's dns_get_record and checking the A or AAAA records, which I guess does not catch meta redirects
#
gRegorLove After it checks for a domain of "localhost" that is
#
bear netaddr is what the domain portion of a URL is resolved into
#
loqi.me created /netaddr (+92) "prompted by tantek and dfn added by bear" (view diff)
#
Loqi ok
#
tantek bear then I'm confused, you said netaddr is requested to be resolved, then also it is what is resolved into?
#
bear sighs
#
tantek is it pre or post resolving?
#
tantek or both?
#
bear netaddr is literally the network address of the host being targeted by a URL
#
bear so that happens during DNS resolution
#
tantek IP address?
#
bear yes
#
bear (answering from the point of view of a web developer now -- don't even want to bring up other transport issues)
#
tantek what is an IP address?
#
Loqi It looks like we don't have a page for "IP address" yet. Would you like to create it?
#
tantek right, I'm answering from that point too
#
gRegorLove what is IP?
#
Loqi IP is the Internet Protocol which is one of the core protocols within the Internet protocol suite https://indieweb.org/IP
#
tantek I think "IP address" is a more used term than "netaddr"
#
bear IP address is an address used to identify a single machine or server on the network
#
loqi.me created /IP_address (+115) "prompted by tantek and dfn added by bear" (view diff)
#
Loqi ok
#
bear I use netaddr because it's referenced by a lot of libraries when parsing urls
#
bear but i'll stop doing that as it's not web dev
#
bear but you do need to seperate "network location" from "ip address"
#
tantek.com edited /netaddr (+87) "basically, IP address, blame libraries" (view diff)
#
aaronpk localhost6.webmention.rocks now resolves to ::1
#
gRegorLove What's the difference?
#
bear localhost is a network location that also happens to be commonly resoved to 127.0.0.1
#
tantek what is a network location?
#
gRegorLove Ahh
#
Loqi It looks like we don't have a page for "network location" yet. Would you like to create it?
#
bear network location is the FQDN part of a URL
#
loqi.me created /network_location (+74) "prompted by tantek and dfn added by bear" (view diff)
#
Loqi ok
#
aaronpk what is FQDN?
#
Loqi It looks like we don't have a page for "FQDN" yet. Would you like to create it?
#
gRegorLove Heh
#
bear FQDN is Fully Qualified Domain Name
#
loqi.me created /FQDN (+67) "prompted by aaronpk and dfn added by bear" (view diff)
#
Loqi ok
#
bear i'm going to have to go back thru them and fill out some details when i'm not work distracted
#
tantek wish when we tunneled like that that Loqi would know enough to go back and wikilink the previous dfn use of the next jargon term
#
gRegorLove Is three a Partially Qualified Domain Name? :)
#
gRegorLove s/three/there/
#
bear nope
#
bear well, if i'm being snarky I would say yes, it's what .io domains get resolved into as their domain registrar is flakey
#
bear but that's a ops/dns joke
#
tantek wow
#
tantek getting back to defensive code
#
tantek gRegorLove: are you calling the dns resolve call on every webmention URL host before you send your webmention
#
bear for the loopback test - I'm wondering if a simple list of domains to avoid would work
#
tantek not if aaronpk is going to keep creating new ones
#
aaronpk :)
#
tantek bear, i.e. see my attempt here: https://github.com/indieweb/link-rel-parser-php/blob/master/src/IndieWeb/get_rel_webmention.php#L12
#
bear reads
#
gRegorLove tantek: Yes, on each webmention endpoint
#
bear yea, that code would fail on quite a few loopback/localhost domains
#
gRegorLove I don't think I'm checking for loopback on received webmentions yet
#
bear localhost is a convention - the only way to know is to get the ip address of it
#
bear loopback ip addresses are a well defined set
#
gRegorLove I use: filter_var($ip, FILTER_VALIDATE_IP, FILTER_FLAG_NO_RES_RANGE)
#
tantek so ... that ought to be added to /loopback
#
tantek aaronpk, including your couple of test URLs
#
bear most domain-ipaddress libraries have a way of telling if an IP is private
#
bear which is what a loopback address is
#
tantek hmm the localhost6 subdomain didn't resolve for me
#
aaronpk you might not be on an ipv6 network
#
tantek but my server might?
#
aaronpk yeah
#
aaronpk curl http://localhost6.webmention.rocks/ from my server totally tries to contact itself
#
bear checking if the returned IP address is one of the 127.* or 169.* is the safest (or ::1 for ipv6)
#
tantek interesting, neither my local machine nor my server can resolve that domain
#
tantek unknown host localhost6.webmention.rocks
#
aaronpk on the command line, try `dig aaaa localhost6.webmention.rocks`
#
bear ;; ANSWER SECTION:
#
bear localhost6.webmention.rocks. 7199 IN AAAA ::1
#
tantek what is dig?
#
Loqi It looks like we don't have a page for "dig" yet. Would you like to create it?
#
tantek ok I see that ANSWER SECTION too
#
bear dig is a command line tool used to query DNS servers for information about a domain's Zone definition
#
loqi.me created /dig (+133) "prompted by tantek and dfn added by bear" (view diff)
#
Loqi ok
#
gRegorLove bear is racking up the new wiki pages today
#
tantek is DNS server like ATM machine?
#
gRegorLove Heh
#
aaronpk DNS is Domain Name System
#
tantek what is DNS?
#
Loqi domain name server is an abbreviation for Domain Name Server and often used to refer to the configuring thereof https://indieweb.org/DNS
#
tantek huh
#
aaronparecki.com edited /loopback_address (+138) (view diff)
KevinMarks joined the channel
#
tantek gRegorLove: mind starting a new "How To" section on /loopback that documents your techniques for detection / avoidance?
#
bear gRegorLove - yea, I always get a nice spike in metrics when networky things are worked on
#
gRegorLove Loqi needs a "remind me to" feature so I can queue up stuff like that for later. A la Slack
#
bear I thought loqi had a timer?
#
KartikPrabhu i think it does
#
Loqi KartikPrabhu: tantek left you a message 9 hours, 5 minutes ago: here is my loopback detection code - the new function "is_loopback" at the top of https://github.com/indieweb/link-rel-parser-php/blob/master/src/IndieWeb/get_rel_webmention.php - also live on Falcon and re-checked webmention tests to make sure they still all worked.
#
gRegorLove Maybe not "needs" but yeah
#
aaronpk Loq's timer predates Slack by... about 7 years
#
tantek gRegorLove: it's called
#
tantek who is gRegorLove
#
Loqi gRegor Morrill photo/avatar for gRegor Morrill gRegor Morrill is a software developer living in Bellingham, Washington. Pronouns: he/him/his http://gregorlove.com https://indiewebcamp.com/User:Gregorlove.com
#
gRegorLove Sure, now how do I get Loqi to add stuff to my list? :)
#
aaronpk 2 minutes until gRegorLove don't forget to do the thing
#
Loqi I added a countdown scheduled for 2016-11-02 5:43pm GMT+0000 (#5930)
#
bear 8 hours until tell me to review/edit new wiki links
#
Loqi I added a countdown scheduled for 2016-11-02 9:41pm EDT (#5931)
#
tantek User:Gregorlove.com << mind starting a new "How To" section on [[loopback]] that documents your techniques (2016-11-02 in irc) for detection / avoidance?
#
loqi.me edited /User:Gregorlove.com (+150) "/* See Also */ new section" (view diff)
#
Loqi ok, I added "mind starting a new "How To" section on [[loopback]] that documents your techniques (2016-11-02 in irc) for detection / avoidance?" to the "See Also" section of /User:Gregorlove.com
#
gRegorLove Wat
#
gRegorLove Haha
#
aaronpk notice that bear's timer is in his local timezone :)
#
Loqi hahaha
#
bear I did notice that
#
bear aaronpk++ on human centric bot design
#
Loqi aaronpk has 13 karma in this channel (1128 overall)
#
aaronpk thanks /irc-people!
#
tantek waits for !todo kind of like !tell but adds to people's "To Do" section like << does to a "See Also"
#
Loqi There was an error: Array
#
tantek ORLY
#
Loqi gRegorLove don't forget to do the thing
#
tantek gives Loqi an Array
#
Loqi investigates the Array
#
gRegorLove Thanks Loqi
#
Loqi you're welcome
#
gRegorLove If You Give a Mouse an Array
#
tantek hey at least Loqi is not creating / adding to an "Inbox" section on your user page ;)
#
ben_thatmustbeme what triggered that error message from Loqi?
#
aaronpk bad regex on <<
#
ben_thatmustbeme ohh
#
ben_thatmustbeme didn't know << was a thing Loqi parsed
#
Loqi There was an error: Array
#
ben_thatmustbeme what is the usage of that?
#
loqi.me created /account_deletion (+326) "prompted by tantek and dfn added by tantek" (view diff)
#
aaronpk see above
#
bear isn't that the "add a see also link to wiki page" syntax
#
ben_thatmustbeme scrolls up
#
bear lunchifies himself
#
ben_thatmustbeme woah
#
ben_thatmustbeme User:Ben.thatmustbe.me << todo: create a pubsubhub hub
#
loqi.me edited /User:Ben.thatmustbe.me (+48) "/* See Also */ new section" (view diff)
#
Loqi ok, I added "todo: create a pubsubhub hub" to the "See Also" section of /User:Ben.thatmustbe.me
#
ben_thatmustbeme cool
#
ben_thatmustbeme Loqi++
#
Loqi loqi has 1 karma in this channel (409 overall)
#
ben_thatmustbeme aaronpk++
#
Loqi aaronpk has 14 karma in this channel (1129 overall)
#
tantek gRegorLove++ for asking for it (the feature that is, “a "remind me to" feature so I can queue up stuff like that for later. A la Slack”)
#
Loqi gregorlove has 7 karma in this channel (87 overall)
KevinMarks joined the channel
#
loqi.me created /pdf;dr (+92) "prompted by tantek and dfn added by tantek" (view diff)
#
loqi.me edited /pdf;dr (+80) "/* See Also */ new section" (view diff)
#
loqi.me edited /pdf;dr (+62) "tantek added "https://twitter.com/stevefaulkner/status/793399153703419904" to "See Also"" (view diff)
#
aaronpk hm i seem to have lost my notes on setting up my screenshot->micropub workflow
#
@WeeklyMeta https://w3.org/TR/2016/PR-webmention-20161101 Webmention: A W3C Proposed Recommendation via FrontEnd Focus (twitter.com/_/status/793879440824606721)
#
aaronpk why didn't i make a blog post about this
#
loqi.me edited /js;dr (+102) "tantek added "https://twitter.com/jkphl/status/792452368562618369 https://pbs.twimg.com/media/Cv9bNjYW8AAHOac.jpg" to "See Also"" (view diff)
chrisaldrich_ joined the channel
#
tantek aaronpk, check your queue of stuff to write posts about!
#
www.boffosocko.com edited /site-deaths (+629) "/* Upcoming */ TiddlySpace on 2016-12-15" (view diff)
#
sknebel bear, do you see any reason to handle localhost in your Webmention code vs in the system firewall (for systems where you controll that)?
#
tantek sknebel, defense in depth
#
sknebel hacking DNS resolution in requests is a bit annoying to do
#
tantek.com edited /js;dr (+159) "/* See Also */ make the image fit, add alt" (view diff)
#
aaronpk the loopback address doesn't go over the network though, so the firewall doesn't apply
#
sknebel aaronpk: on Linux it does
#
sknebel it's an interface like all others in that regard
#
bear i'm on the fence about this
#
tantek ooh this should be interesting
#
bear if the person (or network person or whoever) has defined a domain to point to localhost then we should honour it
#
bear but
#
bear in the "real world" localhost is invalid IMO for received webmentions
#
tantek we should not honor obvious mistakes are (perhaps unintentional) attempts to access local (webmention sender) resources by the external (webmention receiver) host
#
tantek s/are/or
#
bear right - your saying what I was thinking in a better way
#
Loqi I agree
#
bear things coming into my site need to be checked to make sure the source url is safe and also to clean the target url
#
bear as an attacker could make a target url contain a malformed url
#
tantek feel free to copy liberally to improve the /loopback article
#
tantek right
#
bear I will, gathering up info and chewing it over in my brain now
#
tantek thanks!
#
bear I think I will make it very secure by default and add a debug flag to allow dev tests to use local host
#
bear (i'm realizing that my python libs are now viewed by more folks and used as a source of patterns so I have to be very purposeful in the changes I make to them)
#
sknebel I've firewalled off private RFC 1918 IP space too, but that's because I know I don't have services that might want to use webmention there
#
sknebel published software might be used on an intranet where that isn't the case
#
tantek bear, and that's a good explanation for why I don't open source the rest of Falcon
#
bear yea, I think the pattern should be block everything and only if needed allow a whitelist of ips
#
bear sknebel - thats prudent and sane
#
bear ipv6 means we also have to look into RFC 4193 now
#
aaronpk oh gosh, more JSON alternatives https://tonyarcieri.com/introducing-tjson-a-stricter-typed-form-of-json
#
tantek I'm tempted to start filtering even harsher
#
tantek e.g. if a webmention endpoint is on a different domain than where I discovered it, consider not sending unless it's on a whitelist (bridgy, webmention.io, webmention.heroku etc.)
#
aaronpk hm it sounds like what you actually want is for the endpoint to confirm it is an endpoint and that it handles webmentions for a given domain
#
tantek that's too late
#
tantek e.g. the loopback problem
#
tantek you can'
#
tantek you can't ask the endpoint anything if it's already a malformed URL
#
aaronpk a malformed URL is a different issue
#
tantek use of loopback is a malformation IMO
#
aaronpk you're going to have to check if it's a malformed URL, and if you don't, your HTTP client will fail out anyway
#
aaronpk no malformed means something else
#
KartikPrabhu tjson! <sigh> so many formats
barryf joined the channel
#
bear loopback checking, IMO, is adding a third layer of checks to the domain matching and vouch checks already in place
#
bear *after* you have a good domain, then resolve it to find out if it's a loopback
#
barryf Hello all. I'm almost done with my Micropub.rocks tests but I've hit 804: rejecting an unauthorized access token. I need to generate a token but don't know of a quick way to do so. Does anyone have a live tool I could use to log in and generate one?
#
aaronpk hi barryf
#
aaronpk where do your tokens come from right now? are you using tokens.indieauth.com or your own server?
#
barryf Hey aaronpk. I'm using tokens.indieauth.com
#
aaronpk I can't think of a quick way to do that, but you need to generate a token that doesn't have "create" scope. You could log in to Quill and change the scope that it's requesting for example
#
barryf I thought about hacking together something for that purpose. Sounds like it might be a useful tool. When you say I could change the scope via Quill, is there a way of configuring the scope it requests?
#
aaronpk just fiddling with the URL
#
barryf Ah, got it. I'll give that a try.
#
aaronpk this is one of the reasons I like having my own authorization endpoint. I gave myself checkboxes to change the scope that's granted :) https://media.aaronpk.com/Screen-Shot-2016-11-02-12-05-30.png
#
aaronpk but yeah you should be able to change the URL that quill redirects you to and adjust the scope value that's in the query string
KevinMarks joined the channel
#
barryf Nice! Creating my own endpoint is on my list. Need to finish off my new server software first. Nearly there.
#
tantek.com edited /events/2016-11-02-homebrew-website-club (+597) "/* What */ a bunch of topics for today" (view diff)
#
@LikeTheMammal Ooh, the Webmention API looks sweet. "..notify any URL when you mention it on your site.". https://www.w3.org/TR/2016/PR-webmention-20161101/ (twitter.com/_/status/793905536425074689)
ChrisAldrich joined the channel
#
aaronparecki.com edited /screenshot (+153) (view diff)
chrisaldrich1 joined the channel
#
calumryan.com edited /events/2016-11-16-homebrew-website-club (+408) "/* Where */" (view diff)
barryf joined the channel
#
calumryan.com edited /events/2016-11-16-homebrew-website-club (+206) "/* RSVP */" (view diff)
KevinMarks, tantek, KevinMarks_ and gRegorLove joined the channel
#
tantek hey aaronpk you noted a while ago that cassis didn't autolink .rocks - it's had that since 7 months ago! https://github.com/tantek/cassis/commit/8ca2edbd01cb8c9228741bd0064ca708ffb0398b
#
aaronpk orly
#
aaronpk ah there haven't been any releases since then
#
aaronpk https://github.com/tantek/cassis/releases
#
tantek ahhh ok
#
tantek I keep forgetting to do those
#
aaronpk so i am running an old version
#
gregorlove.com edited /Main_Page (+203) "/* Homebrew Website Club */ +11/16 hwc" (view diff)
#
tantek alright let me sync a few more changes with what I'm selfdogfooding and I can cut a release
#
tantek what's live on my server has been stable for a while, and no API breaking changes
#
aaronpk nobreakingchanges++
#
Loqi nobreakingchanges has 1 karma
#
@yuichigrut Daily Pickup : Webmention is a W3C Proposed Recommendation https://www.w3.org/blog/news/archives/5916?pk_campaign=feed&pk_kwd=webmention-is-a-w3c-proposed-recommendation - via: W3C News https://www.w3.org/blog/news (twitter.com/_/status/793958411616526336)
#
gregorlove.com edited /loopback_address (+929) "How To with some PHP sample code" (view diff)
#
gRegorLove tantek: Added https://indieweb.org/loopback_address#How_To
#
aaronpk gRegorLove: get_dns_record isn't a built-in function, what is that?
#
gRegorLove Typo :) dns_get_record()
#
aaronpk oho
#
gregorlove.com edited /loopback_address (+0) "/* How To */ typo" (view diff)
#
sknebel gRegorLove: you should mention that the resolved IP should be used for all communication -> if you do this check, but then give the full domain to e.g. curl it will redo the resolve process and could get a different answer
#
aaronpk also what's the return value of filter_var?
#
aaronpk true if reserved? true if not reserved?
#
sknebel (really should be in the general description, but I can't come up with a nice way of explaining it right now... really should go to sleep and try tomorrow ;))
#
gregorlove.com edited /loopback_address (+85) "/* How To */ clarify filter_var return value" (view diff)
#
gRegorLove filter_var will return the IP address if it's valid and not in the reserved range, otherwise it returns boolean false.
#
aaronpk aha
#
aaronpk also what happens if the domain is a CNAME to something?
#
aaronpk (I made localcname.webmention.rocks to test but it won't be active for another 15 minutes or so)
#
gRegorLove Ok, no more questions. :) jk
#
aaronpk localcname.webmention.rocks -> CNAME to localhost.webmention.rocks -> A 127.0.0.1
#
gRegorLove I don't think it will catch CNAME. Guess it depends if dns_get_record follows the chain
#
tantek I smell a new test suite
#
bear yea, just like url redirects - you have to follow the whole chain
#
sknebel I looked into teaching python's requests library to run such a check for each resolving it does... have to check if I can find the code for that again
#
sknebel (so it also applies to HTTP 301s etc)
#
gRegorLove sknebel: Interestin re: using the IP. Is the idea an attacker could server a legit IP and then quickly change it to loopback?
#
bear yea, even the python libraries depend on the system's dns resolution for that - it becomes hard to check things
#
aaronpk gRegorLove: that's the basis of the attack i linked
#
aaronpk http://bouk.co/blog/hacking-developers/
#
gRegorLove Ahh, right
#
sknebel bear: in requests there are some hooks you can plug into, but in the end that's among the reasons why I decided to go with just firewalling the process of
#
sknebel *off
#
bear nods
#
aaronpk yeah it seems like this needs to be part of HTTP client libraries
#
bear well, it's a matter of scale
#
aaronpk i mean the other alternative is make sure you don't have any services listening on 127.0.0.1
#
bear ^^ that is the only true solution IMO
#
aaronpk s/services/unauthenticated services/
#
bear it's like an SSL MITM attack
#
bear sure it's possible, but how likely is it for 99% of what your doing
#
tantek or just have a logging service, for weakness/attack detection
#
gRegorLove Hah, from PHP docs "Because of eccentricities in the performance of libresolv between platforms, DNS_ANY will not always return every record, the slower DNS_ALL will collect all records more reliably."
#
bear I think it could be done for webmention and micropub using a "sanity check" helper
#
bear and the sanity checker would need to have a paranoia flag - how insane do you want it to check