#dev 2017-11-22

2017-11-22 UTC
snarfed and [miklb] joined the channel
#
aaronparecki.com
edited /hwcpdx (-37) "december hwcpdx is cancelled"
(view diff)
#
aaronparecki.com
deleted /events/2017-12-06-homebrew-website-club "deleting because it was the off-week for PDX and PDX is cancelled"
[jackysee], snarfed, tantek and renem joined the channel
#
www.funwhilelost.com
edited /exercise (+286) "/* POSSE */"
(view diff)
jjuran, KartikPrabhu, snarfed and [jeremycherfas] joined the channel
#
loqi.me
edited /Indieweb_for_Journalism (+123) "KartikPrabhu added "https://www.poynter.org/news/after-watching-gothamist-and-dnainfo-disappear-journalist-built-tool-help-others-save-their" to "See Also""
(view diff)
#
loqi.me
edited /PASTA (+43) "sgreger added "https://github.com/pastpages/savemy.news" to "See Also""
(view diff)
#
ancarda
Well, my work PC started up this morning without an IPv4 address. Only Google Apps is working >:(
[kevinmarks] and loicm joined the channel
#
petermolnar
you mean you only have an ipv6 address?
#
petermolnar
interesting
#
ancarda
This has happened before and I've just rebooted, but today it's actually not picking up an IPv4 address - not sure how to debug this
#
ancarda
Ubuntu 17.10. This happened the other day too, although a reboot fixed it. In the process of trying to debug what was going on, I disabled IPv4 in Network Manager and that only took affect this morning when I booted up. Took some time to figure out it was just me being an idiot
#
ancarda
I'm not sure why it broke the other day though - but now it's not exclusively using IPv4 DNS servers so it can reach Gmail without any IPv4 support now, which is nice
[mrkrndvs] joined the channel
#
Zegnat
[mrkrndvs], what do you consider “really poor permalinks”?
[jgmac1106], loicm, snarfed, eli_oat, sdfsdfsdf, [markmhendrickso, amz3, [miklb] and tantek joined the channel
#
vanderven.se martijn
edited /authorship (+970) "/* Other theoretical */ When mapping h-entry with h-cards elsewhere in the DOM, it makes sense to do so before grabbing external resources. Discussion prompted by keithjgrant"
(view diff)
#
Zegnat
Hopefully that summarises yesterday’s #microformats talk ^^^
snarfed joined the channel
#
sknebel
yes, think it does
#
aaronparecki.com
edited /Microsub-spec (-216) "remove inline note about paging"
(view diff)
eli_oat and jjuran joined the channel
#
aaronparecki.com
edited /Microsub-spec () "(-586) remove config query from channels section"
(view diff)
#
aaronparecki.com
edited /Microsub-spec (+1074) "/* Actions */ create a channel, and describe response for listing channels"
(view diff)
[cleverdevil] joined the channel
#
vanderven.se martijn
edited /Micropub-brainstorming (+1146) "/* Micropub for Multi-user Sites */ Thinking about this. Micropub dictates a token-endpoint-per-micropub-endpoint instead of the assumed token-endpoint-per-user-identity."
(view diff)
#
Zegnat
Thoughts super welcome ^^^
#
aaronpk
I think I've already done this
#
Zegnat
I couldn’t find any thoughts on it :/ Sorry if I copied some prior discussion
#
aaronpk
I don't think I documented it, but i've already implemented it a couple times
#
Zegnat
Ah, would be great if you can find the time to put that down into writing somewhere then :D
#
Zegnat
My 2018-01-01 idea is to do all posting to my blog over Micropub, and because I will be logging in using vanderven.se/martijn/ I have been reviewing some of the micropub & identity decoupling discussions
#
tantek
that sounds like a commitment :)
#
tantek
what is 2018-01-01-commitments?
#
Loqi
2018-01-01-commitments are implementation and launch commitments publicly made by the IndieWeb community to ship on their personal sites by 2018-01-01 00:00 local time https://indieweb.org/2018-01-01-commitments
#
Zegnat
Yes, let me add that
#
tantek
yes! please do! we only have *ONE* now, from [chrisaldrich]!
#
aaronparecki.com
edited /Micropub-brainstorming (+1179) "/* Micropub for Multi-user Sites */"
(view diff)
#
aaronpk
Zegnat: ^^
#
loqi.me
created /recommendations (+34) "prompted by tantek and dfn added by tantek"
(view diff)
#
Zegnat
tantek, added. Please hold it against me if I do not get it done this year. I apparently had the same commitment last year, haha
#
Loqi
ahahaha
#
tantek
oh boy
#
tantek
Zegnat++ for transparently noting his renewal of his commitment :)
#
Loqi
zegnat has 37 karma in this channel (151 overall)
#
sknebel
Zegnat: yeah, it kind of feels like you'd have to have "profiles" in the community blog and you log in as that profile (which then could do authentication by in turn running indieauth against your main identity), not as the main user
#
Zegnat
aaronpk, yeah I guess that works. It just seems weird to me that we then need a double IndieAuth protocol dance. I also as a user then need to go through the communal blog to revoke a token rather than being able to do it myself.
#
Zegnat
sknebel puts the “double” better then me :)
#
aaronpk
Zegnat: as someone who maintains that community blog, they would want control of the security of their blog
#
sknebel
I guess you could have the micropub endpoint go look up your token endpoint and use that to verify the token, as long as it whitelists the allowed identities
#
sknebel
(and scopes)
#
aaronpk
no I dont think that puts the trust in the right place
#
sknebel
it's ugly, yes. depends on how much you trust the contributors etc
#
aaronpk
imagine the perspective of the person maintaining the community blog
#
aaronpk
I don't want to rely on everyone's individual token endpoints not being compromised, I want to control that myself
#
sknebel
that'd be nice, yes, and you don't get it this way
#
aaronpk
the way I described it you do
#
aaronpk
(not just described, but i've built that a couple times)
#
aaronpk
also nice that there's no need for protocol changes to support it
#
Zegnat
But you already rely on their authorization endpoints not being compromised.
#
aaronpk
only during the login
#
aaronpk
not for every request
#
Zegnat
Whenever a user switches client they do a new login. Login may as well happen for every request, not really a difference
#
aaronpk
there's a huge difference
#
aaronpk
micropub requests happen out of the blue, with no advance notice. I don't want to a) be reliant on hitting an external API to validate every request, and b) I want to be able to revoke tokens I issue
#
tantek
jkphl++
#
Loqi
jkphl has 2 karma in this channel (38 overall)
#
Zegnat
Oh I get that less external calls are good for implementors, aaronpk. I'm only saying that (unless you police when logins are allowed) fear for the token endpoint being compromised is not a reason as the same applies to authorization endpoints.
#
aaronpk
but it *is* because of the relative volume of expected requests to the micropub endpoint vs the authorization endpoint
#
aaronparecki.com
edited /Feedly (+15) "feedly apparently can subscribe to json feeds"
(view diff)
#
tantek
what about jf2 feeds?
#
aaronpk
tries to find one to test
#
aaronpk
404 not found :P
#
aaronpk
er, 404 fragment not found
j12t joined the channel
#
tantek
what is 309?
#
Loqi
HTTP 309 is a redirect to call someone and must only be returned when the path /8675 is requested https://indieweb.org/309
#
tantek
so we have 309 examples but no jf2 examples?!?
#
tantek
cries a little
#
aaronpk
I suspect that's because jf2 feed wasn't specified enough to be obvious how to implement it
#
tantek
like 309 was?
#
aaronpk
I mean it says it right in the dfn :)
#
aaronpk
goes on a survey of subscribing interfaces
#
aaronpk
interesting
#
tantek
what is subscribe
#
Loqi
follow is a common feature (and often UI button) in silo UIs (like Twitter) that adds updates from that profile (typically a person) to the stream shown in an integrated reader, and sometimes creates a follow post either in the follower's stream ("… followed …" or "… is following …") thus visible to their followers, and/or in the notifications of the user being followed ("… followed you") https://indieweb.org/subscribe
bengo joined the channel
#
tantek
definitely worth capturing screenshots of various subscribing interfaces ^^^
#
aaronpk
what's between indieweb examples and silo examples?
#
aaronpk
e.g. all the feed reader apps on https://indieweb.org/feed_reader
#
tantek
I think we have "Services" in some places? or "Tools" maybe?
#
tantek
like "Service Examples"
#
tantek
can't remember where tho
#
aaronpk
feedhq has the same "channel" concept, except they call it "Category"
#
aaronparecki.com
edited /follow (+408) "FeedHQ"
(view diff)
#
aaronparecki.com
edited /follow (+611) "feedly"
(view diff)
#
aaronparecki.com
edited /follow (+770) "feedly details"
(view diff)
#
aaronpk
feedly seems to be covering all the bases I want to cover in a reader
#
aaronparecki.com
edited /follow (+126) "/* Feedly */"
(view diff)
#
snarfed
feedly++
#
Loqi
feedly has 1 karma
#
snarfed
newsblur++
#
Loqi
newsblur has 1 karma
#
aaronparecki.com
edited /follow (+152) "/* Feedly */ follow button"
(view diff)
#
vanderven.se martijn
edited /Micropub-brainstorming (+468) "/* Users using their own identities to log-in to a community blog. */ Mental recalibration required, but makes sense."
(view diff)
#
aaronparecki.com
edited /Microsub-spec (+2531) "/* Actions */ add search action, inspired by the Feedly UI"
(view diff)
[cleverdevil] joined the channel
#
[cleverdevil]
I like to see this activity on the microsub spec ?
#
aaronpk
i'm making these updates as i'm actually building it out too
#
tantek
good methodology
#
aaronpk
first draft was brainstorming, now time to implement
#
[cleverdevil]
Looking forward to seeing it progress. We're a bit stalled out on Together until there's a decent Microsub implementation.
#
aaronpk
cool. with any luck i'll have a prototype of this for you to play with at the end of the week
#
aaronpk
i'm hoping it's minimally functional enough to continue progress on Together
#
[cleverdevil]
Would be nice!
#
aaronpk
I already have it running and collecting things from a few feeds
#
Loqi
[grantcodes] #11 Random Backend Thoughts
[kevinmarks] joined the channel
#
aaronpk
commented :)
#
[cleverdevil]
Oh, very nice.
#
aaronpk
struggling with how to handle uniquely identifying items in microsub
#
aaronpk
anyone want to brainstorm with me?
#
[cleverdevil]
What are your main concerns?
#
aaronpk
when the server is fetching external posts, there are a few situations it might encounter: a uid property, a url property, both, or neither.
#
aaronpk
right now I prioritize uid, then use url if uid is not present, and if neither is present, I hash the content and use that. that's what I use as the unique identifier internally in order to know whether to update an old post or add a new post
#
aaronpk
(both uids and urls are scoped to the feed as well, so two different feeds could both use the uid "1234" and they wouldn't conflict)
#
[cleverdevil]
That seems sane, apart from the case where the content changes for an item with neither a URL or UID property.
#
[cleverdevil]
But, I think that has to be pretty rare.
#
tantek
there are items without URLs?
#
aaronpk
yeah that's an edge case and there isn't really a good way to know whether it's the same post or a new post anyway
#
aaronpk
I can't assume there will always be a url or uid when i'm handling external content
#
tantek
or are you trying to handle arbitrary h-entry items without any properties potentially?
#
[cleverdevil]
You could also consider synthesizing a UID in the case of none being present, either from the URL if present, or the content itself.
#
aaronpk
yeah that's what I do, my current implementation is a hash of the content
#
aaronpk
so this all works fine for my internal storage for deduplication
#
aaronpk
the question is now what do I expose to the microsub client
#
[cleverdevil]
I sort of view the microsub server as a way to simplify as much as possible about building out a reader.
#
[cleverdevil]
So, I wouldn't mind just exposing that directly through.
#
aaronpk
here's an example: say the microsub client wants to hide/block a specific post, it needs a way to refer to that post when talking to the microsub server
#
tantek
besides URL?
#
aaronpk
there may not be a url
#
tantek
er, permalink?
#
[cleverdevil]
Right, it should just use the synthesized UID.
#
tantek
I feel like there are some use-cases / examples you have in mind which are not necessarily obvious
#
[cleverdevil]
(Or the actual UID)
#
tantek
aaronpk, is this for an h-entry without a u-url or u-uid? or something else
#
aaronpk
any post, not necessarily h-entry
#
tantek
I'm looking for specific examples that made you want to generalize to solve that
#
aaronpk
since there's nothing about microsub that limits the server to subscribing to just h-entry
#
aaronpk
I can subscribe to an Atom or RSS feed as well
#
tantek
and those items might not have URLs? at least in RSS?
#
tantek
or have you seen actual RSS files with items without URLs?
[kevinmarks] joined the channel
#
aaronpk
let's talk about the uid vs url case first
#
[kevinmarks]
Can you use a hash url, then everything is still a url
#
aaronpk
in general, if there is a uid, it should be prioritized over url. there are lots of examples of feeds where the URL scheme changes but the uids stay the same. e.g. wordpress uses example.com/?id=400 as a uid but then you can change the URL scheme to /yyyy/mm/slug etc
snarfed joined the channel
#
aaronpk
if there is no uid, then the url is an acceptable identifier for the post
#
tantek
what is GUID
#
Loqi
A GUID or Globally Unique Identifier is a (typically) 128-bit number to uniquely reference or identify something; on the IndieWeb, GUIDs are encouraged in some feed file formats as supposedly more persistent than permalinks but in practice GUIDs have been less portable and harder to preserve/migrate/restore than permalinks https://indieweb.org/GUID
#
aaronpk
I would like this complexity to be removed for microsub clients
#
aaronpk
[kevinmarks]: i'm trying to make this easier not harder
#
tantek
aaronpk, I have wondered about that "e.g. wordpress uses" in practice
#
[cleverdevil]
Clients see JF2, right?
#
aaronpk
so basically I want there to be only *one* way that items are identified between microsub clients and servers
#
[kevinmarks]
Which is canonical with WordPress?
#
tantek
I haven't seen any actual data that shows any use of UID/GUID being more persistent than permalinks
#
tantek
whereas the opposite is documented at /GUID
#
tantek
I think it's that permanence/uniqueness of UID/GUID for posts is one of those aspirational things in feed specs, more than actually useful as intended
#
[kevinmarks]
A hash works, but only if the other site urls are relative
snarfed joined the channel
#
tantek
aaronpk, I'd suggest just "URL" for the " *one* way that items are identified between microsub clients and servers"
#
tantek
and then if you need different kinds of URLs
#
tantek
e.g. the hash URLs kevinmarks mentions, you can make that decision later
#
aaronpk
so if that's the case, then what do I use when the post doesn't have a URL
#
tantek
rather than trying abstract some identifier type that includes URL *and other stuff*
#
tantek
aaronpk you make up a fragmention of where ever you saw the post, and its contents
#
tantek
fragmentions are still URLs
#
aaronpk
so I make one up?
#
tantek
if the item has no URL property, you synthesize one
#
aaronpk
okay, next question
#
tantek
nice thing about fragmention URLs is that they already have a method of resolution defined, and work well in terms of being user browseable
#
aaronpk
say i'm following example.com and it reports a post with a url like example.org/post/100. For the purposes of storing and deduplicating that, it's fine for me to use that URL within the scope of the example.com feed. However, when I report that to the client, it's no longer within the scope of example.com so there's a security risk that the client may allow that post to be overwritten by some other
#
aaronpk
domain
#
tantek
I think only example.org can canonically claim to provide a url like example.org/post/100. if you want to enable example.com to provide an example.org/whatever post, then you start down the path of signatures
#
aaronpk
so this should be captured somewhere, like in h-entry or h-feed, or at least when talking about following feeds
#
tantek
the post would carry a signature with it that it came from example.org, so that example.com or other domain could pass it along
#
aaronpk
the obvious question being what should a consumer do that encounters a post with a URL from a domain different from where the content came from?
#
tantek
I believe that's the reasoning for using signatures in federation, so that a post can be passed along any number of servers (fatping) without necessarily having to contact the original domain
#
tantek
like if aaronpk.com decided to offer a feed of select items from tantek.com?
#
aaronpk
yeah signatures and a key management scheme solve that use case. it's more immediately applicable in the indieweb context for things like in-reply-to and repost-of where the site may be showing expanded content from another domain already
#
tantek
currently consuming code would have to go retrieve the permalink to make sure the content is actually at the claimed location
#
tantek
and that's how we avoid signatures in the common case
#
[cleverdevil]
I don't think that's too large a burden, either.
#
aaronpk
fetching the post?
#
aaronpk
I agree
#
tantek
also I'd agree that's the 99.9% indieweb use-case
#
tantek
and helps build on the primacy of URLs as identifiers for stuff on the web
#
aaronpk
so I need to code this into my feed processing function. if I encounter a URL property from a different domain, I should straight out reject the post?
#
[kevinmarks]
The other issue is that feeds originally did have external urls - they were local text summaries of what they were linking to
#
tantek
no it means you have to retrieve the post to get its properties
#
tantek
you can't trust the asserted properties in the feed
#
[kevinmarks]
Some sites still do this eg daring fireball
#
aaronpk
[kevinmarks]: that sounds like what we've done with bookmark posts
#
tantek
kevinmarks that was only a legacy RSS problem
#
tantek
which we can workaround
#
tantek
no need to drag that legacy RSS issue into feeds in general
#
tantek
aaronpk, similar thing applies to any object btw
#
tantek
e.g. an h-card from another domain, you likely have to go get that h-card
#
[cleverdevil]
Yes, workaround, and coerce as much as possible into sanity so that the client can have a long and productive life ;_
#
tantek
to get its actual name, photo etc
#
aaronpk
okay, so I think I found my first need for a background worker for monocle
#
aaronpk
I was hoping to avoid it, letting monocle respond to websub payloads directly, but it's looking more complicated than that after all
#
aaronpk
so, it will get a websub payload with the contents of a feed (h-entry or Atom/RSS or JSON Feed), and will process each item in the list. if that item has a URL that's on a different domain, it's going to need to bump that into the queue to fetch it from the remote URL
#
[kevinmarks]
Right, originally feeds were bookmarks
#
tantek
link blogs as it wer
#
[kevinmarks]
Otoh, the https://daringfireball.net/feeds/json has url and external_url
#
[kevinmarks]
The uids are the local urls
#
aaronpk
which incidentally means it'll be almost empty to my feed reader since most of those canonical URLs don't have microformats
#
[cleverdevil]
Better, but still not quite right.
#
aaronpk
still this is not the primary use case of a microblog reader
#
[kevinmarks]
<link rel="alternate" type="text/html" href="https://www.washingtonpost.com/news/the-switch/wp/2017/11/21/the-fcc-has-unveiled-its-plan-to-rollback-its-net-neutrality-rules/?utm_term=.f239ae8e96a4"/>
#
[kevinmarks]
<link rel="shorturl" type="text/html" href="http://df4.us/qh3"/>
#
[kevinmarks]
<link rel="related" type="text/html" href="https://daringfireball.net/linked/2017/11/22/fcc-net-neutrality"/>
#
aaronpk
really those links are all bookmarks
#
[kevinmarks]
Is the Atom
#
aaronpk
ANYWAY this was not my question
#
[kevinmarks]
No, some of the posts are local
#
[kevinmarks]
Eg <link rel="alternate" type="text/html" href="https://daringfireball.net/2017/11/twitter_280"/>
#
[kevinmarks]
<link rel="shorturl" href="http://df4.us/qg6"/>
#
[kevinmarks]
<id>tag:daringfireball.net,<2017://1.34278></id>
KartikPrabhu joined the channel
#
aaronpk
[cleverdevil]: btw these XRay URLs are the format that i'm expecting microsub clients to interact with, so I hope it looks relatively easy to consume
#
[kevinmarks]
So Gruber is flipping the rel="related" meaning in atom to on that I think https://tools.ietf.org/html/rfc4287#section-4.2.7.2
#
aaronpk
why isn't my feed parsing library finding the <id> element?
#
tantek
rel=related is a good example of nonsense markup - any use of "rel" means there is a relationship between the links
#
KartikPrabhu
rel=related lol!
#
tantek
the point of rel is to state what that the relationship *is*
#
KartikPrabhu
that is like class="class"
#
tantek
yeah KartikPrabhu it's an astronomy abstraction
#
tantek
add a layer of abstraction because it gives the illusion of adding meaning when it does nothing
#
aaronpk
so this is an example of a feed with no URLs for the permalinks of each item
#
[kevinmarks]
No, the feeds have them, but they are a but odd.
#
aaronpk
oh right rel=alternate is supposed to be the permalink
#
[kevinmarks]
The Atom definition of related is what jsonfeed calls external_url
#
[kevinmarks]
But gruber is using them backwards because gruber
#
tantek
even "external" is a poor name. external to what? where do you draw the boundaries?
#
tantek
does gruber auto-tweet everything he posts?
#
aaronpk
"elsewhere" according t othe spec
#
tantek
maybe it would be easier to use his twitter feed as cleaner substitute for his site feed
#
aaronpk
"external_url (very optional, string) is the URL of a page elsewhere. This is especially useful for linkblogs." https://jsonfeed.org/version/1
#
tantek
also redundant
#
aaronpk
"If url links to where you’re talking about a thing, then external_url links to the thing you’re talking about."
#
[kevinmarks]
He also has entries that point to his podcast entries on the same domain
#
tantek
they just didn't want to call it "about" because the RDF folks call it that
#
tantek
and there was too much bad blood to re-use "about" like that
#
[kevinmarks]
As external_url
#
tantek
"the thing you're talking about" might be on your own site
#
tantek
and thus NOT an *external* URL at all
#
tantek
so yeah, crappy name
#
tantek
though not as meaningless as "related"
#
aaronpk
according to jsonfeed, gruber's Atom feed has the two URLs backwards
#
aaronpk
his rel=alternate should link to his own permalink, and rel=related should link to the external post
#
aaronpk
from https://jsonfeed.org/mappingrssandatom "link with rel="alternate" maps to url in JSON. If rel="related" is used for links to an external site, in JSON Feed those map to external_url."
#
tantek
right
#
aaronpk
so i'm not going to use https://daringfireball.net/feeds/main as a good example of an Atom feed
#
tantek
no don't
#
tantek
I'd say use an Atom feed from snafed
#
tantek
snarfed:
#
[kevinmarks]
That's what I was saying
#
tantek
as I'd expect snarfed's Atom feeds to be precisely correct
#
tantek
and if you do discover a problem, he'll likely fix it quickly
#
[cleverdevil]
Yeah, this is perfect, aaronpk, in terms of a simple format that a Microsub client can consume.
#
[kevinmarks]
But it is a good example of a site with multiple kinds of posts
#
aaronpk
[cleverdevil]: ?
#
aaronpk
would it be reasonable to map JSONFeed's external_url to the h-entry bookmark-of property?
#
aaronpk
it seems like that's the intended use of it
#
[kevinmarks]
I'm laughing wryly at "correct" here
#
[kevinmarks]
This was my life from 2003 to 2007
#
[cleverdevil]
So, I actually built a custom JSON Feed extension to map some things out in the hopes that Manton would adopt it in for the Micro.blog timeline - https://cleverdevil.io/content/all/?_t=microblog
#
[cleverdevil]
See the `_meta` extension.
#
tantek
aaronpk - not really
#
tantek
like I said "external_url" is really a poorly named "about"
#
tantek
and is also too vague to be usefrul
#
tantek
e.g. it could be the thing you're replying to
#
tantek
you can't assume it's a bookmark post
#
aaronpk
looking at the daringfireball posts, most of those have original content that he wrote, and usually contain a blockquote from the site. that looks an awful lot like my own bookmark posts
#
tantek
they're far beyond a bookmark post
#
tantek
they had their own structure
#
[kevinmarks]
Yes, they're mostly bookmark posts. He links to the permalink with a star
#
tantek
i.e. just because an article has quotations with citations, doesn't make it a bookmark post
#
aaronpk
okay anyway let's get back to my actual problem
#
aaronpk
this got super derailed
#
tantek
aaronpk, any time you try to use RSS or Atom or some random old feed as an example, you will get super derailed
#
aaronpk
I don't really want to debate the meaning of what is a bookmark vs what is a quote or whatever
#
tantek
because there is so much historical nonsense there
#
aaronpk
well I don't really want to completely disregard Atom and RSS since there are so many existing blogs with those formats that I want to follow
#
[kevinmarks]
That's fair. The question is how much of this messiness you want to model, and how much you're happy with pruning off to fit into your model
#
tantek
it's hard to avoid those kinds of debates if you're trying model something that is poorly named "external_url"
#
tantek
aaronpk, no one said completely disregard
#
aaronpk
forget external_url, that isn't part of the original discussion
#
tantek
just stop prioritizing them first in your design
#
[kevinmarks]
And I'm being a little unfair because I know a lot of edge cases
#
tantek
otherwise you'll never get even a v0
#
tantek
because they have endless ratholes
#
tantek
that you'll get stuck in
#
aaronpk
so, immediate example of where this is going to happen: indienews
#
tantek
start with making it work with h-feed (and possibly jf2 since those should be a direct mapping)
#
tantek
and only later, after you've got that whole flow figured out, look at what you need to do in addition for feed file formats
#
aaronpk
since it's an aggregator, all the h-entrys on news.indieweb.org have url properties on different domains
#
aaronpk
so if I follow https://news.indieweb.org/en in my reader, it should avoid using the content of the h-entrys it gets in the websub notifications, and instead go fetch the individual URLs
#
tantek
right. exactly why fatpings were not really that useful
#
tantek
because you have to go fetch the individual URLs anyway to get the real substance (as it were)
#
aaronpk
definitely defeats the purpose of fat pings for that, but the fat pings will still be useful for following blogs directly
#
tantek
yeah I could see that
#
aaronpk
this also sounds like a *great* thing for the microsub server to do, to remove all this complexity from individual reader interfaces
#
aaronpk
so, that solves part of the problem. the remaining issue is how should I handle uniquely identifying posts. right now h-entry lists both uid and url properties.
#
aaronpk
when I encounter an h-entry, I need to know whether i've seen it before, and update the existing one or add it new
#
aaronpk
right now I prioritize uid and if that's not present I use the url. that's stored internally and I look up the post by that identifier
#
tantek
that seems reasonable based on intuitively what I feel like I've seen from existing h-entrys in the wild
#
aaronpk
so since the goal of microsub is to simplify consuming content when building a reader interface, I want to avoid clients having to do that logic of "use uid if present otherwise use url" etc, especially since some clients might not end up implementing that and we wouldn't know
#
tantek
I presume you've captured that in a "Goals" section in the spec?
#
aaronpk
no but that's a good idea :)
[chrisaldrich] joined the channel
#
aaronparecki.com
edited /Microsub-spec (+9) "/* Design Goals */ Goals->Considerations"
(view diff)
snarfed joined the channel
#
aaronpk
so that begs the question of whether the microsub server should be generating its own IDs for the client to use when deduping items and referring to items at the server
#
aaronpk
I'm kind of leaning towards just saying that the server should include its own "id" property, and that can be any string value such as the URL of the post or an internal database ID
#
tantek
might a client talk to multiple microsub servers?
#
tantek
then it might be better to not do that
#
tantek
and instead, literally just use the URLs of the items
#
aaronpk
but then we're back to the question of what if there is no URL
#
tantek
and specify an algorithm for canonically synthesizing them if missing (using fragmentions)
#
tantek
that way two microsub servers would synthesize the same synthetic permalink for an item lacking a URL
#
tantek
and you could test it
#
aaronpk
that seems unlikely to work
#
tantek
fragmentions work because of this
#
aaronpk
fragmentions aren't guaranteed to be unique over time when applied to feed pages
#
aaronpk
okay let's back up a step
#
aaronpk
there are two use cases I have in mind where I need the client to be able to refer to a particular post within a channel
#
aaronpk
finding new items in the channel, such as "show me things added to the channel after this post X"
#
aaronpk
or "hide this post"
#
aaronpk
hide/delete
#
aaronpk
any other interactions the client would do with the post would be doing things like replying to it, or liking it, which it does at the micro*pub* endpoint and uses the post's URL
#
[cleverdevil]
I see why this is a bit of sticky decision.
#
[cleverdevil]
I don't think you're going to come up with a single solution that handles every single edge case and use case.
#
[cleverdevil]
Might be better to focus in on solving the problem for use cases that are most important.
#
tantek
algorithmic synthetic URLs as noted should be enough for the new items / hide items
#
aaronpk
that's not the only issue
#
aaronpk
if i'm doing the thing where I prioritize the uid if present and use url only if there is no uid, that means I have to tell clients how to refer to posts by either of those properties
#
tantek
ignore uids that are not URLs
#
tantek
becuase they're being produced by a system built with bad assumptions
#
aaronpk
that still doesn't really solve my problem
#
tantek
uid should only be a hint for *which* URL to use
#
tantek
not a separate value
#
tantek
(pretty sure that's 100% true in practice in existing h-entry / h-feed that have any uid)
#
aaronpk
which URL out of ...?
#
aaronpk
oh like if there are multiple values for the actual url property?
#
tantek
some with tracking garbage etc
#
tantek
or short urls
#
aaronpk
oh gosh I dont think I have any tests in XRay for that
#
aaronpk
pretty sure it always just uses the first
#
aaronpk
oooookay soooo
#
aaronpk
this has been a fun hour and a half
#
tantek
consider yourself lucky nobody mentioned 14
#
aaronpk
is there any point to giving clients the uid of a post?
#
aaronpk
maybe I just don't return that in the response at all
jjuran joined the channel
#
aaronpk
that sounds like a nice simplification
#
tantek
right, just give clients the permalink
#
tantek
that the server has done the hardwork to figure out if necessary
#
aaronpk
that gives clients only one way to refer to a post, which is better
#
aaronpk
the only trick is I don't think I want to make up a URL if there isn't actually one in the item, because I don't want to be in a situation where clients are hesitant to use the value for fear of accidentally misleading users to click on a thing that doesn't exist
#
[cleverdevil]
Perhaps you could generate it in such a way that its clear the URL is synthesized?
#
[cleverdevil]
That way clients can choose what to do with the URL.
#
[cleverdevil]
Or you could include an additional property indicating that the URL is made up.
#
aaronpk
that'd be one option, but does mean clients have to add a string check every time they use the URL
#
[cleverdevil]
That's not much of an issue, IMO.
#
[cleverdevil]
You could also just use two different properties.
#
aaronpk
IMO the easiest thing that would probably have the least consequence if ignored would be to prepend a "#" to an arbitrary string, so clients could check "does the URL begin with '#'? if so, don't show the URL", and if they forget to add that check, clicking that in a browser will not navigate away to some unexpected place
#
aaronpk
but yeah the two different properties idea was what I was saying earlier where the microsub server could just always use its own local ID for the client to refer to posts
#
[kevinmarks]
The point is to make one that resolves - so it could be one that resolves via you (or hasharchive or archive.org)
#
www.boffosocko.com
edited /Indieweb_for_Journalism (+564) "/* Articles relating to Indieweb and Journalism */ Poynter article about savemy.news"
(view diff)
#
aaronparecki.com
edited /Microsub-spec (+1065) "/* Design Goals */ new section"
(view diff)
#
aaronpk
I guess if the microsub server makes up a URL for a post that exists on its domain and actually displays the post at that URL, then that seems like a pretty good fallback behavior
#
aaronpk
does place more of a burden on the server, but it's not entirely unreasonable
#
[kevinmarks]
One thing that feed serving did tend to get right was etag/last-modified and 302
#
[kevinmarks]
So if you do have to poll, you can exit early on either side