#social 2017-07-12
2017-07-12 UTC
timbl joined the channel
# 
xmpp-social [ajordan] https://www.w3.org/wiki/SocialCG/2017-07-12 is empty
# xmpp-social [ajordan] cwebber2, aaronpk: ^^^?
# xmpp-social [ajordan] Telecon tomorrow?
# xmpp-social [ajordan] Do we have an agenda somewhere other than that page? If there isn't any clear purpose I have other things I'd rather do during that time
prtksxna, prtksxn__, xmpp-social, timbl, cwebber2 and jmcbray joined the channel
# puckipedia hm, that's in about 45 mins right?
# puckipedia yeah, thought of that
# puckipedia and also maybe the extension I added (there's 2 or 3 or so)
# puckipedia currently in a train back home
# 
ben_thatmustbeme sandro, yes, it caused a really bad echo when you were unmuted
RRSAgent joined the channel
# 
RRSAgent logging to http://www.w3.org/2017/07/12-social-irc
Zakim joined the channel
# ben_thatmustbeme waves to Gargron
# puckipedia present+
# ben_thatmustbeme present+
# Gargron present+
# puckipedia aaronpk: websub: there are now two or more implementation imports for each role (pub/sub/hub)
# Gargron i've been recently thinking about sticking HTTP signatures on top of WebSub subscription requests so the hub could attach a user identity to a callback URL (optionally)
# puckipedia aaronpk: no new reports from the implementations in the GNU social space, Mastodon would be really nice to get
# Gargron it's not so much that a user identity is useful because a callback URL is instance-wide but because there is no other concept of "instance" than the domain field on a user, that's basically what i want to use it for
# ben_thatmustbeme go to websub.rocks and test there
# puckipedia aaronpk: we need more reports to go further through the process
# ben_thatmustbeme (that part cut out of me)
# Gargron mastodon is on pubsubhubbub 0.4
# ben_thatmustbeme Gargron, it should be the same, we worked hard to make sure any pubsubhubbub is websub compliant
# ben_thatmustbeme for 0.4 anyway
# puckipedia aaronpk: about PuSH 0.4, it should be compatible with WebSub
# Gargron i am pretty sure websub.rocks worked with mastodon since the very start
# puckipedia Gargron: I don't know how important it is, the application of this is limited. There's a spec called HTTP signatures, came up before related to ActivityPub
# puckipedia q+
# puckipedia Gargron: we can reuse the keys to do salmon keys, and for access control. for WebSub subscriptions, it'd be nice to link the subscriptions to instances or servers
# puckipedia Gargron: domains are [side effects], ... domain whitelisting, blacklisting. somehow link a callback url to the instance it comes from.
# puckipedia cwebber2: if you're adding signatures via http signatures, it'd be backwards compatible
# puckipedia Gargron: it'd be an extra piece of information, and wouldn't be required
# puckipedia aaronpk: why isn't the URL itself enough to use? the domain is part of the callback url, right?
# puckipedia Gargron: not needed, it can be on other domains, etc, be forwarded, and instances can be on subdomains and have forwarded the webfinger from the root
# puckipedia Gargron: we only send private statuses to instances if their domain is approved. but the callback url can be different. Domains that have a different host don't receive the private statuses
# 
cwebber2 puckipedia: mostly in response to http signatures; I recently implemented server to server authorization on Koreg, which uses json web tokens with the key behind it. And I think it might be a slightly better option because it's already in RFC, whereas http signatures might expire. As for federation of websub urls, it sounds a bit hacky to use a random identity, for example a first follower
# cwebber2 Gargron: yes you're right, it's hacky, but that's because this hasn't been a use case in the original protocol designs. They didn't consider an instance (user?) to be an entity by itself and have any agency. Doesn't have to be a random user, can be the first user that followers, but if different user in the future there has to be another method because salmon, etc. Re: JWT I dunno, but HTTP Signatures are dead simple to implement and
# 
aaronpk https://tools.ietf.org/html/draft-cavage-http-signatures-06 is the signature spec we're talking about right?
zatnosk joined the channel
# cwebber2 puckipedia: yes, but also usually activitypub works on the other way, like you specifically determine which users to send it to, which can be a collection of all your followers, so the server pushes out the messages rather than a server subscribing to and endpoint that delivers the messages of that user
# puckipedia scribenick: puckipedia
# puckipedia sandro: this is no reason to slow down websub, right?
# puckipedia Gargron: indeeds
# puckipedia sandro: I don't know http signatures very well, but this doesn't solve distribution. You still have to look up the key, can't you just check the hub url?
# puckipedia Gargron: not quite. the http signature contains the key id, algorithm, and signature (concatenation of a specific set of headers in an order). the key id can be anything. In case of mastodon, the key id is the account resource identifier
# puckipedia Gargron: which is most likely to be in the database. So you can just look it up in the database. if it isn't equal, you can look it up remotely
# puckipedia sandro: by the same mechanism of storing the public key, you could also store the hub url
# puckipedia sandro: your extension would be an add an 'on behalf of'. you could also add a webfinger field to where the subscriptions should be sent
# puckipedia Gargron: there's no connection between webfinger and signature url. The websub subscription contains a callback url, it's not in webfinger
# puckipedia Gargron: the hub server is supposed to validate the url. but, that's nothing to do with who's doing the subscription. and that's the key of what http signatures offers. it's not enough to say it's 'on behalf of'. what if a malicious user sends the request? That's what http signatures are for
# puckipedia sandro: would someone want to raise an issue to [explore how to solve this issue?]
# puckipedia cwebber2: sounds like you just volunteered, sandro :)
# puckipedia cwebber2: so one thing I wanted to understand is slightly [tangental], the purpose of the publicInbox endpoint is to avoid the problem of where you have to do a lot of delivery to a lot of users
# puckipedia cwebber2: we did this in request of diaspora, do you think there are other cases that are not the publicInbox case that aren't handled by that delivery mechanism, that we're really missing?
# puckipedia Gargron: I understand that ActivityPub is very general, in the way it was created with the client API as well, it's oriented to writing messages to specific users
# puckipedia Gargron: in the case of adding activitypub, we won't get a whole new addressing. it's delivered to the followers and the people mentioned, in the case of public posts and private posts
# puckipedia Gargron: if I make a private post, would my server have to send 15000 requests to all servers? my view is that the publicInbox would be used for private posts too. the server would use its knowledge of the followers to do delivery to the specific individuals
# puckipedia q+
# puckipedia cwebber2: my impression is where thing only would be used publicly isn't right, my issue is how we'd addrses this
# puckipedia cwebber2: you'd have to rely on the other server having a good sense of followers
# cwebber2 puckipedia: in the way that I implemented and think of how it should be? your inbox is basically your home feed, so every message that goes to followers of a person, you're addressed to that post. I don't really trust the collections on other servers, I don't use that information. I don't use that to determine if you follow someone. For example, if you block someone, it shouldn't be delivered to that person
# puckipedia scribenick: puckipedia
# puckipedia cwebber2: so partly one way this might tie in is, I agree with puckipedia with that, specifically, the server it's delivering list, and I think a public inbox that also lists which people to deliver it to, instead of a server keepign track of who it should deliver to
# puckipedia cwebber2: you might not want to expose a block activity with a person. you may be federating a block list, without sending that to hostile people. another one, used as an unfortunate [??], there's been this conversation about 'advisory privacy' type things [?]. The challenge of ostatus private messages is because of a lack of explicitness
# puckipedia cwebber2: I worry that would go away if you post to a publicInbox and not specifying who it'd go to
# puckipedia Gargron: there's concerns about synchronizing follower lists, if e.g. a block has not been federated properly, and the person shouldn't be following the other anymore
# puckipedia Gargron: the main problem of the "leaky" privacy is the fact that the status of a message being private or not is bolted on, and not part of the protocol itself
# puckipedia Gargron: if the parser doesn't know what to do with the bolted on private message it defaults to public. this will be solved by ActivityPub, as public/private is part of the protocol
# puckipedia Gargron: the greatest approach would be to explicitly state which places a message would go to, and each place storing which users were targets. however, it's a problem of data modelling. storing a list of thousands of people that are unique for each individual message would be an explosion of storage and complexity. the way Mastodon models is that there are messages, followers, and your home feed is a sum of the messages of your followers
# puckipedia cwebber2: I was wondering if there was a publicInbox which has a list of audiences, would that be satisfactory? e.g. being explicit being a dealbreaker
# puckipedia Gargron: what would the remote server do with the explicit information? For notifications, we could store it. But messages haven't been made to be put in 12k people. as for home feeds, they're based on follower status. even if you get a list of targets, while it is possible to append it to the home feed in redis (cache layer), as soon as that expires
# puckipedia Gargron: e.g. being gone for 2 weeks, it has to be rebuilt based on the database. so those messages would be lost
# puckipedia cwebber2: I'm starting to understand the database side. Seeing how puckipedia and I implemented it, the inbox is a large collection, an ordered list you are tacking items onto. but my impression is, what you're saying, is that in MAstodon there's no such 'inbox' structure, you have each post with who its followers are, you build the inbox from that
# puckipedia cwebber2: so if you unfollow someone, old statuses disappear too?
tantek joined the channel
# puckipedia Gargron: there's two things: accounts, and statuses. Accounts follow other accounts, and are followed by others. this is how it's encoded into and from OStatus
# puckipedia Gargron: there is no concept of a polymorphic table you store the entire inbox into
# puckipedia Gargron: so yeah, that's mainly where the project comes from. The system wasn't build for this massive home feed for every user that stores each thing for each user
# puckipedia cwebber2: it'd be very complicated to calculate, right? would you think to moving a collection for each user would be maybe more efficient?
# ben_thatmustbeme side note, Gargron would love to get your thoughts / feedback on this https://discourse.joinmastodon.org/t/federation-with-indieweb/434
# puckipedia Gargron: it's pretty efficient. when a status appears, we trigger so-called FanOutOnWrite. it looks at the followers of the account who posted it (local or remote), for each follower we put the message on their home feed, the write-layer cache, a redis sorted set that contains the IDs of the statuses on each user online for the last 2 weeks
# puckipedia Gargron: the expensive query is when the feed is cleared out completely, when we have to look at all the messages of the people they're following
# puckipedia cwebber2: I see why the DB structure incentivizes a certain pattern. I feel like I haven't completely thought it through. so, well, I don't know if I have anything else
# puckipedia Gargron: to reiterate, that system is kinda really efficient, when I was building Mastodon, I was researching how other systems implemented it. there's an article on a blog called something dot scalability. Twitter shared something about the architecture. the method I implemented is how Twitter does it as well, it's storage efficient and also very fast
# puckipedia Gargron: you can also do advanced filtering on the home feed
# puckipedia cwebber2: I suggest we stew on these structural differences and what it means and move on to further topics
# puckipedia (Pub)
# ben_thatmustbeme sandro best i have have been able to find is pronounce it sort of like "crew-g" or "crew-ig"
# ben_thatmustbeme from audio clips on the internet
# cwebber2 puckipedia: one weird thing I had was side effects of specific objects... eg user A of server A sends Like to server B on server B, I accidentally handled side effects twice because it was delivered multiple times. So I limited it to only when the user is notified that it gets the side effects
# Gargron tantek: i am pretty sure it just works and you could try it yourself with someone's public profile
# cwebber2 Gargron: so I'm trying to make progress in AP in mastodon as well, and the way I'm doing this is adding the json representations of objects first, and it turns out it's kind of complicated. the pull request has been WIP for like 20 days, which is pretty long. some of the roadblocks are eg sensitive content, which I've added a boolean property, and the Tag type, which I've just added a Tag type. What's still problematic is which
# cwebber2 internal visibility levels should correspond to to/cc objects. It's not clear how to represent that. It's implementation specific but I'd like help/review. But one thing is when to show Create of the object or just the Object. And this again has to do with the internal storage; there is basically a url structure where you basically retrieve a status. So a status can actually be something like an announce of another note, so either
# puckipedia
{type: Announce, id: https://b.example.com/b#object, object: https://a.example.com/a#object}# puckipedia scribenick: puckipedia
# puckipedia cwebber2: so one thing I wanted to say is that this challenge of Creates being kind-of as .. activitypub, as a spec comes from pump.io, which had separate create objects. when it started to turn into ActivityPub it was called ActivityPump, written by oshepherd, familiar with the history of this space
# Gargron :100:
# puckipedia cwebber2: originally, they wanted to remove Create, because it was very artificial. if it's just an object, you may as well ship it
# puckipedia cwebber2: the problem is that not everybody agreed with it (at least Evin[?]), you should always have the verb of the Activity, and when you look at someone's feed you should always see activity, some kind of subject, and some kind of object
# puckipedia s/subject/actor/
# puckipedia cwebber2: so that's the reason the Create activity is there. We talked about this, because webmention doesn't do anything like this. we had talked about removing it, but we got strong pushback from Evin, and from my own implementaion, I wouldn't mind dropping it
# puckipedia we resolved to keep it in, and added the whole goofy mechanism that if you drop an object in an outbox it'd be wrapped in a Create activity. some history
# Gargron so outbox should contain create/announce. the IDs would correspond to local statuses, and the status URIs would return the create/announce as well, and nothing would return the notes standalone - would that be fine?
# puckipedia cwebber2: not as short as I implied it was going to be. I feel your pain on the create wrapping stuff. if it was just me, we would make it optional. maybe we *should* make it optional, if there's enough implementors
# cwebber2 puckipedia: I think that the Create is actually pretty useful in some cases, because you get an outbox that is just activities, and each activity contains an object. that kind of IMO makes more sense, because you've got Create, Updates and Deletes. I think if just posting objects I think it would be less clear. I don't think an object is required to have any attributedTo or anything, so I think the actor is the only official way to
# ben_thatmustbeme lol, tantek, dual telcon wielding
# cwebber2 Gargron: but they both currently require an id, and it's not clear that both should be resolvable. but both id should resolve to the actual id of the object, so the question actually becomes when the corresponding user has a unique identifier, if queried by itself should it show the Create/Announce or the actual object. the consistent way its the outbox has the Create/Announce, and if you pull back the Create/Announce and nothing has
# cwebber2 puckipedia: so there are multiple ways to fix this, for example one of the ways you can do it is by having always a url which is the url of the Create activity or the Announce activity, and always have the id inside, and the fragment is inside, you can get the Create around it and store that too, returning only the Note it contains, so you could have a fragment identifier or have a sub-id
# Gargron so.... activity = real id. object = fragment id inside the activity
# puckipedia scribenick: puckipedia
# Gargron if that's what you mean that could work, makes sense
# puckipedia cwebber2: fragment identifiers are tricky, because there's no way it can be resolved. Kroeg assumes you can find the top-level object and look through it. that's nice, but not guaranteed
# puckipedia cwebber2: fragment identifiers are best used where you have no guarantee you can retrieve it independently. so I would advice each object having a full-fledged identifier
# puckipedia cwebber2: one solution is, I'm going to type it up on IRC
# cwebber2 https://mastodon.example/post/123 <- object
# cwebber2 https://mastodon.example/post/123/create <- "artificially generated" Create object
# puckipedia cwebber2: so you cna have your base object, and since Mastodon is generating things from its highly structured database, so you can generate a Create object for that, that's dynamically generated. that'd be the cleanest result for what I understand is Mastodon's design
# puckipedia cwebber2: I kind of like fragment identifiers, and I kind of hate them. there's one person here who hates this usage, it can be a bit tricky. If you want people to be able to resolve it the best way is to give it a separate url
# puckipedia cwebber2: answering tsyesika; say, you had an object like this
# puckipedia cwebber2: I think that's the cleanest design for Mastodon's case, I wonder how you feel about that, Gargron
# puckipedia Gargron: I think that would probably work, didn't [..] want to add additional routes, but that's going to work, it's not the worst thing
# puckipedia (Gargron: I could probably help with the implementation?)
# puckipedia (tsyesika: Mastodon would probably not do the ActivityPub client API?)
# puckipedia woops, missed a bit
# puckipedia Gargron: in case of the /announce, the url without the /announce would be the remote one
# puckipedia (did I drop? I don't hear anything)
# puckipedia . o O ( crickets: *cricketing* )
# puckipedia ( seriously, is there anyone speaking? I don't hear anything and my Mumble is not showing anything )
# puckipedia reconnects
# puckipedia cwebber2: my suggestion of the artificial create would be only for the Creates, but for the others they wouldn't have to be dynamically generated. I understand your feeling of ickyness for the particular url structure
# Gargron yeah sorry about making this discussion too implementation specific
# 
tantek commented on https://github.com/swicg/general/issues/5 btw
# ben_thatmustbeme Gargron: its okay, implementation info is always good
# RRSAgent I have made the request to generate http://www.w3.org/2017/07/12-social-minutes.html trackbot
# puckipedia Gargron: so I could probably poke at the ActivityPub implementation for Mastodon
# Gargron puckipedia: https://github.com/tootsuite/mastodon/pull/3844
# puckipedia yeah, got that in front of me
# puckipedia lemme see if I still have the local mastodon instance
# Gargron saving AP users into db gonna be challenging because while some attributes overlap AP users also need some of their collection URIs stored as well
# puckipedia for remote users, you could also just return the ID of the user instead of storing all the metadata
# aaronpk too late, just did https://github.com/tootsuite/mastodon/issues/4173
# puckipedia cwebber2: wouldn't that just be the audience in the object itself
# puckipedia bto/bcc is kinda unclear to me still
# Gargron cwebber2: yeah honestly publicInbox + to: followersURI is what i had in mind
# puckipedia wait here's the Mastodon instance I had running locally
# Gargron aaronpk: tantek: as i said, websub.rocks/publisher works just fine
# Gargron you can enter https://mastodon.social/@Gargron.atom into the field and wait until i toot something
# aaronpk Gargron: it's more about having the report on file than whether it works with websub.rocks. we need the PR with the checkboxes https://github.com/w3c/websub/blob/master/implementation-reports/PUBLISHER-TEMPLATE.md
# Gargron mastodon is publisher, hub and subscriber
# Gargron and yes feel free to file
# Gargron the hub is advertised in the Atom XML
# Gargron all updates are in Atom
# Gargron that's the publisher template. the hub one:
# Gargron subscribes with a secret are our main thing. don't remember if it works without secret
# Gargron ignoring unrecognized params is no problem
# Gargron re-subscribing anytime
# Gargron unsubscribing is fine
# Gargron verification is fine
# Gargron lease request is supported, but a min/max range is enforced
# Gargron matching content type: yes, full contents: yes, signature: yes, it uses the PubSubHubbub 0.4 signature method, whatever that one was. hmac sha256 on shared secret probably
# Gargron sends only a diff
# Gargron subscriber:
# Gargron expects hub link in atom
# Gargron rejects invalid topic URLs
# Gargron creates subscriptions fine
# Gargron uses a secret (same as above)
# Gargron specifies a lease
# Gargron callback URL is unique per subscription
# Gargron callback URL miiiight be guessable
# Gargron unsubscribes
# Gargron dunno about the redirect ones or different rel=self
# Gargron returns 2xx on delivery
# Gargron verifies signature
# Gargron and the last ones, yes
# puckipedia Gargron: so I'll grab some food, wait for the dependencies to install, then I'll take a peek at the PR about activitypub
# Gargron that's all - sorry for spamming the channel didn't expect there to be so many items
timbl joined the channel
# cwebber2 Gargron: puckipedia: https://github.com/w3c/activitypub/issues/242
# cwebber2 https://github.com/w3c/activitypub/issues/242#issuecomment-314880073 might be the easiest way to make this both feasible and simple too
# puckipedia cwebber2: well, blocks only promise no interaction
# puckipedia anyways, imo above would basically be the server using an internal mechanism to federate activities
xmpp-social joined the channel