#social 2017-11-20
2017-11-20 UTC
# 
eprodrom Well, getting headers right for signatures is really error-prone
# eprodrom And library support in languages I've seen so far is sparse
# eprodrom It's an RFC draft
# eprodrom That's all
# eprodrom I'm prepared to be pleasantly surprised
# eprodrom That's good too
# eprodrom I just remember it being a hassle with OAuth 1.0
# eprodrom People can usually find a way to misinterpret directions so they get their signature wrong
# eprodrom gotcha
# eprodrom I don't know what's supposed to happen there
# eprodrom Is the key just another property on the AS2 representation of the object?
# 
cwebber2 eprodrom: http://sebsauvage.net/paste/?0efddcfe6cb03228#W7yhotg5rY7ltL8vOSfOBh0MZaCnoKsEpfS8unxsEjA= see the publicKey section
bengo, rowan, h, cdchapman, timbl, npdoty and eprodrom joined the channel
# eprodrom cwebber2: I added a new issue to AP2
# eprodrom I think it's just a typo
xmpp-social joined the channel
# puckipedia <eprodrom> Well, getting headers right for signatures is really error-prone <- surprisingly in C# it was pretty simple, I had control over every header (including Date!)
# puckipedia hm. I think there's a bug in Mastodon somewhere
# puckipedia I think it has to do with Delete blasting
# puckipedia this post: tag:mastodon.social,2017-06-09:objectId=8499030:objectType=Status <- I think this got deleted. I got two "Undo/Announce" for kellerfuchs, three from cwebber2, 17 from .. .someone else, and 1 from Gargron
# puckipedia this is a dedupe bug for me though
# puckipedia but... 17 times!!
# puckipedia I think it's once per linked server that also had someone that boosted the post
# puckipedia except for Gargron, who is on the originating server
cdchapman and timbl joined the channel
# puckipedia "so why doesn't the event queue system work" ... "oh, I forget to poke it to start processing events"
rowan joined the channel
cdchapman, timbl, h and eprodrom joined the channel
# eprodrom cwebber2: hey
# eprodrom I hope you don't mind that I'm filing issues on AP
# eprodrom I don't think they're normative
# eprodrom I'm just trying to get them out there while I can
# eprodrom huh
# eprodrom What does that mean? "awesome"?
# eprodrom Is it a response to something one of us said?
# eprodrom I should hang out in this channel more
# eprodrom ha
# eprodrom cwebber2: I think the "MAY" wording for the content types is fine. It says, I think, to prefer JSON-LD but that you can accept AS2 type
# eprodrom So it's slightly loosening the requirements
# eprodrom I think it's fine
# eprodrom Cool
# eprodrom I think that was my last read-through so you probably won't see any more issues from me
# eprodrom Oh wait
# eprodrom I think I might have one more little beef
# eprodrom It's this: "The server MUST remove the bto and/or bcc properties, if they exist, from the ActivityStreams object before storage and delivery, ..."
# eprodrom I think that how I store objects is none of your concern
# eprodrom RIght
# eprodrom I agree
# puckipedia maybe change storage to presentation?
# eprodrom Maybe even move it to the privacy considerations
# eprodrom But there's another tricky part
# puckipedia makes guess on what the tricky part is
# eprodrom Which is that, if a server receives an activity for a user...
# eprodrom ...it may want to verify that the activity was addressed to that user.
# eprodrom was that your guess?
# puckipedia knew it. solution: if you get a signed request to the inbox, just trust that it was meant to be delivered there
# puckipedia this is actually needed for proper Mastodon interop
# eprodrom Why
# puckipedia as it sends Like/Follow/Accept/Reject with no audience whatsoever
# eprodrom Ha
# eprodrom It should not do that
# puckipedia ^
# puckipedia no way to prove it isn't :P
# eprodrom Lemme check what pump.io does
# puckipedia and yeah, I strip bto/bcc before presentation, as I consider that the functional equivalent to before storage+delivery
# eprodrom pump.io removes bto/bcc before remote delivery
# eprodrom So that's good
# eprodrom I think it leaves it on in storage though
# puckipedia cwebber2: .... I have noooooo LD signatures right now
# eprodrom That way if someone who is bcc'd tries to read the activity, they have access to it
# puckipedia ^ yep
# eprodrom The server shouldn't say what?
# puckipedia eprodrom: bcc/bto
# puckipedia ah lol
# eprodrom Right
# puckipedia hooray for stress
# eprodrom The spec shouldn't say storage, agreed
# eprodrom I think presentation is another question
# eprodrom Probably good to filter it for everyone but the author
# puckipedia probably true
# eprodrom That's the typical email behaviour, e.g.
# puckipedia egh I need to redo most of the handler chain (and flattening) in the Twitter AP bridge
# puckipedia which is totally expected but still annoying
# eprodrom TWITTER AP BRIDGE
# eprodrom WAS IS DAS
# puckipedia eprodrom: a c2s AP api that allows you to use AP clients to talk to Twitter
# eprodrom puckipedia: f*** yeah
# eprodrom That's great
# eprodrom Bravo
# eprodrom I have that on my TODO list! I am taking it off right now!
# eprodrom cwebber2: that is good fun
# eprodrom cwebber2: maybeeeee make a note about bcc and bto in https://www.w3.org/TR/activitypub/#security-sanitizing-content ?
# eprodrom cwebber2: yes
# eprodrom Agreed
# eprodrom But you were talking about _presentation_
# eprodrom I think maybe making that a security/privacy note is probably enough
# eprodrom "bto/bcc is private info that the author does not want shared with the world. Don't show it to anyone but the author."
# eprodrom cwebber2: will do
# eprodrom cwebber2: https://github.com/w3c/activitypub/issues/280
# eprodrom cwebber2: I guess the main thing I want to make sure of is that I don't stumble over something in this spec 2 months from now where I'm like, uh, this is unclear and/or wrong
# eprodrom So even those this is late in the day, I'd rather say it now than a week from now
# puckipedia idea: hooking the Twitter -> AP bridge I'm writing into a GNU Social server
# eprodrom cwebber2: uh
rowan joined the channel
# eprodrom cwebber2: I have another teensy one
# eprodrom https://www.w3.org/TR/activitypub/#inbox-forwarding <-- here
# eprodrom Huh
# eprodrom Well, it mentions bto and bcc
# eprodrom But we just said that bto and bcc are stripped before remote delivery
# eprodrom So they'll never show up there
# eprodrom I'll make this another issue
# eprodrom Oh thanks
# eprodrom cwebber2: " This problem isn't best demonstrated with an example. "
# eprodrom That's very humble
# eprodrom "There are probably better ways to demonstrate this problem, but I'm feeling lazy so I'm going to give you an example."
# eprodrom So, OK, I think this is a really smart way to do it
# eprodrom Although
# eprodrom umm
# eprodrom Is the activity re-distributed as-is, or is it wrapped in an Announce activity?
# eprodrom cwebber2: yes, I think that's fine
# eprodrom hmm
# eprodrom Well, it is a share
# eprodrom Maybe a share by the server instead of a share by the user
# eprodrom Yeah, I agree
# eprodrom So servers that are validating input to the inbox need to make an exception for this
# eprodrom Also, I think you're right about having an inbox for collections
# eprodrom 1.1
# eprodrom It's also the best way to do groups
# eprodrom Cool
# eprodrom Right
# eprodrom Nice
# eprodrom Right
# eprodrom So, I'm very happy about this
# eprodrom What I'm not happy about is the Big Problem we don't know about yet
# eprodrom But there's only one way to find that
# eprodrom Which is to get so popular that we make it happen
# puckipedia okay, I have a fake AP server that, when you use the proxy endpoint to get a twitter url, gives you an AS2 object representing a twitter post :D
# puckipedia see how it translated the mentions and links as well :P
timbl joined the channel
# puckipedia then I'm going to hook this so it piggy-backs on the Kroeg authentication, and run it for myself :P
# puckipedia of course not :P
# puckipedia actually one sec
# puckipedia going to recreate it
# puckipedia fixing a few small issues I noted
# puckipedia fun fact: I changed a thing then remembered it was cached
# puckipedia egh I feel like ther's a Kroeg bug somewhere
# puckipedia I didn't get that mention
# puckipedia cwebber2: wait did you get the like back
# puckipedia hm
# puckipedia :D
# rowan yo puckipedia i'm so into this bridge you're making
# puckipedia so the biggest problem I'm looking at now is: do I keep track of the user's inbox locally, or do I just let it use the home_timeline
# puckipedia cwebber2: I got that activity. I don't know what's in it yet, but I did get it
# puckipedia ah, it's the reply with the tweetdeck/mastodon c2s bridge
rowan joined the channel
# Gargron cwebber2: i know it's cool and all but you should realize how much bigger and more complicated the mastodon frontend JS would have to be to speak C2S while staying the same
# Gargron anyway, puckipedia: https://github.com/tootsuite/mastodon/issues/5761 i just experienced this on a ~massive~ scale after deleting a toot that received a lot of boosts
# Gargron (my toot, that is)
# puckipedia aha
# Gargron 250,000 queued jobs :thinkhappy:
# puckipedia Gargron: you mean that you OH GOD
# Gargron it almost cleared up now but
# puckipedia do you have a dump of the instances those jobs are going to, just wondering
# Gargron this needs to be fixed
# Gargron well it would be instances whose users reblogged that toot
# Gargron what happens is
# Gargron i send out a delete to my followers. their servers delete my toot and check if it had any reblogs. for each reblog, it sends out an Undo to *their* followers
# Gargron etc
# Gargron puckipedia: so there's a massive overlap between who receives those Undos
# Gargron actually it's not just undos that get forwarded afaik
# Gargron delete also does
# puckipedia hmm
# Gargron yeah, otherwise original would remain on those end nodes
# Gargron but yeah anyway so the receiving inboxes of all of those things don't get deduped
# Gargron before sending
# Gargron i mean, they can't be fully deduped *anyway* because one server cannot know if another has already sent this to this recipient
# Gargron so some wasted work is always gonna be there
# puckipedia but so you would receive it once per federated instance that boosted it
# puckipedia ehm, hm
# Gargron but: 1) the author of the toot should prolly not get their own delete
# Gargron 2) you should try to forward the delete only once per inbox, instead of once per author of reblog
# Gargron per inbox
# Gargron if you know what i'm saying
# puckipedia the question is, can't we implicitly do the unboosting? like, we don't need the unboost to be federated if we get the delete
# Gargron this is true if we assume that deleting the target of an Announce should delete the Announce implicitly
# Gargron if you tell me that's normative i'll assume as such
# Gargron cwebber2:
# puckipedia Gargron: well, you'd be announcing a deleted post otherwise
# erincandescent I think that's notionally fine?
# Gargron that's the thing, you *could* announce a deleted post for all i know
# erincandescent Yeah
# Gargron AP is very generic
# puckipedia well, you'd be announcing a Tombstone
# erincandescent Theres no reason you couldn't announce a tombstone. It'd be weird, but you could do it
# puckipedia which, tbh, is not useful behavior
# Gargron ok so
# Gargron no cwebber2 we do the Undo Announce right now, puckipedia is suggesting we *omit* that if the original was deleted, propagating *only* the delete instead
# puckipedia and even if it didn't
# puckipedia then the timeline of that user would just sshow "announced [a tombstone]"
# puckipedia I mean for me it's more a coffin
# puckipedia "⚰ here lies [post ID]"
# Gargron ok so we can reduce this traffic by half i guess
# Gargron aww
# Gargron cute
# Gargron ok so omitting the Undo->Announce, leaving only Delete, will literally reduce the traffic by 1/2
# puckipedia more than that!
# Gargron then we also need to add a check that if we're about to forward the delete back to its author, don#t
# Gargron that'll reduce a bunch of useless pings also
# puckipedia originally: status.boosts.select(a => a.boosted_by.followers.group_by(instance).count).sum()
# puckipedia now: way less than that, like a total of 1 boost's worth of requests
# puckipedia because the delete is generic and not for a specific follower (and might as well have as:Public in audience, imo!), while undo/boost is
# Gargron oh we also gotta track who we received forward from, and omit forwarding it back *there*
# Gargron tho i dont see how delete is "generic". we still need to forward it for every boost we've done so it goes to the right followers. the only dedup there is shared inboxes
# puckipedia imagine if one instance has 10 people, all with the same 10 followers on 5 different instances. now, a post gets deleted. now instead of 5*10 requests going out for the undo, only 5 requests go out for the delete
# Gargron but this is not how it happens
# Gargron it's probably how it should happen if a correct deduplication algorithm is implemented
# Gargron im considering if it would just be better to literally forward the delete to every known inbox once
# Gargron regardless of followers, minus who sent it and who original author was
# puckipedia and then have the receiving one filter out the forwarding
# Gargron >minus who sent it
# Gargron yes
# puckipedia I mean, if it's already gone on your instance, don't forward it further
# puckipedia so you get like, N deletes, where N is the amount of servers of the federation, and send out N deletes?
# puckipedia anwyays I am moving to bed now