#social 2019-03-10
2019-03-10 UTC
Guest84_ joined the channel
# 
jdormit[m] dansup: kaninii's suggestion was to not forward activities whose object is an actor. Was that the three lines of code you mentioned earlier? I don't understand why that would be a problem - do you know what he meant?
# jdormit[m] (forward as in inbox forwarding)
xmpp-social and Gargron joined the channel
# dansup jdormit[m]:
# dansup [14:25:51] <kaniini> why is it posting
# dansup [14:25:55] <kaniini> an activity to itself
# dansup [14:30:40] <kaniini> the problem is here
# dansup [14:30:55] <kaniini> they are delivering a remote activity to themselves
# dansup [14:30:58] <kaniini> voila infinite loop
# dansup this is from #pixelfed-dev btw, kaniini and pleroma are boycotting this channel lol
# dansup on freenode
# dansup jdormit[m]: anyways, pixelfeds issue with pleroma was that the Accept object needs to contain the same id as the one pleroma sent in the Follow
# dansup so the fix was to return the Follow object that pleroma sent as the Accept object in the reply
timbl joined the channel
# fr33domlover Hi people!
# fr33domlover Long time ago I asked here, which application objects should become ActivityPub actors?
# fr33domlover There was some discussion but no clear decision, on my part at least, and now I'd like to ask again and really decide
# fr33domlover What would be a good requirement for making something an actor?
# fr33domlover - Requiring its own inbox?
# fr33domlover - Needs to have its own list of followers?
# fr33domlover etc.
# fr33domlover Example: In an issue tracker people may wish to get notified on a specific issue, so issues needs to have lists of followers - does it mean issues will need to be actors?
# dansup fr33domlover: hi
# dansup I assume you are talking about forgefed or git
# dansup each project could have its own actor
# fr33domlover dansup, yeah I mean not just each project, but also each ticket, maybe even smaller things
# dansup nope, each project actor should have an inbox to consume any related activity
# fr33domlover dansup, I'm writing a question at socialhub with more details ^_^
# dansup otherwise you need a route for the inbox of every ticked
# dansup ticket*
timbl and vitalyster joined the channel
# fr33domlover dansup, yeah I guess that means a lot of requests and storage (unless there's a way to declare a shared inbox). But if tickets aren't actors, there needs to be a way to follow a specific ticket and maintain a list of its followers!
# fr33domlover (and specifically, *not* have to list every single ticket's followers collection URI in a project's JSON-LD representation, otherwise some projects will have huge representations :p)
# dansup oh god, that would require a public/private key for every issue
# dansup how is that scalable?
# fr33domlover dansup, yeah but that can be solved by having a shared key, I'm already doing that in Vervis
# dansup you need an actor for a user, project, issue, merge request, ect
# fr33domlover dansup, the question is can we also do that with the inbox URI
# fr33domlover and then it can really be nice
# dansup i think you are approaching this wrong, you can have a collection on an object without making a new actor
# dansup like mastodon polls
# fr33domlover dansup, yes I know but just need to be sure that it's as easy to use as the regular "followers" collection etc.
# fr33domlover dansup, hmmmm why would projects be actors then? We can just let people be actors and that's it
# fr33domlover Idk
# dansup because a user has many projects
# fr33domlover dansup, and a project has many tickets, what's the difference? ^_^
# dansup i mean, creating an actor for each issue seems excessive
# fr33domlover I don't mind making projects be actors
# fr33domlover I just want to have a set of rules for how to decide
# fr33domlover Not just arbitrarily
# dansup or each issue could have a pseudo actor which is a route that the backend could translate into a project issue
# dansup like a query string of the project actor
# dansup im not sure how mastodon or other projects would interpret that though
# fr33domlover By some intuition, actors are things that post stuff
# fr33domlover So that's humans and bots basically
# fr33domlover Projects don't post stuff, they don't initiate action
# fr33domlover Project changes are done by human users
# fr33domlover (Or bots, in some special cases I guess)
# fr33domlover By that intuition, only humans/bots would be actors
# fr33domlover On the other hand, if an actor is anything that can be followed and have its own followers/followed collection...
# fr33domlover dansup, cwebber said sharedInbox was a mistake but Idk why :P
# cjslep[m] fr33domlover it's going to be an arbitrary rule no matter what model you decide to follow.
# fr33domlover It could really help here lol
# dansup its not a mistake
# fr33domlover dansup, I mean as in he regrets putting it in the spec idk
# cjslep[m] Shared inbox is a mistake because it breaks the direct delivery paradigm of AP
# dansup fr33domlover: I am working on a federated git project too, these are just my opinions
# fr33domlover dansup, sure :)
# dansup cjslep[m]: not really
# cjslep[m] Shared inbox makes it more like delegation delivery that is email
# dansup if shared Inbox wasn't a part of AP, it would be no better than oStatus lol
# fr33domlover sharedInbox helps when delivering something to a large number of actors hosted on the same server
# cjslep[m] There is no "404" returned for the accounts in the shared inbox that are failed to be delivered. The sender only knows this class of information if it directly delivers.
# dansup I have 210 distinct sharedInbox followers on my pixelfed.social account, it doesnt make sense to deliver the same object multiple times to the same domain
# cjslep[m] This the sender can only "hope" that it's delivery succeeds for all actors and doesn't get a receipt reply listing what was successful and what wasn't.
# dansup you only consume sharedInbox if the implementation supports it
# fr33domlover cjslep[m], hmmm but how would we otherwise do things?
# cjslep[m] Also fr33domlover I'm of the opinion that "actors are entities that act in the Fediverse". Forge projects make sense. Forge issues.... not so much.
# dansup thank you cjslep[m]
# dansup that is what I was trying to tell them
# fr33domlover Haha :)
# fr33domlover I have that intuition too, but, I want to challenge it
# cjslep[m] (again it's always going to be an arbitrary rule. If you don't like mine, that's fine. But the alternative rule will be no better/worse)
# fr33domlover cjslep[m], in which way do projects act, that tickes don't? Think about something like Mantis, Redmine, etc. which is only an issue tracker software
# cjslep[m] I think AP is missing a "instance" actor that represents the whole server.
# dansup cjslep[m]: isnt that a sharedInbox or Application ;)
# cjslep[m] sent a long message: cjslep[m]_2019-03-10_09:27:08.txt <https://matrix.cybre.space/_matrix/media/v1/download/cybre.space/oEUdafJHMEIPmLYsGhgcxSnl>
# dansup cjslep[m]: btw, have you tried pixelfed? registration is open https://pixelfed.social
# cjslep[m] Yes but codified as a proper actor and governed in an AP extension. SharedInbox is a URI attached to all actors on an instance.
# cjslep[m] I think I had an account from when registration was open earlier... Let me find it
# cjslep[m] https://pixelfed.social/cj is me
# dansup oh cool
# dansup oh I should tell pleroma that they are vulnerable to xss via svg avatars lol
# cjslep[m] But yes fundamentally sharedInbox puts a black box around delivery; an actor would be able to send a reply back to the originating actor with Accept and Rejects to say "here's the sharedInbox destinations I successfully resolved and here's the ones that failed".
# dansup i disagree, the black box is the internal representation of the object graph that each implementation can handle as they please
# cjslep[m] Sure, but the caller still needs to update its internal representation to reflect the reality of it's peer
# cjslep[m] Right now sharedInbox is all or nothing success, which I don't think is realistic.
# dansup yeah but that all or nothing is a feature. you can reject an object without a response
# dansup if your implementation is written properly
# dansup and handle it too
# cjslep[m] I agree. I mean to say "all of nothing fine grained (per delivery) vs all or nothing en Masse (per N deliveries)"
# cjslep[m] It's helpful to have the fine grained rejections due to "account not found" or "moved" (let user know about those accounts or automatically forward a retry delivery), etc. All those nitty gritty real world edge cases.
# dansup well, I think that is a consequence of the generality of the spec, it really depends on your use case
# fr33domlover Oh by the way, I just realized something: Tickets don't need keys if they don't deliver anything
# fr33domlover All changes to a ticket/whatever are done by a human user who sends the action
# fr33domlover The ticket doesn't actually POST stuff
# dansup fr33domlover: thats what I was saying lol, just reference them internally with a unique actor url
# dansup cjslep[m]: what do you think of pixelfed?
# fr33domlover dansup, maybe I just won't make them actors at all, projects are independent entities but tickets are tied to specific projects semantically. That would be an okay setup if there's a sane scalable way to implement followers per ticket etc.
# fr33domlover I'll work on it ^_^
# dansup fr33domlover: sounds good :)
# cjslep[m] dansup my notes above doesn't change the fact that w/o sharedInbox its a direct delivery mechanism, and w/ shared inbox there's now an added layer, which is a new dimension that fundamentally makes delivery a shared effort instead of solely being the senders effort
# fr33domlover dansup, cjslep[m] I'm also wondering, when you add someone as a project collaborator, should you send an Offer first to get their Accept, or instantly send them an access token etc. like in GitLab/Gogs (you get a notification, not asked for approval iirc)
# fr33domlover I'm just wondering due to the cons of automatically giving access
# cjslep[m] It constrains future design decisions and could make some design choices fundamentally incompatible with sharedInbox
# dansup cjslep[m]: yeah but mastodon is a great example of why sharedInbox is a real world benefit
# fr33domlover (Side note: sharedInbox makes the most benefit when there are big instances and many people who follow you use them)
# dansup that reminds me, I should renew socialhub.network before it expires
# fr33domlover (I wish that wasn't the case :P)
# cjslep[m] I won't deny that. A fundamental problem of a federated spec is that implementations move faster than the thinking, which makes future iterations harder and slower
# cjslep[m] Than the spec-thinking*
# dansup cjslep[m]: not really, an implementation does not need to have a sharedInbox, pixelfed doesnt yet
# cjslep[m] Neither does go fed, but now and for all of time when developing extensions we have to think "...and what about sharedInbox?"
# dansup cjslep[m]: so pixelfed actually benefits from sharedInbox by grouping domains and delivering to distinct instance inboxes
# cjslep[m] I'm not doubting the real world benefit. I am doubting it's a scalable long term solution when it comes to more people extending AP in unforseen ways.
# cjslep[m] I think it's great Mastodon folks put the hard work and effort into it to quickly allow themselves to scale. I don't want to diminish that
# dansup thats not a problem at all because each implementation can handle what they want, how they want
# cjslep[m] dansup you're right but it sure would suck to lose this real world benefit in the process of doing so
# dansup cjslep[m]: i'm pretty drunk right now, im not sure what you mean
timbl joined the channel
# cjslep[m] Another problem is that the object crash doesn't match the reality of the delivery. In a direct delivery scenario, you can look at the to, cc, etc and know how it got there. SharedInbox means we have no idea which endpoints this object was actually sent to. The object graph doesn't record the sharedInbox-ness of it
# cjslep[m] Also dansup I like PixelFed, I'm just not a big insta- kind of person personally.
# cjslep[m] I think PixelFed is incredibly polished and it's very slick to use.
# cjslep[m] I would probably use it more if I knew of an Android app that catered to it
timbl joined the channel
# dansup cjslep[m]: yeah that is true, I guess it really depends on the implementation. Writing a full library like gofed, I can see why you are concerned
# cjslep[m] Also dansup get some sleep. :)
# dansup cjslep[m]: sleep is for the weak
# dansup I'm fortunate enough to set my own schedule
# cjslep[m] Haha. That's a nice thing to have. I'm envious! Also I'll have to think on this sharedInbox stuff more and begin to write some of these thoughts down as notes.
# cjslep[m] Also dansup when is PixelFed going to support cat-mode Easter egg?
# dansup cjslep[m]: funny you should mention, I've been slowly adding the April Fools "feature" https://mastodon.social/@dansup/101628838432252297
# dansup nobody has noticed yet lmao
# dansup the timeline UI will resemble facebook UI for the day, with an opt-out
# dansup hello pixelbook
# cjslep[m] Lol!
# fr33domlover Hmmm weird, looks like there isn't any RDF vocabulary for signature/encryption algorithms
# fr33domlover The closest thing is the JOSE list but it just has textual ID codes, no URIs
# fr33domlover Second best is the XML Signature spec, which does have IDs, but it feels weird to use stuff not meant for RDF, and the list there is also very limited
# fr33domlover And the hypothetical IANA registry for that spec doesn't exist
# fr33domlover uses custom IDs for now, maybe we can get some sane IDs for this somewhere under w3id.org
mayel, timbl, decentral1se, cwebber2 and cwebber joined the channel
# Gargron cwebber: Could we encode a bloom filter in JSON-LD?
# Gargron For example, if you want to distribute the message to followers, but don't want to put tens of thousands of URIs into the JSON, could you create a bloom filter, put that in the JSON, and have the receiving server check its own state against the bloom filter to ensure the wrong people don't get the message?
# nightpool[m] :o
# nightpool[m] why not just followers.group_by(&:public_inbox).each_slice(500) or something?
# Gargron Would we really want to do that for every message, nightpool[m]
# nightpool[m] maybe only private ones?
# nightpool[m] this is leading me to believe we should invest time in getting thib's async web code working
# nightpool[m] if we're going to keep running into scalability challenges like this
# fr33domlover wonders if the solution is to keep instances small
# nightpool[m] doesn't really help
# fr33domlover nightpool[m], well, the more instances we have per amount of users on the network, the more we can distribute the computation. Say, we'd exchange sending a huge JSON file with concurrent HTTP requests to many small instances, on each of which you'd have only a small number of followers
# Gargron that's a lot of HTTP requests then, which also sucks
# nightpool[m] there's also still a big jump from "deliver the same payload to every follower instance" to "deliver a customized payload listing every follower to each instance"
# fr33domlover Gargron, yeah but it's like a mailing list server sending email to all subscribers. Also, is it a problem to send many HTTP requests? Take your time, put them in a background job etc.
# fr33domlover Many requests may still suck but it could all happen in parallel
# fr33domlover Without clogging one huge server's queue etc.
# nightpool[m] 10,000 http requests is going to suck no matter how you parallelize it
# fr33domlover Yeah true
# fr33domlover is glad he doesn't have 10,000 followers :P
# nightpool[m] but the idea that big servers need to be more performent then smaller ones isn't insane
# nightpool[m] which is why I said
# nightpool[m] > this is leading me to believe we should invest time in getting thib's async web code working
# nightpool[m] re: mastodon specifically
# nightpool[m] and, to improve delivery, use something like Typhoeus
# fr33domlover nightpool[m], which part of the delivery process does Mastodon do serially/sync that you'd like to do async instead? Just curious :) for my own project as well
# nightpool[m] we don't do any of it serially
# nightpool[m] or synchronously
# fr33domlover I suppose sending those 10,000 requests asynchronously could be an improvement if not already done, like, run each POST in a separate (hopefully lightwight) thread, or at least one such thread per target server
# fr33domlover (Actually even multiple requests to the same server could be done async, although I suppose this doesn't happen when servers provide a sharedInbox)
# nightpool[m] yes we currently use a thread based worker queue per request
# nightpool[m] I'm saying that that's too heavyweight if we're going to be increasing the amount of requests we do by an order of magnitude, and we should be using something like Typhoeus instead
# nightpool[m] for our web server, we use Puma, which is a hybrid forking/threaded server iirc
# fr33domlover Idk Ruby but, say, in Haskell threads are very lightweight and it's trivial to launch 10,000 threads for those 10,000 HTTP requests etc.
# fr33domlover nightpool[m], hmm yeah typhoeus sounds like the right thing, run requests in parallel ^_^
# nightpool[m] but there have been suggestions to use something fiber-based
# nightpool[m] Ruby had threads and fibers
# nightpool[m] threads are OS threads
# nightpool[m] so not very lightweight :P
# fr33domlover Haha yeah
# fr33domlover But Typhoeus has a 200 concurrency limit
# fr33domlover So I guess it has a way to run many requests concurrently?
# fr33domlover Whether the implementation is based on fibers, or on carefully managing many open TCP connections, or something else, I suppose it could provide the concurrency benefits
# nightpool[m] yes it uses the built in Curl async framework
# nightpool[m] or, the async API that is built in to curl
# cjslep[m] Has anyone tried thinking of using the inbox forwarding mechanism to get a server to deliver to intended recipients it owns, more easily? At first glance, I have my doubts. But maybe someone with a better perspective can come up with something workable.
# fr33domlover cjslep[m], hm intended recipients it owns?
# fr33domlover (What do you mean? :)
# cjslep[m] Yes, the idea is to reexamine inbox forwarding to see if there is a way it can also support the solution that is currently solved by sharedinbox.
# cjslep[m] something like: 1 HTTP request to a client server -> trigger inbox forwarding -> ??? -> client delivers to intended recipients.
# cjslep[m] last step addendum: *but in such a way its only to the intended recipients residing on its server.
# cjslep[m] But , as I said, I have my doubts about the "???" step.
# fr33domlover cjslep[m], hmm do you mean something like this: (1) deliver to 1 actor on server A (2) somehow let server A know that it can locally deliver to the other recpients of that activity, that it locally owns
# fr33domlover I was wondering earlier about such a thing :P
# cjslep[m] Well, yes, that description is the overall goal behavior. Right now that perfectly describes sharedInbox.
# fr33domlover cjslep[m], well, what other way is there? I mean, no matter how you trigger that local sending, if you only deliver in 1 request then you get 1 response, one HTTP status etc. and not one status per delivery
# fr33domlover If you want to get a separate delivery status for each recipient, but in 1 request/response, we'll need to decide how to encode all the statuses
# cjslep[m] I was just playing with the idea of removing divide between sharedInbox vs non-sharedInbox delivery, uniting the benefits (and drawbacks?) of both. I'm not giving it too much thought.
# fr33domlover cjslep[m], well yeah that's what I was thinking too. Like, replace the sharedInbox with (1) use some actor's inbox property (2) Have a way to hint that this inbox is shared
timbl and GDorn joined the channel
# rialtate[m] > <@irc_fr33domlover:cybre.space> cjslep[m], well, what other way is there? I mean, no matter how you trigger that local sending, if you only deliver in 1 request then you get 1 response, one HTTP status etc. and not one status per delivery
# rialtate[m] Doesn't matter much if the response is always 202 anyway
dmitriz joined the channel; vitalyster left the channel