#social 2019-04-09
2019-04-09 UTC
vitalyster, Simounet, xmpp-social, timbl, eprodrom and dmitriz joined the channel
# 
fr33domlover I suppose any result other than getting 2xx http status in the response?
# fr33domlover e.g. TLS error, connection timeout, response timeout, 5xx error, 4xx error.. all of these should make us remember the activity and retry delivery later?
# fr33domlover I'm wondering because 4xx means client error
# fr33domlover So I'm wondering, maybe for 2xx and 4xx, consider delivery done
# fr33domlover And retry on any other result (connection failure, 5xx error, etc.)
# fr33domlover jdormit[m], how do you do it in your library? ^_^
# fr33domlover Actuallt 4xx may get a special status: consider deliveries done, however if we keep getting 4xx from an inbox for a few months, we may wish to remove the actor from follower lists
# 
jaywink[m] I'd consider anything not 2xx delivery error - why would 4xx be a success?
eprodrom joined the channel
# fr33domlover jaywink[m], I agree after months of 404 you might want to give up on that actor, what I mean is that perhaps it's a waste to retry. For examle if you get 5xx it's a server error, maybe you'll retry in an hour and it will succeed. But 4xx is a client error; if you retry, you'll probably get the same result. Your headers will still be bad, or you still get 404 because that actor doesn't exist anymore etc. I
# fr33domlover suppose it's an optimization then: If you get 4xx, don't do periodical retries, just remember the event so that if months of failed attempts pass, you could do something about it. Hmmm would you keep retrying for 4xx the same way as for 5xx or for connection timeout etc.?
# fr33domlover 4xx is kind of like the server saying "I see your activity, and I reject it"
# fr33domlover So in that sense it's like successful delivery; the other side just decided it doesn't want your stuff, it's like physical mail where you deliver a letter to someone's hands and they just throw it away
eprodrom and timbl joined the channel
# jaywink[m] servers sometimes act in mysterious ways and the developers coding them in even more mysterious ways - as said, I would retry anything not 2xx :) better not try to be too clever imho. but for sure don't retry for months. I'd do an exponential backoff strategy, eventually giving up.
# fr33domlover jaywink[m], hmm good points! In my code far it's linear because linear is easier ^_^ I'll figure out a way to make it exponential :p
vitalyster left the channel
# fr33domlover jaywink[m], another question: after a long time passes and all deliveries fail, and you give up, do you remove the actor from local follower lists? Pro: Avoid potentially many failed deliveries (say a server with 10000 users goes down and all requests give 404, removing from follower lists will save you tons of useless requests); Con: If the error is something like Unauthenticated, maybe the target server
# fr33domlover has a bug in the HTTP signature code, we wouldn't want to remove people from follower lists just because of that :P
# fr33domlover wonders
eprodrom joined the channel
# jdormit[m] fr33domlover: I am just implementing delivery so not quite decided. But given that I'm in PHP land so all the handling happens in a synchronous context within a single request, and it's a library so I can't rely on the existence of an external queuing service, I'm thinking I'll just fire off the requested asynchronously and not wait for them to complete...
# jdormit[m] That's not ideal and will mean my services will be perhaps slightly less reliable than other ones, but I don't see an alternative - PHP doesn't even have a way to do multithreading in a web server context as far as I know
eprodrom joined the channel
# nightpool[m] fwiw I strongly strongly recommend against that
# jaywink[m] fr33domlover (IRC): I haven't got that far with AP, but in diaspora protocol land I think it's a common thing to stop delivering to servers after X time of failures. Still need to implement that logic in Socialhome :) I'll probably follow the same kind of logic for AP. I'm not sure however whether to track it per actor or down to the server. Anyway, this is going a bit far away from your question.
# jaywink[m] probably makes sense to not deliver until end of time to actors who have disapeared, that wont scale
# fr33domlover jdormit[m], I'm sure people's PHP apps have a way to handle this. I imagine if you don't have concurrency in PHP, you can do the requests asynchronously using some external service or program. If you don't want your library to require Redis, you can write a generic backend interface and let people choose the implementation: Redis, or native limited one in PHP, or whatever else implements the interface
# nightpool[m] well I guess if you're firing off the request asynchronously it's not a huge deal, but doing requests synchronously is basically unsalable in practice
# nightpool[m] there are ways to work around this as a library—provide hooks for people to add the jobs to their own queuing system and test against something else, or whatever
# fr33domlover jaywink[m], the spec says per actor, which is the general case. I guess with HTTP you can do per server, but not everything: If you get a connection timeout, good chance it's a whole server problem. But if you get 503 internal error, it may be an exception thrown in the server's code, maybe just for that specific actor
# fr33domlover jaywink[m], for simplicity so far I do this per actor, but, before delivery I group the recipients by server, and I try 1 request per server. If I get a TLS error or connection timeout etc. I don't try the other actors, assuming it's a whole-server thing
# fr33domlover (It's the same amount of requests as remember-per-server, the difference is that it uses more DB space I suppose)
# jdormit[m] Yeah I could build some hooks in there I guess and then provide the fire and forget thing as a default. My use case is a WordPress plugin which also lacks any sort of queuing system or control over the deployment environment lol
# fr33domlover (Just to clarify, I do the first-try-one-request-per-server every time, on every batch of deliveries, regardless of how long a server/actor has been unreachable)
# jdormit[m] The PHP ecosystem is one of those dumpster-fire-all-the-way-down kind of situations
# fr33domlover jdormit[m], hmm I suppose PixelFed, Friendica, Nextcloud etc. use some queuing system? I mean, I guess even if someone uses a library for AP, they still want to send requests async the same way
timbl joined the channel