#social 2017-10-25
2017-10-25 UTC
# 
cwebber http://sebsauvage.net/paste/?3e4040d2d0a1cbe9#PNU3MKqPYnqv7eOvlba738cKI9oHaExlc961HWkk8TI= very frustrating
# cwebber whether or not there's a problem with a continuation barrier isn't even deterministic
# cwebber or not within parameters I know how to control
# cwebber I'm not having a problem against my own test server but I am against puckipedia's ... which, it's definitely Guile's (or my) fault somehow, but I don't know how
# cwebber it's probabalistic here... I run that piece of code several times and it fails sometimes and not others
# puckipedia hm. that's the https one?
# cwebber nope, the http
# cwebber one
# cwebber this is what's causing it to "freeze" up
# puckipedia huh
# puckipedia anyways, ehm, that server is going down
# cwebber puckipedia: actually, let's try it against the https one
# puckipedia feel free to make an account on kroeg.azurewebsites.net
# cwebber where do I do that?
# puckipedia ehm, https://kroeg.azurewebsites.net/auth/register
# puckipedia it does nothing with the email
# cwebber https://kroeg.azurewebsites.net/auth/oauth?id=https%3A%2F%2Fkroeg.azurewebsites.net%2Fseadub is making it unhappy :)
# puckipedia it most definitely is valid though :P
# puckipedia oh huh
# puckipedia it doesn't do checking, so it needs the oauth parameters
# puckipedia response_type=token, redirect_uri=whatever (will append fragment)
# puckipedia it actually wants a state too
# puckipedia anyways
# puckipedia gnight :P
# cwebber gnite puckipedia
# cwebber OH
# cwebber could it be the call to FCNTL even when it isn't https?
# cwebber wait no that wouldn't be it
# cwebber I have no idea.
timbl, rowan and xmpp-social joined the channel
# xmpp-social [ajordan] Excellent
xmpp-social, dlehn, rowan and timbl joined the channel
# cwebber hi
# puckipedia I moved my entire site to ActivityPub :P still have to migrate content though
# cwebber puckipedia: cool!
# cwebber I suppose we have a SocialCG meeting in an hour, eh?
# puckipedia good point
# cwebber btw, just in case someone wanted to verify thta it does run under https now
# cwebber https://test.activitypub.rocks/ try with user https://chaos.dustycloud.org/u/tester and JNbn70qGRGTlYm8ZHPmVzQwtfj5ahAHnz9JVpwsy
# cwebber I think I developed a hunch as to where (though not why) the continuation barrier things were happening...
# cwebber puckipedia: it may actually not happen over https on your site now that https is working, I'm not sure. I should give it a try now I suppose
# cwebber I'm hopping in the shower and etc now though
# puckipedia right
# puckipedia ouch
# cwebber puckipedia: ouch?
# puckipedia it's not doing very well on the tests :P
# cwebber oh, did it run further than it did before?
# puckipedia yep, it got all the way to "Fetching the inbox" manual test
# cwebber well that's better, though I'm guessing you don't know why the tests are all failing
# puckipedia yeah, gonna run it against my local instance with some debugging turned on
# cwebber if things aren't hitting that bug that frees me up to add the debuggging stuff your equested
# puckipedia ah, that's also good :P
# puckipedia it'd be nice to see why it gives an unknown server error
# cwebber that one's probably a bug on my end
# cwebber but here it is
# cwebber oh it looks like it got html back
# cwebber and tried to read it as an as2 object
# cwebber I guess it should warn the user about that
# puckipedia yep. it threw a 403
rowan, rowan_ and alanz joined the channel
# puckipedia joins socialcg mumble
# xmpp-social [ajordan] Whew okay
# cwebber too
# xmpp-social [ajordan] Meeting?
# nightpool waves
# cwebber aaronpk, you joining us? :)
# nightpool cwebber: just happened to see your post on masto, decided I had time to jump in and felt guilty for not having been to a meeting recently
# nightpool :D
# cwebber trackbot, start meeting
# 
RRSAgent logging to http://www.w3.org/2017/10/25-social-irc
RRSAgent joined the channel
Zakim joined the channel
# cwebber scribenick: aaronpk
# cwebber present+
# puckipedia present+
# nightpool present+
# alanz present+
# cwebber https://test.activitypub.rocks/
# cwebber aaronpk: we talked about a couple of minor editorial edits, and someone submitted a full review, but also yesterday we got a new implementation in the wild... twitch.tv has a new webhook platform using websub so that's exciting
# cwebber ack ajordan
# 
csarven cwebber: https://test.activitypub.rocks/ is a 502 for me.
# cwebber csarven: oops
# cwebber csarven: no idea what happened will look post-meeting
# cwebber csarven: this version is bridging to my own local testdev instance for just the moment, heh
# 
rhiaro icymi I ripped media upload out and put it on the cg wiki yesterday https://www.w3.org/wiki/SocialCG/ActivityPub/MediaUpload
# cwebber ack ajordan
# cwebber rhiaro, yay
# cwebber q?
# cwebber ack ajordan
er1n joined the channel
# er1n o/
# cwebber q
# cwebber q?
# 
jaywink FYI. here is the diaspora protocol account migration entity, for reference: https://diaspora.github.io/diaspora_federation/entities/account_migration.html
# cwebber https://w3c-ccg.github.io/did-spec/
# cwebber q?
# nightpool (to close the loop async: msatodon does parse a Delete
{ Actor } as deleting an entire actor and all of their content, but doesn't expose this to users in any way))er11n joined the channel
# puckipedia q+
# cwebber ack puckipedia
# cwebber
{"to": ["did:example:d20Hg0teN72oFeo0iNYrblwqt#services/ActivityPub"]}# nightpool q+
# cwebber ack nightpool
# nightpool can someone link the veris(sp?) protocol? googling doesn't seem to be turning anything up
# nightpool q+
# cwebber ack nightpool
# cwebber q?
# cwebber ack aaronpk
# cwebber scribenick: aaronpk
# cwebber aaronpk: this feels like going down quite a rabbit hole... I'm a bit confused because everything going around this point has been based around HTTP and very web-like protocols, and up until this point we've been talking about very web-like things
# cwebber aaronpk: I realize DNS can go down and you're stuck, but you're very likely to have issues with these other systems and we might not know what this is. one benefit of https is dns is not tied to a server, you can move it to a new server etc without having to change your identity... so it seems like http/dns are very natural fits for this, esp in the context of the web, this is the social web community group
# cwebber ack ajordan
# cwebber q?
# cwebber q?
# nightpool q+ PurgeActor
# cwebber ack nightpool
# cwebber q+ to talk about rewriting ids
# cwebber ack cwebber
# nightpool rewriting IDs from something like https://example.org/ID to something like https://new.server/authority/https://example.org/ID is probably doable
# nightpool +1 for deferring, but maybe give a quick summary?
# cwebber haha
# nightpool sgtm
# cwebber aaronpk++
# nightpool aaronpk++
# RRSAgent I have made the request to generate http://www.w3.org/2017/10/25-social-minutes.html trackbot
rowan and tantek joined the channel
# 
aaronpk minutes are up: https://www.w3.org/wiki/SocialCG/2017-10-25/minutes
# cwebber oh no wonder
# cwebber csarven: it's back up
# xmpp-social [ajordan] tantek: o/
# cwebber https://test.activitypub.rocks/ is being port-forwarded from my machine via ssh
# cwebber I can run it on the server for real but
# cwebber I have it local so I can do debugging while working with puckipedia
# cwebber the ssh connection died... guess autossh didn't automatically bring it back up
# cwebber runs `cmatrix -u 10' to keep the ssh connection from timing out from inactivity.. that should do it, right? ;)
# nightpool cwebber: just so I'm clear, the test suite doesn't have any S2S stuff yet, right?
# alanz Is Zakim a bot?
timbl_ and rowan joined the channel
# xmpp-social [ajordan] Yep!
# alanz quite funky, is it open source, available somewhere?
# alanz found it, nvm
# puckipedia the first actual other person followed me from Mastodon towards my Kroeg instance! and, well, https://dl.puckipedia.com/up/companioncube.png
# puckipedia bit later, I got it working nicely: https://dl.puckipedia.com/up/woo_avatars.png
# puckipedia is happy
# cwebber nightpool: it has S2S kind of "prompt'y" tests where it asks you to try things, and then describes the behavior you should see and you check if it's correct
# cwebber in the future it may do it in a more automated fashion
# cwebber the SocialWG agreed that since we have real interoperable implementations this was good enough for us to get to PR
# cwebber I'd still like the automated tests tho
# cwebber now to get back to cleaning some stuff up... gonna add the debugging feature puckipedia asked for and also fix the websocket timeout issue
# puckipedia yay :)
# cwebber nightpool: aaronpk: btw I still ack that it's another layer to add, but one thing I forgot to add is, depending on the http
# cwebber er
# cwebber on the did method you use, it may be http
# cwebber veres one is, and iirc there are ipfs network layers that are
# cwebber but definitely that's not true for bitcoin.
# cwebber also true probably if using a resolver
# cwebber nonetheless even if that's true ack that it's a new abstraction layer to absorb
eugr joined the channel
# eugr hm
# eugr hey guys
# eugr so i'm working on cobalt, my take on mediagoblin territory. i'm still a bit unsure about how to schema the storage for AP stuff
# cwebber hi hi eugr
# cwebber want my opinion on how I would have done it if I started mediagoblin over?
# eugr how have y'all done it on your end? i want to try a different approach to mastodon's full normalization
# eugr like, it was such a bother in masto because AP payloads had to be generated on the fly
# eugr so right now i am considering an activities table (account_id:bigint, payload:json)
# eugr in postgres ofc
# cwebber eugr: yeah using jsonb I'm guessing?
# eugr yeah
# cwebber eugr: you'll also want an opportunity for private fields
# puckipedia my plan is to use PostgreSQL as a huge tuple store, but currently I have a table
{ entity_id: string, is_owner: boolean, payload: jsonb, type: string }# eugr wait is json and jsonb different things?
# puckipedia eugr: jsonb uses binary json internally
# cwebber eugr: jsonb is waaay better than json
# cwebber b-trees
# eugr oh okay
# cwebber eugr: as for the private fields, eg a password
# eugr what private fields?
# cwebber password hash
# puckipedia cwebber: tbh, I fixed that by using an out-of-band mechanism for that
# eugr um no why tho. it's just for activities, no?
# eugr i still have an accounts table
# puckipedia eugr: I store *everything* in the object store. each Person is in the store as well
# eugr i don't think i want to do that
# puckipedia actually, the Person gets Created by itself as well :P
# cwebber eugr: ok, what I was going to say is that I'd link in other tables to each of the linked data/ as2 objects depending on their type
# cwebber so for example an Account table may link to the Person recrod
# cwebber record
# cwebber and hold some of the "common" field
# cwebber s
# puckipedia eugr: you're still going to have to be able to process external users
# eugr hm
# cwebber eugr: then some of the data you expand dynamically
# cwebber eugr: here's the other major thing I'd do
# cwebber eugr: you actually want to do content based addressing of objects
# eugr oh boy it's still a mess, no matter which approach you take... normalization at least made a lot of sense to me at the start
# eugr okay so
# puckipedia also my suggestion: expand/re-compact with a specific AP context when storing in the DB
# eugr table "objects" (uri:string payload:jsonb)??
# cwebber eugr: I'd instead do this
# puckipedia eugr: if you want to be able to cache external objects, also a flag is_owner :P
# eugr what is is_owner?
# eugr is that like local:boolean?
# puckipedia yep
# eugr i would call it local :P
# puckipedia so you know what you can evict :P
# cwebber (uri:string cas_object:cas_hash) -> (cas_hash, payload:json)
# puckipedia eugr: tomato, tomato :P
# eugr cas_object? wut
# cwebber eugr: so think of git
# cwebber objects are immutable hashes
# puckipedia hmmm
# eugr eh
# cwebber but each branch points at an object
# puckipedia https://github.com/puckipedia/Kroeg/blob/master/Kroeg.Server/Models/APEntity.cs#L12-L21 for reference, in Kroeg
# cwebber so there's a reason for this
# eugr git repos get really big over time tho exactly for this reason. why would i store outdated versions of objects?
# puckipedia ... it's signatures, isn't it
# cwebber eugr: read this https://github.com/w3c-dvcg/ld-signatures/issues/7#issuecomment-316997653
# puckipedia bingo
# eugr i find this hard to process
# eugr :/
# puckipedia eugr: basically, you can deconstruct incoming objects, and reconstruct them for signatures
# cwebber also hey
# cwebber everyone loves content addressed storage right?
# eugr okay here's a thing
# cwebber dlongley / dlehn: btw how do you store these references in your db, since I think you do that
# cwebber if you're storing your documents, how do you actually save which revision is being pointed to in json
# eugr i need to have some old fashioned models in my application because anything else requires me to code common functionality myself
# eugr because everything expects normal models
# cwebber eugr: you using something like mongo or rails?
# cwebber er
# cwebber django or rails
# cwebber or something else standard-framework like that?
# eugr like, my Video model has an uploader via Shrine
# eugr yes rails
# eugr if i encode that stuff in a weird way i'm not gonna be able to use any existing tooling
# eugr so, i imagined that i would build a local-only application first
# eugr in an old school wa
# eugr y
# puckipedia I use EntityFramework, the standard .NET ORM, but then basically hacked my own data storage on top. so I can easily poke something else (e.g. IPFS) over it
# eugr and then i would add AP on top
# cwebber eugr: if I were you I'd design for federation from the get-go... adding later is always harder
# cwebber eugr: but, it may not be in this case
# cwebber because at least you know what to expect
# cwebber a lot of the pain comes from learning about all the corner cases of federation after the fact
# cwebber of most of your structure being there
# eugr so hooking AP on top means, be able to serialize those models into AP, which i've successfully implemented for Actor and Video
# eugr it also means the fiddly bits, payloads
# eugr like Create, Delete, Add, Remove
# puckipedia you know. I really need to fix Kroeg's IDs
# puckipedia also I shouldn't give Links IDs
# eugr so my outlook so far is
# eugr when certain actions on my models are performed, i need to fire off the correct activitypub activity, and record the json, and send out the json to subscribers
# eugr make sense?
# puckipedia yep
# eugr so i thought my activities table would allow me to create an outbox by simply SELECT * FROM activities WHERE account = ?
# eugr so let's see, how would i change that to incorporate your suggestions
# cwebber eugr: mostly true, though you may develop some things dynamically
# cwebber eg, let's imagine you make a collection
# cwebber is that whole collection stored in your database as one object
# puckipedia eugr: I have another table to map a collection entity to its objects
# cwebber or is it, itself, dynamically generated partially
# cwebber eg imagine you and I are sharing a photos collection
# cwebber do you serialize all that pagination and everything into the json field?
# cwebber probably not, there's probably some dynamic additions
# cwebber *eugr is unhappy with me*
# eugr ughhh
# eugr let me think about this
# cwebber I'm just the messenger don't shoot me!
# eugr let's say i want to add a video to a playlist
# eugr it's an Add activity that references a Video and a Playlist
# cwebber yep
# eugr but it does not have to contain either except for ID
# cwebber right
# cwebber though, you do need to know effectively for signatures a tuple of
# eugr other than that, i don#t think any Collections are actually needed to be stored like that
# cwebber [id to object, revision it was signed with, was it "embedded"]
# cwebber eugr: but what about *the collection itself*
# eugr Outbox is dynamically generated
# cwebber inbox/outbox, obviously those are
# eugr if you access a Playlist, that would be a dynamic model
# cwebber my argument is that's also true for the playlist
# cwebber yes
tantek joined the channel
# eugr the Playlist would reference Video items which are also dynamic
# cwebber eugr: what I'd do is allow each type to provide a method to dynamically expand itself in ruby
# eugr wut
# cwebber ok ignore me :)
# cwebber well
# cwebber I'm just talking about how I'd structure it
# cwebber I think Collections are one clear case but
# cwebber I think there may be others where you may end up dynamically adding some things...
# cwebber but tbh I can't think of them off the top of my head
# puckipedia Person endpoints?
# puckipedia I dynamically generate those
# eugr one thing that bothers me even if i implement this all so far it's local-content-only
# eugr i would need to normalize remote videos and actors into the same dynamic tables
# eugr like on mastodon
# eugr i don't think there's a way around that
# puckipedia do you need all the info a normal ActivityPub actor has
# cwebber eugr: yes there's some more dynamic expansion stuff done on local things only
# cwebber but for remote things you present the object "as you got it"
# eugr ideally i would only re-reference remote objects by ID
# eugr never embedded
# puckipedia I have a flag to do so :P
# cwebber eugr: but what happens if you got an object that was signed tthat itself embedded an object :)
# cwebber then you need to reproduce the embedded object which was as-you-saw-it-when-you-got-it
# cwebber hence CAS
# eugr okay, how do i do CAS
# puckipedia eugr: you hash the objects as they come in, then you store those in one table, then have the metadata (ID, etc) in another table, pointing to a specific hash
# eugr hash how
# eugr sha1
# eugr md5
# eugr something else?
# eugr normalization before?
# puckipedia normalize before, of course. you can use the thing that json-ld signatures does
# eugr how will that actually help me if an object comes in referencing another by ID
# eugr i won't be able to know which hash to use
# puckipedia you only need the older ones to recreate signatures
# eugr yeah how would i recreate the signature?
# puckipedia by looking up the ID, which then returns a list of "it embedded this object, which had this hash, which embedded this object, ..."
# eugr wow ok i dont get this at all after all
# cwebber eugr: if they don't embed it then it doesn't affect what their content was
# eugr i'll do the thing without hashes first
# cwebber eugr: if you think of it in triples
# cwebber if you embed it, the flattened triples will include all the content if it's embedded
# cwebber but if you don't it will just say "it goes to this here..." but say nothing about the contents
# cwebber eugr: it's tricky! sorry for throwing so much at you but you asked what the right way to do it is
# eugr i dunno folks there's not that much to be done with signatures tbh
# cwebber well there's a lot more potential for signatures than just for forwarding objects but I have a feeling if I start talking about it I'm going to look like an even crazier person deep in teh weeds than I already do right now ;)
# eugr i am not building a general purpose server
# eugr like
# eugr this thing is gonna ignore everything things it can do
# eugr *except
# eugr like it's gonna understand Videos, and eventually playlists
# eugr it should ideally understand comments (replies to Videos)
# eugr it would send out non-Video activities so that e.g. mastodon would get notified about new uploads
# eugr but it's not gonna do microblogging
# eugr the home feed will only display video subscriptions
# cwebber ok :)
# cwebber well! you can ignore my recommendations
# cwebber you asked for thoughts on the right structure and I laid forth my tale-of-the-choices-I-wish-I-made ;)
# cwebber but "great is the enemy of good" so!
# eugr oh hey
# eugr what about the @context property
# eugr can children have it?
# eugr should i store it in the objects table
# puckipedia eugr: I would suggest expanding/compacting whatever object you get with your own context
# puckipedia and then you can just skip storing the context
# puckipedia actually, I think you can directly compact any input you get with your own context
# puckipedia so yeah, just that JSON-LD's compact, with e.g. a Mastodon-like @context
# puckipedia this way you can just use default json handling to read it, and not have to worry about someone thinking "eh, I'll use toot:manuallyApprovesFollowers instead of manuallyApprovesFollowers"
# 
Loqi Tantekelik made 1 edit to [[Socialwg/2017-10-31]] https://www.w3.org/wiki/index.php?diff=104850&oldid=104834
# cwebber yeah you can compact to a local context without needing to expand
# cwebber (expansion is automatically done for you as an intermeidate step where necessary)
# eugr so i'm gonna generate UUIDs for activity IDs, i guess
# eugr but they need to be URLs so how should i structure them
# puckipedia whatever you want
# eugr /@:username/:uuid or
# puckipedia I would probably say /@:username/activity/:uuid, so you can have /@:username/video/:uuid etc
# eugr just /payload/:uuid
# eugr ok
# puckipedia eugr: btw, https://github.com/tootsuite/mastodon/pull/5534
# puckipedia took me a while to figure out why my DMs weren't arriving on Mastodon's side
# cwebber I'm doing some goofy experimentation trying ot get the websocket ping stuff working so test.ap.rocks is going up/down for a bit
# puckipedia I've worked around the Mastodon issue on Kroeg, but only on my puckipedia.com instance
# cwebber woo websocket pinging is working
# cwebber let's see how long https://test.activitypub.rocks/ stays alive now ;P
# cwebber I'm gonna go play some Cataclysm. Hopefully in 15mins it's still up :)