#social 2017-07-14
2017-07-14 UTC
# 
saranix Happy 1.5 billion seconds everyone
# saranix lol
# xmpp-social [ajordan] Hahahahaha
prtksxna and xmpp-social joined the channel
# puckipedia so I was thinking, it'd probably be possible to limit the overhead of string IDs by having a map [uri -> internal id] and another map [internal id -> object data]
# puckipedia e.g. on my test instance two 8-bit ints would be 16 bytes, and the entire two ids concatenated would be 96 bytes
# puckipedia assuming e.g. mastodon.social, and everyone on there follows 10 people and everyone exactly posted the average statuses ... just the table mapping entities to collections would be 2GB
# puckipedia ajordan: I mean, e.g. my collection table is in the format [https://lol.puckipedia.com/users/puckipedia/outbox contains https://lol.puckipedia.com/puckipedia/status/asdfasdf]
# puckipedia if I were to store the ID string separately and give entities an integer primary key I could store it like [12312 contains 898373]
# puckipedia and then of course I could improve efficiency of storing by, just before serializing the data into the table, replacing ID strings with their internal representation (if possible)
# puckipedia hm, could be
# puckipedia well, I would like to figure out if it would give a performance boost (because I don't know how well postgresql likes text/varchar as primary key)
# puckipedia okay :< let's not do that for now
# puckipedia git reset --hard
# puckipedia tbh I just pressed "undo changes" in visual studio
# puckipedia let's build an admin entity info screen
prtksxna joined the channel
# puckipedia . o O ( activitypub over gopher )
prtksxna joined the channel
# puckipedia cwebber2: huh, I looked at init-as-objects.sql in pubstrate
# puckipedia and well, that's also a way to fix it :P
# puckipedia well, I was just wondering, but the idea of having a bigserial id, then the id is just an index inside the jsonb
# puckipedia cwebber2: that depends, I wonder if PostgreSQL can optimize on id retrieval? If I were to do this in Kroeg I'd add a computed column(is that a thing?)
# puckipedia reasoning is mostly because I use an ORM, Entity Framework Core, and hack it to do some jsonb queries
# puckipedia optimally I should make a small query rewriter that can rewrite e.g. .Where(a => a.Data["object"].Contains("https://example.com")) to "where \"SerializedData\" @> '
{"object": "https://example.com"}'::jsonb"# puckipedia so currently on my Update blacklist is attributedTo, created, id, deleted, type, object, and actor
# puckipedia ooh bleh trying to federate with distbin is a clusterfuck
# puckipedia ... Kroeg somehow started to generate its own IDs for distbin items
# puckipedia because they don't have IDs in distbin itself :<
# puckipedia I should probably refuse to generate IDs when flattening remote items
# puckipedia fixed that
# puckipedia ... :D: I broke distbin by replying to a Kroeg post
# puckipedia https://distbin.com/activities/be734dc2-3a3b-4487-a835-6913d9c19cb5.json I can guarantee users/puckipedia has an inbox
# puckipedia it's just that it GETs it with the wrong accept header
# puckipedia application/ld+json; profile="https://www.w3.org/ns/activitystreams#"
# puckipedia well it's a MUST in ActivityPub spec that the Accept header must be correct. it isn't, so..... :PPP
# puckipedia "A key-value pair in the body of a JSON-LD document whose value is null has the same meaning as if the key-value pair was not defined."
# puckipedia well
# puckipedia Q: how can "An ID explicitly specified as the JSON null object, which implies an anonymous object (a part of its parent context)" work then?
# saranix puckipedia: postgres can index on text columns just fine (default is btree hashtable-like index). varchars are crappy for other reasons, avoid them, always use text instead. I didn't understand the other pg ques but also FYI pg has jsonb indexing too
# puckipedia ah, right
# puckipedia huh. id being null is invalid?
# puckipedia ... yuup
prtksxna_ joined the channel
# puckipedia I can read the mastodon activitypub data, and probably even process it
# saranix hmm.. https://www.w3.org/TR/social-web-protocols/ points to http://www.w3.org/TR/activitystreams but it seems there is only https://www.w3.org/TR/activitystreams-core/ and https://www.w3.org/TR/activitystreams-vocabulary/ -- I'd file an issue https://github.com/w3c-social/social-web-protocols/issues but I don't have a github login
# saranix ajordan++
timbl and ben_thatmustbeme joined the channel
# 
ben_thatmustbeme Wouldn't count on postactiv for AP then either
# xmpp-social [ajordan] Wow
# xmpp-social [ajordan] Wouldn't do it myself but I certainly sympathize