#dev 2023-06-14
2023-06-14 UTC
ahappydeath, nertzy, [Jo]1, [calumryan]1, [Ana_R]1, [Jo]2, Loqi__, vikanezrimaya, bret, eitilt, sebbu2, omz13, strugee-, sebbu3, strugee_ and Guest3712 joined the channel
# 
Saphire epoch: is there something wrong with nginx? When not on a shared host, at least
Loqi__, sebbu2 and [aciccarello] joined the channel
# 
[aciccarello] [jacky] there's a wm npm package with a cli. Might be useful prior work
Loqi__, [jacky], geoffo, Loqi, win0err, gxt__, jan6, [jeremycherfas], heyyyy and ahappydeath joined the channel
# 
vikanezrimaya TIL postgres supports indexing on jsonb subfields: https://stackoverflow.com/questions/47318001/postgres-create-index-on-attribute-of-attribute-in-jsonb-column
# vikanezrimaya that's nice, because I am kinda thinking of using Postgres for storing data, but I don't want to lose on storing MF2-JSON
pharalia joined the channel
holiday_medley joined the channel
# superkuh The best databased is the file system.
# superkuh -d
[timothy_chambe] joined the channel
# vikanezrimaya superkuh: database is indeed the best filesystem, because it's almost always a first-class citizen in terms of API maturity; but the indexing and searching is somewhat lacking
# vikanezrimaya that's where Postgres may or may not come in. Search by categories? Easy:
# vikanezrimaya `SELECT url FROM vika.mf2_json WHERE mf2['properties']['category'] @> '["IndieWeb"]'::jsonb;`
# vikanezrimaya I just tested this by importing everything I have under /posts/ into a local Postgres instance, and apparently I have 54 posts in my blog tagged with "IndieWeb".
# vikanezrimaya I may or may not need to work on case-folding this query though...
# vikanezrimaya Additionally I may need indexes created for these queries
# [timothy_chambe] Reposting this here in #Dev for any onging chatting 👇
# [timothy_chambe] FYI: not sure when, but once the BlueSky effort sat Federation are solid enough, I am strong considering setting up a Blue Sky "Personal Data Server" which as best I can see, is more or less akin to a Mastodon server. I was planning to do this as "bluesky.indieweb.social" - and it would be an exercise in learning the tech, understanding what if any moderation occurs at the PSD level and to help that platform evolve. It would
# [timothy_chambe] of course explore if there was any facility to bridge the AT Protocol federated world to the ActivityPub world, as we also host the Mastodon based Indieweb.social site, that is now just under 11,000 users.
# [timothy_chambe] The Mastodon server for indieweb.social was born out of discussions in this message group, so seems appropriate to see if this sister-project could launch with members of this community too.
# [timothy_chambe] https://blueskyweb.xyz/blog/5-5-2023-federation-architecture
# [timothy_chambe] If anyone would be interested in admin this with me LMK.
# vikanezrimaya Bluesky has a centralized discoverability layer problem, doesn't it? Searching and indexing is centralized, and without that federation doesn't mean much, as somebody can be easily excluded from the conversation by the search and aggregation layer excluding them from feeds...
# vikanezrimaya Sovereignity on your data is a nice concept, but it has no meaning in conventional social networks as long as the incoming flow of data is still under somebody else's control. That's why we need independent search engines and self-hostable feed readers (such as Aperture, that allows to use Microsub -- or just a simple RSS reader)
# vikanezrimaya I so want a Microsub server for such a long time, but got no time nor energy for implementing it. So I'm stuck with an RSS reader for now...
# vikanezrimaya Note: I just realized that postgres JSON matching means I can match on arbitrary amount of properties, essentially constructing a skeleton of an MF2 thing I want to find:
# vikanezrimaya `SELECT count(*) FROM vika.mf2_json WHERE mf2 @> '
{"type": ["h-entry"], "properties": {"category": ["IndieWeb"]}}'::jsonb;`Gorro_Rojo[theyi, Seirdy_, [0x3b0b], vladimyr, bkil, usop[m], olaf[m], shreyanjain[m], JaeItIts[m] and driftwood[m] joined the channel
geoffo joined the channel
# vikanezrimaya postgres provides so much opportunities for directly querying data... I wish I could expose this power through the Micropub endpoint
# vikanezrimaya I think I just got full-text search working... and it's blazing fast
drmr joined the channel
# vikanezrimaya `CREATE INDEX fulltext ON vika.mf2_json USING GIN (to_tsvector('english', mf2['properties']['content'][0]['html']));` -- this cuts a query on the database from ~90ms to ~2ms, and EXPLAIN shows it using an index search instead of sequential
# vikanezrimaya amazing
jonnybarnes joined the channel
# vikanezrimaya [snarfed]: > at least won't be: considering how much compute and bandwidth will probably be required for that, this will be out of reach for many, and this is an easy target for (effectively) centralizing the data flow.
# vikanezrimaya I do admit that fediverse has problems with discoverability due to how it works and how it refuses to have indexing (I heard fediverse-specific search engines sparked a controversy there)
# vikanezrimaya I personally do experience these problems, as there is no full-text search among content visible to one's instance (only #tags), and even that doesn't seem perfect, because one instance may not see the entire fediverse -- only those parts that ever interacted with it in some way
# vikanezrimaya (for example, if someone follows or is followed by a remote user, or interacted with their posts)
# vikanezrimaya Because of that, the instance one joins is somewhat important if one wants to see a good feed -- it's better to join a bigger instance with a bigger federation graph to get a bigger and more diverse federated timeline... aaaand we're back to square two -- essentially a semi-centralized approach
# vikanezrimaya At least that's how I understand this problem. Please correct me if I'm wrong on some points
# [snarfed] Bluesky goal isn't to somehow let small hobbyists like you or me run a service that indexes and processes an entire social network's data in realtime for a few dollars as a side project. that's impossible. but the goal definitely is to let _multiple independent_ people/organizations do it
# vikanezrimaya Well, are there any organizations that either already run do servers or have committed to run these servers as soon as federation kicks in? Besides the bluesky developers, of course.
# vikanezrimaya I wonder if there's any adoption.
# vikanezrimaya My fediverse bubble seems to have a negative opinion on Bluesky (in part because it's founded by the founder of Twitter)
# vikanezrimaya I wonder if that's entirely justified, or bluesky is worth salvaging in case I am presented with the opportunity to destroy any parts of the Internet I don't like
# [snarfed] bluesky was kicked off by a few people at Twitter, including Jack, but realistically it was _founded_ by Jay Graber. https://snarfed.org/bluesky-corporate-ownership-and-structure
tei_ joined the channel
# vikanezrimaya https://fireburn.ru/posts/kittybox-postgres-experiment-1 -- saved some of my experiments I was running in Postgres into a post for my future reference
tei_ joined the channel
# vikanezrimaya This could probably replace my current filesystem backend if I index things by URLs and not just the primary key (to make fetching posts by pretty URLs faster) and make a separate table for parent-child relations (so I can more easily JOIN on it and not have row-level locks when inserting many posts in a row into a feed)
# 
[tantek] prediction: Bluesky is / will be more indexing/search "friendly" than Mastodon (cultural differences among early adopters), and thus once there are more AT Protocol servers federating with each other, the Bluesky "community" will figure out cross-instance indexing & search, and *that* will draw A LOT of people over (I saw how much search on Twitter impacted adoption there when that happened, even only as a 3rd party service, which
# [tantek] The Mastodon "community" (as much as there is one) will either dig their heels in even deeper about no indexing/search by default, in a very self-defeating way IMO, or there will be a renewed debate and perhaps a fedi-split among search-friendly Masto instances vs anti-indexing/search instances
# vikanezrimaya I will probably join bluesky the moment community-operated non-corporate-controlled indexers with a good view of the network (i.e. a view of the network no worse than the instance controlled by bluesky developers) appear
# vikanezrimaya I don't exactly trust the developers, but thankfully in these scheme I may not need to trust the developers, as long as the underlying protocol is sound (didn't read specs yet, so I don't know)
eitilt joined the channel
[Caleb_Hearth] joined the channel
# [Caleb_Hearth] My goal is to directly federate to AT/Fedi on my personal site and not need to have separate presences.
# [Caleb_Hearth] BF is very nice but my goal is an “in-house” solution, which I’d probably open source.
# [snarfed] I did that a while back for Microsub -> legacy RSS reader services, https://indieweb.org/Baffle , but this would be different
gRegor joined the channel
# [Caleb_Hearth] Sorry work called me away. Yeah I’m not that excited to implement AP myself, especially since I’d like to still be able to use Ivory so I’d need to _also_ implement the Mastodon API at least in part.
# [tantek] I can't decide if I want to bother with "sign up for the https://developer.twitter.com/en/portal/products/free to have it unsuspended."
# capjamesg 😄
geoffo, tei_ and [snarfed] joined the channel
# [snarfed] not with a bang but with a whimper: demoted the Twitter button on https://granary.io/ 's homepage just now
tei_, tei_1, holiday_medley and [benatwork] joined the channel
ahappydeath, gxt__ and gnoo joined the channel