#dev 2020-05-22
2020-05-22 UTC
# 
[schmarty] lol that browsers don't support Atom/RSS as if they aren't XML
# 
KartikPrabhu is confused about "browsers support JSON but not Atom/RSS"? what's that all about?
[tw2113] and gRegorLove__ joined the channel
# 
[tw2113] i mean, i visited [tantek]’s feed URL in Firefox and see this https://cloudup.com/cVbIv25il1z which looks pretty well supported in some way to me
gxt__ joined the channel
# KartikPrabhu right, so browsers support Atom/rss as well as they do JSON
# [tw2113] I’d argue that Firefox does miles better with both formats, cause i see https://cloudup.com/cpIht5Sb1-b for JSON
# KartikPrabhu yes it does. But the point being it is not Atom/RSS vs JSON
# KartikPrabhu also FF only does the nice display if the Atom/RSS has some styling information
# KartikPrabhu for example it does not render my Atom feed
geoffo and [kimberlyhirsh] joined the channel
xsteadfastx and [KevinMarks] joined the channel
# 
[KevinMarks] Browsers used to support rss/atom
# [KevinMarks] As far as I know they've never supported JSON-LD
# [KevinMarks] A declarative playlist as part of html would be good.
# [KevinMarks] Yes we have the vestiges of SMIL but that got lost in version 2
# [KevinMarks] Buried deep away from users is support for the chained m3u stuff for live streaming, but there's no real visibility into it
# KartikPrabhu h-feed + audio/video post
# KartikPrabhu I don't see why playlists are different from image/photo stream markup
xsteadfastx, [tantek], gRegorLove__ and [snarfed] joined the channel
# [snarfed] jacky just fyi, looks like your jsonfeed (eg https://v2.jacky.wtf/feeds/json/all ) may have a bug. content_html and content_text should be strings, not lists of strings. https://jsonfeed.org/version/1#items
xsteadfastx joined the channel
# dansup I had to disable federation support on pixelfed.social because of a DoS like federation pattern of huge bursts of activities. Watching the access log was wild
# dansup Mastodon has a circuit breaker that has exponential backoff, which means you see huge bursts from remote servers retrying delivery. Its been a rough day
[schmarty] joined the channel
# [schmarty] Oh wow
# [snarfed] reminds me ofhttps://github.com/tootsuite/mastodon/issues/4486
# dansup The biggest issue is that we validate http signatures in inbox requests, so I moved that to a queue job so bursts wont bring everything down
# dansup [snarfed]: yeah that has affected my @pixelfed account, it has over 5k followers. Fetching is easier to handle than inbox POSTs
# dansup gargron has a lot of followers, but most are on mastodon.social. If there was 400k followers with their own instance, I think even masto.social would have problems with fanout
# [schmarty] lol some of the comments in that thread! I found myself angrily adding thumbs-down emojis
# dansup yeah, I envy laravel who disabled issues in their main repo
# dansup also https://github.com/laravel/framework
# dansup we have something similar, https://github.com/pixelfed/ideas && https://github.com/pixelfed/support
# dansup they used to close a lot of issues without explanation, now they redirect users to forums or discord for common questions/support
# [schmarty] dansup: so does this traffic appear to be legit, just overly intensive due to resolving signatures?
# dansup its legit, pixelfed.social has a lot of fedi connections, with over 10k users it gets a lot more traffic than I expected. 400k requests in 4 hours
# [schmarty] whew!
# dansup jacky: fediDB is really coming along, today a pleroma dev tested it and they are impressed. Can't wait to open the public beta next week! 😁 https://fedidb.org/rb/mgc4EcwQvSbXhxZBbW9W
gRegorLove_ joined the channel
# [schmarty] Do you think the signature checking is mostly network-bound with key fetching? is it also a big CPU hit?
[prtksxna] joined the channel
# 
[prtksxna] 🙌
# dansup its both, especially for blind signature rotation, if the first validation fails it will refetch the key and check again
# dansup We do cache the key, but if it fails we will refetch in case its been changed
# dansup the lack of sharedInbox is the biggest culprit, I'm working on that now
# [schmarty] Makes sense. Also, yeowch!
# dansup Yeah, thankfully I was able to disable federation support, its kind of ironic that its only affecting the biggest pixelfed instance
treora, gRegorLove_ and [LewisCowles] joined the channel
strugee joined the channel
# [schmarty] For what it's worth, websub + fetch original could also lead to thundering herd issues. I think that's (part of) why websub has the option to deliver the whole content to subscribers?
[KevinMarks], nickodd, swentel, gxt__, moppy and jeremych_ joined the channel
# 
jeremycherfas Does anyone here know that feed parser https://github.com/kurtmckee/feedparser ?
[LewisCowles] joined the channel
# jeremycherfas After months of waiting, I finally got a reply from the developer of Newsblur, about adding h-feed support, and he suggested taking it up with the developer of the parser he uses. But I'm not sure I know how best to phrase a request or what it might take. It would be great if someone could take a look and advise me.
# jeremycherfas Looks like the issue has already been raised and closed https://github.com/kurtmckee/feedparser
# jeremycherfas !tell kevinmarks have you seen any further progress on kurtmckee's feedparser?
# jeremycherfas !tell ben_thatmustbeme I just got a somewhat lackadaisical reply from Samuel Clay to the continuing saga of trying to get Newsblur to parse h-feed. I guess you got it too. Is there any point in keeping the issue alive?
loicm and [KevinMarks] joined the channel
# [KevinMarks] I haven't, I forked it just before he started deleting things and splitting it into multiple files
# [KevinMarks] I then spaced on making it parse h-feed myself
[Murray] joined the channel
# 
[Murray] Having a weird issue when reading my own RSS feeds with The Old Reader (I subscribe to make sure they're working). Seems each time the feed reader checks my URL it pulls back 10 posts, no matter if anything new has been posted. My feeds are quite new but there's 5 years content, so there's plenty of "backlog" for it to pull in. It's effectively slowly backfilling my feed lists within the reader, which is not behaviour I've ever seen before.
# [Murray] Anyone have any idea what might be causing it? I'm using Gatsby, and given everything going on right now spending a bit of time working on the site, so there are plenty of rebuilds, but the RSS feed structure itself seems consistent. I'm also subscribed to other feeds using the exact same setup but I'm not seeing similar behaviour, so think it must be something on my end.
# [KevinMarks] Did you paginated the feeds with next/prev links
jeremych_ joined the channel
# jeremycherfas There just doesn't seem to be any interest from Newsblur in doing h-feeds. And yet, because it has Python plugins for sharing, it would help to turn it into a useful social reader.
dckc, KartikPrabhu, garrettw_, loicm, geoffo, [LewisCowles], jolvera, [jgmac1106], nickodd and [KevinMarks] joined the channel
# [KevinMarks] ↩️ *it has seen
# [KevinMarks] [Murray] it may well be that your Reader is gradually adding posts to its archive, and doing so in batches, so each time it gets to your feed it skips the ones it he's
gRegorLove, [email096], swentel, [LewisCowles], [snarfed] and betodealmeida joined the channel; nickodd left the channel
# [LewisCowles] So I'm looking at this WithKnown upload. Maybe I just need to code some things and go through some pain, but I think I'll make something separate to Known first, and then just see if my implementation is effective.
# [LewisCowles] I think every upload chunk needs:
# [LewisCowles] • a unique invariant (could be SHA of upload as calculating needs knowledge of the file)
# [LewisCowles] • sequence of chunk (unless sequential transfers had to be guaranteed)
# [LewisCowles] • size of chunk (maybe)
# [LewisCowles] • storage for the data
# [LewisCowles] • . Also needs some shared scratch space to avoid asking / storing n places to
swentel, flex14, KartikPrabhu, [chrisaldrich] and gRegorLove joined the channel
[dmitshur], [snarfed] and [Murray] joined the channel
[tw2113] and [tantek] joined the channel