#dev 2020-05-22

2020-05-22 UTC
#
[schmarty]
lol that browsers don't support Atom/RSS as if they aren't XML
#
KartikPrabhu
is confused about "browsers support JSON but not Atom/RSS"? what's that all about?
[tw2113] and gRegorLove__ joined the channel
#
[tw2113]
i mean, i visited [tantek]’s feed URL in Firefox and see this https://cloudup.com/cVbIv25il1z which looks pretty well supported in some way to me
#
[tw2113]
chrome doesn’t render a nice UI like that, it shows plaintext, but that’s about the same as JSON anyway in that it shows plain text
gxt__ joined the channel
#
KartikPrabhu
right, so browsers support Atom/rss as well as they do JSON
#
[tw2113]
I’d argue that Firefox does miles better with both formats, cause i see https://cloudup.com/cpIht5Sb1-b for JSON
#
[tw2113]
your move, Google
#
KartikPrabhu
yes it does. But the point being it is not Atom/RSS vs JSON
#
KartikPrabhu
also FF only does the nice display if the Atom/RSS has some styling information
#
KartikPrabhu
for example it does not render my Atom feed
geoffo and [kimberlyhirsh] joined the channel
xsteadfastx and [KevinMarks] joined the channel
#
[KevinMarks]
Browsers used to support rss/atom
#
[KevinMarks]
As far as I know they've never supported JSON-LD
#
[KevinMarks]
A declarative playlist as part of html would be good.
#
[KevinMarks]
Yes we have the vestiges of SMIL but that got lost in version 2
#
[KevinMarks]
Buried deep away from users is support for the chained m3u stuff for live streaming, but there's no real visibility into it
#
KartikPrabhu
h-feed + audio/video post
#
KartikPrabhu
I don't see why playlists are different from image/photo stream markup
xsteadfastx, [tantek], gRegorLove__ and [snarfed] joined the channel
#
[snarfed]
jacky just fyi, looks like your jsonfeed (eg https://v2.jacky.wtf/feeds/json/all ) may have a bug. content_html and content_text should be strings, not lists of strings. https://jsonfeed.org/version/1#items
xsteadfastx joined the channel
#
jacky
thank you [snarfed]
#
dansup
I had to disable federation support on pixelfed.social because of a DoS like federation pattern of huge bursts of activities. Watching the access log was wild
#
dansup
Mastodon has a circuit breaker that has exponential backoff, which means you see huge bursts from remote servers retrying delivery. Its been a rough day
[schmarty] joined the channel
#
Loqi
[valentin2105] #4486 Mastodon can be used as a DDOS tool
#
dansup
The biggest issue is that we validate http signatures in inbox requests, so I moved that to a queue job so bursts wont bring everything down
#
dansup
[snarfed]: yeah that has affected my @pixelfed account, it has over 5k followers. Fetching is easier to handle than inbox POSTs
#
dansup
gargron has a lot of followers, but most are on mastodon.social. If there was 400k followers with their own instance, I think even masto.social would have problems with fanout
#
[schmarty]
lol some of the comments in that thread! I found myself angrily adding thumbs-down emojis
#
dansup
yeah, I envy laravel who disabled issues in their main repo
#
Loqi
[laravel] laravel: A PHP framework for web artisans
#
[snarfed]
yeah a lot of tech-superiority "you're holding it wrong" comments in that issue. and then fortunately also actual maintainers disagreeing with them and trying to help
#
jacky
how do they broker discussion? do they have forums?
#
Loqi
[pixelfed] ideas: Issues board used for Pixelfed suggestions
#
dansup
they used to close a lot of issues without explanation, now they redirect users to forums or discord for common questions/support
#
[schmarty]
dansup: so does this traffic appear to be legit, just overly intensive due to resolving signatures?
#
dansup
its legit, pixelfed.social has a lot of fedi connections, with over 10k users it gets a lot more traffic than I expected. 400k requests in 4 hours
#
dansup
jacky: fediDB is really coming along, today a pleroma dev tested it and they are impressed. Can't wait to open the public beta next week! 😁 https://fedidb.org/rb/mgc4EcwQvSbXhxZBbW9W
gRegorLove_ joined the channel
#
[schmarty]
Do you think the signature checking is mostly network-bound with key fetching? is it also a big CPU hit?
[prtksxna] joined the channel
#
dansup
its both, especially for blind signature rotation, if the first validation fails it will refetch the key and check again
#
dansup
We do cache the key, but if it fails we will refetch in case its been changed
#
dansup
the lack of sharedInbox is the biggest culprit, I'm working on that now
#
[schmarty]
Makes sense. Also, yeowch!
#
dansup
Yeah, thankfully I was able to disable federation support, its kind of ironic that its only affecting the biggest pixelfed instance
treora, gRegorLove_ and [LewisCowles] joined the channel
#
[tantek]
lol [schmarty] I had to go look at the issue to see which comments you thumbs downed
#
Loqi
[ashfurrow] > Can the sending server be trusted about that? Yeah that's for sure been my concern. We've seen instances running malicious code modify payloads when federating before.
#
[tantek]
"modify payloads when federating" (!!!) <--- yeah that might be the shortest summary critique of the "pump" model that I've seen (in contrast to WebSub style notify + fetch original)
strugee joined the channel
#
[schmarty]
For what it's worth, websub + fetch original could also lead to thundering herd issues. I think that's (part of) why websub has the option to deliver the whole content to subscribers?
[KevinMarks], nickodd, swentel, gxt__, moppy and jeremych_ joined the channel
#
jeremycherfas
Does anyone here know that feed parser https://github.com/kurtmckee/feedparser ?
#
Loqi
[kurtmckee] feedparser: Parse feeds in Python
[LewisCowles] joined the channel
#
jeremycherfas
After months of waiting, I finally got a reply from the developer of Newsblur, about adding h-feed support, and he suggested taking it up with the developer of the parser he uses. But I'm not sure I know how best to phrase a request or what it might take. It would be great if someone could take a look and advise me.
#
jeremycherfas
Looks like the issue has already been raised and closed https://github.com/kurtmckee/feedparser
#
Loqi
[kurtmckee] feedparser: Parse feeds in Python
#
jeremycherfas
!tell kevinmarks have you seen any further progress on kurtmckee's feedparser?
#
Loqi
Ok, I'll tell them that when I see them next
#
jeremycherfas
!tell ben_thatmustbeme I just got a somewhat lackadaisical reply from Samuel Clay to the continuing saga of trying to get Newsblur to parse h-feed. I guess you got it too. Is there any point in keeping the issue alive?
#
Loqi
Ok, I'll tell them that when I see them next
loicm and [KevinMarks] joined the channel
#
[KevinMarks]
I haven't, I forked it just before he started deleting things and splitting it into multiple files
#
Loqi
[KevinMarks]: jeremycherfas left you a message 1 hour, 11 minutes ago: have you seen any further progress on kurtmckee's feedparser?
#
[KevinMarks]
I then spaced on making it parse h-feed myself
[Murray] joined the channel
#
[Murray]
Having a weird issue when reading my own RSS feeds with The Old Reader (I subscribe to make sure they're working). Seems each time the feed reader checks my URL it pulls back 10 posts, no matter if anything new has been posted. My feeds are quite new but there's 5 years content, so there's plenty of "backlog" for it to pull in. It's effectively slowly backfilling my feed lists within the reader, which is not behaviour I've ever seen before.
#
[Murray]
Anyone have any idea what might be causing it? I'm using Gatsby, and given everything going on right now spending a bit of time working on the site, so there are plenty of rebuilds, but the RSS feed structure itself seems consistent. I'm also subscribed to other feeds using the exact same setup but I'm not seeing similar behaviour, so think it must be something on my end.
#
[KevinMarks]
Did you paginated the feeds with next/prev links
#
[Murray]
No, they're just one long list
jeremych_ joined the channel
#
jeremycherfas
There just doesn't seem to be any interest from Newsblur in doing h-feeds. And yet, because it has Python plugins for sharing, it would help to turn it into a useful social reader.
dckc, KartikPrabhu, garrettw_, loicm, geoffo, [LewisCowles], jolvera, [jgmac1106], nickodd and [KevinMarks] joined the channel
#
[KevinMarks]
↩️ *it has seen
#
[KevinMarks]
[Murray] it may well be that your Reader is gradually adding posts to its archive, and doing so in batches, so each time it gets to your feed it skips the ones it he's
gRegorLove, [email096], swentel, [LewisCowles], [snarfed] and betodealmeida joined the channel; nickodd left the channel
#
[LewisCowles]
So I'm looking at this WithKnown upload. Maybe I just need to code some things and go through some pain, but I think I'll make something separate to Known first, and then just see if my implementation is effective.
#
[LewisCowles]
I think every upload chunk needs:
#
[LewisCowles]
• a unique invariant (could be SHA of upload as calculating needs knowledge of the file)
#
[LewisCowles]
• sequence of chunk (unless sequential transfers had to be guaranteed)
#
[LewisCowles]
• size of chunk (maybe)
#
[LewisCowles]
• storage for the data
#
[LewisCowles]
• . Also needs some shared scratch space to avoid asking / storing n places to
swentel, flex14, KartikPrabhu, [chrisaldrich] and gRegorLove joined the channel
#
GWG
Going to try to add activity Icons based on Compass (the project) data before I leave the house next... which I am planning
#
GWG
So, if you see a post with a little car or walking man...
#
GWG
I'm doing that
[dmitshur], [snarfed] and [Murray] joined the channel
#
[Murray]
[KevinMarks] Yeah that makes sense, though it's odd I've never seen it before with any other feeds. I've reached out to their support just in case.
[tw2113] and [tantek] joined the channel