2022-09-10 UTC
jacky, superkuh, geoffo, angelo, nertzy, bterry, mro and tetov-irc joined the channel
mro, geoffo, vilhalmer and walkah joined the channel
# 18:46 Loqi angelo has 12 karma in this channel over the last year (14 in all channels) # 18:47 capjamesg (as in, if this is something people want, I'll publish it on GitHub) # 18:49 angelo i'd be interested in any domains with a representative hcard # 19:20 capjamesg I'm at 7,000 documents and I am still crawling domains whose first character is between A and C. # 19:59 angelo whoa you can pack an entire mirror of a site into a single .warc [jgarber] joined the channel
# 20:04 [jgarber] capjamesg I may have missed this earlier, but what's the IndieWeb search you're working on? jjuran, geoffo and [aciccarello] joined the channel
# 22:14 capjamesg angelo Is indieweb.rocks open to contributions? I'd love to help out how I can. # 22:16 angelo i've wanted to calculate total page weight of indexed sites for a while.. just came up with a wget command to warc a page with its assets and am reporting its file size # 22:16 angelo aaronpk your homepage is clocking in at 75MB; is that right? # 22:17 aaronpk i just checked in chrome and a full load is 6mb for me # 22:19 angelo wget -EHkp aaronparecki.com --warc-file="apk.warc" --no-warc-compression # 22:19 angelo running that will give you a full tree of assets it downloaded # 22:20 angelo i had an embedded youtube iframe /being inserted at page load/ and wget found it and added it to the warc anyway # 22:21 angelo so it's probably some kind of unrealistic upper bound # 22:23 angelo i guess that makes sense.. the feature is for archival purposes, not page weight determination tetov-irc joined the channel
# 22:32 angelo capjamesg that'd be awesome.. let me write some build instructions and you can start by getting it built on your machine geoffo joined the channel
# 22:47 IWDiscordGateway <capjamesg> It does require a cryptocurrency wallet, which I created for the specific purpose of exploring this. # 22:47 IWDiscordGateway <capjamesg> They have interoperable clients built by different people for events, news, and other things. # 22:48 IWDiscordGateway <capjamesg> The main reader is owned by a business but it seems as though anyone can build one. # 22:49 IWDiscordGateway <capjamesg> Cryptocurrency requirements aside, I am intrigued to see different ways of thinking about data ownership for social media. # 22:51 angelo that page-weight lib is exactly what i wanted and is working well for all sites so far.. it gets a permalink on your site just fine but it's hanging on your homepage aaronpk # 22:51 aaronpk my home page uses http long polling to bring in posts in real time # 22:52 aaronpk it's probably unnecessary but it's kind of cool :) if you're on my home page and I post something it'll just show up right at the top # 22:52 angelo it did add a "XHR" section to the data returned for one of your permalinks.. do you use the same tech on individual posts? # 23:04 angelo capjamesg farcaster looks like so many similar projects; what appealed to you? # 23:11 angelo capjamesg i was going to warn you about crawling my site.. i've got a ridiculous number of generated pages in the form of software releases.. someone's crawling them right now.. there's no bottom to that pit # 23:54 Loqi [GoogleChrome] lighthouse: Automated auditing, performance metrics, and best practices for the web.
IWDiscordGateway <capjamesg> angelo Do you have a list of the sites you are crawling?
angelo where there's a will there's a node lib: https://www.npmjs.com/package/page-weight