#dev 2025-04-15

2025-04-15 UTC
troojg, stefen2, sebsel_, hidjy[d]_, aaronpk_, _hidjy[d]_, claudine, sebbu2, okCiel, srushe, immibis, rossabaker, roxwize, capjamesg and rrix joined the channel
#
mandaris https://mandarismoore.com/2025/04/14/updates-to-the-theme-playlab.html
#
mandaris [edit] https://mandarismoore.com/2025/04/14/updates-to-the-theme-playlab.html
nemonical, cow_2001, zoglesby and athenaeryma joined the channel
#
aaronpk some bot is pulling down avatars at like 500 per second
#
aaronpk https://media.aaronpk.com/2025/04/15063756-3589.png
#
capjamesg 😬
#
[tantek] Yikes
#
aaronpk from about 6 different IP addresses, but all coming from https://nerdy.dev/ as a referer
#
capjamesg https://nerdy.dev/ex-googler has 1k likes.
#
capjamesg [edit] https://nerdy.dev/ex-googler has 1k likes.
#
capjamesg And it is pulling from webmention.io links for avatars.
#
capjamesg like https://webmention.io/avatar/cdn.bsky.app/fd6ca4bf7a454acb8e315b7ed59b53e6d7b1ad9322d557e6d69d4081d45f0703.jpg
#
capjamesg [edit] like https://webmention.io/avatar/cdn.bsky.app/fd6ca4bf7a454acb8e315b7ed59b53e6d7b1ad9322d557e6d69d4081d45f0703.jpg
#
capjamesg That post got a lot of traction.
#
aaronpk maybe it's not bots?
#
capjamesg If it is the same six IPs, it feels like it would be?
#
capjamesg Maybe a crawler gone wrong?
#
aaronpk omg so every page load is pulling 400 photos https://media.aaronpk.com/2025/04/15064150-4934.png
#
capjamesg Yeah.
#
aaronpk so probably it's not bots, i'm just seeing only a few IPs because it floods the logs
[snarfed] joined the channel
#
[snarfed] Ugh, sorry
#
[snarfed] Why is http://wm.io serving avatars? Shouldn't those be from Mastodon/Bluesky?
#
aaronpk no, it downloads a copy of them, resizes them to a consistent reasonable size, and serves those URLs instead
#
aaronpk hotlinking them from elsewhere was a giant mess
#
[snarfed] Oh wow interesting ok, didn't realize
#
aaronpk they're actually hosted on s3, but I kept the URLs as webmention.io URLs and the server sends a 301 redirect
#
[snarfed] (and agreed, definitely downsides of hotlinking)
#
aaronpk probably what I should do is use a subdomain on webmention.io that's pointed directly at the s3 bucket instead
#
[tantek] That image needs to go on /facepile 😆
nemonical joined the channel
#
aaronpk alright, the API now returns avatar URLs on a subdomain which is pointed to a cloudfront hosted website in front of the s3 bucket, so future crawls of the api will mean people archive the new URLs
#
aaronpk all so that my logs don't get filled with 301 redirect requests 😂
#
aaronpk omg the nerdy.dev site is already serving the new urls
#
aaronpk it must be loading the webmention api on each pageload too 😂
duanin2 joined the channel
#
aaronpk haha yeah when i visit the page, i see a request to webmention.io from a Deno user agent, which means his server is pulling down the api request each time
#
aaronpk well cool that means my server is out of the hot water for now
#
capjamesg aaronpk++
#
Loqi aaronpk has 57 karma in this channel over the last year (131 in all channels)
[schmarty] joined the channel
#
[schmarty] aaronpk++
#
Loqi aaronpk has 58 karma in this channel over the last year (132 in all channels)
#
[schmarty] CDN++
#
Loqi CDN has 1 karma over the last year
#
[schmarty] i was pretty impressed with this video on making a simple resizing/reformatting image proxy in about 20 minutes in PHP/Laravel. https://www.youtube.com/watch?v=lq_YlAOoLT8
#
aaronpk oh yeah this isn't just a resizing proxy, it archives the file based on the content hash, so if the file at the URL ever changes the new file gets a new archived URL
#
[schmarty] heck yeah! i didn't mean to imply anything about wmio's image proxy, just ADHD-brain pulling up this related thing i watched yesterday. 😂
#
[schmarty] i absolutely want to set up something like you described for the indieweb webring
#
aaronpk sometimes i am amazed that webmention.io still runs
#
[artlung] aaronpk++ fast fixing! amazing indeed
#
Loqi aaronpk has 59 karma in this channel over the last year (133 in all channels)
#
aaronpk it's almost 13 years old now
jacensekai joined the channel
#
aaronpk yay, requests have dropped off https://media.aaronpk.com/2025/04/15081548-6426.png
#
capjamesg Wow, that is a lot of requests±!
barnaby and [KevinMarks] joined the channel
#
[KevinMarks] I had to nuke http://svgshare.com after Google stopped honouring daily billing limits and I got a $900 hosting bill for some crawlers
ttybitnik joined the channel
#
aaronpk whoa
[Murray], jjuran_, ttybitnik, thegreekgeek, [aciccarello], [Joschi_Kuphal], nemonical, thegreekgeek_, gRegor, [tantek] and [tw2113] joined the channel
#
[KevinMarks] hmmm https://bsky.app/profile/mackuba.eu/post/3lmv3kwdvwc2c