#dev 2026-11-07
2026-11-07 UTC
Xlongdivision joined the channel
troojg, rMdes, GWG, CRISPR, Xlongdivision, grufwub, k3can81, oxtyped and Maxpm joined the channel
# 
mcepl aaronpk: https://spectrum.ieee.org/ai-in-cybersecurity-php2shell-vulnerability … I think that is really the point: we are now forced to pay 40 years of accumulated technical debt. Do you remember all those "// TODO Fix later" comments, all those decisions “This is just a small project, PHP is good enough.”? And now we have Wordpress (and Wikipedia).
chimo, win0err, barnaby, deathcomestotime, rMdes-bnc, [burgeonlab], [trovster], dustinm`, troojg, bterry, [snarfed] and [pcarrier_com] joined the channel
# 
WinnerWind [pcarrier_com]: re: #cafe it's really really difficult to tell what is resource abusive and what isn't resource abusive just by looking at headers
# [pcarrier_com] Then it’s the wrong approach.
# [pcarrier_com] How about you actually track resource usage instead?
# WinnerWind i actually did
# [pcarrier_com] OK?
# WinnerWind i would send a blog post, uh, open it in chromium or something, or ill make a PDF copy
# WinnerWind in case you can't view the link
# WinnerWind the point is that the bots are *relentless*
# [pcarrier_com] And my point is, block things that abuse the service, not things you’re guessing might abuse the service from looking at headers statically?>
# WinnerWind well the alternative is tools like Anubis and Cloudflare Bot Fight Mode which people ALSO have reservations about
# WinnerWind if we had a magic sword that only stabbed things that abused the service and did not harm anyone else, we would use that sword
# WinnerWind that sword does not exist
# [pcarrier_com] Well I feel like you’ve just listed the 6 top UAs
# [pcarrier_com] Think the 6th is OK, so could just automatically block the first 5 and call it a day?
# WinnerWind [pcarrier_com]: as a web admin i should not be expected to be looking at my logs like a hawk
# WinnerWind im a student, i have better things to do
# [pcarrier_com] I literally said “automatically”?
# WinnerWind also, iocaine isn't JUST Looking at UAs
# WinnerWind [pcarrier_com]: sorry, missed that. bear with me here, it's 22:48
# WinnerWind nam shub of enki kind of does that already
# WinnerWind it looks for patterns in the most offending requests and then automatically blocks those
# WinnerWind if im not mistaken, you have a problem with the fact that tools like iocaine and anubis aren't "open web". forgive me, for being cynical, but the open web doesn't really *Exist* anymore. we can be as open as we like, but i don't think any webserver deserves to be hammered for terabytes at a time in some fight to be "open"
# [pcarrier_com] Well, I can’t load https://iocaine.madhouse-project.org/ in brave or google chrome or firefox or curl, so I’m just gonna move on. I’ll keep treating those types of tech as an affront to the open web, but you do you 🙂
# WinnerWind none of your browsers work? really?
# WinnerWind not a single one?
# [pcarrier_com] nope.
# WinnerWind i think that's a problem with your computer, ive never heard of iocaine EVER doing that
# WinnerWind iocaine usually breaks for one browser/OS combination at a time
# [pcarrier_com] Again I’ve already argued for blocking based on traffic patterns, so “don’t tolerate siphoning of terabytes” is moot.
# WinnerWind moot how?
# [pcarrier_com] Moot in that we agree that abusers should be excluded?
# WinnerWind i'd be more than happy to block based on traffic patterns rather than a request by request basis, but unless there's a ready made tool that people can easily enough plug into their reverse proxy setup (like how anubis and iocaine work), it's all theory
# WinnerWind the only reason anubis and by extension iocaine are even talked about is because they *exist*
# [pcarrier_com] well I’d consider building an alternative, but I cannot access the website so I have no idea how they integrate 🙂
# WinnerWind well, the fact that alternative doesn't exist is a sign that either someone should build it, no one has considered it, or people have tried it and failed
# WinnerWind anubis and iocaine integrate in similar ways
# WinnerWind (i have opinions on anubis. id ont think a JS challenge is a particularly user friendly way of protecting websites)
# [pcarrier_com] Well, if I ever hear someone is interested in a solution that blocks solely based on a threshold per ASN+UA+time interval, I’ll build something for them.
# WinnerWind it would be an interesting thing to see, ill be honest
# WinnerWind but it would have to be something better than basic rate limiting
# [pcarrier_com] Elaborate?
# WinnerWind nginx has a rate limiting module that allows you to rate limit if a specific IP targets a server by sending n requests in t minutes
# WinnerWind that can be easily gamed by just sending <n requests in t minutes
# [pcarrier_com] My solution tracks requests and bytes transferred per ASN+UA per interval, and when the volume over the last n intervals exceeds a particular threshold, it blocks all traffic from that ASN+UA.
# WinnerWind a smart bot crawler will just crawl your site less
# [pcarrier_com] Right, if your goal is to fight well-behaved crawlers our interests diverge, sorry.
# WinnerWind i think there are some crawlers i just don't want to see, ever
# WinnerWind and i think some people here share that sentiment, too
# WinnerWind just because a crawler is "well behaved" doesn't mean it has good intentions, perse
deathcomestotime and [artlung] joined the channel
# 
Loqi A curlable URL is one where you can use the curl command line tool to request the URL via http(s), and receive back HTML with visible contents https://indieweb.org/curlable
# 
[artlung] For any given site, are you delivering traffic to entities you would like to is priority one. When entities/user agents use unreasonable amounts of resources, whatever the nature of that, we deploy tooling to slow or prevent such traffic. The problem is that the second part can cause secondary effects on the second goal.
# WinnerWind i just don't see the point in having good faith diplomatic solutions for crawlers when in my experience many of them are anything but
# [pcarrier_com] Well, I literally couldn’t access your blog… Anyway, will publish `blockade` on my GitHub in the next few hours.
# WinnerWind ive even seen crawlers ignore robots.txt like its a suggestion
# WinnerWind yes, i am deeply sorry that you could not access my blog
# WinnerWind that IS a problem, i am not saying it isnt
# WinnerWind but i would rather not degrate my internet connection for my entire family just to let one person access my blog
# WinnerWind i am willing to work with [pcarrier_com] to help me diagnose what exactly is going on so i can send a report
# [pcarrier_com] What happens in #indieweb-cafe stays in #indieweb-cafe, but it was in the context of my calling those types of tools anti-web 🙂
# WinnerWind personally im not sure how i feel about being called a "maniac" for this, felt a bit bad faithed to me, just putting that out there
# [pcarrier_com] Sorry wasn’t directed at the individual using those tools, merely the tool itself.
# [pcarrier_com] To provide more nuance, my response to the suggestion to support iocaine: https://srv.us/gwuwftnokuanm5tvmis5efayum7un536
# WinnerWind what is xmit?
# Loqi It looks like we don't have a page for "xmit" yet. Would you like to create it? (Or just say "xmit is ____", a sentence describing the term)
vikanezrimaya joined the channel
# [pcarrier_com] My hosting platform sorry.
# [pcarrier_com] https://xmit.co
# WinnerWind looks really neat!
# [pcarrier_com] thanks! it’s not a huge community but we’re all happy campers AFAICT
marc_in_- joined the channel
# [pcarrier_com] Well, I have a solution based solely on usage-per-ASN+UA ready to test. GitHub release binaries coming shortly. If anybody wants to give that a go, I’ll be more than happy to assist with the setup, specially if you’re open to contributing docs/a blog post for your particulars 🙂 No need to point it’s relatively easy to circumvent, very much a point in identifying who circumvents how :
# [pcarrier_com] https://github.com/pcarrier/blockade
# [pcarrier_com] It’s a 1MB binary that uses very little memory and doesn’t have any external dependency other than a directory to write to. Sits between an HTTP(S) load balancer/proxy (nginx, caddy, etc.) and your actual HTTP/1 server, whatever it is.
# [pcarrier_com] And you can denylist agents you don’t like WinnerWind 🙂
# [pcarrier_com] Would be happy to denylist all known bad actors out of the box
jone42, deathcomestotim1, pstuifzand, dustinm`, barnaby, rrix, vriska, Elleo, WinnerWind, vikanezrimaya_, ttybitnik, JoshuaMorris and troojg joined the channel