Blocking the Bad Bots, Welcoming the Good Ones
Our client’s website was down. It looked like a denial-of-service attack, but a malicious actor wasn’t responsible for the outage. A bot was. A poorly controlled AI crawler was making hundreds of requests a second for the pages on the site. That’s what crawlers do – they collect data from websites to feed search engines…
