EN World remains calm amidst a plague of AI bot swarms
The longstanding tabletop news site and forum is periodically shutting down access to prevent overloads and potentially deter data-hungry scrapers.
On October 5, EN World shut down. Not for long — it was back up by the morning of October 6, but it would be closed to all but logged-in readers two days later, the morning of October 8. EN World, the long-running outlet/forum run by Russ Morrissey (also known by online handle Morrus) fully restored public access later in the day, posting an excellent piece about Morrissey’s visit to CERN, where he was able to talk to the scientists about playing tabletop games and experimenting with antimatter.
The cause of all the shut downs, semi-closures, and login walls? Bots. “We're seeing increasingly large waves of bots, largely composed of AI scrapers descending on the site en-masse,” Morrissey told Rascal via email. “A couple of weeks ago, the server was down at about 9am (UK time). At that time, traffic is usually fairly low, as the US is still asleep, so I'd have expected to see maybe a couple hundred registered users browsing and a few thousand guests. Instead, what I saw was 18(!) humans and 90,000 bots. And I don't think those 18 humans were seeing much.”
These bots are not looking for anything specific; they are simply scraping large caches of writing to add to their dataset,a banal evil without prejudice or malice. According to Morrissey, this isn’t a new problem for EN World. Bots typically occupy space on the forums and news articles, scraping additions by both writers and users. But Morrissey isn’t sure why these scrapers have suddenly become more aggressive over the past month. “It's been happening for a long time. Months at least, maybe a year or so. Earlier on it was manageable. In the last month, the sheer volume has increased to a degree that we don't have the resources to cope with [it],” he explained. “This week it was 150,000. Last night, as I type this, it was nearly 200,000. These bots overwhelm the server's resources, making it impossible for actual humans to access it, and eventually crash it.”
Websites can deploy limited techniques to prevent AI bots from completely disabling the functionality of the site through sheer numbers. “We can try to block them by IP, but as I understand it from our server admin, they are constantly switching IPs to bypass the blocks,” Morrissey wrote. While the AI swarm “certainly feels” similar to DDoS attacks, Morrissey described the coordinated data scraping as essentially having the same effect: “Taking the site down, while scraping up its content, and then serving it up to their own users without sending us any traffic… It feels like it should be illegal.”
The bots aren’t affecting server fees, at least — intentionally shutting down the site allows EN World to stop serving content to the bots, which prevents them from accessing, scraping, tapping, and sharing both news stories and user-generated forum content. “Ironically,” Morrissey wrote, the fees might be affected “if there were fewer of them… We have our own co-located dedicated server, so we're lucky in that sense, but it does mean we also can't just draw on other resources… The main issue is database connections and processing power. Having your site down isn't too much of a bandwidth problem, so it's more about the hardware and software just being able to cope with it.”
When asked what the plan was for stymying the AI swarm, Morissey said that he’s still looking at options. Currently, shutting down the site is the easiest way to defer an attack. “Maybe that will discourage them if we're not a fruitful source of scrapable content. But having a closed site is somewhat counter-productive and defeats the aim of having a site at all.” He wrote that while requiring registered users to access the site was “better than nothing,” he admitted that “site logins aren't that much of a long-term defense against bots — they'll figure it out.”
Ultimately, Morrissey isn’t too worried. While there’s no immediate solution, he and the ENWorld team are testing out different ways to shut down the attacks and keep the site both safe and usable for the thousands of TTRPG forum visitors who discuss gaming news, new titles, and everything in between. “We'll get there,” Morrissey assured Rascal. “We've been here 26 years now and survived all sorts of catastrophes and massive shifts in the online landscape. We've overcome them, and we'll overcome this.”
