Comment

Comment on Black Mirror AI

<- View Parent

Hexarei@beehaw.org ⁨2⁩ ⁨months⁩ ago

Wait what? I am uninformed, can you elaborate on the ScummVM thing? Or link an article?

source

Sort:hotnew top

gaael@lemm.ee ⁨2⁩ ⁨months⁩ ago
From the Fabulous Systems (ScummVM’s sysadmin) blog post linked by Natanox:

About three weeks ago, I started receiving monitoring notifications indicating an increased load on the MariaDB server.

This went on for a couple of days without seriously impacting our server or accessibility–it was a tad slower than usual.

And then the website went down.

Now, it was time to find out what was going on. Hoping that it was just one single IP trying to annoy us, I opened the access log of the day

there were many IPs–around 35.000, to be precise–from residential networks all over the world. At this scale, it makes no sense to even consider blocking individual IPs, subnets, or entire networks. Due to the open nature of the project, geo-blocking isn’t an option either.

The main problem is time. The URLs accessed in the attack are the most expensive ones the wiki offers since they heavily depend on the database and are highly dynamic, requiring some processing time in PHP. This is the worst-case scenario since it throws the server into a death spiral.

First, the database starts to lag or even refuse new connections. This, combined with the steadily increasing server load, leads to slower PHP execution.

At this point, the website dies. Restarting the stack immediately solves the problem for a couple of minutes at best until the server starves again.

Anubis is a program that checks incoming connections, processes them, and only forwards “good” connections to the web application. To do so, Anubis sits between the server or proxy responsible for accepting HTTP/HTTPS and the server that provides the application.

Many bots disguise themselves as standard browsers to circumvent filtering based on the user agent. So, if something claims to be a browser, it should behave like one, right? To verify this, Anubis presents a proof-of-work challenge that the browser needs to solve. If the challenge passes, it forwards the incoming request to the web application protected by Anubis; otherwise, the request is denied.

As a regular user, all you’ll notice is a loading screen when accessing the website. As an attacker with stupid bots, you’ll never get through. As an attacker with clever bots, you’ll end up exhausting your own resources. As an AI company trying to scrape the website, you’ll quickly notice that CPU time can be expensive if used on a large scale.

I didn’t get a single notification afterward. The server load has never been lower. The attack itself is still ongoing at the time of writing this article. To me, Anubis is not only a blocker for AI scrapers. Anubis is a DDoS protection.

source