Comment on Performance issues?
acockworkorange@mander.xyz 2 days agoDid you upgrade Lemmy on the 13th or something? Could be a new bug.
Also, did site traffic increase at all? Maybe there’s an AI scraper messing up.
Comment on Performance issues?
acockworkorange@mander.xyz 2 days agoDid you upgrade Lemmy on the 13th or something? Could be a new bug.
Also, did site traffic increase at all? Maybe there’s an AI scraper messing up.
Sal@mander.xyz 2 days ago
I did not update or change anything in the past few days.
But, now that you mentioned an AI scrapper I looked into the logs and noticed some heavy requests to the API from a specific IP.
Requests look consistent to scraping - just consistently and continuously issuing GET requests to different API endpoints.
I have started denying their requests and it is the first thing that seems to have actually helped!
I don’t want to speak too early but I think you may have identified the cause. Thanks!
Sal@mander.xyz 2 days ago
I just realized that it is not a ‘scraper’, the requests came from the server that I am using to provide an interface to the site as an Onion site. The amount of requests was suspiciously high so maybe a bot is scraping through Tor. I will leave it off for a few days and see if I can turn it back on later.
acockworkorange@mander.xyz 2 days ago
That’s a level of sophistication I hadn’t seen - using Tor to scrape. Hopefully they give up soon.