I loved scraping until my ip was blocked for botting lol. I know there’s ways around it it’s just work though
Comment on Chad scraper
bill_1992@lemmy.world 1 year ago
Everyone loves the idea of scraping, no one likes maintaining scrapers that break once a week because the CSS or HTML changed.
camr_on@lemmy.world 1 year ago
pennomi@lemmy.world 1 year ago
I successfully scraped millions of Amazon product listings simply by routing through TOR and cycling the exit node every 10 seconds.
camr_on@lemmy.world 1 year ago
That’s a good idea right there, I like that
AlecStewart1st@lemmy.world 1 year ago
This guy scrapes
aBundleOfFerrets@sh.itjust.works 1 year ago
lmao, yeah, get all the exit nodes banned from amazon.
pennomi@lemmy.world 1 year ago
That’s the neat thing, it wouldn’t because traffic only spikes for 10s on any particular node. It perfectly blends into the background noise.
Touching_Grass@lemmy.world 1 year ago
You guys use IP’S?
camr_on@lemmy.world 1 year ago
I’m coding baby’s first bot over here lol, I could probably do better
synae@lemmy.sdf.org 1 year ago
Token ring for me baybeee
dangblingus@lemmy.world 1 year ago
Or in the case of wikipedia, every table on successive pages for sequential data is formatted differently.
Matriks404@lemmy.world 1 year ago
Just use AI to make changes ¯_(ツ)_/¯
anarchy79@lemmy.world 1 year ago
Here take these: \\
Matriks404@lemmy.world 1 year ago
¯_(ツ)_/¯\\ Thanks
DigitalPaperTrail@kbin.social 1 year ago
spite can be a great motivator, though
Anonymousllama@lemmy.world 1 year ago
This one. One of the best motivators. Sense of satisfaction when you get it working and you feel unstoppable (until the next subtle changes happens anyway)
archomrade@midwest.social 1 year ago
I feel this