Comment on Chad scraper

• anteaters@feddit.de ⁨1⁩ ⁨year⁩ ago

  Mind blowing stuff

  source
• madcaesar@lemmy.world ⁨1⁩ ⁨year⁩ ago

  No I mean more what is the use case where it would be worth scrapping on a massive scale?

  source

  • oatscoop@midwest.social ⁨1⁩ ⁨year⁩ ago

    When the data is on multiple sites or sources.

    API licenses can be expensive, and some sources might not even have an API.

    source

    • madcaesar@lemmy.world ⁨1⁩ ⁨year⁩ ago

      I get the concept but a concrete example. What company could possibly want to pay for scraping a site?

      Some dude as a hobby I get it, but what, like Amazon will pay some guy to scrape competition prices or something?

      source

      • Sotuanduso@lemm.ee ⁨1⁩ ⁨year⁩ ago

        Maybe you’ve got a small company involved in toy buying and reselling, and they want to scrape toy postings from ebay etc. so that they can scroll through a database of different postings and sort it by price or estimated profit or whatever.

        source
      • oatscoop@midwest.social ⁨1⁩ ⁨year⁩ ago

        I can’t imagine data scraping is something companies will quickly admit to, considering the legal issues involved. It was also the norm for a long time – APIs for accessing user generated data is a relatively new thing.

        As for a concrete example: companies using chatGPT. A lot of useful data comes from scraping sites that don’t offer an API.

        source