Can we estimate the size of the Internet ? If so, what size would it be (in GB, TB or whatever) ?

Comments

• Hackworth@piefed.ca ⁨2⁩ ⁨weeks⁩ ago

  IDC forecasts that the amount of data generated each year will grow to 175 zettabytes by 2025. It further estimates that a total of 22 zettabytes of digital storage will be shipped across all storage media types between 2018 and 2025, with nearly 59 percent of this capacity being provided by the hard drive industry.

  https://en.wikipedia.org/wiki/Zettabyte_Era

  source

  • RememberTheApollo_@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    I wonder how much of this data will be overlapping copies.

    source
  • Buage_@piefed.social ⁨2⁩ ⁨weeks⁩ ago

    Interesting, thanks for your response!

    source
• Zwuzelmaus@feddit.org ⁨2⁩ ⁨weeks⁩ ago

  42 of course.

  source
• AlecSadler@lemmy.dbzer0.com ⁨2⁩ ⁨weeks⁩ ago

  My friend works at AWS and has single customers with hundreds to thousands of petabytes.

  In whole sum, I think it would be pretty unfathomable.

  source

  • bhamlin@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    Yeah we only use fathoms for depth, not data.

    source
• SkyNTP@lemmy.ml ⁨2⁩ ⁨weeks⁩ ago

  Define “Internet” to answer your question more meaningfully.

  Are you counting cached copies of data (Cloudflare etc), or only the original copy? Only the source data or the data copied on the receiving end (webbrowser etc) too? Data at rest on a computer hard drive or data being transmitted or both? Are you only including world wide web (websites) or any data (gaming, streaming, fileshare). Humans or bots or both? Public web or dark web? Any computer physically connected to a network or only those data accessible publicly?

  source

  • Jimmycrackcrack@lemmy.ml ⁨2⁩ ⁨weeks⁩ ago

    Would it be easier to be simply defined as any data on any storage device that is connected to the internet and accessible that way? This would include things such as those CloudFlare cached copies and anything beyond simply WWW and make no distinction between humans and bots or dark or public. I guess that would make it pretty hard to make any even remotely educated guess though since so much it would not be accessible to anyon trying to count it.

    source
• Ceruleum@lemmy.wtf ⁨2⁩ ⁨weeks⁩ ago

  About 64 kilobytes.

  source

  • DebatableRaccoon@lemmy.ca ⁨2⁩ ⁨weeks⁩ ago

    Not 42KB?

    source

    • bhamlin@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

      Nah, 64kb should be enough for anyone.

      source
• Rhynoplaz@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

  Follow up question: what percentage of that do you think is porn?

  source

  • Jackhammer_Joe@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    Not enough

    source
  • edgemaster72@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

    Yes

    source
  • bold_omi@lemmy.today ⁨2⁩ ⁨weeks⁩ ago

    101%

    source
  • MissingGhost@lemmy.ml ⁨2⁩ ⁨weeks⁩ ago

    Way less than 20 years ago.

    source
• slazer2au@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

  This was asked less then a week ago
  lemmy.world/post/53033969

  but depending on where you draw the line on what is the internet then we are likely in the Exobyte territory.

  source

  • Buage_@piefed.social ⁨2⁩ ⁨weeks⁩ ago

    Oh I didn’t see, thanks for letting me know!

    source
• cypherpunks@lemmy.ml ⁨2⁩ ⁨weeks⁩ ago

  lmgtfy? estimates vary a lot of course but currently tend to range from tens to hundreds of zetabytes (10^21^ bytes, or trillions of gigabytes).

  en.wikipedia.org/wiki/Zettabyte_Era

  source

  • Buage_@piefed.social ⁨2⁩ ⁨weeks⁩ ago

    That’s really impressive!

    source
• Ephera@lemmy.ml ⁨2⁩ ⁨weeks⁩ ago

  Something I always find kind of puts the scale into perspective, is that all these AI companies have a huge interest in storing as much of the internet’s data as possible, because they need access for training and it ensures they continue to have access to relatively high-quality (it’s only going to get worse with more and more content on the internet being generated).

  And yet, even with their trillions of investor money, they evidently don’t have it all stored. Because they still regularly hammer all kinds of webpages with an insane number of requests, when they do their trainings.

  source

  • fuckwit_mcbumcrumble@lemmy.dbzer0.com ⁨2⁩ ⁨weeks⁩ ago

    They’re checking the websites for change. They’re apparently too stupid to tell websites that change often from ones that are more static.

    They’re not using it live for training. That’d be far to slow for their training.

    source
  • Buage_@piefed.social ⁨2⁩ ⁨weeks⁩ ago

    That’s a good point! I forgot about AI companies stealing as much as data as they can..

    source
• twjolson@lemmy.world ⁨2⁩ ⁨weeks⁩ ago

  Yes, you can estimate anything.

  source
• unitedwithme@lemmy.today ⁨2⁩ ⁨weeks⁩ ago

  Idk but I’m about to eclipse 200TB myself! 😅

  1. 6x 3TB in RAID10 = 18TB raw (9TB usable)
  2. 10x 8TB in RAID5E = 80TB raw (64TB usable)
  3. 8x 16TB in RAID5 = 128TB raw (112TB usable)

  AKA 226TB raw or 185TB usable

  source
• HubertManne@piefed.social ⁨2⁩ ⁨weeks⁩ ago

  this feels like a perfect llm question. It may have it wrong. It quite likely will be wrong or at least not exact. But it will give you a feel good answer that is going to be correct with enough proximity tolerance.

  source

  • bold_omi@lemmy.today ⁨2⁩ ⁨weeks⁩ ago

    Never slop. Never outsource thought to LLMs.

    source
• schnurrito@discuss.tchncs.de ⁨2⁩ ⁨weeks⁩ ago

  We had the same question just a few days ago: discuss.tchncs.de/post/69097816

  source