Submitted 5 hours ago by Buage_@piefed.social to [deleted]
I already know it’s not that easy and that the Internet is constantly expanding, but I really wonder if it’s possible to very roughly estimate its size
this feels like a perfect llm question. It may have it wrong. It quite likely will be wrong or at least not exact. But it will give you a feel good answer that is going to be correct with enough proximity tolerance.
IDC forecasts that the amount of data generated each year will grow to 175 zettabytes by 2025. It further estimates that a total of 22 zettabytes of digital storage will be shipped across all storage media types between 2018 and 2025, with nearly 59 percent of this capacity being provided by the hard drive industry.
Interesting, thanks for your response!
42 of course.
Follow up question: what percentage of that do you think is porn?
Not enough
Yes
Something I always find kind of puts the scale into perspective, is that all these AI companies have a huge interest in storing as much of the internet’s data as possible, because they need access for training and it ensures they continue to have access to relatively high-quality (it’s only going to get worse with more and more content on the internet being generated).
And yet, even with their trillions of investor money, they evidently don’t have it all stored. Because they still regularly hammer all kinds of webpages with an insane number of requests, when they do their trainings.
They’re checking the websites for change. They’re apparently too stupid to tell websites that change often from ones that are more static.
They’re not using it live for training. That’d be far to slow for their training.
That’s a good point! I forgot about AI companies stealing as much as data as they can..
This was asked less then a week ago
lemmy.world/post/53033969
but depending on where you draw the line on what is the internet then we are likely in the Exobyte territory.
Oh I didn’t see, thanks for letting me know!
lmgtfy? estimates vary a lot of course but currently tend to range from tens to hundreds of zetabytes (10^21^ bytes, or trillions of gigabytes).
That’s really impressive!
SkyNTP@lemmy.ml 45 minutes ago
Define “Internet” to answer your question more meaningfully.
Are you counting cached copies of data (Cloudflare etc), or only the original copy? Only the source data or the data copied on the receiving end (webbrowser etc) too? Data at rest on a computer hard drive or data being transmitted or both? Are you only including world wide web (websites) or any data (gaming, streaming, fileshare). Humans or bots or both? Public web or dark web? Any computer physically connected to a network or only those data accessible publicly?