Internet Archive Passes 10 Petabytes

Devin Coldewey, NBC News

Internet Archive hits 10 petabytes of saved Net culture
Archive.org
26 OCT 2012

A banner celebrates 10 petabytes of data at an Archive.org event.

There are plenty of things on the Internet today that won't matter tomorrow, but this online world of ours is also a powerful way to store and browse all kinds of data from the past. The non-profit Internet Archive, which is dedicated to preserving all kinds of culture digitally, has recently hit an amazing milestone: 10 petabytes of stored data.

The collection comprises everything from World War II newsreels to classic literature to public domain music, and of course, the famous "Wayback Machine," with which you can visit many popular websites through the years.

This week saw the addition of the 10,000,000,000,000,000th byte to the pile — that's 10 petabytes, 10,000 terabytes, or 10 million gigabytes. That particular byte was likely mixed in with the recent and enormous 80-terabyte crawl of the Web's most popular sites. You could just visit the sites themselves, of course, but researchers looking to systematically analyze the content of millions of webpages can't be sitting around waiting for them to load. Huge downloadable archives like this are invaluable for their purposes.

Archive.org's blog post also notes that they have put up what they describe as "the first complete literature of a people," specifically that of the Balinese. It's an impressive feat and guarantees the survival of these vital historical documents. The effort is reminiscent of another, which is attempting to fully document endangered languages.

Such culturally important projects are a good reminder of the Internet's value apart from news and entertainment.

The Archive is free to browse, of course, and recently added the option to download files via BitTorrent, which should help keep costs down and accessibility up.

Devin Coldewey is a contributing writer for NBC News Digital.

[end snip]

I hope they've got a good index!!!

About a year or so ago, one tech news writer offered his opinion that "nobody could ever have enough data to use a 1 TB drive," and suggested that if you got one of the then new 1 TB ones you'd never have to buy another drive. (He has someone to do backups for him, of course.)

Including backups, I've got 8 TB worth of hard drives in regular use, a little over 2/3 filled, so the organization dedicated to "saving just about everything" has barely 2,000 times what I have (????) at my fingertips. A couple of thousand users with similarly greedy demands for data could match what they've done on purpose????

Considering that I don't have much music, and NO VIDEOS** on my drives, it seems likely that some here probably are using about that much - or at least a third of it if they don't keep backups.

** 2 hours of analog TV (VHS) takes about 5 GB, so 200 movies would fill 1 TB(?). We have (or I could say "she has") about 270 movie DVDs (commercial) and about the same in "home videos" on DVDs, but none that I feel compelled to backup for "her."

Is it really reasonable that they're still selling desktop computers with 100 GB (and smaller) hard drives?

And why don't I get more done now than when I only had 70 MB of hard drives and 640 KB RAM?

One more drive and I could save the MOABS thread!!!!

John