Tech: life durability of memory sticks?????

Subject: Tech: life durability of memory sticks?????
From: GUEST,Mr Red
Date: 27 Sep 11 - 06:13 AM

No not write cycles. Read and lifetimes in general, just sitting powered or unpowered.

OK the application is documenting Buskers in my Town - and giving the files to our museum. I have over 4Gb and counting.

So, (I am an electronic engineer), the question is how long could I anticipate data to remain? The SD or stick would be just sitting in a draw or powered and being read. The museum may have a display PC or PVR cycling.

I will post the "Contact Jugglers" video on MySpace soon - they asked to see it that way.

Stroud is rich in buskers, street traders and canvassers on a Saturday, and the folkie in me is in collecting heavan.

OK. Any clues where I can chase the longevity data? Google is misleading. Wiki better but I struggle.

Mr Red

Subject: RE: Tech: life durability of memory sticks?????
From: GUEST,999
Date: 27 Sep 11 - 06:27 AM

The AData 512MB stick is advertised to hold data for 10 years.

Some other sites say much the same about various USB devices. IMO, if the stuff is important, get it on paper that hasn't been bleached. Or stone.

Subject: RE: Tech: life durability of memory sticks?????
From: saulgoldie
Date: 27 Sep 11 - 06:45 AM

I suggest what I heard sometime back was the industry standard of three backups in two different locations for anything you *can't lose.* Two locations, because what if something happens to one location--fire, flood, theft? Three copies, because even with only three, you may have issues with "version control." You will have to meticulously keep track of them, or you will go nucking futz trying to remember which version is current and which one you made which modifications to. Most professional backer-uppers I am aware of have calendars and procedures for this. By the way, the major operating systems now have built-in backup features.

If you are concerned with the media itself, I would suggest also using another media type, like CDs. CDs are dirt cheap, anymore--less than a dime in bulk. And as we know from our music, CDs last a long time. I have many that are over two decades old. Remember NOT to mark them with a Sharpie. The ink can bleed through and damage your data. There are special pens specifically for this purpose. USB external hard drives are also very useful, and not too expensive. But I am answering more questions than you asked.

I will further investigate your question about media longevity.

Saul

Subject: RE: Tech: life durability of memory sticks?????
From: JohnInKansas
Date: 27 Sep 11 - 07:40 AM

k-1: Technology for putting sound recordings or video on paper is not well developed, and every time I put one of the rocks in my player it bends the needle. One maker advertised the ultimate in storage security with their WOM (Write-Only-Media) Drive some years ago, but it apparently never made it to production.

John

Subject: RE: Tech: life durability of memory sticks?????
From: GUEST,999
Date: 27 Sep 11 - 07:43 AM

lol

Subject: RE: Tech: life durability of memory sticks?????
From: Bernard
Date: 27 Sep 11 - 09:10 AM

Didn't Kelloggs develop cereal drives? I think they were a bit flaky, though.

I can think of some applications for Write Only Media...!! Nuff said!

Unfortunately, my experience is that flash media isn't to be trusted for anything other than carrying data from one machine to another - they can fail without warning and there's no way to recover your stuff... and it doesn't seem to matter how cheap or expensive they are.
Even if the manufacturer offers a warranty, what help is that when your data is lost forever?

No storage medium is 100% reliable, but data from failed hard drives can usually be rescued, although the service can be expensive.

It's also worth remembering that most pen drives and memory cards use FAT32, which limits the size of individual files to 4Gb irrespective of the drive capacity.

I carry two cards around with my camera, but I also have a portable card reader with internal 80Gb hard drive, so that I can back up my photgraphs soon after I've taken them without needing access to a computer - and it's far more convenient than a laptop.

Subject: RE: Tech: life durability of memory sticks?????
From: JohnInKansas
Date: 27 Sep 11 - 10:13 AM

I can think of some applications for Write Only Media ...

With the newer NTFS format, Microsoft says that they can apply something like two dozen different attributes to a file, and not just the old Read-Only, Archive, Hidden, System ones. (In Vista, Command Prompt, "Attrib /?" returns an additional "I = not content indexed" Attribute, but I've never found names for any of the others.) It has been suggested that the attribute applied to Backups made with their built-in Backup utilities generally has the Write-Only attribute attached, since almost nobody's ever been able to restore anything from one of them.

If the problem is just recording the data and getting it back to a storage place like a Hard Drive, and/or getting it to the library, most people would consider thumb drives or memory cards sufficiently secure. I wouldn't, at present, consider them a good way to archive anything for long-term storage (measured in years) until they've been in use long enough to develop some experience with how well they'll hold the data. The predicted data stability is pretty good, but so were the claims made for recordable CDs and DVDs until people learned otherwise.

The sticks/thumb drives etc don't seem like something that would be too practical for a library to keep as a master, due to the ease of "dropping one in a crack" so that it disappears among other reasons, although the library could easily transfer what's given to them on one to another medium that they believe is of archival quality.

I would suppose that part of the question depends on whether the one providing the data is responsible for it being on a "permanent" medium, or needs only safe transport and delivery with the library taking care of the storage problem.

John

Subject: RE: Tech: life durability of memory sticks?????
From: pavane
Date: 27 Sep 11 - 02:34 PM

I certainly wouldn't reply on them for long term storage. I have had 4 die on me so far, with no warning. So I only use them for data transfer.

Subject: RE: Tech: life durability of memory sticks?????
From: GUEST
Date: 27 Sep 11 - 03:31 PM

Perhaps not all things are menat to be kept and that's why all media fail to some extent or another.

Given past history, the long term solution might be to save your data in rocks.

Subject: RE: Tech: life durability of memory sticks?????
From: saulgoldie
Date: 27 Sep 11 - 03:36 PM

I discussed media durability with a coworker, and he said that most recent vintage CDs and DVDs have a life span of over 80 years. I expect to be pushing up daisies by then. And I will speculate that some or much of the world's coastal cities will be under water. YMMV.

Saul

Subject: RE: Tech: life durability of memory sticks?????
From: Artful Codger
Date: 27 Sep 11 - 06:34 PM

Don't you find it preposterous that the best modern media can only last 80 years, while copyrights remain in effect for at least 125 years? That essentially means that nothing except what the megacorporations find wildly "commercially viable" is likely to survive long enough to enter the public domain. We already see a similar situation with much sheet music from the 1800s, which may be stored in public repositories like the British Library, but is too frail to be accessed by the public and is too expensive to scan for dissemination over the web. And few private collectors who hold copies make this material available to the public domain that technically should have access by right, because it would diminish the market value of their collectibles. It is a gross mistake to believe that everything worth saving will continue to be replicated (faithfully) onto new media before the old media become obsolete, unreadable or privately hoarded--just think of all the once-popular films that have been irretrievably lost or degraded beyond restoration. Why do we tolerate this? When corporations sit on their acquired rights and assets like a dog in a manger, why do we condone or tolerate this?

Subject: RE: Tech: life durability of memory sticks?????
From: JohnInKansas
Date: 27 Sep 11 - 08:21 PM

Note that the "vintage CDs and DVDs" that "have a life of over 80 years" are pressed disks or produced on industrial burners with lasers powerful enough to make mechanical pits in the material where the data is recorded. The recordable CDs and DVDs you can "burn" at home record the information in a film that's more like a badly developed photo film (if anybody remembers those) than to the grooves and bumps in vinyl (or for CDs, in really hard plastic?).

Any material "photosensitive" enough to be changed by the low powered lasers used in computer burners can be changed in ways that degrade the data stored on them by any number of other things like heat or temperature changes, sunlight or low level uv emissions from some interior lighting, or chemicals ranging from cleaning materials to medicines, or by contaminants from gerbil dander or cockroach footprints.

In the interval while CDs offered cheaper "bit storage" than alternatives, over a period of a few months I "archived" about 360 CDs that were then stored in jewel boxes, inside drawers, in the dark, in a quiet part of the air conditioned office area. Approximately five years later, hard drive capacities with much larger sizes were economically available, so I copied the CDs back to a backup HD.

All of the CDs were "verified" when made. Approximately 10% of the CDs were completely unreadable when I pulled them out to transfer the data. Another 10% had some, but not all files recoverable. Only about 80% appeared to be fully readable after just 5 years in "benign" storage.

Because I had deliberately made the CDs so that each individual file was on three or more CDs (mixed more or less at random), my actual data loss was quite low, but loss would have been quite annoying if I hadn't had an unusually high level of redundancy in the disks.

For comparison, at the same time that I transfered all the contents from the CDs, I went through my accumulation of old 3.5" floppy disks that had been stored similarly for 10 to about 18 years, and found about 40% of those completely unreadable with almost no disks from which all files on the disk could be recovered. (But floppies were never considered particularly reliable even when they were about all we had.)

Note too that several individual files from the CDs were each larger than the total contents of a folder containing all the files from ~60 3.5" floppies. (Just in case you're making a five year plan for getting the storage space you may want in 2016.)

And anyone who thinks that "writing it in stone" is more permanent should take a walk through any 100 year old cemetery and see how many of the stones can be read as cearly as when new.

John

Subject: RE: Tech: life durability of memory sticks?????
From: GUEST,.gargoyle
Date: 27 Sep 11 - 09:34 PM

Thank you all

Everything is built on shifting sand.

Only print - seems to endure (surface/ink)

I am ready to explore some 5.5 floppys this winter.

Sincerely,
Gargoyle

Backup - and print - if you have not visited ARCHIVE.ORG - recently .... it is well worth a return.

Subject: RE: Tech: life durability of memory sticks?????
From: GUEST,Grishka
Date: 28 Sep 11 - 09:58 AM

Three copies on different media is the best idea, but only if you check each of them every year with a checker software that reads every bit. If one of the media has errors (or takes very long to spit out the correct data), duplicate one of the other copies.

Subject: RE: Tech: life durability of memory sticks?????
From: Mr Red
Date: 29 Sep 11 - 10:28 AM

Thanx Guys

I have seen CD's (HHD in the UK) that claim somewhat like 100 years. If I was in my 20's I might consider trying to test that one. You never know!

I am well aware of the rule of three and the spread-bet methodology. And cycling on a (say) 5 year basis staggered.

The advantage of SD &/or memory stickie thingies is size. Triplicating the files &/or the medium means that there is more capacity to try it. And at relatively low cost.

The museum is prepared to accept DVD and hold while the county government get their act together on digital archiving poicy. And capacity. I hold files on HDD and will transfer to another HDD because video eats up memory - fast. 1Tbyte is look mighty small!!

I have one DVD's worth in MP2 and given that the source is from MP4 I doubt that my personal files in uncompressed AVI (converted from MP4) will be much better on resolution. Given the nature of the environment - the audio is good enough. I use an Alesis Videotrak which is better for audio than Video.

Subject: RE: Tech: life durability of memory sticks?????
From: JohnInKansas
Date: 29 Sep 11 - 10:34 PM

It's a little hard to visualize what total amount of stuff you're planning to deliver to the library, but it appears that you may be talking about a fairly large number of devices for SD &/or Memory Sticks.

A disadvantage of the small size is the difficulty of affixing physical labels so that users can find the one wanted. Sorting through even a small bunch of individual devices to find the ones wanted can mean a significant amount of handling for all of them, unless you've got numbered slots to put them in, in a cabinet or file box. A "proper storage" at least partly negates the small individual size.

SD cards generally require inserting the cards into a reader of some kind, and while they're fine while they live in a phone or camera, when you have to pull them out and stuff them in somewhere else, there's some risk of mechanical damage. Many memory sticks have retractable connectors and/or covers. The retractables are more durable due to breakage of some kinds of the covers, but not all of them "lock extended" securely, with the result that they get abused by people trying to get them in the socket. Careful selection of a single specific kind of stick can help some.

SD & memory sticks in my local retail market seem to "sweet spot" at about 20 GB, beyond which the price per bit goes up by a fair amount as capacity increases. A TB of 20GB devices is 50 parts to label and keep track of, and at the retail $28/device a TB of storage runs up to $1400. I haven't looked for quantity prices, but that seems rather steep when you can get a portable USB 1TB hard drive for about $120 at single unit retail.

With 50 devices, you'll probably want at least one "index device" to keep track of what one to pull out when you want a particular data track.

A single 1TB portable HD can use the same USB connection as the thumb drives, and can have an index file up front on the same device.

I haven't seen portable HDs larger than 1TB easily available, and although the "desktop USB external HDs" run up to 5 TB fairly economically, the desktop units I've used have failed miserably when moved around and otherwise handled, even with meticulous care to be sure they're never moved while running and with careful shock protection in transit. (3 of 5 failed in one or two trips each, when used while parked out on the road.)

Using Western Digital and Seagate Portable External USB Drives, with 3 1TB, 4 500GB, and a couple of 300 GB drives, some around 6 years old, I've yet to see a failure (still being careful with handling). Both of these makers have USB-3 units available now, with 1 TB at a little over $100 each. I don't have a USB-3 port as yet, and at USB-2 it takes a few hours to copy a half-T, but retrieving individual files is about as fast as for the internal HDs I have. USB-3 will eventually be available and claims 10x speed over USB-2, so it's probably worth looking for the faster drives so that you'll have the better speed when the computers catch up. (Note that this isn't a particular recommendaton for the makers I've named. That's just what's handy in my local markets.)

This isn't meant to argue with what you've planned - but bunches of little parts being handled by people you don't get to train carefully always made me nervous when I had to be "responsible" for results. Now that I'm fully retired and irresponsible, the final decisions are up to others - like you.

John

Subject: RE: Tech: life durability of memory sticks?????
From: Mr Red
Date: 30 Sep 11 - 08:00 AM

As I say - getting the MP2 files or DVD equivalent to the museum is easy. 4.7Gb per DVD and I am only up to to 1 so far. They have asked me to hold original copies myself - which I would do anyway. And.... see below

Now- On-line storage - any experience? names might help me.

I have another more exacting project where a professional transfer service in NZ (I have family there) can transfer 9.5 (yes 9.5mm) film to TIFF files for me - at a cost I have yet to discover.

The upshot is 17000 files totaling 17 Tbytes which is an improvement on the 62 Tb for colour TIFFS - the family video is over 70 years old and in B&W but rather precious since dad died when I was 9 months old. It is a playlette done through the cine club of the time. Dad being the driving force.

Logistically On-Line is the easiest to administer. The NZ end can upload as his capacity (5Tb at last count) and bandwidth/Gb limit allows, and I can download at a pace I can process. We are talking years for my time.

Subject: RE: Tech: life durability of memory sticks?????
From: Mr Red
Date: 03 Oct 11 - 09:48 AM

I have a Corsair Voyager GT (16K) memory stick and - I have more problems with it than a Verbatim 4K, I bought it because it claimed superior burst speed. The Verbatim is faster by any measure and that was given to me by Martin of Webfeet - ideal site for UK dancers.

this is what Corsair say on lifetime:

There is no definitive answers to your questions sorry but the warranty is for 10 Years and if the drive fails in that time frame we will replace or repair it. Additionally the Flash we use has an expectancy of 10K writes per cell so if you write something to the drive every day for 10 tears you would still not use the hole write availability

I left the spelling as sent - you gotta larf ain't ya?

Subject: RE: Tech: life durability of memory sticks?????
From: Newport Boy
Date: 03 Oct 11 - 10:35 AM

Re your NZ project. If you spread this out over time, www.dropbox.com might be a solution. It's online backup & storage and is free for 2GB, tiered charges above that. I've used it extensively for the past year for relatively small files, and found it very reliable. It's very easy to use.

Once set up, any file you place in your Dropbox folder appears automatically in the remote Dropbox folder(s). By copying into and out of these folders, and deleting files once transferred, you could pass all the files through.

I'm undecided about on-line storage generally - if it's for the long term, you are relying on the provider staying in business.

Phil

Subject: RE: Tech: life durability of memory sticks?????
From: JohnInKansas
Date: 03 Oct 11 - 03:22 PM

A recent business announcement won't help you much, since it's something only available in Japan; but the "quoted capacity" may suggest something of what you need for storage:

Toshiba to sell 5TB home media server in Japan

"Toshiba said the server can hold 15 days of digital broadcasts from six channels, and stream them to its TVs and tablets."

"By Jay Alabaster | 03 October 11"

Desktop USB Drives are available with 5TB storage, and might help with the home storage; but as has been discussed you'd likely want redundant backup regardless of media used.

John

Subject: RE: Tech: life durability of memory sticks?????
From: Bernard
Date: 04 Oct 11 - 07:25 AM

There are pro's and con's with 'mirrored' drives, which is the simplest form of redundancy...

In theory, when one drive fails, the other still has all your stuff.

Other people say that the more likely scenario is that a software error will simply be replicated and you've still lost the lot. I'm inclined to think it is more likely to occur in a mirrored RAID system drive.

I'm happy to rely on a NAS with mirrored drives (Network Attached Storage) for at least one of my backup devices, and USB drives for others.

Interestingly, I have a couple of LG-NAS (LG as in the manufacturer) units with pop-up USB slots... insert a memory stick and it will back it up for you. I've yet to find out how to restore, though...! Maybe it's a real example of John's 'Write Only Memory'...!!

Subject: RE: Tech: life durability of memory sticks?????
From: JohnInKansas
Date: 04 Oct 11 - 01:29 PM

Bernard -

All Microsoft OS versions have included instructions for making system backups. The latest versions have called the backups "Recovery Disks."

Beginning when my first machine had nothing but DOS 3.5 on it, I've made one of their backups for every computer I've had. The first one filled 12 or 13 1.4 MB floppy disks, including a few data files I'd made before I got around to the backup.

Progressing through Windows 3.1, 3.11, 3.11WG, Win95, Win98, Win98SE, Win2K, WinXP, and now Vista, I've made their backups, carefully following instructions, on floppies (including IOMEGAs), tape, CDs, DVDs, and separate dedicated hard drives. (In most cases I made separate backups to at least two different kinds of media.)

I have NEVER successfully restored anything from any of those backups.

For one of the Win95/98 versions, about 3 years after the version was released, Microsoft published a warning that "backups made using Windows Backup almost never restore. About three years after WinXP came out, a "tech note" reported that "many backups fail to restore." I never found any further explanation in either of those two cases.

I recently made a full system backup for Vista on a dedicated HD, reformated (NTFS) to "virgin" before doing the backup, with nothing else on the drive, and a couple of months later my system HD failed. When I hooked up the Backup HD all I could get was "no backup found on designated drive."

While I never needed to restore anything for a couple of the "backups" so I can't testify in criminal court that none of them ever worked, odds are very good that Microsoft has never produced a reliable System Backup in any user OS since Hard Drives were first in common use.

In the Vista case, a new copy of Norton Ghost didn't do much better. Although it indicated compatibility with Vista, Vista didn't really like it, and "instructions" weren't fully compatible with Vista's "new features."

It's possible that one of the Microsoft Server OS versions did something better. RAID striping or other "redundancies" haven't produced many comments about failures, and are used where recoveries are frequent enough to show up failures; but those are a little out of sight for most of us.

For the majority of users, the best we can do is:

Keep the original installation disks to restore the OS and programs. (Problematic for the OS and pre-installed programs, since few OEM builders provide disks now.)

COPY data files to a redundant drive. (XCOPY still works in Vista, although I'm not sure yet about Win7/8.)

John