Hard Disk Warning!

Posted on by Larry

[ This article was first published in the September, 2008, issue of
Larry’s Final Cut Pro Newsletter. Click here to subscribe. ]


For those of you who are archiving your products on hard disks, please pay close attention!

Recently, I was talking to an engineering manager of a well-known hard disk company discussing how we were supposed to archive tapeless media for the long-term.

NOTE: The company he works for is not important, because I’ve since corroborated this issue with two other hard disk companies. This is a hard disk issue, NOT a specific vendor issue.

Executive Summary

Magnetic signals recorded on a hard disk are designed to be refreshed periodically. If your hard disks stay on, this happens automatically. However, if you store your projects to a removable hard drive, then store that hard drive on a shelf, unattached to a computer, those magnetic signals will fade over time… essentially, evaporating.

According to what I’ve been told, the life-span of a magnetic signal on a hard disk is between a year and a year and a half. The issue is complex, as you’ll see, but this is a MUCH shorter shelf-life than I was expecting.

The way to keep the files on your hard disks safe is to connect the hard drive to your computer every six months or so and, ideally, copy all the files from one drive to another. Failing that, use a program like Micromat’s TechTool Pro, or ProSoft’s Drive Genius, to do a complete scan of your hard drive. Doing so will replenish any magnetic signals that are starting to fade.

The Technical Details

The issue of magnetic fading has been a little-talked about concern in the hard disk industry for a while, but it is complex and simple answers are hard to find. Here are some comments that one engineer sent me that may help you understand this better.

This has been something that has come up over the years, in the quest for higher density hard drives, before we crossed the 1GB per drive barrier. At that time, there was concern because the earth’s magnetic field could affect data integrity, and many measures were designed to compensate for this effect. At that time and since, there was also concern for the magnetic charges on the disk platters.

 

Older mainframe storage systems continuously read each sector; this read automatically refreshed any sectors that were margina. Servo tracks are also embedded within the data tracks, so any signals out of margin were easily detectable. Loss of servo info would render the disk useless, as the controller would not know where the head really was on the disk surface.

 

Improvements have been made in head technology, media magnetic qualities, and recording techniques to make the data integrity less subject to outside influence, and to the fading of the magnetic encoded bits on the disk surface. Sort of like when cheap magnets eventually lose their magnetism. Better materials deliver longer life.

 

So…in the interest of checking things out, simply reading every sector on a disk actually is preventative; if the controller within the disk detects any marginal data in either the servo tracks or the data bits recorded on the surface, the controller will automatically rewrite the data to the sector. If it cannot, this block is mapped out, again, automatically, and a substitute data block will be mapped in. All modern disks do this for you today. A simple read cycle of every sector or data block is all that is necessary.

 

In the 80’s and early 90’s this was NOT automatic, and that is where things like defragging and bad-block mapping were the norm. Today, a lot of this is done automatically for you by the disc itself, or the file system manager in the OS. Things are constantly being moved around.

 

So, the scanning is simply reading every sector of the disk surface. The act of copying all the files from one disk to another disk would almost accomplish the same thing. With this latter method, unused parts of the disc would not be read again.

 

As for duration: I would say that the disks could lose data if not used for a long period of time. Doing this surface scan every year or two is preventative. It is hard to define a point in time when a failure due to degraded media occurs. I do have drives here that have not been spun up for several years, and they are fine, but I have heard from many who do have issues after leaving the drive in storage for several years. Sometimes it is actual data fade; sometimes it is power supply related; sometimes it is due to extremes in temperature or humidity.

 

All things age, some more gracefully that others.

 

Bottom line, revisiting your storage archives periodically is some assurance that what you have saved away is really still there. You know the old adage, the only thing worse than no backup, is a backup that can’t be used.

Larry again: Frankly, I was stunned with this news and wanted to share it with you as soon as I could. Please make sure to check your hard drives regularly. Keep multiple backups. And consider using tape as a backup while these archiving issues get sorted out.

Here’s an article I wrote recently on backing up to tape that may be useful to you.


Bookmark the permalink.

20 Responses to Hard Disk Warning!

Newer Comments →
  1. Don Lewis II says:

    Thanks for the updated info., Larry. What’s the best way to scan those shelved HDs?

  2. Kirk Lohse says:

    Larry,

    Thanks for reposting this. What about thumb drives? Has there been any research into their “shelf life”? Or is it the same difference as with HDs?

    Thanks!
    K Lohse

  3. Ben says:

    Hi Larry, What about SSD drives? They are becoming cheaper and perhaps viable for permanent backups if they are reliable.

  4. Larry Jordan says:

    Krik and Ben:

    I have not read any research on either SSD or thumb drives. At thins point, I would be cautious. If I learn anything I will post it.

    Larry

  5. PassingBy says:

    Interesting topic, and thanks for reminding me of it. I would, however, question the ability of a Read pass to ‘refresh’ the magnetic domains.

    I’m an old radio tech, and as I understand the theory on the matter, the data has to physically re-written (moved to an empty section of the medium) to restore the magnetic intensity of the data-point.

    Personally, I keep a spare drive in a removable caddy, and clone a needful volume to the spare, then back to it’s original disk.

    It actually takes less time that a bit by bit media check, *fully rewrites | restores every data, incorporates a bad block check (AFAIK), and has the added benefit of doing a very effective de-fragmentation all the while.

    Easy to do in the back-ground. Thanks for the heads up!

    I’m off to do all mine right now!

  6. Rand says:

    Thanks for the heads up.
    So if simply reading the files helps keep them fresh
    can one simply use DiskWarriors ‘Check All Files & Folders’ function?
    Rand

    • Tim Johnston says:

      Yeah, I too would like to know if using DiskWarrior’s ‘Check All’ function will serve this purpose…?

  7. Tim Johnston says:

    Exactly which function in Tech Tool Pro is the one that will read (and therefore refresh) the entire hard disk? Surface Scan?

  8. I really love your site.. Very nice colors & theme.

    Did you create this amazing site yourself? Please reply back
    as I’m hoping to create my own personal website and would like to find out where you got this from or exactly what the theme is named. Cheers!

  9. Scott Higgins says:

    Larry
    it seems this is very hidden info and the new tedious but safe blu-ray data disc 128gb becomes a very useful archive.Why is this magnetic fade fact so hidden?
    cheers and thank you for the site
    scott

Newer Comments →

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends:

FCPX Complete

NEW & Updated!

Edit smarter with Larry’s latest training, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.