When you have a hard disk connected to your computer, all is good. It periodically checks to be sure all data is secure.
However, when you unplug a hard drive (also called HDD) and store it on a shelf, the magnetized bits that store your data can, over time, lose their charge and its associated data. This process is called “data rot.”
The general rule is that HDDs containing data can be safely stored on a shelf for 3-5 years before problems of data loss or sticky heads (“stiction”) arise. So to prevent problems we need to “reenergize” the data on the disk.
Here’s how.
IMPORTANT NOTE: This requires using Terminal, because I don’t know of an existing utility that does this. This isn’t hard, I’ve run it myself. But, if you are not comfortable using Terminal, get a tech-savvy friend or consultant to help.
NOTE: If the drive does not have an enclosure, use a drive dock (like this Voyager from OWC) to connect it to your computer.
sudo dd if=/dev/r— of=/dev/null bs=2048k status=progress
NOTE: In my case, I used: sudo dd if=/dev/rdisk11 of=/dev/null bs=2048k status=progress. That “r” before the name of the drive is important.
Terminal will ask for your password, then display a progress report on the process (red arrow above).
This command reads every sector on the disk – even if the sector is empty and even if it’s not a Mac format disk. Writing is not part of the pass since the drive firmware logic will handle sector reassignments if there are low level questionable reads. The purpose for this is twofold: keep the heads cleared of any parked “stiction” opportunities and validate the sectors on the disk via drive’s the built in block management logic.
This does not erase anything, nor write new data.
NOTE: The “sudo dd” option forces the drive to hit all of the drive’s sectors, not just those specific to any one filesystem/partition.
Most hard disks transfer data between 50 – 250 MB/second. The duration of this re-energizing process depends upon the speed of the drive and the amount of storage to be reviewed. This table can help you calculate the duration.
Time to Process 1 TB of Storage
| Hard Drive Speed | Hours / TB |
|---|---|
| 50 MB/s | 5.5 hours |
| 100 MB/s | 2.8 hours |
| 150 MB/s | 1.9 hours |
| 200 MB/s | 1.4 hours |
| 250 MB/s | 1.1 hours |
From the table above, multiply the hours per terabyte times the total capacity of your hard drive to get an estimate of how long this will take. If your computer goes to sleep, it will take longer.
EXTRA CREDIT
I’m grateful for the technical assistance of Tim Jones, Chief Solutions Architect at OWC, in providing and verifying the Terminal instructions.
24 Responses to How to Prevent or Repair Hard Drive “Data Rot”
DriveDX (Mac, $20, free download) will do a full surface scan of SMART-enabled hard drives. I’ve used this for years for this purpose. It only works with SMART-enabled drives where it can access SMART functions, but that is most drives. It’s great to have a command line alternative.
– – –
(Then, Tod wrote back) Of course, having just “recommended” it, DriveDX failed to scan the drive I just mounted. The drive was “interrupting” the scan. It may be an issue with Ventura power saving. I have not scanned drives since installing Ventura. I have not diagnosed the problem yet and I am using your command line. We’ll see how it goes.
Tod:
Thanks for sharing this. I had not heard of DriveDX. Here’s the link:
https://binaryfruit.com/drivedx
As a note – given Tod’s note, be sure to confirm that DriveDX performs a full disk scan before purchasing it.
Larry
DriveDX does indeed do the full drive scan, probably using the “dd” command under the hood.
https://binaryfruit.com/drivedx/help#ui-self-tests
>Full (Long / Extended) Self-test
>Longer, much more comprehensive and thorough version of the Short self-test, does >read scan of the entire media, with no time limit. Can be used to validate the results >of the Short self-test because Short self-test time constraint may not provide >sufficient test time to identify a fault condition.
As you mentioned in the article, Larry, using one of OWC’s drive docks is a great idea, as they support SMART reporting.
Dave:
This is excellent news, because DriveDX takes the fear out of using Terminal. Thanks.
Larry
I haven’t used DriveDX for this function yet, but I have been using it for a few years now to monitor some of the other drive health metrics. I haven’t had any drive issues recently, but it’s nice being able to keep an eye on any warnings or potential failures before they happen, in addition to stats like temp and terabytes written.
Hi Larry, In terminal I had a command line ending: username$. I had to log out before doing the disc operation. When I logged in again I had my name, in a different format, followed by “MacBook-Pro-2 ~ % ”
This will not action the command.
I have a screenshot.
Any help? Thanks, Randal (UK)
Randal:
Nope. I am not a Terminal expert in the least. I got this line from Tim. Perhaps another reader can suggest a workaround.
Larry
Hi Larry,
Thank you the great tip.
However, in the terminal command screen capture the r is missing from the if=/dev/rdisk.
Bruce
Bruce:
You eagle-eyed maven! You are correct. The command will run without the “r” – however it is more effective if the “r” is included. I checked with Tim.
Larry
Warning! If one edits this command line improperly one could erase a drive. The dd command (data duplicator) copies the source to the target, overwriting the target. In this case, the target is “null” so it is overwriting nothing. Clever. But if one were to replace “null” with a drive specification, you would overwrite that target drive. It seems like an unlikely mistake but be forewarned.
Tod:
Yup. Terminal is very powerful. That’s why the command is set to only read the drive.
I always triple-check that the text I’m entering in Terminal is correct.
Larry
Thanks, Larry, for this invaluable advice.
I’m being ultracautious when I need to mess with only-one-copy of contents drives. I signed up to iDrive.com where 5 terabytes of cloud storage cost only $9.95 for the entire first year. (Instead of $99.50).
Although it’s agonizingly slow, I back up the entire drive into the iDrive cloud before I do anything with the drive’s content. And now, I’ll perform your Terminal function before that.
BTW: iDrive also provides auto-backup but ignore that as we have Apple’s better Time Machine. So do not download/install the iDrive app.
David:
There’s nothing wrong with caution. Remember, you only need to run this Terminal script if the drive is stored on the shelf. If it attached to your computer, the operating system maintains the data.
Also, if you backup the drive, the process of backing up also re-energizes the data to the copy.
Larry
For many years now I have stored ALL my media on an enclosure with 4 spinning drives. The enclosure is set to Raid 0.
I have two identical 64 TB Raid O enclosures. I clone the two additional Raids to the master once a week or so.
Before this I had three 20 tb Raid O enclosures.
So far i have never had any data loss/rot problems. When one of the drives failed (this happened on the 20 tb system), I was able to continue working on the second raid IMMEDIATELY, while I replaced the bad drive, reformatted the raid and updated the media from the third back up to the original newly formatted Raid
Retrieving any of the media from the cloud, or from (for example) a Raid 5 enclosure, would have held up my work flow by at least a day.
Barry:
What you report is perfectly normal. And a great way to backup your data.
Data rot is only an issue when one ore more drives are unplugged from an enclosure and stored in a shelf. After a few years, dat can start to disappear.
This is what re-energizing is designed to fix.
Larry
Update on DriveDX. My issue with drives “interrupting” DriveDX scans appears has been solved by disabling disk sleep (sudo pmset disksleep 0), something I’ve been meaning to do in any case to prevent hard drives from constantly spinning down. The DriveDX SMART scans appear to be faster than the terminal copy. I presume this is because the drive is operating autonomously. Activity Monitor shows no disk activity even though the drive is clearly very active.
Hi Larry – Many thanks for all these great tips you share so readily.
I had a problem using this one with some of my HDDs that show as ‘Container’ between the two lines you show in your examples. Both the top line and the ‘Container’ line show things like disk4 and then, say, disk5 but neither work – typically I get something like “dd: /dev/rdisk5: Operation not permitted”. I could post more complete info but I can’t see how to include screen shots here.
I greatly appreciate your help and many thanks in advance, Brian.
Brian:
I can’t help here as much as I’d like, as Terminal is something I am not proficient in.
However, when selecting the drive, you need to select the TOP level, the device itself, not the container. Then read the name in the Devices field.
Larry
Thanks Larry. As I mentioned, your tip works perfectly for me provided, in Disk Utility, a line showing ‘Container’ does not appear between the two lines you show in your examples. (I have quite a few HDDs as I look after the archives of a number of groups, schools etc. thus my interest in all this.)
Now that I’m aware of the ‘rot’ risk I’ll stop pondering Terminal and try out DX. I’ll come back and post my experience in case someone else hits this too.
Brian:
Just as a reminder, the Container feature only applies to drives formatted as APFS. This is best used for SSDs. HFS+ is the strongly preferred choice for spinning hard disks. Most of the time, then, you should not need to see a container. Also, keep in mind that this re-energizing ONLY applies to spinning hard disks. There is no reason to use it for SSDs or SSD RAIDs.
Larry
Thanks Larry, as I work through all my spinning drives it’s becoming clear that this works great for Mac OS Ext and NTFS but not for APFS which, as you point out, have this ‘container’. I don’t know why I formatted some of those drives as APFS but in every other respect they seem to work fine.
I had a large WD SSD late last year but it provoked frequent unexpected reboots of my M1 MacBook Pro so I sent it back.
One other little thing: yesterday I wanted to stop the process on a 1TB drive. Ejecting using Finder got rid of the drive in the sidebar but it was still going in Terminal. Ejecting within Disk Utility ‘failed’ as ‘it is currently in use’. In desperation (and knowing I had another good copy of that drive) I restarted my machine which had the desired effect. Might you perhaps know of a more elegant way of halting the process? Many thanks in advance.
Brian:
APFS was designed, from the beginning, for SSDs, not spinning hard disks. If at all possible, backup the data on any HDD (spinning hard drive), then reformat it to HFS+. It is too complex to explain all the reasons why, but HFS+ is always the best option for HDDs.
As for Terminal, Control+C is the traditional way to stop an on-going process.
Larry
Useful comparative review of DX & alternatives:
https://www.uubyte.com/blog/the-best-7-hard-drive-monitors-for-mac-ensure-optimal-performance-and-data-security/
Steve:
Thanks for the link.
Larry