Caution: Storage Speeds Can Vary Dramatically and Cause Problems

As I was writing an article on compression speed using the 4.3.1 version of Apple Compressor, I needed to transfer some files from one hard disk to another. More specifically from a single, spinning media hard disk to an external SSD connected via USB3.

Things started out well – then they got weird. Then, they got REALLY weird.

BACKGROUND

All storage devices have maximum speeds which can vary based on a number of factors:

• How they are connected (Thunderbolt 1, 2, 3, or USB 2, 3, 3.1, 3.1 Gen 2…)
• How full they are (empty spinning hard disks are faster)
• Whether they using spinning media or SSDs (SSDs are faster)
• Whether they use one or more disks (RAIDs contain multiple hard disks and are faster than a single drive)
• The overall efficiency of the Finder at copying files

It has long been reported that the Finder is not particularly efficient at copying files between drives, though moving files within the same hard disk is almost instantaneous.

The computer doing the transferring was an iMac running OS X 10.12.3.

The spinning media drive transferred data around 75 MB/sec, while the SSD transfered data around 115 MB/sec; as measured by the Blackmagic Disk Speed Test.

NOTE: The specific brands of drives are not the issue, directly, so just stay with me a for a bit.

I was transferring 143 GB of files, the smallest of which was 2.5 GB. Due to the way the Finder works, small files transfer more slowly than large files. These are really big files, so they should transfer faster than “normal.”

WHAT HAPPENED?

Um, I don’t know, really, but here’s what happened in the first two minutes. The spinning media drive (in blue) started playing back data at 45 MB/sec, while the SSD started recording it at 150 MB/sec.

NOTE: The key numbers are Data read/sec (playback) and Data written/sec (recording). Both are measured using Activity Monitor, which is stored in your Utilities folder, under the Disk tab.

This transfer shouldn’t be asymmetrical – you can only record data at the speed it is played back. The SSD was recording data three times faster than it was playing back. There are always measurement issues when a transfer first starts, but this lasted almost two minutes.

Then, pulsing started. The fastest sections (the fat ones) pushed data at close to 120 MB/sec, while the slower sections dragged at 17 MB/sec. Notice also that the “saw-tooth edge” of the spinning media drive indicates significant speed variations, while the SSD drive records at a more steady pace.

Then, things just got insulting – speeds slowed to 8 MB/sec. This data rate is far too slow to support even SD video!

NOTE: I consider 75 MB/sec the minimum speed necessary to support reasonable HD video editing. Faster is, obviously, better; but steady and constant speeds are essential, too.

ANALYSIS

During the transfer, Time Machine was turned off and all other applications were quit. Nothing should have interfered with this transfer running as fast as possible.

I’ve known for a long time that spinning media drive speeds will vary slightly when transferring data, depending upon how long it takes the heads to find the right spot on the disk. But, I’ve never measured anything like this.

Worse, if you were editing and this happened, you’d be dropping frames left and right, and never think that the hard disks you were using were slowing down so severely. Still, I’m not sure where the problem is:

• Is it with the spinning media drive?
• Is it the Finder?
• Is it some unknown background process soaking up CPU attention?
• Is it how data transfers are measured using Activity Monitor?

One thing I noticed was that when I transferred a large file from the internal Fusion drive to the SSD, I got consistent speeds with much less fluctuation. So, this may point the finger toward the spinning media drive being the cause of the problem.

SUMMARY

I don’t have a solution, but I do have some recommendations:

• If you are getting dropped frames in your video editing application, open Activity Monitor and measure the data transfer speed in the Disk tab. Watch the data rate when you are dropping frames. If this is too low, you will have a transfer problem with your storage. (The exact minimum speed will depend upon codec, frame rate and image size.)
• SSD drives are supposed to be faster, but they can only be as fast as the weakest link in the data transfer chain. Notice that I’m using an SSD drive here, but not getting anywhere close to its “maximum rated speed,” due to other factors.
• It is a good idea to test the speed of any hard drive you use for video editing periodically to make sure they are running OK. You should get at least 80 MB/sec from any hard drive or SSD. Anything less means it is time to either call Support or get a replacement unit.

There are two free utilities that are excellent for testing hard disk speeds:

• AJA System Speed Test
• Blackmagic Design Disk Speed Test

In all cases, when dropped frames or slow data transfers start occurring, ignorance is not bliss. And, as always, let me know what you think.

---

Bookmark the permalink.