[ This article was first published in the January, 2011, issue of
Larry’s Monthly Final Cut Studio Newsletter. Click here to subscribe. ]
I’ve had the pleasure of testing a variety of truly high-speed RAIDs over the years, and I’ve spent a lot of time worrying about whether I was getting the absolute maximum speed from each unit.
However, as I was investigating how Final Cut Pro handles multiclip editing, it struck me that, after a certain point, the speed of your storage doesn’t really matter. Which means that we need to pay attention to more than just the raw speed of our storage systems.
Let me back up for minute, though, and explain where maximizing your speed is essential.
First, how you connect your storage to your computer determines transfer speed more than anything else. Take a look at the table below. These were measured about a year ago using the same computer, same hard disk and same test software (AJA System Test).
| Connection | Data Transfer Speed |
|---|---|
|
USB 1.0 or 2.0
|
10 – 15 MB / second
|
|
FireWire 400
|
20 – 25 MB / second
|
|
FireWire 800
|
45 – 80 MB / second
|
|
iSCSI (Gigabit Ethernet)
|
75 – 95 MB / second
|
|
eSATA (single drive)
|
75 – 95 MB / second
|
|
Internal iMac drive
|
50 – 60 MB/sec
|
|
Internal MacPro drive
|
70 – 90 MB/sec
|
The key point isn’t the specific numbers, but the impact that your connection protocol has on transfer speed. You could have the world’s fastest hard disk, yet, if you connect it via USB, all that speed is wasted.
Second, single hard drives, while by far the most popular– and cheapest — method for storing video data for editing, suffer from two significant limitations. They can’t protect your data if the drive fails, and they have a maximum transfer rate of less than 100 MB/second.
This speed is fast enough for most single camera editing, but not fast enough for any significant multicam editing.
NOTE: Regardless of how you set the RT Menu in the Timeline, editing multiclips requires the full data stream from the hard drive for each angle in your multiclip. The RT Menu only reduces the strain on your CPU, not your storage.
For anyone using single drives as their main storage, speed is everything and you need to make sure you are connecting your hard drive using the fastest protocol it supports.
But the situation isn’t necessarily the same for RAIDs. Different factors come into play.
IMPORTANT NOTE: This article assumes you are directly connecting your storage to your computer. If you are connecting storage to a server, then speed is critical because the RAID needs to support more than one user.
If you connect your RAID via FireWire 800 (you should not even THINK about connecting a RAID via FireWire 400!) the maximum speed you can get from your RAID is limited by the FireWire bus to about 85 MB/sec. Even if you have a blazingly fast RAID, FireWire will limit the speed.
If you connect your RAID using iSCSI, your speed will be limited by the Gigabit Ethernet protocol to about 105 MB/second. This is fast enough for many single-stream video formats, but can be limiting for multiclip editing. Personally, I like iSCSI for its flexibility allowing me to use this same device on laptops and iMacs, as well as towers. iSCSI is faster than FireWire, but no speed demon.
It isn’t until we get into eSATA, Mini-SAS, or Fibre-channel that the connection protocol starts to match the speed the RAID is capable of sustaining.
At this point we need to be careful, lest we fall into a marketing trap; which is what got me started on this whole musing in the first place.
THINKING ABOUT RAIDS
If you are copying files, the speed of the copy is dependent upon your slowest device. So, if you have a blindingly fast RAID copying to a single drive, the speed of the copy is determined by the single drive, not the RAID.
Render files, even renders of multiple layers of video, play at the same rate as a single stream of the video in the sequence you are editing because the process of rendering creates a single video clip integrating all those different streams. That’s the magic of a render file — its the same size and speed as the video format of your sequence.
The speed of rendering is determined by the speed of your processor, not your storage. Render playback speed is the same as a single video clip.
In fact, once you have a hard disk with sufficient speed to play back two simultaneous streams of your video, your hard disk is “fast enough” for almost all editing, with one exception: multiclips.
A multiclip requires a storage system fast enough to play every video clip (also called a “stream”) in the multiclip at full speed. So, depending upon how many streams and which format, you will quickly exceed the capability of a single drive, and start to tax a RAID.
However, there is an upper limit to the speeds you actually need; which is what I discovered when testing multi-clips.
Let’s do some simple math.
Final Cut Pro supports up to 16 simultaneous streams in a single multiclip. (You can link over 100 clips, but they don’t all play at the same time, they are simply tracked.)
The highest-quality, and largest, HD video files we can create are 1080p square pixel ProRes 422 HQ files.
NOTE: Yes, HDCAM SR files are larger, but transcoding them to ProRes 422 HQ is visually lossless and reduces files sizes close to 90%. You would use HDCAM SR files for final mastering, not for rough editing using multiclips.
For most transcoding I prefer ProRes 422. So, let’s do examples with both.
In other words, the MAXIMUM speed you would use from any RAID for a fully-loaded multiclip is less than 450 MB/second. And, more practically, you would probably need less than 300 MB/second.
Granted, this is far faster than a single drive can support; or even a three drive internal RAID in a MacPro. However, there are many, many RAIDs on the market that far exceed these speeds.
Which means that you need to pay attention to more than maximum speed.
Here are other things to consider in deciding which RAID to buy:
1. How noisy is it?
2. How far can the RAID be placed from your computer?
3. How much total storage does it have?
4. How low is the seek time?
5. What’s the experience of the manufacturer?
6. What’s the duration of the warranty?
7. How long does it take to get a replacement drive?
8. Does it require any special software to run?
9. Does your computer provide an ability to connect to it?
10. What does it cost?
The older I get, the more noise matters to me. The old XRAID sounded like a jet taking off. It was awful. If companies expect me to connect a RAID to my computer in an edit suite, they need to make these units are REALLY quiet.
A while back, a RAID company sent me a unit to test (this was not AccuSys) and the interface cables were 18″ long. I don’t have enough desk space as it is. Short cables are useless. I use RAIDs for their speed and storage, not because I like staring at them on my desktop.
Video devours storage. I will take a slower RAID with more storage than a faster RAID with less – assuming both are the same price.
Seek time is how long it takes the heads to get to a specific piece of data on the hard disk. With multiclips, low seek times are essential, since the RAID essentially needs to play all these files simultaneously. This means the drive heads need to move VERY quickly between all these files so that frames don’t drop.
Since there is no way I can benefit from a RAID that goes faster than 500 MB/sec – multiclips in Final Cut don’t need that much speed – I don’t see any reason to pay extra for speed I’m not using.
There’s an analogy that applies here. Car speedometers routinely display speeds in excess of 100 MP/H. Yet very few of us would EVER drive at that speed. A car that goes fast is good for the ego, but may not be practical.
Same thing with a RAID. It may go really, really fast. But at a certain point, you don’t need the speed. A fast engine, by itself, isn’t the answer. You also need a comfortable place to sit and a steering wheel that works.
You get the point.
