Understanding Storage and Media Bandwidth [u]

[ This article was first published in November, 2008. Updated: July 2009, May 2010, June 2017, and significantly revised November, 2024. ]

Now that the shift to digital media is complete and the on-going race to ever higher resolutions continues to pick up speed, it is increasingly important that we understand the role storage and codecs play in the overall performance of our editing systems.

There are three key concepts to keep in mind:

1. The two main categories of storage are: Spinning media (HDD) and solid-state media (SSD). Each has advantages and limitations.
2. Storage is not infinite in speed, nor unlimited in capacity.
3. File sizes and bandwidth requirements increase as frame size, frame rate and bit depth increase.

STORAGE

It seems obvious, but it is very easy to ignore:

• The speed of your storage depends, in part, on how it is connected to your computer.
• The type and configuration of your storage affects speed and capacity.
• Different codecs create files at wildly different file sizes from the same source media.
• Dropped frame errors and stuttery performance are generally caused by a storage system that is too slow for the media you are editing.

HARD DISK DRIVES VS. SOLID STATE DRIVES

Hard disk drives (HDD) are what made digital media possible in the first place. Each drive consists of multiple spinning platters which store data magnetically on each disk. These are highly-complex, mechanical units with lots of moving parts. Hard drive capacity continues to increase into the dozens of terabytes per drive, while prices continue to drop.

Advantages

• Vast capacity
• Low price

Disadvantages

• Slow speeds
• Fragility, data can be lost if unit is dropped
• Latency, the time it takes to move the drive heads from Point A to Point B
• Size
• Heat

Solid-State Drives (SSD) are the latest storage speed demons. There are two categories: SATA and NVMe; NVMe is the fastest. While they don’t hold nearly as much as an HDD, they are far, far faster. Data is stored on solid state media, which means no moving parts are required to read (playback) or write (save) data. SSDs continue to increase in capacity, while lowering in price, but nowhere close to the capacity or cost per TB of an HDD.

Advantages

• Vast speed
• Ruggedness
• Small size
• Zero latency (no moving parts)
• Almost no heat
• Affordable, but not cheap

Disadvantages

• Much less capacity than an HDD
• More expensive that an HDD for the same capacity
• NVMe SSDs are more expensive than SATA SSDs, but much faster

STORAGE BANDWIDTH (SPEED)

Storage “bandwidth” is the speed at which a storage device transfers data between itself and the computer. We also call this the storage “data transfer rate.” This speed is dependent upon:

• How the storage system is connected to your computer
• Whether it is an SSD or HDD
• If it is an SSD, whether it is a SATA or NVEm SSD
• If it is an HDD, the number of drives it contains, how fast those drives spin, how full those drives are

Approximate Bandwith by Hardware Type

Hardware	Bandwidth
SINGLE HDD	100 – 220 MB / second
SATA SSD	Up to 400 MB / second
NVMe SSD	Up to 6,000 MB / second

 

HDDs hold more, but SSDs are faster.

STORAGE LIMITATIONS

Because HDDs are mechanical devices with many moving parts, they have more limitations than all-digital SSDs. Here are several things to keep in mind:

1. In the past, protocols like FireWire and USB 2 were slower at transferring data than an HDD disk. So, how we connected the drive made a difference in its speed. Today, protocols like Thunderbolt and USB 3.1 Gen 2 are far faster than a single HDD drive. So the speed of our storage is based more on the number of drives they contain than how the drives are connected to the computer.
2. RAIDs are devices that contain multiple drives; either HDD or SSD. The speed of a RAID is dependent upon how it is configured.
3. As frame sizes increase, file sizes get bigger and bandwidth needs to be faster.
4. As frame rates increase, file sizes get bigger and bandwidth needs to be faster.
5. As bit-depth increases, file sizes get bigger and bandwidth needs to be faster.
6. Most editing codecs – ProRes, DNxHD, GoPro Cineform, R3D – create large files which are optimized for editing performance, while retaining the maximum image and color quality.
7. Most distribution codecs – H.264 and HEVC – are optimized for small file size, by reducing the amount of color data, image quality, and editing efficiency of a clip.
8. The more data you store on an HDD, the slower it goes. An HDD drive is fastest when it is empty. When an HDD drive is totally full, it neither plays back, nor records. I strongly recommend keeping about 20% free space on any HDD drive.

RAIDs – FASTER SPEEDS, MORE CAPACITY

To improve on the speed of a single HDD, or to improve on the storage capacity of a single SSD, RAIDs were invented. A RAID is a collection of matching drives that combine the speed and capacity of multiple drives, but act – to the computer – as though it was a single unit. In general:

• RAID 0 is the fastest, with the greatest capacity, but all data is lost if even one hardware drive dies.
• RAID 1 is the slowest, but guarantees that all data is safe if one drive dies.
• RAID 4 is optimized for SSD RAIDs and protects all data if one drive dies.
• RAID 5 is optimized for HDD RAIDs and protects all data if one drive dies.
• RAID 6 is optimized for HDD RAIDS – and not recommended for SSD RAIDs – and protects all data if two drives die at the same time.
• RAID 1+0 provides higher speeds for SSD RAIDs than any format except for RAID 0, yet protects all data if one drive dies.

RAID Speed & Capacity Calculator

RAID Type	Multiplier
RAID 0	(# of drives)
RAID 1	1
RAID 4	(# of drives – 1)
RAID 5	(# of drives – 1)
RAID 1+0	(# of drives / 2)

 

To calculate the speed or capacity of a RAID, multiply the Multiplier by the speed or capacity of each device. For example:

• A 4-drive HDD RAID 5 containing 8 TB drives, would have bandwidth of: (4 drives – 1 drive) * 200 MB/second = 600 MB/second and a capacity of (4 drives – 1 drive) * 8 TB = 24 TB.
• A 4-drive SATA SDD RAID 0 containing 2 TB drives, would have bandwidth of: (4 drives) * 400 MB/second = 1,600 MB/second and a capacity of (4 drives ) * 2 TB = 8 TB.
• A 4-drive SATA SDD RAID 1+0 containing 2 TB drives, would have bandwidth of: (4 drives / 2 ) * 400 MB/second = 800 MB/second and a capacity of (4 drives / 2 ) * 2 TB = 4 TB.

NOTE: Here’s a more detailed explanation of RAID levels such as RAID 0 or RAID 5.

CONNECTION PROTOCOLS

Before we look at codecs directly, there’s one more set of numbers we need to understand: How you connect storage to your computer. The connection protocol has a dramatic effect on performance. It’s important to note that both your storage and computer need to support the same connection protocol. Buying a Thunderbolt 5 device won’t deliver Thunderbolt 5 speeds, unless your computer also supports Thunderbolt 5. Both ends of the connection must support the same protocol.

Data Transfer Rates By Connection Protocol

Protocol	Data Transfer Speed
ThunderBolt 5	Up to 6,000 MB / second
ThunderBolt 3/4	Up to 2,850 MB / second
ThunderBolt 2	Up to 1,400 MB / second
USB 3.1 Gen 2	Up to 1,000 MB / second
10-Gig Ethernet	Up 1,000 MB / second
1-Gig Ethernet	Up to 110 MB / second
USB 3.1 Gen 1	Up to 400 MB / second
Legacy Formats – Generally Too Slow
FireWire 800	70 – 80 MB / second
FireWire 400	20 – 25 MB / second
USB 2.0	10 – 15 MB / second
iSCSI	75 – 95 MB / second

 

Keep in mind that these speeds define the size of the “pipe” that carries the data. The actual data speed is dependent upon they type of drive connected, how many drives are connected, how the drives are configured and, in the case of HDDs, how full the drives are.

With all this background, let’s see how this applies to video editing.

VIDEO FORMAT DATA TRANSFER REQUIREMENTS

New codecs seem to be released every week. Some are optimized for smaller files, other to give a camera manufacturer something proprietary to talk about, others to preserve more color or image data. The good news is that, as the media industry continues to migrate to NVMe SSD storage for primary editing, storage bandwidth becomes less important. However, storage capacity remains critical.

Why? Because virtually all digital filmakers consider storage to be “free,” when compared to film, and therefore shoot far more media than ever before. This means that, as DITs and editors, we need to make sure we have enough storage capacity before the next project rolls in the door. Making this issue even more complex, media storage varies widely by codec – as does image quality and editing efficiency.

NOTE: If you are editing a single camera project with a modern computer, editing efficiency – while meaningful – won’t slow you down much. But, if you are doing multicam editing, it can have a major impact on the performance of your system.

Here’s how to read the table below.

Format indicates the frame size, frame rate and codec.

Store One Hour indicates how much hard disk space it takes to store an hour of material in a particular format. This allows you to estimate what your total storage needs are based upon the amount of material shot. Remember, to add additional space for work files and free space.

Single Stream indicates how much data, on average per second, is required by a single stream of that format. (Measured in megabytes per second: MB/s.)

Editing indicates the approximate bandwidth needed for normal “single-camera” editing in that format.

Multicam indicates the approximate bandwidth needed for a 4-camera multicam edit in that format. (Using proxy files will decrease this number significantly.)

Here’s the key point: The table below helps you to match the required data rate of your video format with the speed of your storage, indicated in the table above.

Bandwidth & Capacity by Video Format

Video Format	Store 1 Hour	Single Stream (MB / second)	Editing (MB / second)	Multicam (MB / second)
SD – rarely used today
DV NTSC / PAL	13 GB	3.75	7.5	15
DVCPRO-50	27 GB	7.5	15	30
Uncompressed 8-bit (Beta SP)	72 GB	20.2	41	85
Uncompressed 10-bit (DigiBeta)	96 GB	26.7	54	110
ProRes 422 (NTSC or PAL)	19.5 GB	5.25	12	24
HD – 720p formats will be smaller
HDV (25 mbps) 60i	13 GB	3.75	7.5	15
AVCHD 1080p/30	Up to 10.8 GB	1.5 – 3.0	6.0	12
AVCCAM 1080p/30	Up to 10.8 GB	1.5 – 3.0	6.0	12
AVC-Intra 1080p/30	45 GB	12.5	25	50
XDCAM HD (50 mbps)	28 GB	7.75	16	32
XDCAM EX 1080p/30	19 GB	5.2	12	24
DVCPROHD 1080p/30	45 GB	12.5	25	50
ProRes 422 (Proxy) 1080p/30	20 GB	5.6	12	24
ProRes 422 1080p/30	66 GB	18.1	36	72
ProRes 422 (HQ) 1080p/30	99 GB	27.5	55	110
ProRes 4444 (no alpha) 1080p/30	148 GB	41.25	85	170
R3D 1080p/30 *	137 GB	38	76	150
HDCAM 1080p/60	834 GB	237	474	948
UHD & 4K
UHD ProRes Proxy/30	82 GB	22.75	46	92
UHD ProRes 422/30	265 GB	73.7	150	300
4K ProRes 422/30	283 GB	78.6	150	300
4K/24 R3D *	586 GB	163	326	650
Sony RAW 4K *	520 GB	300	600	1200
6K & 8K
6K/24 R3D *	660 GB	183	360	720
6K ProRes 422/30	636 GB	176.75	354	707
8K ProRes 422/30	1,131 GB	314.25	628	1,257

 

NOTES

• R3D, Sony and RAW files have a large number of variations based upon compression rate, frame size, frame rate, and the amount of the sensor used. Consult the websites for specific Sony and RED cameras to determine the exact data rates for the media format you plan to shoot.
• All AVC video formats are converted (transcoded) to ProRes422 during ingest into Final Cut Pro, but not other NLEs. So, while the AVC source video is small, the converted ProRes are much larger.
• ProRes is a variable bit-rate encoder, so file sizes vary depending upon format, image size, and frame rate. The HD specs for ProRes are  taken from Apple’s ProRes white paper. File sizes decrease slightly for 720p files, or slower frame rates.
• For the latest on storage requirements for Blackmagic RAW files, please visit this website.
• Here’s a link to learn more about the storage speed and requirements of ProRes RAW.

SUMMARY

When it comes to compressing media, there are no hard-and-fast numbers because compressed file size depends upon six factors, each of which vary by movie:

• Codec of the compressed file
• Frame size of the compressed file
• Frame rate of the compressed file
• Bit depth of the compressed file
• Compression bit rate of the compressed file
• The amount of movement between frames in the source file

To wrap up, here are three key takeaways:

• Understanding the relationships between storage speeds, storage capacity, and video formats greatly simplifies storage budgeting and problem-solving.
• As frame rates or frame sizes increase, storage speeds and capacity increase exponentially.
• When in doubt, assume you’ll need more storage. Modern storage – especially Thunderbolt devices – will be fast enough, but you’ll always need to hold more media.

---

Bookmark the permalink.