Dakota McLearn sent me a great question, which sparked this article:
I am in the middle of setting up our content team out of our office here in Las Vegas.
We are considering buying the G-Speed Studio XL but my only concern is can I get several editors to work of the one data storage device? Is there a way to network the storage to efficiently be able to edit off several computers at once?
I’ve been getting this question a lot recently, and finally decided to write up a more detailed answer.
THE SHORT ANSWER
Well, it depends.
THE LONGER ANSWER
Gosh, this is complicated.
Let’s tackle this in sections:
THE HARDWARE
The G-Technology Studio XL RAID is a stand-alone RAID that connects to your computer via Thunderbolt 2. I like G-Tech. They make good gear.
This means that in order to have multiple users connect to this storage, you’ll need to connect it to a server. This could be as simple as a desktop Mac running with personal file sharing turned on in the Sharing preferences pane; as illustrated above.
However, while this works – in fact, I use it here at the office – it is not high-performance.
A better choice is to dedicate a Mac specifically as a server, which can handle the demands of multiple users at the same time.
For small workgroups, Mac Minis are commonly used. However, for more intense operations, the purchase of a system specifically designed as a server makes more sense – however, servers, especially with lots of storage, are not cheap.
I’ve used systems from Synology, ProMax, EditShare, and others. If you use a Mac Mini, find an older one – unless Apple updates them soon – that has 4 cores. You’ll need to buy one used.
Servers come with their own storage, which means the G-Tech unit is not needed, unless you are using a Mac Mini, in which case you would need the G-Tech system. Determining factors in buying a server include:
If you have older Ethernet cabling, you’ll need to upgrade it (meaning pull new wires) to Cat-6 cable or better.
AN ALTERNATE HARDWARE APPROACH
While servers provide the highest performance, they are also the most expensive.
An alternative approach is using a NAS (Network Attached Storage) or a SAN (Storage Area Network). This is a stand-alone unit that acts as a server and storage all in the same box. These tend to be less expensive than a server, but may not have the performance you need.
SAN vendors include 1Beyond, Synology and LumaForge.
NETWORK PERFORMANCE
I’ve used that term “performance,” but what does it mean? Performance, in the sense I use it here, refers to the ability of the storage system (server and storage, or NAS) to deliver files to individual editors so that they can get their work done without being delayed by the storage system.
Storage is measured three ways, when it comes to media:
Of these three, the most obvious is capacity, but the most important is bandwidth. Seek times are highly relevant for multicam editing.
A typical video codec used in Mac-based editing is ProRes. ProRes has a variety of “flavors,” but the two most common are ProRes 422 and ProRes 4444. If you are editing 1080p video at 30 frames per second, each stream of ProRes 422 video requires about 18 MB/second; ProRes 4444 requires about 40 MB/second.
Data rates (bandwidth) vary depending upon the frame size of the video, the frame rate of the video and the version of ProRes (codec) you are using; along with the number of streams of video required by each editor. Multicam editing requires far more bandwidth than traditional editing.
NOTE: Here’s a link to Apple’s White Paper on ProRes which details bandwidth requirements for different frame sizes and versions of ProRes. The key bandwidth information starts on page 21.
If your server is connected to the network switch using 1 gigabit Ethernet, that means that the server can feed data to the network at about 105 MB/sec. (Yes, the Thunderbolt connection is FAR faster, but the gating factor on speed is how the storage system is connected to the network through the switch.)
This means that if every editor and the storage system is connected to the switch via 1 gigabit Ethernet and each editor needs enough bandwidth to smoothly deliver two streams of video (which is typical in non-multicam video editing), then the storage system can support up to three editors using ProRes 422, but only two editors using ProRes 4444.
NOTE: This assumes that no other network activity occurs when editors are editing. In real-life, this example may be a bit optimistic.
If any of the editors are doing multicam editing, they will need to use ProRes Proxy files, and, even then, they would be limited to about four cameras in the multicam edit.
If frame sizes increase, so do bandwidth needs. It is VERY easy to exceed the speed limitations of 1-gig Ethernet. Faster performance requires rewiring your network to support 10-gig Ethernet or fiberoptic connections. This significantly increases the bandwidth of your network, but also increases the cost of cabling, server connections and the network switch.
NOTE: For example, a single stream of UHD (4K) ProRes 422 30 fps video is 66 MB/sec. ProRes 4444 requires 166 MB/sec, which exceeds the speed of the network itself!
AN ALTERNATIVE
Clearly, if you have a workgroup, a high-performance server with plenty of storage, with edit bays cabled using fiber is the best option. But this isn’t cheap.
An alternative is to attach a high-performance RAID to each workstation, then use the server solely to transfer files from one system to the next, rather than to try to edit directly from the server.
While this is not cost effective for more than a few editors, it can save a ton of money on server and cabling costs for groups of four or less.
SUMMARY
All too often, it is easy to think: “Wow! I have a large RAID here. How can I get everyone to use it?” This is a great question, but unless you understand how the whole system fits together, you will be spending money for the wrong storage and wondering why everyone is having problems.
There are many excellent storage companies and value-added resellers (VARs) that support them who can spend time looking at your editing needs and make recommendations on the products that are most appropriate to your situation.
Planning does take a bit of time, but it can save you wasting money and time and massive headaches by asking the right questions before you start buying gear.
As always, let me know what you think.
2 Responses to The Perils and Pitfalls of Sharing Storage
While you state that simply duplicating needed footage to the workstation that requires it is better for fewer edit bays, doesn’t the extremely low price of hard drive storage make this a better option if building a capable network is too much for the situation? The only complication would be the sharing of the master project file in Premier or Final Cut. I did this recently in Premier by cloud sharing only the project file. All it required was a quick re-link of files on either side. Of course a bit of coordination about who was working on it when was needed. “Save As” was a big help. Final Cut X (my preferred NLE) seems to be a little more difficult moving just the project files but I could be wrong. With FCPX I usually ship an entire drive with all the assets.
William:
I agree, thought needs to be given when sharing project files, first, because only one editor can use the project file at a time, and second in terms of keeping everyone on the current version.
For small files that change frequently, The Cloud or slower server work great.
Larry