Cloud-Based Archiving May NOT Be a Good Idea

My article on storage last week prompted a very interesting conversation with Rony Sebok, VP for 1 Beyond. She pointed out that Cloud-based storage sounds like a great idea, until you run the numbers.

I was so intrigued by her comments that I wanted to write them up for this week’s newsletter. So, I sent her the following questions via email. I think you’ll find her responses fascinating.

NOTE: Over the 20+ years 1 Beyond has been in business, they’ve evolved as a company. They started out building DV Editing systems, then direct-attached storage, then shared storage, then file-based workflow products for TV and Film (Data Wranglers, On-set Review stations), then LTO-Archive products and now they’ve added Recording, Streaming, Conferencing products and cameras. It’s this breadth of experience and exposure that gives them a unique position in the market.

They are now focusing on four key areas: Cameras, Recording/Streaming/Conferencing, Storage, and Archive, as they see these to be the biggest growth areas. Everything they sell, they manufacture.

Website: 1Beyond.com

OUR DISCUSSION

Larry: Cloud-based storage provides media creators with an unlimited capacity to store their files. What’s the problem with this?

Rony Sebok: The big difference between Cloud-based archiving and local archiving (in this case LTO tape) is 3 things:

1. With Cloud storage you pay for the same storage over and over again (via their monthly fees). You may find inexpensive Cloud storage (eg $5/TB/month). But if you wish to keep those assets long term, that TB will cost you every month and every year: $60 the first year, $60 the next year and so on. By the end of 5 years you’ve paid $300 for that TB.

Local archiving storage you only pay for once. At current prices, LTO-6 tape is $26.50 per 2.5 TB tape, that’s $11/TB. LTO-7 tape is $100 per 6 TB tape, that’s $17/TB. So, that 1 TB will only cost you $11-17 – for the life of that tape! 5 years later you still have that asset and you only had to pay $11-17 for it, not $300 and counting.

Now think about your video library. You don’t have only 1 TB. People these days are running out of storage and they have 48 TB+ on their NAS/SAN storage. All of a sudden that $300 and counting is looking more like $14,000 and counting.

NOTE: $300 per TB for five years times 48 TB equals $14,400.

2. Transferring and retrieving files to/from local storage is FAST. You can write to LTO-7 tape at 260 MB/sec (that’s 2080 mbps). That’s 2 gigabits.

I don’t know many post-houses with that kind of internet bandwidth. Most of us have 10 mbps upload speeds. But even at 100 mbps speeds, copying to the Cloud — and, for that matter, retrieving files from the Cloud, will take 10-20x longer. And with the files sizes we are working with now (HD, 4K), an hour of video can be anywhere from 100 GB (that’s ProRes 220 mbps) to 1575 GB (that’s Phantom Flex 4K RAW). ProRes 4444 XQ 4K is 764 GB/hour.

The copy time for that hour to LTO-7 would be 50 minutes. Copy time to/from Cloud at 100 mbps would be 17 hours. And that’s just 1 hr of footage! Would you want to wait 17 hours to get your file back?

Larry Notes: The fastest uplink I can get where I live is 2.0 mbps. At that rate, transferring that same file would take me about a week.

3. Someone else has your data, and there are no guarantees they will be around in 5-10 years. The Cloud storage providers I’ve seen all have fine print disclaiming responsibility for the data and recommending you keep a backup.

Larry: Regarding this “last-mile” connection, why not just upgrade your connection speed to the Internet?

Rony: This is a nice thought. You’ll pay for that extra speed. And, in most locations it’s just not available. We tried to get fibre to our office and couldn’t.

Larry: What’s the difference between short-term file storage and long-term archiving?

Rony: If you just want to back up data temporarily and you plan to delete and reuse the space, then the situation is different. You could get a set amount of Cloud storage and reuse it (if you don’t care about how long it takes to backup/restore).

Instead, we’d recommend a backup NAS (local disk storage that may not be as fast as your SAN, but provides backup) or LTO rather than Cloud. You’ll get access to the data faster if/when you do need it.

You may choose to use Cloud storage for smaller files (eg. project files) or graphics or to share a finished sequence and then remove it when the recipient receives it.

Larry: Cloud storage costs just pennies a gigabyte, while buying an LTO system costs thousands of dollars. Where does this make sense?

Rony: The cost of a 1 Beyond ThunderTape 3 with LTO-7 drive is $4,795. This pays for itself in less than 2 years if you’re archiving 6 TB of data a month. For most, that’s just 3 days of shooting a month (at 2 TB/day).

Larry: An LTO-7 tape cartridge holds 6 TB of data. Assuming we add 6 TB of new data a month, what does your spreadsheet illustrate?

Rony: 6 TB takes 7 days at 24hrs/day to upload at 100 mbps. 6 TB takes 6.5 hours to archive to tape. If you look at the specific costs:

• By the end of year 2 you’ve not only paid off the 1 Beyond ThunderTape 3, but you’ve saved $1,800.
• By the end of year 3, you’re ahead by $11,590. You could buy a brand new 1 Beyond ThunderTape and still be ahead $6,800; assuming prices haven’t dropped (which they usually do)

For long term archive or raw camera assets, I really don’t see a better alternative right now.

I also want to mention that you can lease the 1 Beyond ThunderTape 3 (or NetDrive). For those who prefer to pay monthly and not have the capital outlay of the $5K tape drive, it’s possible to do it that way.

Then you could really do “apples to apples” comparing just monthly costs. The monthly cost of the drive and the tapes would be MUCH LESS than the monthly cost of Cloud storage, once the amount you have in the Cloud exceeds X TB.

I haven’t done all the math, yet, so I can’t make the complete argument. But, suffice it to say that if people prefer to pay monthly and not have the capital outlay of the $5 K tape drive, it’s possible to do it that way and then the savings from LTO are IMMEDIATE. You don’t have to wait 2 years to get your money back.

Larry: Are there on-going costs with LTO tape, the way there are with Cloud storage?

Rony: No. If you purchase a 1 Beyond ThunderTape 3 (or the 1 Beyond NetDrive), you can purchase an extended warranty and then there are no costs during the term of the warranty. You’ll need to purchase an LTO cleaning cartridge (one cartridge). This costs $50 and should last for at least 3 years (good for 50 cleanings).

Larry: How about security, which is more secure?

Rony: I would suggest making a backup copy (two tapes) of all data and keeping one off site. As we discussed above, the tape media is very inexpensive, using two tapes doesn’t change the math, much. As I mentioned above, the Cloud vendors do not guarantee they won’t lose your data or that it won’t be compromised.

Having control over your own data seems to me to be the most secure.

Larry: What advice do you have for media creators trying to figure out their long-term archive strategy?

Rony: Backup and archive are necessities in our business. We’ve all had experiences where data is lost due to hard drive failure (or know someone with this experience).

It’s worth differentiating between short-term backup of work in progress or assets you will not need in the future, and archives of assets that have value that you may need in 1 to 15 years. You may decide to use a second hard disk for short-term backup. LTO, at present, is the best longer-term archive medium from a cost and speed of access standpoint (assuming you are accumulating TB’s of data a month).

Any raw footage that would be difficult or costly to reshoot would be something I’d suggest archiving to tape right away. LTO tape costs less than a hard drive per GB and this backup could then be your long-term archive, also. I recommend backing up footage to hard drive and LTO right away. You can work off the hard drive and you have the tape as backup.
If you don’t need the footage long term, you can always re-use the tape (just format it and then it’s free to use again).

If you want a long term archive, I’d suggest making a second LTO copy of the assets you want to keep longer. Keep one tape in-house and another offsite.

SUMMARY

Archiving the massive files media creators have to work with remains difficult. LTO tape is a solid option, but Apple has not optimized the Finder to make working with tape easy. It would make our media life a lot easier if Apple would support tape as part of the operating system.

The Cloud is easy, but high fees over the long term and bandwidth limitations make it inaccessible or unaffordable.

On top of all of this, we still need to deal with the never-ending obsolescence of technology. Codecs and video formats die over night. As well, we need media asset management systems that allow us to track the hundreds of thousands of files we are archiving so that we can actually find them again when we need them.

I’m grateful for Rony’s comments and for allowing me to share them with you.

As always, I’m interested in your comments, as well.

---

Bookmark the permalink.