First Look: YoYottaID – Powerful Software for Archiving [u]

[ Updated Jan. 27, 2018, with comments from the developer and screens shots from a new build of the software.]

[Click here to read my product review disclosure statement.]

YoYottaID is software that allows you to index, track, backup, transcode, conform, archive and restore media and other project files.

I’ve heard about YoYottaID for years, but have never worked with it until I borrowed an mLogic mTape LTO-drive, which came bundled with YoYotta. After working with the software for a few days, here are my impressions, along with an interview with its creator: Martin Greenwood, CTO of YoYotta.

EXECUTIVE SUMMARY

YoYotta ID is powerful software for media archiving and transcoding hampered by a very awkward interface. Unless you use it daily, you’ll need to create notes reminding yourself how to use it. However, once you learn the interface, there is very little in media management that YoYotta does not appear to handle.

YoYotta is modular. The base package supports media backups to hard drives, indexing media into local databases and searching for files. LTO tape support requires YoYotta LTFS which is £360 or a £200 upgrade. Other modules are available for handling automated tape libraries, conforming media from an archive into a project, and transcoding media.

NOTE: As a policy, I don’t share my reviews with developers prior to publishing them. However, within 24 hours of the release of this review, Martin Greenwood implemented various changes in a new v2.3 (524) build, which will be a free update. I am very impressed by his response.

System requirements: macOS 10.9 or later. LTO drive needed for archiving. Supported drives include those from: 1Beyond, HPE, IBM, mLogic, and Quantum.

Product: YoYottaID
Developer: YoYotta
Website: https://yoyotta.com
Price: YoYottaID starts at £160 for basic media backups. LTO support requires YoYotta LTFS at £360 (roughly $510 (US)). A 10-day free trial is available.

WHAT IT DOES

YoYotta does a lot! Depending upon the modules you purchase, YoYotta can:

• Index, backup and track camera media to hard disks
• Index, backup, track and restore all project files to LTO tape
• Find media needed for a project in a backup or archive, copy it to your hard disk and link it into an existing project (Conform)
• Transcode media into a wide variety of camera formats – ARRI, Sony, RED, Canon and GoPro – into a wide variety of media formats: DPX, MOV, MP4, MXF.
• Handle CDL file processing, 3D LUTs, resizing images, creating DPX sequences, and audio processing for DPX files.

NOTE: Click here to read the details about transcoding in YoYotta.

The Automation module allows YoYotta to control one library with up to 4 LTO drives and a virtually unlimited number of tapes. Supported libraries include: HPE, IBM, MagStor, Oracle, Overland, Qualstar, Quantum, Spectra, StorageTek or Tandberg.

Automated LTO library hardware is far beyond my budget, so I was not able to test any of the automated functions, though the software I’m working with had that module installed.

NOTE: Click here to read more about it’s Automation features.

A more immediate benefit to many filmmakers is YoYotta’s ability to conform a project. What conforming allows you to do is drop in an AAF, ALE or EDL list into the app. It will then search its database for all the referenced clips, then create a list of all drives or tapes that are needed to restore these files to a locally-attached hard drive for editing. At which point, you can click a button and it will find and download all the necessary media.

The Conform option can also, if you want, create new, smaller clips from the source material so that you can save space by not copying vast of amounts of media from the tape back to your hard disk.

“YoYottaID Conform will trim ALEXA Mini MXF, ARRIRAW, DPX, RED R3D, Apple ProRes (including 4444 XQ), Sony F700/F5/F55/F65 MPEG2, SStP, RAW and XAVC files. When trimming ALEXA Mini MXF YoYottaID will automatically create individual ARRIRAW files. Trimming ProRes requires macOS 10.11 or newer.” (YoYotta website)

NOTE: Click here to learn more about its Conform options.

ARCHIVING – THE HEART OF THE MATTER

However, for me, while the other features are useful, the core reason for buying YoYotta is to enable media backup, archiving and restore. The only problem is that the interface is designed for people who use it everyday, because the rest of us are going to get lost.

From my perspective, YoYotta’s interface gets in the way of its power.

Installation is easy. Purchase the modules you need from the YoYotta website then download the 119 MB file. Drag it into the Application folder and installation is complete.

Click the green box at the top to access and install the YoYotta software license.

YoYotta will verify your license via the YoYotta servers, but, after that, you don’t need to be connected to the Internet to use the application.

ARCHIVE OVERVIEW

YoYotta does not use menus, as you can see above. All system control is from 18 buttons or icons in the interface itself.

Some of these icons have tool tips that appear as you hover over them. Most do not. This means you need to click something to figure out what it does. (I sent my console log twice to YoYotta before I realized I should not click a green envelope button.)

Update: Jan. 18. Martin Greenwood writes:

The green envelope button in the log panel has now been replaced, there is a new Send log… dialog. See the first screenshot above.

Also, I’ve added more Tool Tips to the icons.

The Question Mark button, see the screen shot below, accesses the Help files. I’ve added extra steps to the Help, which goes though the different parts of the interface.

Larry continues…

(Click to see larger image.)

Here are the steps to archive a folder, or group of folders. The numbers in the screen shot above correspond to the numbers in the list below.

Steps to Archiving

1. Insert and format a tape – click the LTFS button at the top (Button 1a verifies that the tape formatted correctly.)
2. Click the Book to create a new database to store file and folder records
3. Click the Gear on the top left to enable creating thumbnails as part of the archive process.
4. Drop the folder(s) into the Job pane to create a new project  and queue it for archiving
5. Click the right-hand Plus to add a destination
6. Click Edit Paths to specify if and where to save archive reports
7. Click the Green Arrow to start archiving

If you use the software every day, this may not seem too intimidating. But for a new or occasional user, this is incomprehensible. The icons are scattered all across the interface. The design is clean, simple and impossible to figure out.

It would make life a LOT easier for occasional users to create a simple “Archive Checklist” menu with links to each part of the interface that we need to access prior to archiving a tape. It should list each step in order and allows you to jump to that part of the program by clicking a menu item. Setting system defaults to quickly and successfully create an archive will help as well. There are far too many settings that need to be configured before an archive can be achieved.

It would also be very helpful for the Green Arrow to stay dim until all menu options are properly completed.

THE DETAILS

Let’s see how this works in practice. I inserted an LTO-7 tape into an LTO-8 mTape drive. This allows me to store 9.5 TB of data on an LTO-7 tape. (LTO-8 tapes, which are much more expensive, allow storing 12 TB per tape.)

Click the LTFS button at the top. This allows us to control up to four LTO drives; if you have the Automation module installed.

Click the Format button near the top to erase and format a tape. This process takes a couple of minutes, but only needs to be done once per tape.

YoYotta requires a six letter/digit code to identify each tape. This is a critically important step because every archived file is linked to that tape ID, allowing YoYotta to find any file on any tape at any point in the future.

The Exclamation mark at the bottom of the main screen indicates when formatting is complete.

Click the Book icon, also at the bottom, to create or access the archive database.

(Click to see larger image.)

The database is a list of all the tapes in the library, along with all the files on them. As this is our first archive project, it is empty.

Before we start archiving, we need to create a new database. Click the Gear icon in the lower right of the Project window and click Add Project.

Here, I created a new project called “Webinar Archives.” Once a project is complete, you can close the database window.

NOTE: Just as with formatting a tape, if all your files relate to the same project database, you only need to create a database once. Whether to keep all your files in one large database or spread them across multiple databases depends upon your particular requirements.

BEGIN THE ARCHIVE PROCESS

Once a tape is formatted and a database is created, the process of archiving is straight-forward: Drag the folder(s) you want to archive into the Job section of the main interface.

The Path pane displays all the files that will be archived, while the stats panel in the center shows what I’m backing up. For example, this folder contains FCP X libraries, all of which point to linked media.

Here’s another example where I’m archiving a single media folder containing 23 ProRes 4444 files requiring 873 GB to store.

If you make a mistake, select the folder you want to remove and click the minus ( – ) button in the top left. Sigh… Not to be confused with the minus button in the top RIGHT.

If you want to generate PDF reports, showing thumbnails of what you are archiving, click the Edit Paths button in the top right.

(Click to see larger image.)

This allows you to create ALE, CVS, MD5 or PDF reports – along with the ability to email them to yourself or your producer – illustrating what was archived, when and where.

NOTE: Again, you only need to set up the Reports folder once. After that, new reports are created with each archive and added to the Reports folder automatically.

UPDATE – Jan. 18. In my original review, I wrote that the text box in the lower left was for archive notes. This is incorrect. As Martin explains:

(Click to view larger image.)

Rather than for notes as you described, the box in the Edit Paths panel allows the user to define extra folders that will be created when copying to the destination. In the new build there is now a label above the box. See the screenshot above. Plus there are more details in the Adding Destinations support page.

Larry continues…

When you’ve added all the folders you want, click the Green arrow to start the archive process.

Oops. Apparently I forgot to add a destination.

Click the Plus button in the top RIGHT corner to add a Destination. Here, I selected the formatted tape in the mTape drive. (I assigned it the “MT2345” ID when I formatted it.)

NOW click the Green arrow to start the archive process.

NOTE: From beginning to end, watching movies on the YoYotta website, writing these instructions, this first archive took slightly longer than two hours to complete.

The actual archiving process of writing files to tape is fully automated. You can either watch the on-screen feedback or, my personal preference, go get something to eat or take a nap.

LTO-7 and LTO-8 drives copy media to an LTO tape about 300 MB/second. This speed varies depending upon the files. Large files copy more quickly than smaller files. This is due to the nature of LTO drives and how they write files.

When I was copying thousands of smaller files, they transferred around 60 MB/sec. Copying large media files transferred around 300 MB/sec. Most of the time, you are not waiting around for the LTO drive to finish. Rather, while the software and tape drive are archiving, you are able to concentrate on other work.

As a general rule, allow about 10 minutes per 100 GB for LTO-7 or LTO-8 archiving.

VERIFYING

When YoYotta is done copying files, it then verifies that the copies are precisely correct. As it does so, it creates a green checkbox indicating that everything is fine.

Here are the FCP X Library files that I archived.

However, when I go into the database to look for the library, I can’t find it. I can find MOV and JPG files, but not the library files themselves.

The software says that it copied and verified them, but the database does not make them easy to find or restore.

NOTE: YoYotta’s website advises that all LTO tapes should be verified using MD5. Yet the default setting is Quick Verify. This places a big burden on the new user to know the difference and have the ability to find the setting to change it. Default settings should be set to provide optimum results with the least amount of changes from the user.

UPDATE – Jan. 18. Martin writes:

The Quick Verify preference is off when you first install the app, however after that it’s stored persistently, so once enabled it will stay on. This can be confusing, so in the upcoming v3 release this setting applies to the job, is shown in a clearer way and can be stored into presets. The v3 release (a free update) will also have a Finder style view for restoring material which will be a lot more intuitive than the current datable view.

AN INTERVIEW WITH MARTIN GREENWOOD

To help me better understand YoYotta, as part of this review, I emailed a series of questions to Martin Greenwood, CTO of YoYotta, about the software. Here’s what I learned.

Larry: How would you describe YoYotta?

Martin: YoYotta creates workflow software for film and television customers based globally. We developed a macOS app called YoYottaID that manages your digital assets on location, in production and post.

By the way the Yotta in our name comes from the largest scientific unit. A yottabyte is 1024 bytes or a million million TB. It may be some time before we reach this amount of data, but our users already create projects that have petabytes (1000 TB) of source footage.

Larry: Why did you write YoYotta?

Martin: Initially for backup and verification of camera cards on location, but in 2011 a cross platform filing system called LTFS was introduced for LTO-5 tapes. So we added LTFS + LTO drive support to YoYottaID, then later on added Automation to control the robot inside LTO tape libraries.

Originally tape archives were created in post houses at the end of projects, but now it’s performed as early as possible during production and even on location. Along the way new cameras with higher resolutions, frame rates and dynamic range have arrived, drastically increasing the amount of data.

Larry: What’s the best way for new users to get started with the software?

Martin: Download the YoYottaID demo and follow this video, to test archiving a project with your hardware: vimeo.com/127943273

Don’t leave your archiving until your storage is full ! When new footage arrives keep it separate from the project files on your storage and archive it straightaway. Link to the footage from your grading or editing app, this way you only need to routinely backup project files and new renders.

Larry: What do you advise as “best practices” for filmmakers who need to archive their projects for the long term?

Martin: We suggest that you don’t store any content on the computer, always use external storage. Make at least two copies of all your material. Initially this may be on drives, but as soon as possible make LTFS copies. Store the sets of tapes in different locations. Consider migrating your archive to newer LTO generations, although previous LTO hardware will be available for some time.

Larry: Why should someone use YoYotta, instead of, say, Carbon Copy Cloner?

Martin: Products like Carbon Copy Cloner and SuperDuper work very well for making copies of your system storage. They also allow you to revert system changes or to get a corrupted Mac booting again. In addition to copying, YoYottaID also extracts metadata, creates thumbnails, verifies all copies and makes PDF, CSV and checksum reports.

Whilst this is happening you can use the Transcode option to make DNxHD and ProRes for editorial or H.264 for web approval.

Larry: Key to any archiving software is the ability to find a particular file on a particular tape, especially years after it was stored. What kinds of record keeping and searching do you support?

Martin: All the camera metadata is extracted and stored in a database, so in addition to searching by file name you can also locate material by timecode, codec and resolution. The checksums are also stored making it easy to verify that every time the file is restored it is the same as the original.

Larry: How is YoYotta priced and why did you pick this pricing model?

Martin: YoYottaID LTFS is around $500 for a permanent license with free updates and the license can easily be moved between multiple computers. You can even lend your license to a freelancer or colleague for a fixed period of time after which it returns to you.
Users appreciate the price and flexibility. Right now YoYottaID LTFS is bundled free with the purchase of mTape.

Larry: What is provided in the YoYotta bundle with mLogic, and what additional features are available from your website?

Martin: The bundle includes a full copy of YoYottaID LTFS allowing you to create industry standard LTFS archives of your projects. If you shoot a lot of footage each day and have two mTape units, then YoYottaID LTFS can create two identical copies in parallel, saving time.

In addition to Transcode there is also a Conform option. With this you can drop in editorial timelines (EDL, AAF, ALE or FCP XML) and selectively restore from tape just the clips needed for online or grading. Conform can also trim camera raw files based on the edit creating shorter clips without conversion.

Larry: Where can we go to learn how to use the software?

Martin: You will find videos and support topics on our website at: yoyotta.com/support.html

There is also information about LTO, LTFS, Mac and related computer hardware.
Any questions please get in touch support@yoyotta.com

Larry: In one paragraph, describe why someone should buy YoYotta?

Martin: Powerful, flexible and affordable. That’s not a paragraph… how about, made in Greenwich, London!

SUMMARY

YoYotta is a powerful archiving utility designed for someone that will use it every day with the time to spend reading manuals and watching videos to understand it.

What YoYotta needs are:

• Default settings that allow us to successfully archive a folder without having to create a database, set a destination or configure reports. It should also automatically index an archive and make sure verification is set to MD5 (the default is Quick Verify). Call this “Get Started Mode.” YoYotta is very strong on customization, very weak on helping new users.
• Reports and destinations enabled by default.
• Automatic indexing of tapes; why this is a separate step I do not understand.
• The Green Arrow button should not activate unless all settings are correct for a successful archive; as well, error messages about why the Green Arrow doesn’t work need to be clearer.
• All icons need Tool Tips when the cursor hovers over them.
• Menu items, using words, should be available for key functions such as creating a database, setting a destination, or finding a file.
• Help files that explain whether to create one or more databases for tracking multiple projects, how to find and restore a single file, or how to enable reports. (I looked and couldn’t find any.)
• Better menus. The lack of menus means that we need to spend time deciphering 18 icons and buttons to get the software to work.
• Group buttons into more intuitive groups. Then, label the groups. Buttons are not grouped by function and key buttons – such as tape formatting, license registration or archive start – have no pop-up hint that describes what they do.

There is no easy way to determine if you have the system properly setup. For example, I have still not gotten an archive report to print to PDF, I can’t find specific files on a tape, and thumbnails for MOV files were only generated for the first 60 seconds. I don’t think this is because the software doesn’t work, but the interface makes it impossible to figure out if you have it set up correctly or determine what went wrong.

I am convinced that YoYotta does everything it says it does. However, new users should allow a couple of days to watch all the videos and read the online help before doing any project-critical work. The 10-day free trial will be invaluable in your learning process.

However, I also need to point out the speed with which YoYotta responded to many of the comments in this review. I expect the software to evolve quickly over the next few weeks.

---

Bookmark the permalink.