First Look: YoYottaID – Powerful Software for Archiving [u]

Posted on by Larry

[ Updated Jan. 27, 2018, with comments from the developer and screens shots from a new build of the software.]

[Click here to read my product review disclosure statement.]

YoYottaID is software that allows you to index, track, backup, transcode, conform, archive and restore media and other project files.

I’ve heard about YoYottaID for years, but have never worked with it until I borrowed an mLogic mTape LTO-drive, which came bundled with YoYotta. After working with the software for a few days, here are my impressions, along with an interview with its creator: Martin Greenwood, CTO of YoYotta.

EXECUTIVE SUMMARY

YoYotta ID is powerful software for media archiving and transcoding hampered by a very awkward interface. Unless you use it daily, you’ll need to create notes reminding yourself how to use it. However, once you learn the interface, there is very little in media management that YoYotta does not appear to handle.

YoYotta is modular. The base package supports media backups to hard drives, indexing media into local databases and searching for files. LTO tape support requires YoYotta LTFS which is £360 or a £200 upgrade. Other modules are available for handling automated tape libraries, conforming media from an archive into a project, and transcoding media.

NOTE: As a policy, I don’t share my reviews with developers prior to publishing them. However, within 24 hours of the release of this review, Martin Greenwood implemented various changes in a new v2.3 (524) build, which will be a free update. I am very impressed by his response.

System requirements: macOS 10.9 or later. LTO drive needed for archiving. Supported drives include those from: 1Beyond, HPE, IBM, mLogic, and Quantum.

Product: YoYottaID
Developer: YoYotta
Website: https://yoyotta.com
Price: YoYottaID starts at £160 for basic media backups. LTO support requires YoYotta LTFS at £360 (roughly $510 (US)). A 10-day free trial is available.

WHAT IT DOES

YoYotta does a lot! Depending upon the modules you purchase, YoYotta can:

NOTE: Click here to read the details about transcoding in YoYotta.

The Automation module allows YoYotta to control one library with up to 4 LTO drives and a virtually unlimited number of tapes. Supported libraries include: HPE, IBM, MagStor, Oracle, Overland, Qualstar, Quantum, Spectra, StorageTek or Tandberg.

Automated LTO library hardware is far beyond my budget, so I was not able to test any of the automated functions, though the software I’m working with had that module installed.

NOTE: Click here to read more about it’s Automation features.

A more immediate benefit to many filmmakers is YoYotta’s ability to conform a project. What conforming allows you to do is drop in an AAF, ALE or EDL list into the app. It will then search its database for all the referenced clips, then create a list of all drives or tapes that are needed to restore these files to a locally-attached hard drive for editing. At which point, you can click a button and it will find and download all the necessary media.

The Conform option can also, if you want, create new, smaller clips from the source material so that you can save space by not copying vast of amounts of media from the tape back to your hard disk.

“YoYottaID Conform will trim ALEXA Mini MXF, ARRIRAW, DPX, RED R3D, Apple ProRes (including 4444 XQ), Sony F700/F5/F55/F65 MPEG2, SStP, RAW and XAVC files. When trimming ALEXA Mini MXF YoYottaID will automatically create individual ARRIRAW files. Trimming ProRes requires macOS 10.11 or newer.” (YoYotta website)

NOTE: Click here to learn more about its Conform options.

ARCHIVING – THE HEART OF THE MATTER

However, for me, while the other features are useful, the core reason for buying YoYotta is to enable media backup, archiving and restore. The only problem is that the interface is designed for people who use it everyday, because the rest of us are going to get lost.

From my perspective, YoYotta’s interface gets in the way of its power.

Installation is easy. Purchase the modules you need from the YoYotta website then download the 119 MB file. Drag it into the Application folder and installation is complete.

Click the green box at the top to access and install the YoYotta software license.

YoYotta will verify your license via the YoYotta servers, but, after that, you don’t need to be connected to the Internet to use the application.

ARCHIVE OVERVIEW

YoYotta does not use menus, as you can see above. All system control is from 18 buttons or icons in the interface itself.

Some of these icons have tool tips that appear as you hover over them. Most do not. This means you need to click something to figure out what it does. (I sent my console log twice to YoYotta before I realized I should not click a green envelope button.)

Update: Jan. 18. Martin Greenwood writes:

The green envelope button in the log panel has now been replaced, there is a new Send log… dialog. See the first screenshot above.

Also, I’ve added more Tool Tips to the icons.

The Question Mark button, see the screen shot below, accesses the Help files. I’ve added extra steps to the Help, which goes though the different parts of the interface.

Larry continues…


(Click to see larger image.)

Here are the steps to archive a folder, or group of folders. The numbers in the screen shot above correspond to the numbers in the list below.

Steps to Archiving

  1. Insert and format a tape – click the LTFS button at the top (Button 1a verifies that the tape formatted correctly.)
  2. Click the Book to create a new database to store file and folder records
  3. Click the Gear on the top left to enable creating thumbnails as part of the archive process.
  4. Drop the folder(s) into the Job pane to create a new project  and queue it for archiving
  5. Click the right-hand Plus to add a destination
  6. Click Edit Paths to specify if and where to save archive reports
  7. Click the Green Arrow to start archiving

If you use the software every day, this may not seem too intimidating. But for a new or occasional user, this is incomprehensible. The icons are scattered all across the interface. The design is clean, simple and impossible to figure out.

It would make life a LOT easier for occasional users to create a simple “Archive Checklist” menu with links to each part of the interface that we need to access prior to archiving a tape. It should list each step in order and allows you to jump to that part of the program by clicking a menu item. Setting system defaults to quickly and successfully create an archive will help as well. There are far too many settings that need to be configured before an archive can be achieved.

It would also be very helpful for the Green Arrow to stay dim until all menu options are properly completed.

THE DETAILS

Let’s see how this works in practice. I inserted an LTO-7 tape into an LTO-8 mTape drive. This allows me to store 9.5 TB of data on an LTO-7 tape. (LTO-8 tapes, which are much more expensive, allow storing 12 TB per tape.)

Click the LTFS button at the top. This allows us to control up to four LTO drives; if you have the Automation module installed.

Click the Format button near the top to erase and format a tape. This process takes a couple of minutes, but only needs to be done once per tape.

YoYotta requires a six letter/digit code to identify each tape. This is a critically important step because every archived file is linked to that tape ID, allowing YoYotta to find any file on any tape at any point in the future.

The Exclamation mark at the bottom of the main screen indicates when formatting is complete.

Click the Book icon, also at the bottom, to create or access the archive database.


(Click to see larger image.)

The database is a list of all the tapes in the library, along with all the files on them. As this is our first archive project, it is empty.

Before we start archiving, we need to create a new database. Click the Gear icon in the lower right of the Project window and click Add Project.

Here, I created a new project called “Webinar Archives.” Once a project is complete, you can close the database window.

NOTE: Just as with formatting a tape, if all your files relate to the same project database, you only need to create a database once. Whether to keep all your files in one large database or spread them across multiple databases depends upon your particular requirements.

BEGIN THE ARCHIVE PROCESS

Once a tape is formatted and a database is created, the process of archiving is straight-forward: Drag the folder(s) you want to archive into the Job section of the main interface.

The Path pane displays all the files that will be archived, while the stats panel in the center shows what I’m backing up. For example, this folder contains FCP X libraries, all of which point to linked media.

Here’s another example where I’m archiving a single media folder containing 23 ProRes 4444 files requiring 873 GB to store.

If you make a mistake, select the folder you want to remove and click the minus ( – ) button in the top left. Sigh… Not to be confused with the minus button in the top RIGHT.

If you want to generate PDF reports, showing thumbnails of what you are archiving, click the Edit Paths button in the top right.


(Click to see larger image.)

This allows you to create ALE, CVS, MD5 or PDF reports – along with the ability to email them to yourself or your producer – illustrating what was archived, when and where.

NOTE: Again, you only need to set up the Reports folder once. After that, new reports are created with each archive and added to the Reports folder automatically.

UPDATE – Jan. 18. In my original review, I wrote that the text box in the lower left was for archive notes. This is incorrect. As Martin explains:


(Click to view larger image.)

Rather than for notes as you described, the box in the Edit Paths panel allows the user to define extra folders that will be created when copying to the destination. In the new build there is now a label above the box. See the screenshot above. Plus there are more details in the Adding Destinations support page.

Larry continues…

When you’ve added all the folders you want, click the Green arrow to start the archive process.

Oops. Apparently I forgot to add a destination.

Click the Plus button in the top RIGHT corner to add a Destination. Here, I selected the formatted tape in the mTape drive. (I assigned it the “MT2345” ID when I formatted it.)

NOW click the Green arrow to start the archive process.

NOTE: From beginning to end, watching movies on the YoYotta website, writing these instructions, this first archive took slightly longer than two hours to complete.

The actual archiving process of writing files to tape is fully automated. You can either watch the on-screen feedback or, my personal preference, go get something to eat or take a nap.

LTO-7 and LTO-8 drives copy media to an LTO tape about 300 MB/second. This speed varies depending upon the files. Large files copy more quickly than smaller files. This is due to the nature of LTO drives and how they write files.

When I was copying thousands of smaller files, they transferred around 60 MB/sec. Copying large media files transferred around 300 MB/sec. Most of the time, you are not waiting around for the LTO drive to finish. Rather, while the software and tape drive are archiving, you are able to concentrate on other work.

As a general rule, allow about 10 minutes per 100 GB for LTO-7 or LTO-8 archiving.

VERIFYING

When YoYotta is done copying files, it then verifies that the copies are precisely correct. As it does so, it creates a green checkbox indicating that everything is fine.

Here are the FCP X Library files that I archived.

However, when I go into the database to look for the library, I can’t find it. I can find MOV and JPG files, but not the library files themselves.

The software says that it copied and verified them, but the database does not make them easy to find or restore.

NOTE: YoYotta’s website advises that all LTO tapes should be verified using MD5. Yet the default setting is Quick Verify. This places a big burden on the new user to know the difference and have the ability to find the setting to change it. Default settings should be set to provide optimum results with the least amount of changes from the user.

UPDATE – Jan. 18. Martin writes:

The Quick Verify preference is off when you first install the app, however after that it’s stored persistently, so once enabled it will stay on. This can be confusing, so in the upcoming v3 release this setting applies to the job, is shown in a clearer way and can be stored into presets. The v3 release (a free update) will also have a Finder style view for restoring material which will be a lot more intuitive than the current datable view.

AN INTERVIEW WITH MARTIN GREENWOOD

To help me better understand YoYotta, as part of this review, I emailed a series of questions to Martin Greenwood, CTO of YoYotta, about the software. Here’s what I learned.

Larry: How would you describe YoYotta?

Martin: YoYotta creates workflow software for film and television customers based globally. We developed a macOS app called YoYottaID that manages your digital assets on location, in production and post.

By the way the Yotta in our name comes from the largest scientific unit. A yottabyte is 1024 bytes or a million million TB. It may be some time before we reach this amount of data, but our users already create projects that have petabytes (1000 TB) of source footage.

Larry: Why did you write YoYotta?

Martin: Initially for backup and verification of camera cards on location, but in 2011 a cross platform filing system called LTFS was introduced for LTO-5 tapes. So we added LTFS + LTO drive support to YoYottaID, then later on added Automation to control the robot inside LTO tape libraries.

Originally tape archives were created in post houses at the end of projects, but now it’s performed as early as possible during production and even on location. Along the way new cameras with higher resolutions, frame rates and dynamic range have arrived, drastically increasing the amount of data.

Larry: What’s the best way for new users to get started with the software?

Martin: Download the YoYottaID demo and follow this video, to test archiving a project with your hardware: vimeo.com/127943273

Don’t leave your archiving until your storage is full ! When new footage arrives keep it separate from the project files on your storage and archive it straightaway. Link to the footage from your grading or editing app, this way you only need to routinely backup project files and new renders.

Larry: What do you advise as “best practices” for filmmakers who need to archive their projects for the long term?

Martin: We suggest that you don’t store any content on the computer, always use external storage. Make at least two copies of all your material. Initially this may be on drives, but as soon as possible make LTFS copies. Store the sets of tapes in different locations. Consider migrating your archive to newer LTO generations, although previous LTO hardware will be available for some time.

Larry: Why should someone use YoYotta, instead of, say, Carbon Copy Cloner?

Martin: Products like Carbon Copy Cloner and SuperDuper work very well for making copies of your system storage. They also allow you to revert system changes or to get a corrupted Mac booting again. In addition to copying, YoYottaID also extracts metadata, creates thumbnails, verifies all copies and makes PDF, CSV and checksum reports.

Whilst this is happening you can use the Transcode option to make DNxHD and ProRes for editorial or H.264 for web approval.

Larry: Key to any archiving software is the ability to find a particular file on a particular tape, especially years after it was stored. What kinds of record keeping and searching do you support?

Martin: All the camera metadata is extracted and stored in a database, so in addition to searching by file name you can also locate material by timecode, codec and resolution. The checksums are also stored making it easy to verify that every time the file is restored it is the same as the original.

Larry: How is YoYotta priced and why did you pick this pricing model?

Martin: YoYottaID LTFS is around $500 for a permanent license with free updates and the license can easily be moved between multiple computers. You can even lend your license to a freelancer or colleague for a fixed period of time after which it returns to you.
Users appreciate the price and flexibility. Right now YoYottaID LTFS is bundled free with the purchase of mTape.

Larry: What is provided in the YoYotta bundle with mLogic, and what additional features are available from your website?

Martin: The bundle includes a full copy of YoYottaID LTFS allowing you to create industry standard LTFS archives of your projects. If you shoot a lot of footage each day and have two mTape units, then YoYottaID LTFS can create two identical copies in parallel, saving time.

In addition to Transcode there is also a Conform option. With this you can drop in editorial timelines (EDL, AAF, ALE or FCP XML) and selectively restore from tape just the clips needed for online or grading. Conform can also trim camera raw files based on the edit creating shorter clips without conversion.

Larry: Where can we go to learn how to use the software?

Martin: You will find videos and support topics on our website at: yoyotta.com/support.html

There is also information about LTO, LTFS, Mac and related computer hardware.
Any questions please get in touch support@yoyotta.com

Larry: In one paragraph, describe why someone should buy YoYotta?

Martin: Powerful, flexible and affordable. That’s not a paragraph… how about, made in Greenwich, London!

SUMMARY

YoYotta is a powerful archiving utility designed for someone that will use it every day with the time to spend reading manuals and watching videos to understand it.

What YoYotta needs are:

There is no easy way to determine if you have the system properly setup. For example, I have still not gotten an archive report to print to PDF, I can’t find specific files on a tape, and thumbnails for MOV files were only generated for the first 60 seconds. I don’t think this is because the software doesn’t work, but the interface makes it impossible to figure out if you have it set up correctly or determine what went wrong.

I am convinced that YoYotta does everything it says it does. However, new users should allow a couple of days to watch all the videos and read the online help before doing any project-critical work. The 10-day free trial will be invaluable in your learning process.

However, I also need to point out the speed with which YoYotta responded to many of the comments in this review. I expect the software to evolve quickly over the next few weeks.


Bookmark the permalink.

4 Responses to First Look: YoYottaID – Powerful Software for Archiving [u]

  1. Carl Williams says:

    If my experience is any indicator, the answer to most of your tape / archival questions is “LTFS”. Since Yoyatta (and most other Mac backup tools) is dependent on LTFS to access tape, it can only do what LTFS allows. That’s why there are so many manual steps when writing archives using tape. While all of the other features surrounding media manipulation are nice, you can achieve them using the generic open source FFMpeg tool suite and a wrapper such as iFFmpeg or ff•Works ($25 or so). I gave Yoyatta:ID a try and it’s far from novice friendly and suffers from the same LTFS issues that we ran into just trying to use LTFS from the command line. For now, I’m sticking with 256GB SanDisk SD cards and my safe deposit box for archival since my shoots are usually 200GB or smaller when wrapped.

  2. Derrick says:

    Larry,

    There is a bit of misinformation in this article and the mTape article regarding the new LTO-7 type M format (also known as M8, which allows you to use 9tb of space on an LTO-7 cartridge). You state in your review that it’s as simple as putting an LTO-7 tape into the LTO-8 deck will format it into the M8 format. But nowhere in your review do you say how you accomplished this. I had to do some deep digging, since LTFS is all over the map, and there is seemingly contradictory information depending on which vendor you seek information from – especially in regards to the implementation of M8. What I concluded, is that unless you have a tape library system (not a standalone tape drive like the mLogic one) then you have no way of initializing a tape for M8, and will need to wait to purchase “pre-initialized” M8 tapes from vendors as they become available. I reached out to both YoYotta and mLogic for more information. mLogic simply pointed me to YoYotta to find out more information – here is what Martin from YoYotta has said on the matter as of 03/06/18. “We are still testing Type-M Formatting.
    Note that Type-M is quite different to other LTO formats as the tapes will not be readable in future drive generations.
    So we would recommend using LTO-8 tape for any long term archives.” So I am a bit confused as to how you accomlished 9.5tb of space with using an mTape with the YoYottaID software. Larry, you’ve always proven to be extremely thorough, articulate, and well researched. But now I am bit dismayed because I upgraded our LTO-6 drive to LTO-8 purely based on the two articles you wrote regarding LTO-7 m8 capabilities. I am still holding out that there is some simple step that I missed out on, and that you would be able to clarify so that I can take advantage of this capability. If not, then I hope you will update your article to reflect what’s possible currently with mTape and YoYotta, regarding LTO-7 type M. Thank you.

    • Larry says:

      Derrick:

      Thanks for writing.

      As I was writing this article, I was working with both Yoyotta and mLogic. As LTO-8 tapes had not yet shipped all my testing and screen shots were done using LTO-7 tape.

      When I formatted the tapes, I used the default settings that Yoyotta provided. The LTO-7 tape formatted perfectly and delivered the through-put that I was expecting, around 300 MB/second. What I did not check was the total capacity of the tape, as that didn’t occur to me until I read your comment above.

      I no longer have the drive or the software, having returned them after my testing was over. But, in thinking about it, it is possible, that this formatted the LTO-7 tape on an LTO-8 drive in the LTO-7 format. Which would mean that total storage capacity was only 6 TB, not 9.5. I will reach out to the team at mLogic to see if I can learn more.

      However, I think you were wise to upgrade from LTO-6 to LTO-8 anyway, because LTO-8 drive no longer read LTO-6 tapes. This means that all tapes in LTO-6 format will need to be converted to the latest version of LTO in order to be readable in the future. Also, you make an EXCELLENT point that Type-M, the hybrid LTO-7 format, is not a long-term storage option. Like you, I would recommend that people avoid it. Stay with plain LTO-7 or LTO-8 formats for the security of knowing it will be readable on multiple devices going forward.

      Larry

  3. Pingback: YoYottaID - Poderoso Software para Archivar - Marcass

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends

Final Cut Pro X 10.4

FCPX Complete

Edit smarter with Larry’s brand-new webinars, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.