[ This article was first published in the July, 2010, issue of
Larry’s Final Cut Pro Newsletter. Click here to subscribe. ]
One of the fun things about researching my weekly webinars is all the new technology I get to learn and reflect upon.
I was introduced to tapeless media when I was doing a project last year using both P2 and XDCAM EX formats, media that is shot on cards. Without a doubt the most important point to keep in mind is that you MUST copy the entire contents of the card to its own folder on your hard disk. Don’t just copy a few files. Copy everything.
Second, put the contents of each card in its own folder. And how you name that folder is important. (More on that in a minute.)
As part of my recent research for the Tapeless and DSLR Media webinar, I sent a note to a variety of editors asking for their thoughts on working with tapeless media. Specifically, I wanted to learn what problems they were running into.
I was fascinated by the results. Take a look.
Stephen Kanter
I’ve just finished 2 DSLR jobs, and the big issues are:
1. Ignorance of the actual shooting spec. Is it 29.97 or true 30? Is it progressive, progressive segmented frame or interlaced? Not one of the people I have worked with knew the answers.
2. Workflow path for ingest. On both jobs, users just used the raw files. One literally dragged them into a project and cut in H.264, discovering only at the end that they couldn’t get the movie exported properly. Another used Compressor to transcode to ProRes. However it is possible this was done wrongly, as FCP and Compressor assumed 29.97 interlaced, when I suspect the true frame rate was 30. Why, oh why, did they not use Log and Transfer with new Cannon plug-in? Again, if you know the answer to 1. above, feel free to use other methods. But no one knows 1.
3. Cards must be copied and backed up AS IS with folder structures intact. Fee photographers seem to understand this. Plus, tracking original filename if they choose to change clip name.
Shane Ross
#1 – Importing from the cards, then erasing the cards. not backing up the cards at all…only keeping the imported media. That means that they have no backup. Or, if they backup THAT footage, then it is tied to that NLE…or makes it difficult to get to another. PLUS, the imported footage might be larger than the originals, therefore the backup takes up more space. H.264 to ProRes, or AVCHD to ProRes…the originals take up less space.
#2 – Backing up only part of the cards. Like only the VIDEO and AUDIO folders of P2, or only the H.264 files of the Canon DSLR cards. Because FCP needs the FULL file structure in order to import this footage via Log and Transfer. And the advantage of using Log and Transfer with Canon DSLR footage is that it adds TIME OF DAY code from the camera, instead of having each and every clip have a start time of 00:00:00:00 like MPEG StreamClip or Compressor does. And it assigns the clips the REEL number of the backup folder you backed up the footage into.
I’m sure there are more, but those are the biggies.
Robbie Coblentz — MidSouth Duplication
Not having an adequate backup strategy thought out to prevent accidental erasure or non-copying. We build in redundancy at almost every point here, including long term archive to LTO.
The sometimes fragile nature of all solid state and hard drive media can’t be underscored enough. Hard drives will fail. SD cards will go belly up. This stuff is still a bit bleeding edge.
You also need to have a standard way of labeling assets to make reconstructing in the future easier. We mark all our footage data folders with a Date_Client_Project tag. Makes it easier for everyone to figure out what someone else was doing.
[However, I am very impressed with] how awesome it is to be able to grab a shot off a hard drive without having to cue up a tape, shuttle to the appropriate segment and capture. Log and transfer is great!
Steve Oakley – Digital Illusionist
[Tapeless is] not any faster than tape. To just dump a card, its maybe 4X to 6X faster then RT which seems great, but once you figure in backup to at least one more drive, that’s cut in 1/2. tapeless is great for news, for super quick turn work where you don’t care about anything except getting something cut ASAP.
The next problem is having duplicate media file names like MVI_1234
Shooting double system sound is extra gear, and extra work. any time saved shooting tapeless was just eaten up syncing sound, PluralEyes or not. DSLR’s don’t have timecode, they don’t even put the Time-of-Day as the timecode of a clip, it always starts at 00:00:00:00. incredibly stupid. I’m hoping a third party will come along with a utility to take the file creation time and use it to set ToD TC into the file. Pat Pending.
[As a concern,] long-term backup will be a problem because tape is cheap and reliable. I’ve got 15 year old beta SP’s that play fine. how many 15 year old hard drives do you have working or even have an interface you can connect them to to try ? thankfully 2 TB drives are down to $129 now to back up all the 320-500G drive’s I’ve got right now. Until there is reliable long term digital storage format where the media is very stable, and has an interface that’s going to be accessible.
Ben Daube
[When it comes to] labeling and Backup/Archive, you must be ruthlessly disciplined in labeling and boringly consistent in copying/backing up/archiving.
[On the plus side, how] refreshingly easy and liberating the process is!
And the instant review/playback; no cards to lose with built-in memory; and no more dropouts or tape caught in rollers, yeah!!
Jarle Leirpoll
I use the SxS cards in my EX-1 camera, and I will NEVER go back to tape. Transferring the files to disc is lightning fast, as these cards (and the express card slot) have an incredible speed. I also have a RAID 0 on my laptop, so the file transfer is really quick.
Duplicate file names will not happen if you make sure to give your cameras different prefixes. I have my own camera that spits out files with name starting with JL (my initials), and any other camera I use will be set to something else.
I use Premiere Pro CS5 to edit this material, so no transcoding is necessary. Just copy to drive, import to Premiere and edit.
Backup can easily be done while editing, so no extra time is used for that.
All in all, the file based workflow is MUCH faster and easier than tape.
The SxS cards are not cheap, but I think of them as a part of the camera, not as a kind of “tape”. They’ll be used thousands of times before they wear out. I have two 16 GB cards and two 8 GB cards, and I have never filled up all of them in one day.
Also, for long time storage of my projects, I use external drives. Hard drives have been cheaper than DV tape per gigabyte for many years now, and the difference grows bigger every year. And opening a project from tape is not easy… Opening it from the hard drive is quick and easy, so making new versions is very fast.
I don’t really care how long my hard drives will last. I just bought a few 2 TB RAIDs, and copied all my old projects onto two of them (yes, I want backup). My data never stays long enough on one media (drive) for it to go bad. I expect this to be the case for many years, until we get insane
capacity solid state drives fort permanent storage. Petabyte flash memory has been mentioned…I highly recommend file-based workflow. And I really, really hate to work
with tape now.
Noah Kadner
The biggest mistakes people make who are new to tapeless are:
- Not making backups of the entire original media card.
- Not understanding the structure of the media card and how to repair broken clips with the built-in repair services of the cameras; though not all cameras have this.
- Editing directly with the H.264 or Motion JPEG files generate by many DSLR camera rather than transcoding to ProRes or another more suitable editing format first.
In reflecting on their comments, there are many advantages to going tapeless:
But there are also downsides:
I’ve learned that with Final Cut Pro 7, always use Log & Transfer to ingest your media.
Sony recently updated their XDCAM Transfer utility software so all ingesting (the process of bringing media into Final Cut) can be done using Log & Transfer. [See the note from Andy Mees below for more information on this.]
Also, for DSLR footage, Canon recently released a free new utility called — EOS Movie Plug-in — that allows you to transcode your DSLR footage using Log & Transfer. This utility has three big benefits:
If you want to see step-by-step illustrations on how to work with the following tapeless media formats, check out my Webinar on Tapeless & DSLR Media. In it we cover:
If you want to know how to convert from one video format to the next, check out my webinar on Video Transcoding Compression Techniques.
UPDATE – Aug. 10, 2010
Peter Darg adds:
Thank you for review of handling tapeless media in the July newsletter.
One discovery about the new technology which I found to my regret was shooting a scene on a Sony EX-1 and then ejecting the sxs card as soon as the red LED turned green.
I wanted to check everything on my MacBook Pro. However, to my horror the file was not complete and could not be read by Sony software or FCP.
Conclusion: don’t be in a hurry to eject the card even if the LED has turned green.
Give the card at least another minute in the camera slot after the green to completely finish writing all the data (just as a good baker leaves the cake in the oven an extra minute to let it brown a little).
Dan Novak adds:
I just wanted to send a quick note regarding one of your latest articles “Thoughts on working with tapeless media”.
I have an older Mac Book Pro and a Panosonic DVX100B which I connect via mini DV/Firewire cable (I know kind of old school).
I capture the footage in iMovie (its quick and easy) and use the laptop as a back up until the project is complete.
To get the footage from the laptop to my Desktop for editiing I connect the two computer together using a Firewire 800 cable and transfer it via Target Disk Mode.
This process really cuts down on the capturing time if I were to record to tape.
Thanks for all the great articles.
Richard Krentz also sent his comments:
Larry I see you’re as solid as FCP, always good to read your news letter, I have never came away without some good knowledge. On the tapeless media, i just use it once and throw everything away, I am only interested in what comes out the first time, it is so easy to just shoot some more, I think its the era we are in at the present, shoot, use and throw, that’s how I see it, I am using the new Iphone4 and the new Sony Handycam, you need to talk about that, those two do it all, edit Iphone4 in the Imovie app or edit the Sony in FCP, its the coming thing, you are right its all about the story unless its 3D and wild effects, loved your semiar in Vancouver — will try for a webinar soon.
James Salkind adds:
I just finished reading the summer Newsletter, and I have a couple of comments/suggestions, particularly regarding tapeless workflow. First, as almost all the commentators mentioned, one big mistake users make is not properly backing up the original media. A great tool for streamlining this workflow is ShotPut Pro — (www.imagineproducts.com/index.php?main_page=product_info&cPath=1&products_id=2). It copies and verifies all manner of tapeless media, and it does it to multiple drives at the same time. This function is great for creating working and archival drives simultaneously.
Second, as was also mentioned, duplicate file names are an issue, especially with DSLR cameras. For instance, on the Canon 5D, all movies have the “MVI_” prefix, then a sequential number from 0001 to 9999. You can manually reset the camera to 0001, but not to any other number, so on a multi-camera shoot, it’s fairly easy to get duplicate file names among the cameras. One work-around is to organize the cards by camera — name the folder that the card is copied to by camera/date/card# — and then add the camera number to the clip name. “MVI_1234” would become “CAM1-MVI_1234”. It’s important to only have one card open at a time in the Log & Transfer window, otherwise FCP will automatically change the file numbering to prevent duplicate file names.
Lastly, regarding batch export of still images, I have found no automatic method to remove the colons, other than hiring an intern/assistant.
Keep up the good work with the Newsletter. It’s always informative and thought provoking.
UPDATE – Aug. 12, 2010
Don Hertz adds:
There is a known issue with the Log and Transfer tool and the Canon EOS plugin where it randomly truncates some clips it is importing. It is very important for an editor to take a look at the final length of the clips in the bin and make sure they match the original – re-encoding those that do not. This is especially important if the editor was not involved with filming and may not realize a key take or additional B-roll is missing.
I’ve had many clips cut in half or even a quarter of their original length. The problem seems to happen less if you don’t touch your system while it is doing the transfer but I’ve still had it happen. I haven’t been able to get a response from Canon yet on the issue but a Google search pulls up many others having the same problem.
Larry replies: Thanks, I didn’t know this.
Larry replies: Peter, Dan, Richard, James, and Don — thanks for writing!
UPDATE – Aug. 20, 2010
Andy Mees sent me the following correction:
Was just pointed at the July/August edition of your newsletter, and whilst browsing the this months selection of tips and tricks there was one thing that stood out a little for me in the Tapeless Media section where you noted:
“Sony recently updated their XDCAM Transfer utility software so all ingesting (the process of bringing media into Final Cut) can be done using Log & Transfer.”
I was wondering if that might need some clarification as “XDCAM Transfer” is actually Sony’s own stand-alone transfer application and is not really related to the Log and Transfer interface in FCP (except in basic function).
What I think you meant is Sony’s XDCAM EX Log and Transfer Plugin (recently updated to version 1.2) which allows XDCAM EX users to ingest using FCP’s Log and Transfer (instead of having to use Sony’s aforementioned XDCAM Transfer app). And again, just for clarity, when speaking of XDCAM ingest you might want to note that the XDCAM EX Log and Transfer plugin does not allow “all” ingesting to be done by Log and Transfer, it works only for those ingesting media recorded by Sony’s XDCAM EX based systems … for instance, it won’t work for those trying to ingest XDCAM HD (or SD) from XDCAM optical media.
Hope it helps, and keep up the good work with the newsletter.
Larry replies: Andy, thanks for sending this. I appreciate the clarification. I was referring to the plug-in.
