FCP X: Divide Large Libraries [u]

[Updated Aug. 12, 2014, with additional tips on improving performance.]

For smaller projects it is perfectly fine to store all media and projects in the same library. But, for larger projects, this can cause problems. While the specific number of open files varies based upon RAM and project complexity, a rough number that I use to determine when a Library is getting full is around 3,000 clips.

Creating multiple libraries for large projects can simplify media management and improve the performance of Final Cut Pro X.

However, before we start moving media around, we need to define two key terms:

• Managed media
• External media

Managed media is media which is stored inside the library bundle on our hard disk. External media is media which is stored in a folder outside the library bundle, but linked to the folder using “symlinks.” Symlinks are a high-power version of an alias.

In general, there is no performance difference between Managed and External media; assuming the hard disks they are stored on are roughly the same speed. However, if you plan to use the same media in more than one Library, external media will decrease the amount of storage space you use. This is because when FCP uses media from one library in a project stored in a different library, it will copy the media file if the media is stored inside a library (Managed) but only copy the link to that file if the media is stored outside the library (External).

So, for very large projects, storing media outside the library will provide greater flexibility and reduced file storage requirements. Storing all media inside the Library on your hard disk is easier and simpler to manage and backup.

Final Cut has several media management functions built-into it:

• Consolidate a Library
• Copy an Event to a Library
• Move an Event to a Library

Consolidate a library is the best option when media is stored in multiple locations and you want to store everything in one spot.

Copying media is the best option when you need to have multiple copies, say to share with another editor.

Moving media is the best option when you want to break up a large library into multiple pieces.

NOTE: The techniques described here require Final Cut Pro X 10.1.2 or later.

WHY THIS MATTERS

First, there is a limit to how many clips FCP X can have open at one time. While this numbers in the thousands, I’ve had several editors contact me who’s clips numbered in the tens of thousands! Finding simple ways to manage media means that we need to limit the number of clips that are open at one time.

Final Cut allows us to have multiple libraries open at once and to edit media from any open library into any project. However, when a clip is edited from, say, Library 1 into a project which is stored in Library 2, FCP makes a copy of the Library 1 media into Library 2. This makes sense because if you deleted Library 1 and all its media because you are done with it, you don’t necessarily want to delete the media which is referenced by projects stored in Library 2.

But, for large projects, copying media between libraries negates the advantage of storing media in separate libraries in the first place AND it takes up more disk space.

So, Final Cut provides a second option: When you edit media which is linked to Library 1 (External media) into a project which is stored in Library 2, only the link to the media is copied, not the media itself.

This means that when we use External media, we can use thousands of clips stored in dozens of libraries and, provided that all media is stored external to any library, performance won’t suffer and our hard disks won’t fill with duplicate media.

NOTE: Again, to obtain the best performance, only open the libraries that you need, and keep the total number of open clips and libraries as small as possible.

UPDATE: THOUGHTS ON PERFORMANCE

Apple made major improvements to system performance in the 10.1.2 update. So, as a start, if you are experiencing slow-downs, be sure to upgrade. After that, while there are many elements that affect the performance of Final Cut, the three big ones are:

• The amount of RAM installed
• The number of clips that are open in the Browser
• Other software running in the background

Final Cut uses as much RAM as you have installed, so adding more RAM – assuming your system supports it – is always a good idea. As well, other applications or utilities can divert system resources, which also slows FCP down.

However, this article looks at using multiple libraries as a way to improve performance.

NOTE: Another factor affecting performance is network speed, if you are editing clips or accessing libraries store on a server, rather than editing clips or libraries stored locally.

THE IDEAL APPROACH

Ideally, we should think out media management before we start importing our first clip. This is very easy to say and almost impossible to do. Still, if you are someone who can actually plan that far ahead, creating multiple libraries and storing media externally will vastly simplify media management for large projects.

After you create a new library, and BEFORE you import any clips, select the Library and choose File > Library Properties (type Control + Cmd + J). This opens the Library Properties section of the Inspector

Click the Modify Settings button and click the Media menu.

Select Choose from the menu and navigate to the folder where you want all media for that library stored. In this example, I’ve created a folder on my 2nd Drive RAID called “External Media Folder,” and selected that.

NOTE: You can name this external folder anything you want. Pick a folder name that helps you remember what’s inside. Final Cut Pro will work with any folder name that the Finder supports.

Now, when you import clips into that Library, Final Cut will store the media in this external folder and create links to the media in the Library itself.

Cool. But who can plan that far ahead?

CREATE NEW LIBRARIES

If you’re like most editors, you won’t realize you need multiple libraries until you are drowning in files and wondering why FCP X is so slow. The answer is your libraries are too big and need to be divided up.

NOTE: Libraries and the clips they contain require RAM to open. When you have thousands of clips in a single library, you can exceed the RAM that FCP has available. Divide your media into separate libraries, and only open the libraries you need for the work you are doing right then. Get in the habit of closing libraries you don’t need to reduce the amount of RAM FCP uses to track all your clips.

Remember a project can contain media from any library, whether that library is open or not.

We want to create external media, so that we can edit any clip from any library into any project without duplicating media files in the library. This is actually quite easy – but not intuitive. The secret is to create the new libraries first, then move the media into it.

Here’s how.

Let’s say that we have one big library: “Main Library” and we want to divide this into three new Libraries: “Library 1,” “Library 2,” and “Library 3.”

NOTE: Remember, you can use any name for a library. I’m just trying to be really obvious here.

First, create a folder on your external hard drive to store the media for each library you want to create. Here, for example, I’ve created three folders on my 2nd Drive:

• Library 1 External
• Library 2 External
• Library 3 External

Next, create three new libraries in Final Cut Pro X using File > New > Library. Store the Library files wherever you want – again, I recommend never using your internal drive – and name them whatever you want. These Library files will not contain media, just links to the media.

For this example, I created Library 1, Library 2 and Library 3.

Select each of the three new libraries – one at a time – go to Library Properties (Control +  Cmd + J) and change their Storage Locations so that Library 1 Media is stored in Library 1 External.

Set Library 2 to store its media to Library 2 External.

Set Library 3 to store its media to Library 3 External.

UPDATE: IMPROVING PERFORMANCE

Something I just learned is that Final Cut can point multiple libraries to the same media folder. In fact, Apple recommends that when you need to create multiple libraries to minimize the number of clips open at the same time for a specific project, have all the libraries point to the same external media folder.

When importing media, FCP X assigns unique identifiers to distinguish clips from one another.  This means that clips with the same name shot on different days or using different cameras won’t overwrite each other. This is why it’s important to use the media management commands — like Consolidate — within FCP X because the app can properly track all links to media files wherever they are located.

Storing all media files from multiple libraries into one folder is the same as outlined above, except that when you set the Media location using Library Preferences, point each Library to the same external media folder. While storing media in multiple folders works perfectly well, creating a single external folder makes it easy to locate media and assures that there’s no confusion about where new media files should be imported.

While I think the idea of storing all media for a specific project in a single external folder makes a great deal of sense, I will create separate media folders to store media for projects that are not related to each other.

MOVE MEDIA

While you can drag media from one event in one library to another event in a different library, since we want to take a big project and break it into smaller groups, it is MUCH easier to do this by moving media contained in entire events.

Here, in the Main Library, I’ve created three events – one for all the media that I want to move to each of the three libraries.

NOTE: You can create multiple events for each library, because each event name will be retained when you move it to a new library. Also, when you move media files all keywords, ratings and metadata move with them.

Now, organize your media by dragging it into the appropriate event. To keep things simple for this article, I’m dragging three clips from an event in the Main Library, into the event for Library 1.

Let’s pretend that all our media is now organized and ready to move. The process is the same for each event and library.

Select the Event containing the media you want to move to a new Library. Choose File > Move Event to > [Name of Library]. In this example, I’m moving the Event “Media for Library 1” into “Library 1.”

Final Cut gives us the option to move camera original, optimized or proxy files. in general, because you don’t want to keep recreating optimized or proxy media, you should move everything.

NOTE: If the media is inside the library, it will be moved. If the media in the Main Library is linked, only a link will be created.

Open the Background Tasks window to see the status of your media. (This is optional, but fun to watch.)

Because we set Library Preferences to store all media external to the Library, Final Cut moves the media to the external folder – “Library 1 External” – while creating symlinks in Library 1 that point to the media location.

Here we see that the clips were moved from the Main Library to Library 1.

While the media was moved OUT of the Main Library into Library 1 External. We have successfully created external media from an existing library!

SUMMARY

If media is stored externally to the original Main Library, Final Cut will only move symlinks to the new libraries. No media will be moved from its external location. This is still a good idea, because we need to reduce the size of the open Library.

If media is stored inside the original Main Library, Final Cut will move the media to the media location specified in Library Preferences, which, in our case, is an external folder and create symlinks in the new library.

Either way, you end up with as many libraries as you need, the ability to open and close libraries as needed and no duplication of files or hard disk storage space.

Very cool.

---

Bookmark the permalink.