Create Adjustable Grids for Remote Performance Videos

Posted on by Larry

Mark Suszko presented me a challenge that took me most of the day to figure out; and even then, there’s no easy solution:

I decided to create a performance video of the Springfield Uketopians (Link). I have questions that revolve around design principles of setting up the grid and streamlining the process of aligning the incoming shots, every one of which I found I had to manually scale, crop, color-correct, and align to the audio guide track

By the way, after completing two of these, I’ve learned that Organizing and pre-treating the footage is important. I demanded everyone shoot landscape with their phones and webcams, and they got tips on camera eye-line, backlighting, and etc. issues. A few still did it wrong. Everyone participating had a variety of bandwidth and computer issues. I had everyone use VLC to pre-compress their footage at home to mpeg-4 and they posted those direct in a dedicated facebook group project discussion thread, where I pulled them out and worked with them. A few stragglers sent theirs in at the 11th hour by a free FTP service. You might want to give tips on a better workflow than that, I dunno…

I first aligned each track by audio-only; once I had a 20-odd-track stack, I superimposed a grid generated from Apple Motion. That was my guide for hand-placing the thumbnails, scaling them, cropping them, and figuring out where they might look best according to look space and other design ideas, trying to keep the cropped to fit the established “mullions”. I was not entirely successful at this, this time, as I’m sure you noticed, and I hope you’ll have tips to make this better/more efficient when you make your training video. My audience loved every second, but I mostly see all the little mistakes – that’s our way, as editors. I didn’t want to spend another week fixing it, the players were anxious to see something.

I wonder if there’s easier ways to do this in a slicker manner, using pre-generated video walls with drop zones in Apple Motion, for example. If I’m going to do one of these every 2 weeks, I need to be able to do it faster, with less fiddly adjusting.

I will say again, what a thrill it is to hear the first few harmony tracks start coming together to build a mix. It’s electric. I get now how recording studio engineers feel, doing that with pro musicians. And I mentioned before, whichever person you look at in the matrix, their audio jumps forward a little bit, even though I swear their levels are all set the same. Human perception is tricky.


Mark, congratulations on tackling this. Productions like this are fun to watch but really, really difficult to do well.

Let me share some ideas on how to simplify creating your grids by using Motion to create generators for Final Cut Pro X. However, the actual image placement and audio mixing will be almost impossible to automate.

OVERVIEW

The easiest way to do this is to create a grid with drop zones in Motion and save it as a Final Cut Generator. However, the problems with drop zones – and they are big – are that:

Having seen your video, where every shot needed framing and color adjustments, plus audio playback from every clip, drop zones – though fast – won’t work. This means that the easiest way to simplify production is to create a grid template, which can be adjusted for width and color, to help get started.

The first step is to figure out how many images you need to see on screen at once. This tells us what size each image needs to be. In your case, you had 20 images onscreen in a 1080p project. This would look best, given horizontal video, in a grid five images across by four images down.

The math says each image should then be 384 x 270. (1920 / 5) x (1080 / 4). So we need to create a 5×4 grid where each rectangle is 384 x 270 pixels. This isn’t a 16:9 aspect ratio, which means that each image will be slightly cropped. But, since every image is full screen, you have plenty of pixels to work with.

CREATE THE GRID

Create a new Final Cut Generator. Set the Preset to your frame size (Broadcast HD 1080 in my example). Because this grid won’t be animated, leave the frame rate at the default and keep the duration short. You can change the duration to whatever you need in Final Cut.

To make seeing the grid easier, select the Project in the Layers panel, then go to Inspector > Properties and change the Background color to something bright.

Using the Line tool, draw a black horizontal line that goes across the frame. Press the Shift key while drawing so the line is forced to be horizontal.

Right-click the line in the Layers panel and choose Make Clone Layer. This creates a new shape that mimics the original.

Create two cloned lines for a total of three.

Use Inspector > Properties > Y Position to set the vertical position of each horizontal line. For these three lines, set Y Position to:

Select the first (master) line and create four more clone lines. Then, using the Inspector, set Rotation to 90° and change the X Position to:

NOTE: FCP X sets the 0,0 point at the center of the frame. Since the middle image sits equally across the center, that means that half its width is on each side of zero. (384 / 2 = 192). That puts a vertical line on both sides of that image at ±192. From there, it’s easy to calculate the last line: 192 + 384 = ±576.

We now have a grid to place our images into.

NOTE: The reason for using clones is that when we change the width or color of the master line, all the clones immediately change to match. This is MUCH faster for making changes. Prove this for yourself by changing the master line.

CREATE DROP ZONES

If you are lucky enough that all your shots are framed properly and synced, you can add drop zones to this template to simplify adding images later.

NOTE: You add video to these drop zones when this template is opened in Final Cut.

ADD ADJUSTMENT CONTROLS

We use Rigs in Motion to add controls to elements that we want to adjust in Final Cut. For this grid, I want to adjust the line width and color.

NOTE: You can rig any parameter – the process is the same.


(Click to see larger image.)


(Click to see larger image.)

To help you keep things straight, change the names in the Rig at the top of the Layers panel to:

FINAL CUT PRO X

Open Final Cut

The new grid will be in the Generators browser.

Place this above all the clips in your timeline so the lines remain visible. Here, I placed it over a solid blue color so you can see the lines.

With the grid selected in the Timeline, go to the Generator Inspector to adjust the controls you created for line width and brush color. Because we used a master line with clones, as you adjust these settings, all the lines will change at the same time.

EXTRA CREDIT

While it is possible to create a grid that can vary in columns and rows, it is far easier to simply create different generators for different layouts.

This grid must be at the top layer of your project, otherwise the video clips will block the lines.

You can also create specific rigging controls to limit the width of the line, select between specific colors, or animate the grid lines so they wiggle. Any parameter in Motion can be added to a rig for control in Final Cut.


Bookmark the permalink.

12 Responses to Create Adjustable Grids for Remote Performance Videos

  1. Mark Suszko says:

    WOW! Thanks for the Easter present! So many musicians and musical groups like mine have lost their ability to play together or give concerts due to Covid-19, so making these multi-screen music performance videos has become our main way to perform together while apart (zoo, hangouts, Skype, etc. have too much lag to keep in time and play music).

    Thanks for this tremendous effort, Larry! We’ll be putting this method to work often!

    • Mark K says:

      Hi Mark,

      From one Mark to another, THANK YOU for posting this question! And a big thanks to Larry for writing it up. I’ve actually been in the exact same boat–I play in a full-time orchestra in Ohio, and I volunteered to assemble some virtual performances once I saw one released by Colorado Symphony and thought to myself, “I bet I could fumble my way through that.” It does present unique challenges.

      For anyone who might be closing in on their deadline and is stubborn about updating their OS like me (currently running MacOSX 10.13.6 High Sierra): if you don’t already have Motion, you need the most recent version of MacOSX in order to install it. At this point in time, I’m not ready to update, so I found a slightly more complicated, but still Larry-inspired, solution using custom generators within FCPX.

      Initially, I dragged a custom generator into my timeline, then made it white and adjusted the opacity so I could see both the individual musicians’ videos and the grid “stencil” (that’s what I’m calling it now) as I edit. After that, I dragged one custom generator into the timeline FOR EACH VERTICAL AND HORIZONTAL LINE in the grid stencil. In order to make each generator appear as a line, I simply adjusted the X or Y “scale” percentage (0.1% and 0.2% respectively, in my case), then I used the transform function to reposition as necessary. This was really helpful for me, because it still allowed me to determine line positions mathematically, plus I figured I could copy and paste attributes if I needed to adjust line thickness, although I never tested that particular theory.

      One important note, however: due to the LARGE number of concurrent videos in the timeline and the LARGE number or custom generators, once I had all the videos in place, I could not enable them all without the Viewer going completely black. A little unnerving at first, but I combined all the custom generators for my “stencil” into a compound clip and–VOILA!–I was able to see all my videos as intended. As long as the compound clip is positioned above the individual videos in the timeline, it should work similarly to Larry’s fix.

      Again, thank you so much for sharing! Hopefully my experience will help one person on an older machine, but regardless, this was really amazing to read right now.

  2. Derek Roff says:

    Thanks for the great tutorial, Larry. You make a convincing argument that drop zones won’t work for a project as complex as Mark’s. Given the manipulations that he has to do for each clip, I’m wondering if there is anything he can do to speed the process of placing his clips in the grid. What came to mind was creating template clip for each of the 20 rectangles in the grid, with the proper scaling and position attributes. Would it work to then copy and paste the scaling and position attributes from a given grid position template clip to a specific music clip for the chosen time of one arrangement of the grid? In other words, I would copy the attributes from template clip A1 to the clip with Bill and Mary, from B1 to the clip for Suzy, for C1 to the clip for George, and so on for all twenty cells on the grid.

    That leaves many tasks still needed for each performer, but I imagine that if this would be a quick way to get the scale and position set for each clip, that would be an advantage.

    • Larry says:

      Derek:

      An interesting suggestion. However, this also makes moving a clip from one cell to another tricky.

      I’m interested to see if other readers have ideas as well. Thanks for sharing yours.

      Larry

  3. Tim Nolte says:

    Hi Larry!

    Like you, I was intrigued by this problem and spent the day working on it. I created a bunch of video wall templates and a grid overlay, shared for free at http://www.boldervideo.com/fcp-templates. They’re extensively documented in a PDF manual found there. I also discovered a bunch of things about drop zones in the process!

    The short version is that nesting your clips in compound clips, then embedding those compound clips in the drop zones, should give all of the control Mark (or anyone else) needs.

    In more detail:

    1. Videos in drop zones CAN be scaled and repositioned, however, the option only shows up in Motion if there’s something in the drop zone. To be able to access the controls in Final Cut, you have to add dummy images to the drop zones in Motion, then publish the pan and scale controls from the Image Inspector.

    2. There are two ways for a drop zone video to have effects. Either create a compound clip, trim the clip, add the effects you want, then add it to the drop zone, OR add the untrimmed clip to your project, add effects, and add it to the drop zone.

    3. Compound clips in drop zones “live link” to their originals in the browser. This is a strong argument for nesting any clips you want to use in a drop zone inside a compound clip. This means changes, including trimming, grading, effects, etc. will be updated in the drop zone.

    4. You can control where a clip starts in a drop zone by which frame you click in the Event Browser when adding it. If you create a compound clip containing just the portion of the clip you want in the drop zone, simply click the very beginning of that compound clip’s filmstrip in the browser.

    5. No, drop zones don’t play audio, but if you create a bunch of compound clips that start at the right frames as I suggest, you can simply add the audio from those compound clips to the project timeline, connecting them so they line up with the start of the video wall generator.

    It’s also possible to use clips in the project timeline to populate drop zones, however, this creates lots of issues.

    1. In theory, the frame you click in the timeline SHOULD be the first frame used in the drop zone. HOWEVER, there is a bug in Final Cut that causes trimmed timeline clips that are added to drop zones to be double-trimmed. I.e., if you trim 5 seconds off the head of a clip in your timeline, then click a frame 3 seconds in when adding it to the drop zone, the drop zone will play it starting from 8 seconds in (5+3)! If one insists on adding timeline clips to drop zones, the workaround is to make sure the clips are untrimmed when adding them to the drop zone.

    2. Clips playing in drop zones don’t “live link” to their corresponding timeline clips. They reflect whatever state and effects were on the clip when you added it to the drop zone. If you want to add new effects, you must re-add the clip to the drop zone.

    Again, hopefully your readers (including Mark Suszko) find these templates helpful! They’re at http://www.boldervideo.com/fcp-templates.

    Tim Nolte
    Bolder Video

    • Larry says:

      Tim:

      This is a REALLY helpful response. Thanks for taking the time to research and write it up!!

      I’ll drop Mark a note asking him to read your reply – and thanks for sharing the templates! I look forward to playing with them.

      Larry

  4. Dennis Dirksen says:

    Larry, thank you. At some point could you do a grid tutorial for 4k? I know the instructions are the same just different numbers.

    • Larry says:

      Dennis:

      The numbers are easy to calculate. A UHD (4K) frame size is 3840 x 2160. Divide the number of columns into 3840 and the number of rows into 2160.

      Larry

  5. Bryan Davis says:

    Hi Larry.

    This is an interesting solution, but slightly different to one that I came up with. At time of writing, in early August 2020, I’ve now edited 16 weekly “virtual big band” shows, each of 45-60 mins in length, and with a band ranging in size from 18 to 39 pieces (the latter for one single collaboration tune, thankfully!). The show is called “Virtual Birdland” with Arturo O’Farrill and the Afro-Latin Jazz Orchestra from NYC, if anybody is interested to see the results. We’re on every Sunday night at 8:30pm ET on Facebook and YouTube. Our most recent episode can be viewed at: https://www.youtube.com/watch?v=CIpb6hQxcT0

    For our purposes, I needed not only to have the full band onscreen, but also present smaller sub-sections of it as the music demanded. Therefore, the generator/dropzone solution is not really practical. I need to sync the 18+ individual videos to the master audio once (we’re home recording and videoing separately, and an audio engineer is mixing the tracks) and then be able to cut to different views throughout each arrangement.

    My initial solution was to align and crop each video clip to a series of grids I prepared as graphics in Photoshop, and save those positions as Effects Presets in FCPX. That was certainly a quick-and-dirty solution, but it was the best I could come up with on the fly at extremely short notice. I’d never edited a video of this sort before and suddenly had to edit 35 mins worth of tunes in 3 days! That got me through the first few weeks but I soon realized that there needed to be a more flexible way to hone the position of the individual clips, within each little box, as different instruments necessarily frame themselves in different ways. Then I learned of the existence of Motion!

    My subsequent solution was to use Motion to create a series of Grid Effects plugins, with masked areas to fit the individual boxes in each grid, and rigged so I could choose the appropriate position (i.e. Row 2, Col 3 in a 6×3 grid) from a dropdown, and quickly resize and reframe the clip as necessary using Published Parameters in FCPX. When I come to a point in the music when the view needs to change, I simply make a Blade cut in the clips, apply a different grid, and deactivate the unnecessary clips for that section of the music. That way, everything stays organized, in sync, and can be adjusted fairly easily if necessary.

    Along with this, I’ve found it necessary to do a certain amount of pre-processing to the individual musician’s clips before adding to the main timeline: any color grading necessary, and retiming many of them – even though we all video at “30FPS”, we have a variety of cameras with a lot of cell phones in the mix along with some DSLRs etc. 30FPS and 29.97PFS are NOT the same when you’re dealing with long clips – our tunes are typically 6-8 minutes long, so a 30FPS clip in a 29.97FPS timeline gets a frame out of sync with the audio every 33.33 seconds, as you’ll realize. I’ve been very grateful for some of your tips on Compressor along the way! Thank you!

    • Larry says:

      Bryan:

      This is a GREAT tip! If you want to write it up in a bit more detail and add some screen shots to illustrate your process, I would be DELIGHTED to share this in my weekly newsletter.

      Thanks!!

      Larry

      • Bryan Davis says:

        Larry:

        Thanks. I’ll do that as soon as I have a bit of spare time next week. I’m currently editing this Sunday’s show, so I’m slammed from now until then!

        A secondary tip, that I forgot to mention, for those using these techniques for music videos, particularly those they’ve freshly recorded the music for: – Import the click track for your recording as a secondary audio file in FCPX. This gives you a clear visible and audible reference to help you cut to the beat, while editing, which can then simply be removed when you’re done. If you don’t want to hear the click, simply max the volume on the click track then deactivate it – you’ll see the clicks in the waveform but not hear anything. For those of us who come at this from the musical side of things, it’s a decent substitute for the music timeline we’re familiar with from our DAWs.

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends

Final Cut Pro X 10.4

FCPX Complete

Edit smarter with Larry’s brand-new webinars, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.