Mark Suszko presented me a challenge that took me most of the day to figure out; and even then, there’s no easy solution:
I decided to create a performance video of the Springfield Uketopians (Link). I have questions that revolve around design principles of setting up the grid and streamlining the process of aligning the incoming shots, every one of which I found I had to manually scale, crop, color-correct, and align to the audio guide track
By the way, after completing two of these, I’ve learned that Organizing and pre-treating the footage is important. I demanded everyone shoot landscape with their phones and webcams, and they got tips on camera eye-line, backlighting, and etc. issues. A few still did it wrong. Everyone participating had a variety of bandwidth and computer issues. I had everyone use VLC to pre-compress their footage at home to mpeg-4 and they posted those direct in a dedicated facebook group project discussion thread, where I pulled them out and worked with them. A few stragglers sent theirs in at the 11th hour by a free FTP service. You might want to give tips on a better workflow than that, I dunno…
I first aligned each track by audio-only; once I had a 20-odd-track stack, I superimposed a grid generated from Apple Motion. That was my guide for hand-placing the thumbnails, scaling them, cropping them, and figuring out where they might look best according to look space and other design ideas, trying to keep the cropped to fit the established “mullions”. I was not entirely successful at this, this time, as I’m sure you noticed, and I hope you’ll have tips to make this better/more efficient when you make your training video. My audience loved every second, but I mostly see all the little mistakes – that’s our way, as editors. I didn’t want to spend another week fixing it, the players were anxious to see something.
I wonder if there’s easier ways to do this in a slicker manner, using pre-generated video walls with drop zones in Apple Motion, for example. If I’m going to do one of these every 2 weeks, I need to be able to do it faster, with less fiddly adjusting.
I will say again, what a thrill it is to hear the first few harmony tracks start coming together to build a mix. It’s electric. I get now how recording studio engineers feel, doing that with pro musicians. And I mentioned before, whichever person you look at in the matrix, their audio jumps forward a little bit, even though I swear their levels are all set the same. Human perception is tricky.
Mark, congratulations on tackling this. Productions like this are fun to watch but really, really difficult to do well.
Let me share some ideas on how to simplify creating your grids by using Motion to create generators for Final Cut Pro X. However, the actual image placement and audio mixing will be almost impossible to automate.
OVERVIEW
The easiest way to do this is to create a grid with drop zones in Motion and save it as a Final Cut Generator. However, the problems with drop zones – and they are big – are that:
Having seen your video, where every shot needed framing and color adjustments, plus audio playback from every clip, drop zones – though fast – won’t work. This means that the easiest way to simplify production is to create a grid template, which can be adjusted for width and color, to help get started.
The first step is to figure out how many images you need to see on screen at once. This tells us what size each image needs to be. In your case, you had 20 images onscreen in a 1080p project. This would look best, given horizontal video, in a grid five images across by four images down.
The math says each image should then be 384 x 270. (1920 / 5) x (1080 / 4). So we need to create a 5×4 grid where each rectangle is 384 x 270 pixels. This isn’t a 16:9 aspect ratio, which means that each image will be slightly cropped. But, since every image is full screen, you have plenty of pixels to work with.
CREATE THE GRID
Create a new Final Cut Generator. Set the Preset to your frame size (Broadcast HD 1080 in my example). Because this grid won’t be animated, leave the frame rate at the default and keep the duration short. You can change the duration to whatever you need in Final Cut.
To make seeing the grid easier, select the Project in the Layers panel, then go to Inspector > Properties and change the Background color to something bright.
Using the Line tool, draw a black horizontal line that goes across the frame. Press the Shift key while drawing so the line is forced to be horizontal.
Right-click the line in the Layers panel and choose Make Clone Layer. This creates a new shape that mimics the original.
Create two cloned lines for a total of three.
Use Inspector > Properties > Y Position to set the vertical position of each horizontal line. For these three lines, set Y Position to:
Select the first (master) line and create four more clone lines. Then, using the Inspector, set Rotation to 90° and change the X Position to:
NOTE: FCP X sets the 0,0 point at the center of the frame. Since the middle image sits equally across the center, that means that half its width is on each side of zero. (384 / 2 = 192). That puts a vertical line on both sides of that image at ±192. From there, it’s easy to calculate the last line: 192 + 384 = ±576.
We now have a grid to place our images into.
NOTE: The reason for using clones is that when we change the width or color of the master line, all the clones immediately change to match. This is MUCH faster for making changes. Prove this for yourself by changing the master line.
CREATE DROP ZONES
If you are lucky enough that all your shots are framed properly and synced, you can add drop zones to this template to simplify adding images later.
NOTE: You add video to these drop zones when this template is opened in Final Cut.
ADD ADJUSTMENT CONTROLS
We use Rigs in Motion to add controls to elements that we want to adjust in Final Cut. For this grid, I want to adjust the line width and color.
NOTE: You can rig any parameter – the process is the same.
(Click to see larger image.)
(Click to see larger image.)
To help you keep things straight, change the names in the Rig at the top of the Layers panel to:
FINAL CUT PRO X
Open Final Cut
The new grid will be in the Generators browser.
Place this above all the clips in your timeline so the lines remain visible. Here, I placed it over a solid blue color so you can see the lines.
With the grid selected in the Timeline, go to the Generator Inspector to adjust the controls you created for line width and brush color. Because we used a master line with clones, as you adjust these settings, all the lines will change at the same time.
EXTRA CREDIT
While it is possible to create a grid that can vary in columns and rows, it is far easier to simply create different generators for different layouts.
This grid must be at the top layer of your project, otherwise the video clips will block the lines.
You can also create specific rigging controls to limit the width of the line, select between specific colors, or animate the grid lines so they wiggle. Any parameter in Motion can be added to a rig for control in Final Cut.
18 Responses to Create Adjustable Grids for Remote Performance Videos
WOW! Thanks for the Easter present! So many musicians and musical groups like mine have lost their ability to play together or give concerts due to Covid-19, so making these multi-screen music performance videos has become our main way to perform together while apart (zoo, hangouts, Skype, etc. have too much lag to keep in time and play music).
Thanks for this tremendous effort, Larry! We’ll be putting this method to work often!
Hi Mark,
From one Mark to another, THANK YOU for posting this question! And a big thanks to Larry for writing it up. I’ve actually been in the exact same boat–I play in a full-time orchestra in Ohio, and I volunteered to assemble some virtual performances once I saw one released by Colorado Symphony and thought to myself, “I bet I could fumble my way through that.” It does present unique challenges.
For anyone who might be closing in on their deadline and is stubborn about updating their OS like me (currently running MacOSX 10.13.6 High Sierra): if you don’t already have Motion, you need the most recent version of MacOSX in order to install it. At this point in time, I’m not ready to update, so I found a slightly more complicated, but still Larry-inspired, solution using custom generators within FCPX.
Initially, I dragged a custom generator into my timeline, then made it white and adjusted the opacity so I could see both the individual musicians’ videos and the grid “stencil” (that’s what I’m calling it now) as I edit. After that, I dragged one custom generator into the timeline FOR EACH VERTICAL AND HORIZONTAL LINE in the grid stencil. In order to make each generator appear as a line, I simply adjusted the X or Y “scale” percentage (0.1% and 0.2% respectively, in my case), then I used the transform function to reposition as necessary. This was really helpful for me, because it still allowed me to determine line positions mathematically, plus I figured I could copy and paste attributes if I needed to adjust line thickness, although I never tested that particular theory.
One important note, however: due to the LARGE number of concurrent videos in the timeline and the LARGE number or custom generators, once I had all the videos in place, I could not enable them all without the Viewer going completely black. A little unnerving at first, but I combined all the custom generators for my “stencil” into a compound clip and–VOILA!–I was able to see all my videos as intended. As long as the compound clip is positioned above the individual videos in the timeline, it should work similarly to Larry’s fix.
Again, thank you so much for sharing! Hopefully my experience will help one person on an older machine, but regardless, this was really amazing to read right now.
Mark:
This is a REALLY cool tip! Thanks for writing it up.
Larry
Thanks for the great tutorial, Larry. You make a convincing argument that drop zones won’t work for a project as complex as Mark’s. Given the manipulations that he has to do for each clip, I’m wondering if there is anything he can do to speed the process of placing his clips in the grid. What came to mind was creating template clip for each of the 20 rectangles in the grid, with the proper scaling and position attributes. Would it work to then copy and paste the scaling and position attributes from a given grid position template clip to a specific music clip for the chosen time of one arrangement of the grid? In other words, I would copy the attributes from template clip A1 to the clip with Bill and Mary, from B1 to the clip for Suzy, for C1 to the clip for George, and so on for all twenty cells on the grid.
That leaves many tasks still needed for each performer, but I imagine that if this would be a quick way to get the scale and position set for each clip, that would be an advantage.
Derek:
An interesting suggestion. However, this also makes moving a clip from one cell to another tricky.
I’m interested to see if other readers have ideas as well. Thanks for sharing yours.
Larry
Hi Larry!
Like you, I was intrigued by this problem and spent the day working on it. I created a bunch of video wall templates and a grid overlay, shared for free at http://www.boldervideo.com/fcp-templates. They’re extensively documented in a PDF manual found there. I also discovered a bunch of things about drop zones in the process!
The short version is that nesting your clips in compound clips, then embedding those compound clips in the drop zones, should give all of the control Mark (or anyone else) needs.
In more detail:
1. Videos in drop zones CAN be scaled and repositioned, however, the option only shows up in Motion if there’s something in the drop zone. To be able to access the controls in Final Cut, you have to add dummy images to the drop zones in Motion, then publish the pan and scale controls from the Image Inspector.
2. There are two ways for a drop zone video to have effects. Either create a compound clip, trim the clip, add the effects you want, then add it to the drop zone, OR add the untrimmed clip to your project, add effects, and add it to the drop zone.
3. Compound clips in drop zones “live link” to their originals in the browser. This is a strong argument for nesting any clips you want to use in a drop zone inside a compound clip. This means changes, including trimming, grading, effects, etc. will be updated in the drop zone.
4. You can control where a clip starts in a drop zone by which frame you click in the Event Browser when adding it. If you create a compound clip containing just the portion of the clip you want in the drop zone, simply click the very beginning of that compound clip’s filmstrip in the browser.
5. No, drop zones don’t play audio, but if you create a bunch of compound clips that start at the right frames as I suggest, you can simply add the audio from those compound clips to the project timeline, connecting them so they line up with the start of the video wall generator.
It’s also possible to use clips in the project timeline to populate drop zones, however, this creates lots of issues.
1. In theory, the frame you click in the timeline SHOULD be the first frame used in the drop zone. HOWEVER, there is a bug in Final Cut that causes trimmed timeline clips that are added to drop zones to be double-trimmed. I.e., if you trim 5 seconds off the head of a clip in your timeline, then click a frame 3 seconds in when adding it to the drop zone, the drop zone will play it starting from 8 seconds in (5+3)! If one insists on adding timeline clips to drop zones, the workaround is to make sure the clips are untrimmed when adding them to the drop zone.
2. Clips playing in drop zones don’t “live link” to their corresponding timeline clips. They reflect whatever state and effects were on the clip when you added it to the drop zone. If you want to add new effects, you must re-add the clip to the drop zone.
Again, hopefully your readers (including Mark Suszko) find these templates helpful! They’re at http://www.boldervideo.com/fcp-templates.
Tim Nolte
Bolder Video
Tim:
This is a REALLY helpful response. Thanks for taking the time to research and write it up!!
I’ll drop Mark a note asking him to read your reply – and thanks for sharing the templates! I look forward to playing with them.
Larry
Larry, thank you. At some point could you do a grid tutorial for 4k? I know the instructions are the same just different numbers.
Dennis:
The numbers are easy to calculate. A UHD (4K) frame size is 3840 x 2160. Divide the number of columns into 3840 and the number of rows into 2160.
Larry
Hi Larry.
This is an interesting solution, but slightly different to one that I came up with. At time of writing, in early August 2020, I’ve now edited 16 weekly “virtual big band” shows, each of 45-60 mins in length, and with a band ranging in size from 18 to 39 pieces (the latter for one single collaboration tune, thankfully!). The show is called “Virtual Birdland” with Arturo O’Farrill and the Afro-Latin Jazz Orchestra from NYC, if anybody is interested to see the results. We’re on every Sunday night at 8:30pm ET on Facebook and YouTube. Our most recent episode can be viewed at: https://www.youtube.com/watch?v=CIpb6hQxcT0
For our purposes, I needed not only to have the full band onscreen, but also present smaller sub-sections of it as the music demanded. Therefore, the generator/dropzone solution is not really practical. I need to sync the 18+ individual videos to the master audio once (we’re home recording and videoing separately, and an audio engineer is mixing the tracks) and then be able to cut to different views throughout each arrangement.
My initial solution was to align and crop each video clip to a series of grids I prepared as graphics in Photoshop, and save those positions as Effects Presets in FCPX. That was certainly a quick-and-dirty solution, but it was the best I could come up with on the fly at extremely short notice. I’d never edited a video of this sort before and suddenly had to edit 35 mins worth of tunes in 3 days! That got me through the first few weeks but I soon realized that there needed to be a more flexible way to hone the position of the individual clips, within each little box, as different instruments necessarily frame themselves in different ways. Then I learned of the existence of Motion!
My subsequent solution was to use Motion to create a series of Grid Effects plugins, with masked areas to fit the individual boxes in each grid, and rigged so I could choose the appropriate position (i.e. Row 2, Col 3 in a 6×3 grid) from a dropdown, and quickly resize and reframe the clip as necessary using Published Parameters in FCPX. When I come to a point in the music when the view needs to change, I simply make a Blade cut in the clips, apply a different grid, and deactivate the unnecessary clips for that section of the music. That way, everything stays organized, in sync, and can be adjusted fairly easily if necessary.
Along with this, I’ve found it necessary to do a certain amount of pre-processing to the individual musician’s clips before adding to the main timeline: any color grading necessary, and retiming many of them – even though we all video at “30FPS”, we have a variety of cameras with a lot of cell phones in the mix along with some DSLRs etc. 30FPS and 29.97PFS are NOT the same when you’re dealing with long clips – our tunes are typically 6-8 minutes long, so a 30FPS clip in a 29.97FPS timeline gets a frame out of sync with the audio every 33.33 seconds, as you’ll realize. I’ve been very grateful for some of your tips on Compressor along the way! Thank you!
Bryan:
This is a GREAT tip! If you want to write it up in a bit more detail and add some screen shots to illustrate your process, I would be DELIGHTED to share this in my weekly newsletter.
Thanks!!
Larry
Larry:
Thanks. I’ll do that as soon as I have a bit of spare time next week. I’m currently editing this Sunday’s show, so I’m slammed from now until then!
A secondary tip, that I forgot to mention, for those using these techniques for music videos, particularly those they’ve freshly recorded the music for: – Import the click track for your recording as a secondary audio file in FCPX. This gives you a clear visible and audible reference to help you cut to the beat, while editing, which can then simply be removed when you’re done. If you don’t want to hear the click, simply max the volume on the click track then deactivate it – you’ll see the clicks in the waveform but not hear anything. For those of us who come at this from the musical side of things, it’s a decent substitute for the music timeline we’re familiar with from our DAWs.
I’m a little late to the game on this one, but there are some really interesting ideas here as I’ve been wondering if there are ways to streamline the process I’ve been using for the past several months, although I’m not sure if Motion will make my scenario easier or more difficult, but here’s what I’ve been doing…
I’ve been publishing weekly choir videos for my church, which began with a big multi-song Easter Cantata back in April, and continued through to mostly weekly productions of a single song for each Sunday. Now I’m back doing another big one for our Christmas Cantata.
I haven’t really gotten into Motion, as I’m not up on that at all, but instead I’ve just been doing this fairly manually with a series of compound clips. I get my videos in a variety of different resolutions and formats (usually standard 720p or 1080p, but there are always a couple of oddballs who record in panoramic 1080p (e.g. 2400×1080) or other such odd resolutions.
What I’ve been doing is:
1. Place each of the videos into a compound clip that’s pre-sized for the grid. That’ usually either 320×270 for a 6×4 grid (when there are more than 20 performers), or 384×270 for a 5×4 grid. At first I used those exact resolutions for the compound clips, but then realized that maybe after cropping and zooming I might be losing some resolution, so I opted to create the compound clips at full 1080p, with the appropriate aspect ratio (e.g. 1536×1080 or 1280×1080), although I don’t know how much that matters.
2. Take each compound clip and use the built-in crop tool to frame each performer correctly.
3. Optionally, if necessary, apply filters for lighting and colour correction to each compound clip.
4. Once all of the compound clips are ready, place them into the main project timeline, stacked.
5. Use the Transform scaling adjustment to reduce them to 25%
6. Use the Transform X/Y positioning to place each one in the appropriate spot by number (e.g. top left corner of a 5×4 grid ends up being -768(X) x 405(Y).
7. Align clips in the timeline to line up audio.
8. Trim and apply effects as appropriate. In some cases these are simple fades in and out while in other cases I’ve added keyframed vignette and positioning effects to move soloists or groups of soli performers into appropriate positions (e.g. when just the sopranos are singing by themselves).
Originally I was doing the video and audio together, although at this point somebody else on my team is doing the audio in Logic Pro and then sending me the resulting file, which I just drop into the timeline as a standalone audio track and mute all of the in-video audio.
Anyway, the most cumbersome part of this process is creating all of the compound clips, especially since there doesn’t seem to be a way I’ve found to standardize the odd resolutions, so for each one I have to click the drop-down, select Custom, type in 1280 or 1536, and make sure I click the 23.98 dropdown to set it to 30fps.
Adjusting and cropping each of the performers can get a bit cumbersome too, but I really can’t see any practical way to automate that, as it’s something that needs to be eyeballed.
Anyway, I’d be happy if anybody has any better ideas…. Should I be using Motion to create a generator for this? Would that makes things easier, or would it simply complicate my life even further?
Thanks for any feedback you can provide!
Jesse:
This is a GREAT write-up! Thanks for taking the time.
A couple of notes:
* A FAST way to solve the compound clip creation issue is to select the compound clip in the Browser and type Cmd + D. This duplicates it. Double-click the compound clip to open it, and swap out the duplicated clips for new ones.
* A Generator in Motion may make this easier – use drop zones in Motion as placeholders, then add the video to the drop zone in FCP X.
Experiment with this by creating only two drop zones. If this works, you can create the full grid. No sense wasting time until you see if the concept of drop zones and generators makes sense for your project.
Larry
Thanks for your quick response!
That compound clip trick was a big help… For some reason it never occurred to me that something had to be IN the compound clip in order to duplicate it. I had originally tried creating an empty compound clip with the specs I wanted, but the “Duplicate” option never did anything in that case. Your tip about swapping out the duplicated clips was the key “aha” moment for me 🙂
Now the only thing I wish there was an easier way to deal with is adding multiple parallel clips into the main project timeline all in one fell swoop, since I’m frequently adding at least 20 for every video. Selecting and dragging multiple clips in of course adds them sequentially, and so far I haven’t found any ways to make them show up in parallel, which I guess isn’t surprising as it’s probably not something that most FCP users need to do all that often, but if you have any suggestions for how to smooth over that rather tedious task I’d be glad to hear them.
I’m still confused about the limitations of drop zones, however… from what I’m reading above, it sounds like they aren’t going to be as flexible for adding effects — especially movement effects. While most of the performances are fairly static, every so often I want to move the performers around, especially when there are solos or lengthy single-voice sections. Right now I’m just using keyframes with X/Y positioning changes, since I’ve generally been okay with linear movements.
I really need to take the time to learn Motion at some point; I know so little about it right now that I’m not even sure if there are aspects of it that would make it a better tool for some of these effects, but as it always seems to go, I only think about these things when I’m up against a time crunch 😀
Jesse:
I don’t understand what you mean by “parallel.” Contact me directly via email – as this Comment thread will run out of room – and I’ll help if I can.
A drop zone is “a clip you replace with another clip.” They can’t be created in FCP X, but can be created in Motion. If you constantly create the same grid over and over, they can speed the work.
Larry
This column was vital to a project I finished a few days ago. Thanks to Larry and also to Bryan Davis, and everyone else who commented, it really helped.
I wrote down what what I ultimately did for a friend, so here it is:
Everything was done in Apple Compressor, Apple Motion, Final Cut, fxfactory’s Griddy, and some work on the title card in Affinity Photo.
Preserving image quality and color depth was a priority
The audio came from a CD we recorded in 1995. (I played bass, my wife, flute)
The CD audio was converted from 44.1khz to 48khz with Apple Compressor. That was the audio master. a 4 second front was added to the front for time for the title card.
I always obtain a score or lead sheet, or conductor’s score, orchestral reduction-whatever is available that has the most information musically-it helps to lay out a game plan for the whole piece.
It helped that we had a finished audio track of the piece-the musicians perform it yearly. All the choir and orchestra were sent a copy of the CD audio to sing along with, but none of their audio was used.
The instructions sent to the choir and orchestra we kept as simple as possible-there’s only so much you can ask -it helped that the piece was light-and they were asked to have fun with it.
I watched a bunch of tutorials and finished products, and corresponded with a few who had already been down this path-I’ve been an editor a long time (started on a Steenbeck!), but virtual choirs are a unique beast.
Everyone was given a 1-week window after Thanksgiving to submit their clips. I allowed 2 weeks to assemble the footage-and should have planned on twice that.
Conforming all submitted Clips: 2 steps
1st Compressor, then FCPX
All clips went through Compressor, to arrive a general consistency-because frame rates, file formats, clip sizes and colors are all over the map
Compressor output settings:
Audio: 48khz,
29.97 FPS
Video Colorspace: Rec.2020
Progressive
Uncompressed
This step was easy. I could put all the clips into a queue and let it run overnight. This is why I went uncompressed: I was concerned with the computer’s ability to play 62 clips all at once, and didn’t know what would happen when all the tiles were assembled. A past colleague from the 90s had a theory that compressed formats slowed a computer down unpacking and repacking compressed video. For this project I was using a 10 core iMac Pro with OWC SSD external storage, so I went uncompressed and was happy with the results.
Finishing that, In FCPX I created a 3840x 2160 rec.2020 library for the project, and a project file for every clip. I didn’t wan’t to be syncing or color correcting or stabilizing in Griddy-I wanted finished clips when I got to that stage. So, in FCPX:
Retiming for sync to the master 48khz wav.)
Reframe and crop: output square 1080×1080 tile.
Color correction
Video noise reduction
Stabilize when necessary
4 second slug at the front of each clip for title card
Organizing and tracking the footage:
In the computer’s download folder clips were tagged according to dates they arrived (they were submitted to dropbox, & downloaded from there).
A spreadsheet was kept with original filenames, formats, names, download dates, whether they were choir or orchestra, & columns to check off completed steps for each.
I tried to keep original filenames to the end, except when clips came in without performer’s names, which were added.
Overkill?, Maybe but it was necessary for me to keep a handle on it all. Our social media people would need to know who was in the piece for tagging.
Daily backups, of course,
One thing I didn’t know who to do: make a bunch of individual compound clips (required by Griddy) without doing them one by one
Each full page (orchestra, men, whole group) was output for the entirety of the song, and then brought in and a finished product was cut together at the end. The frame and flashing Christmas lights at the end were built in Apple Motion, rendered out with an alpha channel, and keyed in.
here’s a link to the clip:
https://drive.google.com/file/d/1screq9wN61XxhDTEbDCkaBRcc9rDXB7I/view?usp=sharing
David:
This is SO COOL! Thanks for your write-up.
Your using an existing audio track was genius, because the mix from remote footage would drive you nuts. I especially like your treatment of the conductor – bouncing around the frame in all those different hats.
Lovely.
Larry