Frame Rates are Tricky Beasts

Posted on by Larry

creativity

My goal in this article is to discuss the challenges in converting frame rates.

If everything you shoot, edit and output is a single frame rate, then don’t change anything. This is the ideal way to work. However, as you start to integrate elements that originate at different frame rates, frame rate conversion rears its very ugly head.

DEFINITIONS

Think of a video clip as a series of wooden children’s blocks connected by a piece of string. Each block represents a frame of video. As we pull the string, tugging the blocks along in a line, the frame “rate” represents the number of blocks (or images or frames) that pass an observer each second. Frame rate is measured in frames per second; “fps.”

Changing the speed of a clip is NOT the same as changing the frame rate.

This difference is significant. The first is easy, the second is hard.

We change the speed of a clip to create a visual effect. We change the frame rate of a clip to match the settings of our clip to the project. If you don’t need to match settings, don’t mess with changing frame rates.

SETTING BOUNDARIES

There are two sides to a frame rate discussion:

There is a lot of debate as to which is the “best” frame rate. Some feel that 24 fps is more “cinematic,” while 60 fps is more “real.” As you should know by now, there is no “best.” Just as there is no “best” car, camera, or restaurant; there are simply choices.

Converting to a 24 fps frame rate will NOT make your movie look “filmic.” It will, generally, just make it look worse. The “cinematic look” is a combination of: lenses, lighting, depth of field, shutter speed, shutter angle, motion blur and frame rate. Changing the frame rate only affects the frame rate, not the look.

There are no right answers, just louder voices.

Also, to keep this article to a manageable length, I will ignore:

These special cases don’t alter the basic rules of frame rates, though they can complicate understanding.

THE BASIC RULES

Whether you use Adobe, Apple, Avid, or any other video editing software on Macs, PCs or mobile devices, the basic rules of frame rates remain the same:

HOW WE GOT TO TODAY

In the early days of film, say 1890 – 1915, all cameras were hand-cranked. During this time, frame rates wandered from 8 fps to 30 fps, often in the same scene. In those days, the value of a camera operator was not based on their composition, but on the consistency of their cranking.

NOTE: This is one of the reasons comedies were so prevalent in the early days of film. Speed changes are inherently comedic and physical comedy does not require dialog.

As films grew in popularity and profitability, standards developed allowing cameras to be cranked by a motor, rather than by hand. Also, at this time, the industry settled on a frame rate of 18 fps.

Why? Because film was expensive and producers were, um, cheap. 18 fps provided the illusion of smooth movement without wasting a lot of film, and money.

This standard continued up until the advent of talkies which exploded on the scene in 1927 with The Jazz Singer. The problem was that 18 fps was not fast enough to support high quality audio. This frame rate yielded audio roughly equivalent to a telephone call.

So, a new frame rate standard needed to be developed – and the industry chose 24 fps.

Why? Because film was expensive and producers were still, um, cheap. 24 fps provided the illusion of smooth movement and relatively high-quality sound without wasting a lot of film, and money.

NOTE: Sound quality continued to improve over time, not by increasing the frame rate, but by shifting audio from an optical track to a magnetic track.

When video arrived, in the 1930’s, we had a major timing problem. How to get the TV receiver to “pulse” in sync with the transmitter? The solution was AC power. All across the US, power “pulsed” at 60 cycles per second.

Television engineers adopted this “universal” pulse as the basic timing circuit for video. Since video in those days was interlaced, where a single frame (complete image) was composed of two fields (a portion of the image consisting of all the odd or even scan lines), each field pulsed at 1/60th of a second.

Ta-dah! 30 fps video.

Except, over time it was discovered that high-voltage electricity “evaporated” from transmission lines when the cycle rate was too high. 50 cycles preserved more power over distance than 60 cycles (now called Hz). So, when much of the world was rebuilt after World War II, the utility companies, to save money and power, dropped the cycle rate to 50 Hz.

From there, the video industry derived 25 fps video, because interlacing was still in vogue.

So, at the dawn of the HD era in the early 1990’s, we had three principle frame rates: 24, 25, and 30 (which, with the advent of color was slightly modified to 29.97 fps, because why should this story be particularly simple?)

And, as we all know, with the rise of HD, our industry came together as a group and standardized on a single frame size and single frame rate.

– – –

Sigh… No such luck.

At last count, we now have nine different frame rates: 23.98, 24, 25, 29.97, 30, 48, 50, 59.94 and 60. (And, yes, 100 and 120 fps are knocking on the door. Please keep that door shut…!)

No WONDER we’re all confused. We’ve been handed a veritable Gordian Knot of frame rates!

CONVERSION OPTIONS

Frame rate conversion is the process of duplicating or removing frames such that, when the clip is played in a sequence that matches the frame rate of the clip, all action appears at “normal” speed. (I put “normal” in quotes because I couldn’t figure out an easy or good way to define normal.)

Most of the time, video editing software will automatically handle frame rate conversions. And, most of the time, I suggest you not worry about it, because, most of the time, it will look fine.

REMEMBER: “Camera-native frame rates always look better than converted frame rates.”

When it comes to frame rate conversions, there are easy options and hard options. Following the wooden block analogy I introduced earlier, we can’t just stretch frames to different rates because each frame is made of wood, not Silly Putty. Instead, we change frame rates by inserting or removing entire blocks.

EASY: 50 fps to 25 fps – or 60 fps to 30 fps

Assuming the video is progressive, conversion simply deletes every other frame.

If the video is interlaced, one field is deleted while the scan lines in the other field are duplicated. (Yes, this option reduces image quality. That’s one reason I hate interlaced video.)

EASY: 29.97 fps to 59.94 fps – or 25 fps to 50 fps.

Here every frame is duplicated. This does not create slo-mo because the video plays back at 50 fps, yielding the same movement as playing 25 fps video in a 25 fps project.

This does not significantly degrade movement quality, but movement will look more fluid if you shot 50 fps (or 60) originally.

MOSTLY EASY: 24 fps to 25 fps

The traditional way of converting 24 fps to 25 fps is increasing the speed of the 24 fps material 4%. This allows all frames to be displayed and, while the action is a bit faster, it isn’t so much faster that the audience will perceive it.

NOTE: Yes, this speed change means we need to speed the audio as well. There’s no free lunch.

HARD: 24 fps to 29.97 fps

This was done traditionally when converting films for television broadcast using a telecine.

Here, we need to create, essentially, six “new” frames every second. (The difference between 24 fps and 30 fps.) But we are dealing with children’s blocks here, we can’t create new images, we can only create new frames that contain existing images.

There are several ways to do this, depending upon whether you are working with interlaced or progressive images. The interlaced methods are quite complex and involve duplicating specific fields, not just frames.

But, here’s a simple method to illustrate the process. Take a group of four frames, then duplicate the last frame in the group. Over 24 frames this creates 6 new frames.

When played back at 30 frames a second, most viewers won’t notice the duplicated frame. However, for the discerning, your action will slightly stutter every five frames. This illustrates why you want to avoid converting frame rates.

HARD: 60 fps to 24 fps

Three quick reminders:

NOTE: Optical flow seeks to do just that, invent new frames. However, while good in theory, the results are often worse than not using optical flow.

Here’s an example of how this could be handled: We need to remove 36 frames from every second of video. Since both 60 and 24 are divisible by 3, we can divide each second into three “blocks,” or sections.  This means that a  20 frame block in the source clip needs to be converted into an 8 frame block in the destination clip. To do this:

As you can see, asymmetrical trimming (remove 1 frame, then remove 2, then remove 1…) gets us to the frame rate we need, but at the expense of potentially adding jitter to movement; say during an actor’s walk or as a car drives through a scene.  Whenever we convert frame rates asymmetrically, we run the risk of damaging the movement in the clip.

A SIDEBAR ON VIDEO COMPRESSION

Video can be compressed in one of two ways:

I-frame formats include: ProRes, GoPro Cineform, AVC-Intra and the DNx family of codecs.

GOP formats include: AVCHD, H.264, HDV and most formats that generate very small file sizes.

As I was writing this article, it occurred to me that camera-native GOP-format video will probably suffer more from image degradation as you change frame rates than video that was shot as I-frame media.

I haven’t tested this, and would like to hear other opinions, but if you are seeing blurry images when changing frame rates by small increments, I would suspect your video format is too blame.

SUMMARY

Frame rates are complex. However, a little planning ahead can simplify headaches. The best option – always – is to shoot the frame rate you need to output.

And, keep in mind, that a “film-like look” does not necessarily require a “film-like frame rate.”


Bookmark the permalink.

45 Responses to Frame Rates are Tricky Beasts

← Older Comments
  1. Mahesh says:

    I work on fcpx. I have a project at 1080p/30fps. I have to work with mix footage, 4k/25 fps, 720p/120fps and 240fps. What happens when i bring this on the timeline, will the 120fps and 240fps be brought down to 30fps, there by slowing down my footage or it will retain the same frame rate???

    • Larry says:

      Mahesh:

      By default, all your clips will play at normal speed, neither sped up nor slowed down.

      However, you can switch your high-frame rate clips to slow-motion by selecting the clip, then choosing Modify > Retime > Automatic Speed.

      Larry

  2. Russ Carlin says:

    One of the biggest mistakes you can make with weddings, is if you’re shooting multiple angles for speeches, but one camera is say 24fps and another 25fps, it’s a nightmare as the audio sync has a complete spaz and can’t sync it fully.

    I think you can get away with candid/b-roll footage at slightly different rates. Whenever audio is involved just make sure you’re same frame rate or at least a direct divide of each other!

    Don’t know why I wrote this, guess if someone starting out sees it and it helps one person then it was worth it. I learned this lesson early on!

  3. Nangko says:

    Great article!

    Quote: And, keep in mind, that a “film-like look” does not necessarily require a “film-like frame rate.”

    Do you have an article about getting the “film-like look” without dropping to lower frame rates?

  4. James Nicholson says:

    What about cameras that record 23.98fps? If I’m working the footage into a true 24fps timeline, is 23.98fps going to convert better than 25fps?

    • Larry says:

      James:

      As with all these conversions, always test your results before the deadline is so close that change is impossible.

      That being said, my understanding is that there are very few cameras that shoot “true 24 fps.” Most shoot 23.978 fps. And, yes, 23.978 would convert more smoothly than 25 fps.

      Larry

  5. John says:

    Great article!
    I am somewhat of a rookie at all of this. I create videos for the web (mainly Youtube) that are shot with a couple of drones and (believe it or not), a couple of cell phone cameras. I set both of my drones shoot at 30fps but the actual framerate of the footage as shown in Davinci Resolve is 29.97fps.

    The cell phones seem to shoot (natively) at some kind of variable frame rate (which I have no control over): Davinci Resolve shows some of the cell phone clips being 30fps and others 29.97.

    I’ve used both the 30fps and 29.97 footage together on 30fps timelines and also 29.97 and it has looked fairly good for the most part.

    What are you thoughts on this situation and what should my project framerate really be considering the ever so slight difference in the source footage framerates?

    Thanks!!!

    • Larry says:

      John:

      Essentially the difference is drop-frame, vs. non-drop-frame timecode. Given your description, I’d set your project to 29.97 and you should be fine.

      Larry

  6. Bernard says:

    Thank you very much for great article! I have just one more thing to ask: if my footage is 1080p50 and desired output is 1080p25, should I edit video in 50fps sequence end export it as 25fps or use 25fps sequence from the start? And, either way, should I use frame sampling in export or some other option (frame blending or optical flow), or perhaps add some motion blur to compensate for stretching the frame durations?

  7. Frank says:

    Thanks for the article. I paid a lot for a telcine company to take my old Super 8 films and convert them to digital files. When I received them back in the form of 30fps, I was disappointed to find that “Premiere Pro” pulled them down into two different patterns. One pattern was logical and the movement was acceptable. The other pattern was awful and created terrible jutter. I was able to look at the frames and see that the bad pattern was as follows: Frame 1, Frame 2, Frame 3, Frame 3, Frame 3. That is single frames and then a triplicate! After much research, I have decided to get the my 18fps film in the form of an non-pulled down 24P file. This file contains only one frame each but will play back too fast. How can I take this 24P archive copy of my films and get it to slow down and be saved so that I can play my films with a natural motion. I am a complete novice but I am trying hard to figure this out so that I can enjoy my movies at the proper speed without the nauseating jutter.

    • Larry says:

      Frank:

      Keep in mind that if you are posting your Super8 films to the web, or playing them on the computer, the frame rate is “irrelevant.” By that I mean that the web and computers can play any frame rate without restriction. So, for the smoothest playback, transfer your films at 18 fps, which matches the speed they were shot.

      For broadcast, or cable, or DVD, then we must work with one of the existing broadcast frame rates – these distribution media are much less flexible. This means transferring your films at 23.98, 24, 25, 29.97, 30, 50, 59.94 or 60 fps. The trick is that these frame rates ALWAYS involve pull-down, or duplicating an existing frame. Using interlacing, we can create fields, rather than frames, but interlacing looks terrible when played back on the web or a computer. The trick is to find a frame rate that 18 divides evenly into – which, theoretically, would me 18, 36, 54, or 72 fps. Which don’t exist. So you do the best you can.

      The easiest, but may not be the best, is a cadence that convert 18 fps to 24 fps: Frame 1, frame 2, frame 3, repeat frame 3, then do that pattern again: 4, 5, 6, 6 – 7, 8, 9, 9 and so on. This provides the easiest conversion with a slight stutter in movement.

      Again, pick the frame rate you need based upon how you are distributing the finished work and if you are using the web – transfer at 18 fps. As with all things, experiment with a short film to see what works the best for you.

      Larry

      • Frank says:

        Larry, thank you for this explanation. The company I used had no way to take the Super 8 shot at 18fps and output it to 18fps. The closest option they had was to output it to 24P without pulldown. I would like to play it both from a computer and from a 4k telivison via the “Plex” application. So my problem is, now that I have all the single frames with no pulldown in 24P, is there a way for me to use slow motion in some program to bring the single frames down to regular 18fps speed with no duplication of frames and RECORD this slow motion version so that I can play it back and it will look almost exactly like 18 fps with no duplication and very little jutter? If this is not possible they can sell me either a Tiff uncompressed image sequence, or a compressed JPG image sequence. However, is there some kind of program I can buy that will take image sequences and create movies out of them at 18fps? As you can see, this is really complicated for a total novice like me to understand. I just am trying to figure out what is the best way to try to get the motion and speed to look like 18fps with as little juttering as possible since the 30fps with pulldown looks absolutely awful regarding juttering. Thank you so much.

        • Frank:

          Whichever option you choose will be a time-consuming mess. The IDEAL option is to find a firm that can output your film at 18 fps and redo the work. This will be the best and fastest, but not cheapest route.

          There’s no way slow-motion will do what you want, because what you need to do is drop some frames (the excessive duplicates) without dropping other frames. Slow motion will change the speed of all frames equally. Again, not what you want.

          If you can’t re-digitize, then get them to send you an image sequence of TIFF images (which are the highest quality). However, they will take a lot of storage space, which is the tradeoff for high quality. Then, you can manually go thru and remove the frames you don’t want. There may be some Automator workflows you can use to automate this, but it will be time-consuming.

          Once you have all the stills removed that you don’t want, you can use Apple Compressor – or Adobe Media Encoder – to turn it back into a real movie again.

          Larry

          • Frank says:

            Larry let me specify. They took my original Super 8 films, color corrected, grain removal, etc. and brought this frame by frame into 24P WITHOUT ANY PULLDOWN. So the 24P is the same frames with no duplication. They told me I can play the 24P file and it will be exactly like playing the original Super 8 film itself…except it will be faster because it is 24P rather than 18fps. They then took the 24P and “pulled down” to 30fps and gave me that. So to be clear, I have one 24P file with no pulldown (no duplicate frames) and one 30fps file with pulldown. The pulldown for the 30fps was done in Premiere Pro but for some reason that program gave two different patterns and one involved a triplicate with singles which looks just awful with the jutter. So I have this 24P file that contains only single frames and I am trying to figure out how I can make this work so that it plays at the same speed as 18fps. So I was wondering if I could simply put 24P to some percentage slow motion and save it so that it plays close to 18fps speed. Again, the 24P file has not been pulled down and contains no duplicate frames at all. I am not sure I explained it correctly to you before. Thanks again.

          • Larry says:

            Frank:

            Ah! This is easy, then. All you need to do is change how the NLE you are editing the clip in determines the Frame Rate.

            In Premiere, it’s Modify > Interpret Footage. In FCP X it’s Modify > Retime > Automatic Speed. Make sure your project is set to 18 fps before you make this change.

            Larry

    • Larry says:

      Frank:

      Ah! This is easy, then. All you need to do is change how the NLE you are editing the clip in determines the Frame Rate.

      In Premiere, it’s Modify > Interpret Footage. In FCP X it’s Modify > Retime > Automatic Speed. Make sure your project is set to 18 fps before you make this change.

      Larry

  8. Scott says:

    Awesome information – thank you Larry….

    I’m just starting to work with DaVinci Resolve and am a little confused between the 1. ‘Timeline Frame rate’, the 2. ‘Playback frame rate’ and the 3. ‘Video format’…

    For example if I shot the original footage at 30fps then all three of these settings should be at 29.97 yes?

  9. Raul Tello says:

    Hey Larry first of all I’m a huge fan of you from Mexico! Well this is my doubt:

    I shoot in 30fps with my gh5 in AlL-I and then when I go to Adobe Media Encoder to convert it to a decent codec for editing in Premiere Pro I choose DNXHD but there is no 30fps option so instead I choose 29.97 fps for convert, editing and output.

    What are your recommendations? I do videos in general for an “Auto Body Shop”…

    Raúl.

    • Larry says:

      Raul:

      There is, essentially, no difference between these two. Whether you are going to the web or broadcast, converting 30 fps to 29.97 fps is fine.

      Larry

  10. David says:

    Hi Larry, I am having my short film make into a DCP for festivals. They are having issues with my drone footage stuttering, dropping and duplicate frames during playback. What should I be doing in my timeline to prevent this? Do I need to compress the drone footage before using it? Create compound clips? Here’s their message – “Panning aerial shot from 00:00:58:16 to 00:01:09:03 contains dupe frames, possibly due to speed change. Extremely stuttering playback due to varying cadence (2 move 1 freezes, 2 move 1 freezes, 1 moves 1 freezes; repeat).”

    • Larry says:

      David:

      I know the cause, but not the fix. This “stuttering,” though I would not call it “extreme,” is always created when you use software – Final Cut, Premiere – to change the speed of a clip. Because all that slow motion is doing is playing the same frame more than once. Depending upon the speed you select, you can play this in an even sequence (i.e. 50% speed means playing each frame twice, 25% speed means playing each frame four times) or a variable speed (33% means playing one frame twice, two frames once).

      The only perfect solution is to shoot the frame rate you need to deliver. But, after the fact, this is impossible.

      I would contact your DCP company and ask them how to resolve the issue of slow motion created using the NLE.

      Larry

← Older Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends

Final Cut Pro X 10.4

FCPX Complete

Edit smarter with Larry’s brand-new webinars, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.