Frame Rates are Tricky Beasts

Posted on by Larry

creativity

My goal in this article is to discuss the challenges in converting frame rates.

If everything you shoot, edit and output is a single frame rate, then don’t change anything. This is the ideal way to work. However, as you start to integrate elements that originate at different frame rates, frame rate conversion rears its very ugly head.

DEFINITIONS

Think of a video clip as a series of wooden children’s blocks connected by a piece of string. Each block represents a frame of video. As we pull the string, tugging the blocks along in a line, the frame “rate” represents the number of blocks (or images or frames) that pass an observer each second. Frame rate is measured in frames per second; “fps.”

Changing the speed of a clip is NOT the same as changing the frame rate.

This difference is significant. The first is easy, the second is hard.

We change the speed of a clip to create a visual effect. We change the frame rate of a clip to match the settings of our clip to the project. If you don’t need to match settings, don’t mess with changing frame rates.

SETTING BOUNDARIES

There are two sides to a frame rate discussion:

There is a lot of debate as to which is the “best” frame rate. Some feel that 24 fps is more “cinematic,” while 60 fps is more “real.” As you should know by now, there is no “best.” Just as there is no “best” car, camera, or restaurant; there are simply choices.

Converting to a 24 fps frame rate will NOT make your movie look “filmic.” It will, generally, just make it look worse. The “cinematic look” is a combination of: lenses, lighting, depth of field, shutter speed, shutter angle, motion blur and frame rate. Changing the frame rate only affects the frame rate, not the look.

There are no right answers, just louder voices.

Also, to keep this article to a manageable length, I will ignore:

These special cases don’t alter the basic rules of frame rates, though they can complicate understanding.

THE BASIC RULES

Whether you use Adobe, Apple, Avid, or any other video editing software on Macs, PCs or mobile devices, the basic rules of frame rates remain the same:

HOW WE GOT TO TODAY

In the early days of film, say 1890 – 1915, all cameras were hand-cranked. During this time, frame rates wandered from 8 fps to 30 fps, often in the same scene. In those days, the value of a camera operator was not based on their composition skills, but on the consistency of their cranking.

NOTE: This is one of the reasons comedies were so prevalent in the early days of film. Speed changes are inherently comedic and physical comedy does not require dialog.

As films grew in popularity and profitability, standards developed allowing cameras to be cranked by a motor, rather than by hand. Also, at this time, the industry settled on a frame rate of 18 fps.

Why? Because film was expensive and producers were, um, cheap. 18 fps provided the illusion of smooth movement without wasting a lot of film and money.

This standard continued up until the advent of talkies, which exploded on the scene in 1927 with The Jazz Singer. The problem was that 18 fps was not fast enough to support high quality audio. This frame rate yielded audio roughly equivalent to a telephone call.

So, a new frame rate standard needed to be developed – and the industry chose 24 fps.

Why? Because film was expensive and producers were still, um, cheap. 24 fps provided the illusion of smooth movement with relatively high-quality sound without wasting a lot of film, and money.

NOTE: Sound quality continued to improve over time, not by increasing the frame rate, but by shifting audio from an optical track to a magnetic track.

When video arrived, in the 1930’s, we had a major timing problem. How to get the TV receiver to “pulse” in sync with the transmitter? The solution was AC power. All across the US, power “pulsed” at 60 cycles per second.

Television engineers adopted this “universal” pulse as the basic timing circuit for video. Since video in those days was interlaced, where a single frame (complete image) was composed of two fields (a portion of the image consisting of all the odd or even scan lines), each field pulsed at 1/60th of a second.

Ta-dah! 30 fps video.

Except, over time it was discovered that high-voltage electricity “evaporated” from transmission lines when the cycle rate was too high. 50 cycles preserved more power over distance than 60 cycles (now called Hz). So, when much of the world was rebuilt after World War II, the utility companies, to save money and power, dropped the cycle rate to 50 Hz.

From there, the video industry derived 25 fps video, because interlacing was still in vogue.

So, at the dawn of the HD era in the early 1990’s, we had three principle frame rates: 24, 25, and 30 (which, with the advent of color was slightly modified to 29.97 fps, because why should this story be particularly simple?)

And, as we all know, with the rise of HD, our industry came together as a group and standardized on a single frame size and single frame rate.

– – –

Sigh… No such luck.

At last count, we now have nine different frame rates: 23.98, 24, 25, 29.97, 30, 48, 50, 59.94 and 60. (And, yes, 100 and 120 fps are knocking on the door. Please keep that door shut…!)

No WONDER we’re all confused. We’ve been handed a veritable Gordian Knot of frame rates!

CONVERSION OPTIONS

Frame rate conversion is the process of duplicating or removing frames such that, when the clip is played in a sequence that matches the frame rate of the clip, all action appears at “normal” speed. (I put “normal” in quotes because I couldn’t figure out an easy or good way to define normal.)

Most of the time, video editing software will automatically handle frame rate conversions. And, most of the time, I suggest you not worry about it, because, most of the time, it will look fine.

REMEMBER: “Camera-native frame rates always look better than converted frame rates.”

When it comes to frame rate conversions, there are easy options and hard options. Following the wooden block analogy I introduced earlier, we can’t just stretch frames to different rates because each frame is made of wood, not Silly Putty. Instead, we change frame rates by inserting or removing entire blocks.

EASY: 50 fps to 25 fps – or 60 fps to 30 fps

Assuming the video is progressive, conversion simply deletes every other frame.

If the video is interlaced, one field is deleted while the scan lines in the other field are duplicated. (Yes, this option reduces image quality. That’s one reason I hate interlaced video.)

EASY: 29.97 fps to 59.94 fps – or 25 fps to 50 fps.

Here every frame is duplicated. This does not create slo-mo because the video plays back at 50 fps, yielding the same movement as playing 25 fps video in a 25 fps project.

This does not significantly degrade movement quality, but movement will look more fluid if you shot 50 fps (or 60) originally.

MOSTLY EASY: 24 fps to 25 fps

The traditional way of converting 24 fps to 25 fps is increasing the speed of the 24 fps material 4%. This allows all frames to be displayed and, while the action is a bit faster, it isn’t so much faster that the audience will perceive it.

NOTE: Yes, this speed change means we need to speed the audio as well. There’s no free lunch.

HARD: 24 fps to 29.97 fps

This was done traditionally when converting films for television broadcast using a telecine.

Here, we need to create, essentially, six “new” frames every second. (The difference between 24 fps and 30 fps.) But we are dealing with children’s blocks here, we can’t create new images, we can only create new frames that contain existing images.

There are several ways to do this, depending upon whether you are working with interlaced or progressive images. The interlaced methods are quite complex and involve duplicating specific fields, not just frames.

But, here’s a simple method to illustrate the process. Take a group of four frames, then duplicate the last frame in the group. Over 24 frames this creates 6 new frames.

When played back at 30 frames a second, most viewers won’t notice the duplicated frame. However, for the discerning, your action will slightly stutter every five frames. This illustrates why you want to avoid converting frame rates.

HARD: 60 fps to 24 fps

Three quick reminders:

NOTE: Optical flow seeks to do just that, invent new frames. However, while good in theory, the results are often worse than not using optical flow.

Here’s an example of how this could be handled: We need to remove 36 frames from every second of video. Since both 60 and 24 are divisible by 3, we can divide each second into three “blocks,” or sections.  This means that a  20 frame block in the source clip needs to be converted into an 8 frame block in the destination clip. To do this:

As you can see, asymmetrical trimming (remove 1 frame, then remove 2, then remove 1…) gets us to the frame rate we need, but at the expense of potentially adding jitter to movement; say during an actor’s walk or as a car drives through a scene.  Whenever we convert frame rates asymmetrically, we run the risk of damaging the movement in the clip.

A SIDEBAR ON VIDEO COMPRESSION

Video can be compressed in one of two ways:

I-frame formats include: ProRes, GoPro Cineform, AVC-Intra and the DNx family of codecs.

GOP formats include: AVCHD, H.264, HDV and most formats that generate very small file sizes.

As I was writing this article, it occurred to me that camera-native GOP-format video will probably suffer more from image degradation as you change frame rates than video that was shot as I-frame media.

I haven’t tested this, and would like to hear other opinions, but if you are seeing blurry images when changing frame rates by small increments, I would suspect your video format is too blame.

SUMMARY

Frame rates are complex. However, a little planning ahead can simplify headaches. The best option – always – is to shoot the frame rate you need to output.

And, keep in mind, that a “film-like look” does not necessarily require a “film-like frame rate.”


Bookmark the permalink.

102 Responses to Frame Rates are Tricky Beasts

← Older Comments
  1. JR says:

    Hi. I have a 9 year old Sony AVCHD cam capturing at 29.97 fps. It still works great, but I have to film my nephew’s wedding next month. A nightmare scenario is it dies from old age during the wedding. So I want to buy a new videocam to be safe. I’ll put the older videocam on a tripod right behind the bride/groom/pastor, and move around the sides and with the new one. For the best looking editing mix from the two cams, should I buy another 29.97 videocam or a 60 fps model?

    • Larry says:

      JR:

      Buy a new camera that shoots 59.94, that is easy to reduce to 29.97 in the edit, without stutter. Also, make SURE your older camera shoots progressive. If it shoots interlaced, I would suggest you not use it. Interlacing is designed for TV sets, Progressive is designed for the web.

      Finally, it goes without saying, so I’m gonna say it, that both camera should shoot the same frame size.

      Larry

  2. Jayson Bodnar says:

    Thanks.

  3. Jennifer says:

    Thanks for this article. I have two questions:
    What happens when moving 30fps to 29.97, and what steps need to be taken for audio sync? The footage uses Prores for some and DNx for others..

    I’m also wondering if work delivered digitally at 29.97 will have a problem being screened via digital projectors in Europe. Will 50 Hz make a difference?

    • Larry says:

      Jennifer:

      Audio is not synced to frame rate. Audio is synced to samples per second. So, theoretically, you should be OK. As with all things in media, test to be sure.

      If the projector can’t handle 29.97 fps, then, yes, you’ll have stuttering in your image during playback.

      Larry

  4. DONALD T. WOODARD says:

    Larry,
    Love the article and right with you on most everything. Arguing with a videographer who insists on shooting/editing at 24 when we are broadcasting at 720/59.94.
    I’ve had to try & explain that when they give us 24frame media that our playback machine has to interpolate the frames not in the original media.
    I have to disagree with you on one point you made to Laurie
    March 30, 2020 at 9:36 am
    Laurie:

    That’s one way. Another is to drop the movie into a 59.94 timeline in Premiere. MAKE SURE!!! however, whether they want 59.94 PROGRESSIVE or INTERLACED. Most broadcasters are using interlaced at that frame rate.

    Not True…
    All TV outlets using 1280×720 are PROGRESSIVE at 59.94 ALL OTHER BROADCAST TV outlets using 1920×1080 are INTERLACED at 59.94. NBC and CBS chose 1080i, while ABC and FOX chose 720p. 1080p is Sat or Web delivery…

    • Larry says:

      Donald.

      You are correct. I should have been more precise, as both Fox and ABC are progressive networks.

      Thanks for the correction.

      Larry

  5. Michael Saracho says:

    Hi Larry,

    I’m an assistant editor working on Avid Media Composer. I have 6 fps footage and I want to convert to 24 fps in the Avid. I created a sequence and placed a Timewarp effect and dropped it to 25%. It does what I ask but it cuts off the clip to almost 25% in length. I think I know why but I would like your opinion, thank you!

    • Larry says:

      Michael:

      While I don’t know Avid, I do know what you are trying to do and you have an incorrect setting.

      You’ve told Avid to play the clip 4X faster, which would play the clip at 24 fps – and shorten the duration to 1/4 the length of the original. Why a shorter duration? Because you are playing the clip faster than normal, which means it will end more quickly.

      But, I suspect, you don’t want the clip to play faster, you want to play the clip at normal speed, but in a 24 fps sequence. This means that Avid needs to retime the clip so that each single frame plays for 4 frames each. This stretches the length of each frame, without changing the duration of the clip.

      The first option is called a “speed change,” the second option is called: “clip retiming.” Check your user guide to see how this is done on an Avid.

      Larry

  6. James says:

    Thanks for this Larry.

    Question: Will the final result look the same if I shoot in 50fps and output to 25fps as if I shoot 25fps and output to 25fps (assuming I don’t use slow mo etc)? Or will there be a difference in how the final product will look due to the different shooting frame rate in each case?

    • Larry says:

      James:

      Shooting 50 and outputting 25 will look the same as shooting 25. Converting 50 to 25 simply means your NLE will drop every other frame.

      Larry

  7. Michael says:

    I work a lot with old Egyptian films, as they are broadcasted on today’s satellite networks, and noticed that they are sped up an average of 4%. I can confirm this by comparing the music and pitch of the high quality CD recordings with the songs in the films. I think it’s because of the 24fps to 25fps.

    It’s unfortunate this happened to their films. Is this something common in other parts of the world? Have you seen this speeding up of films in America, Bollywood, or elsewhere? I would love to hear your insight. Thank you.

    • Larry says:

      Michael.

      You are correct. The traditional way to convert 24 fps to 25 is to speed everything up 4%.

      While this somewhat disrespects the sound track, it makes for the smoothest video playback because no frames are added or dropped.

      Larry

      • Michael says:

        Thank you. With the advent of new technologies that solved this issue, do you know if this method of speeding up is still used today? My hunch is the satellite networks that play these films are not complaining about the fact that 7 minutes have been shaved of the films, leaving more time for ads.

        • Larry says:

          Michael:

          I don’t know for sure, but I suspect this practice is still followed – mainly because it does the least damage to the video. However, they MAY use pitch shifting on the sound track to compensate. My guess is that they don’t.

          Larry

  8. Brandon Tucker says:

    Hello Larry,

    Thank you for the article. Yours is maybe the clearest explanation I’ve found of the problem.

    I am trying to edit together old home movies shot mostly at 16fps and 18fps (but with a few at 24fps and 29.97fps). As you can imagine, that diversity has proven troublesome. But I keep thinking there must be SOME WAY to group these clips together without messing up the natural speed or accepting compromise solutions like frame sampling/dropping— if only because I can open and play them all separately on my computer just fine. It seems there must be SOME WAY on earth to translate that individual smoothness of the source videos into a single watching experience.

    I know that you said “Projects/sequences can only have one frame rate”, but in a way, I can’t believe that means there is no way of getting separate fps into a single viewing experience. As an analogy, if a professional were making a documentary on the history of movies across the ages with diverse fps (like Marc Cousins’ The Story of Film: An Odyssey), I can’t imagine they would just accept frame sampling/dropping or unnatural speeds or feel beholden to smashing everything into the one frame rate of their editing software’s timeline. Of all the crazy things movies can do, accepting this situation as inevitable seems inconceivable. Why not just have seconds 1-5 at 16fps and seconds 6-10 at 24fps and jump back and forth like that? What technologically prevents this? Why should uniform frame rate be infallible?

    If a computer can play each clip at its source’s frame rate individually, there must be SOME WAY to get them all together in a single experience—even if that maybe takes a little extra work or requires some extra software. Why or why not do you agree with my reasoning? I would deeply appreciate your thoughts, Larry. Thank you very much.

    • Larry says:

      Brandon:

      Thanks for your question. Frame rates are tricky. However, it is absolutely true that a project or sequence can have only one frame rate. Period. No exceptions. What producers do, if needed, is convert the frame rate of old movies to the frame rate of your project. There are a variety of technical reasons for this limitation, which all revolve around how each NLE counts frames from one edit to the next. NLEs don’t edit based on time code, they edit based on frame counts. And if the frame count were to change, all the edit positions would shift. (By counting frames, rather than timecode, it becomes very easy to support multiple timecodes from source clips in the same project.)

      You can – with the right hardware or software – change frame rates without changing the perceived speed of a clip. It doesn’t need to speed up or slow down. However, this requires using the right gear to get the frame rates to match.

      Larry

← Older Comments

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends:

FCPX Complete

NEW & Updated!

Edit smarter with Larry’s latest training, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.