Frame Rates are Tricky Beasts

Posted on by Larry

creativity

My goal in this article is to discuss the challenges in converting frame rates.

If everything you shoot, edit and output is a single frame rate, then don’t change anything. This is the ideal way to work. However, as you start to integrate elements that originate at different frame rates, frame rate conversion rears its very ugly head.

DEFINITIONS

Think of a video clip as a series of wooden children’s blocks connected by a piece of string. Each block represents a frame of video. As we pull the string, tugging the blocks along in a line, the frame “rate” represents the number of blocks (or images or frames) that pass an observer each second. Frame rate is measured in frames per second; “fps.”

Changing the speed of a clip is NOT the same as changing the frame rate.

This difference is significant. The first is easy, the second is hard.

We change the speed of a clip to create a visual effect. We change the frame rate of a clip to match the settings of our clip to the project. If you don’t need to match settings, don’t mess with changing frame rates.

SETTING BOUNDARIES

There are two sides to a frame rate discussion:

There is a lot of debate as to which is the “best” frame rate. Some feel that 24 fps is more “cinematic,” while 60 fps is more “real.” As you should know by now, there is no “best.” Just as there is no “best” car, camera, or restaurant; there are simply choices.

Converting to a 24 fps frame rate will NOT make your movie look “filmic.” It will, generally, just make it look worse. The “cinematic look” is a combination of: lenses, lighting, depth of field, shutter speed, shutter angle, motion blur and frame rate. Changing the frame rate only affects the frame rate, not the look.

There are no right answers, just louder voices.

Also, to keep this article to a manageable length, I will ignore:

These special cases don’t alter the basic rules of frame rates, though they can complicate understanding.

THE BASIC RULES

Whether you use Adobe, Apple, Avid, or any other video editing software on Macs, PCs or mobile devices, the basic rules of frame rates remain the same:

HOW WE GOT TO TODAY

In the early days of film, say 1890 – 1915, all cameras were hand-cranked. During this time, frame rates wandered from 8 fps to 30 fps, often in the same scene. In those days, the value of a camera operator was not based on their composition skills, but on the consistency of their cranking.

NOTE: This is one of the reasons comedies were so prevalent in the early days of film. Speed changes are inherently comedic and physical comedy does not require dialog.

As films grew in popularity and profitability, standards developed allowing cameras to be cranked by a motor, rather than by hand. Also, at this time, the industry settled on a frame rate of 18 fps.

Why? Because film was expensive and producers were, um, cheap. 18 fps provided the illusion of smooth movement without wasting a lot of film and money.

This standard continued up until the advent of talkies, which exploded on the scene in 1927 with The Jazz Singer. The problem was that 18 fps was not fast enough to support high quality audio. This frame rate yielded audio roughly equivalent to a telephone call.

So, a new frame rate standard needed to be developed – and the industry chose 24 fps.

Why? Because film was expensive and producers were still, um, cheap. 24 fps provided the illusion of smooth movement with relatively high-quality sound without wasting a lot of film, and money.

NOTE: Sound quality continued to improve over time, not by increasing the frame rate, but by shifting audio from an optical track to a magnetic track.

When video arrived, in the 1930’s, we had a major timing problem. How to get the TV receiver to “pulse” in sync with the transmitter? The solution was AC power. All across the US, power “pulsed” at 60 cycles per second.

Television engineers adopted this “universal” pulse as the basic timing circuit for video. Since video in those days was interlaced, where a single frame (complete image) was composed of two fields (a portion of the image consisting of all the odd or even scan lines), each field pulsed at 1/60th of a second.

Ta-dah! 30 fps video.

Except, over time it was discovered that high-voltage electricity “evaporated” from transmission lines when the cycle rate was too high. 50 cycles preserved more power over distance than 60 cycles (now called Hz). So, when much of the world was rebuilt after World War II, the utility companies, to save money and power, dropped the cycle rate to 50 Hz.

From there, the video industry derived 25 fps video, because interlacing was still in vogue.

So, at the dawn of the HD era in the early 1990’s, we had three principle frame rates: 24, 25, and 30 (which, with the advent of color was slightly modified to 29.97 fps, because why should this story be particularly simple?)

And, as we all know, with the rise of HD, our industry came together as a group and standardized on a single frame size and single frame rate.

– – –

Sigh… No such luck.

At last count, we now have nine different frame rates: 23.98, 24, 25, 29.97, 30, 48, 50, 59.94 and 60. (And, yes, 100 and 120 fps are knocking on the door. Please keep that door shut…!)

No WONDER we’re all confused. We’ve been handed a veritable Gordian Knot of frame rates!

CONVERSION OPTIONS

Frame rate conversion is the process of duplicating or removing frames such that, when the clip is played in a sequence that matches the frame rate of the clip, all action appears at “normal” speed. (I put “normal” in quotes because I couldn’t figure out an easy or good way to define normal.)

Most of the time, video editing software will automatically handle frame rate conversions. And, most of the time, I suggest you not worry about it, because, most of the time, it will look fine.

REMEMBER: “Camera-native frame rates always look better than converted frame rates.”

When it comes to frame rate conversions, there are easy options and hard options. Following the wooden block analogy I introduced earlier, we can’t just stretch frames to different rates because each frame is made of wood, not Silly Putty. Instead, we change frame rates by inserting or removing entire blocks.

EASY: 50 fps to 25 fps – or 60 fps to 30 fps

Assuming the video is progressive, conversion simply deletes every other frame.

If the video is interlaced, one field is deleted while the scan lines in the other field are duplicated. (Yes, this option reduces image quality. That’s one reason I hate interlaced video.)

EASY: 29.97 fps to 59.94 fps – or 25 fps to 50 fps.

Here every frame is duplicated. This does not create slo-mo because the video plays back at 50 fps, yielding the same movement as playing 25 fps video in a 25 fps project.

This does not significantly degrade movement quality, but movement will look more fluid if you shot 50 fps (or 60) originally.

MOSTLY EASY: 24 fps to 25 fps

The traditional way of converting 24 fps to 25 fps is increasing the speed of the 24 fps material 4%. This allows all frames to be displayed and, while the action is a bit faster, it isn’t so much faster that the audience will perceive it.

NOTE: Yes, this speed change means we need to speed the audio as well. There’s no free lunch.

HARD: 24 fps to 29.97 fps

This was done traditionally when converting films for television broadcast using a telecine.

Here, we need to create, essentially, six “new” frames every second. (The difference between 24 fps and 30 fps.) But we are dealing with children’s blocks here, we can’t create new images, we can only create new frames that contain existing images.

There are several ways to do this, depending upon whether you are working with interlaced or progressive images. The interlaced methods are quite complex and involve duplicating specific fields, not just frames.

But, here’s a simple method to illustrate the process. Take a group of four frames, then duplicate the last frame in the group. Over 24 frames this creates 6 new frames.

When played back at 30 frames a second, most viewers won’t notice the duplicated frame. However, for the discerning, your action will slightly stutter every five frames. This illustrates why you want to avoid converting frame rates.

HARD: 60 fps to 24 fps

Three quick reminders:

NOTE: Optical flow seeks to do just that, invent new frames. However, while good in theory, the results are often worse than not using optical flow.

Here’s an example of how this could be handled: We need to remove 36 frames from every second of video. Since both 60 and 24 are divisible by 3, we can divide each second into three “blocks,” or sections.  This means that a  20 frame block in the source clip needs to be converted into an 8 frame block in the destination clip. To do this:

As you can see, asymmetrical trimming (remove 1 frame, then remove 2, then remove 1…) gets us to the frame rate we need, but at the expense of potentially adding jitter to movement; say during an actor’s walk or as a car drives through a scene.  Whenever we convert frame rates asymmetrically, we run the risk of damaging the movement in the clip.

A SIDEBAR ON VIDEO COMPRESSION

Video can be compressed in one of two ways:

I-frame formats include: ProRes, GoPro Cineform, AVC-Intra and the DNx family of codecs.

GOP formats include: AVCHD, H.264, HDV and most formats that generate very small file sizes.

As I was writing this article, it occurred to me that camera-native GOP-format video will probably suffer more from image degradation as you change frame rates than video that was shot as I-frame media.

I haven’t tested this, and would like to hear other opinions, but if you are seeing blurry images when changing frame rates by small increments, I would suspect your video format is too blame.

SUMMARY

Frame rates are complex. However, a little planning ahead can simplify headaches. The best option – always – is to shoot the frame rate you need to output.

And, keep in mind, that a “film-like look” does not necessarily require a “film-like frame rate.”


Bookmark the permalink.

102 Responses to Frame Rates are Tricky Beasts

Newer Comments →
  1. Standards conversion requires a complex algorithm. The best seamless transcoding system with the least artifacts is Cinnafilm’s Tachyon software. I have no connection to the company it is simply the best. Cinnafilm.com

  2. Ryan Horner says:

    Good article, thank you. But what if you have a 29.97 fps and it needs to be converted to 25 fps for the euro standard. Can this be done in premier or should we go to a place that uses hardware? I have 13 chapters that the client wants in PAL. Possible?

    • Larry says:

      Ryan:

      It is possible, but it may not be easy.

      I realize that I need to quality my comment that it is easier to go from a faster frame rate to a slower one. 29.97 to 24 or 29.97 to 25 are DIFFICULT conversions if you have a lot of movement.

      If nothing is moving – think interviews – the conversion is relatively easy and painless. Or, if the action is moving violently – think chase scenes – the conversion is also easy, because you’ll never see the small jitters. It’s the part in the middle that is awkward. Smooth, obvious, gliding movement is the hardest to convert.

      What I would do is create my project in Premiere or FCP X and export a high-quality master file. Then, using other software to change the frame rate. I would try Adobe Media Encoder (for Premiere) or Compressor (for Final Cut) first. If the movement looks smooth to you, then they ARE good. There are no “hidden specs” you need to worry about.

      But, if the movement looks jerky, you’ll need to use more powerful tools which rebuild your existing frame rates into something different. Check into Cinnafilm, listed above, or Amberfin.

      Larry

  3. Chris says:

    Instead of “normal”, you can say natural.

  4. Richard Hale says:

    FYI
    Your description of 18fps as a silent era film rate is an over simplification. Often the projectionist, who could also determine the speed, was given a frame rate cranking speed to use when projecting the film. In some cases films were cranked at lower speeds when shot to conserve film and then instructed to be played back at that speed – more ‘flicker’ but more economical.
    Another thing to note is although a film is shot at 24 fps, when played back a shutter is used (a three or two bladed) which interrupts each frame projecting 3 or 2 flashes of each frame on the screen resulting in an illusion of 72 or 48 fps. A telecine projector designed to convert 24fps to 30fps video uses a 5 bladed shutter to achieve the conversion eliminating flicker in the video.
    When film spread to the home and school market sound speed was set at 24fps and silent speed at 16fps. Almost all home movies were shot at 16 fps until Super 8mm came along in 1965 and set the standard to 18fps, also accommodating 24fps for compatibility with the industry sound standard.
    When converting home ‘film’ movies to video and maintaining the original speed of the film without flicker first, a video image of every frame is made in sequence. Using Super 8mm as an example (18fps). these images are then wrapped into a video movie file which would play back frame for frame naturally at 18fps. Import this into a 24 fps Editing Project and the software will create a file with a frame sequence as 1,2,3,3,4,5,6,6 etc. This produces an acceptable conversion retaining most of the original film look.

    • Larry Jordan says:

      Richard:

      Great comments – thanks for sharing!

      Larry

    • Mike says:

      Larry and Richard,

      Jonathan Erland of Pickfair Institute has a great demo he’s presented most recently at HPA and SMPTE Hollywood 2015 that goes into these details with great explanation and creative presentation. Very entertaining, very informative.

    • D Gary Grady says:

      To add a bit of historical trivia to what Richard says, each crank of the camera (or projector) handle moved 8 frames or six inches of film through the gate, so two cranks per second yielded 16 frames. Cameramen held competitions to come closest to exactly two cranks (one foot of 35mm film) per second. In practice, exhibitors often showed films faster than 16 fps to cram more showings into a day.

      In Larry’s defense, the notion that 18 fps was the silent standard is now quite common, though less so than the notion that Academy Aperture for sound movies was 1.33:1 (it was 1.375:1; 1.33: was the silent standard however) or that the scope (anamorphic) ratio is 2.35:1 (it’s been 2.39:1 since SMPTE changed it in 1970). And all this is admittedly highly picky and pedantic, but I’m old, and what else have I got?

      Another quibble: The multi-bladed shutter addresses the problem of flicker, but it does not have the motion-smoothing effect of a higher frame rate, so 24 fps with a two-bladed shutter doesn’t look like 48 fps, and 24 fps seen on 60i television still looks looks like 24 fps in terms of movement, not like 60i.

  5. Jamie LeJeune says:

    For all the tricky conversions I’ve been sending my footage to Isovideo. They use very powerful algorithms running on customized multi-GPU systems to do seamless standards conversions to a degree of quality I’ve never seen anywhere else. For example, converting interlaced 29.97 masters to 24.00 progressive for DCP or 23.976 progressive for cinematic looking BluRay. Not only do they accomplish a conversion with zero jitter they also add exactly the right amount of motion blur to the resulting frames so that they look identical to media captured with a standard 180 degree shutter. Also, conforms are a breeze because Isovideo integrates perfectly in a file based workflow. I can send a set of interlaced 29.97 clips and get back a set of 23.976 progressive clips with the same exact length and file names so that they relink instantly in my NLE or grading application. They also do fantastic de-interlacing and down/up scaling. I used to go to local post houses for hardware conversions, but since finding Isovideo, I use them for all my footage conversion. I’m not affiliated with them, but I am a huge fan and as they have made my life in post so much simpler I do hope they remain in the business. None of the other conversion options that I’ve found compare to them. BTW if you do check them out, be warned that their website looks like something from 10 years ago, but despite their design sense being out of date their tech is definitely cutting edge.

  6. Andy says:

    I think that there is a way to convert from any rate to any other rate by generating new frames using interpolation of every pixel between existing frames treating existing frames as key frames.
    After all this is digital media and not a physical film.

    Twixtor and other software does that.

    • Larry says:

      Andy:

      While true, the process of creating new frames – which Apple calls “optical flow” – is not always smooth or reliable. Creating new pixels is easy, creating the RIGHT pixels is very much harder.

      Larry

  7. Justin says:

    As always, a very thoughtful explanation to a non-intuitive subject matter that is often riddled with misinformation. I just finished a project whereas I shot 4K at 60fps (actually 59.94p) on one camera and 30 fps on the other two (29.97p).

    I want to conform them all (in the latest version of FPX) to 24p but there is also an option for 23.98p. Is there a difference (between 23.98p and 24p)? Is one easier for FPX to conform from the above two frame rates?

    Thanks so much for your time and insights,
    Justin

    • Larry says:

      Justin:

      It would be MUCH better to conform both to 30 fps (or 29.97) than 23.98 and you will probably get much better results. In general, cutting the frame rate in half is easy. Converting to anything else is not – and very likely to create motion jitter due to the frame rate conversion.

      Unless you plan to distribute your movie theatrically, you don’t need to convert to 24 fps – and, even then, it may not be necessary. Check with your distributor for supported frame rates. Digital files are much more flexible about frame rates that traditional mechanical devices.

      If you are going to the web, absolutely do NOT convert your frame rates to 24, 30 fps will look perfectly fine without the risk of resampling frame rates.

      Larry

  8. Patrick says:

    I have a Phantom 4 Pro drone, and in a few months I will be filming scenes of fish being caught. These scenes will make up probably less than five minutes of a movie intended to be distributed theatrically. Obviously, these are action scenes with water splashing and writhing fish. The lense is wide angle and the drone will not be able to get as close to the action as I would like. This makes me think I should shoot in as high resolution as possible to enable zooming in postproduction.

    My frame rate choices for C4K and 4K in H.265 are: 24, 25, and 30.

    For H.264: 24, 25, 30, 48, 50, and 60.

    If I drop down to FHD or HD in either file format I can increase frame rates to as high as 120.

    Considering post production conversion of frame rates and capturing fast action, what would be the best way to capture these images?

    • Larry says:

      Patrick:

      The CRITICAL thing is to confirm deliverable settings with your editor or distributor. They will probably want a frame rate of 23.976 fps, but 24 and even 48 would not be unheard of.

      Second, do a test now based on their specs – shoot traffic or some other common moving object and send it to them for review. With this much lead time, it is easy to find problems and solutions.

      That being said, you are correct: if you plan to zoom into the footage, shooting at a larger frame size will improve quality. Faster frame rates are only needed if you plan to do high-quality slow motion. Equally important, record at the highest bit rate your drone will support. Too much compression will degrade any image – especially those that are blown up to a very large screen.

      Overall, though, allow time for tests to make sure your codec, frame size and frame rate yield the quality and technical stability that your clients require.

      Larry

  9. Ben says:

    Great article, one I will pass on as an essential resource to others.

    Just one thing. This point you made here:

    “Frame rates are irrelevant on the web. Streaming and downloadable files can play at any frame rate”

    That’s untrue as a generalization, although from certain points of view it is the case.

    Frame rates are extremely relevant. In fact, I am on a project right now where the only actual complication IS the fact that the web is involved.

    Here’s some considerations (regardless if one has a quick solution for any of them or not, the fact that they exist make frame rates relevant)

    – Streaming providers offer limited support
    – Software and display hardware often does less than ideal conversions
    – Bandwidth basically doubles when it comes to encoding – a useless waste if the content is simply being frame doubled at the sending end and halved at the receiving.
    – YouTube for example will do this with content ingested at 50fps:
    – offer it at 60fps
    – offer it at 30fps
    – once stream complete and processed, the 50fps version will then be available to download
    – Already compressed video makes conversions much harder, putting the emphasis back on the source.

    There are others, but overall, in considering frame rates – if it’s to be used for the web, one really should consider these factors and others as to their impact on the overall product / brand

    • Larry says:

      Ben:

      I understand what you are saying, but I disagree.

      Frame rates are not relevant when playing media files on a computer; a computer can play any frame rate. That is still true. However, some social media sites, in order to simplify their compression workflow, may decide to limit the frame rates they will accept or transcode all media into a limited number of frame rates.

      One other clarification, file size is not directly proportional to frame rates. It was 15 years ago, but it is not now due to the prevalence of inter-frame encoding.

      I also agree that changing the frame rate of an already compressed file will yield very poor results, due to the math underlying the compression.

      Thanks for your thoughts.

      Larry

  10. abejah maginley says:

    how about converting 25fps to 23.978. for some reason the audio keeps drifting.

    • Larry says:

      Abejah:

      Generally, audio drift is caused by changing the sample rate, instead of the frame rate. Make SURE the sample rate stays consistent at 48kHz.

      Larry

Newer Comments →

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends:

FCPX Complete

NEW & Updated!

Edit smarter with Larry’s latest training, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.