Review: Topaz Video AI – A Wonder at Deinterlacing and Up-scaling Video

[Read my product review disclosure statement here.]

UPDATE CAUTION (May 16, 2023): There is a bug in Topaz Video AI 3.2.5 which makes deinterlacing confusing. This bug is fixed in version 3.2.6.

Topaz Video AI creates amazing results. It isn’t perfect. It requires serious computing power, takes time to process, and its interface can be confusing. But, when you learn how it works, it creates results that will drop your jaw to the floor.

Topaz Video AI is used for deinterlacing video, upscaling low-resolution video to a higher resolution, removing excessive motion blur and creating super smooth slow motion. All with the goal of improving image quality.

The folks at Topaz Labs gave me a one-month license to review the current version (3.1.11) of their software. I’ve spent the last three days working with it and here’s what I learned.

EXECUTIVE SUMMARY

The key features in Topaz Video AI include:

• Deinterlacing. The deinterlacer in Topaz Video AI is outstanding and good enough on its own to justify purchasing the program.
• Up-scaling (also called “up-rezing” or “up-resing”). This adds back detail that was lost due to the small size of the original image.
• Removing motion blur. This gives footage the feeling it was shot at high-frame rates. While not generally used for drama, it makes action and sports footage much more “real” and clear.
• Creating extreme slow motion. The slowmo created by Video AI is far better than optical flow, with consistent and clear details. However, it takes time to process.
• Image stabilization. Frankly, stabilization is a complete mess. It takes – I did the tests – up to 53 times longer to stabilize a clip in Video AI than in Final Cut Pro. Premiere Pro yields results similar to FCP. This is not a good app to use for stabilization.

In addition to image stabilization, Video AI has other issues such as: confusing presets, long processing times on slower systems and a lack of clear guidance providing a recommended way to accomplish a task. For instance, an explanation on how to create super slow motion is found in the online Product Roadmap; which makes no sense. A “roadmap” is not a tutorial.

Clearly, like all software wrapped around AI, Topaz Video AI is a work in progress. What it does well is amazing. What it has problems with can be solved using other tools.

Would I recommend the program? Yes, but be sure you have a fast computer to run it on.

Developer: Topaz Labs
Product: Topaz Video AI
Website: https://www.topazlabs.com/topaz-video-ai
Price: $299 (US) one-time purchase

TEST GEAR AND RESULTS

I tested Video AI on an M1 Pro 16″ MacBook Pro running macOS Ventura 13.3. I was using the latest version of Topaz Video AI: v3.1.11. All media was stored on the internal drive of my Mac.

I created a series of 24 tests for the software, ran each test twice and averaged the results in this table. The most important column has numbers in bold: “Time to Convert 1:00 of Media.” This normalizes the results of each task so that regardless of how long the source clip is, we can compare results between tests.

NOTE: There were four tests which ran a ridiculously long time. I only ran those once because, over time, I realized the app did not vary much in how long it took to process a task.

Column definitions:

• Task. A somewhat cryptic description of what I did. (For example, “Deint.” means “deinterlace”.)
• Source clip duration. The duration of the source clip in minutes and seconds (mm:ss)
• Time to Convert: This converts actual results into normalized results. For example, a ten-second clip that took a minute to process means that if that clip were one-minute long the processing time would be six minutes.
• Average. The average of the two test durations
• Standard Deviation. The smaller this number, the more consistent the two test durations
• Test 1. The duration of the first test
• Test 2. The duration of the second test
• Note. Mostly a note to myself on something I wanted to write about

NOTE: Here’s a one-page PDF of all my test results.

INSTALLATION & UPGRADING

Downloads are from the TopazLabs website. Installation is straight-forward and works as you would expect.

The software also notifies you, after you start the app, when an upgrade is available. Upgrading doesn’t work as well.

After beginning the update process, the installer displays a screen saying: “Close Applications and Install.” When I clicked this, it immediately displayed this same message a second time.

On the M1 MacBook Pro, leaving this second warning open would start the installation, closing it would hang the install. On an Intel iMac, the installer hangs regardless of what you do with this second warning. With no other applications running, the installer warned that it was “Waiting for other installations to complete.” After waiting ten minutes or so, the installation finally completed.

So, if you are upgrading, be patient. I saw this problem on three different computers.

OPERATION

When you first open the application, this screen appears. Drag one or more video clips into this panel. Or, click the Browse button and navigate to a folder.

This is the main operating screen for Topaz Video AI. Imported clips are displayed in the lower left, preview is in the top center and settings are on the right.

By default, all settings are turned off so you don’t change something by accident. The system defaults to processing an image at the same size it was imported.

To get started, select an appropriate setting from the Presets menu in the top right corner. You can also manually adjust the settings. Since the presets did not do what I needed, I set them manually – except for always choosing “Deinterlace footage…” from the Presets menu because there’s no obvious way to deinterlace manually.

Click the blue Preview button (right red arrow) to preview a short segment of the video to see the results of that setting. To change the size of the image in the Preview panel, click the Scale button (left red arrow). To generate a longer preview duration, the default is 2 seconds, select it from the Duration menu (middle red arrow).

This preview window displays the source (left) and the clip after the settings are applied – deinterlacing in my example (right). You can play the preview so you see the processed video. This is much better than simply looking at a still image of the finished effect.

This preview is a big help in assuring your clips looks the way you expect before spending the time to export it. The image difference is immediately apparent.

To save time, but not shown here, you can trim and crop clips so you only process that portion of a clip that you need.

To preview as a split screen, click the icon under the left red arrow. The default is side-by-side, which is accessed by clicking the icon under the right red arrow.

When things look good, click the blue Export button at the bottom right.

NOTE: Export is disabled in the free trial version of the software.

The system then displays how long the export will take, as well as calculate the current frames processed each second. These numbers vary based upon the complexity of the settings applied, the frame size of the media and its duration.

Video AI supports adding clips in a batch, with separate settings for each clip. Once exporting starts, the status of each job is tracked under the timeline.

NOTE: I just imported 81 clips and deinterlaced them all in a batch; keeping the frame size the same as the original footage. I selected all 81 and applied the same settings to each. The setup took less than five minutes and the resulting clips – which took several hours to calculate – look amazing.

Topaz Video AI exports 14 different video codecs, depending upon your need. While the default setting is ProRes 422 LT, it puzzles me that ProRes 422 is not on this list. ProRes 422 LT is a 10-bit full-raster codec with an emphasis on smaller file sizes. This makes sense if you are dealing with low-resolution legacy images.

But ProRes 422 is ubiquitous. So, for that matter, is ProRes 4444. Both are missing in this supported codec list. ProRes 422 LT is perfectly adequate, but ProRes 422 would be a better choice.

NOTE: Topaz support tells me that support for ProRes 422 is coming in a future upgrade.

COMPARING RESULTS

This is why we use Topaz Video AI. Whether you are integrating legacy media, or repurposing broadcast media from CBS, NBC or PBS for the web, interlaced video is a fact of life in media today; in spite of our fervent wish it was gone.

The image above on the left is an interlaced NTSC 4:3 source image enlarged three times, using an NLE, to fill a 1920 x 1080 timeline. The image on the right is the source image deinterlaced, enlarged and enhanced using Topaz Video AI. Both images are the same size, but converted from the rectangular pixels of SD video into the square pixels of computers.

NOTE: SD video is a 4:3 rectangle. HD is a 16:9 rectangle. SD uses rectangular pixels, HD uses square pixels. (Yes, I know there are a few exceptions in HD, we will ignore them.) This means that SD video displayed in an HD timeline has black “pillar boxes” on the left and right sides of the image. These black bars allow us to see the source image without it looking horizontally stretched. I removed these pillar boxes from all my screen shots.

When you look at the source image (left) and compare it to the deinterlaced image using Topaz Video AI, the differences are stark and eye-catching. Look at the detail in the skier, the texture in the flying snow and definition of the ground snow. There is no comparison between these two images. Deinterlacing makes a huge difference in improving image quality.

However, the preset to get here is not helpful. You should choose Deinterlace…, then manually switch the Output to the original size in the setting immediately below. (I’ll have more on presets in a bit.)

The next reason to use this software is to improve the resolution (up-scale) a clip. This is typically done when converting legacy SD media for HD projects. Here, again, depending upon how you up-scale the clip, the results are lovely.

Take special notice of the texture in the flying snow and on the ground.

NOTE: While Topaz Video AI does an excellent job upscaling, the more you enlarge any digital image, the worse it looks. Up-scaling an SD clip to 4K will not look good, no matter what software you run it through.

However, while the presets seem to suggest that you deinterlace and up-scale at the same time, my tests show that image quality is improved if you deinterlace to the same size first, then up-scale in a second step.

While this two-step process takes longer – more on that in a bit – the results are worth it. While they are subtle in this still, they are obvious during playback. The image on the right shows the difference between these two screen shots; calculated in Photoshop.

Another feature worth experimenting with is removing motion blur. While this would not generally be used in dramas, it significantly improves image quality for sports and action scenes. What I discovered is that it brought back detail in clothing, as well as the snow on the ground.

I should mention that the quality of the final image depends upon the quality of the source. Video AI works magic, but not miracles. In this scene from a series I directed for WCVB-TV in the 1980’s (“Point of Law with Arthur Miller“), look at the “echoes” to the right of the text and his head.

If we look at this more closely, after deinterlacing, up-scaling and enhancing, the ringing is still there. Arther also suddenly developed six fingers in his left hand.

AVOID STABILIZATION

Another feature of Topaz Video AI is image-stabilization. Video AI provides two stabilization options Auto-crop and full-frame.

Whether I was stabilizing a full 4K frame, or down-scaling 4K to 1080p, I could not tell a difference in results between these two settings. Additionally, I could not tell a difference between the results of these  two settings and the source clip.

Worse, calculating image stabilization ranged from 20 to 50 minutes PER SOURCE CLIP MINUTE to stabilize the image!

Out of curiosity, I ran these  same two tests in Apple Final Cut Pro and they took 12 seconds to calculate and display. Plus, it actually stabilized the image.

There is no reason – not even using batch processing – to use this feature as currently implemented.

A NOTE ON FFmpeg AND ProRes

Topaz Video AI improves the image using AI inside the application, but sends the results to FFmpeg for video processing. I asked Topaz Support why they use FFmepg.

“FFmpeg is a video encoding tool that powers many popular video editing applications and conversion tools. It supports complex filtering and stacking of multiple effects, and offers us the ability to intake many codecs that are not supported by default in Windows and macOS.

“Topaz Video AI adds its AI models into the chain of processing steps that FFmpeg performs when encoding video files and actually gives users the option to run AI models directly from the command line for further customization. Clicking the three-dot menu next to any export job in the Topaz Video AI queue and selecting “FFmpeg command” will reveal exactly what command the application is running to generate enhanced output.

“FFmpeg does frequently update with new features and performance improvements, and the builds included with Topaz Video AI are updated often to maintain pace with the upstream project.”

In the past, Apple made a big deal that FFmpeg had not licensed the ProRes codecs; rather, it reverse engineered them. It warned of quality problems… or worse.

When I asked Topaz Labs support about this they said: “That’s correct, in the past there was no official way to encode ProRes via FFmpeg even on macOS. This has changed with the addition of prores_videotoolbox to FFmpeg as of version 5.0. This is what we are using on macOS versions of Topaz Video AI, both to ensure complete spec compatibility and to make use of hardware acceleration on M-series processors.

“There is no current implementation of a licensed ProRes codec for FFmpeg builds on Windows, so we continue to use the popular prores_ks implementation, which is reverse engineered but has historically been able to operate without issue with playback and editing software on both operating systems.”

I use FFmpeg every week in my webinars and have done so for years, without problems. But it is good to know that, on the Mac version at least, we are dealing with the real thing.

OK, SO JUST HOW FAST IS THIS?

Here’s what these numbers mean:

• To deinterlace an interlaced NTSC DV (720×480) clip takes 2.5 minutes per minute of source video.
• To deinterlace and, at the same time, upscale the clip takes 4.1 minutes per minute of source video, regardless of whether you are going to 720p or 1080p.
• However, for better image quality, you should deinterlace first, then up-scale. When you do, this two-step process takes a total of 6.8 minutes (2.5 + 4.3) per source minute. I don’t know why it takes as long to up-rez a deinterlaced clip as it does to deinterlace and up-rez the same clip.

I tried this again with a different clip. (The first clip ran 1:22, the second ran 0:53.) The normalized results were identical.

• To remove motion blur from an interlaced NTSC DV clip without changing the image size, takes 10.5 minutes in a single pass,
• To remove motion blue from an interlaced NTSC DV clip in two passes for better image quality takes 11 minutes. The two-pass version looks better.

SIDEBAR ON INTEL MACS

All these tests were run on a 16″ MacBook Pro with an M1 Pro SoC. For comparison I ran one of these tests on a 2017 iMac with an i5 Intel CPU.

• Deinterlacing an NTSC DV clip tool 2.5 minutes per source minute on the M1 Pro
• Deinterlacing the same clip on the iMac took 12.4 minutes. Roughly 5 times longer on the Intel CPU. Your mileage will vary, but Intel systems will take a lot longer.

So, if you are running Topaz Video AI on older systems, you need to budget more time for these jobs to complete.

CREATE SMOOTH SLOW MOTION

Another feature where Video AI excels is creating very smooth and clean slow motion. I have never liked the results of using optical flow. It almost never created anything I could use.

Video AI is just the opposite. It creates smooth, fluid and detailed slow motion. However, it takes a while. BUT!! There’s a very cool workaround.

• If you deinterlace a clip and create the slow motion at the same time, it takes 47 minutes per source minute.
• If you deinterlace the clip first, then, in a second pass, create the slow motion, it takes 13.1 minutes per source minute.

I have no idea why the time savings are as great as this – 3.6 times faster! I’ve run this test multiple times with the same results.

Clearly, as you do more with a clip, it takes longer. For example:

• Deinterlacing the clip (step 1) and adding slow motion (step 2) takes 13.1 minutes per source minute.
• Deinterlacing, adding slow motion and removing motion blur takes 18 minutes per source minute.
• Deinterlacing, adding slowmo, removing blur and up-scaling to 720p takes 44.3 minutes per source minute.

The visual results are amazing, but plan for this process to run overnight.

CHECKING SPEEDS WITH ACTIVITY MONITOR

If you look at Activity Monitor during deinterlacing, you’ll notice that Topaz Video AI isn’t doing much of anything (bottom red arrow) and FFmpeg is hardly working (top red arrow) The CPU is loafing and the GPU is asleep.

NOTE: The maximum CPU speed on this system is 1200%. The maximum GPU speed is 100%.

I asked Topaz Support about this and was told: “Activity monitor is not a reliable source of GPU/ CPU [activity. Why?]

“[Because] Video AI uses coreML for all its inferences. CoreML uses other APIs/services to use the neural engine and GPU. So when running the app there would be elevated GPU usage that can be tracked using GPU history view but may not show up at the process level.

“If you hover over the model [names in the interface] you will see a description for what each model has been trained for. ‘Low quality’ refers to the input file, the output file should always be high quality.”

As proof, look at how busy the CPUs and GPUs are when stabilizing a clip.

AREAS WHERE TOPAZ CAN IMPROVE

In many ways, Topaz Video AI amazing. Its deinterlacing is stellar, the slow motion is glossy and the up-scaling makes a silk purse out of a sow’s ear. (Um, that means it does a really good job.)

But here are areas where the application can improve:

• There should be a single preset that just deinterlaces. This prevents people from accidentally up-scaling. This would be the go-to choice for two-step image processing.
• There are two flavors of HD: 720 and 1080. Defining a preset as HD is confusing, especially because it defaults to 1920 x 1080. Popular broadcast networks in the US also use 720p.
• It would be useful to customize what gets appended to the exported file name – for example “_deinterlaced” – instead of the default AI model names appended by Video AI; especially because the AI model names make no sense to users.
• Because speed varies widely depending upon the order you do stuff, presets should be applied in a stack where, for example, you can set deinterlacing to be done first, with other actions following in a separate pass.
• The online help needs “Best Practice” recipes that tell us the steps to take simply and quickly to complete a task, without going into long explanations on why. (Save the explanations for the user manual.)
• Enhancement settings should be clearly labeled as applying to the source clip, not the result.
• Changing the input quality setting should be more obvious.
• A help note should mention that processes complete faster when preview is turned off.
• Provide the ability to boost vibrance, not saturation, on upscaled clips. Up-rezed clips tend to come out a bit desaturated.
• Topaz Labs writes that filters are “stackable.” However, the processing order of filters displayed in the interface, does not match the order in which effects are processed. Nor is there a clear guide on how or why to stack clips.

A NOTE ON COMPUTER HARDWARE

Topaz Video AI benefits from running on powerful computers. Apple silicon systems, with their built-in neural engine will be vastly faster than an Intel system. (In one quick test an Apple silicon system was five times faster than the same task and media run on an Intel iMac.)

As well, Video AI will benefit from higher core counts on Apple silicon systems. However, it does not need excessive amounts of RAM or storage. 32-64 GB of RAM and 1-2 TB of storage – which is what I recommend for editing in general – will be enough.

NOTE: If you plan to do a lot of video processing to improve image quality, you will benefit from the power in a Mac Studio M1 Ultra system.

SUMMARY

Topaz Video AI delivers dramatic results and holds a great deal of promise for the future. It is not cheap, but Topaz Labs provides a free trial where you can experiment with your  footage to see how it enhances your own media. I found this trial very useful in deciding whether to spend the time to review the software in detail.

Topaz Video AI does not work miracles. But it can transform your video into something that looks magical. I recommend it.

EXTRA CREDIT

Here’s a one-page PDF with all my notes and results.

---

Bookmark the permalink.