The Basics of HTTP Live Streaming [u]

UPDATE – Oct. 11, 2020

Publit.io is cloud-based Digital Asset Management software. It handles storage, processing and delivery of images, videos & rich-media files for modern web apps. With it’s URL-based API, you can resize, format, transform, watermark your media files and more. Recently, they released an article explains HLS Streaming, a necessary codec for mobile media. This is a good backgrounder. Here’s the link.

This provides a good follow-on to the information in my article, below.

UPDATE – Dec. 2, 2022

The Porch published “Live Stream Setup and Tech Guide for Beginners” – which covers getting gear together for live streaming at the consumer level. Their article includes different tips on how to take advantage of live streaming, including how to set up your studio at home and a comprehensive overview of cameras and mics. Here’s the link.

UPDATE – April 29, 2023

Updated Apple Compressor screen shots to version 4.6.3 and made text corrections for current practices in the Compressor section. The rest of this article was not touched.

---

The research for this article started when some of my subscription users started complaining that they could only see a few minutes of one of my longer webinars before they needed to reset their browser. At first, I thought this was caused by bad programming on our part. But, further research made me realize that iOS devices only stream about 10 minutes of continuous video content when they are connected to a cellular data network, then they stop.

Period.

This article explains why. (If you want a more technical explanation, read this Apple Support Note.)

NOTE: If any of the following conditions are true, you can ignore this article:

• You stream all your videos off YouTube, Vimeo, or other commercial streaming service
• Your videos run less than 10 minutes
• No one watches your videos on an iOS mobile device (apparently, Android devices don’t have this limitation).

Understanding Live Streaming isn’t easy, but it isn’t impossible, and this article provides a cookbook you can follow which makes a lot of it fairly simple.

SOME BACKGROUND

There are two types of web video:

• Progressive downloads
• Streaming video

Progressive downloads are single files that are stored on a website. When someone wants to view them, the files are downloaded via a browser to a computer or mobile device for viewing. Downloads can be stored locally on the device doing the watching.

Streaming video is fed from a server and watched via a browser without the file actually being transferred from the server to the viewer. YouTube and Vimeo are two excellent examples of streaming video.

In the early days of web video, in the mid-1990’s, creating a high-quality progressive download video required rocket science. Today, though, we have the process nailed. However, as I am discovering, this is not the case for streaming video, which is what this article talks about.

THE ISSUE

The problem is mobile devices. Here are two scenarios involving a 30-minute training video, which is stored in a 120 MB file:

Scenario 1: Watch on a computer. When you fire up your browser to watch the program, the request goes out over your Internet connection and the file starts downloading. Except, you get a phone call about five minutes in, so you put the program on hold. Realizing that the call is going to take a while, you quit the browser and put your computer to sleep.

Scenario 2: Watch on a mobile device. Same program, same five minutes, same phone call.

• Total time spent watching: 5 minutes.
• Total file size watched: 20 MB
• Total file size downloaded: 120 MB

The problem is that your computer has unlimited access to the Internet, while your phone most often has a data plan that charges by the megabyte transferred.

You only watched 5 minutes, therefore your phone should have downloaded only 20 MB of the program.  But it didn’t. It downloaded the whole thing because it was a progressive download. You were just charged for downloading 100 MB of data that you never used.

Worse, the phone network just carried 100 MB of data that was never used. Now, multiply that by the number of users on your cell phone data network watching video and you begin to see the size of the problem. MASSIVE amounts of data are downloaded and not being used.

To prevent this problem, iOS devices will only stream the first ten minutes of video material, then they stop. If your video is shorter than ten minutes, no problem. Everything is fine. But if your video is longer than ten minutes, the user needs to open the video again in their browser and navigate to where it left off. Annoying.

THE SOLUTION

The solution lies in compressing and segmenting your video into very short chunks, about ten seconds each, then streaming those chunks sequentially to your phone. Because you are not downloading the entire file, when you stop viewing, or simply pause the video, no additional data is transferred. This frees up huge amounts of network bandwidth. When you restart the stream, the chunks pick up where you left off.

All the heavy lifting for this to work happens at the web server, so end-users don’t need to do anything different, provided they have a reasonably current browser. Even better, this new streaming protocol senses the speed of your data connection and sends files that are optimized for that speed. This minimizes stuttering and choppy playback.

This new video protocol is called HTTP Live Streaming and was invented by Apple. Google and Microsoft each have something similar, though with a different name. This streaming protocol is supported by any HTML/5-compliant browser and requires no changes on the part of the end user.

NOTE: It is more accurate to call HTTP Live Streaming a “media streaming communications protocol.” First, you compress your files, then the Live Streaming protocol defines how to segment them and generates a playlist to adaptively choose a stream based on available bandwidth. Apple has publicly documented the protocol so it is available to all browsers and devices.

While all the streaming is handled by the web server supporting this format requires major changes in how we compress videos.

NOTE: For those that want a technical briefing, here’s the WikiPedia link:
http://en.wikipedia.org/wiki/HTTP_Live_Streaming

The good news is that we can use Apple Compressor to create the files required for HTTP Live Streaming and the quality is potentially indistinguishable from a progressive download file. Another good thing is that this protocol can be viewed by both computers and mobile devices so that once you’ve created media files to support this protocol, you can use it for all your viewers.

But there are a lot of changes you need to make to get this to work:

• Because Live Streaming requires multiple versions of the same file, total streaming file sizes will be 3-4 times larger than a single file, requiring more website storage
• You can not retro-fit this to existing compressed files. You need to recompress from high-quality master files. (This is another great case for keeping good archives of your master files.)
• You are no longer uploading a single file, you are uploading a folder containing hundreds, even thousands, of smaller files.
• The process of compression is trickier, as you’ll see in a minute.
• Linking to the video files stored on your website is fairly simple. However, if you are using a CDN or Cloud storage, this becomes much trickier.

EXECUTIVE SUMMARY

You need to recompress your master file using Apple Compressor for HTTP Live Streaming. This creates the small chunks that get streamed by the server.

You need to transfer the folder containing these files to the website that is hosting your videos, then, following the instructions that Apple provides as part of the compression process, change the links on the webpage to point to a specific file in this compressed files folder.

For videos hosted and streamed from a local website, this works great. For videos hosted on The Cloud, for example, Amazon servers, I haven’t figured out how to get this to work yet.

IMPORTANT NOTES

1. HTTP Live Streaming does not work with video-only, or audio-only files. You must have audio in your clips, even if it is silence. The compression process fails when compressing video-only clips.
2. HTTP Live Streaming can also be used for live events which this article does not cover.

COMPRESSION

There are two types of files that need to be created:

• The compressed MPEG-4 files
• The segment, or chunk, files that are derived from the compressed file

While any compression software can create the MP4 files, the only application that I’ve found that also creates the segment files – called “HTTP Live Streaming” (HLS) – is Apple Compressor 4.

Here’s how this works in Apple Compressor 4.6.3.

Start Compressor 4 (Compressor 3 does not support this HLS).

Here, for example, I’ve loaded one of my recent webinars into Compressor for compression.

In the Settings tab, open the BUILT-IN folder and locate the HTTP Live Streaming folder.

NOTE: There are two versions. The one indicated by the red arrow (above) compresses using H.264. The one displaying “(HEVC)” compresses using HEVC. H.264 is appropriate for most situations because of its wide compatibility. Only use HEVC compression if you are instructed to do so. HEVC will also take longer to compress.

Drag the entire folder on top of the name of clip you want to compress (red arrow above). Seven different settings are applied to the master clip.

NOTE: These seven versions of your file range from extremely high-quality, large frame size high-bandwidth files to small frame size, low-bandwidth audio only. This allows the server to instantly vary the quality of video playback based on the connection speed of your phone. If your viewer suddenly drives into a dead spot, the server switches to low-bandwidth chunks until the data speed improves; at which point the server automatically switches back to the higher-quality version. Each chunk runs ten seconds.

The compression settings range from audio only – at the top – to high-speed, high-quality broadband – second from the top – then into other image resolutions and data rate options.

NOTE: You MUST use all seven of these settings, you can not select between them.

EVEN BIGGER NOTE: Do not alter any of these settings in any way. I’ll have more on that in a bit. They are specifically optimized for HTTP Live Streaming.

This process generates seven different compressed versions of your movie in 10 second chunks. Each version starts with the same name, but adds a specific numbered extension to differentiate between the different versions.

Each of these seven formats creates a folder containing hundreds of chunks which need to be stored in a master folder.

NOTE: Yup, you guessed it. DO NOT change these names!

So, the next step is to create that master folder. I long ago standardized on storing these in a folder I named the same as the webinar. You can name this folder anything and store it anywhere. However this folder will be what you ultimately transfer up to the web, so make sure it doesn’t contain any spaces and try to use exclusively ASCII characters.

Create one master folder for each source video.

The files displayed in Compressor are all work files. Compressor needs to create these in order to create the HLS folder. But it doesn’t need these once the HLS folder is complete. So I store these to my customary compression location, but delete them as soon as the HLS folder is processed.

A CRITICAL FINAL STEP

Before you click the Start Batch button, you have one more setting to configure. You need to tell Compressor the name and location of the HLS master folder.

In Compressor, click the Job tab at the top of the Inspector, then scroll all the way to the bottom to the Action section. All these defaults are fine, except click Choose to display the standard Mac file picker. Navigate to and open the HLS master folder.

With the HLS master folder open, enter a file name at the top which becomes the master file name for all the thousands of chunks you are about to create. For my webinars, I name the master file name the same as the folder.

Remember, no spaces and only use ASCII characters. Click the Save button when you’ve entered a file name and selected the destination folder.

This file name and destination will now be used for the job. The rest of the settings in this section are fine.

NOTE: If you don’t complete this step, all the little chunks won’t be created and you’ll need to start all over again. I learned this from personal experience.

Then, as usual, click the Start Batch button to start compression. On my M1 Pro MacBook Pro, it takes a couple of hours to create all these files. So be patient.

AFTER COMPRESSION

Once compression is complete, if you were to open the HLS master folder (which you don’t need to do unless you are curious) you would see a folder for each compression format.

Inside each of those subfolders, you will see hundreds of clips – each 10 seconds long – spanning the entire duration of your video.

MOVING TO THE WEB

Also, shown in the screen shot above, a special file is added to the master folder, called the “.m3u8” file. This is the playlist controller that tells the server which files to play and where they are located.

(Click to see larger image.)

Here’s what the m3u8 file looks like. Notice, on the right side, how the image size varies allowing your movie to play even when cell bandwidth drops.

Also in this folder is a ReadMe file that provides very specific directions on how to post this file to your website. For example:

In case you are curious, this is what this looks weblinks look like for my movie. This Read Me file should be given to your webmaster so they know exactly what links to use when posting.

UPLOADING

Once all compression is complete, it is time to upload the HLS version of your movie. However, in this case, you upload the ENTIRE FOLDER that contains all these files to wherever you store videos on your website.

BIG NOTE: Just a reminder that this process is NOT necessary if you are using YouTube or other commercial video streaming services for your videos. Only for video streamed from a local website.

A FLY IN THE OINTMENT

So far, everything is working great. However, remember that I said not to change any of the compression settings? Well, ah, that causes me a problem. Because I want to add a watermark, maybe scale the image. And none of that is possible given the default settings in Compressor.

In the past, I would use Job Chaining, which allows the output of the first job to become the input of the second job. The problem is that while this is a great idea, it isn’t reliable. I’ve struggled with Job Chaining for years and finally given up.

Instead, I create an intermediate file of my movie with all watermarks added; plus any other effects. I save this as a ProRes 4444 file, which is, essentially, lossless. Then, I use this intermediate file to create the HLS files.

The good news is this works perfectly. The bad news is that Compressor takes a LONG time to create that intermediate file. I don’t know why. But it takes as long, or longer, to create an intermediate ProRes 4444 file as it does to create the HLS chunks.

So, I start the job and do something else for a few hours until it is complete.

SUMMARY

HTTP Live Streaming is specifically designed to reduce network bandwidth required for users of cellular mobile devices to watch videos on their phones, while at the same time allowing video quality to fluctuate to compensate for changing data rates across the network.

If your viewers watch your videos on a computer with a high-speed internet connection, they will see top-quality images without worrying about data rates or any other technical issues.

However, like all web videos, compression is only half the story. The other half is working with your web team to post the videos. The best place to start, after the files are transferred, is the ReadMe created by Compressor when the job is complete. When I followed its instructions and posted the video to my website, everything worked great.

As always, I’m interested in your comments.

---

Bookmark the permalink.