The Basics of HTTP Live Streaming

Posted on by Larry

The Basics of HTTP Live Streaming

[ This article is a work in process, because I haven’t figured out all the answers yet.
In some cases, I’m still working on the questions. ]

The research for this article started when some of my subscription users started complaining that they could only see a few minutes of one of my longer webinars before they needed to reset their browser. At first, I thought this was caused by bad programming on our part. But, further research made me realize that iOS devices only stream about 10 minutes of continuous video content when they are connected to a cellular data network, then they stop.

Period.

This article explains why. (If you want a more technical explanation, read this Apple Support Note.)

NOTE: If any of the following conditions are true, you can ignore this article:

Understanding Live Streaming isn’t easy, but it isn’t impossible, and this article provides a cookbook you can follow which makes a lot of it fairly simple.

SOME BACKGROUND

There are two types of web video:

Progressive downloads are single files that are stored on a website. When someone wants to view them, the files are downloaded via a browser to a computer or mobile device for viewing. Downloads can be stored locally on the device doing the watching.

Streaming video is fed from a server and watched via a browser without the file actually being transferred from the server to the viewer. YouTube and Vimeo are two excellent examples of streaming video.

In the early days of web video, in the mid-1990’s, creating a high-quality progressive download video required rocket science. Today, though, we have the process nailed. However, as I am discovering, this is not the case for streaming video, which is what this article talks about.

THE ISSUE

The problem is mobile devices. Here are two scenarios involving a 30-minute training video, which is stored in a 120 MB file:

Scenario 1: Watch on a computer. When you fire up your browser to watch the program, the request goes out over your Internet connection and the file starts downloading. Except, you get a phone call about five minutes in, so you put the program on hold. Realizing that the call is going to take a while, you quit the browser and put your computer to sleep.

Scenario 2: Watch on a mobile device. Same program, same five minutes, same phone call.

The problem is that your computer has unlimited access to the Internet, while your phone most often has a data plan that charges by the megabyte transferred.

You only watched 5 minutes, therefore your phone should have downloaded only 20 MB of the program.  But it didn’t. It downloaded the whole thing because it was a progressive download. You were just charged for downloading 100 MB of data that you never used.

Worse, the phone network just carried 100 MB of data that was never used. Now, multiply that by the number of users on your cell phone data network watching video and you begin to see the size of the problem. MASSIVE amounts of data are downloaded and not being used.

To prevent this problem, iOS devices will only stream the first ten minutes of video material, then they stop. If your video is shorter than ten minutes, no problem. Everything is fine. But if your video is longer than ten minutes, the user needs to open the video again in their browser and navigate to where it left off. Annoying.

THE SOLUTION

The solution lies in compressing and segmenting your video into very short chunks, about ten seconds each, then streaming those chunks sequentially to your phone. Because you are not downloading the entire file, when you stop viewing, or simply pause the video, no additional data is transferred. This frees up huge amounts of network bandwidth. When you restart the stream, the chunks pick up where you left off.

All the heavy lifting for this to work happens at the web server, so end-users don’t need to do anything different, provided they have a reasonably current browser. Even better, this new streaming protocol senses the speed of your data connection and sends files that are optimized for that speed. This minimizes stuttering and choppy playback.

This new video protocol is called HTTP Live Streaming and was invented by Apple. Google and Microsoft each have something similar, though with a different name. This streaming protocol is supported by any HTML/5-compliant browser and requires no changes on the part of the end user.

NOTE: It is more accurate to call HTTP Live Streaming a “media streaming communications protocol.” First, you compress your files, then the Live Streaming protocol defines how to segment them and generates a playlist to adaptively choose a stream based on available bandwidth. Apple has publicly documented the protocol so it is available to all browsers and devices.

While all the streaming is handled by the web server supporting this format requires major changes in how we compress videos.

NOTE: For those that want a technical briefing, here’s the WikiPedia link:
http://en.wikipedia.org/wiki/HTTP_Live_Streaming

The good news is that we can use Apple Compressor to create the files required for HTTP Live Streaming and the quality is potentially indistinguishable from a progressive download file. Another good thing is that this protocol can be viewed by both computers and mobile devices so that once you’ve created media files to support this protocol, you can use it for all your viewers.

But there are a lot of changes you need to make to get this to work:

EXECUTIVE SUMMARY

You need to recompress your master file using Apple Compressor for HTTP Live Streaming. This creates the small chunks that get streamed by the server.

You need to transfer the folder containing these files to the website that is hosting your videos, then, following the instructions that Apple provides as part of the compression process, change the links on the webpage to point to a specific file in this compressed files folder.

For videos hosted and streamed from a local website, this works great. For videos hosted on The Cloud, for example, Amazon servers, I haven’t figured out how to get this to work yet.

IMPORTANT NOTES

  1. HTTP Live Streaming does not work with video-only, or audio-only files. You must have audio in your clips, even if it is silence. The compression process fails when compressing video-only clips.
  2. HTTP Live Streaming can also be used for live events which this article does not cover.

COMPRESSION

There are two types of files that need to be created:

While any compression software can create the MP4 files, the only application that I’ve found that also creates the segment files is Apple Compressor 4.

NOTE: In researching this article, I contacted Telestream, Sorenson Media, and Adobe to ask whether their software – Episode, Squeeze, or Adobe Media Encoder – creates these segment files. So far, the only software I’ve found that creates these is Apple’s Compressor.

Here’s how this works.

Start Compressor 4 (Compressor 3 does not support this feature).

Here, for example, I’ve loaded one of my recent webinars into Compressor for compression.

In the Settings tab, open the Apple folder and locate the HTTP Live Streaming folder.

Drag the entire folder on top of the clip you want to compress. Seven different settings are applied to the master clip.

NOTE: These seven versions of your file range from extremely high-quality high-bandwidth files to low-bandwidth audio only. This allows the server to instantly vary the quality of video playback based on the connection speed of your phone. If your viewer suddenly drives into a dead spot, the server switches to low-bandwidth chunks until the data speed improves; at which point the server automatically switches back to the higher-quality version.

Look at the compression settings, they range from audio only – at the top – to high-speed, high-quality broadband – second from the top – then a wide range of other quality and data rate options.

NOTE: You MUST use all seven of these settings, you can not select between them.

EVEN BIGGER NOTE: Do not alter any of these settings in any way. I’ll have more on that in a bit. They are optimized for HTTP Live Streaming. Leave them alone.

This process generates seven different compressed versions. Each file starts with the same name, but adds a numbered extension to differentiate between the different versions.

Next, before you start compression we need to create a destination for all these files. I long ago standardized on storing all my compressed files in a folder I created on a second drive called: “Compressed Files”

Because compressing for Live Streaming creates hundreds of new files – these “chunks” – it is also a good idea to create a new folder specifically for each video you want to compress.

So, in this screen shot, inside my Compressed Files folder, I created a folder for this webinar, called: “Webinar101”. You can call this folder anything you want, however, since this folder will be transferred to your website, be SURE that the folder name does not contain any spaces, and try to use ASCII characters.

Now that you’ve created the special folder for these files, we need to change the destination so Compressor puts these files into that folder.

While you could change each file individually, it is faster to just select the task tile (the one that has the image of the video), then right-click anywhere in the blue area and select Destination > Other.

A CRITICAL FINAL STEP

Before you click the Submit button, you have one more setting to configure. Select the task containing all the streaming compression settings, then go to the Inspector and click the Job Action tab.

Click the “Do Nothing” pop-up menu and select Prepare for HTTP Streaming.

In the Destinations text box, click Choose and select the folder you created to store the compressed files. (In my example, this was Webinar101.)

Then, at the top of the dialog, give the compressed files a name. Remember, no spaces and only use ASCII characters. Click the Save button when you’ve entered both a file name and destination.

The destination will now be used for the compression. The rest of the settings in this window are fine.

NOTE: If you don’t do this step, all the little chunks won’t be created and you’ll need to start all over again. I learned this from personal experience.

Then, as usual, click the Submit button (twice) to start compression.

On my MacPro, creating these seven versions of a one-hour webinar takes about six hours, because there is a LOT of work going on.

AFTER COMPRESSION

When compression is complete, open the folder you created. Inside you’ll find a wide variety of stuff.

You can move all the files ending with “mp4” (highlighted in yellow) out of the folder. They were needed during the process, but are not needed now.

DO NOT MOVE the file ending “m4a” (highlighted in red). This is the audio-only file and is necessary for this system to work.

MOVING TO THE WEB

As the segments are created, a special file is also built, called the “.m3u8” file. This is the master playlist controller that tells the server which files to play.

Here’s what the m3u8 file looks like. And, no, I don’t understand what this is saying either.

Inside each of the scene segment folders are all those 10-second chunks I mentioned earlier. Each of these is a very short movie, fully compressed and ready to stream.

Also in this folder is a ReadMe file that provides very specific directions on how to post this file to your website.

In case you are curious, this is what this looks weblinks look like for my movie. This Read Me file should be given to your webmaster so they know exactly what links to use when posting.

NOTE: For instance, here is the specific HTML code I used to playback a test clip from my website:

UPLOADING

Once all compression is complete, it is time to upload your movie. However, in this case, you upload the ENTIRE FOLDER that contains all these files to where ever you store videos on your website.

BIG NOTE: Just a reminder that this process is NOT necessary if you are using YouTube or other commercial video streaming services for your videos.

AN ALTERNATIVE APPROACH

Apple provides an HTTP Live Streaming template that, potentially, makes this process easier. Just choose File > New Batch from Template (Shift+Command+N), then select HTTP Live Streaming from the Template Chooser.

That creates all the necessary settings in an empty task window. So far, so good.

But the process breaks down when I add a file.  Notice all the red boxes? That indicates where the file names conflict. So, if you are using this template, you need to manually update all the file names to prevent one version erasing the other. For me, this is too much work, which is why I avoided the Template Chooser when I first presented this workflow.

However, once you’ve updated the file names – as I did here by adding numeric extensions – using the template saves you a step when selecting the pop-up menu in the Job Action tab of the Inspector. For me, the extra steps required by the Template Chooser don’t justify the time savings in the Inspector. But, now you know two ways to create these settings.

A FLY IN THE OINTMENT

So far, everything is working great. However, remember that I said not to change any of the compression settings? Well, ah, that causes me a problem. Because I want to add a watermark, maybe scale the image. And none of that is possible.

Unless you understand Job Chaining in Compressor, in which case, this becomes very, very cool.

JOB CHAINING

I’ve written about job chaining before – Compressor 3/4: Job Chaining – but I’ve never really found a good use for it, until now.

What job chaining allows you to do is to take the output of one compression job and use it as the input for a second. For instance, I want to add a watermark to my video first, then compress it for streaming second. Job chaining makes this easy. Here’s how to do it.

Add your video to Compressor as usual.

I’ve created a custom compression setting that converts the file to ProRes 422 (to take advantage of its high quality and minimal compression) and add a watermark, using the watermark filter.

NOTE: Here’s a article that explains how to add a watermark – Compressor 3: Add a Watermark to Video

Next, right-click directly on the blue compression setting you just added to the clip. This displays a contextual menu – select the bottom choice: New Job with Target Output.

This creates a new job that uses the output of the first job. In other words, FIRST I add the watermark (and any other filters I want) THEN the file gets compressed for Live Streaming.

Apply the HTTP Live Streaming settings to the chained clip and proceed as before.

I’ve done several test movies using custom filters for the top job and it all works great.

SUMMARY

HTTP Live Streaming is specifically designed to reduce network bandwidth required for cellular mobile devices to watch videos, while at the same time allowing video quality to fluctuate to compensate for changing data rates across the network.

If your viewers are always watching your videos on a computer with a high-speed internet connection, they will see top-quality images without worrying about data rates or any other technical issues.

However, like all web videos, compression is only half the story. The other half is working with your web team to post the videos. The best place to start, after the files are transferred, is the ReadMe created by Compressor when the job is complete. When I followed its instructions and posted the video to my website, everything worked great. Now, I’m turning my attention to figuring out how to get this to work on Amazon’s servers.

I’ll have more on this as I learn more. As always, I’m interested in your comments.

 

Visit our website to see Final Cut Pro Training & more!


Bookmark the permalink.

30 Responses to The Basics of HTTP Live Streaming

Newer Comments →
  1. Larry:

    GREAT article. Very interested in the next step.

    We have been publishing a TWO HOUR HD video each week since November 2009 on how to convert cars to electric drive. We host this on Amazon’s Cloud using S3. We also do a blog using EC2 and WordPress. We use the JWPLayer to show the videos.

    Big fan of Amazon. I’ve written you on this topic before.

    You might find it interesting to learn. We tried selling commercial advertising on the video series as regular video commercials for several years. The companies offering components for electric vehicles actually tended to be small mom and pop shops and video production was a bridge too far for most and buying advertising kind of a new concept for most.

    We gave it up. Instead we became dealers for the components we use and like on the show and put up a web store selling batteries, motors, controllers etc. about a year ago.

    In May we sold $200,000 gross in components and we are growing about 12% per month. I think this is the new model for monetizing instructional/informational videos. Sell the frying pans while teaching cooking.

    Hanging on every word of your Amazon investigation.

    Jack Rickard
    EVtv.

  2. Nice job of researching and presenting a highly technical explanation in a manner that is much easier to comprehend.
    I seem to recall a guest on your show a few weeks back that offered this compression service for his clients. He touched upon the need for multiple versions of video in order to effectively deliver on multiple devices and platforms.

    Was he one of your resources? Might be a good idea to repurpose his expertise and invite him back for a more in depth discussion.

  3. Robert says:

    You should check into using the Wowza servers on Amazon’s service. I believe that Wowza will do the segmenting for you, all you have to provide is a H.264 file. Also I believe that there are AddOns that will give you the various broadband versions.

  4. Wowza sounds like a quick way to get rid of $995, to do what I’m hoping Larry shows us how to do anyway. Better, you can let them markup ALL your Amazon charges. Good work if you can get it.

    Amazon already has streaming. I just need to learn how to use it and the work flow with Final Cut Pro X.

    Jack Rickard

  5. Jeff says:

    Hi Larry.

    I was testing your demo video to see how it performs on different web browsers. It worked ok on Safari and iCab on Mac, the same went for my iPhone.

    But when I tested on Firefox, Opera and Chrome on Mac (all latest versions installed), they didn´t seem to work. Actually on Firefox I had an error message that says that “video format or MIME type is not supported”.

    So, what is missing on my browsers? Is necessary to have something special installed to see the videos?

    Thanks…

    • Jeff:

      Thanks for the feedback. At this point Google, who owns FireFox, does not support H.264/MPEG-4 videos within FireFox because they don’t want to pay royalties on the codec. Instead, they’ve created a far less competent codec called “WebM.” In order for our videos to support FireFox, we would need to create an entirely different version just using WebM. It is far easier to change browsers than to create two different encodings of our training.

      Larry

  6. Thad Macnamara says:

    Great research and writing! One question: Is the limitation on file size only on phones, or does it apply to iPads, eg a sales force in the field?

  7. Amazon has a new product – Elastic Transcoder or something like that. https://console.aws.amazon.com/elastictranscoder/home?region=us-east-1#getting-started:

    I kind of like the idea of OFFLOADING this little chore to Amazon and not spending a lot of time on my machine doing it myself. Since we host on Amazon anyway, it would appear they can handle the transcode chores to make files for various platforms. How about a tutorial how to do THAT.

    Jack Rickard

    • Robert says:

      Yes Wowza does cost a bunch of money but if you are using the Amazon instance I think it is a lot cheaper. As a university we do get a discount of the purchase of the server so that is why we use it. But I like the fact that I only need one H.264 video to serve out Flash, HTML 5, HTTP Streaming(Flash and Apple) and Silverlight without having to encode all of the different versions.

  8. Peter Lonsdale says:

    Great article Larry. It begs the question: How were we to know about this if it wasn’t for you?!

    Thanks,

    Peter

  9. Great article – thanks for digging into this subject. This explains how some of my end users have been experiencing the cutoff after 10 mins of playback.

    Have you figured out how Vimeo is doing their back end? Nice because I upload one 5000 KBPS file, and they transcode. Wondering what encoder/workflow they use? Might be part of their mojo….

  10. Matt says:

    Larry – Great article. We’ve taken a stab at the topic as well, and we cover some points that you don’t, such as browser and device support. For example, we look at how Android supports http live streaming. Feel free to read through our article and excerpt anything you find of value. http://features.encoding.com/http-live-streaming-hls/

    Many thanks!

Newer Comments →

Leave a Reply

Your email address will not be published. Required fields are marked *

Larry Recommends

Final Cut Pro X 10.4

FCPX Complete

Edit smarter with Larry’s brand-new webinars, all available in our store.

Access over 1,900 on-demand video editing courses. Become a member of our Video Training Library today!

JOIN NOW

Subscribe to Larry's FREE weekly newsletter and save 10%
on your first purchase.