(This is Part 1 in a multi-part series on video transcoding and the web. For the rest of the series, take a look at the first post. It was edited on April 27 to include the Ogg format and codecs.)

A typical video file is made up of one or more video streams and one or more audio streams. Each video or audio stream has a codec, and the file itself has a container format.

A codec (compressor/decompressor) is a software encoding that allows a stream to be compressed for storage or transmission, and decoded to a raw or readable format. Most, but not all, codecs are lossy. Examples of familiar audio codecs include MP3 (technically, MPEG-1 audio format 3) and AAC (technically, MPEG-4 part 3). Familiar video codecs include DivX and Xvid (both implementations of MPEG-4 ASP).

A container format (sometimes called a wrapper format) is a file format that can contain video and audio streams (along with other types of streams, like text). Common container formats include MOV (Quicktime), and AVI.

In this post, I will highlight a few interesting codecs and formats. The discussion will be far from comprehensive – there other good codecs and formats. My goal is to discuss the best ones, but there may be others that I’m not aware of. (Leave a comment if you want to suggest one!)

Formats and Codecs

Video codecs

• MPEG-2 is commonly used for DVDs and ATSC broadcasts. MPEG-2 doesn’t have particularly efficient compression, so unless you’re creating a playable DVD or broadcasting over the air, you probably won’t create MPEG-2 video files.

• MPEG-4 ASP is a much more interesting format, and it commonly goes by the names XviD or DivX. These codecs provide reasonably good compression, wide compatibility, and a reasonable CPU load for encoding/decoding. DivX is owned by DivX, Inc., while XviD is GNU licensed.

• H.264 is also known as MPEG-4 AVC. H.264 is a more advanced video codec than ASP/XviD (hence the AVC acronym), and provides better compression. How much better? Don’t quote me on this, but IIRC it provides about 35-50% better quality for the same file size. Unfortunately, it has two disadvantages when compared to MPEG-4 ASP. First, H.264 decoders are a little less common than XviD decoders, so compatibility isn’t quite as widespread. Second, encoding and decoding H.264 is more processor intensive than XviD. That said, H.264 is probably the best quality video codec on the market today, or at least is tied for this honor.

• H.263 is, unfortunately, not just an incremental step down from H.264. It is about 8 years older, and provides far worse quality. This is unfortunate, because H.263 is used in at least two prominant places. First, most Flash video is encoded with H.263, including just about everything on YouTube and MySpace. Second, H.263 is often used by cell phones to play or capture video. Despite the low quality, H.263 makes perfect sense if you want to put video on the web and don’t want to use the codec that will be described next – VP6.

• VP6 is a proprietary codec developed by On2 Technologies. VP6 has two major advantages. First, it offers great compression, comparable to H.264. Second, VP6 is used by Flash 8, so it is a great candidate for video that will be played back on the web. The downside is that VP6 can’t be encoded for free; it requires commerical software, like Flash 8, or the On2 Flix Engine. But this may actually not be a downside, as I’ll discuss in a future article on licensing and royalties.

• WMV describes several Microsoft codecs. Confusingly, WMV 7 is also known as WMV1, WMV 8 is also known as WMV2, and WMV 9 is also known as WMV3. (The latter name refers to the codec’s FourCC code, and I think the former name refers a corresponding version of Windows Media Player.) These codecs are not bad, and with Flip4Mac, they are no longer restricted to Windows machines.

• Theora is a truly open-source video codec based on On2’s VP3 codec, as a part of the Ogg project. It provides comparable quality to MPEG-4 ASP (e.g. XviD), and it is BSD-licensed. Did I mention that it is open-source (unlike any of the MPEG, Microsoft, or On2 codecs)? I’ll discuss this in more detail in a future post.

Audio codecs

• MP3 is probably the most famous codec of all time. It is a part of the MPEG-1 standard (MPEG-1 audio layer 3). This codec provides adequate quality at a bitrate of about 128kbps. However, distribution of MP3-encoded content is not free, as will be discussed in a later post.

• For this reason, AAC is a better audio codec than MP3 for most uses. AAC-encoded content is free to distribute, which is one of the reasons why iTunes chose AAC for its content. AAC also provides better compression than MP3 – 96kbps AAC is generally considered equal to 128kbps MP3.

• Vorbis is an Ogg audio codec that is similar in quality to MP3, or perhaps slightly better. Like Theora, it is truly free (unlike AAC or MP3).

• FLAC and Apple Lossless are two lossless codecs that provide about 50% size savings compared to uncompressed audio. FLAC is part of the Ogg project, and I’m not sure who developed Apple Lossless.

• AC-3 is the Dolby Digital audio codec that can store 5.1 channel audio.

Formats

• MOV is the Quicktime container format. It is a reasonably good format, with pretty widespread compatibility (every computer with iTunes installed also has Quicktime) and decent codec support. One major advantage of Quicktime is that MOV files can be played back in many browsers (though not all).

• MP4 (MPEG-4 part 14) is based on Quicktime and is very similar, but it supports some codecs and encoding options not supported by MOV (like advanced H.264 profiles). Most media players can play MP4, though not in a web browser.

• AVI is an old format (1992!) that is somewhat outdated today. It has pretty good codec support, though it isn’t able to handle some modern codecs very efficiently, including XviD and H.264. Ask yourself why you want to use AVI instead of a more modern format.

• ASF is a proprietary Microsoft format. If you’re primarily using a Microsoft codec and Microsoft players, ASF may be a good option.

• OGG is a free, open-source format released under the BSD-license that typically is used with Vorbis, Theora, FLAC, and other Ogg codecs. However, the Ogg container format can wrap other formats (like MP3 and various MPEG-4 codecs).

• FLV is the Flash Video format, and offers by far the best web browser compatibility – likely in the 99% range for folks with up-to-date computers. Of course, there are several versions of Flash Player, and so if you’re on the bleeding edge, the number may be lower. Unfortunately, FLV only supports two codecs: H.263 (bad) and On2 VP6 (great but commercial). That said, if the web is your medium, FLV is the obvious choice.

So what should I use?

That depends on what you want to do, of course. If you want to put video on the web, there is one easy option and two good options.

• Easy web video: FLV/H.263. Low quality, high compatibility, easy to produce. A fine option when your input quality is low, or quality doesn’t matter, but otherwise not recommended.

• Good web video (1): MOV/H.264. Great quality, decent compatibility, somewhat complex to produce. H.264 lends itself to a really high degree of optimization, and MOV is picky about its H.264. I guess Apple wants people to create their H.264 videos using Quicktime Pro, so other H.264 encoders have trouble with certain settings. Quicktime Pro creates beautiful H.264 videos, but this is a Mac-only solution, and integrating with the Quicktime API is more difficult than integrating with ffmpeg for server-side solutions.

• Good web video (2): FLV/VP6. Great quality, high compatibility, not free. On2 Flix Engine is fairly affordable for businesses, but cost prohibitive for individuals. This is an attractive option for many businesses, with no downside that I can see except for the up-front cost.

If you aren’t producing video for the web, there are many more options. Ogg/Theora is a great option if you want good quality and royalty-free distribution. MP4/H.264 and MP4/XviD are also good options, though not truly free. WMV 9 (WMV3) could be valid for Windows applications. There are more options and fewer constraints for non-web video, and so it is hard to make generic recommendations.

In my next post, I’ll discuss tools used in video transcoding. Stay tuned!