It’s been 2 months since I’ve posted about Zencoder. You may be glad to know that the project is still alive and is nearing completion. It has moved a little slower than we initially expected (we had hoped to finish in Nov/Dec), but the end is in sight. So if you are interested in a complete video transcoding solution for your web application, request more info at zencoder.com. And if you’ve inquired in the past, and you are interested in a demo, let us know: we’re starting our online demos this week.

But on to the sneak peak.

Scaling

Zencoder scales both horizontally and vertically to a nearly arbitrary limit. You can scale vertically (obviously) by increasing the speed of your machines: the faster the CPUs, the faster the transcoding, and any Linux or OS X machine should work. And you can scale horizontally by increasing your number of servers, as high as your licensing allows.

You can scale the same way on EC2, by moving from small instances to large instances, or by increasing your number of instances.

What’s important is that Zencoder works the same whether you run it with 2 servers or with 200 (or more). The system will happily distribute your jobs across however many servers you have. (Note that it will do this by distributing each job to a different server, not one job across many servers – the latter approach can finish each individual job more quickly, but the Zencoder approach should have a somewhat higher overall throughput, since it is a bit more efficient. And it is significantly simpler, of course.)

This means that Zencoder can grow with your application. If you underestimate your traffic, or your site grows exponentially, just add another server or fire up a new EC2 instance.

Finally, you don’t even need to restart Zencoder to add additional servers, so expansion is almost completely seamless. If you’re running on dedicated hardware, just configure Zencoder on a new server and start it up: it will begin working immediately. On EC2, things are even easier: click the big green “Launch new worker” button, and you’ll have a new server up and running in about 3 minutes.

Reliability

On the reliability front, Zencoder is built with three goals in mind.

First, errors should be few and far between. This goal is obvious, and is shared with most software (except for some software coming out of Redmond, WA). And Zencoder is a reliable system in this respect. The code is reasonably compact, and it is well tested (both by unit tests and by humans).

Second, if any (or all) pieces become unavailable, your website should continue to work. In other words, if a user uploads a video at your site, and Zencoder is unavailable for some reason, your website should continue functioning as-is. Similarly, if your site is down, and Zencoder finishes a job, your website should receive the job as soon as it comes back up. As a result, we have designed the system such that you can pull the plug on any piece – or multiple pieces – and as soon as everything is back online, the system picks up right where it left off.

Third, no job should ever get lost in the system. This is accomplished through integrity checks – between the queue and each transcoding node, and between the Zencoder system and your application. You can even wipe the Zencoder queue, and if your application is still waiting on any jobs, they will be automatically recreated.

The result is that Zencoder is almost a self-healing system. It isn’t perfect (yet), but it is robust, reliable, and scalable.

So how does Zencoder do its video transcoding? What codecs and formats does it support? And what about the critical issue of codec licensing?

Licensing

Let’s start with the question of licensing. Codecs are licensed in three main ways.

1. Commercial software (On2, Nellymoser)
2. Patents and standards requiring licenses (MPEG-2, AAC, MP3)
3. Free or unenforced formats and codecs (Ogg, Matroska, AVI)

If you’re going to transcode and distribute video, category 3 is easy. Category 1 is straightforward, and these commercial codecs work just fine with Zencoder.

Category 2 poses the real problem. In my estimation, three of the most important video codecs – MPEG-4, H.264, and MPEG-2 – and three of the most important audio codecs – AAC, MP3, and AMR – all require licensing direct from their patent holders and/or licensing authorities. Those six invaluable codecs are represented by four separate licensing bodies (representing hundreds of patent holders), each of which has separate terms, prices, minimum fees, etc. And when you move to the second tier of codecs, things get even more complicated.

Zencoder takes care of this for you. With Zencoder, you receive licenses to these and other critical codecs. We’re still working out the specifics, but when Zencoder launches, we plan on including licenses to 10-20 patented codecs and formats, covering all of the major ones and several in the second or third tier.

Compatibility

Zencoder can handle every major video codec, audio codec, and container format. And we do this legally, taking care of most licensing issues for you.

Here is a partial list of formats and codecs that we support.

• H.264
• FLV
• MP4
• 3GP
• VP6 (with commercial license)
• MP3
• AAC
• MPEG-4 Video, XviD
• MPEG-2
• AVI
• Ogg, Theora, Vorbis
• Nellymoser (with commercial license)
• And 100+ more

The goal is to decode everything that you can throw at Zencoder, and to encode everything that you have a good reason to encode.

Slantwise has been working on a video processing product called Zencoder for the last several months. Zencoder is a distributed multimedia processing system, written in Ruby, that is powerful, flexible, and scalable. It also integrates easily with any application that needs video transcoding. The product itself is in its final stages of testing, and will launch soon, so head over to http://zencoder.tv and fill out the contact form if you want more info.

In this first “sneak peek” post, we’ll look at Zencoder from a high level. Where does it run? How does it integrate with your application?

Two Hosting Options

Zencoder is a server, or collection of servers, that you host. The first server is the “mind” of Zencoder, which queues jobs and handles all communication with your application. The other servers are “workers” that do the actual video transcoding.

You can host Zencoder on your dedicated hardware or on Amazon EC2. Both work equally well, but we think that the EC2 approach is especially interesting, and Zencoder does some cool things with EC2. (More on that in a few weeks.) But you can mix-and-match EC2 instances and dedicated servers, so you don’t have to choose. You could use dedicated hardware for most of the transcoding work, and use EC2 for overflow work. Or use EC2 for most of your work, but one ultra-fast dedicated machine for high-priority jobs.

Easy Integration

Zencoder integrates via REST web services, so it is completely technology-agnostic. Your application can be written in Java, Ruby, PHP, .NET, Python, Erlang, Ada, or Assembly, as long as it can communicate via HTTP. It can be a web application, server software, or anything else that can send and receive HTTP requests.

To submit a job to Zencoder, you just need to send a POST request to your Zencoder server with the basic details of the job: input file, job id, and task/recipe. Zencoder will then do its work and send a request back to your application with the results of the job: status, output file attributes, output file location.

So your application only needs to do two things to integrate with Zencoder:

• Send a HTTP request to your Zencoder server when you want to trigger a new transcoding job
• Handle a HTTP request from Zencoder with the results of a job

We also provide a Ruby on Rails plugin that handles these things for you; so if you’re running Rails, this work is done for you. You just have to install the plugin and configure a few things, and your application is ready to go. Eventually, we plan on providing similar integration classes for Java and PHP. But even if you aren’t using one of these technologies, the integration work is pretty simple on your end, and we’ll help you through it.

RVideo is now available as a Ruby gem. Install with:

sudo gem install rvideo

(RVideo depends on other tools for transcoding, like ffmpeg, so you’ll probably need to install a few other things as well. See the Documentation for a little more detail.)

I’ve tagged this release as 0.9.0. It is still beta-quality code, so test thoroughly. If you run into problems, let me know – I’ll be deploying RVideo to a live app soon, so I want to squash any bugs as much as you do. :)

What is it?

RVideo is a Ruby library for video/audio transcoding. It provides a clean Ruby interface to transcoding tools like ffmpeg, and can easily be extended to support more tools. At this point, only ffmpeg and flvtool2 are supported, but more will follow.

1
2
3

transcoder.execute(recipe, {:input_file => "/path/to/input.mp4",
      :output_file => "/path/to/output.flv", :resolution => "640x360"})

Details

To inspect a file, initialize an RVideo file inspector object. See the documentation for details.

A few examples:

1
2
3
4
5
6
7
8
9
10

  file = RVideo::Inspector.new(:file => "#{APP_ROOT}/files/input.mp4")

  file = RVideo::Inspector.new(:raw_response => ffmpeg_inspection_response)

  file = RVideo::Inspector.new(:file => "#{APP_ROOT}/files/input.mp4",
                                :ffmpeg_binary => "#{APP_ROOT}/bin/ffmpeg")

  file.fps        # => "29.97"
  file.duration   # => "00:05:23.4"

To transcode a video, initialize a Transcoder object.

1
2

  transcoder = RVideo::Transcoder.new

Then pass a command and valid options to the execute method

1
2
3
4
5
6
7
8
9
10

  recipe = "ffmpeg -i $input_file$ -ar 22050 -ab 64 -f flv -r 29.97 -s"
  recipe += " $resolution$ -y $output_file$"
  recipe += "\nflvtool2 -U $output_file$"
  begin
    transcoder.execute(recipe, {:input_file => "/path/to/input.mp4",
      :output_file => "/path/to/output.flv", :resolution => "640x360"})
  rescue TranscoderError => e
    puts "Unable to transcode file: #{e.class} - #{e.message}"
  end

If the job succeeds, you can access the metadata of the input and output files with:

1
2
3

  transcoder.original     # RVideo::Inspector object
  transcoder.processed    # RVideo::Inspector object

If the transcoding succeeds, the file may still have problems. RVideo will populate an errors array if the duration of the processed video differs from the duration of the original video, or if the processed file is unreadable.

RVideo supports any transcoding tool with a command-line interface; adding a new tool just means writing a class for the tool that subclasses RVideo::AbstractTool. It also means that you need to use common sense to avoid attacks. For example: don’t run RVideo as a privileged user. Control your input recipes, and don’t accept user-submitted recipes. (RVideo is pretty well protected from these problems; you can’t execute a command that isn’t identified by a transcoder tool class, so `rm -rf *` won’t work. But it pays to be cautious.)

More info

See the RVideo Google Code site for more info, including links to Documentation and a Google discussion group. Use these to file tickets, discuss, etc. (The SVN repository is currently at Rubyforge, but I may move it to Google Code.)

Contribute

I would love help on this project. If you want to help out, there are a few things you can do.

• Use, test, and submit bugs/patches

• We need a RVideo::Tools::Mencoder class to add mencoder support. (Someone has started on this, so let me know if you’re interested in helping and I’ll put you in touch.)

• Other tool classes would be great – On2, mp4box, Quicktime (?), etc.

• Eventually, it would be great to (optionally) use the processing feedback provided by ffmpeg etc. to get real-time progress updates (e.g. 20% complete, 40% complete, 90% complete). (More info)

• Submit other fixes, features, optimizations, and refactorings

It’s been a while since I posted about RVideo. Consider this a pre-release announcement.

RVideo is a Ruby gem that makes video and audio transcoding a bit easier. The gem wraps various video transcoding tools – ffmpeg and flvtool2 upon release. But it is extensible, so that other tools (like mencoder) can be added as needed. Transcoder instructions are specified by passing a recipe to RVideo, along with custom values.

For example:

1
2
3
4
5
6
7

recipe = "ffmpeg -i $input_file$ -r 29.97 -vcodec xvid -s $resolution$ $output_file"

transcoder = RVideo::Transcoder.new
transcoder.execute(recipe, {:input_file => "original.mov", 
                            :output_file => "processed.mp4", 
                            :resolution => "640x360"})

The resulting transcoder object will include information about the job, including metadata for the output file, etc. The execute method raises a variety of exceptions when it doesn’t work (e.g. input file not found, unsupported formats, could not save the output file), so you’ll want to run #execute in a begin/end block and handle each exception as needed.

I’ll go into more detail in a later post, so look for more updates soon. Expect the gem to launch sometime in October.

Starting today, with Flash Player 9 Update 3 Beta 2, Flash Player will support H.264 video and AAC audio. This is great news for sites hosting online video. It is the equivalent of Microsoft announcing that IE7 would now render web pages exactly like Firefox. In other words, once this becomes reality, online video sites will really only need to worry about one format.

UPDATE: One of the engineers on Flash Player has posted a detailed account of the new Flash Player. The implementation looks well conceived. As expected (below), you will still need to worry about MPEG licensing issues with H.264.

H.264 is a mpeg-4 video codec that provides the best video compression widely available today. That means that H.264 allows better quality video that other codecs, when comparing files of the same size. AAC is a mpeg-4 audio codec (not an Apple codec!) that is a bit better than mp3 and has better licensing terms. (You have to pay royalties when distributing mp3-encoded content, but you don’t with AAC.) See my earlier post on formats and codecs for more info.

This move is great for online video, and bad for On2. Until now, On2’s VP6 codec was by far the best codec available in the Flash Player. VP6 and H.264 are both good codecs, though H.264 has the edge in my experience. The bigger issue is cost. On2’s Flix Engine software is commercial and isn’t cheap, while free H.264 encoders are available (x264). Expect to see less VP6 content over the next year.

A few caveats, though.

First, H.264 encoders may be free, but H.264 is not (strictly speaking). If you make money from H.264 in one way or another, you’ll need to pay royalties. This is true whether you sell a H.264-encoded content, a H.264 encoder or decoder, or make money through other means (subscriptions or advertising). Fortunately, there are minimums – for instance, if you have less than 100,000 subscribers, you don’t need to pay royalties. See the MPEG-LA FAQ for details. This alone may make On2’s one-time cost an attractive option for some businesses.

There is an outside chance that Adobe may have a licensing arrangement that takes care of this, which would be great for content creators, but don’t count on it.

Second, this Flash 9 update will take time to proliferate. It took 9-12 months for Flash 9 to reach 90% market penetration. So unless you’re willing to force your users to upgrade, don’t drop VP6 or H.263 support today.

Third, H.264-encoded video is more compressed than other video, and so it takes more processor power to watch. Most computers these days are plenty fast for H.264, but some users may see their CPUs spike while watching H.264 video. The good news is that early reports say that the new Flash Player will make use of multiple cores on multi-core processors, though this may be a Zompire Dracularius.

Fourth, H.264 has five levels plus sub-levels. Each level allows for better quality and better compression. What level will Flash Player support? The lower levels (1-1.3) are still better than most competitors, but they don’t make use of the codec’s full potential. The Quicktime format, for instance, only supports level 1.3 (if I remember correctly) – not too shabby, but not as good as the MP4 format. It would be great to see Flash Player support at least level 2 H.264. (More info on levels)

Caveats aside, this sounds like great news for video content creators. Keep watching the wires as more details unfold. And watch for updates to RVideo and Spinoza (our forthcoming video transcoding service), which will support Flash/H.264.

Here are the slides from my RailsConf presentation today on building a video transcoder workflow. I’ve taken out the video; if you’re really interested in seeing it, send me an email. I’ve also created a version without any images at all in order to save space.

Video transcoding slides, with images (4.5M)

Video transcoding slides, no images (207K)

I’d also love your feedback from the talk if you were there.

(This is part 3 in a series on video transcoding. Parts 1 and 2 covered video formats and codecs, and transcoding tools, respectively.)

Ruby on Rails is a great technology that gives you a lot for free. You get ORM, MVC, templating, HTML helpers, URL rewriting, database schema management, a development web server, prototype, script.aculo.us, deployment automation, a unit test framework, etc. etc. etc. But what you get falls into two basic categories: tools to handle a HTTP request, and tools to make the application easier to manage. In other words, Rails helps you out when something is either triggered directly by an HTTP request, or when it is triggered directly by a developer. This is because the Rails environment fires up and tears down with every HTTP request.

Unfortunately, a video transcoding system can’t be run in either of these ways.

First, video transcoding can’t be done within the space of an HTTP request. Transcoding jobs can take several minutes (or hours), and you can’t expect your users to wait that long. If a request takes longer than a few seconds, a significant number of users will cancel the request. Beyond this, since Rails isn’t thread-safe, every video transcoding job would cause a Mongrel instance to block.

Second, video transcoding could theoretically be triggered manually by an operator, but this solution isn’t responsive or scalable. What happens when your application takes off, and you have dozens or hundreds of files per day?

In my next few posts, I will examine three approaches to asynchronous processing, and will discuss three ways to host your asynchronous system. The use case will be video transcoding, but the approaches themselves could be used for just about any time-consuming action: expensive calculations, large PDF creation, data processing, etc.

1. Database polling with a daemon – This is the simplest approach, and it isn’t a bad one. All you need is one or more transcoding servers with access to your main Rails application database, and maybe some shared disk space. The transcoding servers query the database for unassigned jobs, and the first server to select the job takes it. Then, when the work is done, the transcoding servers update the database with the new state.

2. Message queue polling – Instead of polling a database, this solution uses a message queue (like Amazon SQS). The base application passes a message to the message queue announcing the new job, and the transcoders poll the queue looking for jobs. This approach is a little more complex than the first approach, but is a little more secure, robust, and scalable.

3. Reactor-pattern system – This solution doesn’t involve polling at all. Instead, a controller accepts jobs and assigns them directly to individual workers. This approach is the most complex of the three, but will satisfy the purist’s desire to avoid polling.

I’ll discuss each of these approaches in their own articles in detail. There certainly are other approaches; if you have a favorite that you don’t see here, post a comment.

Finally, I’ll look at three ways to host these approaches: local to your main application, in its own dedicated hosting environment, and using Amazon Web Services.

Stay tuned for the next post. I’m at RailsConf right now, but I’ll try to get something posted within the next few days.

(Update: I gave a talk on video transcoding at MinneBar, and the slides are now available online.)

Video on the web is a hot topic these days. Hundreds of people want to be the next YouTube, and thousands more are making use of user-submitted in some other way.

Unfortunately, putting video on the web is a total mystery to some developers, and may seem deceptively simple to others. The former don’t know where to start, and couldn’t tell a codec from a container, or ffmpeg from mencoder. The latter know the fundamentals, which don’t look too tough, but would have trouble putting together a scalable, robust, production-worthy system.

I’m not an expert, but I have been a part of several projects which do video transcoding. So in my next several posts, I will outline video transcoding for the web from a variety of angles.

• Part 1: Formats and codecs – an overview of the various video codecs, audio codecs, and container currently available.

• Part 2: Tools – what are the free (e.g. ffmpeg) and commercial (e.g. On2 Flix) tools that can be used to build a transcoding system?

• Web application integration – once I’ve settled on codecs and tools, how do I put everything together into a working system?

• Legal and licensing issues – some prominent codecs and formats are commercial; some are completely free; and some of the most popular occupy an ugly middle ground that require royalties.

• RVideo – Slantwise is working on a Ruby-based video transcoding library called RVideo. I’ll outline its capabilities and the various design decisions we made (and are making) along the way.

A few parts of this series will have a Ruby on Rails focus, but most of the information is generic. So if Ruby isn’t your thing, you may still want to stick around.