(This is part 3 in a series on video transcoding. Parts 1 and 2 covered video formats and codecs, and transcoding tools, respectively.)
Ruby on Rails is a great technology that gives you a lot for free. You get ORM, MVC, templating, HTML helpers, URL rewriting, database schema management, a development web server, prototype, script.aculo.us, deployment automation, a unit test framework, etc. etc. etc. But what you get falls into two basic categories: tools to handle a HTTP request, and tools to make the application easier to manage. In other words, Rails helps you out when something is either triggered directly by an HTTP request, or when it is triggered directly by a developer. This is because the Rails environment fires up and tears down with every HTTP request.
Unfortunately, a video transcoding system can’t be run in either of these ways.
First, video transcoding can’t be done within the space of an HTTP request. Transcoding jobs can take several minutes (or hours), and you can’t expect your users to wait that long. If a request takes longer than a few seconds, a significant number of users will cancel the request. Beyond this, since Rails isn’t thread-safe, every video transcoding job would cause a Mongrel instance to block.
Second, video transcoding could theoretically be triggered manually by an operator, but this solution isn’t responsive or scalable. What happens when your application takes off, and you have dozens or hundreds of files per day?
In my next few posts, I will examine three approaches to asynchronous processing, and will discuss three ways to host your asynchronous system. The use case will be video transcoding, but the approaches themselves could be used for just about any time-consuming action: expensive calculations, large PDF creation, data processing, etc.
1. Database polling with a daemon – This is the simplest approach, and it isn’t a bad one. All you need is one or more transcoding servers with access to your main Rails application database, and maybe some shared disk space. The transcoding servers query the database for unassigned jobs, and the first server to select the job takes it. Then, when the work is done, the transcoding servers update the database with the new state.
2. Message queue polling – Instead of polling a database, this solution uses a message queue (like Amazon SQS). The base application passes a message to the message queue announcing the new job, and the transcoders poll the queue looking for jobs. This approach is a little more complex than the first approach, but is a little more secure, robust, and scalable.
3. Reactor-pattern system – This solution doesn’t involve polling at all. Instead, a controller accepts jobs and assigns them directly to individual workers. This approach is the most complex of the three, but will satisfy the purist’s desire to avoid polling.
I’ll discuss each of these approaches in their own articles in detail. There certainly are other approaches; if you have a favorite that you don’t see here, post a comment.
Finally, I’ll look at three ways to host these approaches: local to your main application, in its own dedicated hosting environment, and using Amazon Web Services.
Stay tuned for the next post. I’m at RailsConf right now, but I’ll try to get something posted within the next few days.
Nice articles series. Now i’m waiting for the 4th part to see what solution will be implemented. I’m not an expert but looks like “Reactor-pattern system” would be not so difficult to put in place using ffmpeg and backgroundrb. One background server ready to accept batch jobs that could encode using ffmpeg … ;-)
Can’t wait for the next article.
Thanks !
Use the O/S Luke.
Just batch the job in the background using the appropriate batch control command on the O/S.
Get your long running job into another process and get back to rendering an answer back to your user.
A simple system where you randomly ‘ssh’ into your worker server and run a simple shell ‘batch’ command would work just fine in the majority of cases. If you want to get sophisticated get your worker servers to log their name and current run load in the database on a regular basis and do your random pick from that.
Unix in particular is very good at batch processing. It is what it was designed for. Make it earn its keep.