Starting today, with Flash Player 9 Update 3 Beta 2, Flash Player will support H.264 video and AAC audio. This is great news for sites hosting online video. It is the equivalent of Microsoft announcing that IE7 would now render web pages exactly like Firefox. In other words, once this becomes reality, online video sites will really only need to worry about one format.

UPDATE: One of the engineers on Flash Player has posted a detailed account of the new Flash Player. The implementation looks well conceived. As expected (below), you will still need to worry about MPEG licensing issues with H.264.

H.264 is a mpeg-4 video codec that provides the best video compression widely available today. That means that H.264 allows better quality video that other codecs, when comparing files of the same size. AAC is a mpeg-4 audio codec (not an Apple codec!) that is a bit better than mp3 and has better licensing terms. (You have to pay royalties when distributing mp3-encoded content, but you don’t with AAC.) See my earlier post on formats and codecs for more info.

This move is great for online video, and bad for On2. Until now, On2’s VP6 codec was by far the best codec available in the Flash Player. VP6 and H.264 are both good codecs, though H.264 has the edge in my experience. The bigger issue is cost. On2’s Flix Engine software is commercial and isn’t cheap, while free H.264 encoders are available (x264). Expect to see less VP6 content over the next year.

A few caveats, though.

First, H.264 encoders may be free, but H.264 is not (strictly speaking). If you make money from H.264 in one way or another, you’ll need to pay royalties. This is true whether you sell a H.264-encoded content, a H.264 encoder or decoder, or make money through other means (subscriptions or advertising). Fortunately, there are minimums – for instance, if you have less than 100,000 subscribers, you don’t need to pay royalties. See the MPEG-LA FAQ for details. This alone may make On2’s one-time cost an attractive option for some businesses.

There is an outside chance that Adobe may have a licensing arrangement that takes care of this, which would be great for content creators, but don’t count on it.

Second, this Flash 9 update will take time to proliferate. It took 9-12 months for Flash 9 to reach 90% market penetration. So unless you’re willing to force your users to upgrade, don’t drop VP6 or H.263 support today.

Third, H.264-encoded video is more compressed than other video, and so it takes more processor power to watch. Most computers these days are plenty fast for H.264, but some users may see their CPUs spike while watching H.264 video. The good news is that early reports say that the new Flash Player will make use of multiple cores on multi-core processors, though this may be a Zompire Dracularius.

Fourth, H.264 has five levels plus sub-levels. Each level allows for better quality and better compression. What level will Flash Player support? The lower levels (1-1.3) are still better than most competitors, but they don’t make use of the codec’s full potential. The Quicktime format, for instance, only supports level 1.3 (if I remember correctly) – not too shabby, but not as good as the MP4 format. It would be great to see Flash Player support at least level 2 H.264. (More info on levels)

Caveats aside, this sounds like great news for video content creators. Keep watching the wires as more details unfold. And watch for updates to RVideo and Spinoza (our forthcoming video transcoding service), which will support Flash/H.264.

Juggernaut is a clever “server push” solution for Rails that’ll work in nearly all browsers.

With Juggernaut, your server can push updates into an already rendered page. Push a useful alternative to Ajax style polling, especially when your application is event-based with an update cycle that is irregularly/unpredictably timed.

Chat is the most common example of using push, though it’s easy to imagine other collaborative applications where this would work.

I’ve spent the evening pouring over the code; the Juggernaut site and README contain useful documentation to get you started; I’ll use this post to describe how Juggernaut actually works behind the scene.

Juggernaut Internals

Juggernaut’s implementation is clever, works in better than ninety five percent of all browsers, and is both small-enough and straightforward-enough to be approachable/understandable to most Rails programmers.

Juggernaut is made up of three main elements: (1) in-browser/client side elements, (2) a stand alone push server and (3) Rails elements to tie everything together.

In-Browser/Client Side Elements (media/juggernaut.as ; media/juggernaut.js)

Juggernaut works by embedding a Flash object into pages that receive push messages. It accomplishes this by establishing an long-lived outbound TCP connection to the oush server, which lives on a separate port than the web server.

Unlike a browser initiated HTTP connection that is closed after the page renders, the Flash initiated TCP connection remains open, waiting for messages. (Long-lived HTTP connections don’t work well with Rails because of Rails’ oft-discussed single-threaded implementation; this approach side-steps that potential problem.)

The push server sends message down the already established connection. Messages are strings, often a rendered RJS partial, containing JavaScript that is eval’d in the page. So, borrowing an example from the README, you could use Prototype’s Insertion object you could update the “chat_data” div with the following message string:

new Insertion.Top('chat_data', '<li>#{input_data}</li>');

The Flash component is tiny – the ActionScript is only 38 lines!—and really straight forward: it simply opens the TCP connection (using ActionScript’s XMLSocket object to connect to host/port that are passed in as arguments to in the the with host and port params passed in in the enclosing HTML) and connects an event handler to hand off messages that it receives to a JavaScript method in the browser that runs eval on the message string.

Modern browsers, designed for HTTP/1.1, limit the number of connections to a host. IE, for example, won’t allow more than 2 simultaneous connections. By using keep-alive to allow a single connection to handle multiple requests browsers are able to avoid the time/overhead of connection setup and takedown. Using the Flash VM to handle the connections is handy in this regard because it doesn’t use one of your previous few connections.

Stand Alone Push Server (media/push_server)

Conceptually, the push server is straightforward: it receives messages (usually from the web server) and forwards it to the clients.

Messages are generated based on events that happen “elsewhere” (from the perspective of the client receiving the message). For example: we’re both in a chat room; you enter a message that is delivered to the web server using Ajax. The web server receives the message from you, which should ultimately be displayed in my browser, and forwards it to the push server for delivery to me in my browser.

The push server receives messages serialized to JSON over a standard TCP socket (that it opens for listening at startup) and not via DRB. Presumably this is to more easily allow non-Ruby processes to send messages, though there are implications with session that I’ll discuss below in the Broadcast/Channels section.

Rails Components (lib/juggernaut.rb ; lib/juggernaut_helper.rb)

Juggernaut’s Rails components tie everything together.

Form helpers instantiate the Flash object with the appropriate host/port parameters and include the necessary JavaScripts to evaluate events. Using the helper is a one-line non-event:

  <%= listen_to_juggernaut_channels :chat_channel %>

A couple of Juggernaut class methods make relaying messages easy. For example this code…

Juggernaut.send_data(data, channels])

...opens a new socket to the push server, passes the message on, and closes/cleans-up.

Broadcast/Channels

When creating messages, you choose whether to send them to a single user (via the Juggernaut.send_to method) or to broadcast them (via the Juggernaut.send_data method).

Single user message recipients are identified by a unique id. Unless otherwise specified, this is set to session.session_id. If you plan to create messages outside of the Rails framework, it’s probably handier to use another unique ID can be easily associated with the user; using Rails internal session identifier outside of Rails seems more complicated than necessary. The ID column from the user table seems like a natural choice.

For broadcast, Juggernaut uses a “channel” metaphor. The HTML helpers pass the channels that the page/user subscribe to the Flash object, which in turn sends them to the push server when it establishes its connection.

Sessions and Security

The push server uses a “shared secret” string to restrict unauthorized users from pushing messages out to end users.

Client access control is optional. If turned on, it works at a connection level (i.e., you can either connect to the push server or not). It’s implemented in a clever way: when the Flash player connects to the push server it includes its Rails session.session_id when it sets up the connection. The push server then makes an HTTP request to an authenticated page within the Rails app using this session id. If it succeeds (and gets a HTTP response code of 200) then the user is considered authenticated.

Some TODOs

From looking at the code, it appears that if the client’s connection to the push server is lost that it won’t automatically reconnect. There is a “num_tries” configuration directive for setting up the number of times to try to reconnect after a failure, but it’s unused in the Flash. Similarly, the “num_secs” directive, which sets the number of seconds to try between delivery attemts, is unused.

Conclusion

All in all, it’s a very elegant solution to an interesting problem.