So, I decided to take a look at the #io2011 live keynote stream with captioning on YouTube. (Note that the link may be dead after today…)
The captions default off, so the viewer will have to click the CC icon in the lower right to turn them on.
The captions come to the user along with the live stream, so if you wish to look back at previous captions you are limited to the point at which you came into the video. You can control up and down movement with two arrows on the lower right, but there is no indication of where you are relative to the available captions. Each speaker change is indicated by a white box around red text.
According to Gigaom, Google is taking the output of the live CART provided for the event and streaming it to the YouTube channel. This story is supported by the fact that the operator backspaced and corrected some text while I was watching.
I doubt it is a technical limitation of the API from YouTube, but it really irks me to see captions in ALL-CAPS unless it indicates yelling. The DCMP Captioning Key is very clear on the subject:
“Mixed case characters are preferred for readability. However, capital letters are used for an individual word or a single phrase to denote empasis or shouting.”
In this case, I’m sure it was the CART provider who chose or was instructed to use all capital letters.
The sync with the video was very good – in fact, in a few places the captioning was about 3 words ahead of the sound, which probably means that the speaker was working from a script that the CART provider had access to. At any rate, there doesn’t seem to be a noticeable delay. Interestingly, pausing the live video feed does not pause the captioning.
The interface offers 3 font size choices, and the standard YouTube auto-translate language choices. I’m waiting to hear if the API will support multiple language feeds in realtime.
One more thing: the direct link and embed codes do not appear to support the realtime captions at this time – they are missing the “Live right now” text and the CC button.
This is very likely because the caption stream is coming as its own frame on the html page, hosted from io-captions.appspot.com (Google’s App Engine host).
The word is that the App Engine code will be released as open-source, so we should be able to find it here in the next few weeks.