timesheets.js :: Media Annotations

SMIL Timesheets can easily be used to add HTML subtitles to a video stream. This has been applied to the Mozilla Firefox 4 beta "first run" page:

As an open source project and community,
we need your feedback to make the next generation browser.

Meet some of the team that has been hard at work
on the Firefox 4 project.

I work on Web Standards like HTML 5, File APIs and IndexedDB.

We’re going to make the best and most secure browser out there!

We work on testing Firefox 4 beta.
We’d like to know what you think, we’d like your feedback.

You’re going to notice a feedback button
on the upper-right corner of your browser…

…that you can use to tell us
what you like and don’t like about Firefox 4.

Let me know what you think.

Im working on the Firefox 4 beta program.

I work on content security policy for Firefox 4.

We work on crash bugs.

I’m David and I work on developer tools for Firefox 4,
and I’d really like to know what you think.

We need as much feedback as possible.

I work on the Firefox 4 Sync servers.

Firefox Sync lets you take your Firefox experience with you
wherever you go.

And we’re really looking forward to your feedback.

I work on JetPack, which is a project
to make it easier to build Add-Ons and to make Add-Ons easier to use.

I worked on the new Add-ons manager in Firefox 4.

We’d really like to know your feedback
about what you love and hate about Firefox 4, so let us know.

Please help us and make Firefox better!

We need your feedback.

View this demo in full page.

HTML Markup

Each subtitle is a plain HTML paragraph:

<div smil:timeContainer="excl" smil:timeAction="display" smil:dur="1:09">

  <video smil:syncMaster="true" smil:timeAction="none" controls="true" preload="auto">
    <source type="video/webm" src="http://videos-cdn.mozilla.net/firefox4beta/Firefox_4_beta.webm" />
    <source type="video/ogg"  src="http://videos-cdn.mozilla.net/firefox4beta/Firefox_4_beta.ogv" />
    <source type="video/mp4"  src="http://videos-cdn.mozilla.net/firefox4beta/Firefox_4_beta.mp4" />
      Your browser does not support the HTML5 &lt;video&gt; tag,
      please upgrade to a modern browser to see this demo.
  </video>

  <p smil:begin="0:00.00">
    As an open source project and community, <br />
    <a href="http://mozilla.com/firefox/beta/feedback/">we need your feedback</a>
    to make the <a href="http://mozilla.com/firefox/beta/">next generation browser</a>.
  </p>
  <p smil:begin="0:04.93" smil:end="0:10"> … </p>
  <p smil:begin="0:11.14"> … </p>
  <p smil:begin="0:15.57"> … </p>
  <p smil:begin="0:18.32"> … </p>
  <p smil:begin="0:23.00"> … </p>
  <p smil:begin="0:27.00"> … </p>
  <p smil:begin="0:30.04"> … </p>
  <p smil:begin="0:31.00"> … </p>
  <p smil:begin="0:33.00"> … </p>
  <p smil:begin="0:35.63"> … </p>
  <p smil:begin="0:37.05"> … </p>
  <p smil:begin="0:41.01"> … </p>
  <p smil:begin="0:42.64"> … </p>
  <p smil:begin="0:44.63"> … </p>
  <p smil:begin="0:48.06"> … </p>
  <p smil:begin="0:49.85"> … </p>
  <p smil:begin="0:55.32"> … </p>
  <p smil:begin="0:57.57"> … </p>
  <p smil:begin="1:01.74 smil:end="1:04"> … </p>
  <p smil:begin="1:05.00"> … </p>

</div>

You can easily add links and images into your subtitiles, and style the whole thing with CSS. As the video is fully transcripted in HTML, it’s already searchable and indexable.

Timing: syncMaster

standard way

The main trick here is to use the <video> element as the time container’s sync master (see the syncMaster attribute): the time container uses the media player time instead of an internal timer.

<video smil:syncMaster="true" smil:timeAction="none" […] />

A <excl> time container is handy because we don’t have to specify any duration or end time for every paragraph: as each subtitle is hidden as soon as the next one is shown, we only have to specify an end time to get a pause between two subtitles. However, the syncMaster attribute can be used within <par> and <seq> time containers as well.

We have to specify timeAction="none" to exclude the <video> element from the <excl> time container, and keep it visible all the time. I don’t really understand how an audio or video element could have a non-null timeAction attribute… but that’s how I’ve understood the syncMaster spec.

non-standard alternative

A simpler but non-standard approach could be to use a custom attribute directly in the time container node, to select an <audio|video> element as the main timer:

<div smil:timeContainer="excl" smil:mediaSync="video" […] />

The mediaSync attribute value is a CSS selector, and the time container will use the result of querySelector() to get the related element. The syntax is a bit simpler (especially here, since we have only one <video> element in the web page), and this allows to put the <audio|video> element outside of the time container. This attribute is used in the next example.

HTML Markup

Timing: syncMaster

standard way

non-standard alternative

CSS Markup