How to display a transcript in sync with a media file?

My understanding of SMIL is that it plays a block of text for a certain duration, and that's it. I want all the text to be visible at all times, and for the user to be able to start playback at any arbitrary point in either the media or transcript. Imagine a two-column design, with the transcript in one column and the media in the other; the text is persistent, but what section is highlighted changes in time with the media. Is this possible with SMIL? I don't have any experience with SMIL, just did some quick research.

i'm not a front-end web developer, and i don't know how precise javascript timers are but the following scheme may be a potential solution. use a javascript function starts the media clip and a timer, everytime the timer fires it highlights a particluar portion of the page displaying the transcript. the transcript would need to be broken out into container elements that contain the timestamp as the container id. when the timer fires, it gets the appropriate transcript container and sets the background color, or some other css style to produce the desired effect.

That would work for the single case of playing the media from the beginning, but it wouldn't allow users to start from an arbitrary point in the transcript (e.g., line 483), and have the media play from that point (e.g., 10 minutes in). Is there a way to have Javascript look at the sentence's time stamp and then tell the media file to play from that point? Or at worst, extract what point in media file you are currently at, and throw that to a function which then highlights the appropriate sentence?

Ah, I understand better. I was confused by the "scroll" part — I had that mentally eclipsing the "entire transcript" part, since I thought of a window of text scrolling the transcript by. Unless this is a very short media file, the transcript window will be much larger than the media window. SMIL can put text in one spot and video (or audio) in another spot, and keep them synchronized. You can start in the middle, go back, etc. — still synchronized. But you're right, that wouldn't necessarily make the text navigable. So, are you thinking of this page, which shows a very large text transcript in its entirety, and the media window off to the right (for an example), slides down the page along with a changing, green-highlit sentence, with its baseline roughly colinear with the vertical middle of the media file? I would rate that as being significantly more difficult than having an immobile media window, with the text scrolling alongside and not the entire transcript viewable. You might want to have the transcript confined to a window, with scrollbars. I realize that's not exactly what you asked, but you'd have your work cut out for you otherwise. I'm starting to like askmehow's answer better than mine. When I did text captioning of a movie, I did line repeats in parts of it. Frame 34 Line A Line B Line C Frame 35 Line B Line C Line D If you're going with Javascript, you'll have to choose your format carefully to make sure that the play can support the functions you need. It looks like you'd need an onclick or something similar for each line in the transcript. That would have to then trigger some Javascript function which takes the id of the transcript block which called it and either lookup against times, or somehow decode the id into a time (id="03020145" meaning 3 hours, 2 minutes, and 1.45 seconds in), then try to manipulate the player and send it to that time. That part is probably fairly easy in Javascript compared to, say, grabbing the controls on the media player, fast-forwarding a bit, and then expecting that to change how the transcript is displayed. There, you would want a repeating Javascript event which would look at the player, try to extract where it is, and then synchronize the display of the transcript. Furthermore, that repeating event would have to know not to try to interfere when someone has just jumped ahead or back in the transcript, probably through setting a flag, good for a second or two. What a sticky situation. If you do go the Javascript route, your most critical part will be making sure the player supports the functions you must have. That looks like the decision point to me.

When I say "the entire text should be visible", all I mean is that it is persistent and able to be seen at any time. That rules out SMIL with its durations (right?), but I really did have a basic "text in the left frame, media in the right frame" idea. So, yes, sorry for the confusion. The entire transcript would not literally be completely visible; it just wouldn't appear at some point and disappear at another like in closed captioning or slide presentations. I had a doh! moment when I read "make sure the player supports the functions", and then went to check the functions QuickTime supports. Lo and behold, there are http://developer.apple.com/documentation/QuickTime/Conceptual/QTScripting_JavaScript/bQTScripting_JavaScri_Document/QuickTimeandJavaScri.html#//apple_ref/doc/uid/TP40001526-CH001-DontLinkElementID_17 functions. My thinking now is: Each sentence would have a button next to it. When clicked, a function would read the start time for that sentence (pulled from the time stamp), and set SetTime() as that time. Conversely, going to a point in the media would call a function that gets the time from GetTime() and then highlights the sentence with a time stamp range that covers that time. The text could scroll using anchor tags, and Mootools or similar could be used to make that scrolling smooth. Now my question is, "is this the best implementation?" Flash looks to have similar time functions that can be made externally available to Javascript, so that's another possible route. Maybe this is more easily done server-side somehow (PHP, or Perl, etc.)? HTML 5 is going to have new media elements, but, while cool and simple, wouldn't work for any user not running cutting-edge beta software. http://people.w3.org/~dom/archives/2009/02/the-beauty-of-htmlmediaelement/ has an interesting HTML 5 post that I'll file away for 2012 when IE 16 finally supports HTML 5. Thanks a bunch askmehow and adipocere. I'm not quite at a solution yet, but ideas are flowing.