I’ve been working a great deal on Flash video related projects the past several years, and just this week I ran across and solved a puzzling issue for a client where the duration value being injected into a very large .flv file kept coming up far short of its real world value. I talked it through on Stefan Richters great mailinglist FLASHMEDIA and thanks to Asa Whillock at Adobe on the FMS team, came to a conclusion, so I’m consolidating it all here for the benefit of others.

The video in question was over 8 hours in length, 8 hours 26 minutes and 26 seconds to be exact. It was encoded using the On2 VP6 CODEC and clocked in at 1.4GB in size. Not the typical average length and size for an FLV by any means, but I thought it certainly should be supported as it plays fine when streamed, or progressively downloaded, but the duration value was wrong. The other metadata values were correct, just duration.

Unfortunately this file was to be hosted on bitgravity’s network using their advanced-progressive delivery – a customized high performance, scalable version of the pseudo-streaming technique made popular by Stefan Richter and others as described here. So this compounded the problem as the bitgravity player requires the duration to be correct in order to calculate the proper position of the scrub/seek bar and to keep track of the offset of the beginning of the video when jumping/seeking which causes new requests for the video to be sent out, where a server side process looks at the video and its embedded keyframe metadata as well as the duration, recalculates the proper header, and begins sending data from the approximate requested keyframe position onward. So having an incorrect duration really fouled up playback in this specific instance of the bitgravity player.

Each tool I tried using to inject/view the metadata, including: flvtool2, flvtool++ and jflvtool all inserted incorrect values – and the results varied from tool to tool. I was going to try Burak Kalaycis FLVMDI metadata injector tool, but as luck would have it, Burak is also on Stefans mailing list, and he chimed in and mentioned that FLVMDI has some other limitations on the file size of videos that it can process (currently limited to 700MB), and that I would probably hit that issue first. So I didn’t run it through FLVMDI – but Burak did also confirm that his tool also suffers from the same problem, where the duration value would be incorrect, and if he gets a chance to fix it, he would update us on that.

The key issue does indeed relate to the length or duration of the .FLV itself and the FLV container format and specs for how the duration metadata value is stored. For videos that are under approximately 4.5 hours, everything is fine. However, once a video exceeds a little over 4 hours and 39 minutes, this is when the duration values appear incorrect with these tools, and all of them report back a value of around 16777.990 seconds in duration, a little more than half of the 8 hours 26 minutes of the video I was working with, which should have reported 30386 seconds.

I assumed that the problem might have something to do with the extra long duration, so thats when I began doing some research on the maximum length of FLV’s. I determined thanks to some help from Asa Whillock that the problem lies in how the duration value is stored in the .FLV container and the publicly available specs that were available at the time to third party authors that developed these applications. Originally the spec (prior to FMS 1.0) only designated a 3 byte integer type (UI24) to store the duration in milliseconds, or just enough to store a value corresponding to (256*256*256) 16.7 million or 16,777,990 milliseconds or 16777.990 seconds or 4 hours, 39 minutes, 37 seconds. So any video with a duration greater than that that can’t be represented with the single 3 byte sequence as the original FLV spec listed. Asa pointed out that in the recent spec they added an extended 7 byte UI8 tag named “TimeStampExtended”. This tag is to immediately follow the original 3 byte UI24 “TimeStamp” tag, and then both values are used to store the time. In this arrangement the original “TimeStamp” tag acts as the lower 24bits of the duration in milliseconds, and the “TimeStampExtended” acts as the upper 8 bits of the duration in milliseconds. This results in a 4 byte representation when combined, (256*256*256*256) so a maximum of 4.2 billion or 4,294,967,296 milliseconds or 4294967.296 seconds or 1193 hours, 2 minutes and 47 seconds or nearly 50 days straight worth of video. This would certainly allow just about any videos duration to be stored properly in the metadata.

So it looks like the solution to the the problem is to make sure the duration is stored using not only the “TimeStamp” tag but also the “TimeStampExtended” tag in order to store the full value of the duration for any video that exceeds 4 hours, 39 minutes and 37 seconds, otherwise the duration will be truncated to only the first 3 byte representation. To me this looks like a fairly easy fix (although the devil is always in the details). Burak is already aware of the issue, and I have gone ahead and emailed the info to Chris Norman of Facebook who wrote the c++ flvtool++ implementation, as well as Norman Timmler over in Germany who wrote the flvtool2 in Ruby. I still need to write the author of the java based jflvtool. Fernando Florez of Funciton Communications mentioned a colleague of his has a Python based metadata injector that also suffers from this issue, but they are working on a patch and will try to release that as open source at some point. So I have hope we will have a third party tool capable of handling videos of this length in the short term.

If it helps anyone else working on similar tools, the exact reference to the “TimeStampExtended” tag is on page 9 of Adobes Video File Format Specification Version 9

I haven’t yet had a chance to see if any commercial products like Sorenson Squeeze, On2 Flix, and the Flash CS3 Video Encoder correctly store the duration for videos over 4:39:37 in length – but will try to validate that and update this article once I’ve had a chance. I’m making an assumption the Adobe’s encoder does support it properly.

Thanks to Asa Whillock of Adobe for input and validation on this, as well as fellow Adobe Community Experts Fabio Sonnati and Stefan Richter and his invaluable FLASHMEDIA mailing list and to Burak Kalayci and Fernando Florez for their input and thoughts. Thanks guys!

UPDATE: 09/02/08 – I found another nice FLV metadata injector: YAMDI – http://yamdi.sourceforge.net/ but it also appears to suffer from the same bug.

Its been several days and I still have yet to hear back from Bitgravity about their FLVMDU tool that their CTO Barrett Lyon posted about in the comments. I emailed him directly right after the comment was posted, as well as through support as he noted and got a ticket #1678, but have yet to receive a response from Barrett or their support staff. Chalking it up to the holiday weekend.

UPDATE: 09/03/08 –I spent some extra time tonight trying to locate a copy of the Bitgravity FLVMDU tool and with a little careful url guessing, was successful and found this link to the tool – it also includes the source under a GNU license which is nice. This version is dated 06-18-2008 as version 1.0.0. Unfortunately after running it through the same series of tests on a video with a duration greater than 4 hours 39 minutes and 37 seconds, it also fails to write the proper duration on such a long video. So it appears that bitgravity needs to also update their utilities to support the “TimeStampExtended” tag. There was a note to contact a “pclarke” at bitgravity in regards to bugs – so I’ve dropped them a note and pointed them at this article. Hopefully they will be able to fix things as of right now every FLV metadata injector tool I’ve tested suffers the same problem.

UPDATE: 09/03/08 12:37AM – I found one more nice flv metadata injector and dumper called flvmeta available here http://code.google.com/p/flvmeta/. The source, and pre-made binaries for windows and Mac OS X are available, but it too appears to suffer from the bug I’ve been exploring since it doesn’t appear to support the “TimeStampExtended” tag either. Looks like everyone completely missed this in the spec, or just didn’t implement it for some reason.

UPDATE: 09/03/08 12:24PM – I heard back from Perry Clarke at Bitgravity in response to my email to him about the bug/lack of support for “TimeStampExtended” in their FLVMDU tool. Perry noted, “I haven’t reproduced the issue you report though I can believe it’s a problem. I’ll look into it when I have time.” So if you rely on FLVMDU when working with bitgravity, you’ll probably need to pester Perry or their support team to implement proper support in FLVMDU if your video is longer than 4 hours 39 minutes 37 seconds, especially if you are using their advanced progressive http delivery for videos – as that service requires the keyframe/metadata hinting and proper metadata duration tag otherwise it wont work properly. If they don’t update the tool, you’ll need to deliver your video in smaller chunks (less than 4 hours 39 minutes 37 seconds). I’d recommend smaller chunks anyway given the memory usage.

UPDATE: 09/08/08 7:40PM – Last week I heard back from Dan Weatherford of Facebook and the primary author of flvtool++ He was very responsive in working with me to get in the proper support for the “TimeStampExtended” tag for proper durations on long videos. We made a number of test files from a variety of all the popular FLV exporters and transcoders and Dan was able to implement the full spec in the latest version of flvtool++ 1.1.2 available here: http://mirror.facebook.com/facebook/flvtool++/ The funny thing was that we found was that not a single tool, none of the flv metadata injectors out there, nor any of the FLV encoders we tested put in the proper duration in “TimeStampeExtended” format. Not ffmpeg, not Squeeze 5, not On2 Flix, not even Adobe Flash CS3 Video Encoder (standalone or Quicktime Exporter) and even a few upcoming apps currently in beta that I have access too – none of them inserted the proper duration in “TimeStampExtended” format for videos over 4 hours 39 minutes and 37 seconds. I thought this was especially odd for the Adobe apps – so it just looks like everyone totally overlooked this. Thanks so much to Dan Weatherford for his time in implementing proper support into flvtool++. Its currently the only tool I know of that properly supports “TimeStampExtended”

PRINT THIS ENTRY

18 Responses »

  1. Safari 3.1.2 Safari 3.1.2 Mac OS X  10.5.4 Mac OS X 10.5.4
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_4; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.20.1

    Hey there,

    We have a killer open source tool called FLVMDU which will do some pretty amazing things with metadata injection.

    If you email support@bitgravity.com we’ll be happy to help you with any issues.

    Cheers,

    Barrett Lyon
    CTO, BitGravity

  2. Safari 3.1.2 Safari 3.1.2 Mac OS X  10.5.4 Mac OS X 10.5.4
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_4; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.20.1

    Hi Barrett,

    Thanks for the comment – much appreciated.

    I’d love to get a chance to play with the FLVMDU tool – it wasn’t clear from your comment if your FLVMDU tool properly supports the “TimeStampExtended” tag for videos over 4 hours, 39 minutes and 37 seconds that I described above, but I’ll assume it does given your comment.

    If so, that would be great for two reasons:

    1. My client is using your services and would have a quick solution until they can properly split the video into smaller chunks, as having a video this size delivered progressively, even using the advanced-progressive really requires a huge amount of memory on the clients end.

    2. It would be great to point others to your tool as far as I know it would be the only one capable of supporting the full extended FLV metadata spec.

    Do you have a URL where we could grab a copy of FLVMDU?

    I’ve dropped a note to your support staff as well at the address you noted and look forward to a response.

    Thanks -Rob

    [See my updates in the bottom portion of the post above for more info on this. -Rob]

  3. WordPress 2.6.2 WordPress 2.6.2
    Incutio XML-RPC -- WordPress/2.6.2

    […] except mvflvServer its compabitle other playback clints like vlc and mplayer, works with flv files over 4 hours 39 min 37 seconds long and avoides sending the client keyframe metdata for quick streaming […]

  4. Firefox 3.0.1 Firefox 3.0.1 GNU/Linux GNU/Linux
    Mozilla/5.0 (X11; U; Linux i686; fr; rv:1.9.0.1) Gecko/2008072820 Firefox/3.0.1

    Hi,

    I’m the author of FLVmeta, and I’ve just stumbled upon this post.
    I’m glad to hear that you gave a try to my tool, and I am therefore interested in implementing the feature you’ve described here.
    I suggest we discuss about it in the open FLVmeta mailing list http://groups.google.com/group/flvmeta-discuss, or directly by email with me.

    To be honest, it’s actually the first time that I’ve heard about this tag, I didn’t know the FLV spec had been updated πŸ™‚

    The bugfix should be pretty trivial, but it would be nice if you could provide me with sample FLV files (huge files, I guess) so I can test it properly and validate it with you.

    Regards,
    Marc

  5. Safari 3.1.2 Safari 3.1.2 Mac OS X  10.4.11 Mac OS X 10.4.11
    Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.22

    Hi,

    the recent versions >= 1.1.2 fail to compile here (MacOS 10.4.11).

    I already followed the instructions on http://tech.howcast.com/?p=6 about byteswap.h.

    However recent versions also want endian.h, and including any kind of (_)endian.h on my machine in the build dir didn’t help:

    g++ -o AMFData.o -c -ggdb -O3 -Wall -I. -I/sw/include/boost-1_35 AMFData.cpp
    serialized_buffer.h: In member function ‘uint16_t serialized_buffer::get_u16_be()’:
    serialized_buffer.h:55: error: ‘__bswap_16’ was not declared in this scope
    serialized_buffer.h: In member function ‘uint32_t serialized_buffer::get_u24_be()’:
    serialized_buffer.h:61: error: ‘__bswap_32’ was not declared in this scope
    serialized_buffer.h: In member function ‘uint32_t serialized_buffer::get_u32_be()’:
    serialized_buffer.h:67: error: ‘__bswap_32’ was not declared in this scope
    serialized_buffer.h: In member function ‘uint64_t serialized_buffer::get_u64_be()’:
    serialized_buffer.h:73: error: ‘__bswap_64’ was not declared in this scope
    scons: *** [AMFData.o] Error 1
    scons: building terminated because of errors.

    thx

    c

  6. Safari 3.1.2 Safari 3.1.2 Mac OS X  10.4.11 Mac OS X 10.4.11
    Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10_4_11; en) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.22

    version 1.1.6 builds w/o probs or additional headers on MacOS 10.4.11.

    Some quick tests reveal that it even gives correct results πŸ˜‰ — no endian issues.

    thx

    c

  7. Safari 3.1.2 Safari 3.1.2 Mac OS X  10.5.5 Mac OS X 10.5.5
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_5; en-us) AppleWebKit/525.18 (KHTML, like Gecko) Version/3.1.2 Safari/525.20.1

    Yes, actually I had an email with Dan about the endian issues, and he quickly knocked out version 1.1.6 yesterday for us. πŸ™‚

  8. Firefox 3.0.3 Firefox 3.0.3 Windows XP Windows XP
    Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.3) Gecko/2008092417 Firefox/3.0.3 (.NET CLR 3.5.30729)

    Did anyone try the newest 1.0.7 version of FLVmeta ?

    I still have to get some feedback about this new feature.

    thanks !

  9. Google Chrome 1.0.154.48 Google Chrome 1.0.154.48 Windows Vista Windows Vista
    Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/525.19 (KHTML, like Gecko) Chrome/1.0.154.48 Safari/525.19

    Hey there,

    Glad to have found you . I also plan to serve humongous files through BG. But the thing I am most impressed is how you got an 8hr file to 1.4 GB. Was that the On2 magic ? I have 90 min files that clock in at 1 GB when encoded in MP4. The source files I get are in MPEG-2 format and tend to be even larger, at about 4GB – 7 GB

    Thanks for any leads you can give me.

    Jo

  10. Safari 4.0 Safari 4.0 Mac OS X  10.5.6 Mac OS X 10.5.6
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-us) AppleWebKit/528.16 (KHTML, like Gecko) Version/4.0 Safari/528.16

    Ahh – the test files were pretty much designed so that they would be small in size for testing this specific bug/issue. The original 4 hour video with real content at 640×480 was indeed well over a gigabyte in size. The only thing I can suggest is to lower your bitrate, framerate, keyframes and width/height dimensions as low as you can tolerate for your needs, and used either On2-VP6 or H.264 – those are the parameters you’ll need to compromise on in one way or another to get a smaller file. Also, you mentioned you are getting them in MPEG-2 format. If you can get a copy straight from either the film source, or in a CODEC that is lossless or hasn’t already been through a lossy process (Apples 422, or uncompressed 10bit, similar CODECS), you’ll get a better cleaner encode that should encode a slight amount smaller since it would have less artifacts interfering with the encode process of the target CODEC.

  11. Firefox 3.0.7 Firefox 3.0.7 Windows Vista Windows Vista
    Mozilla/5.0 (Windows; U; Windows NT 6.0; fr; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7

    Hello,

    Does anybody know a Linux CLI tool to inject onCuePoint events in existing FLV files ?
    flvtool2 seems to be the only one but it’s really too sloooow.
    yamdi or flvmeta are great but they don’t support this feature.

    Thank you.

  12. Firefox 3.0.7 Firefox 3.0.7 Windows XP Windows XP
    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (.NET CLR 3.5.30729)

    Does anybody know what happens with really long videos encoded in the FLV container using h.264. After reading through the specs I see a problem with the fact that the AAC Composition Time is still only 24 bits, and features no extended time stamp.

    I am in the process of writing a meta injector for live content streamed using h.264 to Flash… Guess I’ll find out in 3 hours or so when I get back from the pub πŸ™‚

  13. Firefox 3.0.8 Firefox 3.0.8 Windows XP Windows XP
    Mozilla/5.0 (Windows; U; Windows NT 5.1; fr; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8 (.NET CLR 3.5.30729)

    I think I will implement cuePoint insertion in flvmeta1.1, that feature has been asked more than often.

  14. Firefox 3.0.10 Firefox 3.0.10 Mac OS X  10.5 Mac OS X 10.5
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.10) Gecko/2009042315 Firefox/3.0.10

    @George:

    I succeeded by means of forcing a few tags. Example:

    flvtool++ -nomerge -tag framerate “25.000000” \
    -tag videocodecid “avc1” -tag audiocodecid “mp4a” \
    -tag audiosamplerate “22050.000000” -tag stereo “false” \
    -tag width “320.000000” -tag height “240.000000” \
    in.mp4 out.f4v

  15. Firefox 3.0.10 Firefox 3.0.10 Mac OS X  10.5 Mac OS X 10.5
    Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.0.10) Gecko/2009042315 Firefox/3.0.10

    @George
    Sorry, I completely forgot that you have to put the h264 in a flv container first!
    Here’s the complete stanza to cheat h264 content into a flv container with metadata:

    ffmpeg -i in.mp4 -f flv -vcodec copy -acodec copy intermediate.flv
    flvtool++ -nomerge -tag framerate β€œ25.000000β€³ \
    -tag videocodecid β€œavc1β€³ -tag audiocodecid β€œmp4a” \
    -tag audiosamplerate β€œ22050.000000β€³ -tag stereo β€œfalse” \
    -tag width β€œ320.000000β€³ -tag height β€œ240.000000β€³ \
    intermediate.flv out.f4v

  16. Firefox 3.0.11 Firefox 3.0.11 Windows XP Windows XP
    Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11

    Hey, I’m not exactly like you guys here – not programmer but I just like the flv. format. Just wanted to say I had same issue with large files till I stumbled unto “yamdi.” It stands for Yet Another MetaData Injector and it does what the author of this article asks – works for files greater than 800MB. It addes the correct time duration for files greater than 4 hrs. It is literally a simple command line. Also its pretty fast, I’m pretty sure its alot faster than “flvmdi.” Check him pretty cool – especially for newbies. http://yamdi.sourceforge.net/

  17. Firefox 3.0.13 Firefox 3.0.13 Ubuntu 8.10 Ubuntu 8.10
    Mozilla/5.0 (X11; U; Linux i686; ca; rv:1.9.0.13) Gecko/2009080315 Ubuntu/8.10 (intrepid) Firefox/3.0.13

    Hi, I’ve experimented exactly the same problem. Somehow, my it’s a little more hard to fix for me because I’m working with a sequence of movies. I has some little movies (each movie has 10 seconds aprox.). A XML file say to flash wich movie and how long must be played and flashplayer plays that sequence. It never stops and it must be hours playing that sequence. So, after 4h 30minutes flashplayer’s sound and video becomes crazy. There’s no error messages on my system so I can ensure that it’s flashplayer’s cause. And if you turn off and then turn on the fullscreen mode, it works fine again.

    As you said, flashplayer has a counter from 0 to 16777990. Would’nt that counter be reset to 0 when I’m reading a new movie? How could I force reset the counter? It seems to be possible to reset it because I think that it’s what fullscreen does. I’m clearing all variables I know without any success.

    As you would see, I cannot set TimeStampExtended… I’m working with XML files!

    Any help will be great. Thank you in advance.

    PD: I’m so sorry for my english, I’m spanish and my english suck’s!

  18. Firefox 3.5.3 Firefox 3.5.3 Windows Vista Windows Vista
    Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.3) Gecko/20090824 Firefox/3.5.3 (.NET CLR 3.5.30729)

    Where I can find a win32 build or any build for the FaceBook flvtool++?