A while back I posted whining about the lack of decent video support in Java, and the fact that try as I might I just couldn't seem to find a decent one. I wrote this as part of my struggles trying to get video implemented properly in Quelea. The good news is that the trunk version of Quelea seems to have decent, cross platform and reliable video that's compatible with any format you throw at it. And it's all thanks to the VLCJ / VLC guys. The bad news is that, well, it's not as obvious or as easy as it might first look.
Before I go into details however, I think an honourable mention goes to Xuggler. It was the first package I really tried that seemed to be getting me somewhere - with really quite little effort after watching the tutorials I had a basic video application going that could play most file types. It couldn't do any seeking, pausing or the like but that I thought would be relatively easy. And, guess what - once again I was completely wrong. See, Xuggler isn't a high level video API. You don't point a video file and canvas at it and tell it to play one on the other. Instead, you have to open each packet in the file, check what stream it comes from, see if it's a video or an audio packet, deal with it appropriately and make sure the timestamps of the two match up (that's the really hard part)...
Initial results looked positive. But before long the video and audio on the videos drifted slightly out of sync. If there was a significant pause on the video because it was fetching it from disk or something like that, this problem got worse still. There were all sorts of corner cases as well that would've taken months to track down and cope with, both in terms of streaming, file formats, sync issues and so on. The guys on the discuss page were very friendly, and there were excellent tutorials provided - but as time wore on it became clearer and clearer Xuggler wasn't going to be for me. Beyond a certain point you were always going to be on your own, and I needed to go way beyond that point!
That's not to say Xuggler's useless, far from it. If you're doing any low level video stuff (transcoding, converting between formats, analysing frame / sound contents etc.) it's probably the best out there for Java. But that wasn't what I needed to do. The crux of the matter is, Xuggler just isn't really designed to be used in this way. It's not a high level video API, nor will it ever be.
At this point I really did think all my options were out. JMF was dead, no-one on the face of this earth can seem to get FMJ working / it's half dead too, VLC/VLCJ kept bringing the VM down, Xuggler was aimed at the wrong area for me, and to top it off my JavaFX2 hopes were dashed by finding out video-wise it only supported FLV, at least for now.
Rather than keep trying to find new things to use (I'd done a lot of Googling by now!) I decided to revisit my options and see if there was anything I'd missed. On the JMF and FMJ front this seemed very unlikely. The conclusions I came too seem to be reflected by many others for very valid reasons - there's not a lot you can say about projects that genuinely seem dead and don't work properly. And as for JavaFX, well it really does just support FLV and I really do need much more than that! I'd already taken a long, hard look at Xuggler; this left me with VLCJ and more specifically asking the following two questions:
This didn't take too long to find out: http://code.google.com/p/vlcj/wiki/Crashes
In fact, I'd already found and read the link before, but I'd brushed it aside because it basically seemed to say that there wasn't much hope with ever getting the thing to work 100% reliably. At least not without resorting to some out of process magic which was a route I really didn't want to go down.
After getting to this point though I did start to look down the out of process route, and it's this approach that I'm currently using in Quelea. In Windows it appears to work absolutely fine, I haven't managed to test anywhere else yet but in theory there's no reason why it shouldn't work on MacOS or Linux (potentially more native code might need to be executed on Mac to get it to work; this won't be an out of the box solution on Mac. But more on that later hopefully!)
To avoid creating one huge post, the code and more details about it will follow. But if you've been after a run down of how to do video bits in Java and come unstuck, I hoped this has helped.
In conclusion? If you want low level video manipulation, use Xuggler. If you want to play videos nicely in a variety of formats, use VLCJ with out of process players.
Before I go into details however, I think an honourable mention goes to Xuggler. It was the first package I really tried that seemed to be getting me somewhere - with really quite little effort after watching the tutorials I had a basic video application going that could play most file types. It couldn't do any seeking, pausing or the like but that I thought would be relatively easy. And, guess what - once again I was completely wrong. See, Xuggler isn't a high level video API. You don't point a video file and canvas at it and tell it to play one on the other. Instead, you have to open each packet in the file, check what stream it comes from, see if it's a video or an audio packet, deal with it appropriately and make sure the timestamps of the two match up (that's the really hard part)...
Initial results looked positive. But before long the video and audio on the videos drifted slightly out of sync. If there was a significant pause on the video because it was fetching it from disk or something like that, this problem got worse still. There were all sorts of corner cases as well that would've taken months to track down and cope with, both in terms of streaming, file formats, sync issues and so on. The guys on the discuss page were very friendly, and there were excellent tutorials provided - but as time wore on it became clearer and clearer Xuggler wasn't going to be for me. Beyond a certain point you were always going to be on your own, and I needed to go way beyond that point!
That's not to say Xuggler's useless, far from it. If you're doing any low level video stuff (transcoding, converting between formats, analysing frame / sound contents etc.) it's probably the best out there for Java. But that wasn't what I needed to do. The crux of the matter is, Xuggler just isn't really designed to be used in this way. It's not a high level video API, nor will it ever be.
At this point I really did think all my options were out. JMF was dead, no-one on the face of this earth can seem to get FMJ working / it's half dead too, VLC/VLCJ kept bringing the VM down, Xuggler was aimed at the wrong area for me, and to top it off my JavaFX2 hopes were dashed by finding out video-wise it only supported FLV, at least for now.
Rather than keep trying to find new things to use (I'd done a lot of Googling by now!) I decided to revisit my options and see if there was anything I'd missed. On the JMF and FMJ front this seemed very unlikely. The conclusions I came too seem to be reflected by many others for very valid reasons - there's not a lot you can say about projects that genuinely seem dead and don't work properly. And as for JavaFX, well it really does just support FLV and I really do need much more than that! I'd already taken a long, hard look at Xuggler; this left me with VLCJ and more specifically asking the following two questions:
- Why did it seem to crash the VM every so often whenever I did anything complicated?
- Could this be avoided?
This didn't take too long to find out: http://code.google.com/p/vlcj/wiki/Crashes
In fact, I'd already found and read the link before, but I'd brushed it aside because it basically seemed to say that there wasn't much hope with ever getting the thing to work 100% reliably. At least not without resorting to some out of process magic which was a route I really didn't want to go down.
After getting to this point though I did start to look down the out of process route, and it's this approach that I'm currently using in Quelea. In Windows it appears to work absolutely fine, I haven't managed to test anywhere else yet but in theory there's no reason why it shouldn't work on MacOS or Linux (potentially more native code might need to be executed on Mac to get it to work; this won't be an out of the box solution on Mac. But more on that later hopefully!)
To avoid creating one huge post, the code and more details about it will follow. But if you've been after a run down of how to do video bits in Java and come unstuck, I hoped this has helped.
In conclusion? If you want low level video manipulation, use Xuggler. If you want to play videos nicely in a variety of formats, use VLCJ with out of process players.
Comments
Post a Comment