I’ve been getting more and more uncomfortable with how little I understood what was going on behind the scenes in Cinder-WMFVideo, and although it’s served me well over the years there’s been enough little quirks here and there that I decided to try and do something about it. After failing to make heads or tails of the existing code, it turned out much simpler to just implement a new media player instead
Currently it copies frames to the CPU which means it’s not as performant as it could be, but there is a fast path available via DXGI, so once i learn what the hell that is I might try and add that a bit later. It can come in handy to have access to the pixel data sometimes as recently discussed with @morphogencc so my goal is to have this player offer both paths.
All the usual warnings apply, I just knocked this together in an evening or two so while all care was taken, i can’t guarantee zero leaks or data races, but hopefully it still comes in handy for someone anyway.
OK after another night of head scratching and MSDN scrolling we’ve now got a hardware accelerated fast path merged, with any luck this brings us to feature parity with the current defacto standard player with the added bonus of CPU pixel access, however minus the years of battle testing.
Because of the nature of the thread synchronisation / locking when it comes to the fast path, the API is slightly different when using each of the different paths, though I doubt anyone would be switching back and forth often enough for it to cause any real discomfort. I’ve stuck to the cinder Surface / Texture idiom so it should be pretty familiar.
// CPU Path
if ( _player->CheckNewFrame ( ) )
{
// Only true if using the CPU render path
if ( auto surface = _player->GetSurface ( ) )
{
// If you need to modify the pixels
}
// Or
_texture = _player->GetTexture ( );
}
// GPU Path
// Only true if using DXGI path
if ( auto lease = _player->GetTexture ( ) )
{
// You can now use this texture until `lease` goes out
// of scope (it will Unlock() the texture when destructing )
gl::draw ( *lease, getWindowBounds ( ) );
}
The sample application shows how both methods can be consumed side by side should anyone want that but as I said I suspect the dominant use case will be picking one method and sticking with it, but more than happy to take any advice / suggestions for sensible defaults / API design etc
Just wanted to express my appreciation of your work @lithium . While the video rendering path of my software isn’t the highest priority, it remains a high importance.
One thing I’d want to ask upfront is if you’ve run into WMFCinder throwing spurious exceptions and whether your own implementation exhibits any similar behavior? I’ve been assuming that it’s perhaps just a part of the underlying WMF platform, and WMFCinder appears to function fine even with the odd exception.
Could you elaborate a bit on what kind of exceptions you’re seeing? My implementation is generally pretty quiet, though you will see the occasional _com_error from, say, setting a negative playback rate on a video that doesn’t support reverse playback, but these appear to be internal to the IMFMediaEngine and don’t affect playback or stability.
Ever since the great Nahimic debacle of 2021 it’s hard to know what’s responsible for which exceptions anymore
Oh no, so far I’ve seen nothing like that, and those kind of indecipherable / unpredictable exceptions are exactly what made me want to start fresh in the first place.
There’s a lot less code in my version which is always a good thing, and most of the heavy lifting is handled by IMFMediaEngine so if microsoft can’t get it right themselves then what hope do the rest of us have?
Just plugged this into the project I’m working on and it works great! Legit seeing a 10x+ improvement in performance over trying to kludge WMF into playing nicely with OpenCV!
I’m volunteering myself for battle testing, so I’ll be sure to report back
I’m stuck representing the video path in the SimplePlaybackApp sample correctly. How do we set the CINDER_PATH macro? Ideally I’d like to load the video from the project’s assets folder, but I’m not able to set that path or load that asset for AX::Video::MediaPlayer either.
You’re not the first person to get caught by that, I thought i was being clever pointing the sample to the user’s cinder installation so I didn’t have to provide a test video, but apparently not ;). The CINDER_PATH macro gets injected by the sample’s cmake file, just for future reference.
But to your question, the constructor just takes a DataSourceRef so any usual cinder method of constructing those will work:
MediaPlayer::Create ( loadAsset ( "SomeAsset.mp4") ); // For a assets/SomeAsset.mp4
MediaPlayer::Create ( loadFile ( "c:/Path/To/SomeFile.mp4") ); // For a fully qualified path
MediaPlayer::Create ( loadUrl ( "http://someserver.com/SomeStream.mp4") ); // For a streamed url
I also got stuck in the cmake process, so I did not use it. I cloned your repo into the Cinder blocks folder, failed to get cmake to work, and then linked to the src directory instead. Perhaps this is why I am having so much difficulty.
Yeah cmake can be pretty annoying to get going but once it’s all set up it’s definitely better than the alternative. This thread has some good info about getting going on windows.
That said, this is a source only library and I handle all linking via #pragma comments so just dumping the source files into your project should be enough to get started. Did that not work either?
Sorry, I need to straighten out my Cinder forum user accounts. Seems I have one account with autologin on my Mac from years ago and a more recent one on my PC.
It looks like you’re not compiling one (or more) of the source files. Do you want to put your project somewhere and I can have a look at it?
I do recommend getting cmake going at some stage though as all of these problems just magically go away, but for now, making a new test app in tinderbox and dragging all of the .h and .cxx files from the AX-MediaPlayer clone into the solution should be all you need to do.
for now, making a new test app in tinderbox and dragging all of the .h and .cxx files from the AX-MediaPlayer clone into the solution should be all you need to do
Ah, I thought that I did that, but I bet that while I did drag the AX-MediaPlayer source files to the project directories, I did not put them into the solution. I typically use xcode so VS also confounds me. I will push on using cmake.
Hi @lithium this looks exciting, really looking forward to checking this out! A fresh new look at accessing video playback via the platform-provided WMF is great.
I’m curious how you usually run this - do you use cmake from the command line and generate a Visual Studio project, or open a specific folder in Visual Studio, ignore, etc? There are a number of approaches to go about configuring and building with cmake settings and I wanted to go with the way you’ve been using.
Hey mate, happy to hear it may come in handy for you.
I tried to follow the existing cmake cinder block style so that you should be able to just add AX-MediaPlayer to the BLOCKS list when calling ci_make_app (assuming it lives in your {cinder_path}/blocks folder), or just selecting it as usual from TinderBox.
If all that fails, you can just dump the source files into your project, it’s laid out to potentially support other platforms / backends (i.e implemented via pimpl) but as it stands it’s just windows + wmf so every source file needs to be compiled, meaning just copying everything into your source tree should be fine.
Happy to take advice about this if there’s better ways to make itself available to a wider range of project formats though.
Got it working! With the command line I posted here for others who are trying to get the cmake build working.
Also tried out a 4K / 30fps video and it played very nicely at 60FPS on my PC.
Thanks again @lithium for putting this together! So far it looks like the best option for video on windows when you don’t want large dependencies, and I’ll try to put it through the wringer in upcoming projects.
For ease of other people testing, I’d suggest adding a Visual Studio project created from Tinderbox that plays nicely with the age-old way cinder is built on windows (although I do personally prefer cmake these days). I can send you a PR for this if it helps.
The other thing I’m looking at is if there is some format that WMF can play where we can have video with alpha channels, as this always comes up. I did see there is a H.265 / HEVC Video Decoder that should theoretically allow for this, but I recall using MF Transforms to be quite tricky.
It would be great to finally get a proper WMF video player in cinder core … the past attempts have fallen short but it is still the best native path.
