AX-MediaPlayer - Cinder Block for Windows Video Playback

AX-MediaPlayer

I’ve been getting more and more uncomfortable with how little I understood what was going on behind the scenes in Cinder-WMFVideo, and although it’s served me well over the years there’s been enough little quirks here and there that I decided to try and do something about it. After failing to make heads or tails of the existing code, it turned out much simpler to just implement a new media player instead :wink:

Currently it copies frames to the CPU which means it’s not as performant as it could be, but there is a fast path available via DXGI, so once i learn what the hell that is I might try and add that a bit later. It can come in handy to have access to the pixel data sometimes as recently discussed with @morphogencc so my goal is to have this player offer both paths.

All the usual warnings apply, I just knocked this together in an evening or two so while all care was taken, i can’t guarantee zero leaks or data races, but hopefully it still comes in handy for someone anyway.

Cheers,

A.

8 Likes

OK after another night of head scratching and MSDN scrolling we’ve now got a hardware accelerated fast path merged, with any luck this brings us to feature parity with the current defacto standard player with the added bonus of CPU pixel access, however minus the years of battle testing.

Because of the nature of the thread synchronisation / locking when it comes to the fast path, the API is slightly different when using each of the different paths, though I doubt anyone would be switching back and forth often enough for it to cause any real discomfort. I’ve stuck to the cinder Surface / Texture idiom so it should be pretty familiar.

// CPU Path

if ( _player->CheckNewFrame ( ) )
{
    // Only true if using the CPU render path
    if ( auto surface = _player->GetSurface ( ) )
    {
        // If you need to modify the pixels
    }
    // Or
    _texture = _player->GetTexture ( );
}

// GPU Path
// Only true if using DXGI path
if ( auto lease = _player->GetTexture ( ) )
{
    // You can now use this texture until `lease` goes out
    // of scope (it will Unlock() the texture when destructing )
    gl::draw ( *lease, getWindowBounds ( ) );
}

The sample application shows how both methods can be consumed side by side should anyone want that but as I said I suspect the dominant use case will be picking one method and sticking with it, but more than happy to take any advice / suggestions for sensible defaults / API design etc

Cheers,

A

2 Likes

Just wanted to express my appreciation of your work @lithium . While the video rendering path of my software isn’t the highest priority, it remains a high importance.

One thing I’d want to ask upfront is if you’ve run into WMFCinder throwing spurious exceptions and whether your own implementation exhibits any similar behavior? I’ve been assuming that it’s perhaps just a part of the underlying WMF platform, and WMFCinder appears to function fine even with the odd exception.

Cheers,

Gazoo

Thanks mate :slight_smile:

Could you elaborate a bit on what kind of exceptions you’re seeing? My implementation is generally pretty quiet, though you will see the occasional _com_error from, say, setting a negative playback rate on a video that doesn’t support reverse playback, but these appear to be internal to the IMFMediaEngine and don’t affect playback or stability.

Ever since the great Nahimic debacle of 2021 it’s hard to know what’s responsible for which exceptions anymore :confused:

Thank you @lithium ^^

In WMFCinder…?

This type of exception:

Oh no, so far I’ve seen nothing like that, and those kind of indecipherable / unpredictable exceptions are exactly what made me want to start fresh in the first place.

There’s a lot less code in my version which is always a good thing, and most of the heavy lifting is handled by IMFMediaEngine so if microsoft can’t get it right themselves then what hope do the rest of us have? :wink:

Just plugged this into the project I’m working on and it works great! Legit seeing a 10x+ improvement in performance over trying to kludge WMF into playing nicely with OpenCV!

I’m volunteering myself for battle testing, so I’ll be sure to report back :wink:

Thanks so much for all of your work here!

Awesome, that’s really encouraging. Thanks for putting it through its paces for me.

Given that I’m pulling out the rug from my own project, I’ll also see about exchanging WMFCinder for your framework @lithium . Can’t wait to try it!

I’m stuck representing the video path in the SimplePlaybackApp sample correctly. How do we set the CINDER_PATH macro? Ideally I’d like to load the video from the project’s assets folder, but I’m not able to set that path or load that asset for AX::Video::MediaPlayer either.

You’re not the first person to get caught by that, I thought i was being clever pointing the sample to the user’s cinder installation so I didn’t have to provide a test video, but apparently not ;). The CINDER_PATH macro gets injected by the sample’s cmake file, just for future reference.

But to your question, the constructor just takes a DataSourceRef so any usual cinder method of constructing those will work:

MediaPlayer::Create ( loadAsset ( "SomeAsset.mp4") ); // For a assets/SomeAsset.mp4 
MediaPlayer::Create ( loadFile ( "c:/Path/To/SomeFile.mp4") ); // For a fully qualified path
MediaPlayer::Create ( loadUrl ( "http://someserver.com/SomeStream.mp4") ); // For a streamed url  

I also got stuck in the cmake process, so I did not use it. I cloned your repo into the Cinder blocks folder, failed to get cmake to work, and then linked to the src directory instead. Perhaps this is why I am having so much difficulty.

Yeah cmake can be pretty annoying to get going but once it’s all set up it’s definitely better than the alternative. This thread has some good info about getting going on windows.

That said, this is a source only library and I handle all linking via #pragma comments so just dumping the source files into your project should be enough to get started. Did that not work either?

The linking seems to be working, but for good measure I’ve taken your advice and dumped the source files into the project.

I’ve put the video “bbb.mp4” into the assets folder of the project and added this file via VS Solutions Explorer. When I try to run (in setup()):

    auto fmt = AX::Video::MediaPlayer::Format().HardwareAccelerated(_hardwareAccelerated);

    _player = AX::Video::MediaPlayer::Create(loadAsset("bbb.mp4"), fmt);

I get the error:

1>TestAXMediaPlayerApp.obj : error LNK2019: unresolved external symbol "public: static class std::shared_ptr<class AX::Video::MediaPlayer> __cdecl AX::Video::MediaPlayer::Create(class std::shared_ptr<class cinder::DataSource> const &,struct AX::Video::MediaPlayer::Format const &)" (?Create@MediaPlayer@Video@AX@@SA?AV?$shared_ptr@VMediaPlayer@Video@AX@@@std@@AEBV?$shared_ptr@VDataSource@cinder@@@5@AEBUFormat@123@@Z) referenced in function "public: virtual void __cdecl TestAXMediaPlayerApp::setup(void)" (?setup@TestAXMediaPlayerApp@@UEAAXXZ)

Sorry, I need to straighten out my Cinder forum user accounts. Seems I have one account with autologin on my Mac from years ago and a more recent one on my PC.

It looks like you’re not compiling one (or more) of the source files. Do you want to put your project somewhere and I can have a look at it?

I do recommend getting cmake going at some stage though as all of these problems just magically go away, but for now, making a new test app in tinderbox and dragging all of the .h and .cxx files from the AX-MediaPlayer clone into the solution should be all you need to do.

for now, making a new test app in tinderbox and dragging all of the .h and .cxx files from the AX-MediaPlayer clone into the solution should be all you need to do

Ah, I thought that I did that, but I bet that while I did drag the AX-MediaPlayer source files to the project directories, I did not put them into the solution. I typically use xcode so VS also confounds me. I will push on using cmake.

Thank you!!