Cinder-VLC + alpha

CinderBlocks
1

Hello,
I like Cinder-VLC because it’s very robust with a lot of video in same time (in contrary with quictime + hap) !
I can’t active alpha channel !
I tested with quicktime animation with alpha, quicktime png and avi + apha
but it does not work !
Any idea ?

void SimplePlaybackApp::draw()
{
    gl::clear( ColorA( 0.5, 0, 0, 1.0) );

    gl::enableAlphaBlending();
    gl::TextureRef mVideoTextureRef = moviePlayer->getTexture();

    if (mVideoTextureRef)
    {
        gl::color(ColorA::white());
        gl::draw(mVideoTextureRef, mVideoTextureRef->getBounds() , getWindowBounds());
    }
}

Other question, does sound work ?

Thanks

2

Check if the texture’s internal format is GL_RGBA or equivalent (like GL_ARGB or GL_BGRA). Chances are it’s only GL_RGB and simply does not decode the alpha channel. If you have the source code of the video decoder, that’s where I would look as well.

3

https://github.com/AndreaMelle/Cinder-VLC
https://github.com/AndreaMelle/Cinder-VLC/blob/master/src/CVLCMoviePlayer.cpp
Texture is GL_RGBA !

I’m curious !
What’s player video was used for the magnificent project Samsung CenterStage please ?

Thanks

4

hey @colin a bit of topic but do you have the correct dll & lib files for this addon?

Thx in advance.

5

A bit late to the table, but in reply to Colin’s question: the video player for Samsung CenterStage was written from the ground up by the mighty Andrew Bell. It’s completely custom and not available anywhere. You also need a pretty beefy computer to run it at its 12K resolution.

6

nevermind converted them using this tutorial
https://wiki.videolan.org/GenerateLibFromDll/

7

I used the same tuto : https://wiki.videolan.org/GenerateLibFromDll/

Whahou 12k resolution for the video player for Samsung CenterStage !
Very impressive ! Bravo Andrew !

8

I believe the res was roughly 8k x 4k, and some crazy compression was necessary to play them in real-time (.dds images that were decoded on the gpu), as well as a more than hefty compression time (a one minute video was somewhere around 30gb), so obviously not a general solution but was amazing that he got it done. :slight_smile: Modern day top of the line graphics cards have a much easier time doing this.

9

It seems somehow the holy grail for creative coding frameworks especially on Windows.
On OSX I had very good experiences with large resolutions when using cinders builtin QTKit player using ProRes files.
On Windows I use DXT files most of the time but this doesn’t work well if the resolutions are very large and when clients want to update the files themselves. Therefore I was looking into the VLC player.
It’s kinda frustrating that the VLC player plays large (4K) files easily but when using the cinder block the fps is super low.
I suspect that this has todo with the fact that on windows most decoding works with directx which doesn’t exchange well with OpenGL?

10

You can leverage the WGL_NV_DX_interop extension on most modern cards (even AMD and intel implement it despite being an nvidia extension) to share hardware accelerated video from directx to gl. Cinder-WMFVideo goes this route, though I can’t vouch for the quality of the block since i’ve never used it. but i can vouch for the technique itself.

11

I’ve actually used the wmf block ~2 years ago, for a 4k video full of particles, and it was pretty smooth on windows.
Can’t remember exactly what graphic card it was, but probably a titan black.

12

OK, here goes… first a rant, then a recommendation.

The CinderWMF does a decent job, but it’s limited in what it can do. For instance, it doesn’t allow you access to the decoded textures, because it maintains a small pool of textures that DirectX needs to have control over. Handing those textures to the user would negatively impact the hardware video decoding process.

I’m not sure the block handles the small implementation differences between NVIDIA, AMD and Intel GPU’s equally well. I remember being unable to run it on several machines with GPU’s from different vendors. This is probably due to the flags used to create the DirectX surfaces.

The way the block works, is to create a Windows Media Foundation player that runs in a hidden window. WMF is the only way to have access to hardware accelerated video on the Windows Platform* (which is a bloody shame really). The block uses a DirectX9 backend to create a pool of surfaces that are then shared with OpenGL through the WGL_NV_DX_interop extension. Each time a frame has been decoded, the main thread is notified and the surface/texture is then locked to prevent concurrency errors. This is quite an expensive operation that unfortunately reduces the performance of video in OpenGL compared to DirectX, but not by a lot. OpenGL can then render the texture, after which it is unlocked and DirectX can write a new video frame to it.

Because DirectX runs in a thread of its own, we need at least 3 surfaces to avoid stalls, and OpenGL can not hold on to a texture for too long. The block has no mechanism to create extra surfaces when needed, for example if you want to capture screenshots or thumbnails from the movie. The fact that it needs a scheduler running in a hidden window, which is tied to the refresh rate of the monitor, causes OpenGL to often wait for vertical sync twice, before rendering the video frame, causing sync issues. Rewriting the scheduler is not an easy task, though.

People on NVIDIA laptops with Optimus (like yours truly) are in bad luck, because the Intel integrated GPU supports a DirectX9 backend, but the NVIDIA discrete GPU requires a DirectX11 backend. The block does not provide that.

As you can tell, I have been looking into video on Windows for quite a bit. I turned grey because of it and my doctor has made me promise never to work on it again.

With that being said, I do believe the block is the best option for running video on Windows while the platform does not provide access to hardware accelerated video decoding from OpenGL and I would like to sincerely thank the fine people at Stimulant and Philippe Laulheret from Second Story for their pioneering work.

-Paul

*) technically, the older DirectShow also allows access to hardware accelerated video, but it’s an older API that is no longer properly supported on modern systems (64-bit Windows 10, for example).

1 Like
13

I had tried CinderWMF recently, but could not get alpha channel to work. Are there any codec+format that preserves alpha when used with CiWMF?

Thanks,
Bala.

14

No, probably not. You can find a list of supported formats here.

15

*) technically, the older DirectShow also allows access to hardware accelerated video, but it’s an older API

It’s an older API but it checks out

(I’ll show myself out)