This is my second post here and I’m in the process of testing Cinder on the Raspberry Pi 3.
Before I worked with OF on Linux platforms but since Cinder now supports it too, I’m interested and want to check it out and help where I can.
For now I’m especially interested in Gstreamer related things. I tried to compile a basic examples but the linking failed.
How far is the implementation of Gstreamer for Linux in Cinder ? Maybe someone can point me in the right direction and I can contribute to get it to work.
the basic GStreamer player functionality should already work across the Cinder supported Linux variants ( e.g RPi / Jessie, TK1, *buntu etc. ) . That said there are a lot of different things to take in to account, especially due to the different nature of the supported devices and variants but also because of the changes in recent versions of GStreamer itself.
This might get a bit long so apologies beforehand.
One main thing is that we have different versions of GStreamer shipping with the different flavors of Linux that Cinder can run on – This makes it trickier to construct/find a unified API that works equally well on all the different Linux variants and devices. This is especially true when you want to take advantage of more experimental features like the gst-gl components for example. This API moves some processing from the cpu to the gpu which makes things faster and will also allow for things like zero-copy gpu transfers in the future. This API has been in flux up until now across the different GStreamer versions ( 1.4, 1.6 and now 1.8 for Xenial ) but there is already initial Cinder support for it.
In parallel with this there is support for the more ‘traditional’ ( and slower ) path where most of the processing happens on the cpu. Depending on your requirements / hardware and GStreamer version though this might be already good enough. I would consider this path stable and I m using it for two installations that I just got finished with GStreamer 1.8 and Xubuntu 16.04.
The path decision happens based on the detected GStreamer version so for >= 1.6 the gl path is chosen and for smaller versions the cpu one.
The RPi is also a special case because of the OMX plugin and the need for moving as much as possible to the gpu but last I checked ( it has been some months already ) on Jessie, GStreamer was still at 1.4.4 . Arch provides the latest packages but the ones provided by pacman do not have properly enabled gl support on the RPi so there also you would have to go with the cpu part. The only solution for testing the gl path in these devices currently would be to compile and install GStreamer from source.
The video player is something that I am planning to work again in the next period and any suggestions/testing/remarks are more than welcome. Which brings me to your linking error statement – What distribution are you trying to test stuff on and what kind of linking error are you getting? The gst player is currently mapped to the qtime namespace so you should try to compile and run the QuickTimeBasicSample for starters. The gst libraries are pulled in automatically for you through our CMake configuration so you shouldn’t need any additional configuration as long as you have GStreamer installed in your system. Btw if you don’t get any linker errors when building the BasicSample for example, this means that all of Cinder’s dependencies ( including GStreamer ) are properly found on your system so if that builds, then the video sample should also build for you.
Hope these info give a first overview of the current status – It is a challenging topic in general and It would be great to get more people involved in this so please feel free to come back with more on this!
first of all thanks for the detailed answer.
I decided now to work with Cinder’s Gstreamer implementation and build my own special player on top of GstPlayer.
As you suggested I tried to compile the QuickTimeBasicSample but got some linking errors too.
After digging into the cmake files if found the problem.
This pull request #1493 should resovle these. Tested on Ubuntu 16.04 and RPI3 Jessi.
Although it runs now, the whole OS crashes after a couple of seconds when running the QuickTimeBasic example with gstreamer version 1.8.1.
True, the video source files were missed from build process on the new cmake configuration. I also realized this today.
I would suggest to force the cpu path to start with so that you are sure that you have at least basic things running properly. For now you would have to overwrite this variable to false and recompile Cinder. This option should eventually come from the CMake configuration process in the future.
I tried to get Gstreamer working on the RPI with full HD videos.
With the following pipeline it worked totally fine : gst-launch-1.0 uridecodebin uri=file:///full_HD_file.mp4 ! queue ! glimagesink
So trying to apply this to the Cinder pipeline and still using appsink I found this post suggesting to use glcolorscale to get a gl:memory buffer for the appsink.
This works much better on the RPI now, with less CPU usage, but still it is only possible to play 720p videos smooth. Any suggestions on this ?
Testing it on Ubuntu 16.04 the cpu path works as you sad. Didn’t had the time to inspect the gpu path here, as RPI is my priority.
Against which version of GStreamer are you testing this exactly? As far as I can tell it must be a 1.4.* which is nice to see that the gl path is at least working there since it has been a lot of upstream development in between.
Unfortunately the 720p for smooth playback on the Pi was also a wall that I hit when working on this with appsink. I didn’t have much time to see what could be optimized at that point specifically for the Pi but its definitely something that needs to be addressed in the near future.
I came across this gst-omx example that can provide a good reference point for the Pi, but from a first glance I don’t see something completely different in the logic of things. One thing for sure is the managing of the incoming buffers on the gl path which definitely needs to be cleaned up / fixed in our case but other than that the idea is the same.
Nevertheless it might be a good reference point for and it also uses a different approach ( signals vs callbacks ) for passing the buffers that might worth exploring ( supposedly the overhead of using signals is larger than the callbacks alternative but most of the recent examples use this approach so it would need some benchmarking ) .
On the the RPI the Gstreamer version is 1.4.4 so far.
For me 720p is sufficient for now, but I’m gonna look in to it in more detail when I have more time.
Currently working on porting your OF addon ofxGstVideoSyncPlayer to Cinder, while adding some more features to GstPlayer.
One of this features is adding signals for EOS, ready, ect. Not sure if it’s the right way to put them into the GstPlayer class to be consistent with the cinder way. Commit
I like signals for callbacks such as these. One trick that we’ve learned is that you get nice auto-completion and organized docs if you start the methods with getSignal*(), which is also the pattern that cinder code uses (like here).
thanks for the answer. Looks like I did it right so far.
Since Cinder has now GST and QT I’m wondering if it would make sense to create a movie interface both implementations share. Something like Moviebase without the GSt or QT related things.
This would probably be worth discussing in another forum post before adding to much to GST. Or maybe it has been discussed before and I didn’t find it.
We’re heading in that direction (cross-platform ci::video api). Right now however the focus is on getting platform-specific implementations in decent shape, and getting 0.9.1 wrapped up. I expect there to be much public discussion a little ways down the road…