Syphon Texture to Surface

Using Cinder
1

Hello.

I receive the video data from Syphon and I can draw it on screen as texture.

ciSyphon::ClientRef client;

gl::TextureRef mTex;

======

setup():

client = ciSyphon::Client::create();
client->setup();
client->bind();

======

update():

 mTex = client->fetchFrame();

======

draw():

gl::clear( Color(0, 0, 0 ) );
gl::draw(mTex);

================

It’s easy. But now I want to send mText to OpenCV for face detection. And that’s a problem because OpenCV works with Surface only. I don’t understand how to pass data from Texture to Surface. Maybe it’s not the best solution because of CPU and GPU but I don’t know another way. All examples I found doesn’t work. I would be very appreciate for any help.

2

You can use the createSource () method from Texture2d and the apply the cv::toOcv method to use it with OpenCV.

ci::Surface mySurface =mTex->createSource();
cv::Mat ocvImage = cv::toOcv(&mySurface); //May be just mySurface instead of &mySurface
3

Thank you so much. The next step is to find my face :grinning:

4

Just a heads up, you may want to have a look at what ciSyphon is doing internally, because from my limited understanding of how it works, this solution will receive the frame as CPU side pixels (which is what you want), upload them to the GPU, only for you to pull the pixels back down again. All of this causes a lot of stalling, so you might be better off modifying the library to give you access to the pixels prior to creating an OpenGL texture if you find performance becomes a problem.

5

Yes, you’re right. Frames are freezing, lagging and so on… performance is poor. But I’m afraid it would be very difficult for me to understand what is going on in Syphon under the hood. I have about 5 years experience in Python, but I’m total newbie in C++ and aspecially Obj-C. But I’ll try. Maybe you will see a new Syphon fork on git hub. :sunglasses:

6

So, I spent a day for it.
1). There is no way to get CPU pixels from Syphon.
2). As far as I understood OpenCV can work on GPU but not in the current version for Cinder 0.9.1.
3). I learned to write Syphon textures to Fbo, and I can draw it on screen as well. But I don’t know how it could be useful for my problem. :slight_smile: Maybe OpenCV can get frames from Fbo?
4). Sorry for my English, it’s not my native language :slight_smile:

7

You are right, Syphon is GPU, that’s why it is fast. Why do you need it, though? Can’t you capture the camera stream directly from Cinder?

If not, look into PBO read back, that might help get rid of the lag you are experiencing. Check out this sample:

-Gabor