Applying a shader to a texture after gl::draw?

Using Cinder
1

Hello,

I’m currently working on a project that takes in n number of videos, slices them up into squares and draws them at somewhat random positions on the screen.

Basic movie loading, getting texture(s) from each movie and drawing using gl::draw

setup()…
qtime::MovieGlRef movie = qtime::MovieGl::create (path);
…

update()…
movieTexture = movie->getTexture();
…

draw()…
gl::draw (movieTexture, src, dest);
…

Everything is looking and performing somewhat as expected, playing 10 videos at once is taking about 28-30% CPU on my Macbook Pro (2.3 GHz i7 w/ a GeForce GT 750M) (4 year old Macbook) on a debug build.

I’m at a point now where I’d like to add a shader and apply it to the texture(s) that are being drawn… Just trying to understand how cinder::gl/OpenGL work. My OpenGL plumbing chops are a bit rusty (currently reading lots of bits and pieces online and digging into “the” OpenGL book).

So my question is:

  1. Can a shader be applied to a texture after it is drawn using gl::draw ?
  2. I’m currently looking at gl::draw (const Texture2dRef &texture, const Area &srcArea, const Rectf &dstRect) implementation and from what I understand if I wanted to add a shader to a texture I would basically need to roll my own version of gl::drawTexture which would have my custom shader in addition to what the original IMPL is doing.

Also where is this https://github.com/cinder/Cinder/blob/bcff0b9df2aff499971a08f707f52d787747f8d9/src/cinder/gl/draw.cpp#L300 “stock” shader located? Is it baked in to the library binary as text or is it generated on the fly? Going to dig a little more for it.

I obviously have lot more to take in and wrap my head around to understand how things actually get bound/drawn etc. so any help would be appreciated.

Thanks

2
  1. No, but that doesn’t matter, because
  2. You can use a stock shader, your texture, and gl::drawSolidRect() to achieve what you need.

The stock shaders are generated based on a gl::ShaderDef() and are cached in the gl context. To have a look at what they can generate, try passing different gl::ShaderDef()s to the generation functions:

auto def = gl::ShaderDef().color().lambert();
auto vs = gl::env()->generateVertexShader( def );
auto fs = gl::env()->generateFragmentShader( def );

std::cout << vs << "\n===\n" << fs << std::endl;

To do what you need, you’ll have to do something like:

{
    // gl::TextureRef movieTexture comes from your video
    
    Rectf destRect { 0, 0, 320, 240 };
    Area sourceArea { 0, 0, 160, 120 };
    Rectf uvs = movieTexture->getAreaTexCoords( sourceArea );
    
    gl::ScopedGlslProg shader ( gl::getStockShader ( gl::ShaderDef().texture( movieTexture ) ) );
    gl::ScopedTextureBind tex0 ( movieTexture );
    gl::drawSolidRect( destRect, uvs.getUpperLeft(), uvs.getLowerRight() );
}

Your uvs might need to have their y components swapped if it comes in upside down. Also note that i’ve wrapped it in its own scope so that the gl::Scoped* classes can do their thing. This is also a prime candidate for instancing, where you can draw all the rectangles at once by providing an index into a texture array as a custom attribute, but i suggest you only worry about that if / when performance becomes an issue.

Hope it helps,

A.

1 Like
3

@lithium thanks for the reply, very helpful.

generateVertexShader() and generateFragmentShader() are very useful.

Wrote a test that switches between gl::drawSolidRect using a shader and gl::draw and it works just fine.

if (useCustomShader) {
    
    gl::ScopedGlslProg shader (gl::getStockShader (gl::ShaderDef().texture (txtr_)));    
    gl::ScopedTextureBind tex0 (txtr_, 0);
    
    const Rectf uvs = txtr_->getAreaTexCoords (src.getInteriorArea());
    gl::drawSolidRect (dest, uvs.getUpperLeft(), uvs.getLowerRight());
}
else {
    gl::draw (txtr_, src.getInteriorArea(), dest);
}

But as soon as I try to add my own custom shader I get a black screen.

glsl in setup()


glsl = gl::GlslProg::create (gl::GlslProg::Format()
.vertex (CI_GLSL (150,
    uniform mat4 ciModelViewProjection;

    in vec4 ciPosition;
    in vec2 ciTexCoord0;
    out highp vec2 TexCoord;
    
    void main( void ) {
        gl_Position = ciModelViewProjection * ciPosition;
        TexCoord = ciTexCoord0;
    }
 ))
.fragment (CI_GLSL (150,
    out vec4 oColor;
    uniform sampler2D uTex0;
    in vec2	TexCoord;
    
    void main( void ) {
        oColor = vec4 (1) * texture (uTex0, TexCoord.st);
    }
)));

draw()


if (useShader) {
    
    gl::ScopedGlslProg shader (glsl);
    glsl->uniform ("uTex0", 0);
    
    gl::ScopedTextureBind tex0 (txtr_, 0);
    
    const Rectf uvs = txtr_->getAreaTexCoords (src.getInteriorArea());
    gl::drawSolidRect (dest, uvs.getUpperLeft(), uvs.getLowerRight());
}
else {
    gl::draw (txtr_, src.getInteriorArea(), dest);
}
4

This is because quicktime uses the texture target GL_TEXTURE_RECTANGLE rather than GL_TEXTURE_2D. In your shader, change uTex0 to be of type sampler2DRect and you should be fine.

1 Like
5

@lithium thanks, that was it.