Android: Sampling from movie texture does not work

Developing Cinder
1

I am converting iOS app to Android. iOS version has a working displacement map texture rendering, in which the displacement map is a movie texture. On Android, however, sampling from a movie texture returns zeros.

Below I removed the displacement part and simply am drawing the movie texture using a primitive shader

    cinder::android::video::MovieGlRef		mMovie;
   gl::TextureRef			mDisplaceTexture;
...
//setup
        mMovie = cinder::android::video::MovieGl::create( moviePath );
        mMovie->play();
...
//update
       mDisplaceTexture = mMovie->getTexture();
...
//draw
                gl::ScopedColor col(Color::white());
                gl::ScopedModelMatrix modelScope;
                gl::ScopedTextureBind a (mDisplaceTexture, 0);
      
                gl::ScopedGlslProg shaderScp1( glsl );
                glsl->uniform("mDisplaceTexture", 0);
                 gl::drawSolidRect( Rectf(vec2(0),getWindowSize() ) );

The vert shader:

#version 100
uniform mat4 ciModelViewProjection;

attribute vec4		ciPosition;
attribute vec2		ciTexCoord0;
attribute vec4     ciColor;
varying vec2	Vertex;

void main( void )
{
    Vertex.xy 	= ciTexCoord0;
    gl_Position = ciModelViewProjection * ciPosition;
}

The fragment shader:

#version 100

//uniform sampler2D   mCamTexture;
uniform sampler2D   mDisplaceTexture;

varying highp vec2	Vertex;

void main( void )
{
   // highp float displace = dot(texture2D(mDisplaceTexture,vec2(Vertex.x, 1.0 - Vertex.y)).rgb, vec3(0.3, 0.6, 0.1))*0.5;
    //gl_FragColor = texture2D(mCamTexture, Vertex + vec2(displace));
    gl_FragColor = texture2D(mDisplaceTexture, Vertex);
}

This draws a black screen. No GlError thrown. The camera feed, asset pictures are displayed fine with this approach.

Playing displace texture as follows works too:

                gl::color(Color::white());
                gl::draw( mDisplaceTexture, Rectf(vec2(0),getWindowSize() ) );

What shall I look into?

Thanks.

–8

2

Try using sampler2DRect instead. This is required on Windows and OS X, so perhaps also on Android. Movies often use a GL_TEXTURE_RECTANGLE target that uses absolute texture coordinates instead of normalized ones.

3

Thanks Paul.

sampler2DRect breaks the shader with

10-23 17:18:29.667 12346-12391/ru.scriptum.augmentedtheatre A/cinder: |fatal  | void cinder::app::AppBase::executeLaunch()[193] Uncaught exception, type: cinder::gl::GlslProgCompileExc, what: FRAGMENT: Fragment shader compilation failed.
                                                                      ERROR: 0:3: 'sampler2DRect' : Reserved word. 
                                                                      ERROR: 0:3: 'sampler2DRect' : Syntax error:  syntax error
                                                                      ERROR: 2 compilation errors.  No code generated.
                                                                      
                                                                      
                                                                       1: #version 100
                                                                       2: uniform sampler2DRect   mCamTexture;
                                                                       3: uniform sampler2DRect   mDisplaceTexture;
                                                                       4: varying highp vec2	Vertex;
                                                                       5: void main( void )
                                                                       6: {
                                                                       7:     highp float displace = dot(texture2DRect(mDisplaceTexture,Vertex.yx).rgb, vec3(0.3, 0.6, 0.1))*0.5;
                                                                       8:     gl_FragColor = texture2DRect(mCamTexture, Vertex + vec2(displace));    
                                                                       9: }
                                                                      10:

–8

4

I have no experience with this, but here it shows that the texture target is GL_TEXTURE_EXTERNAL_OES, so I would try uniform samplerExternalOES mCamTexture instead.

https://github.com/cinder/Cinder/blob/android_linux/src/cinder/android/video/VideoPlayer.cpp#L70

5

If that is the case, you may have to enable the extension:
#extension GL_OES_EGL_image_external

6

@gabor_papp and @paul.houx : Thanks, guys, but still no luck:

10-23 19:45:28.895 28622-28982/ru.scriptum.augmentedtheatre W/Adreno-ES20: <core_glFramebufferTexture2D:2561>: GL_INVALID_ENUM
10-23 19:45:28.895 28622-28982/ru.scriptum.augmentedtheatre A/libc: Fatal signal 6 (SIGABRT), code -6 in tid 28982 (Thread-2390)

Shader:

#version 100
#extension GL_OES_EGL_image_external : enable
//uniform sampler2D   mCamTexture;
uniform samplerExternalOES   mDisplaceTexture;

varying highp vec2	Vertex;

void main( void )
{
    highp float displace = dot(texture2D(mDisplaceTexture,vec2(Vertex.x, 1.0 - Vertex.y)).rgb, vec3(0.3, 0.6, 0.1))*0.5;
    //gl_FragColor = texture2D(mCamTexture, Vertex + vec2(displace));
    gl_FragColor = texture2D(mDisplaceTexture, Vertex);
}

–8

7

Are you rendering to an Fbo that has this texture bound as a target?

8

No:

Also, just checked the GL_OES_EGL_image_external extension is available.

–8

9

@paul.houx, @gabor_papp Actually, I am taking it back. While banging on this problem, I was trying to convert a texture into a surface into a texture, and that is what was causing this SIGABRT. I removed that craziness, and now everything works.

So, GL_OES_EGL_image_external all the way to the bank.

Thanks a lot.

–8

1 Like