Unable to render an Android WebView into an external texture shared between C++ and Java code

Viewed 2010

I'm currently trying to render an Android WebView content into a texture that can be used in a C++ application using JNI and the NDK. I don't figure out why it doesn't work as I expect.

I've read a lot of documentation all over the web, and here is what I have for now:

C++ side:

// Create the texture
uint texture;
glGenTextures(1, &texture);

// Bind the texture with the proper external texture target
glBindTexture(GL_TEXTURE_EXTERNAL_OES, texture);

// Create the EGLImage object that maps the GraphicBuffer
int usage = GraphicBuffer::USAGE_HW_TEXTURE | GraphicBuffer::USAGE_SW_READ_OFTEN | GraphicBuffer::USAGE_SW_WRITE_OFTEN;
auto gralloc = new GraphicBuffer(width, height, PIXEL_FORMAT_RGBA_8888, usage);
EGLClientBuffer clientBuffer = (EGLClientBuffer) gralloc->getNativeBuffer();

EGLDisplay display = eglGetDisplay(EGL_DEFAULT_DISPLAY);
EGLint attrs[] = { EGL_IMAGE_PRESERVED_KHR, EGL_TRUE, EGL_NONE };

auto eglImage = eglCreateImageKHR(display, EGL_NO_CONTEXT, EGL_NATIVE_BUFFER_ANDROID, clientBuffer, attrs);
glEGLImageTargetTexture2DOES(GL_TEXTURE_EXTERNAL_OES, eglImage);

In the pixel shader, I use the new type of sampler specific to external textures:

// See https://developer.android.com/reference/android/graphics/SurfaceTexture.html
#extension GL_OES_EGL_image_external : require
precision mediump float;

uniform samplerExternalOES uDiffuseMap;
varying vec2 vVertexUV;

void main(void)
{
    gl_FragColor = texture2D(uDiffuseMap, vVertexUV);
}

To check that I can properly sample the texture, I've already did some tests filling the GraphicBuffer with specific data doing like that:

unsigned char *pixels = nullptr;
gralloc->lock(GraphicBuffer::USAGE_SW_READ_OFTEN | GraphicBuffer::USAGE_SW_WRITE_OFTEN, (void **) &pixels);
std::memcpy(pixels, data, width * height * 4);
gralloc->unlock();

And I confirm that it's work as expected, the data written in the GraphicBuffer is the data retrieved when I sample the external texture in the pixel shader.

Now, let's see how it's done Java side to render the WebView in the same texture:

Java side:

CustomRenderer.java (implements GLSurfaceView.Renderer):

@Override
public void onSurfaceChanged(GL10 gl, int width, int height)
{
    // glTextureId is given by the C++ through JNI and correspond 
    // to the ID returns by glGenTextures() call
    surfaceTexture = new SurfaceTexture(glTextureId);
    surfaceTexture.setDefaultBufferSize(width, height);
    surface = new Surface(surfaceTexture);
}

@Override
public void onDrawFrame(GL10 gl)
{
    synchronized(this)
    {
        surfaceTexture.updateTexImage();
    }
}

public Canvas onDrawViewBegin()
{
    surfaceCanvas = surface.lockCanvas(null);
}

public void onDrawViewEnd()
{
    surface.unlockCanvasAndPost(surfaceCanvas);
}

CustomWebView.java (extends WebView)

@Override
public void draw(Canvas canvas) {
    if (customRenderer == null)
    {
        super.draw(canvas);
        return;
    }

    // Returns canvas attached to OpenGL texture to draw on
    Canvas glAttachedCanvas = customRenderer.onDrawViewBegin();

    if (glAttachedCanvas != null)
    {
        // Draw the view to provided canvas
        super.draw(glAttachedCanvas);
    }
    else
    {
        super.draw(canvas);
        return;
    }

    // Notify the canvas is updated
    customRenderer.onDrawViewEnd();
}

I willingly removed all error checks to have the simplest code ever.

The result that I get with this code let me think that the texture is not written by the Java side.

Do you know what happens? Did I do something wrong?

1 Answers
Related