Rendering
This example shows and explain how rendering works in combination with the Wikitude SDK Native API. There are two methods of rendering available in the Wikitude Native SDK. We call them internal and external rendering. Internal means the rendering view (OpenGL ES, Metal) is setup by the Wikitude SDK and the SDK user can define custom rendering, that is executed by the Wikitude SDK. On the other hand external rendering means the SDK user sets up the rendering view on his own and integrates the Wikitude SDK into this rendering setup.
Rendering APIs
The Wikitude Native SDK for Android currently supports the following RenderingAPIs:
- OpenGL ES 3
- OpenGL ES 2
How the Rendering API can be selected can be seen at Rendering API selection for External Rendering or Rendering API selection for Internal Rendering.
External Rendering
To activate external rendering you need to pass an object implementing the ExternalRendering
interface to the constructor of the WikitudeSDK
class. In the following example this object will be an instance of the ExternalRenderingActivity
class which you can find in our sample application under the package com.wikitude.samples.rendering.external
.
public class ExternalRenderingActivity extends Activity implements ImageTrackerListener, ExternalRendering {
private WikitudeSDK wikitudeSDK;
...
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
wikitudeSDK = new WikitudeSDK(this);
In the method onRenderExtensionCreated
which is defined by the ExternalRendering
interface we receive a RenderExtension
instance as a parameter. We pass that parameter to our OpenGL renderer which extends GLSurfaceView.Renderer
. We also create a Driver
which calls the Renderer 30 times per second to draw the current frame.
@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension) {
glRenderer = new GLRenderer(renderExtension);
view = new CustomSurfaceView(getApplicationContext(), glRenderer);
driver = new Driver(view, 30);
setContentView(view);
}
The following code shows a very basic implementation of a GLSurfaceView.Renderer. Please note that the first thing to do in every method is to call the WikitudeSDK RenderExtension, otherwise the WikitudeSDK
won't be able to render the camera frame or perform any image recognition.
public class GLRenderer implements GLSurfaceView.Renderer {
private RenderExtension wikitudeRenderExtension = null;
private TreeMap<String, Renderable> mOccluders = null;
private TreeMap<String, Renderable> mRenderables = null;
public GLRenderer(RenderExtension wikitudeRenderExtension) {
wikitudeRenderExtension = wikitudeRenderExtension;
/*
* Until Wikitude SDK version 2.1 onDrawFrame triggered also a logic update inside the SDK core.
* This behaviour is deprecated and onUpdate should be used from now on to update logic inside the SDK core. <br>
*
* The default behaviour is that onDrawFrame also updates logic. <br>
*
* To use the new separated drawing and logic update methods, RenderExtension.useSeparatedRenderAndLogicUpdates should be called.
* Otherwise the logic will still be updated in onDrawFrame.
*/
wikitudeRenderExtension.useSeparatedRenderAndLogicUpdates();
}
@Override
public synchronized void onDrawFrame(final GL10 unused) {
GLES20.glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
GLES20.glClearDepthf(1.0f);
GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT);
if (wikitudeRenderExtension != null) {
// Will trigger a logic update in the SDK
wikitudeRenderExtension.onUpdate();
// will trigger drawing of the camera frame
wikitudeRenderExtension.onDrawFrame(unused);
}
if (mOccluders.size() > 0) {
Iterator itOccluders = mOccluders.entrySet().iterator();
while (itOccluders.hasNext()) {
TreeMap.Entry pairOccluder = (TreeMap.Entry)itOccluders.next();
Renderable renderable = (Renderable)pairOccluder.getValue();
renderable.onDrawFrame();
}
}
if (mRenderables.size() > 0) {
Iterator itRenderables = mRenderables.entrySet().iterator();
while (itRenderables.hasNext()) {
TreeMap.Entry pairRenderables = (TreeMap.Entry)itRenderables.next();
Renderable renderable = (Renderable)pairRenderables.getValue();
renderable.onDrawFrame();
}
}
}
@Override
public void onSurfaceCreated(final GL10 unused, final EGLConfig config) {
if (wikitudeRenderExtension != null) {
wikitudeRenderExtension.onSurfaceCreated(unused, config);
}
mOccluders = new TreeMap<>();
mRenderables = new TreeMap<>();
}
@Override
public void onSurfaceChanged(final GL10 unused, final int width, final int height) {
if (wikitudeRenderExtension != null) {
wikitudeRenderExtension.onSurfaceChanged(unused, width, height);
}
}
public void onResume() {
if (wikitudeRenderExtension != null) {
wikitudeRenderExtension.onResume();
}
}
public void onPause() {
if (wikitudeRenderExtension != null) {
wikitudeRenderExtension.onPause();
}
}
public synchronized void setRenderablesForKey(final String key, final Renderable renderbale, final Renderable occluder) {
if (occluder != null) {
mOccluders.put(key, occluder);
}
mRenderables.put(key, renderbale);
}
public synchronized void removeRenderablesForKey(final String key) {
mRenderables.remove(key);
mOccluders.remove(key);
}
public synchronized void removeAllRenderables() {
mRenderables.clear();
mOccluders.clear();
}
public synchronized Renderable getRenderableForKey(final String key) {
return mRenderables.get(key);
}
public synchronized Renderable getOccluderForKey(final String key) {
return mOccluders.get(key);
}
}
Rendering API selection for External Rendering
For external rendering the SDK needs to know which Rendering API was used so the correct API calls can be made internally. If this is not explicitly set the SDK will expect an OpenGL ES 2 context.
To set the Rendering api use NativeStartupConfiguration.setRenderingAPI(RenderingAPI...).
Make sure that only one RenderingAPI is set otherwise otherwise an IllegalArgumentException will be thrown once the SDK starts.
Internal Rendering
To activate internal rendering you need to pass an object implementing the InternalRendering
interface to the constructor of the WikitudeSDK
class. In the following example this object will be an instance of the InternalRenderingActivity
which you can find in our sample application under the package com.wikitude.samples.rendering.internal
.
public class InternalRenderingActivity extends Activity implements InternalRendering, ImageTrackerListener {
...
private WikitudeSDK wikitudeSDK;
...
@Override
protected void onCreate(final Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
wikitudeSDK = new WikitudeSDK(this);
...
In the method provideRenderExtension
which is defined by the InternalRendering
interface we create an instance of our CustomRenderExtension
and return it.
@Override
public RenderExtension provideRenderExtension() {
renderExtension = new CustomRenderExtension();
return renderExtension;
}
The CustomRenderExtension was defined like in the following code snippet. All defined methods will be called in the appropriate methods of the WikitudeSDK
renderer which extends the standard Android GLSurfaceView.Renderer
.
public class CustomRenderExtension implements GLSurfaceView.Renderer, RenderExtension {
private Target currentlyRecognizedTarget = null;
private StrokedRectangle strokedRectangle;
@Override
public void onDrawFrame(final GL10 unused) {
if (currentlyRecognizedTarget != null) {
strokedRectangle.onDrawFrame(currentlyRecognizedTarget);
}
}
@Override
public void onSurfaceCreated(final GL10 unused, final EGLConfig config) {
strokedRectangle = new StrokedRectangle();
}
@Override
public void onSurfaceChanged(final GL10 unused, final int width, final int height) {
}
public void onResume() {
}
public void onPause() {
}
@Override
public void useSeparatedRenderAndLogicUpdates() {
}
public void setCurrentlyRecognizedTarget(final ImageTarget currentlyRecognizedTarget) {
currentlyRecognizedTarget = currentlyRecognizedTarget;
}
}
Rendering API selection for Internal Rendering
For internal rendering the SDK needs to know which Rendering API context it shall create. If this is not explicitly set the SDK will create and use an OpenGL ES 2 context.
To set the Rendering api use NativeStartupConfiguration.setRenderingAPI(RenderingAPI...).
The SDK will try to create the set Rendering APIs in the following order: OpenGL ES 3 > OpenGL ES 2.
e.g.:
NativeStartupConfiguration startupConfig = new NativeStartupConfiguration();
// try to create OpenGL ES 3 context, if OpenGL ES 3 is not supported on the device create an OpenGL ES 2 context
startupConfig.setRenderingAPI(RenderSettings.RenderingAPI.OPENGL_ES_3, RenderSettings.RenderingAPI.OPENGL_ES_2);
// try to create OpenGL ES 3 context, if OpenGL ES 3 is not supported on the device there will be undefined behaviour
startupConfig.setRenderingAPI(RenderSettings.RenderingAPI.OPENGL_ES_3);
// try to create OpenGL ES 2 context
startupConfig.setRenderingAPI();
To check which RenderingAPI was created the InternalRendering interface defines a onRenderingApiInstanceCreated(RenderingAPI) callback.
@Override
public void onRenderingApiInstanceCreated(RenderSettings.RenderingAPI renderingAPI) {
String renderingAPIName = renderingAPI == RenderSettings.RenderingAPI.OPENGL_ES_3 ?
"OpenGL ES 3.0" : "OpenGL ES 2.0";
Log.v(TAG, "Rendering connection was created with rendering API " + renderingAPIName);
}