Rendering
This example shows and explain how rendering works in combination with the Wikitude SDK Native API. There are two methods of rendering available in the Wikitude Native SDK. We call them internal and external rendering. Internal means the OpenGL view is setup by the Wikitude SDK and the SDK user can define custom rendering, that is executed by the Wikitude SDK. On the other hand external rendering means the SDK user sets up the OpenGL view and integrates the Wikitude SDK into this rendering setup.
Rendering APIs
The Wikitude Native SDK for Android currently supports the following RenderingAPIs:
- OpenGL ES 3
- OpenGL ES 2
How the Rendering API can be selected can be seen at Rendering API selection for External Rendering or Rendering API selection for Internal Rendering.
External Rendering
To activate external rendering you need to pass an object implementing the ExternalRendering interface to the constructor of the WikitudeSDK class. In the following example this object will be an instance of the ExternalRenderingActivity class which you can find in our sample application under the package com.wikitude.samples.rendering.external.
public class ExternalRenderingActivity extends Activity implements ImageTrackerListener, ExternalRendering {
private WikitudeSDK mWikitudeSDK;
...
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mWikitudeSDK = new WikitudeSDK(this);
In the method onRenderExtensionCreated which is defined by the ExternalRendering interface we receive a RenderExtension instance as a parameter. We pass that parameter to our OpenGL renderer which extends GLSurfaceView.Renderer. We also create a Driver which calls the Renderer 30 times per second to draw the current frame.
@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension) {
mGLRenderer = new GLRenderer(renderExtension);
mView = new CustomSurfaceView(getApplicationContext(), mGLRenderer);
mDriver = new Driver(mView, 30);
setContentView(mView);
}
The following code shows a very basic implementation of a GLSurfaceView.Renderer. Please note that the first thing to do in every method is to call the WikitudeSDK RenderExtension, otherwise the WikitudeSDK won't be able to render the camera frame or perform any image recognition.
public class GLRenderer implements GLSurfaceView.Renderer, RenderExtension {
private RenderExtension mWikitudeRenderExtension = null;
protected Target mCurrentlyRecognizedTarget = null;
private StrokedRectangle mStrokedRectangle = null;
private StrokedRectangle.Type mStrokedRectangleType = StrokedRectangle.Type.STANDARD;
public GLRenderer(RenderExtension wikitudeRenderExtension) {
mWikitudeRenderExtension = wikitudeRenderExtension;
/*
* Until Wikitude SDK version 2.1 onDrawFrame triggered also a logic update inside the SDK core.
* This behaviour is deprecated and onUpdate should be used from now on to update logic inside the SDK core. <br>
*
* The default behaviour is that onDrawFrame also updates logic. <br>
*
* To use the new separated drawing and logic update methods, RenderExtension.useSeparatedRenderAndLogicUpdates should to be called.
* Otherwise the logic will still be updated in onDrawFrame.
*/
mWikitudeRenderExtension.useSeparatedRenderAndLogicUpdates();
}
@Override
public void onDrawFrame(final GL10 unused) {
if (mWikitudeRenderExtension != null) {
// Will trigger a logic update in the SDK
mWikitudeRenderExtension.onUpdate();
// will trigger drawing of the camera frame
mWikitudeRenderExtension.onDrawFrame(unused);
}
if (mCurrentlyRecognizedTarget != null) {
mStrokedRectangle.onDrawFrame(mCurrentlyRecognizedTarget);
}
}
@Override
public void onSurfaceCreated(final GL10 unused, final EGLConfig config) {
if (mWikitudeRenderExtension != null) {
mWikitudeRenderExtension.onSurfaceCreated(unused, config);
}
mStrokedRectangle = new StrokedRectangle(mStrokedRectangleType);
}
@Override
public void onSurfaceChanged(final GL10 unused, final int width, final int height) {
if (mWikitudeRenderExtension != null) {
mWikitudeRenderExtension.onSurfaceChanged(unused, width, height);
}
}
public void onResume() {
if (mWikitudeRenderExtension != null) {
mWikitudeRenderExtension.onResume();
}
}
public void onPause() {
if (mWikitudeRenderExtension != null) {
mWikitudeRenderExtension.onPause();
}
}
public void setCurrentlyRecognizedTarget(final Target currentlyRecognizedTarget) {
mCurrentlyRecognizedTarget = currentlyRecognizedTarget;
}
public void setStrokedRectangleType(StrokedRectangle.Type strokedRectangleType) {
mStrokedRectangleType = strokedRectangleType;
}
}
Rendering API selection for External Rendering
For external rendering the SDK needs to know which Rendering API was used so the correct API calls can be made internally. If this is not explicitly set the SDK will expect an OpenGL ES 2 context.
To set the Rendering api use NativeStartupConfiguration.setRenderingAPI(RenderingAPI...).
Make sure that only one RenderingAPI is set otherwise otherwise an IllegalArgumentException will be thrown once the SDK starts.
Internal Rendering
To activate internal rendering you need to pass an object implementing the InternalRendering interface to the constructor of the WikitudeSDK class. In the following example this object will be an instance of the InternalRenderingActivity which you can find in our sample application under the package com.wikitude.samples.rendering.internal.
public class InternalRenderingActivity extends Activity implements InternalRendering, ImageTrackerListener {
...
private WikitudeSDK mWikitudeSDK;
...
@Override
protected void onCreate(final Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
mWikitudeSDK = new WikitudeSDK(this);
...
In the method provideRenderExtension which is defined by the InternalRendering interface we create an instance of our CustomRenderExtension and return it.
@Override
public RenderExtension provideRenderExtension() {
mRenderExtension = new CustomRenderExtension();
return mRenderExtension;
}
The CustomRenderExtension was defined like in the following code snippet. All defined methods will be called in the appropriate methods of the WikitudeSDK renderer which extends the standard Android GLSurfaceView.Renderer.
public class CustomRenderExtension implements GLSurfaceView.Renderer, RenderExtension {
private Target mCurrentlyRecognizedTarget = null;
private StrokedRectangle mStrokedRectangle;
@Override
public void onDrawFrame(final GL10 unused) {
if (mCurrentlyRecognizedTarget != null) {
mStrokedRectangle.onDrawFrame(mCurrentlyRecognizedTarget);
}
}
@Override
public void onSurfaceCreated(final GL10 unused, final EGLConfig config) {
mStrokedRectangle = new StrokedRectangle();
}
@Override
public void onSurfaceChanged(final GL10 unused, final int width, final int height) {
}
public void onResume() {
}
public void onPause() {
}
@Override
public void useSeparatedRenderAndLogicUpdates() {
}
public void setCurrentlyRecognizedTarget(final ImageTarget currentlyRecognizedTarget) {
mCurrentlyRecognizedTarget = currentlyRecognizedTarget;
}
}
Rendering API selection for Internal Rendering
For internal rendering the SDK needs to know which Rendering API context it shall create. If this is not explicitly set the SDK will create and use an OpenGL ES 2 context.
To set the Rendering api use NativeStartupConfiguration.setRenderingAPI(RenderingAPI...).
The SDK will try to create the set Rendering APIs in the following order: OpenGL ES 3 > OpenGL ES 2.
e.g.:
NativeStartupConfiguration startupConfig = new NativeStartupConfiguration();
// try to create OpenGL ES 3 context, if OpenGL ES 3 is not supported on the device create an OpenGL ES 2 context
startupConfig.setRenderingAPI(RenderSettings.RenderingAPI.OPENGL_ES_3, RenderSettings.RenderingAPI.OPENGL_ES_2);
// try to create OpenGL ES 3 context, if OpenGL ES 3 is not supported on the device there will be undefined behaviour
startupConfig.setRenderingAPI(RenderSettings.RenderingAPI.OPENGL_ES_3);
// try to create OpenGL ES 2 context
startupConfig.setRenderingAPI();
To check which RenderingAPI was created the InternalRendering interface defines a onRenderingApiInstanceCreated(RenderingAPI) callback.
@Override
public void onRenderingApiInstanceCreated(RenderSettings.RenderingAPI renderingAPI) {
String renderingAPIName = renderingAPI == RenderSettings.RenderingAPI.OPENGL_ES_3 ?
"OpenGL ES 3.0" : "OpenGL ES 2.0";
Log.v(TAG, "Rendering connection was created with rendering API " + renderingAPIName);
}