Documentation

Image Recognition

This example shows how to recognize images in the viewfinder and overlay it with images.

For a better understanding, here are some terms that will be used in the following and other section of this documentation related to vision-based augmented reality.

  • Target: A 2D target image and its associated extracted data that is used by the tracker to recognize an image.

  • Target Collection: An archive storing a collection of 2D targets that can be recognized by the tracker. A target collection can hold up to 1000 targets. Target collections are stored as .wtc files

  • ImageTracker: The tracker analyzes the live camera image and detects the 2D targets stored in its associated target collection. Multiple trackers can be created, however only one tracker can be active for recognition at any given time.

Simple Image Tracking Android

In this section we will go through the code of the SimpleClientTrackingActivity, which you can find in the example application under the package com.wikitude.samples.recognition.client. We will discuss general concepts on how to use the Wikitude Native SDK as we go along, please don't skip this section even if you are for example only interested in cloud recognition.

The WikitudeSDK class is structured to be used in a standard Android activity and to make use of the activities life cycle events. We will use interfaces to communicate to the WikitudeSDK which type of rendering method we would like to use and to provide the necessary callbacks for Client- and Cloud-Trackers.

First let us have a look at the declaration of the activity class. 

public class SimpleClientTrackingActivity extends Activity implements ImageTrackerListener, ExternalRendering {
...
}

We subclass the standard Android activity and implement the interfaces ImageTrackerListener and ExternalRendering. Later on when we create the instance of the WikitudeSDK we will pass the this pointer of our activity to the WikitudeSDK constructor and this way indicate our chosen type of rendering. In this example we will use external rendering, for details on how to setup the rendering and the difference between internal and external rendering please read through the section on rendering.

The next step we will take is create an instance of the WikitudeSDK class. This and the NativeStartupConfiguration are the only objects you will need to create by yourself, every other object will be created by factory methods of the WikitudeSDK instance.

IMPORTANT: Do not instantiate any other class out of the Wikitude Native SDK other than the WikitudeSDK and the NativeStartupConfiguration.

We are now going through each method of the SimpleClientTrackingActivity class, starting with onCreate. In the onCreate method we instantiate an instance of the WikitudeSDK, and an instance of the NativeStartupConfiguration, which will hold our license key. If you do not have a license key yet, read this chapter on how to obtain a free trial key. After that we are ready to propagate the onCreate life cycle event to the Wikitude SDK. It is very important that you do propagate onCreate, onPause and onResume otherwise the Wikitude SDK won't be able to properly manage its resources which will lead to unexpected behavior.

After we called the WikitudeSDK onCreate method the SDK is initialized and we are now able to create a TargetCollectionResource and an ImageTracker. To do that we get the TrackerManager from the WikitudeSDK instance and call createTargetCollectionResource passing the url to the WTC file. Since we are loading an asset on the device we indicate that by starting the url with the string file:///android_asset/ and add the path to the file starting from the asset root directory.

To get notified when the Target Collection Resource finished loading, we can implement the second parameter of the createTargetCollectionResource. The onError method is called when the file at the specified url could not be loaded and onFinish is called when loading was successfully completed. At this point we can create the ImageTracker that will use this newly created Target Collection Resource by calling the createImageTracker method from the TrackerManager.

To get notified when the Tracker finished loading, recognized a target and so on we register the Activity which implements ImageTrackerListener as a Listener by passing this as the second parameter to the function.

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    mWikitudeSDK = new WikitudeSDK(this);
    NativeStartupConfiguration startupConfiguration = new NativeStartupConfiguration();
    startupConfiguration.setLicenseKey(WikitudeSDKConstants.WIKITUDE_SDK_KEY);
    startupConfiguration.setCameraPosition(CameraSettings.CameraPosition.BACK);
    startupConfiguration.setCameraResolution(CameraSettings.CameraResolution.AUTO);

    mWikitudeSDK.onCreate(getApplicationContext(), this, startupConfiguration);

    mTargetCollectionResource = mWikitudeSDK.getTrackerManager().createTargetCollectionResource("file:///android_asset/magazine.wtc", new TargetCollectionResourceLoadingCallback() {
                @Override
                public void onError(int errorCode, String errorMessage) {
                    Log.v(TAG, "Failed to load target collection resource. Reason: " + errorMessage);
                }

                @Override
                public void onFinish() {
                    mWikitudeSDK.getTrackerManager().createImageTracker(mTargetCollectionResource, SimpleClientTrackingActivity.this, null);
                }
            });
}

The next important method to look at is the onRenderExtensionCreated method. Since we decided to use the external rendering functionality by implementing ExternalRendering, the WikitudeSDK provides us with a RenderExtension. The RenderExtension interface exposes the same methods as the standard GLSurfaceView.Renderer. In your custom GLSurfaceView.Renderer the first thing to do in every method is to always call the same method on the provided RenderExtension. To be able to that we pass the RenderExtension to the constructor of our Renderer, create our SurfaceView, initialize a Driver and set our SurfaceView as our content view.

@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension) {
    mGLRenderer = new GLRenderer(renderExtension);
    mView = new CustomSurfaceView(getApplicationContext(), mGLRenderer);
    mDriver = new Driver(mView, 30);
    setContentView(mView);
}

Next we will have a look at the ImageTrackerListener interface. The onErrorLoadingTargets method will be called like the name suggest when the Wikitude SDK wasn't able to load the targets from the target collection resource into the image tracker. This is called, for example, when the loading of the file was successful, but the contents were corrupted or somehow incompatible with the image tracker. The onTargetsLoaded method will be called once when the targets were successfully loaded into the tracker. When the image tracker first recognizes a target it will call onImageRecognized providing you with the recognized target name. When the image tracker starts tracking this target it will call onImageTracked continuously until it loses the target and finishes tracking with a call to onImageLost.

The ImageTarget object provided in the onImageTracked method contains information about the tracked target like the name, the distance to the target and most importantly the matrices which describe where on the camera frame the target was found. Since we need this information to draw on the target we pass the object to our renderer in the onImageTracked method and remove it when we lose the target in the onImageLost method.

@Override
public void onTargetsLoaded(ImageTracker tracker) {
    Log.v(TAG, "Image tracker loaded");
}

@Override
public void onErrorLoadingTargets(ImageTracker tracker, int errorCode, final String errorMessage) {
    Log.v(TAG, "Unable to load image tracker. Reason: " + errorMessage);
}

@Override
public void onImageRecognized(ImageTracker tracker, final String targetName) {
    Log.v(TAG, "Recognized target " + targetName);
}

@Override
public void onImageTracked(ImageTracker tracker, final ImageTarget target) {
    mGLRenderer.setCurrentlyRecognizedTarget(target);
}

@Override
public void onImageLost(ImageTracker tracker, final String targetName) {
    Log.v(TAG, "Lost target " + targetName);
    mGLRenderer.setCurrentlyRecognizedTarget(null);
}

Extended Image Tracking Android

Extended tracking is an optional mode you can set for each target separately. In this mode the Wikitude SDK will try to continue to scan the environment of the user even if the original target image is not in view anymore. So the tracking extends beyond the limits of the original target image. The performance of this feature depends on various factors like computing power of the device, background texture and objects.

Based on the previous sample, to enable Extended Tracking for a tracker you need to provide a String array which defines which targets should be extended. In this sample we simply set a wildcard * so that all targets in this tracker are extended. 

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    ...
    mTargetCollectionResource = mWikitudeSDK.getTrackerManager().createTargetCollectionResource("file:///android_asset/iot_tracker.wtc",
            new TargetCollectionResourceLoadingCallback() {
                ...
                @Override
                public void onFinish() {
                    ImageTrackerConfiguration trackerConfiguration = new ImageTrackerConfiguration();
                    trackerConfiguration.setExtendedTargets(new String[]{"*"});
                    mImageTracker = mWikitudeSDK.getTrackerManager().createImageTracker(mTargetCollectionResource, ExtendedClientTrackingActivity.this, trackerConfiguration);
                }
            });
}