Documentation

Object Tracking

Introduction to Object and Scene Recognition

Object Recognition and Tracking extend the capabilities of the Wikitude SDK to recognize and track arbitrary objects for augmented reality experiences. The feature is based on Wikitude's SLAM engine that is used throughout the SDK for any kind of tracking the environment. Object Recognition and Tracking let you detect objects and entire scenes, that were pre-defined by you. Suitable objects include

  • Toys
  • Monuments and statues
  • Industrial objects
  • Tools
  • Household supplies

Recognition works best for objects that have only a limited number of changing/dynamic parts.

Scene Recognition

With version SDK 8 the object recognition engine can also be used to recognize larger structures that go beyond table-sized objects. The name Scene recognition reflects this in particular. The new image-based conversion method allows for Object targets that are a lot larger in size, that can be successfully recognized and tracked.

  • Rooms
  • Face of buildings
  • Squares and courtyards

Important: Learn how to create Object Targets

Make sure to read the chapter on how to create Object Targets before using Object Recognition on your own.

Simple Object Tracking

The Simple Object Tracking sample should just give you a rough idea of how object tracking works with the Wikitude Native SDK. We will track a toy fire truck and add a stroked cube as well as an occluder cube to demonstrate its use.

Object Tracking means that instead of tracking an image we track 3-dimensional objects, using information that has been extracted from a video of that object. This is why instead of a Wikitude Target Collection (.wtc) we use a .wto-file (Wikitude Object Target Collection) in this sample.

To use object tracking with the Wikitude Native SDK we have to make our Activity implement the ObjectTrackerListener interface and we will need to create an ObjectTracker as well as a StrokedCube and an OccluderCube using the GLRenderer class.

public class ObjectTrackingActivity extends Activity implements ObjectTrackerListener, ExternalRendering

After creating and initializing the Wikitude Native SDK object we create a TargetCollectionResource that points to the firetruck.wto (Wikitude Object Target Collection) file and hand it over to the TrackerManager's createObjectTracker method to create the object tracker, passing the Activity as the listener.

final TargetCollectionResource targetCollectionResource = wikitudeSDK.getTrackerManager().createTargetCollectionResource("file:///android_asset/firetruck.wto");
wikitudeSDK.getTrackerManager().createObjectTracker(targetCollectionResource, ObjectTrackingActivity.this, null);

In the onObjectRecognized method we allocate a SrokedCube instance and an OccluderCube instance and register it with the renderer under the name of the recognized target.

@Override
public void onObjectRecognized(ObjectTracker tracker, final ObjectTarget target) {
    [...]

    StrokedCube strokedCube = new StrokedCube();
    OccluderCube occluderCube = new OccluderCube();

    glRenderer.setRenderablesForKey(target.getName(), strokedCube, occluderCube);
}

Correspondingly, the cube instances are unregistered again when the target is lost.

@Override
public void onObjectLost(ObjectTracker tracker, final ObjectTarget target) {
    [...]

    glRenderer.removeRenderablesForKey(target.getName());
}

The cubes are updated within the onObjectTracked method. The instances are retrieved from the renderer using the supplied target name and are subsequently updated using the information stored in the ObjectTarget parameter. The 0.5 offset on the Y-axis accounts for the origin being on the bottom of the object rather than the center in this particular map.

@Override
public void onObjectTracked(ObjectTracker tracker, final ObjectTarget target) {
    StrokedCube strokedCube = (StrokedCube)glRenderer.getRenderableForKey(target.getName());
    if (strokedCube != null) {
        strokedCube.projectionMatrix = target.getProjectionMatrix();
        strokedCube.viewMatrix = target.getViewMatrix();

        strokedCube.setXScale(target.getTargetScale().getX());
        strokedCube.setYScale(target.getTargetScale().getY());
        strokedCube.setZScale(target.getTargetScale().getZ());
    }

    OccluderCube occluderCube = (OccluderCube)glRenderer.getOccluderForKey(target.getName());
    if (occluderCube != null) {
        occluderCube.projectionMatrix = target.getProjectionMatrix();
        occluderCube.viewMatrix = target.getViewMatrix();

        occluderCube.setXScale(target.getTargetScale().getX());
        occluderCube.setYScale(target.getTargetScale().getY());
        occluderCube.setZScale(target.getTargetScale().getZ());
    }
}

Finally, similar to the other samples we implement onTargetsLoaded and onErrorLoadingTargets to simply log to the console to provide information about the loading process.

@Override
public void onTargetsLoaded(ObjectTracker tracker) {
    Log.v(TAG, "Object tracker loaded");
}

@Override
public void onErrorLoadingTargets(ObjectTracker tracker, WikitudeError error) {
    Log.v(TAG, "Unable to load image tracker. Reason: " + error.getMessage());
}

If you add all this to your code, an orange cube will be displayed around the fire truck if you look at it. Since the occluder cube is always at the same position as the cube, you can't see the cubes sides that are farther away from the camera since the occluder blocks the view.

Simple Object Tracking

Extended Object Tracking

Warning: This feature is marked as deprecated since 9.12.0 and will be removed in future releases.

Extended tracking is an optional mode you can set for each target separately. In this mode the Wikitude SDK will try to continue to scan the environment of the user even if the original target object is not in view anymore. So the tracking extends beyond the limits of the original target object. The performance of this feature depends on various factors like computing power of the device, background texture and objects.

Based on the previous sample, to enable Extended Tracking for a tracker you need to provide a String array which defines which targets should be extended. In this sample we simply set a wildcard * so that all targets in this tracker are extended.

@Override
protected void onCreate(Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);
    ...
    final TargetCollectionResource targetCollectionResource = wikitudeSDK.getTrackerManager().createTargetCollectionResource("file:///android_asset/firetruck.wto");
    ObjectTrackerConfiguration trackerConfiguration = new ObjectTrackerConfiguration();
    trackerConfiguration.setExtendedTargets(new String[]{"*"});
    objectTracker = wikitudeSDK.getTrackerManager().createObjectTracker(targetCollectionResource, ExtendedObjectTrackingActivity.this, trackerConfiguration);
}

The ObjectTrackerListener includes the method onExtendedTrackingQualityChanged which provides updates to the quality of extended tracking. In the sample this is used to show an indicator with the quality.

@Override
public void onExtendedTrackingQualityChanged(final ObjectTracker tracker, final ObjectTarget target, final int oldTrackingQuality, final int newTrackingQuality) {
    EditText trackingQualityIndicator = findViewById(R.id.tracking_quality_indicator);
    switch (newTrackingQuality) {
        case -1:
            trackingQualityIndicator.setBackgroundColor(Color.parseColor("#FF3420"));
            trackingQualityIndicator.setText(R.string.tracking_quality_indicator_bad);
            break;
        case 0:
            trackingQualityIndicator.setBackgroundColor(Color.parseColor("#FFD900"));
            trackingQualityIndicator.setText(R.string.tracking_quality_indicator_average);
            break;
        default:
            trackingQualityIndicator.setBackgroundColor(Color.parseColor("#6BFF00"));
            trackingQualityIndicator.setText(R.string.tracking_quality_indicator_good);
    }
    trackingQualityIndicator.setVisibility(View.VISIBLE);
}

To stop extended tracking ObjectTracker.stopExtendedTracking() can be used.

@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension) {

    ...

    stopExtendedTrackingButton = findViewById(R.id.stop_extended_tracking_button);
    stopExtendedTrackingButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(View v) {
            objectTracker.stopExtendedTracking();
        }
    });
}