Documentation

Object Tracking

Introduction to Object and Scene Recognition

Object Recognition and Tracking extend the capabilities of the Wikitude SDK to recognize and track arbitrary objects for augmented reality experiences. The feature is based on Wikitude's SLAM engine that is used throughout the SDK for any kind of tracking the environment. Object Recognition and Tracking let you detect objects and entire scenes, that were pre-defined by you. Suitable objects include

  • Toys
  • Monuments and statues
  • Industrial objects
  • Tools
  • Household supplies

Recognition works best for objects that have only a limited number of changing/dynamic parts.

Scene Recognition

With version SDK 8 the object recognition engine can also be used to recognize larger structures that go beyond table-sized objects. The name Scene recognition reflects this in particular. The new image-based conversion method allows for Object targets that are a lot larger in size, that can be successfully recognized and tracked.

  • Rooms
  • Face of buildings
  • Squares and courtyards

Important: Learn how to create Object Targets

Make sure to read the chapter on how to create Object Targets before using Object Recognition on your own.

Simple Object Tracking

The Simple Object Tracking sample should just give you a rough idea of how object tracking works with the Wikitude Native SDK. We will track a toy fire truck and add a stroked cube as well as an occluder cube to demonstrate its use.

Object Tracking means that instead of tracking an image we track 3-dimensional objects, using information that has been extracted from a video of that object. This is why instead of a Wikitude Target Collection (.wtc) we use a .wto-file (Wikitude Object Target Collection) in this sample.

To use object tracking with the Wikitude Native SDK we will need an ObjectTracker member and implement handlers for its events. We also add a StrokedCube and an OccluderCube as members.

void objectTargetRecognized(wikitude::sdk::uwp::ObjectTracker^ objectTracker_, wikitude::sdk::uwp::ObjectTarget^ objectTarget_);
void objectTargetTracked(wikitude::sdk::uwp::ObjectTracker^ objectTracker_, wikitude::sdk::uwp::ObjectTarget^ objectTarget_);
void objectTargetLost(wikitude::sdk::uwp::ObjectTracker^ objectTracker_, wikitude::sdk::uwp::ObjectTarget^ objectTarget_);
wikitude::sdk::uwp::ObjectTracker ^ _objectTracker;
StrokedCube* _cube;
OccluderCube* _occluder;

When we start the Wikitude Native SDK in OnNavigatedTo we create a TargetCollectionResource with firetruck.wto and hand it over to the TrackerManager::createObjectTracker method to create our object tracker.

page->_resource = page->_sdk->getTrackerManager()->createTargetCollectionResource("file://./app_assets/firetruck.wto");
page->_resource->load(ref new LoadCompletionHandler([wr](bool success_, Error ^ error_) {
    SimpleObjectTrackingPage ^ page = wr.Resolve<SimpleObjectTrackingPage>();
    if (success_) {
        auto objectTracker = page->_sdk->getTrackerManager()->createObjectTracker(page->_resource, nullptr);
        // ...
    }
}));

In the onRender event handler we have to call OccluderCube::Render and StrokedCube::Render, so they will be drawn once the application is running.

void SimpleObjectTrackingPage::onRender() {
    //...
    if (_occluder) {
        _occluder->Render();
    }
    if (_cube) {
        _cube->Render();
    }
}

The object tracker is now ready, but its event handlers aren't there yet. We need to implement a few methods first.

ObjectTracker:RecognizedTarget: events are fired when an object is recognized. Here we instate the OccluderCube and the StrokedCube and dismiss the firetruck image hint.

void SimpleObjectTrackingPage::objectTargetRecognized(wikitude::sdk::uwp::ObjectTracker ^ objectTracker_, wikitude::sdk::uwp::ObjectTarget ^ objectTarget_) {
    dropDownAlert->dismiss();
    auto cameraManager = _sdk->getCameraManager();
    _cube = new StrokedCube(_renderer->deviceResources(), 0.02f);
    _cube->updateVerticalFieldOfView(cameraManager->getVerticalFieldOfView());
    _occluder = new OccluderCube(_renderer->deviceResources());
    _occluder->updateVerticalFieldOfView(cameraManager->getVerticalFieldOfView());
}

The majority of the work is done in ObjectTracker::TrackedTarget event handler. Here we update the scale and position of the OccluderCube and the StrokedCube while we are tracking the fire truck.

void SimpleObjectTrackingPage::objectTargetTracked(wikitude::sdk::uwp::ObjectTracker ^ objectTracker_, wikitude::sdk::uwp::ObjectTarget ^ objectTarget_)
{
    if (_cube) {
        _cube->updateModelMatrix(objectTarget_->getMatrix());
        auto scale = objectTarget_->getTargetScale();
        _cube->setScale(scale.x, scale.y, scale.z);
    }
    if (_occluder) {
        _occluder->updateModelMatrix(objectTarget_->getMatrix());
        auto scale = objectTarget_->getTargetScale();
        _occluder->setScale(scale.x, scale.y, scale.z);
    }
}

The next callback-function to implement is ObjectTracker::LostTarget, which delete the cubes object, and show again the hint.

void SimpleObjectTrackingPage::objectTargetLost(wikitude::sdk::uwp::ObjectTracker ^ objectTracker_, wikitude::sdk::uwp::ObjectTarget ^ objectTarget_)
{
    dropDownAlert->show();
    if (_cube) {
        delete _cube;
        _cube = nullptr;
    }
    if (_occluder) {
        delete _occluder;
        _occluder = nullptr;
    }
}

If you add all this to your code, an orange cube will be displayed around the fire truck if you look at it. Since the occluder cube is always at the same posititon as the cube, you can't see the cubes sides that are farther away from the camera since the occluder blocks the view.

Extended Object Tracking

Warning: This feature is marked as deprecated since 9.12.0 and will be removed in future releases.

Extended tracking is an optional mode you can set for each target separately. In this mode the Wikitude SDK will try to continue to scan the environment of the user even if the original target object is not in view anymore. So the tracking extends beyond the limits of the original target object. The performance of this feature depends on various factors like computing power of the device, background texture and objects.

Based on the previous sample, to enable Extended Tracking for a tracker you need to provide a String vector which defines which targets should be extended. In this sample we simply set a wildcard * so that all targets in this tracker are extended.

auto objectTrackerConfiguration = ref new ObjectTrackerConfiguration;
objectTrackerConfiguration->setExtendedTargets(ref new Platform::Collections::Vector<Platform::String^>{ "*" });

The ObjectTracker class includes the UpdatedExtendedTrackingQuality event which provides updates to the quality of extended tracking. In the sample this is used to show an indicator with the quality.

void ExtendedObjectTrackingPage::onUpdatedExtendedTrackingQuality(wikitude::sdk::uwp::ObjectTracker ^ sender_, wikitude::sdk::uwp::ObjectTarget^ target_, wikitude::sdk::uwp::ExtendedTrackingQuality oldTrackingQuality_, wikitude::sdk::uwp::ExtendedTrackingQuality newTrackingQuality_)
{
    _dispatcher->RunAsync(CoreDispatcherPriority::Normal, ref new DispatchedHandler([=]() {
        auto resourceLoader = Windows::ApplicationModel::Resources::ResourceLoader::GetForCurrentView();
        switch (newTrackingQuality_)
        {
        case ExtendedTrackingQuality::Bad:
            trackingQualityIndicator->Background = static_cast<SolidColorBrush^>(this->Resources->Lookup("bad_red"));
            trackingQualityIndicatorText->Text = resourceLoader->GetString("TrackingQualityBad");
            break;
        case ExtendedTrackingQuality::Average:
            trackingQualityIndicator->Background = static_cast<SolidColorBrush^>(this->Resources->Lookup("avg_yellow"));
            trackingQualityIndicatorText->Text = resourceLoader->GetString("TrackingQualityAverage");
            break;
        case ExtendedTrackingQuality::Good:
            trackingQualityIndicator->Background = static_cast<SolidColorBrush^>(this->Resources->Lookup("good_green"));
            trackingQualityIndicatorText->Text = resourceLoader->GetString("TrackingQualityGood");
            break;
        }
        trackingQualityIndicator->Visibility = Windows::UI::Xaml::Visibility::Visible;
    }));
}

To stop extended tracking, ObjectTracker provides the stopExtendedTracking method. This is helpful if a currently running extended tracking session should be stopped to recognize one of the original targets again. In this example, the method is called when the user taps a Button.

void ExtendedObjectTrackingPage::stopExtendedTracking_Click(Platform::Object^ sender, Windows::UI::Xaml::RoutedEventArgs^ e)
{
    _objectTracker->stopExtendedTracking();
    stopExtendedTracking->Visibility = Windows::UI::Xaml::Visibility::Collapsed;
}