--- Android --- Android Javascript API Android Native API --- iOS --- iOS Javascript API iOS Native API --- Windows --- UWP Native API --- Unity --- Unity HoloLens Unity --- Extensions --- Flutter Cordova Xamarin --- Services --- Studio User Manual Studio API --- Expert Edition --- Expert Edition Unity

Wikitude SDK iOS Native API

Object Tracking

Introduction to Object and Scene Recognition

Object Recognition and Tracking extend the capabilities of the Wikitude SDK to recognize and track arbitrary objects for augmented reality experiences. The feature is based on Wikitude's SLAM engine that is used throughout the SDK for any kind of tracking the environment. Object Recognition and Tracking let you detect objects and entire scenes, that were pre-defined by you. Suitable objects include

• Toys
• Monuments and statues
• Industrial objects
• Tools
• Household supplies

Recognition works best for objects that have only a limited number of changing/dynamic parts.

Scene Recognition

With version SDK 8 the object recognition engine can also be used to recognize larger structures that go beyond table-sized objects. The name Scene recognition reflects this in particular. The new image-based conversion method allows for Object targets that are a lot larger in size, that can be successfully recognized and tracked.

• Rooms
• Face of buildings
• Squares and courtyards

Simple Object Tracking

The Simple Object Tracking sample should just give you a rough idea of how object tracking works with the Wikitude Native SDK. We will track a toy fire truck and add a stroked cube as well as an occluder cube to demonstrate its use.

Object Tracking means that instead of tracking an image we track 3-dimensional objects, using information that has been extracted from a video of that object. This is why instead of a Wikitude Target Collection (.wtc) we use a .wto-file (Wikitude Object Target Collection) in this sample.

To use object tracking with the Wikitude Native SDK we have to make our UIViewController conform to the WTObjectTrackerDelegate protocol and we will need a WTObjectTracker member. We also add a StrokedCube and an OccluderCube as members.

@interface WTSimpleObjectTrackerViewController () <WTWikitudeNativeSDKDelegate, WTExternalOpenGLESRenderingProtocol, WTObjectTrackerDelegate>

In -viewDidLoad:, default initialize the StrokedCube and the OccluderCube and set up a WTTargetImageView with the image of the fire truck.

When we start the Wikitude Native SDK in -viewDidAppear: we create a WTTargetCollectionResource with firetruck.wto and hand it over to the WTTrackerManager's -createObjectTrackerFromTargetCollectionResource:delegate: method to create our object tracker.

NSURL *objectTargetCollectionResourceURL = [[NSBundle mainBundle] URLForResource:@"firetruck" withExtension:@"wto" subdirectory:@"Assets"];
self.targetCollectionResource = [self.wikitudeSDK.trackerManager createTargetCollectionResourceFromURL:objectTargetCollectionResourceURL completion:^(BOOL success, NSError * _Nullable error) {
    if ( !success )
    {
        NSLog(@"Failed to load URL resource. Reason: %@", [error localizedDescription]);
    }
    else
    {
        weakSelf.objectTracker = [weakSelf.wikitudeSDK.trackerManager createObjectTrackerFromTargetCollectionResource:weakSelf.targetCollectionResource delegate:weakSelf];
    }
}];

In the -renderBlock method we have to add self.occluderCube and self.renderableCube to the render call, so they will be drawn once the application is running.

[self.occluderCube drawInContext:[self.renderer internalContext]];
[self.renderableCube drawInContext:[self.renderer internalContext]];

The object tracker is now ready, but its delegate (our UIViewController) isn't there yet, because it doesn't conform to the WTObjectTrackerDelegate-protocol. We need to implement a few callback methods first.

-objectTracker:didRecognizeObject: gets called when an object is recognized. We then set self.isTracking to true, hide WTTargetImageView and log the name of the object to the console.

- (void)objectTracker:(WTObjectTracker *)objectTracker didRecognizeObject:(WTObjectTarget *)recognizedObject
{
    NSLog(@"recognized object '%@'", [recognizedObject name]);
    _isTracking = YES;
    [self.targetImageView hide:YES];
}

The majority of the work is done in -objectTracker:didTrackObject:. Here we update the scale and position of the WTOccluderCube and the WTStrokedCube while we are tracking the fire truck.

- (void)objectTracker:(WTObjectTracker *)objectTracker didTrackObject:(WTObjectTarget *)trackedObject
{
    self.occluderCube.yTranslation = 0.5f;

    self.occluderCube.xScale = trackedObject.xScale;
    self.occluderCube.yScale = trackedObject.yScale;
    self.occluderCube.zScale = trackedObject.zScale;

    [self.occluderCube setProjectionMatrix:trackedObject.projection];
    [self.occluderCube setModelViewMatrix:trackedObject.modelView];

    self.renderableCube.yTranslation = 0.5f;

    self.renderableCube.xScale = trackedObject.xScale;
    self.renderableCube.yScale = trackedObject.yScale;
    self.renderableCube.zScale = trackedObject.zScale;

    [self.renderableCube setProjectionMatrix:trackedObject.projection];
    [self.renderableCube setModelViewMatrix:trackedObject.modelView];
}

The next callback-function to implement is -objectTracker:didLoseObject:, which logs the name of the lost object to the console and sets self.isTracking to false again.

- (void)objectTracker:(WTObjectTracker *)objectTracker didLoseObject:(WTObjectTarget *)lostObject
{
    NSLog(@"lost object '%@'", [lostObject name]);
    _isTracking = NO;
}

Finally, similar to the other samples we implement -objectTrackerDidLoadTargets: and -objectTracker:didFailToLoadTargets: by simply logging to the console what happened.

- (void)objectTrackerDidLoadTargets:(WTObjectTracker *)objectTracker
{
    NSLog(@"Object tracker loaded");
}

- (void)objectTracker:(WTObjectTracker *)objectTracker didFailToLoadTargets:(NSError *)error
{
    NSLog(@"Unable to load object tracker. Reason: %@", [error localizedDescription]);
}

If you add all this to your code, an orange cube will be displayed around the fire truck if you look at it. Since the occluder cube is always at the same posititon as the cube, you can't see the cubes sides that are farther away from the camera since the occluder blocks the view.

Extended Object Tracking

Extended tracking is an optional mode you can set for each target separately. In this mode the Wikitude SDK will try to continue to scan the environment of the user even if the original target object is not in view anymore. So the tracking extends beyond the limits of the original target object. The performance of this feature depends on various factors like computing power of the device, background texture and objects.

Based on the previous sample, to enable Extended Tracking for a tracker you need to provide a String array which defines which targets should be extended. In this sample we simply set a wildcard * so that all targets in this tracker are extended.

weakSelf.objectTracker = [weakSelf.wikitudeSDK.trackerManager createObjectTrackerFromTargetCollectionResource:weakSelf.targetCollectionResource delegate:weakSelf configuration:^(WTObjectTrackerConfiguration *objectTrackerConfiguration) {
    objectTrackerConfiguration.extendedTargets = @[@"*"]; /* Consider all object targets as extended */
}];

The WTObjectTrackerDelegate protocol includes the method -objectTracker:didUpdateExtendedTrackingQualityForTarget:fromQuality:toQuality: which provides updates to the quality of extended tracking. In the sample this is used to show an indicator with the quality.

- (void)objectTracker:(nonnull WTObjectTracker *)objectTracker didUpdateExtendedTrackingQualityForTarget:(NSString *)extendedTarget fromQuality:(WTExtendedTrackingQuality)oldTrackingQuality toQuality:(WTExtendedTrackingQuality)newTrackingQuality
{
    UIColor *extendedTrackingQualityBarTintColor = nil;
    NSString *extendedTrackingQualityText = nil;    

    if ( WTExtendedTrackingQuality_Bad == newTrackingQuality )
    {
        extendedTrackingQualityBarTintColor = [UIColor colorWithRed:1 green:0.204 blue:0.125 alpha:1];
        extendedTrackingQualityText = @"Bad";
    }
    else if ( WTExtendedTrackingQuality_Average == newTrackingQuality )
    {
        extendedTrackingQualityBarTintColor = [UIColor colorWithRed:1 green:0.851 blue:0 alpha:1];
        extendedTrackingQualityText = @"Average";
    }
    else
    {
        extendedTrackingQualityBarTintColor = [UIColor colorWithRed:0.420 green:1 blue:0 alpha:1];
        extendedTrackingQualityText = @"Good";
    }

    if ( extendedTrackingQualityBarTintColor
        &&
        extendedTrackingQualityText)
    {
        dispatch_async(dispatch_get_main_queue(), ^{
            [self.navigationController.navigationBar setBarTintColor:extendedTrackingQualityBarTintColor];
            [self.navigationItem setPrompt:[NSString stringWithFormat:@"Extended tracking quality: %@", extendedTrackingQualityText]];
        });
    }
}

To stop extended tracking, WTObjectTracker provides -stopExtendedTracking. This is helpful if a currently running extended tracking session should be stopped to recognize one of the original targets again. In this example, the method is called when the user taps a UIButton.

- (IBAction)stopExtendedTracking:(UIButton *)button
{
    [self.objectTracker stopExtendedTracking];
    [self updateExtendedTrackingVisibility:NO];
}