Client Recognition

This example shows how to recognize images in the viewfinder and overlay it with images.

For a better understanding, here are some terms that will be used in the following and other section of this documentation related to vision-based augmented reality.

Simple 2D Client Tracking iOS

In this section we will go through the WTSimpleClientTrackerViewController code, which you can find in the example applications folder Native Examples/Controller/ClientTracking. We will discuss general concepts on how to use the Wikitude Native SDK as we go along, please don't skip this sample even if you are only interested in cloud recognition for example.

Please refer to the Setup Guide section of this documentation to find detailed information on how to integrate the Wikitude Native SDK into a Xcode project.

In this example we will use external rendering, for details on how to setup the rendering and the difference between internal and external rendering please read through the section on rendering.

The WTWikitudeNativeSDK class is structured to be used in a standard iOS view controller as a model of the MVC pattern. This class is central starting point for anything related to the Wikitude Native SDK. It provides factory methods to create different kinds of tracker and to receive information of those through there delegate objects.

A typical usage would be to have a strong reference to a WTWikitudeNativeSDK object in a view controller and to define this view controller to be conform to the WTWikitudeNativeSDKDelegate protocol

@interface WTSimpleClientTrackerViewController () <WTWikitudeNativeSDKDelegate, WTClientTrackerDelegate>

The WTWikitudeNativeSDKDelegate protocol defines methods that can be used to react on SDK specific events. It is the chosen way to communicate e.g. runtime errors that can not be returned directly from an API call. Such errors might contain information about camera authorization status changes and inconsistent rendering setups.

To receive client tracker information about recognized/tracked or lost targets, this view controller is also conform to the WTClientTrackerDelegate protocol. All methods in the mentioned protocol are optionals thus only provide information from the SDK but don't require any action from the application. Additionally this protocol provides information about the loading status of a particular client tracker object.

The next step we will take is create an instance of the WTWikitudeNativeSDK class. This is the only object that you have to create manually. All other object will be created by factory methods of the WTWikitudeNativeSDK object.

IMPORTANT: Do not instantiate any other class out of the Wikitude Native SDK other than the WTWikitudeNativeSDK manually. All object somehow play together in a well defined manner and that why they are internally controlled by the WTWikitudeNativeSDK object and its sub components like WTTrackerManger and WTCaptureDeviceManager.

A WTWikitudeNativeSDK object can be instantiated at any time and does not depend on any particular application lifecycle event. To integrate internal processes into the hosting application, -start:completion and -stop are used. These two methods should either be called from - viewWillAppear: | - viewWillDisappear: or -viewDidAppear: | -viewDidDisappear:. To refine the startup behaviour of the Wikitude Native SDK, the first block parameter of -start:completion: contains an already created and setup object of type WTStartupConfiguration. Public properties of this object can be used to further define how the Wikitude Native SDK should start e.g. its camera. Typical settings from this object are the camera preset which defines in which resolution the device camera should be started or what's the desired target rendering frame rate. The second block parameter contains a BOOL indicating if the given properties could be taken into account during the final startup phase. If an error occurred, isRunning is set to NO and the given NSError object contains more information about what went wrong.

The second block parameter is also the ideal starting point to create any tracker object from the Wikitude Native SDK if the running state is YES. To load client tracker objects, a target collection is needed. These target collections are represented through a .wtc file (Wikitude Target Collection). The .wtc file can be packed into the application bundle and a file URL retrieved using the NSBundle method -URLForResource:withExtension:subdirectory: or uploaded to a server and downloaded from the Wikitude Native SDK.

To get notified when the client tracker finished loading, its delegate method -clientTracker:didFinishedLoadingTargetCollectionFromURL: is called or -clientTracker:didFailToLoadTargetCollectionFromURL:withError: if the .wtc could not be loaded.

[self.wikitudeSDK start:^(WTStartupConfiguration *startupConfiguration) {
        startupConfiguration.captureDevicePosition = AVCaptureDevicePositionFront;
    } completion:^(BOOL isRunning, NSError * __nonnull error) {
        if ( !isRunning ) {
            NSLog(@"Wikitude SDK is not running. Reason: %@", [error localizedDescription]);
        }
        else
        {
            NSURL *clientTrackerURL = [[NSBundle mainBundle] URLForResource:@"magazine" withExtension:@"wtc" subdirectory:@"Assets"];
            self.clientTracker = [self.wikitudeSDK.trackerManager createClientTrackerFromURL:clientTrackerURL extendedTargets:nil andDelegate:self];
        }
    }];

Next we will have a closer look on how to get on tracker specific data. The WTClientTrackerDelegate provides three methods that will be called when the tracker found, lost or tracked targets.

The WTImageTarget object provides information about which target changed and what changed for the target, e.g. its position which would be represented through a OpenGL like model view matrix.

- (void)baseTracker:(nonnull WTBaseTracker *)baseTracker didRecognizedTarget:(nonnull WTImageTarget *)recognizedTarget
{
    NSLog(@"recognized target '%@'", [recognizedTarget name]);
    _isTracking = YES;
}

- (void)baseTrakcer:(nonnull WTBaseTracker *)baseTracker didTrackTarget:(nonnull WTImageTarget *)trackedTarget
{
    [self.renderableRectangle setProjectionMatrix:trackedTarget.projection];
    [self.renderableRectangle setModelViewMatrix:trackedTarget.modelView];
}

- (void)baseTracker:(nonnull WTBaseTracker *)baseTracker didLostTarget:(nonnull WTImageTarget *)lostTarget
{
    NSLog(@"lost target '%@'", [lostTarget name]);
    _isTracking = NO;
}


- (void)clientTracker:(nonnull WTClientTracker *)clientTracker didFinishedLoadingTargetCollectionFromURL:(nonnull NSURL *)URL
{
    NSLog(@"Client tracker loaded");
}

- (void)clientTracker:(nonnull WTClientTracker *)clientTracker didFailToLoadTargetCollectionFromURL:(nonnull NSURL *)URL withError:(nonnull NSError *)error
{
    NSLog(@"Unable to load client tracker. Reason: %@", [error localizedDescription]);
}

Extended 2D Client Tracking iOS

Based on the above to enable Extended Tracking for a tracker you need to provide a String array which defines which targets should be extended. In the sample we simply set a wildcard * so that all targets in this tracker are extended. 

self.clientTracker = [self.wikitudeSDK.trackerManager createClientTrackerFromURL:clientTrackerURL extendedTargets:@[@"*"] andDelegate:self];

3D Client Tracking iOS

In this example we will take a look at how to use the WTTrackingMapRecorder and 3D tracking configured WTClientTracker to do 3D Tracking. If you haven't already done so please read through the first section on 2D Tracking before continuing here. Essential concepts like how to setup rendering or how to use a Wikitude Tracker are explained there and won't be repeated here. Similar if you don't already know why we need to record a Tracking Map or what to expect from 3D Tracking in general please read through the introduction to 3D tracking.

Depending of the use case, an App that uses the Wikitude 3D Tracking technology might include an already recorded Tracking Map or needs to create such a Map each time the App is used. If a certain object should be recognisable, than it is not necessary to let the user record a new Tracking Map each time the App is used. A map can be recorded using this example applications Record Map functionality and packaged into the final App. As this example does not require a certain object to be available for 3D Tracking, a new Tracking Map is recorded every time, including objects that are currently available in your vicinity.

The first change in this example is already at the end of -viewDidLoad. The delegate object of WTTrackingMapRecorder is set to this view controller. Objects of type WTTrackingMapRecorder should not be created manually but only retrieved through the factory method of a WTWikitudeNativeSDK object. The WTTrackingMapRecorderDelegate protocol offers methods to get information about a running Tracking Map recording session. We use those methods in this example to indicate when a new Tracking Map contains enough information and the recording session can be stopped.

- (void)viewDidLoad
{
    [super viewDidLoad];

     // ... Wikitude SDK initialization ...

    [[self.wikitudeSDK trackingMapRecorder] setDelegate:self];
}

To start a Tracking Map recording session, WTTrackingMapRecorder offers -startTrackingMapRecording:. This method calls a block of type WTTrackingMapRecordingStartupHandler once the recording session started. The block object offers more information about if the session could be started and if not, why. The Tracking Map recording is started once the start button of a UIAlertAction is tapped.

- (void)startTrackingMapRecordingInternal
{
    [self setNavigationItemPrompt:nil withNavigationBarTintColor:nil];

    if ( ![[self.wikitudeSDK trackingMapRecorder] isRecording] )
    {
        __weak typeof(self) weakSelf = self;
        [[self.wikitudeSDK trackingMapRecorder] startTrackingMapRecording:^(BOOL isRecording, NSError * _Nullable error) {
            if ( !isRecording )
            {
               // ... react to start issues. The NSError objects contains more information about the issue
            }
            else
            {
                // ... the recording session is running and a Tracking Map is recording ...
            }
        }];
    }
}

During the recording, the WTTrackingMapRecorderDelegate method -trackingMapRecorder:didChangeTrackingMapRecordingQualityFrom:toQuality: is called every time the quality of the new Tracking Map changes. Recording should continue until enough information could be gathered. The more informations are included in a Tracking Map, the more likely it is to recognise objects from the recorded scene in a later detection phase. In this example, the navigation bars tint color and prompt are updated accordingly.

- (void)trackingMapRecorder:(WTTrackingMapRecorder *)trackingMapRecorder didChangeTrackingMapRecordingQualityFrom:(WTTrackingMapRecordingQuality)oldTrackingQuality toQuality:(WTTrackingMapRecordingQuality)newTrackingQuality
{
    UIColor *trackingMapRecordingQualityBarTintColor = nil;
    NSString *trackingMapRecordingQualityText = nil;

    _currentTrackingMapRecordingQuality = newTrackingQuality;

    if ( WTTrackingMapRecordingQuality_Bad == newTrackingQuality )
    {
        trackingMapRecordingQualityBarTintColor = [UIColor colorWithRed:1 green:0.204 blue:0.125 alpha:1];
        trackingMapRecordingQualityText = @"Bad";
    }
    else if ( WTTrackingMapRecordingQuality_Average == newTrackingQuality )
    {
        trackingMapRecordingQualityBarTintColor = [UIColor colorWithRed:1 green:0.851 blue:0 alpha:1];
        trackingMapRecordingQualityText = @"Average";
    }
    else
    {
        trackingMapRecordingQualityBarTintColor = [UIColor colorWithRed:0.420 green:1 blue:0 alpha:1];
        trackingMapRecordingQualityText = @"Good";
    }

    if ( trackingMapRecordingQualityBarTintColor
        &&
        trackingMapRecordingQualityText )
    {
        [self setNavigationItemPrompt:[NSString stringWithFormat:@"Recorded Tracking Map Quality: %@", trackingMapRecordingQualityText] withNavigationBarTintColor:trackingMapRecordingQualityBarTintColor];
    }
}

To stop a running Tracking Map recording session, -stopTrackingMapRecording:completion: can be used. This method requires the name of the Tracking Map file that will be written and a WTTrackingMapRecordingCompletionHandler completion handler that is called once the save operation finished. The completion handler contains more information about if the file could actually be written and, in case of success, also returns it's final file URL.

[[self.wikitudeSDK trackingMapRecorder] stopTrackingMapRecording:@"3d_tracking_map" completion:^(NSURL * _Nullable trackingMapURL, NSError * _Nullable error) {

    [weakSelf setNavigationItemPrompt:nil withNavigationBarTintColor:nil];
    if ( nil == trackingMapURL )
    {
        NSLog(@"Unable to save recorded Tracking Map. Reason: %@", [error localizedDescription]);
    }
    else
    {
        [weakSelf.navigationItem setRightBarButtonItem:nil animated:YES];
        weakSelf.clientTracker = [[weakSelf.wikitudeSDK trackerManager] create3DClientTrackerFromURL:trackingMapURL andDelegate:weakSelf];
    }
}];

The Tracking Map save operation is executed asynchrony in a background thread and might take some time to finish. The completion handler is called once this background activity finished. In this example we use the completion handler to immediately load the recorded Tracking Map using the WTTrackerManager factory method -create3DClientTrackerFromURL:andDelegate:. Once the WTClientTracker object is created, it can be used as an already known 2d WTClientTracker.