Instant Tracking
The following sections detail the instant tracking feature of the Wikitude Native SDK by introducing a minimal implementation, showcasing the simplicity the Wikitude Native SDK provides.
Introduction
Instant tracking is an algorithm that, contrary to those previously introduced in the Wikitude SDK, does not aim to recognize a predefined target and start the tracking procedure thereafter, but immediately start tracking in an arbitrary environment. This enables very specific use cases to be implemented.
The algorithm works in two distinct states; the first of which is the initialization state. In this state the user is required to define the origin of the tracking procedure by simply pointing the device and thereby aligning an indicator. Once the alignment is found to be satisfactory by the user (which the users needs to actively confirm), a transition to the tracking state is performed. In this state, the environment is being tracked, which allows for augmentations to be placed within the scene.
The instant tracking algorithm requires another input value to be provided in the initialization state. Specifically, the height of the tracking device above ground is required in order to accurately adjust the scale of augmentations within the scene. To this end, the example features a range input element that allows the height to be set in meters.
During the initialization, another parameter can be set which influences the alignment of the instant tracking ground plane. This ground plane is represented by the initialization indicator and can be rotated in order to start instant tracking at e.g. a wall instead of the floor.
Please refer to the WTInstantTrackerConfiguration reference for detailed information.
Basic Instant Tracking
The Instant Tracking example provides a minimal implementation of the instant tracking algorithm. First we have to make some additions to our UIViewController. In order to use instant tracking, the UIViewController has to conform to the WTInstantTrackerDelegate protocol and we will need a WTInstantTracker and a WTInstantTrackingState as members.
@interface WTInstantTrackerViewController () <WTWikitudeNativeSDKDelegate, WTInstantTrackerDelegate>
@property (nonatomic, strong) WTInstantTracker *instantTracker;
@property (nonatomic, assign) WTInstantTrackingState trackingState;
Initialize both the WTInstantTracker and the WTInstantTrackingState after checking if the WTWikitudeNativeSDK is running in viewDidAppear:.
self.instantTracker = [self.wikitudeSDK.trackerManager createInstantTracker:self configuration:nil];
self.trackingState = WTInstantTrackerInitializing;
A WTInstantTracker can, minimally, be instantiated with just the previously generated tracker instance, although supplying drawables to be rendered in both the initialization state and the tracking state is advisable for any practical use case. Therefore a StrokedRectangle instance is generated.
self.renderableRectangle = [[StrokedRectangle alloc] init];
self.renderableRectangle.scale = 0.45;
To use our StrokedRectangle for the initialization of WTInstantTracker, we implement the instantTracker:didChangeInitializationPose: callback function of the WTInstantTrackerDelegate.
- (void)instantTracker:(nonnull WTInstantTracker *)instantTracker didChangeInitializationPose:(nonnull WTInitializationPose *)pose
{
[self.renderableRectangle setProjectionMatrix:pose.projection];
[self.renderableRectangle setModelViewMatrix:pose.modelView];
}
This callback function supplies a WTInstantTracker and the current WTInitializationPose, the latter of which can be used to set the projection matrix and model view matrix of the StrokedRectangle so it is displayed properly during the initialization state.
Next we need a means to transition from one state to the other. For this task we provide the toggleInstantTrackingState: function which we conveniently call on a button click. self.trackingState holds the value of the current state.
- (IBAction)toggleInstantTrackingState:(id)sender
{
if ( [[self.instantTrackingButton currentTitle] isEqualToString:@"Start Tracking"] )
{
if ( WTInstantTrackerInitializing == self.trackingState )
{
[self.instantTrackingButton setTitle:@"Start Initialization" forState:UIControlStateNormal];
[self.instantTracker setActiveInstantTrackingState:WTInstantTrackerTracking];
}
}
else
{
if ( WTInstantTrackerTracking == self.trackingState )
{
[self.instantTrackingButton setTitle:@"Start Tracking" forState:UIControlStateNormal];
[self.instantTracker setActiveInstantTrackingState:WTInstantTrackerInitializing];
}
}
}
We now draw an orange rectangle while the instant tracker is initializing, but we want the rectangle to be blue once we're in the tracking state.
- (void)instantTracker:(nonnull WTInstantTracker *)instantTracker didChangeState:(WTInstantTrackingState)newState
{
_trackingState = newState;
if (_trackingState == WTInstantTrackerInitializing) {
[self.renderableRectangle setColor:[UIColor colorWithRed:1.00f green:0.58f blue:0.16f alpha:1.0f]];
} else {
[self.renderableRectangle setColor:[UIColor colorWithRed:0.41f green:0.60f blue:0.76f alpha:1.0f]];
}
}
In the WTInstantTracker's callback function instantTracker:didChangeState: we control the color of the StrokedRectangle for a better visual representation. However this only changes the color of the rectangle and not its position, which is why we need the instantTracker:didTrack: callback function to update the projection matrix and the model view matrix of the rectangle.
- (void)instantTracker:(nonnull WTInstantTracker *)instantTracker didTrack:(nonnull WTInstantTarget *)target
{
[self.renderableRectangle setProjectionMatrix:target.projection];
[self.renderableRectangle setModelViewMatrix:target.modelView];
}
Next, we provide the updateDeviceHeightAboveGround: function to set the deviceHeight property of the WTInstantTracker and connect it to a UISlider. While this change is, strictly speaking, not required, we strongly recommend every application to supply the device height accurately by this method or another for the Wikitude SDK to provide an accurate scale.
- (IBAction)updateDeviceHeightAboveGround:(UISlider *)sender
{
[self.instantTracker setDeviceHeightAboveGround:@(sender.value)];
}
Lastly we have to make the height slider invisible when the state changes from initialization to tracking, so set its alpha value correctly in instantTracker:didChangeState:.
dispatch_async(dispatch_get_main_queue(), ^{
if ( WTInstantTrackerTracking == newState )
{
self.deviceHeightAboveGroundSlider.alpha = 0.0;
}
else
{
self.deviceHeightAboveGroundSlider.alpha = 1.0;
}
});
The example outlined in this section renders an orange rectangle augmentation when in initialization state as its indicator and a corresponding blue rectangle augmentation when in tracking state. While the example is quite trivial, we believe it serves the purpose of familiarizing the reader with the core concepts of instant tracking well.
Instant Scene Picking
The instant tracking feature further allows for 3D points to be queried from the underlying point cloud structure. This section is concerned with showcasing this feature based on the corresponding sample of the sample application.
To utilize this feature a 2D input position on the screen is required. A UITapGestureRecognizer is added to the view controller for that purpose.
@interface WTInstantScenePickingViewController () <WTWikitudeNativeSDKDelegate, WTInstantTrackerDelegate, WTExternalOpenGLESRenderingProtocol, UIGestureRecognizerDelegate>
@property (nonatomic, strong) UITapGestureRecognizer *tapGestureRecognizer;
_tapGestureRecognizer = [[UITapGestureRecognizer alloc] initWithTarget:self action:@selector(handleTapFrom:)];
[self.view addGestureRecognizer:_tapGestureRecognizer];
_tapGestureRecognizer.delegate = self;
The handleTapFrom:recognizer function is implemented to query the tap position, scale it using the main screen scale to convert from points to pixels and pass the resulting coordinates to the convertScreenCoordinate:toPointCloudCoordinateOnQueue:completion function. These input coordinates have their origin in the top-left corner of the screen and are within the interval [0, screen_width_in_pixels) and [0, screen_height_in_pixels). Within the completion handler function, a boolean value and a 3D point is received. The former informs on whether the operation completed successfully or not; the latter contains the result in case of success, nil in case of failure. Note that the query will fail whenever no point cloud position can be found for the input coordinate within a specific interval. For successful queries a StrokedCube instance is generated and placed at the resulting position as an augmentation.
CGPoint tapPosition = [recognizer locationInView:self.view];
CGFloat screenScale = [[UIScreen mainScreen] scale];
CGPoint tapPositionScaled = tapPosition;
tapPositionScaled.x *= screenScale;
tapPositionScaled.y *= screenScale;
[_instantTracker convertScreenCoordinate:tapPositionScaled toPointCloudCoordinateOnQueue:[NSOperationQueue currentQueue] completion:^(BOOL success, WTPoint3D* _Nullable pointCloudCoordinate) {
if (pointCloudCoordinate) {
StrokedCube *targetAugmentation = [[StrokedCube alloc] init];
targetAugmentation.uniformScale = 0.05f;
targetAugmentation.xTranslation = pointCloudCoordinate->x;
targetAugmentation.yTranslation = pointCloudCoordinate->y;
targetAugmentation.zTranslation = pointCloudCoordinate->z;
[self.renderableCubes addObject:targetAugmentation];
}
}];
Finally, running the sample allows cube augmentations to be placed on screen touch when in tracking state.
instant scene picking on the floor of the Wikitude offices