June 7, 2018
Track, Save and Share: A Developer View on SDK 8More than 6 years ago in April 2012 the very first version of the Wikitude SDK found its way to our first customers. So now, after XYZ releases in total, what can one expect from the 8th major market version of an augmented reality SDK? In short: a lot. Wikitude SDK 8 represents a change as significant as SDK 5 was when we started to offer native APIs for the Wikitude SDK. Some of the features in this 8th version made it necessary to rethink software architecture choices from the past and re-write major parts of the SDK to give it a structure that supports future challenges and meets the requirements of future projects.
In this article, I will share insights about the technical background of the key changes and features that are new in Wikitude SDK 8.
Instant Targets – making augmented reality persistentBack in early 2017, the Wikitude SDK was the first SDK to provide developers with the ability to use Instant Trackers (aka markerless tracking, aka Positional tracking) for a wide range of devices. Some of this functionality is now available as part of our SMART feature, which wraps ARKit and ARCore and Instant Tracking into a single API. Positional tracking is a fantastic capability, but it is not persistent at all. Users can place content freely but will have to re-do this every time they start another session.
Wikitude’s Instant Tracking included the ability of visual re-localization right from the beginning (once tracking was lost, it would pick up again in case it detected an area already covered). With SDK 8 Wikitude now introduces the ability to save Instant Tracking sessions in the form of Instant Targets. The API allows you to serialize the current session and use this target to load as a target later on – the same way used with other trackers in the Wikitude SDK. Object Tracker recognizing Object Targets, Image Tracker recognizing Image Targets and now, Instant Tracker recognizing Instant Targets.
Reworked AR engineThe introduction of Instant Targets is accompanied by a major upgrade to our Instant Tracking engine – actually to the entire 3D SLAM engine that is powering Instant Tracking, Object Tracking, and all Extended Tracking modes in the background. Anything SLAM-related was revisited and improved – the algorithms have been updated to the latest generation of SLAM algorithms and incorporate findings from recent research.
While ARKit and ARCore are deploying sensor-fusion based systems, we saw that there is still a great potential in vision-only systems, that don’t require detailed calibration of camera and IMU. Of course, sensor-fusion systems have their advantages when it comes to fast, rotational-dominant movements like fast 180° turns.
Apart from this, the new 3D SLAM engine reaches extremely good results in tracking benchmarks even compared to ARKit and ARCore. In our tests, we could see a reduction of tracking error by up to 60% compared to SDK 7.2.
The general benefits of the new SLAM engine you will enjoy are:
- Higher precision of tracking the environment in general
- Instant Tracking more robust for pure lateral movements with low parallax in the beginning
- Tracking can cover greater areas
Going beyond Object Recognition with Extended Object TrackingFor Object Recognition SDK 8 has many changes included. Both the quality and accuracy of the initial recognition and tracking improved substantially, particularly for 360° tracking scenarios. The version of Object Recognition in SDK 8 is fundamentally better compared to our previous versions. Part of the improvements are due to the new 3D SLAM engine, but a great part is also due to the new recording process for Object Targets introduced with SDK 8.
Previously, Object Targets were created by uploading a video of the object. While this method worked properly for many of our customers, we identified weaknesses with this approach:
- Extending Object Targets was cumbersome as it required to produce a new video of the entire object
- The size of the video files was limited and as a direct consequence, the video quality could only be average.
- For some objects, it is hard to create a proper video
With this approach the quality of the resulting Object Targets will already be a lot better compared to the video-based approach (e.g. if you create an Object Target from a video and compare it with an Object Target created from still images of that very same video, the new image-based conversion will produce a more accurate Object Target).
Precise Object Targets obviously also result in higher accuracy. However, the new conversion method made it necessary to adapt the internals of the wto file format and wtos created using image-based conversion will not be backward compatible.
We also saw that many objects from customers consist of mirrored faces or have some sort of symmetry in it. The long sides of the toy fire truck that you find in Wikitude’s sample app are nearly identical but mirrored. The new image-based conversion has been developed so it can detect these cases and identify symmetrical faces correctly.
Speaking of new images: this method also allows you to upload images from the object under different conditions. Using images of the same object in front of a light background and in front of a dark background (or adding variants of an object in direct sunlight and in overcast conditions) will increase the stability of recognition in changing light conditions.
Scene Recognition – tracking LARGE objects and scenesThis brings me to the next great (!) addition of Object Recognition in SDK 8 – Scene recognition. As a direct consequence of the new conversion method, suitable objects can now be considerably larger in size. In our tests, we mapped objects like monuments, entire house facades, and castles that cover areas over 2400 square meters.
Unity Live PreviewAs a result of the Universal SDK, we can now finally offer Live Preview for Unity, for both using attached cameras or Unity Remote for testing your experiences directly within the IDE. No more building and transferring your app to an attached device for your testing. Live Preview works for all tracker types – even Instant Tracking using the Unity Remote application. Unity developers can now work using this feature on their macOS or Windows machines.
Some additional cool featuresThe camera obviously plays a vital role in every augmented reality experience. With the Wikitude SDK, you already have several ways to control the camera streams available on a device. From changing camera (back/front) to setting focal distances, there already is a great variety available. With SDK 8 we are adding two additional items that you can manually control if needed. Developers can now tell the camera which area of the camera image should be used to calculate exposure time and set focus. In a sample, we demonstrate that usage in a tap-to-focus scenario.
With this release, Android developers can use Gradle 3 (used in Android Studio 3) without any limitations.
Endless AR possibilitiesWikitude SDK 8 is a big step for us to make the life of AR developers easier and help them create new ways of AR experiences. We are proud to announce the beta version of SDK 8 today and will continue to increase stability and performance in the upcoming weeks.
Start developing with Wikitude SDK 8Getting started with SDK 8 has never been easier! Here’s how:
Download SDK 8 and sample app (included in the download package)
Select your license plan
Got questions? Our developers are here for you!
Help us spread the news on Twitter, Facebook and Linkedin using the hashtag #SDK8 and #Wikitude.