June 7, 2018
Track, Save and Share: A Developer View on SDK 8More than 6 years ago in April 2012 the very first version of the Wikitude SDK found its way to our first customers. So now, after XYZ releases in total, what can one expect from the 8th major market version of an augmented reality SDK? In short: a lot. Wikitude SDK 8 represents a change as significant as SDK 5 was when we started to offer native APIs for the Wikitude SDK. Some features in this 8th version made it necessary to rethink software architecture choices from the past and re-write major parts of the SDK to give it a structure that supports future challenges and meets the requirements of future projects. In this article, I will share insights about the technical background of the key changes and features that are new in Wikitude SDK 8.
Instant Targets – making augmented reality persistentBack in early 2017, the Wikitude SDK was the first SDK to provide developers with the ability to use Instant Trackers (aka markerless tracking, aka Positional tracking) for a wide range of devices. Some of this functionality is now available as part of our SMART feature, which wraps ARKit and ARCore and Instant Tracking into a single API. Positional tracking is a fantastic capability, but it is not persistent at all. Users can place content freely but will have to re-do this every time they start another session. Wikitude’s Instant Tracking included the ability of visual re-localization right from the beginning (once tracking was lost, it would pick up again in case it detected an area already covered). With SDK 8 Wikitude now introduces the ability to save Instant Tracking sessions in the form of Instant Targets. The API allows you to serialize the current session and use this target to load as a target later on – the same way used with other trackers in the Wikitude SDK. Object Tracker recognizing Object Targets, Image Tracker recognizing Image Targets and now, Instant Tracker recognizing Instant Targets. Instant Targets is a 3D representation of whatever a user tracked in a session – in fact, the file format in which the information is stored is identical to the files generated for Object Tracker (wto). This could be really anything – internally we tried plain images, objects, and smaller scenes with good results.
Reworked AR engineThe introduction of Instant Targets is accompanied by a major upgrade to our Instant Tracking engine – actually to the entire 3D SLAM engine that is powering Instant Tracking, Object Tracking, and all Extended Tracking modes in the background. Anything SLAM-related was revisited and improved – the algorithms have been updated to the latest generation of SLAM algorithms and incorporate findings from recent research. While ARKit and ARCore are deploying sensor-fusion based systems, we saw that there is still a great potential in vision-only systems, that don’t require detailed calibration of camera and IMU. Of course, sensor-fusion systems have their advantages when it comes to fast, rotational-dominant movements like fast 180° turns. Apart from this, the new 3D SLAM engine reaches extremely good results in tracking benchmarks even compared to ARKit and ARCore. In our tests, we could see a reduction of tracking error by up to 60% compared to SDK 7.2. The general benefits of the new SLAM engine you will enjoy are:
- Higher precision of tracking the environment in general
- Instant Tracking more robust for pure lateral movements with low parallax in the beginning
- Tracking can cover greater areas
Going beyond Object Recognition with Extended Object TrackingFor Object Recognition SDK 8 has many changes included. Both the quality and accuracy of the initial recognition and tracking improved substantially, particularly for 360° tracking scenarios. The version of Object Recognition in SDK 8 is fundamentally better compared to our previous versions. Part of the improvements are due to the new 3D SLAM engine, but a great part is also due to the new recording process for Object Targets introduced with SDK 8. Previously, Object Targets were created by uploading a video of the object. While this method worked properly for many of our customers, we identified weaknesses with this approach:
- Extending Object Targets was cumbersome as it required to produce a new video of the entire object
- The size of the video files was limited and as a direct consequence, the video quality could only be average.
- For some objects, it is hard to create a proper video