Dev to Dev

3D Tracking for large-scale scenes

Update (August 2017): Object recognition, multi-target tracking and SLAM: Track the world with SDK 7

As part of our mission to keep augmenting the world around us, our tech team has been extending the capabilities of our 3D tracking. We started simple with the first public beta version of this feature a few months back, recognizing and tracking small-scale spaces. As things progress very quickly, Wikitude’s computer vision team is taking a step forward to track and map bigger environments or “large-scale” scenes as we like to call it.

In this post we will provide an insight on Wikitude’s 3D tracking technology, share our next move on the 3D tracking road and provide a hands on demo of our large scale feature, the WikiWings app for Android.

The Wikitude cross-platform 3D Tracking technology

Tracking objects and environments in 3D is a complex task. Particularly if this is done without depth sensors and by using just a single front facing the camera as found in the majority of smartphones out there today. We, humans, are actually at an advantage as we are equipped with two eyes (cameras) for sensing depth and understanding the third dimension.

The Wikitude team has been working tirelessly in the past three years with the aim to create a common base for recognizing and tracking three-dimensional objects, structures, and spaces. The market requirements range from being able to recognize objects of a few centimeters in size up to positioning a device in a sequence of rooms and corridors stretching several 100s of meters. There is no “one” computer vision algorithm out there that can support this broad set of requirements and use cases – yet. As the pioneer in the mobile augmented reality industry with a razor sharp focus on technology only, we will continue to address the demand for varying “flavours” of 3D recognition and tracking. The result of this approach is a strong common core for 3D tracking, which serves as the common base for a number of use cases.

The Wikitude 3D Tracking engine

In the past weeks, another building block of our 3D tracking has evolved, and today we are excited to share with you the second part of our journey to map and track the world around us.

From “small-scale” to “large-scale”

Wikitude’s first 3D tracking beta released a couple of months ago was the initial public release for mapping and tracking objects and environments on a small scale, such as an office desk for example, as previously described in our blog. We have gathered feedback from our developer community in the past weeks and worked on an updated version of our small-scale 3D tracking. Interested developers can request our updated (December 1st, 2015) version of the beta on our 3D tracking page feature page.

As a next step, we are preparing our next public releases for tracking and mapping larger indoor spaces to navigate users, display augmentations in rooms or show points of interest inside buildings.

To demonstrate the basics of our large scale 3D tracking feature already, our development team took the Wikitude office as an initial test environment. The video below is a first hands-on example of the current capabilities of our SLAM based 3D tracking applied to indoor navigation and localization.

The first step of the above demo is to identify objects and physical structures of the room that will provide key feature points to be tracked. As the user moves around, feature points are captured and become the base for the forming map on the device, see the box in the bottom right corner on the device in the video above. Once the algorithm has tracked key features of our office, it’s time to augment the scene! In the technical demo, we demonstrate a simple augmentation of an animated 3D model using our Native iOS SDK.

Large scale in action: use cases with the Wikitude 3D tracking

The demand for technology that “understands” and enhances the larger spaces around us inside of shopping malls, public buildings, airports, train stations etc. is tremendous. Often times we wonder what’s the shortest way to a departure gate inside of airports and train stations, where we can get the best deals in large shopping malls or find the nearest Starbucks, or locate a piece of machinery on complex industrial sites, even indoor gaming is frequently being requested for. Here are some of the many use cases where the Wikitude 3D tracking can be applied.

Mockup 3D Tracking LivingRoom

One of the coolest applications of our large scale 3D tracking is the ability to make rooms highly interactive. This feature allows users to hunt for flying dragons in your living room, fight living creatures in your kitchen or follow an alien in a shopping mall. Any room can become the scene of your game!



What if we could see a design idea of a building structure in real time? Or plan industrial spaces before a single brick is being moved? Architects can use Wikitude’s large scale 3D tracking to display their plans on their client’s tablet, helping them easily visualise what things will look like upon completion of the project.


Indoor navigation (proof of concept)Indoor navigation - augmented reality app

Tracking and mapping indoor spaces enables powerful indoor navigation. Locating deals in the maze of big shopping centers, leading passengers to their boarding gates inside of airports are only the beginning.



Check out WikiWings, Wikitude’s large-scale demo app

The Wikitude large-scale capabilities will be available in our SDK soon, however, you can get an early taste of it already by downloading the WikiWings demo app for Android.

wikiwings logo

Screenshot 3d tracking app

(We have already shot quite a number of dragons at our office ;)

Update (August 2017): Start developing with Wikitude SDK 7

Getting started with SDK 7 has never been easier! Here’s how:

Help us spread the news on Twitter, Facebook and Linkedin using the hashtag #Wikitude.

Dev to Dev

Wikitude SDK and Android Studio

Update (August 2017): A developer insight into Wikitde SDK 7- Object recognition, multi-target tracking and SLAM based instant tracking

Android Studioandroid-studio-logo-1 the new IDE by Google for developing Android apps was announced back at Google I/O 2013. The adapted IDE, which in its core is based on IntelliJ IDEA, has been available since December last year. This year Google I/O also brought news, that the Android NDK will be fully covered as well. Something we are looking forward to use internally.

Google made it clear from the beginning that Android Studio will replace the existing Android Developer Tools (ADT) for Eclipse and that Google intends to cease support for Eclipse somewhen in the future. As we know now since the 26th of June from a blog post by Jamal Eason, Product Manager Android at Google, support for Eclipse will stop entirely by end of this year (2015).

How does that relate to you as a developer?

  • Migrate your projects to Android Studio projects
  • Get familiar with Android Studio. For example, check out the new memory or CPU profiler – an improvement over the DDMS feature in Eclipse
  • Get to know the new build system gradle and say goodbye to ant. You can start using the gradle plugin in Eclipse as a first step of the migration.
  • Have a look at the new parameter applicationID instead of package name
  • The format of Android libraries changes from a Java Archive (.jar) to an Android Archive Library (.aar), this has many advantages especially if you plan to distribute your application using the Google Play Store.

The last point from the list above of course also affects us at Wikitude. Internally we have already made the switch and are purely using Android Studio for any Android application development. For the Wikitude SDK will be do this switch starting with SDK 5, where we are fully embracing Android Studio. The Android library will be shipped as an Android Archive Library (.aar file). Sample projects for Android will be based on Android Studio.
Releasing the SDK by end of July will give you enough time to switch your own projects to Android Studio in time and don’t have to worry that the Wikitude SDK would not work.

Dev to Dev

Moving from PhoneGap to the Cordova Plugin

Hi folks,

Careful followers of Wikitude will have noticed in the past few days, we’ve moved away from PhoneGap and are now offering a Cordova plugin. As many of you know, Apache Cordova is the open-source spin-off of PhoneGap (after the acquisition by Adobe). Fear not, we haven’t given up on PhoneGap, but have renamed the Wikitude PhoneGap Plugin to the Wikitude Cordova Plugin.

PhoneGap vs. Cordova

In the past, the difference between Cordova and PhoneGap has raised more than a few questions and caused a bit of unintentional confusion, especially as both projects walked hand-in-hand in the beginning. Cordova was PhoneGap and PhoneGap was Cordova. Many people tried to compare Cordova to being the engine powering PhoneGap, very similar to Webkit acts as the engine for Safari or other browsers. Personally, I thought that PhoneGap and Cordova were too close, that a good differentiation was possible.

That all changed when PhoneGap introduced PhoneGap Build – the first service offering by the PhoneGap team, allowing to build PhoneGap apps in the cloud. Here, it became readily apparent that PhoneGap was moving towards providing a development framework and additional services based on Cordova.

Being under the Apache umbrella, Cordova is an open-source project to the core. What this means is that every other company can take the Cordova base and start building their own development framework on top of it. And companies did. There’s currently a great selection of development frameworks based on Cordova that all have their specialities and dedicated developer groups.

Look at Intel, who integrated Cordova in their Intel XDK framework, SAP making their SAP Mobile Platform compatible with Cordova or Telerik enabling their AppBuild product with Cordova. And there are are many, many more out there. This is great for providers of PhoneGap Cordova Plugins as this means that all of those frameworks can use and work with your existing plugin.

Screen Shot 2015-05-05 at 14.22.48

For us, it was time to honor this industry shift and reflect it by renaming our plugin to the Wikitude Cordova Plugin. Having a look at the data, Google Trends also thinks that it’s a good idea (see above). You can now find the renamed repositories for the Wikitude Cordova plugin on Github under wikitude-cordova-plugin and wikitude-cordova-plugin-samples.

While the name has changed, the quality you’ve come to know and expect from Wikitude will stay the same.

Head on over to the Download section to give our Cordova Plugin a spin, and let us know what you think in the comments below!


Phil – Wikitude Product Manager

PS: A word on PhoneGap Build – although Adobe relaxed their policy on 3rd party plugins for Build, we’re unable to provide you with direct integration, as our plugin is over the file size limit Adobe has imposed. Once Adobe ups the files size, we’re ready to go.