Category: SDK releases

Update: It has been some time since the launch of Wikitude – Unity 3D Beta Plugin. Check out the updates on all AR development frameworks supported by Wikitude.

The SDK 5 is now available for download!

In the past months our development focus for the Wikitude SDK was on executing our ambitious plans for the next major version – 5.0, released at the end of August. Today we are lifting the curtain on one particular feature: the Wikitude SDK Plugins API!

Over and over customers approached us with the desire to enhance the functionality of the Wikitude SDK with features from other areas of computer vision. Optical character recognition (OCR), face detection and recognition of QR codes were demanded repeatedly. All these features have two things in common: first they also require access to the camera image, and second they can compliment the AR experience built with the Wikitude SDK.

Embedding and integrating some of those libraries on our own would have been an option, but it would have bloated the SDK, and we were sure, that we wouldn’t be able to cover all customer requirements. The idea then was born to link external libraries to the SDK through a common interface, which then lead to the concept of plug-ins for the wikitude SDK.

Technically a plugin is a class, either written in C++, Java or ObjC, that is derived from the wikitude Plugin base class. Beside lifecycle handling and options to enable and disable the plugin, the Plugin class has two main methods that you can override

cameraFrameAvailable, which is called each time the camera has a new frame
update, which is called each time the wikitude SDK renders a new frame

/* Derive from this class for custom plugin implementations */
class Plugin {
   public:
      Plugin();
      ~Plugin();
      string identifier() const; // returns a unique plugin identifier
      bool processesColorCameraFrames(); // returns true if the plugins wants to process color frames instead of bw
 
      void setEnabled(bool enabled_);
      bool isEnabled();
 
      string callJavaScript(string javaScriptSnippet); // evaluates the given JavaScript snippet in the currently loaded ARchitect World context.
 
   protected:
      void initialize(); // called when the plugin is initially added to the Wikitude SDK
      void pause(); // called when the Wikitude SDK is paused e.g. the application state changes from active to background
      void resume(uint pausedTime_); // called when the Wikitude SDK resumes e.g. from background to active state. pausedTime represents the time in milliseconds that the plugin was not updated.
      void destroy(); // called when the plugin is removed from the Wikitude SDK
 
      void cameraFrameAvailable(const Frame&; cameraFrame_); // called each time the camera has a new frame
      void update(const vector recognizedTargets_); // called each time the Wikitude SDK renders a new frame
 
   protected:
      string      _identifier;
      bool        _enabled;
}; 

With those methods in place your plugin will be able to read the full camera image for your own purpose, where the YUV image is also processed in Wikitude’s computer vision engine.

In case you have the wikitude SDK running with ongoing image recognition, the plugin API will populate the RecognizedTarget in the update method once an image has been recognized. The plugin can then work with class RecognizedTarget, which wraps the details of the target image in the camera view. With that you can read out the pose of the target image and use it for your purposes. Additionally, the call contains the calculated distance to the recognized target.

class RecognizedTarget {
   public:
      const string&    getIdentifier() const; // the identifier of the target. The identifier is defined when the target is added to a target collection
      const Mat4&      getModelViewMatrix() const; // the model view matrix that defines the transformation of the target in the camera frame (translation, rotation, scale)
      const Mat4&      getProjectionMatrix() const;
      const float      getDistanceToCamera() const; // represents the distance from the target to the camera in millimeter
};

Passing values from within the plugin to the JavaScript part of your augmented reality experience is done via the addToJavaScriptQueue() method of the Plugin class. Using this function will execute any JavaScript code in the context of your augmented reality experience.

We hope you like the first release of the Plugins API and can build powerful extensions for the Wikitude SDK. We already have ideas how to further develop the concept like generic Positionables, which you can pass to the Wikitude SDK (e.g. pose of something you recognize and track) or sharing the render loop with plugins.

The Wikitude SDK version 4.1.1 is now available for download. This stable release is purely a maintenance and stability update, with no new added features. The update is available for Android and iOS platform and can be downloaded from www.wikitude.com/download

We highly recommend developers using 4.1.0 install this update to ensure optimal results. If you’re using an older version, please see the migration guides for Android, and/or iOS in our documentation “how to upgrade to 4.1.x”.

The update also includes a revamped iOS setup guide, a guide on how to include the Wikitude SDK in Android Studio and a setup guide for using The Wikitude Cordova plugin with Intel XDK.

A big thanks to all that have reported outstanding issues and feedback.

Changelog for the Wikitude SDK 4.1.1

Fixes
- Windows version of 3D Encoder
- `onError` handler now has more details about what failed
- Loading of ClientTracker works as expected when no `onLoaded` trigger is set
- openInBrowser can load .html files from external storage (Android only)
- an issue with inverted axis when using image recognition in combination with the front camera (iOS only)
Updated
- Documentation clarifications and fixes
- Terms of Service