--- Android --- Android Javascript API Android Native API --- iOS --- iOS Javascript API iOS Native API --- Windows --- UWP Native API --- Unity --- Unity HoloLens Unity --- Extensions --- Flutter Cordova Xamarin --- Smart Glasses --- Epson Vuzix --- Services --- Studio User Manual Studio API --- Expert Edition --- Expert Edition Unity

Wikitude SDK Epson

Plugins API

This guide consists of multiple sections, first we discuss Wikitude SDK Plugins in general, than we talk about platform specifics and how to register a plugin with the Wikitude SDK and then we go through each of the sample plugins included with the Wikitude Example Applications.

About Wikitude SDK Plugins

A plugin is a class, or rather a set of classes, written in C++ that allows extending the functionality of the Wikitude SDK. While there is a Plugin base class that offers some of the main functionality, a plugin itself can have multiple different optional modules that allow for more complex concepts to be implemented. The following table gives a very brief overview of the distribution of responsibilities of the plugin related classes.

class	purpose
Plugin	Main class of a plugin implementation. Derive from this class to create your plugin. It handles the application lifecycle and main plugin functionality. It provides access to various parameters of the Wikitude SDK and provides access to the camera frame and the recognized targets. It further owns and manages all the optional plugin modules.
ImageTrackingPluginModule	Optional module to allow for custom image tracking implementations. Derive from this class to implement your own image tracking algorithm to work in conjunction with the Wikitude SDK algorithms.
InstantTrackingPluginModule	Optional module to allow for custom instant tracking implementations. Derive from this class to implement your own instant tracking algorithm to work in conjunction with the Wikitude SDK algorithms.
ObjectTrackingPluginModule	Optional module to allow for custom object tracking implementations. Derive from this class to implement your own object tracking algorithm to work in conjunction with the Wikitude SDK algorithms.
CameraFrameInputPluginModule	Optional module to allow for frame data to be input into the Wikitude SDK. Derive from this class to implement your own camera frame acquisition. The supplied frame data supplied can be processed and rendered by the Wikitude SDK.
DeviceIMUInputPluginModule	Optional module to allow for sensor data to be input into the Wikitude SDK. Derive from this class to implement your own sensor data acquisition. The supplied sensor data supplied will be used by the Wikitude SDK for its tracking algorithms where applicable.
OpenGLESRenderingPluginModule	Optional module to allow for custom OpenGL ES rendering. Available on iOS and Android.
MetalRenderingPluginModule	Optional module to allow for custom Metal rendering. Only available on iOS.
DirectXRenderingPluginModule	Optional module to allow for custom DirectX rendering. Only available on Windows.

Each of the optional modules can be registered by calling the corresponding function of the Plugin class.

An important thing to remember when working with plugins is that they need to have a unique identifier. If an attempt is made to register a plugin with an identifier that is already known to the Wikitude SDK, the register method call will return false.

Plugin Base Class

class Plugin {
public:
    Plugin(std::string identifier_);
    virtual ~Plugin();

    virtual void initialize(const std::string& temporaryDirectory_, PluginParameterCollection& pluginParameterCollection_);
    virtual void setSDKEdition(SDKEdition sdkEdition_);
    virtual void pause();
    virtual void resume(unsigned int pausedTime_);
    virtual void destroy();

    virtual void cameraFrameAvailable(common_code::ManagedCameraFrame& managedCameraFrame_) = 0;
    virtual void deviceRotationEventAvailable(const DeviceRotationEvent& deviceRotationEvent_);
    virtual void deviceOrientationEventAvailable(const DeviceOrientationEvent& deviceOrientationEvent_);
    virtual void prepareUpdate();
    virtual void update(const RecognizedTargetsBucket& recognizedTargetsBucket_) = 0;

    virtual const std::string& getIdentifier() const;

    virtual void setEnabled(bool enabled_);
    virtual bool isEnabled() const;

    virtual PluginType getPluginType() const;

    virtual bool canPerformTrackingOperationsAlongOtherPlugins();
    virtual bool canUpdateMultipleTrackingInterfacesSimultaneously();

    ImageTrackingPluginModule* getImageTrackingPluginModule() const;
    InstantTrackingPluginModule* getInstantTrackingPluginModule() const;
    ObjectTrackingPluginModule* getObjectTrackingPluginModule() const;

    CameraFrameInputPluginModule* getCameraFrameInputPluginModule() const;
    DeviceIMUInputPluginModule* getDeviceIMUInpputPluginModule() const;

    OpenGLESRenderingPluginModule* getOpenGLESRenderingPluginModule() const;
    MetalRenderingPluginModule* getMetalRenderingPluginModule() const;
    DirectXRenderingPluginModule* getDirectXRenderingPluginModule() const;

protected:
    void setImageTrackingPluginModule(std::unique_ptr<ImageTrackingPluginModule> imageTrackingPluginModule_);
    void setObjectTrackingPluginModule(std::unique_ptr<ObjectTrackingPluginModule> objectTrackingPluginModule_);
    void setInstantTrackingPluginModule(std::unique_ptr<InstantTrackingPluginModule> instantTrackingPluginModule_);

    void setCameraFrameInputPluginModule(std::unique_ptr<CameraFrameInputPluginModule> cameraFrameInputPluginModule_);
    void setDeviceIMUInputPluginModule(std::unique_ptr<DeviceIMUInputPluginModule> deviceIMUInputPluginModule_);

    void setOpenGLESRenderingPluginModule(std::unique_ptr<OpenGLESRenderingPluginModule> openGLESRenderingPluginModule_);
    void setMetalRenderingPluginModule(std::unique_ptr<MetalRenderingPluginModule> metalRenderingPluginModule_);
    void setDirectXRenderingPluginModule(std::unique_ptr<DirectXRenderingPluginModule> directXRenderingPluginModule_);

    void iterateEnabledPluginModules(std::function<void(PluginModule& activePluginModule_)> activePluginModuleIteratorHandle_);

protected:
    std::string     _identifier;
    bool            _enabled;

    mutable std::mutex          _pluginModuleAccessMutex;
    std::set<PluginModule*>     _availablePluginModules;

private:
    std::unique_ptr<ImageTrackingPluginModule> _imageTrackingModule;
    std::unique_ptr<InstantTrackingPluginModule> _instantTrackingModule;
    std::unique_ptr<ObjectTrackingPluginModule> _objectTrackingModule;

    std::unique_ptr<CameraFrameInputPluginModule>   _cameraFrameInputModule;
    std::unique_ptr<DeviceIMUInputPluginModule>     _deviceIMUInputPluginModule;

    std::unique_ptr<OpenGLESRenderingPluginModule> _openGlesRenderingModule;
    std::unique_ptr<MetalRenderingPluginModule> _metalRenderingModule;
    std::unique_ptr<DirectXRenderingPluginModule> _directXRenderingModule;
};

While we will not go over every function of the Plugin class and all the optional module classes, the following sections will present sample plugins that should convey most of the concepts and methods involved in creating your own plugin.

Information about Recognized Targets

If the Wikitude SDK is running with active image-, instant- or object recognition, the plugins API will populate the RecognizedTargetsBucket in the update method with the currently recognized targets. The plugin may then use the corresponding target objects to acquire data, most importantly the pose, and use it for further processing.

class RecognizedTargetsBucket {
public:
    virtual ~RecognizedTargetsBucket() = default;

    virtual const std::vector<ImageTarget*>& getImageTargets() const = 0;
    virtual const std::vector<ObjectTarget*>& getObjectTargets() const = 0;

    virtual const std::vector<InstantTarget*>& getInstantTargets() const = 0;
    virtual const std::vector<InitializationPose*>& getInitializationPoses() const = 0;

    virtual const std::vector<Plane*>& getPlanes() const = 0;
    virtual const Matrix4& getViewMatrix() const = 0;
};

Platform Specifics

Registering Plugins

Barcode and QR code reader

This samples shows a full implementation of the popular barcode library ZBar into the Wikitude SDK. As ZBar is licensed under LGPL2.1 this sample can also be used for other projects.

ZBar is an open source software suite for reading bar codes from various sources, such as video streams, image files and raw intensity sensors. It supports many popular symbologies (types of bar codes) including EAN-13/UPC-A, UPC-E, EAN-8, Code 128, Code 39, Interleaved 2 of 5 and QR Code.

Face Detection

This samples shows how to add face detection to your Wikitude augmented reality experience using OpenCV.