Cloud Recognition

This example shows how to recognize images on a cloud server and then overlay it with augmentations utilizing the CloudTracker class.

For a better understanding, here are some terms that will be used in the following and other sections of this documentation related to vision-based augmented reality.

For both Cloud Recognition samples below we will use external rendering if you don't know what that means please go through the section on rendering before starting here.

The CloudTracker is able to run in two modes, we call them on-click and continuous. In On-Click mode a single recognition cycle will be executed, in continuous mode the recognition will be started in a variable interval. You can find both examples discussed in the following in our sample application under the package com.wikitude.recognition.cloud.

Regional server endpoints

Before we get started please note that you have to choose which regional-distributed Wikitude server the SDK should contact.

The cloud recognition server region can be selected by calling setCloudRecognitionServerRegion of the TrackerManager. Every CloudTracker created after this call, will use the chosen region. In the following code snippet we will change the Cloud Recognition server location from the default setting of Europe to China.

    _wikitudeSDK.getTrackerManager().setCloudRecognitionServerRegion(TrackerManager.ServerRegion.CHINA);

On-Click Cloud Recognition

We will now go through the class OnClickCloudTrackingActivity of our sample application, starting with the onCreate method. In onCreate after creating an instance of the WikitudeSDK, we obtain the TrackerManager and call create2dCloudTracker passing our authentication token and target collection id. On the CloudTracker instance returned by the TrackerManager we call registerTrackerEventListener passing this to register our activity as a listener for this particular CloudTracker. For this to work, we implemented CloudTrackerEventListener on our activity.

public class OnClickCloudTrackingActivity extends Activity implements CloudTrackerEventListener, ExternalRendering {

    ...

    @Override
    protected void onCreate(Bundle savedInstanceState) {
        ...
        _wikitudeSDK.onCreate(getApplicationContext(), startupConfiguration);
        _cloudTracker = _wikitudeSDK.getTrackerManager().create2dCloudTracker("b277eeadc6183ab57a83b07682b3ceba", "54e4b9fe6134bb74351b2aa3");
        _cloudTracker.registerTrackerEventListener(this);
    }

CloudTrackerEventListener defines callbacks which enable the CloudTracker to communicate its status. On one hand we receive notifications about the loading process and on the other hand we receive updates concerning the tracking process. Let's first have a look at the methods concerning the loading process. The method onTrackerFinishedLoading will be called, when the CloudTracker was successfully loaded and initialized. In this example we do not have a reason to react on that, but you could for example do some initializations to be ready when the CloudTracker recognizes a target. The onTrackerLoadingError method will be called when there was a problem loading the CloudTracker. We'll receive an errorMessage, containing more information about the problem. For debugging purposes we just log the error for now. In a real world project, were you already tested your app and know it won't be some small mistake like for example a wrong target collection id, you probably want to remove the log and for example try to load the CloudTracker again.


    @Override
    public void onTrackerFinishedLoading(final CloudTracker cloudTracker_) {
    }

    @Override
    public void onTrackerLoadingError(final CloudTracker cloudTracker_, final String errorMessage_) {
        Log.d(TAG, "onTrackerLoadingError: " + errorMessage_);
    }

We had a look at the loading process callbacks above. Now let's move on to the callbacks concerning the tracking process. The methods onTargetRecognized as well as onRecognitionSuccessful will be called when the CloudTracker first recognizes one of your targets. In this example we do not react on the onTargetRecognized call, because we do not care which of our targets were recognized, but again in a real world example you could prepare your renderer for the target to be tracked. The onRecognitionSuccessful method is unique to the CloudTracker and not available in the ClientTracker interface. It will be passed a JSONObject containing meta information you defined for this target when creating your Cloud Archive. For now we will just display the contents in a standard Edittext ui component so you can have a look.

After the call to onTargetRecognized the next calls will be to onTracking. On this calls the CloudTracker will pass an instance of the RecognizedTarget class containing information about the current target. Besides general information about the target like the name of the target and distance to the target, this object also contains matrices which define the location of the target in the current frame, we pass the object to our renderer instance so it can be used to render something on or around the target. If the 'CloudTracker' isn't able to track the current target any longer it will call the onTargetLost method. In this case we will remove the last RecognizedTarget object from our renderer since it won't be updated anymore until the CloudTracker recognizes a target again.

The last method onRecognitionFailed will be called if the last recognition attempt was unsuccessful, in this case we will display an error message to the user so that she/he tries again.

    @Override
    public void onTargetRecognized(final CloudTracker cloudTracker_, final String targetName_) {

    }

    @Override
    public void onTracking(final CloudTracker cloudTracker_, final RecognizedTarget recognizedTarget_) {
        _glRenderer.setCurrentlyRecognizedTarget(recognizedTarget_);
    }

    @Override
    public void onTargetLost(final CloudTracker cloudTracker_, final String targetName_) {
        _glRenderer.setCurrentlyRecognizedTarget(null);
    }

    @Override
    public void onRecognitionSuccessful(final CloudTracker cloudTracker_, final JSONObject jsonObject_) {
        if (jsonObject_.toString().length() > 2) {
            runOnUiThread(new Runnable() {
                @Override
                public void run() {
                        EditText targetInformationTextField = (EditText) findViewById(R.id.on_click_cloud_tracking_target_information);
                        targetInformationTextField.setText(jsonObject_.toString(), TextView.BufferType.NORMAL);
                        targetInformationTextField.setVisibility(View.VISIBLE);
                    }
            });
        }
    }

    @Override
    public void onRecognitionFailed(final CloudTracker cloudTracker_, final int errorCode_, final String errorMessage_) {
        Toast toast = Toast.makeText(getApplicationContext(), "Recognition failed!", Toast.LENGTH_SHORT);
        toast.show();
    }

To be able to actually start the recognition we define a button and set an anonymous OnClickListener which calls the CloudTracker recognize method.

@Override
public void onRenderExtensionCreated(final RenderExtension renderExtension_) {
    ...
    Button recognizeButton = (Button) findViewById(R.id.on_click_cloud_tracking_recognize_button);
    recognizeButton.setOnClickListener(new View.OnClickListener() {
        @Override
        public void onClick(final View view_) {
            _cloudTracker.recognize();
        }
    });

}

Continuous Cloud Recognition

On-Click recognition is useful in some particular cases, but more often than not you probably want to use continuous recognition. For continuous cloud recognition we set an interval in which the CloudTracker automatically calls the recognize function.

Since we want to make sure the CloudTracker finished loading and is ready to be used before we start recognitions, we we'll use the onTrackerFinishedLoading callback to start the continuous recognition. To do that we call startContinuousRecognition on the CloudTracker instance passing an interval in milliseconds. Since mobile internet can be quite bad, we recommend an interval between 1500 and 2000ms.

@Override
public void onTrackerFinishedLoading(final CloudTracker cloudTracker_) {
    cloudTracker_.startContinuousRecognition(1500);
}

As already mentioned before internet quality isn't always the best on mobile devices. If the Wikitude SDK notices that the cloud recogntion interval was set to low, the onRecognitionInterruption callback method will be called. The SDK will pass an adjusted interval which should be more appropriate for the current connection speed. In our example we'll reset the CloudTracker interval by calling startContinuousRecognition passing the suggested interval.

@Override
public void onRecognitionInterruption(final CloudTracker cloudTracker_, final double suggestedInterval_) {
    cloudTracker_.startContinuousRecognition(suggestedInterval_);
}