Documentation

Hardware Control

Camera Settings

The first part of the series will present a general overview of what is possible with the Wikitude SDK.

Camera Position

The camera position defines if the front, back or default camera should be started. This can be set by using NativeStartupConfiguration.setCameraPosition or CameraManager.setCameraPosition.

Camera Focus Mode

The camera focus mode defines which focus mode is used internally by the camera. It can be set by using NativeStartupConfiguration.setCameraFocusMode or CameraManager.setFocusMode. The camera focus mode can be one of the following modes:

  • Continuous: Is the default if the device supports it, in this mode the camera will try to refocus automatically when necessary.
  • Once: Can force the camera to refocus once on the current view. If you need to change focus you can set this value again. Each time the value ONCE is set, the camera tries to focus on the current scene again
  • OFF: Is disabling auto focus and will set the focus to infinity focus if not specified by the manual focus distance. This mode is a prerequisite for setting the manual focus distance.

Manual Camera Focus

The manual camera focus allows to set the focus manually and is only available if the camera2 is enabled in the NativeStartupConfiguration and the device supports it. To check if manual focus is available use CameraManager.isManualFocusAvailable The focus distance ranges from 0 to 1 where 0 is closest focus possible and 1 is at infinity focus. It can be set by using NativeStartupConfiguration.setCameraManualFocusDistance or CameraManager.setManualFocusDistance.

Camera2 API usage

The SDK can use either the legacy camera API (deprecated Google sind Android 5.0+) or the new camera2 API. The camera2 API enables a lot more manual camera features like the manual focus. Since the camera2 API is not fully working on some devices it is disabled by default and can be enabled by using NativeStartupConfiguration.setCamera2Enabled.

Exclusive camera2 features currently are:

  • Manual Focus

Camera Resolution

It can be set by using NativeStartupConfiguration.setCameraResolution and not be changed during runtime. The recommended values are the following:

Recommendations Geo Image Tracking Extended Tracking Cloud Recognition Instant Tracking
CameraSettings.CameraResolution AUTO AUTO SD_640x480 SD_640x480 SD_640x480

This setting will only be used on armv8 devices, all other architectures will default to SD_640x480 .

Camera Zoom

Digital zoom can be controlled by using CameraManager.setZoomLevel. To get the maximum zoom level use CameraManager.get.

Flashlight

The flashlight can be turned on and off by using CameraManager.enableCameraFlashLight and CameraManager.disableCameraFlashLight.

Focus at point of interest

To check if the device supports setting the focus point of interest CameraManager.isFocusPointOfInterestSupported can be used.

The camera focus point of intereest can be changed using CameraManager.setFocusPointOfInterest. Setting the focus point of interest is currently only supported with the focus mode ONCE.

The parameters given to this method call are the x and y coordinate in pixels at which the underlying camera should focus at. The coordinates have to exist on the render surface view. The actual values can either be choosen manually or received through the usage of a e.g. View.OnTouchListener where getX and getY of the MotionEvent can be used. Device orientation changes are considered in the internal SDK implementation.

Expose at point of interest

To check if the device supports setting the exposure point of interest CameraManager.isExposurePointOfInterestSupported can be used.

The camera exposure point of interest can be changed using CameraManager.setExposurePointOfInterest.

The parameters given to this method call are the x and y coordinate in pixels at which the underlying camera should focus at. The coordinates have to exist on the render surface view. The actual values can either be choosen manually or received through the usage of a e.g. View.OnTouchListener where getX and getY of the MotionEvent can be used. Device orientation changes are considered in the internal SDK implementation.

Camera Controls

The CameraManager allows you to switch between front and back camera, continuous focus and one time focus as well as control the zoom. We created a activity_camera_control.xml layout file, were we define some simple ui components for all exposed settings, please browse the sample app code if you would like to have a look at this file.

With the layout file in place we are ready to connect the UI components and the CameraManager.

The first component we connect is the flash light on/off switch, were we set a anonymous OnCheckedChangeListener in which we retrieve the CameraManager from the WikitudeSDK and enable or disable the flash light by calling enableCameraFlash or disableCameraFlash.

For the zoom SeekBar we first set the maximum value of the SeekBar to the maximum zoom value returned by the CameraManager, to create a wider range of values we multiply the returned value by 100. Then again we set a anonymous Listener to react on the user input. Since we widened the value range, we need to divide the current Seekbar value by 100 before we pass it back to the CameraManager by calling setZoomLevel.

The manual focus SeekBar is very similar to the zoom SeekBar with the only difference that the range of values is always 0-1. We set the range of the SeekBar from 0-100 to widen the range. Then again we set a anonymous Listener to react on the user input. Since we widened the value range, we need to divide the current Seekbar value by 100 before we pass it back to the CameraManager by calling setManualFocusDistance. Using the manual focus only works if no auto focus is enabled. This means that to use manual focus the camera focus mode has to be CameraSettings.CameraFocusMode.OFF. The manual focus SeekBar is only shown in the sample if the focus mode is set to CameraSettings.CameraFocusMode.OFF because of this behaviour. Manual Support works only when using the camera2 api which can be enabled in the ArchitectStartupConfiguration by the calling config.setCamera2Enabled(true). Please note that the camera2 api has issues on several devices which is why it is disabled by default.

    @Override
    public void onRenderExtensionCreated() {

        ...

        Switch flashToggleButton = (Switch) findViewById(R.id.flashlight);
        flashToggleButton.setOnCheckedChangeListener(new CompoundButton.OnCheckedChangeListener() {
            @Override
            public void onCheckedChanged(final CompoundButton buttonView, final boolean isChecked) {
                if (isChecked) {
                    cameraManager.enableCameraFlashLight();
                } else {
                    cameraManager.disableCameraFlashLight();
                }

            }
        });
        flashToggleButton.setEnabled(false);

        SeekBar zoomSeekBar = (SeekBar) findViewById(R.id.zoomSeekBar);
        zoomSeekBar.setMax(((int) cameraManager.getMaxZoomLevel()) * 100);
        zoomSeekBar.setOnSeekBarChangeListener(new SeekBar.OnSeekBarChangeListener() {
            @Override
            public void onProgressChanged(final SeekBar seekBar, final int progress, final boolean fromUser) {
                if (progress > 0) {
                    cameraManager.setZoomLevel((float) (progress + 100) / 100.0f);
                }
            }

            @Override
            public void onStartTrackingTouch(final SeekBar seekBar) {

            }

            @Override
            public void onStopTrackingTouch(final SeekBar seekBar) {

            }
        });
        zoomSeekBar.setEnabled(false);

        focusRow = (TableRow) findViewById(R.id.tableRow4);
        SeekBar focusSeekBar = (SeekBar) findViewById(R.id.focusSeekBar);
        focusSeekBar.setMax(100);
        focusSeekBar.setOnSeekBarChangeListener(new SeekBar.OnSeekBarChangeListener() {
            @Override
            public void onProgressChanged(final SeekBar seekBar, final int progress, final boolean fromUser) {
                cameraManager.setManualFocusDistance((float)progress/100.0f);
            }

            @Override
            public void onStartTrackingTouch(final SeekBar seekBar) {
            }

            @Override
            public void onStopTrackingTouch(final SeekBar seekBar) {
            }
        });
    }

    @Override
    public void onCameraOpen() {

        ...

        cameraPositionSpinner.setEnabled(true);
        focusModeSpinner.setEnabled(true);
        flashToggleButton.setEnabled(true);
        float maxZoomLevel = wikitudeSDK.getCameraManager().getMaxZoomLevel();
        if (maxZoomLevel > 1) {
            zoomSeekBar.setEnabled(true);
            zoomSeekBar.setMax((int) maxZoomLevel * 100 - 100);
        }
    }

Since we use two Spinners to control the camera position and focus mode, we don't go with anonymous listeners, but let the activity implement onItemSelectedListener and handle both spinners in the same method. For both focus mode and camera position CameraManager exposes setters to pick a value. We use the appropriate enums provided by the CameraSettings class to choose from the available settings.

    @Override
    public void onRenderExtensionCreated() {

        ...

       Spinner cameraPositionSpinner = (Spinner) findViewById(R.id.cameraPosition);
       ArrayAdapter<CharSequence> adapter = ArrayAdapter.createFromResource(CameraControlsActivity.this, R.array.camera_positions, android.R.layout.simple_spinner_item);
       adapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
       cameraPositionSpinner.setAdapter(adapter);
       cameraPositionSpinner.setOnItemSelectedListener(CameraControlsActivity.this);
       cameraPositionSpinner.setEnabled(false);

       Spinner focusModeSpinner = (Spinner) findViewById(R.id.focusMode);
       adapter = ArrayAdapter.createFromResource(CameraControlsActivity.this, R.array.focus_mode, android.R.layout.simple_spinner_item);
       adapter.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item);
       focusModeSpinner.setAdapter(adapter);
       focusModeSpinner.setOnItemSelectedListener(CameraControlsActivity.this);
       focusModeSpinner.setEnabled(false);
    }

First we check if the camera is already stared since we cannot make changes to the camera if it is not started. If CameraFocusMode is set we check if the device and camera configuration support manual focus and depending on that show the manual focus SeekBar or show a toast explaining why it is not available.

    @Override
    public void onItemSelected(final AdapterView<?> adapterView, final View view, final int position, final long id) {
        if (mIsCameraOpen) {
            switch (adapterView.getId()) {
                case R.id.focusMode:
                    switch (position){
                        case 0:
                            cameraManager.setFocusMode(CameraSettings.CameraFocusMode.CONTINUOUS);
                            if (focusRow != null) {
                                focusRow.setVisibility(View.GONE);
                            }
                            break;
                        case 1:
                            cameraManager.setFocusMode(CameraSettings.CameraFocusMode.ONCE);
                            if (focusRow != null) {
                                focusRow.setVisibility(View.GONE);
                            }
                            break;
                        case 2:
                            cameraManager.setFocusMode(CameraSettings.CameraFocusMode.OFF);
                            if (cameraManager.isManualFocusAvailable()) {
                                if (focusRow != null) {
                                    focusRow.setVisibility(View.VISIBLE);
                                }
                            } else if (!mCamera2Enabled || Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP_MR1) {
                                Toast.makeText(this, "Manual Focus is not supported by the old camera API. The focus will be fixed at infinity focus if the device supports it.", Toast.LENGTH_SHORT).show();
                            } else {
                                Toast.makeText(this, "Manual Focus is not supported by this device. The focus will be fixed at infinity focus.", Toast.LENGTH_SHORT).show();
                            }
                    }
                    break;
                case R.id.cameraPosition:
                    if (position == 0) {
                        cameraManager.setCameraPosition(CameraSettings.CameraPosition.BACK);
                    } else {
                        cameraManager.setCameraPosition(CameraSettings.CameraPosition.FRONT);
                    }
                    break;
            }
        } else {
            Log.e("CAMERA_OPEN", "camera is not open");
        }
    }

Focus and exposure point of interest are not controlled through UI elements but user input. For this the sample sets an OnTouchListener to the surfaceView. Setting the focus point of interest is currently only supported with FocusMode.ONCE active.

Important: the coordinates used to set the focus or exposure point of interest have to be coordinates of the view that is displying the camera frame, not coordinates of the screen. In the sample this view is the customSurfaceView.

The following code shows such an implementation:

customSurfaceView.setOnTouchListener(new View.OnTouchListener() {
    @Override
    public boolean onTouch(final View v, final MotionEvent event) {
        cameraManager.setFocusPointOfInterest(event.getX(), event.getY());
        cameraManager.setExposurePointOfInterest(event.getX(), event.getY());
        return false;
    }
});