MAD Gaze GLOW: Android SDK

Changelog

[08/28/2020][v1.1.0] Supporting MAD Gaze GLOW Plus

[03/03/2020][v1.0.0] First launch.


Build History

Date GLOW-SDK (.aar) UVCCamera (.aar) USBSerial (.aar) Demo Project (.zip)
08/28/2020
03/03/2020

Getting Started

  1. Download UVC Library file

    , GLOW Library file and USB Serial Library file .

  2. Import downloaded modules into your project.

    1. Open your Android Studio project
    2. File > Project Structure > Modules > Press the button ‘+’ in the top left
    3. Select Import .JAR/.AAR Package
    4. Select downloaded UVC Library file
    5. Press Next and finish the Import
    6. Press Apply to synchronize the project
  3. Repeat the Step (2) with GLOW Library file.

  4. Repeat the Step (2) with USB Serial Library file.

  5. Add repository in build.gradle (the root level).

    allprojects {
    	repositories {
    		...
    		maven { url 'http://raw.github.com/saki4510t/libcommon/master/repository/' }
    		//add this line
    		...
    	}
    }
    
  6. Set Target SDK to 27 in build.gradle (the root level).

    android {
    	...
    	defaultConfig {
    		...
    		minSdkVersion 21
    		targetSdkVersion 27 //set this to 27 or below 27.
    		...
    	}
    }
    
  7. Add the depdencies in app/build.gradle

    implementation("com.serenegiant:common:1.5.20")
    implementation("com.squareup.okio:okio:2.1.0")
    //add these line before UVC Library and GLOW Library
    
    implementation project(':libuvccamera')
    implementation project(':usbserial')	
    implementation project(':GLOW-devkit-release')
    
  8. Synchronize the Gradle.

  9. Includes the following permissions in AndroidManifest.xml

    <uses-permission android:name="android.permission.CAMERA"/>
    <uses-permission android:name="android.permission.RECORD_AUDIO"/>
    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
    
  10. Request permissions above programmatically, please check Android API Doc:

  11. Set all your activities in Landscape. (Optional)

  12. You are good to go!


Connection Control

Setup for Connection Control

Bind SplitUSBSerial.onStart() and SplitUSBSerial.onStop() to your activity to handle GLOW connection.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
}
@Override 
protected void onStop(){
	super.onStop();
	SplitUSBSerial.getInstance(this).onStop();
}

Connection Status

To check the connection of GLOW and connected device has been established.

boolean isConnected = SplitUSBSerial.getInstance(this).isDeviceConnected();

Connection Callbacks

SplitUSBSerial.getInstance(this).setConnectionCallback(new ConnectionCallback(){
	@Override
	public void onConnected() {
	//The connection between the glasses and connected devices has been successfully established.
	}

	@Override
	public void onDisconnected() {
	//The connection is disconnected.
	}

	@Override
	public void onError(int code) {
	//Something happened (See table below)
	}
});
Erorr Code Description
1 The connected device was not MAD Gaze branded.
2 The prompted USB Permission has been denied.

Screen Control

Setup for Screen Control

Bind SplitUSBSerial.onStart() and SplitUSBSerial.onStop() to your activity to handle GLOW connection.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
}
@Override 
protected void onStop(){
	super.onStop();
	SplitUSBSerial.getInstance(this).onStop();
}

Configure Screen Status

To turn on the screen programmatically.

//Turning on screen
SplitUSBSerial.getInstance(this).turnOnScreen();

To turn off the screen programmatically.

//Turning off screen
SplitUSBSerial.getInstance(this).turnOffScreen();

Configure Screen Brightness

Brightness Level: 0 (Almost Dark) - 7 (The Brightest)

To get the current brightness level of the screen.

//Get the current screen brightness level.
int screenBrightness = SplitUSBSerial.getInstance(this).getBrightness();

To set the brightness level of the screen.

//Setting the brightness of the screen at 4. (0-7)
SplitUSBSerial.getInstance(this).setBrightness(4);

Configure 3D Mode

To check if 3D mode is now turned on.

//Check if 3D Mode is now turned on.
boolean isOn = SplitUSBSerial.getInstance(this).Mode3D.is3DModeOn();

To turn on the 3D mode programatically.

//Turn on the 3D Mode
SplitUSBSerial.getInstance(this).Mode3D.turnOn();

To turn off the 3D mode programatically.

//Turn off the 3D Mode
SplitUSBSerial.getInstance(this).Mode3D.turnOff();

Camera Control

Setup for Camera

  1. Configure the activity orientation to Landscape.

  2. Grant following permissions in your applications.

    android.permission.CAMERA
    android.permission.RECORD_AUDIO
    android.permission.WRITE_EXTERNAL_STORAGE
    
  3. Insert a CameraView into your layout file.

    <com.madgaze.smartglass.view.SplitCameraView
    	android:layout_width="match_parent"
    	android:layout_height="match_parent"
    	android:id="@+id/splitCameraView"
    	android:layout_centerInParent="true"/>
    
  4. Set Camera Resolution to 1280x720.

    SplitCamera.getInstance(this).setPreviewSize(SplitCamera.CameraDimension.DIMENSION_1280_720);
    
  5. Set Camera Frame Format to MJPEG.

    SplitCamera.getInstance(this).setFrameFormat(CameraHelper.FRAME_FORMAT_MJPEG);
    
  6. Bind the created SplitCameraView to SplitCamera for video preview.

    SplitCamera.getInstance(this).start(findViewById(R.id.splitCameraView));
    
  7. Monitor the connectivity and start preview when GLOW is connected.

    SplitCamera.getInstance(this).setCameraCallback(new SplitCameraCallback() {
    	@Override
    	public void onConnected() {
    		//USB Camera (GLOW) is connected.
    		SplitCamera.getInstance(MainActivity.this).startPreview();
    		//Start the preview immediately when it is connected.
    	}
    
    	@Override
    	public void onDisconnected() {
    		//USB Camera (GLOW) is disconnected.
    	}
    
    	@Override
    	public void onError(int code) {
    		//Code 1: There is no connecting MAD Gaze Cameras.
    	}
    });
    
  8. Bind onResume(), onDestroy(), onStart(), onStop(), onPause() to your activity.

    @Override
    protected void onDestroy() {
        super.onDestroy();
        SplitCamera.getInstance(this).onDestroy();
    }
    
    @Override
    protected void onStart() {
        super.onStart();
        SplitCamera.getInstance(this).onStart();
    }
    
    @Override
    protected void onStop() {
        super.onStop();
        SplitCamera.getInstance(this).onStop();
    }
    
    @Override
    protected void onResume() {
        super.onResume();
        SplitCamera.getInstance(this).onResume();
    }
    
    @Override
    protected void onPause() {
        super.onPause();
        SplitCamera.getInstance(this).onPause();
    }
    
  9. Congratulation! You are now ready to see the Camera Preview from GLOW with your application.

Code Example (Complete)

  • MainActivity.java
@Override
protected void onCreate(Bundle savedInstanceState) {
	super.onCreate(savedInstanceState);
	setContentView(R.layout.activity_main);
	SplitCamera.getInstance(this).setPreviewSize(SplitCamera.CameraDimension.DIMENSION_1280_720);
	SplitCamera.getInstance(this).setFrameFormat(CameraHelper.FRAME_FORMAT_MJPEG);
	SplitCamera.getInstance(this).start(findViewById(R.id.splitCameraView));
	SplitCamera.getInstance(this).setCameraCallback(new SplitCameraCallback() {
		@Override
		public void onConnected() {
			//USB Camera (GLOW) is connected.
			SplitCamera.getInstance(MainActivity.this).startPreview();
		}

		@Override
		public void onDisconnected() {
			//USB Camera (GLOW) is disconnected.
		}

		@Override
		public void onError(int code) {
			//Code 1: There is no connecting MAD Gaze Cameras.
		}
	});
}

@Override
protected void onDestroy() {
	super.onDestroy();
	SplitCamera.getInstance(this).onDestroy();
}

@Override
protected void onStart() {
	super.onStart();
	SplitCamera.getInstance(this).onStart();
}

@Override
protected void onStop() {
	super.onStop();
	SplitCamera.getInstance(this).onStop();
}

@Override
protected void onResume() {
	super.onResume();
	SplitCamera.getInstance(this).onResume();
}

@Override
protected void onPause() {
	super.onPause();
	SplitCamera.getInstance(this).onPause();
}
  • activity_main.xml
<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.madgaze.smartglass.view.SplitCameraView
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:id="@+id/splitCameraView"
        android:layout_centerInParent="true"/>

</RelativeLayout>

Photo Capture

Take Photo

SplitCamera.getInstance(this).takePicture(new TakePictureCallback() {
	@Override
	public void onImageSaved(String path) {
		//When image is saved succesfully
	}

	@Override
	public void onError(int code) {
		//When captured image is not saved successfully.
	}
});

Video Capture

Start Recording

SplitCamera.getInstance(this).startRecording();

Stop Recording

SplitCamera.getInstance(this).stopRecording();

Video Recording Callbacks

SplitCamera.getInstance(this).setRecordVideoCallback(new RecordVideoCallback() {
	@Override
	public void onVideoSaved(String path) {
		//When video is saved succesfully
		//path: the file location has been saved.
	}

	@Override
	public void onError(int code) {
		//When captured video is not saved successfully.
	}
});

Preview Frame Data

To retrieve video(perview) data for image processing.

SplitCamera.getInstance(this).setOnPreviewFrameListener(new AbstractUVCCameraHandler.OnPreViewResultListener() {
	@Override
	public void onPreviewResult(byte[] yuv420sp) {
		//Retrieve video data in YUV420sp format

		/* Just an example how to convert the data to Bitmap */
		int width = SplitCamera.getInstance(this).getPreviewWidth();
		int height = SplitCamera.getInstance(this).getPreviewHeight());
		
		ByteArrayOutputStream out = new ByteArrayOutputStream();
		YuvImage yuvImage = new YuvImage(yuv420sp, ImageFormat.NV21, width, height, null);
		yuvImage.compressToJpeg(new Rect(0, 0, width, height), 50, out);
		byte[] imageBytes = out.toByteArray();
		Bitmap image = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
		//Handle with your *image* data
	}
});

Configurables

Resolution Settings

  • To set the preview and recording resolution of the video. (Default: 1280x720px)

    SplitCamera.getInstance(this).setPreviewSize(SplitCamera.CameraDimension.DIMENSION_1280_720);
    
    Width Height Configuration
    1600 1200 SplitCamera.CameraDimension.DIMENSION_1600_1200
    2592 1944 SplitCamera.CameraDimension.DIMENSION_2592_1944
    2048 1536 SplitCamera.CameraDimension.DIMENSION_2048_1536
    1920 1080 SplitCamera.CameraDimension.DIMENSION_1920_1080
    1280 720 SplitCamera.CameraDimension.DIMENSION_1280_720
    640 480 SplitCamera.CameraDimension.DIMENSION_640_480
    1024 576 SplitCamera.CameraDimension.DIMENSION_1024_576
  • You can always check the current width and height with SplitCamera.getPreviewWidth() and SplitCamera.getPreviewHeight() repspectively.

    //Get the Current Preview Width
    SplitCamera.getInstance(this).getPreviewWidth();
    
    //Get the Current Preview Height
    SplitCamera.getInstance(this).getPreviewHeight();
    

Format Settings

  • To set preview format of the video. (Default: YUYV)
    SplitCamera.getInstance(this).setFrameFormat(CameraHelper.FRAME_FORMAT_MJPEG);
    
    Format Configuration
    YUYV CameraHelper.FRAME_FORMAT_YUYV
    MJPEG CameraHelper.FRAME_FORMAT_MJPEG

Sensors Control

Setup for Sensors

Bind SplitUSBSerial.onStart() and SplitUSBSerial.onStop() to your activity to handle GLOW connection.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
	//Register your sensors listener here
}
@Override 
protected void onStop(){
	super.onStop();
	SplitUSBSerial.getInstance(this).onStop();
}

Register Sensors Callback

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent sensor) {
			float x = sensor.values[0];
			float y = sensor.values[1];
			float z = sensor.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, TYPE_OF_SENSOR, CAPTURE_INTERVAL);

*Note: You should register your callback with onStart() and after SplitUSBSerial.onStart() function.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
	
	SplitUSBSerial.getInstance(this).registerSensorListener(...); //Sensor 1
	SplitUSBSerial.getInstance(this).registerSensorListener(...); //Sensor 2
	SplitUSBSerial.getInstance(this).registerSensorListener(...); //Sensor 3
}

TYPE_OF_SENSOR

Sensor Name TYPE_OF_SENSOR
Accelerometer MAD_SENSOR_TYPE_ACCELEROMETER
Gyroscope MAD_SENSOR_TYPE_GYROSCOPE
Magnetometer MAD_SENSOR_TYPE_MAGNETIC_FIELD

CAPTURE_INTERVAL

Default Value: 10000 (which is equivalent to 10000µs === 0.01s)

Capturing Sensors Data

To start capturing the data from sensors

SplitUSBSerial.getInstance(this).startSensorsCapturing();

It shall activate every sensors at once, which means normally you should not execute this funtion more than once even you have to capture data from multiple sensors.

*Note: You shall capture sensor data after onConnected() to ensure the connection has been established successfully.

SplitUSBSerial.getInstance(this).setConnectionCallback(new ConnectionCallback(){
	@Override
	public void onConnected() {
		SplitUSBSerial.getInstance(this).startSensorsCapturing();
	}
	.
	.

});

To stop capturing the data from sensors
``` java
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

It shall deactivate every sensors at the same time. You should always include this function call when this app is going background or terminated.

Accelerometer

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent accelerometer) {
		//Start receiving data when startSensorsCapturing() is executed.
		float x = accelerometer.values[0];
		float y = accelerometer.values[1];
		float z = accelerometer.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, MadSensorManager.MAD_SENSOR_TYPE_ACCELEROMETER, 10000);
.
.
//To start capturing data from sensors
SplitUSBSerial.getInstance(this).startSensorsCapturing();

//To stop capturing data from sensors
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

Gyroscope

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent gyro) {
		//Start receiving data when startSensorsCapturing() is executed.
		float x = gyro.values[0];
		float y = gyro.values[1];
		float z = gyro.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, MadSensorManager.MAD_SENSOR_TYPE_GYROSCOPE, 10000);
.
.
//To start capturing data from sensors
SplitUSBSerial.getInstance(this).startSensorsCapturing();

//To stop capturing data from sensors
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

Magnetometer

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent magnetometer) {
		//Start receiving data when startSensorsCapturing() is executed.
		float x = magnetometer.values[0];
		float y = magnetometer.values[1];
		float z = magnetometer.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, MadSensorManager.MAD_SENSOR_TYPE_MAGNETOMETER, 10000);
.
.
//To start capturing data from sensors
SplitUSBSerial.getInstance(this).startSensorsCapturing();

//To stop capturing data from sensors
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

Troubleshooting

  • Q: Why error "Failed to find byte code for com/felhr/usbserial/UsbSerialInterface$UsbDSRCallback" is showed when compiling the codes?

    Please ensure the gradle version is configured to greater or equal to 3.2. There is a bug that not able to compile the bytecodes for Gradle version 3.1 or below.

  • Q: Why the USB device is not detecting after completing all steps stated above?

    Excluding the possible causes of hardware issue, you will need to provide permission of Camera and Microphone on Android 9.0 devices.

    <uses-permission android:name="android.permission.CAMERA" />
    <uses-permission android:name="android.permission.RECORD_AUDIO" />
    

    To request permissions approval from the users, please find article below: Request App Permissions:

  • Q: Why videos are not successfully recorded?

    You may need to include permission of Writing External Storage into your application in order to save the videos to the storage.

    <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
    
  • Q: Why the permission dialog never prompt on my Android 10 device?

    It is because the native bug doesn’t allow you to prompt the dialog on Android Q devices with Target SDK newer than 28. You have to set Target SDK at 27 or below in order to make it work.

  • Q: Why the library "com.serenegiant:common:1.5.20" cannot be resolved?

    You can download the library files below and reimport the library with Step 2 in Getting Started.

Disclaimer

This library is licensed under the

.