MAD Gaze GLOW: Android SDK

Changelog

[08/28/2020][v1.1.0] Supporting MAD Gaze GLOW Plus

[03/03/2020][v1.0.0] First launch.

Build History

Getting Started

1. Download UVC Library file here, GLOW Library file here and USB Serial Library file here.

2. Import downloaded modules into your project.

   1. Open your Android Studio project
   2. File > Project Structure > Modules > Press the button '+' in the top left
   3. Select Import .JAR/.AAR Package
   4. Select downloaded UVC Library file
   5. Press Next and finish the Import
   6. Press Apply to synchronize the project

3. Repeat the Step (2) with GLOW Library file.

4. Repeat the Step (2) with USB Serial Library file.

5. Add repository in build.gradle (the root level).

   allprojects {
   	repositories {
   		...
   		maven { url 'http://raw.github.com/saki4510t/libcommon/master/repository/' }
   		//add this line
   		...
   	}
   }
   

6. Set Target SDK to 27 in build.gradle (the root level).

   android {
   	...
   	defaultConfig {
   		...
   		minSdkVersion 21
   		targetSdkVersion 27 //set this to 27 or below 27.
   		...
   	}
   }
   

7. Add the depdencies in app/build.gradle

   implementation("com.serenegiant:common:1.5.20")
   implementation("com.squareup.okio:okio:2.1.0")
   //add these line before UVC Library and GLOW Library
   
   implementation project(':libuvccamera')
   implementation project(':usbserial')	
   implementation project(':GLOW-devkit-release')
   

8. Synchronize the Gradle.

9. Includes the following permissions in AndroidManifest.xml

   <uses-permission android:name="android.permission.CAMERA"/>
   <uses-permission android:name="android.permission.RECORD_AUDIO"/>
   <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE"/>
   

10. Request permissions above programmatically, please check Android API Doc: https://developer.android.com/guide/topics/sensors/sensors_position

11. Set all your activities in Landscape. (Optional)

12. You are good to go!

Connection Control

Setup for Connection Control

Bind SplitUSBSerial.onStart() and SplitUSBSerial.onStop() to your activity to handle GLOW connection.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
}

@Override 
protected void onStop(){
	super.onStop();
	SplitUSBSerial.getInstance(this).onStop();
}

Connection Status

To check the connection of GLOW and connected device has been established.

boolean isConnected = SplitUSBSerial.getInstance(this).isDeviceConnected();

Connection Callbacks

SplitUSBSerial.getInstance(this).setConnectionCallback(new ConnectionCallback(){
	@Override
	public void onConnected() {
	//The connection between the glasses and connected devices has been successfully established.
	}

	@Override
	public void onDisconnected() {
	//The connection is disconnected.
	}

	@Override
	public void onError(int code) {
	//Something happened (See table below)
	}
});

Screen Control

Setup for Screen Control

Bind SplitUSBSerial.onStart() and SplitUSBSerial.onStop() to your activity to handle GLOW connection.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
}

@Override 
protected void onStop(){
	super.onStop();
	SplitUSBSerial.getInstance(this).onStop();
}

Configure Screen Status

To turn on the screen programmatically.

//Turning on screen
SplitUSBSerial.getInstance(this).turnOnScreen();

To turn off the screen programmatically.

//Turning off screen
SplitUSBSerial.getInstance(this).turnOffScreen();

Configure Screen Brightness

Brightness Level: 0 (Almost Dark) - 7 (The Brightest)

To get the current brightness level of the screen.

//Get the current screen brightness level.
int screenBrightness = SplitUSBSerial.getInstance(this).getBrightness();

To set the brightness level of the screen.

//Setting the brightness of the screen at 4. (0-7)
SplitUSBSerial.getInstance(this).setBrightness(4);

Configure 3D Mode

To check if 3D mode is now turned on.

//Check if 3D Mode is now turned on.
boolean isOn = SplitUSBSerial.getInstance(this).Mode3D.is3DModeOn();

To turn on the 3D mode programatically.

//Turn on the 3D Mode
SplitUSBSerial.getInstance(this).Mode3D.turnOn();

To turn off the 3D mode programatically.

//Turn off the 3D Mode
SplitUSBSerial.getInstance(this).Mode3D.turnOff();

Camera Control

Setup for Camera

1. Configure the activity orientation to Landscape.

2. Grant following permissions in your applications.

   android.permission.CAMERA
   android.permission.RECORD_AUDIO
   android.permission.WRITE_EXTERNAL_STORAGE
   

3. Insert a CameraView into your layout file.

   <com.madgaze.smartglass.view.SplitCameraView
   	android:layout_width="match_parent"
   	android:layout_height="match_parent"
   	android:id="@+id/splitCameraView"
   	android:layout_centerInParent="true"/>
   

4. Set Camera Resolution to 1280x720.

   SplitCamera.getInstance(this).setPreviewSize(SplitCamera.CameraDimension.DIMENSION_1280_720);
   

5. Set Camera Frame Format to MJPEG.

   SplitCamera.getInstance(this).setFrameFormat(CameraHelper.FRAME_FORMAT_MJPEG);
   

6. Bind the created SplitCameraView to SplitCamera for video preview.

   SplitCamera.getInstance(this).start(findViewById(R.id.splitCameraView));
   

7. Monitor the connectivity and start preview when GLOW is connected.

   SplitCamera.getInstance(this).setCameraCallback(new SplitCameraCallback() {
   	@Override
   	public void onConnected() {
   		//USB Camera (GLOW) is connected.
   		SplitCamera.getInstance(MainActivity.this).startPreview();
   		//Start the preview immediately when it is connected.
   	}
   
   	@Override
   	public void onDisconnected() {
   		//USB Camera (GLOW) is disconnected.
   	}
   
   	@Override
   	public void onError(int code) {
   		//Code 1: There is no connecting MAD Gaze Cameras.
   	}
   });
   

8. Bind onResume(), onDestroy(), onStart(), onStop(), onPause() to your activity.

   @Override
   protected void onDestroy() {
       super.onDestroy();
       SplitCamera.getInstance(this).onDestroy();
   }
   
   @Override
   protected void onStart() {
       super.onStart();
       SplitCamera.getInstance(this).onStart();
   }
   
   @Override
   protected void onStop() {
       super.onStop();
       SplitCamera.getInstance(this).onStop();
   }
   
   @Override
   protected void onResume() {
       super.onResume();
       SplitCamera.getInstance(this).onResume();
   }
   
   @Override
   protected void onPause() {
       super.onPause();
       SplitCamera.getInstance(this).onPause();
   }
   

9. Congratulation! You are now ready to see the Camera Preview from GLOW with your application.

Code Example (Complete)

• MainActivity.java

@Override
protected void onCreate(Bundle savedInstanceState) {
	super.onCreate(savedInstanceState);
	setContentView(R.layout.activity_main);
	SplitCamera.getInstance(this).setPreviewSize(SplitCamera.CameraDimension.DIMENSION_1280_720);
	SplitCamera.getInstance(this).setFrameFormat(CameraHelper.FRAME_FORMAT_MJPEG);
	SplitCamera.getInstance(this).start(findViewById(R.id.splitCameraView));
	SplitCamera.getInstance(this).setCameraCallback(new SplitCameraCallback() {
		@Override
		public void onConnected() {
			//USB Camera (GLOW) is connected.
			SplitCamera.getInstance(MainActivity.this).startPreview();
		}

		@Override
		public void onDisconnected() {
			//USB Camera (GLOW) is disconnected.
		}

		@Override
		public void onError(int code) {
			//Code 1: There is no connecting MAD Gaze Cameras.
		}
	});
}

@Override
protected void onDestroy() {
	super.onDestroy();
	SplitCamera.getInstance(this).onDestroy();
}

@Override
protected void onStart() {
	super.onStart();
	SplitCamera.getInstance(this).onStart();
}

@Override
protected void onStop() {
	super.onStop();
	SplitCamera.getInstance(this).onStop();
}

@Override
protected void onResume() {
	super.onResume();
	SplitCamera.getInstance(this).onResume();
}

@Override
protected void onPause() {
	super.onPause();
	SplitCamera.getInstance(this).onPause();
}

• activity_main.xml

<?xml version="1.0" encoding="utf-8"?>
<RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="match_parent"
    android:layout_height="match_parent">

    <com.madgaze.smartglass.view.SplitCameraView
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        android:id="@+id/splitCameraView"
        android:layout_centerInParent="true"/>

</RelativeLayout>

Photo Capture

Take Photo

SplitCamera.getInstance(this).takePicture(new TakePictureCallback() {
	@Override
	public void onImageSaved(String path) {
		//When image is saved succesfully
	}

	@Override
	public void onError(int code) {
		//When captured image is not saved successfully.
	}
});

Video Capture

Start Recording

SplitCamera.getInstance(this).startRecording();

Stop Recording

SplitCamera.getInstance(this).stopRecording();

Video Recording Callbacks

SplitCamera.getInstance(this).setRecordVideoCallback(new RecordVideoCallback() {
	@Override
	public void onVideoSaved(String path) {
		//When video is saved succesfully
		//path: the file location has been saved.
	}

	@Override
	public void onError(int code) {
		//When captured video is not saved successfully.
	}
});

Preview Frame Data

To retrieve video(perview) data for image processing.

SplitCamera.getInstance(this).setOnPreviewFrameListener(new AbstractUVCCameraHandler.OnPreViewResultListener() {
	@Override
	public void onPreviewResult(byte[] yuv420sp) {
		//Retrieve video data in YUV420sp format

		/* Just an example how to convert the data to Bitmap */
		int width = SplitCamera.getInstance(this).getPreviewWidth();
		int height = SplitCamera.getInstance(this).getPreviewHeight());
		
		ByteArrayOutputStream out = new ByteArrayOutputStream();
		YuvImage yuvImage = new YuvImage(yuv420sp, ImageFormat.NV21, width, height, null);
		yuvImage.compressToJpeg(new Rect(0, 0, width, height), 50, out);
		byte[] imageBytes = out.toByteArray();
		Bitmap image = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
		//Handle with your *image* data
	}
});

Configurables

Resolution Settings

• To set the preview and recording resolution of the video. (Default: 1280x720px)

  SplitCamera.getInstance(this).setPreviewSize(SplitCamera.CameraDimension.DIMENSION_1280_720);
  

  Width	Height	Configuration
  1600	1200	SplitCamera.CameraDimension.DIMENSION_1600_1200
  2592	1944	SplitCamera.CameraDimension.DIMENSION_2592_1944
  2048	1536	SplitCamera.CameraDimension.DIMENSION_2048_1536
  1920	1080	SplitCamera.CameraDimension.DIMENSION_1920_1080
  1280	720	SplitCamera.CameraDimension.DIMENSION_1280_720
  640	480	SplitCamera.CameraDimension.DIMENSION_640_480
  1024	576	SplitCamera.CameraDimension.DIMENSION_1024_576

• You can always check the current width and height with SplitCamera.getPreviewWidth() and SplitCamera.getPreviewHeight() repspectively.

  //Get the Current Preview Width
  SplitCamera.getInstance(this).getPreviewWidth();
  
  //Get the Current Preview Height
  SplitCamera.getInstance(this).getPreviewHeight();
  

Format Settings

• To set preview format of the video. (Default: YUYV)

  SplitCamera.getInstance(this).setFrameFormat(CameraHelper.FRAME_FORMAT_MJPEG);
  

  Format	Configuration
  YUYV	CameraHelper.FRAME_FORMAT_YUYV
  MJPEG	CameraHelper.FRAME_FORMAT_MJPEG

Sensors Control

Setup for Sensors

Bind SplitUSBSerial.onStart() and SplitUSBSerial.onStop() to your activity to handle GLOW connection.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
	//Register your sensors listener here
}

@Override 
protected void onStop(){
	super.onStop();
	SplitUSBSerial.getInstance(this).onStop();
}

Register Sensors Callback

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent sensor) {
			float x = sensor.values[0];
			float y = sensor.values[1];
			float z = sensor.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, TYPE_OF_SENSOR, CAPTURE_INTERVAL);

*Note: You should register your callback with onStart() and after SplitUSBSerial.onStart() function.

@Override 
protected void onStart(){
	super.onStart();
	SplitUSBSerial.getInstance(this).onStart();
	
	SplitUSBSerial.getInstance(this).registerSensorListener(...); //Sensor 1
	SplitUSBSerial.getInstance(this).registerSensorListener(...); //Sensor 2
	SplitUSBSerial.getInstance(this).registerSensorListener(...); //Sensor 3
}

TYPE_OF_SENSOR

CAPTURE_INTERVAL

Default Value: 10000 (which is equivalent to 10000µs === 0.01s)

Capturing Sensors Data

To start capturing the data from sensors

SplitUSBSerial.getInstance(this).startSensorsCapturing();

It shall activate every sensors at once, which means normally you should not execute this funtion more than once even you have to capture data from multiple sensors.

*Note: You shall capture sensor data after onConnected() to ensure the connection has been established successfully.

SplitUSBSerial.getInstance(this).setConnectionCallback(new ConnectionCallback(){
	@Override
	public void onConnected() {
		SplitUSBSerial.getInstance(this).startSensorsCapturing();
	}
	.
	.

});

To stop capturing the data from sensors
``` java
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

It shall deactivate every sensors at the same time. You should always include this function call when this app is going background or terminated.

Accelerometer

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent accelerometer) {
		//Start receiving data when startSensorsCapturing() is executed.
		float x = accelerometer.values[0];
		float y = accelerometer.values[1];
		float z = accelerometer.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, MadSensorManager.MAD_SENSOR_TYPE_ACCELEROMETER, 10000);
.
.
//To start capturing data from sensors
SplitUSBSerial.getInstance(this).startSensorsCapturing();

//To stop capturing data from sensors
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

Gyroscope

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent gyro) {
		//Start receiving data when startSensorsCapturing() is executed.
		float x = gyro.values[0];
		float y = gyro.values[1];
		float z = gyro.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, MadSensorManager.MAD_SENSOR_TYPE_GYROSCOPE, 10000);
.
.
//To start capturing data from sensors
SplitUSBSerial.getInstance(this).startSensorsCapturing();

//To stop capturing data from sensors
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

Magnetometer

SplitUSBSerial.getInstance(this).registerSensorListener(new SplitSensorEventListener() {
	@Override
	public void onSplitSensorChanged(MadSensorEvent magnetometer) {
		//Start receiving data when startSensorsCapturing() is executed.
		float x = magnetometer.values[0];
		float y = magnetometer.values[1];
		float z = magnetometer.values[2];
	}

	@Override
	public void onSplitAccuracyChanged(MadSensor sensor, int accuracy) {

	}
}, MadSensorManager.MAD_SENSOR_TYPE_MAGNETOMETER, 10000);
.
.
//To start capturing data from sensors
SplitUSBSerial.getInstance(this).startSensorsCapturing();

//To stop capturing data from sensors
SplitUSBSerial.getInstance(this).stopSensorsCapturing();

Troubleshooting

• Q: Why error "Failed to find byte code for com/felhr/usbserial/UsbSerialInterface$UsbDSRCallback" is showed when compiling the codes?

  Please ensure the gradle version is configured to greater or equal to 3.2. There is a bug that not able to compile the bytecodes for Gradle version 3.1 or below.

• Q: Why the USB device is not detecting after completing all steps stated above?

  Excluding the possible causes of hardware issue, you will need to provide permission of Camera and Microphone on Android 9.0 devices.

  <uses-permission android:name="android.permission.CAMERA" />
  <uses-permission android:name="android.permission.RECORD_AUDIO" />
  

  To request permissions approval from the users, please find article below: Request App Permissions: https://developer.android.com/training/permissions/requesting

• Q: Why videos are not successfully recorded?

  You may need to include permission of Writing External Storage into your application in order to save the videos to the storage.

  <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
  

• Q: Why the permission dialog never prompt on my Android 10 device?

  It is because the native bug doesn't allow you to prompt the dialog on Android Q devices with Target SDK newer than 28. You have to set Target SDK at 27 or below in order to make it work.

• Q: Why the library "com.serenegiant:common:1.5.20" cannot be resolved?

  You can download the library files below and reimport the library with Step 2 in Getting Started. https://file.madgaze.cn/sdk/files/libraries/others/com.serenegiant.common.1.5.20.aar

Disclaimer

This library is licensed under the MAD Gaze - Terms of Use.