MAD Gaze GLOW & GLOW Plus: Scene Tracking SDK for Unity

v2021.2 (Release Date: 06/18/2021)

  • Updates [Scene Tracking SDK]
  1. Standardize application flow for Scene Tracking
  2. Developers can now customise the Animation and Virtual Planes on their apps.
  3. Different callbacks among all state changes are now available.
  4. Two control modes are now available, Arbitary-Placement mode and Center-Placement mode.


Build History

Date Package File (.unitypackage) Demo Project
06/18/2021
02/10/2021
08/19/2020

Capabilities

Scene Tracking SDK for Unity is a proprietary technology that provides stunning AR experience can be used to recognize table-sized plain surface patterns and place the target to the tracked surface.

The recognition does not rely on an extra object or particular images but it tracks features points of the physical environment.


Prerequisite

*This SDK is applicable to both MAD Gaze GLOW and MAD Gaze GLOW Plus model.

The application shall be

  • an Android application
  • with Target SDK at 27 (Android 8.1 ‘Oreo’)
  • with Minimum SDK at 27 (Android 8.1 ‘Oreo’)
  • with Landscape Left/Right orientation

Tested Unity Editor Versions

  • Unity 2019.4.1
  • Unity 2019.3.12
  • Unity 2019.2.3

Experiencing

  1. Download Sample Unity Project .
  2. Open downloaded project with Unity 2019.3.12f1.
  3. Switch Platform to Android
  4. Open Player Settings, Check MADGaze/Demo/Scenes/SceneTrackingDemo, and uncheck other scenes.
  5. Plug your phone, do the Build & Run.

Application

Scene Tracking consists of 3 key states, 1) Plane Searching, 2) Plane Detected and 3) Tracking.

Stage Description
1. Plane Searching Searching for a flat surface to build a Virtual Plane on top of the surface.
2. Plane Detected Virtual Plane is now created, and ready to enter the Tracking state.
3. Tracking Objects can now be placed on the Virtual Plane using Screen Space Position.

1. Plane Searching

  • Developers can enable Scene Tracking by calling MADGazeARManager.Instance.startTracking(), which can be located in Controller > SlamDemoController.

    MADGazeARManager.Instance.startTracking();
    
  • This step needs to be triggered manually, developers should ensure the user has already put the glasses on and is looking at a flat surface before entering to Plane Searching state.

  • A Animation will be played once the plane searching stage has started.

  • The SDK will turn on the smart glasses camera, analyse the captured videos and start searching for a flat surface.

  • The Animation will dismiss automatically once the flat surface has successfully been detected.

  • Developers can customise the Animation for Plane Searching state (See

    ), and also register a callback with the code snippets as below.

    MADGazeARManager.Instance.addPlaneSearchingCallback(()=> {
    	//Plane Searching is now started.
    });
    

2. Plane Detected

  • The Virtual Plane will be built and displayed when the flat surface is successfully detected and located.
  • The Animation of Plane Searching will dismiss automatically, and the PlaneGroup will now be shown.

PlaneGroup is a group of the 3D objects in the form of Plane, to indicate a flat surface that user can place the objects.

MADGazeARManager.Instance.addPlaneDetectedCallback(()=> {
	//Virtual Plane is now mapped.
})

3. Tracking

  • If the target plane is TRACKED, user can place the 3D models on the virtual surface without overlapping. The overlapped 3D models will behave differently depends on the selected control mode. (See

    )

    • Developers can also register callbacks to monitor if the 3D model is successfully placed. (Check Controller > CustomSlamPlaceModelController)
    public override void onPlaceModelSuccess(){
    	GameObject obj = getCurrentGameObject();
    	//Get the 3D model that was sucessfully placed.
    }
    
    public override void onPlaceModelFail(){
    	GameObject obj = getCurrentGameObject();
    	//Get the 3D model that was not successfully placed.
    }
    
  • If the target plane is NOT TRACKED, the Animation will be played until the target plane is successfully tracked again.

  • Developers can customise the Animation for Target Plane Lost state (See

    ).


Methods of Control

Center-Placement Mode

In Center-Placement mode, the 3D Model will always be placed at the center of the screen, therefore the placement position will vary according to the different head rotation of the user. For example, if the user want to place the 3D model on the left side of the screen, he/she will need to rotate their head in anti-clockwise direction, vice-versa.

Placement of 3D model on the Virtual Plane will be triggered when user tap on the screen.

Arbitary-Placement Mode

In Arbitrary Placement mode, the position of the 3D model is determined by the tap position of fingers on the screen. First, user needs to tap and keep their finger on the screen to show the 3D model, they can then adjust the location of the model through slight movement of their fingers, once confirm they can release the fingers to place the 3D model at the current position.

Placement of 3D model on the Virtual Plane will be triggered at the point the user release their finger from the screen.


Configurables

Turn On/Off Camera Preview

Enabling Camera Preview may distract users from AR Experience. We recommend developers to provide this option in the application, but allow users to decide whether to turn the camera preview on or off.

This API does not shut down the camera, but the camera preview will be hidden in background and not shown to the user.

To turn on the camera preview,

MADGazeCamera.showCameraPreview = true;

To turn off the camera preview,

MADGazeCamera.showCameraPreview = false;

Change Control Modes

By default, the control mode is set to Center Placement Mode.

To understand the differences between Control Modes, please see

.

Switch to

CustomSlamPlaceModelController.switchToCenterPlacementMode();

Switch to

CustomSlamPlaceModelController.switchToArbitaryPlacementMode();

Change Animations

In Scene Tracking, there are two built-in Animations that may show up in the following conditions,

  1. Plane Searching

    Suggest users to slightly move their head in order to search for the best surface to build a virtual plane.

  2. Tracking - Target Plane Lost

    Inform users that the tracked plane is lost and could not be detected. User needs to look at the starting point again.

The corresponding GameObject(s) will be enabled/disabled in above situations automatically.

Developers can change the animations with the following code snippets.

GameObject planeSearchingHint;
GameObject targetLostHint;

.
.

MADGazeARManager.Instance.setTrackingHints(planeSearchingHint, targetLostHint);

If developers need to change the prefabs while building apps with demo scene, it can be done in Controller/SlamDemoController (PlaneSearchingPrefab and TargetPlaneLostPrefab).

Create/Change a 3D Model to Clone

To use your custom 3D models, you will need to create a prefab with your 3D model from MAD Gaze SDK before applying it to Scene Tracking.

  1. Open the menu MAD Gaze from the Unity Editor, and Select Create CustomObject

  2. A default object with the plane is now created and inserted into the active scene.

  3. Delete the default Cube object.

  4. Put your 3d model into CustomObject.

  5. Adjust the desired size and angle of the 3D Model. This will exactly be how does it look like when user places the object on the virtual plane at the first moment.

  6. Export CustomObject to a Prefab.

  7. Replace the CloneTarget (Controller > CustomSlamPlaceModelController > CloneTarget) to exported prefab.

  8. Done.

Change Appearance of Virtual Planes

Virtual Planes can be found under PlaneGroup (World > Target > PlaneGroup).

Developers can change and adjust the GameObject(s) in PlaneGroup, or simply replace the material on the GameObject(s) of PlaneGroup.

Turn On/Off Virtual Planes

Virtual Planes can be found under PlaneGroup. (World > Target > PlaneGroup).

Developers should never disable the GameObject(s) of PlaneGroup. Disabling the PlaneGroup will result in malfunction of 3D models placement on the plane.

To turn on the Virtual Planes,

for (int i = 0; i < planeGroup.transform.childCount; i++)
{
	planeGroup.transform.GetChild(i).gameObject.GetComponent<Renderer>().enabled = true;
}

To turn off the Virtual Planes,

for (int i = 0; i < planeGroup.transform.childCount; i++)
{
	planeGroup.transform.GetChild(i).gameObject.GetComponent<Renderer>().enabled = false;
}

Sample Scene

You can check SceneTrackingDemo in MADGaze/Demo/Scenes.


Troubleshooting

  • Q: Why the permission dialog never prompt on my Android 10 device?

    It is because the native issue doesn’t allow you to prompt the permission dialog on Android Q devices with Target SDK newer than 28. You have to set Target SDK at 27 in order to make it work.

  • Q: I want a more stable recognition, what should I do?

    Scene Tracking will recognise all the images captured from camera, the recognition will become more stable while the capturing images have higher contrast.

    You can also download the AR Marker below.



Changelog

v2021.2 (Release Date: 06/18/2021)

  • Updates [Scene Tracking SDK]
  1. Standardize application flow for Scene Tracking
  2. Developers can now customise the Animation and Virtual Planes on their apps.
  3. Different callbacks among all state changes are now available.
  4. Two control modes are now available, Arbitary-Placement mode and Center-Placement mode.

v2021.1 (Release Date: 02/10/2021)

v1.3.0 (Release Date: 08/19/2020)

  • First Release
    1. : This SDK allows user to recognize table-sized plain surface patterns and place the target to the tracked surface.
  • Updates
    1. : Support USB Connection with MAD Gaze Adapters.

v1.1.0 (Release Date: 07/10/2020)

  • First Release
    1. : This SDK allows users to control the application via operating with head rotation.

v1.0.0 (Release Date: 06/29/2020)

  • First Release
    1. : This SDK allows users to access the camera with codes.

Disclaimer

This library is licensed under the

.