On-Board Training Project


On-Board Training isĀ an innovation project I led which reduced the operator learning curve by a significant 35%, using cutting edge technologies like Augmented Reality, speech recognition and semantic analysis and has assisted the operator in performing complex operations with greater efficiency

This demo involves understanding the operation and providing contextual information using multiple technologies like projecting on the wind screen, using Heads-Up displays or speech and gesture recognition

2 3 4 IMG_0111 IMG_0351

IMG_0134 IMG_0352


Updates to Training using Augmented Reality


I have been working on improving the training demos of using AR technology for training new operators .

Here is a demo android app which augments 3D animations of work instructions over the printed OMS sheets.

It has touch and speech based interaction, which is quite useful for porting it to HMDs like Google Glass, Epson Moverio etc.


Another demo is using the Vuzix wrap 920AR for radiator assembly training, along with Kinect for gesture and speech recognition.

Current Research


1) 3D Reconstruction of real objects and environments

Application to be used in reverse engineering and factory layout planning .


Toolkits used:- PCL, Kinect SDK, Skanect, Meshlab.

2) Loader Visibility Enhancement:-

An application to stitch video from 4 cameras and identify and remove loader arms and display the resulted video to the user to enhance is view.


Toolkits used:- OpenCV, Matlab

In-Cab training application


An application which has a gesture login (hand wave) and speech command interaction to navigate all through the menu. A special Hand position recognition is also used to identify the control the user is working on.


Toolkits used:- Kinect SDK, Microsoft Speech SDK

Languages:- C#