Masangga.net Jurnal Tugas Akhir

Drawn Into Details #day0

have u ever heard about Augmented Reality?

maybe, most all of you have heard about this, though u can type "augmented reality" keyword in search engine,surely you can find many explanations about it

here's the reference may help you to understand what Augmented Reality really is..

1. Wikipedia

2. blog kak whe (explains a lot)

so what's this all 'bout?

currently, I'm researching this topic as a undergraduate thesis for my graduation,  my thesis title is

"Hand Tracking using Camshift and Kalman Filter Method in Augmented Reality, study case : Virtual Object Interaction Based on Touch in Real Time"

Nowadays, Augmented Reality is one of the breakthrough technologies that provides new paradigm into computer vision nor human-computer interaction that virtual and real object can be fused in the user interface, AR Technology has been used in various fields, such as game industry, medical, education etc.

Many useful interaction options have been developed for AR systems. Handheld or hand-controlled devices have been implemented in many systems for object visualization and manipulation. Multidimensional input devices would be more intuitive interaction tools for the 3D interface by directly mapping the user’s intentions to the application and providing more efficient interacting methods (Sundin & Fjeld, 2009). Computer mice with six degrees of freedom (DOFs), physical panels and pens tracked with sensors of six DOFs, joysticks and PHANToMTM are the usual devices to interact with 3D computer-generated objects. Compared to these devices in which specific hardware is required, the devices tracked using computer vision techniques are more flexible and cheaper. Vision-tracked fiducial markers have been widely used to register the virtual objects onto the real world and interact with the virtual objects (Chong, Ong, & Nee, 2007). For interaction via 2D interfaces, methods to track a uniform color have been proposed in some AR systems (Yuan, Ong, & Nee, 2004), with which a handheld stylus can be designed.

In recent years, the research issue of interacting with virtual interfaces andobjects using hands directly has been explored in some studies due to its nat-ural and flexible properties (Ha & Woo, 2006). The interactions with hands can
be roughly classified into two groups—device-assisted hand interaction and bare-hand interaction. For the systems in the first group, specific devices, such as pinchgloves with fiducial markers (Piekarski & Smith, 2006) and the PHANToMTM device with a deformable foam ball (Pihuit, Kryt, & Cani, 2008), have been designed and used. In bare-hand interaction, the natural features on the hands, such as the skin color, shape, and so on, are detected and tracked by processing the images captured by the cameras. Ha and Woo (2006) observed that the users prefer bare-hand interaction over interaction with the help of other devices.

Methods to be used so that users can interact with virtual objects should be required for AR is natural and intuitive, therefore, the detection and tracing of the hand (hereinafter referred to as hand tracking) is expected to connect the virtual world(AR systems) and the real world..

There are various hand tracking methods such as Mean Shift[2], Particle Filtering[3],  Camshift [4],  nor Kalman Filter [5]

Camshift Algorithm, one of the popular algorithm that has been used in developing hand tracking , basically we can call it "mean-shift improvisation" because it adapts "mean-shift" to track the object.  The main difference from both of methods that Meanshift Algorithm's using static-probabilistic distribution, and Camshift's using dynamic-probabilistic distribution.

But Camshift algorithm has weaknesses, the algorithm the tracking process may fail if occlusion happen between objects, and color-disturbation in large scale that may occurs in background.

To solve this problem, combining Camshift algorithm and Kalman Filter is required to give prediction about hand location.
Kalman Filter is recursive estimator that needs previous state and present measurement to estimate present state. Kalman Filter can predict hand location accurately  despite of similarity between hand color and background

 

In my Final Project, I want to combine these two methods (kalman + camshift) to track the hand in AR environment so that human hand can interact with virtual objects in AR Environment.