Research Works

Gesture Based Interaction in Immersive Virtual Reality

Recent development in virtual reality (VR) interaction with 3D camera and sensors like kinect, range camera, leap motion controller etc., has enabled opportunity in development of human computer interaction (HCI) application. Hand gesture is one of the popular ways that people use to interact with the computer. Even automatic hand gesture recognition appears as a suitable means for interacting with virtual reality systems. This research work focuses on the study and analysis of the application based on gesture interaction technology in virtual reality. Customizing gestures for pointing, grabbing, zoom in/out, swap were defined and implemented in unity 3D with leap motion SDK. The effectiveness of the hand gesture was analyzed through recording user experience and questionnaire.

Constrained Procedural Reconstruction of Architectural 3D Models from 2D Images

This research work devoted to 3D architectural reconstruction using structured representation and rules derived from a single view photograph using limited information. We present a novel modeling framework to generate real-world 3D buildings from a single facade layout that adapt to the real footprint data automatically. Facade components are extracted from the facade layout and organized as a repetitive shape tree. A meaningful grammar representation is automatically extracted from the hierarchical facade subdivision. We extend the previous approaches of procedural building models to a constraint-based framework for the recovery of the hidden parts of the building. We then provide an interactive editing process for updating of the structural topology given a different view of the building. We demonstrate our framework on several real-world buildings with challenging footprints and we show that the procedural representation can generate similar buildings to the original that are used to populate a virtual city.

Semantic Rendering based on Information Hierarchy for Urban Navigation

Natural scenes from the real world are highly complex, such that the modeling and rendering of natural shapes, like mountains, trees and clouds, are very difficult and time consuming and require a huge amount of memory. Intuitively, the critical characteristics of natural scenes are their self- similarity properties. Motivated by the self-similarity feature of the natural scenes that surround us, we present a hardware accelerated fractal based rendering method for natural environments. To illustrate the problem that classical geometry has in dealing with natural objects, we considered the basic fractal example as the Mandelbrot set which is a 2D structure. We examined the serial algorithm of this set and devised a parallel algorithm for implementation on a massive parallel graphics processing unit (GPU) using the computer unified device architecture (CUDA) programming model. We also considered the modeling of 3D fractals such as terrains and evaluated its performance both in terms of execution time and hardware acceleration. Performance is evaluated in terms of execution time and it was observed that a parallel implementation of the method on a GeForce GTX 650 GPU is on average 2X times faster than its sequential implementation. The running behavior of the system at various system states is also evaluated to strongly support our approach.

An Image Based Approach for Reconstruction of Ancient Architectures by Extracting and Arranging 3D Spatial Components

The objective of this research is the rapid reconstruction of ancient buildings of historical importance using a single image. The key idea of our approach is to reduce the infinite solutions that might otherwise arise when recovering a 3D geometry from 2D photographs. The main outcome of our research shows that the proposed methodology can be used to reconstruct ancient monuments for use as proxies for digital effects in applications such as tourism, gaming, and entertainment, which do not require very accurate modeling. In this article, we consider the reconstruction of ancient Mughal architecture including the Taj Mahal. We propose a modeling pipeline that makes an easy reconstruction possible using a single photograph taken from a single view, without the need to create complex point clouds from multiple images or the use of laser scanners. First, an initial model is automatically reconstructed using locally fitted planar primitives along with their boundary polygons and the adjacency relation among parts of the polygons. This approach is faster and more accurate than creating a model from scratch because the initial reconstruction phase provides a set of structural information together with the adjacency relation, which makes it possible to guess the approximate depth of the entire structural monument. Next, we use manual extrapolation and editing techniques with modeling software to assemble and adjust different 3D components of the model. Thus, this research opens up the opportunity for the present generation to experience remote sites of architectural and cultural importance through virtual worlds and real-time mobile applications. Variations of a recreated 3D monument to represent an amalgam of various cultures are targeted for future work.

An Approach for 3D Building Model Retargeting based on Structural Inference

We introduce an approach to generate many variations of a 3D building that looks similar to a 2D facade image. Computing such new facade variations has many unique challenges and we propose a new method to allow a casual user to quickly and easily generate many variations of an architecture given its structural inference. We demonstrate interactivity and variability of the initial test results of our approach using many examples modeled after real-world architectures exhibiting various forms of variations.

Collimation Using Transparent Projection Screen for Augmented Environment HUDs

HUD is a visual media technology in which 3D virtual objects are integrated into the real world in less intrusive way. It can provide information about the status and location of the surrounding environment in real time. This paper proposes an AR-HUD system to visualize the “out-of-the-window” scene and spatially align the real and superimposed objects i.e. collimation in accordance with the user’s line of sight. This is achieved through mapping the frames of references of the real site and the HUD system. The interaction between the HUD and the user is provided using an optimized head tracking approach. Observation of the people and vehicle movements in the outside world is dynamically represented in our AR HUD system. The system is further enhanced with the use of transparent projection screen with varying voltage regulation capability to adjust the transparency of the screen for observing distant objects. To evaluate the feasibility of this system, we have considered a construction site scenario which is populated with buildings, equipments, workers, and vehicles. Preliminary evaluation of the initial field tests quantify the performance of our system, compared with the real far view.

Methodology and Evaluation of Terrain Fusion Based on Different Resolution 2D Images

Currently, digital terrain models (DTMs) are important constituent of any urban navigation system. When modeling the earth's surface in digital form, there occurs voids or gaps in the terrain data. This may result in an erroneous representation of the geographic information system. Terrain fusion is suggested as a solution to fill the voids in the data representation. This paper focuses on this idea and presents a novel terrain fusion methodology and evaluation using satellite imagery (SRTM) and 2D map representation (NDM). We use iso-elevation contour map, a compact feature based representation to evaluate the reference terrain (SRTM) and the fused terrain (SRTM and NDM). We experiment with these criterions of aerial difference, structural similarity and height deviation to estimate the accuracy of fusion methodology. The result shows that the proposed methodology can improve the accuracy in the terrain matching.

Balancing usability and security in gesture based locking systems

Information Security is an important aspect to be considered in any authentication systems including locking systems. Balancing usability and security has been researched more than a decade. Improving usability and security parallelly needs to be considered due to increasing theft and online frauds. This research focuses on using mid-air based gesture as a solution to the above problem and discusses its advantages over existing ones.