Research Areas

Human Cognition and Next Generation Computing

This powerpoint presentation outlines the approach of our work, defining the centrality of human semantics and cognition in the evolution of next-generation information systems. As processors and communications become faster and storage becomes larger, computational barriers to finding patterns in complex data diminish. The next challenge will be in providing systems that enhance human problem solving and decision-making. This presentation identifies some key areas in this emerging field.

B. Rogowitz, " Human Cognition and Next Generation Computing" Keynote presentation, Joint Technical University of the Netherlands "Winterschool," February, 2009.

Use of Color in Visualization and UI Design- why perception matters:

Visualization involves mapping data onto visual dimensions. Understanding the perceptual effects of these visual choices can help enhance the salience of features and ensure that the structural features of the data are not unwittingly misrepresented.

Two color maps have been applied to exactly the same data. The rainbow colormap does not map equal data values equally. The "perceptual" colormap to the right has luminance increase monotonically above sea level and decrease monotonically below sea level. This simple mapping reveals the structure in the data.

The goal of this research area is to conduct experiments on the visual impact of spatial and color dimensions, then incorporate this guidance as rules in visualization systems. Although these perceptual rules apply generally, a key focus of this work was to develop an approach that could be applied to interactive systems that allow the user to do “what-if” experiments on the fly. The patent “Interactive Rule-Based Systems” provides a method for presenting the users with rule-based selections, on-the fly, based on user behaviors. One implementation of this patent was PRAVDA, a Perceptual Rule-Based Architecture for Visualizing Data Accurately, which provided guidance to users on the use of color maps that depend on the structure of the data, the users’ task, and the visual perception of color scales. A second implementation was CRAFT, a Color Rule and Font Tool, for selecting visible and effective colors, fonts and patterns in applications and web pages.

Bergman, L.D., B.E. Rogowitz, L.A. Treinish, “A Rule-based Tool for Assisting Colormap Selection,” Proceedings of IEEE Conference on Visualization, 1995.

Rogowitz, Bernice and Lloyd Treinish, “How NOT to Lie with Visualization,” Computers in Physics, Issue 3, May/June,1996, 268 - 273.

Rogowitz, Bernice and Lloyd Treinish, "Data Visualization: The End of the Rainbow" IEEE Spectrum 35, Issue 12, pp. 52-59, December, 1998

Rogowitz, Bernice E., Alan D. Kalvin, Adar Pelah, Aron Cohen, Which Trajectories through which Perceptually Uniform Color Spaces Produce Appropriate Color Scales for Interval Data? invited paper, IS&T Color Imaging Conference 7, Scottsdale, AZ, November 1999.

Rogowitz, Bernice and Alan Kalvin, "The 'Which Blair Project:" A Quick Visual Method For Evaluating Color Maps," Proceedings. IEEE Visualization Conference, San Diego, CA, October 21-26, pp. 183-188, 2001

Four different color maps applied to Tony Blair's image. Luminance (1) and Colored luminance (2) color maps provide good representations of the data. The Iso-luminant (3) and Rainbow (4) colormaps provide depleted or distorted renderings.

Interactive Data Exploration

The goal of the Diamond and Opal visualization systems was to provide powerful mathematical analytics integrated with interactive visualization. The analysts’ workbench version of this system was licensed to BMDP and SPSS, and the API version was integrated into several IBM applications, including the IBM Fraud and Abuse Management System. These libraries were important tools in many projects across a wide range of disciplines, such as Financial Risk Analysis and Bioinformatics. Merging the multidimensional parametric visualization of Diamond with 3-D visualization provided eye-opening understanding of how patterns in parameter space were painted out in the 3-D model, and vice versa, enabling novel applications, such as the collaboration on Cardiac modeling with Johns Hopkins University. A stand-alone version of this software is available on Source Forge, under the name ViVA (Visualization and Visual Analysis) Workbench.

The Visualization and Visual Analysis (ViVA) Workbench provides multiple linked visualizations plus an extensive math library supporting interactive visual analysis and visualization.

Finding Patterns across Multiple Data Types:

D.L. Gresh, B.E. Rogowitz, R.L. Winslow, D.F. Scollan, C.K. Yung “WEAVE: A System for Visually Linking 3D and Statistical Visualizations, Applied to Cardiac Simulation and Measurement Data,” Proceedings of the IEEE Conference on Visualization 2000.

An application that dynamically links a 3-D visualization of the heart with multivariate visualizations of the model parameters, enabling insight across different data types. Here there are two different classes of finite elements, distinguished by the value of calcium in the subspace (bottom), which occupy common spatial location in the 3-D heart in a tesselated pattern (top left), and have a very small range of the variable Ispace (top right).

Image Semantics and Search- using perceptually-determined features to index images

In a novel set of experiments, we looked at the question of digital libraries from a unique perspective. Instead of characterizing an image by simply enumerating its visual aspects, she conducted perceptual experiments that revealed the visual dimensions human observers used when judging image similarity. Using multidimensional scaling and hierarchical cluster analysis, we were able to identify key building blocks, which could be used to define a semantic language for images, which could serve to drive a perceptually-based image retrieval system, ISee.

B.E. Rogowitz, T Frese, J. Smith, CA Bouman, E Kalin, “Perceptual Image Similarity Experiments,” Conference on Human Vision and Electronic Imaging, 1998

A Mojsilovic, J. Gomes, and B. Rogowitz, “Semantic-friendly Indexing and Quering of Images Based on the Extraction of the Objective Semantic Cues”, International Journal of Computer Vision, 2004 Pages 79-107

A. Mojsilovic and B.E. Rogowitz, "Semantic Metric for Image Library Exploration”, IEEE Transactions on Multimedia, Volume 6, Issue 6, 828-838, 2004.

J. Chen, T. Pappas, A. Mojsilovic, B Rogowitz, ”Adaptive Image Segmentation Based on Color and Texture,” Proceedings of the International Conference on Image Processing, 2005.

Multidimensional scaling shows that human observers conceptualize images along two main axes: Natural vs. man-made (bottom left to top right) and non-human vs. human (top left to bottom right).

User-Generated Semantic Tags for Multimedia Search

InSight is a social software system that allows the on-line community to add semantic "tags" to subsets of information in complex media, such as videos. Since these user-created "micro-tags" are each associated with a URL, they can be used for search and composition. The idea is to give people access to information hidden within videos. Our experiments show that when a community of users is allowed to see and edit each others' tags, performance increases, and the number of tags decreases. This idea can be extended to give semantic tags to information hidden within other complex objects, such as web pages, analyses, or code.

.

In InSight, users can tag a video clip and also tag spatial objects within that clip. Each tag is a URL, which can be searched. this gives users the ability to find information hidden deep within a video. Community editing can improve tag quality.

Rogowitz, Bernice E. and Mercan Topkara, "Tags, Micro-Tags and Tag Editing: Improving Internet Search," Proc. SPIE (Conference on Human Vision and Electronic Imaging), Volume 7240, 65, January 2009.

Topkara, Mercan, Bernice Rogowitz, Steve Woods, Jeff Boston "Collaborative Editing of Micro-Tags," ACM CHI Works in Progress, April, 2009.

Perceiving the Texture, Geometry, and Quality of 3-D Objects

H. Rushmeier, B.Rogowitz, C. Piatko, “Perceptual Issues in Substituting Texture for Geometry,” Proceedings of the SPIE Conference on Human Vision and Electronic Imaging, 2000.

B. E. Rogowitz, H. Rushmeier “Are Image Quality Metrics Adequate to Evaluate the Quality of Geometric Objects? Conference on Human Vision and Electronic Imaging VI, Rogowitz, Pappas (eds.), Proc. SPIE Vol. 4299, 2001.

Display Technology

We have worked on visual aspects of display technology, including the development of a psychophysical technique for measuring perceived flicker on CRT displays that became a standard in IBM and contributed to the DIN standard. Our xperiments in the perceptual effects of spatial and color sampling contributed to the design of the earliest LCD displays, before they were even constructed.

Haptics

We have developed a novel haptic device that can be used to allow users to feel, and more important, edit surfaces in visualization and virtual reality environments.

B. E. Rogowitz, P. Borrel, "Virtual hand: a 3D tactile interface to virtual environments" Proc. SPIE (Conference on Human Vision and Electronic Imaging), Vol. 6806. January, 2008.