Research

Urban streets constitute the nervous system of a city and the issues related to streets may impact various aspects of city life such as transportation, public health, pollution, economy, and tourism. Given that an efficient monitoring of wide area urban streets is in need, manual monitoring is impractical; hence automatic street monitoring might be more feasible. Towards this end, automatic street monitoring solutions using visual data have been proposed, such as traffic flow analysis, road damage detection, street cleanliness, graffiti detection, air pollution detection, and water leakage detection. Most of these solutions require a visual dataset along with a machine-learning algorithm to train a model capable of detecting specific objects. This project investigates various object detection and classification applications to analyze street scenes at a metropolitan scale.

This project is to design and develop a platform, dubbed “Translational Visual Data Platform (TVDP)”, to collect, manage, analyze urban visual data which enables participating users connected not only to enhance their individual operations but also to smartly incorporate visual data acquisition, access, analysis methods and results among them. Specifically, we focus on geo-tagged visual data since location information is essential in many multimedia applications and provides a fundamental connection in managing and sharing data among collaborators. Furthermore, our study targets for an image based machine learning platform to prepare users for upcoming era of machine learning and AI applications. TVDP will be used to pilot, test, and apply various visual data intensive applications in a collaborative way. New data, methods, and extracted knowledge from one application can be effectively translated into other applications, ultimately making visual data and analysis as an infrastructure. The goal is to make value creation through visual data and their analysis as broadly available as possible, thus to make social and economic problem solving more distributed and collaborative among users. 

There is a consensus today that the market potential of the emerging Internet of Things (IoT) is enormous and it will globally change the way we live and work. However, IoT applications have no value till they have data, yet device owners are nervous about disclosing their personal data to unknown parties. Until this chicken-and-egg problem is solved, application developers will struggle to break free and unlock the potential that IoT represents.

This project proposes to develop a new platform, named the I3 Marketplace, to integrate IoT applications with financial transactions and brokerage concepts for IoT data and services (including marketing and pricing) so that IoT sensor data, access to actuators, and relevant data processing/analytics functions (through APIs) can be efficiently listed, searched, and traded among users (i.e., sellers, buyers, and brokers). Our I3 Marketplace effort envisions a world where application developers can gain access to the myriad of sensors and/or actuators that others have deployed and connected to the network, and sensor owners can take the initiative and deploy intelligent sensors in anticipation of an emerging and independent application market that will utilize their data for the benefit of its users. With the I3 Marketplace platform, IoT device owners will be allowed to access/trade their different kinds of sensor data and actuator access with many different vendors to create a supporting environment so advanced data analytics programs can be efficiently developed and supported in a multivendor-multi-owner device environment. The I3 Marketplace also provides methods to make it much easier to develop and deploy IoT applications and devices by maximizing the level of data reuse and interoperation among different applications. If successful, this project could spur the creation of new companies and business models related to providing data brokering and real-time streaming data processing services for end users, in the context of smart cities.

The I3 Marketplace is being developed as a unique interdisciplinary collaboration between the USC Viterbi School of Engineering (through CCI and IMSC centers) and the USC Marshall School of Business (through its CTM institute).

Large-scale spatial-visual search faces two major challenges: search performance due to the large volume of the dataset and inaccuracy of search results due to the image matching imprecision. First, the large scale of geo-tagged image datasets and the demand for real-time response make it critical to develop efficient spatial-visual query processing mechanisms. Towards this end, we focus on designing index structures that expedite the evaluation of spatial-visual queries. Second, retrieving relevant images is challenging due to two types of inaccuracies: spatial (due to camera position and scene location mismatch) and visual (due to dimensionality reduction). We propose a set of novel hybrid index structures based on R*-tree and LSH, i.e., a two-level index structure consisting of one primary index associated with a set of secondary structures. In particular, there are two variations to this class: using R*-tree as a primary structure (termed Augmented Spatial First Index) or using LSH as primary (termed Augmented Visual First Index). We experimentally showed that all hybrid structures greatly outperform the baselines with the maximum speed-up factor of 46. 

In a disaster, fast initial data collection is critical for first response. With the wide availability of smart mobile devices such as smartphones, a dynamic and adaptive crowdsourcing on disaster and after disaster has been getting attention in disaster situation awareness. We have been working on maximizing visual awareness in a disaster using smartphones, especially with constrained bandwidth resources. Specifically, We are currently performing a federally funded international joint project (US NSF and Japan JST joint) about data collection in disasters using MediaQ.