A critical part of AutoSense is enabling rich, real-time context inferencing using the sensors on the chestband and mobile phone. Real-time inferencing is needed to respond to errors and anomalies in the signals and display questionnaires to study participants in the field. The former is critical to collecting valid field data for scientific analysis, and the latter is critical for both collecting information from participants in the field and training or validating inferencing algorithms using participant responses.
We have several field studies planned that leverage AutoSense to study stress in the natural environment. Stress is a leading cause of mental and physical health problems. To reduce the occurrence of stress-related health problems, behavioral scientists need a better understanding of stress in the natural environment. Thus, AutoSense focuses specifically on inferring contexts associated with stress (e.g., stressors and coping behaviors). The following list represents the stress-related contexts we plan to infer in the field in real-time.
We are working on real-time machine learning algorithms that infer a person's stress levels from their physiology. This work addresses two major challenges:
low-power respiration band that is part of AutoSense to search for a signature for smoking. The figure below compares the respiration signature of a person breathing quietly while working to a person smoking a cigarette. Clearly, smoking has a respiration signature that is different from other common respiration patterns, and we are working towards identifying this pattern in real-time to infer smoking. This research is being conducted by Somnath Mitra, Amin Ahsan Ali and Mahbubur Rahman.
mobile phone microphone and the low-power respiration sensor. These properties include when the user is in conversation, when the user is speaking, when others are speaking, the number and genders of conversation participants, and the tenor of the conversation (serious, joyfyul, formal, informal, etc.). This research is being conducted by Amin Ahsan Ali and Mahbubur Rahman.
Several of these features will require the use the microphone on the AutoSense mobile phone. However, using the microphone has several drawbacks. First, the microphone is a relatively high-power sensor. It cannot be kept running continuously without sacrificing the battery life of the mobile phone. In addition, mobile phones are frequently kept in the pocket or purse. This means the microphone will often be covered such that it will be difficult to obtain a high quality audio signal for inferencing.
Thus, our current efforts focus on using the signal from the low-power respiration sensor to detect conversation. The microphone can then be used sparingly to infer specific properties of the conversation described previously (e.g. gender of conversation participants). The figure below shows that conversation has a different respiration signature than quiet breathing. Furthermore, this signature is also different from the smoking signature shown earlier. We are working towards identifying this pattern in real-time to infer smoking.
As there are many types of interruptions, we focus specifically on interruptions from 1) ecological momentary assessments (EMA) and 2) conversation. The former will enable scientists to reduce the burden of EMA on study participants EMA, and the latter will enable development of stress/interruption-aware people and devices. People need better awareness of stress to help them manage everyday stress. Likewise, stress-aware devices will be be capable of predicting and responding to stressful events before the user can. For example, a stress-aware mobile phone will be capable of holding all calls from the boss - a stressor for some individuals - when the user is already quite stressed.
To detect interruptions from conversation, we leverage the conversation inferencing described previously. In addition, we hypothesize interruptions will coincide with a unique physiological "startle" response. Thus, our efforts focus on detecting the startle response and conversation from respiration signals, as well as merging these two events to enable high accuracy detection of everyday interruptions. This work is led by Andrew Raij in collaboration with Amin Ahsan Ali and Mahbubur Rahman.
 D. McFarlane. Comparison of four primary methods for coordinating the interruption
of people in human-computer interaction. Human-Computer Interaction,
 D. McFarlane and K. Latorella. The scope and importance of human interruption
in human-computer interaction design. Human-Computer Interaction, 17(1):1–
 B. P. Bailey and S. T. Iqbal. Understanding changes in mental workload during
execution of goal-directed tasks and its application for interruption management.
ACM Trans. Comput.-Hum. Interact., 14(4):1–28, 2008.
 M. Danninger, E. Robles, A. Sukumaran, and C. Nass. The Connector Service:
Representing Availability for Mobile Communication, chapter 19, pages 235–56.
Computers in the Human Interaction Loop. Springer London, 1 edition, 2009.
 S. Iqbal and B. Bailey. Investigating the effectiveness of mental workload as
a predictor of opportune moments for interruption. In Conference on Human
Factors in Computing Systems, pages 1489–1492. ACM New York, NY, USA,
 J. Fogarty, S. Hudson, C. Atkeson, D. Avrahami, J. Forlizzi, S. Kiesler, J. Lee,
and J. Yang. Predicting human interruptibility with sensors. ACM Transactions
on Computer-Human Interaction (TOCHI), 12(1):119–146, 2005.
 J. Fogarty, S. Hudson, and J. Lai. Examining the robustness of sensor-based statistical
models of human interruptibility. In Proceedings of the SIGCHI conference
on Human factors in computing systems, pages 207–214. ACM New York, NY,
 J. Ho and S. Intille. Using context-aware computing to reduce the perceived burden
of interruptions from mobile devices. In Proceedings of the SIGCHI conference
on Human factors in computing systems, pages 909–918. ACM New York,
NY, USA, 2005.
 E. Horvitz and J. Apacible. Learning and reasoning about interruption. In ICMI
’03: Proceedings of the 5th international conference on Multimodal interfaces,
pages 20–27, New York, NY, USA, 2003. ACM.
 A. Kapoor and E. Horvitz. Experience sampling for building predictive user models:
a comparative study. In CHI ’08: Proceeding of the twenty-sixth annual
SIGCHI conference on Human factors in computing systems, pages 657–666,
New York, NY, USA, 2008. ACM.
 Maurer A et al ‐ Activity recognition and monitoring using multiple sensors on different body positions ‐ 2006.
 Bonomi, AG et al ‐ Detection of Type, Duration, and Intensity of Physical Activity Using an Accelerometer ‐ 2009
 Bao, L; Intille, S S ‐ Activity Recognition from User‐Annotated Acceleration Data ‐ 2004