Workshop Poster and Full Workshop Proceedings
Thank you to everyone who made the workshop a thought provoking day. Please take a look at our poster summarizing our discussion ( pdf ), and the workshop proceedings ( pdf ).

List of Participants
Christiane Robbins - University of Southern California
Position Paper

Eric Baumer, Bill Tomlinson - University of California, Irvine 
Advocating Polytheoretic Evaluation of
Interactive Art and New Media

Eric Paulos - Intel Research
HCI Cannot Be Used To Evaluate Art

Tim Coughlan, Peter Johnson - University of Bath 
Support for Creativity in Musical Collaboration
and Creator / Technologist Interaction

Charlotte Wiberg, Johan Bodén, Kalle Jegers 
- Umeå University
Cross-Media Interaction Design

Steven Dow, Blair MacIntyre - Georgia Tech 
New Media Collaboration through
Wizard-of-Oz Simulations

Karl Flieder - University of Applied Sciences, Graz, Austria
Learning from an Extended Context of 
Patterns in Science of Design

Ryan Shaw - University of California, Berkeley
Collaborations in Border Zones

Mary Flanagan, Helen Nissenbaum, Jim Diamond, Jonathan Belman - Hunter College and New York University 
HCI and New Media Arts:
Human Values Methods & Evaluation

Call for Participation ( pdf ) 
Practice and research in art and science today requires great attention to context as well as deep, exhaustive investigation. Through increasing demand for work that is both rigorously disciplinary and responsive to conditions outside of the discipline, art and science practitioners continue to search for innovative methodologies.

This workshop will focus on three areas in HCI/New Media collaboration: 
Contributions to Evaluation and Methodology
What methods are amenable to cross-disciplinary application? Where do practices within HCI or New Media stand most in need of influence from the other field's methods, and where do incompatibilities lie?

Informing Reflective Practice
What features of disciplinary practice are brought into relief through HCI/New Media collaboration? What techniques are appropriate when studying HCI/New Media collaborations? What kinds of research findings or innovations in practice do these collaborations promote?

Identifying Critical Issues in Collaboration
How do HCI/New Media collaborations deal with institutional constraints, like Institutional Review Boards. Can the diverse goals and expectations of collaborators be satisfied effectively?

Our intended audience includes researchers in HCI, industry experts and designers practiced in art/science collaboration and New Media artists whose work incorporates interaction.

Individuals interested in participating should submit a position paper, 2-4 pages in length. Participant position papers should either (i) situate the participants’ interests and background among the themes of the workshop, or (ii) report on preliminary research findings or case studies in HCI/New Media collaboration.

Papers should conform to the CHI 2007 Extended Abstracts format (http://www.chi2007.org/submit/eaformat.php). To participate, one author is required to register for the workshop and one day of CHI 2007.

Workshop Extended Abstract ( pdf )

Successful collaborations between New Media Arts and HCI tend to develop hybrid techniques that promote balanced contributions from both disciplines. However, since many of these collaborations are one-off or highly dependent on the researchers/artists involved, systematic discussions of the role and impact of the various evaluation techniques and methodologies are missing. This workshop is aimed at practitioners from both HCI and the Arts as a venue to discuss the contribution that one another’s techniques have made to their own practice, evaluate critical issues in HCI/New Media Collaboration, and examine ways that existing approaches can be extended for a deeper role in practice, design, and research.

Introduction

Practice and research in art and science today requires great attention to context as well as deep, exhaustive investigation. Through increasing demand for work that is both rigorously disciplinary and responsive to conditions outside of the discipline, art and science practitioners continue to search for innovative methodologies.

Similar to media rich and often interactive New Media art, more work in HCI is exploring artful interaction [4, 6], tools to support creative engagement [9], and constructed user experiences meant to elicit creative responses [3, 7, 10]. Projects such as these often move outside familiar modes of investigation, and HCI researchers / practitioners rely on the skills of collaborators. Correctly structured, these kinds of collaborations can push practitioners to augment the refined evaluative skills from the arts with the rigorous methodologies of scientific research.

Work in HCI that explores artful interaction [4,6], tools to support creative engagement [9], or sited interaction through pervasive computing struggles to produce quantitative evaluation of qualitative experience. Artists and designers sometimes struggle to apply training in individualized technical mastery to the more collaborative, often dematerialized processes of work outside the gallery context.

Prior models and theories of collaboration between artists and scientists often produced outcomes applicable only to one or the other discipline, or worse, ill-formed products that performed poorly in both areas. Our argument is for collaboration as a way of making all practitioners more competitive, attentive, and responsive within their own fields. Methodology (or "process") and evaluation (or "critique") are key areas of disciplinary difference that deserve to be examined and clarified, rather than transcended or obscured.

A reciprocal relationship can be created between the practices of art and science that preserves disciplinary distinctiveness while challenging all participants in the areas where their respective disciplines are weakest. [1, 2, 5, 8] This workshop is meant to serve as a forum for exploration of new projects and an opportunity for discussion of how HCI/New Media collaboration can extend existing techniques as well as open new possibilities in research and design.

Highlighting Contributions in Evaluation

A fundamental aspect of HCI research is matching projects to appropriate metrics of evaluation. Most metrics cleave the space fairly cleanly into qualitative or quantitative elements, leaving out much of the nuance in the increasingly complex experiences provided by HCI. In contrast, structured art criticism thrives on this nuance, highlighting and leveraging points of complexity that can convey deeper meaning for an engaged audience. And rather than provide subjective or ephemeral suggestions for improvement, techniques from the arts address specific issues (context, function, process, audience) that remain largely overlooked in HCI evaluation. A major workshop theme will be to explore the appropriateness of methods of evaluation across disciplines.

Informing Reflective Practice

Another major theme will explore how HCI/New Media practitioners can better study their own modes of collaboration. We will examine key elements in successful collaborations, with a focus on common traits that make techniques amenable to application across disciplines. Specifically, we will explore familiar HCI techniques including rapid prototyping and scenario-based design along with newer forms of collaboration like mashups and social bookmarking.

In addition to specific techniques, we will examine whether existing models of HCI collaborations with other disciplines (e.g. Psychology, Sociology) are informative when working closely with New Media. Are the interactions between people in the Arts and HCI just another manifestation of the power and problems of multidisciplinary collaboration? Are these essentially the same as in other cases, or is there something special about this particular disciplinary interaction?

Arts practitioners may feel constrained when imposing a formalized HCI approach on the artistic process, feeling instead more comfortable with a more naturalized or "organic" process. In collaborations, formal HCI techniques do provide additional, objectifiable requirements that can be debated and tasks that can be allocated and evaluated. In the best cases, these hybrid formal processes also ensured a more intentional and invested final product.

More generally, there remains the problem of ensuring "buy-in" to new methods and evaluative criteria across collaborative teams. For example, how can collaborators accomplish a smooth inclusion of documentation for the sake of collaboration, without inhibiting progress and intuitive flow?

Finally, how might public presentation and communication of highly situated HCI/New Media projects be made compelling to new audiences. What aspects of HCI are suited for a gallery or performance space, and which aspects of New Media presentation might aid in the reception of cutting edge research?

Identifying Critical Issues in Collaboration

For projects across disciplines with very different, multiple, and perhaps conflicting goals, participants need to be acutely aware of the needs and expectations of their collaborators. This workshop explores whether HCI/New Media collaborations are predominantly informing new kinds of artistic expression or creativity in human-computer interaction. Do members of the same team have similar opinions on the ultimate goals and directions of their collaboration? Are there aspects of HCI/New Media collaborations can help them stand without the need for a contextualization meant to appeal to a discipline specific audience?

HCI/New Media collaborations often have to satisfy a myriad of institutional constraints. The functions of collaboration may be drastically different when expressed as a vehicle for teaching as compared to research. Depending on the hosting agency or department, rules regarding compliance with Institutional Review Boards are particularly important. Human subject permissions are very seldom considered in the arts. Participation in artworks by special populations and children is often encouraged without any kind of informed or parental consent process. It is not uncommon for New Media pieces to collect data about participants (surveillance camera streams, web traffic/histories, biometric information) all without the bureaucratic controls imposed on similar research when performed solely by HCI practitioners. Post performance discussions (effectively focus groups) are also common practice, and are again not considered human subject research. At what point does HCI/New Media collaboration come under the purview of an IRB? Is the inclusion of HCI researchers enough to make collaboration a controlled pursuit? And if so, are there inevitable changes in how the Arts will respond to these new institutional framings and strictures?

Immediate and Long Term Workshop Goals

The goal of this workshop is to introduce HCI researchers and New Media practitioners to new modes of evaluation, expanded methodologies, and expose points for mutually constructive future collaboration.

This workshop, by bringing together a diverse group of practitioners, researchers, and artists provides a strong venue for establishing connections between communities, for sharing information about successful modes of collaboration especially within institutional contexts, and as a setting for a critically engaged audience to develop models for future work.

Within this context, the workshop will produce a working draft focused on the core themes, informed by participant presentations and discussion, to suggest areas for future research and expanded practice. Workshop participants will be solicited to provide longer versions of their position papers for future publication.

References

1. Adamczyk, P.D., Campbell, R. and Fineberg, J. 
   Public and Virtual Social Spaceshttp://orchid.cs.uiuc.edu/people/adamczyk/pvss/, 2006.
2. Adamczyk, P.D., Chambers, E., Campbell, R., Fineberg, J. and Hamilton, K. The Palace of Projects
   http://orchid.cs.uiuc.edu/people/adamczyk/final/, 2005.
3. Buchenau, M. and Suri, J.F., Experience Prototyping. in Conference on Designing Interactive Systems (DIS), (2000), 424-433.
4. Crabtree, A., Benford, S., Greenhalgh, C., Tennent, P., Chalmers, M. and Brown, B., Supporting Ethnographic Studies of Ubiquitous Computing in the Wild. in DIS, (2006), 60-69.
5. Dix, A., Ormerod, T., Twidale, M.B., Sas, C., Gomes da Silva, P.A. and McKnight, L., Why bad ideas are a good idea. in Proceedings of the First Joint BCS/IFIP WG13.1/ICS/EU CONVIVIO HCI Educators’ Workshop “HCIEd.2006-1 inventivity: Teaching theory, design and innovation in HCI”, (2006).
6. Edmonds, E., Candy, L., Fell, M., Pauletto, S., Weakley, A. The studio as laboratory: Combining creative practice and digital technology research. International Journal of Human Computer Studies, 63 (4-5). 452-481.
7. Gaver, B., Dunne, T. and Pacenti, E. Cultural Probes. interactions, 6 (1). 21-29.
8. Hamilton, K., Adamczyk, P.D., Levin, M.S. and Long, L., Mobile Mapping for Everyday Spaces. in Proc. Conference of the College Art Association, (New York, NY, 2007),http://www.art.uiuc.edu/projects/mobilemapping/
9. Jennings P. and Giaccardi, E. Creativity support tools for and by the new media art community. Schneiderman, B. ed. NSF Workshop Report on Creativity Support Tools, Washington, DC, 12-14 June, 2005, 37-52.
10. Paulos, E. and Jenkins, T., Urban Probes: Encountering Our Emerging Urban Atmospheres. in Conference on Human Factors in Computing Systems (CHI), (2005), 341-350.