Multimodal research is crucial to a number of different areas of technology development, from human computer interaction research that focuses on multimodal computer interfaces through multimodal modelling for human robot interaction to artificial intelligence, where the goal is to automatically understand multimodal human behaviour in context. Applications in these areas call for a well developed theory of multimodality and interaction and good models that can be utilized in building technology.

In order to be able to build these theories and models, the development of certain technological tools would be helpful. The most obvious of these is annotation tools that can deal with large amounts of multimodal or mixed media data. As a tool set this is reasonably well developed, with recent developments allowing for guided machine learning and automated annotation. They still have to be made useful for multimodal research since their development has largely been driven by the need for well annotated corpora of multimodal interaction for robotics and affective computing. Corpus tools are an obvious extension of annotators, with a focus on being able to search for similarly coded features in large multimodal datasets. Because one of the end goals is to be able to establish the alignment of different features across different modalities, visualization tools are another area where technological development could help further research in multimodality. What remains a challenge is complex visualization and the combining of different data types, particularly in real time. Tools here are relatively restricted because combining time aligned data types with non-time aligned data types is more difficult.

In this paper, we first discuss the different areas where multimodal research is being applied in technology and then look at how technology can be used for tool development to further research in multimodality by looking at the development, analysis and subsequent uses of two multimodal corpora.