Wim Pouw, Mark Dingemanse & Asli Özyürek

Gesture network analysis reveals structural properties of change in evolving languages in the lab

We introduce gesture network analysis (Pouw & Dixon, 2019) as a quantitative, reproducible, automatable method for understanding how communicative bodily expressions undergo structural changes during simulated language evolution.

Iterated learning paradigms have been used to understand the constraints of evolving communication systems (e.g., Motamedi, Schouwstra, Smith, Culbertson, & Kirby, 2019; Raviv, Meyer, & Lev-Ari, 2019; Sato, Schouwstra, Flaherty, & Kirby, 2019). Such experimental paradigms aim to simulate evolving language dynamics that naturally occur in larger populations and on longer time scales, e.g., the evolution of sign language over generations of users (e.g., de Vos and Nyst, 2018; Senghas, Kita, & Özyürek, 2004; Sandler, Aronoff, Meir & Padden, 2011).

The promise of iterated learning paradigms is exemplified by experiment 1 of Motamedi et al. (2019), wherein pairs of participants learn to associate a set of concepts (e.g., hand-cuffs, prison etc.) with silent gestures. The first generation of participants learns these silent gestures from seed participants asked to spontaneously produce silent gestures for a set of concepts. After learning the gesture, the pair belonging to the first generation is asked to interact in a director-matcher task to solidify their gestures to be communicated to a next generation of participants. This procedure is repeated up to the 5th generation for 5 unique chains of participants. Systematicity, measured in terms of Shannon entropy of hand-coded gesture content, decreased over generations, indicating simpler gesture content being used as the communication system evolved.

Recently Pouw & Dixon (2019) have showcased the potential of combining time series analysis with network analysis so as to gain understanding how kinematics of different gestures interrelate in structural ways on the level of a gesture ensemble (a larger set of gestures). The method uses dynamic timewarping (e.g., Sato et al., 2019) to measure the kinematic similarity of gesture trajectories, and based on this, builds distance matrices in which each gesture is compared to each other gesture. It then uses network visualization and topology and complexity measures to probe structural properties of gesture networks.

We used gesture network analysis to reanalyze the data of Motamedi et al. )2019; experiment 1). We extracted kinematic positional data for both hands and the head with Openpose (Cao, Simon, Wei, & Sheikh, 2017) and performed dependent multivariate Dynamic Time Warping (Giorgino, 2009) on each gesture pair, retrieving a distance measure of how well the trajectory of each gesture matches the trajectory of another gesture. Then we constructed distance matrices for networks of gestures within a generation and probed whether the structure of these networks changed over the generations. Through hierarchical clustering analysis we find that gesture networks become less cluttered (i.e., agglomerative clustering coefficient increases, p = .012), suggesting that gestures become less similar to their nearest neighbors over the generations. Additionally, we find that Shannon entropy of the network connections go down, suggesting that the gestures kinematic relations become more similar to each other (p = .008), i.e., increases in systematicity.

Our analyses show how gestural kinematics evolve in structural ways in language evolution. As a fully automated, reproducible method, gesture network analysis can be scaled to very large datasets, supplementing hand-coded procedures. More importantly however, gesture network analysis provides new ways to show how structure may emerge within the interrelationships of communicative tokens. We will discuss further analyses in our presentation, as well as shortcomings and promising avenues of application. Shortcomings of the current analysis are that it will judge a gesture to be different from another gesture if the order of otherwise identical segments are reversed in another gesture. A human rater might judge such gesture phrases to be similar in form. Additionally, this analysis is only sensitive to changes in gesture form (kinematic trajectories), not content.