Projects

Projects are in four areas:

1. Beautiful teamwork – connect with team members:

Measuring communication in teams for happier, higher performing teams. The quality of teamwork is also dependent on background music, immersion into nature, and well-designed architectural spaces.

In this project we study communication in teams through face-to-face and online social media interaction measured with the Happimetrics toolkit. The goal is to improve PERMA of participants in the team.

Subproject: Personal Job Coach (Predicting personalities and preferences from essays and prefered images)

In this project we aim to predict personality from text. The goal is to analyze text written by a person, and to deduct their preferences and characteristics such as risk-taking, FFI, moral values and ethical principles.

Subproject: Finding Characteristics of Long-Covid Sufferers from Online Social Media

Originally developed as a Master's thesis at HSLU, analyze  a large Reddit dataset of Long Covid suffers in combination with a similarly sized control group, to identify features, personality characteristics, and personal values of sufferers of Long Covid from their words and posting behavior on Reddit.

Subproject: Detecting lies

In the winter term of the COINs seminar, a team of students from LUT created a datasert where people told lies in front of a camera. The goal of this project is to build a machine learning model that will detect lies based on body posture and facial expression.

2. Beautiful music – connect musicians and audience:

Measuring the impact of music on musicians and audience, with the goal of improving musical performances leading to a better experience for the audience,  learning and teaching how to play musical instruments and perform in live concerts.

The goal of this project is to develop a multimodal emotion sensing toolkit for musicians that can be used to improve music education and musical performances. Our approach uses the happimetrics toolkit described above, which combines emotion sensing by face emotion recognition (FER), body signal analysis using the Happimeter, a smartwatch-based solution, and plant-based emotion sensing. 

Applying our approach to music education, we have already run a series of pilots with jazz musicians at concerts and rehearsals, for instance at the Berklee College of Music. Our most recent project is with Jazzaar, led by Fritz Renold, a 31-year music project located in Aarau, Switzerland. There young talented musicians are brought together with prominent jazz musicians for an intensive week of playing. The goal is for the young musicians to learn from the stars as role models through observation, and to perform with them in a series of concerts after a week of intensive rehearsals.

Research Goals

1. Understand the emotional process of the student while participating in face-to-face and online music lessons, in concert rehearsals, jam sessions, and in concerts

2. Understand the emotional process of the teacher while participating in face-to-face and online lessons

3. Understand the emotional process of the audience while listening to a concert

4. Applying the principle “the best against the rest” develop best practices guidelines for optimal teaching, rehearsals, and playing in concerts

5. Develop a virtual mirroring toolkit that will increase student and teacher effectiveness, by showing them how far they deviate from best practices, and what they can do to get close to these best practices.

3. Beautiful nature – connect the human to nature:

Measuring the impact of humans (performing Eurythmy) on plants and measuring the impact of plants on humans. Towards that goal, a device that measures changes in the action potential of plants in response to human emotions shown near the plant, will be built, consisting of dedicated hardware and software.

The Eurythmy subproject measures the influence of human interaction on plants. Salad, beans, and tomatoes are treated with eurythmy. The change in action potential is measured for untreated, single treated, and multiple treated plants using the plant spikerbox. Additionally, the change in action potential is also measured between different persons doing eurythmy.

In a second project a machine learning model for emotion tracking with the plant spikerbox and basil is developed. This is combined with team experiments. The action potential change for different emotions for a large cohort of study subjects is measured and taken as input data to build the model. In a later phase the goal is to build dedicated hardware for emotion tracking with a plant. This will be a non-intrusive, privacy respecting emotion tracking device which can be flexibly used for many different settings.

Subproject Happy-Cow

Orginally developled as a Master's thesis at HSLU, further develop an app to identify stress and happiness of cows from their body posture, facial expressions, and thermal body values.

4. Beautiful architecture – connect the human to the built environment:

Measuring the impact of well-designed and poorly designed architectural spaces on human wellbeing, and teamwork performance.

The goal of the “beautiful architecture” project is to measure the influence of the built environment on happiness and teamwork. This will include measuring architectural influence both on the macro building level, and the micro room level. We will measure emotions through FER in interaction with different types of buildings, for instance in museums, churches, or supermarkets, applying the 14 design patterns of biophilic design.

On the team level we will measure communication intensity and emotions for comparable teams in different rooms, for instance small workgroup rooms, large shared office space, or in the park. We hypothesize that the more a team room resembles a landscape in the park, the higher will the productivity of the team be.