Principal investigator: Grazia Cicirelli

Start: 10/10/2018

End: 12/03/2020

The BESIDE project is a POR Puglia FESR-FSE 2014-2020 project, developed in cooperation with several industrial partners (Loran Srl, CVS Srl, E-COM Srl, Ideasviluppo Srl, Senior Srl, Visvisol Srl, IRTE Spa, Progetto Vita) and research units (Università di Bari, CNR).

The BESIDE project aims to develop an ICT-based system for monitoring elderly people suffering from neurodegenerative diseases such as the Alzheimer’s disease or dementia, while living at home or at residential care settings.

Once diagnoses are established, daily observations of the subjects are mandatory to prevent further clinical issues due to the loss of motor skills. This phase is often hard to be delivered to subjects forced at home or hospitalized in crowded structures. For this reason, the use of automatic vison systems, able to capture subject motor skills and to extract information about the disease progress, can considerably reduce the risk of further lethal injuries due to falls.

The ISP group is responsible for the development of a vision-based system for the functional monitoring of elder people with neurodegenerative diseases in order to evaluate their risk of fall.

Subjects monitoring is performed by a set of three low-cost cameras, namely the Hik Vision DS-2CD2385FWD-I, which has a 4K resolution at 20 fps. Three cameras are placed around three sides of a volume of interest (front camera, left camera, and right camera).

Within this volume, the subject performs three exercises whose outputs are expressed in terms of time duration and gait/posture measurements.

These tests are:

• Balance test: the subject stands with parallel feet for few seconds, then moves one foot ahead and tries to keep its position for further seconds;
• Walking test: the subjects walks for 4 meters from C_Sx to C_Dx and back;
• Sit to stand test: the subject stands up and sits five times keeping his arms crossed.

These tests give a representation of simple daily activities, which may become increasingly harder for elderly persons suffering from neurodegenerative disease. It is easy to understand that continuous monitoring of these activities in time is useful to determine the progress of the disease and to assess the risk of falling. Proper video processing techniques are thus fundamental to extract information from the captured videos and to automatically label the subjects abilities.

Low level information to be extracted from videos mainly regards relative movements of body parts. For this reason, it is useful to segment the shape of the subject under analysis in simple skeleton (skeletonization). This result can be achieved following these three steps:

1. Video synchronization: the three videos are manually analyzed to find common features, e.g. people movements, which enable the detection of the relative time shifts needed to synchronize the three videos;
2. Activity selection: synchronized videos are further separated, depending of the specific test performed;
3. Skeleton extraction: skeletonization is then performed singularly on each video by applying the OpenPose library based on deep neural networks.

Preliminary processing results are reported in Figure, where skeletons are displayed in colors. These examples are obtained for the three views and for the three tests of interest (sit-to-stand, balance and walking test).