There are two cameras being used in this multi-agent system, one is the web cam of the computer the other, the camera of the UAV, the first camera helps the doctor to control the flying device and also uses AI for computer vision mechanisms that allow the pilot to see himself, while performing his tasks, the second camera is in the drone, that will treat the Photoplethysmogram (PPG) signal obtained allowing to know the BPM and other important medical information via the face detected by the done camera. The level of light influences the accuracy of the readings, but after tests even in low light environment the levels obtained are almost the same as the one perceived by medical instruments like infra-red (IR) thermometers or oximeters.

The system has several contents to facilitate the primary care diagnosis like recording sessions, playing a serious game to get the important rest state of the patient. The visual test can be done with an eye gaze controller, helping the healthcare professional to know where the user is looking during a virtual consultation. A gesture control was also implemented to facilitate the usability of people with visual disability or without a mouse or keyboard among other possibilities. Another feature is the multi video chat room allowing senior doctors to teach skills and how to use the system. For the pulse measurement, an API receiver mechanism from the developed mobile application in java to get the cardiac frequency via infra-red (IR) camera was implemented. The video chat constructed mainly via NodeJS allows to interact with the patients with mixed reality (MR) and obtain the vital signs data on real time, helping the doctor to decide if an emergency procedure should be triggered. The health state values are obtained on real time, and the video chat gives the possibility to the doctor to provide instructions to the patient. The drone can facilitate the process of reaching several areas, providing important information like the location coordinates and the vital signs of a user, the research would allow an innovative approach of performing general clinic activities when compared with traditional methods and can be very helpful for situations of contagious diseases or cases that require constant monitoring. The AI mechanisms of the virtual elements can be used to control drones and also for reporting vital signs of a user after ROI detection.

Application developed with serveral technologies like PHP and Python Flask

To create the database use the 127_0_0_1.sql

Requires an application web server to be running