Technical Details
Key Characteristics of HuBotVerse
Security and Privacy:
The framework features a security and privacy control module utilizing password-based authentication. The users' operation data will be stored in their accounts to ensure security. This can also help reduce mutual influence among users in the meantime. General users can access the webpage and view environmental sensing data, and robot perception data, while only authorized users can access the robot control dashboard and remotely control robots.
Evidence:
Users can control the robots or actuators through a web browser by registering an account through the website. User information is stored in a built-in database (MongoDB) in Django and authenticated by the Django authentication system during login. Each user can be assigned different types of permissions. Users with read-only access could only view the web page of `multi sensors'(see Fig. 1). Individuals with robot control access can operate one or more specific robots (see Fig. 2).
To prevent unintended input, users must activate the keyboard functionality by clicking the `Turn on the keyboard' button. If abnormal states of the environment are discovered, the system can automatically inform security professionals to deal with emergencies.
Fig. 1 Smart House application: Example of environment monitoring for users with read-only access.
Fig. 2 Robot Control application: Example of robot control using the web-based control panel for users with robot control access.
User-Friendliness:
The framework is developed based on a modular and layered architecture, targeting at easy integration of various devices, functionalities, and services for different robotic applications. The framework supports multiple interaction modalities and has integrated various types of user interaction interfaces to ensure both accessibility and user-friendliness.
Evidence:
Our framework's design facilitates effortless interaction with highly complex robots, such as humanoid robots. An example of this interaction is demonstrated with an in-house customized humanoid robot (named `SimNao') using the control panel on the web service provided by the IoHRT framework, as shown in Fig. 3. Additionally, various types of HRI interfaces can be utilized to acquire human control commands to control different types of robots, including but not limited to Joystick controllers, Haptic controllers, Handheld controllers, and wearable motion-capture devices. These interfaces enable seamless interaction between human operators and robotic things.
The proposed framework has an easy-to-use control panel, which can enable users to add, delete, view, and manage various types of robotic platforms.
Fig. 3 Interaction with a humanoid using a user-friendly control panel on the web service provided by the IoHRT framework.
(a) Overview of the control panel.
(b) Illustration of the motion control of the in-house customized humanoid robot, called `SimNao'.
Compatibility:
A key design consideration for the framework is its compatibility with a wide array of robotic platforms, actuators, and sensors. To facilitate this, data structures used for information exchange should conform to predefined protocols.
The framework should also be designed to be lightweight and executable on various operating systems. Additionally, an upgrade pathway should be provided for researchers with developer access.
Evidence: The proposed framework is capable of integrating diverse types of HRI interfaces and robotic platforms, which may include commercial or custom-built robotic arms, mobile robots, mobile manipulators, and humanoid robots. For ROS-based robotic systems, a ROS and TCP/IP connection package with client and server nodes can be used to easily integrate those systems with our framework. The processed information can then be transmitted to the web service for visualization.
Fig.5. Sketch of the data flow during human-in-the-loop control, which demonstrates the compatibility of different types of HRI interfaces.
Manageability:
The framework is expected to be easy to manage by streamlining the device customization process for users. It should support diverse combinations of interfaces, strategies, and platforms while offering adaptability for various applications.
The modular design of the architecture reduces system integration efforts when changing tasks, interaction interfaces, environment perception modules, and robotic platforms. Different combinations of interaction interfaces, mapping strategies, and robotic platforms can be chosen for different applications without redesigning the teleoperation system.
Evidence:
As illustrated in Fig. 6 (b), users can effortlessly add, view, edit, and delete sensors, robots, and actuators. Whenever a new device is created, its information is automatically saved as a new collection in MongoDB. All messages sent and received by the device are stored in this collection. Users can define the degree of freedom(DoF) and the joints' limits of the newly added robot.
Fig. 6 (c) depicts the creation of a Mycobot280 robot arm (Elephant Robotics) with ID Mycobot280_manipulator_0, which has six DoF for end-effector pose control, along with an additional DoF for controlling the gripper angle. The edit webpage function enables users to modify these parameters easily.
Illustration of the user control panel on the web page and the experimental setup of a case study. (a) Interaction with a humanoid using a user-friendly control panel on the web service provided by the IoHRT framework. (b) Illustrate the user interface for adding, viewing, editing, and deleting robots. (c) Illustrate the process of adding specifications for a new robotic system.
Training and Support:
Comprehensive documentation and tutorials are made available to support users in integrating their customized robotic platforms into our system. Ultimately, the software architecture should be open-sourced to benefit the broader robotics community.
Evidence:
We implement mechanisms for regular evaluation and feedback from users to identify areas of improvement, address issues, and continuously enhance the user experience of using the framework. We provide references and learning materials, which can enable users to make good use of open APIs for third-party integrations. Documentation and tutorials are publicly available on our project website and will be updated regularly. We envision that our proposed framework can enable users to integrate their customized robotic platforms into the system.