This research explored how existing industrial robotic systems could be upgraded for safer and more collaborative Industry 5.0 applications without requiring manufacturers to completely replace legacy equipment. Our proposed approach focused on augmenting traditional industrial robots with external sensing, digital-twin technology, and real-time workspace awareness so that they could operate more safely around human workers.

The project centered on developing a digital-twin framework in Siemens Tecnomatix Process Simulate using FANUC M-410iB industrial robots and a simulated manufacturing workspace. The proposed system incorporated 3D LiDAR and vision-based cameras to monitor the shared workspace, with YOLO-based computer vision and LiDAR-generated occupancy data supporting real-time human tracking. The digital twin was intended to visualize robot behavior, human presence, and dynamic safety zones within one simulation environment.

A major part of the research involved evaluating how human-robot collaboration could be implemented safely using ISO/TS 15066 protective-separation requirements. The project considered variables such as human speed, robot speed, robot stopping time, sensor reaction time, and stopping distance to determine how the robot should respond as a person approached the workspace. The broader goal was to enable behaviors such as reduced-speed operation or a monitored stop based on real-time proximity.

The research also examined the role of IIoT and sensor integration in connecting physical equipment with the digital twin. Future implementation plans included streaming live or near-real-time sensor data into Siemens Tecnomatix, validating robot and workspace positions, and comparing virtual safety behavior with real-world testing. This would allow the digital twin to support early-stage validation of human-robot interaction before exposing operators to physical risk.

Overall, the project proposed a cost-conscious path for manufacturers transitioning from Industry 4.0 to Industry 5.0 by extending the useful life of existing robotic infrastructure rather than relying entirely on new collaborative robots. The research emphasized digital twins, external sensing, human-aware safety, and modular upgrades as a way to make industrial automation more adaptive, collaborative, and sustainable.

Technologies / Research Areas: Siemens Tecnomatix Process Simulate, FANUC M-410iB, Digital Twins, Industry 4.0/5.0, Human-Robot Collaboration, LiDAR, Machine Vision, YOLO-Based Human Tracking, IIoT, ISO/TS 15066, Collaborative Robot Safety

This research explored a more accessible approach to precision agriculture by investigating whether crop-health information normally obtained through costly multispectral imaging could instead be estimated using standard RGB imagery. Our team focused on the Normalized Difference Vegetation Index (NDVI), a commonly used indicator of vegetation health, with the goal of developing an AI-based method that could make crop monitoring more affordable for farmers with limited access to specialized imaging equipment.

Our methodology began with a MAPIR Survey3W multispectral camera, which allowed us to collect RGB and near-infrared image data and generate NDVI values for comparison. We worked through camera calibration, image collection, dataset creation and annotation, and preparation for training an artificial neural network capable of estimating NDVI from regular RGB images without requiring the near-infrared band.

During the project, we successfully captured multispectral images and generated NDVI measurements through the camera’s post-processing workflow. This also helped us identify an important limitation of the initial process: manually selecting files for post-processing would not scale well to real-time or large-volume agricultural monitoring. That became a key motivation for moving toward a more automated RGB-based approach.

The broader goal was to develop a low-cost crop-monitoring system that could eventually support real-time analysis using consumer-grade cameras. Future directions included live NDVI monitoring, testing an RGB-only workflow, integrating the system with drones for larger-scale crop monitoring, and using a neural network to automate NDVI estimation and analysis.

Technologies / Research Areas: Multispectral Imaging, NDVI, RGB/NIR Imaging, Artificial Neural Networks, Dataset Creation, Image Processing, Precision Agriculture

The integration of SLAM with YOLO object detection represented a significant advancement in robotics, particularly in enhancing environmental awareness and object interaction. Previous research demonstrated the potential of using renderers as recognition tools in robotic systems, thereby improving functionality and versatility. This study focused on implementing SLAM and YOLO with the Unitree Go1 Quadruped Robot Dog, exploring the synergy of environmental mapping and real-time object recognition to enhance the robot's spatial awareness and interaction capabilities. By evaluating the performance and precision of the Unitree Go1 through this integrated approach, which incorporated ROS and LIDAR with a Raspberry Pi 4, the aim was to develop a system that effectively mapped surroundings while accurately identifying and labeling key objects. The investigation was conducted in a controlled environment to assess the Go1's potential for diverse applications such as monitoring factory and warehouse environments.

The research began with the installation of ROS2 on a Docker container using Raspberry Pi OS. A basic talker program was run to verify the system setup, followed by the configuration of RViz and other necessary components. To integrate LiDAR with the Unitree Go1 robot, a LiDAR sensor was set up and calibrated to ensure that sensor readings were properly displayed in SLAM for effective environmental mapping. ROS was used to generate a 2D or 3D pixelated map of the surroundings.

For the YOLO integration, the YOLOv8 model was installed and tested on a separate Raspberry Pi. This model was then integrated with SLAM to enhance the robot’s mapping capabilities. Using the YOLOv8 camera, we identified and tagged objects in real time. The lab layout was used to create a 2D map via SLAM, while YOLO was tested for object identification, including people, tables, boxes, and chairs. We assessed the accuracy of object recognition before tagging results in the SLAM map. This process is aimed at improving applications in spaces such as factories, warehouses, and other automation environments.

The research explored several potential applications of the integrated system. In surveillance, the quadruped robot was used to monitor factory, warehouse, and automation spaces. It provided the ability to quickly map and identify objects in areas unsafe for human presence, improving security by detecting unauthorized objects or individuals in real time. Additionally, in smart homes, the robot could identify household objects, monitor activities, and detect potential hazards, offering users real-time alerts and insights to improve home security and efficiency.

Our research made significant progress in configuring LiDAR with RViz and the RP LiDAR puck. We successfully established a wireless connection between the Unitree Go1 robot and our personal devices, enabling more efficient and accurate testing. Moving forward, we plan to continue developing a labeled SLAM map using YOLOv8 and testing it in real-world environments, particularly in factory and manufacturing settings.

Technologies / Research Areas: ROS, RViz, SLAM, YOLOv8, RP LiDAR, Raspberry Pi, Unitree Go1, Computer Vision, Object Detection, Environmental Mapping

This research explores the applications of Large Language Models (LLMS) as it investigates the implementation of ChatGPT with the UnitreeGo1 Robot Dog, specifically focusing on voice prompts to instruct the UnitreeGo1 Robot Dog. This utilization simplifies the process of controlling the robot dog, allowing users with limited knowledge to execute commands. Additionally, those with coding experience can utilize this to expedite software development. Research in this technology holds immense potential for enabling users to explore and experiment with different functionalities of the robot dog, contributing to enhanced comprehension and testing abilities within the field of robotics. 

We achieved this by referencing DroneBlocks' UnitreeGo1 Programming YouTube playlist. While following these tutorials, I learned how to establish an API key for the communication between Go1 and the program's terminal through the additional Wi-Fi networks we set up. Throughout the process, we also learned about ROS (Robot Operating System) and its structure of nodes and packages. We learned how to translate and integrate different program files while navigating various programming repositories, such as GitHub and Hugging Face.

Once we accomplished implementing ChatGPT into the robot's program, we began to test its accuracy by providing commands. For instance, we asked ChatGPT, "Turn the LED lights the color of the sky," and it resulted in making Go1's LED lights sky blue. In addition, we asked it, "Do the Cha Cha slide," and the robot began dancing to the left, right, and turning while quickly switching its LED lights to the colors of the rainbow. However, we realized that qualitative versus quantitative verbiage was more efficient when providing commands. For instance, the robot dog would execute "Go forwards a little bit," more accurately than "Go forwards thirty-six inches," when prompted. 

The ideal applications of this research would be in robot efficiency testing, helping people with disabilities, robot-human interaction education in schools, and smart home automation.

Our research began in January 2024, and I had the honor to present with my partner at the Spring Symposium of Undergraduate Research in April. We plan on adapting a seamless communication between WhisperAI and the Unitree Go1 Robot Dog in our upcoming semesters for research.

Technologies / Research Areas: ChatGPT, Whisper AI, ROS, Python, API Integration, Unitree Go1, Natural Language Interaction, Human-Robot Interaction, Large Language Models