Know How’s A Multi modal Agent System for Converting Manufacturing
A multi-modal AI framework designed to automatically transform unstructured manufacturing videos into standardized digital manuals, referred to as "Know-how Heritage." Unlike conventional video-to-text approaches, our system integrates Range of Motion (ROM) and Hand-Object Interaction (HOI) analysis to extract quantitative data on ergonomic postures and finegrained tool manipulations.
Multi-modal Tacit Knowledge Analysis: Designing a mechanism that extracts quantitative data on ergonomic postures and fine-grained tool manipulations by integrating Range of Motion (ROM) and Hand-Object Interaction (HOI) analysis into the video processing pipeline.
Automated SOP Generation: Implementing an LLM-driven system that synthesizes multi-modal insights—including temporal, physical, and verbal data—to automatically generate standardized, context-aware "Know-how Heritage" manuals.
Quantitative Performance Validation: Verifying the proposed framework through extensive experiments with 3,000 iterations on manufacturing datasets, demonstrating superior performance in factuality and structural coherence using G-Eval and METEOR metrics.
Generation of Critical Information and Sharing Mechanism for Robot Collaboration Mission Success - Ph.D study
This dissertation investigates collaborative mission success in multi-robot systems by proposing a priority-based real-time information generation and sharing mechanism, validated through experiments using TurtleBot robots in ROS2 and Gazebo simulation environments
Information Generation and Sharing: Designing a mechanism that enables robots to generate and share mission-critical information in real time to support collaborative decision-making.
Multi-Robot Coordination: Implementing autonomous task allocation and synchronized actions among multiple robots to ensure successful mission execution.
Simulation-Based Validation: Verifying the proposed approach through extensive experiments using TurtleBot robots in ROS2 and Gazebo environments.
논문 삽입 영상_250610_1st.mp4논문 삽입 영상_250613_2nd.mp4Advanced Deep Learning Applications in Unmanned Aerial Vehicles (UAVs)
We are dedicated to enhancing the autonomy and reliability of UAVs by leveraging cutting-edge deep learning models such as Convolutional Neural Networks (CNN), Long Short-Term Memory (LSTM) networks, and Transformers.
Model Predictive Control (MPC): Utilizing deep learning to predict and optimize UAV flight dynamics, ensuring precise control and stability during complex maneuvers.
Anomaly Detection: Implementing advanced neural networks to monitor UAV systems in real-time, swiftly identifying and addressing anomalies to prevent potential failures.
Autonomous Mission Execution: Developing intelligent algorithms that enable UAVs to perform missions autonomously, adapting to dynamic environments without human intervention.
Deep Learning-Driven Autonomous Ground Vehicles (AGVs) Missions
Our exploration extends to terrestrial platforms, focusing on the deployment of deep learning techniques to enhance the operational capabilities of AGVs.
Path Planning and Navigation: Employing deep reinforcement learning to enable AGVs to navigate complex terrains efficiently, learning optimal paths through continuous interaction with their environment.
Obstacle Detection and Avoidance: Utilizing sensor fusion and neural networks to accurately detect obstacles and execute real-time avoidance maneuvers, ensuring safe operations.
Mission Optimization: Applying machine learning algorithms to optimize task execution, improving the efficiency and effectiveness of AGV missions.
Mission Simulation for Autonomous Platforms in Aerospace & Defense
Recognizing the critical role of simulation and optimization in mission success
Comprehensive Mission Planning: Designing algorithms that consider various mission parameters to formulate optimal plans for autonomous platforms.
High-Fidelity Simulation Environments: Developing and utilizing advanced simulators to model real-world scenarios, facilitating the testing and refinement of autonomous systems under diverse conditions.
Integration of Unmanned Aerial and Ground Vehicles (UAVs and UGVs): Exploring cooperative strategies between aerial and ground units to accomplish complex missions synergistically.
