“PolyU Develops Intelligent Human-Robot Collaboration System for Advanced Manufacturing Tasks”
A research team at The Hong Kong Polytechnic University (PolyU) has developed an advanced system for human-robot collaboration, aimed at advancing the principles of Industry 5.0. The system, termed the “Mutual Cognitive Human-Robot Collaboration Manufacturing System” (MC-HRCMS), enhances the ability of robots to perceive complex environments in real time and accurately interpret human intentions. Key functionalities include skill transfer and self-learning through demonstration, as well as the automation of process code generation and adjustments to ensure precise task execution. The system has been successfully applied in high-end manufacturing tasks, such as autonomous drilling in large aircraft and the disassembly of electric vehicle batteries, representing a significant advancement in the development of a human-centric approach to smart manufacturing.
Objectives of the Research
The primary objective of this research is to achieve seamless integration between human adaptability and machine precision, thereby fostering a symbiotic relationship between human operators and robots. Under the direction of Professor Zheng Pai, an Associate Professor in the Department of Industrial and Systems Engineering, the MC-HRCMS departs from traditional pre-programmed operations to emphasize holistic perception and analysis. Utilizing multimodal sensing data—including visual, haptic, language, and physiological signals—the system supports comprehensive environmental awareness and autonomous decision-making.
Enhancements in Human-Robot Interaction
The MC-HRCMS features significantly enhance the safety and efficiency of human-robot interactions within complex manufacturing environments. Tailored to meet the needs of various leading companies through industry partnerships, the technology has demonstrated successful deployment across a range of challenging scenarios involving precision tasks.
Collaborative Models in Manufacturing
Professor Zheng has noted that the manufacturing sector is increasingly adopting collaborative models that necessitate flexible automation. The research team intends to establish a new paradigm focusing on multimodal perception and cross-scenario skill transfer, enabling robots to evolve into intelligent agents rather than merely functioning as tools. This transition offers smart factories alternative pathways that diverge from conventional automation methods.
Novel Planning Framework for Personalized Manufacturing
To tackle the challenges associated with personalized manufacturing, often marked by extensive assembly and disassembly processes, the team has introduced a novel vision-language-guided planning framework. This framework integrates Large Language Models (LLMs), Deep Reinforcement Learning (DRL), and Mixed-Reality Head-Mounted Displays (MR-HMD), which collectively enhance the execution capabilities for personalized and unpredictable production tasks.
Innovation in Task Comprehension
A notable innovation within this framework is the combination of visual data and language-based instructions, which allows robots to comprehend complex task semantics and communicate effectively with human operators. The use of head-mounted devices not only facilitates real-time data acquisition but also offers intuitive guidance to workers, thereby transforming the human-machine interaction experience.
Future Plans in Human-Robot Collaboration
Looking ahead, Professor Zheng and his team plan to investigate advanced topics such as self-configurable human-robot networks and mechanisms for skill transfer, with the aim of further improving human-robot collaboration. Their ultimate goal is to develop a deeply human-centric intelligent manufacturing system that contributes to societal advancement in a technology-driven and empathetic future.
Recognition and Funding
In recognition of his contributions to the field, Professor Zheng received funding from the Excellent Young Scientists Fund by the National Natural Science Foundation of China in 2024, leading the RAIDS research team in these innovative pursuits. Additional information regarding the team’s projects is available on their website.
(Source: The Hong Kong Polytechnic University)
