YOU ARE IN MY HEART: Enhancing Human-Robot Collaboration-Safety through Human Emotion-Based Perception in Smart Manufacturing Contexts

Abstract

The development of safe human-robot collaborative systems is crucial to the future advancement of the manufacturing industry. This is particularly relevant to companies that already utilize robots alongside human workers in collaborative tasks such as product co-assembly. These pre-existing large robots allow industrial sectors to create very precise products at a rate that a human could not keep up with or would be very dangerous to attempt. However, as industry standards require workers to continually be in closer proximity to these robots with powerful capabilities, it is vital that workers are kept safe, especially when entering the robot’s envelope or workspace.  The proposed approach in this paper allows both the human and robot to make the best use of their respective capabilities, while constantly monitoring the human worker’s emotions in order to prevent accidents. This additional layer of understanding allows the robot to respond appropriately to its human counterpart’s feelings.  Having robots take over the tedious, repetitive tasks enables the human workers to focus on controlling the more complicated decision-based operations. This allows industries to reduce the possibility of human error incurred by mind-numbing work, while ensuring the safety and uplifting employee engagement. Allowing robots to have an understanding of the nuances within subjective human emotions allows them to respond to the expression of a given feeling appropriately. This subjectivity of human behavior in human-robot collaborative contexts has not been extensively studied in the field. We explore the potential results of robots appropriately responding to human emotions in a manufacturing setting. To allow for safe human-robot interaction and collaboration, we employ a transfer learning-based method of teaching the robot to discern various human emotional states. This method allows the robot to be able to understand and properly respond to its human partner’s emotional states, while collaborating with a human on a job. Within our approach, the speed of the robot’s actions changes in response to the human’s perceived feeling. For example, detected sadness or discomfort in the human causes the robot to slow down, while perceived happiness and satisfaction result in the robot increasing the pace of its work to match its partner’s attitude. Our method allows for greater understanding between a collaborative human-robot pair, leading to both robot-leveraged efficiency and a reduction in safety risks in the workspace. The results of our experiments show how a robot can correctly detect real-time changes in human expressions and apply this understanding towards safely co-assembling a project with a human. We also discuss future work for our study. The real-world experimental video is available at: https://youtu.be/v488i96LPE8.