Integration of computer vision and strain sensor for human–machine interfaces

Abstract

Soft strain sensors play a crucial role in advancing human-machine interfaces, facilitating seamless interaction between humans and machines. However, these sensors encounter specific challenges that impede their widespread application. These challenges arise from their reliance on nanotechnologies and specialized fabrication techniques, resulting in limitations such as limited reproducibility, susceptibility to environmental noise, and relatively short lifetimes. To overcome these challenges, our study introduces a novel computer vision-based optical strain (CVOS) sensor system. This system utilizes a readily fabricated soft silicone substrate embedded with micro-markers, along with a compact camera for precise marker detection. The CVOS sensor system not only addresses the aforementioned challenges but also empowers real-time multiaxial strain mapping, enabling the simultaneous detection and measurement of strain in multiple directions. By incorporating the capability of multiaxial strain mapping, our sensor system can capture a broad range of deformation information that was previously difficult to obtain using conventional strain sensors. This expanded functionality opens up new application fields, including the measurement of complex 3-axis body rotation motion and other intricate deformation patterns. Our experimental results validate the effectiveness and versatility of the proposed CVOS sensor. The sensor demonstrates high sensitivity, ensuring accurate and reliable measurements even in real-world conditions. Furthermore, its long-term operation highlights its robustness and durability. In conclusion, our study presents a promising approach for highly sensitive and adaptable systems that can meet the diverse demands of real-world applications.