Measuring User Experience in XR: A New Approach to Hardware Prototyping
October 25th, 2024
Jamil Joundi
The development of complex hardware often requires significant time and resources, especially during the prototyping phase, where physical models are built, tested, adjusted, and rebuilt. This iterative process can be labor-intensive and slow. However, extended reality (XR) technologies, including virtual reality (VR), offer a new approach to streamlining hardware prototyping. This study, conducted in collaboration with Voxdale and Idevax, explores whether VR can serve as an effective tool for evaluating digital prototypes, potentially shortening development cycles. By testing a case involving an automated vaccination robot, researchers compared participants' experiences with both physical and virtual versions of the device. Physiological responses, such as heart rate, breathing rate, and skin conductance, were measured, alongside participants’ subjective perceptions of usability. The results showed that, while VR prototypes were similar to physical ones in terms of usability and attractiveness, the physical prototype scored higher for hedonic qualities related to identity and stimulation.
The journey of hardware development is complex and time-consuming, especially during the prototyping phase. This stage often requires building physical models, testing, making adjustments, and rebuilding. For complex hardware, this iterative process can take months or even years, consuming significant time and resources. However, in today's world of immersive technologies, extended reality (XR)—including virtual reality (VR)—offers promising potential to streamline these processes.
Can VR Revolutionize Prototyping?
One of the critical research questions in this use case was whether VR could serve as a tool to address the challenges of hardware prototyping. The goal was to see if VR could significantly shorten the iterative process by allowing designers and engineers to evaluate digital models that closely mirror physical prototypes, without having to build them. This approach not only saves time but also resources.
In VR, changes to prototypes can be made instantly, with real-time visualization and testing, eliminating the need to repeatedly build and test physical versions. But for this to be effective, the reactions and feedback gathered from users interacting with a VR prototype must be similar to those obtained from physical prototypes.
Research Focus
To test this hypothesis, we examined whether individuals would experience a digital representation of a prototype in VR similarly to how they experience a physical high-fidelity prototype. The focus of our study was a case where participants interacted with both a physical prototype of an automated vaccination robot and its virtual counterpart in VR.
We measured participants' physiological responses, including heart rate, breathing rate, and skin conductance, and collected data from post-experience questionnaires to assess their perception of the product and its usability.
Experiment Design
The study included 28 participants who underwent a vaccination procedure with both a physical and virtual version of the automated vaccination device. Before each procedure, physiological sensors were attached to the participants to measure heart rate, breathing rate, and skin conductance.
The physical prototype was a high-fidelity automated vaccination device, while the VR experience used a Vive Pro Eye VR headset paired with a state-of-the-art gaming computer. The virtual experience was made realistic by incorporating a 3D-printed armrest that inflated in sync with its virtual counterpart, making the VR simulation feel as close to the physical version as possible.
Key Findings
Physiological Responses:
Heart rate increased during the anticipation period, especially before the injection in the physical prototype. Interestingly, this stress response wasn’t mirrored in the VR condition.
Breathing rate was consistently higher during the real-life procedure compared to the VR session.
Skin conductance remained consistent across both conditions, indicating similar levels of stress during the procedure, whether it was real or virtual.
Perception & Usability:
Participants found the pragmatic qualities (functionality and ease of use) and attractiveness of both the physical and VR systems to be quite similar.
However, the physical prototype was rated higher for hedonic qualities related to identity and stimulation. Participants felt the real-life system was more professional, creative, and engaging.
The usability of both systems was rated as equally high.
Participant Feedback:
Interviews revealed that the most stressful moments were right before and during the vaccination, particularly when the armrest inflated to secure the arm. The reaction to seeing the robot for the first time was generally neutral to positive, with trust in the device growing after the experience.
Conclusion: The Future of XR in Prototyping
The study results show that while there are minor differences, people generally respond similarly to both high-fidelity physical prototypes and VR simulations. This suggests that VR can be a powerful tool in the prototyping phase, offering significant advantages by reducing the time and resources needed to test hardware designs.
By incorporating XR technologies into the hardware development process, companies could enhance efficiency while still gathering valuable user feedback. This opens the door to faster, more flexible iterations and could become the new standard in design and engineering practices for complex hardware.
For the curious heads who want to know the nitty-gritty details of all the research outlined above - here is the full report.