LimbX
Augmented Reality (AR) - controlled Supernumerary Limb*
Massachusetts Institute of Technology Funding
*Supernumerary robotic limbs (SRLs) are robotic appendages worn by a human user
My Role
I led the usability studies for this project. I designed the protocol, led 21 user studies, and synthesized the findings. I collaborated with some amazing engineers at Harvard college, Alice and Aida, who led the engineering development of LimbX. To read more about how it was designed and engineered, visit this page.
How might we control a limb that is not biologically our own?
Supernumerary Robotic Limbs (SRLs) could enhance human capabilities by expanding the range and number of actions achievable. However, current control interfaces for SRLs demand significant cognitive effort from users.
Our innovation is an augmented reality (AR)-controlled continuum SRL. It enables users to issue high-level commands via eye-gaze input, supplemented by secondary voice commands.
Existing Methods
Gesture
-
Difficult to use when hands are occupied with other tasks
Autonomous
-
Removes user autonomy
-
Less applicable for manual repetitive tasks
Joystick
-
Difficult to use when occupied with other tasks
Our Hypothesis
AR control will provide a more optimal and intuitive user interface, leveraging eye gaze and voice commands.
-
Intuitive user control to decrease learning curve
-
Computation burden shifted from user to controller system
-
Improved dexterity given continuum SRL
Usability Testing Goal
To compare the Hololens AR-controlled system with a joystick-controlled system.
Quantitative Metrics
-
Accuracy - measured by the distance of the end effector from the centre of the target
-
Efficiency - time taken to reach target
Qualitative Metrics
-
Comfort
-
Intuitive / ease of use
-
Responsive
-
Embodiment
Test Setup
A series of trials were conducted in which 21 participants engaged with the two systems, in which participants moved the SRL to specified target positions along the X-Y plane.
The experiment was restricted to being conducted on the table-top and the X-Y plane to focus the experiment on comparison of the AR system with the joystick-based control system.
Participant engaging with the Joystick-controlled supernumerary limb (control)
Participant engaging with the AR-controlled supernumerary limb
Results
Quantitative Findings:
Joystick-controlled SRL demonstrated greater overall accuracy (distance), while there was no statistically significant difference in efficiency (time)
Qualitative Findings:
Participants indicated excitement for AR control system, but prefer the joystick, given the alignment with existing mental models.
Participants felt that the AR-controlled SRL offered a greater sense of embodiment, while the joystick-controlled SRL offered a greater sense of familiarity and responsiveness
Limitations
Our hypothesis was not strongly supported through these experiments, likely due to several limitations that could be improved for future studies.
Design Limitations
-
Need to allow users to make fine adjustments to position in addition to one-shot commands
-
Need more precise servo motors / robot movement to allow fine adjustments
-
Need to improve the tension and reduce friction of the tendons in order to improve the reliability of the hardware
Experiment Limitations
-
Inaccuracy of plane calibration
-
Physical movement of the target when the distance between the end effector and target was more than anticipated
-
Delay between control and actuation due to network issues
