Evaluating Deep Networks for Detecting User Familiarity with VR from Hand Interactions

M Li, N Zafar, NK Banerjee… - 2024 IEEE International …, 2024 - ieeexplore.ieee.org
2024 IEEE International Conference on Artificial Intelligence and …, 2024ieeexplore.ieee.org
As VR devices become more prevalent in the consumer space, VR applications are likely to
be increasingly used by users unfamiliar with VR. Detecting the familiarity level of a user
with VR as an interaction medium provides the potential of providing on-demand training for
acclimatization and prevents the user from being burdened by the VR environment in
accomplishing their tasks. In this work, we present preliminary results of using deep
classifiers to conduct automatic detection of familiarity with VR by using hand tracking of the …
As VR devices become more prevalent in the consumer space, VR applications are likely to be increasingly used by users unfamiliar with VR. Detecting the familiarity level of a user with VR as an interaction medium provides the potential of providing on-demand training for acclimatization and prevents the user from being burdened by the VR environment in accomplishing their tasks. In this work, we present preliminary results of using deep classifiers to conduct automatic detection of familiarity with VR by using hand tracking of the user as they interact with a numeric passcode entry panel to unlock a VR door. We use a VR door as we envision it to the first point of entry to collaborative virtual spaces, such as meeting rooms, offices, or clinics. Users who are unfamiliar with VR will have used their hands to open doors with passcode entry panels in the real world. Thus, while the user may not be familiar with VR, they would be familiar with the task of opening the door. Using a pilot dataset consisting of 7 users familiar with VR, and 7 not familiar with VR, we acquire highest accuracy of 88.03% when 6 test users, 3 familiar and 3 not familiar, are evaluated with classifiers trained using data from the remaining 8 users. Our results indicate potential for using user movement data to detect familiarity for the simple yet important task of secure passcode-based access.
ieeexplore.ieee.org