Researchers
have developed a virtual reality application where a range of 3D modeling tools
can be opened and controlled using just the movement of a user's hand.
The researchers, from the University of
Cambridge, used machine learning to develop 'HotGestures'—analogous to the hot
keys used in many desktop applications.
HotGestures give users the ability to
build figures and shapes in virtual reality without ever having to interact with a menu,
helping them stay focused on a task without breaking their train of thought.
The idea of being able to open and
control tools in virtual reality has been a movie trope for decades, but the
researchers say that this is the first time such a 'superhuman' ability has
been made possible. The results are reported in the journal IEEE Transactions on Visualization and Computer
Graphics.
HotGestures give users the ability to build
figures and shapes in virtual reality without ever having to interact with a
menu, helping them stay focused on a task without breaking their train of
thought. Credit: University of Cambridge
Virtual reality (VR) and related
applications have been touted as game-changers for years, but outside of
gaming, their promise has not fully materialized. "Users gain some
qualities when using VR, but very few people want to use it for an extended period
of time," said Professor Per Ola Kristensson from Cambridge's Department
of Engineering, who led the research. "Beyond the visual fatigue and
ergonomic issues, VR isn't really offering anything you can't get in the real
world."
Most users of desktop software will
be familiar with the concept of hot keys—command shortcuts such as ctrl-c to
copy and ctrl-v to paste. While these shortcuts omit the need to open a menu to
find the right tool or command, they rely on the user having the correct
command memorized.
"We wanted to take the concept
of hot keys and turn it into something more meaningful for virtual
reality—something that wouldn't rely on the user having a shortcut in their
head already," said Kristensson, who is also co-Director of the Centre for
Human-Inspired Artificial Intelligence.
Instead of hot keys, Kristensson
and his colleagues developed 'HotGestures', where users perform a gesture with
their hand to open and control the tool they need in 3D virtual reality
environments.
For example, performing a cutting
motion opens the scissor tool, and the spray motion opens the spray can tool.
There is no need for the user to open a menu to find the tool they need, or to
remember a specific shortcut. Users can seamlessly switch between different
tools by performing different gestures during a task, without having to pause
their work to browse a menu or to press a button on a controller or keyboard.
"We all communicate using our
hands in the real world, so it made sense to extend this form of communication
to the virtual world," said Kristensson.
For the study, the researchers
built a neural network gesture recognition system that can recognize gestures
by performing predictions on an incoming hand joint data stream. The system was
built to recognize ten different gestures associated with building 3D models:
pen, cube, cylinder, sphere, palette, spray, cut, scale, duplicate and delete.
The team carried out two small
studies where participants used HotGestures, menu commands or a combination.
The gesture-based technique provided fast and effective shortcuts for tool
selection and usage. Participants found HotGestures to be distinctive, fast,
and easy to use while also complementing conventional menu-based interaction.
The researchers designed the system
so that there were no false activations—the gesture-based system was able to
correctly recognize what was a command and what was normal hand movement.
Overall, the gesture-based system was faster than a menu-based system.
"There is no VR system
currently available that can do this," said Kristensson. "If using VR
is just like using a keyboard and a mouse, then what's the point of using it?
It needs to give you almost superhuman powers that you can't get elsewhere."
The researchers have made the source code and dataset publicly available so that designers
of VR applications can incorporate it into their products.
"We want this to be a standard way of interacting with VR," said Kristensson. "We've had the tired old metaphor of the filing cabinet for decades. We need new ways of interacting with technology, and we think this is a step in that direction. When done right, VR can be like magic."
No comments:
Post a Comment