Communication with your computer – show it your fingers

Communication with your computer – show it your fingers
Researchers from the Max Planck Institute for Informatics and Aalto University calculate the easiest way to interact by gestures of hand and fingers using a camera-based input device

Small smartphones, dusty environment … communication with a computer is not always as efficient as using a keyboard on a desk. Physically small devices like smart watches or augmented reality glasses have per se limits of tactile input. On top of that: Does any existing input device make adequate use of the incredible dexterity of the hand?
Scientists at the Max Planck Institute for Informatics in Saarbrücken and Aalto University in Helsinki explore and calculate how to control the computer by gestures. For the first time, efficient text entry is possible by moving fingers in mid-air.
So far, there is no intention to adopt this system for air-guitars.

Human computer interaction increases with the digitalization of everyday life. There is even a separate branch of computer science investigating this user interface, which in-cludes results from physiology and psychology.
The scientists are researching ways of human computer interaction beyond the clas-sical input; text entry without a keyboard is one of the challenging tasks.
Owing to recent achievements in computer vision, modern algorithms can recognize even multi-finger gestures in a video stream. The research groups around professors Antti Oulasvirta and Christian Theobalt used this as a basis to develop a set of hand and finger gestures that can control a computer and even enable efficient text entry in mid-air, eliminating the necessity of tactile devices. To this end, the scientists developed a model that makes use of the dexterity of the human hand and takes also into account its physiological particularities. Anna Feit at Aalto University says: „Using our model, we optimize the gestures regarding high performance, anatomical constraints, and learnability.“
Test subjects could write up to 22 words per minute (wpm) on average, peak per-formers even 38 wpm. The model, however, predicts an even further increase of up to 55 wpm. Moreover, the model precisely predicted the rate of fingerspelling and showed fingerspelling to be non-optimal for human computer interaction.

„The next challenge will be the capture and utilization of multi-hand input.“ Srinath Sridhar from the MPI-INF points into the next future. „But first we need better computer vision and gesture recognition methods.“

Further Information can be found on the project page:
handtracker.mpi-inf.mpg.de/projects/HandDexterity
Insight into the experiments and more detailed explanations are given here.

contact:
Srinath Sridhar
ssridhar@mpi-inf.mpg.de
group of Prof. Christian Theobalt
theobalt@mpi-inf.mpg.de
Max-Planck-Institut für Informatik
Tel.: +49.681.3925-4057

Anna Feit
anna.feit@aalto.fi
group of Prof. Antti Oulasvirta
antti.oulasvirta@aalto.fi
Aalto University, Finland

Press contact:
Bertram Somieski
somieski@mpi-klsb.mpg.de
Max Planck Institute for Informatics
Max Planck Institute for Software Systems
Joint Administration
– public relations –
Tel +49.681.9325-5710