Researchers from the Swiss Higher Technical School of Zurich (ETH) have used this physical feature of human hands in their invention - a two-sensor bracelet that allows you to interact directly with virtual objects.
Virtual reality is already widely used in various fields, but it is mainly needed to absorb content, not to create it. But Swiss scientists are on the way to address this injustice.
One of the ETH professors, Christian Holtz, sees VR technology as a potential replacement for desktop PCs. In his opinion, “decoupling from the screen” will make it possible to fully utilize the virtual space: to place “working tools” around oneself and interact with it in the most natural way - by hand.
Each finger vibrates differently
Why are there still no virtual computers with an unlimited number of monitors and peripherals? Holtz says it's all about the absence of normal interaction mechanisms between humans and VR objects. Now VR systems are equipped not only with helmets / goggles, but also with special controllers and cameras to record the user's movements. What kind of productivity and focus at work can we talk about when you have to interact with such a sophisticated system?
« ( ), . », — .
The second problem is that when typing on the keyboard, the hands move differently than when playing tennis, so it is difficult for the camera to capture and correctly interpret the movement of the limbs. Moreover, in this case, there is no physical return.
Holz's team resigned themselves to the fact that additional passive interfaces were indispensable, but decided to rethink them, turning fingers and their unique vibrations that occur when touching any hard surface into such an interface. To implement the idea, TapID technology was created, which will be presented at the IEEE VR conference at the end of March 2021. Holz and his team will bring TapID prototypes there in the form of multisensor rubber bracelets.
Sensors in the bracelet register the moment a finger touches any surface and tell the system which finger was used by the person. To do this, the sensor analyzes the vibration at the base of the hand and, based on the characteristics of this vibration, determines which finger is being used at the current time. The bracelet works in conjunction with a camera installed in VR glasses and an advanced neural network developed specifically for TapID. Together, they allowed the researchers to achieve extremely high precision in recording "keystrokes" on virtual keys. The developers of TapID have already demonstrated their brainchild in work on the example of two applications: a virtual keyboard and a virtual piano.
Smart watch will help people play the virtual piano
Holz is especially fond of the virtual piano: he considers it the most successful application for demonstrating the capabilities of TapID, because it is important to take into account both the accuracy of the position of the fingers and the timing of the clicks (after all, extra sharps will appear in the melody, and it will not work to get into the rhythm) ... TapID technology does an excellent job of recording the time of pressing by using a bracelet instead of cameras.
But a specialized bracelet may not be needed. The ETH research team tested the development on 18 smart bracelets and made sure that any of them can become the basis for TapID, because all such devices have a gyroscope.
Holtz also believes that the development of his team will simplify the development of the so-called mobile virtual reality.
“TapID is portable and can be used even on the go. You can create a virtual workspace around you anywhere and anytime, ”added the professor.
Christian Holz dreams of a future in which people can use a virtual space to collaborate without feeling any of the constraints of hardware. And TapID, it seems, should be the key to the rapid development of technology in this direction.