🙎🏽 🚁 ⛱️ Tangible user interface: object recognition when working with a multi-touch system 😄 📻 🤛🏼

We will share our experience with the Tangible User Interface and tell you how to recognize markers by dot patterns. You will learn how to make spectacular visualizations using the display and infrared frame, as well as what pitfalls can be encountered when working with TUIO.

A note from the partner of the MAI IT center and the organizer of the “ VR / AR & AI ” master's program - PHYGITALISM .

When solving scientific, educational or business problems, the problem often arises of finding the simplest and most understandable visualization of complex processes and data. Now this problem is being solved through computer technology and mobile devices. One approach for human interaction with digital information is the tangible user interface (TUI) or tactile user interface .

, // , . TUI , , , , .

TUI — Phygital Platform . - .

, , . , . , - . , .

, ObjectViz, TUIO. , , .

, :

. . , ( ). , , . , . , , , : , , .

, ( , id, , ). , — , — , .

— () , , , , . , , .

G5S (Ultra-Slim) Multi-Touch Screen, 32 . TUIO (). :

 {"Id":15237,
     "Timestamp":397449,
     "Touches":[{
                    "Id":0,
                    "Position":{
                        "X":0.480208337,
                        "Y":0.5842593},
                    "Acceleration":{
                        "X":0.00208333344,
                        "Y":0.00370370364},
                    "Type":1},
                 {  
                    "Id":1,
                    "Position":{
                        "X":0.4859375,
                        "Y":0.484259248},
                    "Acceleration":{
                        "X":0.00208333344,
                        "Y":0.00370370364},
                    "Type":0},
                {
                    "Id":2,
                    "Position":{
                        "X":0.5140625,
                        "Y":0.551851869},
                    "Acceleration":{
                        "X":0.00208333344,
                        "Y":0.00370370364},
                    "Type":0}],
     "Count":3}

ID ();
Id ( );
Position ( X Y);
Type , (0 ; 1 , ; 2 );
Count

: , [0,1]. , :

, .

, . , , 0, .

, 100% . .

M A D = \frac{1}{n} \sum_{i = 1}^{n} | L_{i} - \bar{L} |

$MAD = \frac{1}{n}\sum_{i=1}^n|L_{i} - \overline{L}|$

\bar{L} = \frac{1}{n} \sum_{i = 1}^{n} L_{i}

$\overline{L} = \frac{1}{n}\sum_{i=1}^n L_{i}$

D = \underset{i}{max L_{i}} - \underset{i}{min L_{i}}

$D = \underset{i}{\operatorname{\max {L_i}}} - \underset{i}{\operatorname{\min {L_i}}}$

: $MAD$ , $L$ $D$ , .

, , 2–3 . , , , Id, 20 .

, , — . . , .

, . . , , — .

, , , .

;
( ).

, .

$V_0 V_s V_l$ ( $s$ — shortest, $l$ — longest). $(V_0 V_l)$ , , $(V_0 V_s)$ . $V_s$ , .

, :

| \begin{matrix} (V_{l x} - V_{0 x}) & (V_{l y} - V_{0 y}) \\ (V_{s x} - V_{0 x}) & (V_{s y} - V_{0 y}) \end{matrix} | > 0 \Rightarrow

$\begin{vmatrix} (V_{lx} - V_{0x}) & (V_{ly} - V_{0y})\\ (V_{sx} - V_{0x}) & (V_{sy} - V_{0y}) \end{vmatrix} > 0 \Rightarrow$

(V_{l x} - V_{0 x}) (V_{s y} - V_{0 y}) - (V_{l y} - V_{0 y}) (V_{s x} - V_{0 x}) > 0

$(V_{lx} - V_{0x})(V_{sy} - V_{0y}) - (V_{ly} - V_{0y})(V_{sx} - V_{0x}) > 0$

, , , . , . .

, d. , , , , — , .

, , , , d. .

, — , , d, .

, , . , d. -, . . $O(V+E)$ , $V$ — , $E$ — .

;
:
- $v$ , $DFS(v)$ ;
- .

$v: DFS(v)$ :

$v$ ;
$v$ ;
v $u$ :

$u$ , $DFS(u)$ .

— , .

, ( , ). , s , .

s_{k} = \frac{1}{3} \sum_{i = 1}^{3} | L_{k i} - \bar{L_{i}} |

$s_k = \frac{1}{3}\sum_{i=1}^3|L_{ki} - \overline{L_{i}}|$

R e c o g n i z e d M a r k e r N u m b e r = \underset{k}{argmin} (s_{k})

$RecognizedMarkerNumber = \underset{k}{\operatorname{argmin}} (s_k)$

$L_{ki}$ — , $L_{i}$ — .

, , , .

, . , , , , . , , , — , , .

, . :

, . , "" . : , .
, , , .
, , — . , - , , , .

, .

, , , , Id . , , . Type.

, . , .


Active
Passive

, .

, ; , . Active ( = ) Passive ( = ). , , , .

: Added, Updated, Unstable, Removed. , .


Added	, . UI .		-	Updated
Updated	, , .			Updated, Unstable, Removed
Unstable	, , .			Unstable, Updated, Removed
Removed	, , . UI ,	-	-	-

n	n+1
Added	Updated
Updated	Removed
Updated	Updated
Unstable	Updated	ID
Unstable	Removed
Unstable	Unstable
Removed	-	.

, , (. . 10), , , : . 4,5.


Marker	,
Passive	,


New	, Type = 0
Active	, Type = 1
Lost	, , Type = 2

, — .

, , , , . , , .

x = \frac{m_{a} m_{b} (y_{1} - y_{3}) + m_{b} (x_{1} + x_{2}) - m_{a} (x_{2} + x_{3})}{2 (m_{b} - m_{a})}

$x = \frac{m_a m_b(y_1 - y_3) + m_b(x_1 + x_2) - m_a(x_2 + x_3)}{2(m_b - m_a)}$

y = - \frac{1}{m_{a}} (x - \frac{(x_{1} + x_{2})}{2}) + \frac{(y_{1} + y_{2})}{2},

$y = -\frac{1}{m_a} (x - \frac{(x_1 + x_2) } {2}) + \frac{(y_1 + y_2)} {2},$

m — , :

m_{a} = \frac{y_{2} - y_{1}}{x_{2} - x_{1}}

$m_a = \frac{y_2-y_1}{x_2-x_1}$

m_{b} = \frac{y_{3} - y_{2}}{x_{3} - x_{2}}

$m_b = \frac{y_3-y_2}{x_3-x_2}$

φ . α₁ Oy . , — Oy . $V_{lx}, V_{ly}$ — $V_l$ , $V_o V_l$ ( ) . $\varphi$ :

φ = α_{1} - α_{2}

$\varphi = \alpha_1 - \alpha_2$

a l p h a = \arctan \frac{v_{x}}{v_{y}}

$alpha = \arctan{\frac{v_x}{v_y}}$

$\alpha$ , , , $-π ≤ θ < π.$

, .

, , , .

, TUI , — , ( C#).

— !

Tangible user interface: object recognition when working with a multi-touch system

More articles: