Still from the film "Ivan Vasilievich Changes His Profession"

? , - . — , . . — , , , .

- .

(backbone) — , , , — , . InsightFace — Open Source .
(embedding) — , . 128 512. , 512. : . , , , — . , ( ) 1, — -1 ( 0).
Embedding- — , , ( ) .
— , ( ) , — , — . $W^TX + b$ , $X$ — ( ), $W$ — , $b$ — . softmax.

, , . — , , . — ( — ), — . .

: (metric learning) .

L o s s = \sum_{i = 0}^{N} [| | f_{i}^{a} - f_{i}^{p} | |_{2}^{2} - [| | f_{i}^{a} - f_{i}^{n} | |_{2}^{2} + α],

$Loss = \sum_{i=0}^N[||f_{i}^a - f_{i}^p||_{2}^2 - [||f_{i}^a - f_{i}^n||_{2}^2 + \alpha],$

$f^a$ — anchor, , ;
$f^p$ — positive, ;
$f^n$ — negative, ;
$\alpha$ — , .

, , $\alpha$ - .

, ( — ), , , embedding- .

Triplet Loss , “ ”, , , SotA , . , Triplet Loss (fine-tuning) . , .

, , , , , . :

Classification

— , ( 512), — ( , ). — () , . , “ ” . : , , . . , :

: , , , . , . , , Triplet Loss — , , (hard sampling), .

— , , : Softmax Cross Entropy — . Softmax Loss.

Softmax Loss

Softmax, . :

$W$ — ()
$X$ — embedding-,
$b$ — bias,

, ( , embedding- ): $z =W^TX + b$ . Softmax( $\sigma$ ) :

σ (z)_{j} = \frac{e^{z_{j}}}{\sum_{i = 0}^{C} e^{z_{i}}}

$\sigma(z)_j = \frac{e^{z_j}} {\sum_{i=0}^C e^{z_i} }$

— Cross Entropy, Softmax Loss :

L_{S o f t m a x} = - \frac{1}{N} \sum_{i = 1}^{N} l o g \frac{e^{W_{y_{i}}^{T} x_{i} + b_{y_{i}}}}{\sum_{j = 1}^{C} e^{W_{j}^{T} x_{i} + b_{j}}},

$L_{Softmax} = - \frac {1}{N} \sum_{i=1}^N log\frac {e^{W^T_{y_i}x_i + b_{y_i}}}{\sum_{j=1}^C e^{W_j^Tx_i + b_j}},$

$y_i$ — . , , 42, 42- .

— , .

: $W^TX + b$ — : $b = 0$ , $W^TX$ . , . :

$X$ $F\times B$ , $B$ — ( ), $F$ — ( 512).
$W$ $F\times C$ , $C$ — .

$W^T$ $X$ $C\times B$ , .

, :

c o s (θ) = \frac{d o t (u, v)}{| | u | | | | v | |}

$cos(\theta) = \frac {dot(u,v)}{||u||||v||}$

, : — , . , , ( $||W_i|| = 1$ ), $X$ s (scale), :

W^{T} X = s * c o s (θ)

$W^TX = s*cos(\theta)$

Scale is our first of two hyperparameters. Fixing it for all vectors leads to the fact that they are now located on the hypersphere. In a 2D version, it looks like this (one color - one class):

Image from the ArcFace article , a toy example for demonstration: each class is highlighted with its own color, each point on the circle is a separately taken image, the middle vector of each class is connected to the center for clarity. Note that the classes are connected "without gaps".

Let's rewrite Softmax Loss (now called Normalized Softmax Loss, N-Softmax) with these observations in mind:

L_{N S o f t m a x} = - \frac{1}{N} \sum_{i = 1}^{N} l o g \frac{e^{s * c o s (θ_{y_{i}})}}{e^{s * c o s (θ_{y_{i}})} + \sum_{j = 1, j \neq y_{i}}^{C} e^{s * c o s (θ_{j})}}

$L_{N Softmax} = - \frac {1}{N} \sum_{i=1}^N log\frac {e^{s*cos(\theta_{y_i})}}{e^{s*cos(\theta_{y_i})} + \sum_{j=1, j\neq y_i}^C e^{s*cos(\theta_j)}}$

. N-Softmax .

Margin-Based Loss

, , — . softmax loss, . , (decision boundary), ( ). . ? () , . — (decision margin). — margin — , : scale — . 2D ( ArcFace):

— margin, — margin

, .

, margin ( $m$ ) :

. $cos(\theta)$ $cos(m*\theta)$ . : Large-Margin Softmax Loss SphereFace. , , Margin-based loss. :

$L_{S p h e r e F a c e} = - \frac{1}{N} \sum_{i = 1}^{N} l o g \frac{e^{s * c o s (m * θ_{y_{i}})}}{e^{s * c o s (m * θ_{y_{i}})} + \sum_{j = 1, j \neq y_{i}}^{C} e^{s * c o s (θ_{j})}}$
$L_{SphereFace} = - \frac {1}{N} \sum_{i=1}^N log\frac {e^{s*cos(m*\theta_{y_i})}}{e^{s*cos(m*\theta_{y_i})} + \sum_{j=1, j\neq y_i}^C e^{s*cos(\theta_j)}}$
margin . $cos(\theta)$ $cos(\theta) - m$ . : AM-Softmax CosFace , , , . :

$L_{C o s F a c e} = - \frac{1}{N} \sum_{i = 1}^{N} l o g \frac{e^{s * c o s (θ_{y_{i}}) - m}}{e^{s * c o s (θ_{y_{i}}) - m} + \sum_{j = 1, j \neq y_{i}}^{C} e^{s * c o s (θ_{j})}}$
$L_{CosFace} = - \frac {1}{N} \sum_{i=1}^N log\frac {e^{s*cos(\theta_{y_i}) - m}}{e^{s*cos(\theta_{y_i}) - m} + \sum_{j=1, j\neq y_i}^C e^{s*cos(\theta_j)}}$
margin : $cos(\theta)$ → $cos(\theta + m)$ . ArcFace. ArcFace :

$L_{C o s F a c e} = - \frac{1}{N} \sum_{i = 1}^{N} l o g \frac{e^{s * c o s (θ_{y_{i}} + m)}}{e^{s * c o s (θ_{y_{i}} + m)} + \sum_{j = 1, j \neq y_{i}}^{C} e^{s * c o s (θ_{j})}}$
$L_{CosFace} = - \frac {1}{N} \sum_{i=1}^N log\frac {e^{s*cos(\theta_{y_i}+m) }}{e^{s*cos(\theta_{y_i}+m) } + \sum_{j=1, j\neq y_i}^C e^{s*cos(\theta_j)}}$
, ArcFace AirFace. Margin , ArcFace, , ( $arccos(cos(\theta)) = \theta$ ). , , ( — ), , $\theta$ , $(\pi - 2\theta)/\pi$ , :

$L_{A i r F a c e} = - \frac{1}{N} \sum_{i = 1}^{N} l o g \frac{e^{s * (π - 2 (θ_{y_{i}} + m)) / π}}{e^{s * (π - 2 (θ_{y_{i}} + m)) / π} + \sum_{j = 1, j \neq y_{i}}^{C} e^{s * (π - 2 θ_{j}) / π}}$
$L_{AirFace} = - \frac {1}{N} \sum_{i=1}^N log\frac {e^{s*(\pi - 2(\theta_{y_i}+m))/\pi}}{e^{s*(\pi - 2(\theta_{y_i}+m))/\pi} + \sum_{j=1, j\neq y_i}^C e^{s*(\pi - 2\theta_j)/\pi}}$

margin — , — ( ArcFace).

Margin & Scale

- , , — scale (s) margin (m), . , AM Softmax $s = 30$ , $m = 0.35$ , ArcFace — $s = 64$ , a $m = 0.5$ , CosFace (, , AM Softmax) $s = 64$ , a $m = 0.35$ . , , “ ”, .

— AdaCos, — scale margin . :

Margin scale — , .
scale margin, margin scale.
scale .

— scale

2 20 , , Y — , , X — . , , , :

scale ( , , margin 0), — margin scale=30. , scale, “” , margin X. , — scale margin, - , ? AdaCos scale ( ). , : $s = \sqrt{2}*\ln{(C-1)}$ , $C$ — . s [10, 25], .

AdaCos — scale. , scale , . , , .

margin , , , ? . X , Y — ( N-softmax $cos(\theta)$ ):

: CosFace N-softmax , ArcFace — . SphereFace, $\frac {\pi}{margin}$ , $\frac{\pi}{3}$ . ArcFace — target logit, , , . , , ( $\frac{5\pi}{6}$ ). , (, ) , :

# cosine - cos(theta)
# phi - cos(theta + m)
# th - cos(math.pi - m)
# mm - sin(math.pi - m) * m
if easy_margin:
  phi = torch.where(cosine > 0, phi, cosine)  
else:
  phi = torch.where(cosine > th, phi, cosine - mm)

easy_margin , :

Easy margin , $\frac{\pi}{2}$ N-Softmax ( $cos(\theta)$ ), not easy margin . , , , $\frac{\pi}{2}$ , “”, , , .

, . — " " ArcFace. , ( 4 8 ):

Loss	LFW	MegaFace, Rank1 @ $10^6$	MegaFace, Tar @ Far $10^{-6}$
AM-Softmax/CosFace	99.33	0.9833	0.9841
ArcFace	99.83	0.9836	0.9848
SphereFace	99.42	0.9743	0.9766

( 3 ), LFW, — - :

Loss	Resnet50-MSC	MobileNet-MSC	Resnet50-Casia	MobileNet-Casia
AM-Softmax/CosFace	99.3	97.65	99.34	98.46
ArcFace	99.15	98.43	99.35	99.01
SphereFace	99.02	96.86	99.1	97.83

, ArcFace SotA .

. (margin) , . AM Softmax ( ) ArcFace ( ). , , , AirFace.

:

SphereFace https://arxiv.org/abs/1704.08063

AM Softmax https://arxiv.org/abs/1801.05599

CosFace https://arxiv.org/abs/1801.09414

ArcFace https://arxiv.org/abs/1801.07698

AirFace https://arxiv.org/abs/1907.12256

:

Deep Face Recognition: A Survey https://arxiv.org/abs/1804.06655

A Performance Evaluation of Loss Functions for Deep Face Recognition https://arxiv.org/abs/1901.05903

Your face is not a king! Loss functions for face recognition problem

Softmax Loss

Margin-Based Loss

Margin & Scale

:

:

More articles: