🧕🏾 🏔️ 🔺 Analysis and construction of ROC-curves: communication with the radar 🐙 🌙 🕐

Formulation of the problem

Many have heard of the ROC curve, which is often used in ML . Decoding this abbreviation, we get that ROC ( receiver operating characteristic ). Translated from English, this means RHP ( receiver performance ). This concept is borrowed from the theory of signal detection. The ROC curve can be associated with a radar station ( radar ), considering it from the point of view of object detection. Let's describe this more formally.

The radar sends out pulses that bounce off objects. The reflected signal is received by the receiving antenna of the radar ( Fig. 1 ). If there is any object located in the detection zone ( DZ ) ^ 1 , then the reflected signal will be higher than the detection threshold $\ lambda$ and this will mean the presence of the object (D_1) . If the reflected signal is below the detection threshold, then this means that there is no object (D_0) .

^ 1 The radar detection zone is the area of space within which the radar ensures the detection of objects with the probabilities of correct detection not worse than the required ones.

H_1 , H_0 . — . , (one) , .

$\ Large f _ {(X | H_0)} (x│H_0), \ f _ {(X | H_1)} (x│H_1) \ \ \ \ (1)$

, , , (one) . ,

$\ Large E [X│H_0] <E [X│H_1] \ \ \ \ (2)$

, - , :

Fig. 2 Conditional densities (1) — . 2 (1)

. 2 $\ eta_0, \ eta_1$ — H_0 H_1 . , (2) , , η_0 <η_1 .

, , . D_1 D_0 . . 2 λ, D_1 D_0 , :

Fig. 3 Conditional density distributions (1) with the designation of the threshold level λ and the decision-making ranges D₀ and D₁ — . 3 (1) c λ D₀ D₁

$\Large \begin{array}{l} P_{d}=P\left(D_{1} \mid H_{1}\right)=\displaystyle \int_{D_{1}} f_{X \mid H_{1}}\left(x \mid H_{1}\right) d x \\ P_{f a}=P\left(D_{1} \mid H_{0}\right)= \displaystyle \int_{D_{1}} f_{X \mid H_{0}}\left(x \mid H_{0}\right) d x \end{array} (3)$

(3) :

– , , .. .
$P_{fa}$ – , .. , , , .. .

(3) , :

, , , P_d $P_{fa}$ .

, P_d $P_{fa}$ , ROC- (. Receiver Operating Characteristic, ).

- (, . Signal-to-Noise Ratio, . SNR ), :

$\Large S N R=10 \log _{10}\left(\frac{P_{\text {signal }}}{P_{\text {noise }}}\right)[dB] \ \ \ (4)$

$P_{signal}$ – ;
$P_{noise}$ – .

(4) $P_{signal}=1$ , $P_{signal}=σ^2$ – , :

$\Large S N R=10 \log _{10}\left(\frac{1}{\sigma^{2}}\right)[d B]\ \ (5)$

ROC-

$\Large \begin{array}{c} f_{X \mid H_{0}}\left(x \mid H_{0}\right)= \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{x^{2}}{2 \sigma^{2}}} \\ f_{X \mid H_{1}}\left(x \mid H_{1}\right)=\frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(x-1)^{2}}{2 \sigma^{2}}} \end{array} \ \ \ (6)$ $\Large S N R \in[-1,5]\ \ \ (7) \\ \Large \lambda \in[0,1]\ \ \ (8)$

(6) , , , .. $X \mid H_{0} \sim \mathcal{N}\left(0, \sigma^{2}\right)$ $X \mid H_{1} \sim \mathcal{N}\left(1, \sigma^{2}\right)$ , $\mathcal{N}\left(\mu, \sigma^{2}\right)$ – $\mu$ $\sigma^{2}$ . η_0, η_1 . (6) , , η_0= Ε[X│H_0 ]=0 , η_1= Ε[X│H_1 ]=1 .

(7) , $\sigma^2$ . (5) $\sigma$ :

$\Large \begin{array}{c} S N R=10 \log _{10}\left(\frac{1}{\sigma^{2}}\right)=10 \log _{10}\left(\sigma^{-2}\right)=-2 \cdot 10 \log _{10} \sigma=-20 \log _{10} \sigma \\ -\frac{S N R}{20}=\log _{10} \sigma \\ \sigma=10^{-\frac{S N R}{20}} \ \ (9) \end{array}$

(7) , :

$\Large \begin{aligned} &\sigma \in\left[\left.10^{-\frac{S N R}{20}}\right|_{S N R=5},\left.10^{-\frac{S N R}{20}}\right|_{S N R=-1}\right]\\ &\sigma \in\left[10^{-\frac{5}{20}}, 10^{-\frac{-1}{20}}\right]\\ &\sigma \in\left[\sqrt[4]{\frac{1}{10}}, \sqrt[20]{10}\right] (10)\end{aligned}$

SNR ROC- :

$\Large \overrightarrow{{S N R}_{\text {values }}}=(-1,0.5,1,1.5,2,2.5,3,3.5,4,4.5,5)^{T} \text { (11) }$

(11) $\scriptsize \frac{1}{2}$ , .. $\small ∆_{SNR}=\frac{1}{2}$ . ROC- P_d $P_{fa}$ , λ∈[0,1] . (3) (6) , :

$\Large \text { ROC }_{\text {curve }}(\sigma, \lambda)=\left\{ \begin{array}{r} P_{f a}= \displaystyle\frac{1}{\sqrt{2 \pi \sigma^{2}}} \displaystyle \int_{\lambda}^{+\infty} e^{-\frac{x^{2}}{2 \sigma^{2}}} d x \\ P_{d}= \displaystyle\frac{1}{\sqrt{2 \pi \sigma^{2}}} \displaystyle \int_{\lambda}^{+\infty} e^{-\frac{(x-1)^{2}}{2 \sigma^{2}}} d x \end{array} \right. (12)$

$\sigma$ – ROC- ROC-.

$\Large \overrightarrow{\lambda_{\text {values }}}=(0,0.01,0.02, \ldots, 0.98,0.99,1)^{T} (13)$

(13) $\small \frac{1}{100}$ , .. $\small ∆_λ=\frac{1}{100}$ . (13) 101 , .. 101 . , .

ROC-.

Python.

#   
from scipy.stats import norm
from scipy.misc import derivative
import numpy as np
import pandas as pd

import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
%matplotlib inline

#   SNR  lambda
SNR_values = np.arange(-1, 5.5, 1/2) 
lambda_values = np.arange(0, 1.01, 1/100) 

# ,    sigma    SNR
def snr_to_sigma(SNR):
    return np.power(10, -SNR/20)
  
# ,    Pfa  Pd   sigma  lambda
def roc_curve(sigma, lambda_):
    return 1 - norm.cdf(lambda_/sigma), 1 - norm.cdf((lambda_ - 1)/sigma)

#  DataFrame      Pfa, Pd, SNR
data = []
for SNR in SNR_values:
    for lambda_ in lambda_values:
        Pfa, Pd = roc_curve(snr_to_sigma(SNR), lambda_)
        data.append([Pfa, Pd, str(SNR)])
        
data = pd.DataFrame(data, columns=['Pfa', 'Pd', 'SNR'])

fig, ax = plt.subplots(figsize=(18, 14))
sns.set_context('poster')
plt.title('$ROC$-     $SNR$', fontsize=40)
plt.xlabel('$P_{fa}$', fontsize=40)
plt.xticks(np.arange(0, 0.7, 0.05), fontsize=30)
plt.yticks(np.arange(0.5, 1, 0.05), fontsize=30)
plt.ylabel('$P_d$', fontsize=40, rotation=0, labelpad=40)
sns.lineplot(x='Pfa', y='Pd', hue='SNR', 
             data=data, palette='gist_ncar', ax=ax)

ax.annotate(" $\lambda$", xy=(0.15, 0.81),  xycoords='data',
            xytext=(90, 150), textcoords='offset points',
            size=30, ha='left', 
            arrowprops=dict(arrowstyle="->", color='black',
                            connectionstyle="arc3,rad=0.1"))
plt.grid(color='black', linestyle='--', linewidth=1, alpha=0.3)

plt.legend(fontsize='small', shadow=True, title='$SNR(dB)$',
           borderpad=0.9, fancybox=True);

$P_{fa}=0.5$ ;
, ROC- ;
;
ROC - . , .

(8) . , ROC-, , , [-10,10] .

#  DataFrame      Pfa, Pd, SNR
data2 = []
for SNR in SNR_values:
    for lambda_ in np.arange(-10, 10.1, 1/10) :
        Pfa, Pd = roc_curve(snr_to_sigma(SNR), lambda_)
        data2.append([Pfa, Pd, str(SNR)])
        
data2 = pd.DataFrame(data2, columns=['Pfa', 'Pd', 'SNR'])

fig, ax = plt.subplots(figsize=(18, 14))
sns.set_context('poster')
plt.title('$ROC$-     $SNR$', fontsize=40)
plt.xlabel('$P_{fa}$', fontsize=40)
plt.xticks(np.arange(0, 1.05, 0.1), fontsize=20)
plt.yticks(np.arange(0, 1.05, 0.1), fontsize=20)
plt.ylabel('$P_d$', fontsize=40, rotation=0, labelpad=40)
sns.lineplot(x='Pfa', y='Pd', hue='SNR', 
             data=data2, palette='gist_ncar', ax=ax)

ax.annotate(" $\lambda$", xy=(0.15, 0.81),  xycoords='data',
            xytext=(90, 110), textcoords='offset points',
            size=20, ha='left', 
            arrowprops=dict(arrowstyle="->", color='black',
                            connectionstyle="arc3,rad=0.1"))
plt.grid(color='black', linestyle='--', linewidth=1, alpha=0.3)

plt.legend(fontsize='small', shadow=True, title='$SNR(dB)$',
           borderpad=0.9, fancybox=True);

, P_d $P_{fa}$ . , SNR , , (9) . , (6) SNR .

# ,         sigma 
def conditional_density(x, sigma):
    return norm.pdf(x/sigma), norm.pdf((x - 1)/sigma)
  
  
  
#  DataFrame         SNR
data3 = []
x_range = np.linspace(-5, 5, 100)
for SNR in SNR_values:
    for x in x_range :
        cond_dens1, cond_dens2 = conditional_density(x, snr_to_sigma(SNR))
        data3.append([cond_dens1, cond_dens2, str(SNR), x])
        
data3 = pd.DataFrame(data3, columns=['cond_dens1', 'cond_dens2', 'SNR', 'x'])

fig, ax = plt.subplots(figsize=(20, 10))
sns.set_context('poster')
plt.title('      $SNR$', 
          fontsize=30)
plt.xlabel('$x$', fontsize=40)
plt.xticks(np.arange(-5, 5.5, 0.5), fontsize=20)
plt.yticks(np.arange(0, 0.45, 0.05), fontsize=20)
plt.ylabel('  ', fontsize=20, 
           labelpad=30)

sns.lineplot(x='x', y='cond_dens2', hue='SNR', 
             data=data3, palette='gist_ncar', ax=ax)

sns.lineplot(x='x', y='cond_dens1', hue='SNR', 
             data=data3, palette='gist_ncar', ax=ax, legend=False)

ax.legend(fontsize='small', shadow=True, title='$SNR(dB)$',
           borderpad=0.9, fancybox=True, loc='upper left')

ax.annotate("$f_{X|H_0}(x|H_0)$", xy=(-0.6, 0.35),  xycoords='data',
            xytext=(-90, 20), textcoords='offset points',
            size=30, ha='right', 
            arrowprops=dict(arrowstyle="->", color='black',
                            connectionstyle="arc3,rad=0.1"))

ax.annotate("$f_{X|H_1}(x|H_1)$", xy=(1.6, 0.35),  xycoords='data',
            xytext=(100, 20), textcoords='offset points',
            size=30, ha='left', 
            arrowprops=dict(arrowstyle="->", color='black',
                            connectionstyle="arc3,rad=0.1"))

plt.grid(color='black', linestyle='--', linewidth=1, alpha=0.3);

. 7 , SNR , , , , . P_d :

data4 = []
for SNR in SNR_values:
    for lambda_ in lambda_values:
        Pfa, Pd = roc_curve(snr_to_sigma(SNR), lambda_)
        data4.append([Pfa, Pd, str(SNR), lambda_])
        
data4 = pd.DataFrame(data4, columns=['Pfa', 'Pd', 'SNR', 'lambda'])

fig, ax = plt.subplots(figsize=(16, 12))
sns.set_context('poster')
plt.title(' $P_d$  $\lambda$   $SNR$', fontsize=40)
plt.xlabel('$\lambda$', fontsize=40)
plt.xticks(np.arange(0, 1.05, 0.1), fontsize=20)
plt.yticks(np.arange(0.5, 1.05, 0.1), fontsize=20)
plt.ylabel('$P_d$', fontsize=40, rotation=0, labelpad=40)
sns.lineplot(x='lambda', y='Pd', hue='SNR', 
             data=data4, palette='gist_ncar', ax=ax)

ax.annotate(" $\lambda$", xy=(0.5, 0.85),  xycoords='data',
            xytext=(-10, 50), textcoords='offset points',
            size=20, ha='right', 
            arrowprops=dict(arrowstyle="->", color='black',
                            connectionstyle="arc3,rad=-0.2"))
plt.grid(color='black', linestyle='--', linewidth=1, alpha=0.3)

plt.legend(fontsize='small', shadow=True, title='$SNR(dB)$',
           borderpad=0.9, fancybox=True);

P_d λ=0 SNR=5 . :

fig, ax = plt.subplots(figsize=(16, 12))
sns.set_context('poster')
plt.title(' $P_{fa}$  $\lambda$   $SNR$', fontsize=40)
plt.xlabel('$\lambda$', fontsize=40)
plt.ylabel('$P_{fa}$', fontsize=40, rotation=0, labelpad=40)
sns.lineplot(x='lambda', y='Pfa', hue='SNR', 
             data=data4, palette='gist_ncar', ax=ax)

ax.annotate(" $\lambda$", xy=(0.5, 0.35),  xycoords='data',
            xytext=(-10, 90), textcoords='offset points',
            size=30, ha='right', 
            arrowprops=dict(arrowstyle="->", color='black',
                            connectionstyle="arc3,rad=0.2"))
plt.grid(color='black', linestyle='--', linewidth=1, alpha=0.3)

plt.legend(fontsize='small', shadow=True, title='$SNR(dB)$',
           borderpad=0.9, fancybox=True);

λ=0 , $\small \frac{1}{2}$ ,

$\large \left.\frac{1}{\sqrt{2 \pi \sigma^{2}}} \int_{\lambda}^{+\infty} e^{-\frac{x^{2}}{2 \sigma^{2}}} d x\right|_{\lambda=\eta_{0}}$

, $\small \frac{1}{2}$ , .

, ROC- (6) , P_d $P_{fa}$ .

$\Large \tan \left(\alpha_{\lambda}\right):=\frac{\left(\frac{d P_{d}}{d \lambda}\right)}{\left(\frac{d P_{f a}}{d \lambda}\right)} \ \ ,(14)$

$\Large \begin{aligned} &\tan \left(\alpha_{\lambda}\right)= \frac{\frac{d}{d \lambda}\left(\frac{1}{\sqrt{2 \pi \sigma^{2}}} \displaystyle \int_{\lambda}^{+\infty} e^{-\frac{(x-1)^{2}}{2 \sigma^{2}}} d x\right)}{\frac{d}{d \lambda}\left(\frac{1}{\sqrt{2 \pi \sigma^{2}}} \displaystyle \int_{\lambda}^{+\infty} e^{-\frac{x^{2}}{2 \sigma^{2}}} d x\right)}=\frac{-\frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(\lambda-1)^{2}}{2 \sigma^{2}}}}{-\frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{-\frac{(\lambda)^{2}}{2 \sigma^{2}}}}=\\ &=\frac{e^{-\frac{(\lambda-1)^{2}}{2 \sigma^{2}}}}{e^{-\frac{(\lambda)^{2}}{2 \sigma^{2}}}}=e^{-\frac{(\lambda-1)^{2}}{2 \sigma^{2}}+\frac{(\lambda)^{2}}{2 \sigma^{2}}}=e^{\frac{-\lambda^{2}+2 \lambda-1+\lambda^{2}}{2 \sigma^{2}}}=e^{\frac{2 \lambda-1}{2 \sigma^{2}}} \ \ \ (15) \end{aligned}$

(15) :

$\ Large \ begin {array} {l} \ displaystyle \ lim _ {\ lambda \ rightarrow- \ infty} \ left (\ tan \ left (\ alpha _ {\ lambda} \ right) \ right) = \ lim _ {\ lambda \ rightarrow- \ infty} \ left (e ^ {\ frac {2 \ lambda-1} {2 \ sigma ^ {2}}} \ right) = 0 \\ \ displaystyle \ lim _ {\ lambda \ rightarrow + \ infty} \ left (\ tan \ left (\ alpha _ {\ lambda} \ right) \ right) = \ lim _ {\ lambda \ rightarrow + \ infty} \ left (e ^ {\ frac {2 \ lambda-1} { 2 \ sigma ^ {2}}} \ right) = + \ infty \ end {array} (16)$

(15) (16) , ROC- + ∞ + ∞ . , $λ = -∞ \$ tan⁡ (α_λ) = 0 , , α_λ = 0 ° . λ = + ∞ , tan⁡ (α_λ) = + ∞ , .. α_λ = 90 ° ( ).

# ,    Pfa  Pd   sigma  lambda
def dif_roc_curve(sigma, lambda_):
    return (derivative(lambda x: 1 - norm.cdf(x/sigma), lambda_, dx=1e-10),
           derivative(lambda x: 1 - norm.cdf((x - 1)/sigma), lambda_, dx=1e-10))


data5 = []
for SNR in SNR_values:
    for lambda_ in [0, 0.5, 1]:
        dPfa, dPd = dif_roc_curve(snr_to_sigma(SNR), lambda_)
        Pfa, Pd = roc_curve(snr_to_sigma(SNR), lambda_)
        tan = dPd/dPfa
        for i in np.arange(0.01/tan, 0.2/tan, 0.01/tan):
            data5.append([tan*(i - Pfa) + Pd, i, str(SNR), lambda_])
        
data5 = pd.DataFrame(data5, columns=['y', 'x', 'SNR', 'lambda'])

fig, ax = plt.subplots(figsize=(18, 14))
sns.set_context('poster')
plt.title('$ROC$-  $3^{}$ ', fontsize=40)
plt.xlabel('$P_{fa}$', fontsize=40)
plt.xticks(np.arange(0, 1.05, 0.1), fontsize=20)
plt.yticks(np.arange(0, 1.05, 0.1), fontsize=20)
plt.ylabel('$P_d$', fontsize=40, rotation=0, labelpad=40)
sns.lineplot(x='Pfa', y='Pd', hue='SNR', legend=False,
             data=data[data['SNR']  == '5.0'], palette='gist_ncar', ax=ax)
sns.lineplot(x='x', y='y', hue='lambda', linestyle='--',
             data=data5[data5['SNR']  == '5.0'], 
             palette='dark:b', legend=True,  ax=ax)

plt.grid(color='black', linestyle='--', linewidth=1, alpha=0.3)

plt.legend(fontsize='small', shadow=True, title='  $\lambda=$',
           borderpad=0.9, fancybox=True, loc=4);

Fig. 10 ROC curve SNR = 5 with three tangents at the points. — . 10 ROC- SNR=5 .

. 10 $\ left (P_ {fa} (0), P_ {d} (0) \ right), \ left (P_ {fa} (0.5), P_ {d} (0.5) \ right), \ left (P_ {fa } (1), P_ {d} (1) \ right)$ .

ROC- (12) . ROC- . , , SNR , . , ROC-, .

- . , . 1. , --, 1968, , . , . . . . . ., « », 1972, 744 .
Tyapkin V.N. Fundamentals of building radar stations of radio-technical troops: textbook / V.N. Tyapkin, A.N. Fomin, E.N. Garin [et al.]; under total. ed. V.N. Tyapkin. - Krasnoyarsk: Sib. Feder. un-t. - 2011 .-- 536 p.

Additionally

Link to github with source code.

Analysis and construction of ROC-curves: communication with the radar

Formulation of the problem

ROC-

Additionally

More articles: