, ML- , . ML-, , . - , , ML-.
, ( ).
Let's start with the most commonplace reason for metric degradation in production - technical errors . For example, the predictions of the model in offline tests and in production may differ due to the difference in the code used to preprocess images. There are some simple ways to protect yourself from this shame:
- . (, , ) , . . : , - , ( – , – ).
- -. , -. , , -, , ML- .
- . , , ML-, , (, ). , , COVID-19 . , , . ( ).
— , . .
, (ROC-AUC = 0.53, ), . , ML- . , ML- - , , .
, . , , . , , , - . ML- Out-of-Distribution . , , ML-. .
. , , (, ). , (, ML- - ) . ML covariate shift concept drift.
, , ML- , . , , , , .
, , . ( , , ) , . , - . , , : .
, . -. - . ML- , , .
, . – , .
, ( ) ML- , - . ML- . -, , .
, ML- – ML-. , , :
- ( -, , )
- ML- Out-of-Distribution-
- ,
- , –