How image search works in Dropbox

If you need to find a photo taken at a picnic a few years ago, you can hardly remember the name that the camera automatically assigned to the file when it was taken , for example, 2017-07-04 12.37.54.jpg . You look through everything - photographs, their sketches, trying to identify objects or signs of what you are looking for - and it does not matter whether you are looking for a lost photo or want to find a decent snapshot in the archives for presenting a new project.

It would be great if Dropbox could independently view all the images and select the ones that best match the words in the description! This is exactly the task we set ourselves when creating the image search function .

, Dropbox , , , , Dropbox .

---

, , , Dropbox.

: , () q j, s , , :

s = f(q, j).

- , , . : .

, . , . 

:

• , , ;

  
  
  
  
  

• , , ;

  
  
  
  
  

• , , - .

  
  
  
  
  

— 2012 . Krizhevsky . ImageNet hallenge. , , , , Open Images ImageNet, , TensorFlow PyTorch, , . , :

, , . , , , , , ? , ?

, , , .

. j_c . C- , C — ( ). , , , .

— . — word2vec — Mikolov . 2013 . Word2vec , , , , . d- , d .

, word2vec. , , :

1. q d- q_w, . w, — c.

   
   
   
   
   

2. cⁱ_w. m̂i = q_w - cⁱ_w — i- . -1 1 , . ( mi = max(0, m̂i)), , .

   
   
   
   
   

3. , q_c = [m1 m2 ... mC], C- , , — , , .

   
   
   
   
   

3 — - , q_c = q_wC, C — cⁱ_w.

q_c — s = q_cj_c.

. . - , s = q_cJ, J j_c , s — .

. , — : , , .

, . , [0,35–0,62 0,70], .

, .

  EfficientNet, OpenImages. 8 500 . , . , Dropbox.

TensorFlow   ConceptNet Numberbatch. , , , , . : dog chien , .

AND, . , , beach ball, . , OR , beach ball (beach AND ball) OR (beach ball). , .

, , J . , , , , . ().

J , Dropbox Nautilus.

, Nautilus (forward index), (, ) , (inverted index), (posting list) , . :

, doc_1 doc_2 , . doc_3 , , . 

, , . doc_1 , doc_2, doc_1 .

. j_c . .

, :

1. q_w C q_c, . C — , , .

   
   
   
   
   

2. , q_c, . , , .

   
   
   
   
   

3. j_c q_c, s. , .

   
   
   
   
   

- , . 10 000 10 000 , 40 . , 10 000 . , 40 80 . , , !

, , , , m̂i , 5 000 . , 10 .

, , , . s = q_cj_c, q_c — 10 000 , j_c — 10 000 , . , s. 

q_c j_c , . , 10 q_c 50 j_c . :

• 10 000- 50 , 50 . ; 50 ( ) 50 ( ) 300 .

  
  
  
  
  

• 10 000, 50 , 200 . , 500 80.

  
  
  
  
  

• q_c 10 , 10 — . , .

  
  
  
  
  

, . , , , — .

-. , OCR- , .

?

, , Dropbox. . , , . , , , , , " , " .

, , . , "Machine Learning Deep Learning", NVIDIA.

, :

• Data Scientist

  
  
  
  
  

• Data Analyst

  
  
  
  
  

• Data Engineering

  
  
  
  
  

• Fullstack- Python

  
  
  
  
  

• Java-

  
  
  
  
  

• QA- JAVA

  
  
  
  
  

• Frontend-

  
  
  
  
  

• 
  
  
  
  

• C++

  
  
  
  
  

• Unity

  
  
  
  
  

• -

  
  
  
  
  

• iOS-

  
  
  
  
  

• Android-

  
  
  
  
  

• Machine Learning

  
  
  
  
  

• "Machine Learning Deep Learning"

  
  
  
  
  

• " Data Science"

  
  
  
  
  

• " Machine Learning Data Science"

  
  
  
  
  

• "Python -"

  
  
  
  
  

• " "

  
  
  
  
  

• 
  
  
  
  

• DevOps