Clever Geek Handbook

Azure Custom Vision without Azure, or "where is their mask?" How we recognized the mask on the face (and other parts of the body)

Among the set of examples for Azure on GitHub , one very interesting was found: image recognition on the Raspberry Pi, offline. The authors suggest preparing a machine learning model in one of the Azure cloud services, then transferring it to a computer that has no Internet connection most of the time, after which the pattern recognition will work offline. The developers have prepared a project for two platforms: ARM32 (Raspberry Pi itself) and AMD64 (but without webcam support).







In the wake of the coronavirus hype, we decided to adapt the example to recognize whether a person is wearing a mask or not. It immediately became clear that the steps to adapt the example were not entirely obvious. In this material, we will sequentially consider how this example, firstly, can be modified to work with our machine learning model, and secondly, it can be transferred to the AMD64 platform. Let us assume that our computer, on which the model is running, controls the turnstile: if a person comes wearing a mask, the turnstile opens, if not, it remains closed.







It's Complicated...



Almost every cloud provider has a set of machine learning tools, and the barriers to entry are often low.







Azure Computer Vision, , : API, . Azure Custom Vision, , . , . - , , .







, . , "" , :







  • , - , ;
  • ;
  • .


, :







  • ;
  • , . , "" 100 ?


?



, " ", " ". — . , " ". , Azure Custom Vision, , , .







: Azure IoT Edge . , :







  • ;
  • — ;
  • , .. , ;


Azure IoT Edge



IoT Edge . , IoT Edge — Linux, "" , , , Docker- . IoT Edge Azure IoT Hub. IoT Edge , , Azure, IoT Edge , .







"" IoT Edge Linux, 2019 IoT Edge Windows 10 Enterprise LTSC — .







IoT Edge , "" Azure:







  • Azure Functions ( )
  • Azure Stream Analytics ( )
  • Azure Machine Learning ( "")
  • Azure Custom Vision ( )
  • Azure SQL Server ( "")


Custom Vision. "" .







IoT Edge . , (), Microsoft HTTP, .. web-, .







Microsoft



. , , .













-, USB , IoT Edge.







IoT Edge? , , IoT Hub, ( ) IoT Hub. "" .







IoT Hub. IoT Edge "" , . , Container Registry ( ).







, , :













  • Camera: . "" ( ! ) HTTP ;
  • AI: , . , , "" . Custom Vision;
  • Display: , . , , , ( ).


, IoT Hub.







- , . , - Raspberry Pi , .







, . AMD64 , - - , , . , "" -, ARM32.







. AMD64 "" SenseHat, Raspberry Pi, , SenseHat, , - .







(Custom Vision)



Custom Vision ( Azure) New Project.







  • Name — ;
  • Description — ;
  • Resource — Create new Cognitive Services;
  • Classification Types — Multilabel, / , — ( " ")
  • Domains — General (compact).


:







  • , MaskOnFace;
  • (" "), MaskNotOnFace — - , (, -, );
  • , , (Negative).


.







Train, Advanced training , . Performace .







, , .







Probability Threshold — , . Probability Threshold = 90% , , 90%. , . , (/), "" " " "" "". , - " 89%". , "" "". Probability Threshold 90%, " 89%" "", " 91%" "".







, , "" "". , , . , .







  • Recall () ;
  • Precision () — .


— .







, .







(Publish), (Export — Dockerfile — Linux). IoT Edge.









- ( — ) AVX (, AVX , , grep avx /proc/cpuinfo) — tensorflow. AVX, , . . Intel NUC Core i5.







Ubuntu 20.04 LTS. , IoT Edge ( Microsoft?), .







, Edge. (~).







:







sudo apt-get update
sudo apt-get install wget nano


:







wget https://packages.microsoft.com/config/ubuntu/20.04/prod.list
mv prod.list microsoft-prod.list


sources.list.d, "" Microsoft:







sudo cp ./microsoft-prod.list /etc/apt/sources.list.d/


Microsoft GPG:







wget https://packages.microsoft.com/keys/microsoft.asc
cat microsoft.asc | gpg --dearmor > microsoft.gpg
sudo cp ./microsoft.gpg /etc/apt/trusted.gpg.d/


( — , apt " "):







sudo apt-get update
sudo apt-get install moby-engine


IoT Edge:







sudo apt-get install iotedge


, ( Ubuntu 20.04 — ), . Ubuntu (18.04 — , , ) (, , ). GitHub. Latest release dpkg. libssl :







wget http://archive.ubuntu.com/ubuntu/pool/main/o/openssl1.0/libssl1.0.0_1.0.2n-1ubuntu5_amd64.deb
sudo dpkg -i libssl1.0.0_1.0.2n-1ubuntu5_amd64.deb
wget https://github.com/Azure/azure-iotedge/releases/download/1.0.10.4/libiothsm-std_1.0.10.4-1_ubuntu16.04_amd64.deb
sudo dpkg -i libiothsm-std_1.0.10.4-1_ubuntu16.04_amd64.deb
wget https://github.com/Azure/azure-iotedge/releases/download/1.0.10.4/iotedge_1.0.10.4-1_ubuntu16.04_amd64.deb
sudo dpkg -i iotedge_1.0.10.4-1_ubuntu16.04_amd64.deb


IoT Edge , Azure. , :







sudo nano /etc/iotedge/config.yaml


, Azure device_connection_string provisioning. , , IoT Edge Azure.







Azure



IoT Edge IoT Hub, Azure. — .







IoT Hub IoT Edge . , , :









IoT Edge IoT Hub, .









IoT Edge :







  • . -, USB. HTTP;
  • , ( / / ). HTTP, ;
  • (-). -, .


:













, , .







! Windows 10, IoT Edge.




! Docker : bcdedit /set {current} hypervisorlaunchtype Auto .



! Hyper-V VirtualBox. , VirtualBox, : bcdedit /set {current} hypervisorlaunchtype off, "" Hyper-V Docker.

, :







git clone https://github.com/Azure-Samples/Custom-vision-service-iot-edge-raspberry-pi.git


, :







git checkout 6b3540f9b31121321f9e75d8df0ed86397c9324f


Visual Studio Code (File — Open Folder) . , . , .









, , . , , , . , , .







. — . — , , .







, .







.env



() , Azure:







  • CONTAINER_REGISTRY_ADDRESS="__.azurecr.io"
  • CONTAINER_REGISTRY_USERNAME="_"
  • CONTAINER_REGISTRY_PASSWORD=""


deployment.template.json



  • modules — camera-capture — env — RESIZE_WIDTH — 640, RESIZE_HEIGHT — 480
  • modules — camera-capture — settings — image — ${MODULES.CameraCapture.amd64}
  • modules — sensehat-display — env — THRESHOLD — value — 0.9
  • modules — sensehat-display — settings — image — ${MODULES.SenseHatDisplay.amd64}, createOptions — HostConfig "PortBindings": { "8000/tcp": [ { "HostPort": "8000" } ] }
  • modules — image — settings — image — ${MODULES.ImageClassifierService.amd64}


deployment.template_AMD64



.







ameraCapture\amd64.Dockerfile



AMD64 -, , , . .







CameraCapture.py



( Raspberry Pi), , . time.sleep(1.0) 1.0 0.1.







ImageClassifierService\amd64.Dockerfile



AMD64. .







labels.txt



, . Custom Vision, :







MaskNotOnFace
MaskOnFace


model.pb



Custom Vision — .







SenseHatDisplay\amd64.Dockerfile



, .







SenseHatDisplay — . AMD64 SenseHat , , , , SenseHat, , - Python, .







.







SenseHatDisplay\module.json



platform , amd64:







"amd64": "./amd64.Dockerfile",


SenseHatDisplay\app ()



, . -, .







  • custom.js
  • index.htm
  • jquery.js
  • style.css
  • DisplayManager.py
  • MessageParser.py


- "". , :









, - — , . , , . http://127.0.0.1:8000.









.







  • VS Code View — Command Pallette — Azure IoT Edge: Set Default Target Platform for IoT Edge Solution amd64;
  • Azure IoT Edge: Build and Push IoT Edge Solution. , IoT Edge;
  • IoT Edge, config/deployment.json Create Deployment for Single device, , ;
  • ( -!). IoT Edge Extension Start Monitoring D2C Message, .




IoT Edge UI - http://localhost:8000, -, . , , .













— , .







, , MaskNotOnFace, , .







- ...



, , IoT Edge . ( , , , AVX / tensorflow ). IoT Edge . . , :







  • : iotedge list. ( );

  • : iotedge logs _. , .




, IoT Edge, , ( Azure), IoT Edge .







?



IoT Edge , .







, , IoT Edge, , , , , " " - SQL Edge, .







IoT Edge Azure, .







— , , — .







, Azure , , , .







Microsoft — .







.







— , . sergant (at) quarta.ru.







More articles:


All Articles