By profession I am an editing director, and applied programming is like a hobby in my free time.
At some point, the idea came to combine work with a hobby, I read an article on Habré on recognizing objects in pictures using Core ML, and this is how it all started. I will share my modest experience and problems that you can face when developing applications working with Core ML.
The fact is that almost a third of the work of a video editor consists in the routine search for video sequences from the sources, which must be furled every time in search of a contextual plan for offscreen text, in my opinion this does not carry any creative component, especially when you have been doing this for 15 years). Well, I thought, what if I write a software that will go through the folder with the source code, recognize objects, carefully "put" them in the database. Further, at the moment of searching for video fragments for the so-called "jeans", a search word is entered, for example "Sun", and everything that is in some way is transferred to the editing system.
The idea matured, was going to stack, I decided to write in Swift, trained models of Core ML itself, SQLite database. At first glance, the idea seemed easy to implement, like nothing complicated.
Very quickly I threw in the main code that pulls frames from the video, recognizes objects using the Resnet50 model , which was recommended by Yabloko on their website, it worked very quickly and allowed you to adjust the percentage at which the object was considered recognized. The code itself is quietly distributed on the same apple.com for everyone. I connected the SQLite.swift library , wrapped its functions in my methods, everything works!
! 1000- mvc- , Stackoverflow. . , . , , , .
, Grand Central Dispatch (GCD) - Apple, , - ! ! .
, 70 , - ! ! 420- , - windows mobile, , , , …. 420 ! , , ! ? …. … 420 … .
, ( ) ! VNCoreMLRequest, ML- , , . «» copyCGImage, , generateCGImagesAsynchronously, , «» try catch.
. , . : YOLOv3 Resnet50, , . .
Apple Xcode Create ML , , , .
, - . - EDL XML. , «» , . , EDL , , , , , , , , . XML! : , , , , , ! , , XML, , -. Adobe Premiere XML. . , - , , , , . , String Xcode. , , . ! html-. , , ) ! XML , dobe Premiere, , , , Final Cut Pro ( ),
, , , , .
, , , , . Apple Silicon, ML 16x, . Mac App Store, Videoindex.
,