Hello, Habr! Together with Harmony OS, we introduced the fifth version of HMS Core, a set of tools that you can use to develop applications for the Huawei ecosystem. We added new possibilities for working with content, focused on data security, interaction between devices and expanded the possibilities for AI tools - we talked about all this in detail at our HDC.Together conference , and in this article we will give an overview of new HMS capabilities.
Tools for working with ML and AI
The main services for working with AI are included in the ML Kit and allow you to work with text, voice, pictures, AR / VR technologies. In HMS Core 5.0, we increased the number of supported languages to 50 and can translate between 20 languages, while both text and voice recordings are accepted as input. Also ML Kit can be used to filter out spam and any obscene images.
With ML Kit it is possible to convert voice recordings to text format and vice versa, and we have extended this functionality with Video Course Creator, which automatically creates educational courses: it takes as input the educational program, teacher's comments and preset audio and video parameters - during the broadcast, the teacher can refer to the necessary materials and all recorded materials are assembled into a single course.
We have made a special emphasis on face recognition, gestures and poses. Thanks to this, using ML Kit, you can perform face authentication - the system is connected via API and can be used for quick access to applications without any additional actions.
You can work with AI and ML tools at different levels:
- HiAI , , NPU-.
- Ability Gallery AI .
AR/VR
Our AR engine analyzes information about lighting, plane, object shape, surface type, and can search for specified objects in space. Separately, the system can build a 3D scheme using anchor points and track human movements, gestures and facial expressions. For example, 21 points are allocated to determine the position of the hand, and 23 points are allocated to the position of the body. Now the system can recognize 6 poses and analyze 2 people at once.
With the fifth version, HMS Core CameraKit provides various shooting modes: wide aperture, portrait mode, HDR, background blur, supernight mode and others like them. It also became possible to use AI in photo and video shooting for pre-selection of filters and color correction.
Collaboration of devices
The new version of HMS Core has several new engines for collaboration between different devices. We have simplified interactions between devices and smart things using the DeviceVirtualization engine. It converts peripheral devices into virtual objects and allows the phone to work with them in the same way as with normal functions.
In the same direction, we developed the Cast Engine , which allows the phone to work in multi-screen mode and transfer images to TV in high definition. We also plan to add the ability to work with Cast Engine through other applications.
Another new OneHop Kit engineworks with NFC and allows you to securely transfer data between devices with one touch. With it, you can sync Huawei devices with each other, transfer files and open applications on other devices with the same settings as on the main one. So far, it only works between Huawei phones and tablets, so we provide a Share engine for communication with other devices , which provides speeds up to 80 Mb / s via Bluetooth.
Video and audio
In the new version, we have developed several new SDK tools for working with audio and video. Audio Kit and Video Kit support major file formats and codecs and can be easily integrated into the application and read, save and delete lists, play sequences, adjust playback speed. All audio and video are protected by encryption algorithms.
The Audio Kit now has audio engine to record sound in high quality with the optimization of delay and other functions of the instrument. The video engine, in turn, supports the main protocols HTTP, HTTPS, HLS, DASH. It also allows you to organize streaming from third-party services using the Video Kit WisePlayer SDK.
Based on our serviceHuawei Meetime has launched the Caas Kit for video calls, which provides HD picture playback and transmission even with unstable connections. The engine also allows you to share the screen, draw doodles and, using virtualization technologies, transfer images to any other devices, which can be used for business conferences.
Pictures and 3D rendering
Image Kit has appeared for image processing - it offers more than 20 filters and visual effects, including animation with rendering. It includes 2 SDKs: Image Vision SDK for working with color filters and Image Render SDK for using animation effects.
Scene Kit is designed to work with 3D objects: it uses the method of physically correct rendering (PBR) and allows applications to connect via API and receive 3D models of complex objects. The engine offers three scenarios: SceneView for shared scenes (non-AR), ARView for shared AR scenes, and FaceView for working with faces in AR scenes.
Analytics and security
In Huawei Core 5.0, we launched the Dynamic Tag Manager (DTM) tagging system to track the marketing activity of users: it integrates with both Huawei services themselves and with third-party platforms for sending and processing data. Using the DTM, you can dynamically update tags in the web user interface, track specific events, and send data to third-party analytics platforms. Our systems focus on security, which is why the DTM engine is also used as an antifraud system to track suspicious activity.
One of the features of the new version of HMS is face authentication using LocalAuthentication Engine... It works with an infrared camera that builds the model from the reference points and authenticates with the ML Kit.
Finally, we paid attention to content protection: for audio files, cache encryption is used, for video - an anti-lytic system and authentication during playback, and for working with a camera - end-to-end encryption. All data HMS stores in a trusted TEE environment , within which encryption is performed.
Where can I find out more?
All recordings of the HDC.Together conference are available here . Here you can listen to reports on platform updates, the main scenarios for working with HMS tools and ask technical questions on the Huawei Developer Forum or Stackoverflow with the huawei-mobile-services tag .