Distributed training on Amazon SageMaker
Amazon SageMaker makes it easier and faster to train large models and process large amounts of data. The new Distributed training on Amazon SageMaker product enables distributed training and supports both data and model parallelism. This requires minimal code changes. Now you can easily split the data into parts and train on different GPUs. You can also split the model itself so that multiple GPUs are used for training. This is useful for large models where GPU alone is not enough.
More details here
Amazon SageMaker Clarify
Machine learning models are often a black box. It is difficult to understand why the model produced this or that result. Amazon SageMaker Clarify can help you understand how the models work and the factors that influenced each outcome. This is especially important for auditing models.
In addition, the data may not be balanced. The historical data on which ML models are trained is incomplete and often biased. For example, if earlier people at a certain age took out few mortgages, then a model trained on such data may refuse mortgages to people of this age in the future. Which will only reinforce the bias. Amazon SageMaker Clarify allows you to identify these types of imbalances and bias in your data. This makes the models work better for everyone.
More details here
Amazon SageMaker Debugger
Amazon SageMaker Debugger is a handy tool for debugging and profiling models, collecting and analyzing training data, generating reports, and visualizing metrics. SageMaker Debugger has received many updates this year, as well as a completely redesigned user interface.
Details here
Amazon SageMaker JumpStart
Amazon SageMaker JumpStart allows you to quickly deploy an off-the-shelf solution or ML model. Already 15 solutions are available for tasks such as handwriting recognition, demand prediction, fraud and malicious user detection, and much more. In addition, SageMaker JumpStart lets you deploy one of the 150 open source ML models from the TensorFlow Hub and PyTorch Hub in a few clicks.
Details here
Amazon SageMaker Edge Manager
A toolkit (MLOps) to turn your smart devices into edge smart devices that can run cloud-trained models, collect telemetry, and send sample data back to the cloud for retraining. Amazon SageMaker Edge Manager can also help you monitor the health of your fleet of devices and update models optimized with SageMaker Neo.
The SageMaker Edge Agent is a small runtime hosted on a device that can run models, collect telemetry, and send sample data back to the cloud.
SageMaker Neo is a tool that optimizes your models for low power devices already included in the agent runtime.
SageMaker Edge Dashboards help you monitor the health of your devices by drifting models.
More details here
Amazon Redshift ML
Now you can train models and do inference directly in SQL queries to Redshift, thanks to the integration with SageMaker AutoPilot, which will prepare the data and select the most suitable algorithm. And all further predictions can already be made using the resources of the RedShift cluster without unloading data from it.
This makes it easier for the developer or analyst to work with data and removes steps such as uploading data to staging storage, starting the training process, hosting the model, and the prediction process.
More details
Amazon Neptune ML
Amazon Neptune ML is a new feature for graph-based managed databases - Amazon Neptune. Built using the Deep Graph Library , it can improve accuracy by up to 50% compared to other libraries and algorithms that do not specialize in graph data sources.
Amazon Neptune ML can predict both missing nodes (classification node) and edge (weight regression).
Amazon Lookout for Metrics
A new service that automatically detects anomalies in your business metrics without requiring any development or machine learning skills. The service can connect to data sources such as Salesforce, Marketo, Google Analytics, Slack, Zendesk and many others.
It can be used to monitor, find and alert about anomalies, and is also capable of showing the potential cause of an anomaly on data such as: web page views, customer churn rate, daily active users (DAU), transactions, mobile app installs and many others. ...
Plarix has already got access to the preview and has tested this service to analyze its data.
“We experimented with our user acquisition data to understand how the service works and it quickly identified and grouped anomalies enabling us to work faster and better,” - Mikhail Artyugin, CTO at Playrix.
More details here
Russian-language Twitch session
Today the next Russian-language twitch stream will take place, now on the novelties in the field of machine learning. We remind you that streams are taking place on the key days of AWS re: Invent. Streams are prepared and conducted by leading AWS solution architects, who choose the most interesting and useful from the news and announcements of the multi-hour conference. For those who have not connected yet - the registration link .
More on the topic:
AWS re: Invent. Main announcements of the first day (Part 1) of
AWS re: Invent. The main announcements of the first day (Part 2)