Tools for Competitors in Machine Learning Competitions

Machine learning competitions are a relatively new phenomenon.

It appeared as a result of the development of artificial intelligence technologies.

At the moment it is actively developing and attracting many interested people.

Benefits for the organizers of the competition:

• A large number of qualified people who work on their task and try to solve it better than others
• Relatively small (in comparison with hiring specialists) financial costs
• The solution to the problem, the highest quality and most suitable for it

And the competitors also benefit:

• Public recognition of high qualifications
• Cash prizes
• And just the pleasure of participating and winning

In this article, I want to consider several tools that can help participants organize the process better and more efficiently, increase the likelihood of winning, and, in general, become a more qualified specialist.

Let's get started!

Determined

A platform for training deep learning models.

• Accelerated training of models using state-of-the-art distributed training, without changing the model code
• Automatic search for high quality models, with advanced hyper-parameter settings - from the creators of Hyperband
• GPU GPU,
• , , , -
• DL-
• ,

Compose

.

• ,
• Featurepools
• EvalML

Featuretools

.

EvalML

AutoML , .

• Featuretools Compose end-to-end ML-

Pandas Profiling

DataFrame Pandas.

• df.describe() — df.profile_report()
• HTML-
• :
• : , ,
• : , Q1, , Q3, , ,
• : , , , , , , ,
• : ,
• : ,
• : ( , ), (, ) (ASCII)
• : , , , EXIF

Tpot

, .

• , ,
• Python
• Scikit-learn

Shap

- ML-.

Feature-engine

ML-.

• ,
• , , , , ,

Lale

-.

• ,
• —
• — JSON -
• —

Biome

.

• — , ; ; ;
• — , ; (, , , , , ), (PDF, Word, Excel, HTML, E-mail ); , ; , ,
• — , ;

DataSketch

.

PyTextRank

.

• ( , )
• ( )

Joblib

.

• —
• Pickle ,

Shampoo

.

• ,
• , ,

Michelangelo

Uber.

• API

Hasty.ai

.

Cortex

.

• API

Weights & Biases

.

• -
• —
• -
• —
• —

SpeedRun

ML-.

• Weights & Biases
• - Weights & Biases
• ,
• matplotlib

Great Expectations

— , .

Keras Tuner

-.

NanoEdge AI Studio

AI-, MCU C .

• MCU C
• Arm Cortex-M
• (1-20kB RAM/Flash)
• (1-20ms M4 80MHz)
• AI
• C
• , MCU
• ML

LabelBox

End-to-end .

• API (Python, GraphQL) SDK
• : ,
• : , 30 FPS ,
• : , ,
• API

LabelML

ML- .

• (2 )
• , -s, -
• Tensorboard
• API

PyCaret

Low-code ML-.

• —
• -
• ( 60 )
• ( , , , - )

CometML

• , ,
• — , -, , , ,
• — , ,

ClearML

ML- (MLOps).

• ,
• — , , , , ,
• GPU/CPU ,
• — ; ; ,

, , ,

• Classical music
• Good mood

Conclusion.

Of course, just describing the tools is not enough to always win.

Success depends on many other factors - knowing where and when to use a particular tool or not, what are the limitations, how the tools can be combined, etc. etc.

I hope that nevertheless this article will be useful for you and your participation in the competition will become more fruitful and effective.

Forward to victories!