Scientists from DeepMind have compiled a Curated Resource List of educational materials for those who want to connect their lives with AI and machine learning. I call this collection "ontol" - a list of what forms the picture of the world on a given issue, ranked by importance and compiled by a living person, a specialist who is reputationally responsible for this list (so that there is no marketing and biased bullshit in it).
As planned, if a dozen of the best companies in the field of AI ask their leading specialists (each) to make a selection of the best materials that formed them as specialists, then we will receive an array of collections (list of top 10/100 resources + the name of the author) and based on this, will draw interesting conclusions (a) on the quality of the materials, which should be taught first of all b) on the quality of specialists who can highlight the main thing c) something else). This is how we will "mark up" all open text / video in the field of AI. Then we'll tackle other topics: food, trust, life's work, family, cooperation, cognitive distortions, and so on - what forms the picture of the world.
Test the beta.ontol.org prototype and subscribe to the @Ontol channel
Table of contents
Safety
Neuroscience
Natural Language Processing
Machine Learning
Deep Learning
Reinforcement Learning
Unsupervised Learning and Generative Models
21 Definitions of Fairness and Their Politics (video) - Arvind Narayanan discusses the various definitions of justice and their compromises that they represent to society.
Fairness and Machine Learning Book (book, video) - An overview of fairness in topics related to machine learning.
Harvard University's Justice Course (video) - In-depth and exciting lectures on justice and moral philosophy ( Translation ).
NeurIPS 2017 Tutorial on Fairness in Machine Learning (video) - Solon Barokaz and Moritz Hardt discuss in detail the sociotechnical elements of justice in machine learning.
The Trouble with Bias - NeurIPS 2017 (video) - Keith Crawford discusses the ethnic implications of bias in artificial intelligence systems.
Safety
AGI Safety Literature Review (Publication) - An excellent review of the general AI safety literature prior to 2018 with hundreds of references for further study.
AI Alignment newsletter by Rohin Shah - A weekly newsletter summarizing the latest work on AI security.
AI safety YouTube channel by Robert MIles (video) - Educational and entertainment videos introducing audiences to the key concepts of general AI safety .
Concrete Problems in AI safety (publication) - A useful overview on the safety of artificial intelligence, the original and the article has already become a classic in the field of AI security.
Human Compatible: Artificial Intelligence and the Problem of Control by Stuart Russell (book) - A must-read book on AI security by the original AI.
Theory and fundamental concepts
3Blue1Brown Youtube channel (video) - A great series of tutorials. Video from scratch on linear algebra and neural networks is especially useful.
A 2020 vision of Linear Algebra (Gilbert Strang, MIT) (video) - Briefly re-summarizes the whole course on linear algebra with technical details: how linear algebra is applied in real life, and especially in the field of machine learning.
Andrew Ng's Machine Learning course (online course) - The first very practical and extensive machine learning course. Since the course is on the Coursera platform, your assignments can be evaluated, and assistants and other students can help you with the course material.
Causal Inference in Statistics: A Primer(online preprint) - A great introduction to causal conclusion. This is a preprint of the full version of the latest book.
Causal Inference: What If (online book) - A new book on causal conclusion.
David MacKay, information theory course videos (video) - Covers a wide range of areas in McKay's corporate style lectures.
David MacKay's Course on Information Theory, Pattern Recognition, and Neural Networks (video) - Legendary David MacKay's Course on Information Theory, Data Pattern Detection, and Neural Networks.
Decision-theoretic foundations for statistical causality (online article) - An alternative way of formulating causal derivation operations.
Deep bayes summer school lectures and lab materials(video) - Lectures and practical tasks on probabilistic modeling and Bayesian training.
Elements of Causal Inference: Foundations and Learning Algorithms (online book) - This book introduces the reader to causal inference in an easy and accessible way.
Essence of Linear Algebra (3blue1brown) (video) - Gives a very good idea of the key ideas in linear algebra without going into technical details. Accompanies a traditional linear algebra textbook or college course.
Francis Bach's blog - Useful tips and tricks, in-depth analysis of various machine learning concepts.
Human intelligence enterprise course ( course materials) - History of human intelligence.
Is the Abstract Mathematics of Topology Applicable to the Real World? (video) - Introduction is an excellent description of the basics of topology. The workshop convincingly describes certain applications.
KhanAcademy courses (video) - A great beginner's introduction to statistics, probability theory, and the math required to understand machine learning.
Learning from Data course - Caltech (video) - A neat introduction to machine learning. A very clear explanation of the topics.
Lecture Notes on Monte Carlo - A brief explanation of the Monte Carlo method.
Mathematics for Machine Learning (book) - An excellent book covering the basic math concepts required for machine learning.
MIT Machine Learning course (online course) - An excellent 2006 course on the basics (and now history) of machine learning before deep learning and many levels of abstraction became mainstream.
Nando de Freitas Course on Machine Learning (video) - Useful course and presentation on machine learning.
Princeton Companion to Mathematics (book) - Probably the most striking mathematical source of all. The book provides a detailed overview of the most important concepts in modern mathematics, with no background in the self-proclaimed bedtime story format - fun, easy to understand, and intuitive.
Project euler(Problem Solving Community) - A series of complex mathematical problems and problems from computer science to activate the brain. They are very interesting to solve, and the acquired knowledge will help you in your career in the field of deep learning.
Statistical Learning Theory course (Online) - A free course in machine learning fundamentals for people with math education taught by Professors Hasti and Tibshirani.
Strang All the Key Ideas of Linear Algebra in 1 Lesson (video) - Laconic, comprehensive.
The Book of Why (chapters from the book) - An easy introduction to causal inference and a historical digression into its development.
Neuroscience
Brain Inspired Podcast - A podcast that merges neuroscience and artificial intelligence.
Center for Brains Minds + Machines Summer School Lectures (video) - Lectures from the famous Woods Hole Summer School on Computational * Cognitive * Neurobiology (more on high-level cognition, behavior and links to machine learning).
Computational Cognitive Modeling @ NYU (slides and texts) - An overview of computational approaches to modeling human cognition, closely related to artificial intelligence and machine learning.
Computational models of the neocortex (Class notes) - Interdisciplinary and cutting edge.
Lectures from Methods in Computational Neuroscience Woods Hole Summer School(video) - Lectures from the famous Woods Hole Summer School on the computational * system * of neurobiology (more on the cycles and system properties of the brain)
Marr's Levels of Analysis (Vision, 1982, Chapter 1) (chapter from the book) - Perfectly explains with examples using examples useful algorithms like EM. Serves as a great addition to Bishop's book.
MIT Brains, Minds, and Machines Summer Course (video) - A graduate-level course at the intersection of cognitive science, neurobiology, and artificial intelligence.
Probabilistic Models of Cognition (interactive tutorial) - An interactive tutorial that describes the use of a probabilistic model for creating and modeling human-like behavior.
The challenge of understanding the brain: where we stand in 2015(publication) - A good overview of neurobiology in terms of biology.
Theoretical Neuroscience (online book) - A popular introduction to theoretical neurobiology.
Natural Language Processing
A Code-first Introduction to Natural Language Processing (video) - An introduction to natural language processing for people with a technical background.
A Primer on Neural Network Models for Natural Language Processing (publication) - A clear overview of how neural networks are used in natural language processing.
CS224n: Natural Language Processing with Deep Learning (video) - Stanford Course on Modern Natural Language Processing .
NLP Progress (list of datasets and results) - A community-driven website listing a large number of tasks, datasets, and modern natural language processing results.
Oxford / DM NLP Course 2017(Course of Lectures) - Advanced Natural Language Text Processing Lecture Course given at Oxford by DeepMinders.
Speech and Language Processing (book) - An authoritative reference to natural language processing - now in 3D and available online.
The Annotated Transformer (blog post) - Great introduction to the dominant model of natural language processing.
Machine learning
Amii's Coursera Machine Learning: Algorithms in the Real World Specialization (online course) - An excellent overview on the formation and identification of machine learning problems and their solutions.
Bayesian Reasoning and Machine Learning (online book) - Fundamentals of Probabilistic Reasoning and Modeling.
David MacKay, Gaussian Process Basics (video) - The most accessible and straightforward introduction to the Gaussian process.
David MacKay's book "Information Theory, Inference, and Learning Algorithms" (book) - David MacKay presents a unique perspective on the relationship between information theory, inference, and learning. The style of his writing is unique, as is the humor in the book.
Getting into machine learning (blog) - A blog for those who want to do machine learning.
Lecture notes on Machine Learning - Herbert Yager lecture notes on machine learning. Describe the many basics and standards of machine learning topics. Very well written (almost like a book).
Machine Learning at UBC 2012 (Video) - 2012 Machine Learning Course from the University of British Columbia.
Machine Learning, Probability and Graphical Models (Sam Roweis) (video) - Great explanation of graphical models by the legendary Sam Roweis.
Ranking of ML online courses (list of resources) - A fairly complete overview of the top online machine learning courses.
Stanford's Machine Learning Course (video) - An introduction to the machine learning course.
Sunday classics(list of resources) - A collection of classic works on all topics in machine learning, cognitive science, statistics, information theory, neurobiology, artificial intelligence, signal processing, operations research, econometrics, etc.
WEKA: a workbench for machine learning (online resources) - A large set of free software tools for familiarizing with data, data visualization, classification, regression, choice of characteristics and the basis of data science. I constantly use these resources to teach others to see patterns in the data and appreciate how much the system can see and use these and more complex patterns.
David MacKay, all videolectures (video) - The name of David McKay is well known in this field, especially in the field of statistics and probabilistic machine learning.
Deep learning
Andrej Karpathy blog / hacker guide (blog entry) - A very affordable introduction to neural networks. Also on his blog, you can find practical tips that are applicable to life.
An overview of gradient descent optimization algorithms (blog post) - An exhaustive post exploring the main options for gradient descent used to optimize neural networks
Chris Olah blog (blog) - Chris Ola's approach can be called very educational for exploring key concepts (such as understanding concepts and elements) in machine learning at a deep level. Chris is passionate about education and writes great posts.
Crash Course AI (video) - Helpful well-prepared introductory series. Probably the best for schoolchildren and beginners.
CS231: Convolutional Neural Networks for Visual Recognition (Stanford) (video) - Great notes on the link: cs231n.github.io A nice continuation of Andrew Eun's course, which plunges us deeper into convolutional neural networks (this was briefly mentioned at the end of the previous course) and represents more advanced concepts, such as generative models, deep reinforcement.
CS231n: Convolutional Neural Networks for Visual Recognition (Stanford's legendary CNN lectures) (video) - An excellent overview of both classic and early work on convolutional neural networks that form the basis for much of the work with visual data
Deep Learning at Oxford 2005 (video ) - 2015 Oxford Course in Deep Learning.
Deep Learning Book - An extensive introduction to the basics of deep learning by some of the pioneers in the field.
Deep Learning Indaba Practicals (Colabs) - There are tutorials that have been developed and tested in humans over the years to teach deep learning from fundamentals to advanced topics such as building an automatic differentiation framework or learning a generative adversarial network.
Dive into Deep Learning (book) - An excellent format that turns the study of key concepts of machine learning into a fun and interactive lesson.
DL + RL course with UCL(video) - This course covered many topics related to deep learning and reinforcement learning. It consisted of two, mostly separate, paths: one in deep learning and one in reinforcement learning, which could be studied separately.
EEML ( first / second edition ) Lab materials (Colabs) - Lectures and practical exercises on probabilistic modeling and Bayesian learning.
EEML slides from lectures (slides) - Slides for last year's EEML lectures (unfortunately, no entries). They cover a large amount of material from introduction to more complex presentations.
Full Stack Deep Learning(online course) - Deep learning models don't exist in a vacuum. This course covers practical aspects of deep learning, such as an implementation model, infrastructure, debugging, and even preparation for deep learning interviews.
Intro to machine learning talk at Lviv workshop ( one , two ) (lectures) - Introduction to machine learning. It introduces a theory on the basis of which a deep learning mechanism can be built.
Khipu videos and practicals + github (video + slides) - Materials from Khipu - videos and practical exercises for students to complete.
Lilian Weng's blog(blog) - Lillian's blog contains posts on a variety of topics ranging from teaching curriculum, self-control learning, meta-learning, and more. The posts themselves are not too detailed, sometimes they go too deep into specialization, but quite often they are updated with new information that appeared after the release of the original post.
MIT 6.S191 Intro to Deep Learning (video and tutorials) - Introductory course of the Massachusetts Institute of Technology in deep learning and information systems.
Online journal - A peer-reviewed online journal that allows you to create informative visualizations and code, including to facilitate understanding of research papers and increase transparency and reproducibility.
Parallel distributed processing(online book) - A classic for anyone who wants to understand the roots of deep learning back when it was "connectionism".
Practical Deep Learning for Coders (Online Course) - Recommended by friends of other technical specialties (like physics and math) as a great introduction to deep learning.
Stanford's NLP with Deep Learning Course (online course) - Useful for anyone who wants to start learning natural language processing.
Sutton and Barto's Reinforcement Learning (tutorial) - This is the tutorial of all Reinforcement Learning tutorials. It is built from very basic things to advanced topics. Accompanies lectures by David Silver.
Reinforcement learning
Alberta RL 4-course Specialization (online course) - Four sequential reinforcement learning courses ranging from Bandits to Function Approximation (NNs), Gradient Method, and Average Reward.
CS330: Metalearning and Multitask (video) - Provides an overview of recent work in the field of meta-training and multitasking. An inspirational and very useful video for keeping up with current ideas in the field.
David Silver, Introduction to Reinforcement Learning (video) - Picks up ideas from the textbook Sutton and Barto: Why should we think about these issues? How do the ideas we have discussed relate to? etc.
David Silver's RL Course from UCL(video) - Useful for anyone who wants to learn about reinforcement training.
Emma Brunskill RL Course (video) - Reinforcement training video lectures from Emma Brunskill at Stanford.
OpenAI blog (blog) - Affordable presentations of basic and advanced reinforcement learning algorithms.
Reinforcement Learning: an Introduction (2018 edition) (book) - This is the same introductory book on reinforcement learning. Rich is excellent at explaining the fundamental concepts of reinforcement learning, and he also walks the reader all the way to advanced open research problems.
UofA / Amii Coursera RL Specilization by White and White(online course) - Project of the University of Alberta - reinforcement learning research center. Adam White is associated with Deep Mind; A holistic and well-designed series of courses that provides the most important basics of reinforced learning.
Spinning Up in Deep RL (code) - An educational resource created by OpenAI makes learning deep reinforcement easy.
Unsupervised Learning and Generative Models
Ermon's graphical models course at Stanford (compendium) - Covers a large number of probabilistic methods.
How to Use t-SNE Effectively (interactive tutorial) - This is an interactive and in-depth journey to all the pitfalls of using tSNE, which has become one of the most used low-dimensional data attachments.
Mathematicalmonk Youtube videos (video) - Awesome explanation using examples of useful algorithms like EM. A great addition to Bishop's book.
Monte Carlo Gradient Estimation in Machine Learning (publication) - Useful for those doing reinforcement learning or generative modeling.
Reproducing kernel Hilbert spaces in Machine Learning(course materials) - Suitable for those interested in generative modeling and not only.
Variational inference a feview for statisticians by David Blei (publication) - The best explanation of variational methods in the context of generative modeling.
Other
Meta- training Chelsea Finn's Multi-Task and Meta-Learning Course (video) - Video lectures on multitasking and meta-training.
Philosophy of
Goodman (1955). The New Riddle of Induction. (chapter from the book) The philosophical premises of inductive bias and why it is difficult to draw conclusions and introduction. Lex Fridman's
Science
AI podcast (video) - Conversations with diverse and impressive guest speakers.
Stanford Physics lecture series by Leonard Susskind (video) - An excellent resource for learning about many important areas of modern physics, including classical, statistical and quantum mechanics. These lectures do not imply great background knowledge; Leonard can introduce and explain complex ideas in an accessible and fascinating manner.
Computer Science
Mike Bostock interactive visualisations - Mike Bostock's interactive visualizations .
Probability in high dimensions - A comprehensible book about “ideas at the intersection of probability theory, analysis and geometry that arise in a wide range of contemporary problems in various fields.”
Robotics
Strogatz nonlinear dynamics course (video) - Video course on nonlinear dynamics.
Thank you Ale Blankmer for the help with the translation.
Find out the details of how to get a high-profile profession from scratch or Level Up in skills and salary by taking SkillFactory's paid online courses:
- Machine Learning Course (12 weeks)
- Learning Data Science from scratch (12 months)
- (9 )
- «Python -» (9 )