Do-it-yourself neural network from scratch. Part 1. Theory

Hello. My name is Andrey, I am a frontend developer and I want to talk to you about such a topic as neural networks. The fact is that ML technologies are penetrating deeper into our lives, and a lot has been said and written about neural networks, but when I wanted to understand this issue, I realized that there are many guides on the Internet about how to create a neural network and they look like in the following way:





  1. Take Tensorflow





  2. We create a neural network





More details are scattered in chunks all over the internet. Therefore, I tried to put it together and present it in this article. I'll make a reservation right away that I am not a specialist in ML or biology, so in some places I may be inaccurate. In this case, I will be glad to receive your comments.





While I was writing this article, I realized that I was getting a rather voluminous longread, so I decided to split it into several parts. In the first part we will talk about theory, in the second we will write our own neural network from scratch without using any libraries, in the third we will try to apply it in practice.





Since this is my first post, they will appear as the moderation progresses, after which I will add links to all parts. So let's get started.





Do-it-yourself neural network from scratch. Part 2. Implementation





Do-it-yourself neural network from scratch. Part 3. Sad Or Happy?





. - . .





, . . , , , , , , , . 





, , , . , .





. , , , , . - - - .





: , - .





, . . .





, , , , , , .





- . , , , .





IT

, , ? 





, , , , . , . 





, , - , .





, , . , , .





, .





, , . . , .





, , , . . , . , . .





- , . 





, , . , , , , . , , . , . .





.

, , . . , 0 1 0,5 0. . :





f (x) = 1 / (1 + e ^ -x)

0 1. , , > 0.5, , - .





, . , , ( ), , , . , -0,5 0,5. 





 





1 * 0,43 + 0 * 0,18 + 1 * -0,21 = 0,22





, , .





sigmoid(0,22) = 1 / (1 + e^-0,22) = 0,55





0,60.





, , 0,60, .





, . , , . , .





. , .





error = 0.60 - 0 = 0.60





, . , .





sigmoid (x) dx = sigmoid (x) (1 - sigmoid (x))





delta = 0.60 * (1 - 0.60) = 0.24









weight = weight - output * delta * learning rate





weight - , output - , delta - , learning rate - , , . - , . learning rate 0,3. :





w = 0,22 - 0,55 * 0,24 * 0,3 = 0,18





:





w = 0.47 - 0.60 * 0.24 * 0.3 = 0.43





, , , . , . . :





error = 0.18 * 0.24 = 0.04





, , , , . .





, , . , .








All Articles