The LEGO constructor is often used to create the case of some mechanism with a "raspberry" engine. Most often, this tandem is used in various robots, drones, all-terrain vehicles. Enthusiast Daniel West went the other way and created a machine with Raspberry Pi and LEGO to automatically sort the parts of this set. Naturally based on artificial intelligence. Without AI, irons will probably not be able to work soon.
Under the cut - a description of the mechanism of the sorter, as well as a few more interesting projects based on "raspberries".
Interestingly, the sorting machine itself is made from 10,000 LEGO blocks! It took ... two years to create it.
This car is a real LEGO Cinderella. She is able to sort any part of the constructor into one of 18 containers at a speed of "one cube in two seconds". What's more, the sorter is able to recognize every LEGO brick ever created, including those that it hasn't yet come across. This versatility distinguishes the system from previously created machines for a similar purpose.
What's under the hood
The machine runs on the following hardware:
- Raspberry Pi 3 Model B +
- Raspberry Pi V2 camera module;
- 9 engines (controlled via a servo multiplexer that communicates with the Raspberry Pi via I 2C );
- 6 LEGO motors (controller-controlled L298N electric motors using digital I / O ports on the Raspberry Pi).
As already mentioned, the sorter is based on artificial intelligence algorithms. Overall, a fairly classic convolutional neural network that is used to classify images. We have already described in the blog on Habré how this helps scientists in their work.
Neural network training
Daniel West trained his neural network using images of 3D models of LEGO bricks. At the same time, the developer faced the problem of lack of data for training the neural network. Real quality images were not enough, and synthetic images did not give correct results. In the end, only a combination of synthetic and real images helped to achieve success. The neural network was able to recognize LEGO bricks with high accuracy, even if it had not previously interacted with them.
By the way, to collect data, the enthusiast left the car to run the parts through the scanner for several days. As a result, I assembled a dataset of approximately 300,000 unmarked LEGO images for AI processing. Daniel spoke in more detail about the work of AI and its training in a separate very visual videoand described the process in the text .
As software, the enthusiast used Blender, an open source software for creating 3D computer graphics, and Tensor Flow, an open source machine learning software library from Google. He was also helped in his work by the community of makers from the LEGO constructor - Rebrickable.
Run, detail, run
The development is a real miniature conveyor machine. The mass of the parts is fed through the caterpillar conveyor into the so-called "input bucket". Here the pieces go one after the other, so that strictly one LEGO block is sent for "identification".
The order is achieved by a vibrating platform that is powered by a LEGO motor. By vibrating, the platform distributes the parts, making sure that they do not lie on top of each other and come for scanning one by one.
The Raspberry Pi camera module captures video of each block, which processes the Raspberry Pi 3 Model B + and sends it wirelessly to a more powerful computer, where it operates a neural network to classify the parts. The data processed by the neural network is sent back to the sorting machine so that it can push the part into one of 18 containers using a series of self-driving gateways.
While the project is more suitable under the heading "Don't try to repeat this at home", because the author has done a great jobfor AI training. But, you see, the thing is interesting.
Raspberry Pi in action
And here are some more recent interesting projects with the participation of "Malinka".
RoboScan Film Scanner
Working with film is becoming more and more difficult due to the ever-widening gap between technologies. A project called RocoScan uses a Raspberry Pi with a digital SLR camera. The device not only scans the image from the old film, but also automates the process of recording images into a separate folder.
Minecraft style server case
This glowing familiar Minecraft block of ore is actually a Minecraft server! Inside is a Raspberry Pi 4 along with a 128GB SSD running the Paper MC SMP. The beauty!
Crust cutter
The device we deserve! If you are a fan of perfect sandwiches, you know that there is no place for toasted and crusty crusts. This machine on "raspberry" will exclude from your life the routine getting rid of sandwich imperfections.
Automated chicken coop
Obviously, this can be used not only for chicken coops. But one enthusiast has automated the opening and closing of chicken coop doors to protect his farm at night. It is based on a Raspberry Pi and several timers that initiate the opening and closing of doors in the morning and evening. Does your chicken coop run on python?
Do you have a favorite project on Malinka? Share in the comments!
