But senior developers did it last year.
You may remember that we talked about how you can greatly improve the performance of a conventional agricultural harvester if you use neural networks to recognize crops and obstacles and a robot for autopilot. All this (except for the Nvidia processors and some more hardware) is our development. And the joy is that in some southern regions of the country the harvesting season is over, and our combines performed better than expected. Glory to the robots!
This year we delivered several hundred blocks from a powerful graphics core (for neural networks), cameras, hydraulic pumps or CAN modules for steering. If last year the agricultural pilots were in trial operation, now we are talking about serial models. And they did it.
Moreover, they did better than we expected. In addition, not all features were included in the release. In fact, the core remained in the release, but this alone allowed us to get a very noticeable economic effect.
Of course, there were some surprises. But let's be more specific, with numbers and examples.
You can see the 2 megapixel camera from above. NVIDIA TX2 in a special casing and with a huge heatsink is mounted below in the sub-cabine space. The screen is in the cockpit.
What is it about
An agricultural harvester is similar in complexity to a church organ. When there is a combine operator and an assistant in the cab, one steers (holds the edge), and the second controls the reel, winds, drums and generally monitors the collection. The third person at this time can do the shipment on the move to the truck traveling nearby. The fourth is watching the obstacles. In the era of the USSR, there were two in the cockpit, then there was only one. As a result, he either steers or collects grain normally. Standing still, it does not work out normally to collect grain, so he steers. Our first post is about how everything is cleverly twisted there and why combines regularly grind people, crash into tractors and power lines running across the field .
The second feature is that each of the roles, even if you perform it without stopping, is very monotonous and requires constant vigilance. It's like looking at the track 10 hours a day, provided that you need to catch literally two moments a day when you need a quick reaction.
The third feature is that combine harvesters often prefer to harvest faster with less efficiency (since payment goes for the tonnes shipped), rather than getting the maximum grain per hectare.
The series includes features of edge retention (the combine itself monitors how it drives and steers itself) and collision avoidance (the combine carefully looks around and predicts the movement of everything it sees - from people to tractors). Here, the developments of the unmanned tram after driving around VDNKh were very useful. The field is much calmer with obstacles. About video analytics is here...
A separate unit is engaged in training neural networks (photographing situations and marking up data) in order to determine where which crop is, what the fallen wheat looks like, and so on. Since there are no training samples, we go to the fields and shoot ourselves. This is important because the same crop visually differs from variety to variety and from climate to climate.
Another division is engaged in the development of hardware. We have a radar of our own design for diesel locomotives and trams, but the harvester only has cameras, because they need to be equipped as cheaply as possible. A complex story is the analysis of control protocols (sometimes lost together with the manufacturer, and then you need to reverse it) or the installation of hydraulics to mix our signal into the steering. Computing modules on each combine are autonomous.
What happened this year
It was not engineers and specially trained testers who went into the fields, but ordinary hard workers, for whom all this was developed. They were harvesting a real crop. And yes, they tested our system for strength, like in that joke about the chainsaw.
Most of all, we were afraid that it was the end users (combiners) who would interfere with the implementation, because they would feel a threat to their work. But everything turned out well. They understand the limitations of the autopilot, understand what is needed in the cockpit, understand how it unloads them and what exactly they can do better. Their production increases, which means that the earnings for the harvesting increase. And significantly: by about 10-15%. They want to work with our robot in pairs. On the same farm, they fought for cars with him.
The person is completely unloaded. He turns on the system, releases the steering wheel, at best controls the operation of the car, at worst - sits on the phone. Harvesting weeks for combine operators are work-sleep-work-sleep-work. In principle, there is no strength for anything, because in a month you need to earn for six months. Our pilots began to tell them that they still had strength for homework. Cars began to look better, because after work I wanted not to fall and fall asleep, but it was possible to do maintenance. Those who deliberately chose a longer working day said that it was easy to work two hours more. They would have worked harder, but not at night: dew.
Here are a couple of pictures that neighboring machine operators saw from their cars and then climbed to see how everything worked:
“Sits there, drinking tea, bastard! *** [why] is a machine operator needed now? " Then they looked that the robot still can not do everything, and realized that it was just like a new combine with a couple of features. And calmed down.
There are numbers:
- The productivity of the shift in time has increased - the harvester does not get tired. This might seem like a 10-15% numerical advantage, but it's much more interesting. This is because it gives you three extra days of cleaning. This means that if the weather is bad (torrential rains, in which the grain sprouts or crumbles), the crop will not be lost, but will be harvested entirely with a much greater probability.
- . , . , - , . 100 % . .
- . , - . , — , , .
- , . , . , ( 1,6 «» ).
- : , ( ). , .
It turns out that the machine operators and managers unanimously say that the work has become easier. Those who did not have grain handlers thought to buy them in addition. Different farms have different daily norms, usually 20-25 hectares. We saw that they calmly bet 30, and people do not exhaust themselves. For some, this turned out to be an opportunity to reduce the harvester fleet for the next year: there would be no need to purchase two or three machines. I know it sounds very strange, but two farms (out of a hundred) said they would do just that.
The harvesting began with what German Gref tried in the Peschanokop agrarian group (on a large serial batch) and said that he mastered it in three minutes. We are proud of this video. If the president of the bank has coped, then the hard workers in the fields will do it for sure.
Well, if suddenly someone frees up several hundred officials in the region, then you can quickly retrain them into equipment operators. Think about it. Rusagro signed a contract for 240 vehicles. Many farms have ordered additional kits for the whole park next year.
Not without surprises
By the way, how the men quickly mastered the technique. The farm took four sets to test, we arrived, installed them on the machines, and did the commissioning. We haven't completely calibrated the odometry, because it needs daylight. We decided to do it in the morning. In the morning, the installers come, and the men on these machines with the default settings have already entered the field, everything has been initialized and they are shooting from the cab what the robot is doing. It may even have been an Instagram stream for the rest of the field. On the one hand, of course, I wanted to do the calibration, but on the other hand, it's nice that the men started everything themselves and everything went up to the default settings.
After the first days in many farms, they either began to believe in our robot, as in an omnipotent intellect like the Terminator, or simply tested for strength. One way or another, one farm decided to try cleaning at night. Sometimes people think that a robot should think like a human. We were somewhat disappointed that he was doing worse at night. The fact is that in the combat release there is no night cleaning: this requires additional training and slightly different data processing algorithms. So far, we guarantee normal night work only with sufficiently wide illumination (it is like that on foreign combines three or four years ago almost everywhere), but here our testers went into the field on a domestic combine from 16 years ago with a narrow flickering cone of light in front. Since most of the farms do not practice night cleaning, we postponed this feature for the next year.
The second place where expectations were overestimated was difficult dust conditions. For example, harvesters, when they follow one another, raise clouds of dust. The wind is sometimes such that a gust of dust blows a cloud of dust to the combine, which is behind. Since he is guided not by an expensive radar, but by an ordinary camera, he cannot see what is ahead. Visibility in dust clouds is six meters. At such moments, our system ceases to see: it signals to the operator and releases control. We were even told that the lidar refused. But there is no lidar on the combine. The machine operators swore: "How can he not see?" In the end, this is what somehow calmed them about work. A person remembers and understands: to the combine - 10 meters, we are driving at such and such a speed, the gust will carry in a minute, nothing will happen, if longer - we have to get up. And Pyotr Mikhalych will definitely not get up in front. Probably.The system does not have life experience and has not drunk with Mikhalych for years, so in this case it stops the car and releases control. So, once again, human intelligence defeats the bot.
The release did not include auto rotations. This is the feature that incredibly amazed all combine harvesters, but it turned out to be the most difficult to test: with a huge header width, you need to build a lot of hypotheses about what falls out of sight. Each machine has its own characteristics. Plus, it also requires a complex control system in order to set a route in advance or program it somehow. We are in favor of natural control: you clicked the switch on the robot, you drive the car, you go out onto the field, you start cleaning. The robot says: "Man, let me steer," you click the second switch - and it drives. You need to turn - just do it, the co-pilot releases control, then looks for a new edge. When it finds it, it again asks to return control. Everything is intuitive and simple. As a result, we entrusted people to turn between the aisles.With automation, we are waiting for the end of tests on difficult terrain. Typically, driving is up to five kilometers long, that is, the turns themselves take less than 1% of the combine's operating time.
There is only one camera on the harvester, because the priority is the price. The second does not greatly increase the price, but it greatly increases the load on computations, and a computational unit with 4 Tflops is a large part of the cost of hardware. The camera looks to the left at the header. There are several exotic schemes (rarely used in the Russian Federation), when everything interesting happens not only on the left, but also on the right. In general, we have two main cleaning systems: the paddle and the shuttle. Work with a bend: a piece of the field is cut off, and everything is mowed around it in a tapering rectangle or circle, striving towards the center. Accordingly, we do it counterclockwise. By shuttle: from one side they call in and drive back and forth - like on a printer. Empty runs are formed between penetrations, but you can take a section of any shape. So, for complex-shaped fields there are more optimized schemes,which require looking in both directions. Perhaps, in one of the next releases we will offer switching between cameras.
"Canadian" scheme.
Then - editing. In the middle of summer, in the midst of quarantine, our combines for the first time since last year suddenly saw and noticed everything. More precisely, they probably noticed it back that year, but they began to order kits right before the harvest (this is possibly due to lending to farms with short terms). As a result, we traveled around the country under quarantine conditions, which left a unique imprint on the work of specialists in neural networks. The installers sat somewhere for 14 days, somewhere they walked almost in spacesuits, but they managed to cope with 50 farms (and there are still the same).
Faced with technical difficulties: on some foreign models (and one Russian) the layout of the compartments is extremely dense. The difference is how between looking under the hood of a Zhiguli and looking into a MacBook. Dimensions were sent from the fields, we urgently ordered new brackets or new mounting systems for a specific model.
Due to the pandemic, the equipment supplier was changed. The valve bodies were from a German manufacturer. They told us in March: "Come back in four months." It was panic, because the whole story depended on it. We found a Russian manufacturer, they understood everything, turned out to be easy-going and made us hydraulics. Promptly, but also not without surprises and not without nerves when setting the task, of course. But until this year, we did not believe that this was possible in Russia at all.
What's next
Robots harvested cereals: wheat, barley, oats, rye in the southern regions. There has not yet been serial operation on corn and sunflower (this is later in agronomic terms). We are also interested in rapeseed and soybeans. Rape is Central Russia, while we wait for harvesting there. Soya - Siberia, Altai, southern Siberia, Khabarovsk Territory, this is very soon.
Word of mouth does not stand still. Over the past month and a half, about a dozen very large holdings from the first 50 have come with their custom requests. Some are already purchasing test kits for this harvest. Someone makes our own TK and special wishes for us - we will think in the offseason. The tasks are to tie up the monitoring of the yield (the combine counts the grain in telemetry and sees the coordinates, that is, you can take data on the yield of soil plots up to a meter), monitoring the operation of the combine (sending telemetry to the center). Some farms come only to the figure, for many it is important for crop rotation that critical points in the fields are marked. It is important to understand the annual average yield and value the “cash” each year. Analytics are needed in order to roughly understand the loading of tractors and equipment: buy or reduce.There are many nuances up to ordering fuel and lubricants before the season: these are all unpleasant prepayments. As a major head of a large enterprise said: “We work with the market. We do not control the market. In order to earn more, we can only reduce the cost. If we do not reduce it, they will eat us stupidly. "
The life span of the combine is written 10–12 years (but we often see 2005, this year was even set for 2001). We will retrofit them all. Because while the horse is alive, they ride it. When the repair becomes more expensive than the cost of a new one, they take a new one. It ends, by the way, with the fact that the old combine becomes a donor of spare parts for others of the same kind. Yes, this is a sagging liver and broken legs, but they work for a year or two. Then it all decays.
And another great thing is the feedback that with this economy the combine pays off faster. A car costs, for example, 25 million rubles (depending on the manufacturer and model). The farm thinks: five years - for domestic cars, a foreign car pays off in eight years. Now - about minus a year.
Now we will finish cleaning this year on production models and several more experimental ones, we will bring the economy together and will be published in international economic reviews. Our modules will be installed both in this form - a separate box on the harvesters, and as integrated devices in new harvesters, if everything ends well. We made it. The two and a half years of the team's life seem to change the world a little.
PS If your agronomist is not on Habré, and he is interested, you can find contacts here: promo.cognitivepilot.com and discuss in detail for which combine, what specific set of equipment is needed, how much it costs, and how you can quickly see it - experience.
PS