Harvesting of forage crops begins and we are actively mastering forage harvesting equipment.
A forage harvester is a technically more complex and powerful machine. Several vehicles for harvesting (tractors with a trailer, trucks, silage trucks) go together with it. Only experienced machine operators, who have several years of work behind them, are allowed to work on such equipment.
Working on a combine harvester while harvesting fodder corn is similar to driving in a car in dense fog, only instead of fog along the way, there is a tall green wall of plants from which a wild boar, a pole or a person can jump out. Grinding a person (the story is in my last article), combine operators turn gray and can no longer work. In addition, in this "green fog" one must be able not to crash into a nearby silo truck, monitor the loading accuracy of the silage with a trunk up to 7 meters long, from which 50-60 kg of silage fly out per second, and fill the van evenly so that it does not drove half empty here and there.
In fact, one combine harvester works for three, monitors the corn harvesting process (one workplace), drives the machinery (second workplace), loads the silo truck (third workplace). As a result, something suffers. If you do not manage well, you can break expensive equipment (the minimum price of a forage harvester is 16 million rubles, there are models of 50 million each), so the quality of harvesting and loading usually deteriorates.
We automate most of the work, now I will tell you what difficulties we overcome and what we are doing.
There is no visibility
The height of ripe corn is on average 2-2.5 meters, the height of the combine is 2.5 meters, the machine operator is most often at a level slightly higher than a person standing on the ground and sees only plants in front of him, he essentially does not see beyond his nose, and so on Throughout the whole working day, and this is a 12 hour shift, few can withstand such stress on the eyes and keep the pace of assembly, and hogs or a pole can jump out on the way!
There are combine harvesters, for example KRONE, with a telescopic cabin that rises to a height of 3 meters, or Acros RSM-142 with a height of 4 meters, but this is rather an exception to the rule.
Therefore, only experienced combine operators who have worked for 3-4 seasons can work behind such a combine.
Losses during work
Unlike grain harvesters, a forage harvester cannot store the crop in the bunker, but immediately gives it out through the unloading trunk to a vehicle traveling nearby. As I said, the speed of silage ejection can reach 50 kg per second, depending on the model of the combine, the length of the trunk is 5-7 meters. I drove a little sideways from the car or slowed down and lost half a ton of silage in 10 seconds. On average, up to 7-10% of the crop remains in the field. Nobody collects the lost silage any more, the losses on fuel and lubricants will be much greater. The video shows how the silage spills out if you move the trunk a little.
Uniform filling of the van
In a team with one combine harvester up to 7 cars, one car is filled in about 20 minutes. They can make over 10 trips to the silo in a day. It all depends on the length of the delivery shoulder. If the field is near the pit, then the shoulder is short and you can make more trips for unloading, if further, the trips become less and here it becomes very important to correctly fill the van (silo truck) so that it does not carry air. Brought half of the trailer, count it lost on fuel, the operator must direct the trunk as correctly as possible in order to evenly fill the silo truck.
What can we automate in forage harvesters?
As already mentioned above, only experienced combine operators are allowed to work on such combines. Almost all effective machine operators were trained under the USSR, after which the technological secrets of this civilization were lost. Specifically, training has weakened, and the young people who come, naturally, work worse. This is normal for almost all working professions, but specifically here our robot is in the greatest demand, primarily because of this effect.
Second, our robot sees the edge, obstacles on the field, sees other equipment. The combine operator's work is simplified as much as possible, now he can monitor the technical condition of the combine, the calibration of the corn fraction, which is most suitable for this type of livestock.
Also, our robot can completely and evenly, and most importantly, fill the silo truck accurately, this reduces losses on fuel and lubricants and reduces losses during harvesting to zero.
The normal is the loss of 7% of the harvest during harvesting, if they are reduced to zero, then since silage is mainly done not for sale, but for feed for our livestock, we can reduce the sowing for corn by 7% and give it for wheat, which is more profitable. For example, if 1000 hectares are allocated for corn, of which 70 hectares are allocated for wheat, then even according to the minimum estimates, with a harvest of 30 centners per hectare, we will collect 210 tons, which is about 1.5 million rubles of direct profit. At the same time, due to a more complete assembly, the consumption of fuel and lubricants is reduced by reducing the number of trips from the silo pit to the field.
How we train neural networks to analyze corn harvest scenes
Maize is harvested using special attachments for row crops. For this case, a mode of operation is provided in which the combine tries to keep the header in the middle of the row spacing. Having received a segmentation map of the frame (scene) and knowing the position of the header, you can find the so-called vanishing point and calculate the deviation of the header tooth from the required position.
Unlike wheat, where we follow the edge of the harvested field and drive the combine along the edge, here the task of the neural network is to see the row spacing between the plants. The neural network perfectly sees the rows between the corn, unlike a person, the camera is located above the crop, and at the same time it does not get tired.
Images come to us in the format of a video stream, or as separate images. The data is processed and stored in the Cognitive Agro Data Factory. In addition to raw data from the camera, there are also target tagged frames, which, if necessary, can be added to the training dataset.
For a more accurate selection of possible harvesting scenes, it is necessary to correctly assemble the training dataset. In addition to real images obtained from the camera during the harvesting of crops, an approach is used to generate synthetic images using the augmentation procedure based on natural images.
The network receives a 3-channel RGB image as input. Further, in the process of training the neural network, brightness / color distortions in HSV space, local channel distortions in HSL space are applied to the input tensor - the procedure for adding artificial shadows, geometric distortions and adding noise. The selection of augmentation parameters is a non-trivial procedure that requires a detailed analysis of scenes obtained in real conditions.
The trained network is capable of producing segmentation maps that define the inter-row space.
Silage cleaning
In addition to silage, forage harvesters are also used for collecting silage using a pick-up attachment. The harvesting process is similar to corn, the only difference is that the harvester goes through the felling. The swath may not differ in color from the total weight, and the combine operator may often be idling.
For example, the video above shows that the swath is practically indistinguishable from grass and how accurately the combine operator needs to drive the machine along the swath and at the same time constantly observe the process of loading the haylage into the truck. The work is very intense, by the end of the 12-hour shift, the combine operators simply fall off their feet, they can cope with a full shift only with 3-4 seasons of work experience.
If the operator is inexperienced, then idling is a loss on fuels and lubricants. A forage harvester, in contrast to a grain harvester, spends only 10% of fuel and lubricants on the movement of the combine itself, the remaining 90% goes to cutting, chopping, broaching, throwing silage or haylage. Therefore, it is very important to work with the combine fully loaded.
The neural network does an excellent job with the roll, an example of how the network sees rolls.
Here our system can help the combine operator in guiding the combine through the felling, our robot can also completely and evenly, and most importantly, fill the silo truck accurately, this reduces losses on fuel and lubricants and reduces losses during harvesting to zero, the combine operator becomes the operator of the combine, and takes all the work on self-autopilot.
Now we are developing fodder corn, sunflower, haylage. At the end of the season, if you're interested, we'll tell you how our robots coped with the forage harvesters.
PS If your agronomist is not on Habré, and he is interested in it, then you can find contacts here: promo.cognitivepilot.com and discuss in detail for which combine which specific set of equipment is needed, how much it costs, and how you can quickly see and test it.