Detecting the state of traffic lights at level crossings to fix traffic violations

Recently we were faced with an interesting, seemingly simple, but ambiguous task of detecting the states of railway traffic lights to fix traffic violations. The problem turned out to be that the connection to railway automation is a complex procedure that requires a lot of approvals, and the laying of the corresponding cables requires capital costs. An alternative is to detect the state of a traffic light from a CCTV camera and record the fact of traffic violations according to the data from the camera.



Having previously set a goal and agreed on a plan for testing the software and hardware complex for detecting traffic violations at regulated railway crossings, we chose the Engineering Center of the Oktyabrskaya Railway as an object for testing. A full-scale model of a railway crossing was implemented there, on which a camera and a complex for fixing violations were installed:







The process of debugging and testing video analytics with the traffic light on on the layout



Objectives:

• detecting the state of a traffic light at a crossing in the camera's field of view
• fixation of vehicle passages through the crossing with recognition and recording of numbers at a red traffic light

The composition of the used hardware and software complex:

1. IP camera, 2 MP
2. Industrial PC Outdoor Box Micro, 2 GB RAM, CPU Intel Atom x5, VPU Movidius
3. Switch
4. EDGE software for detection and recognition of state registration plates of vehicles

Traffic light state detection algorithm:

1. Receiving RTSP stream from a camera
2. Cutting a stream into frames
3. Framing the detection area of ​​the status of signal lamps
4. Obtaining the brightness value for each lamp in N-frames / seconds
5. Binarization of brightness values ​​by threshold
6. Checking the operation of lamps in antiphase over the entire analyzed interval of N-frames / seconds
7. Return of traffic light state with a delay from real time in N-frames / seconds

Before binarization, the check if the lamps are in antiphase looks like this:







After binarization:



0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1

1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0 1 1 1 1 0 0 0 0



Demonstration of detector operation:



Detecting the state of a railway traffic light using object video analytics



Problems that we encountered while implementing:

1. You cannot simply detect the color value or the amount of red in the area, because at night the camera turns on the IR illumination and the image becomes black and white.
2. Even when measuring the brightness value, you cannot use data from only one lamp, since blinking, for example, by emergency stop signals of a vehicle or a direction indicator, will fix the reflection in the lamp and a smooth change in brightness up and down. It is clear that it is possible to be tied to the wavelength, but this does not negate the simple possibility of remotely influencing the state of the traffic light from the outside.
3. Sometimes traffic lights work unpredictably and it is necessary to minimize random emissions after binarization, for example, a situation is acceptable when two lamps are lit red at the same time or the brightness values ​​change stepwise with a constant but less attenuation.

Expectation and reality in the work of railway traffic lights



Solution advantages:

1. Lack of capital costs and approvals for the laying of cable routes
2. No need for integration with railway automation
3. Availability of the possibility of quick start-up in the field (installation of cameras, marking of zones, industrial operation)
4. Availability of the possibility of post-processing data on the video archive
5. Stability of operation at different times of the day (when comparing measurements of changes in brightness over a period and color detection)