"We learned about your tracking system and want to see it in action, right now, as we are returning to St. Petersburg in the evening!"Although we were in the office, preventive measures did not allow anyone except employees to enter the building, so we continued the telephone conversation and worked out an action plan taking into account the "quarantine" measures.
The customers on the other end of the line were the guys from Black Anchor Productions Ltd. and Parallax Digital. They once again described the task that was supposed to be solved using our tracking (motion capture) system, indicated the desired characteristics of the system, and we agreed on the first series of online tests.
Without online this year, nowhere, and therefore, before conducting tests on their site in St. Petersburg, it was important for the customer to understand the capabilities of the system as a whole and determine the possibility of integrating it into their processes. Therefore, together we have defined the conditions in which it is necessary to test the system and methods of visualization and presentation of test results.
Home tests
After that, with a charge of cheerfulness, we went to prepare the system and conduct, in fact, the tests themselves, which we managed to miss during the quarantine and "removal". The task was to show the work of the system in statics with the presence of various noises that were available to us at that time and without them, and the same tests in dynamics. At the same time, at the output, our system had to give out 3D coordinates (due to radio modules) and rotation angles (due to inertial sensors).
They sent all the results - now, as they say, "the ball is on their side." After several weeks of feedback and clarification on our part of the required characteristics and working conditions, and on their part - of the current and potential capabilities and modes of operation of our system (of course, it was not without clarifying tests with the arrival of new information), we received an invitation to come to offline tests to the cultural capital of our Motherland - St. Petersburg, he is Peter, he is St. Petersburg, and the old people will say - Leningrad. Then their general director Kirill Andreevich called us and said:
"Come tomorrow to St. Petersburg to our site for a week, we will help you with accommodation"It's hard to refuse such a good offer, so we put together a system, took tickets for the night train and rushed to St. Petersburg! As a travel lover, I was doubly glad to leave somewhere more than 100 km from Moscow after more than six months of quarantine and isolation, and even to such a beautiful city!
Reconnaissance in force
We arrived at the Moskovsky railway station, where we were met and settled in an apartment near the customer's site by the charming Anna Karelkina, the operational manager of Black Anchor Productions & Parallax Digital.
We settled in our new home, some of our team members even managed to get some sleep and arrived at the site by the appointed time. Then everything is like a fog ...)
The site is located in the historical center of the Petrograd district, in the television center of JSC "Teleradiocompany Petersburg" - a full-fledged shooting pavilion, in which the real shooting of the program took place at the moment we first went there.
A huge pavilion 17 * 30 meters with high ceilings - 10 meters, which we miss so much at our training ground. I think many thought about when watching movies and TV shows“How is it all filmed? How is it going? " ... And so we got into the body - behind the scenes and saw with our own eyes how television and film magic is carried out. Therefore, if you ask me about some details of the negotiations on that first day, I can hardly remember anything, as I watched the filming process with curiosity.
By the way about the filming process. He was not quite ordinary because we were on the site not just with the magicians, but the magicians of the first level! (well, or what gradation they have there). This is a new breakthrough technology in the field of filmmaking that allows you to reduce not only the production time, but also its budget.
The idea of combined shooting appeared quite a long time ago, and if we talk about it in its classical sense, referring to already tested and proven methods, then combined shooting really significantly expands the visual capabilities of cinematography, and allows you to reduce production costs and facilitate the work of the film crew. It is often used when direct filming of a scene is too expensive compared to combined. In this way, for example, you can avoid the cost of shooting acting scenes on location on an expedition, especially in hard-to-reach places.
The qualitative advantage of the new technology on our market, which is now being developed by the Black Anchor Productions & Parallax Digital team, is the Parallax effect - a change in the apparent position of an object relative to a distant background, depending on the position of the observer. On the use of the physical and optical properties of this "cosmic" effect - in the literal sense - after all, this term came to us from astronomy, where it is used to measure the distance to distant objects (in particular in parsecs), made a bet by Jon Favreau and Disney + and created with her help the series "Star Wars: The Mandalorian", which became a real breakthrough in the film industry. As the leaders admitted, this was one of the things that inspired them to start working in this direction. It was then that it became clear that the use of breakthrough and newest technologies is the real credo of the Black Anchor team, and how could it be otherwise - for this they have all the necessary specialists, whose staff is constantly expanding, and works not only on the set, but also in other cities of Russia and abroad.
For them, researching new things and finding the practical application of their own software and hardware developments in the video, event and film industry is one of the priority tasks, and the spirit of the pioneers is a lifestyle.
At the moment, Parallax Digital is one of the very few creative and scientific associations that I know about, making such a serious bet on creating their own software, I think this is the approach that brings our teams closer together.
In the process of observing the filming process, the advantages of this type of content production immediately became clear:
- , , Epic Games — Unreal Engine, , , . realtime LED – , 4000 lm, 16K.
- vfx , osc dmx AR CGI , .
- – , – , , . – , , , .
- , – , «».
Camera tracking
In order for the virtual scene on the screen to change its angle, following the change in the position of the lens in the tracking area, a so-called Genlock is needed - this is a method in which the video output of one source (or a specific signal) is used to synchronize other image sources together. This helps to ensure that the signals coincide in time at the merging or switching point. To do this, it is necessary to know the data on the movement of the camera and send it in real time to the server for synchronization with the virtual scene. Here the main characters are tracking systems. I mentioned the peculiarities of the working conditions of optical and inertial systems in my last article "Castle to drink in" extreme "conditions or how we took part in the show" DOZOR ", however, this is a slightly different task than a one-time mockup of actors - the system must track camera movements throughout the filming process and under almost any shooting conditions. The ideal tracking system for such an occasion - versatile, reliable, accurate and bulletproof!
Specialists from Black Anchor prod. tested all tracking solutions available on the market, however, we did not find a universal and fully suitable for the largest number of scenarios and conditions, so at the moment they use different systems, choosing the right one based on the conditions in which it should work not only stably, but also with a submillimeter accuracy. This means that there is a great opportunity to be the first to make the most versatile and suitable solution for camera tracking tasks in filmmaking!
With a high degree of probability, such a task is feasible for a hybrid tracking system (combining 2 or more systems), since in this case it is possible to emphasize all the advantages and minimize all the disadvantages that the systems have separately.
It is such a hybrid solution that our SCM Tracking team is developing! With the help of our software, we improve and combine the data sent by radio inertial modules. Thus, our software fights against the systematic error of the inertial system and the presence of the noise component of ultra-wideband radio modules (UWB). To do this, our arsenal includes non-linear algorithms for complex filtering based on our own motion models. The radio system implements the Time Difference of Arrival (TDoA) algorithm. Its essence is as follows. Some of the modules, here called anchors, are stationary along the perimeter of the zone with known coordinates, synchronize with each other and emit a periodic signal. On people, their body parts or other objects, the position and orientation of which must be tracked, modules are attached, called markers,which work exclusively to receive these signals. Because the moments of emission of signals to the marker are unknown, it measures the differences in the moments of time of their reception. This approach allows providing an unlimited number of markers, and a long range of their work - a large working area.
Theory, of course, is good, but what is in practice?
So, back to the site!
We met the main members of the Black Anchor prod team. and determined the test plan for the first day, taking into account the filming process. We were given a place on the site where we can position ourselves, our PCs and system - a full-fledged workplace!
On the first day, we were able to demonstrate the operation of the system in a small area with the output of the results on the screen of our laptop, and also to check if there was a mutual influence of our system with the equipment on the site - the equipment did not significantly affect the operation of our system, as well as our system on the equipment ... The whole secret lies in UWB radio modules, which operate at a dedicated frequency in the microwave range. By the way, already now the UWB module is included in mobile devices of the new generation, which opens up great opportunities for data transmission, etc., but we already know and are able to work with UWB).
In the next couple of days, we managed to launch the system on a larger area in front of the screen in a quieter environment on the site (the volume of filming and the occupancy of the site was less than on the first day) and, together with the customer's team, synchronize with the virtual scene and evaluate the smoothness and accuracy of the camera angle. with moving the marker. To do this, we placed 4 tripods in the working area in front of the screen and placed 8 anchor modules on them, as shown in the photo.
The modules were powered by ordinary external 2300 mAh batteries, which was enough for a full working day. The marker was attached either to the camera or to a tripod when the camera was in use.
During the tests, we saw the whole workflow with our own eyes, discussed and looked at the current decisions of the guys - the most reconnaissance in force and "caste maidens in the field", what more could you want?
And this is just the beginning! So…
The development team shone us in their future plans: in the future, the priority will shift towards the creation of our own hardware and software systems, which will allow the system to be deployed on any site, and the complex will be completely mobile and will allow it to work without loss of quality and accuracy of tracking systems. Anna, seeing us off, said that in a broader sense, the further vector of the company's development will be determined depending on the needs of the market and the film industry as a whole - and hinted,that they intend to show their progress in this direction, including on the example of their own short film - work on its production has already been completed - the most extreme stage remains - post-production - but as we already understood - it will not take much time - the technology allows you to see almost the final "Picture" is already in the process of its shooting, which is perhaps another important nice and cool feature.
Special thanks to the Black Anchor prod team. for a really warm welcome (including a cultural program), the opportunity to see the "client's pain" and understand what parameters and characteristics of the system need to be modified for a camera tracking case for virtual filmmaking tasks.
As a result, we determined our next steps: we finalize certain parameters and return to the St. Petersburg site to conquer it!
To be continued ...