When you first watch the videos on Denis Shiryaev's channel , they seem wonderful. You can take a walk in New York as it was in 1911, or take a ride on the Wuppertal Cable Car at the turn of the 20th century, or witness the birth of traffic across the Leeds Bridge in 1888.
Shiryaev's YouTube channel is a showcase for his company Neural Love, based in Gdansk, Poland, which uses a combination of neural networks and algorithms to process historical images. Some of the earliest surviving films have been cleaned, repaired, tinted, stabilized, adjusted up to 60fps, and upscaled to vivid 4K resolution.
For viewers, it almost feels like time travel. “This is what our clients and even YouTube commentators have consistently pointed out,” says Elizabeth Peck, one of Shiryaev's colleagues at Neural Love. It gives you more of a sense of the real life of those times. "
But these vivid videos and images did not impress everyone. Digital Upscaling Specialistswho watched their work on YouTube say that in 2020 they make the past interesting for viewers, but for some art historians it creates a lot of problems. Even adding color to black and white photography is hotly controversial.
“The problem with colorization is that it makes people think of these photos as some kind of simple window into the past, which they really are,” says Emily Mark-Fitzgerald , assistant professor at the University of Dublin's School of Art and Cultural History.
Elizabeth Peck says Neural Love is clearly showing customers the huge difference the company sees between "rebuilding" and "improving." They view the removal of scratches, noise, dust and other defects found during processing as a less ethical process of scaling and staining. “We're really bringing the film back to its original state,” she says.
However, not all are of this opinion. Luke McKernan, Lead Curator of News and Moving Images at the British Library, was particularly scathing about Peter Jackson's 2018 World War I documentary They Never Get Older., in which frames from the Western Front were scaled and colored. He argued that creating a more modern look to the footage undermined its value. “This is nonsense,” he wrote. “Colorization does not bring us closer to the past; this widens the gap between the present and the past.
The colors that suddenly spill over the streets of 1910s New York are not from celluloid itself; this information was never recorded there. "
Neural Love uses several different programs to work with its videos, fixing and improving one stage after another. They use open source software DeOldify to colorize their clipswhose developers Jason Antik and Dana Kelly came close to creating a versatile image restoration and coloring tool that would handle the entire process on its own. “However, it was very difficult,” says Kelly. "A lot of training and a lot of failed experiments."
According to Emily Mark-Fitzgerald and other historians, DeOldify and Neural Love can give stunning looks, but they risk distorting the past rather than illuminating it. “Even as a photo historian, I look at these processed images and I think they are pretty exciting,” she says. “But my next thought is always:“ Why do I have such a reaction? And what is the real person who made the restoration of the photographs? What information did he add? And which one did he hide? "
However, DeOldify and Neural Love see their tools as a means of bridging the understanding gap created over a century of technological advancement. Their technology is a means of making jagged and jagged images appear modern. But for historians it's all about the time difference. “Understanding this difference provides insight into the past,” McKernan writes. Without this, there will be no true sympathy, only false feelings. The film, which appears to have been shot last week, belongs only to last week. "
Elizabeth Peck compares Neural Love to an installation at the Salvador Dali Museum in Florida, which manipulates the artist's images, forcing visitors to take selfies: "This is more acceptable for the modern generation, which is used to interacting with media in a different way."
“We see our work as an adaptation of the original, as was the case with the works of Shakespeare (meaning the film “ Romeo and Juliet ” in the interpretation of Baz Luhrmann) or translation of literature into another language,” adds Shiryaev. “Transforming the original has artistic merit, but the original content is still an art form in its own right (and deserves to be perceived as such). Our work is aimed at raising awareness of the originals and not questioning their authenticity or artistic value. "
Antique and Kelly have no illusion that the images processed by DeOldify will be historically accurate. Their doubts are related to the practical aspects of neural network training. Ensuring accuracy in film painting is “literally an impossible challenge,” says Antik. DeOldify uses modern imagery to train its AI, which is a big weakness because it can dress people in snapshots of the past in blue jeans.
Neural Love explains to its clients, “Colorized and anti-aliased images can be historically inaccurate,” says Peck. "This is done by a neural network that makes the most likely estimate based on the vectors present in the movie."
To prevent anyone from taking DeOldify images at face value, users can leave a watermark on any image that they adapt with the software. However, since DeOldify is open source, many developers using it do not bother adding a watermark. “We have no control over what the rest of the world does to him,” says Antik. "All we can do is just try to be kind of opinion leaders."
The original videos still exist, of course, and Shiryaev's YouTube videos are broken down into video processing steps so that no one can mistake them for originals. Kelly compares them to reading written diaries, not illegible scribbles.
For historians, however, there is a gap between the limitations and trade-offs of software, and the assumptions anyone can make when stumbling across images on social media.
On the Internet, they said, these images are "detached" from how and why they were taken, as well as how and why they were changed. “There is something gained in these images, but there is also something that is lost,” says Mark-Fitzgerald. “And I think we need to talk about these two things. Getting people interested is one thing, but you need to critically evaluate what you see and not passively absorb everything that gets into your Twitter feed. "