Rapid advances in machine learning allow a completely new look at the past: Companies such as DeOldify or neural.love use neural networks to clean photos and historical films of scratches, flickering and wobbling, and up to 54 to extrapolate and recolor images per second and 4K resolution. The results are astounding, but historians and media scholars are skeptical, reports Technology Review in its current issue.

The idea of ​​using machine learning to do this To process old pictures and films and to color them automatically is technically obvious. Coloring is, however, much more difficult for an AI than to distinguish between cat and dog pictures, for example. In order to continuously come closer to reality, the AI ​​needs a calculable measure of the quality of the coloring. The simplest way to compare the colored image pixel by pixel with the original color image works rather poorly than right. The catch with this method: “Neural networks have a tendency to cheat,” says Dana Kelley from the start-up DeOldify. They tend to produce “averaged images” as far as possible in terms of color.

Fake or reality? With his software of the same name, which 2018 was published as an open source project, the start-up had its first technical breakthrough that circumvents these difficulties: DeOldify uses a technology that is now mainly associated with deepfake videos: Generative Adversarial Networks (GANs). In the meantime, Antic has licensed a revised version of its software to the genealogy platform MyHeritage – which aims to attract potential new customers by processing historical family photos.

This article is from issue 1 / 2021 of the Technology Review. The booklet is from 17. 12. 2020 available in stores and directly in the heise shop. Highlights from the magazine:

To not only have photos, Instead, transforming moving images from historical films into new-looking videos is even more necessary than true-to-life coloring: the film must be freed from scratches and flickering, extrapolated to a higher resolution, then colored and finally post-processed. In order to automate all of these steps, the Polish start-up neural.love has set up a software pipeline that relies partly on open source solutions and partly on software developed in-house.

No reconstructions The results look amazingly real – but they are completely Not. Therefore neural.love repeatedly emphasizes that the videos from the computer are not reconstructions of the historical material. “We don’t want to replace the old films,” says Elizbeth Peck from neural.love. “But we’re making it more accessible”.

Lorenz Engell, media scientist at the Bauhaus University Weimar, does not want to know anything about the “rhetoric of the closer, of the more tangible”. There is no evidence whatsoever that processed images are perceived by viewers as more authentic, he says. The photo historian Anton Holzer grants the new computer-generated images of the story “a certain fascination”, but from a scientific point of view, he also views the technical processing with due skepticism. “Of course, that slightly distorts the view of the original document,” he says. “Is it our goal to read this historical material without historical breaks, as if what can be seen in the picture only happened yesterday?”

The new picture (11 Pictures) Children deliver vegetables to be sold in a market in Boston. The picture, restored and colored by DeOldify, looks almost idyllic in color …

(Image: DeOldify) (wst)