A group of computer scientist at University of MarylandÂ has designed a new algorithm that incorporates artificial neural networks to simultaneously apply a wide range of fixes to corrupted digital images.
The research team from the University of Bern in Switzerland, tested their algorithm by taking high-quality, uncorrupted images, purposely introducing severe degradations, then using the algorithm to repair the damage.
The researchers presented their findings on December 5, 2017, at the 31st Conference on Neural Information Processing Systems in Long Beach, California.
Matthias Zwicker, the Reginald Allan Hahne Endowed E-Nnovate Professor in Computer Science at UMD and senior author of the research presentation said, “Traditionally, there have been tools that address each problem with an image separately. Each of these uses intuitive assumptions of what a good image looks like, but these assumptions have to be hand-coded into the algorithms. Recently, artificial neural networks have been applied to address problems one by one. But our algorithm goes a step further–it can address a wide variety of problems at the same time.”
Matthias Zwicker said, “This is the key element. The algorithm needs to be able to recognize a good image without degradation. But for an image that is already degraded, we can’t know what this would look like, so instead, we first train the algorithm on a database of high-quality images. Then we can give it any image and the algorithm will modify the imperfections.”
Zwicker said that, “Several other research groups are working along the same lines and have designed algorithms that achieve similar results. Many of the research groups noticed that if their algorithms were tasked with only removing noise (or graininess) from an image, the algorithm would automatically address many of the other imperfections as well. But Zwicker’s group proposed a new theoretical explanation for this effect that leads to a very simple and effective algorithm.”
Zwicker explained, “When you have a noisy image, it is randomly shifted or jittered away from a high-quality image in all possible dimensions. Other degradations, such as blurring for example, diverge from the ideal only in a subset of dimensions. Our work revealed how fixing noise will bring all dimensions back in line, allowing us to address several types of other degradations, like blurring, at the same time.”
Zwicker added that, “To recognize high-level features, the algorithm needs context to understand what is in the image. For example, if there is a face in an image, it’s likely that the pixels near the top are probably hair. It’s like assembling a jigsaw puzzle. If you’re only looking at one piece, it’s hard to place that part of the image in context. But once you find where the piece belongs, it’s much easier to recognize what the pixels represent. It’s quite clear that this approach can be pushed much further still.”
Zwicker also said that the new algorithm, while powerful, still has room for improvement. Currently, the algorithm works well for fixing easily recognizable “low-level” structures in images, such as sharp edges.
The researchers hope to push the algorithm to recognize and repair “high-level” features, including complex textures such as hair and water.