The faces of the forefathers were recreated using artificial intelligence

The original faces of the forefathers and Jesus Christ, depicted at the beginning of the 16th century by the Russian master Dionysius the Wise in the monastery of the Kirillo-Belozersky Museum-Reserve, were recreated using a neural network by specialists from the Space Research Institute of the Russian Academy of Sciences. The chief specialist of the IKI RAS, Alexander Andreev, told MK how the work was carried out.

The frescoes of the Cathedral of the Nativity of the Virgin Mary at the Ferapontov Monastery of the Museum-Reserve, which is located in the city of Kirillov, Vologda Region, are the only paintings of the great Dionysius that have survived to our time almost in full from the beginning of the 16th century.

– The story began in 2008. We decided to join the restorers who worked on the cathedral to help them restore the faces of Jesus Christ depicted on the dome, Eve and the forefathers depicted on the drum of the dome,” says Alexander Andreev.

According to him, they decided to start with the drum (the round base on which the dome seems to rest) because its frescoes are usually difficult to see from the ground.

Over time, some losses have appeared in the painting, for example, the Pantocrator himself (Jesus Christ in the image of the Judge) has white spots on his face from nails that were once driven in; the angels depicted below him, in the upper part of the dome drum, have their wings cut off.

“The frescoes with the archangels were damaged in the 18th century, when they decided to break through the windows under the dome,” explains the scientist. – Just below are the faces of Adam, Eve, and other forefathers, also partially erased. Our task was to first take high-resolution images of them using multispectral imaging, and then “invite” the neural network to restore the gaps.

Having taken many high-resolution photographs of the frescoes and trained the Stable Diffusion neural network on them, the scientists began virtual restoration. So far, out of 12 faces, artificial intelligence has restored seven.

The simplest task, according to the “teacher” of the neural network, Alexander Andreev, was to remove the white spots that were left by nails on Panktokrator’s face. With their help, something was attached under the dome in ancient times.

Then the face of Noah’s ancestor Methuselah was restored, who, according to the Bible, is the oldest person who lived 969 years. Surprisingly, his face was not too badly damaged by time on the wall of the Ferapontov Monastery – there are only cracks and the paint layer on the beard, cheeks and halo has been slightly lost.

“We had to work with Noah’s great-grandfather Enoch,” says the scientist. – His eyes were almost erased by a branched crack, but the neural network restored everything. Noah, as far as restoring his image goes, I’m proud of him. He had large losses on his face and around and inside the medallion. Everything has been restored, only the brightness of the background still needs to be consulted with restorers.

Regarding Eva, according to an IKI employee, the neural network made a small mistake – in particular, her right eye turned out to be greatly shifted relative to the position that we would expect to see.

“This is my oversight,” says Andreev, “it was necessary to teach AI better.”

By the way, experienced restorers to whom we showed our works, including the face of Eve, Seth, Enos, Ared, said that they are very similar to the work of Dionysius, at least “they don’t hurt the eyes.”

– What did your neural network do that ordinary restorers couldn’t do in reality?

– You see, the situation with restoration in reality is very complicated. Often the most experienced restorers do not undertake this for many reasons. For example, Italians usually cover everything with a protective film, while other schools of restorers preserve walls in a special way. As for the frescoes of the Ferapontov Monastery, the question is also open. But with the help of our work, we will be able to show the guests of the monastery, everyone interested, what the frescoes of the master Dionysius were like before they were damaged.

Next up for scientists is to recreate the faces of Adam and Abel using a neural network.

By the way, in addition to the images of the forefathers in the same monastery, scientists managed to decipher another mystery of antiquity – these are the texts that were inscribed on the scrolls of saints in the altar part of the cathedral. They were irretrievably erased about 200 years ago, and now, with the help of ultraviolet light and computer image processing, specialists have restored them.