Now scientists from the Bethge Lab in Germany have built an AI system that can learn a painting’s style and apply it to other images.
The results look as though Pablo Picasso and Vincent van Gogh painted their own version of the image.
According to Leon Gatys, PhD student and the lead on the paper published in the open-source journal arxiv, this is the first “artificial neural system that achieves a separation of image content from style.”
Scroll down to see how it works and the beautiful images it created.
The AI could take the style of one image, like the swirls and dots of Vincent van Gogh's most famous painting 'Starry Night' and make another image adopt this style, like a very sophisticated photo filter.
The altered photo of the river looks as though van Gogh was standing at the banks of the Neckar river instead of at his window in Saint Remy de Provence, the original setting for 'Starry Night.' Gatys wrote that the system gives us a mathematical basis for understanding how humans perceive and create art, especially because it mimics biological vision and the human brain.
The AI is made of stacked layers of computing units that imitates the interconnected structure of neurons, called a convolutional neural network (CNN). The scientists had to first 'train' the system to recognise the features of famous paintings. The graph below shows how the AI comprehended the features of 'Starry Night' and changed the photo of the river based on this knowledge.
The AI system is like an assembly line -- each layer is responsible for one thing. It took Pablo Picasso's cubist style from 'Seated, nude female' and broke it down layer by layer.
Using Picasso's characteristic sharp angles and muted colours, the AI transformed the photo of a quiet riverfront into a stark, dystopian painting. This kind of photo transformation is called 'non-photorealistic rendering.'
The AI system's lower layers identify the painting's simple details like dots and strokes, while the upper layers recognise it's more sophisticated features like colour use.
The AI didn't just take colours and place it in corresponding areas on the photo. It could make sense of the original photo's shadows and highlights and actually 'understand' the scene, in a sense.
The AI was also able to learn the elements that make up Edvard Munch's portrait 'The Scream' and apply it to the landscape photo without adding in the painting's main figure.
The buildings are still recognisable, but re-imagined with Munch's style and the painting's colours.
The results aren't always pretty. For example, the abstract style of Wassily Kandinsky's 'Composition VII' ...
... didn't translate well on the landscape photo. It's difficult to find the buildings and the river in the altered image.
The AI's parameters could also be changed to emphasise or understate the effect of the painting's style. This graph shows what the rendered images would look like at different levels of intensity. The top right rendering would be if Kandinsky's painting filter was at a very low setting, while the bottom left image shows the effect at a very high setting.
Business Insider Emails & Alerts
Site highlights each day to your inbox.