Look closely at a face in a painting and you might think you know what emotion the person is experiencing. A smile suggests happiness. A furrowed brow might signal anger. Widened eyes could mean fear or surprise.
But what happens when you ask lots of people the same question? Do they see the same emotion, or something more complicated?
In our recent research, we tested how consistently people interpret faces and their emotions depicted in paintings.
Emotions are central to how we experience and navigate the world around us.
Because facial expressions are an important way emotions are communicated, psychologists have spent decades studying what faces can tell us about emotional states.
One of the main tools for studying emotions has been photographic databases like the classic Pictures of Facial Affect.
In these images, actors pose using standardised combinations of facial movements to represent six basic, widely recognised emotions: happiness, sadness, anger, fear, disgust and surprise.
These passport photo type images are incredibly valuable for studying how people recognise clear, prototypical emotion expressions.
However, expressions do not always fit neatly into one of these six categories.
To understand the more nuanced ways people interpret facial expressions, researchers also need examples that go beyond clear-cut expressions.
However, asking someone to pose with a convincing layered emotional expression for a photograph is not straightforward.
So researchers have found examples from films or everyday life, while also introducing additional emotion labels and other ways of measuring how a facial expression is understood, such as how positive or negative and how strongly that emotion is felt.
Take, for instance, Leonardo da Vinci’s Mona Lisa.
There has been over 500 years of debate over whether she is smiling or not.
Research has found that perceptions of her gaze can change depending on how the painting is viewed, and that her eyes can appear to smile even when viewed on their own. It is a striking example of how much information can be packed into a single painted face.
The face in a painting is the result of deliberate artistic decisions, observation and traditions, while tapping into the ways our faces naturally move when we express emotion.
In our research, we selected 116 faces from European paintings dating from around 1400 to 1900. Before testing, we standardised the images by making sure that all faces were clearly visible and similar in how they were positioned.
We then cropped the faces from the paintings, removing the wider scene and body language so that participants were judging the expressions themselves.
We presented these isolated faces to 152 participants, who viewed them individually and selected which of the six basic emotions best described the expression.
Our findings revealed that agreement exists on a sliding scale. On one end of this spectrum, some faces produced a clear consensus: viewers consistently selected the exact same emotion.
We classified 61 of the 116 faces in this way as showing a classic “basic emotion”. But the other 55 faces produced consistent patterns of responses across two or three emotional categories.
One painting that garnered such a response was Joan of Arc by Jules Bastien-Lepage (1879).
The painting shows Joan in her family home in Domrémy, France at the moment she is told by the saints she must fight against the English in the Hundred Years War.
The full scene, along with our cultural knowledge of who she is and the heavy fate awaiting her, provides a vivid narrative context that helps us decode her psychological state.
The chair beside her spinning wheel is upturned, telling us that she has been suddenly interrupted.
When we look to the trees, we see the faint traces of the apparitions of the saints who have assigned her the terrifying burden of the role she must play in the defence of France.
But when the context was stripped away and participants viewed Joan’s isolated face, they didn’t agree on one single emotion.
People most often saw fear, but many also saw sadness or surprise.
For the participants, Joan’s expression did not convey a single, flat emotion but several plausible interpretations at once.
These complex, yet non-random patterns of response, to Joan’s face and the other 54 paintings, suggest that paintings offer interesting new avenues for the exploration of how people perceive emotion.
Artworks then could offer researchers a wider range of facial expressions to investigate. (The Conversation)