On Shape And The Computability of Emotions

  Following on from my previous post about Kandinsky's colour theories, I decided to look into some studies that had more psychological proof to them.   Kandinsky's views may be as they are, however they have been proven to have very little scientific basis and were largely based on his personal experiences as a synesthete.  I decided to pursue other studies which had more scientific/psychological basis.  One example that I have found to be very  informative is On Shape and the Computability of Emotions, a (peer-reviewed!) Stanford paper researching the emotional reactions in humans regarding shapes of varying angularity or roundness.  In the paper's abstract, Lu, Suryanarayan et al. (2012) explain how they "investigated how shape features in natural images influence emotions aroused in human beings.  Shapes and their characteristics such as roundness, angularity, simplicity and complexity have been postulated to affect the emotional responses of human beings in the field of visual arts and psychology.  However, no prior research has modelled the dimensionality of our emotions aroused by roundness and angularity."

The research paper specifies that the work is "an attempt to systematically investigate how perceptual shapes contribute to emotions aroused from images through modelling the visual properties of roundness, angularity and simplicity using shapes.  Unlike edges or boundaries, shapes are influenced by the context and the surrounding shapes to influence perception of any individual shape. "  This would be useful to my own are of study because, particularly in the instance of logos, shape can be key in telling the consumer what the brand is about - for example, companies such as IBM and FedEx, which provide technical services, are very square and robust, whereas companies that have a more personable approach like Starbucks or many craft beer brands have circular logos.

  The team of researchers used the International Affective Picture System, which was previously developed at the Center for the Study of Emotion and Attention at University of Florida who curated a set of colour pictures which could be used to measure emotional responses based on valence (emotions from a negative to positive scale)  and arousal (emotionally reacting to stimuli), thus creating a framework for visual studies such as this one (see Fig.5).  The Stanford researchers added a new layer to this by introducing a "dimensional representation"(Lu, Suryanarayan et al, 2012) whereupon the factor of dominance (i.e strength of the emotion and subsequent control over a person) was brought in (see Fig. 6).  "From the perspective of neuroscience studies, it has been demonstrated that the dimensional approach is more consistent with how the brain is organised to process emotions at their most basic level.  Dimensional approaches also allow the separation of images with strong emotional content from images with weak emotional content."  (Lu, Suryanarayan et al, 2012).  The researchers note that this developed framework has been used in a previous study on video content, but that "static images, with less information, are more challenging to interpret."

Fig 1. Visual example of the dimensional approach used

Lu, Suryanarayan et al (2012) studied the elements of "roundness-angularity and simplicity-complexity characteristics" in shapes due to the fact that they have already been proven by psychologists to have an effect on people in test groups.   The main focuses within this specification are line segments ("short straight lines generated by nearby fitting pixels"), angles, (acute or obtuse - an image is more angular if it has more acute angles), continuous lines ("generated by connecting line segments having the same orientation") and curves ("the collection of which are used to measure the roundness of an image").  Using these, the paper proposes to find a relationship between the aforementioned characteristics and an emotional response in a viewer.  Statistical calculations were applied to gauge which visual information had what impact, such as "mean and standard deviation of line segments" to work out the mass of a line form or calculating to what extent a curve fits into an ellipse as a whole.  I won't go into those to much as it gets very maths-based, but essentially calculations of various shapes and angles are used by the researchers to determine what kind of shape criteria was most present in an image of high arousal (one example was a burning building) vs an image of low arousal (a field and sky).

Fig.2 High arousal images

Fig. 3 Images with high arousal are shown to have lower mean value of the length of line segments

One clear finding of the study was that images defined as positive had a higher number of curves in them, supporting the existing theory argument that circles are a less aggressive shape.  "To examine the relationship between curves and positive-negative images, we calculated the average number of curves in terms of circularity and fitness on positive images…and negative images.  Positive images have more curves with 60% -100% fitness to ellipses and a higher curve count. " (Lu, Suryanarayan et al, 2012, )  It can be seen below in Figs 5 and 6 showing measures of valence (see Fig.1 for chart).  Images with high valence are those with high numbers of ellipses and smoother contours.  This is echoed if we look back at Fig.2, which shows evident ellipses and a "flow" of connected contours.

Fig. 4 Low valence

Fig. 5 High valence

Fig. 6 Comparison table

In Fig.7, it is evident that the number of curves in positive images is higher than that of negatives in almost every instance.  This means that, on the scale in Fig. 1, the dominance of emotions that are aroused by images with a higher curve content are stronger.  So a figure with more curves can evoke a reaction of, say, anger, while a negative image with less curves can make a person feel "sad" which has less emotional control.

So the shape features in arousal images had now been defined, so in order to relate this to emotions a secondary experiment was carried out using the IAPS dataset, a collection of images with the pre-define d visual data as calculated in the first stages of research.  The IAPS dataset has 2 "subsets" - Subset A includes images of faces and bodies, which could not be used as these are known to evoke an immediate emotional reaction in all people that is much stronger than that of a different subject matter.  Now I know why perfume and jewellery advertisements use faces to compel people to purchase the product! The remainder of Subset A was used however.  Subset B, another IAPS collection, "includes eight categories namely fear, disgust, anger, sadness, amusement, awe, contentment and excitement. ..Subset B is a commonly used dataset, hence we used it to benchmark our classification accuracy." (Lu, Suryanarayan et al., 2012)

Despite the initial findings, the paper concludes that it did not find definite answers in regard to specific emotions linked to specific shapes.  However it's obvious that the first steps have been taken and some answers have been identified.  "We empirically verified that our proposed shape features indeed captured emotions in the images.  The area of understanding emotion in images is still in it's infancy and modelling emotions using low-level features is the first step toward solving this problem. "  It is also written that "emotions evoked by images cannot be well represented by shapes alone and can definitely be bolstered by other image features including their colour, composition and texture." (Lu, Suryanarayan et al. 2012)  However findings based on elements of shape and how they relate to valance and arousal were identified:
"...we found that angular count, fitness, circularity, and orientation of line segments showed higher correlations with valance, whereas angle count, angle metrics, straightness, length span and orientation of curves had higher correlations with arousal." (Lu, Suryanarayan et al, 2012)  This is summarised in the table below.

Fig 7

This article has been useful research as I now have a reference for designing visual content that could emotionally engage people on a deeper level.  It was also helpful in highlighting the necessity of colour and shape working together to convey and/or arouse emotions, as shape alone cannot do a feeling justice.

-

References

Lu, Suranayaran et al, 2012.  On Shape and the Computability of Emotions [online] Stanford University.  Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Images

Fig. 1 Figure 1: Example images from IAPS (The International Affective Picture System) dataset [15]/Figure 2: Dimensional representation of emotions and the location of categorical emotions in these dimensions (Valence, Arousal, Dominance)  [online image] Available at: http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Fig 2. Figure 5: Perceptual shapes of images with high arousal [online image] Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Fig 3.  Figure 9: Images with high mean value of the length of line segments and their associated orientation histograms [online image] Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Fig 4 Fig 4: Perceptual shapes of images with low valence [online image] Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Fig 5. Figure 3: Perceptual shapes of images with high valance [online image] Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Fig 6.  Table 2 and Table 3 [online image] Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]

Fig. 7 Table 4: Significant features to emotions. [online image] Available at:
http://infolab.stanford.edu/~wangz/project/imsearch/Aesthetics/ACMMM2012/lu.pdf
[Accessed 7th October 2014]