The Future of Measuring Emotions [Expert Interview]

An interview with Dr. David Garcia, Medical University of Vienna and Complexity Science Hub Vienna, about research into using social media data to better understand human expression of emotions.

As more of our lives take place online – producing huge amounts of data – social science researchers have the chance to develop new tools for understanding human behavior. InfluencerDB has partnered with research institutions around the world to provide data for use in this research field. Dr. David Garcia is the Group Leader at the Medical University of Vienna and Complexity Science Hub Vienna, directing research into how people express emotions online. InfluencerDB provided Dr. Garcia with a dataset of millions of geotagged images from Instagram. InfluencerDB’s Director of Global Communications, Ariel Dekovic, recently sat down with Dr. Garcia to discuss the direction of his inquiry.

You are a computer scientist, but you’re not the average computer scientist. Tell us about how you are approaching your research.

I have studied computer science originally, but now I don’t use that term a lot. I’m more interdisciplinary, working with social sciences and applying complexity science to social behavior. We use the term computational social science to describe the field of modeling human behavior through digital traces – basically, learning about humans through what they do online. It’s a new twist on the social sciences.

My most recent research is on collective emotions, the way that people share emotions with each other and have a mutual emotional state that is more than just the state of the individuals. Because we share it with others, it lasts longer and it has long-term consequences in people’s lives. I try to depart from the idea of emotions as purely individual phenomena and use these data sets to analyze emotions at large-scale. It is commonly held that when we share our emotions with other people, the more intense they are. This is the typical example that psychologists use to differentiate animal emotions from human emotions. Animals express these emotions, but the persistence and reach of these emotions is very different. Humans need something more social than individual.

How does that play out in online expressions of emotion, for example with the recent #metoo movement, where women shared examples of sexual harassment and abuse online?

There we have instances of women openly disclosing a very negative emotional experience and usually, this brings similar disclosures by others. This might fit one of the theories of socialization of emotions. This triggers the expression of emotions, and this creates bonding and a sharing of beliefs and purposes.

A post shared by TIME (@time) on

For individuals disclosing examples in #metoo, they don’t feel better immediately. They actually feel worse, because they relive these negative experiences. But what we observe in the long-run is that this actually helps, because it allows the victims to connect with others, and, in the long-run, it has a more indirect, but very reliable positive effect.

So is collective emotion sharing ultimately a positive thing?

It’s definitely not as simple as saying it’s good or it’s bad. It definitely can have good consequences and bad consequences. Shared emotions can be used for very negative purposes. That’s exactly what terrorists are trying to do when they attack – to create massive fear – or what Nazis were doing with political propaganda – to trigger collective emotions. Like any social phenomenon, it can have large-scale negative consequences.

How are you using the data from InfluencerDB in your research?

This data allows us to measure emotion in a different way. Before this, when we were working with social media, we were almost always processing text. We didn’t have image data. But now we can do things like evaluate emotion through facial expressions or color profiles. The research question is still relatively open, but it is the method and nature of the data that is very novel in this case.

I want to first develop some new measurement techniques, and then validate them. If the validations point to a signal, then we can apply them to see how events unfold over social media. For instance, after the terrorist attacks in Paris in November 2015, we analyzed a large number of tweets that followed the event. They were not related to the attack, just all tweets by anyone on any topic. We saw that, of course, there was a spike of negative emotions, but there was also a drop of positive emotions. It’s clear that people are not able to express positivity in other situations after such an event.

We can investigate how this expresses itself in images. If we see fewer smiles on pictures, how long does this last? What are the long-term effects, and so on? So we can use this to validate the methods, and then actually begin to use them.

There has been little research into what constitutes ‘well-being indicators’ in images – color, smiles etc., – and I’ll use these to validate the research and then look at some of these other factors, such as the number of people in a photograph. I’m really open though – we don’t need to inherit what has been designed in an experiment with a small set of pictures. We can be a bit more free and try what works.

So you are trying to develop a new methodology for understanding and tracking human emotional well-being?

We are trying to develop a method that tries to capture the success of surveys to a certain extent. We are trying to overcome the limits that surveys have. For instance, surveys cannot be done too fast or too often, because they are very expensive and they take time for people to receive the questionnaire and answer it. Surveys also have pure methodological limitations.

For example, the way you ask how satisfied a person is with his/her life affects the answers, or answers also differ depending on what day you ask the questions. So what we are trying to do is not to replace surveys, but to have an alternative measurement to surveys that will have its strengths and some limitations.

The strength with this type of research is that it’s faster and cheaper and can be done at scale. But it also has limitations. We have self-selection of who is on the social media platform, as well as self-filtering.

I think the key is to combine the approaches and not simply take one instead of the other. There are values in this methodology, just as there are limitations, and if we can take the sum of the methodologies, this is the strongest approach.

Will the work have a specific application for other fields?

In the short-term, I can only see the scientific outputs that we will create. But in the medium-term, I assume that policy makers and governments could recognize if something is going wrong. Pictures on Instagram could help guide policy makers better than the tools they have right now.

I’m looking at large aggregates, but the private sector, on the other hand, calls for personalization. Perhaps you can detect where something is needed the most. For instance, if you are going to advertise in the health sector, this could be used for this.

Do you think that authenticity is something that can be measured or validated?

That’s an interesting question, but I don’t have an answer for that yet. Perhaps we can help with that in one way. If we fail a lot, then the signal may not to be found because people aren’t being authentic, and it’s some kind of display rather than a reflection of actual emotion.

In some ways, what you’re asking is if you can measuring honesty, that is: Is what you are doing and feeling actually being represented in your images? That kind of question can be addressed through matching with individual surveys, but we’re not going to do it right now because it’s a very complicated question.

A post shared by Sophie Gray (@wayofgray) on

Part of the research is the observation over a long period of time and the observation for variation during catastrophic events to see if there is a correlation in how people express themselves. However, this is a question that likely can only be addressed and not answered.

Where will the biggest advances in research come in the next ten years as a result of having social media data?

What I’m trying to create is better measurements of population level statistics. The biggest improvements are hard to predict. What I see is that we’re becoming a bit more mature with expectations about this data. There has been a lot of excitement about big data, and expectations have been a bit inflated without substantial contributions. Perhaps the good thing is that we’re getting down in the curve of the hype, and we’re starting to get where things are actually solid and can make a change. There may be improvements that are not as flashy but can have an impact on society. My personal bet is on the expression of personal well-being.

Why are you excited about this partnership with InfluencerDB?

Before Facebook acquired Instagram, it was possible to retrieve a lot of data for research. There were a few papers that used it. But since the acquisition, it’s extremely complicated to get sufficient data to answer research questions.

The API is much more oriented towards developers. This kind of design decision is making it nearly impossible for research. It’s not that they are trying to avoid it, it’s just that they don’t seem to care – it’s not in their business model.

But what’s really nice is that you guys are getting all the data, and you are interested in research. You are overcoming the short-sightedness that Instagram has and this is precisely what is making me excited again about Instagram as a venue for research.

InfluencerDB