Social Media: Can Sentiment Analysis help monitor your social media feed?

Something that we as entrepreneurs have realized is that social media can be a huge time sink. However, in modern times it is not something that you can ignore. You want to know what your clients think and say about your business and stay on top of it. As your organization grows, the time that you have left for social media decreases. That does not mean that the importance of social media decreases. On the contrary, it is becoming ever more vital to your business. What role does sentiment analysis play in this?

Why does sentiment matter?

An important aspect that you, as a business owner, want to know is whether people are happy with your company, or whether they are not happy at all. In other words, you want to know the sentiment of every post that involves your company.

Being able to measure the sentiment of your social media channels can help you give insight into the effectiveness of your marketing campaign. Alternatively, it can measure the outcome of a new strategy for handling customer support cases. Another use case could be prioritizing replies to messages with negative sentiments. Replying quickly to negative cases is very important and can help avert significant damage to your brand. Especially if you know how to react!

The suggestions above are only the tip of the iceberg and sentiment analysis is applied in many other use cases. Now ask yourself this question: How can knowing the sentiments of our communication assist you in your daily tasks? If that triggered an idea, don’t hesitate to reach out and spar with one of our experts!

The role of AI in sentiment analysis for social media

Now that the importance of sentiment analysis is clear, we can have a look at the role of AI. How does it work, and what can we expect from it?

It is said that Artificial Intelligence (AI) does not have emotions. And that is true at the time of writing this post. However, that does not mean that AI cannot detect emotions. In fact, AI is becoming better at recognizing emotion in textual data as well as from facial expressions and audio every year. As a result, it is incorporated in many applications around the world.

How does a computer detect sentiment?

Before we continue we have to clarify that the below approach is a simplified depiction and some technical terms and steps are intentionally left out in order to keep things concrete and to the point.

Let’s first ask ourselves the question, how do we asses sentiment as humans? Let’s have a look at the following sentence:

I really really LOOOVVVEEE your products!!!!

There are a few indicators that tell us that this is a very strong sentiment. The repetition of the word ‘really’ shows us that the next word is important when it comes to defining the sentiment of the text. In this case that would be the word love. Love is in general associated with a positive sentiment. Therefore, ‘really really love’ indicates to us that there is a strong positive sentiment. Next to the repetition of ‘really’, the word ‘love’ is also written with uppercase characters and has several of its characters repeated more than twice. Lastly, we can observe that there are multiple exclamation marks at the end. This is another sign that this sentence contains a strong sentiment.
Using these properties as guidelines for determining the sentiment together with a dictionary of positive and negative words already gives a good workflow for a computer to identify the sentiment of a sentence.

Let’s summarize the rules that we have established:

• Word repetition strengthens the sentiment.
• Character repetition strengthens sentiment.
• Words in all or mostly uppercase letters have a stronger sentiment.
• Based on a dictionary we determine whether words are positive, negative or neutral.
• Certain punctuation, such as ! and ? may indicate stronger sentiment.
• Repetition of punctuation may indicate stronger sentiment.

Another rule that we need to consider is negations. Let’s look at another sample sentence:

I am not very happy

If we apply our previously determined rule set we come to the following conclusion:

Summing up the effects we conclude that the sentence ‘I am not very happy’ is substantially positive. Of course, we instantly see that this conclusion is not correct and that the sentiment is mostly negative instead.

We can append a rule where we state that words like ‘not’ and ‘no’ negate the next word with a positive or negative sentiment value:

The new conclusion is that the sentiment is substantially negative. Lastly, there is one more case that we want to address and that is the difference between the following sentences:

I am not very happy

and

I am very not happy

The latter sentence has a stronger negative sentiment compared to the first sentence. In order to incorporate this effect in our sentiment rules, we need to adapt the negation-word rule (‘not’, ‘no’ etc.) to decrease the effect of words that amplify sentiment.

Once this last rule is in place we have a simple but relatively effective approach to determine the sentiment of a sentence. We can now go to a programmer and tell him to make our sentiment analysis program!

Can we do better?

The rule-based approach is already able to determine the sentiment of basic sentences. Sadly, in the real world, most sentences are not as nicely structured as in our examples. They may also contain slang and various typing mistakes. This is especially the case on social media! As a result, it is virtually impossible to come up with a set of rules that covers every possible exception.

The most modern approach is the use of an Artificial Neural Network. This is a technique where brain cells are simulated to teach a computer to perform tasks such as predicting values and classifying information. A huge benefit of these neural networks is that they do not adhere to sets of rules but rather learn patterns.

By showing it many examples of sentences and telling the model what their sentiment value is it learns to determine the sentiment of texts. This approach even works if it has never seen the specific sentence before. In order to get the best accuracy, it is best to show it data from the social media platforms that you intend to use the model on or at least present it in the same format.

Once it has seen enough data it can correctly determine the sentiment for virtually any text that it is presented in your social media feed. And the best thing is that we did not even have to come up with a set of complex rules ourselves!

The limitations

No system is perfect and thus sentiment analysis also has some fields where it does not perform as good as we were hoping. By being aware of the pitfalls we can anticipate the unwanted side effects of applying these techniques to our use cases.

One of these difficulties is sarcasm. Even for humans detecting sarcasm in texts can sometimes be very challenging. Especially on social media sarcasm is not unheard of. Luckily, sarcasm detection is still an active research field and improvements are invented every year.

Another problem is that language develops over time. Think about it, many words that were frequently used in the 1990s are out of fashion today. Even the meaning, and therefore its sentiment value, can change drastically! In order to avoid a degrading performance of these models, we need to keep updating the models over time.

And lastly, it is in a human’s nature to make mistakes. In this case, we are talking about spelling and typo mistakes. Depending on the techniques that are used, spelling mistakes may offset some systems to inaccurately determine the sentiment.

Text Mining Series

This post is part of our text mining series. This series offers insights on how various text mining techniques can be applied to our businesses or everyday activities. By presenting a concrete case and a simplified text mining approach we help you grasp the mysteries that lie behind the word Artificial Intelligence. All posts in this series can be found in the text mining series category