At Nexxt, we are always on lookout for better ways to understand our users. While traditional techniques such as using word-cloud to visualize prominent words in a text is quite common, they are very limiting in capturing the nuances in opinion.

Recently, we have been playing with a proof-of-concept for a visualization called ‘EmojiCloud’. The idea is to create word-cloud like visualizations that reflects the moods of the user from their shared open-ended responses. The model can implicitly understand the mood from the text and generate a chart to reflect the overall mood.

In this post, I will walk you through the whole process with a sample use case. At the end, you will be able to generate EmojiCloud for your own text files.

Step 1: Understanding Emotions

Our first challenge is to detect the mood from an open-ended response written by the user. We will use a model called “DeepMoji” that has been specifically trained for this task.

DeepMoji

When we use social media, it’s pretty common to use emojis to express our mood. Bjarke Felbo et al. used this insight to design an interesting technique called “DeepMoji”.

Their idea is pretty simple. We have millions of people tweeting their opinion along with emojis on Twitter. If we take those tweets and create pairs of (text, emojis) from them, we will have a huge labeled dataset of emojis without any manual annotation effort. This is depicted in the GIF below.

Now a model can be trained on this data to predict emojis for any given text. This trained model can be directly used for emoji prediction as well as be used for transfer learning to tasks such as sentiment, emotion and sarcasm detection.

DeepMoji was released on September, 2017 and was originally written in Keras with a Tensorflow 1.0 backend and Python 2.0.

We ported it to Python 3 and migrated the models to Tensorflow 2.0. We also have added support for direct installation from pip and added wrappers for ease of use. It’s open source and available here:

https://github.com/nexxt-intelligence/DeepMoji

You can also try out emotion prediction interactively from here: https://deepmoji.mit.edu

Emoji Prediction with Deep

First, we will install the DeepMoji package for prediction and wordcloud and matplotlib libraries for plotting using pip.

Now, we import the DeepMoji model and use it to predict the emojis for a list of opinions in a file. The sample file contains examples of user opinions on working from home.

Step 2: Generate EmojiCloud

Here, we create a wrapper class that takes in a list of emojis and uses wordcloud package to visualize the emojis instead of words.

Now we create an object of EmojiCloud by passing the font path. Default fonts available on your local systems might not support all the unicode emojis. So, we use the Symbola font to render the emojis. You can download the font from here and extract the zip archive to get Symbola.ttf file. Then, you need to place it in the same directory where you create this python script and specify the path using font_path argument.

You can also control the color of the EmojiCloud generated using the color parameter. You can use any of ‘yellow’, ‘blue’, ‘green’ and ‘grey’ as the color value with the default being ‘yellow’.

Try it Yourself

You can follow the above steps using this colab notebook and generate EmojiCloud for your own files.

Limitations of DeepMoji

While DeepMoji predictions are mostly relevant, it does suffer from problems due to its training procedure on weakly-labeled tweets in the wild. We have observed few cases where certain triggers caused a specific prediction.

For example, the word “traffic” is mentioned in a positive opinion given below:

It is better for our streets and traffic that not everyone is going to work all at the same time.

DeepMoji predicts the above positive opinion as angry(😠). This could be attributed to the fact that it saw most association of “traffic” to anger when it was trained on tweets.

The Nexxt Iteration

Although EmojiCloud was a fun prototype to play around with internally, there are some severe limitations related to ambiguous and/or sarcastic use of certain emojis. With some lessons learnt, the Nexxt Intelligence team is working on a more robust model tailored to textual emotions by incorporating psychological models of human emotion.

We think this will be a powerful way to understand our users, and we’d love to hear how you might extend this idea in the comments below