The main trend of modern information flows is the maximum simplification of presentation. Even words are replaced by sets of characters or small images – emoji. Recent studies show that in Finland over 60% of Internet users use these icons, in Britain and Germany – about 50%, Japan, America and Brazil – a little less than 40% of Internet users.

Every year, the number of emojis in communication increases, and the pictures themselves reflect socially important issues. In 2014, developers gave users the opportunity to choose emoji skin tone to avoid discrimination. Icons can now be found in court rulings. For example, a British judge, establishing the mother’s intentions to return for her son, analyzed the meaning of  “?” and used it in the ruling to simplify the understanding of the case for the child. An Israeli court recently interpreted a message^[1] as a consent to a lease agreement, rather than an offer to discuss its terms, simply because of the availability of “icons that conveyed great optimism.” Picking up the trends, Facebook replaced “like” button with reactions (“? ❤️ ? ? ? ?”), Instagram created an emoji slider to rate “stories”. The popularity of the icons is so great that the Oxford Dictionary named “?” the word of the year, and on July 17 the whole planet celebrates World Emoji Day.

However, is everything as rosy as it seems at first glance? The answer is sad: no, because emojis are increasingly being used to circumvent restrictions on the distribution of illegal content. How do we determine the improper use of emoji, should social networks respond and are they able to do so? So let’s try to find out.  

Online bullying ?

Bullying and harassment are quite common for social networks and social media. However, it is becoming increasingly difficult to fight them, because the ingenuity of violators is gaining momentum. Thus, a French court considered the case of using “” in a conversation with an ex-girlfriend and concluded that emoji is a “direct and real threat.” In another case, criminal proceedings were initiated against a 12-year-old girl for “meet me in the library  ? ?”.

However, popular platforms responded right away. In 2016, Apple, Google, Microsoft and Facebook (later – Instagram and Twitter) have decided to replace the original image of the weapon “” with a toy weapon image “?”. Even in the update to the Olympic Games, social networks banned rifle emoji, although shooting was portrayed exclusively as a sport. Such measures have partially reduced the use of weapon icons for online bullying, but have not stopped it. For example, a Brooklyn teenager has recently threatened the police[2] and replacing a real gun with a toy one did not change the message. Moreover, the list of emoji threats is updated daily and already includes “?,, ?”.

It is common to use emoji to denote coordinated games aimed at leading to suicide. The danger is that it is impossible to track images through social media searches (most platforms do not have this feature or are just starting to develop it). In Ukraine, in particular, a very popular game “blue whale” spread in the network by #blue_whale та “?”. [3] While the hashtags could be tracked in time, the situation with the icons was much more complicated. In the end, the problem was partially solved by blocking Russian social media through which the game spread. However, effective emoji regulation has not been introduced yet.

The public condemnation campaign on Facebook also revolved around “”, which is interpreted as “feeling fat”. Users complained that being overweight is not an emotional state, and such an image violates the rights of people suffering from eating disorders and obesity (bullying about appearance). Activists collected 16,000 signatures on a petition to remove this emoji and Facebook removed the icon from its list of emoticons.

Less obvious is the use of innocent images for the purpose of online bullying. An example is the US case concerning letters of dismissal containing “???”. The plaintiff’s lawyer stressed that in this way the company expressed joy over the woman’s dismissal. Although the court concluded that such messages were inappropriate and offensive, interpreting irony or other hidden message is not an easy task. That is why emojis only complicate the content interpretation, in particular for social networks.

Eventually, online bullying through the lens of emoji is gradually reaching a global level. The Russian government has repeatedly tried to influence camera operators to remove images of the LGBT community, and China still insists on removing the Taiwanese flag icon. However, developers are trying to maintain a balance between the ability to have heated debates and the protection of human rights.  

Sexual harassment and content?

Platforms do not want to become a tool for distributing obscene content. Therefore, the new policy of advanced social networks prohibits this type of content. The form does not matter: be it texts, images, videos or any other materials. Facebook community standards provide for the blocking of accounts for emoji or combinations thereof distributed as suggestions or requests for sexually explicit acts. The ban also covers attempts to cover naked body parts with small images.

The three icons that social media directly mentioned in the explanations for the new policy were “?, ?, ?” due to “the general sexual innuendo of [such] emojis”. In particular, they are used to obtain nude photos, to arrange intimate relations or during sexting. Despite the position of the platforms, according to statistics, most obscene requests or proposals are accompanied by “?”, “?” or “?”. However, the list of emoji with sexual meanings is not limited to six images. Depending on the context, they include:

• ?, ?, ?, – designation of male genitals;
• ?, – designation of buttocks or female genitals;
• ? – request to send nude images;
• ?, ? – request to send sexually explicit videos;
• ?, ? – an offer of sexually explicit Skype session.^[4]

Also sexually explicit messages and posts are created using combinations of emoji or their combination with other characters. The most common examples are:

• ?ing, ♋ – designation of intimate relations;
• ?? – offer of sexually explicit conversations;
• ??? – request to touch breasts;
• ??, ??? – proposal of intimate relations;
• ???- striptease request or offer.

In fact, the issue is much larger than the usual use of “?” in private correspondence. Today, emojis are used even in the context of sex slavery and human trafficking. Recently, a US court dismissed a “???” message as evidence. However, the lawyers stressed that this decision only provides space for changing the format of communication (changing text for emoji), and therefore helps to hide crimes using small images.

At the same time, emojis often help draw attention to socially important topics. For example, Weibo (the Chinese equivalent of Twitter) has long been heavily censoring the #MeToo movement.[5] This forced activists to change the hashtag to #RiceBunny, and later even use “??” (pronounced “me too” in Chinese). On the one hand, this situation illustrates the need for contextual analysis of any information, even if it has sexual content. On the other hand, the Weibo phenomenon once again points to the ineffectiveness of existing content filtering methods. Social media are not able, if necessary, to respond quickly to the wave of banned materials.

Hate speech, calls for violence ?

Calls for violence can take many forms and contexts – emojis are no exception. For example, in 2016, USA, Canada and Australia were hit by a wave of crimes committed by perpetrators in clown costumes. Afterwards, Apple antagonized users with its release of emoji – users associated the icon with a public call to commit such crimes. However, the story sank into oblivion, and the image regained its usual meaning.

The situation with the use of the “?” symbol is more dangerous. In the United States, racist organizations use it in their posts as a symbol of “white supremacy.” However, the icon is so common that it is very difficult, if at all possible, to track abuse. In addition, radical groups have recently started accompanying their own ideas with “?” to discriminate against non-whites. Originally, emoji was a sports symbol meaning: “I will do anything for my team”, but its content has been distorted and now even peaceful publications are the subject of numerous complaints.

No less popular was“?” (“Pepe the Frog”), which neo-Nazis used in their posts and Twitter account names. Accordingly, emoji gradually began to be associated exclusively with the activities of radical organizations. This served as an impetus to tighten restrictions on its use in social media. However, this is not the end of the story. “?” was replaced by a rather unexpected image – “”.  The use of this icon is probably associated with the color characteristics and the need to bypass social media filters adapted to classic discriminating emojis. The platforms have not yet developed a mechanism for tracking this image.

In addition, dehumanization (deprivation of human characteristics) has been a negative trend for a long time. Examples include the use of “?, ?” in messages about the members of Muslim communities, the publication of “?, ?, ?, ?, ?, ?, ??” to denote members of racial, ethnic or national groups. The response to this problem on the part of social networks is a ban on the distribution of content that identifies a person with certain animals or objects.

Emojis often carry a negative connotation due to their use in the context of hostility. For example, the Brazilian protests against Islam, called “vomitaço”, were accompanied by “, ” (the emoji means “vomiting”, which shouts like the riot slogan). The government has repeatedly sent requests to Facebook for regional blocking of the icon, but the platform has abandoned such practices. In the absence of bans, activists began distributing images through publications and comments turning a small icon into a “weapon against Islamism”. Although the frequency of use of this emoji has decreased, it is a consequence of social processes rather than the platform’s struggle against hostility and discrimination.

But the biggest problem is that the algorithms work against social networks. Facebook’s “? and ?” reactions, which show dissatisfaction with the content, only help to distribute illegal content (through which the post goes up in the news feed, gaining even more views). For example, in the case of the spread of anti-Islamic sentiment in Brazil, it was the negative reactions that contributed to the rapid and large-scale spread of hate speech.  

What should we do about it❓

Currently, the number of problems slightly exceeds the number of solutions. The topical issue is increasing the popularity of posts that receive negative reactions and often have banned content. The stumbling block is the lack of a unified interpretation of emoji among users. Moreover, sometimes users do not update social media applications for a long time. This leads to discrepancies in emoji lists and numerous misunderstandings that reach the level of hate speech or bullying. After all, contextual interpretation is sometimes difficult even for qualified social media staff. For example:

• ???‍ – is interpreted as “a crocodile was shot by a police officer”
• ?‍?? – is interpreted as “anonymous police threat”

Today, the platforms are at a crossroads: due to the popularity of emoji, social networks cannot abandon them, but they are also not able to fully regulate their use. Prohibited content filters are not adapted to the search for posts by icons; and even if such a feature already exists, they are not able to interpret the context. In fact, the situation is similar to the interpretation of hate speech in text form – statements that are legal in one case are prohibited in another. At the same time, moderators cannot evaluate the entire array of information fast enough.

Although high hopes are placed on artificial intelligence, filling the system with the necessary data requires a lot of time and human effort (first, the analysis is carried out by moderators providing “instructions” that help identify misuse). At present, it is difficult to name a more effective mechanism.

However, not everything is as bad as it seemed at first glance. Instagram has already created an emoji search engine and made it possible to distribute icons with hashtags, simplifying the identification of banned material. Snapchat was also affected by the fight against the misuse of emoji. Now, when distributing videos, users cannot use icons for threats (“?, ?, ?”) or dehumanization (“?, ?”). The platform warns that in case of intentional violations, the account may be blocked.

Facebook has developed an emoji interpretation system to help moderators detect misuse of icons faster and more effectively. Images are grouped into the categories such as “bullying”, “attack, harm, call to action”, “hate speech” (which includes the popular “?”), “sexual harassment”, etc. Facebook emphasizes that some symbols, on the contrary, indicate “protected characteristics”:  and national flags. Improper use can lead to posts or even an account being blocked.

Advanced social networks have created emojis (“?️‍?️, ?, ?, ?, ?”) that are used as a “signal” in comments under posts with online bullying (“I Am A Witness” campaign). As a result, more than 110,000 emojis and 400,000 reactions were distributed in a few months, allowing the platforms to deal more effectively with banned content. In addition, children anti-drug campaigns have been launched with the help of emoji, and support icons have been created for online bullying victims.

In conclusion, the prevention of emoji bullying, harassment and calls for violence has set the wheels in motion. Despite the difficulties in the contextual assessment of the content of posts and messages, social networks are developing new policies adapted to the recognition of regional values of icons, hidden subtext, etc.

However, this does not mean that the list of controversial issues involving emoji is exhausted. In particular, since 2018, the below issues have become relevant: copyright to these images, admissibility, social and political icons, as well as the launch of applications that use prohibited emoji. The number of issues for discussion is increasing exponentially almost every day, and the only thing that will surely save the world is emoji “respect” shared online.

^[1] Message: “Good morning? we want a house???‍✌️☄️?️ just need to go over the details… Please let me know what time works best for you”.

^[2] Text of one of the posts: “Hands up ?????‍??”.

^[3] Coordinators collected private information about those involved in the game threatening them and their relatives if they did not complete the tasks (the final task was to commit a suicide).

^[4] Skype is often used for private sexual communications (although now it is also possible on Facebook).

^[5] A movement to combat harassment through the spread of such stories online.