How to Assess the Impact of Digital Diplomacy?

Thursday, December 3, 2015

As any diplomat or embassy communication officer using social media knows by now, impact assessment is the “holy grail” of digital diplomacy. What type of information diplomats can best deliver through social media, how deep their message can reach into the target audience, and what forms of communication are most suitable for engaging the foreign public? In other words, how effective is social media for conducting public diplomacy online?  In a paper I co-authored with Lu Jiang and published in Digital Diplomacy: Theory and Practice, we addressed this question by comparing the digital strategies pursued by the European Union’s Delegation and the embassies of Japan and the United States (U.S.) in Beijing on the Chinese micro-blogging website Sina Weibo (a hybrid version of Twitter and Facebook). To probe the relevance of digital diplomacy at the embassy level, we investigated the conditions under which social media would be able to perform three key aspects of public diplomatic engagement: agenda setting, presence expansion and conversation generating (see Fig 1). Each dimension speaks to an important aspect of exerting influence: message content, informational reach, and mode of engagement with the audience.



 Fig 1: Dimensions of Impact Assessment


  • The first dimension, agenda setting, relates to the extent to which social media platforms enable diplomats to set the agenda of discussion with their target audience. Information dissemination has always been a central task in public diplomacy. Public diplomacy helps build a certain image of the country for foreign audiences by directing the latter’s attention to certain topics, while downplaying others through well-selected news. Diplomats can thus construct an issue as salient and worthy of attention for their audience by repeatedly providing relevant information on that issue.

  • The second dimension of social media’s impact is presence expansion. If a government aims to develop a good relationship with a foreign audience, it first needs to be “out there” in the relevant public sphere. Diplomatic presence does not necessarily lead to a better image or favourable opinion, but without enough exposure, the public diplomatic strategy will ultimately fail. In the digital age, presence expansion becomes an even more critical condition for diplomats to make their voice heard.

  • The third and final dimension is conversation generating. Good public diplomacy can no longer be monologue- but dialogue-based. It has to facilitate a two-way or multidirectional communication between parties and to stimulate collaboration initiatives. Two-way conversations allow diplomats to readjust the focus of their agenda, reduce misinformation, and enhance mutual understanding. It is this particular feature that enables social media to realize the goal of public diplomacy in a different way from traditional methods.


1. Agenda Setting


A close look at the three Weibo accounts reveals important patterns and regularities (Fig 2-4). The design of Weibo feeds shows a crafty combination of informative texts, lively pictures and hyperlinks to policy papers. A balance is struck between the effort to stimulate the interest of Weibo users in an entertaining manner and the need to maintain an authoritative stance and provide official messages. Each Weibo diplomatic account follows a specific content pattern. Consider, for instance, the Weibo feed of the EU account, which starts each morning with famous European paintings or quotes from renowned writers followed by more elaborate and policy oriented contents. The same holds true for the other two embassies, which offer daily or weekly information on fixed topics. However, the focus of each account is slightly different, a fact that reflects the different agenda setting strategies and styles pursued by the three diplomatic actors. For example, Japan exerts a strict control over its agenda as suggested by the topics it chooses to post about, but equally important by those it chooses not to post about on Weibo. Compared with Japan and the EU, the posts of the US embassy are much more engaged in delivering "hard" political messages, especially those with potentially high social impact for the Chinese people.



Fig 2-4: Weibo entries


2. Presence Expansion 


While agenda setting measures influence from the perspective of the message senders, presence expansion looks at the same issue from the perspective of the message receivers. Weibo statistics allow us to accurately capture two important channels of presence expansion: reposting (how many times a post is circulated among users) and commenting (whether a post generates discussion or not).  In order to track the propagation routes of the US, EU and Japanese posts, we randomly picked two posts with min 200+ impressions for each of the three agencies. The results uncover a diversified pattern of propagation. Figure 5 shows, for instance, how a single message gets around. The cluster in the centre represents the original information source. Each dot in Figure 5 represents a node in the propagation route, and the curves connecting each repost refer to the direct relationship between a user and his/her (influential) follower who reposts the message from him/her. Denser dots mean more reposts, and larger clusters signify greater influence. Figure 6, on the other hand, shows the repost layers of these six messages. As it can be seen from the figure, if a Weibo post has more than 200 reposts or comments, reposting usually happens at about four levels. The stronger the impact of a message, the greater the number of repost layers. Two or more repost layers suggest the influence of the message reaches beyond the immediate group of followers to a wider range. It is in this way that public diplomacy is able to expand its presence on social media.


Fig 5: Route of reposting


Fig 6: Layers of reposting




3. Conversation Generating  


While the capacity of social media to facilitate interaction between the message sender and the audience is well recognized, the mechanism that enables this interaction is not fully understood. Theoretically, the conversation process involves a repetitive circuit (see Figure 7) of information providing (It), receiving comments and reposts (IIt), providing feedback,  readjusting information (IIIt & It+1) and making new comments (IIt+1). I, II, III represent different actions taken by EU, US and Japanese diplomats, whereas t and t+1 refer to successive stages of repetition of this cycle. Agenda setting and presence expansion have been discussed above as It (It+1) and IIt (IIt+1). Here we focus on IIIt: providing feedback to the audience. This constitutes a pivotal step in the conversation process because without it, the repetitive process will likely stop after a one-shot conversation. The three agencies provide feedbacks to other Weibo users in two major forms: by reposting on commenting on tagged feeds, or by commenting on Weibo users’ responses to their original Weibo feeds. With respect to the first mode of feedback, we found that none of the three diplomatic accounts reposted or commented on Weibo feeds that tagged them within the chosen timeframe of analysis. We also found no solid evidence of official responses to Weibo users’ queries. These findings may be disappointing to those defending the transformative role of social media in public diplomacy, as they raise doubts about the extent to which foreign publics are able to engage diplomats in two-way conversations. However, our research also reveals that the interactive dimension of social media has a broader scope of application, with additional diplomatic implications. For example, although the message senders rarely engaged directly the audience, exchanges nevertheless took place between the members of the audience, a finding that brings up the issue of community-building as a possible metric of impact assessment.  


Fig 7: Layers of reposting




To conclude, our analysis shows that digital diplomacy is being primarily used as an instrument of information dissemination and much less for engaging the audience in a two-way dialogue, at least in the three case studies that we have examined. To be sure, certain interactive features of social media are still present, such as the feedback mechanism (e.g., the audience’s comments and reposts), which enables embassies to learn about the opinions of their audience and readjust their tactics and strategies accordingly. It should be also added that social media can help deliver a strong message in a highly effective manner, but it cannot act as a substitute for good strategy planning, relationship-building and crisis managing, the well-established marks of professional diplomatic conduct. A full version of this study is available here.



Please reload

Prof. Corneliu Bjola

I'm an Oxford scholar seeking to make sense of "unknown unknowns" in international diplomacy, a tech geek constantly on the lookout for the next Cool Thing, and an unrepentant Big Lebowski fan ("lotta ins, lotta outs, lotta what-have-you's..").

Follow Me
  • Twitter Social Icon
  • Pinterest Social Icon
  • LinkedIn Social Icon
Other Posts

In Virality we Trust! The Quest for Authenticity in Digital Diplomacy

October 15, 2019

Please reload

Search By Tags