The counterterrorism and CVE community has long questioned the effectiveness of counterspeech in countering extremism online. While most evaluation of counterspeech rely on limited reach and engagement metrics, this paper explores two models to better measure behavioral change and sentiment analysis. Conducted via partnerships between Facebook and counter-extremism NGOs, the first model uses A/B testing to analyze the effects of counterspeech exposure on low-prevalence-high-risk audiences engaging with Islamist extremist terrorist content. The second model builds upon online safety intervention approaches and the Redirect Method through a search based “get-help” module, redirecting white-supremacy and Neo-Nazi related search-terms to disengagement NGOs.
Over the last 10 plus years there have been huge global efforts and investments in the Preventing and Countering Violent Extremism (PVE/CVE) sector. This is particularly the case since 2014 with global awareness of the digital savvy nature of the so-called Islamic State and their international recruitment strategy. Large waves of government and private funding have subsequently worked to support organizations, activist networks and marketing teams trying to counter online propaganda and recruitment. This form of strategic communication has largely been done through online messaging campaigns, commonly referred to as “alternative narratives,” “counter-narratives” and/or “counterspeech.”
However, the golden question remains as to whether or not these programs, local or global in their reach, have had a positive measurable impact on the audiences they aim to reach. Beyond measuring the basic metrics of reach and engagement, can these primarily online efforts show behavioral change and/or sentiment shift in the intended target audience exposed to this content? Could exposure to counterspeech in at-risk or radicalized audiences perhaps have the unintended consequence of further radicalization, or act as a catalyst to the radicalization process? How best can private tech companies work with non-governmental organizations (NGOs) and experts in the PVE/CVE space? This article aims to analyze these questions through two testing models of counterspeech deployment.
For the purposes of this article, we will refer to online PVE/CVE campaigns and initiatives as “counterspeech,” with the understanding that this type of content can include both preventative or alternative messaging as well as more down-stream and directed counter-messaging. This article explores two approaches for measuring theories of change in audiences exposed to counterspeech that go beyond basic reach and engagement metrics, based on partnered research and testing carried out on Facebook. Testing and analysis were led by the Facebook Counterterrorism and Dangerous Organizations Policy Team in coordination with internal data scientists, community integrity engineers, market teams and safety researchers with privacy and legal reviews of both methodologies before their deployment.
Privacy and legal reviews have existed at Facebook for many years and are needed in order to launch any new on-platform program and tooling and have become even stronger in recent years. Privacy Review at Facebook includes technical validation, internal and external consultations and a review of documentation and implementation procedures. Review processes look closely at any usage of data, transparency measures for the user interfacing, assess risks and put safeguards in place to address any concerns before a program or tooling feature is allowed to launch. Internal stakeholders involved in Privacy Reviews include key reviews from product, policy, communication, design, legal, user experience and research team leads – among others depending on the nature of the project.
Both models were launched and tested in partnership with NGOs, who developed the counterspeech content and programmatic efforts independently of work with Facebook. The ultimate purpose of this research is to assess the efficacy of different counterspeech and counter-extremism efforts online in order to test the viability of methodologies for proactively surfacing counterspeech content to identified “at risk” audiences at scale and to provide constructive interventions. The first model uses an A/B testing format, building off of campaign deployment traditionally done through ads marketing tools. The A/B test focused on testing both softer preventative content and harder counter-content in order to assess behavioral change aimed at audiences with early at-risk indicators around violent Islamist extremist content in English and Arabic. The second model builds upon existing online safety intervention models that Facebook has developed and combines an approach with the Redirect Method to measure the effectiveness of online searches that trigger support toward offline disengagement practitioners.1 This redirect initiative aimed at audiences searching for white supremacy and neo-Nazi groups and/or individuals and was initially launched in the USA, again with a localized NGO partnership.
The aim of presenting the results of these two methodologies is not to compare them with each other, but to explore the efficacy of two different approaches in disseminating counterspeech online to at-risk target audiences and see where, and what type, of impact is measurable. Therefore, while the violent extremist ideologies and geographies targeted in each model are different, that should not detract from the ultimate goal of testing new models for deploying counterspeech. Recognizing violent extremism takes different forms around the world, we chose to focus on countering violent Islamist extremism and white supremacy/neo-Nazism trends as they are currently recognized by international researchers and international government bodies as the two largest violent extremist threats with an online nexus.2 A range of other Facebook programs tackle a broad international scope of violent extremism.3
Based on the findings from testing both models, this research article argues that:
- There is no data to support the theory that exposure to counterspeech negatively impacts behavior or sentiment indicators. There was no data showing “increased radicalization” based on either test.
- Based on the A/B test, behavioral change in the wider target audience was not statistically significant overall, however, among the higher risk population identified within the target group, positive shifts of decreased engagement with violent extremist content were notable.
- Focusing on behavioral signals to define an at-risk audience more accurately ensures that campaigns deployed are reaching the intended audience, thereby cutting out the noise around results and ensuring cleaner data for analysis.
- One-off signals of a shared piece of violent extremist content are not clean indicators for an individual’s sympathies or membership in a violent extremist group.
- Based on the Redirect Initiative model, we have initial results of exponential increases in audiences finding online resources leading them to off-platform engagement with disengagement practitioners and resources. The findings show that an initial passive online search can lead to active offline engagement and initial findings of positive sentiment shifts.
