Tracking coordinated inauthentic behavior through memes with GraphXR
Click here to interact with GraphXR View
memetic influence offers boutique intelligence, technologies, and data for cross-platform analysis of coordinated inauthentic behavior. This organization was founded by researcher Mitch Chaiet to provide content and analysis regarding misinformation. They develop new media technology for the analysis of media manipulation campaigns. Their disinformation research has been featured by Forbes and NBC.
How did memetic Influence start?
I graduated in December 2019, straight into the COVID-19 pandemic, as part of the last in-person graduation before we were all forced to stay in. With lockdown on the rise, my backburner research on disinformation suddenly came to the forefront as a rapid rise in coronavirus misinformation awashed social media platforms.
This led me to start a blog called “memetic influence” — writing about the influence of memes and other content I found circulating online. I had been part of a group of researchers at University of Texas (UT) called Good Systems, which was a collection of 35 different professors around the school researching disinformation. My focus was on memes and screenshots, specifically how images spread subversive communication. In other words, how do traditional metrics fail in tracking what we would call disinformation?
Click here to interact with GraphXR View
I started finding live examples of coronavirus disinformation online, and mimicked the writing style of reports I found from companies and professionals in the industry. Those became of interest to the OSINT community, such as expert Henk Van Ess who shared some of my articles. We were lucky enough to connect with many of the professional journalists and researchers who cover disinformation to gain a broad understanding of the current tools in the space. From there, we built a prototype for tracking screenshots of tweets back to the original URL, and that became the basis for Sourcerer. It grew from a blog to a hacky startup/freelance business in about 6 months.
What are you working on?
We’ve put out three methods for analyzing the spread of imagery. With Maltego, we’re able to analyze the spread of multiple images using reverse image search, and automatically cross reference the paths of propagation for a body of content. Using compression analysis, we can fingerprint and analyze how images get more compressed over time as they spread online, and determine a hierarchy of how they spread between platforms. With Sourcerer, we’re able to track screenshots of social media posts back to the original source programmatically and analyze the spread of narratives within. GraphXR allows us to easily analyze all of this broad-scope network data immersively and entirely in-browser with no external tools.
In 2018, when I was still in college, Congress subpoenaed Facebook for a dataset of 3500 Russian Facebook ads that the Internet Research Agency bought on Facebook to try and target the election. I was friendly with the research group at UT analyzing them. In this dataset, at least half of the images they used for the ads were “memetic.” Each one was lifted from somewhere on the internet and reposted elsewhere as content for Facebook pages that the Russians created.
For in-depth analysis of the Russian Facebook Ads: https://mitchaiet.github.io/RussianAds/
When this dataset was released, qualitative researchers took the images and started tagging them for content review, as is standard for visual studies. They would discover insights such as 8% of the ads targeted Hispanic men, and 90% of the imagery used in those ads featured Hispanic men. Then, there were those looking quantitatively at which demographics were most engaged in terms of ad spend vs clicks, say between African American men, African American women, Hispanic Men, etc. Those are two styles of analysis that were typical for this kind of dataset.
Click here to interact with GraphXR View
I realized that there was a third method of analysis I could do, given my background knowledge of the spread of memes. I started using reverse image search to create a spider web of how these images traveled across the internet. Normally with reverse image search, you input one image and get a list of results. You can do that with a single image on a website and look through where that image appeared online. We needed an upgrade for analyzing 3500 images. Using Tineye’s API and Maltego, we were able to cross-reference the patterns of propagation of all of the images, and uncover sources like blogs, community forums, and hashtags where images were pulled from programmatically in order to be reused as part of the IRA’s information operations campaign.
Some ads had nothing to do with each other in the dataset, but reverse image search analysis showed the images used in them all traced back to a single source. When analyzing the reposting mechanisms at scale, we were able to find that they were tracking authentic content across these communities, and looking at images that already had high engagement among their target demographics. Then, they would pull content that was already popular and create their own fake pages, a tactic called a butterfly attack.
What were the most concerning trends you found?
The most concerning pattern is their ability to mimic legitimacy, to the point where images from fabricated pages were spread authentically to other spaces by the target audience. They would watermark imagery the reposted from elsewhere on the internet with their own brand logos. Then, their captured audiences would propagate the tagged imagery into authentic, demographically aligned spaces, leading individuals to build trust in the brand and engage with the fabrication further by liking the page, seeking out the associated websites, or reading their articles. On an individual level, if you saw a watermark on a meme in a group that you’re a part of, and then you went and liked that page without realizing it’s a fabricated source, you’d also be targeted more heavily with disinformation fitting your demographic. That was one of the most interesting patterns regarding how they leveraged both watermarks and memetic spread into expanding their audience for propaganda.
Click here to interact with GraphXR View
Who inspires you?
Memes and imagery have been my little niche into the much broader world of open source intelligence, which involves platform manipulation, geopolitics, cybersecurity, and journalism. I just happened to find my little niche and happened to be innovative within that. I look up to a lot of the researchers in the space who are pioneering network analysis at scale, such as Kate Starbird.
What have you recently discovered?
We’ve found a couple new examples of coordinated inauthentic behavior by visualizing a dataset of images used by the Russian Internet Research Agency to “hack” Facebook, along with their patterns of spread using reverse image search via TinEye. Looking at connection-driven data through graph technology enabled us to trace back and visualize where a particular image came from. Some stories that quickly emerged involved looking at the authentic spread of a “fake” African-American pride rally, intertextual sourcing patterns targeting Hispanic demographics, and the amplification of imagery associated with inflammatory topics.
Spread: Authentic Propagation of “Fake” African-American Pride Rally/Protest
Demographic Target: African-American | Stance: Pro-African-American
Click here to interact with GraphXR View
In this example, we have evidence that imagery created by the Russian Internet Research Agency to promote a fabricated “rally” was picked up organically from Facebook, and spread to a thematically similar authentic hashtag on Twitter, #nohate. First, the Russians created the fake brand Black Matters (BM). They then target African Americans and got them to like the page. Afterwards, they created a series of 5 ads and an event on Facebook for a “rally” in Charlotte, NC targeting African Americans in the city.
TinEye Results suggest the banner used in the Facebook ads appeared on Twitter (12/3/2016) crawled from the hashtag #nohate. This means that at least one person was presented with the ads, and spread the imagery authentically across platforms. Mimicking the behaviors of a social group is called a Butterfly Attack, and narratives spreading from an inauthentic source to authentic spaces is called Trading Up The Chain.
Sourcing: Intertextual Sourcing Patterns for Content Targeting Hispanic Demographics
Demographic Target: Hispanic-American | Stance: Pro-Hispanic
Click here to interact with GraphXR View
In this example, we have evidence that imagery used to target Hispanic demographics under the fabricated Facebook page Brown Power was sourced from open, authentic sources with and without attribution. Authentic hashtags and forums served as data sources, often with demographically-linked topics. First, the Russians created the fake brand Brown Power. They then created a manifest of existing sources which Hispanic audiences authentically engage with. They pulled content from those sources, and reposted them under their own brand. They then targeted Hispanic Americans with ads using that content and got them to like the page.
Two different ads, with two different captions, featuring the same image indicate that multiple people were running this page and pulling content from the same repository.
Tineye results suggest the picture used these two ads was found in two places online, www.lowridermagazine.com, and on Pinterest under the tag “chicano-culture.” One of the ads includes a rare source attribution from the Russians; “Source lowrider” indicates the image was certainly sourced from the Lowrider forum and not the social network. The other ad with a different caption does not name a source, indicating that more than one person is using the exact same content source to run ads on this particular page.
Amplification: inflammatory narratives promoted using authentic imagery
Demographic Target: African-American | Stance: Anti-Cop
All of the ads in this cluster were run under a page called “Black Matters US” which was a brand the Russians created to mimic Black Lives Matter. The ads feature a single meme, and are targeted at topics like Black Power, Racial equality, Social justice, and Black Panther Party. Looking at the Tineye results, you can see the image traveled along hashtags like “BluKluxKlan” “BadCops” “PhilandoCastille” “JimCrow” — it’s a perfect, novel example of how the Russians amplified divisive content.
While a feat to navigate this large dataset, these stories were made possible by exploring graph technology with GraphXR & Neo4j — we can visualize how Russians utilized content already proven to be popular among certain demographics, how they edited that content to convey their target messages and spread it through their owned parody pages. You can read up on the latest research at memetic influence.
Mitch Chaiet is a creative technologist from Austin, TX currently focused on Open-Source Intelligence and Disinformation/Media Manipulation Research. He graduated from the Moody College of Communication in 2019, where he helped launch the college’s innovation, entrepreneurship, and immersive media programs. After creating the most-liked meme at UT, and studying it’s spread to other universities, he joined the Technology and Information Policy Institute to complete a thesis reverse-engineering the spread of networked content used by the Russian Internet Research agency to “hack” the 2016 US election. This led to inclusion in UT Good Systems’ Disinformation Research Group. His subsequent research into pandemic disinformation lead to unearthing novel patterns of platform weaponization, which leveraged the Wayback Machine, a hole in Facebook’s content moderation systems, and screenshots of harmful content to spread CoronaVirus conspiracies. He has provided disinformation analysis to NBC and Forbes. Mitch runs memetic influence, inc to build tools which help journalists and researchers fight disinformation, provide content, data, and analysis of internet events and support law firms, newsrooms, and researchers with OSINT investigations. Connect with Mitch on LinkedIn and learn more on GitHub.