Joe Rogan Experience #1263 - Renée DiResta

Episode Summary

Main Topics Discussed

• The Origins of Online Disinformation Research: Renée DiResta's initial research began in 2015, observing anti-vaccine activity in California and how small groups disproportionately amplified messages on social media, often through coordinated and deliberate attempts to game algorithms.
• Evolution of Online Manipulation Tactics: From early observations of algorithmic amplification, the focus shifted to how terrorist organizations like ISIS used bots and social media to spread propaganda, eventually leading to the awareness of Russian interference.
• The Internet Research Agency (IRA): Discussion of Adrian Chen's 2015 New York Times exposé on the IRA, a Russian entity using social media to target various facets of American culture, initially experimenting with Russian and Ukrainian audiences before pivoting to Americans.
• Social Media as a Vulnerable Ecosystem: How the consolidation of audiences onto a few major platforms (Facebook, Twitter, Instagram) made them prime targets for propagandists, trolls, and state intelligence services.
• IRA's Deceptive Tactics: The use of "sock puppets" – real people pretending to be someone else – to engage in online discourse, mimic American trolls, and make online arguments more vicious.
• IRA's Operational Sophistication: Insights into the IRA's structure, including "stand-ups" where employees (often young, internet-savvy millennials) discussed granular details of targeting specific American communities (e.g., using culturally appropriate memes for Black LGBTQ+ pages).
• Tribe-Building Strategy: The IRA's long-term approach of creating and nurturing distinct online "tribes" (e.g., LGBTQ+ pride, various segments of the Black community, different far-right groups) over years, subtly inserting political narratives by appealing to in-group dynamics.
• Account Repurposing and "Sleeper Accounts": How the IRA would rebrand underperforming or dormant accounts (e.g., "Nuts News" to "Army of Jesus") to reach new audiences or act as "sleeper accounts" to be activated and politicized later.
• Impact on Public Trust: The long-term consequence of these disinformation campaigns, making it difficult for people to discern what's real and fostering a sense of paranoia and distrust in the information environment.
• Political Motivations: The clear pro-Trump and anti-Hillary Clinton stance of the IRA's political content, aiming to erode support for traditional Republicans and influence the 2016 US election with specific geopolitical goals like the removal of sanctions.

Key Insights & Memorable Moments

• Renée DiResta's journey into researching online manipulation began with a relatable personal concern: wanting to fit in with parents on preschool lists as someone who vaccinates.
• The realization that the "internet is vulnerable to this type of manipulation by anyone" was a pivotal moment in understanding the broader implications.
• The distinction between a "bot" and a "sock puppet" (a real person pretending to be someone else) highlights the human element in coordinated deception.
• The detailed description of IRA's internal operations, including their "stand-ups" where they would discuss culturally nuanced messaging, revealed a surprising level of sophistication and understanding of American online communities.
• The vivid example of an account evolving from "Nuts News" (featuring Kermit the Frog memes) to "Homer Simpson" memes, and finally to "Army of Jesus" underscores the IRA's pragmatic and adaptive approach to audience engagement and long-term strategy.
• Joe Rogan's observation that "they figured out how to use this to make people argue against each other" perfectly encapsulates a key impact of these operations.
• DiResta's point that even if disinformation campaigns are caught, "it makes it really hard for people to know what's real after the fact," revealing a significant downside that benefits the manipulators.

Notable Quotes or Revelations

• On the early observation of manipulation: "small groups were able to kind of disproportionately amplify messages on social channels."
• Defining the vulnerability: "can we think about ways in which the internet is vulnerable to this type of manipulation by anyone and then and then come up with ways to stop it."
• On IRA operatives: "it's kind of wrong to call them bots because they are real people they're just not what they appear to be... a sock puppet is the other way that we can refer to it."
• Regarding IRA's cultural understanding: "if you're targeting black lgbt people make sure you don't use white people in your in your image and your meme because that's going to like trigger them."
• On the core strategy: "they were building up tribes so they were really working to create distinct communities of distinct types of americans."
• Explaining the subtle influence: "every now and then there would be a post that would reinforce like as black people we don't do this and so or as lgbt people we don't like this and so you're building this rapport."
• On the impact of getting caught: "it makes it really hard for people to know what's real after the fact. It leaves you a little bit off balance."
• A stark political observation: "there is not on facebook and instagram there was not one single pro hillary post" from the IRA.

Overall Themes

• The Fragility of the Information Ecosystem: The discussion highlights how easily social media platforms, initially designed for connection, can be weaponized to sow discord and manipulate public opinion.
• Sophistication of State-Sponsored Disinformation: The IRA's tactics demonstrated a deep understanding of human psychology, American culture, and social media algorithms, moving far beyond simple bot networks.
• Erosion of Trust and Truth: A central theme is the deliberate effort to create confusion and make it difficult for citizens to distinguish between authentic and fabricated information, ultimately undermining societal cohesion.
• Amplification of Division: The IRA's strategy of identifying and reinforcing existing societal divisions, rather than creating new ones, showcases a cunning approach to exploit vulnerabilities for political ends.
• The Long Game of Influence: Disinformation campaigns are revealed not as short-term attacks but as sustained, multi-year efforts to build relationships and subtly shift narratives over time.