Inside Meta’s Efforts to Prevent School Shootings

Mass murderers have long attracted macabre groupies, but, like many ugly human tendencies, the phenomenon has been accelerated and amplified by the internet. According to law enforcement, the Texas girls were participants in the T.C.C., short for True Crime Community, an online fandom that celebrates perpetrators of mass violence. In the past two years, at least nine attacks and sixteen disrupted plots have been linked to the T.C.C., according to the Institute for Strategic Dialogue, a nonprofit think tank with a focus on hate and extremism.

In the aftermath of the arrests, law enforcement announced that the girls’ plot had been discovered through their online activity. “Everyone was, like, ‘Oh, so the F.B.I. really is spying on us?’ And, no, it’s not the F.B.I.,” Baimbridge said. The initial tip had come from a little-known team inside Meta that has spent the past decade honing a method to identify potential school shooters. Algorithmic social media, which has helped create the conditions for the spread of online content that might inspire such shootings, is reckoning with how it can become part of the solution.

Before Emily Vacher was recruited by Meta, in 2011, she was an F.B.I. agent specializing in crimes against children. At Meta, Vacher began working with the law-enforcement-outreach team, which she now helps lead. In the immediate aftermath of high-profile incidents, such as the Sandy Hook school shooting, in Newtown, Connecticut, the team would make contact with law enforcement to offer assistance. “You know, are there any co-conspirators? Is there any other impending danger they need to know about?” Vacher said.

As an agent, Vacher had spent time working with the behavioral-analysis unit in Quantico, the heavily mythologized home of the Bureau’s criminal profilers. There, she studied the work of Mary Ellen O’Toole, a longtime profiler with a specialty in school shooters. O’Toole had come to realize that many people who commit mass violence telegraph their intentions in advance, a phenomenon she referred to as “leakage.” School shooters, it turned out, were particularly prone to it: an F.B.I. study of active-shooter attacks between 2000 and 2013 found that nearly nine out of ten shooters under the age of eighteen had leaked their intent.

If would-be perpetrators were dropping hints about imminent violence online, Vacher thought, it should be possible to detect and stop them—“to think about disrupting attacks, not just investigating them,” she said. In 2018, she helped assemble a group of people with expertise in school shootings, including former journalists and police investigators, to study the online footprints of past attackers and to identify behavioral patterns. They worked with a team of engineers to build systems that use A.I. tools in order to scan for signals. Content flagged by the automated systems is passed along to a human for further review of the user’s online behavior: the posts they make and share, their interactions with others, the prompts they feed into A.I. chatbots. When there is evidence of “substantial steps, overt acts,” as Vacher put it—a list of targets or a manifesto, say—the investigations team provides relevant information to law enforcement. Last year, Meta did so around eight hundred times worldwide.

The process can unfold rapidly. According to prosecutors, last fall, a student at Benjamin N. Cardozo High School, in Queens, posted a photograph on Instagram of schoolwork with a text overlay that read “boutta shoot the school up.” The post was flagged by Meta, which passed the user’s information along to the F.B.I., which in turn notified the N.Y.P.D. After law enforcement matched the I.P. address associated with the post to a phone number and a name, they searched the student’s backpack and found a loaded pistol. Just over two hours after posting the picture, the student was taken into custody. (The student was charged with gun possession and making a terroristic threat.)

But few cases are as straightforward as that one. Publicly announcing an attack in advance is “part of the whole lore in this community,” Vacher said, “but they can’t be so overt that they just immediately get caught.” (Meta’s community standards ban threats of violence, in addition to content that “glorifies, supports or represents . . . multiple-victim violence.”) Young people, adept at evading content bans, know how to speak in a densely coded language that often outpaces moderation tools built to target more explicit forms of radicalization. “The swastika is easy to understand,” Katherine Keneally, the senior director of threat analysis and prevention at the Institute for Strategic Dialogue U.S., said. “The Atomwaffen Division logo, that’s obvious. But if there’s, like, an A.I.-generated version of a person who committed a shooting in the Philippines doing a TikTok dance with a number behind them that signifies the number of people killed in the attack—that’s a lot more difficult to understand.” Keneally said that many social-media companies have downsized their trust-and-safety teams in recent years, allowing more T.C.C. posts to slip through the cracks.

Instead of using perpetrators’ names, T.C.C. posters may invoke Arlene (Eric Harris’s nickname for the shotgun he used in the Columbine attack) or Smiggles (a screen name used by Adam Lanza, the Sandy Hook shooter). When people began to catch on that Meta’s moderation tools were flagging profiles that used widely known images of shooters, they began using childhood photos instead. “You have to have someone who can recognize that, O.K., this is not the normal Adam Lanza picture we’re used to seeing—this is Adam Lanza much younger,” a member of Meta’s investigations team told me. As with any fan culture, the lore and language are self-referential and constantly evolving. “Any time there is an actualized attack, the imagery, the iconography, anything associated with the attacker—the music they listened to, the clothes they wore, the specific weapons they used, the things they wrote—tends to get recycled back into the canon,” the team member said.

The details of past attacks become a template for future violence: in the moments before a fifteen-year-old girl in Wisconsin killed a classmate, a teacher, and herself, in December, 2024, she posted a photo of someone in a bathroom stall, flashing a hand signal associated with white supremacy. A month later, a seventeen-year-old boy in Tennessee posted a similar image before killing a classmate and himself. An act of violence doesn’t need to be high-profile to make an impact. An incel-motivated stabbing at a school in Mexico City last year barely made the news in the U.S., yet it quickly became absorbed into the T.C.C. symbology.

The vast majority of young people who post in T.C.C. communities won’t commit violence, of course. In 2012, I wrote about the Columbiners, a kind of precursor to the T.C.C. The Columbiners congregated on Tumblr, where they posted swoony, heart-covered memes about Harris and Klebold. When I spoke with Columbiners, I was more worried about them than about anything they might do; their memes struck me as a clumsy and self-consciously edgy way of expressing their rage and pain and feelings of alienation. “The experience of engaging with TCC narratives is often deeply emotional for online users,” a report from the I.S.D. notes. One worrying sign is when someone moves from romanticizing or idolizing school shooters to identifying with them, according to Vacher: “They’re not just saying, ‘Oh, this guy’s amazing.’ They’re buying the clothes that that shooter wore, or they’re finding the gun that the shooter used, or they’re getting floor plans for the school, or buying tactical gear.”

Social-media companies must weigh competing claims of privacy, liability, and caution when deciding what to report to law enforcement. Overreporting runs the risk of shunting troubled kids into the criminal-justice system merely for expressing morbid thoughts. But the consequences of underreporting can be dire. Last June, the automated-review system at OpenAI flagged an account used by Jesse Van Rootselaar, a teen-ager whose conversations with ChatGPT featured repeated discussions of gun violence, including mass-shooting scenarios. A team reviewed Van Rootselaar’s activity but decided against reporting it to law enforcement, evidently concluding that it did not constitute an immediate or credible threat. OpenAI shut down the account, but Van Rootselaar soon created another one, on which the discussions continued, albeit in a slightly more coded way: the planned shooting was hypothetical, Van Rootselaar now said. The chatbot gave advice about how not to get flagged again. “You can still be twisted. Just be clever about it,” it advised. During the following eight months, Van Rootselaar “discussed building and using firearms and homemade explosives, fantasized about becoming a notorious mass killer, and asked about the timing of previous real-world school shootings,” according to reporting by Mother Jones. In February, after killing her mother and brother, Van Rootselaar opened fire at Tumbler Ridge Secondary School, in British Columbia, killing six people, and then herself. The province and multiple victims’ families have filed lawsuits against OpenAI. “I am deeply sorry that we did not alert law enforcement,” Sam Altman, the C.E.O., wrote in an open letter.

Meta’s investigations team is just “one small piece” of the work of preventing school shootings, Vacher told me. “This is very much a societal problem. We need schools and parents and mental-health professionals and teachers and law enforcement.” But, after speaking with members of the company’s team—sharp, sincere people doing potentially lifesaving work—I couldn’t help feeling that a kind of cognitive dissonance underlay the whole project. “It’s all well and good to report specific cases to law enforcement, but at the end of the day the platforms are also playing a significant role in the spread of this content that is radicalizing people to engage in violence,” Keneally, of the I.S.D., said—namely, by creating algorithmic feeds that amplify extreme content while also decreasing funding for trust-and-safety programs. (TikTok, she noted, is the mainstream platform on which T.C.C. content seems to spread the fastest and farthest. When reached for comment, a TikTok U.S. spokesperson said that the platform removes content that violates its community guidelines, permanently bans accounts dedicated to glorifying attackers, and takes action against additional accounts that may be operated by the banned user.)

The more time I spent in this world, with its strange mixture of coy menace and clout-driven nihilism, the more I thought about Columbine. Twenty-seven years later, the attack on a school in the Denver suburbs retains an archetypal mystique for generations of troubled young people. Harris and Klebold, early to catch on to the self-mythologizing possibilities of new technology, obsessively documented themselves and their fascination with violence, both online and off. It’s as if they were native speakers of a language that hadn’t quite been invented yet. ♦