It has now been nearly a decade since Donald Trump first became president of the United States. We have become so used to his uncanny hold over the American political system that it is easy to forget how shocking and unexpected it was at the time. As I argued ten years ago, still fresh out of graduate school, this failure was not limited to the journalistic class; social scientists were also caught badly off-guard.
Donald Trump now looks as though he might finally be loosening his grip over America. He is less popular than at any point over the last decade. He is likely to experience a severe shellacking at the upcoming midterms. Soon, attention will turn to the 2028 presidential primaries.
But our collective failure to anticipate Trump’s rise has recently been on my mind in a perhaps unexpected context: As social scientists grapple with the ways in which artificial intelligence will transform our economy and our democracy, they are falling back into some of the same errors that led them to miss the imminent rise of populism: At a moment of disjunctive change, they are uncritically looking at recent data to predict what might happen in the future.
—Yascha
In the wake of the Great Recession, economists faced tough questions: How could they have missed the imminent collapse of the world economy? Of what use were all their intricate models if they couldn’t predict such a huge systemic crisis?
Just as economists failed to predict the Great Recession, so political scientists failed to predict a series of momentous political upsets. The most obvious is the dizzying rise of Donald Trump. But the story is bigger than that: In recent years, political scientists have overestimated the forces of stability time and again, failing to foresee Brexit, the chaos wrought in the Philippines by Rodrigo Duterte, and the serious threat posed to Polish democracy by its populist government, among other developments.
To be sure, it would be too much to ask political scientists to have predicted each of those twists and turns. For one, academics aren’t in the forecasting business. For another, there is such a thing as contingency: When the unlikely comes to pass, even correct models of the social world fail to see around the next historical corner.
And yet it would be a mistake to quibble away the failure of scholars to recognize that, say, one of the most seismic American political events in a half-century might be on the horizon. A year ago, most political scientists would simply have dismissed the suggestion that somebody like Trump could win the Republican nomination and prove competitive in the general election. Worse still, the dominant theories in the field actually seemed to predict just about the opposite of what eventually happened.
In one of the most respected recent works on presidential nominations, for example, Marty Cohen and his co-authors argue that, as their title has it, The Party Decides. In their view, party elites use endorsements and campaign contributions to help favored politicians in primaries, effectively preselecting approved candidates. And yet an outsider with virtually no institutional backing put up stiff resistance against Hillary Clinton in the Democratic primaries, and an even greater outsider with even less institutional backing swept the competition aside to become the Republican nominee for the presidency of the United States. In short, reality upended what we thought we knew even in areas where dominant theories did seem to make clear predictions.
How could this have happened?
Over the last three decades, political scientists have made impressive strides in understanding how the political world functions, especially in periods of relative stability. American politics is a good case in point. Scholars of the judiciary have shown to what degree the past ideological alignment of Supreme Court justices predicts their decisions in future cases. Scholars of Congress, meanwhile, can anticipate with great accuracy how representatives are going to vote on a pending bill by looking to such factors as their party ID, their past votes, or their biggest donors.
Three interlocking methodological shifts account for much of that progress.