The Anatomy of a Presidential Forecast

To put it bluntly, predicting presidential elections is a hard business to be in. On a basic level, the number of observations is small – there have only been 19 elections since the end of World War II and the proliferation of public polling, which is less than the 30 observations required to generate a normal distribution – and the stakes are large that the backlash for incorrect prognostications is immense (see here).

Given these realities, it is important to be open and honest about how these forecasts are constructed and the limitations that they possess. In the spirit of transparency, all of the code and data used in the making of these forecasts will be publicly available on my GitHub. And before we get into the weeds, I want to give a special thanks to Nate Silver for popularizing the field of political forecasting and FiveThirtyEight for publishing the data that serves as the basis of my models, as well as G. Elliot Morris and Lakshya Jain for all the amazing work that they continue to do at FiveThirtyEight and Split Ticket

To start, I first downloaded all of the presidential polls from this cycle from FiveThirtyEight's GitHub I then restructured this data so that it would be easy to analyze. Crucially, I only kept the polls with valid data for both Biden and Trump. This restriction eliminated other potential matchups (e.g. Harris vs. Trump or Biden vs. Hayley). I additionally exclude third party candidates in my analysis for now, only keeping Biden's and Trump's share, but as the situation with ballot access becomes clearer, I'll revisit this issue.

As for the actual polling average, I decided to use an exponential decay as a clean way to incorporate all of the data, while reducing the impact of stale polls. I had considered employing a simple weighted moving average consisting of all polls conducted over the past 2 weeks or month, but given that polls contain predictive power months before an election and the dearth of polling in some races, I wanted to err on the side of inclusivity when it came to adding polls into the average.

In the actual specification, each poll is assigned a weight that equals the mathematical constant e raised to the power of a negative lambda (λ) multiplied by the number of days that have passed since the poll was last conducted. This expression is then multiplied by the square root of the poll's sample size.

Once we calculate the weights for each individual poll, we then divide the weight by the sum of all weights, such that the new, normalized weights add up to 1. To construct the averages, we simply sum the product of each candidate's support by the weight of the poll. This approach has the additional benefit of storing a value for the amount of polling at a given time – the sum of the weights – that would be useful when we add in economic indicators and fundraising numbers to future iterations of this forecast.

To determine the optimal lambda, I trekked from 0 to 1 in 10,000 steps, calculating the polling averages on election day for a set of elections from 1998 to 2023. This data was from FiveThirtyEight's GitHub and included congressional, gubernatorial, and presidential polling as well as the corresponding election results. I was seeking to maximize the R² and minimize the mean absolute error. The lambda optimized for R² was 0.0606, while the lambda optimized for mean absolute error was 0.0632. The lambda that I decided to use in the 2024 polling averages is simply the average of these two – which turns out to be 0.0619.

I want to be clear that this is a naive and datamined approach in determining the optimal lambda. There is no theoretical reasoning behind 0.0619, and as such, a healthy dose of skepticism is warranted. If anyone has a better way of determining this lambda, please feel free to let me know. The second part of the weighting function requires taking the square root of the sample size. I choose this modification as a poll's margin of error varies not with its sample size, but with the square root of its sample size. Moreover, FiveThirtyEight also makes this adjustment in their polling averages. One limitation of this approach is that I don't take into account the historical quality or bias of the various pollsters, and I do hope to add this into future versions of these averages.

Presidential elections are decided at the state-level, where victories in each state are aggregated by the Electoral College, so any proper presidential forecast must determine each candidate's standing in each state. However, not all states are polled equally, and in fact, some states are not polled at all. To get around this problem, this model makes use of the national polling averages and results of the last election. Specifically, a candidate's two-way share in the current cycle's national polling average is divided by their two-way share of the national vote in the last election in order to ascertain changes in support at the national level. This value is then multiplied by their two-way share of the vote in each state (to map the national trends to the individual states), and then multiplied by the sum of two candidate's current national polling averages in order to account for undecided and third-party voters. Finally, this calculated value is inputted as a poll conducted two weeks prior to the current date. This way if a state is frequently and recently polled, the model will prioritize the state-level polling, but when there is limited polling availability, the model will default to the state's historical partisan lean, adjusted for any nation-wide change in sentiment.

This, of course, is an ad hoc fix to this issue, but I believe it to be relatively reasonable given the data constraints. The choice of a two week lag is purely arbitrary, but I believe that it strikes a fair balance in creating an average that is responsive to new state-level polling while being accommodative to swings in national polling. Some alternative approaches include revising past state-level polls in relation to the national polling averages or even a demographic-based multilevel regression with post-stratification (MRP) that extrapolates polling in states without polling based on states with polling.

Once we are able to create state-level polling averages for each candidate, we now have to translate said averages into a probabilistic chance of victory. To do this, using the same historical dataset of polling and election results that was referenced above, we create polling averages based on the aforementioned formula (including the optimized lambda) for each race. We then compared these polling averages to the actual election results. Through this analysis, we are able to determine that the standard deviation of the difference between our estimate of the two-way margin and the actual result is 5.356 points. Using this estimate, and assuming a normal distribution, we can now convert any polling lead into a percent chance of victory.

This method is not without its drawbacks. For instance, we are comparing polling averages constructed on election day to the actual results, so we are certainly underestimating the variance that occurs between now and election day. Moreover, we aren't distinguishing between races with limited polling (like House races) and races with plentiful polling (like Presidential races), so in some sense, the variance is probably overstated. Furthermore, there is an argument to be made in favor of using a beta distribution, which is better suited for outcomes that range between 0 and 1, instead of using a normal distribution, but frankly, I have limited experience with beta distributions. On the balance, I believe my approach is a reasonable methodology to generate probabilistic forecasts for each state, but I do agree that some improvements can be made.

The next step is to aggregate all of the state-level polling data into a national model. One potential approach would be to run a standard Monte Carlo analysis, whereby I simulate the election 10,000 times, assuming that each state is independent of each other, and then tally up the results. A crucial flaw in this approach is the assumption that polling misses are independent of each other – i.e. if the polls were off in Michigan, we would also expect them to be off in Pennsylvania as well. As a remedy, I presume a 50% pairwise correlation between the polling misses, such that 50% of the variation stems from the national environment, and 50% of the variation stems from idiosyncratic state factors. This is perhaps an overly simplified approach to solving this problem, as it ignores that the correlation between certain pairs of states might be lower or higher than the 50% threshold that I set. Moreover, 50% is a purely arbitrary value, and in future versions of this forecast, I hope to better explore both empirical and theoretical improvements to this adjustment.

Now that all the nitty gritty details are sorted, we simply run the model 10,000 times, tallying up the Electoral College votes for both Joe Biden and Donald Trump, and taking count of how many times each candidate wins and how often a tie occurs. As the most recently produced forecast, Joe Biden wins 27.59% of the time, while Donald Trump wins 71.97% of the time. A tie occurs in 0.44% of all scenarios. Normally, I wouldn't feel the need to defend specific forecasts, but because the forecast is perhaps more bearish on Biden than the consensus, I thought it might make some sense to talk through the numbers.

In 2020, Joe Biden won the popular vote by 4.5%. He won Wisconsin, the tipping point state, by 0.6%, which means that if he did a tad more than 0.3% worse (and Donald Trump did 0.3% better), in all likely Joe Biden would have lost the election. Currently, Biden is on track to lose the popular vote by around 0.5%, so he's doing 5 points worse than in 2020 – an election that he barely won. If we assume that the split between the popular vote and Electoral College stays the same at 3.9% (which to be fair is a questionable assumption), then Biden would have to gain 4.4 points in the polls between now and November to have a 50% chance of winning. In this light, a slightly lower than 30% chance of victory for Joe Biden seems perfectly reasonable.

I hope you enjoyed reading this methodology post, and if you have any questions, comments, or concerns, please feel free to reach out to me via text at (832) 499-5541, email at ahv2@rice.edu, or on X (née Twitter) @aidanvyas.