xpt-lib-2025-followup: Existential Risk Persuasion Tournament Follow-Up Package

This repository contains all the code used to produce the analysis in the XPT Follow-Up report from the Forecasting Research Institute (FRI). See the origin XPT report here. Running the scripts within this repo locally requires an active internet connection and a Google Account.

Files

• R/main.R runs all the code needed to replicate the results in the report.
• The numbered scripts in R can be run separately to produce specific parts of the analysis.
• QuestionInfo.csv contains full question text, resolution criteria, and clarifications that were issued to forecasters. For some details, you must refer to the XPT report (pdf).
• Data (loaded from the original xpt-lib repository)
  • expertsG1_anon.csv: user IDs for the experts and their specialty (or specialties).
  • supers_anon.csv: user IDs for Superforecasters
  • questionMetadata.csv: metadata for the forecasting questions (resolution year, question type, answer units, etc)
  • forecasts_anon.csv: forecasts from experts and Superforecasters
  • The public-survey directory contains the responses from public surveys, which were put out in many parts
  • survey_column_matches.csv maps the questions from the public survey (column names in public_supplement files) to the corresponding questions in the main tournament
• The local data folder contains
  • naive_extrapolation.csv: Extrapolated forecasts to each resolved question derived by projecting forward from the last available data using a constant growth rate
  • naive_no_change.csv: Simple forecasts reporting the most recent historical base rate available at the time of the tournament
  • resolutions.csv: Table with question resolutions

Notes

Note that the platform we used to host the XPT occasionally had a small amount of measurement error in tracking when forecasts were submitted. For instance, on questions where we asked the user to submit not just their median forecasts but forecasts of several quantiles (the 5th, 25th, 75th, and 95th percentiles), the platform occasionally registered these forecasts as submitted at slightly different times, a few seconds apart. Our code addresses these errors, but in a handful of cases this caused us to exclude valid forecasts from our analysis.

We do not provide specific timestamps (only calendar dates) in this data release to ensure forecaster anonymity. So, to replicate the results we released in the report as closely as possible, we’ve replaced timestamp-related variables with the variables forecastid, stage, and timestampid, and altered our code to use these proxy variables in place of regular timestamps. 'forecastId' marks sets of forecasts (eg. multiple quantiles) that our original code treated as a single forecast, 'stage' identifies the stage of the tournament that the forecast was submitted during, and 'timestampId' replaces all timestamps registered by the platform with an ordinal number.

There are a few minor discrepancies between the results we produced in the original report and those you will find if you rerun our programs using this data release. These discrepancies are the result of how different statistical programs store information about very large numbers. Feel free to contact us if you have any questions about discrepancies between the report and this data release.

Questions

Please feel free to contact zach@forecastingresearch.org with any questions about the tournament or the analysis herein, or if you wish to contribute.