Unsurprisingly, a new paper looked at the forecast accuracy of more than one million prediction questions from Polymarket, Kalshi, Manifold and the Metaculus forecasting platform. The beauty of the new analysis is that it covers a wide range of areas from business to elections to geo-politics. An event predicted to happen p% of the time happens about p% of the time. Can’t beat it.
Using approximately one million contracts and solved queries across four collective forecasting platforms, twelve domains, and time horizons from hours to years, we evaluate the calibration and accuracy of collective forecasts in real-world settings. We found that the forecast probabilities were overall highly aligned: events assigned the probability of pp% occurrence is approx pp% time. This broad pattern is evident across platforms and many domains, although detailed analysis reveals a systematic tendency for overestimation or underestimation in areas of particular importance. While this does not mean that forecasting platforms provide certainty about individual events, our findings suggest that aggregation of large-scale assessments contains useful information about real-world uncertainty; when examined in aggregate, the predictions made by the platform provide an informative estimate of the probability of an event.
The post Accurate Prediction Market appeared first on Marginal REVOLUTION.