Methodology
About the site
Maxim Lott and John Stossel have covered prediction market odds since 2007. This site is a journalistic project which aims to be an index for prediction market odds that is resistant to attempts by a single trader to move the odds. No trading takes place here; instead, this site displays liquidity-weighted averages of the many prediction markets.
Why Prediction Markets are Great
Starting in the 1600s, stock markets began to revolutionize the way society allocated resources to innovators. Today, prediction markets use the same market-driven powers to transform how we predict the future.
Predicting the future is important. Election predictions help people decide what investments to make, if those investments depend on future laws or regulations. It also can help inform individuals about what government action to expect in their own lives. It can indicate which candidates are most electable. In the future, as markets get more advanced, they may predict who will be best on metrics like economic growth, air quality, conflicts, preserving freedom, etc.
Prediction markets have also branched out into many areas, predicting climate, entertainment award winners, Federal Reserve rates, geopolitical events.
Prediction markets have been shown to be extremely accurate. They are accurate for many reasons, such as:
Incentives. Market traders who let personal biases impact their trades pay the price. Traders are incentivized to take the most clear-eyed, rational, fact-based view possible. This differs dramatically from incentives of pundits and the news media, which are optimized around entertainment, and clicks, rather than accurate predictions.
Survival of the most accurate. When inaccurate or biased traders do enter these markets, their capital is relatively quickly transferred to more accurate predictors. This leads to more accurate predictions for the public.
Wisdom of crowds. In 1907, scientist Francis Galton analyzed a survey of hundreds of villagers to guess the weight of an ox, and he noticed that their average answer was extremely accurate, even as most individuals were way off. Since then, that finding has been repeatedly replicated. Markets benefit from the same effect – wisdom from diverse people, who each may know only a few things about the issue at hand, average into extremely accurate prices. Taking the average of the crowd’s estimates is much more accurate than trusting one single “expert.”
This site is a journalistic project that aims to provide the public with a reliable prediction market index. The site simply averages different prediction markets together using a methodology intended to prevent any single trader from moving the average.
PredictionMarketOdds.com implements the following methods:
Taking the median of 60 samples within each hour
Market manipulators can spike prices for a few minutes, but it is expensive to keep odds elevated for a long time. Therefore, for each prediction market, this site takes one observation every minute, and then, each hour, posts the median observation for that hour. A would-be manipulator would have to push a price up, but typically would have to maintain it for at least half an hour. Generally, there will be 60 observations in an hour; if there are fewer than 20 in an hour, that candidate’s listing on the exchange is not used in the average for that hour. This site also varies the sampling second within each minute, rather than observing at a fixed second, making it harder for would-be manipulators to game the timing.
Liquidity Weighting of Different Prediction Markets
When an index uses a simple average, manipulators can move odds by trading on thin, easily-moveable markets. On the other hand, when trade volume is used for weighting, traders or exchanges can engage in "wash trading" to artificially inflate volume. To avoid those pitfalls, this site weights markets based on liquidity, meaning the dollar value of shares immediately available for purchase near our calculated market price. This requires would-be manipulators to put their capital at risk to move the weights. Additionally, the site does not take just momentary liquidity, but rather observes liquidity for every hour over a period of up to 30 days, and takes the median. Thus, a would-be manipulator of the weights would have to expose their capital to other traders for 16 days in order to be sure they’d shift the weights. If fewer than 30 days of historical liquidity is available, this site uses as much historical liquidity data as has been collected. After liquidity is determined for each exchange, and adjusted on a log scale (discussed below) this site multiplies each exchange’s hourly median odds by its liquidity-based weight, and then adds the results together.
Limiting the dominance of any single market
If only one market were dominant, it could constitute a massive share of the available liquidity. This could make manipulation easier, because a would-be manipulator would only have to access a single exchange. It can be harder for a manipulator to alter odds across many exchanges. That's because the markets differ; PredictIt still has caps on the amount each trader can invest in a contract ($3,500 per contract per trader) and this makes it more manipulation-resistant. UK markets also exist in a different regulatory environment and have different location, banking, and legal requirements. Averaging different sources of knowledge is more accurate at predicting the future. For the above reasons, when 3 or more markets are in the average, this site limits the weight of any one market to 35% of the total. For the same reasons, while this site gives greater weight to more liquid markets, it scales it using log(1 + liquidity / $1,000), meaning, for example, before the 35% cap is applied, $10,000 in liquidity earns about 3.5 times the weight of $1,000, not ten times.
Using all markets with serious liquidity
We exclude "bookies" who set house odds in non-transparent ways. As more markets come online and gain serious liquidity, we plan to add them.
Calculation of each market's Prediction
For each contract, this site calculates the average price one would have to pay to buy 250 contracts and, separately, the average price one would receive from selling 250 contracts, based on the exchange's orderbook. It then averages those two averages, and that is the probability recorded for the observation for that minute.
Edge cases and closing markets
We exclude contracts without enough liquidity to buy and sell 250 contracts, or with excessively wide spreads (More than 8 percentage points between the average prices to buy and sell 250 contracts).
For contracts identified as resolved, we record that source’s odds as 99.9% for a win or 0.1% for a loss, without requiring an active order book. For one-sided books, we estimate 0.1% when there are no executable bids and 250 contracts can be bought at an average price below 12¢, or 99.9% when there are no executable asks and 250 contracts can be sold at an average price above 88¢. The intention is that, as markets resolve to a yes or no, the odds reflect what’s happening.
Ensuring market odds are current
In order to ensure that odds are current, this site checks timestamps or recent order-book activity to establish freshness. Each candidate listing on an exchange needs at least 20 valid observations/ hour, and at least two contributing exchanges. Otherwise, we publish no new odds for that hour.
No guarantees
A manipulator willing to lose money could still move the odds on exchanges, and this index's methodology cannot prevent that. But the methodology is designed to make manipulation of the index harder and more expensive than it would be for any given single market. Use these odds at your own risk – this site makes no guarantees regarding uptime, accuracy, methodology, or anything else.