Brier Score

Keeping a forecast record lets you evaluate the confidence behind your predictions once the results are known. For a game or an interest-rate decision you follow, you can record the probability you assign to each outcome and later compare it with what happened. The Brier score summarizes the errors across those forecasts, giving more weight to confident mistakes.

That score becomes more useful alongside a reference forecast on the same questions. An estimate based on how often an event happens provides a simple benchmark, and market prices offer another comparison. Scoring both sets of probabilities lets you see whether your estimates improved on the reference, even when you favored the same outcomes.

What is a Brier score?Copy Anchor Link

A Brier score measures how close probability forecasts came to the outcomes, with lower scores indicating smaller errors. For a Yes/No question, it squares the difference between the probability assigned to Yes and the result, coded as 1 for Yes or 0 for No. The score for a forecasting record is the average of those squared errors.

The score lets you compare forecasters who pick the same winner but have different confidence in that choice. Someone who gives a team a 55% chance of winning leaves almost as much probability for a loss. If the team loses, that forecast receives an error of 0.3025, compared with 0.9025 for someone who assigned a 95% chance of winning.

The team loses, an outcome coded 0

55% on a win

Loss, 0Win, 1

Gap0.55Squared error0.3025

95% on a win

Loss, 0Win, 1

Gap0.95Squared error0.9025

Squared errorLargest possible error, 1

Calculated this way, the score ranges from 0 to 1. A score of zero requires assigning 100% to the outcome that happens on every question. Even accurate estimates of uncertain events produce errors when the less likely outcome happens.

Glenn W. Brier introduced the score in a 1950 paper on weather forecasts. It has since been used to evaluate forecasting tournament participants and machine-learning models that predict probabilities. Traders can use the same calculation to compare their estimates with market prices interpreted as probabilities.

How do you calculate a Brier score?Copy Anchor Link

To calculate a Brier score, express each probability as a decimal, subtract the outcome, square the difference, and average the results. A 70% forecast enters the calculation as 0.70. Code the outcome as 1 if the event happened and 0 if it did not.

The binary Brier score formula is:

$$\text{Brier score} = \frac{1}{N}\sum_{i=1}^{N}(p_i-o_i)^2$$

Here, N is the number of forecasts, p is the probability assigned to Yes, and o is the observed outcome. The small i identifies each forecast in the record.

Suppose you give the Lakers a 70% chance of winning a game. If they win, the squared error is:

$$(0.70-1)^2 = 0.09$$

If they lose, it is:

$$(0.70-0)^2 = 0.49$$

Across several games, each forecast contributes one error to the average. Suppose you record these probabilities before four Lakers games:

The errors add to 0.53. Dividing by four gives a Brier score of 0.1325. Game 2 contributes most of the total because you favored a win and the team lost.

Squared error by gameSum0.53

1234

AverageBrier score0.1325

0.53 ÷ 4

A spreadsheet can serve as a Brier score calculator. With probabilities in column B and outcomes in column C, enter =(B2-C2)^2 in the error column and fill it down. Then take the average of those errors. A cell entered as 70% already stores 0.70. If you entered the number 70 instead, divide it by 100 before calculating the error.

With squared error, reporting the probability you believe minimizes your expected score. Doubling the distance between a forecast and the outcome quadruples its contribution to the score.

Squared error

0.0900.090.36

Game 1Game 2

00.30.6

Distance between forecast and outcome

What is a good Brier score?Copy Anchor Link

A good Brier score is lower than a relevant reference forecast's score on the same questions. On the binary 0 to 1 scale, predicting 50% every time scores 0.25. A constant forecast based on how often events happen can score much lower, making it a more demanding reference.

Suppose 20 of 100 companies in a hypothetical sample announce a product by a specified deadline. Predicting 20% for every company produces an error of 0.64 on each announcement and 0.04 on each non-announcement:

$$\frac{20\times0.64+80\times0.04}{100}=0.16$$

That 0.16 score comes entirely from using the overall event rate, also called the base rate, without identifying which companies would announce a product.

A constant forecast matching the record's overall Yes rate r scores r times (1 minus r). Here is how that reference changes with the event rate:

An event rate of 90% gives the same 0.09 reference score as 10%. The reference also helps judge machine-learning models when classes are imbalanced. On a dataset where only 5% of cases are positive, predicting 5% for every case already scores 0.0475. A model scoring 0.06 on that dataset has more squared error than the constant forecast.

Brier score of a constant forecast

0.047500.04750.090.160.210.25

5%10%20%30%50%90%

0%50%100%

Share of events that happen

The table uses the event rate in the completed record, making it a retrospective benchmark. A baseline you could actually have forecast would use information available beforehand, such as the announcement rate in an earlier sample. Reference class forecasting explains how to choose a relevant group of past events for that estimate.

Interpreting scores of 0.1 and 0.2Copy Anchor Link

For binary forecasts reported on the 0 to 2 scale, divide the score by two before using this table.

What is a Brier skill score?Copy Anchor Link

The Brier skill score measures the reduction in squared error compared with a reference forecast. A skill score of 0.25 means 25% less error than the reference. Zero means an equal score, and a negative Brier skill score means more error than the reference.

Divide the forecast's score by the reference's score, then subtract that ratio from 1:

$$\text{Brier skill score}=1-\frac{\text{forecast Brier score}}{\text{reference Brier score}}$$

Suppose your score is 0.12 and the reference scores 0.16. Your skill score is 1 minus 0.12 divided by 0.16, or 0.25. Against a reference scoring 0.09, the same 0.12 score has roughly one-third more squared error and gives a skill score of about −0.33.

Against a 0.16 referenceSkill score0.25

Reference0.16

Your forecasts0.12

0.04 less

Against a 0.09 referenceSkill score−0.33

Reference0.09

Your forecasts0.12

0.03 more

Reference's Brier scoreYour Brier score

Brier score vs Brier skill scoreCopy Anchor Link

The raw Brier score measures error, so lower is better. The skill score measures improvement over a reference, so higher is better. It cannot exceed 1, but it has no fixed lower limit. It is undefined if the reference score is zero because the calculation would divide by zero.

A historical base rate tests whether your estimates add information beyond the frequency of similar past events. To assess your forecasts against the market, use the price available when each forecast was made as the reference.

Why is the Brier score a proper scoring rule?Copy Anchor Link

A proper scoring rule gives you the best expected score when you report the probabilities you actually believe. With a strictly proper scoring rule, any other report worsens the expected score. Brier and logarithmic scoring both have this property, so exaggerating your confidence worsens your expected result.

Suppose your assessment is that Bitcoin has a 70% chance of finishing a specified day above a stated price. To work out the expected score, weight the two possible errors by the chances you assign to them:

The middle row's expectation is 0.70 times 0.09, plus 0.30 times 0.49, giving 0.21. Reporting 90% helps if Yes happens, but the larger error if No happens outweighs that advantage in your expected score.

Your belief: 70% that the event happens

Report 50%Expected error0.25

If Yes0.70 × 0.25 =0.175If No0.30 × 0.25 =0.075

Report 70%Expected error0.21

If Yes0.70 × 0.09 =0.063If No0.30 × 0.49 =0.147

Report 90%Expected error0.25

If Yes0.70 × 0.01 =0.007If No0.30 × 0.81 =0.243

The same result holds for any probability. If p is the probability you believe and q is the probability you report, the expected score is:

$$\text{Expected Brier score}=(q-p)^2+p(1-p)$$

The first term is zero when q equals p and positive for any other report. The second term stays the same however you report because it comes from the uncertainty in the event itself.

Expected error under a 70% belief

0.200.200.250.30

50%70%90%

Probability you report

(q − p)² + p(1 − p)p(1 − p) = 0.70 × 0.30 = 0.21

Even someone who knows that each event has a true 70% chance expects a score of 0.21 when reporting those probabilities accurately. The remaining 30% chance of a No outcome accounts for the larger errors that occur despite an accurate estimate.

Does the Brier score measure calibration?Copy Anchor Link

Calibration means that events assigned a given probability happen about that often. The Brier score penalizes mismatches between those probabilities and frequencies, while rewarding forecasts that separate more likely events from less likely ones. A low score can therefore accompany miscalibrated probabilities.

Imagine forecasting whether a selected team will win each of 100 basketball games. The selected team wins 50 of them. Predicting 50% for every game is calibrated across this record and gives a Brier score of 0.25.

Now suppose you can separate those games into two groups. In 50 games, the selected team is an underdog and wins 10 times. In the other 50, it is a favorite and wins 40 times. Forecasting 20% for the first group and 80% for the second matches the observed frequencies within both groups. The Brier score falls to 0.16.

Forecasting 15% for the underdogs and 85% for the favorites exaggerates the difference between those same groups and gives a score of 0.1625. That improves on the 0.25 from predicting 50% throughout, even though the probabilities are miscalibrated. Predicting 20% and 80% still does better, at 0.16.

50% on every gameBrier score0.25

All games

20% and 80%Brier score0.16

UnderdogsFavorites

15% and 85%Brier score0.1625

UnderdogsFavorites

0%20%50%80%100%

ForecastShare of the group's games wonGap

The Brier score decomposition, often called the Murphy decomposition, describes these contributions as:

$$\text{Brier score}=\text{reliability}-\text{resolution}+\text{uncertainty}$$

The three records show what each term contributes:

• Reliability measures the mismatch between forecast probabilities and observed frequencies. The calibrated records have zero mismatch. The 15%/85% record adds 0.0025 for missing each group's frequency by five percentage points.
• Resolution measures how much event frequencies differ across forecast groups. Both two-group records separate the same 20% and 80% win rates, reducing their scores by 0.09. The constant 50% record has no such separation. This differs from market resolution, the process of deciding a contract's outcome.
• Uncertainty comes from the overall event rate. It stays at 0.25 for all three records because the selected teams win half the games.

The difference between 0.16 and 0.1625 comes entirely from the reliability penalty. A calibration curve shows where those mismatches occur in a personal record, such as 80% forecasts succeeding only 60% of the time.

How does the Brier score compare with log loss?Copy Anchor Link

Log loss penalizes a confident error more heavily as the probability assigned to the actual outcome approaches zero. Its penalty keeps growing without limit, while a binary Brier error is capped at 1.

For one resolved question, log loss is minus the natural logarithm of the probability assigned to the outcome that occurred. In classification, this calculation is also called cross-entropy loss. If you forecast a 70% chance of Yes, a Yes result gives a log loss of about 0.357. A No result uses the remaining 30% and gives about 1.204.

The difference becomes especially visible when an unlikely event happens:

Moving from 5% to 0.1% increases the Brier error by about 0.096. Log loss more than doubles. A sequence of extremely confident errors can therefore affect comparisons much more under log loss.

Binary Brier error1 at 0%

0.500.51

0.25 at 50%0.9025 at 5%0.998001 at 0.1%

100%10%1%0.1%

Probability given to what happened

Log lossInfinite at 0%

00246

0.693 at 50%2.996 at 5%6.908 at 0.1%

100%10%1%0.1%

Probability given to what happened

Good Judgment Open uses Brier scoring, while Metaculus builds its scores on the log score. The log score takes the logarithm without the minus sign, so higher is better. Metaculus restricts binary forecasts to 0.1% through 99.9%, keeping the penalty finite even when a forecaster is as confident as the platform permits.

Is the Brier score the same as MSE?Copy Anchor Link

The binary Brier score is mean squared error, or MSE, applied to probabilities and 0-or-1 outcomes. For these inputs, the two calculations give exactly the same number. MSE also applies to other predictions, such as the temperature at the end of a day.

Brier score vs AUC, accuracy, ECE, and RMSECopy Anchor Link

Accuracy counts correct classifications, while AUC measures how well predictions rank cases. These measures answer different questions about the same forecasts:

For a simple accuracy comparison, suppose two forecasters always choose the eventual winner in four games. One assigns each winner 55% and the other assigns 80%. Both have a 100% hit rate on those four games. Their Brier scores are 0.2025 and 0.04, respectively.

55% on each winner

Hit rate4 of 4

Brier score0.2025

80% on each winner

Hit rate4 of 4

Brier score0.04

Brier score of a constant 50% forecast, 0.25

If their favored team loses a later game, both picks count as wrong, but the 80% forecast receives an error of 0.64 and the 55% forecast receives 0.3025. The probabilities determine how much each result changes the overall score.

Why do some Brier scores range from 0 to 2?Copy Anchor Link

Some scoring systems add the squared errors for every possible outcome. On a binary question, that counts both Yes and No and produces exactly twice the score obtained by counting Yes alone. The resulting scores range from 0 to 2, with zero still best.

For a 70% Yes forecast that resolves Yes, the binary error is 0.09. Summing both options gives:

$$(0.70-1)^2+(0.30-0)^2=0.18$$

Good Judgment Open sums the errors across answer options. A score of 0.18 there can describe the same binary forecasts as a score of 0.09 on the 0 to 1 scale. Dividing by two converts between the scales for the same binary record. Constant 50% forecasts score 0.50 under Good Judgment Open's convention.

A 70% forecast of Yes, and Yes happens

Yes, coded 1Squared error0.09

70%

No, coded 0Squared error0.09

30%

01

Yes only, 0 to 1 scaleScore0.09

Both options, 0 to 2 scaleScore0.18

ForecastOutcomeGap

Calculating errors for multiclass forecastsCopy Anchor Link

For a question with more than two possible outcomes, multiclass scoring adds the squared errors for all of them. Exactly one outcome must occur, and the probabilities must add to 100%. Suppose a central bank can cut rates, leave them unchanged, or increase them. You assign those possibilities 50%, 30%, and 20%. If rates stay unchanged, the error is:

$$(0.50-0)^2+(0.30-1)^2+(0.20-0)^2=0.78$$

The outcome that happens gets 1 and the others get 0. For a record of several questions, calculate each question's total error this way, then average those totals.

Rates stay unchanged

Cut, coded 0Squared error0.25

50%

Unchanged, coded 1Squared error0.49

30%

Increase, coded 0Squared error0.04

20%

01

Sum, 0 to 2 scaleScore0.78

ForecastOutcomeGap

Equal probabilities across three outcomes give a score of two-thirds on the 0 to 2 scale, regardless of which occurs. With more possible outcomes, spreading probability equally puts less on the one that happens and raises this reference score.

For machine-learning evaluations, scikit-learn documents a default range of 0 to 1 for binary tasks and 0 to 2 for multiclass tasks. A multiclass result from that implementation therefore uses the summed errors in this example.

What do Polymarket and Kalshi Brier scores mean?Copy Anchor Link

The Polymarket Brier score and Kalshi's reported scores measure the average squared difference between market prices, treated as probabilities, and settled Yes/No outcomes. Contracts priced near their eventual outcomes contribute small errors, so scoring near close or including many near-certain contracts can bring the average far below 0.25. Averaging errors across an event's separate contracts also gives a smaller number than summing them for the full event.

A venue score can fall below the 0.096 reported for combined superforecaster predictions in ForecastBench's 2025 study because the two evaluations use different questions and forecast times.

The Polymarket accuracy pageCopy Anchor Link

The Polymarket accuracy page says its metrics use resolved markets and price snapshots from one month, one week, one day, twelve hours, and four hours before resolution. Its methods text leaves the headline Brier score's snapshot unspecified. The published number therefore cannot be matched to a particular forecast time from that explanation.

The page also shows the shares of markets resolving Yes and No. The Yes share gives an event rate for a retrospective base-rate reference. If the same markets and weights enter both calculations, forecasting that share on every contract supplies a more relevant benchmark than assuming half resolve Yes.

What changes Kalshi accuracy figuresCopy Anchor Link

Kalshi's August 2026 working paper reports a Brier score of about 0.087 three months before close. The score rises to 0.126 at one day as short-lived markets enter the sample, then falls below 0.02 at close. The calculation excludes combination contracts and uses the contracts with prices available at each forecast time.

Restricting the one-day calculation to 35,562 markets already open a month ahead reduced the score from 0.126 to 0.039. The authors attribute the gap to the sports and crypto markets that open in their final day, when their outcomes remain harder to forecast.

Excluding near-certain prices also raised the score. In a sample without sports, mentions, or intraday markets, keeping only prices above 2 cents and below 98 cents raised the close-time score from 0.030 to 0.077.

The date used to measure time remaining mattered too. For elections, measuring one month before election day instead of one month before contractual close raised the score from 0.045 to 0.079. Some contracts closed weeks after the vote, so the original clock counted prices recorded with more information available.

Counting events or individual contractsCopy Anchor Link

Imagine an award with five nominees and exactly one winner. Their probabilities of winning are 60%, 20%, 10%, 6%, and 4%. If the favorite wins, the five binary contract errors sum to 0.2152 for the event's full forecast. Dividing by five gives a per-contract average of 0.04304. The four losing nominees contribute small errors because they were assigned low probabilities of winning.

Five contractsSum for the event0.2152

Winner at 60%0.1620%0.0410%0.016%0.00364%0.0016

Per contractAverage0.04304

0.2152 ÷ 5

Kalshi's paper averages over individual contracts, which corresponds to 0.04304 in this example and gives more weight to events with more contracts. Polymarket's methods text leaves its unit of averaging unspecified.

Matching contracts and pricesCopy Anchor Link

A comparison of forecasting accuracy across venues needs contracts asking the same question, with prices recorded at the same time before the event. Two markets found through a search of titles and resolution rules can concern the same Apple product launch but ask different questions. One might settle Yes on an announcement by a deadline, while another requires the product to be available for sale. Matching contracts means aligning the event, threshold, deadline, and settlement criteria, including which source decides the result.

The price definition matters as well. A last trade can differ from the midpoint between the best buy and sell offers in an order book. Using the same definition for both venues keeps a difference between price measures out of the forecast comparison.

Even at the same time before close, contracts can differ in how much uncertainty remains. A contract already priced near its eventual outcome contributes less error than one still trading around 50%.

Yes priceBoth resolve Yes

100%0%50%100%

3070

Days before close

7 days before closeSquared error

Uncertain until late7 days before close55%Squared error0.2025

Near certain for weeks7 days before close95%Squared error0.0025

What do published superforecaster Brier scores measure?Copy Anchor Link

In the 2025 ForecastBench paper, superforecasters' median predictions scored 0.096, compared with 0.121 for the general public's median predictions. These were forecasts aggregated across participants on a 200-question subset across several horizons. The 0.096 is the score of the combined forecasts, rather than an average participant's score.

The same group's median predictions scored 0.001 in Healthcare and Biology and 0.175 in Science and Tech, with 48 and 14 scored entries respectively. These category results show how much the question set changes the score.

The study used the binary 0 to 1 scale. For resolved Yes/No questions on that scale, constant 50% forecasts score 0.25. For unresolved market questions, however, ForecastBench measured distance to a later community probability. Matching that probability gives zero error even while the outcome remains uncertain. Its reported score therefore combines errors against settled outcomes with agreement with later forecasts.

The earlier Good Judgment study by Mellers and colleagues used the 0 to 2 scale, which doubles raw binary scores. Its main comparisons also standardized scores within each question before averaging because participants chose which questions to answer. Those standardized results measure relative performance on each question, rather than raw squared error. The superforecaster benchmarks show how individual scores changed between the first and final weeks of questions.

What do Good Judgment Open and Metaculus scores mean?Copy Anchor Link

Good Judgment Open's Brier scores measure squared probability errors, with lower scores better. The Metaculus Peer score compares logarithmic scores with other participants, with higher scores better. Their score names refer to these different calculations:

On Good Judgment Open, your standing forecast continues to receive a daily score until you update it or the question closes. Metaculus also weights forecasts by how long they stand.

A negative relative Brier score on Good Judgment Open means you performed better than its comparison group. A negative Brier skill score means you performed worse than the chosen reference, because the skill-score formula measures the reduction in error.

Relative Brier ScoreGood Judgment Open, against the crowd

Less errorMore error

Negative0Positive

Brier skill scoreAgainst a reference forecast

More errorLess error

Negative, no limit01

How can you track your own Brier score?Copy Anchor Link

To track your Brier score, record each probability before the answer is known and score it after the question resolves. A spreadsheet can hold your forecasts, outcomes, and errors. For a comparison with market prices, pair forecasts and prices from the same times and apply the same outcomes and averaging rule to both.

Each entry needs enough information to reconstruct what you forecast:

Use the Brier score calculation in separate error columns for your probabilities and the reference probabilities. For forecasts you update, the average depends on whether you count each submission, each day, or only the final estimate.

ForecastResolves Yes

60%, error 0.1690%

Each day countsScore0.145

Each submission countsScore0.085

Final forecast onlyScore0.01

Day 1Day 10

Suppose a forecast stands at 60% for nine days, rises to 90% for the last day, and then resolves Yes:

You can also score a single forecast from seven days before each event, using questions for which a forecast existed then.

If you score every day, first average the daily errors within each question. Averaging those question scores then gives every question equal weight. Pooling all the daily errors instead gives longer-running questions more weight because they contribute more rows.

Keep the number of resolved questions beside the average. In a ten-question record, a 90% forecast resolving No instead of Yes changes the average score by 0.08.

Does a lower Brier score mean more profit?Copy Anchor Link

A lower Brier score means smaller probability errors under the scoring rules used. Trading profit also depends on the price paid, position size, transaction costs, and when a position is closed. Those inputs do not enter the Brier calculation.

Suppose you estimate an 80% chance that an event contract will settle Yes and return one dollar. Buying at 60 cents gives an expected value of 80 cents minus 60 cents, or 20 cents before costs, if your estimate is correct. Buying at 90 cents gives an expected loss of 10 cents under the same estimate.

Your 80% probability receives exactly the same Brier score in either case, while the entry price changes the economics of the position. Comparing your estimate with the price's implied probability answers a different question from scoring the forecast after resolution.

Buy at 60¢Expected profit+20¢

60¢ to 80¢

Buy at 90¢Expected profit−10¢

80¢ to 90¢

Yes, pays $180%No20%Price paid

Squared error at either priceIf Yes0.04If No0.64

The Polymarket leaderboard ranks traders by profit and loss. Those results depend on trade prices and position sizes as well as forecasts. Keeping a forecast record alongside a trading record lets you examine whether an error came from the probability estimate, the price paid, or another part of the trade.