Wisdom of Crowds
The wisdom of crowds explains how combining forecasts can improve accuracy and why shared mistakes can survive even in a large group.
When people disagree about what will happen, it can be difficult to know whose forecast to trust. One person may understand a part of the story that others have missed, while overlooking evidence they have seen. Combining their estimates offers a way to use that scattered knowledge without first having to identify the most accurate forecaster.
Crowd wisdom depends on how people form and combine their judgments. An average can draw on separate assessments from people who never reach agreement, while an average of copied predictions simply repeats the original guess. In prediction markets, people's ability and willingness to trade also affect how much influence their judgments have on the price.
What is the wisdom of crowds?
The wisdom of crowds is the idea that combining judgments from several people can produce a more accurate answer than relying on one person. Each contributor knows something and misses something. When their mistakes differ, pooling their estimates lets overestimates and underestimates offset one another.
People can submit answers separately, with an average or another rule producing the group's estimate afterward. The improvement over individual judgments is called the wisdom of the crowd effect. It is a form of collective intelligence that can work without the participants ever meeting or reaching agreement.
For example, people forecasting a football match might follow different teams. Someone who knows about the home team's injuries may give it a lower chance of winning than someone who has watched the visitors struggle against its formation. Both assessments contribute to the average forecast.
An injury report and a tactics sheet give forecasters different pieces of evidence about the same football match.
What are some examples of the wisdom of crowds?
Examples of crowd wisdom include estimates of an ox's weight and the number of jelly beans in a jar, forecasts of award winners and sports draft positions, and employee markets on company outcomes.
| Example | How judgments were combined | Observed result |
|---|---|---|
| Galton's ox contest | Mean of 787 weight estimates | Matched the ox's dressed weight to the nearest pound, using the corrected archival figures. |
| Columbia jelly bean experiment | Mean of 73 guesses | Came closer than 71 of the students in Mauboussin's 2007 classroom exercise. |
| Columbia Academy Awards contest | Most popular prediction in each category | Got 11 of 12 winners right; the best students got 9 in the same year's awards exercise. |
| 2005 NFL draft | Average of seven experts' predicted draft positions | Had less total squared error than every expert for the first 12 picks in Scott Page's comparison. |
| Corporate markets at Google, Ford, and an unnamed company | Prices from employee trading | Improved on the firms' expert forecasts, with reductions in mean squared error of up to 25% in Cowgill and Zitzewitz's study. |
Galton's ox experiment
At a livestock exhibition in Plymouth, entrants guessed how much an ox would weigh after slaughter and dressing. Francis Galton's 1907 analysis of 787 entries found that combining their guesses gave a close estimate. The entrants included experienced butchers and farmers.
The actual dressed weight was 1,197 pounds. The crowd's average estimate also rounded to 1,197 pounds, while its median was 1,208 pounds, an error of 11 pounds. The median is the middle estimate after sorting all the answers. These figures follow Kenneth Wallis's examination of Galton's archives.
In the corrected figures from Galton's ox contest, the rounded mean of 787 guesses matches the dressed weight of 1,197 pounds. The median guess is 11 pounds higher at 1,208.
Galton's original report printed the weight as 1,198 pounds and the median as 1,207 pounds, the figures many retellings still use. Wallis traced those differences to errors in the published account. Galton reported the mean in a later reply to a reader.
The jelly bean experiment
Investment strategist Michael Mauboussin asked 73 Columbia Business School students to guess how many jelly beans were in a jar in 2007. Their average guess was 1,151, against an actual count of 1,116. Only two students came closer than the average, according to his report and published guesses.
Of the 73 students guessing a jelly bean count, only the two highlighted came closer than the class average. That average was 1,151, just 35 beans above the actual count of 1,116.
Students submitted their guesses independently, so each answer reflected an assessment made before seeing classmates' numbers.
How does the wisdom of the crowd work?
Averaging lets overestimates and underestimates offset one another. Suppose four people forecast that a basketball team will score 80, 100, 110, and 130 points. In this hypothetical game, the team eventually scores 100.
| Forecaster | Predicted score | Error compared with the final score |
|---|---|---|
| A | 80 | 20 points too low |
| B | 100 | Exactly right |
| C | 110 | 10 points too high |
| D | 130 | 30 points too high |
| Average | 105 | 5 points too high |
Adding the forecasts and dividing by four gives 105. That estimate is closer than three of the four individual predictions. Nobody had to identify the accurate forecaster beforehand.
Four hypothetical basketball forecasts of 80, 100, 110, and 130 points average to 105. The low estimate partly offsets the high ones, leaving the average five points above the final score of 100 and closer than three of the four predictions.
What is the diversity prediction theorem?
The diversity prediction theorem states that the squared error of a crowd's average equals the average individual squared error minus the spread of the predictions around their own average. In Scott Page's formulation, spread means the average squared distance from the group's mean. Squaring an error means multiplying it by itself, so misses in either direction count toward the total instead of canceling out.
For the basketball forecasts, the individual squared errors are 400, 0, 100, and 900, averaging 350. The squared distances from 105 are 625, 25, 25, and 625, averaging 325. Subtracting 325 from 350 leaves 25, the squared error of the group's five-point miss.
Averaging the four hypothetical basketball forecasts reduces squared error from an individual average of 350 to 25. The reduction of 325 is the average squared distance between each forecast and the group's mean of 105.
This identity holds for any set of numerical estimates. It guarantees that the average's squared error is no greater than the average individual squared error. That comparison says how large the errors are, rather than how many people the crowd beats. With forecasts of 99, 99, 99, and 143 for a final score of 100, the average is 110. It loses to three people, yet its squared error of 100 is below the average individual squared error of 463.
For a hypothetical final score of 100, the forecast of 143 raises the average individual squared error to 463. The combined forecast of 110 has a smaller squared error of 100, even though the three forecasts of 99 each miss by only one point.
More disagreement alone need not improve the forecast. Moving the lowest and highest basketball predictions equally farther apart increases both the spread and the individual errors, leaving the average unchanged.
Hong and Page modeled people finding solutions together, asking whether varied approaches could outperform selection for individual ability. Their “diversity trumps ability” result depends on model assumptions and remains disputed. The diversity prediction theorem applies to a different task, combining numerical estimates the contributors have already made.
What are Surowiecki's conditions for crowd wisdom?
James Surowiecki's four conditions for crowd wisdom are diversity of opinion, independence, decentralization, and aggregation. His 2004 book The Wisdom of Crowds popularized the idea that a group can reach an accurate answer using knowledge scattered among its members.
Consider people forecasting whether a new phone will reach customers before the holiday shopping season.
The unfinished phone, empty packaging, and software work on the laptop belong to the same launch. People responsible for different parts of that work can contribute different evidence about whether the phone will ship on time.
| Condition | What it contributes | What goes wrong without it |
|---|---|---|
| Diversity of opinion | A supplier follows component shipments while a software specialist assesses unfinished work. | Everyone relies on the announcement date and overlooks other reasons for delay. |
| Independence | Each person forms a judgment from the evidence they have. | Later contributors copy a popular forecast and suppress their own doubts. |
| Decentralization | People close to different parts of the project use their local knowledge. | A central planner assumes every supplier and team faces the same constraints. |
| Aggregation | A defined rule combines the estimates into one forecast. | Useful observations remain scattered, or the loudest participant determines the answer. |
People with different sources of knowledge can still lose their independence. Several specialists might work in different places yet all copy the same analyst. Their geographic separation then does little to prevent social influence from making their forecasts similar.
A supplier, software specialist, and retail buyer put a hypothetical phone's chance of shipping before the holidays at 40%, 55%, and 70%, averaging 55%. When all three copy one analyst, their average becomes the analyst's 80%, with none of their separate evidence contributing.
Everyone also needs to answer the same question. An announcement, the first shipment, and widespread availability can happen on different dates. Averaging estimates of those dates mixes several targets, just as comparing contracts with different resolution criteria mixes different outcomes.
Is crowd wisdom just the law of large numbers?
The law of large numbers explains part of crowd wisdom. With independent guesses drawn from a stable population, adding more people makes the average settle near that population's average guess. It approaches the correct answer only when the population's errors balance around the truth.
Suppose forecasters tend to overestimate a team's score by five points. A larger crowd's average settles around a forecast five points too high. More participants make the answer more consistent while leaving that bias intact.
For these hypothetical crowds, the range containing 95% of average forecasts is 95.2 to 114.8 points with four people and 104.8 to 105.2 with 10,000. The narrower range still centers on 105, five points above the final score of 100.
Shared errors also limit the gain. If everyone copies the same forecast, averaging their submissions returns that forecast however many people join. People who consult some of the same sources have partly shared errors, so adding them still helps, but less than adding equally accurate people with independent evidence.
Suppose individual errors have a standard deviation of 10 points, a measure of how widely they vary across repeated forecasts. If those errors are independent and centered on zero, the average of 100 forecasts has an error standard deviation of 1 point. If every pair of forecasters' errors has a correlation of 0.25, meaning their errors tend to move together, the corresponding figure is about 5.07 points. Increasing that second crowd to a million people brings it only to about 5 points.
In this hypothetical example, averaging 100 forecasts reduces the error standard deviation from 10 points to one when errors are independent. With a correlation of 0.25 between every pair of errors, it falls only to about 5.07 points, and a million forecasts still leave about five.
The diversity prediction theorem describes the error of an existing set of estimates. The law of large numbers describes what happens as more observations arrive under its assumptions. A small group can benefit from averaging, and a very large one can retain a shared mistake.
What does the Condorcet jury theorem say about majority voting?
The basic Condorcet jury theorem describes majority voting on a question with two possible answers. If voters decide independently and each has the same probability of being right above 50%, adding voters makes the majority more likely to be correct. As an odd-numbered group grows, that probability approaches 100%.
If voters independently favor the wrong answer more often than the right one, adding voters strengthens the wrong majority. Voting also gives a narrow preference and near certainty one vote each, discarding the difference in confidence.
For a question with two possible answers, 101 independent voters give the majority about a 98% chance of being right if each voter's chance is 60%. If each voter's chance is 40%, the majority is right only about 2% of the time.
Is the wisdom of the crowd real?
Yes. In a 2019 study, the crowd's answers were more consistently accurate across questions than individual participants' answers. Camelia Simoiu and colleagues collected more than 500,000 answers to 1,000 questions across 50 subject areas. The crowd ranked ahead of about 65% of individual responses on average across questions, and about 85% of individuals when performance was combined across a subject.
About 65% of individual responses were less accurate than the crowd's answer, on average across 1,000 questions in Simoiu and colleagues' study. When accuracy was assessed over a whole subject, about 85% of participants ranked below the crowd.
The researchers used the median for numerical answers and the most popular answer for categorical questions. The crowd's rank varied widely, and it fell below the middle of the group on some questions.
Most questions tested knowledge or estimation, while five of the 50 subject areas asked for probabilities of future events such as election and sports results. Those forecasts were assessed after the outcomes became known. One way to compare probability forecasts with outcomes is a Brier score, which penalizes confident mistakes more heavily than cautious ones.
Does averaging forecasts beat the best forecaster?
A combined forecast can beat most contributors without beating the best one. Its advantage is that the best forecaster often changes between questions, while an average draws on several people's judgments. A record of accurate forecasts offers a way to choose a stronger group before the next question arrives.
A different forecaster comes closest in each of these three hypothetical basketball games, while the average forecast ranks second each time. Adding up the misses puts the average first overall at 10 points, compared with 30 to 40 for the four individual forecasters.
In a study of select crowds, Albert Mannes, Jack Soll, and Richard Larrick tested a strategy of ranking people using evidence of ability, such as recent accuracy, and averaging a small group near the top. Groups of five performed well across simulations and 90 archival datasets.
Track-record weighting has also helped in forecasting tournaments. In Atanasov and colleagues' study of 261 geopolitical events, market prices beat a simple average of probability forecasts. Team forecasts beat the markets when researchers gave more weight to recent estimates and stronger records, then adjusted the combined probabilities using past results. Such adjustments can include extremizing, which moves a combined probability farther from 50%.
How should you combine a crowd's estimates?
An arithmetic mean gives every numerical estimate equal weight. A median is useful when a few extreme guesses would dominate that mean. When contributors have comparable forecasting records, selection or weighting can give better judges more influence.
| Method | How it combines answers | When it is useful |
|---|---|---|
| Arithmetic mean | Adds estimates and divides by their count. | A straightforward baseline for quantities or probabilities, especially when estimates occupy a similar range. |
| Median | Takes the middle ordered answer, or averages the middle two. | Limits the effect of extreme guesses, though it also gives little influence to an informed minority. |
| Geometric mean | Averages on a proportional scale; for two positive estimates, it is the square root of their product. | Suits positive quantities whose estimates differ by large multiples. |
| Weighted average | Gives some forecasts more influence using a stated rule. | Uses evidence about contributors' past accuracy or how recently they updated. |
| Most popular answer | Chooses the option selected by the largest number of people. | Combines named choices, such as award winners, where averaging the names has no meaning. |
For the basketball forecasts of 99, 99, 99, and 143, the median is 99 and the mean is 110. If the final score is 100, the median misses by one point and the mean by ten. The unusually high forecast shifts the mean while leaving the middle answers unchanged.
One forecast of 143 pulls the mean of these four hypothetical basketball predictions to 110, ten points above the final score of 100. The median is 99, matching the other three forecasts and missing by only one point.
The geometric mean handles a different kind of spread. Two forecasts of ticket sales, 10,000 and 1,000,000, have an arithmetic mean of 505,000 and a geometric mean of 100,000. The geometric mean lies ten times above the lower estimate and ten times below the higher one. It expresses a midpoint in ratios, which may suit uncertainty spanning several orders of magnitude.
For hypothetical ticket sales forecasts of 10,000 and 1,000,000, the arithmetic mean of 505,000 splits the gap into equal amounts. On the ratio scale, the geometric mean of 100,000 divides the gap into two tenfold steps.
Estimating weights introduces its own errors. In research on the forecast combination puzzle, statistically estimated optimal weights often performed worse than equal weights because the estimates were imprecise. Comparing the two methods on later questions tests whether weights fitted to past results help with future forecasts.
When does the wisdom of crowds fail?
Crowd wisdom fails when errors reinforce one another or when the group loses access to useful information. Adding participants helps little if they repeat the same mistake, suppress disagreements, or submit answers for reasons unrelated to accuracy.
| Failure | Why the combined answer suffers |
|---|---|
| Shared bias or sources | People make similar errors, which survive averaging even in a large crowd. |
| Missing information | Nobody contributes a fact that would change the forecast. |
| Copying earlier choices | An information cascade hides later participants' private evidence. |
| Unequal social influence | A central person's estimate dominates revisions, tying the group's accuracy to that person. Experiments show both gains and losses. |
| Pressure to agree | Groupthink discourages objections and examination of alternatives. |
| Distorted incentives | People submit forecasts that protect their reputation or please an audience. |
| Poor aggregation | An extreme answer dominates a mean, or the rule combines answers to different questions. |
Cowgill and Zitzewitz found optimism bias in internal markets at Google and Ford, even though the markets improved on expert forecasts overall. Employees' trading combined useful information with a tendency to favor positive company outcomes. More experienced traders tended to trade against the bias, and the inefficiencies generally shrank over time.
Analysts might understand a crypto project's code, user demand, and competitors while none knows that a security audit has found a serious flaw. Their launch forecasts may differ while all assuming the audit will pass. Averaging cannot incorporate an undisclosed finding.
Three analysts' forecasts for a hypothetical crypto project launching on schedule range from 60% to 80% and average 70%. The average still assumes the security audit will pass because none of the analysts knows it found a serious flaw.
Incentives shape the submitted answers too. A commentator rewarded for a memorable prediction has a reason to report an extreme estimate. An analyst who expects criticism for disagreeing with colleagues has a reason to stay close to their forecasts, even when their own assessment differs.
How is crowd wisdom different from groupthink?
Crowd wisdom comes from combining people's information. Groupthink is a decision process in which pressure for agreement interferes with examining alternatives. A group can preserve disagreement in its collected estimates even when it needs to reach one decision.
| Concept | What happens in the group |
|---|---|
| Crowd wisdom | Different estimates contribute to an aggregate that can be more accurate than individual answers. |
| Groupthink | Members hold back objections or fail to examine alternatives because agreement takes priority. |
| Herding behavior | People copy others' choices to learn from them, fit in, or protect their reputation. |
For example, an election research team might collect forecasts privately and find that almost everyone expects a close race. Their agreement may reflect the available evidence. If members privately expect a close race but report a comfortable lead because their manager favors that forecast, the apparent consensus conceals their doubts.
Five members of a hypothetical election team privately expect a lead averaging two points but report forecasts averaging eight. The reported answers cluster around the manager's preferred eight-point lead, hiding the closer race in their own assessments.
Strangers can also copy visible choices without meeting or pressuring one another. When they treat earlier choices as evidence strong enough to override their own information, an information cascade can form.
Charles Mackay collected accounts of collective delusions and episodes of enthusiasm or fear in Madness of Crowds. The phrase describes collective mistakes broadly, while groupthink and information cascades name specific processes that can produce them.
What is an information cascade?
An information cascade occurs when people follow earlier choices regardless of what their own information says. Each person concludes that the visible actions of others outweigh what they privately know. Later choices then stop revealing that private information.
Imagine people publishing their forecasts for a match one after another. The first two receive favorable reports about one team and predict that it will win. A third receives a contrary report but follows their forecasts, judging their combined evidence to be stronger. A fourth sees three predictions of a win, with no indication that the third person had contrary evidence.
The third forecaster repeats two earlier predictions of a home win despite receiving a report that favors the visitors. The fourth person sees three matching forecasts, with no sign of the contrary report.
In the classic model by Bikhchandani, Hirshleifer, and Welch, a short run of matching choices can start this process. Later people may follow for individually reasonable reasons while their separate observations never reach the group.
Anderson and Holt's laboratory experiment documented cascades in a controlled setting. Participants received private evidence from draws out of a hidden urn, then publicly guessed which of two urns was being used. Earlier guesses were visible to later participants. When the early choices created enough evidence to outweigh a private draw, cascades formed in most such rounds.
Herding covers several reasons for following others, including a desire to fit in or protect a reputation. An information cascade is the narrower case where the apparent information in earlier choices overrides private evidence. Several people can also predict the same winner after independently reading an injury report. Their agreement comes from shared public evidence, with nobody copying anyone else, so it is neither herding nor a cascade.
In financial markets, earlier purchases can raise the price a later buyer has to pay. With an unchanged forecast, following those buyers offers a smaller expected gain, as in the rate-cut example. Avery and Zemsky's model shows how price adjustment can prevent herding when traders' private information is only about the asset's value.
Does social influence make a crowd less accurate?
Social influence can make a crowd less accurate when everyone moves toward one poorly informed person. It can improve the group estimate when people with weaker answers revise toward people with better ones. Experiments distinguish these effects by changing whose estimates participants see before revising their own.
In a 2011 experiment by Lorenz and colleagues, 144 students estimated factual quantities over repeated rounds and received payments for accuracy. Some saw other participants' answers or the group average. Their estimates became more similar and their confidence rose, without a corresponding improvement in collective accuracy.
A 2017 experiment by Becker, Brackbill, and Centola recruited 1,360 people. Participants revised their estimates after seeing the average answers of people they were connected to. Where everyone had the same number of connections, the group median's average error fell by 12% after two revisions. More accurate participants tended to revise less, so others moved toward their better estimates.
In groups where one person had many more connections, revisions followed that person's initial answer. That influence improved the group estimate when it moved answers toward the truth and worsened it when it drew them away. Participants' estimates became more similar in both cases.
In the two simplified networks, each line joins people who see one another's estimates. Everyone on the left has four connections, while each of the seven outer people on the right sees only the central person's estimate.
A forecasting group can record estimates before its members see one another's answers, then compare the original and revised forecasts once outcomes are known. Keeping both sets lets the group measure changes in accuracy as well as changes in agreement.
Does social media reflect the wisdom of the crowd?
Post counts and likes are not a crowd's estimate of what will happen. The posts can address different questions, one author can contribute many times, and visible reactions influence later ones. The evidence for a forecast comes from the observations and arguments within the posts.
After a film trailer appears, viewers might praise the cast, criticize the effects, or report that tickets have sold out at a local cinema. Of those posts, the report about ticket sales bears most directly on opening-weekend revenue.
Only the neutral post in these hypothetical trailer reactions reports ticket sales, noting that every Friday showing at one cinema has sold out. Praise for the cast and criticism of the effects express positive and negative sentiment without that sales information.
A few prolific authors can account for much of a discussion. In Pew Research Center's 2019 study, the most active 10% of US adult Twitter users produced 80% of tweets. A rise in post volume can therefore reflect existing participants becoming more active, as well as additional people joining in.
Visible approval also changes later reactions. In Muchnik, Aral, and Taylor's randomized experiment on a news-aggregation site, giving a comment one initial up-vote raised its final rating by 25% on average.
Reading the Bluesky posts behind a trend helps distinguish an observation from a reaction to it. Several posts quoting one interview or sales report draw on the same evidence, even when different authors share them.
Sentiment analysis summarizes the tone of collected text. The shares of posts labeled positive, neutral, or negative describe that language, as does a sentiment score. Neither gives the probability of an outcome. A post can praise an actor while predicting weak ticket sales for the film. The label for the post as a whole does not identify which subject drew praise.
How do prediction markets aggregate crowd knowledge?
Prediction markets aggregate judgments through trading prices, with influence shaped by traders' resources, risk tolerance, and willingness to hold positions. A price therefore need not equal the simple average of their probability estimates. Buy and sell orders in the order book determine the terms at which the next person can trade.
Consider a hypothetical contract paying $1 if the Federal Reserve cuts its policy rate at a specified meeting, and zero otherwise. Someone who estimates a 60% chance calculates an expected payment of 60 cents. Buying at 40 cents gives an expected profit of 20 cents before costs. Buying at 70 cents gives an expected loss of 10 cents under the same assessment.
As the price rises, the contract becomes less attractive to someone whose forecast has not changed. Another trader who assigns a 30% chance may be willing to sell. Their two views do not force the market price to their 45% average, because the size and price of their orders affect the result.
For a hypothetical contract paying $1 on a rate cut, the buyer's 60% and seller's 30% forecasts let both expect a gain before costs at any price above 30 cents and below 60 cents. Their 45% average corresponds to one possible price among many in that range.
In Wolfers and Zitzewitz's analysis, a price read as a probability equals the traders' average belief under particular assumptions. When beliefs are correlated with wealth, the model gives wealthier traders more influence.
Trading volume measures how much was traded during a period, including repeated trades based on the same information. When you compare trading activity across events, several busy rate-cut markets can reflect responses to one inflation release.
Two venues can reach different prices for the same question because each has its own participants and available orders. Clinton and Huang's study of the 2024 US presidential election found identical contracts diverging across exchanges during the campaign's final five weeks. The settlement rules of two markets chosen for comparison establish whether their prices refer to the same outcome. Contracts with different thresholds ask different questions.
A shared trading price still allows sharp disagreement about what will happen. The buyer and seller accept the same transaction price while placing different values on the contract.
Do prediction markets rely on a crowd or an informed minority?
A study of two years of Polymarket trading found that a small skilled minority moved prices toward eventual outcomes. In the 2026 working paper described by Yale Insights, about 3% of accounts earned profits significantly above what chance would predict. Those traders also reacted faster to scheduled news.
The wider crowd supplied people to trade with and opportunities to profit from correcting mispriced contracts. Other participants' losses helped reward informed trading, giving people a financial reason to gather and act on information.
To test for skill, the researchers replayed each account's trades 10,000 times while randomly changing whether it bought or sold. They kept the markets, timing, prices, and trade sizes fixed, then compared actual profits with the simulated profits. They also divided events randomly into two sets. Of the accounts classified as skilled in one half, 44% were also classified as skilled in the other.
Choosing whether to buy or sell at random in 10,000 replays of a hypothetical account's trades produces profits centered on $0. Only about 30 replays exceed the $1,100 profit from the account's original trades.
A Polymarket leaderboard ranks results produced by forecasting, entry prices, position sizes, and luck. A large trader's purchase reveals willingness to commit substantial money at a price, while the buyer's reasoning remains unknown unless they disclose checkable evidence.
A trader copying a Polymarket purchase may pay more than the person they follow. In the rate-cut example, an early buyer expects a 20-cent profit before costs at a 40-cent entry price. A copier who pays 70 cents expects a 10-cent loss under the same 60% forecast.
A 60% chance of receiving $1 gives both buyers an expected payment of 60 cents on the hypothetical rate-cut contract. Paying 40 cents leaves the early buyer an expected profit of 20 cents before costs, while the copier paying 70 cents expects a loss of 10 cents.
Market Accuracy
Studies of prediction market accuracy show how well prices forecast real events and how Polymarket and Kalshi compare with other forecasts.
Superforecasting
Superforecasters' methods help you turn evidence into probability estimates and review your predictions against what actually happens.