April 16, 2008
Prediction Markets Reach Corporate Board Rooms
By Tom Ryan
Companies like Best Buy, General Electric and Hewlett-Packard are using online prediction markets to try to improve forecasting, reduce risk and accelerate innovation by tapping into the collective wisdom of their work force.
Applied to companies, employees make their wagers over the internet using virtual currency, betting anonymously on what they think will happen. The payoff for the most accurate players is typically a modest prize, cash or an iPod.
Popularized by James Surowiecki’s 2004 book The Wisdom of Crowds and used in the past to bet on political elections, news events, video game sales, and movie box-office receipts, prediction markets have often been more accurate than professional pollsters or market researchers. The idea is that the collected knowledge of many people, each with a different perspective, will turn out to be more accurate than an individual or small group or even experts.
For corporations, prediction markets can spur communication and collaboration within a company – much like blogs – yet also add a rigorous measurement to business forecasts, like estimating the sales of a new product or the chances that a project will be finished on time. According to The New York Times, early results are encouraging.
“The potential is that prediction markets may be the thing that enables a big company to act more like a small, nimble company again,” Jeffrey Severts, a vice president who oversees prediction markets at Best Buy, told the newspaper.
Best Buy has experimented with prediction markets on everything from demand for digital set-top boxes to store-opening dates. For example, in the fall of 2006, the prices in a prediction market on whether a new Best Buy store in Shanghai would open on time – in December 2006 – dropped sharply from $80 a share into the $40 to $50 range. Players made yes-no bets, and the virtual dollar drop reflected increasing doubt that the store would open on time. The store wound up opening late, in January 2007, but the warning signs from the prediction market helped prevent further slippage.
Mr. Severts noted that prices in a current prediction market – betting whether new offerings from its Geek Squad service will be introduced on time in June – are in the $90 range, an encouraging sign.
Best Buy plans to move beyond pilot projects in prediction markets to involve more workers throughout the company, starting next month. “It helps on two fronts, the speed and accuracy of information, so that management can move faster to deal with problems or exploit opportunities,” Mr. Severts said.
Discussion Question: What do you think of the potential of prediction markets as a decision marking tool for retail companies?
BrainTrust
Recent Discussions
Discussion Questions
Poll
The collective mind is a very powerful force when used to look at creative ways to solve new or different problems; it brings a new and different perspective to a problem.
But one caveat to remember is, “The mind is a very efficient instrument and once it thinks it has the answer to a problem it quits working on it.” This is true of the individual as well as the group mind.
Group thinking is just like working with teams. In some places teams are more powerful and knowledgeable than individuals. But in many case a single person with greater knowledge will out perform team thinking.
If you are going to use predictive thinking, look at who makes up the group and make sure you have a professional facilitator.
Prediction markets are the next big thing in decision support, including market research and consultancy. Speed, accuracy and less expense are the hallmarks of this approach.
Prediction markets are not really new. What is driving the need for these now? The accelerating rate of change. The faster change (including, but not limited to, technological change) takes place, the more urgent prediction becomes. I “predict” that we’ll also see a rise in the use of scenario planning as a regular tool in the corporate toolbox.
Prediction markets, hand in hand with scenario planning, will be a table-stakes forms of decision support. Workers who are skilled in using these tools will be in high demand. But hey, don’t believe me, ask the prediction market.
Two simple problems with this–there is no validation of The Wisdom of Crowds and there is the risk of self-fulfilling prophecy. If I’m betting a store will open late, I’m likely to take actions to make that forecast come true. This delphi technique assumes that the crowd brings individual knowledge to the game, and it is the collection of that knowledge that leads to a better prediction. A Best Buy store clerk in Atlanta has no clue as to whether a store in Shanghai will open on time or not.
The conspiracy theorists among our readers will also note how easy it would be for the company to manipulate the results that are reported to the players–who says they need to see the actual parimutuel performance? Read John Brunner’s “Shockwave Rider” for an example of societal impact of buying into this.
Online prediction markets are particularly effective for retailers whose product mixes rely heavily on the right demographics to make this an effective tool. No question that the consumer electronics market is ripe for using this approach. However, the approach still needs to be carefully planned and orchestrated somewhat scientifically to get the most meaningful results.
This prediction model is still based upon a forecasting assumption which isn’t accurate. The inefficiencies built into the system create problems with traditional forecasting models causing errors, logjams, and out of stocks in the supply chain. Using the crowd prediction procedures suffers from similar consumer behavior research in that what consumers say they are going to do is not always what they actually do. The new methodologies that rely on forecasting have not resolved either of these issues.
The only way to be accurate is to base the process on replenishment–what the consumers actually buy!
That being said, it is important to have some estimates, guidelines, and historical sales levels to use as a benchmark against which REAL time sales can be monitored. An estimate compared to real time sales assumes a replenishment model not a forecasting model and will be more accurate.
There is an important distinction between prediction and self-fulfilling prophesy.
Crowds do create their own reality but it’s just that–their own reality.
There’s tremendous value in garnering “outsider” opinions and I hope that Liz is correct about an upturn in corporate scenario planning, but in the end we have to cling to the caveat that sometimes the wisdom of the crowd is little more than the echo of the mob.
There is no doubt that polling thoughtful participants who have knowledge of the subject can provide management better insight. The benefit of anonymity allows individuals to express their opinions without the fear of reprisal (Who wants to tell the boss his favorite project is stupid?). The challenge is that this same anonymity leaves the whole system to the risk of “gaming.” Individuals with personal agendas can manipulate the results, either by using multiple logins or, if it is truly anonymous, by merely voting repetitively.
So the challenge becomes, how do you locate participants who have knowledge of a subject, don’t have a personal agenda, and at the same time guarantee their anonymity? This all comes back to a management culture that encourages open communication and convinces employees that their honest answers will not result in retribution. This allows the system to require registration and screen participants based on their knowledge of a subject without discouraging participation.
But if you really do have an open culture in which employees are comfortable being honest, why do you need a system to get their answers?
I think there is a role–and I don’t think it should be anonymous, if you’re going to involve employees. If you’re having employees bet on what’s going to be the hottest toy or the best selling video game or newest fashion trend, wouldn’t you want to know who is consistently good at that–and maybe hire them as a buyer? I see prediction markets as a great way to collect front-line input into “what’s really going on in the store”–and both identify the talent hidden in the ranks, and engage employees in the future of the company.
The idea of getting people involved that come from different perspectives should improve forecasting, but only marginally. One issue not presented was, at what cost? How much time do participants spend betting on the future rather than doing their jobs? For example, how much does it cost companies in wasted time for March Madness, The World Series or The Super Bowl?
Keeping in mind that all forecasts are wrong, why do we continue spending time forecasting? Modern technology provides the ability to run companies off demand. If one knows what was ordered or sold today, then they know what action is required. Demand should drive companies not forecasts.
If managements will listen more to their folks by using prediction markets, great. Most retail firms are top down dictatorships with one-way communication, when there’s any communication. Managements often don’t listen to their shoppers or their employees or their suppliers or their shareholders.
I may be alone here, but the premise of future-casting by banking on the wisdom of the majority strikes me as empirically inane, if not also a little creepy. When did the crowd become so smart? When did the future become so knowable? The notion that asking Tom and Bill and Suzy what is going to happen next is going to yield some sort of manifestation likelihood is no more persuasive than consulting an astrologer. Alchemy is alchemy is alchemy, and you can bet that either hubris or chaos will eventually bite your butt. Caveat predictor.
If you are trying to get ahead of consumers, the last thing you should do is look directly at them–you must look at what outside factors are influencing them and that is how you get ahead of them. That is true prediction.
Feedback loops can be very helpful! Regardless of whether the executive suite heeds it or not, advice from the people in the store is some of the best information companies can get.
I see a slightly different twist on this, and it is a communications concern. If the employees know whether an outcome such as “the store will open late” is likely, why aren’t they telling management that already? Or perhaps more to the point, why isn’t management listening? There’s a reason some of the most effective leaders of our time practice “management by walking around”….
The critical condition behind “The Wisdom of Crowds” is the crowd, the shear volume of interested and educated data points. The challenge for marketers and retailers is to assemble enough data points for predictive markets to be meaningful and, ultimately, predictive.
That said, predictive modeling suffers from the same shortcomings as every other forecasting methodology in that it will be wrong. It can only predict scale and relationships with other demand. In the end, if we know that any forecast can only predict a range of likely sales levels, the critical element for any retailer or marketer is the plan for executing to the forecast, the plan for remaining nimble, and liquid, and responding quickly to the market as it develops.
Frankly, this sounds a little fishy to me. If I’m BP or Shell, I need scenario planning because my business is global and is directly affected by commodity prices and natural disasters. If Best Buy is applying this to a single store opening on time outside of their home market, I’d recommend the Compensation Board reconsider what Best Buy’s CEO and CMO are paid. This sounds like some useless product McKinsey came up with because it sounds sophisticated.
More Discussions
Has Same-Day Delivery Become an Expectation Across Retail?
Are expectations around retail delivery speed in flux?OpenAI Names the Infrastructure Standard for Agentic Commerce
Will OpenAI or Google (or another entity) set the gold standard for agentic commerce?Is Five Below Poised For Even Greater Growth?
Five Below is hitting serious growth numbers. Can this trend continue?
The collective mind is a very powerful force when used to look at creative ways to solve new or different problems; it brings a new and different perspective to a problem.
But one caveat to remember is, “The mind is a very efficient instrument and once it thinks it has the answer to a problem it quits working on it.” This is true of the individual as well as the group mind.
Group thinking is just like working with teams. In some places teams are more powerful and knowledgeable than individuals. But in many case a single person with greater knowledge will out perform team thinking.
If you are going to use predictive thinking, look at who makes up the group and make sure you have a professional facilitator.
Prediction markets are the next big thing in decision support, including market research and consultancy. Speed, accuracy and less expense are the hallmarks of this approach.
Prediction markets are not really new. What is driving the need for these now? The accelerating rate of change. The faster change (including, but not limited to, technological change) takes place, the more urgent prediction becomes. I “predict” that we’ll also see a rise in the use of scenario planning as a regular tool in the corporate toolbox.
Prediction markets, hand in hand with scenario planning, will be a table-stakes forms of decision support. Workers who are skilled in using these tools will be in high demand. But hey, don’t believe me, ask the prediction market.
Two simple problems with this–there is no validation of The Wisdom of Crowds and there is the risk of self-fulfilling prophecy. If I’m betting a store will open late, I’m likely to take actions to make that forecast come true. This delphi technique assumes that the crowd brings individual knowledge to the game, and it is the collection of that knowledge that leads to a better prediction. A Best Buy store clerk in Atlanta has no clue as to whether a store in Shanghai will open on time or not.
The conspiracy theorists among our readers will also note how easy it would be for the company to manipulate the results that are reported to the players–who says they need to see the actual parimutuel performance? Read John Brunner’s “Shockwave Rider” for an example of societal impact of buying into this.
Online prediction markets are particularly effective for retailers whose product mixes rely heavily on the right demographics to make this an effective tool. No question that the consumer electronics market is ripe for using this approach. However, the approach still needs to be carefully planned and orchestrated somewhat scientifically to get the most meaningful results.
This prediction model is still based upon a forecasting assumption which isn’t accurate. The inefficiencies built into the system create problems with traditional forecasting models causing errors, logjams, and out of stocks in the supply chain. Using the crowd prediction procedures suffers from similar consumer behavior research in that what consumers say they are going to do is not always what they actually do. The new methodologies that rely on forecasting have not resolved either of these issues.
The only way to be accurate is to base the process on replenishment–what the consumers actually buy!
That being said, it is important to have some estimates, guidelines, and historical sales levels to use as a benchmark against which REAL time sales can be monitored. An estimate compared to real time sales assumes a replenishment model not a forecasting model and will be more accurate.
There is an important distinction between prediction and self-fulfilling prophesy.
Crowds do create their own reality but it’s just that–their own reality.
There’s tremendous value in garnering “outsider” opinions and I hope that Liz is correct about an upturn in corporate scenario planning, but in the end we have to cling to the caveat that sometimes the wisdom of the crowd is little more than the echo of the mob.
There is no doubt that polling thoughtful participants who have knowledge of the subject can provide management better insight. The benefit of anonymity allows individuals to express their opinions without the fear of reprisal (Who wants to tell the boss his favorite project is stupid?). The challenge is that this same anonymity leaves the whole system to the risk of “gaming.” Individuals with personal agendas can manipulate the results, either by using multiple logins or, if it is truly anonymous, by merely voting repetitively.
So the challenge becomes, how do you locate participants who have knowledge of a subject, don’t have a personal agenda, and at the same time guarantee their anonymity? This all comes back to a management culture that encourages open communication and convinces employees that their honest answers will not result in retribution. This allows the system to require registration and screen participants based on their knowledge of a subject without discouraging participation.
But if you really do have an open culture in which employees are comfortable being honest, why do you need a system to get their answers?
I think there is a role–and I don’t think it should be anonymous, if you’re going to involve employees. If you’re having employees bet on what’s going to be the hottest toy or the best selling video game or newest fashion trend, wouldn’t you want to know who is consistently good at that–and maybe hire them as a buyer? I see prediction markets as a great way to collect front-line input into “what’s really going on in the store”–and both identify the talent hidden in the ranks, and engage employees in the future of the company.
The idea of getting people involved that come from different perspectives should improve forecasting, but only marginally. One issue not presented was, at what cost? How much time do participants spend betting on the future rather than doing their jobs? For example, how much does it cost companies in wasted time for March Madness, The World Series or The Super Bowl?
Keeping in mind that all forecasts are wrong, why do we continue spending time forecasting? Modern technology provides the ability to run companies off demand. If one knows what was ordered or sold today, then they know what action is required. Demand should drive companies not forecasts.
If managements will listen more to their folks by using prediction markets, great. Most retail firms are top down dictatorships with one-way communication, when there’s any communication. Managements often don’t listen to their shoppers or their employees or their suppliers or their shareholders.
I may be alone here, but the premise of future-casting by banking on the wisdom of the majority strikes me as empirically inane, if not also a little creepy. When did the crowd become so smart? When did the future become so knowable? The notion that asking Tom and Bill and Suzy what is going to happen next is going to yield some sort of manifestation likelihood is no more persuasive than consulting an astrologer. Alchemy is alchemy is alchemy, and you can bet that either hubris or chaos will eventually bite your butt. Caveat predictor.
If you are trying to get ahead of consumers, the last thing you should do is look directly at them–you must look at what outside factors are influencing them and that is how you get ahead of them. That is true prediction.
Feedback loops can be very helpful! Regardless of whether the executive suite heeds it or not, advice from the people in the store is some of the best information companies can get.
I see a slightly different twist on this, and it is a communications concern. If the employees know whether an outcome such as “the store will open late” is likely, why aren’t they telling management that already? Or perhaps more to the point, why isn’t management listening? There’s a reason some of the most effective leaders of our time practice “management by walking around”….
The critical condition behind “The Wisdom of Crowds” is the crowd, the shear volume of interested and educated data points. The challenge for marketers and retailers is to assemble enough data points for predictive markets to be meaningful and, ultimately, predictive.
That said, predictive modeling suffers from the same shortcomings as every other forecasting methodology in that it will be wrong. It can only predict scale and relationships with other demand. In the end, if we know that any forecast can only predict a range of likely sales levels, the critical element for any retailer or marketer is the plan for executing to the forecast, the plan for remaining nimble, and liquid, and responding quickly to the market as it develops.
Frankly, this sounds a little fishy to me. If I’m BP or Shell, I need scenario planning because my business is global and is directly affected by commodity prices and natural disasters. If Best Buy is applying this to a single store opening on time outside of their home market, I’d recommend the Compensation Board reconsider what Best Buy’s CEO and CMO are paid. This sounds like some useless product McKinsey came up with because it sounds sophisticated.