Round Table: Forecasting The Potential Of The Market

Forecasting market as a tool can gather abnormal scattered information, but it still faces organizational and legal challenges.

Ren e e Dye2008 April, all executives knew that the information used in making business decisions was often incomplete.

However, decision makers seldom consult with the lower level employees who directly communicate with customers, such as asking them how to effectively promote new products.

In this way, the leader himself deprives himself of the opportunity to gain valuable information, which not only makes them more thorough in analysis, but also reduces the risk of making decisions behind closed doors.

Some executives realize that valuable information is scattered around the company, but they do not know how to get it.

Other executives don't even try to get this information, perhaps because of the firm hierarchy, or because executives suspect that the answers they get may have some sort of color that supports their true or hypothetical views.

The market can solve these problems.

The real prediction market is initially a research field, essentially a small-scale electronic market, usually for all employees.

These markets link returns to measurable future events, such as the sales data of a computer workstation, the number of vulnerabilities of an application, or the application mode of a product 1.

Some enterprises, especially those in the hi tech industry, have seriously adopted the tool of market prediction, and some large enterprises are also trying to use the forecast market.

These prices are priced according to the predicted contracts, which can be interpreted as market aggregate forecasts.

The accuracy of their collective wisdom is often at least inferior to that of experts.

Supporters say that the prediction market can play a role because they can quickly gather information scattered around the company, while allowing participants to extricate themselves from limitations: for example, employees can anonymously complain about project launch date or new product performance, instead of worrying about their career being affected.

In addition, proponents of market forecasts point out that competition among colleagues and aspiration for rewards can motivate people to search for information and make the most informed decisions.

In order to assess the potential of the market and to overcome the organizational and legal challenges that have to be overcome, the McKinsey Co has recently held a roundtable meeting in Dubai.

Participants included Bo Cowgill, head of Google forecast market; Todd Henderson, assistant professor of law school, University of Chicago; Jeff Severts, general manager of Geek Squad, a branch of the US electronic retailer's best buy service; and the author of the book "prediction of market intelligence and other forms of collective wisdom", "group intelligence" (The Wisdom of of).

The discussion was hosted by Ren e Dye, a consultant at McKinsey Atlanta branch.

The following is an edited version of the roundtable discussion.

Bo Cowgill, Google Corporation product manager, is responsible for managing the company's forecast market for two and a half years.

He is also a writer and technical expert who has invested a lot of time to predict the effectiveness and efficiency of the market.

Todd Henderson, assistant professor of law at University of Chicago, served as project manager of McKinsey Co (2001 - 2004).

He is an expert in high technology and telecommunications.

In addition to forecasting market related research, he also co authored Prediction markets for corporate governance with Michael Abramowicz Notre Dame Law Review, University of Notre Dame.

Jeff Severts is vice president and general manager of Geek Squad, a US electronics retailer's best buy service branch.

His work also involves forecasting models, forecasting markets and alternative capital allocation processes.

James Surowiecki is a full-time contributor to New Yorker magazine, contributing to the commercial column of New York magazine.

He is the author of the best-selling book The Wisdom of Crowds, which mainly introduces collective wisdom such as forecasting the market.

"McKinsey Quarterly" (hereinafter referred to as "Quarterly"): let us first discuss the basic assumptions of the market and predict the reasons why the market results are so gratifying.

James Surowiecki: the basic assumption of forecasting the market is that, when appropriate, compared with the opinions of an expert or even a small group of experts, the collective judgement of a large group of people usually gives a clearer picture of what may happen in the future.

In most organizations, there are a lot of information that policymakers can not get.

They may not know the existence of such information, or they do not know who to ask.

Even if they know who to ask, the other person may not want to disclose this information because they are worried about their boss's criticism.

Forecasting the market is a way to collect and gather information scattered in every corner of the company.

It operates in a very similar way to the futures market, and the contract price reflects the collective conventional judgement of the value, such as the level of sales at a certain period of time, or probability, such as the probability of a product reaching a milestone target on a certain date, in percentage terms.

Jeff Severts: our first trial at best buy was inspired by James's writings, which showed that even if it were only a preliminary survey, it abandoned the filtering factors that often distort the truth in the process of information coming from the organization.

At that time, I was managing the gift card business, which is a relatively small part of our business portfolio, but I am particularly interested.

We sent e-mails to hundreds of employees in the company to invite them to predict the sales of our gift cards in February 2005.

We only provide some simple, ready-made data for their reference background.

We received 190 replies and carried out a simple average calculation.

The accuracy rate was 99.5%, and the accuracy rate we paid was 95%.

Later in the year, we launched a similar experiment, inviting 350 employees to predict our holiday sales.

As a result, the errors of these non professionals were only 1/1000, and again exceeded the error of 7% experts.

When the actual results came out and the report was formed, we were surprised when we told participants about the result.

These initial attempts have given us the courage to introduce predictive contracts. So far, we have seen more than 2000 traders making a total of 70000 pactions in 147 contracts.

Todd Henderson: of course, there is a special need to note that, contrary to the survey described by Jeff, a real prediction market allows for more than one time before the market closes, but a continuous multiple prediction, which is very beneficial to the accuracy of the prediction, which can establish a continuous updating information feedback loop.

Quarterly Bulletin: is there any distinction between wise and unwise groups?

Are groups always wiser than experts?

James Surowiecki: group wisdom is not a crusade against experts.

The point that this book should express is that it should not rely solely on the judgement of a person or a very small number of people.

But it is wise for a group to meet certain conditions.

First of all, this group must be diversified so that people can provide different information.

The group needs to be decentralized so that no high-level person can control the answers of the group.

Groups must group people's opinions into a collective conclusion.

Every member of a group must be independent, so that they will only focus on their own information without worrying about the ideas of the people around them.

Quarterly magazine: Google has more applications in forecasting the market.

Bo, can you tell me what the main applications are?

In April of Bo Cowgill:2005, we launched the forecasting market function. Since then, we have raised about 275 issues and completed nearly 80000 pactions.

About 1/4 of our market means are related to demand forecasting. For example, how many people will use Gmail in the next three months?

Almost all Google products have and still have a forecast market for product usage.

Another 30% of the market is related to company performance, for example, can we finish the project on time?

There are still a small number of forecasts that the market is about industry trends, such as mergers and acquisitions that may have a major impact on Google.

About 20% of our market is about who will win the world series.

The interesting forecast market can often attract people to familiarize themselves with how to predict pactions in the market.

This is good because many interesting market participants will then provide liquidity in a serious forecast market.

In addition to finding answers to our questions, we also try to understand the operation of these companies through these markets.

For example, we have been studying the cognitive biases and information of different departments in the company through different types of networks.

Quarterly magazine: what are the most common prejudices?

Bo Cowgill: there are several prejudices that are quite interesting.

I have done some work with two external economists, Eric Zitzewitz and Justin Wolfers, to study these prejudices.

Overall, the market is forecast to perform quite well.

Optimism is one of the most important prejudices.

The favorable result for Google &not, such as obtaining large numbers of users, is slightly higher.

The probability of market prediction is higher than the normal probability.

The reason seems to be the new employees, whose pactions show that they are very optimistic about our company.

Google's share price performance in the stock market also seems to be helping.

When the company's stock performance is excellent, the price that predicts the optimistic outcome in the market will also rise.

People are excited about the company's performance when Google's performance is excellent, so it is more likely to predict that Google will do something good.

We also note that traders are pricing too low in two extreme cases.

When we divide the contract into five different results (for example, the number of users in Gmail), the best and worst results appear more frequently than predicted by the market.

The trading has revealed some insights about how information can be delivered in the company.

Our market shows that cognition has the characteristics of clustering. These clusters are made up of individuals. They are close to each other physically and work closely. This means not only the urban or national level, but also the office floor and other micro level, such as the office desk.

The formation of "cluster" may also involve working in the workplace, participating in the same social activities, or even speaking the same language because they are not sitting together.

But all these factors seem less important than the region.

Jeff Severts: in best buy, we especially notice that employees have a bias that underestimates the strength of their competitors and that they know their competitors well.

When it comes to our main competitors, the accuracy of our prediction market can not be reassuring.

Quarterly magazine: will groups continue to learn progress over time?

Bo Cowgill: according to our experience, they will.

The longer you work in Google and participate in the forecast market, the better you calibrate yourself, and the more likely you are to have a successful paction record.

It is worth noting that, in terms of trading profits, a high position in the company (measured at a distance from the CEO's level) actually seems to put you at a disadvantage.

The market is becoming more and more smart as a whole.

Two and a half years after the opening of the market, the pactions we observed are more accurate than those just opened. As time goes on, we can see a steady improvement.

Quarterly magazine: we just discussed the cognitive bias that might affect these markets.

Are there any other deficiencies or limitations in forecasting the market?

James Surowiecki: we still don't know whether the market is suitable for accurately predicting non continuous innovation or leap.

Or is it possible to predict the market performance only when the variables are relatively clear?

We have no satisfactory answer, because we rarely carry out experiments to verify these problems.

The biggest weakness of the market is that many employees inside the company think that the market mechanism is not intuitive or difficult to understand.

They find it too complicated to use, which limits the number of participants.

Todd Henderson: now, forecasting the market is very easy to use, someone has set up the forecast market software for employees, and makes the interface easy to access as far as possible.

We have found some interesting phenomena.

One of them is that information may exist in unexpected places for policymakers, for example, in the "hidden group" of academia.

This refers to the position in the company because of its personality or in the hierarchy.