Forecasts and mean squared errors (Bold written numbers indicate significant differences at the 10 percent significance level)
The ”Beer Distribution Game“, a business simulation game developed at MIT (Sterman, 1989), is familiar to most students of supply chain management. As all good games, the Beer Game teaches some important lessons. The key lesson from the Beer Game is that sharing of information among the members of a supply chain is crucial for the supply chain performance. Information and knowledge in firms, social networks and supply chains to manage decisions with effects in the future is often dispersed, only available to certain individuals or groups and consists of opinions, personal guesses and evaluations. The human component in information sourcing, adapting and processing has largely been neglected in supply chain research. But the adaption of personal information and beliefs are important sources for adaption to an uncertain and changing market environment (Kirzner, 1973).
All business decisions are based on formal or informal forecasts about the future development of the market environment and the possible consequences of the decisions. So forecasts are a key skill in nearly every business situation. Forecasts vary in target, horizon, method, model, the team involved etc. Many approaches to forecast in fields of business interests neglect the information held by people in the respective firm. Inaccurate forecasts can result in substantial costs for a firm (Spann and Skiera, 2003). Instead of acknowledging the importance of good predictions for business success the managers invest in many cases little time and money and put less attention on the forecasting work. And mostly they neglect the information held by other persons in their own firm, e.g. workers in the production process, and outside the firm, e.g. consumers.
There are two different approaches of forecasting methods: The first approach uses statistical methods to result a prediction out of existing data (e.g. by methods of time series econometrics); the second approach collects new data of the predicted object (e.g. consumer and expert surveys) (Armstrong, 2001). Both approaches have major weaknesses; the first approach needs reliable data - only for known settings predictions can be estimated; the second approach lacks information on who to ask and how to aggregate the answers. Prediction Markets (PMs) are forecasting tools which employ digital network technology in order to aggregate diverse personal beliefs (forecasts) about the future into forecasts that tend to be better than forecasts by individual experts or by statistical forecasting methods.
PMs can overcome these lacks of traditional approaches by potentially using all the information available by allowing people to trade their bets. Formerly this used to be costly; modern hardware and internet based markets have driven down transaction costs rapidly. Nonetheless, incentive systems have to be developed to truthfully reveal the information of all participants. In the end all bets are liquidated at a price according to the actual outcome (Spann and Skiera, 2003). PMs are “...markets that are designed and run for the primary purpose of mining and aggregating information scattered among traders and subsequently using this information in the form of market values in order to make predictions about specific future events” (Tziralis and Tatsiopoulos, 2007).
Von Hayek (1945) assigned markets a dual role. They allocate resources and, through the process of price discovery, they aggregate information about the values of these resources. This information aggregation role is widely accepted for stock markets. Stock market prices are interpreted as the consensus judgment about the value of future corporation earnings (Berg et al., 2003).
PMs are widely used to forecast election results, sport games, box office revenues and a lot more (e.g. Berg et al., 2008b, Gruca et al., 2008, Hartzmark and Solomon, 2006, Servan-Schreiber et al., 2004). In these fields PMs created accurate forecasts and mostly these forecasts are better than the standard operated methods. We applied the principle of PMs to forecast future product sales in a firm. This implementation confirms the idea that PMs are a promising tool to manage supply chains. In our case study they show accurate forecasts, both in absolute and relative terms compared to the standard operated methods.
A short description of the functional principle of PMs is followed by two theoretical foundations of the price formation process and the forecasting ability of PMs. Applications of PMs to different topics are described in the next section. Section five describes the requirements of PMs, the design and the results of our own experiment to forecast future sales volumes. The chapter finishes with a conclusion.
2. Functional Principle of Prediction Markets
The use of experimental markets for forecast purposes is based on the idea that private and public information will be aggregated and published efficiently by these markets (Berlemann, 2004). The information aggregation ends directly in a qualitative or quantitative forecast at the PM. The functional principle behind PMs is similar to the big stock exchanges. Certificates, which represent a forecast of a future event, are traded between the PM participants. The prices of the certificates can be interpreted as a prediction concerning the forecasted event. The possible realisations of the forecasted event have to be transformed into values of the tradable certificates. The transformation function determines the certificate value at any time T (Spann, 2002):
di,T=certificate revenue value,
φ()=transformation function, transforms the realisation of the forecasted eventinto the certificate revenue value,
Zi,T=realisation of the forecasted event i at time T,
I=set of events and
T=point in time of the forecasted event.
The transformation function φ() transforms the possible outcomes Zi,T of the forecasted event into termination values di,T of the certificates. The expected payoff of a certificate for every participant under the information set Ωt is at every time t<T:
Every PM participant has to insert his own expectation concerning the forecasted event into the transformation function to receive his expected payoff of the certificate. By comparing his expected payoff with the market offers and trading accordingly, he achieves expected wins at the PM. By doing so he will change the market price towards his own expectations.
An invertible transformation function -
has the advantage that the trading prices of the certificates can be retransformed into a forecast. The future sales quantity of a specific product shall be forecasted, for example. A possible transformation function for this task can convert every sold quantity unit (QU) into one currency unit (CU) of the certificate. Direct transformation of the trading price into a prediction is possible in this example. Let the trading price at time t be pi,t = 22.34 CU. This implies a forecast of Zi,T = 22.34 QU. The transformation function clarifies that the certificates depend on the outcome of an uncertain future event (Spann and Skiera, 2003). Most PMs use a continuous double auction with or without market makers as trading mechanism. Market makers offer continuously bids and asks for the certificates to increase market liquidity (Hanson, 2009). PMs can be divided depending on the forecasting issue into three types (Spann, 2002):
a. Winner-Takes-All Markets
The prediction of the occurrence or non occurrence of a future event, e.g. the re-election of a candidate, is the simplest example for Winner-Takes-All Markets. Two certificates are traded on such PMs: A represents “Re-Election” and B “No Re-Election”. - Both certificates are combined in a unit portfolio for the price S. The market organiser sells and buys the unit portfolio to/from the participants during the market operation time for the price S. The unit portfolio represents all possible realisations. The termination value of certificate A is S if the candidate is re-elected (dA,T = S) otherwise it is 0 CU (dA,T = 0 CU). Certificate B is worth 0 CU if the candidate is re-elected or S if the candidate is not re-elected. The market organiser buys back all certificates for their final value after the last trading day. The trading prices of the certificates are interpreted as the probability of occurrence of the underlying event in percent if the price S of the unit portfolio is standardised to 100 CU. The prediction of more than two possible states is realised by more than two certificates; one certificate for every possible state. - Example: There is one certificate for every team in the Football World Cup to predict the champion. The price of the unit portfolio, existing now out of 32 certificates, is again 100 CU. The trading prices reflect again the winning probability of the underlying team. The possible prediction of continuous variables with Winner-Takes-All Markets needs non overlapping intervals of the total event space. Each interval represents one subspace of the event and belongs to one certificate. The calculation of the expected value leads to the prediction (it is the sum over the means of the subspaces multiplied with their probability of occurrence).
b. Vote-Share Markets
Vote-Share Markets are used to predict relative figures, e.g. the market share of different products or vote shares. The market share of three different products A, B and C in one market shall be predicted, for example. - Certificate A (B, C) represents the market share of product A (B,C). The unit portfolio includes each certificate once. The termination values of the certificates are calculated by multiplying the actual market share of the product by the price of the unit portfolio S; di,T = vi,TS (vi,T: actual market share). Product A reaches an actual market share of vA,T = 0.25 = 25 % and the price for the unit portfolio is S = 100 CU. Then certificate A has a termination value of 25 CU = 0.25100CU. The trading prices of the certificates are interpreted directly as the expected market share of the underlying product.
c. Markets for the prediction of continuous variables
Also continuous variables can be predicted with the help of PMs. Instead of the construction of non overlapping intervals as shown above, continuous variables can be predicted directly. Example: The total sales quantity of product X (ZT) shall be forecasted. Two certificates are traded on the PM for the direct prediction of the sales quantity. Certificate A represents the sales quantity and certificate B S minus the sales quantity. Certificate B is necessary to create a unit portfolio which is always worth S. S has to be chosen carefully because the forecasted sales quantity has to lie with certainty between zero and S. The typical sales quantity of product X was about 50,000 and it is not expected that the future sales quantity exceeds 100,000. In this case S will be set equal to 100,000 CU. The transformation function converts one sold quantity into one currency unit. The termination value of certificate A is dA,T = ZT and of certificate B dB,T = S - ZT. If the actual sales quantity exceeds 100,000 units, then certificate A has a value of 100,000 CU and B of 0 CU. One participant expects for example that the sales quantity will be 56,000 units, then certificate A has an expected value of dA,T = 56,000 CU = ZT = 56,000 units and B of dB,T = 44,000 CU = S – ZT = 100,000 – 56,000 for him.
Prediction Market Trading
Trading at a PM is divided into two stages. The market organiser sells (or buys) the unit portfolio for the price S during the whole market operation time to (from) the participants at the first stage (the primary market). When at least one participant buys one unit portfolio at the primary market, the participants can trade the single certificates for prices, which may represent their expectations, among each other at the secondary stage (the secondary market). The above description of the termination value structure clarifies that the PM is a Zero-Sum-Game for the organiser. The market organiser sells the unit portfolio for the price S and buys back all certificates at market termination for their final values. The construction of the certificates guarantees that the sum of the termination values is always equal to S. The primary market is a riskless exchange of S CU for a unit portfolio.
The secondary market is the core of the PM. The PM participants trade the certificates among each other at prices that reflect their expectations about the underlying event. Normally a continuous double auction is chosen as market mechanism. This design assures the possibility to create buy or sell orders for the certificates at any time. For the case of orders with higher buy than sell prices a trade takes place. The general rules of continuous auctions are applied for the matching process. Open design possibilities are the selection of the allowed order forms (e.g. market order, limit or stop-limit order) and the design of the order book (e.g. open or closed). The trading at the secondary market contains win and loss possibilities for the participants, if the trading prices differ from the actual termination value of the certificate. One participant j shall try to buy all certificates at the market if he can get them for lower prices than Ej,t[di,T] and to sell all certificates at the market if he can receive a higher price than Ej,t[di,T]. By doing so he gains an expected profit. Only if two participants j and k have different expectations about the event, the participants trade certificates. The relationship Ej,t[di,T] ≠ Ek,t[di,T] must be valid. At time t a PM is cleared if there is no demand for certificates with a price greater than the price of the smallest sell order. The last trading price in this situation represents the collective expectation of the market participants about the future event. The PM produces a new prediction with every trade. Every forecast indicates a different time horizon as time goes by.
PMs show impressive forcasting performances in previous applications - in comparison to alternative forecasting methods. We still do not fully understand the well functioning of PMs. Two different approaches for the theoretical foundation of the prediction process can be found in literature. The first approach is based on von Hayek’s (1945) insight about the dual role of markets. Markets are well known for swapping goods between different persons. Additionally, markets aggregate the diverse information of the traders by the price formation process. The stock value of a company, e.g., is taken as the collective expectation of the company value. So the first approach is based on the theory of rational expectations and efficient markets. The second approach is based on the toolbox approach by Page (2007), which highlights the importance of diverse forecasting groups.
3.1. Classic market theory
A simple theoretical model based on Kyle (1985) is presented in the following to explain the price formation process on PMs (Wolfers and Zitzewitz, 2006b). The PM is organised as a continuous double auction. The occurrence or non occurrence of an event is predicted on the PM. The participants trade a binary certificate, which has a final value of if the event occurs and of if the event does not occur. The expected payoff of the certificate is for every participant his personal probability of occurrence in CU. All traders have an individual expectation ei,j,t,T concerning the probability of occurrence of event i out of the distribution F(ei,j,t,T) and a private wealth of wj. The traders maximise following logarithmic utility function:
The partial differential of the utility function with respect to xj results in the net demand of every participant j:
If the individual expectations of the trader ei,j,t,Texceed the price pi,t, then he will buy the certificate. Otherwise he will sell the certificate. The PM is in equilibrium if the market is cleared. The market clearing price has to equalise the aggregated demand and supply over all participants. The net demand for the certificate has to be equal to zero. The market clearing pricehas to fulfil the following condition:
The expectations are furthermore distributed independently from the wealth. So the equation reduces to:
Market prices are consistent with the mean of the expectations and they are an unbiased predictor for the participants (Wolfers and Zitzewitz, 2006b). The difference between the mean of expectations and the market clearing price is quite small for different types of utility functions (Wolfers and Zitzewitz, 2006b). The market clearing price differs significantly from the mean of the expectations of the traders in the special case of only one single investment decision and uniform distributed information (Manski, 2006).
Transaction costs have to be considered but they do not change the main result. An area of uncertainty of the market clearing price, depending on the transaction costs, appears around the old market clearing price now. The transaction costs are divided in two modes: the information search costs, these are all well-known costs for the information search and the creation of the expectations, and the common transaction costs, which cover all costs of market participation, e.g. fees. The information search costs are nearly the same for every market type, if the same product is traded. Solely the transaction costs partially exhibit great variance between the market types. These costs are quite small for PMs, because PMs operate over the World Wide Web (WWW). - The resulting market clearing price has now an uncertainty of k around the price without transaction costs if all market participants have the same transaction costs k. The new asks and bids differ from the expectations by the factor k. A bid (ask) is higher (smaller) than the expectations. The transaction costs can cause some participants not to trade. The actual market clearing price is now located in a corridor of the size k around the price without transaction costs depending on the distribution of the expectations. The smaller the transaction costs the smaller is the potential deviation between the market clearing price and the mean of the expectations.
3.2. Page’s toolbox theory
The diversity of the expectations of economic agents is assumed in the classic market theory. The classic market theory gives no reasons for the existence of the diversity. In the case of rationality and identical information all economic agents shall have identical expectations. Page (2007) tries to explain the information aggregation process with his “toolbox” approach. Basis of this and other approaches to model human decisions is the assumption of limited rationality instead of complete rationality (Simon, 1982). Page tries to decompose the decision process into its elementary components (Gigerenzer and Selten, 2002). Furthermore, Page explains why predictions, composed of individual predictions of a group of forecasters, are often better than the predictions of the best forecaster within this group. Finally Page gives reasons why PMs show better predictions than polls.
The assumption that cognitive diversity yields in better results of job completion is Page’s basic concern. The diversity can be decomposed in the diversity of the four cognitive tools of decision makers: (i) diverse perspectives or the way of representing situations and problems, (ii) diverse interpretations or the way of categorising or partitioning perspectives, (iii) diverse heuristics or the way of generating solutions to problems and (iv) diverse predictive models or the way of inferring cause and effect (Page, 2007).
The predictive model is based on the concepts of perspectives and interpretations. A perspective is nothing more than a word, which describes a situation, an event or an object. A pack of paper between two covers can be described as a book and we can read it, or we can describe and use it as a doorstopper, if it is heavy enough, or as a missile to banish unwanted persons. It is important that the perspective, the description of the object, indicates its usage for the solution of a specific problem. The perspective differs from person to person and more creative persons have more versatile perspectives than less creative persons.
Perspectives are components of interpretations. Interpretations assign a group of objects, events or situations to a word. Specific attributes of these objects, situations or events are normally not considered. We can categorise persons, who apply for a job, in many directions: age, gender, family status, education and so on. If we just use the two interpretations gender and family status, we get six groups of job candidates: the combination of female and male with single, married and divorced.
The predictive model is a combination of interpretation and prediction. The prediction is the result of the interpretation of an object. For example we classify job candidates for a research job according to their field of study, place of study and exam marks. By doing this we hope to receive a good interpretation of the future research quality of the candidate: good university, useful field of studies and good marks indicates good research work, and so on. As persons have different perspectives they can have different interpretations and use them to receive different predictions in the end.
The predictions from different persons differ because they are based on their diverse predictive models. Therefore the question, how this diversity can be used to create better predictions, is apparent. One obvious idea could be to select the best forecasters. This strategy has two disadvantages: first in the case of long time predictions the selection can only be done with long delay and second there is no reason for the assumption that a good forecaster in one field will be a good forecaster in others too. A good meteorologist, e.g., would not be taken as an investment banker solely because of his good weather forecasts. Page chooses another approach. He explains the phenomena of the “wisdom of crowds”, known since Galton (1907), with the help of a theorem and a “law”. Page’s Diversity Prediction Theorem signifies that the collective prediction error is the average individual prediction error minus the prediction diversity. This theorem indicates the Crowd Beats the Average Law: “the collective prediction is more accurate than the average individual predictions” - (Page, 2007, p. 209). We introduce a few notations to clarify the theorem and the law.
ZT is the observed realisation Z of a metric variable at a future time T. The members of a prediction collective kK forecast individually this variable at time t=T-n. These predictions are denoted vk,t,T. We have only one prediction time t and one prediction horizon n, so in the following we leave the indexes T and t out.
Every prediction is afflicted with errors which are measured as the squared difference between the predicted and the realised value: e=E[(Z-v)2]. The individual prediction error of one member is marked: ek=E[(Z-vk)2]. The average individual prediction error can now be calculated as:. The square of the difference between the mean of the individual forecasts and the realised value is the collective prediction error. The mean of the individual forecasts is. The collective prediction error is calculated as:. At the end we need a measure for the prediction diversity in the collective, which is the mean squared difference between the individual forecasts and the mean of the individual forecasts:. By the help of this notation the Diversity Prediction Theorem indicates: or.
This theorem has a lot of beneficial implications. The quality of the collective prediction is influenced by the average quality of the individual forecasts () and additional high prediction diversity (D) is advantageous. In short the forecasting ability of a collective is as important as prediction diversity. Furthermore the collective prediction is always more accurate than the average prediction of its members. Communication between group members has not to improve the prediction quality. If the prediction diversity D decreases more than the average individual prediction error, the collective prediction error then increases instead of decreases.
The participants of a PM are members of a forecasting collective. In the above description all individual forecasts are weighted equally but the PM participants define the weight of their forecasts with their money at risk. This suggests the assumption that market participants highly confident in their forecasts invest more money and put more weight on their forecasts than less confident participants. This will end up in a reduction of both, the average individual prediction error and the prediction diversity. If the collective prediction error will also decrease depends on the weights, the money at risk, of the market participants. In contrast to polls, which can be responded cost- and riskless, it can be assumed that persons who do not know anything about the forecasted object, are not prepared to invest their own money at PMs. Page calls this the “fools rush out”. The prediction quality can be highly increased by the banishment of the fools. In respect to the importance of the prediction diversity Page is cautious about too much incentives for the good forecasters and too few for bad forecasters, because the small fools are necessary for prediction diversity.
4. Applications of Prediction Markets
A great part of the actual literature describes and analyses realised PMs. We classify the applications according to their main focus into four groups: Policy, Sports, Business, Cinema and Others.
The most known application are probably the PMs to forecast the US presidential elections and further elections in the US by the University of Iowa at the Iowa Electronic Markets (IEM). - The first PM was organised to predict the next US-President 1988 (Forsythe et al., 1992). The PM correctly predicted the win of George Bush Sen. nearly perfect. The last forecast of the PM indicated a 53.2 percent vote share for Bush and 45.5 percent for Dukakis. The final election result was a vote share of 53.2 percent for Bush and 45.2 percent for Dukakis respectively. The forecasts of the PM were in addition very accurate in absolute terms and in relative terms in comparison to the polls. A comparison of the prediction error of the PM and of the polls shows the good relative prediction quality of PMs. The absolute prediction quality is measured by the mean squared error of the forecast against the actual election outcome. The mean squared error of the last PM forecast is 0.00004 and 0.00013 for the best poll (own calculations based on Forsythe et al., 1992, p. 1149).
In real money PMs traders have to invest their own budget. Participation here was restricted to University members and the budget was limited to 500.00 US$ due to an agreement with the US Commodity Futures Trading Commission. This restriction was necessary because otherwise the PM would have been banned by the gambling laws. Thus, PM participants were not a representative sample of the US population. Additionally the traders were predominantly male and educated (Forsythe et al., 1992). A number of scientists, including the three Nobel Prize laureates K. Arrow, T. Schelling and V. Smith, campaign for a separation of PMs from the classical gambling and a licence for Real money PMs (Arrow et al., 2008).
After the first successful PM the IEM organised PMs for all US presidential elections and other political elections in the US. The participation restrictions were lifted over the years. Now everybody with access to the WWW can participate. The number of participants increased from 192 in 1988 to several thousand in 2008. The number of participants had no impact on the forecast accuracy. An analysis of all PMs up to 2008 shows, that the PM predictions were more accurate than the polls in 74 percent of the cases (Berg et al., 2008b, Tables 2, 3 and 4). The PM predictions have furthermore a high consistence with the actual election results. The pairs of values are all very close to the 45°-line. (Berg et al., 2008b, figure 1). The win of Obama 2008 was predicted with a mean absolute error of 1.2 percent points. The PM achieved this small mean absolute error over the time span from June 2006 up to the election in November 2008. Only the last polls have a similar size of error and for longer time distances to the election the errors were significantly larger (Berg et al., 2008a).
After the success in the US PMs have been employed to predict election results in numerous other countries. It is to highlight that PMs are used in countries with more than two relevant parties (among others Canada, Germany, the Netherlands and Austria). The Canadian election 1993 was accurately predicted by a PM. The special feature of this election was that two new parties took part for the first time. The 257 participants had no information about the performance of these two parties at former elections. The mean absolute error was only 0.57 percent points at election eve (Forsythe et al., 1995). The election PMs for Germany (Berlemann and Schmidt, 2001, Hansen et al., 2004), Austria (Murauer, 1997) and Australia (Leigh and Wolfers, 2006, Wolfers and Leigh, 2002) reached comparable results. The PM for the election in the Netherlands produced poor forecasts absolutely and relatively in comparison to the polls (Jacobsen et al., 2000). The authors mention the false-consensus effect as one possible reason. The false-consensus effect describes the curiosity that persons estimate themselves as being representative for the whole group of voters and finally expect a false election result. The number of participants ranged from 21 toover 1,000. The number of participants seems to have no influence on the accuracy of the PMs. Chen et al. (2008) analyse the PMs to forecast the election results in the single states of the US and control these results with the winning probability of the US President candidates, which was forecasted by an additional PM. They show that the PM participants interpret correctly the forecasted results in the States into the winning probability in the US presidential election. Wolfers and Zitzewitz (2004) and Snowberg et al. (2007) research the connection between the winning probability of the US President candidates 2004 with different other events, like the imprisonment of Osama bin Laden or unemployment rates. They show that a better economic development or an imprisonment of bin Laden increases the winning probability of the Republican candidate Bush.
Solely for a part of the German policy PMs has to be mentioned that the correlation between the published polls and the prices at the PMs is significant (Berlemann and Schmidt, 2001). It could be verified for all other PMs that they create independent forecasts. The PMs react to new information and not to new polls (Abramowicz, 2003) and this reaction is faster (Forsythe et al., 1992).
The known betting markets are similar to the PMs because the reciprocal of the odds can be interpreted as the probability of occurrence. Betting markets reached a great distribution in the US to forecast the US presidential elections from 1868 to 1940. Afterwards the betting markets were banned. In 11 of 15 cases the betting markets detected the correct future president. The favourite of the betting markets lost the election in one case only. Three times there was no favourite at the betting markets. The transaction volumes at the betting markets were particularly curiously higher than at the stock exchanges of the Wall Street (Rhode and Strumpf, 2003, 2004). Manipulation attempts were not successful at the betting markets and either at the modern PMs of the IEM. The manipulation had short impacts and the markets reached the pre-manipulation level very quickly (Rhode and Strumpf, 2009).
Betting markets are very popular for sport events. The odds are an indirect prediction. In the following only betting markets where participants trade the odds directly with each other are taken into account. - Participants of sport PMs can react to new information during the event and adjust their forecasts to the new information that might directly affect the outcome. The number of participants is oftensignificantly bigger than 1,000. Most of the PMs for sport events operate with real money and the investment of the participants is unlimited. Some sport PMs operate additionally on the basis of virtual money. Servan-Schreiber et al. (2004) show that virtual and real money PMs to forecast the winner of NFL games have similar accuracy. The forecasted win rates are nearly perfectly correlated with the actual win rates. The correlation is r=0.94 (r=0.96) for the virtual (real) money PM. The additional comparison with opinion pools shows no significant difference in the forecasting accuracy for the 210 analysed games (Chen et al., 2005). The operation with real or virtual money has no significant impact on the forecasting accuracy for sport PMs (Rosenbloom and Notz, 2006).
The PMs predict the winning probabilities in the NFL accurately. The actual winning rates match the predicted ones (Chen et al., 2005, O'connor and Zhou, 2008, Rosenbloom and Notz, 2006, Servan-Schreiber et al., 2004). This relationship is true until the game starts. The actual win rates differ from the predicted ones during the game, especially after new information, e.g. touchdowns or field goals (Borghesi, 2007). Hartzmark and Solomon (2006) detect the disposition effect for the NFL PM. The disposition effect describes the phenomena that persons realise wins faster than losses because they rate wins and losses differently. The transaction prices at the PMs increase after the occurrence of new positive information (touchdown) as expected. Shortly after the significant price increase the prices decrease without new information occurred. This implicates that the traders offered more sell than buy orders shortly after the price increase and realised wins. The prices increase again after the decrease. They rise to their new correct level. The disposition effect appears only if the transaction prices during the game are higher than the pre-game prices. A similar effect cannot be detected if the in game prices are lower than the pre-game prices.
Soccer PMs are also quite popular. PMs predicted the outcome of games (win, draw or loss) during the European Championship 2000 more accurately than the odds from the bookmaker Oddset. A bet at Oddset on the favourite team of the PM yielded positive returns (Schmidt and Werwatz, 2002). The PMs for the European Championship 2004 (Slamka et al., 2008) and the World Championship 2002 (Gil and Levitt, 2007) were accurate too. The initial design of the start depot has a significant influence on the trading frequency. Two PMs with different start conditions, PM1 with only virtual money and PM2 with virtual money and certificates, predicted the World Champion 2006. More than twice as many orders were executed at PM2 than at PM1 (Seemann et al., 2008). The PMs for the English Premiership are semi strong efficient for new goals, which are the most important information next to the game time (Croxson and Reade, 2008). A trader cannot gain a positive return with public information. The PMs were very liquid; the average trading volume was nearly 5.8 mil US$ per game. 44 percent of the volume was executed during the game (Croxson and Reade, 2008). The prices of the PMs during the World Cup 2002 show the same information revelation (Gil and Levitt, 2007). Additionally, price inefficiencies and arbitrage possibilities do not hold longer than 15 seconds. It takes less than 90 seconds to incorporate all new information in the PM prices (Slamka et al., 2008). The trading prices are significantly higher for the scoring team after a goal than before. The trading prices feature a drift. The decreasing game time offers fewer possibilities of new information. The prices for the leading team increase with the decreasing game time on this account (Gil and Levitt, 2007).
The accurate prediction of the game outcome is verified in additional research. An analysis of the PMs for games in the NBA, NHL, NFL, MLB and NCAA - show an accurate prediction though the win rates of the favourite teams are underestimated (Bean, 2005). A single analysis of the NBA and NFL games detects more accurate predictions for the NBA games and the underestimation of the favourites decreases (Borghesi, 2009). The results of English rowing events are accurately predicted (Christiansen, 2007). The PMs for cricket games detect the correct outcome and show efficient information revelation. Only the batting team can score due to the game plan. Thus the trading prices increase for the batting team in anticipation of possible points before the team actually scores. This anticipated increase of the prices reduces with every point scored because the probability of an additional point decreases (Easton and Uylangco, 2007).
The application of PMs to predict economic and business developments and performance figures shows that PMs can reach accurate predictions which are partly better than the standard methods. All described PMs for business events operated as closed groups; the participation was restricted to members of the company or members of special business sections. Siemens forecasted a project termination within the company with the help of PMs. The 62 participants in the PM correctly predicted the delay of the deadline(Ortner, 1998a, b). A small group of 7 to 24 participants forecasted future sales of printers at Hewlett Packard. This small group had the ability to predict the future sales figures more accurately than the standard operated internal methods. The PM traders did not know the internal forecasts (Chen and Plott, 2002). A PM was operated to predict the future sales volume of an unnamed company. The predictions were in 15 of 16 cases more accurate than the internal methods of the company (Plott, 2000). Google used PMs to forecast major business figures and business related figures like number of users of different Google services, general business and hard- and software developments and non business related topics (e.g. sport events) (Cowgill et al., 2008). The PM is an appropriate tool to forecast future developments. The 1,463 traders at the PMs showed significant learning effects during the participation. They show a positive overestimation of the development at their first trades. The overestimation decreases with growing trade experiences. Traders with small spatial office distances have similar expectations concerning the forecasted events. The diversity of expectations grows with increasing spatial distance (Cowgill et al., 2008). The PMs to forecast future gross user acquisitions and user figures of different mobile technologies yielded more accurate predictions than the survey among experts (Spann and Skiera, 2004). Motorola (Levy, 2009) and General Electric (Spears et al., 2009) use PMs in research and development for idea detection. PMs increase significantly the speed and the quality of idea detection.
Cinema and Other PMs
PMs are applied to a wide area of topics. A PM to forecast future infectious disease activity yielded accurate predictions for short time horizons. A time series method reached similar forecast accuracy for time horizons greater than eight weeks (Polgreen et al., 2007). The Hollywood Stock Exchange, a PM to predict the box office results of future cinema movies, is very popular with over 500,000 restricted users. The correlation between forecasted and actual box office revenue is 0.94. The traders detected the future winners of the Oscar awards correctly (Pennock et al., 2001a, b, Spann and Skiera, 2003). The movie PM at the IEM is clearly smaller with about 1,000 traders. But the forecast accuracy is at the same level (Gruca et al., 2005, 2008).
A combination of policy PM predictions and financial market data results in the conclusion, that a higher probability of war with Iraq will increase the oil price and decrease the stock index S&P500. The probability of war was measured with a PM, which forecasted that Saddam Hussein would be ousted until specific dates. A ten percent rise in the probability that Saddam Hussein would be ousted induced a 1 US$ per barrel increase of the oil price and a one and a half percent decrease of the S&P500 (Leigh et al., 2003). Additional PMs forecasted future inflation rates (Berlemann and Nelson, 2002), base rates (Berlemann, 2004), number of hospital patients (Rajakovich and Vladimirov, 2009) and stock prices (Berg and Rietz, 2002).
Summarizing, it can be shown empirically that the prices at PMs react quickly to new information, suffice the law of one price and follow a random walk (Wolfers and Zitzewitz, 2006c). Furthermore, PMs are weak or semi strong information efficient. Traders cannot gain positive returns with the help of public information (Leigh and Wolfers, 2006). The participants of PMs can actually reveal correct information even if they know nothing about the real value of the event (Hanson and Oprea, 2004). The information aggregation capacity of markets is undisputed and is proven by a lot of experiments, although the clear process is still unknown (e.g. Berg et al., 2003, Forsythe and Lundholm, 1990, Forsythe et al., 1982, Plott, 2000, Plott and Sunder, 1982, 1988, Plott et al., 2003).
For successful applications of PMs to forecasting problems several conditions have to be fulfilled. At least some participants need to have information about the forecasted event. This information has to indicate some asymmetric distribution between participants because otherwise no trade is likely to occur ("no trade theorem", Wolfers and Zitzewitz, 2006a). Former applications of PMs have shown that for 15 to 20 participants accurate results are obtained (Chen and Plott, 2002, Christiansen, 2007). Uninformed persons (noise traders) can take part in PMs. They increase market liquidity and offer potential profit opportunities to informed traders. Successful manipulation will be less likely when the number of participants increase. Manipulative orders offer profit possibilities for the informed traders and can therefore actually increase the PM accuracy (Hanson and Oprea, 2004). This effect is shown by experimental analysis by Hanson et al. (2006). The traders recognise that a part of the orders have manipulative intentions and react to it. If traders can influence the forecasted event, it is nearly impossible to prevent successful manipulation.
The transformation of the forecasted event into values of the certificates is elementary for well functioning PMs. A clear and objective transformation function is necessary to convert the possible realisations into values of the certificates. The transformation function has to be published prior to the PM start and may not be changed afterwards. The transformation factor has to be chosen in a way that even small changes of the forecasted event lead to changes in the expected value of the certificate. The prediction of very uncertain or certain events has to be possible and the transformation function has to account for it. In case of very (un)certain events the favourite longshot bias is a popular phenomena. The favourite longshot bias occurs when individual forecasting probabilities indicate an s-shape instead of a linear shape which was first proven for horse betting. The odds for the horses differ from the actual win rates. While the odds are too high for the favourites, they are too low for the longshots. A bet on the favourite (longshot) yields a positive (negative) expected return (Thaler and Ziemba, 1988).
The reward system is important for inducing information search and revealing them in the PM (Sunstein, 2006). Due to legal reasons real money PMs are forbidden almost worldwide. The operation with virtual money instead has, however, no significant effect on the forecast accuracy (Servan-Schreiber et al., 2004). Wins and losses of virtual money PMs are virtually but some prizes can be rewarded to support the extrinsic motivation. Different immaterial and material incentives can stimulate the extrinsic motivation (Spann, 2002). Gifts or money rewards are the most known material incentives. They are given to the most successful traders or a randomly chosen active participant. Immaterial incentives are rank lists or the publication of the best traders. All these incentives support the desired objective function of portfolio maximisation. The traders can also maximise their portfolio due to their intrinsic motivation. The total trading volume also has a positive effect on the individual trading activity. The average trade volume per participant is correlated with total trade volume (Seemann et al., 2008).
The experimental PMs were introduced to forecast future sales figures of different products of a well-known agribusiness company in Germany. - The firm had a high interest in good predictions because of a long time lag between production and selling. 37 people of the firm, mostly belonging to the sales and administration division, were elected to participate in the PMs. The best participants received prizes depending on their final ranking. The markets started about four months before the actual sales’ volumes are known. At the end the market organiser bought back all virtual shares to their termination values that correspond with the actual sales’ volumes.
Four different PMs, one for every product, were installed and operated over the WWW. Every PM operated independently. The PMs were organised as type c) with a small change. Only one virtual certificate was traded in every market. The participants traded with virtual money at the PMs. The virtual certificates were named after the products sold. The virtual share represented the sold quantity of the respective product; the transformation function was 1 CU for every QU sold. If for example 100 QU of product X were sold then the certificate would be 100 CU worth. The participants received 1,000 virtual certificates and more than 1,000times the forecasted sales quantity from the last internal forecast before market start in virtual CUat each market. The combination of virtual certificates and money was necessary because there was only one certificate in each market and no unit portfolio existsoverall. The participants could not buy additional certificates by trading unit portfolios with the market organiser. The initial certificate endowment increased market liquidity. The markets were named here A, B, C and D. All markets were organised in the same fashion.
The PMs were open for approximately four months. All PMs started at the same date and had the same market termination time. 37 people were invited to participate in the PM to forecast the future product sales. 22 people actually participated actively, which means they ordered at least once. The total number of orders was 545. 221 orders expired before market termination and 324 orders were executed. PM B, C and D nearly indicated the same number of orders (138, 147 and 165 orders respectively). Significantly fewer orders were observed for PM A (95). Trading was dominated by five participants. They were responsible for 61 percent of the trades at PM A and up to 86 percent at PM C. The five most active traders posted relatively more non-executed than executed orders. Their proportion at the executed orders was 77 percent relative to 85 percent of the non-executed orders.
The trading activity differed during the operation time. The daily trading volume, orders per day, is shown in Figure 1. The activity decreased over time. In the first 25 trading days a total of 259 orders were offered by the participants. In the next 25 days 172 orders were posted. In the following period 85 orders were offered. The participants placed only 29 orders in the last 26 days before the finaltradingday of the PM. The differences between the four markets were small. The activity decreased significantly. This could be an expression of decreasing uncertainty about the termination value. The possibility of overlapping orders reduces if the traders are more confident about the termination value. The trading success was differently distributed among the participants. 18 participants had the same wealth at the end as in the beginning. Among these 18 participants were the 15 people, who did not trade. The sum of the wins of the 7 most successful traders was equal to the sum of the losses of the 12 least successful traders. The absolute profits and losses were quite small. The best trader made a profit of nearly 3 percent and the worst trader lost about 2 percent.
All market data was collected on order basis. To result forecasts we calculated volume weighted average prices for each day. The forecasts of the PMs were compared to the internal company forecasts (IF). - The mean squared forecast error was calculated to compare both methods in relative and absolute terms. Four updates occurred for the IF during the market operation time. The average daily PM prices and the IF for product C are shown in Figure 2. The actual sales quantity is also shown as a reference for both forecasts. - Both instruments underestimated the total sales quantity at the beginning by nearly the same amount. The forecasts of the PM moved away from the IF over time. The forecasts of the PM were closer to the actual quantity than the IF. The forecasts reached a level nearly similar to the actual sales quantity fifteen days before market termination. The IF reached a similar forecast quality three days before market termination. The last forecast of the PM is an actual sales quantity of 1.001. Because all data is normalised by the actual sales, it is obvious that the forecast of the PM is nearly exact for the last fifteen days. Most of the time the IFs showed a larger deviation, the last IF, however, was 0.994, which was also very accurate in absolute terms.
The average forecast for product C over the whole market duration was 0.8345 at the PM and the IF predicted 0.7747 as shown in the second last column of Table 1. The PM forecast is significantly closer to the actual result than the IF. Bold numbers indicate statistically significant differences (α=0.1). The mean squared errors are measured to determine the absolute forecast accuracy. The average trading price is calculated for every day and the difference between this price and the actual result is squared. Final the mean over the squared differences is determined for the different time spans. The mean squared error gives more weight to larger deviations and treats positive and negative deviations the same way. The mean squared errors are presented for both forecast instruments in Table 1. For product C it is unambiguous that the PM is more accurate over all time spans compared with the IF.
The forecasts and mean squared errors for the additional three products are also shown in Table 1. PMs are more accurate than the IF over all products and all time horizons. In 10 of 16 cases the predictions of the PM are more accurate. IFs are more accurate in five cases of which four cases are product D. And in one case both methods are similar accurate. Product D is the product with the smallest variance of the forecasts and the smallest deviations from the actual sales volume. The internal forecast predicts the sales quantity of product D more accurately for all time spans. The deviations are less than one percent for the internal forecast (last column in Table 1). The PM is better to predict the sales of product A. Though not statistically significant, during the final 50 trading days the IF beatsthe PM.
|last trading day||last forecast||PM||1.0171||0.9848||1.0009||0.9809|
|last 26 trading days||average forecast||PM||1.0191||0.9846||0.9455||1.0257|
|last 51 trading days||average forecast||PM||1.0656||0.9767||0.8793||1.0270|
|last 76 trading days||average forecast||PM||1.0741||0.9631||0.8508||1.0290|
|whole market duration||average forecast||PM||1.0716||0.9735||0.8345||1.0332|
Finally, the hypothesis of independent predictions of the PM is tested. The forecasts of the PMs are tested to be independent from the IFs by regression analysis. The reactions of the PM after an update of the IF are controlled if the PM reacts in the same way as the IF and if the PM only reflects the same information as the IF. First differences of the IFs are regressed on several differences of the PM prices. Different adjustment periods are considered for the PM to control for the velocity of adaption. Four updates occurred for the IF during market operation time. All but four first differences of the IF are zero so four observations enter every regression for the IF. The first regarded difference (time span) of the PM is from one day before the new IF to the day of the IF update. Afterwards the four first differences of the IF are regressed on the four differences of the PM. The results of the regressions are presented in Table 2 for all products. In all cases the PM prices appear to be independent from the IFs.
The changes of PM prices after the time of the IF update are also considered to control for delayed adaption. In a second step we compute the same regression but now we take longer adaption periods for the PM. The second price difference is the difference of the PM prices one day before the publication of the internal forecast with the price one day after the publication instead of the price at publication as above. Additionally the differences with the prices 2, 3, 4, 5, 6 and 7 days after the publication of the IFs are computed. In all but 2 of 32 cases a significant impact of the internal forecasts on the PM prices can be rejected. - A significant relationship is found for product A for 6 and 7 days after the publication of the internal forecast. The factor of the explanatory variable is significant at a level of 0.096.
|Product||Constant||Factor of the explanatory variable||Multiple R²||F-value|
5.4. Possible different applications within a supply chain
PMs can be applied to forecast nearly every future event or object as the above experiment and applications show. There are different forecast purposes within a supply chain; upcoming possible sales quantities at the end of the chain, other production figures (quality of products, output quantities) or prices of inputs or output products. Additionally, forecasts of possible future needs and requirements of consumers in the new product development have a high value for firms.Some of these forecasts can be obtained by analysing actual production figures but these cannot take unexpected changes into account and the new product development can only be improved with new data generating processes. And most of the standard processes do not take the wisdom of the workers in the production process, company members, people outside the firm, consumers and suppliers into account.
To avoid high storage costs and missed vending chances accurate forecasts of upcoming sales are necessary. PMs achieved more accurate forecasts in predicting upcoming printer sales at Hewlett Packard (Chen and Plott, 2002) and product sales in our own experiment than the standard methods. Additionally, PMs generated accurate forecasts in the prediction of future user figures of Google services (Cowgill et al., 2008) and different mobile technologies (Spann und Skiera, 2004). PMs can operate to forecast production figures within a supply chain to enhance the coordination between the chain members. The forecast of production figures in the process, especially product quality, offers the possibility for quick reactions to changes. The workers operating the machines or producing the goods have knowledge about possible changes but normally they do not participate in forecasts. The introduction of a PM to forecast future product quality can enhance the forecast quality.
The forecast of prices of input and output products offers the chance for quick reactions to the changing market environment. It reduces the uncertainty of the price development, if the price forecasts reach good forecasting qualities.The price forecasts offer a high potential to increase the commercial exploitation in the purchase and sales process. The price selection for new developed products will provide great opportunities; it may decide about the success or failure of a product. A PM can be installed to forecast the willingness to pay of the consumers. This offers the chance for a better pricing and reduces the risk of failed products because of their high price.
Another possible application of PMs can be found in the idea generating and selection process in new product development. Different product designs and their sales’ potential can be traded on PMs to select the most promising products. Additionally, ideas can be traded and the participants can add new ideas to the PM to result in a guide to future developments. The incorporation of consumers of the possible products with the help of PMs is easy because PMs operate over the WWW. The consumers of future products will increase the quality of the new product development and will value information and knowledge about future needs of the consumers more highly than the workers in the company will do. For example General Electric (Spears et al., 2009) and Motorola (Levy, 2009) introduced PMs in the idea generating process to accelerate the idea detection process and to raise the quality of idea detection. They wanted to involve the workers in the process and their ideas and expectations about possible problems or consumer needs. They succeeded in both with the help of PMs.
PMs offer an easy and cheap possibility to incorporate the diverse information held by workers in the own firm and additional information held by people outside the firm, for example sales people in an electronic market for printer sales. PMs can react instantaneously to new information and reveal it via the price. The reward system causes the participants to trade correctly on their information basis because correct anticipation leads to portfolio increases otherwise they obtain a loss.PMs offer a high potential to enhance the forecast quality within a supply chain in different fields and to improve the involvement of the supply chain members or persons from outside the supply chain and outside the company in the forecasting process.
The adaption to the changing environment is a key capability for successful firms. The quick reaction to changes is essential and offers high potentials. Reliable forecasts are needed as a base to plan firm’s adjustments. Inaccurate forecasts and/or incomplete adaption to the changing environment can result in substantial costs for the firm (Spann and Skiera, 2003). A good example is Nokia’s adaption to the new trend of smartphones with touch screens. The reasons for missing this trend are unknown to us but obviously they misjudged future consumer needs. Most of the standard forecast methods neglect the wisdom of the people in the production process, in the company, or the wisdom of customers.
PMs are a forecasting tool which can achieve better results in predicting future events as the standard methods. PMs are based on the “wisdom of crowds” principle. The “wisdom of crowds” describes the phenomena that groups achieve better prediction results than the single group members. PMs are similar to stock exchanges on which the participants trade certificates of future events. The group members trade their expectations concerning the future development and determine their weight by the money they are willing to bet. This is a great advantage against surveys. Compared with standard statistical methods PMs use new information and aggregate these efficiently over participants. Participants can use a variety of forecasting methods to create their expectations. The information and results of the forecasting methods are still built from expectations of the participants. PMs “...cannot make a silk purse out of a crowd of sows’ ears” (Page, 2007).
PMs reach good forecasting results in a lot of applications in a variety of fields. The first modern PMs are used to predict election results (e.g. Berg et al., 2008c). Afterwards PMs are applied to sport events (e.g. Bean, 2005), box office revenues (e.g. Pennock et al., 2001c), economic development (e.g. Berlemann, 2004), future disease activity (e.g. Polgreen et al., 2007), business forecasts (e.g. Chen and Plott, 2002), and a lot more. The PMs often achieve better forecasting results than the respective standard methods in the field. The PMs can produce continuous forecasts.
We implement four different internal PMs to forecast future product sales of an agribusiness company. The future sales of the four different products are predicted with a group of 37 persons. 22 persons trade actively at the PMs. The forecasts of the PM are compared with the internal forecasts of the company. The PMs are more accurate than the internal forecasts in 10 of 16 cases. The prediction errors of the PMs are significantly smaller in these 10 cases. The liquidity at the markets is low. Five participants are responsible for nearly 80 percent of the trades. More participants, more active traders, the introduction of a market maker, or more incentives for participants might have increased the participation rate and thereby the number of trades to further improve the quality of the PMs forecasts. Market liquidity indicates the major task for successfully introducing and efficiently operating forecasting PMs.
Grateful acknowledgement is made for financial support by Stiftung Schleswig-Holsteinische Landschaft.
- The risk free interest rate is set to zero and the holding of the certificates is riskless. This implicates for every point in time t<T: pi,t = Et[di,T|Ωi,t] = Et[di,T|Ωi,t]/(1+r).
- The mathematical function φ-1(() can be calculated. So the following relation is valid: Zi,T = φ-1(pi,t) with pi,t = Et[di,T].
- “Lock-In-Option”, “digital options”, “Simplex Options”,, “All-or-Nothing-Options” or “Lottery Options” are different common notations of these certificates (Berlemann, 2004).
- The winning candidate of the election out of a set of candidates can be predicted alternatively. In this case every certificate represents one candidate. The trading prices represent now the winning probability of the candidates.
- These three products A, B and C represent 100 percent of all sold products. Otherwise an additional certificate “Others” is necessary to cover the total event space.
- See section 4 for more details.
- Intermediate solution step: ∁Ej,tUj,t∁xj=ei,j,t,T1-pi,twj+1-pi,txj-1-ei,j,t,Tpi,twj-pi,txj≜0and solving leads to:wjei,j,t,T-pi,t=pi,t1-pi,txj
- Intermediate step: wjpi,t1-pi,t∪0pi,tei,j,t,T-pi,tfei,j,t,Tde=wjpi,t(1-pi,t)∪pi,t1(pi,t-ei,j,t,T)f(ei,j,t,T)de
- Nearly every person with access to the WWW can participate in a PM. Additionally access has no time limits, so the reaction on new information is always possible.
- For the special case that the prediction diversity is zero then the collective error is equal to the average individual error. The prediction diversity can only be zero if all forecasters have the identical prediction.
- The Iowa Electronic Market can be reached under following address: http://tippie.uiowa.edu/iem/index.cfm.
- Only these are very similar to PM. The regular bookmaker bets for example are notconsidered.
- These are all North American sport leagues.
- Due to legal reasons we are not allowed to name the firm and the products. Additionally we have to present the results anonymously. All quantities were normalised by their final amount.
- The internal forecast are an extrapolation of actual sales volume to the end time point.
- A presentation and description of the three additional products is missed here. The results are presented below for all four products.
- The presentation of the regression is missed here, due to the limited space.