Newswise —
The judgments made by policymakers and managers frequently hinge on projections and estimations offered by specialists. To illustrate, if policymakers foresee economic stagnation, they might raise public investments, while companies may launch new products if they expect ample demand. The prognostications of specialists regarding upcoming trends are fundamental in guiding these determinations. Consequently, the precision of specialists is scrutinized, and predictors who have a demonstrated history of precise forecasts could receive substantial professional prospects.
According to a forthcoming paper by Marco Ottaviani, Salvatore Nunnari (Bocconi University, Milan), and Debrah Meloso (Toulouse Business School), a desire to maintain a positive reputation may cause forecasters to provide inaccurate information that could potentially harm their reputation for being knowledgeable. The paper highlights a series of experiments that support this idea.
In their study, the researchers utilized an urn and balls model where each ball was comprised of an outer shell and an inner core. The inner core of each ball was either blue or orange, while the outer shell was also either blue or orange and was opaque, preventing the color of the core from being seen. Two urns were used, each containing 10 balls, representing the quality of information provided by the forecaster. The informative urn held balls where the inner core perfectly matched the outer shell, representing a forecaster with perfect knowledge of the future. The uninformative urn contained balls where the color of the shell was independent of the color of the core, representing a forecaster with no ability to predict the future.
The game consists of two steps: Firstly, a ball is randomly selected from either the informative or the uninformative urn with equal probability, but both the forecaster and the evaluator are unaware of which urn the ball is chosen from. Secondly, the forecaster is shown the color of the outer shell (but not the inner core) of the chosen ball and reports it to the evaluator. The evaluator then examines the color of the inner core and calculates the probability that the forecaster has seen a ball from the informative urn. The forecaster is rewarded based on the evaluator's estimation of the likelihood that the ball was selected from the informative urn.
The forecaster and evaluator have access to a panel that displays the actual distribution of blue and orange cores, which is identical in both urns. The distribution may be either 6/4 (to simulate high uncertainty) or 8/2 (to simulate a lower level of uncertainty). The uninformative urn always contains five blue shells and five orange shells, as shown in the diagram.
In the condition with lower uncertainty (8/2), the forecaster may still believe that a blue core is more probable than an orange core, even after seeing an orange shell. This is because out of the seven orange shells, four of them must have a blue core. Thus, by providing inaccurate information, the forecaster can enhance their likelihood of correctly guessing the core.
The experiment supports the notion that forecasters are motivated to provide inaccurate information to protect their reputation, and that this behavior is more common in situations with greater certainty. Specifically, the study found that in the condition with higher uncertainty (6/4), the rate of misreporting was 51%. In contrast, in the condition with lower uncertainty (8/2), the rate of misreporting increased to 63%.
Prof. Ottaviani notes that the findings of the study have implications for the use of expert advice in managerial decision-making and the creation of markets for professional forecasting. According to the experimental evidence, companies should rely more heavily on experts' advice when the phenomenon being predicted is more uncertain. However, when firms have access to precise information and the variables are less uncertain, the value of expert advice decreases, and it becomes less reliable.
Prof. Nunnari adds that the approach taken by the evaluators also has a significant impact. The study suggests that experts are more likely to provide accurate information when their reputation is closely tied to the accuracy of their predictions. Therefore, evaluators should evaluate experts based on their ex-post accuracy, rather than solely on their advice, as this can reduce the incentives for experts to misreport information.
Debrah Meloso, Salvatore Nunnari, Marco Ottaviani, “Looking into Crystal Balls. A Laboratory Experiment on Reputational Cheap Talk.” Forthcoming in Management Science. DOI: https://doi.org/10.1287/mnsc.2022.4629.
RSS