Too many forecasters are fools or frauds. Forecasting fools should be sent back to school to learn new methods. Forecasting frauds should be fired and possibly prosecuted, depending on how serious their fraud is. Recently, for the first time in history, forecasting frauds have been prosecuted, sending shock waves through the global forecasting industry.
[The following text, by Bent Flyvbjerg, calling for litigation against deceptive forecasters, was first published in 2013 in International Journal of Project Management. Many were skeptical of Flyvbjerg’s proposal to sue forecasters. You cannot lie about the future, some said, so where is the liability? Others said that forecasters had never been sued before, so why now? Two years later, in 2015, the first litigation ever against forecasters happened, for misleading demand forecasts on Sydney’s Lane Cove Tunnel and Brisbane’s Clem Jones toll road and Airport Link. Litigation immediately spread to the US, needless to say. With lawsuits and claims of more than half a billion US dollars in a single case this has sent shock waves through the global forecasting industry, which will likely never be the same. For the full study, with references and notes, see here: http://bit.ly/2eEugA2.]
The risks of major capital project investments in business and government are at least as large, as misunderstood, and as mismanaged as risks in the financial markets. Probably more so, because data about project risks are less available than data about market risks. In finance, biased forecasts and bad risk assessments led to the 2007-2009 market collapse, which exposed financial forecasters’ dubious practices. As a consequence, economists began to discuss the necessity of “firing the forecaster.” This is prudent advice, also for major project investment decisions.
But for major project investment decisions, firing the forecaster may be letting off forecasters too easily. Some forecasts for major projects are so grossly misrepresented and have such dire consequences that we need to consider not only firing the forecasters but suing them, too, for the losses they incur. In a few cases, where forecasters foreseeably produce deceptive forecasts, criminal penalties may be in place, if the damage caused by forecasters is severe enough.
Firing the forecaster may be letting off forecasters too easily.
So far, no forecaster has been sued for an inaccurate forecast, to the best knowledge of the present author. This may well be changing with an AUD$150 million class action suit under way in Australia, against the traffic and revenue forecaster for the Clem Jones toll road in Brisbane for producing “woefully inaccurate” forecasts. The suit is organized on behalf of 700 investors who lost their money when the toll road went bankrupt. The suit is historic as the first of its kind and should therefore be studied carefully by forecasters and forecasting scholars alike.
Nassim N. Taleb, author of The Black Swan, says about deceptive forecasters:
“People … who forecast simply because ‘that’s my job,’ knowing pretty well that their forecast is ineffectual, are not what I would call ethical. What they do is no different from repeating lies simply because ‘it’s my job.’ Anyone who causes harm by forecasting should be treated as either a fool or a liar. Some forecasters cause more damage to society than criminals.”
Management research has recently developed strong theory to help identify the “fools” and “liars” Taleb talks about here. Being academic, the theories use more polite and opaque language, needless to say, describing Taleb’s fools as people who are subject to “optimism bias” and the “planning fallacy” and liars as those practicing “strategic misrepresentation,” “agency,” and the “conspiracy of optimism.” This research has demonstrated that fools and liars constitute the majority of forecasters and that forecasts are used as sinister tools of fraud more often than we would like to think.
Fools and liars constitute the majority of forecasters and forecasts are used as sinister tools of fraud more often than we would like to think.
UCLA professor Martin Wachs found “nearly universal abuse” of forecasting in this manner, and it is common in all sectors of the economy where forecasting routinely plays an important role in policy debates, according to Wachs. Forecasts are used to “cloak advocacy in the guise of scientific or technical rationality,” in Wachs’s classic formulation, and there is no evidence that things have improved since Wachs did his pioneering studies.
However, recent research has also developed the concepts, tools, and incentives that may help curb delusional and deceptive forecasts. The outside view and due diligence, presented above, was developed with this purpose in mind. So was reference class forecasting, which has been officially endorsed by the American Planning Association and made mandatory by government in the UK, Denmark, and Switzerland. Finally, the incentive structures designed by Bent Flyvbjerg, Nils Bruzelius, and Werner Rothengatter in Megaprojects and Risk are means to curb both optimism bias and strategic misrepresentation in forecasting.
Given the knowledge and tools we have today for making better forecasts, there is little excuse for accepting forecasts that are incompetent (made by fools) or fraudulent (made by liars). We have the knowledge and tools to disclose the incompetence and fraud and should clearly do so, given the large financial, economic, social, and environmental risks that are at stake in this type of multibillion-dollar investments. Luckily, government and business are beginning to see this.
A fool with a tool is still a fool
Forecasting fools should be sent back to school. They make errors unknowingly and are therefore likely to be motivated to improve, once the errors are pointed out to them and better ways of forecasting are presented. When in school, they would need to unlearn conventional forecasting methods and to instead learn about optimism bias, the planning fallacy, the inside view, black swans, strategic misrepresentation, political power, principal-agent problems, the outside view, quality control, due diligence, reference class forecasting, incentives, and accountability. Their key teachers should be people like Daniel Kahneman, Nassim Taleb, and Martin Wachs. The main take-away for students would be a realization that a fool with a tool is still a fool, and that, therefore, they must stop being fools.
Forecasting frauds should be fired and possibly prosecuted, depending on how serious their fraud is. Forecasting frauds knowingly bias their forecasts for personal or organizational gain, for instance to obtain approval and funding for their projects. Forecasting frauds are therefore not immediately motivated to change their ways. Fraud is their business model. This is why carrots will be ineffective with frauds and sticks must unfortunately be brought out.
A light stick may be found in professional ethics. Professional organizations like PMI, APM, APA, RTPI, and other professional societies for managers, planners, engineers, and economists could use their codes of ethics to penalize, and possibly exclude, members who do unethical forecasts. Given how widespread unethical forecasts are, it is interesting to note that such penalties seem to be non-existent or very rare, despite codes of conduct that explicitly state that misinforming clients, government, citizens, and others is unacceptable behavior by members of these organizations. By not sanctioning fraudulent forecasts, and thus quietly accepting them, professional organizations become co-responsible for their existence. This needs to be debated openly within the relevant professional organizations. Malpractice in project management should be taken as seriously as malpractice in other professions, like medicine and law. To not take malpractice seriously amounts to not taking the profession of project management seriously.
Forecasting frauds are not immediately motivated to change their ways. Fraud is their business model. This is why carrots will be ineffective with frauds and sticks must unfortunately be brought out
A heavier stick would be to make forecasters financially liable for inaccurate forecasts and to reward them for accurate ones, something that is rarely done today, but which would undoubtedly improve accuracy. If there are no incentives to get forecasts right, perhaps we should not be surprised when they are wrong.
Finally, an even heavier stick would be to prosecute forecasters who, through blatant negligence or deliberate deception, cause serious damage to others with their forecasts, be they government, corporations, or citizens. Here the Sarbanes-Oxley Act-like legislation, which has recently been implemented in many countries around the world, signifies progress. Such legislation is aimed at corporate fraud and in many contexts fraudulent forecasts may be considered part of corporate fraud. Forecasters are therefore no longer as safe from prosecution as they were earlier. This in itself should have a sobering effect on forecasters, if not before then when someone has the good sense to sue a fraudulent forecaster.
Given the knowledge and tools we have today for making better forecasts, there is little excuse for accepting forecasts that are incompetent (made by fools) or fraudulent (made by liars)
[The above text is an excerpt from Bent Flyvbjerg, 2013, “Quality Control and Due Diligence in Project Management: Getting Decisions Right by Taking the Outside View,” International Journal of Project Management, vol. 31, no. 5, May, pp. 760–774. DOI: 10.1016/j.ijproman.2012.10.00. Free pdf with full text here: http://bit.ly/2eEugA2.]
First published on LinkedinBack to top of article