The Problems of Forecasts in Civil Engineering

Historically, the first traffic forecasts dated back to the 1950s and were prompted by the question of how many roads (primarily highways at that time) would be needed. Although the term “predict & provide” was not yet in use, it was heavily focused on the predictability of limits even then. However, the saturation assumptions of the known, elongated S-curves back then were not influenced by environmental issues, traffic safety concerns, or capacity issues but rather by social and economic questions. For example, when will everyone who wants their vehicle will have one? The absence of these other factors in the forecast continued for decades. Generally, it focused on what land use or public transportation policies would contribute to solving transportation problems. It was hardly or not considered that the limit could be influenced as a deliberate policy measure or chosen as a strategic goal to achieve environmental objectives, as would later be the case. Thus, the forecasts were driving the policy. Predict & provide, as it were. So far, so well-known. This paradigm is no longer so dogmatic, monomodal, or monocausal in its use today. And yet, forecasts are still indispensable.

First, let’s talk about terminology: Forecasting literally means knowing something beforehand (prognosis). Less strictly, one could say prediction, and even less exactly, estimation. A forecast and an estimation are similar but have an essential difference. A forecast refers to predictions about future events or developments based on data analysis and statistical methods, generally regarded as valid and reliable in society. On the other hand, estimations and, even worse, prophecies refer to a judgment or assessment based on individual experience and subjective knowledge. An estimation is often less precisely formulated than a forecast because it is based on fewer data and less precise methods. However, it doesn’t say anything about the accuracy of what is estimated or forecasted. Theoretically, it is possible that looking into a crystal ball, reading cards, or observing bird flights could achieve “better” results. But such methods are not scientifically grounded or justifiable.

From science to philosophy and back again

When we take a strict view of the topic of forecasting, we almost inevitably move beyond the realm of science or, more precisely, the domain of knowledge. At its core, science deals with analyzing and interpreting empirical findings. From the apparent randomness that can be observed, counted, measured, and experienced, we try to gain insights either inductively or deductively based on hypotheses and theories using scientific methods. However, we always operate on the “safe” terrain of the past unless we work as mathematicians or logicians in the realm of apriori sciences, which is not the case for planners. We derive our insights from the world of experience. Unfortunately, certainty regarding future events ceases knowledge and judgments based on experience and empiricism (a posteriori). This is where things get interesting, as it raises questions about cause and effect, initial and boundary conditions, causality, and certainty, which can only be expressed in probabilities with the help of empiricism. And since the arc of causal considerations stretches at least philosophically from the pre-Socratics, Plato, Aristotle, Nicholas of Autrecourt, the medieval Hume, to Hume himself, Frege, and Wittgenstein, to name a few, it exceeds the scope here. Let’s keep it brief: There is no absolute certainty here! And there cannot be.

Our experiences are based on observations of the real world. However, unlike the natural sciences, we face a fundamental problem. While they can substantiate their insights through experiments in the laboratory by varying input variables and thus derive valid constructs of causality from the many correlations, this path is either unavailable to us or severely limited. And that brings us right to the topic of forecasting. Analytically speaking, our forecasts have a significantly lower probability of “prior knowledge.” Therefore, we cannot trust them. Only significant causal relationships enable good predictions. Let’s momentarily step away from science and briefly turn to philosophy, not delving into the details of classical questions like Kant’s “What can we hope for?” but rather the question: What can we know? There is indeed something like a strict certainty: for (1) the necessary and (2) the impossible: What is necessary? I must breathe to stay alive. What is impossible? Some things are impossible. A square cannot be a circle. I can only be in one place at a time. Between necessity and impossibility lies the realm called contingency. Contingency refers to chance or unpredictability, which can cause events to unfold differently than expected or predicted. And that’s precisely what happens to us in transportation planning time and again. Whether our forecasts are trend or model forecasts, they are always stochastic and highly contingent. Although inconvenient for us, traffic forecasts are far from being deterministic. One exception is the so-called normative forecasts, which will be discussed later under “magical thinking.”

Of course, we can interpret specific patterns and extrapolate them based on the past. Our databases are becoming more detailed, and our mathematical and statistical methods (e.g., deep learning) are improving. And yes, for the most part, we can live with and plan based on the (vague) results. However, when it becomes tight and threshold values come into play, we must argue for accuracy and precision, which we cannot achieve. Because as a natural, non-repeatable real-time experiment, we can’t isolate and explain all (necessary) determining factors (e.g., traffic volume on a specific link). Disruptive innovations and other non-evolutionary phenomena cannot be anticipated. This brings us directly to the topic of multicausality and the INUS concept.

The INUS concept

The INUS concept is a philosophy and decision theory concept developed by John D. Mackie. INUS stands for “Insufficient but Necessary parts of an Unnecessary but Sufficient condition” and refers to the idea that complex events or phenomena can consist of a combination of necessary but not sufficient conditions. Insufficient means that a cause (for an effect) requires further conditions. Necessary means that the condition alone, without a cause, is not sufficient. Unnecessary means that other causes can also trigger the effect. Sufficient means that the reason and condition are sufficient. In other words, different influencing factors (on an outcome) are necessary but individually not sufficient, or conversely, a product can also occur due to entirely different (currently unknown) reasons. For example, a traffic accident can occur due to bad weather, a faulty brake light, and an inattentive driver. Each of these factors may be necessary to cause the accident, but none alone can explain the accident.

Understanding the INUS concept helps comprehend complex events or phenomena by highlighting that they can arise from a combination of conditions or factors that are necessary but insufficient to explain the event or phenomenon. When we look at the standard input variables for our traffic models or various forecasts, it quickly becomes apparent that we are constantly dealing with significant uncertainties and can’t know the future but can only estimate it at best. One could sarcastically ask whether the glaring forecast errors of some politicians concerning inflation were merely coincidentally wrong or whether some supposedly good traffic forecasts were just coincidentally correct. However, one can also make it a practice to validate all accessible predictions from reputable institutions ex-post. You’ll be amazed. So, what do we do? Rely on gut feelings or roll the dice? No, there are strategies.

Strategies in the battle against randomness

One initial rule could be acknowledging the limited extent of one’s knowledge before making predictions. By doing so, one inherently presents the outcome of the forecast as one of several possible outcomes, fostering a sense of healthy scepticism. These known unknowns fundamentally differ from unknown unknowns, which can be operationalized through various risk management concepts. Known unknowns refer to things we are aware we do not know, whereas unknown unknowns refer to uncertainties of which we are unaware. In other words, known unknowns are uncertainties to which we are conscious, while unknown unknowns are uncertainties to which we are oblivious. Understanding the distinction between these two is crucial for interpreting forecasts and developing appropriate strategies to handle them.

As early as 1921, Frank Knight delved into this issue in his work “Risk, Uncertainty and Profit.” The concept of Knightian probabilities entails a theory of decision-making in situations where it is uncertain whether a specific condition will or will not occur. Unlike classical probabilities, where one can calculate an objective or subjective probability, Knightian uncertainty is the absence of known or computable probabilities. This type of uncertainty may arise when there needs to be more information to calculate a probability or when the situation is too chaotic or complex to predict its outcome. The concept is particularly relevant in cases involving the management of risks associated with significant consequences, such as making decisions on expensive infrastructure or technology investments. In practice, this implies that decision-makers should consider the possibility that there may be no objective or subjective probabilities that can quantify the likelihood of an event. Instead, one must strive to manage uncertainty by evaluating various decision options regarding their potential benefits (maximal possibilities) and risks and losses (costs).

The consideration of subjective probabilities provides a more concrete approach. Subjective probabilities are frequently employed in fields like econometrics and risk management, where calculating objective probabilities for future events proves challenging. They are also utilized in decision theory to model individuals’ or companies’ choices under uncertainty. An example from business administration is the Hurwicz rationality axiom, which suggests that a decision-maker should choose a compromise between maximum subjectively expected utility and maximum minimum utility. This axiom reflects a certain degree of risk aversion.

In contrast, objective probabilities are derived from facts and statistics and do not depend on individual subjective opinions. This is considered state-of-the-art in most traffic models. Business administration, particularly in decision theory, has developed axioms like the Laplace rationality axiom. This concept in decision theory states that a decision-maker should act rationally by selecting the action with the highest expected value. It aligns with the jurisprudence of supreme courts regarding traffic forecasts as well. In this context, realistic expected values should be determined for environmental impact assessments rather than relying on political desires and wishful thinking. In summary, rationality axioms and Knightian probabilities address different aspects of decisionmaking and uncertainty. The mentioned rationality axioms pertain to evaluating options when there is a certain level of uncertainty or subjective/objective probabilities to assess. In contrast, Knightian probabilities deal with situations where uncertainty does not influence the outcome or no objective or subjective probability can be assigned.

Normative forecasts and magical thinking

A few thoughts should be dedicated to a particular form of forecasting. The so-called normative forecast is not classical but rather the design or depiction of a politically desired or undesired future state. Starting from this state, which does not automatically evolve, one can then contemplate what actions should be taken or avoided today to achieve the set normative state. This is fundamentally an exciting approach but must be distinct from reputable methods of futures research which as described above strive to generate plausible expected values and acknowledge uncertainty by indicating the range (statistically expressed as the confidence interval of the expected values). The leap from normative forecasts to magical thinking becomes especially small when failing to describe how this state can realistically be achieved. We are experiencing this right now, for example, in the discussion about a CO2-free world. Setting ambitious goals is one thing, but implementing them in a democratic country, considering legal perspectives (appropriateness, necessity and proportionality), is another.

From theory to practice

Not entirely unrelated to the challenges and strategies mentioned, one crucial topic must be addressed at the end. What makes a good forecast, and why are some forecasts useless? But first, let’s consider the criteria for good predictions, serving as a checklist for the next time you encounter forecasts: Do the authors of the estimates have a deep understanding of the underlying data and trends, as well as the ability to create and analyze complex models? Do they possess experience and expertise in the area to which the forecast pertains? Do they have access to current and accurate data? Is there a clear definition of the question at hand and the goal of the forecast? Do they have the ability to consider appropriately and transparently present uncertainty and risk? Are they free from ideological bias? Is there a critical evaluation and review of the results? Suppose the answer is yes, great! It doesn’t get any better than that. If the answer is no, why is that? Also, consider the following questions: Was sufficient time allocated for the task? Was the appropriate hourly rate paid? Did the lowest bidder win? Was the outcome predetermined before the assignment? Unfortunately, we often know the answers to these questions.

Outlook-learnings

Warren Buffett once said that forecasts reveal more about the person making them than the future. This is a good thing. As planners, we can discern a certain mindset or wishful thinking in every forecast, which also holds a particular value of insight. Combined with the influence of these individuals or institutions, they may be crucial guideposts. In the spirit of Prometheus, the forward thinker, humans can learn from our failures. At least we are credited with that ability. So, what can we learn? Anyone familiar with traffic and economic models knows about the difficulties involved. It is hard to put it more eloquently than Umberto Eco, who, loosely translated, said: The problem (of a good forecast) is as unsolvable as determining the exact location of a ping pong ball in the ocean on Wednesday, which we threw in on Monday. The conclusions drawn from this are modesty and well-founded or justifiable scepticism in dealing with statements, recommendations, calculations, forecasts, demands, etc., mainly regarding decisions that could significantly impact present and future generations. And by impacts, we refer to climatic but also economic and social effects. And, of course, it affects our values, especially regarding the peaceful resolution of differences in opinion.

It is undisputed that forecasts can never accurately predict what will happen in the future. Instead, forecasts should always be used to support decision-making and planning and should never be seen as the only possible truth. Nonetheless, it remains to be seen that planners have livelihoods to sustain, authorities grapple with time and budget limitations, politicians approach matters from their unique perspectives, and scientists continually strive for enhanced understanding. Consequently, we should remain open to reassessing and rectifying our viewpoints as new information and insights emerge.

In conclusion, this article highlights the complexities of traffic forecasting, emphasizing the challenges and limitations in predicting future transportation outcomes. It emphasizes the need to consider multiple factors and their interactions when formulating forecasts. Strategies for addressing uncertainty are discussed, including the importance of acknowledging knowledge limitations, distinguishing between known and unknown factors and considering subjective and objective probabilities. The article calls for a modest and well-founded approach to traffic forecasting, advocating for continuous learning, open-mindedness, and the consideration of uncertainty and risk in decision-making processes. By embracing these principles, policymakers, planners and researchers can navigate the complexities of traffic forecasting more effectively and make informed decisions that positively impact transportation systems.