SIAM News Blog

The Models are Incomplete, the Intuitions are Unreliable

By Karl Kempf

Business decision-making is the focus of my work. I am especially interested in those business decisions where the difference between a good answer and a poor one is a billion dollars or more. Although such decisions are relatively common today, they have only been so for roughly the last 140 years. The Industrial Revolution set the stage for the so-called “big company.” Old businesses merged and reorganized while entrepreneurs formed new businesses to leverage the confluence of mass production and mass distribution, meant to stimulate mass consumption. Standard Oil was established in 1870 and quickly became the largest oil refining company in the world. General Electric resulted from the merging of two small companies in 1892 and grew to become one of the 12 original companies included in the Dow Jones Industrial Average. In 1901, the combination of three companies resulted in U.S. Steel. Capitalized at $1.4B, it was the world’s first billion-dollar corporation, and at one point the largest corporation in the world. The flow of new technologies continued to grow throughout the 20th century and is expanding even more rapidly today. At $486B, Sinopec Limited (the China Petroleum and Chemical Corporation) topped the 2014 list of largest companies. Even at Intel Corporation, whose 2014 revenue was only $56B, billion-dollar decisions are made on a regular basis. Such decisions include determining what features to incorporate in the next product, when and where to build the next factory, how much inventory to hold, and where to hold it, among others.

Figure 1. Wafer with Intel® Xeon® processor E7 v2 family chips, each made of 4.3 billion 22nm transistors. Photo credit: Intel Newsroom.
Over this 140-year time span, the intuition of senior management has played a major role in making big-business decisions. In this case, “intuition” refers to the bringing of one’s entire life experience and learned heuristics to bear on current circumstance. This capability has been developed for millions of years in the limbic or emotional layer of the human brain, augmented only in the last few hundred thousand years by the neocortex or rational layer. These two layers, or systems, are so interconnected that it is impossible to use them independently. It is also important to note that, as most people have likely experienced, sometimes intuition is useful and sometimes it is not. Experts have arrived at a consensus on this point [1]. When a) the situation under consideration has structure, b) decisions made can be evaluated after the fact by some feedback loop, and c) the person supplying an intuitive response has experienced many repetitions, intuition can be very valuable. But when one or more of these conditions are violated, as they are in the vast majority of big-business decisions, intuition is unreliable.

Applied mathematicians would likely recognize that many big-business decisions involve large-scale stochastic combinatorial optimization problems. Instead of the emotional brain, the rational brain, or a combination of both, why not use today’s high-speed digital hardware to solve strong algorithms encoded in modern software? Mathematical optimization techniques can solve large problems in minutes or hours. Powerful search techniques, like genetic algorithms and simulated annealing, can also provide near-optimal solutions. But there are issues. The Turing machine of 1937 became practical with the invention of the transistor in 1947 and the integrated circuit in 1959. The circuit then begat the 2,300 transistor microprocessor of 1971, which morphed into the multibillion transistor processor of today. Similar timelines exist for algorithms and software. However, a few decades of development for the computational approach versus the developmental timeline for purely mental decision-making is apparently inadequate to convert senior managers. This is especially true of those who believe that their superb business intuition propelled them to their current positions. Another issue concerns modeling. Although optimization techniques may quickly return an optimal or near-optimal solution to the model of the business problem, senior managers are correct in thinking no model can possibly include all important factors or relationships. Their skepticism is well founded.

Figure 2. Intel's wafer fabrication facilities in Chandler, Ariz. were recently converted to high-volume 14-nanometer processes, supporting a wide range of high-performance to low-power products including servers, personal computing devices and others for the Internet of Things. Photo credit: Intel Newsroom.

The work of two Nobel Laureates begins to bridge the gap between unreliable intuition and incomplete models. Theoretical economists had assumed that humans exhibit perfect rationality in decision-making, collecting all relevant data and considering all rational alternatives. Herbert Simon showed, specifically in business decision-making, that our rationality in considering possibilities is bounded by incomplete data and inadequate computational resources (mental or electronic), as well as the urgency of time [4]. Amos Tversky and Daniel Kahneman went even further and identified a wide variety of biases in human decision-making [5]. Decision processes are not only bounded, but are also inescapably biased. Yet the Laureates left two important practical questions unanswered. How bad are the results from our bounded, biased processes, and (assuming the results are too bad) how can they improve?

A broad portfolio of decision support projects completed at Intel Corporation allows us to estimate, based on data, an answer to the question “how bad are we?” This is a measure of improvement achieved by migrating from weak methods, which in the vast majority of cases can be characterized as automating intuition with spreadsheets to strong methods using powerful algorithms. We use these results when discussing new projects with our senior management. We commit to a range of 10-15% in improvements to the quality of decisions; this is measured by increasing revenue, decreasing cost, or some combination of both. Regarding the amount of time required to reach a decision, we commit to a range stretching from a 5x decrease in process duration for exploring the traditional number of business scenarios to a 5x increase in the number of business scenarios explored in the traditional process duration. Our most successful projects have realized a 25% improvement in quality and a 10x reduction in duration of the decision process. 

Figure 3. Intel's 300mm 45nm wafers like the one shown here are used to make its newest dual and quad-core processors that are made up of hundreds of millions of the company's new 45nm transistors with Hafnium-based high-k metal gate silicon technology. Photo credit: Intel Newsroom.
When implementing these decision support systems, we have explored three distinct approaches to answering the question “how do we do better?” Initially we employed strong algorithms in brute force mode to try to overwhelm intuitions. After users helped ratify initial conditions and specified goals, our system produced a solution for complex tactical problems, like factory floor execution, that was carried out without modification. With appropriate user training to understand the algorithm and feedback loops to improve it, this worked well. But upon addressing more strategic problems, such as demand forecasting, we began leveraging intuitions in our decision support systems in ways intended to reduce the impact of biases. For example, we employed “wisdom of crowds” and “decision markets” to great advantage. Recently, in building decision support systems for our most senior management, we are making explicit use of intuition. We model and solve the business problem, ultimately providing a small set of very good solutions and a suite of graphical interrogation and what-if tools. Users can apply their business intuitions to select between the candidates or generate and evaluate additional candidates. The analytics inform the intuitions and the intuitions inform the analytics. This third approach has generated very positive outcomes on multi-billion dollar decisions [3], much improved over previous methods.

These are encouraging results, but we are still only in the early stages of understanding how to optimally combine ancient and deeply-ingrained human mental processes with modern computers and algorithms [2]. The desire for faster, better decisions as a competitive business advantage drives our experimental work, but we continue to hope for a strong theory to offer guidance. 

[1] Kahneman, D., & Klein, G. (2009). Conditions for intuitive expertise: a failure to disagree. American Psychologist, 64(6), 515–526.

[2] Kempf, K. (2015). Applied mathematics for business decision making: the next frontiers, presented at 8th Inter. Congress on Industrial and Applied Mathematics, Beijing, 2015, 135-158.

[3] Sampath, S., Gel, E. S., Fowler, J. W., & Kempf, K. (2015). A Decision-Making Framework for Project Portfolio Planning at Intel Corporation. Interfaces, 45(5), 391-408.

[4] Simon, H. A. (1972). Theories of bounded rationality. In C. McGuire & R. Radner (Eds.), Decision and Organization, (pp. 161-176). Amsterdam: North-Holland Publishing Company. 

[5] Tversky, A., & Kahneman, D., (1974). Judgment under uncertainty: heuristics and biases. Science, 185(4157), 1124-31.

Karl Kempf is a senior fellow and director of Decision Engineering at Intel Corporation. He is a member of the National Academy of Engineering, a fellow of both the IEEE and INFORMS, and serves as a research adjunct at Arizona State University.

blog comments powered by Disqus