By Max Gunzburger
As is true for any rapidly developing field that is attracting ever increasing numbers of participants, uncertainty quantification for complex systems (such as those governed by partial differential equations) has become quite a competitive area of research. This being so, one endeavors, in one’s papers, talks, and proposals, to present one’s accomplishments in UQ in as favorable a light as possible. This is often difficult, given that one’s accomplishments may not, in fact, be very impressive. To help those in such difficulty, especially junior researchers just getting into the UQ business, this article provides 15 suggestions for making one’s results look wonderful, when they are actually nowhere near wonderful.
1. Compare your results with the slowest possible version of Monte Carlo, never with someone else’s method. This way, even you will have a difficult time not looking good, as long as you don’t go to a dimension higher than 3, if your method is really bad, or higher than 1, if it is a total disaster.
2. Claim that you can do 1000-dimensional problems and that no one else can do this, but present results for a problem that has an effective dimension of only 4 or less; totally obfuscate what you are doing with lots of mumbo-jumbo so that a naive audience really believes you are doing a 1000-dimensional problem.
3. Tailor your examples so that they are advantageous for your method and disadvantageous for your competitors’ methods, even if all fair examples have the opposite result. bBt heed this warning: finding such examples for a lousy method requires some cleverness on your part, so you may not be able to do so.
4. Invent a fancy name for a method you claim to have invented and that you also claim to be new and transformational, but that really is a well-known method given a new name. This way, everyone will think you are really doing something new and important.
5. Using the same problem as for the previous recommendation, invent a fancy name for another well-known method and again make outlandish claims, but completely fail to mention that your previous claims turned out to be totally unfounded. In fact, don’t mention the previous method at all; this way, no one will be reminded of your previous failure.
6. Tailor your claims to your audience; if the audience is made up of, say, probabilists, make outlandish claims about how good your algorithms are; if they are numerical analysts, make outlandish claims about how your results involve deep and innovative probability. This way, you take advantage of the audience’s ignorance without revealing your own.
7. Never show what happens as the number of samples or other degrees of freedom increase. This way, the lack of convergence of your method is not revealed, and, if you also follow recommendation 8, no one will care because the errors you report are so small.
8. Without pointing out that you are doing so, show only absolute and not relative errors, making sure to neglect to mention that the solution you use for comparison has a minuscule norm, certainly no bigger than 10−5. This way, you can make your non-convergent method look really good by showing that, using only a handful of samples, you can get an error of order 10−5.
9. Blame the data, and not the shortcomings of your method, for its poor performance. Most people will believe you when you say that if you had better data, your results would be better.
10. Always do your calculations on a supercomputer, even if you just use one processor. This allows you to use any of David Bailey’s stratagems to hide the poorness of your results.
11. When comparing your results with experiments, make sure to pick experimental data that comes with huge error bars—so huge, in fact, that your bad results still fit within them. You can then claim that your results always fall within “experimental error.”
12. Give plots that show your method converging to a tolerance of less than 10−12 with just a few samples, but do not mention the preprocessing step that required a humongous number of samples, and certainly do not mention that the method converges to the wrong solution. The 10−12 figure will dazzle your audience to such an extent that they will not think of questioning your claims.
13. Show results for only two parameter dimensions or, even better, for a single parameter, but add that your method obviously scales well to much higher dimensions. Use of the word “obviously” will keep people from questioning you, fearful that, by missing something that is obvious to everyone else, they will look stupid.
14. Use no math. That way, you won’t make elementary mistakes and no one will figure out that you don’t know any math.
15. Of course, if all else fails, show lots of pretty pictures and especially movies, but don’t mention anything about the performance of your methods.
*David Bailey of Lawrence Berkeley National Laboratory initiated a series of missives of this nature with “Twelve ways to fool the masses when giving performance results on parallel computers” (Supercomputing Review, 1991). That paper spawned many imitations. Apologies to him for yet another blatant echo.
Max Gunzburger ([email protected]) is the Frances Eppes Eminent Professor in and chair of the Department of Scientific Computing at Florida State University.