Box seminar from 1996 remains visionary

While cleaning out files this week, I found my long-forgotten notes from “Design of Experiments for Discovery, Improvement and Robustness” co-presented by DOE guru George Box in March of 1996 – over a dozen years ago. The first thing I noticed was the photo roster showing how huge my spectacles and others were in that era.* The ones I have now are so narrow I cannot see, but at least they are fashionable! (There is an added factor: In 1996 I did not need progressively-lensed bifocals, although they would have benefited from the goggly glasses of that time.)

However, even more enjoyable than the chuckle over obsolete fashions were Box’s timeless anecdotes, which I recorded religiously. For example, Professor Box mentioned how his father would comment on trivial differences: “A blind man would be glad to see it.” This drove home a point Box wanted to make on how being statistically significant did not necessarily lead to anything of practical importance. In fact, while working as a statistician at Imperial Chemical Industries (ICI), he banned the use of p-values by their industrial experimenters! (Box advocated the use of confidence intervals, instead.)

In my webinar last month on “10 Ways to Mess Up an Experiment & 8 Ways to Clean it Up”. I made this point (statistical significance versus practical importance) in a slide similar to that shown here. It accomplishes little to achieve a low p value for a change that is so small that it produces nothing of any practical importance. In today's age of robotic experimentation this happens more-and-more often due to the large number of runs -- in the hundreds or even thousands. On the other hand, plenty of experiments are still done in situations where runs are dear and not many can be performed. Then a big difference may be seen that fails the pre-ordained threshold level for p. In that case it often pays to investigate further.

“Even if the probability was 6% of not finding a crock of gold behind the next tree, wouldn’t you go and look?”

-- Quote from “An appendix featuring quaquaversal quotes … that embellish key concepts and enliven the learning process” presented by George E. P. Box, J. Stuart Hunter, and the late William G. Hunter in the second edition of the classic book Statistics for Experimenters: Design, Innovation and Discovery.

*You have to see this web site, at least the goofy glasses shown on the rotating eye-catcher, on spectacles through the ages.