Box seminar from 1996 remains visionary
However, even more enjoyable than the chuckle over obsolete fashions were Box’s timeless anecdotes, which I recorded religiously. For example, Professor Box mentioned how his father would comment on trivial differences: “A blind man would be glad to see it.” This drove home a point Box wanted to make on how being statistically significant did not necessarily lead to anything of practical importance. In fact, while working as a statistician at Imperial Chemical Industries (ICI), he banned the use of p-values by their industrial experimenters! (Box advocated the use of confidence intervals, instead.)
In my webinar last month on “10 Ways to Mess Up an Experiment & 8 Ways to Clean it Up”. I made this point (statistical significance versus practical importance) in a slide similar to that shown here. It accomplishes little to achieve a low p value for a change that is so small that it produces nothing of any practical importance. In today's age of robotic experimentation this happens more-and-more often due to the large number of runs -- in the hundreds or even thousands. On the other hand, plenty of experiments are still done in situations where runs are dear and not many can be performed. Then a big difference may be seen that fails the pre-ordained threshold level for p. In that case it often pays to investigate further.
“Even if the probability was 6% of not finding a crock of gold behind the next tree, wouldn’t you go and look?”
-- Quote from “An appendix featuring quaquaversal quotes … that embellish key concepts and enliven the learning process” presented by George E. P. Box, J. Stuart Hunter, and the late William G. Hunter in the second edition of the classic book Statistics for Experimenters: Design, Innovation and Discovery.
*You have to see this web site, at least the goofy glasses shown on the rotating eye-catcher, on spectacles through the ages.