A recent question from a statistician in Germany led me to the realization that the F-test of analysis of variance (ANOVA) fame is in serious need of an update.

What the F-ratio does

The F-ratio, created by Sir Ronald Fisher around 1925, is a generalization of Student’s t-test for comparing two averages. With the F-ratio a set of k averages can be compared to see if any differ from the others. Say we have k experimental conditions and we collect n observations for some response variable at each of these k conditions. Do the experimental conditions affect the response variable? If so, then we would expect the k averages to differ. If not, then the k averages should all be about the same.

 …