The Three Eras of Statistics

Source: Efron (2012), Large-Scale Inference

1. The era of huge census-level datasets where there were simple but important questions:

• Are more males or females born?
• Is the rate of insanity increasing?

2. The classical period where intellectual giants such as Pearson, Fisher, Neyman, Hotelling, and their successors developed the theory of inference capable of extracting every drop of information from a scientific experiment. The questions addressed still tended to be simple:
   • Is treatment A better than treatment B?
3. The era of mass scientific production, in which new technologies typified by the microarray allow a single team of scientists to produce high-dimensional data. But now the flow of data is accompanied by a very high variety and quantity of questions, which the statistician is tasked with answering using estimates or hypothesis tests.

Microarrays

• For many statisticians, microarrays provided an introduction to Large-scale data analysis. These were revolutionary biomedical devices that allowed the evaluation of individual activity for thousands of genes simultaneously. The underlying idea is to perform thousands of simultaneous hypothesis tests, done with the prospect of finding only a few interesting genes within a haystack of null cases:

The Four Eras of Data

Source: Jeff Leek’s, post

1. The era with not much data: before around 1995, we could generally collect a few measurements at a time (so there was a much lower capacity to collect data compared to the current one). The whole point of statistics was to try to optimize the information from a limited number of samples and try to derive meaningful information with methods such as maximum likelihood and minimum variance unbiased estimators .

2. The era of many measurements on few samples: in this phase there were important developments in the field of biology with the development of microarrays that allowed measuring thousands of genes simultaneously. In this phase the main problem turned out to be that of the previous era but the greater quantity of data causes more noise. Here we see the development of methods for multiple testing and regularized regression to separate the signal (i.e., the useful information) from the noise (fluctuations due to chance).

3. The era of few measurements on many samples This era partially overlaps with the previous one. Large-scale data collections by EMRs and Medicare are examples where there is a huge number of people (samples) but a relatively modest number of measured variables. Here there is a large focus on statistical methods to know how to model different parts of the data with hierarchical models and separate the signal from the noise with model calibration .

4. The era of all the data on everything This is an era that is the one we are in where Facebook, Google, Amazon, NSA and other organizations have thousands or millions of measurements on hundreds of millions of people. The main problem in this era, beyond the simple calculation with such a large amount of data, and the problems of the previous eras, is the risk of tying predictions too closely to the data being studied (crazy overfitting).