Many mathematical disciplines call one result their fundamental theorem. Recall the fundamental theorem of algebra or the fundamental theorem of calculus. There are many more theorems named “fundamental theorems” for subdisciplines as exotic as, for example, ideal theory in number fields. What is missing, though, is something people agreed upon calling the “fundamental theorem of statistics”. Here, I will discuss a few candidates for this title, including the central limit theorem as well Bayes’ theorem, and finally argue that most promising is a historic precursor of the law of large numbers sometimes called “Bernoulli’s theorem”. For non-mathematical readers, this post can be read as a brief tour of some important theorems of statistics and their interpretations with regard to the questions of the “logic of statistics”, the possibility of inference from given data and the distinction of probability and frequency.
(This was originally posted on my old wordpress blog and not imported in full text.)