Category Archives: Frequentist

An interesting fact: Probability 0 means “very rare” but NOT “absolutely impossible”

Consider the following sequence


Using some basic maths we can easily ascertain that the relative frequency of A goes to zero as the length of the sequence goes to infinity (i.e. its limiting value equals zero). Hence the (frequentist) probability 0 means infinitely rare but not absolutely impossible.

(graph source:

Tagged ,

The principles of Frequency Probability (Von Mises)

1) Unlimited Repetition : The theory of probability applies only to a practically unlimited sequence of uniform observations. Hence, the probability applies only to problems in which either the same event repeats itself, or a great number of elements are involved simultaneously. The probability, in a frequentist sense, is not synonymous to trustworthiness or to the degree of belief.

2) The Collective: A probability is attached to a class or a collective but nothing can be said about the probability of a single member of the collective.

3) Relative Frequency: The relative frequencies of certain attributes in a collective become more and more stable as the number of elements is increased and will converge to a fixed limit. This limit is “the probability of the attribute considered within the given collective”.

4) The Principle of Randomness: The limiting values of the relative frequencies must remain the same in all subsets which may be drawn from the original sequence in an arbitrary way. It is however crucial that the probability of the attribute considered in a collective is independent of all possible place selections. Note that if a sequence satisfies conditions 1-3 but not condition 4 then the limiting value will be called “the chance” of the occurence of the attribute rather than its “probability”.

The aforementioned principles of frequency probability are presented in a classic work of one of the most influential figures of the frequentist school of statistical though, Richard Von Mises (see Richard Von Mises (1957), “Probability, Statistics and Truth”, Dover Publications).


update: Michael P. O’Brien provides some further explanation of my summary  here.

Tagged , , ,

Bayesians vs. Frequentists

Lecture by Michael I. Jordan

Tagged , ,