San José State University |
---|
applet-magic.com Thayer Watkins Silicon Valley & Tornado Alley USA |
---|
for Computing Inverse Probabilities |
---|
In the eighteeenth century mathematicians around Europe were working out the details of probability. This always took the the form of given a condition what are the probability of various events occurring. Thomas Bayes was a Presbyterian minister in England at a time when Christian denomination like the Presbyterians were being pursecuted for not supporting the Church of England. Mathematicians and their mathematics from such sources were being denounced. Thomas Bayes decided to enter the dispute. He published a pamphlet defending Isaac Newton.
Thomas Bayes |
---|
In the course of his mathematical studies Bayes realized there was an interesting question in probability theory that was not being answered. That question was, "Given the occurrence of an event what were the probabilities of it coming from various possible sources. For example, suppose the flipping of a coin gave ten heads in a row. What is the probability that the flipped coin is a double headed coin versus a reular coin. This came to be called an inverse probability problem.
Bayes did not fully answer this question but the formula which evolved from his ideas is
where P(C_{i}, E) is the probability of event E given condition C_{i} whereas P(E, C_{i}) is the probability that condition C_{i} is responsible for the event E occurring. This is called the Bayesian Rule although it was developed by Pierre Simon Laplace, the brilliant 18th century French mathematician.
For a regular coin the probability of getting ten heads in row is (1/2)^{10}=1/1024. For a double headed coin the probability is 1.00. Thus the probability that the coin is double headed according to the Bayesian Rule is
The probability that it is a regular coin is
If the coin is flipped an eleventh time and tails comes up the probability that the coin is double headed goes to zero and that it is a regular coin goes to one.
The problem is how should the results of a bayesian computation be interpreted. The answer is that they are in no way probabilities; they are degrees of confidence. The computation of degrees of confidence may be identical to that for probabilities but they are not the same conceptually.
Let D(C_{i}, E) be the degree of confidence that condition C_{i} prevails given only the information that event E has occurred. Then the degree of confidence is defined as
Then the degrees of confidence for two separate events E_{1} and E_{2} may be derived. First in order for those degrees of confidence to be consistent with the above definition they must be given by
However if E_{1} and E_{2} are independent then
For convenience let ΣP(C_{j }, E_{1}) and ΣP(C_{j }, E_{2}) be denoted by S_{1} and S_{2}, respectively.
Then
The event E_{1} could stand for all of the prior events and E_{2} for an additional event. Replacing E_{1} and E_{2} with E_{p} and E_{a} for prior and additional, respectively, the rule for modifying prior degrees of confidence to take into account new information is then
D(C_{i}, E_{p}&E_{a}) = D(C_{i}, E_{p})*D(C_{i}, E_{a})/ [ΣD(C_{j }, E_{p})*D(C_{j }, E_{a})]
This is what is usually called the Bayesian Rule.
Suppose the same event E_{a} occurs over and over again n times. Let D(C_{i}, E_{p}) and D(C_{i}, E_{a}) be abreviated as D_{pi} and D_{ai}, respectively. Then
Let D_{aM} be the maximum degree of confidence for the event E_{a}. It is assumed that this maximu occurs uniquely among the possible conditions. The numerator and denominator of the RHS of the above equation may be divided by D_{aM}^{n} to give
Provided that D_{pM}≠0, it then follows that
lim_{n→∞} D(C_{i}, E_{p}&nE_{a}) = 0 if i≠M
and
lim_{n→∞} D(C_{M}, E_{p}&nE_{a}) = 1
In other words asymptotically the degree of confidence that the condition of world is M approaches certainty. Notably the prior degrees of confidence are asymptotically irrelevant.
Proof:
For i≠M the numerator of RHS includes the less than unity ratio (D_{ai}/D_{aM}) raised to the power n which goes to zero as n increases without bound. The denominator, on the other hand, contains the term ((D_{aM}/D_{aM})^{n}=1 which precludes the denominator going to zero as n increases without bound. Instead the limit of the denominator is D_{pM}. For i=M the limit of the numerator is also D_{pM} and hence, providing that D_{pM} is not zero, their ratio is unity regardless of the value of D_{pM} or any of the other prior degrees of confidence.
This result can be extended.
(To be continued.)
HOME PAGE OF Thayer Watkins |