Bayes' Theorem

(idea) by Actuary X Fri Jan 21 2000 at 19:31:05
This extraordinarily useful theorem tells us that P(B|A) = P(B)* P(A|B) / P(A) ... which is easily seen by multiplying both sides by P(A). P(B|A) is the conditional probability of an event B occurring, given that event A has occurred.
(idea) by ariels Sun Mar 26 2000 at 8:28:19
This theorem is somewhat unintuitive, and often used in this nonobvious way: Suppose we've made an observation O. We hypothesise that O happened because of some hypothesis H (e.g. O can be "a coin toss came up 6 times heads and 4 times tails", and H can be "the coin is a fair coin"). We know how to calculate P(O | H); that's elementary probability. But what we're really interested in (especially if we want to apply the Neyman-Pearson lemma) is the converse probability P(H | O). By Bayes' Theorem, this is just
  P(H | O) = P(O | H) * P(H) / P(O)                        (*)

where we know how to calculate P(O | H); we might also be able to estimate P(H), but usually both P(H) and P(O) are very hard to estimate (you need to make much stronger assumptions about the real world for that, since the luxury of assuming H -- what we'd like to be true -- is gone!). However, for Neyman-Pearson P(O) doesn't matter, and we can (often) "swallow" P(H) into a parameter.

But think what (*) is telling us! It says how to calculate the probability that our hypothesis is true, given what we've observed. No wonder it's so hard to discover P(H) and P(O) -- they tell us how to check if a theory about the real world is true!

Y'know, if you log in, you can write something here, or contact authors directly on the site. Create a New User if you don't already have an account.