Wednesday, December 14, 2005

Inference and Bayes' Theorem

Say you want to calculate the probability that your theory T is correct given observations O. According to Bayes' Theorem, the probability of T given O is

P(T|O) = P(O|T) P(T) / P(O)

where P(O|T) is the probability of O given T, and P(T) is the prior probability of T without accounting for observations O. P(O) is the prior probability of O.

P(O|T) is the degree to which T predicts O. If T doesn't predict O then either (i) T predicts that O is unlikely/impossible, or (ii) T is uncorrelated with O.

Let's consider case (ii) where T is consistent with O but doesn't preferentially predict O.

If T is consistent but uncorrelated with O, then the probability of O given T is just the probability of O. That is P(O|T) = P(O). Consequently, P(T|O) = P(T). In this case, you cannot make any inference of T from O.

More intuitively, the probability that a theory T is inferred by observations O is proportional to the force with which T predicts O. If T doesn't preferentially predict O, you are not justified in making an inference from O to T.

For an Intelligent Design theory to be inferred from the data, it must be specific enough to predict something about the observations.

One objection to this conclusion would be to claim that an arbitrary T can always be fitted to O such that it preferentially predicts O. For example, one might claim that "T is the generic theory that there is an unknown agent that is responsible for O." However, the problem here is that T is no longer an inference from O. It is a paraphrasing of O. (After all, we originally set out to answer the question "what is the unknown agent responsible for O?")

So, how do we know whether or not T is just paraphrasing O? We can know this by counting parameters. If we have N data points, we need N parameters to paraphrase the data without making any inferences. For example, we can always write the next number in a sequence as the sum of the previous number and some parameter tuned to give us the correct answer.

Therefore, an inference is a theory T with fewer parameters in it than O. Since there are fewer parameters in T than in O, some proper subset of O must be predicted by T from the remainder of the data in O.

Generic ID makes no predictions, not even within the existing data we have. It has more free parameters than any amount of data we throw at it. It is, at best, a paraphrasing of the data.

We can certainly infer the action of intelligent agency from some data sets, but only when our intelligent agent theory is predictive in some way.


rob said...

Hi Doc,
I'm interested in the probability end of this post. For a couple of years now it keeps coming up for me.Am I right in thinking that probability is the percentage of possibility? And if that is so aren't the only two numbers that really have meaning are 0 and 100% given one particular event?

If someone tells me I have a sixty percent chance I always think I have a 0 or 100% chance.

How important is probability if a possible chance of 1 in 100 becomes a real event.

Longshots happen,right?

Oh and happy Winter Solstice to you.

Doctor Logic said...

Hi rob,

Probability is usually equivalent to the odds of something happening. It's either going to happen or not, but it's important to know how likely it is to happen, especially if you're in the insurance or casino business!

If you draw five cards from a 52-card deck, the odds of getting four of a kind are 4164 to 1. That means that on average, if you repeatedly shuffle the deck and draw five-card hands, you'll only draw a four-of-a-kind once in 4164 draws. Of course, there are no guarantees, so you could get lucky and get two four-of-a-kind draws in a row. It's just not likely.

If you write "4164 to 1" as 1/4164, you get "0.00024" or 0.024%.

There's another way to think about probability, and thats as a level of confidence in a statement. If I say, "my first draw will be four-of-a-kind," you (and I) should only have 0.024% confidence that I'm right. However, if I draw four Kings on my first draw, then, after my draw, I have 100% confidence in my original statement. If I draw a pair of deuces, then, after my draw, I have 0% confidence in my original statement.

Am I answering your question?

And happy Winter Solstice to you too, rob!

rob said...

Yessir you did. The trick is to know when to play the odds and when to just go for it regardless of the odds.
I guess I'll go back and read that Game Theory theory again while it's fresh in my mind.