Wednesday, December 20, 2006

How Statistics Can Shed Light On Human Behavior

This is a reply to Michael Williams' blog on this article in the NY Times about the "whys" of human behavior. His spam filter didn't like my reply so I posted it here. This phrase from the original article is really what I am responding to:

Research psychologists have known for decades that it is very difficult to determine causation in mental life and thus, of behavior. For one thing, we can never perform an experiment. Take my patient Karen, 50, who spent most of the 1990s smoking crack. She is certain that the decade-long binge would never have happened had her mother not died when she was 12. We will never know if she is right because we cannot rewind Karen’s life, play it again, and see what would have happened if her mother had lived.
Well its true we cannot back up in life to perform experiments, but "mock" experiments can be performed with surprising results. I know the human psyche is extremely complex but other areas of science have proven to be extremely succesful when using sound statistical methods for performing designed experiments. Even if you cannot perform a physical experiment and re enact "what if" scenarios, much can be learned.

I heard this guy from Georgia Tech's ASDL talk at the JMP user conference this year and he was talking about this exact thing. In his field, they cant exactly build missles and aircraft then go perform experiments against live terrorists to track the results, so they use powerful statistical software to run "what if" scenarios. (It's Patrick Biltgen and his podcast is worth listening to on that page as well but its nothing like his speech at the conference.)

I'm not saying that statistics can or ever will be able to figure out the "whys" behind human behavior. But you can get a pretty good idea if you know what you're doing.

So while we might feel that the 'defective father' theory of atheism (for example) doesn't adequately explain the psychological behavior of atheists, the facts are there and.. well I'm no statistician but... let's face it, poor relationship with fathers tend to lead to bad things (like atheism)... Same thing with homosexuals... Same thing with criminals etc...

People dont seem to like statistics to be accurate (because in some cases they dont fit the bill sure). But someone who has no earthly clue about statistics is very likely to reply to me and say "But wait I know a criminal with a good family" or "wait I know a homosexual who grew up in a conservative Christian family", in fact I'm constantly amazed at how 99% of people have absolutely no understanding of how statistics work. But as much as we may dislike the notion... statistics can tell us a lot about the causes of human behavior.

But then again, statistics aren't usually very politically correct. | Read this post in Japanese.

No comments: