“At any given time we know what we are doing….”

Disclaimer: Both Robert MacCallum (e.g., 2003) and George Box (e.g., 1979) have written extensively about the value of models and I will basically steal parrot their ideas in this post. Moreover, I did not sleep much last night…

Let the postmortem on the 2012 election begin! One story will likely involve the accuracy of well-conducted polls and the success of Nate Silver’s methods over “gut-based” methods favored by pundits and campaign workers.  Not surprisingly, I like much of this story as is nicely summed up by this cartoon (thanks to Skip G. for posting this one on “the” Facebook).

But what if Nate Silver was wrong? What if we woke up today and Romney won 303 electoral votes and Obama lost the election?  I think Mr. Silver would have been in a much better position than his “gut-based” critics are in today. The reason boils down to the advantages of models.  It is really useful to have a formalized recipe for prediction. To quote Box (1979): “The great advantage of the model-based over the ad hoc approach, it seems to me, is that at any given time we know what we are doing.”

I happen to like the model for science depicted by George Box (see Figure A1 in his 1976 Science and Statistics paper).  The basic idea is that errors drive the accumulation of knowledge in an iterative cycle. Learning is produced when there is “a discrepancy between what tentative theory suggests should be so and what practices says is so” (p. 791).  In other words, there is something to be gained when predictions from models and empirical facts disagree.  These errors lead to better models, at least in the ideal case.

So if Mr. Silver’s model was gravely wrong, he could have spent the next days and weeks figuring out where his model went wrong.  He has his predicted values and he has the actual values.  He can test alternative models to find ones that outperformed his original model. He is in a good position to learn something.  Compare his plight with that of the gut-based pundit.  How are they going to figure out why their predictions failed?  What are they going to learn?

Moral of my story: Models rule.  I think there might be a bigger lesson in here for “soft” psychology but I am too tired to express it properly.

Advertisements

Author: mbdonnellan

Professor Social and Personality Psychology Texas A &M University

1 thought on ““At any given time we know what we are doing….””

  1. Here’s one example, rather than a bigger lesson: Models of psychopathology are crucial, as it informs diagnosis and treatment, among other things. I recently read for a class an excellent paper by Barlow (2000) where he puts forward a well-researched model of the structure of mood disorders. His model is informative and helps overcome the…issues with the categorical approach of the DSM.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s