Friday
Jun012012

That Way Madness Lies

Prof. Jacqueline Stevens of Northwestern has written an opinion piece for the New York Times in which she says she is sympathetic to Congress's attempts to block the National Science Foundation from funding Political Science research, because Political Scientists have turned out to be very bad at predicting major international events such as the Arab Spring or the fall of the Soviet Union. Here is my response.

This article is ridiculous. As Prof. Stevens quotes Popper saying, it's possible to make accurate predictions only in closed, simple, controlled systems, and we know the world isn't one of those. She's not the only person besides Popper smart enough to have noticed that. There's no way any one person could ever know enough about a particular situation to say "I think there will be a revolution there if a young person chooses to set himself on fire. Otherwise, things will just keep going." Guess who else can't make predictions like that? Biologists. When is this species going to evolve, and how? YOU DON'T KNOW? What the hell is the point of your "science" then?

The "scientific benchmark" is NOT predicting the future. Scientific prediction is about whether your hypotheses "predict" what happens in your data. The point of political science research is to help us understand why things happen the way they do, and to help us produce better policy. Sometimes that works, sometimes it doesn't. Some research really is just confirming trivialities, but it is the nature of research to fail far more often than it succeeds. I believe I've read somewhere that chimps throwing darts are actually BETTER at prediction than economists, meteorologists, and sportscasters, so it looks like we're doing pretty well.

Furthermore, her solution is ludicrous. So sometimes the people handing out money influence what the people receiving the money can study. Wow. What a newsflash. So we're supposed to do what? HAVE A LOTTERY. So instead of there possibly being some inappropriate influence, there will be no vetting process at all.

I'm fine with people criticizing political science. I have some problems with it myself. By all means, let's talk about disciplinary biases towards quantitative research methods; let's talk about how bad a lot of qualitative research is; let's talk about how lots of political scientists don't study anything that would be useful to policy-makers; let's talk about how the peer review system has some major flaws. But as far as criticisms go, the one that we didn't predict big stuff happening and therefore are a bunch of failures is pretty stupid. For one thing, most political science research is probabilistic, not deterministic, which means that even if we wanted to make predictions they'd all be in terms of what's more or less likely to happen, not what WILL happen. Plus, there is a big difference between polling political scientists about what they think will happen, and doing a systematic study of political science research to see how accurate its generalized predictions would be in particular situations. Prof. Stevens doesn't bother with that, but I'd be willing to bet we wouldn't do that badly.

It is important to have government funding for political science research. Why? First, because research is inherently inefficient, thus unlikely to be funded by profit-motivated entities unless there is the possibility of a huge return (e.g. chemistry/pharmaceuticals). Second, because our only client is the government/public. Oh, and which kind of research is less capital-intensive and needs far less grant money? The quantitative numbers-crunching kind that Prof. Stevens thinks is so pointless. So: less government money = EVEN MORE QUANT BIAS.

I know Prof. Stevens is a smart lady, and she is in intelligent company when she complains that Political Scientists aren't predicting big stuff (John Lewis Gaddis is no dummy). To claim in one breath, though, that prediction is impossible and that political scientists are failures for not predicting accurately, is ... unhelpful.

 

I had a thoughtful response to the above. Here it is, with my further response:

@Lindsay, you stated, "The 'scientific benchmark' is NOT predicting the future." Then followed it with, "Scientific prediction is about whether your hypotheses "predict" what happens in your data. The point of political science research is to help us understand why things happen the way they do, and to help us produce better policy."

Help us produce better policy. I am stuck on that. So what does the scientific benchmark do if it does not "predict the future?" How does it help us produce better policy? Aren't we producing better policy so we can deal better with the future?

I will side with Prof. Stevens. Political Science has risen to become sacrosanct around Washington, yet the greatest policy failure—there is no articulated Grand Strategy driving US foreign policy, we bumble from one surprise to the next—is because a bunch of mountebanks have gone to Washington and promised to have the cure, advising from without from think tanks, academia, or newspapers while hiding behind fancy degrees and peer-reviewed papers (a stark exception being the neocons). They have no skin in the game, and when proven wrong claim that it is unfair to be expected to "predict the future" with a model never intended to do that. Guess what? Their models shouldn't be used to guide the formulation of policy either.

Political science has a place, and it has great value. But we should be careful to not ascribe to it a status it does not deserve, or a role it cannot fulfill.

 

LPC: in order to make good policy, we need to know what works and what doesn't. That is the purpose of scientific research: to determine the conditions/factors that seem to affect the outcomes we care about. We then create policy in an attempt to manipulate those factors in order to produce the outcomes. E.g. if we know that non-violent protest movements are far more likely to succeed than violent ones, we can encourage non-violence, oppose protests turning into armed rebellion, etc. If we know from research that intervention in a civil war tends to produce a lasting peace only if the intervenor is willing and able to commit to a significant presence in-country for a significant amount of time, that tells us something about what we may or may not want to do if there's a civil war going on that we care about. That doesn't mean that we can predict when a protest movement is going to happen, or whether it will turn violent, or when a civil war is going to break out. It means we have a POLICY when it happens. I didn't see anyone claiming that epidemiology was pointless when they failed to tell us all that swine flu was going to happen. What epidemiology helps us do is try to put measures in place to prevent epidemics, and to deal with epidemics when they happen.

I disagree with your assessment of the problem. Throughout its entire history the US has only rarely had something that could be called a grand strategy, and this is hardly the result of political scientists as a group (a discipline that has existed only since the 1950s) claiming to have a grand strategy, hoodwinking everyone in Washington with it, and then being wrong. Different presidents have listened to different advisors. That has always been the case. Some of those advisors have been political scientists, and some have not. Some presidents have listened to political scientists, and some have not. Some political scientists have been right, and others wrong. The idea that our current problems are either new or created by the members of a single academic discipline seems a little ... overwrought.

I completely agree that we shouldn't ascribe Political Science magical status ... the same way we shouldn't ascribe doctors magical powers to know what's wrong with us or geologists magical powers to predict earthquakes (can they do that? Not usually). I also agree that it should not be asked to fulfill a role it cannot. What I find ridiculous is the dual claim that prediction is impossible and that Political Scientists are failures because they cannot predict. Like military experts, we advise. Anyone wandering around claiming to make major predictions about wildly complex situations is free to do so, but that is his/her educated opinion. Not science.

One last thing: I'm not saying that prediction is impossible or that we should never attempt it. I'm saying that our ability to predict is limited, should be used carefully and with humility, and is emphatically NOT the benchmark for whether our research is useful to society.

PrintView Printer Friendly Version

EmailEmail Article to Friend

References (36)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
    Response: Updated Focus
    excellent post, very informative. I'm wondering why the other experts of this sector don't understand this. You must proceed your writing. I am confident, you've a great readers' base already!
  • Response
    Response: check over here
    Lovely Web page, Carry on the excellent work. thnx!
  • Response
    999 An important portion of each lady wardrobe are her shoes.
  • Response
    999 A necessary portion of every lady wardrobe are her shoes.
  • Response
    999 An essential part of each woman wardrobe are her footwear.
  • Response
    999 A vital portion of each lady wardrobe are her shoes.
  • Response
    999 A vital element of every female wardrobe are her footwear.
  • Response
    999 An essential part of every lady wardrobe are her sneakers.
  • Response
    Response: ugg outlet online
    999 A vital aspect of each girl wardrobe are her footwear.
  • Response
    Response: ugg outlet sale
    999 An essential half of every female wardrobe are her sneakers.
  • Response
    999 A necessary piece of every female wardrobe are her sneakers.
  • Response
    999 A vital aspect of every lady wardrobe are her sneakers.
  • Response
    999 An important portion of each girl wardrobe are her shoes.
  • Response
    999 An essential element of each lady wardrobe are her footwear.
  • Response
    999 An essential element of every lady wardrobe are her footwear.
  • Response
    999 An important component of each lady wardrobe are her shoes.
  • Response
    999 An essential aspect of each female wardrobe are her shoes.
  • Response
    999 A necessary component of every woman wardrobe are her shoes.
  • Response
    999 A necessary half of each girl wardrobe are her shoes.
  • Response
    Response: cheap genuine uggs
    999 An essential part of each woman wardrobe are her shoes.
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    Response: 2014 ford mustang
    I wanted to start my own blog and I was looking for your suggestions for which sites you think are the best to start one with. . I want to be able to post pictures, text, and music (maybe).. I only want to use a free blog host..
  • Response
    get best search engine optimisation available
  • Response
    The best information on alcoholism detox anywhere
  • Response
    click here for the best Quirk Volkswagen car dealers around
  • Response
    Response: long beach seo
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    Response: Matthew Sample
    lpcohn - Blog - That Way Madness Lies
  • Response
    Response: Pick 6 Leak Proof
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies
  • Response
    Response: Church Staffing
    lpcohn - Blog - That Way Madness Lies
  • Response
    lpcohn - Blog - That Way Madness Lies

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>
« Chasing Tails | Main