Tuesday, May 24, 2005

Grading the exams

I've just finished my final grading for the semester. It is the same every year. You wade through the pile of exams, slinging red ink here and there, thinking to yourself, "My God, this is a disaster. They don't remember anything. They can't do anything. I've been talking to myself in class for two months." But then, when you actually add up the grades, they often do not look so bad. This spring, the average score on the final in my "pre-med" General Physics class was 86 (out of 100), which in fact is pretty dang good; and nobody -- nobody! -- made below a 70.

How is it that your impression while grading is so different from the actual performance of the class? When you are grading a problem, you can grade a correct solution quite quickly. You recognize that the student had the right idea, used the right mathematics, and carried through the calculation successfully. There's the right answer, and the verbal explanation makes sense. Check, check, check. On to the next one. But when there is something wrong with the solution, it takes a long time to figure it out. Where did they go wrong? What was the student up to? Is this a calculational mistake or a conceptual mistake? If the answer to part (a) is wrong, is the answer in part (b) nevertheless correct given part (a) as an input? How many points is this error worth? Et cetera. So you spend 85% of your time dealing with the 15% of the test that they got wrong.

It occurs to me that there is an analogy to be drawn to the news media. Consider, for example, the situation in Iraq. The news stories from Iraq over the last two years have, for the most part, been pretty dire. The infrastructure is in ruins, the museum is looted, US troops and UN people and innocent Iraqis are blown up by huge bombs, the insurgents have tremendous support, foreign jihadis are streaming across the borders, there are no WMDs but there are leftover guns and explosives everywhere, the US does not have enough "boots on the ground" to secure the country, the political process is not working, the Sunnis are not cooperating with the election, the Shiites want to turn the country into another Iran, anti-American sentiment is running high in the region and around the world, US troops are committing atrocities against prisoners and shooting good guys at roadblocks, Fallujah is shot to pieces but Zarqawi remains at large, etc., etc. Some of this, a lot of this, is even true to some extent. There have been some encouraging stories -- the capture of Saddam, the January 30 election -- but they seem to be exceptions and they go by pretty fast. And you say to yourself, "My God, this is a disaster. It's all going to hell."

This impression is, I think, analogous to the impression that I get while I'm grading my exams. It is based on real facts, so it is hard to argue that it is simply false. But it is not a very reliable guide to judgment. The news media obviously focuses on the stuff that is going wrong, because that is exciting and interesting and attracts viewers and readers. It is a truism that "man bites dog" is news while "dog bites man" is not news. If you have thousands of dogs and thousands of men, with nobody biting anybody, that is really, really not news, and you won't read about it in the paper.

Making political decisions based on the general impression left by the fume and flame of the news media would be like my assigning grades based on the general impression I have after grading the tests. My students, I'm sure, would prefer that I step back, take a deep breath, and actually add the thing up.

4 Comments:

Anonymous Dave Bacon said...

Hear! Hear!

News, almost by definition these days, is all about accentuating the negative. And I wonder how in the world this will ever change? It doesn't seem like an easy question.

1:14 PM  
Blogger aram harrow said...

A similar problem arises when comparing economic data across different countries. Government statistical agencies in poor or unstable countries produce less reliable data, which can make it look like things are changing rapidly. The result can be a kind of overweighting of data from poor countries. For example, people have said that low growth means rapidly increasing inequality, but there's evidence that this is just a statistcal artifact.


On an unrelated note, I think it's reasonable to say that the overall picture of Iraq is consistent with the details of daily bombings:
http://www.juancole.com/2004/09/if-america-were-iraq-what-would-it-be.html
though I agree with your point in other contexts.

2:24 PM  
Blogger Joe said...

What would grading papers look like if, instead of starting from 100% and marking off, you started from 0% and marked up?

What would the news look like if we started from fuzzy bunnies and pretty butterflies and zip-ah-dee-doo-dah and only covered nasty things like war and murder toward the end?

Personally, I'm pretty happy that "your government occasionally follows through on its promises and accomplishes some of its goals" is not "news".

11:35 AM  
Anonymous Joe Renes said...

Interesting comment on news from Bruce Schneier:

"One of the things I routinely tell people is that if it's in the news, don't worry about it. By definition, "news" means that it hardly ever happens. If a risk is in the news, then it's probably not worth worrying about. When something is no longer reported -- automobile deaths, domestic violence -- when it's so common that it's not news, then you should start worrying."

http://www.schneier.com/blog/archives/2005/05/should_terroris_1.html

don't know I'd apply to Iraq necessarily, but for local news, it makes a certain amount of sense.

4:57 AM  

Post a Comment

<< Home