Textbook Tests

This video is making the rounds. The professor is discovering to his chagrin that students can get access to test banks and cheat on the answers.

 

At Outside the Beltway, James Joyner (a former professor himself) takes the professor to task for using test bank questions at all:

Crafting the exam isn’t something that ought be farmed out to one’s graduate students — much less the textbook publisher. Testing is part of teaching. It’s a way of forcing students to grapple with the material, extract major points, and understand how the pieces relate to one another.

Well crafted exams are such that having the questions ahead of time wouldn’t constitute cheating.

I don't know that I particularly agree with Prof. Joyner here. I certainly don't do it. In the class I'm teaching now -- which I anticipate will be the last class I teach as an adjunct for any institution; unless I somehow get hired on full-time as faculty somewhere I'm not interested in teaching anymore because I've stopped having fun doing it -- I've set up a class almost completely out of the textbook.

First, I looked at the learning objectives fed to me by the school. Presumably, these relate to some sort of accreditation. Whatever; it's what my employer wants me to teach, so I'll teach it. For instance, if I were teaching a geography class, I might find "political geography" on the list of subjects to address.

Second, I looked at the schedule fed to me. I have nine classes in which to teach a total of about forty subjects. So I split them up more or less evenly between the nine classes. So again, if one of the subjects to address were "political geography" I would look at that in, say, week 3.

Third, I looked through the test bank of objective questions provided to me along with the textbook. I read through them until I found questions that matched up with the learning objectives which I did not consider to be either pathetically easy nor unfairly difficult. I had to make that choice, because my students would always complain that this would be an example of an unfairly difficult "political geography" question:

What is the capital of France?
A) Beijing
B) Paris
C) Washington, D.C.
D) All of the above

So I did have to make a judgment call about the difficulty of the questions.

Fourth, I rearranged the answers to the questions so that the answers were not in the same order as the test bank, just in case a student were able to get a copy of it. So to continue with the example, my question would look like this:

What is the capital of France?
A) Paris
B) Washington, D.C.
C) Beijing
D) All of the above

See how tricky I am?

Fifth, I change the names of the characters in the question so that they are easier to understand, less boring, or suggestive of their roles in the question. When I have a procedural question, the plaintiff's name always begins with "P" for "plaintiff," so there's a lot of people named Peter, Paula, Preston, and Priscilla filing lawsuits -- against people with names like Dawn, Donald, Debbie, and David, because "defendant" begins with "D." Other times I might use a pop culture reference, because that seems to help people remember who the characters are in a given scenario. So to continue my political geography example, my question would go through the transformation process and look like this:

Woo hoo! Homer Simpson won a contest and got tickets to take his family to the capital of France! Where are they going?
A) Paris
B) Washington, D.C.
C) Beijing
D) All of the above

Granted, in this case the pop culture reference doesn't really add much to the question. But in a five-sentence setup for a question about the Statute of Frauds, it is helpful. It also obscures the question from being identical to a question in the test bank, again if the students have access to it. I figure, if the student has enough intelligence to read through my irrelevant cosmetic changes to the question and discern what the real question is and match it up to the test bank and extract the correct answer from the re-ordered answers on my test, that student is probably reasonably smart to begin with and has now devoted more effort to cheating than he would have devoted to actually studying.

Then, as I have done in other classes, I print up all nine quizzes and I hand out the actual quizzes to the students, on the first day of class. Here it is, folks, the entire class. Each week I lecture about the answers to the quizzes. And only the answers to the quizzes. I explain why the correct answers are the correct answers and why the incorrect answers are incomplete. And this is where it gets to be a drag -- the students will insist that the incorrect answers really are correct. "You see, Professor TL, Beijing really is the capital of France if you look at it from an economic hegemony perspective..." No. If you didn't figure out that the right answer is "Paris," you don't get credit.

Full credit, that is. Since I'm teaching at a career college, I have heard and understood the implicit message that I'm not supposed to fail anyone, whether or not they have actually learned or understood anything I've taught or which might be found in the textbooks that it remains an open question as to whether they have been opened.

Like Prof. Joyner, I eventually succumbed to the incentives for objective testing rather than essays, and went the next step to not caring at all about how my students performed, and then took the last step on the Academic Morale Death Sprial^® and stopped caring all that much about whether I reported good or bad grades -- I now only care that there is sufficient variation that it appears that I am doing something to evaluate my students' performance. Thus, a student can "correct" or "amend" a bad answer in class after I explain what the right answer is.

So to complete my "political geography" example, I will make two powerpoint slides for my question, which would look something like this:

I talk about this stuff for a few minutes, and then I do the reveal:

...And all the students furiously scratch down "A" on their amended answer sheets and turn them in to me at the end of class -- for which they will receive 60% credit, the minimum necessary to pass. Thus, if they simply show up, send text messages to their girlfriends or boyfriends the entire class and simply transcribe the answers on the slides, they are guaranteed to pass. If they pick answers through the mechanism of chance and leave the textbook virgin in its shrink-wrap, they will still get one out of four correct for full credit, resulting in a "C" grade for picking answers at random and then showing up.

Indeed, since I will accept amended answers up through the last day of class, and I post my slides on the class website every week, they can actually not even show up to get the correct answers. I don't have enough contempt for them yet that I can bring myself to say "Actually, I prefer it if they don't show up," but for some of them I'm kind of close to that. Not that they've done anything wrong; it's just my profound apathy at play.

Why don't I worry about cheating? Because I've taken all the incentive out of cheating. It would be a hell of a lot more work to get the test bank and decipher my questions than it would be to do the process I've outlined above. And if there's one thing I can count on cheaters to do, it's to take the path of least resistance. If it's more effort to cheat to pass than it is to do the real work, then no one will cheat.

Are the students learning anything this way? Am I being offensively lazy for teaching my class this way? Is this instructor-facilitated cheating? Once upon a time, I would have delved into these answers. But as I approach the event horizon of the Academic Morale Death Sprial^®, my answer to all three of those questions is the same: I don't care.

No one else seems to care about the answers to those questions, either. Not the students, who insist after class that they were really, really thinking that "Paris" was the correct answer and want to vent their immense frustration at confusing themselves, at the wasted effort spent studying for hours and still getting it wrong, at the cruel fact that one little word in the question (viz., "France," which in this case, they apparently mistook for "China") changes the whole answer!

Then they tell me that they're really learning a lot in the class and they're really enjoying it. I smile, and deliver good customer service on behalf of my employer. But inside, I just don't care whether they're enjoying the class or not.

As an added bonus, when the students complain that the question is poorly-worded, I can tell them that I took the question from the test bank provided to me by the authors of the textbook. In other words, I can dodge the blame and pass the buck on that sort of criticism because frankly, I haven't the patience or the interest to hold up a sustained argument about a multiple-choice question anymore. "I'll send the authors a message when the class is over with your complaint" seems to satisfy them -- even though I'm pretty sure that the message will be ignored.

Nor does the administration seem to care about the intellectual integrity of this teaching method. Their incentive, after all, is to get happy customers who will give them repeated business. Failing grades make for unhappy customers. Not the accreditation agencies that seem to tolerate this sort of thing, from whom I've never heard a peep since I started teaching seven years ago as a supplement to my regular career.

If the government doesn't care, the administration doesn't care, and the students don't care, why should I? Well, I don't. I've managed to design a class for which there is little incentive to cheat in the first place. So I don't worry about cheating. Nor do I worry about failing students, for which my employer feels a sharp financial disincentive that it dare not articulate but which it nevertheless communicates to me anyway, because a for-profit career college has customers, not students.

Those customers who are there to learn, to be actual students, are going to learn the actual material, no matter what I do. They come self-motivated and they're the ones who ask good questions and whose faces register light bulbs turning on during my lectures. It doesn't take Super-Teacher to get them working and learning -- and amazingly, these students are on track to get good grades!

Most of them, though, are just there to go through the motions in order to get a piece of paper to which they assign no intellectual import are not going to learn anyway, again, no matter what I do. No one -- not the administration, not prospective employers -- will care about the students' grades in a few years anyway. So if nothing I do will change the real outcome of the class, and the superficial outcome of the class is functionally irrelevant anyway, why should I care?

Getting back to the professor in the capstone class finding out that his students cheat -- there are all sorts of things that can be done to get around it. I don't know that writing one's own objective questions is any better or worse than using questions out of a test bank. I think the real issue is coming up with a way to de-incentivize cheating in the first place, rather than punishing it after the fact. Then the problem never comes up.