TILT

Many instructors in STEM fields struggle with how to design exam questions that assess students’ (1) problem solving strategies rather than computational ability, and/or (2) grasp of complex topics where solving an individual problem would take a student an hour or more to solve.

Richard Felder, faculty emeritus of Chemical Engineering at North Carolina State University, sets out a great approach for these kinds of situations in “Designing Tests to Maximize Learning.”  (Link provided below.)

In short, he says, ask students to outline the solution rather than solve it.

Here’s how he describes the technique in the article:

In my courses, the problems get quite long: by the end of the course, a single problem might take two or three hours to solve completely. There’s no way I can put one of those problems on a 50-minute test, but I still have to assess my students’ ability to solve them. I do it with the following generic problem: 

Given…(describe the process or system to be analyzed and state the values of known quantities), write in order the equations you would solve to calculate…(state the quantities to be determined). Just write the equations—don’t attempt to simplify or solve them. In each equation, circle the variable for which you would solve, or the set of variables if several equations must be solved simultaneously.

The students who understand the material can do that relatively quickly—it’s the calculus and algebra and number-crunching that take most of the solution time. Moreover, I know that if they can write equations that can be solved sequentially for the variables of interest, given sufficient time they could grind through the detailed calculations.

One cautionary note, however. If students have never worked on a problem framed in this manner and one suddenly appears on a test, many of them will be confused and may do worse than they would have if the problem had called for all the calculations to be done. Once again, the rule is no surprises on tests. If you plan to use this device, be sure you work similar problems in class and then put some on homework, and then do it on the test.

In a follow up email where I asked Felder for additional information about this technique, he provided this response:

It works for any quantitative problem in which the students are given a definition of a system, device, mechanical structure, or whatever, and values of some system variables (inputs) are specified. The student’s job is to write or derive equations that relate the system variables to one another, substitute the specified values, and calculate the remaining variables. That’s a generic version of most problems in most undergraduate engineering courses. The approach you’re talking about goes like this:

[Define the system and give values of the input variables.] Write the complete set of equations you would use to calculate __, __, and __, briefly stating what each equation is. Your solution should be a set of n equations in n unknowns. Do no calculus, algebra, or arithmetic—points will be deducted if you do.

The first time I ask students to do it is in an active learning exercise in class, and you can see them struggling to keep their hands from reaching for their calculators. Then I do it again in one or two homework problems, and I dock points from anyone who violates the instruction. At that point I’m ready to put one on the exam, and by then they’re used to it and most have no trouble. When you use this scheme, you can put some sizeable problems on your time-limited tests—it’s the algebra and number crunching that take up most of the time. I know that if my students can come up with the correct set of equations, given enough time they could grind through the other stuff. If I want to find out whether they can differentiate or integrate functions, I’ll put another small problem or two on the exam that only involves those operations.

The article includes a number of other tips for designing tests in quantitative fields, so it’s worth a quick read before designing your next exam:

Richard Felder.  “Designing Tests to Maximize Learning.” http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/TestingTips.htm