TILT

Yup, time to update and reprise the semi-annual post focused on final exams.  We’ve reviewed some newly published or posted resources on testing, selected always apt segments from earlier posts, and winnowed a selection of For Further Information to the five that appear with annotations at the end of this post.

Prepare Students for Conveying Learning 

As “finals week” arrives, students often move into a “preparing for the test” mode which they – and sometimes we as teachers – practice as an activity separated from learning.  Students who engage, therefore, in cramming work to isolate, pin down, list or otherwise master the “information I need to know.”  If practices reflect learning, it’s learning in the realm of “lower order thinking skills” in a Bloom’s Taxonomy context.

As teachers who aspire to engage students in “higher order thinking skills” that we will assess in ways that include final exams, this is a good time to be reminded that students who maintain (or newly engage) learning behaviors – such as summarizing and synthesizing and posing new questions – will come to the end of the course actively linking information, ideas, and concepts to convey new knowledge. Generally, these students come to tests better prepared and less anxious.

Students who are preparing for a final exam will benefit as learners from hearing our suggestions about how to synthesize and study course materials from unit to unit – whether the exam features quantitative or qualitative or mixed methods of assessing their learning related to course outcomes.   To help students move into high-level learning ahead of exams, we might provide students with ideas such as these (inspired in part by a UC-Berkeley resource on alternatives to final exams and by ways I’ve structure learning practices toward review in anticipation of exams):

• Create Crib Sheets – Even if you don’t allow these into the classroom, the process of openly creating abridged/condensed sets of notes will help students to identify vital, nice to know, and can do without categorization of information, to further cluster the vital information into memorable schemas, and to ask questions of peers and teachers in the process.
• Create a Portfolio – Provide students with a course outline as a prompt for gathering their course materials – all manner of notes, assessments and assignments – into one virtual or print folder that they can then review to trace ideas, identify links between readings and lectures in previous assessments, and follow up teacher feedback to review previously completed assignments.  Post-Its are helpful here to code ideas, mark pages, and jot questions across the portfolio.  From this self-assessment, students will be better able to map their own study needs ahead of a final exam.
• Create a [Learning] Map of the Course – Another way for students to integrate course learning across the weeks of a semester, or shorter span of time that an exam addresses, this process asks students to think through how one chunk of a course relates to the next, and the next, and the next. These modified concept maps help students make explicit the cognitive and affective links as they move from reviewing one segment of the course to another.  Marilla Svinicki suggests these guiding questions:
  • How does this topic compare with/relate to what has gone before? 
  • How is it different? similar? 
  • Why is this included in the course? at this point in the course? 
  • What are the main points of a given unit/topic/theme? how do these link to one another? 
  • How do the components relate to the overall outcomes for the course?
• Create a Memorandum or Fact Sheet – for courses where students are expected to communicate ideas an external audience in mind, encourage them to study by organizing “one-pager” write ups for each of several key concepts to be addressed in the final exam.  If students are already working collaboratively – and will continue to be allowed to work this way in preparing for the exam – they can share the task of creating one-pagers to share with peers.  These short papers can include select information and ideas (with reference to where to find out more about it in course materials), major talking points, possible problems/positive outcomes related to particular concepts, and potential applications they recommend.
• And, whether as a recurring activity or to provide a course synthesizing focus, invite students to develop a Course Question Bank of  “short, sharp questions about a topic or subject, and make a parallel list of answers to the questions, or includes learning towards the answers.” In posting questions, students exercise higher level thinking skills.  

Plan for Post-Exam Feedback Opportunities – Even after Final Exams

Research conducted by the UK’s National Union of Students is reflective of findings also reported in the US:  Analysis of student comments points to two main requests, if you will, from students (1) knowing how exams will be rated, reviewed, scored; and (2) having some feedback on the assessment.

As the 2010 NUS report sums this up: “Feedback on assessment plays a crucial role in a student’s learning, self-esteem and future development.” Even at the end of the term.  Just don’t expect them to come to your office to pick up the feedback, and don’t feel as though you only have the option to respond to each student individually.  Instead

• Have students use Google Documents to send in a take home exam and respond directly on that file.
• Ask students who want individualized feedback to provide a self-addressed stamped envelope and one or two specific questions to which you can respond.
• Use Moodle to send out a classwide email or to post a sound file that speaks to the entire class about what made for successful and unsuccessful responses to essay questions, or patterns that emerged in answering multiple choice questions – especially if these impacted exam scoring.
• Talk to students via a podcast about what you see as course content students have “mastered,” followed by naming those bits that need their continued attention, and maybe by a suggestion of two good resources for their future reference given interests in the topic that have emerged in student writing.

Final exams – indeed, all effective exams we compose for a particular course – build from these four characteristics, reported by Mary Piontek (see Resources).  They are:

• valid – provide useful information about outcomes they are designed to test,
• reliable – allow for consistent measurement and discriminating between different levels of performance,
• recognizable –  align testing with outcomes, and with course activities that students have engaged to gain, practice and advance learning,
• realistic – in time and effort allowed to complete the full exam.

A Few Key Guidelines

According to Mary Piontek and Barbara Gross Davis, instructors begin to “develop exams that provide useful and relevant data about their students’ learning”  with these few  guidelines in mind:

• Focus on the most important course objectives you have emphasized during your course – the distribution of items should reflect the relative emphasis you gave to information, ideas, concepts, and/or practices explored; that is, develop exams based on vital course materials and concepts.
  • See General Tips in Test Construction and Test Construction Grid to gain an overview for this prioritizing process, and suggestions for distributing exam questions to include higher order thinking skills (the HOTS of analysis, synthesis and evaluation.
  • And see Alternatives to the Final Exam for ideas about writing- and presentation-based final exams that involve students in exams expecting students to apply higher level learning to real-world audiences requiring analysis and problem-solving skills.  Many of these ideas can be adapted for even final exams that are just weeks away.

• Clarify the exam format, scoring practice to be used, and ways to effectively study for the proposed exam.  Davis offers these considerations about what to convey to students during the two to three weeks leading up to a major exam:
  • Format: What exam format can students expect – take home, writing-based? in-class writing in response to a scenario or case? final exam period test with particular question types addressing all or some course topics?  Brief students on the types of questions they can expect and also on what types of answers you will expect for each general type of question.
  • Scoring: Will narrative responses be criterion referenced, then what are key characteristics you’ll be looking for in distinguishing A from B from C from failing work? Will exams mixing multiple choice and short answer questions be curved or norm referenced? Will the exams be graded by the instructor? TAs? a machine? a mix of human and machine scoring?
  • Studying: How would you advise students to go about studying for the exam – do particular course outcomes come into prominence in the final exam? do specific readings and class sessions map onto those outcomes and, hence, onto the exam?  Remember here, as the teacher, you’re the expert on not just how the field is conceptually organized, but also on how the course works conceptually – so mapping out the links as you see them and passing this information along is part of the academic socialization role linked to your role as a teacher.  In naming how we link course outcomes and course materials, we both model a learning process and cue ways of learning while studying.
  • Organizing:  Does the test begin with questions, problems, scenarios, cases that are both interesting and that “build rather than undermine student confidence” by presenting questions that are “reasonably easy for a prepared student?”  See Exam Checklist for more key guidelines ideas.

• Finally, review the test items you’ve created somewhere in that space between “cooling off” a couple of days after drafting the exam, and a few days ahead of the final exam period.  On returning to the exam draft, you can critique each item for its relevance to what you have actually taught.

Develop Clearly Written Test Directions and Questions / Prompts

Prepare test directions carefully – to provide students with information about the test’s purpose, format, time allowed, and scoring criteria.

• Consider this point from Chris Galdieri, UM alum holding a political science teaching position at a New Hampshire university:

Exam questions are a great place to emphasize inclusive language.  There hasn’t yet been a woman president, for instance, but there’s no reason there can’t be in a question about a hypothetical president on your exam.  Similarly, there’s no reason a hypothetical candidate on a question can’t be named Reyes instead of Smith.

• Avoid extra language in writing multiple choice question stems or essay exam prompts.  Notice the difference in these two examples:
  • According to Tuckman’s model, groups develop through several stages over time.  Furthermore, it contradicts Poole’s activity-track model which has groups switching among several different linear sequences.  Which of the following is not one of the stages identified in Tuckman’s model?
  • Tuckman’s model of group development includes which of the following stages?  Select ALL that apply.
  • In setting out direction for essay exam questions, be specific about what you mean by discuss, analyze, or evaluate, be especially clear about this if you add the word criticallybefore any of these verbs.

Can the discussion you having in mind for a restricted response essay be a one-sided reiteration of key points to be considered in a specified context, or does that response need to be a more robust two-part comparison of perspectives that lead the writer to a recommend a particular action in the specified context?

If you want your students to provide an analysis as the rhetorical structure for an extended response essay promptconsider carefully whether the components you expect in an analysis are known to your students – have you already required use of this rhetorical structure in your course? If yes, remind students of that; if not, set out the parameters for analysis well ahead of the exam so that students may use this format as they prepare for the exam.

Help smooth this by providing students with a list of key words – such as those noted above – that will be used in an exam and specify what those terms imply in terms of structuring responses and delivering content within those responses.

Resources

Best Practices for Designing and Grading Exams – M.E. Piontek, UMichigan Center for Research in Learning and Teaching http://www.crlt.umich.edu/P8_0

Assessing Student Learning Outcomes – UMinnesota Center for Teaching and Learning http://www1.umn.edu/ohr/teachlearn/tutorials/coursedesign/assess/index.html

Designing [Quantitative] Tests to Maximize Learning – Richard M. Felder, engineering http://www4.ncsu.edu/unity/lockers/users/f/felder/public/Papers/TestingTips.htm

Writing Multiple-Choice Questions for Higher-Level Thinking – Learning Solutions Magazine  http://www.learningsolutionsmag.com/articles/804/writing-multiple-choice-questions-for-higher-level-thinking

Writing and Grading Essay Questions – UNC-Chapel Hill Center for Faculty Excellence http://cfe.unc.edu/pdfs/FYC7.pdf

Tags: assessment, learning, wrap