Multiple Approaches for Measuring Student Learning

What evidence do you have that your students are learning?

In addition to student grades, evidence can include graduation and retention data, student capstone experiences and artifacts, or recent graduate employment or professional school attendance data. Making improvements to your teaching based on this data is what Michael Reder refers to as an evidence-informed teaching approach.

Reder – Director of the Joy Schectman Mankoff Center for Teaching and Learning at Connecticut College – urges educators to measure something meaningful to them as a way to determine specifically if their teaching approaches are effective for student learners and learning.

A specific example of something instructors measure at Connecticut College is the intellectual challenge and academic rigor of courses. Faculty participants who opt to participate in this assessment process ask their students to complete a “Supplemental Course Survey” at the end of the semester. (Link to sample survey provided below.) Analyzing data like this provides a snapshot in time about perceived academic rigor of a course. Collecting individual course data every semester allows an instructor to track changes. Drawing multiple course sections allows a unit to build a comparative data analysis to understand impacts of different instructor approaches, or of course-wide adoption of strategic teaching initiatives.

Reder also offers a couple of cautions about measuring learning as part of reminding faculty to triangulate the data gathered so that our conclusions don’t rely on just one measure. The first caution is to remember data doesn’t prove anything. He advises that collecting data about student learning shouldn’t be about proof but rather about understanding. As Reder notes, “The best way to dig deeper and make data multidimensional is to talk with students”.

The second caution is to incorporate more than one measure. A survey is one way to collect data about academic rigor. Supplementary measurements could include qualitative analysis of samples of student work or quantitative analysis drawing on test scores. Additionally, reflective peer review of a course syllabus and/or class session(s), or consultant-facilitated focus groups of students who took a course can yield helpful qualitative data about perceptions of teaching.

Linda Nilson also advocates for incorporating at least one measure beyond end-of-term students ratings of teaching in order to understand student learning and perceptions of leanring. In her recent article, “Measuring Student Learning to Document Faculty Teaching Effectiveness’,” Nilson describes some additional course level measures of student learning to consider employing. Some of these measures may be more or less appropriate depending on a teacher’s discipline and on what the teacher is hoping to measure or understand.

These measures may be direct or indirect, may occur only at the end of course, or take shape as combined pre- and post- course assessments of learning. Some examples:

Indirect measures at the end of the course only. These can take the forms of surveys that ask students to rate their perceived learning or knowledge gains at the end of the course. The intellectual challenge and academic rigor survey described above would be an example of this. Another commonly used template to consider is the Student Assessment of Learning Gains (www.salgsite.org).

Direct measure at the end of the course. These would include assessments such as final exams, capstone assignments, and portfolios. These are graded by the instructor against predetermined standards. Results can be compared between sections and semester-to-semester.

Indirect pre- and post- measures built into the beginning week to reappear again at the end of the course. These can take the form of surveys asking students to rate their confidence in course skills and knowledge at the beginning of class and again at the end of the course. Increases in the posttest scores over the pretest scores provide a gain score for students in each of the areas measured. These scores can be compared semester-to-semester. This is the type of measurement Teaching in Higher Education course instructors developed to ask students to rate their pre- and post-course confidence in their ability to design and teach college courses.

Direct pre-and posttest on course concepts. To do this an instructor would give a version of their final exam on the first day of class as an ungraded, diagnostic tool. Another approach is to ask students on the first day of class to indicate their level of agreement with various statements about the course topic, including some commonly held misconceptions. Ask students to and provide a brief sentence or two of explanation for their answer. At the end of the semester distribute these to students and ask them to comment on their original answers and indicate any changes in thinking. In addition, ask students to provide evidence from the course to support their answer. With either approach, increases over the ungraded pretest scores may be measured as percentage improvement or as a percentage of possible performance.

Choosing appropriate multiple measures of your students’ learning positions you to make rational and informed decisions about your teaching approaches.

Resources Mentioned in Post:

Linda Nilson. “Measuring Student Learning to Document Faculty Teaching Effectiveness.” To Improve the Academy 32 (2013): 287-300. PDF available until 1 February 2014: http://www.scribd.com/doc/197408484/Nilson-Measuring-Student-Learning-to-Document-Faculty-Teaching-Effectiveness

Michael Reder. ‘Cultivating a Culture of Evidence-Informed Decision Making about Teaching and Learning.” Workshop Session. Professional and Organizational Development Conference, 2013.

Reder. “Supplemental Course Survey.” Connecticut College, 2013. PDF available: http://www.scribd.com/doc/197408483/Intellectual-Challenge-and-Adademic-Rigor-Survey

Teaching in Higher Education Instructors. “Pre-Course or Post-Course Confidence Survey.” University of Minnesota Preparing Future Faculty Program, 2011. PDF available: http://www.scribd.com/doc/197408482/Pre-Post-Confidence-Survey-GRAD-8101

Tags: assessment, learning

Featured Posts

Enhance Cognitive Capacity to Maximize Learning

Four Key Strategies Cognitive load refers to the amount of mental effort expended in your brain’s working memory. Learners only have so much working memory available. With attention to cognitive capacity and cognitive load, we support learners in managing the Too Much Information monster. When learning new material, what may be a simple concept for […]
Incorporating Canvas Tools: Helping Students Keep Up

As the University of Minnesota transitions from Moodle to Canvas, CEI will be highlighting some of the ways that instructors can leverage Canvas to meet learning and teaching goals. This is the first of four planned posts for this academic year. As instructors, we want students to come to class prepared, complete assignments on time […]
Online Learning Spaces – (Re)Imagining Discussions

3 Research Resources: (Re)Imagining Discussion in Online Learning Spaces How might we evaluate, measure and grade, online discussion forums? Teachers commonly opt for participation measures such as a counting the number of times and ways people participate, or the weighting of the quality of response content, or build a rubric to guide their review as […]
Developing a Learning Culture and Learning-Centered Climate

3 Resources Cultures of Learning 1. What is a culture of learning? The short answer, posted by an Edutopia blogger, is this: a culture of learning is a collection of thinking habits, beliefs about self, and collaborative workflows that result in sustained critical learning. 2. Rethinking our default definitions of learning is a crucial first step for thinking […]
The Write Situation: Assignment Clarity and Timing

With this post, we’re continuing our (re)introduction of the Teaching with Writing project’s monthly writing tips. See the Teaching with Writing pages on the Center for Writing website for teaching resources, including sample assignments and syllabi. Establish the Situation for Your Assignment “At the heart of every assignment is the rhetorical situation — someone […]