Tag Archives: rigor

PNAS: More walking along new roads and less debating about which roads – Undoing Illusions of Rigor, Part 4

21 Nov

The two headlines linked together tell a powerful story on their own:

Large-Scale Comparison of Science Teaching Methods Sends Clear Message

Active Learning Increases Student Performance in Science, Engineering, and Mathematics

These headlines – the first from a high-profile commentary advocating for more active learning for more learners in response robust findings outlined in the article following that second headline.  The commentary – by Nobel winner and distinguished teacher Carl Weiman – and multi-authored metareview of 225 studies reporting on comparisons of student exam scores in classes under traditional versus active learning conditions – both publications appear in the 10 June 2014 print publication and May 2014 online posting of Proceedings of the National Academy of Sciences.

A brief look at each, in order of publication dates (full citations and links below):

“Active Learning Increases Student Performance in Science, Engineering, and Mathematics”:

Briefly the metareview team reports from examining those 200 plus studies that detailed comparisons of active learning to lecturing in STEM classes that:

  • effect size (how well active learning will work in a range of contexts) indicates active learning results in a 6% increase in exam scores over lecture;
  • effect size (the difference between active learning and lecture results across all studies) indicates a 1.5 times greater chance of failing (D or F grades, course Withdrawls) in lecture vs. active learning classes;
  • the increase in numbers of struggling students remaining in courses making use of active learning will “depress average scores on assessments – meaning that the effect size…for examinations and concept inventory scores may underestimate active learning’s actual impact in the students performed to date”; and
  • “the data suggest that 3,516 fewer students would have failed these STEM courses under active learning,” which links with analysis of “science leavers” (eg, Seymore and Hewitt, Talking about Leaving: Why Undergraduate Leave the Sciences) who “indicate that increased passing rates, higher grades, and increased engagement in [active learning] courses all play a positive role in retention.”

Furthermore:

  • the effect size “held” even if the studies were not classical randomized assignments (which addresses pushback from scholars stating that some studies weren’t rigorous enough to be trusted)
  • the authors likened the effect of lecture to a “stop for benefit” that would happen on a drug trial – if one condition were clearly more efficacious that the other (with some reviewers likened the significance of this report to the US Surgeon General’s research linking smoking to cancer);
  • the authors recommend moving to “second-generation research” that would compare different active learning approaches to each other – the range of definitions of active learning addressed in the studies ranged from as little as 10% of the class session to 100%.

“Large-Scale Comparison of Science Teaching Methods Sends Clear Message”

Two points of data-rich analysis addressed by the meta-study authors provide the basis for Carl Wieman’s PNAS Commentary:

(1) that most STEM courses are still taught via traditional lecture formats even as more effective active learning teaching methods have been overwhelmingly demonstrated in quantitative studies including focuses on exams, grades, concept inventories, and qualitative research involving science leavers, strugglers and persisters, and

(2) that, therefore, it is time to move into next stage research focused on specific ways a broad active learning practices impact postsecondary STEM student learning in order to understand why, as the authors point out, “more is better” when it comes to active learning and student learning in STEM disciplines.

Speaking from the research as well as his dual positional power – lauded researcher in physics and science education, skilled teacher in working with students and colleagues – Wieman offers clear why and a what calls to action for postsecondary educators:

WHY attend to what we do know about active learning?  These approaches to learning engage students in developing expertise through authentic practice and timely feedback: 

Nearly all techniques labeled as active learning include those features known to be required for the development of expertise…; in this case, thinking like an expert in the discipline. The active learning methods are designed to have the student working on tasks that simulate an aspect of expert reasoning and/or problem-solving while receiving timely and specific feedback from fellow students and the instructor that guides them on how to improve. These elements of authentic practice and feedback are general requirements for developing expertise at all levels and disciplines….

The relationship between active learning and general requirements for expertise development may also explain the consistency of the benefits across the different disciplines and levels of courses.

WHAT will inspire us as teachers to make our teaching new?  More research as scholar teachers and researchers work together with students to examine variations in the active learning methods to further discern the relative impacts of particular active learning methods for specific contexts:

One promising direction emerging…is that ‘more is better.’ The highest impacts are observed in studies where a larger fraction of the class time was devoted to active learning. Those high impact studies would suggest that it is reasonable to aspire to teaching that consistently achieves twice the average improvements reported…, e.g., failure rates of only ∼10% and increases in learning of 1 –1.5 SDs. Such improvements in STEM educational outcomes would have major national implications.

In 1996 Craig Nelson raised a question woven into both PNAS pieces, noted here in Nelson’s words as “whether it has already become immoral to teach without extensive use of the active learning techniques that so enhance performance.”  The authors drawn together in this post all assert that (1) it is certainly time to acknowledge the evidence and move away from debating whether to maintain lecture-only teaching practices or actively adopt student-active learning practices, (2) the shift for teachers as for learners will be challenging to make and rewarding in having been made, and (3) these new research (ad)ventures make ethical, scientific and educational sense. Wieman’s description of the situation –

If a new antibiotic is being tested for effectiveness, its effectiveness at curing patients is compared with the best current antibiotics and not with treatment by bloodletting. However, in undergraduate STEM education, we have the curious situation that, although more effective teaching methods have been overwhelmingly demonstrated, most STEM courses are still taught by lectures – the pedagogical equivalent of bloodletting.

– puts me in mind of Nelson’s 2010 experientially- and research-based articulation of the illusions of rigor that holds us to teaching practices anchored in past beliefs, misconceptions about teaching and learning, about roles of teachers and capacities of learners.

In answering Weiman’s closing question –

Should the goals of STEM education research be to find more effective ways for students to learn or to provide additional evidence to convince faculty and institutions to change how they are teaching?

– with Nelson’s writing and Freeman, et al’s findings in mind, I would hope that we who teach and who research learning and teaching are saying “Heck, yes, let’s study how all this works!” to the first clause, and saying “We now have enough evidence, thanks, about why to change teaching across the ranks and disciplines” as we design courses, interact with learners, and shape our research.  

Walking this new road, call it “Illuminations of Vigor” and see it as a by-pass over “Disillusions of Rigor,” lets the “science leaver” I was in college spring down the road to catch up with the learning & teaching in higher education scholar I have become to see where we might travel together with colleagues and future faculty poised to take their own next steps.

Previous Rigor / Vigor Posts

Articles Cited Here

 

 

Tags: ,

← Older Entries