Comments from Bruce Sherwood:

We did do a straight exam comparison at CMU, which alas
we've never written up. It's the only such comparison I know of. We
worked with colleagues who were teaching the traditional E&M course to
put three common problems on our final exams (which also had other
problems tied more closely to course-specific goals and topics, such as
in the M&I case the surface-charge model of circuits). We even went to
the trouble of using exactly the same fonts and page layouts. After lots
of thought, scoring was done on the basis essentially of no partial
credit: you had to get it exactly right (or exactly right except for
trivial arithmetic or copying errors).

On two of the problems there was no difference, though there were
interesting differences in how one got it wrong. "Conservation of errors".

The third problem was quite complex, so complex that those teaching the
traditional course insisted on breaking it up into scaffolded parts. A
long solenoid has a linearly increasing current. Around the solenoid,
with its center at the same point as the center of the solenoid, is a
metal ring of known resistance. Find the field at the center of the
solenoid. So you have to find the emf in the ring, find the current in
the ring, find the field contributed by the ring at its center, and add
that contribution to the field made by the solenoid at its center. This
problem can now be found in our textbook.

Only 7.5% of the traditional students got this problem right, whereas
30% of the M&I students got it right, which is a big effect. If I
remember correctly (Ruth can correct me), it was clear from their work
that the M&I students were in a much better position to attack a very
hard problem of this kind.

The two populations were equivalent as near as we could tell (SAT, GPA,
calculus grades, etc.).

If it is of any use, we can provide you with sample exams from NCSU,
Georgia Tech, and Purdue, which show that the M&I tests are typically
pretty demanding for the respective populations.

While the BEMA results show that M&I students learn the common,
lowest-common-denominator concepts of E&M much better than do students
in the traditional course, to me what is much more important, and for
which no comparison is possible, is all the unfamiliar and vitally
important contemporary physics that the M&I students learn. One can't
make comparisons because none of the following is present in the
traditional course: atomic nature of matter; retardation; surface-charge
model of circuits; atomic theory of magnetism; classical theory of the
interaction of light and matter. One might worry that with all this
contemporary content the students wouldn't know the basics ("physics for
poets"). The BEMA shows that the M&I students learn a contemporary view
of E&M without the cost of skimping on fundamentals -- quite the
contrary. Our course homework and tests show plenty of challenge with
respect to problem-solving, too.

Bruce

P.S. By the classical theory of the interaction of light and matter I
mean this: accelerated charges make radiation; the E field in the
radiation accelerates electrons in matter; these accelerated electrons
re-radiate, which is why you see the wall -- in this picture you see new
light, light dosn't "bounce" off walls. In striking contrast, most
traditional intro E&M textbooks just show you a picture of moving
crossed E and B fields, with no mention of what makes such fields, and
with no mention of what effect these fields might have on matter, and
certainly no discussion of the classical re-radiation process.


P.P.S As I said, we never published that final exam comparison that we carried
out at CMU. In fact, we had argued for including this information in the
BEMA paper, but our colleagues argued that this would be a distraction,
that the paper should focus solely on BEMA. What I wrote you about the
final exam comparison is all I've got to give you on the subject,
unfortunately. But I want to share with you some deeper thoughts on
assessment issues in general, which I'll do later. I have to run off to
give a talk to education folk about the distance ed version of M&I that
I offer to in-service high school teachers, which I presumably told you
about when you visited NCSU.

As a different kind of evidence, I attach final exams in mechanics and
E&M from the M&I courses at NCSU in fall 2005 (I happened to find these
easily; I can give you later exams if that's of interest, and there is
some continual change as we refine the curriculum). You'll see that
they're serious exams, with a very different character from assessments
such as BEMA.

Bruce