Innoeduvation

I don’t like grading. And when I have a large enrollment course, the tedium of it can drive me round the bend. To get through the task with my sanity intact I break it into batches, take breaks, etc.

When the task is complete, I have a new perspective on the course I just taught and the students who took the course. I’m a different person for the experience.

It is likely that this transformation is not a sudden one that occurs when I finally put my green pencil back in its holster. I’m pretty sure I evolve over the course of several days of marking. And, too, I’m pretty sure there are other temporal effects too: I get tired, exasperated, bored, delighted, and even angry over the course of the work. I bet that has an effect on the grades I confer, no matter how careful I might be.

To investigate this I click on the “analytics” button on my LMS’s interface. But what to my astonished eyes should appear but a zillion ways to slice and dice data about students and not a one to measure me.

So why isn’t there a function that would tell me if there is a bias that creeps into my grading over the course of a session or over the course of several days of end of semester grading? The system lets me grade anonymously, of course, but personal bias is not what I am looking for here. I want to extract some meaning out of data that’s already there: is there an advantage to being in the group I mark the first day? Last day? Are you better off getting marked at the beginning of a work session or the end? Simple stuff, really, but stuff that would contribute to reflective practice. I’m not imagining an algorithm that will compensate for my human foibles, but one that will keep me inclined to the same sort of continuous evaluation and performance improvement that we deploy with our students. And maybe some practical wisdom will emerge: “don’t grade for more than two hours at a time, Dan” or “don’t stretch it out for more than two days, Dan.” Those would be useful too.

Canvas folks in the back room? Are you listening? This would be dead easy – the data is already there. A lot of it. Give me more tools for self-monitoring rather than more tools for surveillance of my students.

While using chopsticks at lunch today I found myself musing about how to train a robot to do the same. One could, of course, attempt (likely in vain) to describe how it is done. Perhaps better than that, one could show how it is done. But the thing about using chopsticks is that when your hand-brain-eye system gets it, you know it. It’s a feeling, “I got it!” Suddenly it totally works. What was awkward becomes natural. Those two slivers of bamboo or plastic become an extension of the self. The strength of the “this is working” reinforcement feedback is tremendous.

Which got me thinking about teaching in general. We do a lot of explaining and describing and even some showing. And we tend to couple it with rewards in the form of approval/disapproval, points added or points taken off. In another jargon, we might say that we are inventing reward functions for our students, rewards rooted, ultimately, in the instructor’s approval.

And many of us bemoan the fact that a predominant result of this process is that the students invest a lot of time trying to figure out “what the teacher wants.”

Of course, what we really want is for the student to master the skill at hand but we offer this proxy reward, our approval when the performance is judged right by us.

Somewhere in this is a theory that if we repeat this process often enough, the student will learn what it was we were actually after (that is, they’ll get beyond trying to win our approval and see that actual mastery of the skill is what it is that earns the approval). We vary the inputs and mix showings and tellings of both problem and solution in a hope to optimize this process.

But what about that awesome feeling when your fingers and brain finally “get” how chopsticks work? What if I could be more successful at communicating that feeling and setting up in my students a search for it? My job can then be converted into helping them move toward trajectories that will pass through that experience.

We might still deploy some conventional approaches to explanation, demonstration, and performance assessment, but I think I’ll be doing a better and more effective job to the degree that I can create opportunities for the production and recognition of that tangible feeling of “I got it.”

Photo by Chopstick.JPG: 毛抜きderivative work: Richardprins (talk) – Chopstick.JPG, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=9559743

In an entrepreneurship course a colleague of mine used to have a “pitch and catch” session in which one team would pitch their idea and another team would play the role of investors or critics while the rest of the class watched. The catching team’s job was to help the pitching team sharpen its presentation. The rest of the class would watch the presentations.

I’ve been evolving a variation on this technique in other courses over the last few years. Here’s how it works.

Suppose students in a class of 40 students are working on individual semester-long projects. After about the middle of the semester when a few installments or drafts of the projects have been submitted to me and returned with feedback, we move into pitch and catch mode. During each class session we set aside an hour or so. The basic structure is like a conference at which there are simultaneous sessions running in parallel.

We break the session into four 15 minute rounds. During round one 10 students will pitch and the other 30 will catch. And then in round two, a different set of 10 will pitch and the others will be assigned catching roles. The catchers are shuffled between rounds so students are not simply listening back and forth to one another’s ideas.

Pitching and catching assignments are projected from a page in our LMS as shown below. We also use this page to link catchers to pitchers’ work in progress so they can consult it (and ideally offer feedback) between class sessions.

The “charge” to the catchers is to make their pitcher’s project as good as it can be. I tell them they want to get into a mindset in which THEIR grade is dependent only on the quality of the work done by the three pitchers they will meet with during the session. We talk about the learning outcome we are aiming at here: the capacity to be a helpful, constructively critical teammate/colleague.

The animation below gives a sense of what the classroom movement looks like.

By the end of the session, each student has had fifteen minutes and the attention of three classmates dedicated to her work and she has caught three times. The pitcher gets serious feedback and repeated engagement with interested listeners and each catcher participates with two classmates in a total of 45 minutes of focused conversation about other students’ work.

In one hour this yields 10 student-hours of presentation experience and 30 student-hours of feedback-experience. The pitch-catch conversations tend to be taken rather seriously – with just three critics there’s some informal pressure to participate and do one’s share of the thinking and talking and in most groups I’ve seen a regression to serious rather than the casual.

Following a pitch and catch session, each pitcher has an assignment to “evaluate” the catching she received. They post a short, private reply to the LMS saying who their catchers were and what advice/feedback stood out. This is designed to motivate the pitcher’s taking notes and reflecting on the feedback received and the catchers to give serious advice. The instructions for the assignment mention that they only need cite helpful feedback, that they should attribute it, and that one of the things I read these four is so that when I’m writing letters of evaluation I’ll have concrete things in mind – e.g., “during our pitch and catch sessions, Jesse’s classmates consistently appreciate their feedback on marketing issues.”

Here’s just one example of post pitch-and-catch session reflection:

My assigned catchers were – Q, T, and A. I found T’s input very helpful. My workshop is targeted towards helping entrepreneurs bring design into their startup using a version user centred design thinking and T suggested taking a look at design hackathons. Hackathons are already popular among my audience so using that concepts will help me connect. I also found Q’s input very helpful. She suggested for the prototype step that I could guide participants toward simple mockups. This was helpful because creating an MVP is a common hurdle.

Pitch-and-catch combines the best elements of small group discussion and presentations of work-in-progress, supercharging both in a manner that yields a lot of bang-for-your-buck. It allows us to get serious small class payoffs in arbitrarily large classes by exploiting the insight that not everyone has to hear from everyone and that with sufficient structural constraints we can get high quality conversations happening in parallel.

If one hour of class time were split equally among a class of forty, with half the time hearing about project and half the time offering feedback to peers, on average each student would have 45 seconds to communicate their ideas and each would receive about 45 seconds of feedback. Pitch and Catch yields about 10 times as much student experience.

I still remember things said in some lectures that I heard in the late 1970s, the 80s and the early 90s. In some cases I remember what was said, in some cases I simply think and live differently to this day because of what was said.

Earlier this year David Goodblar wrote a piece in Chronicle Vitae under the title “‘Is It Ever OK to Lecture?’” Mostly the piece just recounts the arguments made in favor of “active learning” vs. lecturing. His take away conclusion is that lecturing is OK if it includes active learning activities. A bit disappointing as a dodge of the question in the title, but it left me wondering whether the whole piece, and much of the active learning conversation, is perhaps grounded in a false premise.

The author characterizes lecturing as just “tell[ing] students what we know.” The standard caricature of lectures is that they are mere conveyance of information with goal being that students remember what we said, but that’s wrong. It might be what far too many instructors do when they are supposed to be lecturing, but a good lecture is something different from articulating information that students are supposed to absorb or write down.

A proper lecture is an interactive experience in which the speaker connects with the minds of the audience and takes them on a journey and changes them in the process. I’m a serious advocate and practitioner of all manner of “active” pedagogy, but I’m disappointed when supporters resort to cartoon versions of alternatives to bolster their case. The argument between active learning and lecturing as it normally happens is between great active learning and mediocre lecturing. It’s easy to win that battle. But out in the wild there’s plenty of really great lecturing and plenty of really tedious active learning too.

Why do we rarely see, in conversations like this, pointers to ideas about how to give really awesome lectures. I think it’s because the conversations are dominated by individuals and organizations that live OFF bandwagons and trends, making their names telling faculty to get with the program or risk being labeled dinosaurs, rather than living FOR education per se. There is material out there – see Chris Anderson on how to deliver a compelling TED talk or folks like Nancy Duarte on great talks more generally or Conor Neill on how to start a speech or so many others just waiting for a little curation – that can help the mediocre lecturer most of us are become the one who can deliver the lecture that a student will remember 30 years later.

When we have that skill in hand we can make smart, strategic pedagogical decisions about when to lecture and when to use other methods.