Small Group Meetings on Zoom

An alternative, with different functionality, to “breakout rooms.”

Zoom, a video-conferencing tool that lots of folks have started to use for teaching recently, has a “breakout room” feature. It has to be enabled in one’s settings (via My Account > Settings on the web sign-in page) and then a meeting host can split a session up into multiple separate conversations.

Here’s another approach that is integrated with the Canvas (Quercus at UofT) platform.

I want to split my class of 64 into groups so that each student can present her work in progress and get feedback from peers. I want every student to have the opportunity to present over the course of a 3 hour class session.

Basics: I’m going to divide the class time into several “rounds” or “sessions” during which several student groups will meet in parallel. During each round a student is either a presenter or a feeder-back. Every student has one role or the other over the course of the class.

First I decide on the timing and group size. The constraint is the number of students. One scenario is 8 sessions with 8 groups in each session and 8 students in each group. If I divide my 180 minutes up evenly (with a little wiggle time for transitions and a break) this gives me 20 minute sessions. Other options:

  • Four 45 minute sessions with 16 groups of 4 students
  • Sixteen 10 minute sessions with 4 groups of 16 students

Once you’ve decided on the number of sessions, take the class roster and assign everyone a session to present in. For example, if I have 8 sessions I assign first 8 students to session 1, next 8 to session 2, and so forth.

Next: in Canvas go to People > +Group Set and call it, say, ROUND1. Then tell Canvas to randomly assign students to N groups where N is the number of groups per session that I will need.

Click the arrow to expand all the groups so you can see who is where. Go to your presentation assignment list to see who presents in round 1. Say the first one is Amal. Scan the students in the group until you find the group Amal is in. Make Amal the group leader and change the name of the group to Amal.

If the next student for round 1 is Bashar, scan the groups until you find the one Bashar is in, make Bashar the group leader and rename the group Bashar.

If the student you are looking for is in an already named group, just swap them with any student in an unassigned group and make this the student’s group as above.

Once you have all the presenters assigned in this round you should see a group set with each group having the name of a presenter.

Now repeat this whole process creating a group set for ROUND2 and so forth.

Once the groups are set up, go to DISCUSSIONS in Canvas and create a discussion topic for each round. The description can be the same for all of them. But when you get to the settings, choose “Make this a group discussion” and then select the group set that you created for that round. Thus the discussion topic “Round 1 Presentations” uses the group set “Round 1.”

When a student clicks on the first discussion topic she will see a notice that says “this is a group discussion and here are the groups you have access to” and then the name of the presenter she will be with that session.

You share the roster with the round assignments so that students know what round they present in along with a schedule saying what time each round happens.

Students create Zoom meetings for their designated time slot and they post the meeting invitation as an announcement in the group workspace that Canvas creates each time you make a group. This announcement then goes out to everyone in the group – all the students who should come to that meeting.

After the meeting, students post feedback for the presenter as posts in the discussion. Canvas keeps track for you of who contributed so you have a record of group participation in the critique session.

The advantages of this method (which no doubt seems logistically challenging but it’s not once you try it and then it’s set up for reuse) are that it systematically gives every student a slice of “the floor” – attention on her work – and that it ties a digital workspace to the conversation. The group space can contain drafts, handouts, pages the presenter creates, ongoing dialog and feedback after the “face-to-fade” synchronous session, etc., mitigating, a bit, the effervescent quality of remote-interaction.

To Grade or Not to Grade

My musings when a friend asked about my thoughts on whether we ought to switch to credit/no-credit for this semester.

At the law school here we are having an advisory faculty vote on this – as I understand it, some law students petitioned for change to credit/nocredit. They tend to be a very anxious bunch. At the Faculty of Information, my home faculty, a few discussions about doing this in particular classes but otherwise current plan is business as usual.

When I think about it, much lands on the side of switching to pass/fail. There’s the fairness of changing horses midstream when students had been marshalling their resources and work habits in light of what they know as their strengths and weakness and such. Maybe I’m banking on acing the final after mediocre midterm, but now I might be thrown for a loop. Or maybe I was struggling to participate, but now I find participation by chat suddenly frees my voice. In short, there is so much unintended, and out of people’s control, movement in factors that can affect the thing we think we are measuring with grades, and so much measurement error introduced by the untested methods we are about to use that the error bars on the grades we come up with will be so large that A might overlap with C and so on.

Counter to this, especially in the case of first year law students is that no small number of external things are keyed to first year grades. It could be argued that credit/no-credit would be a real disservice to the students who were destined to get higher marks. ON THE OTHER HAND, if this crisis forced employers and scholarship providers to look beyond the GPA, that might represent a sizable social good.

And then there is the mindset we are all in which is “adapt and be pro-social.” It’d be nice not to be contradicting that with incentives to maximize one’s own grades in the midst of turmoil.

Selfishly, perhaps, I find myself thinking how much of a nightmare headache it is going to be as I try to be fair at semester’s end. I will be adjudicating a whole bunch of personal situation information that’s below the threshold of official requests for accommodation. There’s always a bit of this, but I expect a lot this round. Is that really what the school wants me to be spending my energy and time on next month?

The employer issue raised above relates to the question of the external utility of grades: are any employers or graduate schools going to apply strict scrutiny for grades earned this semester? Or are they going to know full well that extraordinary circumstances render those grades perhaps a less reliable signal than they are usually taken to be? In a sense, both students and professors would be investing a whole bunch of time to come up with fine grained measurements that nobody is ever going to pay attention to. (WE certainly shouldn’t when it comes to cumulative GPAs if we are honest – course to course variations are wide in the best of times).

Maybe I can get a handle on this with some analogies: sometimes in a restaurant when they mess up your order or have to substitute things they just comp you the meal or part of it.

In the tour-de-france if there’s an accident in the last 3km of the race they just give everyone involved the finishing time of those in the group who make it to the finish line. The point is to avoid dangerous behaviour at the finish AND to not give people time bonuses simply because they avoided getting wiped out by the falling cyclist on the last turn.

Then there is a part of me that thinks we should always be “pass/fail” with a bar that’s a lot higher than D. I’m inherently skeptical that there are meaningful small differences that we can well characterize with things like B+ vs A-. My current grading practice is something like 85-87 (the lower boundary on an A – this is Canada) means competently completed as assigned and then the numbers over that signal impressive extras and numbers below deficiencies and things missing, but I don’t manage much more fine grainedness than that. Since I’ve been grading assignments like that all semester, I’ve done all the “this is great, do more of it!” and “this is short of expectations” formative assessing along the way. A summative “done well enough, let’s wrap this up and move forward” would probably be a smart and responsible move for my professional students in the current circumstances.

Got a Plan? What does the college do when no one shows up in September?

I have not seen anything addressing the following issue, so here I go.

This is college/university acceptance season. The plan over the next month or two at most American colleges and universities was to build the class of 2024. Obviously, a spanner has been dropped into the works. What are the implications?

Most of us are up to our eyeballs trying to adapt to things on the time-scale of hours or days at the moment. When you see things tagged with longer time horizons there are way too many that go no further than “the next two weeks” or “the rest of the month.” The more sober ones are things like “classes online until April this” or, better, “for the rest of the semester and commencement exercises are to be rescheduled.”

But there might be a bigger wave out there beyond the one we can see: the current extraordinary situation might well become the new ordinary for a prolonged period, one that could easily stretch into the fall. If it does, colleges and universities might find themselves with no entering class of 2024. And we might expect higher than normal attrition if what we have to offer in the fall is all online. And maybe some pushback on tuition levels if that’s the product on offer. The astute student will recognize that it’s not just classes that she pays for – the very opportunity to be learning with other people is a big piece of our value proposition.

A very small college with, say, 1000 students and effective tuition revenue per student of about $15,000 would immediately be looking at a missing $3.5 million in tuition revenue from lack of a new class. Add fees and the expected shortfall is easily north of $5m before we even look at attrition among returning students. For an institution with even a moderate endowment that sort of hit means making payroll and bond payments will be difficult. For a school of 2 or 3 thousand students with a more favorable total cost per student the hit could be an even higher percentage of total revenue.*

What’s the Plan?

Maybe schools will be able to lay off all of the Non-academic staff associated with on-campus learning, but caution: that’s an operation that could be hard to rebuild from scratch in the post-Corona era. And there might not be much in the way of net-savings because they will simultaneously be staffing up to support the online alternative.

So maybe they could just layoff faculty, hire more adjuncts. Many of these institutions have already played the adjunct card so there’s not a lot of wiggle room there. And those that take this approach will figure out pretty fast that full time faculty were their product development team not just their pedagogical assembly line workers.

For a decade or two the mindset of administrators has been that online education might be either or both the cost-cutting move and cash-cow activity that they needed. I think they are about to discover that it is neither of these. The venal administrative mindset that has seen instructional faculty as a cost center may have come home to roost.

Why? Because the thing that a fully online college or university will need a lot less of is traditional administrators. Over the past several days faculty around the world have turned on a dime, struggled, and innovated. The results are almost certainly uneven, and it was not accomplished without the able assistance of centers for teaching technology and the like. But, it turns out, the work that instructional faculty do is transformable and the personnel involved are flexible. I hypothesize that the same cannot be said for the layers and layers of administration that have accumulated in most institutions.

The responsible college or university president has to go against her instincts. Those instincts are to sit with other administrators and figure out how to do the job with fewer faculty. The presidents who keep their institutions from ratifying the infamous 2013 prediction by Horn and Christensen that 25% of American colleges will fail in the next decade are the ones who will, instead, sit down with their faculty and figure out how to do the job with fewer administrators.

A Simple Google Drive Hack for Interactive Group Work

My class in human centred design has workshop presentations to do over the next few weeks and so we have to invent so remote analogs to things we would with workshop participants sitting around a table. Here’s one.

Create a Google Doc that is shared for editing with your participants (or whoever has the link).

Create a Google Drawing for each participant. To make it easy to keep track, name the drawing for the participant and put a text box in the drawing with their name.

Publish each drawing. Then, in the Google doc, insert each drawing from your drive on the page, resizing as needed. Adjacent to each drawing insert text such as “Edit drawing” or “Amir’s Drawing.”

Go back to each drawing and click on the Share button. Get the sharing URL for allowing editing. In the Google doc, use this URL to make an “Edit Drawing” link.

The document looks like this (it’s live – go ahead and click the link and edit the drawing):

Participants click on their link and they can modify the drawing. In the Google doc, an “Update” button will pop up when changes are saved in a drawing. Thus, the Google doc displays work of team.

I can go one step further and publish the google doc and get it embed code which I then put on a Canvas page

Remote Small Group Interaction

As we prepare to social distantify our teaching I’m adapting something I use a lot in the “active” classroom to the online context.

I describe the “Pitch and Catch” technique in a previous blog post. Here’s the TL/DR version.

Everyone brings their work-in-progress to class; over four 15 minute rounds every student sits with 3 peers, “pitches” her current draft and gets intense, “iteration forward” feedback*; each round the groups are shuffled so that over the course of 60 minutes everyone sits with 9 different classmates, one round as pitcher and three as catcher. In my classes of 64 students the one hour yields 16 student-experience-hours of having one’s work the intense focus of classmates’ attention and 48 student-experience-hours of give-and-take critique with classmates.

I’ve just adapted this for remote classroom. Here’s my toolkit and workflow.
I built a simple “app” that takes a comma separated list of student names and produces randomized groups of 4 for four rounds. The output looks like this:

undefined

Next I create “Group Sets” in Canvas for each round. This is a little bit labor intensive but barely a half an hour of work for a class of 64. Groups are named so that students can easily connect them back to the table and the round.

Four students are assigned to each one, based on the output of the app above. The pitcher is made the team “leader.”

There are lots of tools we can use for remote video but for this demonstration we use Zoom. The free version allows sessions up to 40 minutes and has a very shallow learning curve; it’s easy to get up to speed with screen sharing, interactive whiteboards, etc.

We make the pitchers in each session responsible for setting up the meeting on Zoom. They fill in some information like this:

And they copy some “invite” text to the clipboard:

Inside Canvas the student goes to the group’s “Home Page” (essentially a section of Canvas pages reserved for members of the group)

and she posts an “Announcement” (just like an instructor can do for the whole class but this one only goes to the group members and the instructor). Each group member sees this and can click on the link to join the meeting at the appointed time.

If the group wants to it can have the host’s machine keep a copy of the meeting. Peer feedback is collected either as answers to questions in Canvas discussion or on a Google form like the one below. We have folks wait until AFTER the session to fill in the form so that it’s the product of some reflection rather than contemporaneous note-taking (which is what we found when folks had the form open during the session).

We do a mail merge with the data (there’s currently a bit of hand massaging needed in Excel to make this happen but we’ll automate that soon) from the feedback form so that each student receives written critique AND we have a “kudos for you” section, an acknowledgment of things other people appreciated during the session. Here’s what the feedback looks like – we either send a PDF through Canvas or paste the text as a Canvas comment.

*a concept developed many years ago at the top of Piedmont Ave with Maia Averett within earshot of Joe Edelman

Wanted: Grading Analytics (about me)

I don’t like grading. And when I have a large enrollment course, the tedium of it can drive me round the bend. To get through the task with my sanity intact I break it into batches, take breaks, etc.

When the task is complete, I have a new perspective on the course I just taught and the students who took the course. I’m a different person for the experience.

It is likely that this transformation is not a sudden one that occurs when I finally put my green pencil back in its holster. I’m pretty sure I evolve over the course of several days of marking. And, too, I’m pretty sure there are other temporal effects too: I get tired, exasperated, bored, delighted, and even angry over the course of the work. I bet that has an effect on the grades I confer, no matter how careful I might be.

To investigate this I click on the “analytics” button on my LMS’s interface. But what to my astonished eyes should appear but a zillion ways to slice and dice data about students and not a one to measure me.

So why isn’t there a function that would tell me if there is a bias that creeps into my grading over the course of a session or over the course of several days of end of semester grading? The system lets me grade anonymously, of course, but personal bias is not what I am looking for here. I want to extract some meaning out of data that’s already there: is there an advantage to being in the group I mark the first day? Last day? Are you better off getting marked at the beginning of a work session or the end? Simple stuff, really, but stuff that would contribute to reflective practice. I’m not imagining an algorithm that will compensate for my human foibles, but one that will keep me inclined to the same sort of continuous evaluation and performance improvement that we deploy with our students. And maybe some practical wisdom will emerge: “don’t grade for more than two hours at a time, Dan” or “don’t stretch it out for more than two days, Dan.” Those would be useful too.

Canvas folks in the back room? Are you listening? This would be dead easy – the data is already there. A lot of it. Give me more tools for self-monitoring rather than more tools for surveillance of my students.

Feeling Learning

While using chopsticks at lunch today I found myself musing about how to train a robot to do the same. One could, of course, attempt (likely in vain) to describe how it is done. Perhaps better than that, one could show how it is done. But the thing about using chopsticks is that when your hand-brain-eye system gets it, you know it. It’s a feeling, “I got it!” Suddenly it totally works. What was awkward becomes natural. Those two slivers of bamboo or plastic become an extension of the self. The strength of the “this is working” reinforcement feedback is tremendous.

Which got me thinking about teaching in general. We do a lot of explaining and describing and even some showing. And we tend to couple it with rewards in the form of approval/disapproval, points added or points taken off. In another jargon, we might say that we are inventing reward functions for our students, rewards rooted, ultimately, in the instructor’s approval.

And many of us bemoan the fact that a predominant result of this process is that the students invest a lot of time trying to figure out “what the teacher wants.”

Of course, what we really want is for the student to master the skill at hand but we offer this proxy reward, our approval when the performance is judged right by us.

Somewhere in this is a theory that if we repeat this process often enough, the student will learn what it was we were actually after (that is, they’ll get beyond trying to win our approval and see that actual mastery of the skill is what it is that earns the approval). We vary the inputs and mix showings and tellings of both problem and solution in a hope to optimize this process.

But what about that awesome feeling when your fingers and brain finally “get” how chopsticks work? What if I could be more successful at communicating that feeling and setting up in my students a search for it? My job can then be converted into helping them move toward trajectories that will pass through that experience.

We might still deploy some conventional approaches to explanation, demonstration, and performance assessment, but I think I’ll be doing a better and more effective job to the degree that I can create opportunities for the production and recognition of that tangible feeling of “I got it.”

Photo by Chopstick.JPG: 毛抜きderivative work: Richardprins (talk) – Chopstick.JPG, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=9559743

Assessing Assessment?

The appalling legacy of “assessment” goes on and on and on. This “frank discussion” at a recent WASC conference is a classic bit of “too little, too late.”

I’m someone keenly interested in the organizational aspects of higher education, especially in questions of how we know we are being as effective and as productive as the world needs us to be. But for most of my career I’ve watched millions of person-dollars squandered on misguided efforts to “document” and “measure” learning. Alongside that I’ve watched the erosion of the intellectual integrity of institutions and individuals as they winked and went through the motions of methods they knew (or should have known) were bogus and would never produce actionable, valid knowledge. We watched as individual faculty members sold their souls for small stipends or to keep on the good side of a dean who might have input into their tenure or promotion case. And those of us who dared to apply our professional training to point out the inanity of the methodological manure being sold to us endured being dressed down for not being team players or having our commitment to students questioned by arrogant small-minded assessment consultants.

A real underlying pathology exposed by the ongoing assessment debacle is the monopoly power of the accreditation agencies. For the last two decades they ranted about accountability in higher education – the one standard they would never have to meet. The hypocrisy of agencies like WASC being immune to serious criticism should be an embarrassment to people who care about higher education.

The simple move of forcing national education accreditation agencies to compete rather than allowing them to enjoy geography-based monopolies would do more for higher education than a thousand conference presentations from people who live off the problem rather than for its solution.

Pitch & Catch: Maximizing Peer Critique and Feedback

In an entrepreneurship course a colleague of mine used to have a “pitch and catch” session in which one team would pitch their idea and another team would play the role of investors or critics while the rest of the class watched. The catching team’s job was to help the pitching team sharpen its presentation. The rest of the class would watch the presentations.

I’ve been evolving a variation on this technique in other courses over the last few years. Here’s how it works.

Suppose students in a class of 40 students are working on individual semester-long projects. After about the middle of the semester when a few installments or drafts of the projects have been submitted to me and returned with feedback, we move into pitch and catch mode. During each class session we set aside an hour or so. The basic structure is like a conference at which there are simultaneous sessions running in parallel.

We break the session into four 15 minute rounds. During round one 10 students will pitch and the other 30 will catch. And then in round two, a different set of 10 will pitch and the others will be assigned catching roles. The catchers are shuffled between rounds so students are not simply listening back and forth to one another’s ideas.

Pitching and catching assignments are projected from a page in our LMS as shown below. We also use this page to link catchers to pitchers’ work in progress so they can consult it (and ideally offer feedback) between class sessions.

The “charge” to the catchers is to make their pitcher’s project as good as it can be. I tell them they want to get into a mindset in which THEIR grade is dependent only on the quality of the work done by the three pitchers they will meet with during the session. We talk about the learning outcome we are aiming at here: the capacity to be a helpful, constructively critical teammate/colleague.

The animation below gives a sense of what the classroom movement looks like.

By the end of the session, each student has had fifteen minutes and the attention of three classmates dedicated to her work and she has caught three times. The pitcher gets serious feedback and repeated engagement with interested listeners and each catcher participates with two classmates in a total of 45 minutes of focused conversation about other students’ work.

In one hour this yields 10 student-hours of presentation experience and 30 student-hours of feedback-experience. The pitch-catch conversations tend to be taken rather seriously – with just three critics there’s some informal pressure to participate and do one’s share of the thinking and talking and in most groups I’ve seen a regression to serious rather than the casual.

Following a pitch and catch session, each pitcher has an assignment to “evaluate” the catching she received. They post a short, private reply to the LMS saying who their catchers were and what advice/feedback stood out. This is designed to motivate the pitcher’s taking notes and reflecting on the feedback received and the catchers to give serious advice. The instructions for the assignment mention that they only need cite helpful feedback, that they should attribute it, and that one of the things I read these four is so that when I’m writing letters of evaluation I’ll have concrete things in mind – e.g., “during our pitch and catch sessions, Jesse’s classmates consistently appreciate their feedback on marketing issues.”

Here’s just one example of post pitch-and-catch session reflection:

My assigned catchers were – Q, T, and A. I found T’s input very helpful. My workshop is targeted towards helping entrepreneurs bring design into their startup using a version user centred design thinking and T suggested taking a look at design hackathons. Hackathons are already popular among my audience so using that concepts will help me connect. I also found Q’s input very helpful. She suggested for the prototype step that I could guide participants toward simple mockups. This was helpful because creating an MVP is a common hurdle.

Pitch-and-catch combines the best elements of small group discussion and presentations of work-in-progress, supercharging both in a manner that yields a lot of bang-for-your-buck. It allows us to get serious small class payoffs in arbitrarily large classes by exploiting the insight that not everyone has to hear from everyone and that with sufficient structural constraints we can get high quality conversations happening in parallel.

If one hour of class time were split equally among a class of forty, with half the time hearing about project and half the time offering feedback to peers, on average each student would have 45 seconds to communicate their ideas and each would receive about 45 seconds of feedback. Pitch and Catch yields about 10 times as much student experience.

To Lecture or Not to Lecture

I still remember things said in some lectures that I heard in the late 1970s, the 80s and the early 90s. In some cases I remember what was said, in some cases I simply think and live differently to this day because of what was said.

Earlier this year David Goodblar wrote a piece in Chronicle Vitae under the title “‘Is It Ever OK to Lecture?’” Mostly the piece just recounts the arguments made in favor of “active learning” vs. lecturing. His take away conclusion is that lecturing is OK if it includes active learning activities. A bit disappointing as a dodge of the question in the title, but it left me wondering whether the whole piece, and much of the active learning conversation, is perhaps grounded in a false premise.

The author characterizes lecturing as just “tell[ing] students what we know.” The standard caricature of lectures is that they are mere conveyance of information with goal being that students remember what we said, but that’s wrong. It might be what far too many instructors do when they are supposed to be lecturing, but a good lecture is something different from articulating information that students are supposed to absorb or write down.

A proper lecture is an interactive experience in which the speaker connects with the minds of the audience and takes them on a journey and changes them in the process. I’m a serious advocate and practitioner of all manner of “active” pedagogy, but I’m disappointed when supporters resort to cartoon versions of alternatives to bolster their case. The argument between active learning and lecturing as it normally happens is between great active learning and mediocre lecturing. It’s easy to win that battle. But out in the wild there’s plenty of really great lecturing and plenty of really tedious active learning too.

Why do we rarely see, in conversations like this, pointers to ideas about how to give really awesome lectures. I think it’s because the conversations are dominated by individuals and organizations that live OFF bandwagons and trends, making their names telling faculty to get with the program or risk being labeled dinosaurs, rather than living FOR education per se. There is material out there – see Chris Anderson on how to deliver a compelling TED talk or folks like Nancy Duarte on great talks more generally or Conor Neill on how to start a speech or so many others just waiting for a little curation – that can help the mediocre lecturer most of us are become the one who can deliver the lecture that a student will remember 30 years later.

When we have that skill in hand we can make smart, strategic pedagogical decisions about when to lecture and when to use other methods.