For TeachLab’s fifth Failure to Disrupt Book Club episode, we look back at Justin’s live conversation with regular Audrey Watters and special guests Scot Osterweil, a game designer and creative director for the MIT Education Arcade, and the esteemed games researcher Constance Steinkuehler. They discuss the history of learning games, their current work, and Failure To Disrupt’s Chapter 4: Testing the Learning at Scale Genres: Learning Games. “I've been studying kids in games for a long time. And oftentimes, when you try to tackle issues of how to treat other people online, how to deal with conflict, how to manage your screen time and also stay physically fit, it's very hard to create interventions around games, that kids just don't spit right back out. They just don't take because there are often layers added on top. They're not authentic to the space. In my efforts, and I'm sure people have done better than me, but in my efforts, it always seems to be colonizing and the kids will ignore me, and it comes off as, mom is wagging a finger saying you need to get up off that screen and go stretch.” - Constance Steinkuehler
For TeachLab’s fifth Failure to Disrupt Book Club episode, we look back at Justin’s live conversation with regular Audrey Watters and special guests Scot Osterweil, a game designer and creative director for the MIT Education Arcade, and the esteemed games researcher Constance Steinkuehler. They discuss the history of learning games, their current work, and Failure To Disrupt’s Chapter 4: Testing the Learning at Scale Genres: Learning Games.
“I've been studying kids in games for a long time. And oftentimes, when you try to tackle issues of how to treat other people online, how to deal with conflict, how to manage your screen time and also stay physically fit, it's very hard to create interventions around games, that kids just don't spit right back out. They just don't take because there are often layers added on top. They're not authentic to the space. In my efforts, and I'm sure people have done better than me, but in my efforts, it always seems to be colonizing and the kids will ignore me, and it comes off as, mom is wagging a finger saying you need to get up off that screen and go stretch.”
- Constance Steinkuehler
In this episode we’ll talk about:
Resources and Links
Watch the full Book Club webinar here!
Check out Justin Reich’s new book, Failure To Disrupt!
Join our self-paced online edX course: Becoming a More Equitable Educator: Mindsets and Practices
Transcript
https://teachlabpodcast.simplecast.com/episodes/bookclub5/transcript
Produced by Aimee Corrigan and Garrett Beazley
Recorded and mixed by Garrett Beazley
Follow TeachLab:
Justin Reich: From the home studios of the Teaching Systems Lab at MIT, this is TeachLab, a podcast about the art and craft of teaching. I'm Justin Reich. Today: the 5th episode of our book club series discussing my book, Failure to Disrupt: Why Technology Alone Can't Transform Education. We're talking about Chapter Four, Testing the Learning at Scale Genres: Learning Games. Scot Osterweil and Constance Steinkuehler joined Audrey Watters and I for a very rich conversation, and we hope you’ll enjoy it as much as we did! Let’s jump right in.
Justin Reich: Welcome everybody, to what is week five of the book club, Failure to Disrupt: Why Technology Alone Can't Transform Education. This week, we're going to be talking about learning games, thrilled that so many of you can join us. And we've got three terrific folks with us today, to talk about learning games. Audrey, who you know from past sessions, Audrey Waters, who's an education technology journalist, and historian and critic. And then two new folks joining us today, and I'll let them introduce themselves a bit. Scot Osterweil, and Constance Steinkuehler.
Scot, maybe we can ask you to start us off. If you can tell us who you are, what kinds of things you work on, and your ed tech story. What's one pivotal experience that you had in encountering education technology, learning games, those kinds of things?
Scot Osterweil: Yeah. I'm Scot Osterweil, my title right now is I'm a research scientist at MIT, and creative director of the MIT Education Arcade. I'm really a game designer by practice, so I've been doing that for over 30 years. And I have so many... I realized, you asked that and I realized that nothing jumped out to me as more salient than anything else.
I guess one I would say that I learned early on, is the one product that I worked on, where we really had sort of a model for how the game would be implemented in the classroom. We thought through that question, not just what's a good game, but what's the environment in the classroom in which it's going to be used? We quickly learned that there is no fidelity. That no two teachers actually deploy the technology the way you had hoped they would deploy it. I mean, some do, some are great, but to me it's an important reality, which is just that classrooms continue to be highly individualistic places. Which I think is probably more good than bad, but that has a huge impact on the way educational technology gets deployed.
Justin Reich: And then we have Constance Steinkuehler, who's extremely generous to join us. She has a bunch of family things that are going on, but we're super excited that she made some time to be here with us. So, can you tell us a little bit about who you are and what kinds of things you're working on these days, and then your ed tech games, learning games story. What was it that catalyzed your interest in these topics?
Constance Stein...: Sure, so I started studying games in 2001, 2002. Not because I was a gamer, but because I was very interested at that time, the internet had just sort of passed about the 50% mark in American households. And I was really interested in what kind of collective, collaborative problem solving online could happen. I'm trained originally in discourse analysis methods for analyzing online social interaction, and so I came to games sort of through the back door, because I was studying these lab environment type spaces, to look at cognition and reasoning in teams, in groups of people. And I got really frustrated, because you could show that different conditions led to different statistically significant effects. But when you opened the transcripts and tried to look at what people were doing, it was very clear that there was very low engagement.
And I felt a lot like it was, I would say it's like hiring the Tabernacle Choir to hum, right? It's hard to theorize and understand what people are able to do in, in this case internet technologies, when you're looking at a space where they don't want to be and they're not trumping. So, Jim [Gee 00:00:03:57], who was a major professor of mine, advised that I switch over to online games, and I downloaded my first one and never went back. I was really captivated by the level of engagement, the level of cognitive difficulty in them, and really fascinated with this question of, if kids and adults, but specifically kids for K through 12 that I'm interested in, teenagers, if they're spending more time on games on average every day than homework, then what are they getting out of it?
So, I really did a lot of work around commercial games. And at that time it was big news to find out that they weren't just simply barren, that they weren't barren intellectually. And from there, moved into policy, et cetera. Right now, we're just finishing up a series of studies on an e-sports program, so it's kind of an enriched e-sports model for kids. And I can talk about that if you want that, that's kind of a little bit different take on the game scene. But yeah, that's how I got into educational games, and games and learning generally.
Justin Reich: So, you weren't a gamer pre-2000, 2001? Your hobbies and interests were elsewhere, and then you were trying to study collaboration, and someone said, "Hey, there's a lot of collaboration that's happening here," and that's how you intersect with it?
Constance Stein...: Yeah. I mean, like a lot of women, I certainly did play at arcades. I grew up during the arcade games and Nintendo, so my family would not allow us a Nintendo. So I gamed, but a lot of us were excluded from an identity as being a gamer. And so, I say well, I didn't really game, but I certainly could crush anyone in the arcade games like Ms. Pac-Man, Tron, Centipede, et cetera. And then Nintendo, whenever I could get to a friend's house, we would play, but it wasn't an identity that I felt was mine or that I was actually allowed into.
So, it wasn't necessarily not doing games, but compared to my husband, Kurt Squire, he grew up on games. He never stopped. He went through whole Sims series, all of that. And for me, and that wasn't a thing. I stopped watching TV, and really engaging in media when I was 17, and kind of looked down my nose at it, until I was in my 30's, and realized that the media had evolved into some very different thing than what I had thought it was like, back in the late 80's, early 90's.
Justin Reich: I have a very distinct memory of a cultural watershed for me, when I was teaching ninth grade in maybe 2004. And I was teaching ninth grade world history, and a young woman, she would've been 14 years old or something like that, came up to me and was like, "Hey, Mr. Reich, do you know about this Civilization game series? I think it would be a great way to connect with this." And she sort of said it out loud, in class, with no particular sense of shame or loss of dignity or something like that. And I was like, we've definitely... I barely would have admitted in my high school ninth grade class, as a white male gamer, that I did that. But if you could just wander into my classroom now as a young woman, and declare that you're really interested in turn-based strategy games, then something has changed within our culture.
So Scot, I wanted to turn to you to see if there are particular reactions to the chapter in the book about learning games, that you had. I think the way that I tried to set up and frame the chapter in Failure to Disrupt, is that in the first half of the book, we look at these three genres of learning at scale. Instructor-guided things like massive open online courses, algorithm-guided things like intelligent tutors, peer learning networks like Scratch and network learning communities, and kind of come to the conclusion that a neat thing about the field of learning games, is that you can both find all three of those things, and you can find some interesting examples of hybridity, more hybridity than you typically find in other parts of education technology. How did that argument ring for you? What did you find misplaced, or compelling, can you start us off just with some of your own take on the chapter?
Scot Osterweil: I was struck, I hadn't thought about that particular sorting of things, although I found it convincing. I guess, I tended to look at games in terms of whether or not they were giving... I'll admit, I sort of lumped games into two categories, of which your first example, Math Blaster was category A, which is just anything that viewed learning as largely Skinnerian. So, you do stuff and you're going to learn it just by doing it.
Justin Reich: Great, let's take a minute to break that down. So, by Skinnerian you mean... Well, let's turn to Audrey for this one. Because Audrey has been working on a book on teaching machines, for which BF Skinner plays a big role. So Audrey, give us the rundown on what Scot means by Skinnerian.
Audrey Watters: Right. Well, this is behaviorism. In particular, I think it's this idea that machines can be used, and computers, but machines in this case. But machines can be used to condition students. And so, you're presented with a series of questions or circumstances in the case of a game. And if you get it right, you get a reward, and that positive behavioral reinforcement is how one learns according to behaviorism. Learning is a behavior, and you want to reward that behavior, in order to enhance it. And so, yes, exactly. You sit someone down in front of [crosstalk 00:09:41].
Justin Reich: Skinner does this with pigeons. He gives pigeon snacks, or maybe they... I don't know if he punished the pigeons at any point.
Audrey Watters: He did not end up punching the pigeons.
Justin Reich: He just wanted to reward the pigeons.
Audrey Watters: Just reward, yes.
Justin Reich: We can pin many crimes on Skinner, but animal cruelty may not be one of them. All right, so Scot argues in a game like Math Blaster, which I'm sure everybody who's our age in the audience immediately knows what Math Blaster is, but younger folks may not remember. It's like this little space invader-like game, where there's a little alien spaceship on the bottom that's shooting things at the top. But the only way you can make the cannon fire is if you answer math problems correctly. And if you answer math problems correctly, you get to keep playing.
Scot Osterweil: The math problem is the obstacle that's preventing you from having fun. The math problem is, if you're a good pigeon and you do your arithmetic right, then we'll let you have the fun of shooting at aliens. Rather than the notion that this is a fun, interesting problem, why don't you sit down and... So to me, I sort of thought, either games treat the core intellectual content of the game, and it doesn't matter whether we're talking to NBA Jam or Call of Duty or one of my games, either treat the intellectual content of a game as interesting and worthwhile, or they treat it as something you've got to get through to have fun.
Justin Reich: So for you, that's the two primary categories of learning games. They're either behaviorist, learning as obstacle to be able to play a game, or you actually find the game and the playfulness in the learning materials that's in there.
Scot Osterweil: And I also, people like Raph Koster would argue that all games are learning games. All games are about mastering a difficult challenge, learning how to master a difficult challenge, and they almost all require some amount of creativity or ingenuity, if they're an interesting game. If it's too easy to master, then it's a boring game. Which probably gets at, you asked if I saw something sort of... I had a different take from you. I guess it's that, you talk a lot and I think rightly so, about issues of transfer, near and far transfer. And so, your look at learning games is largely in the context of how do they align with schooling right? And I guess most people who make learning games are guilty of that, too, of thinking about how does this game align with schooling? Whereas to me, the interesting thing is actually the process that people go through developing mastery and the way that shifts their identity about themselves. I'm cutting to my punchline here, but that I think that the real work we should be doing around any game is helping people recognize the learning they've just done, maybe be able even to verbalize it a little bit. Like it would be great if a kid could say, "Oh, I solved that by controlling for variables." Because kids are controlling for variables all the time when they're solving a game, but they don't know it, and they've never learned to see it that way. And then they never learned to... And then you're right. No transfer's going to occur. There's no way that a kid who does that is going to recognize that that's scientific practice and become a scientist. But if you can help the kids say, "Gee, you were just doing what scientists do when you won Call of Duty," maybe that kid would be interested in pursuing science or history or... leave aside that, just having an enriched and fulfilling life, because they suddenly recognize the power of intellectual activity more broadly.
Justin Reich: That's great. Yeah. So just for people who read less of the chapter or who I didn't describe it very clearly for people, one of the core questions that comes up in learning games is this issue of transfer and transfer is a topic which has interested educational psychologists for at least a century. And transfer works like this. As far as people can tell, when you learn something in one domain, you tend to be pretty good at doing it in that domain. And then when you take what you learn and apply it to something different, it can be very, very frustrating to teachers, to educational psychologists about what a short distance that transfer can go.
One of my favorite stories around this is someone who is coaching schools and math told me a story about a middle-school teacher who spent a whole year teaching her kids about Venn diagrams because they hadn't done very well in Venn diagrams the year before. So they did Venn diagrams all year long, got to the state test, and all of this... You all remember Venn diagrams. They have two little overlapping circles. They go and look at the end of year state tests, which the kids get wrong, and the Venn diagram has two overlapping squares. And the teacher's theory becomes that the students didn't recognize that a Venn diagram problem couldn't just be a circle but could also be a square. And so the following year, she said, "Oh, okay, we're going to still do a zillion Venn diagrams problems, but now we're going to do them with circles and squares and pizzas and hexagons and trains, and anything else that we can think of to make this overlapping transfer work." So a huge question in games is if you spend a whole bunch of time playing Minecraft, you clearly learn how to do all kinds of interesting things in Minecraft. To what extent do you learn how to do things outside of Minecraft? To what extent can you recognize those skills?
And I would say, Scot, what you've just proposed is that maybe one thing that games don't do as much is highlight for you what it is that you're learning and doing in that process. Like, "Hey, Minecrafter, you're doing a transmedia analysis. You're learning about this activity on Reddit and on Wikis and by watching YouTube videos. You could also do that with math. You could also do that with hairstyling and makeup. There's lots of ways that you can do that." But I do think this question of transfer is central. Constance, I know you've thought about this a lot because you've tried to highlight the various ways in which the kinds of things that people do in games have all kinds of resonance potentially in other parts of their lives. How in your work have you thought about people transferring their learning from games to other parts of their life or making games and the environment around games better at doing those kinds of things?
Constance Stein...: Yeah. I mean, I think to build on what Scot's talking about, this notion that... I mean, games are transformative in more ways than simply following a bucket of content, knowledge, skills, and some dispositions, right? And just looking at the comments, I do think it's important to note that if we're going to talk about games and learning, then we have to deal with learning stuff that may not be intended or necessarily what we want students to learn. I think all of that is in play, but I think it's really important, too, to really push back a little bit or at least be cognizant of this model where kids authentically interesting and interested, invested, affinity-driven work is supposed to pay off in classrooms where classrooms are actually supposed to be paying off in their interest-driven, authentic work.
So I just want to push back a little bit because I mean, the topic of transfer... when you can't find transference studies after, what, 25 years in cognitive science field I'm trained in and yet people are learning and applying knowledge to different domains all the time, you have to wonder whether or not our formal definition and our operationalization of that formal definition are really appropriate versus people are just that dumb. I think probably we're not conceptualizing it in very rich ways. I know that I haven't always in my own work. So, just thinking about how do we come up with new ideas, and there are some scholars out there who have, but on the topic of transfer, I think there's something a little bit strange about saying that a game is a design space and what you learn in it may not always transfer outside and yet a classroom is also a design space and what you learn in it may not always transfer outside. I think there are a lot of different ways to think about that.
And one way to think about it, just going back to some really early work that I was doing, because I was coming at it from this traditional transfer model, right, and I was looking at literacy and scientific reasoning. There's some studies I did there in particular where not only were they not transferring it, but they were really resistant to the idea that they should, could, or would, right? Like even calling it, science was really pushed back on by the kids I was studying. They were like, "No, no, no, no, no, no, no." And I think that if you look... I mean, for me, if you look through some of the interview data again and again, what you see is really [inaudible 00:18:41] to what Scot's talking about around, I guess I would say, transformations and an understanding of what one is capable of or who one happens to be in this world.
So it came up particularly... I'm thinking of a couple interviews around leadership because I was looking at multiplayer games. And massively multiplayer games have really fairly robust communities inside them. I would say even more so than other multiplayer genres like MOBAs, right. And that's probably debatable, but in those spaces, people have to think about themselves as a social creature. And one of the amazing things about games... You're both here and there at the same time. And so what Scot was talking about, where there's this moment of reflection to think about like, "What am I doing?" or a moment of pause to think about, "How am I going to respond to that?" Right? "How am I going to deal with someone who just said I can't do my job when I know I can?"
And you've got this moment of pause, in terms of social interaction that you don't get when you're face-to-face. When you're face-to-face, you've got an entire body, face, and all your gestures that are communicating information, whether you like it or not, and you have a lot less time to respond. So in terms of social identity, just as an example of a topic. There's this really important capacity of being both here in the real world and online socially as genuinely interacting with people at the same time. And one thing that came up in interviews was really around this idea of a transformation in who you see yourself to be or what you see your capabilities of being, right. Now, this was early work I did, and all of the interviews, there were all rosy and wonderful and about leadership and diversity. I'm not sure it always has to be so rosy. And now working on the side of more trying to understand what are the causes of disruptive player behavior and harassment, and how do you stop it, what you find is that people absolutely deny that they are both there and here at the same time and deny reflective work on that.
So I think that the sword cuts both directions, but I do think it's a compelling alternative to the standard configuration of learning and transfer that we often come at when we come at it from only a cognitive point of view, if that makes sense.
Justin Reich: Yeah. Go ahead, Scot.
Scot Osterweil: I'm sorry, Justin. No, you go ahead.
Justin Reich: Scot, please, please.
Scot Osterweil: I was going to say what you were saying, Constance, reminded me is we never ask of school what we ask of the things that we talk about outside of school. We never say, "What kind of transfer happens from school to life?"
Constance Stein...: Yeah.
Scot Osterweil: Right.
Constance Stein...: And even more importantly, like... Reading is a good example in that there are reading interventions. Jim does some really wonderful writing about this in his early work. There are reading interventions that literally you can show that you might increase decoding skill, but you end up creating non-readers, right? Now, I want kids to be able to decode because you have to be able to decode to think about content, but I also want them to identify themselves as readers. I want them to think of themselves as voracious readers of whatever it is that they love. So they're connected, but one does not assume the other.
Scot Osterweil: There's an assumption that whatever we've inherited as our model of schooling must be right. So, since I started early in this space, I'm used to people asking, "Are games good for you, or are they good for learning?" No one ever says, "Is Jane Austin or is Shakespeare good for you or good for learning?" I mean, it's just a given because they've always been part of the... And I love both of them, by the way. But they've always been part of the curriculum, so we just assume that everyone's got to read Jane Austen and Shakespeare, and we never interrogate why. And that's not giving new technology a pass. I'm not saying, "Therefore, ergo, it must be fine," but it does show the degree to which we don't examine what we think school should be.
Justin Reich: In your work with schools implementing these games, what are the places that you found where games have been effective in getting teachers to rethink teaching or schools to rethink schools? Just questions coming up along the lines of, perhaps we're disadvantaged the game intervention by not focusing more on changing the nature of the learning environment. To what extent have you seen games be a useful lever for getting communities of educators to think differently or to behave or to organize themselves differently around teaching and learning?
Scot Osterweil: It goes back to my first comment about implementation. It very much depends on how they're used, but when I've seen teachers who actually take the trouble to understand what their kids are doing in the games. They also frequently discover that kids who they thought were unmotivated or uninterested or not understanding actually had capabilities and talents that they revealed in the game. Now, these are probably teachers who wanted to find that. I mean, I'm not sure the degree to which that can work for a teacher who's so fixed in their dispositions that nothing's going to change it, but I think to their credit, I think most teachers want their kids to do well.
Justin Reich: So Constance, I'm curious to get your thoughts on this, especially as it relates to your recent work around e-sports because I feel like one of the ways that... So E-sports for people who don't know about them is for the... And you can disagree if I'm getting this wrong, but it's like taking commercially available games, like League of Legends or Overwatch or other kinds of things like that, creating teams in schools that play against other schools, but because the sort of infrastructure doesn't exist for these sports in the same way as it does for soccer or for field hockey or something like that, the students also end up organizing tournaments and advertising and creating media around the games and things like that.
That there's this whole other infrastructure that ends up existing around the games, which strikes me as... There's usually, my understanding is that there's usually an adult mentor involved in some kind of role. And so it seems to me like it's not just putting a game into an existing space in schools, but kind of an opportunity to create a whole new space in schools in which you might be able to attend to a bunch of these contextual factors that can make learning more rich or feel more connected for students. I don't know. What are you finding in looking at e-sports that connects to these ideas?
Constance Stein...: Yeah, no, I think you've got it well. First, it's worth pointing out that now e-sports has been big in Asia, specifically Korea, for 15 years, 10 years, 15 years now. And really, I never really imagined that it would become a global phenomenon, but right now, I mean, just for the record, there were more spectators of the League of Legends World Championship, not globally, but certainly far outdid the number at before pandemic, NBA finals, which my family is a big basketball family. So it's all to say that there's a lot of talk right now and a lot of energy around e-sports for a variety of reasons. In the open market, outside of education, it's because people see huge profit making opportunities. In the education space, what we're seeing, it's really interesting. I moved from Madison, Wisconsin out to Irvine, California and the first thing that happened when I got to campus was they gave me a tour of our e-sports arena we have.
And we have this winning team in League of Legends and I was totally blown away. I mean, I was just like, I don't even understand where to put this in the world of phenomena. And, but we have a varsity and JV team, we play across multiple titles and now it used to be there was maybe 20 or 30 schools, now it's 170 universities with college scholarships. So now the discussions are evolved and they're around things like how does Title Nine apply? Title Nine applies to everything, including e-sports and issues around who's going to be the governing body. So this is already out the gate. Universities, just to make it super clear, universities now have e-sports teams that are collegiate level that compete and have athletic scholarships attached. So all that set.
Justin Reich: So quickly, in high schools and middle schools too, right? I mean-
Constance Stein...: Yes. Exactly. Exactly. So now what you're seeing is that it's now down into high schools and middle schools even. Middle has been this last 18 months. I was like, Whoa, Nelly. I did not expect that, but really sixth grade and up because you have to be online to play and I think it's possible that the pandemic has maybe escalated some of that, but it was already in play before that and what a stupid word to use. It was already sort of in the process of sort of amplifying. So now all of these high school teams, and there's a whole bunch of for profit, some of which I won't name, but are not at all... I mean, they're, how do I put this? Private efforts that are pay-to-play that really sort of do this hand waving to being good for kids. I mean, I'm in the space of, what? It's my personal commitment. If you're working with kids, it ought to do amazing stuff for them. It doesn't mean it all has to be educational, but it needs to be designed for kids and thought of as for kids and not all of it is.
So it's sort of the Wild Wild West right now and we started, we had involved because there was this effort to make the first ever scholastic e-sports and what that means is that it's an enriched e-sports model. So they had asked me to come in and just first look at their program the first year and understand where would you connect it to content in classes? Where would you connect it to big ideas? And then how do you build activities that would actually do that connecting? So, for example, how do you kids engaged in data science? Because it turns out that understanding your summative data and some of your gameplay data, knowing how to parse that can help you play better and help you compete better. And that really has evolved into us looking at things like tilt and emotional wellbeing and emotional self-regulation, not just content areas, but it's a very-
Justin Reich: Helping the phenomenon of getting super frustrated while playing and having the quality of your gameplay go down significantly because you're frustrated and those kinds of things.
Constance Stein...: Yeah.
Justin Reich: So you're saying making some connections. There may be academic connections, but, and there also may be social, emotional learning connections like here's a place where people have a lot of feelings, let's help them understand their feelings and how to manage their feelings in ways that are productive. I mean, that sounds like right down the line of what middle schools ought to be doing with part of their time.
Constance Stein...: Yeah. I mean, and I have a bigger agenda that isn't necessarily the program's agenda, but my agenda is that I see people acting online in ways to each other that are totally inhumane and game culture is no exception, in fact, it was one of the first places that we saw it really go crazy, really go hyper inhumane. So to me, this is just another place where we might be able to shift the needle a bit toward helping players stop acting in ways that are terrible and hateful to each other. And so we started digging into that some, but one of the interesting parts of this program or one of the reasons e-sports, I think is a provocative space, the downside, the con of using it educationally is that e-sports has played across different titles.
Kids will play the titles they love and that's that. It's very hard to diversify because I would love to see parity in terms of how many women are involved, we're not even at... We're at 30, 35%, not even that close. But you can't go in saying you're going to tell teenage girls that they're going to like League of Legends. Life doesn't work that way. Right? So that's the downside. The downside is that it's hard to figure out how to diversify it in the ways that I want to, but sort of slow going. The positive side is this. I've been studying kids in games for a long time and oftentimes when you try to tackle issues of how to treat other people online, how to deal with conflict, how to manage your screen time and also stay physically fit.
It's very hard to create interventions around games that kids just don't spit right back out and they just don't take because they're often kind of layers added on top. They're not authentic to the space, right? In my efforts and I'm sure people have done better than me, but in my efforts, it always seems to be sort of colonizing and the kids will ignore me and it sort of comes off as mom is wagging a finger saying, "You need to get off of that screen and go stretch." Right? Well, e-sports is interesting because there's this notion of coaches and for the kids that we're studying and this program is now over 700 schools and has a four year high school curriculum attached to it, that's how popular it's gotten. What's so interesting is that these coaches... E-sports is a sport and so coaches are playing and debriefing cheek to jowl with kids.
And so there's this normal place in which an adult or a mentor tells them how to regulate their play, how to regulate their feelings and you can start talking about things like, listen, we can just sidestep the issue of what you ought to do. I can show you in your data that when you behave this way, your entire team falls apart and your performance goes down. So even I can tell you why you ought to be a better person online, but setting all that aside, forget all my values. If you want to be a good player, if you want to be a pro, you have to be able to hold it together and not go toxic on people.
Justin Reich: One set of questions came up that I want to make sure that we got through. It was just about sort of the learning games industry, which Scott, I know is something that you've spent a lot of time thinking about having worked for things with commercial releases, for university based projects and things like that. So I'm just curious, what are your thoughts on the broader industry of learning games? And I think the connection to the theme of the book because it's saying we can't just think about the development of learning technologies exclusively in their learning features or exclusively in their technology feature. It's the whole sort of system that exists around them. So what's your sense of where we are right now? Given that one obstacle used to be that there weren't computers in schools, by the end of the pandemic, a substantial portion of the 57 million kids in the US are going to have a computer in their hands to learn with, is that going to make learning games more viable or are there still sort of other obstacles to be addressed?
Scot Osterweil: No, I mean, I'm not particularly optimistic about sort of market driven good learning games. I would say, and it's gone through a bunch of different cycles, the main, I mean, the issue for games is the same issue for publishing in general, which is to say that the easiest way to sell into schools is to have these giant workforces that only the big publishers can afford. And so what you see with learning games is people come up with interesting or whether good or bad, they come up with some reasonably smallish idea and discover it's almost impossible to make a business out of selling that one idea into a school.
I would say, what exists as a learning games industry are people making apps to sell parents and I think it's always been that way. The goal has always been to sell parents something for their kids improvement. And unfortunately, it happens to coincide with a 30 year period where school has become more and more focused on sort of... It's been more and more conditioned by status anxiety as the middle class fears it's falling farther and farther behind, more and more of parents' interest in their kids' education is about, what's going to give them the leg up in this highly competitive world? Not to do anything good or worthwhile, but just to survive or to get into a college that I think I need them to get into, or whatever it is. And it started in the '90s when I was first getting started, is suddenly there was a shift toward games that were implicitly suggesting that it was going to help your kid get into Harvard. And that's really been, I would say, that's been the industry ever since. It's gone through different fads in terms of this product or that product, but the marketplace has been basically addressing parental anxiety about their kids' learning.
It wasn't that way when I started. I mean, there was a brief period in the early '90s, and it was really because at that point people had computers. There were still people who thought computers were cool to mess around and they wanted their kids to have that same experience of messing around with interesting stuff.
Justin Reich: And so when you have a consumer base that's interested in messing around, there's a wider range of learning experiences that you can create for people, whereas if you have a consumer base that's really focused on status anxiety and making extracurricular happies be things that are career advancing for 11 year olds, then there's just fewer and fewer things that folks can do.
Scot Osterweil: Right, that's right. I mean, I think most of the interesting stuff in the learning game space, and I used to not feel this way, but I think now maybe it's probably because I switched to working in a university, but I do think they're not good enough. But the interesting ideas are being generated by academics, even if they don't necessarily have the art to follow through with them to make good games always or the production know-how. But I think that's the last place where people are still trying to do. And then you'll always get the odd inspired person who somehow manages to work in their basement or in their garage and make something cool, and occasionally one will take off for short periods of time. So it's not that it strictly belongs to the universities at all, but the larger marketplace isn't going to reward those except occasionally by fluke.
Justin Reich: And I think that's a theme that cuts through a bunch of the book, which is if you're looking for places that have the freedom and the runway and the funding to do interesting things in the public interest and in learner interests, it's often easier to create those conditions in universities than in the commercial sector. Although, being a person in the university, I suppose everyone should take that with a grain of salt. I mean, to me, an interesting thing about learning games is that there's all this richness in all kinds of spaces, but still the widest applications of games in schools are things like Kahoot, which turn quizzes into gaming spaces. I think there's a ton of applications of gamified K-5 math happening right now with DreamBox and Misty's Math, and those kinds of things.
Audrey, maybe you could help us by putting some of these games in a longer context. So Audrey's been working on this book, Teaching Machines, which is coming out in February from MIT Press, which you should all read for sure when it comes out. I don't know if pre-orders are available yet, but it's really great. Audrey, help us see how some of the tools that people are using right now as games in classrooms really have ideas and roots that go back to the earliest teaching machines in the United States.
Audrey Watters: Yeah. I mean, as I was saying earlier, I think in education and perhaps in psychology, in educational psychology, we like to say that behaviorism is behind us, that no one's a behaviorist anymore, we all do cognitive science now or neuroscience even, but I would argue that behaviorism really still thrives in ed tech. And in fact, I would say that it's because tech's roots really are in behaviorism, in this idea that we can shape students' behavior through machines, that we can reward them to get them to exhibit the behaviors, to exhibit the things, that we want them to. And I think that something like Kahoot, the fun quiz app, you can really see the way how students are seen as Skinner's pigeons, performing in this experiment in which I guess it's supposed to be fun, but at the end of the day, it's really just a quiz that's got this, I wouldn't even call it the chocolate covered broccoli that you talk about in the chapter. It's not even broccoli.
Justin Reich: It's not even chocolate either. It's insect protein covered with ...
Audrey Watters: It's just a shiny pellet, a pigeon pellet.
Justin Reich: I will say, my students at MIT, I teach a class about education technology, they bring technologies into schools, and I have them use technology in various ways. And so one group did a Kahoot presentation on some topic, and one of the things that the students noted was they had a strong reaction to the Kahoot music being played, which they hadn't heard since high school. But it was basically this conditioned response like, "Oh, the Kahoot music. Now it's time to do a fun thing." So I think that that's a great example. Ann makes the point, Kahoot is routinely on the things of lists they want to do, they seek them out, they report it's more engaging to review material. I mean, I think my MIT students would be great examples of that as well.
And I think the tension for educators thinking about these technologies would be something like, there's plenty of evidence that there are circumstances where playing behaviorists gamified games is seen as a desirable alternative to whatever you were normally doing in class. And on measures of student learning, probably either does nothing or provides some modest benefit to learning. I think the meta-analysis of games in schools leads to the conclusion that gamified things are about as good or maybe even a little better than non-gamified things, depending on what you're comparing them to. I don't know, Audrey. What are your thoughts to that notion that, "Hey, the kids like it"?
Audrey Watters: I mean, I think, again, the person of the Math Blaster! generation, I suppose, is that if you have a choice between doing math worksheets on paper or getting some time in the back of the classroom on the one computer that was in the bathroom of the classroom, that did seem a lot more exciting. But I'm not sure that they're actually substantively that different. It's just that one had some bells and whistles. It was just a different presentation. And it's certainly not the kind of thing, again, that makes you want to do more math, necessarily. It's just a matter of, again, have I done the things I was supposed to do to get this reward?
Justin Reich: Well, we've come now towards the end of our hour together and I hope that folks are starting to see some of the themes that we've been consistently talking about coming together, particularly these ideas of both looking at technologies in their broader context, how does a thing like a learning game fit within a whole ecosystem? If you wanted the learning game to really be powerful and work well, how would you have to change that whole ecosystem? And then recognizing how much the past is still present within our learning technologies. In my examinations of pedagogy, I emphasized ideas around direct instruction and progressive approaches to teaching as important continuities to look for. But earlier, Constance brought up a long history of approaches to transfer, which is worth investigating. Audrey talked about the ongoing presence of behaviorism. So I think there's lots of ways that we can look to the past to better understand the present. So I want to Constance and Scot and Audrey, all three of you, for joining us for a really terrific conversation. Thanks so much for being here and sharing your thoughts with our book group.
Scot Osterweil: Actually, it was fun. It felt like a privilege to be able to talk to your group.
Justin Reich: Thanks, Scot.
Audrey Watters: Thanks for having us.
Constance Stein...: Yeah. Thank you.
Justin Reich: We’re glad you joined us for our Failure To Disrupt Book Club conversation about Chapter Four: Testing the Learning at Scale Genres: Learning Games, with Scot Osterweil and Constance Steinkuehler. We’re grateful to both of them for joining us. And as always, thanks to Audrey Watters for sitting in the cohost chair with me. I'm Justin Reich, thanks for listening to TeachLab. If you’d like to learn more about what we discussed in today’s episode, check out our show notes! You’ll find links about all the great projects we talked about. Be sure to subscribe to TeachLab to get future episodes, and if you like our podcast, leave us a review! Check out my new book, Failure to Disrupt: Why Technology Alone Can't Transform Education, available from booksellers everywhere. You can read reviews of the book, check out related media and sign up for online events at failuretodisrupt.com, that's failuretodisrupt.com. And I want to announce a reboot of our course, Becoming a More Equitable Educator: Mindsets and Practices, taught by myself and Vanderbilt Professor Rich Milner in a free, self-paced online course for educators. Through inquiry and practice, you’ll cultivate a better understanding of yourself and your students, gain new resources to help all students thrive, and develop an action plan to work in your community to advance the lifelong work of equitable teaching.
Justin Reich: If you’ve previously taken this course, we’d love to have you back. Bring your colleagues, form a learning circle in your school or community, or just come and participate in our online community. You can find the link to this edX course in our show notes where you can enroll now. The course launches January 15, and it runs until August 26th, free at your own pace. This episode of TeachLab was produced by Aimee Corrigan and Garrett Beazley. Recorded and sound-mixed by Garrett Beazley. Stay safe, until next time.