How Might Behavioral Science Transform Education?

Nathan: What is the value of using behavioral science in education?

Jayden: I think one of the main values that behavioral science can provide here is being integrated within existing systems. So when we think about changing or shifting education systems, we often think of the word reform. This type of change requires a high degree of political will and usually quite a bit of economic backing, as education is a huge system. These can both be really difficult to organize and mobilize, while behavior change in contrast usually can happen within existing systems. On the flip side, it doesn't require the same level of will of individuals or economic backing to actually see that change when using our methods. So that's one of the big values, is that it is a bit more feasible, or a little more nimble.

Jayden: The second main value that I see is that behavioral science can really bring implicit blind spots, biases, and other social, environmental, and structural barriers to the surface, which might be impeding positive outcomes, but aren't extremely obvious. Behavioral research can help understand some of the underlying reasons why big challenges or big barriers might persist over time. In particular, these can be used to identify specific barriers that implicate disadvantaged or minority populations. There's a lot of data suggesting, for example, that students of color are less likely to access higher education. There's higher attrition and they are less likely to advance into more lucrative jobs after graduation. So there are a lot of very deep embedded barriers that behavioral science can bring to light and give us clues on how we might be able to address them.

Jayden: One last area of value is that behavioral interventions can be quite cost-effective. Our insights can seem small but make a significant difference by changing some choice architecture, changing messaging and framing, or providing simple tools and checklists. These strategies seem extremely small and in many ways they are. They're also effectively costless, which means that testing these interventions is more feasible but they can have quite strong impacts when they're applied at scale.

Nathan: Cool. So you mentioned issues around feasibility and funding. For example, in the for-profit industries, it's smooth to get to that ROI focus really quickly. Who's funding behavioral science interventions in education? Who's seeking you guys out and finding your value?

Marielle: That's a tricky question because from my perspective, behavioral science is already starting to be integrated into governmental systems. Consequently, there is greater awareness about it. It has become a buzzword that's being implemented by a lot of nonprofits and foundations. By and large, they're the ones funding behavioral science in education at the moment. But there's definitely been a lot of research that is starting to be done in academic institutions around the value of behavioral science in education.

Nathan: One thing I'm curious about - you mentioned how sometimes it's easier to get behavioral science units going when profit is involved and when it's a private organization - do you have any knowledge about whether behavioral science has been used at all in charter schools, or for-profit education systems that are generally a bit more targeted towards the bottom dollar?

Nathan: My intuition is that that drive might not be there. And their goals might be of a very different nature, right? Like their interventions might not be to get more students to graduate but maybe just attracting the kind of student that they would want, the kind that can pay tuition, for example.

Jayden: I'm not sure if there's such a clear delineation between the ultimate goals for-profit and non-profit education systems in education, just because the outcomes we want to achieve are very different in education. They are less concrete; ideally, it's more about student outcomes and student learning than profit. It's harder to measure those but I do understand the point around for-profit institutions. They may act on different incentives but education is a unique beast in the sense that we're essentially providing a public good.

Marielle: We haven't worked with private K-12 schools but we've definitely interacted with for-profit university systems and we are trying to understand how we can improve access to their programs and services for a diverse range of students. We're definitely thinking about how they can be part of this systematic reform, but it's not, like Jayden said, focused on trying to increase their bottom dollar. It's more just actually focused on student outcomes. Because they are not necessarily an evil. They're a necessary part of the system.

Nathan: Okay. Going from there, you've talked about how education has different outcomes than other areas where behavioral science has been applied in the past. Education is geared towards student outcomes and increasing learning as opposed to increasing profit. What have you learned by applying a behavioral approach to better understand issues in education and how we can address those issues?

Jayden: Recently we have worked on two main areas of education. The first one is in the context of curriculum adoption. We’ll define curriculum as all of the instructional tools that teachers use to communicate subjects and concepts to students. From a super high level, one key thing we learned in instructional materials research is that there is a really bad choice environment in curriculum development. It is really hard for school districts or educational institutions to choose instructional materials that are best suited to students' needs. Some of the reasons that this is the case is that there are tons of choices to choose from and there aren't clear signals about which are high quality and which are not. There are a lot of incentives from publishing companies, for example, to kind of try to co-op the market. So there are a lot of issues related to power at play.

Jayden: And so through our work, we identified really specific barriers at the behavioral level made by individual decision makers within organizations. Some of these barriers included a tendency towards the status quo. For example, if you've bought a textbook from Pearson for the past 15 years, your next adoption cycle might result in a textbook from that same publisher. This could involve barriers related to teachers being most interested in textbooks that have really nice pictures on the front but may not actually communicate concepts in the most clear way to students. A lot of the issues were actually related to limited stakeholder engagement. Not having really clear strategies to engage stakeholders meaningfully throughout the process to ensure that the ultimate decision reflects their needs was crucial. Some of the consequences of these barriers is that there are poor student outcomes associated with having low quality instructional materials that aren't well suited to their needs.

Jayden: Through this work and by uncovering a lot of these barriers, we began testing interventions, which we are still doing. Some of these are related to improving the choice environment for major decision-makers as well as students and teachers, and others involve providing tools to decision makers to use more evidence and be more deliberative in their decision making. Some of the other interventions are around improving group dynamics amongst decision makers, so that they can arrive at a consensus. Through this work, we've realized that it's a really complex process. Education systems are not going to change very quickly but behavioral science can definitely play a role in at least reviewing some of the barriers and providing some potential solutions that we can test out, outcomes that will hopefully have ultimately positive outcomes for students.

Marielle: I think it is worth mentioning the other project that we're most recently been engaged with, which is teacher preparation. It's important to note that for both these projects, these are in the context of the United States. You can see how the barriers that we have identified to improving education systems are applicable in other education systems around the world. Because it's really around how the decision environment is structured, as well as the factors around it, like your social environment, your financial context, and how that can affect your success as a student.

Marielle: With this teacher preparation work, we were looking at how we can improve the access to high quality teacher preparation programs to a diversity of candidates. Basically how we can increase the options for promising teachers and incentivize them to make the right choices. We looked across the teacher preparation continuum, from the time they decide to become a teacher, how they choose their program—including how they choose a high quality program versus a lower quality program—and then how they go through that program and then get hired in a district as a teacher and continue to teach over time. Part of the issue that this is addressing is that there's a shortage of teachers in the United States but there's also a shortage of a diverse range of teachers from high quality programs. So we're specifically looking at increasing the diversity of candidates that enter the teaching profession. 

Nathan: How do you do that?

Marielle: We're still figuring it out but for our research, we did around a thousand surveys to teaching candidates, admin faculty of teacher preparation programs, district hiring managers, and beginner teachers. We also conducted about a hundred interviews with these different stakeholders to get a sense, more qualitatively, of the barriers that they face in their specific portion of the teacher preparation pathway or continuum.

Marielle: One of the barriers that we identified that were really salient was that there's quite poor information architecture of teacher preparation programs. It's not really clear when you're deciding to become a teacher which programs are high quality. There's not a clear definition for this, what program types are available, what financial aids are available. And there's a lot of reliance on environmental and social factors. Often, people will just choose things that are most geographically accessible to them. They will choose a teacher preparation program that's within their city or their town, or something that's recommended from someone in their network. So this decision is frequently reliant on the most, by default, accessible sources on information, that aren't necessarily considered in a holistic sense to see what would be most impactful or high quality to actually enter into.

Marielle: We also found that there's quite a limited availability of appropriate student support services to manage the teachers’ workload, especially because they often have to do a lot of co-occurring internships on the path to becoming a teacher. Prospective teachers are lacking guidance around financial support, how teachers are able to manage their finances is a bit of a black box because it's a really expensive program to begin with. And one often has to begin with an undergraduate program before even starting teacher’s college. It's already a lot of investment early on, with very little promise of return, which is why a lot of people end up dropping out. And it very much favors people in more privileged circumstances. So really trying to think about ways that our interventions can access a wider diversity of candidates by providing them with the necessary toolkit to enter into this program and succeed is central to our work.

Nathan: Do you think that structural issues like finance and the high costs associated with it are things that behavioral science can address? Or is that beyond our reach?

Marielle: I think at the bare minimum, it can start addressing it. Because, for example, some of the interventions that we came up with address accessibility, not availability, when it comes to finances. This is addressing barriers. It's not necessarily that there aren't these financial opportunities or aids available, it's that they're not accessible, or they're not well known, or teacher candidates find out about them much later, when they've already invested a lot of money in loans or debts, et cetera. So it's really just at the very minimum, providing the information in a comprehensible way, at the right time. 

Marielle: So some of the interventions we came up with were opting in candidates' default to access to information and providing them with workshops. If they've been identified as a low income or potentially vulnerable candidate, giving them the resources that they need early on, in the beginning, so that they have this to fall back on without having to search for it themselves, amongst other things that they probably already have to consider.

Nathan: Nice. I'm going to pivot a little bit. Have you ever seen behavioral science applied poorly in education? Are there any times where behavioral science has gone badly?

Jayden: I think the first thing to call out is that when there are examples of behavioral science being applied poorly, there are usually a couple of reasons at work. I think the first one is that any behavioral intervention should be tested first, before it's scaled up. Whenever we design any interventions, one of the first things we do is consult with stakeholders who are on the ground and have a really clear understanding of the context, as well as an understanding of some of the potential negative effects that an intervention could have on particular groups, so that we don't overlook that in an intervention design.

Jayden: Also, we try not to take shortcuts. Sometimes behavioral solutions can be overly simplistic, and kind of ignore complexity. So trying to strike the right balance between having interventions that can address really challenging issues with something that's real holistic at the same time. So that's always a really hard thing to strike.

Jayden: From one really particular example, there's a great book called Weapons of Math Destruction, which focuses on the role of algorithms and AI when applied to social challenges. In the book, one of the key examples is the Boston public school system. This school system implemented an algorithm that was designed to kind of weed out poor quality teachers in the schools. One of the key inputs to the algorithm was the test scores of students in the elementary schools. This was used as an indicator of how high quality teachers were, because the expectation was that poor quality teachers will probably work in classrooms where children are underperforming. This actually led to a really large percentage of teachers of color being fired from their jobs as a result of their students' test scores. What that algorithm didn't take into account was the fact that most of these teachers were working at Title One schools, which are schools with a really high percentage of low-income students, paired with the fact that the teachers who work in these schools are more likely to be of color.

Jayden: So essentially what I'm trying to illustrate with this example is that when we try to find causation, it's not always there. And sometimes behavioral interventions or other types of interventions don't lead to socially positive outcomes.

Marielle: It's definitely not perfect but I think the best thing you can do is really not to make any assumptions about what the challenges are. Our brain is wired to make assumptions, we have to if we want to easily take in all the information around us. In order to take our tendency to take these shortcuts into account, we try and avoid assumptions whenever we can. Our research questions are generally quite open, and we ask quite a broad range of questions. A lot of very quantitative questions but also a lot of open-ended questions that we can probe deeper into if we find something interesting.

Marielle: With the example that Jayden illustrated, context is really important. Through the projects that we've done, especially with teacher preparation, where the context was so varied, depending on which pathway to teaching they were taking, it's really important to consider where the candidate for example is coming from, and what kind of background they have, and how this will affect their decision making.

Marielle: Getting a good understanding of these contextual factors is really important in applying behavioral science successfully, so to speak, within education.

Nathan: This reminds me of my talk with Brooke, when we discussed applying evidence without a proper framework of considering your values and your norms, which structure the choices that you make when you're applying research. I got the sense that a lot of what you're saying comes out of basically just applying math without fully thinking through what you're doing ahead of time. Or maybe just kind of coming to conclusions based off of biases that are kind of prior to the evidence gathering phase of the process.

Marielle: One thing that sets us apart at The Decision Lab is that we tend to think of things through a behavioral framework but we don't apply it early on. We apply it a lot later on. So we use it to kind of conceptualize our questions but in terms of actually designing interventions, we don't try to shove things through a very limited framework that doesn't really capture a lot of the complexities. So I think that kind of helps us avoid this poor application and mitigate potential unwanted outcomes.

Nathan: Interesting. What are the highest impact areas in education that behavioral science can support?

Jayden: One specific area of interest is financial aid. Very specific interventions like increasing the defaults for a loan package can boost student GPAs and increase the likelihood of participation in the program. Another area is in applications to colleges, where simplifying forms and creating new methods for assisting students can reduce the time it takes to apply. This makes it more likely that busy students, who may be handling jobs or pressures at home, to apply to college, since it is less of a pain to do so. 

Marielle: Another really high impact area is in academic outcomes. Strategies like personalized emails to tutoring sessions can reduce the burden of pursuing college. We increased tutor registration by 34 percent in one of our projects. This most likely helped these students adapt to university and find a place that is a good fit for them. 

Nathan: Okay. That's very interesting. Going forward, what are some ways that behavioral science should be integrated into education? What are some things that may not have been done so far, and ways that we could better integrate behavioral science into the design of education systems and resources?

Jayden: I think one really pertinent area right now is integrating behavioral science into online learning resources. Millions of students right now have been forced into an online learning environment that isn't necessarily well-designed for learning use. So some principles around design thinking, designing online resources could be extremely helpful in increasing student learning. So some really specific examples would be for example, creating reminders for students when they need to complete their assignments, or nudges around when they have to enter class. So just really simple visual cues to communicate student expectations would be really helpful with online tools.

Jayden: The student experience in most online courses can be meaningfully enhanced using evidence based behavioral design to eliminate or ameliorate behavioral barriers that undermine student learning. It’s such a novel environment that is just filled with information. Things like timely reminders to complete assignments, or removing hassles that make simple tasks effortful, can reduce the barriers that face over 25 million students across the United States, especially those that already face challenges, such as low-income or marginalized young people. 

Nathan: What are some examples of how you might tackle information overload? I imagine that's a huge problem for students doing online learning right now, where things seem overwhelming. There seems to be too much information and it's not accessible or clearly presented, just because it's a totally new format that hasn't been well-developed.

Marielle: Kind of breaking things down into manageable chunks is a really big thing in behavioral science, because once people see that it's kind of parceled out into specific things that they can accomplish, it kind of taps into that feedback loop where they feel positively reinforced once they complete a task. So they complete that, and then they go onto the next one and it seems a lot more manageable that way. And it kind of improves your intrinsic motivation, because you feel like you've accomplished something.

Marielle: And I think that's already starting to be applied in a lot of online learning systems like Coursera. I think when schools and universities have transitioned into online learning, they haven't really had the time to adapt to those systems that are already really well established. And so they're really behind. And I think a lot of people can kind of relate to the fact that their professors haven't really adapted to this sort of best practice for online learning as well as courses and online learning systems that have already been online for a very long time.

Nathan: Do you think there's anything we can learn from sites like Khan Academy, or even Wikipedia, or other places online that are well-recognized and manageable? People seem to be able to digest them in a way that they can’t yet with online learning.

Jayden: I mean, those resources are often tapping into principles of good teaching, versus things that are strictly behavioral in nature.

Marielle: This question is probably best answered by going back to the question of how behavioral science should be integrated into the design of education systems and resources. Online learning is obviously a very salient area right now, because a lot of the world is functioning online. But I think over time, we need to start thinking more about how behavioral science can be embedded into the design and process improvement of these education systems. So making sure that you're asking the right questions when you're gathering data about the success of programs or students in order to identify barriers that might be overlooked if you're just looking at test scores, for example.

Marielle: For example, the thing with online learning is that we might be designing really optimal systems to engage students but we might be overlooking the fact that a lot of students are not necessarily in the right context to access these online tools and resources. Whether that is due to being under a lot of financial stress or having to care for children or dependents, it can make it a lot harder to solve problems by just changing defaults and simplifying systems. There's a lot of other external factors that behavioral science allows us the opportunity to consider, which we might otherwise overlook. Embedding behavioral science within the design of systems from early on will ultimately be more powerful than just kind of applying it ad hoc, which is what most practitioners seem to be doing with it right now.

Nathan: That sounds like a great place to wrap this up. Thank you very much for sitting down to talk to me.