Building Better Governments With Behavioral Science: Margarita Gómez

Communicating with public sector stakeholders when conducting research

“I think that when we are starting to work with governments, it is important first, how do we frame our communication? Second, start with something small and then take a little more of a risk because there is a relationship, a trust relationship that we are building between researchers and policy makers. I think that it’s good to start small and then go a little more, go bigger with more ambitious questions. I think that we need to engage them in all the process, to make them part of the process”

The importance of engaging public servants for behavioral science interventions

“I think that it was so important to engage [public servants in the Mexican Federal Government] and show them how they can understand, study and design these policies using the behavioral science lens. Sometimes you can run an experiment and that will be great, but also sometimes you can get this knowledge and design different forms. They might not be tested, but they can learn in the process.”

Convincing others of the merits of behavioral science experiments

“I’m going to talk first about how we engage government. The first thing is, we need to knock on a lot of doors. We need to talk with a lot of people trying to get that buy-in and trying to tell them why it’d be a good idea to run an experiment and use behavioral science. In order to start a project, one of the things that is super important is to find champions. And when I say champions, I mean someone that wants to take the risk to explore and to run an experiment that we can have results, or we can not have any significant results.”

How people behave in different contexts, cultures, and countries

“There are few studies in developing countries in the topic that I study, that is honest behavior. We have few results to really compare if the behavior is different. What I see is that we need to do more. We need to do more experiments in the different levels, lab experiments, field experiments, applied research, more theoretical research, to have more insights about these in developing countries and how people behave in different contexts, different cultures, different types of social interactions. I think that we don’t have yet enough studies to say, and to compare and conclude if the behaviors are different.”

The need for behavioral science research in developing countries

“We don’t want to create this dependency. And I think that we should be building capacities in the [developing] countries. It could be through partnership. It could be creating these specialized units in research centers, in universities, or also creating these units inside the government with the different models that can be followed. That is essential, and it should be also a part of our agenda to create these capacities in these countries. Because it’s not just about the methods, it’s also about how do we see the problems? How do we understand problems and how do we design public policies in a different way using behavioral science.”

Perceptions of an Insufficient Government Response at the Onset of the COVID-19 Pandemic are Associated with Lower Mental Well-Being (Academic paper)

Applying behavioral science to increase compliance and reports of gifts received by public servants in Mexico (Working paper)

Behind and beyond nudging (Online webinar video)

Global Behaviors & Perceptions at the Onset of COVID-19 (Online webinar video)

Blavatnik School of Government, People in Government Lab (Website)

Brooke: Hello everyone and welcome to the podcast, The Decision Lab, a non-profit think tank dedicated to democratizing behavioral science. We conduct behavioral research and consulting projects with clients such as The Gates Foundation, The World Bank, and governments and nonprofits around the world, helping to improve outcomes for all of society. My name is Brooke Struck, Research Director here at TDL and I’ll be your host for today’s discussion.

Brooke: My guest today is Margarita Gomez, Executive Director of the People in Government Lab, located in the Blavatnik School of Government at Oxford University.

Margarita: Since October 2019, I am leading the People in Government Lab, which is the new innovation research center in the Blavatnik School of Government at Oxford University that is focused on how can we improve public servants, motivations, effectiveness, and responsiveness around the world. And why did we think that is important? What we saw is, there is a general trend to have a lot of labs innovation, units, centers focus on designing better public policies, but we have less efforts on applying experimentation, behavioral science, and new methodologies to improve public servants’ performance and the efficient making process.

Margarita: That was the main idea of creating the People in Government Lab. This is an idea that we have been discussing since 2018, more or less. And we found a space that needs to be filled with academics and with practitioners and with more knowledge to try to build the bridge between practice and academia.

Brooke: It sounds like the initiative is still very new. Do you have an example that you can share with us to illustrate, to put some meat on the bones with what you’re talking about?

Margarita: Even though the Lab was created in October 2019, we have been and I personally have been working on how we can apply behavioral science and experimental methods inside of the government. One of the main projects that I did was related to increasing honest behavior in public servants. In Mexico, specifically, we identify some of the policies that the government was already implementing. And we use some insight from behavioral science to make that public servants report the gifts that they received during Christmas. Even though in the law it’s stated that public servants shouldn’t receive these gifts, in cultures like the Mexican, they still receive these gifts during Christmas.

Margarita: And we thought this is really clear behavior, it’s a specific behavior, and we can try to test and to see what we can do to increase compliance with the law and also to improve honest behavior in public servants. This is the first of steps that we have been doing on this. Now we are working in the Brazilian government also to increase motivation of public servants.

Brooke: I’m very curious. Can you share a little bit about what the response was from the Mexican public servants?

Margarita: The behavior that we wanted to change was increasing the reports of gifts that public servants received during Christmas. Just a little bit of context. As I said, this is prohibited by the law but it’s still happening. Because during Christmas it’s okay to receive gifts. It’s culturally accepted. And what we did is, we explored the literature on behavioral science and we saw one thing that was important is, usually gifts in this context are used as a kind of bribe. This is a behavior that we want to change in government because it can influence the behavior of public servants and it can affect their impartiality.

Margarita: The first thing that we did, the first treatment was about just making it simple, telling them that there is a law that says in a simple way that they have to receive it. And we also reduce the friction cost, making a simple form where they can report online the gift that they have received. Just the first one was, making it simple and telling them in a really easy way, what they have to do and timely. So we send them messages during Christmas. The second one was, we told them we use social norms. We told them that some of the colleagues in this case, around 1,000 public servants, have already reported the gifts that they’ve received in the past couple of years.

Margarita: Then we also know because of the literature that making salient the consequence of our behavior can change the behavior of individuals. And when we did this, we told them that receiving a gift could compromise their impartiality. The fourth treatment that we have, it was focused on reciprocity. We told them moral norms and reciprocity. We told them that we already trust them and that we know that they will behave in the proper way, according to the law, that they will report the gift if they receive it. The final treatment was focused more on threatening them, telling them, “If you don’t report this, someone else is going to report the gift that you receive.” Also, we put some eyes in the email. All these emails were sent online. It was already a policy that the government was doing.

Margarita: In total, we sent around 1 million emails to 157,000 public servants in Mexico.

Brooke: One of the things that stands out to me in that example is there’s this interesting back forth and interplay between the behavioral intervention, sort of the more typical structural piece. You mentioned that you were using social norms, there’s many of your colleagues are already exhibiting this behavior. And that’s more typically on the behavioral side of the line, if you will. But you also had one intervention that was very much focused on the structural issue, that there is such and such a penalty associated with this behavior, but you still have the kind of behavioral flavor add into that with adding some eyes in the email to make salient to people. They’re being observed that not only does this structural feature exist, but there’s also this social observation aspect that’s going on. I think it’s very interesting to see those mixed together.

Margarita: I think that when we think about the last treatment, that is the one where we used, someone else can… that is the threat of the sanction, someone else can accuse… you can be sanctioned for this. We more or less thought about increasing cost, the social costs of misbehaving or behaving in a dishonest way. However, when we saw the results, the only two that were significant and statistically significant were first, just making the message simple, making it timely and also reducing the friction costs. So, that’s treatment one. And the second that was also statistically significant was, making visible and salient what could happen if they receive a gift. How their impartiality could be compromised.

Margarita: We did qualitative research. We did some focus groups and some interviews to try to understand a little more about the mechanisms. And the last one that it was the one pointing out that someone else could accuse them. It was very aggressive. And also like public servants felt that it was creating a really negative environment in their offices. The takeaways of these experiments, additionally, to the study that we did and to the findings that we have. It’s also how to talk about honest behavior and corruption. We usually frame that in a really negative way. That is the main communication style.

Margarita: What we found is that public servants find that really aggressive. And instead of promoting the behavior that we are willing to promote, there are a lot of negative reactions to that. We could give these insights to the government because we did all these experiments and run together with the federal government. We gave them some insights and also during all the experiments, they were learning how to design the experiments. They were learning how to design the messages and use behavioral science. And also something really simple but I find that really valuable, they created databases and they systematized information. That also helped them to make some decisions. For example, we found some gifts that were really expensive.

Margarita: One of the protocols that the government established was, okay, these gifts that are super expensive to have an investigation, just to know what is happening here. I think that there were really good policy implications about doing this project together with the government

Brooke: In discussing the interventions, especially treatments that you rolled out. One thing stood out to me. You mentioned that there were five or six interventions, two of them worked. From my own experience, working in government and working with colleagues who are presently in government from the outside. I noticed that some people have a different perspective on that kind of issue. For some, they say, “Well, we tried six things and two of them worked.” For others, they say, “Well, we tried six things and four of them failed.” There was a big aversion to risk within the culture. It’s many, many governments. I think that this is a prevalent problem around the world.

Brooke: And when we’re running experiments, there’s always the possibility that they will fail. That’s a necessary component. We must fail in a certain number of experiments in order to figure out the thing that actually works. But within the context of very culturally or a culture that is very averse to risk, how do we address that challenge? How do we get the space to be safe for public servants to start experimenting so that they can sort of test it out without worrying?

Margarita: I will say that there are different elements. The first one is in our case, at the end, we were trying to understand better dishonest behaviors as a proxy of corruption. We avoid using corruption because we know that it might be more difficult to engage people about that. So we spoke about honest behavior. We reframe our study in order to increase engagement. The other thing that we did, this was a really… We were not studying the big issues, like the big corruption. This was something where we can have some results in a short time. I usually say that it’s just the foot on the door. So it is a specific behavior. We know a lot of non political topics and where we can give some results in a short time.

Margarita: I think that’s when we are starting to work with governments, it is important first, how do we frame our communication? Second, maybe first start with something small and then take a little more of a risk because there is a relationship, a trust relationship that we are building between researchers and policy makers. I think that it’s good to start small and then go a little more, go bigger with more ambitious questions. I think that third, it is to engage them in all the process, to make them part of the process, to also give them something. Because at the end, for example, the experiment doesn’t work, they also can get from these, they can learn, and we are also building capacities.

Margarita: They also know that it’s something that is not really expensive. It is something that it can have a low cost and they can also use these insights in different projects that they have. I think the communication, start small and then go bigger and engage them in all the process.

Brooke: Yeah. That’s very interesting. It relates to a point that I’ve discussed with some other colleagues. And that’s another episode in this season of the podcast, we’ll be touching on that more extensively. The topics that it touches on is what will help nudges to become quite mainstream within government, as opposed to becoming something very niche. What you mentioned here about involving the policymakers in the process is very relevant to this. If we involve them in the process, we develop that internal capacity. We expose them to what all of those steps look like. In some, there’s an appealing aspect to saying, “Okay, well, we work with the policymaker to define the problem.” But then we, as the experimental group as the behavioral scientists, will retreat into our lab setting and we will conduct this on our own.

Brooke: And then when we’re done, we’ll come back and show you the results. But as you mentioned, that doesn’t contribute as much to developing trust with those policy makers. It doesn’t contribute to developing capacity within those contexts. So for behavioral scientists, if we were to always follow this model to help us define the problem, then we will take ownership of that. And we’ll come back to you when we’re done, that could contribute to us isolating behavioral science from the mainstream of what happens in government. As a result, we might find that there are behavioral science units that are dedicated, but also insulated, as opposed to saying behavioral science is one of the tools that the government uses all the time for all kinds of things.

Margarita: Talking about these experiments that we’ve run together with the Mexican Federal Government. I think that it was so important to engage them, not to engage the unit responsible for this. And also to show them how they can understand and study and design these policies using the behavioral science lens. Sometimes you can run an experiment and that will be great, but also sometimes you can get this knowledge and design different forms but they might not be tested, but they can learn in the process. For us also, it was super important that they can create databases. Databases that after that, as researchers, we can use that to go into deeper studies, but also that they can use to make better decisions in the day-to-day. Something that is useful for them. Something that gives them information that is relevant for the decision making process.

Brooke: The creation of databases is an interesting challenge these days. Of course, there’s a lot of visibility around data privacy and concerns around that. There’s a slightly different point that I wanted to pivot to. You mentioned that starting off small is really important, involving government stakeholders and policy makers along the way to expose them to this process. But how can we get the very first project off the ground? In a previous conversation, one of the ideas that you mentioned that stuck with me is this need to have champions. Could you explore a little bit the role of champions in helping to get these initiatives started and their role as the initiative scale?

Margarita: Yeah, I think that is a fascinating topic. How can we engage the government to apply behavioral science and to do applied research. On one side, we have the government on the other side, we have researchers. I’m going to talk first about how we engage government. The first thing is, we need to knock on a lot of doors. We need to talk with a lot of people trying to get that buy-in and trying to tell them why it’d be a good idea to run an experiment and use behavioral science. In order to start a project, one of the things that is super important is to find champions. And when I say champions, someone that wants to take the risk to explore and to run an experiment that we can have results, or we can not have any significant results. But I think it is also a matter of framing if we just focus in the end, I mean the final result, it might be a little sad.

Margarita: I think that it’s more important to focus on the process. In all the research, from the moment one develops the idea together until the end, and in the process, all of us, we can bring a lot of things. What we did in these, in order to engage governments, first of all, is trying to find who can be our champions. Second, we didn’t talk about the final result that we wanted. Specifically, We talk about, this is a process where we both, we are going to learn and we are expecting this. And you can get the results in a short time. This experiment was really good because it was in our short time. Finding these champions is really, really important. Champions that can have the power and the position to move this forward. Also, the resources to support all these experiments.

Brooke: You mentioned also, needing to engage the research community in this. As you just covered, there’s risk aversion in the context of governments towards things that might not succeed and might be seen to not be succeeding. That’s a big worry on the government side. And champions are very important to provide a space for experimentation to take place, to provide a bit of inner cover if you will, for things to proceed, even without the assurance that there’s going to be a demonstrable result at the end. But what about on the research side? How do you see researchers feeling about engaging with the government?

Brooke: And I think especially one of the points of worry I’ve heard from researchers is that they worry that engaging with government will allow their research to be co-opted to government agendas, that there’s an impartiality or a neutrality to academia, that they would be sacrificing in engaging with government. How do you address those worries from the research side?

Margarita: As I said, one is, how can we engage government? Then we have the other side is also, how can we also engage researchers to do more applied work with government? I think that here we have a big challenge because we are talking about the incentives that academics have in universities and that kind of thing. There are some changes going on. I see that they’re more willing to do applied research inside of academia. They’re having a lot of discussion about how can we make relevant the research and academics that we are doing in the universities. And when we say relevant, it is how can this bring some knowledge and some insights to the real problems that our society and our governments are facing. And I see a new trend there, there are more scholars and researchers that want to do applied knowledge.

Margarita: I think that also the fact that we have different Nobel prizes that are in economics, for example, that are recognizing how valuable is to do some applied research that is relevant for the problems that our societies are facing. There is a trend there. Also, in some universities, for example, here in the school of government, we recognize the impact that our research has. Not just regarding how many journals did you publish, but also about how the impact that this research is having in policymakers, in the real world, in specific policies that can be implemented. So I think that is another thing. First, I think there is more willingness from the researchers to work more with governments.

Margarita: Second, I feel that there are some institutions and some universities that are recognizing the importance of having impact and also communicating our knowledge to a broader audience. And the third one, I think that also what I have seen is that there’re new journals that are also publishing some of the research that deal with the problems on the ground. The incentives are moving a little.

Brooke: Part of the identity of many academic scholars is around rigor. Do you feel that there are some ways that the worries about rigor are being mitigated? The worry that let’s take an example, in conducting a study with the government in order to minimize the level of risk and the cost associated with the study would be to take only a small sample because of some political or policy worries. It might be difficult to get a sample that will not be systematically biased, but that’s the way that research needs to go sometimes in the government context. Whereas, from the academic perspective, we say, “Well, that’s maybe not such a rigorous piece of research.”

Brooke: Do you feel that there are some ways that some structures that allow rigor to be protected or embedded a little bit more firmly when we engage with afters in government as researchers?

Margarita: It is good to recognize that there are trade-offs when we work with governments. There are trade-offs, and also there is the need to find a common ground that you usually don’t have to do when you are doing your independent research. Because you can just define your question and then you can just run your experiments. When we are working with governments, I will say that this will be more research for public policy or applied research. We need to discuss what are the relevant questions for them? What are the main problems? What are the relevant questions for us as researchers? Find a common agreement on what would be interesting and get some information to the challenges that they face. And also something that can bring knowledge. In that way, there is a trade-off. We give things to gain others.

Margarita: How can we deal with the temptation of changing the path of the research or changing some of the potentials or framing the different ways on the potential results. I think that in our experience, we have followed some specific steps. First, I think that we need to register, pre-register the experiments that we want to run. What are our hypotheses? Where is what we are looking for? What effect are we waiting to have? And I think that gave us credibility to our colleagues inside of academia, but also it establishes a common agreement with the government. Because it is a statement that what we are doing, it has to be rigorous. It has to follow some scientific steps. That is one of the things that we are doing.

Margarita: The second thing that we are doing is we share data when it’s possible, protecting and following all the standards that we need to protect personal data. But we share the data that we have. One thing that also has been important for us is, a part of the policy recommendations that we can have from these kinds of experiments. We also want to publish in journals and to make this knowledge available for academia for a general audience. And that is important because they understand that they are in that case, there are some steps and some protocols that we need to follow if we want to work together. In my experience, it’s a matter of communication. It’s a matter of understanding the incentives that they have and understanding our incentives, finding a common place, a common agreement. I think that it’s possible to have both. To be relevant in their real world and in policy, and also get good research and bring some new knowledge.

Brooke: In developing new knowledge and this trade-off, or finding common ground that you talk about. There’s attention that I’ve encountered. And maybe one way to phrase it would be like this. In government, There’s a lot of focus on the magnitude of an effect that you can measure. It’s about whether you can make a meaningful difference whereas, a lot of the incentives within academia are focused on statistically significant differences. Those are two concepts, of course, that are related to each other. But the focus is quite different in those two things. A government might say, “Well, even if the effect is statistically significant, the magnitude is so small that it’s not meaningful.” Yes, you’ve demonstrated that it can add a change beyond a launch. We know that this change is real. That is going to happen. The change that you would make is of such a small magnitude that it’s not worth doing.

Brooke: I’m not worried about the risk that this might just be noise in the data. Whereas, on the academic side, there’s much, much more focus on whether what we’re measuring is noise for signal. And so that focus on statistical significance. My sense is that, especially when it comes to journal publications, the focus on statistical significance is very, very intense. And this is how we end up with, for example, the file is, the effect where no result studies are not published. And this leaves a glaring hole in what’s visible as the outputs of the research enterprise. Do you have some thoughts on that, on the tension between meaningful effects and statistically significant effects on no results and how we communicate our work?

Margarita: Governments are looking to have a huge effect on the things that they are doing. But also, what we know is like a lot of the policies that they are implementing, we don’t know what is the real effect that these policies are having. When we put on the table that we can do these kinds of studies and experiments, we will at the end know the real effect that these interventions or policies are having, that is a good framing. I have seen that governments buy the idea. They like to know, and they are willing to know about what are the real effects of the policies that they are having. I already said, I think that it’s important to start small so they can see how we implement the projects, how the method works, that is one challenge that is there. But also that we can deal with that when we work with the government.

Margarita: The second challenge, I think that is bigger. That is about the significance,  the statistical significance of our studies. Maybe a couple of years ago, having a null result, it will be the most sad moment for a researcher. Now, when you are after running an experiment to see that your experiment doesn’t have any result that is statistically significant. What I see now is that this is changing. What I see is that different journals are valuing and considering that it’s important also to publish null results. What is behind this, is that we are now recognizing that having null results doesn’t mean that your experiment didn’t work. It means when we were testing, our hypothesis might not be the ones that are important for the final factor that we want to change, but that we can learn from this process.

Margarita: Having null results can tell us a lot of things, can tell us what has been done and didn’t work. And how can we… it gives us some insights of other things that we can try, because maybe the traditional hypothesis doesn’t work. It gives us information about that. It also gave us information about the process, how these experiments were done, why it didn’t work, what are the reasons. It gives us a lot of knowledge that it might not be the causality but it also gives us information about other things that we can try. I think that the good thing of presenting and publishing null results, we can build above the knowledge that is already there, above some findings that give us information about what works and what doesn’t work.

Brooke: We’ve talked a lot about the relationship between research and government. And I think that it’s valuable to talk about this idea of null results, which means something very different in the research context and the government context. In the research context I think you’ve done a very nice job of articulating some of the challenges of how a null result is perceived and the way that’s evolving. On the government side, a null result means you tried something that didn’t work and you didn’t have an impact. Of course, there are difficult discussions to be had in a government context around that as well, where there’s so much pressure to demonstrate that everything you’ve tried was a success. I’d like to shift a little bit now and pull apart this concept of government.

Brooke: Governments are not the same all around the world, and one of the stark differences that’s been noted in bringing experimental approaches to government is the difference between governments in developed contexts versus governments in developing context. For yourself as someone who’s worked quite a bit with governments in developing contexts, I’d love to pick your brain on that topic. What can we do to improve or increase the use of behavioral science interventions in the context of developing countries?

Margarita: First thing, it is that the challenges are different. In developing countries, we are also facing a lot of lack of, or challenges related with structural barriers that are basic like the rule of law, if we are talking about transportation, the lack of infrastructure, we need to recognize that. And I think that is important in the work that I have done in developing countries. I think that it’s important to tell governments that we recognize that there are challenges that are there that are more institutional and structural challenges. But even if we have that, we still can do something using behavioral science while we are also working in the institutional and structural challenges.

Margarita: What I have seen is the first thing, we can get inspiration from studies around the world mainly in developing countries, but we need to recognize as we study behavioral science in that context matter. That the context is important. In a lot of the studies that we did in developing countries, we always do the scoping of the context. We do a lot of qualitative research to understand really well what are the challenges in these regions? Then we also develop the interventions considering all these insights. I think that there are a lot of things that we can do. I feel really excited about the idea of implementing more behavioral science in developing countries, and also doing some comparative analysis and trying to see how we can implement these interventions in different contexts in these countries.

Margarita: Even though there might be a lot of doubts about how we can apply behavioral science in these countries, I see a lot of opportunities. And I also think that it is a good way to start to change some behaviors. The example that I gave you at the beginning about the honest behavior, when we started to think about this experiment, a lot of people, the ones that were studying corruption for a long time in Mexico, they told us this is not a big issue. That was perfect for us because it’s like, that is what we want is to change specific and little behaviors in the beginning.

Margarita: We started with this specific project in honesty, the experts on corruption, they were saying, so that is not important. But at the end we could change some behaviors. And we also could change some norms inside of the government. One thing that happened as a result of our experiment is that some of the ministers who didn’t receive any gifts in the main entrance of the buildings in the government. And we couldn’t quantify that. It wasn’t good for our experiments talking about numbers, but at the end, this is the kind of behavior that we want to change. There are a lot of opportunities applying behavioral science in developing countries. There are ways to show also that it’s possible.

Margarita: We have done this experiment in honest behavior. We also have done some experiments about increasing women’s participation in rural areas with good results. We have also done some studies about increasing female high school students’ interest in STEM careers. What I see is there are a lot of efforts in developing countries in Latin America, Africa. We can add the behavioral lens and the behavioral perspective to make the projects and the policies that are already implemented better. And we can do different topics. We also were doing some work about police motivation, gender violence is not like substituting or it’s not. I see behavioral science as a complimentary intervention of the ones that the governments are already doing. And I think that that is the right frame.

Margarita: We understand that you are doing all these policies already, but I think that if we use behavioral science, we can have a better approach to the problems and we can have an impact on changing behaviors. I will use that this is a complementary perspective to make better and to improve the policies that governments are already implementing.

Brooke: When you’re working in developing contexts, do you feel that the knowledge base of behavioral science is serving you well? For instance, if we run an experiment with a few hundred undergrads from a psychology 101 course, the population that we will be studying is probably very different from the type of population we would then be working with in the context of behavioral insights with governments in developing countries. Do you have any experience with the kinds of insights that you find in literature being useful or not useful, or needing to be modified in a certain way to translate them from what is often an experimental context that’s quite different from the application context that you’re focused on?

Margarita: I’d say that we don’t know. There are few studies in developing countries in the topic that I study, that is honest behavior. We have few results to really compare if the behavior is different. What I see is that we need to do more. We need to do more experiments in the different levels, lab experiments, field experiments, applied research, more theoretical research, to have more insights about these developing countries that are and how people behave in different contexts, different cultures, different type of social interactions. I think that we don’t have yet enough studies to say, and to compare and conclude if the behaviors are different. I think that we need to do a lot more to see if people behave differently in different contexts and in different countries.

Margarita: We have a lot of… like the studies that we have, are mainly coming from the States from Europe and from the UK. But we don’t have a lot of studies about these behaviors in Latin America, In Africa, in Asia. I see a huge opportunity here from both sides, from academia to try to understand more and see if the behaviors are the same. But also, I see a lot of opportunities to work with governments to see how we can do more of these applied research? I think that with information that we have now, we can’t really conclude if they behave the same way or not. There is an opportunity there.

Brooke: In the context of this opportunity, and I agree with you that there is opportunity there to really expand the knowledge base and to improve the way that governments function, especially in developing contexts. You mentioned earlier the structural constraints on the resources that are available for developing governments or governments in developing countries and the infrastructure that they already have. How do we create a scenario where the use of behavioral insights does just create one more line of dependency between developing countries and developed countries? So the kind of dystopian scenario that I have in mind is that we say, “Well, behavioral science is a really, really valuable tool in these contexts.”

Brooke: And so a bunch of money from governments and foundations gets poured into sending experts from Europe and from North America to Africa, for instance, to conduct a bunch of studies, develop the knowledge base, help to design interventions, to help governments improve their conditions. But what has ended up happening is we have developed the knowledge base and developed the capacity, actually among those experts who are being sent to developing countries. How do we create opportunities for the developing countries themselves to build capacity along with us as money, resources, expertise, get pumped into these contexts?

Margarita: I would say that three main actions that should be implemented, or we should consider to avoid what you are saying. Because we don’t want to create that dependency. First, I think that the international organizations, international systems, universities that I am now in, we should partner with local researchers in order to also develop that capacity in academia. That is one way that we can and I think that we should be doing that in order to develop the knowledge and the capacity in the countries. And also because I think that it’s important to recognize the knowledge that our colleagues in these countries have not just about the techniques, but also about the real context of the country. That will be one.

Margarita: The second one would be, a good example is the unit that I created in Mexico, that it was based in our research center. These units brought together also help to form and to educate students in behavioral science and experimental methods. And now, I think that there is a capacity there that they are also running their experiments, and they are continuing working with governments using these methodologies. So I think that, that will be a second path that we can take. It’s like, how can we help these governments to create their own systems or unit universities or in research centers?

Margarita: And the third one that I will say is a third option, all of them can happen together. The third option could be, how can we also develop knowledge inside of the government? And we have different models here. We have units like centralized units, usually in the presidential or in the cabinet office, that have a group of researchers very well-trained in experimental methods and behavioral science that help all the government to apply these techniques and this perspective. That could be one or also the other model that I have seen is that when you create these units or these experts in the different agencies and in the different ministries. So it can be implemented in all the government.

Margarita: We don’t want to have to create this dependency. And I think that we should be building capacities in the countries. It could be through partnership. It could be creating these specialized units in research centers, in university, or also creating these units inside of the government with the different models that can be followed. That is super essential, and it should be also a part of our agenda to create these capacities in the countries. Because it’s not just about the methods, it’s also about how do we see the problems? How do we understand problems and how do we design public policies too in a different way using behavioral science. I think that is something that we should be spreading around the different governments and also to create more of these capacities in the new scholars and in students that are really interested in applying this methodology.

Brooke: You mentioned a lot of different actors there. There are local governments, there are local researchers, there are research partners in other countries. There are international organizations who are often supporting this kind of work. This seems to be a very important difference from what we were talking about a few months ago, when we were talking about using behavioral science in the context of developing governments. The conversation there that we were having was really about aligning the incentives of two different groups: government workers, and policy makers on the one side and local researchers usually on the other side.

Brooke: Now, we’re up to four, we’ve got the local researchers, the international researchers, the governments, and these international organizations that are often funding the work. Now we have a lot of different incentives that we need to align. How do we find the right champions in this kind of context, where it’s even more complicated to align these different incentives. Who are the right people to identify and to bring on board to help to move these projects forward in a development context?

Margarita: The right person to do this? The right institution. I think that is more about opportunity. I think that if the government is open to do this kind of let me… so I think that it’s like two ways. One way is if the government is willing to do this and it can be with lockout or with researchers in the country. We should be happy that it’s happening. It could be maybe the government willing to do that but with an international entity, I think that it should be also, okay. I will say more from the side of the international entity, I think that it will be smart to invite some locals to work with them. And also sometimes for the locals or for the people that are in the country, it is smart to buy some international researchers, because also it gives a little more pressure to the government when you have international presence to continue this unless you like to publish and to engage with this.

Margarita: I would say that we don’t need to have all of them in a project. I think that it goes in two ways. One, is the willing of the government to be open and to try this research. And on the other side, the willingness of researchers, international researchers, or in-country researchers to do this kind of research. I was still thinking that we have two main incentives: the researchers, the researchers I’ll say in the country or outside the country, and then on the other side, we have the government. I would say that we have more stakeholders and more incentives to align.

Brooke: So, It’s really about finding opportunities.

Margarita: You see, that we need to align all of them?

Brooke: I don’t know that we need to align all of them and certainly not from the beginning. Of course, if we think that we need to build this entire massive edifice before we can start doing anything, then we’re unlikely to get very much off the ground at all. In the same way, the individual projects need to start small and start with things that are low risk and can demonstrate a little bit of return on investment early on. We should think of the same type of approach when we think about building relationships, that maybe our ideal is that in the longer term we have all of these players at the table and working together smoothly and harmoniously for the reasons that we discussed earlier, that allows us to create knowledge jointly.

Brooke: It allows us to develop capacity in an equitable manner. It allows local governments in developing contexts and local societies and developing contexts to have what is truly an appropriate amount of agency in determining how their own public policy problems are scoped and framed. And how the experiments are designed in order to address those problems to develop solutions that ultimately are solutions that they have full ownership of. We don’t want a bunch of Europeans and North Americans going and stepping off the colonial ships and saying, “Here’s the perfect solution to your problem.” We don’t need to have the idea that all of this needs to be set from day one.

Brooke: As you mentioned, you start with the opportunities that present themselves and you engage the individuals who are willing on the problems that seem to lend themselves well to this approach. And from there, we can build. My takeaway from what you’ve said just now, is that we shouldn’t think about starting too big. We should start small with the opportunities that are available to us and with the champions and the leaders who are there ready.

Margarita: Mainly, I think that when we are talking about countries that they haven’t tried, they haven’t done a lot of experiments. They haven’t applied behavioral science to say that we’re building a relationship, we’re creating trust. I think that in that case, both sides, the government and researchers, want to have the best results possible. And when I say the better results is like in the process of doing the research, but also in the final results. And I think that starting small, having something that is low risk, that they also can get some insights about how we do this kind of research and what they can learn in the process. And how can they also capitalize the knowledge that we can create, for me, that is the key.

Margarita: Finding these champions, it opens doors, it opens a lot of doors to do after having low risk projects, to do projects that are more ambitious and projects that have maybe also bigger and deeper questions.

Brooke: As we scale then, it seems like we’re in agreement on this point that we need to start small with the opportunities that present themselves and the champions who stepped forward. And so perhaps the lesson that we can extract from this is that in the developing context, there are additional worries about ownership and agency and capacity that we need to have in mind. Those are not necessarily problems that we’re going to solve from the beginning, but as we scale, we need to keep in mind that those problems can start to arise. And so, we need to have a clear idea in our mind of how that scaling path in a development context might need to be different than the kind of scaling path that we see in the context of a more developed country where the worries about agency and ownership are maybe less acute.

Margarita: Other thing that I was just thinking that it will be, it is valuable, in some cases, when we are talking about developing countries, is that, we can add value, not just with the final result, but also we can add value about helping them to create these databases that is going to be useful for the decision-making process. That was one of the main learnings that we had with this project, with the experiment that we ran with public servants in Mexico. It wasn’t just about the final result. It was also about giving them little takeaways in the process. When they saw that they could have a database that makes their work easier and also that gives them key data for the work, they were also really excited about that.

Margarita: And I think that that is the way that we also for researchers, it is good to think. It’s not just about the final results, it’s like, what can we give them? What takeaways can they have in the process of the research? That was really gratifying for them and also it took a lot of pressure out of us because we’re not just looking at the end of the experiments. We were having some little takeaways and wins in the process.

Brooke: I think it’s very important that those takeaways, those wins be really framed around the needs and priorities of the local actors that you’re working with. One of the concerns that I have is that international organizations, in many ways, they’re under a lot of pressure to demonstrate impact. And one of the offshoots of that, is that they spend a lot of time and resources collecting an incredible amount of data. But the data that they’re collecting is often oriented towards their own organizational and institutional needs. They collect the data that allows them to demonstrate impact to the people to whom they are accountable. But that’s a very different question, a very different problem than the kinds of problems and questions that would drive the way that data collection is scoped by local actors, for instance.

Brooke: We won’t be very cynical about it. An international foundation, that’s looking to make a case to its donors, that the money that they’re donating is worthwhile and that they should continue giving is a very different kind of storyline supported by very different kind of data than local government who needs to make decisions about how it is that they are going to reorient a certain policy. As you mentioned in the Mexican context for instance, a policy around transparency and gift giving.

Margarita: I think that that’s a big change that international organizations, also some researchers, academics, need to do, if we want to do applied research. It is not about us, it’s about them. I usually tell my team, I tell them that, it is about them. It’s like, “What can we do in the process?” That they can keep for them, that they can learn, that also they can build some capacity. And that is a change of mindset because usually we are thinking about, as you just said, we are thinking about, “Okay, how can we give the information or the results that the donor is expecting? Or how can we answer our research question?” And this is first and we already said that, it’s about understanding what are their needs, what are the main priorities, where do they think that we can add value?

Margarita: Once we understand it, how can we connect it with our own interests, our research questions, and maybe about them? How can we generate knowledge and give them some wins in the process? And I think that that is a total change of mindset.

Brooke: Thank you very much for that. I think that’s an excellent note to end on, remembering that when, as researchers, we’re partnering with governments or other organizations looking to have societal impact. It’s really about improving society at the end of the day. We use research to do it, but primarily it’s not about the research.

Margarita: In our case, I think that improving the wellbeing of the society, making better governments, also helping to have better policy makers, better public servants yeah, to improve the wellbeing of the society. I think that this point is really important. How can we make these governments better? How can we increase the well-being of the society? And that means sometimes that we need to give up something if we really want to be relevant and to connect with the realities and the challenges that our societies and our governments are facing around the world.

Brooke: Totally agree. On behalf of TDL and all of our listeners, Margarita, thank you very much for sharing your insights with us today. Before we sign off, is there one last thing that you’d like to share a summary or insight or just something that we didn’t get to touch on today?

Margarita: Maybe I just will say that if some efforts of the people in Government Lab has been done around different universities around the world, there is more willingness and more interest to try to do this kind of work together with the government. I think that it is a great time to do applied research and to make a lot of our basic things and our research relevant to the world.

Brooke: Great. Thank you very much. And for you listeners out there, thank you as well for joining us today. If you’d like to learn more about applied behavioral insights, you can find plenty of materials on our website thedecisionlab.com. There you’ll also be able to find our newsletter, which features the latest and greatest development field, including this podcast, as well as great public content for biases, interventions, and our project work. Thanks very much.

We want to hear from you! If you are enjoying these podcasts, please let us know. Email our editor with your comments, suggestions, recommendations, and thoughts about the discussion.