Off The Record #6: A game to measure honestyAnd the measures which help to decrease lyingBusara CenterBlockedUnblockFollowFollowingMar 28Photo from Unpslash.
comMeasuring honesty is a Behavioral Scientists’ play date: quantifying the difference between reported actions and actual behaviors helps us gauge the validity of our data while also helping us to better understand the behavioral triggers and limits, especially useful in higher-impact instances of dishonesty such as corruption or fraud.
Literature tells us that some people tend to misreport their private information if it is to their material benefit, while others do so even without the pull of material gain.
So, how do we know when someone is lying, and what can be done to reduce it?The experimental designOne method commonly used to measure honesty is the “coin flip”¹ game, which is as simple as it sounds.
We ask respondents to flip a coin 20 times and report their results.
Each time they land on “heads”, respondents receive a small financial incentive (10 KES — $0.
10 USD).
By extension, this means that when respondents land on tails, they are faced with a choice: telling the truth OR receiving the money.
While we cannot infer if any individual decision is deceitful, by looking at the averages across a sample of individuals we can estimate honesty at an aggregate level.
If all respondents report truthfully, the expected proportion of heads is 50% and truthful results would align with a standard bell curve probability such as this one:We will see that that is rarely the case, but the conditions and framing of the task can have surprising impacts on respondents’ willingness to falsify their results.
The research designWe invited 50 residents from the Kibera settlement in Nairobi to our Lab to take part in the coin-flip game.
Number of “heads” reported from our respondentsAs evidenced by the results (left), many more people said they landed all 20 flips on “heads” than what is realistically possible.
Note that the probability of obtaining 20 out of 20 heads flips is more than 1 in 1 million!While this experimental finding is by no means novel (Dan Ariely wrote a whole book on the topic), we decided to use this experiment in the context of the Executive Education course we run with Strathmore University.
The goal was to introduce our students to elements of intervention design aimed at reducing lying (and, by extension, any other dishonest practices that negatively impact others).
Our students were senior-level professionals across various industries ranging from TelComs, to Finance, to Goverment and Public Policy.
Their task was the following:Design an intervention to increase honesty (or decrease the number of heads reported) relative to the control group (the initial 50 respondents mentioned above)Change any one aspect of the game relating to the:ScriptEnvironment of the experimentIntro message, or messages between flips of the coin3.
Do so without changing the measurement (how the respondent enters the result into the computer) or incentivesStudents were split into three groups, each with a faculty partner leading them.
The invention ideasBased on the newly acquired Behavioral theory from the course, each group tested an intervention aimed at reducing lying.
Group 1: Treatment 1 — Monitoring.
Respondents were told “Our Lab Managers are watching you execute this task in the room”, and lab staff were roaming around the lab session and directly observing their actions.
This intervention reflected a common policy response to dishonest actions — to introduce better controls and monitoring systems to catch dishonest acts.
Group 2: Treatment 2 — Framing.
Respondents were told “We anticipate most responses to fall between 8–12 in this exercise.
” This framing creates expectations around what is considered an acceptable answer, and gives respondents a sense of what we might deem as irregular (and presumably dishonest) A policy parallel could be giving quotas for service delivery outputs (i.
e.
fertilizer disbursement, cash transfers etc.
).
Treatment 3 — Environment.
In this treatment, respondents were asked to change seats every five flips of the coin.
Each respondent was paid according to the results from their computer, so this change would cancel most of the agency they have towards their own results.
In this way, there is no personal incentive for individuals to lie as it would benefit someone else who is a stranger.
This mirrors a more complex policy action on rotating procurement seats, but also has an interesting question of cooperation in dishonesty, and how likely are people to contribute to some group pool through lying.
We tested these three interventions on three different groups of 50 respondents from Kibera.
Which intervention do you think was most successful?Share your thoughts with us on Twitter!The resultsComparing to the control (our first study of 50 Kibera residents who received no treatment), our results showed that Treatment 2, (the framing of expected responses between 8 and 12) was the most effective.
Indeed, the other two treatments had no notable variation from the control.
Access our full data hereThe goal of the interventions was to reduce lying which, in practical terms, means moving as many respondents away from the answer of “20 heads” as possible (or moving the mass on the right of the histogram towards the middle).
Treatment 2, which had our enumerator publicly suggest the upper limit of acceptability shows that the modal person selected “12 heads”, presumably as they realised this was the maximum upward limit they could select without being seen to violate the stated norm.
While this was evidently effective at moving the average “lying” downward, it raises the question whether it was the amount or level of lying that was reduced.
Further readingThis interesting study comparing honesty and beliefs about honesty in 15 countries demonstrated that the significant differences in honesty across countries were positively correlated with per capita GDP and that a country’s average honesty correlates with the proportion of its population that is Protestant.
The experiment also elicited participants’ expectations about different countries’ levels of honesty.
Expectations were not correlated with reality and were instead driven by cognitive biases, including self-projection.
With this in mind, for our next “Off the Record”, we shall explore the extent to which different groups are able to forecast experiment results.
A final noteOur goal is to make research accessible in order to start, continue or inform conversations that help us better understand human behavior.
With this in mind, each “Off The Record” post will provide access to the full findings from our studies, freely available here.
As a commitment to Open Science, we’ll continue to keep this anonymized data live at this page for all our on-going research efforts.
This blog post is part of our larger “Off The Record” initiative where we share findings from small-scale research projects, designed to collect initial data and kick-start a conversation.
If you would like to learn more about a specific human behavior or have a research idea you think we could explore for a future “Off The Record”, please reach out to us on Twitter or via email.
Footnote [1]: This game has been used in several previous experiments (e.
g.
Bucciol and Piovesan 2011; Fischbacher and Föllmi-Heusi 2013; Abeler, Becker and Falk 2014; Pascual-Ezama et al.
2015).