Focus group discussion or individual interview? The reality of quantitative interviewing in developing countries

Do you ever find yourself trying to hold a conversation with someone in a noisy, busy environment? Perhaps it’s even in your house. Perhaps there are kids running around, teenagers watching TV, and your relatives have come to stay. It can get crowded. Then there’s a knock on the door. Indeed, someone else – an interviewer – has come to ask for a little of your time. You happily oblige, but there may not be a quiet corner for the interview, and others will inevitably over-hear what you are saying.

This is the reality for many of our respondents, and a common challenge faced by our enumerators. Our populations tend to have large families. Space is often scarce, with one-room houses being commonplace in urban areas. Households in rural areas might have more space, inside or outside, though this space seems to quickly fill up with curious onlookers.

The interview environment is thus not always ideal. This raises the questions: What proportion of interviews is indeed affected by noise and bystanders, and what is the impact of less than ideal interview settings? Does it matter? To what extent does it affect the quality of the data we collect? If so, what are the key concerns?

Our colleague at RTI, Charles Q. Lau, in collaboration with Melissa Baker, CEO of Kantar Public Africa & Middle East, conducted an analysis to answer these questions together with a few other co-authors. The article was published in the International Journal of Social Research Methodology (2016)[1]. Read on for a summary of the findings.

The Results

The findings are based on 15,309 face-to-face in-home interviews representative of the adult populations of five countries in Africa and Latin America (Ghana, Nigeria, Uganda, Brazil, and Guatemala), conducted in 2014 and 2015. The study answered the questions below.

How common are bystanders and noise in the interview context?

Well, it varies. Interviewers do their best to conduct interviews in a private place, out of hearing of others. However, the household context in these countries means this is often not possible. In terms of bystanders, ‘completely private’ interviews were conducted in only 64% of interviews in Brazil, 59% in Ghana, 54% in Guatemala, 53% in Uganda, and 33% in Nigeria. Bystanders are mostly non-family and extended family members, such as neighbours, domestic staff, but also children. In contrast, it appears most spouses have better things to do than listen in to their husband’s or wife’s survey responses.

Most interviews across all countries take place in a ‘quiet and calm’ setting. Even so, children, televisions, telephones and other distractions affect a few of the interviews: between 19% (Brazil) and 45% (Guatemala) were done in more or less noisy surroundings (either a bit of noise, or very noisy and chaotic).

So the one million dollar question is: Do bystanders affect responses to questions?

The good news is that bystander presence has little effect on responses to non-sensitive questions. The analysis found there is little association between presence of onlookers and response distributions about technology-related questions, ‘don’t know’ responses, and survey satisficing (that is, the tendency to answer questions to minimise effort rather than respond in a truthful manner). So, in terms of non-sensitive topics we (and you!) can rest assured that standard interview settings in these countries do just fine for gathering good quality data.

Bear in mind however that this survey covered the topic of technology, which is by and large a non-sensitive topic. Other studies have shown that bystanders do have an effect on responses to sensitive questions, such as domestic violence and drug and alcohol use. For surveys asking sensitive questions, this study highlights the need to carefully consider the interviewing context, given how common it is that respondents are surrounded by bystanders and noise.

Could bystanders actually help to improve data quality for factual questions?

Well, yes, but only if the bystander is the husband or wife. However, most curious onlookers are neighbours, children, or extended family rather than the spouse. So the overall impact on data quality is negligible. Indeed, only 3-4% of interviews in Ghana, Nigeria, Uganda and Guatemala had the spouse present. In Brazil it was 11%. Having said that, among the few spouses present, some of them do chip in with factual information. This was especially the case in Nigeria, where almost half of spouse-bystanders assisted the respondent.

How does the interview environment affect data quality?

Perhaps unsurprisingly, noise has a negative impact on interviewer-respondent interactions. Noisier and more chaotic surroundings are generally associated with lower levels of respondent cooperation, attention and friendliness. However, in terms of the proportion of interviews in our study that were disrupted by chaos and noise, this figure was low: in Brazil, Ghana and Uganda only 2-5% of interviews were conducted in a very noisy and chaotic environment. The equivalent figures for Nigeria and Guatemala were a bit higher, ranging between 11 and 15%.

Having said that, again the good news is that noise and distractions had little effect on data quality itself. Indeed, interviewers seem to know how to cut through the noise! Key quality measures – level of ‘don’t knows’, satisficing, and response distributions – were not significantly associated with interviewing environment. We can therefore be confident that the data we collect is of high quality, indeed reflecting respondents’ attitudes and behaviour rather than the environment.

On that note, I will end this communication and say thank you for reading. That is, assuming you weren’t already distracted halfway through…

[1] Charles Q. Lau, Melissa Baker, Andrew Fiore, Diana Greene, Min Lieskovsky, Kim Matu & Emilia Peytcheva (2016): Bystanders, noise, and distractions in face-to-face surveys in Africa and Latin America, International Journal of Social Research Methodology.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s