Co-operation or freeloading: What is the effect of conditional versus unconditional incentives in an SMS survey?

Written by: Alexandra Cronberg

Introduction

Gifts can be a tricky business. While they may stem from pure generosity and care, they often come with sticky strings. Just ask all the companies that tightly regulate the receipt of gifts from, say, potential clients or partners. Such are human relationships that obligation and reciprocity often govern behaviour and interactions, for better or worse.

In survey research we may draw on the same deep-seated human traits of obligation and reciprocity to get respondents to complete our questionnaires. We can do this by giving an unconditional gift, i.e. incentive, in advance of asking for participation. Indeed, several studies[1] on postal surveys have shown that unconditional incentives do lead to higher response rates compared to giving a gift conditional upon completing the survey, which arguably treats the questionnaire more like a transactional exchange.

The use and administration of incentives is a particularly relevant issue for surveys making use of self-completion questionnaires, such as postal and SMS surveys: These data collection modes do not have the benefit of an interviewer who can coax respondents to take part and therefore need to rely on incentives to a greater extent.

Now, the same studies showing that unconditional incentives in postal surveys lead to higher response have also shown that unconditional incentives are actually not cost efficient. This can be due to undelivered letters or the absence of eligible respondents. Some respondents will also take the incentives, e.g. a voucher attached to an advance letter, without completing the questionnaire. Consequently, few postal surveys in practice administer incentives unconditionally.

With the increasing popularity of SMS surveys, it is pertinent to ask whether unconditional incentives have the same effect on SMS as on postal surveys, and whether it is cost efficient or not. In particular, SMS has the advantage over postal surveys that respondents can easily opt in, meaning cost efficiency may well be improved.

In order to seek the answer to these questions, Kantar Public carried out an experimental study together with the British Council. Read on to find out the results.

This study

The study involved an SMS survey with an experimental design to test the effect of administering conditional versus unconditional incentives. The study also sought to test the feasibility more broadly of using SMS as data collection mode to gather feedback and progress updates from British Council course participants, but that question is the topic for another blog post.

The survey was carried out among course participants in a British Council teacher training course in Ethiopia and the questionnaire comprised 16 questions. The sample consisted of 434 respondents with valid telephone numbers. Respondents were randomly allocated into one of two groups, Group A and Group B. The initial message was successfully delivered to 390 respondents (Group A: 199 resp. and Group B: 191 resp.). Each group was administered the survey as shown in the diagram below.

Group A & B

At the beginning of fieldwork, respondents were sent a message alerting them to the survey. A day later they were then sent another message asking them participate. In order to participate, respondents were instructed to first opt in by responding to the message. For Group A, the questions were then sent out followed by the incentive, provided the respondent completed all 16 questions. For Group B, the incentive was sent immediately after the respondent opted in, which was followed by the questions. The incentive consisted of airtime worth 15 Ethiopian Birr, equivalent to 0.55 US dollars.

Findings

The findings from the study suggest that offering the incentive in advance yields a slightly higher response rate compared to an incentive conditional on the respondent completing all the questions. As shown in the table below, among Group B, 25% completed all the questions whereas in Group A the equivalent figure was 21%.

These figures are broadly in line with surveys of this nature. That said, it is clear that response is still fairly low even among Group B.

How does this impact on cost efficiency? As mentioned above, one advantage of SMS surveys over postal ones is that respondents can easily opt in before any other message or incentive is sent to them. This means that unconditional incentives are only sent to respondents who have a valid telephone number and who are eligible, thus minimising loss. There is, however, still the potential issue of respondents taking the incentive without completing the questionnaire. This problem turned out to be quite a notable one in our SMS survey. Among respondents who opted in, nearly half of Group B (48%) did not complete the questionnaire. That means a large share of respondents took the incentive but ditched the questionnaire. The equivalent proportion who opted in but failed to answer all questions was somewhat higher for Group A (56%). Yet the resulting cost for the airtime incentives overall (and per completed interview) was lower for Group A since we did not allow for any freeloaders.

Putting monetary values to the incentives given to Group A and B, we can see that the total cost for Group B was ETB 15*93=ETB 1,395 (USD 59.30), equivalent to an average of ETB 29 per completed interview. This compares to a total cost of ETB 15 per completed interview among Group A, resulting in a total cost of ETB 15*42=ETB 630 (USD 26.77). Consequently, we might draw the conclusion that cost efficiency is a major concern also for SMS surveys when administering unconditional incentives.

Table

Conclusion

Based on the results from this experimental SMS study among teachers in Ethiopia, we can see that unconditional incentives yielded higher response compared to administering incentives conditional upon completion of the questionnaire. This finding is line with other studies, and re-affirms the view that drawing on respondents’ sense of obligation and reciprocity is more productive than treating survey participation as something of a transactional exchange.

That said, it is clear that a large share of respondents are not that bothered about reciprocity in the face of a free gift, even when first asked for their active participation. In this light, administering unconditional incentives in an SMS survey is arguably not cost efficient, with the average cost of unconditional incentives per completed interview nearly double that of the conditional alternative.

Hence, the sense of obligation and reciprocity may well be part of deep-seated human traits and behaviour, but it seems that in a context of technology and faceless interactions, many respondents will turn into freeloaders. Unfortunately for us social researchers, free airtime does not seem to come with sticky strings.

 

[1] See for example Simmons, E. and Wilmot, A. ‘Incentive payments on social surveys: a literature review’, published by the Office for National Statistics in the UK, 2004. See also Abdulaziz K, Brehaut J, Taljaard M, et al. ‘National survey of physicians to determine the effect of unconditional incentives on response rates of physician postal surveys’. BMJ Open 2015;5: 007166.doi:10.1136/bmjopen-2014-007166

A bit like finding a husband? Success factors for implementing segmentation analysis in social research studies

Written by: Alexandra Cronberg

Introduction

Segmentation analysis is gaining popularity in social research. While it has long been used in market research, this analytical approach can also add value in social research contexts. Specifically, it can help providing an understanding of different needs and motivations among sub-groups in a target population. Consequently it can help donors and agencies tailoring their programmes and interventions and thus increasing the likelihood of success.

There is much to be said about adopting segmentation as an analytical tool into social research. Yet it is also important to recognise the differences between social and market segmentations. This helps to both apply the tool appropriately and to set the right expectations early on. In this light, the blog post here will talk about the main differences between market and social segmentations, and what to bear in mind to ensure segmentation studies are successful in social research.

Now, you might wonder where that husband comes into the picture? Well, bear with me for a moment, but you can think of each segmentation solution as a potential partner. It will all become clear.

Hands holding seeds 2

Examples of segmentation studies

At Kantar Public we have conducted a number of segmentation studies over the last couple of years, including the following projects:

  • Segmentation of the adult population in India, which is one of the countries where open defecation is a major concern. The segmentation explored and helped gain an understanding of people’s toilet acquisition behaviour, drivers, and barriers. The segments identified were Regressives, Conservatives, Prospectives, and Progressives.
  • Farmer segmentation in Tanzania and Mali to understand which African farmers are open to new behaviours. The segments identified were Contented dependents, Competent optimists, Independents, Frustrated escapists, Traditionalists, Trapped.
  • Segmentation of young women and girls at risk of HIV in Kenya and South Africa. This study aimed at understanding risk factors that increase young women’s vulnerability to HIV infection based on behavioural, attitudinal, and demographic variables. The analysis led to segments such as teenage girls just starting to explore sex and relationships; young women in traditional marriages; girls with boyfriends who always use condoms (except when they don’t); and girls with steady boyfriends and sugar daddies on the side.

These projects give a flavour of what social segmentation solutions may look like. The studies have helped our clients to better target their interventions based on the specific needs and drivers of each segment, hence illustrating the value of applying segmentation analysis in a development context.

What is meant by ‘segmentation’ and what should it look like?

Before we move on to the differences and success factors, let’s agree on what is meant by ‘segmentation’. The word segmentation is sometimes used to simply denote splitting a population into sub-categories and presenting analysis by variables such as gender or age group. While this is indeed one type of segmentation, ‘segmentation analysis’ generally refers to sophisticated statistical techniques to segment people based on carefully designed questions and topic areas, and on patterns in the data that are unknown prior to the analysis. Segmentation can be based on a wide range of factors such as socio-demographics, beliefs, attitudes, behaviour, needs, and individual emotional traits. It is this type of segmentation we are concerned with here.

The aim of segmentation analysis is to have segments that are as distinct as possible from each other, while the people within each segment should be as similar as possible. The segments should also be easily identifiable in the population from a practical point of view. Furthermore, a successful segmentation should offer insights, some ‘ah ha!’ experience, and be intuitive enough to strike a chord with the client and stakeholders. If not, the segments are unlikely to gain traction.[1]

How do market and social segmentations differ?

Moving on to the differences between market and social segmentation studies, there are two main differences which I will talk about here.

Firstly, the outcome variables – that is, the factors on which the segmentation is based on – may be less clearly defined in social segmentations than in market ones. While market segmentations generally focus on segmenting the target population on the basis of a single outcome variable and a single behaviour – purchase of a product – social segmentation studies tend to be more complex than that. Social ones often (a) look at multifaceted and socially sensitive behaviours and (b) often try to explain multiple behaviours which each is affected by a different set of drivers and barriers.

As mentioned above, one of the benefits of using segmentation analysis in development is that programmes and interventions can be tailored according to the specific needs and behaviours of the target population. The key outcome variables for a programme may indeed be dependent on the findings from the segmentation analysis. This means that outcome variables may not actually be known or clearly defined at the beginning of a project.

In the context of young women at risk of HIV, there is a multitude of behaviours that lead to increased vulnerability. Risky behaviour may stem from lack of willingness to go out of one’s way to get a condom, lack of confidence to insist on condom use, or the keeping of multiple and/or concurrent boyfriends, to mention but a few. These behaviours, in turn, may be related to opportunities and socio-economic factors. There may also be physical barriers, such as inaccessibility to places providing free condoms, or lack of money to buy them. These factors can all feed into the segments, which subsequently reflect a variety of risk factors and population profiles. The intervention could focus on any one, or more, of these risk factors and drivers.

With complex segmentation studies such as the one of young women at risk of HIV, the analysis is often an iterative exercise where solutions are scrutinised and re-scrutinised as part of the process. In fact, you could say it is a bit like finding a partner or spouse with whom you want to settle down: you might need to meet a few potential partners before you even fully realise what it is you are seeking. Now, some researchers estimated the ideal number of partners to date before settling down is as high as 12![2]

Turning the attention back to segmentation, the multitude of outcome variables and the often complex associations between behaviours, attitudes, and needs further mean that segments produced in social segmentations are unlikely to be as neat as standard market segments.

As for your potential long-term partner, no segmentation solution is perfect. It is thus a matter of deciding what the most important traits are, and focusing on those. Although we may dream of extremely well-differentiated segments, each consisting of highly homogenous groups, we are unlikely to observe such a pattern for the full range of relevant variables. For example, among our young women, social norms and touch points turned out to be less differentiating than behaviour to protect oneself against HIV and also experience of abuse.

On this note, it is worth highlighting the importance of including a sufficient number of behavioural variables in the segmentation. While behavioural variables may not necessarily be more differentiating than attitudinal ones, they tend to have more practical value for identifying the target groups in the population at large. It is therefore important to ensure a sufficient range of relevant behavioural variables are covered.

Success factors

Having talked about segmentation analysis in broad terms, and the main differences between market and social segmentations, we can summarise the learnings for successful social segmentations as follows:

  1. Define as clearly as possible the element(s) (behaviours, attitudes etc.) on which you want the segments to vary, while acknowledging the complexities in social segmentations. Identifying the right segmentation variables is critical for successful segmentations. However, lack of a single outcome variables, and multifaceted relationships between behavioural, attitudinal and demographic variables mean segmentation analysis may involve an iterative process of finding the most suitable solution. It also means that segments may not be as clearly defined as standard market segments.
  2. Make sure the segments are easily identifiable in the population and, if necessary, tilt the balance towards behavioural factors. As for any segmentation, whether in market or social research, it is important that segments are identifiable in the population at large. How will the target groups be reached in practice? Behavioural variables tend to be more useful for this purpose, but this is dependent on the nature of the intervention.
  3. Allow time and resources to find the optimal segmentation solution. Two or three iterations are unlikely to be enough, so it is important to allow sufficient time for analysis. Finding the right segmentation solution is indeed a bit like finding a spouse. None is perfect, and it is only after meeting a few potential partners that one better knows what to settle for.
  4. Align expectations early on since the resulting segments are unlikely to be as neat as standard market segments. In light of the points above, it is important to acknowledge the differences between market and social segmentations, and the expected outputs. Have, and set, the right expectations from the start and segmentation solution will invariably become a smoother exercise.

Social segmentations have immense potential to add value and insight to programme designs, in particular to better understand the needs and drivers across different sub-groups in the target population. Bear in mind the points above, and you will maximise the chances of finding a set of segments that will succeed in making you happy. Perhaps not forever after, but at least until your next programme.

 

[1] I won’t go into the technical details of segmentation here, but it is worth noting that there are several different statistical methods of conducting segmentation analysis. One common analytical approach is Latent Class Analysis (LCA), which for example was used for the HIV related-project. The segmentation analysis is typically used to produces outputs for several different segmentation solutions such as solutions for 3, 4, 5, 6 and 7 segments. When deciding which solution to use, we normally look at the segments based on the segmenting variables and also by cross-tabulating the segments against other variables in the questionnaire. Pen portraits can then be produced of the different segments and to help decide which solution is the most useful ones.

[2] http://www.bbc.co.uk/programmes/p02hl73h