It’s as Easy as A-B-C! How A/B Testing Helped Onboard More Women Customers through Mobile Account Referrals

May 28, 2018

“Refer a friend and get $10 off your next order!”  I would bet that most readers of this blog have received a message like this at some point, encouraging customers to invite friends to use a certain product or service. I would also bet that a much smaller number of readers actually acted on that message.

What prevents more people from sending referrals? Women’s World Banking, in collaboration with behavioral design firm ideas42, set out to answer this question in the context of women’s financial inclusion in Pakistan. Specifically, we wanted to find out how we might get more women to sign up for JazzCash mobile accounts through referrals.

The opportunity: Onboard more women to mobile accounts through referrals

We became interested in this particular design challenge when our analysis of the JazzCash database found that referrals were already a relatively effective channel for onboarding women users. New JazzCash customers sign up for an account in three ways: by themselves directly on their phones, referrals from current JazzCash users, or at a JazzCash agent location.The data showed that a greater percentage of active female than male customers are onboarded through referrals, indicating that something about referrals was already working well for women.

Our team set out to use behavioral science to enable referrals to reach even more new women customers.

The overarching objective of our partnership with Jazz, Pakistan’s leading mobile network operator, is to increase women’s uptake of the JazzCash mobile account. While JazzCash has achieved a commendable 25% growth in its women customer base  in the past year, as of April 2018, still only 18% of all JazzCash accounts were held by women. The 2017 Global Findex underscores the need for solutions to advance women’s financial inclusion in Pakistan, where only 7% of women have an account at a formal financial institution and the gender gap in account ownership has increased in recent years to 28 percentage points.

Our approach: behavioral design & A/B testing

Women’s World Banking partnered with ideas42 to design and test behaviorally-informed text messages to encourage current JazzCash users to send more referrals to women. The first step in this process was to use behavioral mapping techniques to identify the key behavioral barriers limiting the onboarding of new women customers through referrals. These barriers include: the option to invite a friend may not be salient to users; a lack of social proof about sending invitations; no feeling of reciprocity or obligation to send referrals; and forgetting to send the invitation after intending to.

We then developed various text messages to address these barriers, and tested their effectiveness using A/B testing methodology. For those not familiar with this approach, A/B testing involves rapidly testing different variations of an intervention, also called “treatments,” in parallel on various treatment groups of similar profile, in order to compare the effectiveness of Intervention A against that of Intervention B (and C, D, etc.). Our test comprised six different messages sent to six different treatment groups of approximately 35,000 users each, which were compared against a control group that received no messages.

What did we learn?

1. Behaviorally-informed messages are effective in getting more users to send referrals. All six messages had a statistically significant impact on the percent of users sending invitations, compared to the control group, achieving up to a 60% increase. Interestingly, the effect of the messages was greater among women, for all messages except Salience.

2. Women respond more to gender-centric language. Some of our messages used women-centric language, such as specifically mentioning women friends, mothers, sisters, etc., while other messages did not. The test results indicate that when we use women-centric language, women users respond by sending significantly more referrals to women. The behavior of men users does not change significantly whether the message contains gender-neutral or women-centric language.

Why do these results matter?

ext 7 Sending out promotional messages is a low-cost intervention that digital financial service providers can implement relatively easily. For providers that already send promotional messages, tweaking the language of those messages to encourage inclusion of more women comes at zero additional cost. The insight on the power of women-centric language in particular has the ability to help DFS providers such as JazzCash to bridge the gender gap in their customer base by reaching more new women users.

Bringing more women into the formal financial system is not only a market growth opportunity for DFS providers, but also a requirement for successfully meeting the targets Pakistan’s government and State Bank have defined in the country’s 2020 National Financial Inclusion Strategy. The persistent low rate of women’s financial inclusion in Pakistan will hold back progress on these goals if it is not addressed.

We are working with JazzCash to apply insights from this A/B testing project to their business-as-usual promotional messages, in order to scale the results and help accelerate growth of their female customer base. We also have a follow-up test in the works, to build on our initial learnings and understand how combinations of women-centric language, incentive, social norm, and reciprocity messaging might be even more effective at reaching new women customers. Watch this space!

Have you tried using behaviorally informed messages or women-centric language to onboard more women? Let us know! Comment below or tweet at us (@womensworldbnkg and @megflhrty) to tell us about your experience and what you learned.


This project was funded by the Bill and Melinda Gates Foundation.