RFP: Research Firm to Conduct Outcomes Evaluation in Nigeria

Position Summary

Background

Women’s World Banking has collaborated with a commercial bank in Nigeria to design and roll out a digital remittance solution for migrant workers and their families. Women’s World Banking will conduct an outcomes research study on a narrow rollout of this solution (referred to throughout this RFP as the “pilot”) to understand the impact of this remittance solution in Nigeria – for both women customers and the partner FSP, and demonstrate that digitized remittances are a viable strategy to drive women’s financial inclusion and economic empowerment. At the customer level, the research will seek to understand behavioral change and demonstrate that women are benefiting from the remittance solution, and if so, how. At the FSP level, the research will seek to demonstrate that the remittance solution leads to longer-term engagement of the customer with the FSP. The framework used by Women’s World Banking for the outcome research explores changes to women’s economic empowerment through financial inclusion across four dimensions: material, cognitive, perceptional and relational. Focusing on the women’s economic empowerment outcomes of financial inclusion is important to ensure that financial inclusion leads to meaningful and lasting change in migrant women’s access to income and assets, control and benefit from economic gains, and power to make decisions. Furthermore, the study will allow Women’s World Banking to assess the effectiveness of the solution holistically, beyond transaction data, and will provide valuable insights into users’ behaviors. Understanding the barriers to women’s usage of digital financial platforms and how the reduction of these barriers may lead to improved financial outcomes, including reduced friction in the sending of remittances, will enable the FSP and other digital financial services providers to better design and target their platforms to match the needs of these users. Women’s World Banking is seeking proposals from qualified, locally-based research firms to develop and implement the outcomes evaluation.

Women’s Economic Empowerment Framework

Objectives

Women’s World Banking is seeking a research firm to develop and implement an outcomes evaluation. Specifically, the evaluation will assess the medium- to long-term effects of the remittance offered by the bank and on the financial health and economic empowerment of women and their families. Objectives are to:

• Demonstrate that the solution leads to increased usage of remittance services in the longer-term
• Understand customer lifetime value (products purchased, interactions with bank)
• Understand behavioral change at the individual level, and the effect on customer behavior
• Demonstrate that women are benefitting from the services, specifically around financial health and economic empowerment.

Methodology

The outcomes research will be conducted around the time of the pilot deployment of the remittance incubation project in Nigeria, over a two-year period. The districts/zones for the research will be determined and target the pilot (treatment) and non-pilot district (control). The research activities are intended to comprise of the following in both treatment and control districts/zones:

1. Three quantitative surveys with a sample from the FSPs

• Baseline: Before the roll out of the pilot (June 2023)
• Midline: One year after the baseline survey (June 2024)
• End line: One year after the midline survey (June 2025)

2. Analysis of several rounds of administrative customer data received from the FSPs

• Bank administrative data analysis round I: Before the roll out of the pilot (April 2023)
• Bank administrative data analysis round II: Immediately after the roll out of the pilot (August 2023)
• Bank administrative data analysis round III: Approximately one year after the roll out of the pilot (August 2024)
• Bank administrative data analysis round IV: Approximately two years after the roll out of the pilot (August 2025)

The team will design and implement an experimental impact evaluation with a strong preference for the use of a randomized controlled trial (RCT) design to rigorously investigate the medium- to long-term effects of the remittances solution on the empowerment metrics laid out in the figure above. Randomized assignment to treatment and control groups is a rigorous method for estimating the unbiased causal impacts of an intervention by comparing changes measured in treatment participants against those in the control group. Through random assignment, concerns over endogeneity and confounding variables related to sample selection are mitigated. The research team will use a replicable random assignment algorithm for assigning participants into the treatment and control groups, and will stratify on variables whose relevance is dependent on the final proof of concept design and other sample characteristics, which may include gender, location, and remittance-linked country, among other possibilities.

The location for the study is dependent on the areas where the FSP implements their pilot solution. The research team will consider internal and external validity, feasibility, and cost when determining the final geographic boundaries for the study.

The sample of the study will include approximately 3000 FSP customers across treatment (~1500 customers) and control (~1500 customers) groups in the geographies of Nigeria where the upcoming pilot solution will be active. The final sample size will be determined via power calculations, enabled by the first round of data sharing by the partner FSP prior to the roll out of the pilot as described below. The final sample design may also vary from this description, depending in part on the final design of the pilot solution and the geography of the roll out area.

Responsibilities

The responsibilities of the firm include (but are not limited to):

Research planning:

• Collaborating with Women’s World Banking on the research design
• Obtaining IRB approval and local government approvals, as appropriate, including:
  • Organize and submit all documentation to obtain ethical research clearance by local Institutional Review Board.
  • Obtain all permits and local government approvals, as appropriate
  • Create and share informed consent form with Women’s World Banking for approval
  • Share full IRB submission (in English) with Women’s World Banking

 Quantitative surveys:

• Implementing and overseeing quantitative data collection activities, including:
  • Provide input on the questionnaire developed by Women’s World Banking
  • Translate questionnaire into local language
  • Program the quantitative interview guides into software (e.g., SurveyCTO) to administer computer-assisted personal interviews
  • Pilot and test the survey
    • May include expanded Women’s Economic Empowerment module testing with a more intensive question development and administration period with focus group discussions for validation.
  • Record any modifications made to the survey
  • Share pilot results with Women’s World Banking
  • Update survey based on findings from the pilot prior to going to the field
  • Develop relevant documentation for field work, including manuals, interviewer guidelines and other research tools;
• Hiring, training and supervising quantitative research team, including:
  • Select, hire and train a team of interviewers to collect the data (meeting gender, age and language requirements);
    • Ensure deep understanding of the objectives of the research and meaning of the research questions.
    • Women’s World Banking research team may participate in some aspect of the training (e.g., discuss training plan with field coordinator)
  • Supervise enumerators and conduct regular quality checks, providing written reports to Women’s World Banking
  • Provide research plan that describes the training, research team composition and structure, number of interviews to be conducted per day, strategy to ensure data quality
  • Participate in regular progress updates with Women’s World Banking team (frequency and channel of communication to be determined)
• Recruiting all research respondents, including:
  • Engage participants for surveys (firm will partner with bank to contact clients)
  • Assign a unique ID (not a national ID) for all respondents
  • Set appropriate survey participant compensation/incentives
• Sharing data, including:
  • Provide de-identified raw data to Women’s World Banking to review on a regular basis
  • Clean and code collected data
  • Provide final survey tool in English and local language, complete and clean dataset with coding scheme, and any data files associated with the survey.
• Developing implementation report describing fully the methodology and sampling strategy used, response rates, respondent compensation, and any challenges (and changes) encountered during each round of fieldwork.
  • Reports should include descriptive statistics, summary of key findings, and data visualization.

Proposal

The proposal should be no more than 10 pages, and include the elements below:

1. Organizational Capacity and Experience

• Please provide a brief statement of your organization’s capacity to complete this work, listing similar projects completed/relevant organizational experience.

2. Team Structure

• Describe the proposed team required, including team leaders, any statisticians, supervisors and enumerators for fieldwork and how they will be organized, data management staff, and any other others who will be involved in the work. Attach CVs of senior staff who will be leading the work.

3. Organization of Work

• How would you organize fieldwork?
• Describe how to avoid or solve any recruitment-related challenges.
• Describe your system for quality control of interviews and data in the field.
• Describe your system for data entry & quality control of data.

4. Availability

• Is your organization available for the described activities starting in June 2023? If no, list any times where availability is limited.

5. Illustrative budget, including

• Budget options for work in 1, 2, 3 or 4 states
• Budget options for interviewing existing clients only versus recruiting women to be used in comparison group.

Duration of Consultancy

The Consultancy period will be for 3 years starting from as early as April 2023 and ending in December, 2025.

Location of Work

Nigeria

Preferred candidates

Ideal research organizations with skills and competencies in the following areas:

• Experience conducting individual-level quantitative survey with expertise in outcomes evaluation, including ability to conduct and complete large sample phone surveys within periods. Ability to train enumerators and test pilot surveys
• Ability to conduct analysis of data and development of report on results with recommendations
• Ability to conduct quantitative and qualitative interviews in local and regional languages of Nigeria
• Experience conducting research with low-income participants and participants with limited to no literacy, particularly with a focus on women.

Deliverables

Deliverables will include a fully developed research design, sampling strategies, work plan, clean datasets, analysis files, and reports.