The story behind the numbers: Putting the end-user first in impact measurement


Tatiana van Lier, Head of Communications and Stakeholder Management at the DOEN Foundation/DOEN Participaties, wrote this blog for Alliance Magazine:

As an impact first investor, the actual impact of your portfolio and investees is what matters most. Is this company achieving what it aims to do? What does interaction with this company actually mean for its customers? Or, should they adjust their internal strategies in order to be more effective? And if so, how do they determine what direction to take?

Number obsession and case studies
It’s a common dilemma in impact measurement: you can focus on the numbers, such as number of jobs created, number of people with access to clean energy or tons of CO2 emissions saved. But these numbers don’t tell you the real stories of how the lives of people are actually affected. You could consider to do a case study. This will provide you with more insights, but are such stories applicable to the sector as a whole?

Are there any alternatives? Some investors turn to randomized controlled trials (RTC’s), or they hire a consultant. These methods however are not only very expensive, but they are also extremely time-consuming. It can easily take three months to a year to get the data. What if we could really understand the stories behind the numbers, without being too time-consuming or expensive? What if we could easily gain insights in how the company is really affecting lives?

During the EVPA workshop ‘The stories behind the numbers: putting the end-user first in impact measurement’, Michelle de Rijk (impact investment manager at DOEN Participaties), Sasha Dichter (co-founder 60 decibels), Tom Adams (co-founder 60 decibels) and Mansoor Hamayun (founder and CEO of Bboxx) discussed how the Lean Data method can strengthen the sector as a whole, by providing social as well as business insights and enabling benchmarks across other companies in the same sector.

Lean, affordable and scalable
Lean Data is a mobile/techbased impact measurement method, where they speak directly to customers, based on a company’s needs. So how does it work? After the introduction of Lean Data to a company (mostly done by the investor) the kickoff starts. During this first period, the Lean Data team gets to know the company, its clients and its priorities. After the kickoff, the design phase starts. Which tools (phone calls, SMS) are most suitable, which questions need to be asked? Local teams are responsible for data collection, speaking the local language. Lean Data performs the analysis and the result is a straightforward report with lots of customer data, recommended actions and benchmarks.

The team that founded Lean Data at Acumen, has now created a new company ‘60 Decibels’, and has run over 400 projects in 34 countries. More than 100 local people are trained to collect data. Each project is adding to the expertise about impact measurement, as well as the ability to benchmark.

The Lean Data methodology provides insights on and for three dimensions. The end user (1): how does he or she really feel about the product or service of the company. This is valuable information for the company (2) as well, they can adjust and improve their business operation. Finally, the investor (3) gets a real data on the impact of their portfolio company.

Mansoor Hamayun of Bboxx explained that the poorest of the planet spend the most on energy. Energy access is a starting point for many additional series and products. By fighting energy poverty, you are fighting poverty as a whole. His company offers off grid sustainable energy solutions to countries in Africa and the developing world. Thanks to the Lean data methodology Bboxx gained important insights, example given that the use of kerosene by customers of Bboxx dropped from nearly a quarter to one per cent of all customers. Also, the survey found that 14 per cent more children did their homework after dark, and the average time children spend doing their homework daily went up to 12 minutes, roughly amounting to an extra hour of study time a week.

The many questions by the approximately 40 participants of the EVPA workshop clearly shows this is a topic that many people are interested in and struggling with. It was really interesting to hear the different views and witness the very active involvement of all participants. DOEN Foundation has used the Lean Data methodology for several of her own portfolio companies. The results were of great value to DOEN, and in 2019 DOEN Participaties (DOEN Ventures) invested USD 500,000 in 60 Decibels as part of a larger investment round. With this investment DOEN aims to help the company scale so more investors and companies can benefit from listening to the voice of the end-user. Because in the end, that is what counts!