The Global Accelerator Learning Initiative (GALI) report, “What’s Working in Startup Acceleration”, released March 28, 2016, represents the first major report from a collaboration between Social Enterprise @ Goizueta at Emory University, the Aspen Network of Development Entrepreneurs (ANDE) and Village Capital. It is a rare example of an effort to use data to determine the effect of impact accelerator programs, a category of enterprise support that has proliferated in the past ten years growing to over 250 organizations worldwide. The 2015 data collected as a result of this collaboration has been released and provides aggregated data on 4,125 early-stage ventures.
While the number and enrollment of impact focused accelerator programs has grown, their impact on attendees has not be quantitatively demonstrated. In addition, program managers do not have empirical studies that illustrate the relative value of different program components and must instead rely on anecdotal evidence that a particular component is making a positive contribution to their cohorts.
The work of the GALI report represents an exciting new age for impact accelerators. This report falls short of its promise in two critical areas. First, by failing to measure the social impact of accelerated companies and second, by ignoring selection bias. Despite shortcomings, the efforts of GALI have the potential to shed light on and ultimately improve accelerator programs across the social impact space.
Why is social impact different?
A common first question is why do social impact accelerators have to be distinguished from other business accelerators? Business accelerators are often judged by the amount of investment that graduates receive and by high profile exits, either on the public or private market. Even in the non-impact business accelerator space, there have been challenges to determining the impact of accelerator programs. Seed-DB has tracked key metrics for business accelerators including number of companies, level of funding, and value of the exit. Ultimately, accelerators are often most commonly judged by the presence of “name-brand” companies among their alumni.
For social impact ventures, they have the additional difficulty of not only showing improving business metrics but also improving social impact metrics.
Interestingly, this GALI report follows a very similar tack as the traditional accelerator measurements in evaluating financial metrics and does not include any measure of the social impact of the accelerated companies. Instead it focuses on three internal business metrics: number of staff employed, amount of revenue, and amount of investment received.
Should social impact companies also be accessed on the basis of their impact? For example, number of low-income customers served? or percentage decrease in hunger for school children?
One issue inherent in social impact evaluation is that an edu-tech, agricultural, clean water, and solar startup will all have different metrics. One solution is the take advantage of the standardization of IRIS and look at percentage change over the year before.
“What’s Working in Startup Acceleration”
“What’s Working in Startup Acceleration” uses the applicant pools for 15 Village Capital programs with data from a variety of other programs including Points of Light Civic Accelerator to access the impact of each of these programs on participating entrepreneurs by comparing the number of non-founder employees, revenue including earned, borrowed, equity, and philanthropic for the ventures that participated in the program versus the ventures that applied but did not attend.
In this report, GALI sets out to reach two objectives:
- Determine the ‘early’, one-year, impacts of accelerator on three key performance indicators.
- Determine what characteristics distinguish between “high-performance” and “low-performance” programs
By looking at revenue growth, growth in the number of employees, and investment growth, the GALI report sought to show the differences between accelerated and non-accelerated ventures. By looking across 15 of Village Capital’s accelerator programs, it shows a significant impact in investment but not in employee growth or revenue growth (Table 1 below)
Interestingly, when they go to analyze program performance for the 28 programs for which they have data, revenue growth improves significantly. While, there is still a significant difference for investment raised by participating entrepreneurs, there is less of an impact on investment raised in general vs. the results in Table 1. (Box 02 below).
Further examination of the Village Capital programs illustrated two clear groups of high and low performance Village Capital programs (Table 2). In the next section, they try and determine the relative contribution of each program component on the success of the program.
Characteristics of a Successful Village Capital Program
The report uses the programmatic difference between high and low performance programs to attempt to determine which program components were most important to enterprise success.
In this analysis they looked at measures of the following:
- Partner Quality
- Division of Time
- Quality of Applicant Pool
- Later-stage companies
- Financial Training
- Mentor Quality
Considering that the strongest effect was seen in investment, it is curious that they omitted an analysis of potential funders (for instance, using size of fund) to which the entrepreneurs had access.
In this section, they find that the high-performing programs first had
- higher quality applicant pools,
- engaged higher quality partners, and
- provided entrepreneurs more time to work independently.
Weaker effects were seen for mentor quality and networking, and no effect was seen for the stage of the venture or a focus on “financial acumen” during the accelerator program.
These are interesting findings and should inform future research, however, it is difficult to use these results to extrapolate out to the overall importance of each of these components. As the authors state at the beginning of the report: “The main challenge faced when studying the effectiveness of accelerator programs is that different programs seek to accelerate different things.” This means that different programs will also have different admissions criteria. Where one might emphasize a strong financial plan, another might emphasize founder experience or market fit. This makes it difficult to generalize these findings beyond Village Capital.
Inherent in this type of research is the issue of selection bias. Village Capital does not select its cohort randomly. In fact, VilCap, and all accelerator programs, invests time and effort into creating the highest quality candidate pool and selecting the highest quality applicants from each pool. A better comparison, albeit one made challenging if not impossible by small sample sizes, is to compare ventures that were selected and did not attend with those that were selected and did attend. There is a well known study on the outcome of attending elite universities that illustrates that students who were admitted to, but did not attend, schools such as Harvard, do as well as their peers who did attend. GALI has attempted this in tracking those who were accepted but chose not to attend, but unfortunately due to small sample sizes was unable to treat this as a own stand alone comparative group.
How can elite accelerator programs illustrate that they are not only selecting and holding the highest quality talent, but also genuinely accelerating their growth? Hopefully the next GALI report will get closer to this answer.