We have now reached the stage of the first round when we have to rank
shortlisted projects competing for limited funds. Our shortlist is such that
the cumulative requested grant contribution is significantly higher than the
total grant budget available for this round.
Of course, we can expect adjustments to grant requests once the applicants prepare their detailed
business plans, we explore alternative financing options, and we
have serious negotiations with applicants. Maybe some applicants will even drop
out. But nevertheless, we need a rational basis for ranking projects (that
deliver very different types of benefits) against one another.
Our problem was how to compare (for example) a project that delivers access to
affordable genuine pharmaceuticals against a project that delivers increased
fruit yields per hectare, or against a project that opens up new markets for
goat owners? How to incorporate this diversity of benefit into a comparison of
different grant/total investment ratios? Should we monetise everything, do we
maximise leverage as some of our colleagues would argue?
I don't think that there is an acceptable conversion rate for a child's
recovery from illness to extra kgs of apples, and leverage is irrelevant when
we are interested in value for money development impact. Rather, we wanted to
find a methodology that is slightly more sensible (and sensitive) than these
approaches. The rest of the methodology described below is pretty standard stuff, but this is the tricky bit...
We have been working on the ranking/fund allocation methodology for some time now, but now we
have to apply it. Fortunately the development has now reached a point where
we are pretty much ready to pilot it with our shortlist. If this methodology
works (if the applicants get it and it produces the intuitively right results)
then the idea is to roll it out for the second round.
One thing to note is that this approach only works when it is incorporated with the risk adjusted grant methodology we have already adopted.
Otherwise you don't have a rational basis for the "cost to ABIF" column at the final step and you get into difficulties with weighted incomparable criteria and
leverage/impact trade off.
So here is an overview of the methodology we intend to use to rank projects and
allocate limited grant funds. We would of course be very grateful for any
feedback or suggestions for improvements.
Introduction
When we are evaluating projects we are primarily concerned with four
questions:
- Will the product or service deliver profits to the investor and development
impact to the donor?
- What is the potential outcome of the project, how many of our target
beneficiaries could benefit and how much could they benefit by?
- What are the chances of the applicant achieving the theoretical potential,
do they have the necessary capacity and resources?
- Are we giving the right amount of financial support?
The questions above relate to very different properties of the applicant
and the proposed concept. It is not possible, without creating unacceptable
trade-offs or forcing incompatible criteria into the same selection step to
look at all of these questions together. So, the challenge we faced was to come
up with a methodology that allows us to take a sequential and related criteria
approach to evaluating applications against each question, regardless of the
type of product or service that was being proposed... and then to come up with
a ranking that "makes sense" when we take a step back from the
details of the evaluation process (does the ranking tool produce similar
results to our intuitive sense of what makes a good project for ABIF).
The answers to each of these questions will be obtained from the concept note
in the first stage of the challenge fund competition and (in much more detail)
from the application in the second stage. The approach we are taking to
answering the first three questions is set out below, the way that we answer the
fourth is described elsewhere - this is the risk adjusted grant methodology we
have developed.
Commercial and developmental viability
Once we have determined that a product or
service is innovative within the ABIF definition, this is the most important
eligibility criteria related to the concept itself.
ABIF supports private sector investment
where commercial profits and development impact coincide. The Applicant should
demonstrate that the business model that they are proposing will be profitable.
We are not looking for projects that require ongoing subsidy, we do not want
projects that are built on charity or corporate social responsibility. ABIF
financial support is limited to the investment required to get a launch a
product or service.
But we want more than good business ideas, we will only support projects
where the investor will make a good return on his investment and there will be
a tangible development impact as an integral part of the business model. So to
pass this eligibility test, we need to answer two questions:
- Is the product or service commercially viable?
- Does the investment naturally drive the impact logic?
If the answer to both questions is positive, the application moves from
the eligibility evaluation to the assessment process. In this second step we
rank the projects based on the the potential development outcome (the
potential outcome) of the project, which we discount by the project delivery
risk (the capacity of the applicant to deliver that outcome) to arrive at
the expected development return.
It should be noted
that this is not a means to quantify the expected outcome, it is simply a tool
to give us a rational basis for ranking investment projects based on the
development outcomes that we can reasonably expect them to achieve. As well as
having internal value as a management tool, this also gives a framework for us
to use to explain to applicants what we are looking for in a project and how we
decided to accept or reject their application.
Potential development outcome
The potential development outcome is a product of two factors:
- The number of target beneficiaries that could experience a benefit as a
result of the new product or service; and
- The value that the target beneficiaries would place on the benefit that
they experience from adopting the product or service.
While the first factor can be projected (e.g. target market size and
penetration) and measured (e.g. actual sales), to keep things manageable and
avoid a spurious precision at the evaluation stage, we are restricting the
estimation of numbers to orders of magnitude.
However, the second factor is subjective and does not have a constant
value within our target beneficiary population. For example, one community
might value cultivation training much more highly than another, different
people of different ages might value new products differently, according to their
adaptability to new technology.
So the question for us in relation to a given project is how do these
two factors combine? If we look at our entire population of target
beneficiaries, is there a sub-group of sufficient size that values the benefit
sufficiently to incur some cost or effort to consume it, and can the product or
service be delivered in such a way that the cost of delivery is less than the
value that the consumer places on the benefit it generates. The question then
is (to an order of magnitude, anything more precise would be nonsensical given
the quality of data we have available) how big is that group?
We start off by looking at the product or service in theory, based on
what we know of the relevance, affordability and accessibility of the product
or service, the attitudes of the target beneficiary population and the
prevailing market conditions in which they operate. This allows us to place the
project somewhere in a 3 x 3 matrix and give it an associated score:
Potential
development outcome of a project is a product of how many people benefit by
how much.
|
Potential number of target beneficiaries
|
100s
|
1,000s
|
10,000s
|
Potential value
of benefit experienced by target beneficiaries
|
Not very valuable
|
0
|
0
|
1
|
Quite valuable
|
0
|
1
|
2
|
Very valuable
|
1
|
2
|
3
|
This score is
essentially a means to place an evaluation value on the potential development outcome
of the investment project. The values we use depend on the relative and
combined importance we place on the two variables; for example, the
illustrative values in the table above show that we would automatically reject
any project in the top left three cells. We could adjust the value of the other
cells to produce different results... but these illustrative values are just a
start (and probably as good as any other values as a starting point) and we
need to see how they play out in practice.
Project delivery risk
However, like any investment today that should bring returns in the
future, there is a risk factor that should be used to adjust the theoretical
impact downwards. The greater the risk, the greater the downward adjustment.
For ABIF, which operates through the private sector, the risk associated with
achieving the development potential of any given project is a function of the
qualities and capacities of the investment partner and the resources available
to them.
The specific factors that we are interested in are:
- The capacity of the applicant to manage the project and the business;
and
- The capacity of the applicant to reach the potential target beneficiary
group already identified.
Before, we have referred to the same questions as issues of the strength
of the scale agent and the strength of the route to scale.
The scale agent is the applicant, the entity that is responsible for
managing the project and the business; creating the conditions and then
achieving scale. The questions we ask here are to do with the adequacy of the
resources and the timescale for delivering the investment project itself, and
the capacity of the investor to manage the business resulting from the
investment. We look at their financial capacity, their management capacity, and
their presence and reputation in the market.
The route to scale is the physical or virtual intermediary network that
links the scale agent to the target beneficiaries. This can be a
wholesaler/retailer network for a manufacturer, or a provincial/district branch
network of a BMO, or a network of village co-operatives for a processor, or it
can be a mobile phone network or television channel for an information service
provider. Whatever the form of the network, the question for us is how
effective is it at linking the scale agent to the target beneficiaries? We
consider questions such as:
- Does it already exist and is already proven, or will it have to be
created?
- Has the network been used for the proposed purpose before, or is this a
new use?
- Does the scale agent own the network, control the network or does he buy
the facility of the network?
Again, we construct a 3 x 3 matrix, based on our assessment criteria and
the categorisation that we use and give each cell a "discount
factor":
The project
delivery risk is a measure of how confident we are that the applicant can
deliver the theoretical development outcome
|
Capacity of Applicant to manage the project and business
|
Weak
|
Borderline
|
Strong
|
Ability to reach
the target beneficiaries
|
Weak
|
0
|
0.33
|
0.5
|
Borderline
|
0.33
|
0.5
|
0.66
|
Strong
|
0.5
|
0.66
|
1
|
These discount factors are subjective and we continue to refine them
based on experience of comparing projected outcomes with actual outcomes. The
values range from 0, which means that we lack any confidence in the applicant's
ability to deliver the potential development outcome, through to 1, which means
that we are confident that the applicant has the capacity to deliver the
potential development outcome described in the application.
Expected development outcome
The final step in terms of ranking the projects is to discount the potential
development outcome by the development project risk to give us an expected
development outcome that can be used as a basis for ranking.
So we construct a table that shows the expected development return of
the group of investment projects that we want to compare:
Project
|
Potential
development outcome
|
Development
project risk
|
Expected
development outcome
|
Ranking
|
A
|
2
|
1.0
|
2.0
|
1
|
B
|
1
|
0.5
|
0.5
|
5
|
C
|
3
|
0.33
|
1.0
|
3
|
D
|
3
|
0.66
|
2.0
|
1
|
E
|
2
|
0.5
|
1.0
|
3
|
Again, it must be
stressed that the value of the expected development outcome is for ranking purposes, it is not an attempt to quantify the outcome of the
project. As a ranking indicator it allows us to compare the
various outcomes we might expect to see from different investment projects that
are competing for limited funds.
Allocating funds
The first step in allocating funds is to use the risk based approach
described elsewhere as a way of determining the grant to be offered to each
investment project. This methodology gives us a rational basis for offering a
specific grant to a specific investment project. As a result we do not have to
trade off leverage and development outcome as a part of the evaluation process.
We know how much grant a certain project will require to produce acceptable
investment returns, so now we can focus (almost) exclusively on the expected
development return as a basis of ranking and allocating funds. In this way we can
allocate grants to maximise the expected development outcome for available
budget.
Obviously things are not quite so simple as funding projects in ranked
order until we run out of funds. We have to look at the individual and
cumulative investment contributions and identify the optimal way of allocating
funds in order to maximise the expected development outcome for the round. This
may mean allocating funds to "cheaper" projects (in cash terms) that
are ranked lower than a more expensive one that we do not fund.
So, for example, if we take the projects listed above and we assume that
there is a maximum budget for the round of £1,000,000, we would recommend
funding projects B, C and D in order to maximise the expected development
outcome from the portfolio at least cost within the budget:
Ranking
|
Project
|
Expected
development outcome
|
ABIF negotiated cash
contribution
|
Funding
recommendation
|
1=
|
A
|
2.0
|
800,000
|
No
|
1=
|
D
|
2.0
|
400,000
|
Yes
|
3=
|
C
|
1.0
|
300,000
|
Yes
|
3=
|
E
|
1.0
|
400,000
|
No
|
5
|
B
|
0.5
|
200,000
|
Yes
|
Conclusion
One last word... please don't think for one moment that we are claiming
that this is some scientifically rational way to measure the development
outcome of investment projects. It is just supposed to be a simple framework to
structure a comparison and selection process in a way that:
- Brings greater consistency to our project selection judgments broadly
consistent with intuitive sense of what makes a good project for ABIF;
- Makes sense to our Investment Panel and DFID who rightly ask why we
preferred one project to another;
- Allows us to explain what we are looking for to our applicants (and to
justify our funding decision to them); and of course
- Focus our efforts and limited resources on projects that stand the best
chance of achieving development impact!