We have now reached the stage of the first round when we have to rank shortlisted projects competing for limited funds. Our shortlist is such that the cumulative requested grant contribution is significantly higher than the total grant budget available for this round.

Of course, we can expect adjustments to grant requests once the applicants prepare their detailed business plans, we explore alternative financing options, and we have serious negotiations with applicants. Maybe some applicants will even drop out. But nevertheless, we need a rational basis for ranking projects (that deliver very different types of benefits) against one another.

Our problem was how to compare (for example) a project that delivers access to affordable genuine pharmaceuticals against a project that delivers increased fruit yields per hectare, or against a project that opens up new markets for goat owners? How to incorporate this diversity of benefit into a comparison of different grant/total investment ratios? Should we monetise everything, do we maximise leverage as some of our colleagues would argue?

I don't think that there is an acceptable conversion rate for a child's recovery from illness to extra kgs of apples, and leverage is irrelevant when we are interested in value for money development impact. Rather, we wanted to find a methodology that is slightly more sensible (and sensitive) than these approaches. The rest of the methodology described below is pretty standard stuff, but this is the tricky bit...

We have been working on the ranking/fund allocation methodology for some time now, but now we have to apply it. Fortunately the development has now reached a point where we are pretty much ready to pilot it with our shortlist. If this methodology works (if the applicants get it and it produces the intuitively right results) then the idea is to roll it out for the second round.

One thing to note is that this approach only works when it is incorporated with the risk adjusted grant methodology we have already adopted. Otherwise you don't have a rational basis for the "cost to ABIF" column at the final step and you get into difficulties with weighted incomparable criteria and leverage/impact trade off.

So here is an overview of the methodology we intend to use to rank projects and allocate limited grant funds. We would of course be very grateful for any feedback or suggestions for improvements.

Introduction

When we are evaluating projects we are primarily concerned with four questions:

Will the product or service deliver profits to the investor and development impact to the donor?
What is the potential outcome of the project, how many of our target beneficiaries could benefit and how much could they benefit by?
What are the chances of the applicant achieving the theoretical potential, do they have the necessary capacity and resources?
Are we giving the right amount of financial support?

The questions above relate to very different properties of the applicant and the proposed concept. It is not possible, without creating unacceptable trade-offs or forcing incompatible criteria into the same selection step to look at all of these questions together. So, the challenge we faced was to come up with a methodology that allows us to take a sequential and related criteria approach to evaluating applications against each question, regardless of the type of product or service that was being proposed... and then to come up with a ranking that "makes sense" when we take a step back from the details of the evaluation process (does the ranking tool produce similar results to our intuitive sense of what makes a good project for ABIF).

The answers to each of these questions will be obtained from the concept note in the first stage of the challenge fund competition and (in much more detail) from the application in the second stage. The approach we are taking to answering the first three questions is set out below, the way that we answer the fourth is described elsewhere - this is the risk adjusted grant methodology we have developed.

Commercial and developmental viability

Once we have determined that a product or service is innovative within the ABIF definition, this is the most important eligibility criteria related to the concept itself.

ABIF supports private sector investment where commercial profits and development impact coincide. The Applicant should demonstrate that the business model that they are proposing will be profitable. We are not looking for projects that require ongoing subsidy, we do not want projects that are built on charity or corporate social responsibility. ABIF financial support is limited to the investment required to get a launch a product or service.

But we want more than good business ideas, we will only support projects where the investor will make a good return on his investment and there will be a tangible development impact as an integral part of the business model. So to pass this eligibility test, we need to answer two questions:

Is the product or service commercially viable?
Does the investment naturally drive the impact logic?

If the answer to both questions is positive, the application moves from the eligibility evaluation to the assessment process. In this second step we rank the projects based on the the potential development outcome (the potential outcome) of the project, which we discount by the project delivery risk (the capacity of the applicant to deliver that outcome) to arrive at the expected development return.

It should be noted that this is not a means to quantify the expected outcome, it is simply a tool to give us a rational basis for ranking investment projects based on the development outcomes that we can reasonably expect them to achieve. As well as having internal value as a management tool, this also gives a framework for us to use to explain to applicants what we are looking for in a project and how we decided to accept or reject their application.

Potential development outcome

The potential development outcome is a product of two factors:

The number of target beneficiaries that could experience a benefit as a result of the new product or service; and
The value that the target beneficiaries would place on the benefit that they experience from adopting the product or service.

While the first factor can be projected (e.g. target market size and penetration) and measured (e.g. actual sales), to keep things manageable and avoid a spurious precision at the evaluation stage, we are restricting the estimation of numbers to orders of magnitude.

However, the second factor is subjective and does not have a constant value within our target beneficiary population. For example, one community might value cultivation training much more highly than another, different people of different ages might value new products differently, according to their adaptability to new technology.

So the question for us in relation to a given project is how do these two factors combine? If we look at our entire population of target beneficiaries, is there a sub-group of sufficient size that values the benefit sufficiently to incur some cost or effort to consume it, and can the product or service be delivered in such a way that the cost of delivery is less than the value that the consumer places on the benefit it generates. The question then is (to an order of magnitude, anything more precise would be nonsensical given the quality of data we have available) how big is that group?

We start off by looking at the product or service in theory, based on what we know of the relevance, affordability and accessibility of the product or service, the attitudes of the target beneficiary population and the prevailing market conditions in which they operate. This allows us to place the project somewhere in a 3 x 3 matrix and give it an associated score:

Potential development outcome of a project is a product of how many people benefit by how much.		Potential number of target beneficiaries
		100s	1,000s	10,000s
Potential value of benefit experienced by target beneficiaries	Not very valuable	0	0	1
	Quite valuable	0	1	2
	Very valuable	1	2	3

This score is essentially a means to place an evaluation value on the potential development outcome of the investment project. The values we use depend on the relative and combined importance we place on the two variables; for example, the illustrative values in the table above show that we would automatically reject any project in the top left three cells. We could adjust the value of the other cells to produce different results... but these illustrative values are just a start (and probably as good as any other values as a starting point) and we need to see how they play out in practice.

Project delivery risk

However, like any investment today that should bring returns in the future, there is a risk factor that should be used to adjust the theoretical impact downwards. The greater the risk, the greater the downward adjustment. For ABIF, which operates through the private sector, the risk associated with achieving the development potential of any given project is a function of the qualities and capacities of the investment partner and the resources available to them.

The specific factors that we are interested in are:

The capacity of the applicant to manage the project and the business; and
The capacity of the applicant to reach the potential target beneficiary group already identified.

Before, we have referred to the same questions as issues of the strength of the scale agent and the strength of the route to scale.

The scale agent is the applicant, the entity that is responsible for managing the project and the business; creating the conditions and then achieving scale. The questions we ask here are to do with the adequacy of the resources and the timescale for delivering the investment project itself, and the capacity of the investor to manage the business resulting from the investment. We look at their financial capacity, their management capacity, and their presence and reputation in the market.

The route to scale is the physical or virtual intermediary network that links the scale agent to the target beneficiaries. This can be a wholesaler/retailer network for a manufacturer, or a provincial/district branch network of a BMO, or a network of village co-operatives for a processor, or it can be a mobile phone network or television channel for an information service provider. Whatever the form of the network, the question for us is how effective is it at linking the scale agent to the target beneficiaries? We consider questions such as:

Does it already exist and is already proven, or will it have to be created?
Has the network been used for the proposed purpose before, or is this a new use?
Does the scale agent own the network, control the network or does he buy the facility of the network?

Again, we construct a 3 x 3 matrix, based on our assessment criteria and the categorisation that we use and give each cell a "discount factor":

The project delivery risk is a measure of how confident we are that the applicant can deliver the theoretical development outcome		Capacity of Applicant to manage the project and business
		Weak	Borderline	Strong
Ability to reach the target beneficiaries	Weak	0	0.33	0.5
	Borderline	0.33	0.5	0.66
	Strong	0.5	0.66	1

These discount factors are subjective and we continue to refine them based on experience of comparing projected outcomes with actual outcomes. The values range from 0, which means that we lack any confidence in the applicant's ability to deliver the potential development outcome, through to 1, which means that we are confident that the applicant has the capacity to deliver the potential development outcome described in the application.

Expected development outcome

The final step in terms of ranking the projects is to discount the potential development outcome by the development project risk to give us an expected development outcome that can be used as a basis for ranking.

So we construct a table that shows the expected development return of the group of investment projects that we want to compare:

Project	Potential development outcome	Development project risk	Expected development outcome	Ranking
A	2	1.0	2.0	1
B	1	0.5	0.5	5
C	3	0.33	1.0	3
D	3	0.66	2.0	1
E	2	0.5	1.0	3

Again, it must be stressed that the value of the expected development outcome is for ranking purposes, it is not an attempt to quantify the outcome of the project. As a ranking indicator it allows us to compare the various outcomes we might expect to see from different investment projects that are competing for limited funds.

Allocating funds

The first step in allocating funds is to use the risk based approach described elsewhere as a way of determining the grant to be offered to each investment project. This methodology gives us a rational basis for offering a specific grant to a specific investment project. As a result we do not have to trade off leverage and development outcome as a part of the evaluation process. We know how much grant a certain project will require to produce acceptable investment returns, so now we can focus (almost) exclusively on the expected development return as a basis of ranking and allocating funds. In this way we can allocate grants to maximise the expected development outcome for available budget.

Obviously things are not quite so simple as funding projects in ranked order until we run out of funds. We have to look at the individual and cumulative investment contributions and identify the optimal way of allocating funds in order to maximise the expected development outcome for the round. This may mean allocating funds to "cheaper" projects (in cash terms) that are ranked lower than a more expensive one that we do not fund.

So, for example, if we take the projects listed above and we assume that there is a maximum budget for the round of £1,000,000, we would recommend funding projects B, C and D in order to maximise the expected development outcome from the portfolio at least cost within the budget:

Ranking	Project	Expected development outcome	ABIF negotiated cash contribution	Funding recommendation
1=	A	2.0	800,000	No
1=	D	2.0	400,000	Yes
3=	C	1.0	300,000	Yes
3=	E	1.0	400,000	No
5	B	0.5	200,000	Yes

Conclusion
One last word... please don't think for one moment that we are claiming that this is some scientifically rational way to measure the development outcome of investment projects. It is just supposed to be a simple framework to structure a comparison and selection process in a way that:

Brings greater consistency to our project selection judgments broadly consistent with intuitive sense of what makes a good project for ABIF;
Makes sense to our Investment Panel and DFID who rightly ask why we preferred one project to another;
Allows us to explain what we are looking for to our applicants (and to justify our funding decision to them); and of course
Focus our efforts and limited resources on projects that stand the best chance of achieving development impact!