How will you know if you achieved the results? It is not enough to say whether or not you have achieved the results; it is important to know to what extent you have achieved the objectives and results outlined in your project.
The answer is in the next column of the logical framework matrix: the indicators.
Another post on the Logframe methodology:
- Example of a logical framework
- Stakeholder analysis
- Problem tree
- Objective tree
- Analysis of alternatives
- Analytical project structure
- Project Narrative
- Project indicators
- Means of verification
- Project assumptions
This is today’s topic in Ingenio Empresa.
Definition of indicators in the logical framework
The indicators or objectively verifiable indicators (OVI) is a reference measure that indicates the current or final progress of the project.
This is how I define it. Another very relevant definition is from the Logical Framework Methodology manual, which states that:
An indicator is the quantitative and qualitative specification for measuring the achievement of an objective, collectively accepted by those involved in the project as adequate to achieve the corresponding objective.
And to finish defining what an indicator is in the logical framework and to continue with its formulation, I would like to mention three important things:
- Indicators are defined based on the items in the narrative summary. This indicates that there can be multiple indicators for the components, purpose and end of the project; however, it is advisable to use a minimum number of indicators so that with this number you can conclude whether or not the evaluated item was achieved.
- It is important to agree on the indicators and their measurement method with all project stakeholders. In this way, each stakeholder will be able to know the current situation in the achievement of the indicator or to what extent the objective has been achieved.
- Define practical, independent and focused indicators. By practical I mean that which is important and realistic. By independent I mean that you avoid measuring the means used to achieve the objectives, but rather determine to what extent those means allowed you to achieve the objectives. By focused I mean that they have a defined target population, place, time, quantity and quality.
Is the use of indicators important?
Yes, it is. They are important to know how the project is going. In other words, they fulfill an evaluation function for the different moments of the project, either in terms of management or monitoring, which is useful to take the necessary actions towards the achievement of the objectives.
With this clear, let’s see how to define indicators in the logical framework.
How to make the indicators in the logical framework matrix
Step 1: Defining the indicators
Define the indicators for each objective (narrative summary items). To do so, consider the following:
- Goal indicators: With them we identify the change or impact generated by the project (its contribution). They also show the need for additional components or products.
- Purpose indicators: As with end indicators, purpose indicators show the impact of the project. But these differ in that they are a direct effect on the project situation or population.
- Outputs indicators: These are descriptions of the goods or services (project results) in terms of quality, quantity and time.
- Activity indicators: These are the monitoring of activities in terms of resources, time, costs, budget, compliance or personnel. They show the management and effort to build and deliver the goods and services.
When you are creating the indicators, make sure they contain:
- Target population or group: For whom?
- The place: Where?
- Quantity: How much?
- Quality: How is it required?
- The time: When?
Let’s look at an example. We want to increase the employment rate of young people in the city of Bogota Colombia.
- Who: Young people between 18 and 25 get their first job.
- Where? 18 to 25 year olds in the city of Bogota obtain their first job.
- How much? 300 young people between 18 and 25 years old in the city of Bogota obtain their first job.
- How? 300 young people between 18 and 25 years old in Bogota obtain their first job with a contract of at least one year’s duration
- When? Between 2015 and 2017, 300 young people between 18 and 25 years of age in Bogota obtain their first job with a contract of at least one year duration.
Step 2: Setting the criteria
You probably came up with a lot of indicators in step 1. Now we are going to apply a set of criteria to select the best ones.
The United Nations Development Program Evaluation Office (UNDP) proposes this table to filter and select the most appropriate indicators.
The letters correspond to the criteria for filtering and selecting indicators. The UNDP mentions:
- The indicator is clear and understandable
- The information for the indicator is available or can be collected
- The indicator is tangible and observable
- Collecting the indicator data is within the reach of the project team and does not require experts.
- The indicator is representative for the narrative summary item being referenced.
The criteria are up to you. You will probably cover some of the sample criteria mentioned in the UNDP, but the definition of the criteria will depend on the nature of the project.
Step 3: Adding up to select indicators
With the criteria defined, evaluate each indicator by placing 1 if it meets the criteria or 0 if it does not. In the end, the indicators with the highest score will be the ones chosen.
Step 4: Placing the indicators in the column of the logical framework matrix
You now have the indicators for each objective. If you are not satisfied with the result, come up with new indicators and re-evaluate them against the criteria. This is a trial and adjustment exercise
With this we are ready for the next step in the logical framework methodology.
Let’s continue with the example of Colusa Inc: A software company.
Example of indicators in the Logical Framework Matrix
Step 1. We will begin by defining the indicators for each item in the narrative summary.
We will also clarify each indicator in terms of who, where, how much, how and when. We do this to be more confident in the process and to avoid mistakes in this first step. This step should be done as a team with the actors and stakeholders of the project, and those who for some reason are not involved should be aware of what is defined in this step.
I have decided to place the activity indicators at the end, since they are expressed in terms of budget, resources, time or compliance, so the previous table is not useful.
Step 2. We continue with the criteria. As in step 1, this step is done as a team between project management and stakeholders. Everyone should be aware of the criteria defined here.
- Achievable: It is possible to achieve the indicator, i.e. the goal is realistic.
- It is measurable: The information to measure the indicator is easy to collect. In addition, there are formulas to obtain the indicator.
- It is relevant: It is the best way to evidence whether a goal has been achieved.
- It is independent: The indicator is not a measure of the means used to achieve an objective, but a measure of the objective achieved. In addition, the indicator is not related to any other indicator.
Step 3: With the criteria defined, we will evaluate each indicator.
This analysis is done as a group and the permanence, elimination or creation of indicators is defined by consensus.
Why did some indicators score 0?
Indicator 3 Job satisfaction and 4 Staff turnover scored 0 in relevance because it is not estimated to be the best way to measure the objective of staff motivation growth. In fact, we can say that staff turnover is related to staff satisfaction.
However, the relationship is not direct, as employee turnover may be influenced by other factors beyond employee satisfaction in the company. Even so, in this example the project management chooses to keep the indicators, because even though they consider that it may not be the definitive way to measure motivation, it is sufficient to show changes in the long term.
Indicator 5 Customer referrals is 0 because customer recommendations about Colusa’s service are difficult to control. In relevance it has 0 because the company is not interested in having good references. What it is interested in is increasing its market share, which is measured by indicator 2. That is why it is ELIMINATED.
Indicator O.3.1 Mystery shopper script revisions has 0 in relevant because having revisions to a script is not going to ensure that a mystery shopper session program is created and implemented. Also, it scores 0 on independent because the revisions are a means to achieve the objective, i.e., this indicator does not measure whether or not the objective was achieved. It is definitely ELIMINATED.
Indicator O.4.1 Number of tests carried out for the telephone rating mechanism scores 0 for the same points as C.3.1. Doing more or less testing is not going to assure me that I will have the service rating options implemented, plus testing is a means to achieve an objective and not the objective itself. Therefore, ELIMINATED.
Finally, the O.5.1 Testing creation with Wp and Joomla has 0 in relevant. Getting 100% result in the tests is not a way to evidence the implementation of the option to create a website with Wp and Joomla, however it is important to ensure that the objective is achieved.
Note that this indicator is similar to O.4.1, however it is measuring that the result is 100% and not the number of tests done; since obtaining 100% in the tests is vital to ensure quality in the creation of websites for customers, in the independence criterion this indicator gets 1 and not 0.
Step 4. With the indicators ready, this is the partial result in the logical framework methodology:
I have not gone into much detail on the calculation of the cost of project activities because I do not consider that it contributes to the learning of the logical framework methodology. It is a different matter. In fact and in theory, we could indicate other aspects for the follow-up of the activities, especially those that are at $0. But well, on this occasion we will leave it at that and prepare for the next step in the logical framework approach.