TOPIC 4: INDICATORS
Sub-Topics
1) Meaning of Indicators
2) Importance of Monitoring and Evaluation Indicators
3) Types of Indicators
4) Characteristics of a good Indicator
5) Process of selecting Indicators
6) Process and change in Indicators
MEANING OF INDICATORS
What is an indicator?
Indicators are signs that show changes in certain conditions
They are eye openers, markers, units of measure, descriptors, reducers
Five types: Input, Process, output, outcome and impact
An indicator is simply a measurement
Indicators are compared over time in order to assess change
Operation is broken down into design elements such as inputs, activities, outputs, outcomes and impacts referred to as “performance indicators”.
We express indicators in number percentage or even ratios, proportions
IMPORTANCE OF MONITORING AND EVALUATION INDICATORS
1.At the initial phase of a project:
Indicators are important for the purposes of defining how the involvement will be measured It help to pre-determine how effectiveness will be evaluated in a precise and clear manner.
2. During project implementation:
Indicators serve the purpose of aiding program managers assess project progress and highlight areas for possible improvement. Indicators are measured against project goals, managers can be able to measure progress towards goals and inform the need for corrective measures against potential catastrophe (Disasters).
3.At the evaluation phase:-
Indicators provide the basis for which the evaluators will assess the project impact.
Without the indicators, evaluation becomes an uncertain responsibility.
Performance Indicators are used to:
Help organizations to understand their performance levels.
Help setting realistic performance goals.
Help aligning daily work to strategic goals.
Help monitoring progress on a real-time basis.
Help understanding the weaknesses and establishing improvement priorities.
Determine whether an improvement is being made and maintained.
Help benchmark internally and externally.
Identify if staff are doing well and to help them if they are not.
Provide a basis for recognizing team and individual performance.
TYPES OF INDICATORS
Resource Indicators
These concern the budget allocated at every level of the intervention, the human resources mobilized, etc. The measurement of these indicators provides information about the efficiency of the project’s intervention. Example: Grants, Annual Budget, Borrowings, Labour, Input costs, Etc. Activities Indicators To achieve the project objectives, a certain number of activities are implemented. The monitoring system may focus on the level of achievement of these activities. Example: Meetings, Training, Visits, Etc.
Process Indicators:
They relate to the decision-making process and makes it possible to determine, for example, the inclusion and/or participation of actors, the regularity of meetings, the regularity of reports, etc. Example: Existence of reporting rules, Degree of women’s participation in decisions, Frequency of debriefing meetings, Etc. Product Indicators
When an activity is carried out, it generally generates a tangible or non-tangible product. For example, at the end of a training, the number of people trained (product) can be assessed. The product is the immediate and tangible result of the combination of a resource (expenditure) and an activity (training): product: number of trainees. Example: Trainees, Number of text
books produced, Number of credit received ,Etc. The Results Indicators
These reflect behavior. For example, a few months after the training, one can assess how
many participants have applied what they learnt. Or if a project distributes equipment
(product) one can assess how many beneficiaries use this equipment. Example: Number of
people who applied the training received, Use of credit obtained, Number of people who use
the text books,Etc.
Categories of Indicators
Goal level indicators – These are the program of sector objective indicators to which the program is directed .They include targets beyond the scope of the project It is important to scrutinize project appraisals documents and projects operations manual which clearly stipulates the project development documents from which the goal indicates would be derived Purpose level indicators – This are used to define the changes in behavior of project beneficiaries and the way changes in the institution functions as a result of the project outpost Output level indicators- This indicators define deliverables and they are used to establish the terms of reference to a project Activity level indicators – At this level, there are usually the inputs or budgets. This can be established using standard categories like commodities, technical services etc. The budget statements are usually a summary of resources. Quantitative and Qualitative Indicators
Quantitative indicators
Expressed in terms of specific numbers, percentages, proportions and rates of something. Numbers on their own may not be sufficient to indicate e.g range of success and failure likewise percentage by itself does not indicate the size of success therefore if we were to allow the significance of the outcome then we typically require the data on both number and percentage
Qualitative indicators
They provide insights into changes in institutional processes, attitudes, beliefs, motives and behaviors of individuals
A qualitative indicator might measure perception, such as the level of empowerment that employees feel to adequately do their jobs.
Qualitative indicators might also include a description of a behavior, such as the level of mastery of a newly learned skill.
Although there is a role for qualitative data, it is more time consuming to collect, measure, and distill, especially in the early stages.
Qualitative indicators are harder to verify because they often involve subjective judgments about circumstances at a given time.
Qualitative indicators should be used with caution. It is not just about documenting perceptions of progress.
It is about obtaining objective information on actual progress that will aid managers in making more well-informed strategic decisions, aligning budgets, and managing resources.
Proxy Indicators
Sometimes it is difficult to measure the type of indicators directly so we use proxy indicators.
Proxy indicators are used to give at least approximate evidence on performance.E.g.
They are used when direct indicators are not available or visible regularly. If it difficult to conduct periodic household surveys in dangerous housing areas one could use the number of the iron sheets or television antennae as a proxy measure of increased house hold income.
Proxy indicators may correctly take desired outcome but there could be other contributing outcome. Pre-designed indicators
Pre-designed indicators are indicators established independently of an individual country/organization ,program or sector content
For example: a number of development institutions have created indicators to track development goals e.g sustainable development goals indicators ,the united nations development programs ( UNDPS) the World Bank National Development hard book,
The International Monitory Fund (IMF).
(Knec question: What is the importance of indicators)
CHARACTERISTICS OF A GOOD INDICATOR (CREAM)
Performance indicators should be as clear, direct, and unambiguous as possible. Performance indicators should be relevant to the desired outcome, and not affected by other issues tangential to the outcome. The economic cost of setting indicators should be considered. This means that indicators should be set with an understanding of the likely expense of collecting and analyzing the data. Indicators ought to be adequate. They should not be too indirect, too much of a proxy, or so abstract that assessing performance become complicated and problematic. Indicators should be monitor -able, meaning that they can be independently validated or verified Normally we state indicators in terms of quantity, quality and time and sometimes place and costs putting numbers and date on indicators is called targeting.
Variables that affect Indicators
a) Size of the project -The bigger the project the more indicators you need.
b) Availability of resources-Lack of adequate resources affect the choice of indicators
c) The duration of the project- Some projects take longer to complete than others
d) The availability of technical capacity- Indicators need knowledge and technical know how to set
e) Support from top management or institution
PROCESS OF SELECTING INDICATORS
The process of formulating indicators should begin with the following questions:
How can we measure that the expected results are being achieved?
What type of information can demonstrate a positive change?
What can be feasibly monitored with given resource and capacity constraints?
Will timely information be available for the different monitoring exercises?
What will the system of data collection be and who will be responsible?
Can national systems be used or augmented?
Can government indicators be used
Factors to consider when selecting project indicators
Any appropriate M&E indicator must meet particular thresholds. They must be:
1. Precise/Well defined: Probably the most important characteristic of indicators is that they should be precise or well defined. I other words, indicators must not be ambiguous. Otherwise, different interpretations of indicators by different people implies different results for each
2. Reliable: Reliability here implies that the indicator yields the same results on repeated trials/ attempts when used to measure outcomes. If an indicator doesn’t yield consistent results, then it is not a good indicator.
3. Valid: Validity here implies that the indicator actually measures what it intends to measure. For example, if you intend to measure impact of a project on access to safe drinking water, it must measure exactly that and nothing else.
4. Measurable: Needless to say that an indicator must be measurable. If an indicator cannot be measured, then it should and must not be used as an indicator.
5. Practicable: In other cases, although an indicator can be measured, it is impracticable to do so due to the cost or process constraints. An indicator must be able to utilize locally available resources while at the same time being cost effective.
Plan Canada (46) has described the process of indicator development as involving the following elements:
• Definition of the characteristics to be measured
• Identification of the target audience and the purpose of the indicator
• Choosing a framework (i.e. one based on goals, issues, sectors or stress conditionresponse)
• Definition of criteria for selecting indicators • identification and evaluation of a potential indicator on the basis of the selection criteria
• Pilot-testing of the indicator
• Choosing the final set and reviewing the indicator periodically
Many organizations have attempted to define criteria for the construction and selection of indicators, depending on whether they apply to policy, analytical soundness or measurability.
They may also be assessed in relation to factors such as transparency, scientific validity, robustness, sensitivity and the extent to which they are linkable, or according to whether they are relevant to the issue they are intended to describe, whether they relate to changes in
policy and practice or whether they “strike a chord” with their intended audience .
Criteria for Indicators of use for International Purposes
These indicators should be:
• Linked to broadly identified common problems and global priorities
• Appropriate for inter-country comparisons
• Relevant to international initiatives such as SDGs, Agenda 2063 or to international conventions and treaties
• Attractive to a range of sectors, partners and institutions
• Ideally usable for decision-making at different tiers of government
• Based on sound, internationally comparable data that are readily available or easily and relatively inexpensively collected
Criteria for Indicators of use for Local Purposes
These indicators should:
• Be relevant both to individual citizens and to local government –Vision 2030
• Reflect local circumstances
• Be based on information that can be readily collected
• Show trends over a reasonable period of time
• Be meaningful both in their own right and in conjunction with other indicators
• Be clear and easy to understand, in order to educate and inform
• Provoke change (for example in policies, services or lifestyles)
• Lead to the setting of targets or thresholds
Indicators are more to be objectives if they include elements of quality, quantity, time and location
Indicators should be constructed to meet specific needs. They need to be direct
When developing indicators we start with the basic indicators i.e. construct classrooms in Thika technical
Add the quantity e.g increase learning space by 80%. It is required that baseline data be corrected i.e. The situation as it is before the intersection
Add quality e.g 100% completed and operational classrooms
Add time 100% classrooms in use, complete and in use by the end of year 2
Add location 100% complete classroom in use by the end of year 2 at Thika Technical
Example: Basic Sanitation
Indicator: percentage of the population with adequate excreta disposal facilities.
Definition of indicator: proportion of the population with access to a sanitary facility for human excreta disposal in the dwelling or in its immediate vicinity.
Unit of measurement: a percentage.
Measurement variables: the term “sanitary facility” should be defined, for instance as “a unit for the disposal of human excreta which isolates faeces from contact with people, animals, crops or water sources”. The facilities could range from simple, protected pit
latrines to flush toilets with sewerage. The population covered could be defined as that served by connections to sewers, household systems (pit latrines, septic tanks) or communal toilets. The term “immediate vicinity” should also be defined, perhaps as any sanitary facility within 50 meters of a dwelling.
Purpose: the purpose of this indicator is to monitor progress in the access of a population to sanitary facilities. It is important to assess access to adequate excreta disposal facilities, as this is linked fundamentally to the risk for faecal contamination and disease and ill-health
among the population. When disaggregated by geographical area or by socioeconomic status, it also provides evidence of inequalities. Users would include sanitary engineers, planners, public health officials, nongovernmental organizations and others. Linkages: the indicator could be linked to other indicators, such as the proportion of the population with access to adequate and safe drinking-water, or to a health effects indicator such as mortality and morbidity from diarrheal diseases.
Data requirements: data could be obtained from censuses or special surveys and should be disaggregated by (for example) geographical area or urban-rural divide.
1 . Outline factors project team should consider when selecting an indicator in the process of Monitoring and Evaluation.
2. Give 5 reasons a project team would give to jusify the need for developing indicators before conducting M$E
3. Designing project monitoring and evaluation is taken as a challenging task to project teams. Justify this statement
4. Describe the qualities of an indicator that an M$E officer would avoid for when establishing the outcome of a given activity
5. Types of indicators
6. Challenges of Indicators
7. Importance of indicator
