Skip to main content

Building capacity for project monitoring, evaluation, and learning among a wide range of organizations and stakeholders

TCI’s
TARINA Center of Excellence in New Delhi provides capacity building on
project monitoring and evaluation (M&E) to senior- and mid-level managers
of organizations interested in advancing data-based solutions for development
effectiveness. Recently, TCI conducted two M&E workshops, one for managers
of nongovernmental organizations (NGOs) and another for managers of corporate
social responsibility (CSR) projects.  The
former was organized on February 16-17, 2018 with Grameen Development Services
(GDS)
and the Tata Institute of Social Sciences (TISS) in Lucknow, while the
latter was organized with TISS on December 21-22, 2017 in Mumbai.

image

Prof. Prabhu Pingali, Director of TCI, delivering a lecture on M&E systems at the capacity building workshop in Luckow, Uttar Pradesh. 

Spread
over nine technical sessions, both workshops focused on M&E system
fundamentals, from establishing a Theory of Change (ToC) and a proejct Management
Information System (MIS) to understanding evaluation methods, sampling methods,
and data interpretation. A session was also dedicated to teaching managers how
to draft Terms of Reference (ToR) for evaluation studies and examples of
what a good evaluation looks like. Overall, the workshops aimed to provide
practical guidance on how managers can improve the implementation and
efficiency of their projects to maximize impact.

Getting the fundamentals right

The first session of both workshops focused on the following
questions around the fundamentals of M&E:

  • Where are we going?

  • How do we get there?
  • What will tell us if we are moving in the right direction?
  • What will tell us that we have arrived?”

The purpose of these questions is to help
managers define the primary goal of a project, as well as secondary objectives which
are intended to provide a detailed strategy for accomplishing the desired goal.
Furthermore, the session provdied an overview of commonly used M&E terms,
such as project inputs, outputs, outcomes, performance indicators, and
benchmarks. This helped participants understand and appreciate the various
components of monitoring, evaluation, and impact assessment, as well as the
differences between these three concepts.

Establishing a
Theory of Change for efficient M&E

TCI’s workshops emphasized the need for managers to establish a ToC
at the onset of a project, which delineates the main outcomes or results the
project aims to achieve and the main pathways by which it will achieve them. A
clearly defined ToC is key to developing a strong project and M&E system.

Participants worked in groups to develop and present a
ToC for an existing or new project. By the end of the session, they could begin
to appreciate how the ToC lays out the foundation of any project and becomes
the reference point for understanding the successes and/or failures of the
project as it is implemented.

Identifying performance
indicators for measuring change

Measuring
change and the impact of a project over time is often tricky. Defining precise and
consistent indicators is essential for effective M&E. TCI dedicated a full
technical session to the basic rules of measurement. Discussions focused on: identifying
project input, output, and outcome indicators; choosing clear data sources for
these indicators; and determining the frequency of data collection.  

Converting data
into information for better project management

Strategic improvements can be made to project implementation with consistent, reliable,
and quality data. Participants discussed the various methods used within their respective
organizations for gathering data and extracting lessons to improve project management.
These included unstructured field monitoring visits and semi-structured, need-based
visits to project sites, as well as periodic progress reporting. Among the methods shared,
TCI highlighted the need for organizations to build and maintain a flexible,
cost-effective, and participatory project MIS for collecting real-time data in
the field on a regular basis. The granular data and indicators gathered through
an MIS can provide revealing information, allowing for mid-course corrections to
ensure better outcomes.

Comparing
changes: Methods for conducting an evaluation study

The
purpose of an evaluation study is to measure and assess the changes that can be
attributed to the various activities or interventions implemented under a
particular project, which is typically achieved by comparing changes observed
between a treatment group and a control group. This technical session
exposed participants to various evaluation methods and addressed questions such
as the following:

  • How can changes be compared if no baseline survey was conducted?
  • Is there an ideal comparison group?

The
choice of an evaluation method largely depends on the availability of
resources, type of information needed, degree of interruption to the project,
and the optimal way to reach respondents given the literacy, social, and
cultural context of the project location.

From data collection to data analysis

Data is a critical component of any project and so
administration of data becomes an important task for managers. This session reviewed
the technical aspects of collecting, cleaning, managing, and converting data
into information. Discussions focused on the following questions raised by
participants:

  • What is an ideal sample size?

  • What data collection tools should be adopted?

During this session, participants learned
about the advantages of commonly used data collection tools, namely Paper
Assisted Personal Interview (PAPI) and Computer Assisted Personal Interview
(CAPI) tools. They also learned the importance of distinguishing between “good”
data and “bad” data once it has been collected and were asked to engage in an
exercise where they had to identify “good” data from a field survey conducted. Finally,
basic rules for data cleaning were discussed, including the need to ensure
units are consistent, data labels are clear and easy to understand, and duplicate records are removed.

The why, what, and how of evaluation management

It is important to define the expectations of an evaluation study at
the outset, and so it is equally important to develop a clear ToR for the evaluator(s) that will undertake the study.
A ToR defines the objectives, scope, and responsibilities of the evaluator(s).
It also provides a description of the resources available to conduct the
study. Systematic M&E reporting forms an integral part of the evaluation and involves the identification of which findings are to be reported, to whom, at
what interval, and in what format.

From M&E to a learning organization

Project assessments are not always easy to undertake or to
digest and act upon the results when complete. Therefore, in the final
technical session, participants were asked to deliberate on the following
questions:

  • What is the M&E budget in their respective organizations?

  • Who has access to M&E results?
  • With whom are the results shared?
  • Does the organization encourage learning from failures?

During this session, TCI explained that upon finalizing
M&E results, organizations should translate the results into knowledge for
improving project implementation, management, and processes. TCI also emphasized
that in order to initiate change, organizations should aim to become a
“learning organization” by acquiring knowledge and modifying its behaviour to reflect
new insights.

TCI to continue
capacity building efforts

Building the M&E capacities of different stakeholders forms
an integral part of TCI’s work in India. Technical workshops organized so far
have helped to broaden stakeholders’ understanding of M&E systems for development
projects. Similar workshops will continue to be offered to organizations interested
in strengthening their M&E capacities to become effective “learning
organizations.”

By Ruchira Boss

Ruchira Boss (rb838@cornell.edu) is an Assistant Program Officer for TCI-TARINA, based in New Delhi, India.