Module 6.1. Key Concepts

Conflict-sensitive, climate-responsive approaches to monitoring and evaluation (M&E) are critical elements of successful locally led action for climate, peace and security serving two central objectives: accountability and learning.

Module 6.1 introduces key concepts in monitoring, evaluation, accountability and learning, including:

This module will help users:

• appreciate the value of monitoring and evaluation for benefits far beyond required reporting
• balance the importance of accountability with the benefits of learning during a LLACPS project
• consider multi-faceted impact dimensions and indicators relevant to climate, peace and security

Traditional approaches to project planning highlight monitoring and evaluation for the purpose of reporting, with “M&E” often written on its own. Conversely, the “MEAL” Framework – monitoring, evaluation, accountability and learning – emphasises the purpose and benefits of M&E as processes that should be central to effective project planning and implementation, rather than simply a reporting mechanism conducted only when required by funders.

Monitoring, evaluation, accountability and learning (MEAL) supports the design and implementation of a project by providing a framework for understanding what problems the project seeks to address, how well the project is being implemented, what kinds of effects the project is having and what about the project may need to change.

• Monitoring: An ongoing process of collecting information or evidence about a project’s activities and results as well as the broader context. This information can be used to assess progress, risks and assumptions and to make decisions about a project’s implementation. Monitoring information, often in the form of indicators, feeds into evaluation and learning processes. 
• Evaluation: The systematic assessment of a project, either ongoing or completed. Evaluations are often grounded in evaluation questions about the project’s effectiveness and impact, whether or not it has been efficiently and equitably implemented, its relevance to the context and needs, whether it is in line with other projects and the sustainability (or likely sustainability) of positive outcomes.
• Accountability: The responsibility to carry out a project as claimed, and to ensure that the agreed upon deliverables and expectations are satisfactorily met. Donors may require that project managers demonstrate particular outputs, outcomes and results or they will reclaim/withhold funding, but accountability in participatory projects also emphasises the responsibility facilitators hold in relation to program participants and communities.
• Learning: Using insights from evidence and experience to identify successes, challenges, lessons learned, etc. and integrate those into the project and/or future activities. While learning can be informal and ad hoc, more structured learning processes are generally grounded in learning questions.

As highlighted in the Monitoring & Evaluation Note for Addressing Climate-Related Security Risks: Learning and accountability are emphasised to different degrees in different applications, depending on the organisation(s) implementing the project and the donor(s). M&E systems have traditionally stressed accountability to identify results for the purpose of reporting to funders, partners and beneficiaries. When overly focused on accountability, M&E might be seen as having an audit function – especially when funding is dependent on showing particular results and value for money.

However, research carried out by the World Bank has shown that a more flexible approach to learning-by-doing, which includes making errors and even failing, might support long-term success more effectively than a traditional accountability approach. A learning approach requires support for programme adaptation that may include, for example, testing promising innovations – which might fail – and experimenting with high-risk, high-return strategies.

Some approaches use “MEL” (monitoring, evaluation and learning) as a framework, without ‘accountability’ explicitly included, in order to highlight a focus on constructive adaptation rather than what may feel like punitive rigidity in regards to reporting. It is, though, important to maintain responsibility and transparency: flexibility about a project’s exact results and implementation processes, as well as acknowledgement that missteps and unintended consequences are likely in highly complex system and can be learned from, should not equate to ‘immunity’ for project managers and facilitators from being answerable to the impacts of their actions. Accountability continues to be a critical component of meaningful interventions, and is especially important for externally-supported facilitators – who will, to at least some extent, be privileged and/or ‘outsiders’ – involved in locally led action.

In the context of climate, peace and security work, MEAL plays a number of important roles. They include:

• Providing information on and insight into the interplay between climate, peace, and security activities and outcomes. MEAL processes generate evidence on how different activities interact with one another and the context to lead to results, both positive and negative. Good MEAL practice helps with understanding how climate adaptation efforts affect peace and security, and vice versa. 
• Ensuring an intervention is responsive to the context, which may be fluid and dynamic. In many of the places in which climate, peace and security work takes place, the situation may be constantly and even rapidly changing. MEAL processes consider the relationship between the project and the broader context by monitoring not just what the project does, but how it affects the context – as well as how the context may be changing in ways that affect the project.
• Monitoring whether the intervention may inadvertently exacerbate conflict or security issues. Related to the above points, MEAL processes can help a project team to understand how the project may inadvertently be contributing to the conflict or a degraded security situation. By monitoring changes in the context and taking note of unintended consequences from the project, teams can proactively identify and respond to these issues.

The proactive, effective use of MEAL can produce a number of other benefits for locally led action on climate, peace and security. These include:

• Mitigating the waste of time and money. Projects with good MEAL processes are less likely to invest an excessive amount of time and money on activities that don’t work. By increasing buy-in for the project, MEAL processes can also reduce the time needed to (re)engage partners, producing efficiencies in project implementation. 
• Providing evidence about what works for donors. As donors and funders seek to maximize their investments in projects, they will look for those that have evidence that their approaches are working and who can demonstrate an orientation toward learning from progress and using adaptive management to increase project effectiveness. 
• Increase buy-in for the project from donors, participants and other actors. Undertaking transparent and relevant MEAL activities, and producing useful outcomes, can increase buy-in for a project as those involved better understand how it is being implemented and the results it is producing. Partners and communities will be more likely to participate or otherwise support the project, and this can improve a intervention’s effectiveness and sustainability.
• Avoiding unintended negative results. If a project is monitoring its results as well as ongoing changes to the surrounding context, it can better anticipate possible negative outcomes and use early warning information to change course as needed. MEAL is thus a critical part of ‘do no harm’ and conflict-sensitive, climate-responsive approaches.

MEAL, done well, is not a checklist mechanism for reporting completed during the final phase of an intervention like some kind of ‘appendix’ to action; rather, it is a system fully integrated into the entirety of a project.

The value of M&E for a project is dependent on the effectiveness of the action plan: the soundness of the project’s theory of change, the robustness of how outcomes and goals are framed, and the integration of M&E considerations in the strategy. However, a well-designed project will benefit from built-in attention and time to monitoring, evaluation and learning, such that the design will change in response to observations and experiences, and the action plan will become even more effective. The MEL intervention cycle is thus a virtuous cycle for conflict-sensitive, climate-responsive intervention planning.

There are many ways to frame and design MEAL. One results-centred approach makes use of the sustainable livelihoods framework to consider five dimensions that underpin sustainable livelihoods — social, financial, human, natural and physical. This impact dimensions approach to locally led action for climate, peace and security suggests that LLACPS interventions will be seeking to improve at least a few of these areas, and that evaluating the change to individual, community and system resilience is an appropriate way to gauge the project’s successes and failures.

From a peacebuilding perspective, the social dimension is particularly important, as well as aspects of the other dimensions considering the relationships and relative power, resource access and wellbeing between different groups, communities and/or the government. From a climate resilience perspective, there are entry points in each dimension; for example: adaptive capacities in the human dimension; better institutional mechanisms for ensuring protection of natural resources in the social dimension; better natural resource management in the natural dimension; more resilient infrastructure in the physical dimension; and access to micro-insurances for drought in the financial dimension. Effective MEAL will (1) identify the most relevant aspects of each impact dimension for the context and (2) develop a strategy for measuring and analysing the project’s impact on those aspects.

A core component in monitoring, evaluation, accountability and learning is defining and tracking indicators: measures that provide a means for assessing results. Indicators are a key tool for confirming whether the project has achieved its intended outputs, objectives, outcomes, and core goal.

Good indicators are CREAM (Clear, Relevant, Economic, Adequate and Monitorable) and focused on SMART project targets (Specific, Measurable, Achievable, Realistic and Time-Bound). Indicators should have designated means of verification: methods for collecting the necessary data such as pre- and post-surveys with activity participants, key informant interviews, tracking databases, media content analysis, etc.

Indicators can track short-, intermediate-, or long-term impacts, and may be qualitative or quantitative. Qualitative indicators are particularly useful in capturing more intangible results and providing narrative explanation of project results, but are difficult to aggregate and are more prone to bias; quantitative indicators allow for comparative analysis and can be attractive as evidential proof, but are only ‘snapshots’ and may lack or overlook nuance. Effective MEAL approaches make use of a range of indicators.

Intervention indicators are specific to the project’s specified outputs, objectives, outcomes, and core goal. For M&E purposes, each individual project aim should have at least one unique indicator, and may have several. Intervention indicators can be used to assess specific activities; cross-cutting indicators speak to several of the project’s intentions and/or phases.

Context indicators provide information about significant changes in the project’s surrounding context (e.g., a widespread health emergency, a major disaster or a change of government), beyond the scope or control of the intervention. Monitoring external realities is important both to inform necessary changes to the action plan and programme implementation and to help situate and explain the project’s final results. To support conflict sensitivity and climate responsiveness, context indicators can be used to update various assessments conducted during the initial scoping phase.

For Locally Led Action on Climate, Peace and Security, some indicators may be both context and intervention indicators: Since related projects seek to impact conflict and stability issues, indicators that provide information about the evolution and development of conflict and stability might simultaneously showcase project success or failure and explain mitigating or confounding circumstances related to project results.

Interaction indicators provide information on how the intervention may be having an effect on tensions in the area. They provide information on the direct interaction between the project and the peace and conflict factors of its context. Gathering such information often requires reflection on qualitative aspects of project implementation and exploring questions linked to communities’ perceptions of the project. Understanding the interactions between perceptions, impacts and tensions is fundamental to a conflict-sensitive and gender-responsive approach, but can be very challenging. As with all information used in a LLACPS project, data and analysis related to interaction indicators should be carefully and respectfully managed.

Indicators and means of verification that might be relevant to Locally Led Action on Climate, Peace and Security projects include:

• the percentage increase in perceived social cohesion, peace and stability in the project area, as verified by pre- and post-perceptive surveys and focus group discussions
• the number and type of environment- and climate-related conflict incidents reported in the target areas (e.g., disputes over land, water, etc.), as verified by an incident tracking database, local security reports, and media content analysis
• the number of supported climate resilience and/or disaster risk reduction strategies, as verified by project reporting and on-site observation
• improved local capacity to manage natural resources (including through improved community participation), as verified by case studies and capacity assessments
• the proportion of people in two different communities who perceive a project as benefiting both communities equally versus one community more than the other, as verified by surveys and focus group discussions
• the percentage of women participating in the locally led action project, as verified by project records and survey data
• the level of engagement of marginalised groups (e.g., indigenous, youth, displaced), as verified by focus group discussions and survey data

Practical guidance about selecting and designing indicators is presented in Module 6.2.

A core principle for best practice in monitoring, evaluation, accountability and learning is starting from the start: It is important that as many of the MEAL processes as possible are designed at the beginning of a project. This will ensure that there are sufficient resources allocated to these processes in terms of time and money and that everyone is on the same page about what is planned. It also ensures that information about initial conditions – that is, the situation prior to intervention – is collected through a baseline assessment. As discussed in Unit 0, understanding the context of an area where locally led action is to take place will support conflict-sensitive project management, but it also serves to inform MEAL. Baseline information captures data on what a situation is like at the start of the project, so that it can be compared to what things are like later on. This is important for evaluating whether and how a project has contributed to any changes.

Even though participatory approaches make it difficult to know exactly which initial conditions to quantify with determined indicators during initial scoping (given that the focal target of Locally Led Action for Climate, Peace and Security is not determined until the overall project has already been ongoing for a fair while through community mobilisation and collaborative analysis processes), a generalised baseline assessment can still help make MEAL more robust – especially if M&E needs were considered during initial scoping analysis, with attention to a broad range of information about the area.

Additionally, developing MEAL processes at the start of a project increases the likelihood that they will be seen as part of the project itself, rather than an “add on” that is of less importance. When MEAL processes are built into and done alongside project activities, they can add value in terms of supporting project objectives (becoming simultaneously the means and the measurement of project success). For example, if a project seeks to build trust and reduce conflict as part of addressing climate needs, transparent MEAL activities can help increase trust – both in the project and between project participants themselves.

While launching MEAL processes at the same time as the overall project is initiated, it is important not to let “perfect” be the enemy of the “good”. Whether a project is halfway done or there are very limited resources to do MEAL activities, some MEAL work is better than none.

If project-specific monitoring information was not collected at the start of the project (i.e., relevant baseline data about particular initial conditions), evidence about change over time can still be gathered by identifying historical information from other sources or by asking people questions about what a situation was like at that time; that information can then be used to conduct MEAL analysis.

If there are insufficient resources for a full-scale evaluation in the traditional sense with designated and carefully tracked indicators, reflective evaluations can still support project management in meaningful ways. Building in reflection and learning sessions focused on sharing and documenting experiences after key project activities, undertaking internal post-event reviews and/or engaging community members in comparative discussions about the changes they have experience and witnessed is a good way to understand what has happened, what has been effective and what needs to change.