Module 6.3. Participatory Methods

M&E is perhaps the aspect of program management least likely to conventionally include participation, as it is often left until the end, improperly budgeted and completed by external consultants. But participatory approaches can enhance the conflict sensitivity, climate responsiveness and capacity building potential of a project.

As with every component of Locally Led Action on Climate, Peace and Security, collaboration will improve monitoring, evaluation, accountability and learning processes. Module 6.3 considers how LLACPS can integrate participatory approaches to MEAL in projects. It will address:

This module will help you to:

• integrate participation in each step of a LLACPS’ project monitoring, evaluation, accountability, and learning processes
• deploy collaboratively designed surveys as part of ongoing data collection
• engage focus groups and talking circles as sources of monitoring data as well as methods for evaluative analysis

BENEFITS OF PARTICIPATORY MEAL

Monitoring, evaluation, accountability and learning processes will be most effective when they are approached collaboratively. This involves multiple people from diverse groups and backgrounds helping to design and implement MEAL: determining indicators, providing narrative explanation to observed patterns of change, etc. Participatory MEAL might involve:

• Discussing the types of evidence that various groups think would verify whether the project is going well. Different actors will have different understandings and perceptions about which sources of information are accurate, trustworthy and/or valid. Community members may also have a particular way they wish to see project-related data presented or made relevant to them. Creating custom indicators through consensus that reflect the specific context can serve to include a range of epistemologies and worldviews. (See, for example, the Everyday Peace Indicators methodology.)
• Training community members to administer surveys and collect other forms of data pertinent to the project’s monitoring needs.
• Inviting those affected by any unforeseen negative side effects of the intervention to share their experiences, perspectives and needs, working with other community groups and the project manager to address the problem.
• Co-developing learning questions and evaluation frameworks, providing space for a range of participants to address goals and critiques of the project in constructive ways.

Engaging community members in data collection and analysis simultaneously develops local capacity and helps identify key insights for the project. Participatory MEAL can contribute to increased buy-in for the project, more robust learning and strengthened adaptive management. Additionally, community members who have gained experience and awareness of the project by engaging in MEAL processes are better able to support the project after the official LLACPS project cycle ends – enhancing the sustainability of project results.

challenges FOR PARTICIPATORY MEAL

Collaborative approaches to monitoring, evaluation, accountability and learning are not easy. It is time-consuming and expensive to build relationships, create trust and gather communities. Ensuring that community members are able to accurately administer and analyse surveys requires time, preexisting skills and/or training sessions, and robust data security to ensure that respondent privacy and safety are maintained.

And while participatory approaches can inform conflict sensitivity, certain aspects of participation may also lead to or exacerbate tensions, if not well managed.

If community members are invited to participate in the design of a project’s M&E approach but do not feel that the implemented processes reflect their ideas, this may lead to tensions and a lack of buy-in. If diverse groups provide input about which indicators should be used and what kinds of evidence are valid, but disagreement arises between them about validity or priorities, this can contribute to animosity or distrust. Expectation management, skilled facilitation and transparent decision-making are key.

Beyond MEAL design, LLACPS projects should engage participation in the learning processes themselves. Once evidence is collected by monitoring and evaluation activities (such as a collaboratively designed and community-administered perception survey), initial results should be shared with project participants for reflection and discussion.

However, raw data may not be helpful to share, and sensitive information should be carefully handled. If, for instance, certain results indicate that one group benefited from a project more than another, publicising that information might provoke discord between the two groups. Additionally, ‘spoiler actors’ – those who do not wish the project to succeed – may be motivated to retaliate if they see a positive report about project progress. The possibility of these scenarios does not mean that M&E results shouldn’t be collaborative and open; rather, they emphasise the need to engage in iterative dialogue with diverse actors, be aware of potential pushback and develop strategies for managing project-related tensions in order to avoid unintended negative consequences. 

Surveys are a widely used method for data collection. A well-designed survey can gather information about a wide range of topics, and be administered by non-experts – which means they offer an entry point for community members to be actively involved in monitoring.

Any number of questionnaires might be useful for LLACPS projects. Activity surveys can be circulated at the end of events to learn what attendees thought about it. Paired retrospective surveys can measure what participants in a training knew about a topic before and after the workshop. Perception surveys ask community members to share their feelings and impressions about particular groups and/or issues.

In M&E terms, surveys often serve as a verification source for indicators. Participatory survey design involves working with community members to collaboratively develop questions that are appropriate to ask and will provide useful data about the status of selected indicators. Working with a group to write questions will help ensure the prompts are clear to a variety of people, respectfully worded for the local context and likely to elicit helpful responses. The Everyday Peace Indicators worksheet can be independently completed by multiple people with discussion and deliberation after each main step as a structured, inclusive way to approach participatory survey design.

It’s a good idea to involve a range of question styles in a survey: multiple choice, ranking, rating, short phrase responses, and more narrative, open-ended queries. When determining the best question type, bear in mind the evaluation processes: Which types of answers will make confirming the indicator as easy and convincing as possible? Generally, surveys should include quite a few multiple choice questions that are quick to answer and easy to quantify, and just a few qualitative responses: Focus group discussions can help provide qualitative nuance in a more structured format.

The length of the questionnaire should also be carefully considered: While it is helpful to gain extensive insights, M&E processes shouldn’t feel overly burdensome to data collectors or survey respondents – and an excessive amount of data can make analysis unwieldy.

In order to evaluate how a project has engaged and affected various groups differently, it is important to collect information about the respondents themselves. Surveys should thus include a respondent profile section, asking about age, gender, ethnicity, livelihood, occupation and the like. Which aspects of identity will be relevant depends on the indicator and the context. Remember to:

• Treat information sensitively. Where appropriate and possible, surveys should be anonymous; if not, data collectors should clearly explain to participants how their information will be stored and used.
• Avoid collecting data that will not be used. If you ask respondents to share their age, at least some part of the evaluation report should disaggregate results by age.

Monitoring and evaluation is often assumed to be a quite quantitative and bureaucratic process, but in reality, discussion is one of the greatest tools we have for understanding what is going well in a project and what needs to change. Conversation and narrative help to ‘humanise’ the data side of M&E, giving context and purpose to numerical analysis. Facilitated dialogue can be used both as a source of information and a decision-making structure.

Focus groups comprise a set of individuals who come together to provide information and/or feedback about a particular issue. The group should be representative of the community’s diverse identities and interests while being small enough to easily hold a shared conversation. Members of a focus group might be intentionally selected to include persons such as community leaders, interest group spokespersons and local politicians; alternatively, random selection or open invitation can be used to garner a wide range of voices. However the focus group is formed, it is important to recognise the biased perspectives of those who are in the room, and to acknowledge who isn’t in the room. Multiple, diverse focus groups can make the results more robust; this must be balanced with the time and resources required to host quality discussions.

Talking circles are a particular approach to facilitation in which groups are seated or otherwise gathered in a circle to convey equality and respect among participants. Circles might be used as a space for community reflection on how the project will be evaluated and on appropriate methods for collecting monitoring information; they might also be convened to provide feedback on the findings of an evaluation and summarise what the project has learned. Because they bring people together in conversation, talking circles can be a way to build understanding and relationships. However, as with all participatory processes, they must be done in a conflict-sensitive way to avoid aggregating tensions or putting someone’s safety at risk.

Facilitated dialogue can also be used within the core project team: participation isn’t only about engaging external parties, but rather about ensuring multiple voices are heard at every decision-making point. For example, someone on the team might have lead responsibility for MEAL, but that doesn’t mean they should be the only ones determining metrics and verification sources, assessment procedures, etc. A collaborative approach to finalising indicators might involve facilitated dialogue, following a process such as:

• Gathering a group to consider a proposed set of indicators (which might happen after people have individually completed Steps 1-3 of the Everyday Peace Indicators worksheet).
• Discussing each indicator individually, with the facilitator encouraging specific examples or stories that illustrate the benefits or challenges of the indicator.
• Dedicating more time to any suggested indicators that elicit both strongly positive and strongly negative reactions, making use of breakout groups to brainstorm options before the full group makes a final decision on whether to include or discard the potential indicator. The smaller teams might:
  • Consider alternative ways to express the proposed issue.
  • Explore whether it might be integrated into another indicator.
• Addressing whether any indicators should be rewritten/reframed to support community understanding and thus increase engagement.
• Ensuring that everyone feels that the needs and perspectives of those they know are represented by the indicators, agreeing as a group upon the final list of indicators as a set.
  • ‘Temperature checks’ like having everyone in the group put their thumbs up, down, or sideways for each indicator can tell the facilitator how close the group is to consensus on a particular topic.