Ethiopia: Experienced Evaluation Team for Mid-Term Review

The Life & Peace Institute (LPI) is an international centre based in Uppsala, Sweden, that supports and promotes nonviolent approaches to conflict transformation through a combination of research and action. In the Horn of Africa and Great Lakes regions, much of LPI’s work is carried out through engagement with, and support to, civil society and academic institutions, building strategic partnerships with national, regional and international organisations and networks.

LPI is now looking for an experienced evaluator or evaluation team to conduct a mid-term review of its 2017-19 programme in Ethiopia.

Below is an abridged version of the ToR for the Mid-term Review. Please contact charlotte.booth@life-peace.org should you want to have the full version.

Summary of the programme

LPI has supported peace work in Ethiopia since 1991. Through partnerships with traditional leaders, NGOs and church-based institutions, several locally driven peacebuilding initiatives have been supported over the years. Starting in 2009, LPI has worked in partnerships with universities and the Peace and Development Centre (PDC) in facilitating Sustained Dialogue (SD) on university campuses.

The 2017-2019 programme was developed based on past lessons learnt, with the goal: To contribute to reducing negative relational social patterns and support a culture of trust and collaboration between otherwise segregated or conflicted social groups by facilitating dialogue, research and collaborative action on local level and engaging relevant decision-makers and policy implementers to enhance structural conditions for peace.

The programme extended the SD project to several additional universities, and is currently being conducted in Jimma, Haramaya, Gondar, Ambo and Bahir Dar universities, reaching a total of 6,919 students, directly and 35,680 students and university communities, indirectly. In addition to SD, the programme has also worked on applying a Conflict Sensitivity Approach to land administration and management in East Hararghe Zone (CSLA), located in the Oromia National Regional State.

Mid-term Review Purpose: Learning and informing decisions for enhancing performance

The purpose of the mid-term review is to collect and analyse evidence on programme implementation and results, in order to identify significant lessons from past experience and contribute to improving the Ethiopia programme’s relevance and effectiveness for the remaining implementation period.

Evaluation Objectives (EO)

EO 1:To assess the extent to which the strategy, objectives and activities of the Ethiopia programme are relevant to the current situation and stakeholder priorities. (Relevance)

EO 2:To assess to what extent the Ethiopia programme and the key programme model implemented – Sustained Dialogue – have achieved objectives including potential for sustainability. (Effectiveness, Sustainability)

EO 3:To assess how systems for programme monitoring and evaluation allow for measuring change, learning, accountability and adaptive programming

Type of evaluation (timing): Formative evaluation, mid-term into the implementation.

Specific evaluation questions

Evaluation Objective 1 related to relevance:

Evaluation question (EQ 1) Overall Relevance: Is the Ethiopia programme working on the right issues in this context at this time? In particular are SD and Conflict Sensitive Land Adminstration (CSLA) appropriate and relevant in the current Ethiopia context? As CSLA is in the early stages of the project, the evaluation should consider the extent to which the project design and participant engagement respond to the context as the context is evolving; as well as capture early stakeholder perceptions. Data collected should be utilized to revise/improve/clarify theories of change moving forward and make adaptions e.g. regarding inclusion of stakeholders.

EQ 2 Context: What are the possible scenarios for SD work in universities looking forwards given the current environment of political uncertainty?

EQ 3 Relevance according to stakeholders: What is the relevance of SD and CSLA to date as perceived by key stakeholders, including students, partners (including university administrations, student-led fora/organisations), relevant local population and external observers? What are the implications for the programme?

*EQ 4 Participant engagement:*Do participant engagement strategies in the programme allow for a diversity of stakeholders to participate in the SD process, Peace Incubation and CSLA? Who is not reached, but should be, in light of theories of change?

Evaluation Objective 2 related to effectiveness:

EQ 5 Progress: To what extent is SD programming successfully progressing towards their expected outcomes, indicators and current targets as stated in its results framework and defined in the project’s theories of change? What are the factors at the macro level (i.e. current political situation) and the micro level that are impacting on progress? Are there differences between universities and locations? How are the assumptions made in the theories of change manifesting and how can the theories of change be improved? What changes should be made for the SD process to achieve its targeted objectives?

*EQ 6 SD as an inclusive, youth-led process:*How have institutional arrangements affected the degree of influence of students and student bodies over the process? How inclusive are SD sessions and peace actions? What promotes or hinders inclusiveness?

*EQ 8 Moderator capacity and leadership:*To what extent do moderators take ownership and leadership of the SD process? To what extent does the SD process equip them with the skills to implement the activities they are responsible for? In what way does moderator performance (how they uptake the knowledge into their practice as moderators) affect the effectiveness of dialogues?

*EQ 9 Engaging Opinion Leaders:*Is the SD project engaging the right and sufficient number of opinion leaders to influence the larger student community? What are the motivations for their participation? How has the programme affected these opinion leaders, including their attitudes and behaviours and relationships with their peers and the wider environment? What has the project learned in regards to its theory of change for including opinion leaders?

EQ 10 Peace Actions: What have been the results from peace actions, in terms of behavior changes of SD participants and in the campus community more widely? What have been challenges for effectiveness and what lessons can be drawn for future peace action programming?

EQ 11 Peace Incubation: What have been the lessons learned to date concerning the design and implementation of the peace incubation activities?

EQ 12 Partnerships: How has partnership between LPI and PDC functioned in the SD and CSLA projects? Do PDC feel they have received the appropriate support from LPI? How can this be improved? How has partnership worked on the campuses (with administration and student organisations, for instance)?

EQ 13 Sustainability and ownership: To what extent has the programme made progress in promoting the sustainability of the SD process? What have been constraints for building sustainability? What are recommendations moving forward?

Evaluation objective 3 related to quality of the programme learning system:

EQ 14 Monitoring, Evaluation and Learning Processes: Are the Ethiopia programme’s Monitoring, Evaluation and Learning (MEL) processes feasible and useful for fostering accountability and learning, both for LPI and PDC?

Approach of the Mid-Term Review

The evaluation process will use a utilisation-focused evaluation approach, which entails the following, at the minimum:

· Desk review of relevant programme documents and potentially supplemented by a broader review of relevant sector literature

· Collection of primary external data from the field from consultations/interviews with a broad range of stakeholders

· Collection of data from programme and partner staff

· Site visits to selected project locations, in order not only to better understand the context in which the programme operates but also to hear direct feedback from programme participants and communities.

The evaluation should take gender and intersectionality perspectives into account throughout all stages of the evaluation, including in design (evaluation questions), data collection (including ensuring diversity of gender, age, clan, socio-professional, geographical origin) and data analysis (including analysing data for sub-groups of evaluation participants to identify any patterns and significant differences between groups).

The evaluation approach should further include proactive measures to ensure conflict sensitivity throughout all stages of the mid-term review, particularly connected to data collection work where there have been enhanced tensions recently.

Further, the evaluation shall be participatory in approach, which entails:

· LPI and PDC teams shall be involved in evaluation design, particularly in validating any decisions on evaluation scope, translating evaluation questions in data collection tools, participant engagement, and feedback to findings, conclusions and recommendations.

· Preferable would be an interactive evaluation data analysis workshop facilitated by the evaluation team leader.

· Findings and recommendations shall be shared in an interactive utilisation workshop.

Scope

The Mid-Term Review will cover all activities implemented under the programme framework for the time-frame of January 2017 to December 2018.

The Mid-Term Review will make use of existing monitoring data, including baseline and endline studies.

Qualifications and Evaluation team

The Mid-Term Review should be conducted by an experienced evaluator or evaluation team.

The Evaluation Team should demonstrate the following skills, qualifications and characteristics:

· Demonstrated, in-depth experience in the design and implementation of reviews and evaluations in the field of peacebuilding.

· Experience from engaging with dialogue processes.

· Experience with youth-led processes or youth as agents for change in peace processes as the local level is considered a merit.

· Possibility and willingness to access project locations and engage a diversity of project stakeholders (at least one team member).

· Strong analytical skills and understanding of the Ethiopia context, including the environment for civil society engagement.

· Ability to apply gender- and conflict-sensitive approaches in data collection and analysis.

· English proficiency and excellent track record in producing high quality and utilization-focused reports.

· At least one team member must have Amharic and Oromifa language proficiency.

Deliverables

· Inception report and negotiated evaluation plan that details lines of inquiry, definitions, data sources, methods for data collection, methods for data analysis, conflict considerations and time and budget planning.

· Data collection instruments and protocols.

· Draft evaluation report for comments and review.

· Final copy-edited evaluation report following review (no more than 30 pages excluding annexes), including a stand-alone executive summary of no more than 5 pages.

Implementation information

Time-line:The exact time-lined will be agreed upon as part of the inception period. The Mid-Term Review is planned to conduct preparatory steps in January 2019 and data collection in February/early March. The report should be finalised for sharing by the end of March.

Locations and travel:Data collection will require travel to key programme and project locations, in close collaboration with the LPI team and partner organisations.

Activity planning:The detailed activity plan and budget for the Mid-Term Review will be agreed upon based on the methodology proposed in the technical proposal.

Fees

LPI will pay a daily fee for an agreed number of payable days. The number of days will be agreed upon informed by the technical proposal of the evaluation team. The fee will be subject to negotiation.

Please send CVs of all Evaluation Team members, along with your technical proposal (describing your understanding of the ToR, detailing methodology for implementing the ToR, conflict sensitivity considerations, activity schedule and a detailed budget outline for fees and reimbursable costs incurred by the Evaluation Team (e.g. for travel of the evaluation team) to charlotte.booth@life-peace.org by 18 January 2019.

The CVs and Technical Proposals will be assessed on: qualifications and experience of the evaluation team, quality of the methodology proposed and cost effectiveness (value for money).