SHORT TERM CONSULTANT: FINAL EVALUATOR – AYUH, BELIA!
Ayuh, Belia! project was designed with the goal of advancing freedom of expression in Malaysia by increasing young people’s understanding of their roles as stakeholders in advocating for freedom of expression and strengthening the capacity and collaboration between civil society and youth to play a constructive role in public discourse on freedom of expression.
The project has two main objectives:
- Improve Malaysian youth leaders’ understanding and awareness of their roles in advocating for freedom of expression; and
- Increasing Malaysian youth leaders’ ability to effectively advocate for freedom of expression.
Originally launched in 2019 with implementing partners in Malaysia, Ayuh, Belia! project leveraged Internews’ many best practices through its youth programming in the Asia region (and throughout the world) covering a wide variety of issues, including civic engagement. In line with the requirements set forth by the donor, Internews will undertake the evaluation of the work of Ayuh, Belia! from its inception through June 2021.
These terms of reference were developed to guide this substantive evaluation, which should take place in the period from June to July 2021. The primary end users of the evaluation will be Internews and donors supporting the project. Persons and organizations interested in developing similar models of youth-focused projects may also be interested in these findings and recommendations.
EVALUATION PURPOSE AND OBJECTIVES
Internews seeks a consultant to evaluate the performance and success of the project. The evaluation will assess the project’s achievement towards goals and objectives, and look at signs of potential impact of project activities on beneficiaries, as well as sustainability of results. It will also extract learnings and good practices, which could be used to enhance future project engagements and to record the project’s legacy. The evaluation is expected to result in a series of key recommendations to build on results achieved to date and to guide the next phase of Internews’ work.
The evaluation is intended to assess the implementation of activities, the outcomes achieved, and the challenges faced. The evaluation should be completed with reference to the project’s initial framework and other planning/proposal documents, in line with indicators set out in the project’s logframe as it has been adjusted over time.
The evaluation has the following principal tasks:
- Assess performance of the project in terms of relevance, coherence, effectiveness, efficiency, impact, and sustainability;
- Assess project impact on Malaysian youths’ awareness of their role in advocating for freedom of expression;
- Assess project impact on Malaysian youths’ ability to effectively advocate for freedom of expression;
- Analyze underlying factors beyond Internews’ control that affect the achievement of the project results; and
- Provide recommendations for potential future interventions.
EVALUATION SCOPE AND METHODOLOGY
The evaluation covers the project implementation period from September 2019 to June 2021 and should encompass the entire geographical scope of the project, which is Malaysia.
The evaluation should use the Development Assistance Committee of the Economic Cooperation and Development (OCED/DAC) criteria, which is one of the most widely used criteria to review aid effectiveness. The following questions might guide the project’s evaluation, but are subject to discussion and agreement during the period of evaluation design:
- Did the theory of change hold true?
- Has the project responded to the needs of target groups and continued to do so if/when circumstances have changed? (Relevance)
- To what extent are lessons learned from what worked well and less well applied to improve and adjust project implementation? (Relevance)
- How compatible is the project with other interventions in the region or organization where it was being implemented? (Coherence)
- Have the right things been done to achieve the project’s aims? Have the planned project objectives and outcomes been achieved (or are they likely to be achieved)? If not, what is needed to close the gap and improve potential for impact? (Effectiveness)
- Are the resources and inputs converted to outputs in a timely and cost-effective manner? Is the project achieving the planned outputs? Could the outputs be achieved with fewer resource? (Efficiency)
- What results have been achieved in the implementation of the project? What are the contributing factors to the achievement of the project? What are the main challenges/barriers to achieving the implementation of the project? (Achievement)
- What are the tangible outcomes (positive and negative) as a result of this project? Has there been positive or negative impact on beneficiaries as a whole? How has this impact been ascertained? What specific, real difference has the activity made to the beneficiaries? Have certain groups benefited more or been negatively impacted more? (Impact)
- Are the results sustainable? If so, how? (Sustainability)
- How could things be done better in the future? (Future Recommendations)
The methodological approach and design of the evaluation will be developed and agreed between Internews and the evaluator. The evaluator will then be responsible for its objective implementation. The methodological approach should be participatory, inclusive, gender-sensitive and culturally appropriate. The exact evaluation methodology (including the proposed sampling of stakeholders to be engaged) should be presented by the evaluator and defined, discussed, and agreed upon with Internews during the first week of the evaluation period. The evaluation should include, but not necessarily be limited to, the following tools:
- Desk review of relevant documents;
- Individual and/or group interviews with project beneficiaries and stakeholders;
- Online meetings with project beneficiaries and stakeholders; and
- Survey questionnaires to beneficiaries and stakeholders, if relevant.
The evaluator should find ways to ease logistical and time challenges and adapt to the respondents’ schedules and needs, to the extent feasible.
The evaluation and related research should abide by Internews’ policies and protocols related to data protection and security, and be conducted in line with ethical protocols including participant confidentiality and privacy. The secure transmission of data is of great importance and must be ensured at all stages of the evaluation. A non-disclosure agreement will be included in the consultancy contract.
The evaluator is expected to submit an evaluation report that documents the evaluation activities and results, following a clear structure. The evaluation report will contain the main findings, conclusions, and recommendations as well as a recording of the lessons learned. The evaluator will initially prepare a draft report, which will be submitted to Internews for comments. While considering the comments provided on the draft, the evaluator shall use their independent and impartial judgement in preparing the final report.
The final report should be no longer than 20 pages, excluding the annexes and executive summary. The report must include:
- An executive summary highlighting the main findings, conclusions, and recommendations;
- A detailed methodology section, making explicit any limitations to the evaluation;
- An analysis of the project’s performance in terms of relevance, coherence, effectiveness, efficiency, impact, and sustainability;
- An analysis of the technical aspects and main achievement of the project;
- An analysis of the underlying factors that affect the achievement of the project results; and
- Recommendations, being specific and action-oriented, and tailored to all relevant stakeholders
- Advanced university degree, ideally in development, public policy, social science, or evaluations;
- A minimum of 5 years of experience in designing and conducting evaluations;
- Experience with qualitative and quantitative data collection and analysis methods;
- Experience working on or supporting projects implemented in Malaysia;
- Excellent knowledge of M&E methodologies, including complexity-aware monitoring approaches;
- Sound judgement and ability to objectively evaluate projects in terms of results achieved as well as processes (evidenced through previously conducted evaluations and references);
- Fluency in spoken and written English; Bahasa Malaysia proficiency a big plus;
- Excellent inter-generational communication skills/cultural sensitivity and the ability to forge strong cross-cultural relationships and build trust, as demonstrated through previous experience; and
- Excellent conceptual and analytical report writing skills.
Please, read more about the role and apply here.