Short-term Consultant: Final Evaluator, Island Project

Information disorder, including mis-information, dis-information and mal-information, and in particular hate speech, is spreading through conventional media and digital platforms, attacking groups and individuals who are committed to democracy and pluralism. In partnership with local partners in Sri Lanka, Internews implements the Information in Sri Lanka that Advances Democracy (ISLAND) project, which seeks to promote citizen engagement in electoral and democratic process in Sri Lanka by diffusing and exposing mis-, dis- and mal-information, and proving elections-related information quality.

The project has two main expected outcomes:

1. Media and Civil Society Organizations have used fact checking and verification skills they have learned to monitor, expose, and confront elections-related disinformation and hate speech; and

2. Journalists have used journalism skills they have learned to produce accurate, balanced, and responsible media content addressing elections issues.

The following Terms of Reference were developed to guide the evaluation of the ISLAND project, which should take place between June and July 2021. The primary end users of the evaluation will be Internews and donors supporting the project. Persons and organizations interested in developing similar models may also be interested in these findings and recommendations.

EVALUATION PURPOSE AND OJBECTIVES

Internews seeks a consultant to evaluate the performance and success of the project. The evaluation will assess the project’s achievement towards goals and objectives, and look at signs of potential impact of  project activities on beneficiaries, as well as sustainability of resluts. It will also extract learnings and good practices which could be used to enhance future project engagements and record the project’s legacy. The evaluation is expected to result in a series of key recommendations to build on results achieved to date and to guide the next phase of Internews’ work.

The evaluation is intended to assess the implementation of activities, the outcomes achieved, and the challenges faced. The evaluation should be completed with reference to the project’s initial framework and other planning/proposal documents, in line with indicators set out in the project’s logframe as it has been adjusted over time.

The evaluation has the following principal tasks:

• Assess the project impact on practical skills development of local journalists and activists to diffuse and expose mis-, dis- and malinformation, combat hate speech, and effectively report on elections and electoral issues. Focus will be given to the following expected results:
  • Journalists have the tools and know-how to disrupt disinformation by fact checking, verifying information, and investigating disinformation;
  • Journalists and civil society organizations have strengthened capacity to respond to and ultimately reduce hate speech and inflammatory language;
  • Sri Lankan youth are more aware of media and information literacy, informed by both fact-checking findings and an offline campaign; and
  • Journalists have increased capacity to produce higher-quality election-related reporting that is accurate, balanced, and responsible.
• Assess performance of the project in terms of relevance, coherence, effectiveness, efficacy, impact, and sustainability. This also includes an analysis of the performance of project partners and a content analysis of stories produced by the project;
• Analyze underlying factors beyond Internews’ control that affect the achievement of the project results; and
• Provide recommendations for potential future interventions.

EVALUATION SCOPE AND METHODOLOGY

The evaluation should use the Development Assistance Committee of the Economic Cooperation and Development (OECD/DAC) criteria, which is one of the most widely-used criteria to review aid effectiveness. The following questions might guide the project’s evaluation, but are subject to discussion and agreement during the period of evaluation design:

1. Did the theory of change hold true?
2. Has the project responded to the needs of the target groups and continued to do so if/when circumstances have changed? (Relevance)
3. To what extent are lessons learned from what worked well and less well applied to improve and adjust project implementation? (Relevance)
4. To what extent have other interventions (particularly policies) supported or undermined the project intervention, and vice versa? (Coherence)
5. How compatible is the project with other interventions in the country where it was being implemented? (Coherence)
6. Have the right things been done to achieve the project’s aim? Have the planned project objectives and outcomes been achieved (or are they likely to be achieved)? If not, what is needed to close the gap and improve potential for impact? (Effectiveness)
7. Are the resources and inputs converted to outputs in a timely and cost-effective manner? Is the project achieving the planned outputs? Could the outputs be achieved with fewer resources? (Efficiency)
8. What results have been achieved in the implementation of the project? What are the contributing factors to the achievement of the project? What are the main challenges or barriers to achieving the implementation of the project? (Achievement)
9. What are the tangible outcomes (both positive and negative) as a result of the project? Has there been a positive or negative impact on beneficiaries as a whole? How has this impact been ascertained? What specific, real difference has the activity made to beneficiaries? Have certain groups benefited more, or been negatively impacted more? (Impact)
10. Are the results sustainable? If so, how? (Sustainability)
11. How could things be done better in the future? (Future recommendations)

The methodological approach and design of the evaluation will be developed and agreed between Internews and the evaluator. The evaluator will then be responsible for its objective implementation. The methodological approach should be participatory,  inclusive, gender-sensitive and culturally appropriate.  The exact evaluation methodology (including the proposed sampling of stakeholders to be engaged) should be presented by the evaluator and defined, discussed, and agreed upon with Internews during the first week of the evaluation period. The evaluation should include, but not necessarily be limited to, the following tools:

• Desk review of relevant documents;
• Individual and/or group interviews with internal and external stakeholders;
• Online meetings with external stakeholders; and
• Survey questionnaires to internal and external stakeholders, if relevant.

The evaluator should find ways to ease logistical and time challenges and adapt to the respondents’ schedules and needs, to the extent feasible.

The evaluation should abide by Internews’ policies and protocols related to data protection and security, and be conducted in line with ethical protocols including participant confidentiality and privacy. The secure transmission of data is of great importance and must be ensured at all stages of the evaluation. A non-disclosure agreement will be included in the consultancy contract.

DELIVERABLES

The evaluator is expected to submit an evaluation report that documents the evaluation activities and results, following a clear structure. The evaluation will contain the main findings, conclusions, and recommendations as well as a recording of the lessons learned. The evaluator will initially prepare a draft report, which will be submitted to Internews for comments. While considering the comments provided on the draft, the evaluator shall use their independent and impartial judgement in preparing the final report.

The final report should be no longer than 20 pages, excluding the annexes and executive summary. The report must include:

• An executive summary highlighting the main findings, conclusions, and recommendations;
• A detailed methodology section, making explicit any constraints or limitations to the evaluation;
• An analysis of the project performance of the project in terms of relevance, coherence, effectiveness, efficiency, impact, and sustainability;
• An analysis of the technical aspects and main achievement of the project;
• Analysis of the underlying factors that affect the achievement of the project results; and
• Recommendations, being specific and action-oriented, and tailored to all relevant stakeholders (i.e., Internews and the donor).

Please, find more information and apply here.