LogoLogo
- BACK TO THE GFMD WEBSITE+ SUBSCRIBE TO THE GFMD IMPACT NEWSLETTER
  • Welcome to GFMD IMPACT
  • Learning meetings
    • 2025
      • From crisis to opportunity: reimagining media development in the MENA region (Apr. 2025)
        • Meeting Agenda
        • Literature Review
          • Lebanon
          • Palestine
          • Syria
          • MENA region
          • Frameworks, Guidelines and Recommendations
      • Connecting the dots: How to use existing mechanisms to protect media freedom online? (Jan. 2025)
    • 2024
      • GFMD Levant Regional Meeting on Media Support (Dec. 2024)
        • Mapping report
        • Key Recommendations
        • Meeting Agenda
        • Literature Review
          • Lebanon
          • Palestine
          • Syria
          • MENA region
          • Frameworks, Guidelines and Recommendations
      • Post-Summit of the Future Updates and Upcoming Opportunities (Nov. 2024)
      • UNDP–GFMD-SFCG Webinar: Media actors and the (de)construction of the ‘other’ (Oct. 2024)
        • Key insights from the meeting
        • Speakers
        • Literature review
      • Advocating for public interest media and journalism (Aug.-Sept.2024)
        • Donor Workshop: Donor Support for Locally Led Journalism and Media Advocacy (Sept. 2024)
          • Meeting agenda
        • Learning Call: Coordinating policy and advocacy working groups (Aug.2024)
          • Meeting agenda
        • Literature review
      • GFMD Regional Meeting for the wider European region (May 2024)
        • Mapping report
        • Meeting report
        • Meeting agenda
        • Literature review
      • GFMD Latin America & Caribbean Summit (May 2024)
        • Meeting report
        • Literature Review & Case Studies
      • Journalism Cloud Alliance Inaugural Meeting (Apr. 2024)
        • Meeting agenda
        • Speakers
        • Literature Review
        • Press release
      • PRIMED (Feb. - Mar. 2024)
        • Key takeaways
        • Workshop 1
          • Meeting agenda
          • Speakers
          • Literature review & case studies
        • Workshop 2
          • Meeting agenda
          • Speakers
          • Literature review & case studies
        • Workshop 3
          • Meeting agenda
          • Literature review & case studies
    • 2023
      • MENA regional meeting (Nov. 2023)
        • Lit review & case studies
        • Meeting report
      • National Journalism Funds (July 2023)
        • Lit' review & case studies
      • Philanthropy for Ukraine (Apr. '23)
      • Summit for Democracy (March. '23)
        • Media Freedom Cohort: key outcomes and findings
        • Thriving media for resilient democracies
        • Media viability in crisis
        • Other Summit for Democracy Events
      • National consultations on Media Viability (Jan.- Feb. '23)
        • Lebanon
        • Tunisia
        • Namibia
    • 2022
      • Challenges and perspectives for media support programs (Nov. '22)
      • World News Media Congress: Media Viability (Sept. '22)
        • Meeting Report
      • Investing in the future of independent journalism (Sept. '22)
        • Investing in independent media to strengthen democracy
        • Exploring new funding models: thinking brave and bold
        • Editorial sustainability: on ownership and media capture
        • Investing in people
        • Agenda
      • IAMCR 2022 (July '22)
      • World Press Freedom Day (Uruguay, May '22)
        • Principles for effective media assistance
        • Safety of journalists in Ukraine
      • Principles for effective mediadev Belgrade (April '22)
        • Coordination and local perspective
        • Suggestions for improving the support
        • Building digital competences
        • Positive examples
      • MFC Global Conference for Media Freedom (9 Feb. '22)
        • Sustainability & viability
          • How is sustainability defined and measured?
          • Cross-cutting themes
          • Achieving sustainability and viability
          • Initiatives to support media sustainability: an overview for donors and policy
          • Literature review
          • Meeting agenda
        • Renewing the principles for effective media support
          • Process of renewing the principles, development and pathways to implementation
          • Three pillars identified in the consultative process so far
          • Donor perspective- improved partnership
          • Improvements that would benefit the principles process
          • Moving forward with the Principles for Effective Media Development
        • Participation in other MFC events
      • Principles for effective mediadev (Jan. '22)
    • 2021
      • Theories of change & impact measurement (March '21)
        • Meeting agenda
        • Case studies
          • Theories of change in media and governance programmes
          • PRIMED - Protecting Independent Media for Effective Development
          • Measuring the impact of investigative reporting
        • Introduction to GFMD IMPACT
        • Feedback
      • Disinformation - Media support (June '21)
        • Introduction (efficacy of MediaDev responses)
        • 1. Identification responses
        • 2. Responses aimed at producers and distributors
        • 3. Impacting production and distribution mechanisms
        • 4. Supporting the target audiences of disinformation campaigns
        • 5. Evaluating counter-disinformation programmes
          • The Challenges of Evaluating Countering Disinformation Programs: A Working Paper (June 2021)
          • Tips for evaluators
        • 6. Research on disinformation
        • Literature review
        • Feedback
        • Meeting agenda
      • Principles for effective mediadev (Dec. '21)
        • In quotes - What should the principles look like?
        • Feedback
  • Resource centre
  • Activities & services
    • OECD Development Co-operation Principles for Relevant and Effective Support to Media and the Information Environment
    • Help desk
      • Report: Informing the Initial Priorities of IFPIM
      • Literature review on pooled funds
        • IFPIM
    • Learning meetings
  • Policy & Learning Meetings
    • Workshop on Encryption and Media Freedom (June '23)
      • Workshop Report
      • Resources
    • Gender Equality in Media Regulation (May '22)
      • Meeting Report
      • Literature Review
  • RESOURCES
    • Latest data, trends and issues facing journalism and media
    • Guides for donors & funders
    • Declarations
      • Events w/recommendations
    • Policy briefs, working papers & primers
      • Disinformation
      • Gender Equality
      • MediaDev effectiveness
      • Sustainability & viability
      • ... by donors & funders
      • + how to produce a policy brief
    • Academic studies & papers
      • AI, regulation, legislation & design of platforms
      • Disinformation
        • Support to journalism
        • Media & information literacy and fact-checking
      • Ethics & independence
      • Evaluation
      • Gender Equality
      • Investigative journalism
        • Collaborative journalism
      • Journalism, democracy & governance
      • Journalism training
      • MediaDev effectiveness
      • Sustainability & viability
      • Thematic reporting
    • Research & reports
      • AI, regulation, legislation & design of platforms
      • Disinformation
        • Support to journalism
        • Media & information literacy and fact-checking
      • Ethics & independence
      • Evaluation
      • Exiled media
      • Gender Equality in Media Development
      • Investigative journalism
      • Investment & innovation
      • Journalism, democracy & governance
      • Journalism training
      • MediaDev effectiveness
      • Media in fragile contexts
      • Sustainability & viability
        • Business models & start-ups
        • Support for Ukranian Media
      • Thematic reporting
    • Case studies
      • Disinformation
      • Gender Equality
      • Impact measurement, evaluation & learning
        • PRIMED learning questions
        • Measuring social impact
      • Investigative journalism
      • Investment & innovation
      • Research Programmes
      • Sustainability & viability
    • Articles
      • AI, regulation, legislation & design of platforms
      • Data & indicators
      • Ethics & independence
      • Disinformation
        • Support to journalism
        • Media & information literacy and fact-checking
        • Gendered disinformation
      • Gender Equality
      • Investigative journalism
      • Journalism, democracy & governance
      • MediaDev effectiveness
      • Media in fragile contexts
      • Sustainability & viability
        • Business models and start-ups
        • Support for Ukrainian Media
      • Thematic reporting
    • Podcasts
      • Sustainability & viability
    • Evaluation & impact reports
      • Disinformation
      • Emergency & crisis funding
      • Media & information literacy
    • Indicators
    • Data sources
    • Learning tools, interactive maps and applications
    • Glossary
    • Independent Media in Exile
  • OPPORTUNITIES
    • Calls for papers & submissions
    • Events
  • GFMD IMPACT BACKGROUND
    • Consultation (2019)
      • Recipient Perceptions of Media Development Assistance: A GFMD Study (Jan. 2019)
      • Donor feedback on GFMD’s concept for a media development policy Hub - (Sept. 2019)
    • Co-design (2020-2021)
    • Related projects and initiatives
      • Ariadne
      • BBC Media Action research and insight
      • CAMECO
      • CIMA
        • CIMA - Putting Research to Work: Scholars and Practitioners Discuss Greater Collaboration (2016)
      • Civitates
      • Communication Initiative
      • Council of Foundations
      • Dafne
      • Forum Media and Development - FoME (Germany)
      • Global Partnership for Effective Development Co-operation
      • Human Rights Funders Network
      • Humanitarian Journalism
      • Journalism Funders Forum
      • Media Impact Funders
      • Media Impact Project
      • Network for European Foundations (NEF)
      • OECD - DAC Network on Governance (GovNet)
      • Government agencies
  • SUBSCRIBE TO:
    • MediaDev Insider (GFMD IMPACT's newsletter)
    • GFMD's LinkedIn newsletter
  • ABOUT
    • About GFMD IMPACT & FAQs
    • Give feedback
    • GFMD IMPACT regional coordinators
    • GFMD IMPACT pool of consultants
    • About GFMD
    • GFMD coordination
    • GFMD homepage
Powered by GitBook
On this page
  • Theories of change: correlation and causation
  • Draft working paper
  • About the authors
  • 5 key challenges to measuring and evaluating countering disinformation programmes
  • 1. Disinformation is a big, all-encompassing concept.
  • 2. What is a programme’s overall theory of change?
  • 3. Media transparency
  • 4. Trust and truth are hard to measure
  • 5. Making space for non-Western approaches
  • Recommendations
  • 1. Build a community of practice around countering disinformation
  • 2. Carry out an impact evaluation/study
  • 3. Countering disinformation diagnostic tool for program design

Was this helpful?

Export as PDF
  1. Learning meetings
  2. 2021
  3. Disinformation - Media support (June '21)

5. Evaluating counter-disinformation programmes

Summary of the presentations and discussions on "Evaluating counter-disinformation programs" from the June 2021 GFMD IMPACT donor-practitioner-academic meeting on disinformation.

Previous4. Supporting the target audiences of disinformation campaignsNextTips for evaluators

Last updated 3 years ago

Was this helpful?

Theories of change: correlation and causation

Problem: In the breakout group discussions, the issue of attribution correlation and causation was raised as an area where collaborative more work is needed.

Suggestion: Evaluators and researchers should be encouraged to use qualitative research or convenings for discussion that can also be used for outcome harvesting and to further understanding.

The issues around methodological and project and research design questions and how media development projects can measure their effectiveness within theories of change related to governance objectives were discussed in the :

Draft working paper

A was created for the purposes of discussion at the June 2021 GFMD IMPACT donor-practitioner-academic meeting on .

Please leave your comments and suggestions .

Susan Abbott, consultant, co-chair of the Media Sector Development Working Group of The International Association for Media and Communication Research (IAMCR)

Katerina Tsetsura, Ph.D., Gaylord Family Professor of Strategic Communication, University of Oklahoma, Independent Consultant, Research, Measurement, Monitoring, and Evaluation

The following is a summary of the presentation of the draft working paper.

5 key challenges to measuring and evaluating countering disinformation programmes

1. Disinformation is a big, all-encompassing concept.

From an evaluator’s perspective, one of the challenges is knowing what to measure – are we measuring changes to the level of disinformation?

  • Which disinformation is it that we are seeking to counter?

  • What has changed as a result of the programs being implemented?

  • What is a unit of analysis?

  • What or who is supposed to change?

  • Is it an individual person or an institution?

  • What must happen at the societal level?

  • The “disinformation” that’s meant to be countered is often lost in the course of programme implementation.

2. What is a programme’s overall theory of change?

Theories of change are largely underdeveloped. A clear articulation of the change that is supposed to happen as a result of specific inputs is essential.

A simple example:

If we (1) support better journalism and reporting on disinformation, (2) improve newsroom and social media production processes including fact checking and moderation, and (3) train audiences to be literate in assessing media, then we can give the public better tools to interpret and reject disinformation.

In turn, these efforts will help to reduce the impact of disinformation (e.g., on particular topics which were the programme’s focus) and to strengthen disinformation resilience among individuals and communities, a characteristic vital to any democracy.

For more information and recommendations on theories of change see the report from the previous meeting:

3. Media transparency

All the hidden influences that happen in the media can lead to distrust and make it hard to get honest feedback.

Media transparency is a building block for professional media development based on trust between the media and the audience.

The non-transparency practice of journalists and editors worldwide has contributed to losing trust in the media as a social institution in many societies.

4. Trust and truth are hard to measure

Trust is central to any evaluation of countering disinformation efforts.

Countering disinformation is challenging in societies with the fragmented concept of truth, in which polarization and fragmentation, as well as purposely destructive deliberation, perpetuate the fabric of civic discourse.

To what extent can one trust a researcher, an evaluator, a programme implementer in a society of distrust?

How can malign actors utilize the distrust culture to their advantage and turn countering disinformation efforts on its head by diminishing the very nature of societal trust?

5. Making space for non-Western approaches

Disinformation cannot be viewed through the prism of democratic vs. autocratic governments.

What is the extent of our knowledge about countering disinformation in non-western societies?

Who has the right to identify what counts as disinformation and what are the “appropriate ways” to counter disinformation efforts?

Recommendations

1. Build a community of practice around countering disinformation

What? Create a shared vision of what constitutes change.

How?

  1. Set up an open collaborative collective of disinformation researchers.

  2. Develop a shared lexicon of disinformation terminology.

  3. Create shared indicator banks.

  4. Craft common theories of change for countering disinformation programmes.

2. Carry out an impact evaluation/study

What? The media and civil society sector should carry out a comparative, impact study that looks at a variety of countering disinformation efforts to see what has been done thus far and what has been achieved.

How? It should be external and independent. Data collected can inform baseline understandings for future work in this space.

3. Countering disinformation diagnostic tool for program design

What? Produce a diagnostic tool that can be used by the wider community of practice seeking to design civil society and independent media-led countering disinformation programmes.

How? Leverage collective wisdom and understanding that can be used as a construct for carrying out assessments used to develop and design countering disinformation programmes: pressure points, contextual factors, key areas of concern.

Non-transparent practices can be found worldwide (): Media bribery, envelope journalism, or paid news, and media opacity. (Many practitioners around the world also use slang words to refer to this phenomenon: in Russia, in Ukraine, in Peru, and in the USA.)

The notion of truth is increasingly contested, and, as a result, the levels of societal and institutional trust are at an all-time low ().

GFMD IMPACT March event on Theories of change and Impact measurement
Theories of change & impact measurement (March '21)
draft working paper
disinformation
directly in the draft document
About the authors
Theories of change & impact measurement (March '21)
Tsetsura & Kruckeberg, 2017
zakazukha
jinsa
mermelada
pay-for-play
Edelman Trust Barometer, 2021