Evaluation
Academic research into the evaluation of donor and foundation-support to journalism/media development and its consequences for the independence of beneficiaries.
This resource page is a work in progress. Please get in touch to let us know what is missing using this form.
For non-academic research, reports and articles on evaluation go to
Journal of Applied Journalism and Media Studies - ANDREAS MATTSSON
This article examines the development of journalism in Vietnam by exploring documentation from two media aid projects carried out by the Swedish International Development Cooperation Agency (Sida) from 1993 until 2007. The project documents contain fieldnotes, evaluations and reflections from the trainers who were recruited from Swedish media houses to conduct training in Vietnam. A qualitative document analysis was used to examine the content with a conceptual framework built on notions of comparative media systems, global media ethics and the salience of social connections in Vietnam.
The findings explore how the Swedish media aid intervened in the Vietnamese media by contributing to a technological transition of journalism although the training in newsroom management and media ethics were challenged by conflicting journalism ideology and social norms.
This is one of a series of articles published in the special edition, "What's next for media development" (Volume 11) of the Journal of Applied Journalism and Media Studies.
Journalism, Media & Society - JACOB NYARKO, ERIC MENSAH, BASIL HAMUSOKWE
Media development requires substantial funding, and therefore, donors, foreign governments, nongovernmental organizations (NGOs), and foundations have found a way to play an influential role through their financial support. However, it appears that the donors are also fighting a losing battle considering the rapidly changing political-economic structures of contemporary society spearheaded by the very private sector they enhanced. This study generally presents a systematic review of “foreign aid” to Africa as a base “to explore how donor funders and private investment impact media functions.” The work also sheds light on the extent to which donor support impacted the governance system within the media political economy of Africa. It establishes that donors, who are the very saviors, are also a threat to media freedom because they set the agenda for content, resulting in undue influence on the type of stories that are told. As a result, media development becomes constricted.
Center for International Media Assistance - ANDY MOSHER
How are media assistance programmes being monitored and evaluated? How can such M&E be improved? This report from the Center for International Media Assistance finds that useful tools include: gathering baseline data; content analysis; balancing quantitative and qualitative data; and employing outside evaluators to conduct impact assessments. Donors should increase funding for the M&E of media assistance projects and should help develop a shared but adaptable approach to M&E. Organisations could save money by exchanging non-competitive information (such as baseline data) and should ensure that time spent on project M&E is costed.
Journal of Institutional and Theoretical Economics - ANDREAS FREYTAG & JAC C HECKELMAN
Over the past 20 years, democracy promotion has become a pillar of USAID's mission. We assess the success of US aid by testing whether it is enhancing democracy in a panel of 26 transition countries. Using Freedom House Nations in Transit data, we find that aid has generally not been a significant factor in a country's overall democracy score, but has significantly contributed to certain components of the democracy score, namely civil society, electoral process, judicial framework, and media independence. In addition, the impact of aid is found to depend on the number of years of past central planning.
Development Policy Review - ALEX DUNCAN & GARETH WILLIAMS
Politics often explains where development assistance has been effective and where it has not. Yet, until the 2000s there has been little focus by development agencies on political issues. This has recently begun to change with political-economy analysis (PEA) now being more systematically used by development agencies to understand the real world. Nigeria and Bangladesh are two positive examples. Much remains to be done in these countries and more widely, to ensure stronger uptake of PEA. On the supply side this includes getting the ‘product’ right, and better communicating the message. On the demand side, there is a need to take more account of the incentives facing development agencies and to gather more systematic evidence on the operational impact of PEA to date.
Institute of Applied Media Studies (IAM) of Zurich University of Applied Sciences (ZHAW) - CHRISTOPH SPURK & Dr. CARMEN KOCH
The unit on democratisation, decentralisation and local governance (DDLG) within the Swiss Development Cooperation (SDC) is in charge of this thematic domain, as well as of governance as a transversal theme, and of political economy. It is responsible for enhancing thematic quality of respective SDC programmes, for animating knowledge management and joint learning within the DDLG network, for developing SDC policies and positioning SDC at a global level. As part of the collaboration between SDC/DDLG and the Institute of Development Studies (IDS) at the University of Sussex, Brighton, the Institute of Applied Media Studies (IAM) of Zurich University of Applied Sciences (ZHAW), Winterthur, has been mandated to produce a review and an analysis of the Theories of Change used in the fields of Media and Governance. The analysis should look at Theories of Change used by SDC and by Fondation Hirondelle, one of the main organisations in Switzerland working in media support in post-conflict countries. In addition, the Theories of Change used by other international donors or implementing organisations should be investigated and compared to insights stemming from academic empirical research on media effects in the field of governance. Based on this review, conclusions should be drawn with the aim of enabling donors to improve their decision-making on strategies in future programmes on governance with media components. The review should also help to understand more clearly, how media support projects work, how they aim to achieve their objectives on various levels, and how to develop meaningful indicators and conduct insightful evaluations in the future.
II. Research Questions
The review was planned on one hand as being open for any kind of observations emerging from a comparison of the documents, but on the other hand to be conducted under the following specific research questions:
RQ1: What layers of the media sector are usually chosen in the Theories of Change (for example: journalists, media organisations, legal environment, media economics)?
RQ2: Are there specific associations in the Theories of Change; for example between media and elections, or media and decentralisation, or specific media-related objectives within the governance area; for example, transparency, participation, or dialogue?
RQ3: Are there specific differences in Theories of Change between SDC and other donors or implementing organisations?
RQ4: What are the differences between the Theories of Change used by donors or implementing organisations and the research evidence from academic literature?
RQ5: As a conclusion, is it relevant to develop a holistic Theory of Change for the media environment or, more relevant, to create specific Theories of Change; for example, according to specific objectives (democratisation, civic participation, anti-corruption, peace) or according to layers within the media sector or the governance area?
A useful summary is available here:
Christoph Spurk presented during Theories of change and impact measurment - 15 March 2021 meeting , presentation is avalable here:
Palgrave Studies in Communication for Social Change - JESSICA NOSKE-TURNER
This book argues for an overhaul of the way media assistance is evaluated, and explores how new thinking about evaluation can reinforce the shifts towards better media development. The pursuit of media freedom has been the bedrock of media development since its height in the 1990s. Today, citizen voice, participation, social change, government responsiveness and accountability, and other ‘demand-side’ aspects of governance, are increasingly the rubric within which assistance to media development operates. This volume will appeal to scholars and students of media development and communication for social change whilst simultaneously representing a deep commitment to translating theoretical concepts in action-oriented ways.
Global Media Journal: German Edition - JESSICA NOSKE-TURNER
While some form of evaluation has always been a requirement of development projects, in the media assistance field this has predominantly been limited to very basic modes of counting outputs, such as the number of journalists trained or the number of articles produced on a topic. Few media assistance evaluations manage to provide sound evidence of impacts on governance and social change. So far, most responses to the problem of media assistance impact evaluation collate evaluation methodologies and methods into toolkits.
This paper suggests that the problem of impact evaluation of media assistance is understood to be more than a simple issue of methods, and outlines three underlying tensions and challenges that stifle implementation of effective practices in media assistance evaluation.
First, there are serious conceptual ambiguities that affect evaluation design. Second, bureaucratic systems and imperatives often drive evaluation practices, which reduces their utility and richness.
Third, the search for the ultimate method or toolkit of methods for media assistance evaluation tends to overlook the complex epistemological and political undercurrents in the evaluation discipline, which can lead to methods being used without consideration of the ontological implications.
Only if these contextual factors are known and understood can effective evaluations be designed that meets all stakeholders’ needs.
MORE RESEARCH
For non-academic research, reports and articles on evaluation go to
WHAT IS MISSING?
This resource page is a work in progress. Please get in touch to let us know what is missing using this form.
Last updated
