Theories of change in media and governance programmes
THE CASE STUDY
The Swiss Agency for Development and Cooperation (SDC) commissioned research into the use of theories of change in media and governance programmes after a survey of SDC staff who manage media projects showed that measuring effectiveness in media support, building proper M & E systems, and setting realistic expectations was a broadly shared challenge.
Fondation Hirondelle, which began integrating theories of change related to governance into their strategic and project programming around 2017 at the behest of donors, reported that they benefited from participating in the study as it helped identify the gaps, assumptions and factors beyond the scope of working with journalists and newsroom. This evidence helped them become more realistic about what impact is possible in the field of content production and what are reasonable expectations of donors.
The research reviewed:
56 programme documents on media and governance (communication for development programmes were excluded) provided by SDC, Fondation Hirondelle, Free Press Unlimited, BBC Media Action and DW Academy
140 academic journal articles on the empirical studies on media effects
The aims of the research were:
To see if there are similar patterns in the theories of change used in international media support.
To understand the building blocks for media support in the field of governance.
To establish an overview of the research evidence of what actually works in media support, specifically in resource-poor and fragile contexts.
The full report is available to read here:
The Use of Theories of Change in Media and Governance Programmes (2019)
The main findings from the study were:
While all media projects use theories of change, they are mostly a repetition of the logframe.
Media projects should formulate more detailed theories of change, perhaps by using result chains.
Literature review
The study's literature review found only a handful of studies that use reliable methods to measure impact.
These studies found that media development programmes produced very limited evidence of impact in terms of governance, with the exception of some case studies looking at high-profile pieces of investigative journalism.
The study found that it was possible to measure the effect of news exposure on political interest but found no evidence that increased interest led to better governance.
Recommendations
Among other recommendations, the study proposes that the media development sector should look at increased usage of the result chains model as this can help identify what works in the field and what does not.
BREAKOUT GROUP DISCUSSION
Summary
There was broad recognition of media contribution to democratic and societal change albeit in the very long term.
It was also acknowledged that empirically proving a causal relationship between media support and democratic and governance indicators - even with a perfect evaluation and learning process on a project and programmatic level - is a difficult task and may only be possible when taking a much longer view.
Benefits of using theories of change
Some academics felt that expansive theories of change can help policymakers to think about systems in more holistic ways as it considers:
“Who is involved? Who makes change? Who do they collaborate with? Who are the people who are against the change? And so thinking about the context and systems of thinking and people-centred ways.”
Theories of change can also be adaptive and incorporate contribution analysis to understand how much a programme contributed to change.
Using theories of change with 'result chains'
It was suggested that donors should -- while considering the concerns about how and whose theories of change are used -- think more about using 'result chains' to complement theories of change.
The more detailed the theory of change is, the more realistic and relevant to the situation on the ground it has the potential to be.
More detailed theories of change that incorporate 'result chains' are especially important since evidence on what really works in media development is currently limited.
To be effective, theories of change and 'result chains' should be developed in genuine partnership with organisations that will implement, contribute to or benefit from the programmes.
Limits of 'result chains'
While there was a strong case made for focusing on 'result chains' in order to better understand and measure impact, there were words of caution about focusing on measuring impact solely in a linear, cause-and-effect, way. It was noted that a lot of change and a lot of the most important work is not predictable from the outset of a project.
Unrealistic expectations
Some media development practitioners reported that:
They did not feel able to be frank with donors about how big a crisis independent journalism is facing and how hard their work has become. This has led to committing to unrealistic goals in order to access funding.
As a consequence of "shooting for the moon" in order to get funding, change processes are sometimes oversimplified to suit the results frameworks of donors and other aid management formats.
As well as a frank dialogue about what is really achievable, practitioners also urged donors to take a longer-term view.
Macro and micro Theories of Change
There is no “one-size-fits-all” theory of change for media programmes.
Some researchers in the group expressed scepticism about how appropriate general theories of change are for small and short-term media development projects.
Putting a burden of proof for a societal theory of change to individual media support projects was seen as unrealistic.
Donors were advised to avoid having too big expectations when it comes to the impact of media support on governance.
Why donors use theories of change
As well as the benefits of using theories of change noted above, it was recognised that many donors - especially those accountable to the public/taxpayers - will continue to use theories of change as they need to demonstrate how media support is contributing to society and in democratic life to justify support for media development.
This highlights the need for greater understanding between donors, media development practitioners and researchers of their different perspectives about the challenges facing the sector and the choices that we are making based on those different assessments of the problems.
Research collaboration
Operationalise the use of theories of change
There was considerable enthusiasm for more collaborative research with donors and implementers on how to operationalise the use of theories of change in a way that reflects the complex realities of media support.
One suggestion was to look to create different families of theories of change similar to USAID's conflict management and mitigation office manual, which offers different theories of change for different projects in different settings.
Indicators for impact evaluation
There was an interest expressed in working to standardise indicators for impact evaluation across the sector to allow for comparisons and learning from other projects. It was suggested that a critical review of output, outcome and impact level indicators could allow for some of the standardisation to happen.
Some members of the group thought that such attempts might be premature and the sector has some way to go to reach the threshold of common standards and still has a great diversity of strategies and approaches with lots of unspoken assumptions about how change happens in newsrooms and media systems.
References and recommended reading:
Last updated
