Communicating health research (24) Q1. What do we mean by Effective communication of health research? (7) How do we measure it? (2)

8 September, 2022

Dear HIFA colleagues,

In my last message I asked: How might we *measure* the effectiveness of health research communication?

To help answer this question, I looked at the paper by two members of the HIFA Communicating health research group (Rob Terry (TDR/WHO) and Tanja Kuchenmuller (Evidence to Policy and Impact/WHO)):

CITATION: Assessing the impact of knowledge communication and dissemination strategies targeted at health policy-makers and managers: an overview of systematic reviews. Evelina Chapman et al. Health Research Policy and Systems volume 19, Article number: 140 (2021)

The main conclusion was: 'There is limited evidence regarding the effectiveness of interventions targeting health managers and policy-makers, as well as the mechanisms required for achieving impact.'

How did the studies measure 'effectiveness'?

Below are extracts from the full text that (partially) address this question, and a comment from me:


We included outcomes related to the effectiveness of communication and dissemination strategies targeted at managers or policy-makers...

Our primary outcomes were use or uptake of research results, decision-making, adherence to research knowledge (i.e. change in knowledge/awareness) and behavioural change...

Thirteen studies assessed the use or uptake of research results, 14 studies assessed decision-making or changing behaviours, six studies assessed intention to use or apply evidence, 14 studies assessed change in knowledge, and five studies assessed changes in awareness...

Secondary outcomes were those related to understanding, perception and persuasiveness. We considered only objective understanding and not self-reported understanding. Perception referred to how effective an intervention was perceived to be. Persuasiveness considered how likely participants were to make a hypothetical decision in favour of an intervention...

Understanding was assessed by nine studies, perception by seven studies and persuasiveness by three studies, and cost was reported by a single study as a research gap...

Additionally, the included studies assessed outcome measures that were not included in our protocol. These included learning (six studies), attitudes/beliefs (four studies), skills or competencies (three studies), discussion regarding the evidence (two studies), health outcomes (two studies), engagement (two studies), policy changes (one study), value of research evidence (one study), scaling-up of intervention (one study), acceptability (one study), research culture (one study), intention to act (one study), sustainability of evidence-informed policy-making (EIPM) (one study), research coproduction (one study) and credibility (one study).


COMMENT (NPW): The implication is that there are many possible outcomes to consider, and many possible approaches to measurement. As we have discussed previously, there are many aspects to the term 'effectiveness', and the definition of 'effectivesess' will vary from one perspective to another, and from one context to another. Are we able to identify a numerical indicator of 'effectiveness' that can be applied to research communication? It would seem perhaps not. I look forward to hear what Rob and Tanja and others have to say on this topic. Meanwhile I am reminded about a well-known and highly controversial measure - the journal impact factor - which is based on citations.

*If* there is no single numerical indicator of effectiveness of research communication, the implication is that each instance of communication needs to be assessed on a case by case basis, on the basis of whether the communication achieved the desired objective(s).

Meanwhile we can continue to explore the wider questions that frame our discussion:

1. What do we mean by ‘Effective communication of health research to policymakers?’ How do we measure it?

2. What are the different approaches to communicating research (eg academic journals, policy briefs, interaction with policymakers, press releases, social media, infographics, use of video)? What is your experience with these approaches? What works and what doesn't?

3. What is the role of researchers in research communication, beyond publication of their paper? What is the role of other stakeholders (eg communication professionals, editors, media, public health professionals and critical thinkers)

4. What are the needs and preferences of policymakers?

5. What can be done to better support researchers in the communication of health research?

Please do share your experience and thoughts on any of the above, by email to:

Best wishes, Neil

Joint Coordinator, HIFA Communicating health research

Neil Pakenham-Walsh, Global Coordinator HIFA,

Working in official relations with WHO