Setting priorities for knowledge translation of Cochrane reviews for health equity: Evidence for Equity

The number of reports of systematic reviews of research has increased from about 80 a year in the late 1980s to more than 8000 a year today [1]. This makes it very difficult for decision makers to keep abreast of the latest evidence. The Campbell and Cochrane Equity Group is committed to finding ways of helping decision makers access and use the evidence on interventions that has impact on health inequities. Health inequities are avoidable differences in health outcomes [2]. The importance of equity in health, wellbeing and wealth is increasingly accepted globally, and it underpins research, global development goals and reports, and international declarations [3–9]. The Campbell and Cochrane Collaborations, and other groups, such as the Alliance for Health Policy and Systems Research and the International Initiative for Impact Evaluation (3ie), publish systematic reviews of the evidence for what works and what does not. There has been an increased emphasis on health equity in systematic reviews with the establishment of a Campbell and Cochrane Equity Methods Group (Equity Methods Group), whose members have provided a framework [10] and methods [11, 12] for applying an ‘equity lens’ to systematic reviews.

In 2015, the United Nations created the global Sustainable Development Goals (SDGs); a group of 17 goals to be met by 2030 [21]. The topic areas listed above are still relevant to the SDGs. Goal number 3 addresses all health priorities and includes reproductive, maternal and child health; communicable and non-communicable diseases; as well as access for all to safe, effective, and affordable medicines and vaccines [21]. In addition, goal number 10 is to reduce inequalities within and between countries and focuses on eliminating inequities based on age, sex, disability, race, ethnicity, origin, religion, and socioeconomic or other status.

Systematic reviews on these five topic areas were retrieved through a search in the Cochrane Library (via Wiley (http://www.cochranelibrary.com/), up to 2013, Issue 6) using relevant key words in the title field and limiting the start date to 2008. The exact search strategies are reported in Additional file 1.

Two independent screeners reviewed the results section (Data and Analysis) of the Cochrane reviews to identify: a) any statistically significant difference in mortality; b) for any other categorical morbidity outcomes besides mortality, Odds Ratio (OR) or Relative Risk (RR) greater than 2 or less than 0.5; [22] c) all statistically significant continuous morbidity outcomes (SMD, MD) that when transformed into ORs were greater than 2. Surrogate outcomes and non-statistically significant effects were excluded. Details of the population, intervention, comparisons, outcomes, and effect size were extracted.

Negative effect sizes that demonstrated benefit were converted to a positive value, by reversing the scale for continuous outcomes or by taking the inverse of dichotomous outcomes (e.g. >1). All effect sizes were converted to odds ratios to allow for comparison across reviews using the formulae provided in the Cochrane Handbook [23]. The results are described as the “converted effect size and confidence interval”.

Five “Stakeholder Panels” were assembled to participate in the priority-setting exercise, each addressing one of the five condition-related topic areas listed above. For each panel, a chair(s) was recruited based on their expertise in one or both conditions and in conducting systematic reviews. The chair(s) helped identify and approach five other policy makers and researchers (stakeholders) to join the panel. Members of these panels were purposefully selected to ensure a variety of policymakers (e.g. national, regional, civil society, NGO) from both HIC and LMIC, with responsibility in the topic area of their panel and with interest in evidence-based policy making.

Stakeholder panel chairs reviewed the initial list of potential interventions and outcomes and eliminated: a) those which are no longer used b) those which could not be implemented globally due to prohibitive costs, especially in resource-constrained settings c) interventions whose outcomes were not meaningfully important.

Stakeholder panel members were also asked to note any safety concerns. Finally, they were asked to give an overall rating for each intervention (from 1 to 4 where 1 was the least important intervention and 4 was the most important intervention). Instructions given to stakeholder panel members are provided in Additional file 2.

Lastly, the ratings of all panel members were averaged for each intervention and criteria. We converted the average rating into a score out of 100 for ease of interpretation. This step is different from the CHNRI method which calculates the scores divided by the number of received answers to obtain a percentage of agreement [24]. We ordered interventions according to the average overall rating. We provided these rank-ordered lists to all panel members.

With the realisation that single studies, however large, should not drive policy due to the fact that they may not be replicable [25] there has been an exponential increase in systematic reviews. Research community members, especially those working on reducing health inequities, have a responsibility to inform policymakers and their advisors who make decisions on which systematic reviews should be prioritized for knowledge translation for the benefit of the most vulnerable members of their populations. Such global exercises need to be sensitive to major regional differences in needs and perceived priorities.

Our approach differs from other priority-setting exercises because we chose to focus on prioritizing knowledge translation of completed systematic reviews that have the potential to promote health equity. We also involved those who need and use this evidence with researchers and publishers in order to meet the information needs for those making decisions related to equity. The intent is to provide an international platform to deliver summaries from systematic reviews on interventions that impact on health in disadvantaged populations. The target audience includes policymakers, clinicians, regulators, and the general public. This E4E initiative addresses the criticism that Cochrane reviews fail to draw useful conclusions [26] and instead call for more research by prioritizing reviews with potential for health equity impact for knowledge translation and broad dissemination.

Consensus was successfully achieved in identifying the top group of equity-relevant interventions in each of the five pilot areas. The intent was not to focus on specific ranking; rather, it was to provide a matrix across these five criteria to highlight the importance of health equity in decisions on identifying priority interventions given limited resources. The next step is to meet with the relevant Campbell and Cochrane review groups, and other interested systematic review groups, and explore with them whether and how this process can be incorporated into their own priority-setting processes for knowledge translation, as the Cochrane and Campbell Collaborations are both currently developing knowledge translation strategies for their reviews.

Many Cochrane systematic reviews are focused on intervention efficacy, and equity concerns are often more related to intervention implementation and delivery. Therefore, the evidence in the review may not relate to its actual importance in practice. To address this issue, we asked our stakeholders to consider the feasibility of the intervention, deliverability, universality, and effects on health equity. We did not include non-experimental data on harms in this exercise but will include this information in future updates, when available.

Our methods for this priority-setting exercise are similar to those used by other groups, such as Child Health and Nutrition Research Initiative CHNRI [24] and the James Lind Alliance, which uses priority-setting partnerships to develop priorities for ten intervention uncertainties for consideration by research funders [27]. Our approach also aligns with guidance provided by Lavis et al. for health decision makers (policy and programs) which includes using explicit criteria based on the underlying problem and burden of disease and intervention options [28]. Other papers similarly describe priority-setting exercises for research. These methods include surveys and face-to-face consultations and evidence mapping [29].

Informing decision makers should involve providing an easily understood ‘Friendly Front-End’ [13]. Firstly, this derivative summary must provide information on not only the relative effect or statistical significance alone, but also the absolute magnitude of the benefits as well as potential harms, where relevant. Secondly, for those interventions with meaningful, substantive benefit, policymakers also require guidance on: a) ease of implementation of the intervention, including the available capacity and human resources; b) health system requirements and effects on the health system c) universality – i.e., the magnitude of the burden of illness in the country of interest. Finally, policymakers should be informed about whether the intervention will reduce health inequities. There is very little research available on the types of policy summaries and their impact on policy-makers knowledge and decision-making [30].

This method provides an explicit approach to setting priorities by systematic review groups and funders for providing decision makers with evidence for the most important equity-relevant interventions. Sustainability of this E4E special collection will require partnering with Cochrane and Campbell review groups to continuously identify systematic reviews with potential for important impact on health equity. This could be implemented as part of knowledge translation strategies for these organizations. A first step might be to start with interested review groups, and particularly those covering topics that represent a high burden of disease in low and middle income countries.