With the realisation that single studies, however large, should not drive policy due to the fact that they may not be replicable [25] there has been an exponential increase in systematic reviews. Research community members, especially those working on reducing health inequities, have a responsibility to inform policymakers and their advisors who make decisions on which systematic reviews should be prioritized for knowledge translation for the benefit of the most vulnerable members of their populations. Such global exercises need to be sensitive to major regional differences in needs and perceived priorities.
Our approach differs from other priority-setting exercises because we chose to focus on prioritizing knowledge translation of completed systematic reviews that have the potential to promote health equity. We also involved those who need and use this evidence with researchers and publishers in order to meet the information needs for those making decisions related to equity. The intent is to provide an international platform to deliver summaries from systematic reviews on interventions that impact on health in disadvantaged populations. The target audience includes policymakers, clinicians, regulators, and the general public. This E4E initiative addresses the criticism that Cochrane reviews fail to draw useful conclusions [26] and instead call for more research by prioritizing reviews with potential for health equity impact for knowledge translation and broad dissemination.
Consensus was successfully achieved in identifying the top group of equity-relevant interventions in each of the five pilot areas. The intent was not to focus on specific ranking; rather, it was to provide a matrix across these five criteria to highlight the importance of health equity in decisions on identifying priority interventions given limited resources. The next step is to meet with the relevant Campbell and Cochrane review groups, and other interested systematic review groups, and explore with them whether and how this process can be incorporated into their own priority-setting processes for knowledge translation, as the Cochrane and Campbell Collaborations are both currently developing knowledge translation strategies for their reviews.
Many Cochrane systematic reviews are focused on intervention efficacy, and equity concerns are often more related to intervention implementation and delivery. Therefore, the evidence in the review may not relate to its actual importance in practice. To address this issue, we asked our stakeholders to consider the feasibility of the intervention, deliverability, universality, and effects on health equity. We did not include non-experimental data on harms in this exercise but will include this information in future updates, when available.
Our methods for this priority-setting exercise are similar to those used by other groups, such as Child Health and Nutrition Research Initiative CHNRI [24] and the James Lind Alliance, which uses priority-setting partnerships to develop priorities for ten intervention uncertainties for consideration by research funders [27]. Our approach also aligns with guidance provided by Lavis et al. for health decision makers (policy and programs) which includes using explicit criteria based on the underlying problem and burden of disease and intervention options [28]. Other papers similarly describe priority-setting exercises for research. These methods include surveys and face-to-face consultations and evidence mapping [29].
Informing decision makers should involve providing an easily understood ‘Friendly Front-End’ [13]. Firstly, this derivative summary must provide information on not only the relative effect or statistical significance alone, but also the absolute magnitude of the benefits as well as potential harms, where relevant. Secondly, for those interventions with meaningful, substantive benefit, policymakers also require guidance on: a) ease of implementation of the intervention, including the available capacity and human resources; b) health system requirements and effects on the health system c) universality – i.e., the magnitude of the burden of illness in the country of interest. Finally, policymakers should be informed about whether the intervention will reduce health inequities. There is very little research available on the types of policy summaries and their impact on policy-makers knowledge and decision-making [30].
Strengths
Each topic area was co-led by an internationally-recognized “content leader” in the respective content field (i.e., depression, diabetes/obesity, HIV, malaria, and nutrition). Each content leader was teamed up with a Cochrane methodologist with expertise in performing systematic reviews in the same area. Each team was composed of a mixture of researchers and policymakers. The explicit focus on equity was helped by the delineation of the three additional criteria: a) the ease of implementation of the intervention, including the available capacity; b) Health system requirements and effects on the health system c) Universality, or the magnitude of the burden of illness in the country of interest. Consensus was achieved remarkably easily on the assessment criteria. Also, disaggregation of the components contributing to the total score did not show any one of the components driving the total score. This may well be different for a specific country or program where there are political factors and competing programs.
Challenges/weaknesses
It was challenging building the teams as both leaders and team members are in great demand; they are all very busy and typically do not attend Cochrane or other systematic review meetings. There was no financial payment nor academic reward beyond this publication. We initially planned to hold teleconferences but the logistics proved daunting so although we did meet in person or electronically with the leaders, the completion of the worksheets was done asynchronously with the understanding that if there were major disagreements we would set up a teleconference to resolve; however, these were not needed. We had some difficulty getting agreement on the criteria and definitions from our stakeholder panel chairs. As mentioned above, some Stakeholder panel members reported that the wide range of interventions and outcomes made ranking difficult. If the Stakeholder Panels had included different stakeholders this could have changed the priority ranking for some stakeholders. However, since our Panels included diverse individuals and were based on consensus, we feel that the priority lists would have remained similar. Another limitation of our exercise is that we were mostly limited to Cochrane reviews, although the nutrition exercise included some non-Cochrane systematic reviews because the nutrition stakeholder panel chair identified these as interventions with important effects. The other topic areas used only Cochrane systematic reviews. Had additional reviews been included, the results of the exercise may have differed. However, for this exercise we aimed to conduct a priority setting exercise for interventions available in the Cochrane Library.
This paper has focused on the priority setting methods for which the process began in 2013. Since the Cochrane Handbook has not been updated since then, we believe the methods described in this paper would be applicable to the current and future priority setting processes.
Next steps
Our literature will be updated annually and our E4E summaries will be linked to policy briefs provided through Health Systems Evidence (www.healthsystemsevidence.org), a database containing syntheses of research evidence about governance, financial and delivery arrangements within health systems, and implementation strategies. The summaries will also be linked to the International Initiative for Impact Evaluations (3ie) Briefs (http://www.3ieimpact.org/en/evidence/briefs/), which answer policy questions using impact evaluation results.
This priority-setting exercise will be used to identify Cochrane systematic reviews that will be summarized and added to the E4E Special Collection (http://methods.cochrane.org/equity/e4e-series) (Additional file 4: Screenshot of E4E Landing Page and Additional file 5: Screenshot of HIV Topic Landing Page). To date, there are 25 pilot summaries available on the test website. We plan to conduct user testing of the pilot summaries and will revise the summaries based on the results, then develop new summaries for each of the top 10 interventions identified through the priority-setting exercise.