- Systematic review
- Open Access
Does Indigenous health research have impact? A systematic review of reviews
International Journal for Equity in Health volume 16, Article number: 52 (2017)
Aboriginal and Torres Strait Islander Australians (hereafter respectfully Indigenous Australians) claim that they have been over-researched without corresponding research benefit. This claim raises two questions. The first, which has been covered to some extent in the literature, is about what type(s) of research are likely to achieve benefits for Indigenous people. The second is how researchers report the impact of their research for Indigenous people. This systematic review of Indigenous health reviews addresses the second enquiry.
Fourteen electronic databases were systematically searched for Indigenous health reviews which met eligibility criteria. Two reviewers assessed their characteristics and methodological rigour using an a priori protocol. Three research hypotheses were stated and tested: (1) reviews address Indigenous health priority needs; (2) reviews adopt best practice guidelines on research conduct and reporting in respect to methodological transparency and rigour, as well as acceptability and appropriateness of research implementation to Indigenous people; and (3) reviews explicitly report the incremental impacts of the included studies and translation of research. We argue that if review authors explicitly address each of these three hypotheses, then the impact of research for Indigenous peoples’ health would be explicated.
Seventy-six reviews were included; comprising 55 journal articles and 21 Australian Government commissioned evidence review reports. While reviews are gaining prominence and recognition in Indigenous health research and increasing in number, breadth and complexity, there is little reporting of the impact of health research for Indigenous people. This finding raises questions about the relevance of these reviews for Indigenous people, their impact on policy and practice and how reviews have been commissioned, reported and evaluated.
The findings of our study serve two main purposes. First, we have identified knowledge and methodological gaps in documenting Indigenous health research impact that can be addressed by researchers and policy makers. Second, the findings provide the justification for developing a framework allowing researchers and funding bodies to structure future Indigenous health research to improve the reporting and assessment of impact over time.
The nature, quality and value of research evidence in positively impacting and raising Indigenous health status can justifiably be questioned when significant gaps in health outcomes between Indigenous people and benchmark populations persist globally [1, 2]. Questioning the way researchers do business is also being challenged by Indigenous people who continue to voice concerns about being over-researched without corresponding improvements in their health . Relatively unchanged long-term health disparities for Indigenous populations strongly suggest that despite the plethora of health research, corresponding research impacts or benefits for Indigenous people are, at best, modest. As researchers, we have a moral obligation to be responsive to need and ensure that we deliver high impact research evidence from which societal benefits can flow. Research end-users such as policy-makers and practitioners need to know how much confidence they can place in research evidence to make smarter informed healthcare decisions . In tight contemporaneous fiscal environments, ensuring quality evidence also makes sense from a cost perspective. In this paper, we focus on the quality of evidence reported in systematic reviews of Australian Aboriginal and Torres Strait Islander (hereafter respectfully Indigenous Australians) health research as a necessary prerequisite to effecting societal impacts and benefits. A systematic review of reviews on Indigenous Australian health research was conducted to assess the quality of research evidence and the extent to which impact and benefits were reported.
Following Bainbridge et al.  and for the purpose of this review of reviews we consider “research impact” and “research benefit” as discrete terms that are interrelated and interdependent. Research impact is defined as any area of influence flowing from the research endeavour including those that flow from research processes; these can be both positive and negative. Research benefit flows from areas of impact. In Indigenous health research, benefit is broadly defined as any elements of research that are advantageous or good; such as strengthening capacities, opening opportunities, or improving health outcomes that progress the interests that are valued by Indigenous people . Benefit can be intended or unintended, inside or outside the immediate research environment, direct or indirect, tangible or intangible, immediate, short-term or longer-term; but benefit must be positively oriented and represented as elements of value derived from research . Herein we use the term “impact” as the first step for attaining benefits, but look at its positively oriented nature, as in research benefits.
Silburn et al.  noted that the likelihood of favourable research impact and thus benefits is related to the ethical implementation of Indigenous health research processes. Maximising research impact is more probable when: 1) Indigenous people are direct and active participants; 2) issues of relevance to Indigenous people are identified and prioritised; 3) Indigenous peoples’ knowledges and perspectives are incorporated in processes and findings; 4) reporting of findings are meaningful; and 5) all potential end-users are engaged from the outset. Yet, it is unclear whether research is implemented ethically or whether the abundance of research conducted, purportedly to improve Indigenous health, is justified and produces quality evidence that benefits Indigenous people in ways that are meaningful and valued by them .
Aim and hypotheses
This systematic review of reviews extends the author team’s prior work to elucidate the reporting of Indigenous research impact . It synthesises evidence from multiple reviews in a variety of settings to provide information concerning the extent to which impact is examined across different health areas, programs and interventions. Based on the domains of the Research for Impact tool  developed for Australia’s national Indigenous health research organisation, the Lowitja Institute, a set of three a priori hypotheses were developed to test the extent to which published reviews make transparent the impact of research studies on Indigenous health that are included in their reviews. The Tool provides a step-by-step research logic framework designed to increase the value of research in Indigenous Australian health. It is based on the assumption that the implementation of the six objectives in the framework will lead to demonstrable short, medium and longer term impact. The performance or impact of research projects identified in this review of reviews were evaluated against these six objectives: 1) promoting Indigenous leadership and participation; 2) ensuring rigorous and transparent research priority setting; 3) ensuring the study type and design are appropriate to the research question(s); 4) ensuring respectful and rigorous implementation; 5) facilitating knowledge translation into policy and practice; and 6) determining the impact of the research versus its costs. Consistent with these objectives, the hypotheses developed for this systematic review of reviews are:
Reviews address Indigenous Australian health priority needs;
Reviews adopt best practice guidelines on research conduct and reporting in respect of methodological transparency and rigour, as well as acceptability and appropriateness of research implementation to Indigenous Australian peoples; and
Reviews explicitly report the incremental impacts of their included studies and translation of research.
The overarching intent of the systematic review of reviews is to influence the current research reporting and accountability frameworks and expectations towards greater focus on improving the quality of evidence and reporting research impact. This study complements other review of reviews of Indigenous health [7, 8] by focussing explicitly on the impact of research. It also extends the search scope from peer-reviewed to policy initiated reviews (resource sheets), considers reviews’ output over time, and provides characteristics of included studies and institutional affiliations. The review therefore provides researchers, policy-makers, health practitioners and other decision-makers with directions towards greater focus on reporting and demonstrating research impact–the knowledge they need to implement evidence-informed health and wellbeing improvements.
In recent years, health research has been assessed through reviews of the literature. Reviews help to make evidence more accessible by bringing research together in one place . We worked from the premise that judgements about the quality of evidence could best be assessed through systematic review methods. We followed the established PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines  for designing and preparing the manuscript. An a priori protocol defined the eligibility criteria, information sources and search strategy, outlined the data collection process, synthesis of results and quality appraisal strategies.
Studies included: (1) any type of review, for example literature review or meta-analysis or meta-synthesis, meta-ethnography or narrative synthesis, systematic search or bibliometric analysis; (2) focused on Indigenous Australians’ health and wellbeing; (3) published between 1995 and 2014 (inclusive); (4) in the English language; and (5) available as full-text manuscripts.
Indigenous health and wellbeing was defined broadly as “the social, emotional and cultural wellbeing of the whole community in which each individual is able to achieve their full potential as a human being, thereby bringing about the total wellbeing of their community, but not just the physical wellbeing of an individual” . This broad definition meant that studies were included if they addressed a health-related process such as mentoring, health promotion, or a determinant of health such as education or housing initiatives, provided they met all other criteria. Reviews that were international in scope and crossed over Indigenous and tribal populations across various countries (n = 8, 11%) and reviews that considered Indigenous people as a part of a rural or a socially disadvantaged population (n = 3, 4%) were also included, if they reported results separately for Indigenous Australians.
As a quality indicator, this systematic review of reviews was limited to peer-reviewed literature. However, it was acknowledged that grey literature “may contain rich texts and their inclusion may avoid publication bias” . With this caveat in mind, one pertinent sub-section of the peer-reviewed grey literature, known as Closing the Gap Clearinghouse, was included. Resource sheets and issues papers in the Clearinghouse contained evidence on narrowly defined topics commissioned by the Australian Institute of Health and Welfare and the Australian Institute of Family Studies (https://aifs.gov.au/). The aim of the Closing the Gap policy was to achieve equality in health status and life expectancy between Indigenous and non-Indigenous Australians. Resource sheets were subjected to peer review processes and considered to be of high relevance to Indigenous policies and programs.
An accredited librarian working at a leading Australian university designed and carried out a comprehensive search strategy of: (1) 14 electronic databases: Informit, Indigenous Australia, Indigenous Studies Bibliography: AIATSIS, APAIS-ATSIS, FAMILY-ATSIS, Informit Indigenous Collection, ATSIHealth, Campbell Library, EBM Reviews/Cochrane DSR/ACP Journal club/DARE, PsycINFO & PsychARTICLES, Medline, Ovid, CINAHL, Sociological Abstracts, Australian Indigenous HealthInfoNet; and (2) the Closing the Gap Clearinghouse website with a catalogued repository of evidence based resource sheets.
The following terms were searched in either title, abstract or MESH heading: systematic AND Australia AND review AND health AND (Aborigin* OR Indigenous OR “Torres Strait”). MESH terms comprised meta-analysis, oceanic ancestry group, Indigenous health services, Australia; subject terms included meta-analysis, Indigenous peoples OR Aborigines and health. The database search was performed in December 2014. The search was conducted separately for each database and use database specific search strings.
Discussions with experts established that systematic reviews in the field emerged as a common research type only in recent years. Therefore, selected period from 2004 to 2014 (inclusive) and from 1995 to 2014 for Medline were considered appropriate to provide comprehensive coverage of relevant reviews. The search of the Closing the Gap Clearinghouse website was conducted in May 2015.
Data collection and analysis
Information was extracted from each included review for: first author, year, title, institutional affiliation and country of the first author; type of review; period searched (P), number of studies (K) and type of studies analysed; health area; target group: gender, age; reasons for the review; major findings (review authors’ conclusions); assessed research impact; review design; quality of included studies; compliance with Indigenous health research ethics.
Attention was devoted to understanding how researchers reported the impacts of their research for Indigenous people and how such impact was framed. The full texts of the reviews were searched for keywords such as impact, benefit, value, effect, consequence, assess* and translation.
Health areas were categorised using the socioeconomic determinants of health framework by Turrell et al.  and broadly divided into seven themes (areas): biological, psychosocial, health behaviour, healthcare system, health research, health inequality, setting and contexts. The authors of this study categorised reviews into as many health related areas as were relevant. Therefore, if a review fitted into more than one area, all were recorded.
An Excel (Microsoft Corporation) Datasheet 2013 was utilised to record the data items. One review author (IK) extracted the data from included studies and the second author (KT) cross-checked the extracted data. Two authors (IK, KT) independently assessed the included reviews. The coefficient of agreement was k = 85%, z = 16.15, p < 0.001, indicating a substantial level of agreement on extracted and recorded data items . Discrepancies were resolved by discussion.
Quality appraisal comprised a six-step analysis. Step 1: type of review was determined and considered as a quality measure . Step 2: the authors ascertained the use of reporting guidelines, for example Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) or any other relevant guidelines. Step 3: the reviews identified in the previous step were assessed with the AMSTAR tool , a validated measurement tool for assessing the methodological quality of reviews. The tool contains 11 questions (domains) (see Additional file 1). Each included review was assessed against all the domains with assessments labelled as ●, ◐ or ○. Reviews that fully addressed the domain were labelled ●; reviews that addressed the domain to some extent were assigned ◐; and reviews that either did not provide enough information for quality analysis, or the answer could not be ascertained, were assigned ○. Step 4: the authors investigated whether reviews assessed quality of the primary studies they had included. Step 5: the authors examined methodological rigour of the reviews through utilisation of quality appraisal tools informed by ethical standards and guidelines for conducting research with Indigenous people for the purpose of identifying the existence and utility of such tools. Step 6: lastly and importantly, the authors examined whether reviews documented Indigenous leadership and/or participation in research. Full texts of the reviews were examined for acknowledgment of Indigenous origin among authors and cross-checked by two authors (RB, JM) of this review.
Synthesis of results
Due to the heterogeneity of health areas, interventions and outcomes, a quantitative summary measure of the results was not planned. Instead, descriptive analysis was used to synthesise results which were then presented according to the three hypotheses outlined in the introduction.
Reviews included, and published over time
The initial search yielded 911 citations and 37 Closing the Gap Clearinghouse reports. Seven hundred sixty-seven remained after adjusting for duplicates. Of these, 703 citations and 16 Closing the Gap Clearinghouse reports were discarded based on the pre-specified inclusion and exclusion criteria. The full text of the remaining 84 citations (63 journal articles and 16 Closing the Gap Clearinghouse reports) were examined in more detail (Fig. 1).
Overall, 76 studies were included in the qualitative synthesis (see Additional file 2). Of these 76 reviews, 55 were journal articles and 21 were Closing the Gap Clearinghouse reports. The 76 reviews contained evidence from more thanFootnote 1 5636 primary publications including randomised control trials, cohort studies, qualitative studies, and other designs. The average search period of the reviews was 17.5 years (SDFootnote 2 12, MD 17, Min 1, Max 62).
Over the 10-year period from 1995 to 2014, the number of Indigenous health reviews grew on average by 2%. Between 2011 and 2014, however, the average percentage growth was much higher, calculated to be 6% (Fig. 2).
Characteristics of included primary studies
Descriptive studies dominated the list of primary publications, comprising a staggering 83% (n = 1,522). Intervention studies comprised the remaining 17% primary studies; there were no measurement studies found. Only two per cent of the primary studies (n = 31) were randomised control trials (RCTs) and five per cent (n = 100) were non-RCTs comparison studies of strategies or groups.
The demographics of the Indigenous population was considered in some reviews, with 22 reviews gender-age specific. Six reviews focused on women, and only one on men. Children and youth were a focus of thirteen reviews.
Not surprisingly given the focus of this study and search terms, the largest proportion of the reviews came from Australia (70/76; 92%). Twenty-one Australian institutions contributed to the publication pool including [listed in descending order according to the number of published reviews in the field] the Closing the Gap Clearinghouse, James Cook University, Menzies School of Health Research, Baker IDI Heart and Diabetes Institute, University of New South Wales, Deakin University, Flinders University, University of Melbourne, University of Newcastle, University of Queensland, University of South Australia, University of Western Australia, Griffith University, Monash University, Queensland Health Mental Health Services, Southern Cross University, University of Sydney, University of Tasmania, University of Technology Sydney, Urbis Pty Ltd (Fig. 3).
Notably, Menzies School of Health Research was the first institution to start publishing reviews in the field . James Cook University, Baker IDI Heart and Diabetes Institute and University of New South Wales only recently (after 2007) joined the pool of institutions that publish reviews on Indigenous health related topics. The Closing the Gap Clearinghouse contributed the largest number of reviews in the most recent period 2010–14 (Fig. 4). It is worth mentioning that most of the Closing the Gap Clearinghouse reports were produced and reviewed by researchers from national universities and other research institutions.
Hypothesis 1: reviews address Indigenous health priority needs
As mentioned in the methods section of this paper, the health priority areas were categorised according to the Turrell et al.  framework. Figure 4 displays the number of reviews segregated by the seven health related areas: biological, psychosocial, health behaviour, healthcare system, health research, health inequality, setting and contexts. The number attached to a node denotes the number of publications.
Of the 76 reviews, 22 (29%) related to biological health, 15 (20%) health behaviour, 13 (17%) health care system and 13 (17%) psychosocial health, 11 (14%) settings and contexts (Fig. 3). Biological and psychosocial health reviews were among the earliest published in the field. The “setting and context” reviews, covering topics on the social determinants of health such as education, parenting, housing, only gained research popularity since 2012 and had the highest average percentage growth thereafter at 9.1%. Only six reviews were related to Indigenous research output; including analysis of output and strategies for improving health and medical research. Only two reviews investigated health inequality. Table 1 outlines the foci of reviews within these categories.
Hypothesis 2: reviews adopt best practice guidelines on research conduct and reporting in respect of methodological transparency and rigour, as well as acceptability and appropriateness of research implementation to Indigenous peoples
As mentioned in the methods, quality appraisal comprised a six-step analysis of: 1) types of reviews; 2) utilisation of reporting guidelines; 3) methodological quality of reviews; 4) quality of primary studies; and 5) methodological rigour from Indigenous perspective; 6) Indigenous leadership and participation in research.
Step 1. Types of reviews
Our categorisation of review methods, using the verbatim description provided in each review, found that the most frequently adopted approach was systematic synthesis of literature reviews (36/76; 47%). A further 27/76 reviews (36%) were termed “systematic reviews” (Table 2). These claim to have a rigorous methodology for searching, evaluating and reporting standards. The quality of these “systematic reviews” was assessed in step 3 of the analysis. Only four/76 reviews (5%) applied a meta-analysis, and only five/76 (7%) used a qualitative synthesis such as meta-synthesis of the evidence, meta-ethnography, narrative synthesis or an integrative review.
Other forms of reviews included resource sheets, a special type of inquiry published by the Closing the Gap Clearinghouse; literature review (or a review, review of the evidence, review of the literature); contextual review; bibliometric analysis; or brief review. By definition, these forms of reviews did not follow a systematic approach, including identified search engines, specified inclusion and exclusion criteria, and the quality assessment of retained literature.
Step 2. Utilisation of reporting guidelines
Our assessment of the reporting guidelines found that 52/76 reviews (68%) did not report on following any guidelines, and only 20/76 reviews (26%) specified and utilised a guideline such as, for example, PRISMA. Furthermore, 17/27 systematic reviews (63%) did not report on following review guidelines. Among 20 reviews that adhered to the reporting guidelines, 40% related to biological or clinical health; leaving the research on social determinants of health largely of poor methodological quality.
Step 3. Methodological quality of reviews
Our ascertainment of those 20 reviews, which reported following review guidelines, found that 15 (75%) scored “●–fully addressed” on more than six domains of the AMSTAR tool (Table 3). Nineteen of the 20 reviews (95%) were published in the last 5 years suggesting that review authors were only recently started adopting methodological rigour.
Reviews most commonly adhered to the following criteria: 3. Literature search, 4. Grey literature, 6. Study characteristics, 7. Quality assessment, 8. Methodological rigour and 11. Conflict of interest. Most reviews (95%) performed a comprehensive literature search and provided sufficient information, such as search terms, dates searched and justified search restrictions, for reproduction of the study. As well, most stated whether they included grey literature or unpublished studies and reports. In some reviews, grey literature was not included but listed as a limitation [18, 19] or was excluded but not justified . In only three reviews, it could not be ascertained whether authors included any grey literature at all [14, 21, 22]. Most studies (70%) appraised the scientific quality and successfully incorporated the results of the analysis in the conclusions to the review. All reviews (100%) reported the characteristics of the included studies in an aggregated form such as a table, or a graph. As well, 95% (n = 19) of reviews acknowledged sources of funding.
The most poorly addressed domains (less or equal to 50% of reviews fully addressed the domain), were: 1.‘A priori’ design, 2. Study selection and extraction, 5. List of studies, 9. Pooled results, and 10. Publication bias (Table 3). Only 30% of the reviews met the criterion requiring an a priori protocol that defines the search strategy, sets the study selection criteria, outlines quality assessment and data extraction procedures, and plans the analysis of the study results. Half of the reviews did not engage the required number (at least two) of independent data extractors, nor stipulated a consensus procedure for disagreements. Some reviews partly met the criterion by employing one data extractor [23–25]. A staggering 95% of the reviews did not provide a list of excluded studies, reporting only the total number instead. Most reviews (40%) failed to explain the method utilised to combine the findings of primary studies. In some cases, reviews did not provide the method, but acknowledged as a limitation that, for example, meta-analysis as a form of synthesis, was improper to apply due to the heterogeneity [18, 21, 26, 27]. The majority of reviews did not address the domain “Publication bias” or addressed it partially. For example, one mentioned, but did not assess, the potential for publication bias . Other reviews addressed the selection bias, usually as a criterion in the assessment tool [18, 22, 23, 25, 26], or attrition bias  instead.
Results of the analysis show that while the majority of the reviews (95%, 72 of 76) have mentioned search criteria and provided sufficient information for reproduction of the study [reproducible research is a key to new discoveries ], important caveats remain including lack of a priori design, reporting on study selection and extraction, listing of studies and pooled results and assessing publication bias.
Step 4. Quality of primary studies
Our ascertainment of the primary studies found that the most prevalent quality assessment tools were the Quality Assessment Tool for Quantitative Studies , CASP Tool For Qualitative Studies  and National Health and Medical Research Council (NHMRC) levels of evidence scale . Forty nine/76 reviews (64%) did not use the tools to assess the quality of their primary studies. For those that did (27/76; 36%), the assessment included overall quality appraisal by hierarchy of evidence, assessment of the risk of bias, or assessment by study design.
Overall, critical assessment revealed that the majority of studies were of moderate [20–22, 25] to poor quality [23, 33–35], with weak study designs  and lack of consistently strong methodology across the majority of applied criteria . According to the assessment by the hierarchy of evidence, most studies were of level IV, the lowest level of evidence [37–40]; that is, from case series, either post-test or pre-test/[and] post-test; or level III-3  obtained from historical cohort studies, two or more single arm studies or interrupted time series without a parallel control group. Assessments of the risk of bias varied, from low  to moderate  and high  risk. Lack of blinding in some outcomes and insufficient sample size for the primary outcome  were primary reasons for the high risk of bias.
Step 5. Methodological rigour from Indigenous perspective
This step was addressed by determining whether reviews used quality appraisal tools that were informed by ethical standards and guidelines for conducting research with Indigenous people. A very high 95% of reviews (72/76) did not assess the ethics of research from an Indigenous perspective. The four reviews that did so, considered an Indigenous perspective via two means: by developing and utilising a context scale tailored to a specific health area and designed to appraise Indigenous research [37, 38, 44] or applying a general tool, not specifically designed to appraise Indigenous research .
Step 6. Indigenous leadership and participation in research
Indigenous leadership and participation in research was gauged by authorship of reviews as a lead author or as a co-author. Full texts of the reviews were examined for acknowledgment of Indigeneity among authors and cross-checked by two authors of this review. Twenty reviews/76 (26%) acknowledged or documented that their authors were Indigenous; in three reviews Indigenous authors were lead authors. Hence, nearly three quarter of the reviews were considered without Indigenous leadership or without acknowledging their Indigenous leadership or participation.
Hypothesis 3: Reviews explicitly report the incremental impacts of their included studies and translation of research
None of the reviews assessed the impact of research nor explicitly referred to research impact. Having said that, Van Schaik et al.  and Dudgeon et al.  utilised approaches that could potentially be used to capture research impact. Van Schaik et al.  incorporated Indigenous peoples’ specific knowledge by critically reviewing the literature associated with Indigenous peoples’ beliefs about cancer treatment. Dudgeon et al.  applied a three step assessment procedure to gauge the cultural appropriateness, the quality of the program evaluations, and whether the program achieved an effective outcome. Dudgeon et al. , Fromene et al. , Mildon et al. , Ware et al. , Ware et al.  and Nelson et al.  did not explicitly refer to the research impact but emphasised its importance and drew on the local cultural context and knowledge in conducting research, with a particular focus on the relevance of findings and recommendations to Indigenous people.
The authors of this systematic review of reviews noted that a designated journal section, for example “what are the new findings”, “lessons learned” or “what this paper adds”, usually prompted researchers to think explicitly about the incremental impact of their research and research translation. Table 4 provides some examples.
This systematic review of reviews on Indigenous Australian health research analyses the quality of evidence and reporting of research impact in existing health reviews. We argue that if review authors explicitly address each of our three hypotheses, then the impact of research for Indigenous health would be explicated. While reviews are gaining prominence and recognition in Indigenous health research as an important methodological approach to dealing with ever growing amounts of research output, and are increasing in number, breadth and complexity, we found that currently, there is little reporting of the impact of health research for Indigenous people and that few met the criteria expected in the reporting of quality evidence. These findings raise questions about the relevance of these reviews for Indigenous people, their impact on policy and practice and how reviews have been commissioned, reported and evaluated.
Although the priorities for Indigenous health are arguable and vary from region to region, on the whole the findings of this systematic review of reviews partially support the first hypothesis: that reviews of Indigenous health address Indigenous health priority needs. This systematic review of reviews considers health broadly as a whole-of-life view associated with wellbeing of individuals and their communities and, to our knowledge, is among the first reviews that focuses not only on health from the Western perspective, but also on the important social determinants of health. To date, there is still a continuing predominance of reviews focussed on biological or clinical models of health which is promising but not sufficient for addressing Indigenous health. Social and emotional well-being encompasses all aspects of the social and emotional context of the person and their family, the historical and economic factors, including racism, oppression, trauma, grief, loss in its many forms, and the sequelae of the Stolen Generations, and it therefore allows for a more holistic framing of health that needs to be recognised in Indigenous research and its impact . Further reviews of aspects of Indigenous mental health and social and emotional wellbeing are needed. Likewise cultural knowledge and context cannot be fully understood without Indigenous peoples’ involvement in research, participation and leadership which clearly has not been acknowledged and documented sufficiently in the analysed reviews.
Our second hypothesis, that reviews should adopt best practice guidelines on research conduct and reporting in respect of methodological transparency and rigour, as well as acceptability and appropriateness of research implementation to Indigenous peoples, was also rejected. Reviews have progressively become a mainstream methodology in Indigenous health research. Nevertheless, the current analysis reveals a clear misinterpretation as to what is understood by the term “review” and, in particular, “systematic review”. Nearly two thirds (63%) of the systematic reviews, which by definition claim to have a rigorous approach to searching, evaluating and reporting, have not reported nor followed any guidelines. Undertaking a systematic review requires a robust methodology. Authors who endeavour to undertake a systematic review are expected to meet detailed and rigorous requirements and follow guidelines or standards on the conduct and reporting. These include the development of an a priori protocol, which is submitted for peer review prior to the commencement of the review; undertaking rigorous searches which align with the inclusion and exclusion criteria specified in the protocol; and strict alignment with guidelines for the preparation of manuscripts for publication .
What was reported and how it was reported also varies considerably. The majority of the reviews were based on qualitative analyses of descriptive and intervention studies. Cooper et al.  observed that the traditional review process typically “lacks analytical precision because of biases associated with a reviewer’s idiosyncratic perspective, failure to assess the size of the effects reported by studies reviewed, and imprecise combination of the volume of evidence available across the studies reviewed”. According to Cooper et al. , a synthesised approach such as meta-analysis can offer a method that avoids some of the problems of the traditional literary summary. Synthesised approaches were underutilised in Indigenous health reviews. Having said that, reviews concerning Indigenous health are complex in nature with prevalent attribution issues; it is hard to establish the impact of a particular intervention if multiple simultaneous interventions are being delivered. Furthermore, Indigenous health is an area where randomised control trial interventions, with outcomes presented as quantified measures that can be subjected to meta-analysis, are not always ethically appropriate. All of these factors contributed to relatively weak evidence base in Indigenous health research.
The rigour of the research from the perspective of Indigenous people directly relates to scarcity of Indigenous leadership and/or participation. Concerns exist that the reporting of reviews has often been culturally inadequate. A few attempts have been made (four reviews out of 76) to assess the ethical appropriateness and rigour of the research from the perspective of Indigenous people, however, no systematic way of appraisal was found. Authors either developed and utilised a context scale tailored to a specific health area [37, 38, 44] or applied a generic tool, not specifically designed to align with an Indigenous research context . This may reflect either, or both, a lack of awareness of reporting guidelines or tools informed by Indigenous peoples’ perspectives on ethically appropriate and rigorous research processes and outputs, or a perception that guidelines that first emerged from clinical medicine are not applicable in the social sciences where most Indigenous health research is being conducted. As pointed out , absence of such tools might lead to a waste of resources on reviews with limited relevance for Indigenous people. The marriage of health research to needs can only occur with consultation, engagement and the trust of Indigenous communities. We recognise the need for a cultural diversity tool specifically designed for conducting systematic reviews in the context of Indigenous health.
Finally, the third hypothesis, that reviews explicitly report the incremental impacts of included studies and research translation, was also rejected. Even though review authors might state an aim to influence policy or practice, often it was mostly unclear who or what they were attempting to influence, or the impact of their research. Dialogue at the nexus between research, policy, and practice is crucial if systematic reviews are to make a significant impact on practice. Instead, reviews need to be framed in such a way that they feed into the development and refinement of policy and practice guidelines which “interpret results to make knowledge of intervention effects available to a wide audience, including readers who are unfamiliar with the methodological and technical aspects of both primary research and research synthesis” [52, 54]. The identified lack of reporting research impact makes the decision by the Lowitja Institute to develop such a tool that would support planning and reporting impact of research over time responsive to the need and very timely .
While this paper provides a systematic review of the state of the Indigenous health research, because it is a systematic review of reviews, it does not include potentially useful primary studies that have not been included by the identified existing reviews. Reviews published prior to 1995 nor those published since 2015 have not been included. The search strategy could have been improved by adding a more sensitive search filter for Indigenous health related terms. It is possible that some reviews are excluded as a result; however, it is unlikely that the overall findings would be significantly different. We drew on the authors’ knowledge of Indigenous health research to identify additional studies. The authorship included one Aboriginal author (RB) and one Torres Strait Islander author (FWL). We acknowledge the limitations of our approach in addressing the impact of research though analysis of reviews rather than the original studies. We cannot rule out the possibility that original studies reported the impact of their research. However, if such had been reported in original studies, we assumed a high probability of reporting the impact by reviews as well. We encourage further research and analysis of original studies to address this limitation.
We also acknowledge the lack of a benchmark or a threshold figure for testing the research hypotheses. This was mainly due to the absence of any formal numeric criteria. Authors used their professional assessment to gauge whether hypotheses could be rejected or not. Lastly, the methodological quality of included reviews is assessed using the AMSTAR tool designed for studies in any population, not specifically Indigenous due to the nonexistence of Indigenous specific tool at the time of publication.
The findings of our study serve two main purposes. First, we identified knowledge and methodological gaps in documenting Indigenous health research impact that can be addressed by researchers and policy makers. Second, the findings provide the justification for developing a framework allowing researchers and funding bodies to structure future Indigenous health research to improve the reporting of its impact; the framework that underpins principles of Indigenous leadership and participation, research capacity enhancement, methodological rigour of research and knowledge translation.
Not all reviews specified the number of included studies
SD–standard deviation; MD–median
- CASP tool:
Critical Appraisal Skills Programme tool
- Indigenous people:
Aboriginal and Torres Strait Islander people
National Health and Medical Research Council
- PRISMA guidelines:
Preferred Reporting Items for Systematic reviews and Meta-Analyses guidelines
Randomised control trials
Anderson I, Robson B, Connolly M, Al-Yaman F, Bjertness E, King A, Tynan M, Madden R, Bang A, Coimbra CEA Jr, et al. Indigenous and tribal peoples’ health: a population study. Lancet. 2016;388:131-57. No. 10040, 9 July 2016.
Sanson-Fisher RW, Campbell EM, Perkins JJ, Blunden SV, Davis BB. Indigenous health research: a critical review of outputs over time. Med J Aust. 2006;184:502–5.
Bainbridge R, Tsey K, McCalman J, Kinchin I, Saunders V, Watkin Lui F, Cadet-James Y, Miller A, Lawson K. No one’s discussing the elephant in the room: contemplating questions of research impact and benefit in Aboriginal and Torres Strait Islander Australian health research. BMC Public Health. 2015;15:1–10.
Tsey K, Lawson K, Kinchin I, Bainbridge R, McCalman J, Watkin F, Cadet-James Y and Rossetto A (2016) Evaluating Research Impact: The Development of a Research for Impact Tool. Front Public Health. 4:160. doi: 10.3389/fpubh.2016.00160.
National Health and Medical Research Council. Values and ethics: guidelines for ethical conduct in Aboriginal and Torres Strait Islander health research. Canberra: Commonwealth of Australia; 2003.
Silburn K, Macmillan J, White V, Hanley F, Marsh G. Evaluation of the cooperative research centre for aboriginal health (Australian institute for primary care, trans.). Melbourne: LaTrobe University; 2010.
Gomersall JS, Canuto K, Aromataris E, Braunack-Mayer A, Brown A. Systematic review to inform prevention and management of chronic disease for Indigenous Australians: overview and priorities. Aust N Z J Public Health. 2016;40:22–9.
Australasian Cochrane Centre (ACC). Overview of systematic reviews of research into the health of Aboriginal and Torres Strait Islander peoples. ACC; 2015. http://australia.cochrane.org/.
Smith V, Devane D, Begley CM, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11:15.
Moher D, Liberati A, Tetzlaff J, Altman D, The PRISMA Group. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med. 2009;6:e1000097.
The state of Aboriginal Health. http://www.naccho.org.au/about/aboriginal-health/the-state-of-aboriginal-health/. Accessed 07 July 2016.
Bondas T, Hall EOC. Challenges in Approaching Metasynthesis Research. Qual Health Res. 2007;17:113–21.
Turrell G, Mathers CD. Socioeconomic status and health in Australia. Med J Aust. 2000;172:434–8.
Adegbija OO, Wang ZQ. Gender variations in waist circumference levels between Aboriginal and non-Aboriginal Australian populations: A systematic review. Obes Res Clin Pract. 2014;8:E513–24.
Derrick GE, Hayen A, Chapman S, Haynes AS, Webster BM, Anderson I. A bibliometric analysis of research on Indigenous health in Australia, 1972–2008. Aust N Z J Public Health. 2012;36:269–73.
Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J, Henry DA, Boers M. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009;62:1013–20.
Morris PS. A systematic review of clinical research addressing the prevalence, aetiology, diagnosis, prognosis and therapy of otitis media in Australian Aboriginal children. J Paediatr Child Health. 1998;34:487–97.
Laws R, Campbell KJ, van der Pligt P, Russell G, Ball K, Lynch J, Crawford D, Taylor R, Askew D, Denney-Wilson E. The impact of interventions to prevent obesity or improve obesity related behaviours in children (0–5 years) from socioeconomically disadvantaged and/or indigenous families: a systematic review. BMC Public Health. 2014;14:779.
Graham S, Guy RJ, Cowie B, Wand HC, Donovan B, Akre SP, Ward JS. Chronic hepatitis B prevalence among Aboriginal and Torres Strait Islander Australians since universal vaccination: A systematic review and meta-analysis. BMC Infect Dis. 2013;13:403.
Lyons JG, O’Dea K, Walker KZ. Evidence for low high-density lipoprotein cholesterol levels in Australian indigenous peoples: a systematic review. BMC Public Health. 2014;14:545.
Calabria B, Clifford A, Shakeshaft AP, Doran CM. A systematic review of family-based interventions targeting alcohol misuse and their potential to reduce alcohol-related harm in indigenous communities. J Stud Alcohol Drugs. 2012;73:477–88.
Gould GS, Munn J, Watters T, McEwen A, Clough AR. Knowledge and views about maternal tobacco smoking and barriers for cessation in Aboriginal and Torres Strait Islanders: A systematic review and meta-ethnography. Nicotine Tob Res. 2013;15:863–74.
McCalman J, Tsey K, Bainbridge R, Rowley K, Percival N, O’Donoghue L, Brands J, Whiteside M, Judd J. The characteristics, implementation and effects of Aboriginal and Torres Strait Islander health promotion tools: a systematic literature search. BMC Public Health. 2014;14:712.
Shah PS, Zao J, Al-Wassia H, Shah V. Pregnancy and neonatal outcomes of aboriginal women: a systematic review and meta-analysis. Womens Health Issues. 2011;21:28–39.
Ospina MB, Voaklander DC, Stickland MK, King M, Senthilselvan A, Rowe BH. Prevalence of asthma and chronic obstructive pulmonary disease in Aboriginal and non-Aboriginal populations: A systematic review and meta-analysis of epidemiological studies. Can Respir J. 2012;19:355–60.
Clifford AC, Doran CM, Tsey K. A systematic review of suicide prevention interventions targeting indigenous peoples in Australia, United States, Canada and New Zealand. BMC Public Health. 2013;13:463.
McCalman J, Tsey K, Clifford A, Earles W, Shakeshaft A, Bainbridge R. Applying what works: a systematic search of the transfer and implementation of promising Indigenous Australian health services and programs. BMC Public Health. 2012;12:600.
McCalman J, Tsey K, Wenitong M, Wilson A, McEwan A, James YC, Whiteside M. Indigenous men’s support groups and social and emotional wellbeing: a meta-synthesis of the evidence. Aust J Prim Health. 2010;16:159–66.
Begley CG, Ioannidis JP. Reproducibility in science: improving the standard for basic and preclinical research. Circ Res. 2015;116:116–26.
Effective Public Health Practice Project (EPHPP). Quality Assessment Tool for Quantitative Studies. Hamilton: McMaster University; 1998.
Qualitative Research Checklist. http://media.wix.com/ugd/dded87_29c5b002d99342f788c6ac670e49f274.pdf. Accessed 25 Jan 2016.
NHMRC. NHMRC additional levels of evidence and grades for recommendations for developers of guidelines. Canberra: National Health and Medical Research Council; 2009.
McDonald E, Bailie R, Brewster D, Morris P. Are hygiene and public health interventions likely to improve outcomes for Australian Aboriginal children living in remote communities? A systematic review of the literature. BMC Public Health. 2008;8:Article 153.
Day A, Francisco A. Social and emotional wellbeing in Indigenous Australians: identifying promising interventions. Aust N Z J Public Health. 2013;37:350–5.
Bainbridge R, Tsey K, McCalman J, Towle S. The quantity, quality and characteristics of Aboriginal and Torres Strait Islander Australian mentoring literature: a systematic review. BMC Public Health. 2014;14:1263.
Clelland N, Gould T, Parker E. Searching for evidence: what works in Indigenous mental health promotion? Health Promot J Austr. 2007;18:208–16.
Power J, Grealy C, Rintoul D. Tobacco interventions for Indigenous Australians: a review of current evidence. Health Promot J Austr. 2009;20:186–94.
Ivers RG. A review of tobacco interventions for Indigenous Australians. Aust N Z J Public Health. 2003;27:294–9.
Bonevski B, Randell M, Paul C, Chapman K, Twyman L, Bryant J, Brozek I, Hughes C. Reaching the hard-to-reach: a systematic review of strategies for improving health and medical research with socially disadvantaged groups. BMC Med Res Methodol. 2014;14:42.
Bowes J, Grace R. Review of early childhood parenting, education and health intervention programs for Indigenous children and families in Australia. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies (8. Ipn ed. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2014.
Gould GS, McEwen A, Watters T, Clough AR, van der Zwan R. Should anti-tobacco media messages be culturally targeted for Indigenous populations? A systematic review and narrative synthesis. Tob Control. 2013;22. doi: 10.1136/tobaccocontrol-2012-050436
McNamara BJ, Gubhaju L, Chamberlain C, Stanley F, Eades SJ. Early life influences on cardio-metabolic disease risk in aboriginal populations-what is the evidence? A systematic review of longitudinal and case-control studies. Int J Epidemiol. 2012;41:1661–82.
Chang AB, Taylor B, Masters IB, Laifoo Y, Brown ADH. Indigenous healthcare worker involvement for Indigenous adults and children with asthma. Cochrane Database Syst Rev. 2010;(5):CD006344. DOI: 10.1002/14651858.CD006344.pub3.
Dudgeon P, Walker R, Scrine C, Shepherd C, Calma T, Ring I. Effective strategies to strengthen the mental health and wellbeing of Aboriginal and Torres Strait Islander people. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies (12. Ipn ed. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2014.
Dawson AP. Asthma in the Australian Indigenous population: a review of the evidence. Rural Remote Health. 2004;4:238.
Van Schaik KD, Thompson SC. Indigenous beliefs about biomedical and bush medicine treatment efficacy for indigenous cancer patients: a review of the literature. Intern Med J. 2012;42:184–91.
Fromene R, Guerin B, Krieg A. Australian Indigenous Clients with a Borderline Personality Disorder Diagnosis: A Contextual Review of the Literature. Psychol Rec. 2014;64:559–67.
Mildon R, Polimeni M. Parenting in the early years: effectiveness of parenting education and home visiting programs for Indigenous families. Resource sheet no. 16. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2012.
Ware V-A. Housing strategies that improve Indigenous health outcomes. Resource sheet no. 25. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Ware V-A. Improving the accessibility of health services in urban and regional settings for Indigenous people. Resource sheet no. 27. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Nelson A, Abbott R, Macdonald D. Indigenous Austalians [sic] and physical activity: using a social-ecological model to review the literature. Health Educ Res. 2010;25:498–509.
Crisp BR. Systematic Reviews: A Social Work Perspective. Aust Soc Work. 2015;68:284–95.
Cooper H, Hedges LB. Handbook of research synthesis. New York: Russell Sage; 1994.
Littell JH. Lessons from a systematic review of effects of multisystemic therapy. Child Youth Serv Rev. 2005;27(4):445–63.
Arnold M, Moore SP, Hassler S, Ellison-Loschmann L, Forman D, Bray F. The burden of stomach cancer in indigenous populations: a systematic review and global assessment. Gut. 2014;63:64–71.
Azzopardi PS, Kennedy EC, Patton GC, Power R, Roseby RD, Sawyer SM, Brown AD. The quality of health research for young Indigenous Australians : systematic review. Med J Aust. 2013;199:57–63.
Brooke NJ. Needs of Aboriginal and Torres Strait Islander clients residing in Australian residential aged-care facilities. Aust J Rural Health. 2011;19:166–70.
Harlow AF, Bohanna I, Clough A. A Systematic Review of Evaluated Suicide Prevention Programs Targeting Indigenous Youth. Crisis: J Crisis Interv Suicide. 2013;35(5):310-21. doi: 10.1027/0227-5910/a000265.
Morris PS. Randomised controlled trials addressing Australian Aboriginal health needs : A systematic review of the literature. J Paediatr Child Health. 1999;35:130–5.
Guy RW, James S, Smith KS, Su J-Y, Huang R-L, Tangey A, Skov S, Rumbold A, Silver B, Donovan B, Kaldor JM. The impact of sexually transmissible infection programs in remote Aboriginal communities in Australia: a systematic review. Sex Health. 2012;9:205–12.
McCalman J, Bridge F, Whiteside M, Bainbridge R, Tsey K, Jongen C. Responding to Indigenous Australian Sexual Assault: A Systematic Review of the Literature. SAGE Open. 2014;4:2158244013518931.
Minges KE, Zimmet P, Magliano DJ, Dunstan DW, Brown A, Shaw JE. Diabetes prevalence and determinants in Indigenous Australian populations: A systematic review. Diabetes Res Clin Pract. 2011;93:139–49.
Porter C, Skinner T, Ellis I. The current state of Indigenous and Aboriginal women with diabetes in pregnancy: A systematic review. Diabetes Res Clin Pract. 2012;98:209–25.
Woods JA, Katzenellenbogen JM, Davidson PM, Thompson SC. Heart failure among Indigenous Australians: A systematic review. BMC Cardiovasc Disord. 2012;12:99.
Carey TA, Wakerman J, Humphreys JS, Buykx P, Lindeman M. What primary health care services should residents of rural and remote Australia be able to access? A systematic review of “core” primary health care services. BMC Health Serv Res. 2013;13:178.
Chamberlain C, McNamara B, Williams ED, Yore D, Oldenburg B, Oats J, Eades S. Diabetes in pregnancy among indigenous women in Australia, Canada, New Zealand and the United States: a systematic review of the evidence for screening in early pregnancy. Diabetes Metab Res Rev. 2013;29:241–56.
Wakerman J, Humphreys JS, Wells R, Kuipers P, Entwistle P, Jones J. Primary health care delivery models in rural and remote Australia–A systematic review. BMC Health Serv Res. 2008;8:276.
Miller A, Smith ML, Judd J, Speare R. Strongyloides stercoralis: systematic review of barriers to controlling strongyloidiasis for Australian Indigenous communities. PLoS Negl Trop Dis. 2014;8:e3141.
Heffernan E, Andersen K, Kinner S. The insidious problem inside: mental health problems of Aboriginal and Torres Strait Islander People in custody. Australas Psychiatry. 2009;17 Suppl 1:S41–6.
Priest N, Mackean T, Waters E, Davis E, Riggs E. Indigenous child health research: a critical analysis of Australian studies. Aust N Z J Public Health. 2009;33:55–63.
Jervis-Bardy J, Sanchez L, Carney AS. Otitis media in Indigenous Australian children: review of epidemiology and risk factors. J Laryngol Otol. 2014;128:S16–27.
Bowen A, Duncan V, Peacock S, Bowen R, Schwartz L, Campbell D, Muhajarine N. Mood and anxiety problems in perinatal Indigenous women in Australia, New Zealand, Canada, and the United States: A critical review of the literature. Transcult Psychiatry. 2014;51:93–111.
Banbury A, Roots A, Nancarrow S. Rapid review of applications of e-health and remote monitoring for rural residents. Aust J Rural Health. 2014;22:211–22.
Deek H, Abbott P, Moore L, Davison J, Cameron S, DiGiacomo M, McGrath SJ, Dharmendra T, Davidson PM. Pneumococcus in Aboriginal and Torres Strait Islander peoples: The role of Aboriginal health workers and implications for nursing practice. Contemp Nurse. 2013;46:54–8.
Davidson PM, Jiwa M, DiGiacomo ML, McGrath SJ, Newton PJ, Durey AJ, Bessarab DC, Thompson SC. The experience of lung cancer in Aboriginal and Torres Strait Islander peoples and what it means for policy, service planning and delivery. Aust Health Rev. 2013;37:70–8.
Ivers R. Anti-tobacco programs for Aboriginal and Torres Strait Islander people 2011. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2011.
Closing the Gap Clearinghouse. Healthy lifestyle programs for physical activity and nutrition. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2012.
Closing the Gap Clearinghouse. Ear disease in Aboriginal and Torres Strait Islander children. Resource sheet no. 35. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2014.
Closing the Gap Clearinghouse. Strategies and practices for promoting the social and emotional wellbeing of Aboriginal and Torres Strait Islander people. Resource sheet no. 19. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Closing the Gap Clearinghouse. Strategies to minimise the incidence of suicide and suicidal behaviour. Resource sheet no. 18. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Ware V-A, Meredith V. Supporting healthy communities through sports and recreation programs. Resource sheet no. 26. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Pholeros P, Phibbs P. Constructing and maintaining houses. Resource sheet no. 13. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2012.
Strobel N, Ward J. Education programs for Indigenous Australians about sexually transmitted infections and bloodborne viruses. Resource sheet no. 14. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2012.
Harrison L, Goldfeld S, Metcalfe E, Moore T. Early learning programs that promote children’s developmental and educational outcomes. Resource sheet no. 15. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2012.
Closing the Gap Clearinghouse. Increasing employment rates for Indigenous people with a disability. Resource sheet no. 29. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2014.
Ware V-A. Supporting healthy communities through arts programs. Resource sheet no. 28. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2014.
McCuaig L, Nelson A. Engaging Indigenous students through school-based health education. Resource sheet no. 12. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2012.
Higgins D, Morley S. Engaging Indigenous parents in their children’s education. Resource sheet no. 32. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2014.
Rumbold AR, Cunningham J. A review of the impact of antenatal care for Australian Indigenous women and attempts to strengthen these services. Matern Child Health J. 2008;12:83–100.
Christian B, Blinkhorn AS. A review of dental caries in Australian Aboriginal children: the health inequalities perspective. Rural Remote Health. 2012;12:1–11.
Clifford A, Pulver LJ, Richmond R, Shakeshaft A, Ivers R. Disseminating best-evidence health-care to Indigenous health-care settings and programs in Australia: identifying the gaps. Health Promot Int. 2009;24:404–15.
Procter NG. Parasuicide, self‐harm and suicide in Aboriginal people in rural Australia: A review of the literature with implications for mental health nursing practice. Int J Nurs Pract. 2005;11:237–41.
Wise S. Improving the early life outcomes of Indigenous children: implementing early childhood development at the local level. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies (6. Ipn ed. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Osborne K, Baum F, Brown L. What works? A review of actions addressing the social and economic determinants of Indigenous health. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies (7. Ipn ed. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2013.
Sims M. Early childhood and education services for Indigenous children prior to starting school. In: Produced for the Closing the Gap Clearinghouse Institute of Family Studies. Canberra: Australian Institute of Health and Welfare & Melbourne: Australian; 2011.
Eades SJ, Taylor B, Bailey S, Williamson AB, Craig JC, Redman S. The health of urban Aboriginal people: insufficient data to close the gap. Med J Aust. 2010;193:521–4.
We would like to acknowledge Lowitja Institute for providing funding to support this review and James Cook University for in-kind contribution.
This research was supported by the Australian Research Council (ARC) Discovery Indigenous Grant (ID: IN150100011) and James Cook University Research and Innovation Grant “Impact in Indigenous health research: A systematic review”. The contents of the published material are solely the responsibility of the administering institution, a participating institution or individual authors, and do not reflect the views of ARC.
Availability of data and materials
All data generated or analysed during this study are included in this published article [and its supplementary information files].
All authors read and approved the final version of the manuscript.
IK is a Senior Postdoctoral Fellow in Health Economics at the School of Health, Medical and Applied Sciences at CQUniversity and an Adjunct Senior Research Fellow at the Cairns Institute, James Cook University.
RB is a Gungarri Aboriginal woman from South Western Queensland, an Associate Professor and a Senior Principal Research Fellow at the School of Health, Medical and Applied Sciencesat CQUniversity, an Adjunct Senior Research Fellow at the College of Public Health, Medical & Veterinary Sciences and the Cairns Institute, James Cook University.
JM is an Associate Professor and a Senior Principal Research Fellow at the School of Health, Medical and Applied Sciences at CQUniversity, an Adjunct Senior Research Fellow at the College of Public Health, Medical & Veterinary Sciences and the Cairns Institute, James Cook University.
KT is a Tropical Leader/Research Professor (Social Science) at the College of Arts Society and Education and the Cairns Institute at James Cook University.
FWL is a Torres Strait Islander woman with giz from Erub, Mabuiag and Badu in the Torres Strait, a Senior Lecturer at the Australian Aboriginal and Torres Strait Islander Centre and the Cairns Institute at James Cook University.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Kinchin, I., Mccalman, J., Bainbridge, R. et al. Does Indigenous health research have impact? A systematic review of reviews. Int J Equity Health 16, 52 (2017). https://doi.org/10.1186/s12939-017-0548-4
- Indigenous health
- Research impact
- Closing the gap
- Research benefit
- Research value
- Decision making
- Systematic review