Open Access

A rapid equity focused health impact assessment of a policy implementation plan: An Australian case study and impact evaluation

  • Ben F Harris-Roxas1Email author,
  • Patrick J Harris1,
  • Elizabeth Harris1 and
  • Lynn A Kemp1
International Journal for Equity in HealthThe official journal of the International Society for Equity in Health201110:6

DOI: 10.1186/1475-9276-10-6

Received: 25 March 2010

Accepted: 30 January 2011

Published: 30 January 2011

Abstract

Background

Equity focused health impact assessments (EFHIAs), or health equity impact assessments, are being increasingly promoted internationally as a mechanism for enhancing the consideration of health equity in the development of policies, programs and projects. Despite this there are relatively few examples of examples of completed EFHIAs available. This paper presents a case study of a rapid EFHIA that was conducted in Australia on a health promotion policy implementation plan. It briefly describes the process and findings of the EFHIA and evaluates the impact on decision-making and implementation.

Methods

The rapid EFHIA was undertaken in four days, drawing on an expert panel and limited review of the literature. A process evaluation was undertaken by email one month after the EFHIA was completed. An impact evaluation was undertaken two years later based on five semi-structured interviews with members of the EFHIA working group and policy officers and managers responsible for implementing the plan. A cost estimation was conducted by the EFHIA working group.

Findings

The EFHIA made both general and specific recommendations about how the health equity impacts of the policy implementation plan could be improved. The impact evaluation identified changes to development and implementation that occurred as a result of the EFHIA, though there was disagreement about the extent to which changes could be attributed solely to the EFHIA. Those responsible considered the recommendations of the EFHIA in the next versions of their ABHI implementation plans. Factors that influenced the impact of the EFHIA included consolidating understandings of equity, enabling discussion of alternatives, and differing understandings of the purpose of the EFHIA. The EFHIA cost US$4,036 to undertake.

Conclusions

This EFHIA was conducted in a short timeframe using relatively few resources. It had some reported impacts on the development of the implementation plan and enhanced overall consideration of health equity. This case highlights some of the factors and preconditions that may maximise the impact of future EFHIAs on decision-making and implementation.

Background

There is now strong policy support internationally for governments and institutions to routinely assess the health impacts of major policies, plans, programs and projects on health to address health inequalities [19]. Over the past 15 years health impact assessment (HIA) has been promoted as a mechanism through which such assessment can be achieved in a structured and transparent way [1012]. There are now many countries that have extensive experience in the ways in which HIA can add value to policy and planning decision-making processes, with activity occurring in Europe, South-East Asia, Australia, New Zealand and the USA [7, 1326].

HIA enables the systematic consideration of health inequalities early on within the development of policies and other initiatives prior to their implementation [10, 27]. In doing so, HIA becomes a practical policy intervention that can shift the rhetoric of healthy public policy into action [28]. However despite this promise, the experience of the last decade demonstrates that HIA has been difficult to institutionalise within policy development cycles [29]. Further, despite equity being a conceptual driver for HIA's use [27, 30], evidence and commentary suggests this has had limited translation into practice [10, 3135].

The difficulty in using HIA at the policy level has been linked to concerns about HIA fitting within the (often short) timeframes associated with the development, announcement and implementation of policies [3538]. This concern is in part linked to one aspect of the historical development of HIA as a field, as a part of regulatory project impact assessment [3941], which conventionally follows a more structured planning process than policy. That the development and implementation of policy is less linear and less clear than project development poses a challenge to the step-wise process of HIA [4244].

Health equity may be discussed in little detail within HIAs. This may be due to a number of factors. Firstly, there may be few opportunities to describe and discuss what potential health impacts are considered unfair. Secondly, there may be a lack of clarity about which differential health impact should be examined. In other words: how do we know who it is unfair for? Thirdly, it maybe be unclear what changes could remedy this unfairness or injustice [16, 3133, 4547].

These challenges can be compounded by a lack of existing evidence about which groups will be disproportionately affected by the type of proposal being assessed [32, 33]. Explicitly considering the broader determinants of health, beyond biomedical impacts, and engaging communities within the assessment process may help to ensure equity impacts are considered. These may not equate with considering equity, nor do they automatically lead to consideration of differential impacts, fairness or whether unfair impacts could be avoided [16, 31]. Examining differential impacts can add complexity to an already conceptually difficult HIA process that is usually undertaken in an interdisciplinary and intersectoral context [48].

Recognition of the need for a framework for rapidly assessing the health equity impacts of proposed policy and program proposals led to the development of rapid equity focused health impact assessment (EFHIA, see Table 1), informed by earlier work undertaken in developing a framework for EFHIA [46, 47]. This paper presents rapid EFHIA as an approach and details the process and impacts of a rapid EFHIA that was undertaken over four working days on components of a complex state-wide health promotion initiative focusing on the prevention and early detection of chronic disease. It describes the context in which the EFHIA was undertaken, the methods used for the EFHIA and to evaluate the process and impacts of the EFHIA, the findings of the EFHIA and of the evaluation, and conclusions.
Table 1

Health Impact Assessment-Related Terminology

Term

Explanation

Health impact assessment (HIA)

HIA is "a combination of procedures, methods and tools by which a policy, program or project may be judged as to its potential effects on the health of a population, and the distribution of those effects within the population" [27].

Health equity impact assessment (HEIA)

HEIA has been suggested as a means to ensure that the potential impacts of a proposal on health equity is considered prior to implementation [4, 49]. It is related to the notion of health inequalities impact assessment that was originally proposed a decade ago in the Acheson Review in the UK [12, 50]. Despite these calls, specific guidance on how to conduct HEIAs has not been developed and there are ongoing debates about whether it is possible or desirable to conduct an impact assessment focused solely on health equity without considering more general health impacts [51, 52].

Equity focused health impact assessment (EFHIA)

EFHIA is related to HEIA and was developed in response to concerns that (i) consideration of health equity is often limited within HIAs, often being restricted to the realm of professed values and aspirations [31], and (ii) that it was desirable to improve the methods for considering equity within HIA, rather than developing a separate form of HEIA [52]. The term was first used in the Jakarta Declaration on Leading Health Promotion [53] and subsequently in the Bangkok Declaration [54], but was operationalised with the development of the Equity Focused Health Impact Assessment Framework [46, 47, 55] in 2004. EFHIA focuses on improving the consideration of equity and differential impacts at each step of the HIA process [46, 47]. A rapid EFHIA involves scoping the EFHIA so it can be conducted within a limited time frame with limited resources [56].

Context

The NSW Department of Health Australian Better Health Initiative (ABHI) Implementation Plan was developed in 2006 as part of a Council of Australian Governments (COAG) reform package aimed at achieving better health for all Australians through a focus on the prevention and early detection of chronic disease [57]. The implementation plan looked at the implementation of the health promotion-related components of the ABHI in New South Wales (NSW), a state of seven million people in eastern Australia. In NSW, the prevention and early intervention initiatives and their supporting strategies needed to be developed within a short timeframe to enable resources to be allocated within the funding period identified in the COAG agreement.

The draft implementation plan was sent by the NSW Department of Health to key stakeholders for comment. The Centre for Primary Health Care and Equity at the University of New South Wales was included in this process due to its expertise in chronic disease prevention. Centre staff noted that despite equity appearing within the background to the document as a value there was little explicit focus on equity within the strategies and approached the NSW Department of Health about conducting a rapid EFHIA on the initiatives. This was agreed by those developing the initiative within the Department and they were receptive to an EFHIA being conducted provided that (i) it could be done within four working days as the final document needed final approval three days after this deadline, (ii) did not suggest new strategies but made recommendations on how existing strategies could be strengthened or modified, and (iii) did not recommend changes in funding levels. Issues related to Aboriginal health, though important in any consideration of health equity impacts in the Australian context, were excluded from consideration within the rapid EFHIA as these were being covered through a separate Aboriginal Health Impact Statement process.

Methods

Rapid EFHIA methods

A structured approach to screening and scoping the HIA was undertaken by the EFHIA working group using the NSW guide for HIA [56]. The core EFHIA working group was made up of three staff from CHETRE and one NSW Department of Health employee, all of whom were experienced in conducting HIAs. Additionally an expert panel was recruited to undertake the HIA assessment step. Each expert panel member agreed to attend one six hour assessment workshop (Day 2), comment of drafts and participate in two one-hour teleconferences (Days 3 and 4, see Table 2). The expert panel had nine members, seven of whom were able to attend the workshop. They included people with expertise in health equity, early intervention, health promotion, chronic disease prevention, and policy analysis.
Table 2

Timeline for the rapid EFHIA

Day

Day 1

Day 2

Day 3

Day 4

Who

EFHIA working group

EFHIA working group and expert panel

EFHIA working group and expert panel

EFHIA working group and expert panel

HIA step[52]

Screening and scoping

Identification and assessment of impacts (appraisal)

Negotiation and decision-making

Negotiation and decision making

Activity

Screening and scoping report, identification of key documents & organisation of expert panel

Appraisal workshop, drafting report

Teleconference, drafting report

Teleconference, drafting report

The purpose of the screening report was to identify the potential links between the implementation of the initiatives, health improvement and potential health inequities. The screening report determined that the initiatives had the potential to improve health but also to have differential impacts across the population.

The scoping report (included in Additional File 1) established terms of reference for the EFHIA and expert panel, clarified definitions of health and equity, determined the dimensions on which differential impacts were to be considered in the EFHIA (age, gender, place of residence, ethnicity and socioeconomic position), the process by which the EFHIA would be undertaken, and clarification of values and assumptions, especially in defining health, equity and inequity. Of particular importance was a decision to make recommendations that would positively impact on the whole of the population (mainstream approaches) as well as those that would specifically focus on particular groups (targeted approaches). Due to time constraints it was agreed that the assessment process be principally based on expert opinion, supported by a small number of reports providing data on inequity in relation to chronic disease [58, 59].

At the appraisal workshop the screening and scoping papers were discussed, refined and accepted. The group then systematically worked through each of the eight strategies included in the assessment, addressing five specific questions:
  1. 1.

    What is the initiative trying to do?

     
  2. 2.

    Is there evidence of inequity?

     
  3. 3.

    Who may be disadvantaged by the initiative?

     
  4. 4.

    Are there likely to be unanticipated impacts?

     
  5. 5.

    What are the key recommendations for implementation?

     

These questions reflected aspects of the EFHIA framework [46], drew on work that had been undertaken by members of the EFHIA working group on the development of Australian National Health and Medical Research Council Guidelines [60], and are similar to other questions used in other equity audits and HIA screening tools [6164].

To inform their decision-making, the group made an assessment of the potential size of the impact of the initiative on health, the likelihood of the impact and the groups who may be affected. The resulting EFHIA appraisal for each initiative was summarised in one page to facilitate use by the Department in the short timeframe within which they were preparing the implementation plan.

Drafts of the EFHIA report were circulated to members at the end of each day and teleconferences were held early on Days 3 and 4. A draft EFHIA report was sent to the NSW Department of Health on the morning of Day 5. The final document, which incorporated modifications based on comments received from the Department of Health, was sent on Day 7 [65].

Evaluation methods

Process evaluation methods

A brief process evaluation [66, 67] was undertaken through panel members being asked to reflect on the experience via email and what they perceived as the strengths and weaknesses of the process. This was supplemented by a brief discussion one month after submission of the report with the officers and managers responsible for the NSW ABHI implementation plan.

Impact evaluation methods

To evaluate the impact of the EFHIA on planning and implementation, five semi-structured interviews were conducted two years after the EFHIA was completed. The attributes of these interviews are described in Table 3 using the CORE-Q consolidated criteria for reporting qualitative research [68]. The interviewees included policy officers and managers responsible for developing and overseeing the health promotion components of the NSW Department of Health ABHIA Implementation Plan and members of the EFHIA working group.

Each interviewee was asked the following questions:
  1. 1.

    Tell me the story of the New South Wales Australian Better Health Initiative equity focused health impact assessment. (Prompt: And then what happened?)

     
  2. 2.

    What changed as a result of doing the equity focused HIA?

     
  3. 3.

    Was the equity focused HIA a success?

     
  4. 4.

    What is required for an equity focused HIA to be successful?

     

Interviews were audio recorded and then transcribed.

The interviews were analysed qualitatively using a modified version of the analytic method developed by Colaizzi [69, 70]. The major emergent themes from this analysis are detailed in the findings section.
Table 3

CORE-Q Consolidated Criteria for Reporting Qualitative Research [68]

No

Item

Description

Domain 1: Research Team and Reflexivity

 

Personal Characteristics

1

Interviewer/facilitator

Ben Harris-Roxas

2

Credentials

Master of Policy and Applied Social Research, currently enrolled in a PhD

3

Occupation

Research Fellow, University of New South Wales

4

Gender

Male

5

Experience and Training

Has undertaken several qualitative studies, trained in interviewing, qualitative analysis and using NVivo [92]

 

Relationship with participants

6

Relationship established

A relationship existed with all interviewees prior to the interviews

7

Participant knowledge of the interviewer

Knew the researcher has worked on HIA and health equity for several years, have had contact through other activities than the HIA described

8

Interviewer characteristics

Is doing a PhD on EFHIA

Domain 2: Study Design

 

Theoretical Framework

9

Methodological orientation and theory

Analysis based on a modified version of Calaizzi's framework [69]

 

Participant Selection

10

Sampling

Purposive

11

Method of approach

Email

12

Sample size

5

13

Non participation

0

 

Setting

14

Setting of data collection

Participants' workplaces

15

Presence of non-participants

No

16

Description of sample

A mix of those who conducted the HIA and those who were responsible for implementing its recommendations

 

Data Collection

17

Interview guide

Not provided in advance, piloted on 2 interviews not included in study

18

Repeat interviews

No

19

Audio/visual recording

Audio

20

Field notes

No

21

Duration

Mean 24 minutes, Range 20 minutes (min) to 33 minutes (max)

22

Data saturation

Yes, authors decided that saturation was reached after 5 interviews

23

Transcripts returned

No

Domain 3: Analysis and Findings

 

Data Analysis

24

Number of data coders

3

25

Description of the coding tree

No

26

Derivation of themes

Derived from the data

27

Software

NVivo [92]

28

Participant checking

No

 

Reporting

29

Quotations presented

Yes, each participant is numbered when quoted

30

Data and findings consistent

Yes

31

Clarity of major themes

Yes

32

Clarity of minor themes

No

Resource description methods

The resources involved in conducting the EFHIA were estimated by the EFHIA working group and are included to aid future cost utility studies of HIAs.

Findings

Rapid EFHIA recommendations

For each of the eight initiatives included in the EFHIA, a one page summary was included in the report which described in some detail the questions that guided the EFHIA. Additional File 1 includes the summaries for all the initiatives included in the EFHIA.

Evaluation Findings

Process evaluation findings

The data from the process evaluation identified four factors that assisted the EFHIA. These were the support, commitment and openness of the Department of Health to having their plan assessed, the clarity of the instructions from the Department of Health (included in Additional File 1), the structured process the EFHIA followed, and the composition and experience of the expert panel coupled with the ease with which they were able to work together. Because the Centre for Health Equity Training, Research and Evaluation (CHETRE) is recognised as having expertise in this area and had worked with many of the managers on previous projects [71], there was a level of trust which facilitated the conduct of the EFHIA. The structured process made it transparent what commitment was required of the participants and the expert panel in terms of time and scope of activities to be undertaken.

Three major constraints to undertaking the EFHIA were identified. These were the timeframe required, reliance on expert opinion and a limited range of literature rather than a broader range of evidence, and the difficulty in being objective concerning negative or unanticipated consequences that individual members of the EFHIA working group or expert panel strongly supported.

For policy officers and managers the EFHIA provided an opportunity for reflection on how issues of equity had been addressed in the draft implementation plan and how these and other issues could be improved. For example what balance was needed between innovation, which often had high political and professional appeal, and expanding and sustaining existing programs for which there was evidence of effectiveness. In other words, managers had to decide if they should inject more funding and support into a new suit of programs and to neglect existing programs.

Impact evaluation findings

The interviews identified a number of direct and indirect changes to the NSW ABHI implementation plan that occurred as a result of the EFHIA. Though there was a high degree of concordance about the process of the EFHIA (how it was conducted, who was involved, what the major events were, etc.) there was disagreement between the interviewees about the extent of change that occurred as a result of the rapid EFHIA. This disagreement was not fundamental: all those interviewed felt that the EFHIA had some impact on further planning and decision-making. Rather the disagreement was about the extent of change that could be attributed to the EFHIA. This ranged from a very small amount of influence according to some of those interviewed, to what was regarded as a moderate amount of influence by others.

The impact evaluation identified five major themes in relation to the impact of the rapid EFHIA:

1. Changing implementation planning

There were a number of changes to the NSW ABHI implementation plan that were attributed to the EFHIA. The most obvious of these were that the managers responsible for the development of aspects of the plan were asked to re-draft their sections to take the EFHIA into account.

"What we did after we got the equity focused HIA, we gave it to all the managers, and then for each one of their little, sort of, almost section in the plan, we said 'we want you to write a proper plan about how you're going to do it, and we want in your plan, to specifically say how you're going to address the recommendations of this.'

Interviewee 1

Other changes were attributed to the EFIA, though there was greater disagreement between the EFHIA working group and the Department of Health staff about the nature, extent and reasons for the changes. An example of this were the recommended changes to the proposed resource allocation split between urban and rural Area Health Services for a specific activity within the implementation plan, to favour more resources going to rural health services.

I think it actually did have at least one impact that I know of, which was that we had identified that not enough money was being invested in rural areas, although the resources were going to be allocated, the rural and the urban areas were going to get the same resources. And so I understood that... the problem is, for some of the rural areas, what they would have been getting wasn't enough to actually employ someone, so sixty thousand in the thick of an urban area's quite a lot, but in a rural area, it actually doesn't give you capacity. So I understood that what happened was each of the rural area health services was given a larger amount of money than the urban areas, and that they then wrote up their proposals... So it did have that impact.

Interviewee 2

Several interviewees identified this as a change attributable to the EFHIA but others discounted it, as the measure was not implemented in the form originally outlined in the draft implementation plan due to reallocation of funding. This mirrors some of the difficulties that have been found in other HIA impact evaluations in trying to attribute changes solely to a specific HIA [7274].

2. Consolidating understandings of equity

There was a broad agreement between the interviewees that the rapid EFHIA had brought the potential health equity impacts to the fore of the development of the implementation plan and that this was unlikely to have been as clearly addressed without the EFHIA. The interviewees from the Department of Health regarded this focus on health equity issues as a consolidation and focusing of existing knowledge, rather than being transformative or revelatory in nature - it was regarded as possibly under-considered information rather than unknown.

It probably provided a useful tool to make sure that people considered equity issues. Had we not had support for those type of [equity] issues being considered up the line, it probably would have been used as an internal advocacy tool...

...I'm not convinced that [the EFHIA] made people do things differently, because I think that they probably, should've, would've, hopefully would've, done those things anyway. It was nice that it was explicit, rather than left to being implicit.

Interviewee 3

So for me, personally, if you see the change [in] the acceptance of equity as a value determining and influencing people's thinking and work, it's good.

Interviewee 5

HIA's usefulness in consolidating knowledge and understanding of health issues and potential health impacts has been noted in the literature [72, 75], and approaches that explicitly examine health equity impacts seem likely to enhance understandings of health equity as well.

A challenge that was identified by the interviewees from both the EFHIA working group and the Department of Health was that many of the potential health equity issues that could have arisen did not relate to the overall structure or nature of the ABHI initiatives, but to the way the initiatives would be implemented.

...when I was going through the [EFHIA] recommendations, that some of them appeared to have gone beyond just saying what would happen. If you're just trying to provide equity focused [recommendations], they're often about good planning, which I think was probably very apparent to the people during the equity focused HIA going 'what is the good planning in this?'

Interviewee 1

There were also some things in [the EFHIA report] that, I guess, implied, that we wouldn't consider, some issues that I think can be dealt with in careful planning, and careful implementation, and the intention, as I said before, if the [ABHI implementation plan] was really about 'this is the flavour of where we're going with this' we're going to have to obviously have greater implementation plans around each of these strategies, we've only got sixty pages to do it in.

Interviewee 3

This highlights some of the challenges in undertaking assessments of implementation plans. Policies, such as the ABHI as a whole, are necessarily aspirational, setting out areas of activity in broad terms. EFHIAs, and HIAs in general, need to consider implementation as it is at that stage where many unintended and previously unidentified impacts are likely to arise. The NSW ABHI implementation plan included details of how the initiatives would be implemented, but many of the very detailed planning and implementation activities were appropriately determined by operational and service managers. This identifies a tension relevant to all HIAs, but to EFHIA in particular; to what extent should an HIA focus on making recommendations to assist implementation?

Two preconditions seem to be important enablers of EFHIAs of implementation plans: a high degree of understanding of the policy context and processes being assessed; and trust and constructive engagement between the assessors and those responsible for the development and implementation of the implementation plan. This is similar to the findings of studies that have looked at the impacts of HIAs in the Netherlands [76] and impact assessments more broadly [77].

3. Enabling discussion of alternatives

Several interviewees stated that the EFHIA enabled consideration of different ways of achieving the implementation plan's objectives.

...even during the time we were doing it... we were able to enter into some discussions about what might be alternatives. So I think that in these sorts of environments, we've got an opportunity to influence the implementation. It's actually really important to have debate and that's what I think the EFHIA allowed.

Interviewee 2

The identification and assessment of alternatives is an important and under-emphasised part of HIA and impact assessment practice [13, 78, 79]. The development of more formal procedures for generating alternatives that address health inequities, which may then be assessed using EFHIA, would be of use in ensuring health equity is considered earlier in the formulation of policy options.

4. Missed opportunities

There was a degree of ambivalence towards the rapid EFHIA on the part of several of the interviewees, amongst both people from the Department of Health and the EFHIA working group. The terms "lost opportunity" or "missed opportunity" came up several times during the interviews. Whilst all five interviewees acknowledged that the EFHIA had some degree of direct and indirect impact on subsequent activities, three interviewees expressed disappointment that more didn't come from doing the EFHIA, in the form of either more robust consideration of equity in health policy in general or ongoing collaboration with the expert panel.

No, it was really a lost opportunity, I think, to get people engaged, and not only engaged in HIA, but in equity...

Interviewee 1

I think there might have been a missed opportunity... The EFHIA focused too much on issues that would have been addressed at later stages in the planning anyway.

Interviewee 4

This can be attributed, at least in part, to feelings that the Department was not able to be fully involved in the EFHIA due to the competing pressures involved in finalising the implementation plan. The EFHIA was viewed by two of the interviewees as being overly critical of the development of the implementation plan and failing to recognise the time-pressured and politicised context it was being developed in.

...by doing an HIA, if you start then telling people how to do good planning, it's almost like it's a little bit insulting to those who believe they are good planners, rightly or wrongly... So I think there's a fine line between telling people how to suck eggs, when they already know how to suck eggs, but doing it in a different way.

Interviewee 3

I think people felt when recommendations came in, that they saw as a critique, or not that they were a critique, because different... They were like 'Oh, but it wasn't a proper plan anyway, it was just, you know, we were just trying to get the money, and that was our goal at that time, just get the money, and we said we'd do this, but not sure if we really will'.

Interviewee 1

This suggests that the involvement of stakeholders and decision-makers in the process of EFHIA is more than an ideological commitment to participation and representation; it is critical in enabling it to have an impact on decision-making and implementation [80, 81]. There are of course significant, and possibly insurmountable, tensions between the rapid processes required if EFHIA is to inform policy development and implementation in a timely fashion and the need to engage stakeholders fully in the process of conducting the EFHIA.

5. Differing conceptualisations of the purpose of the EFHIA

Although all participants described changes that were attributable to the EFHIA, in general the Department of Health staff described the EFHIA as making a more modest contribution to the development and implementation of the implementation plan than the EFHIA working group did. This difference may be attributed to greater involvement and familiarity with the ongoing development and roll-out of the implementation plan or differing understandings about the role and purpose of the EFHIA.

We didn't have a shared understanding of why we were undertaking it. Our purposes were probably different from CHETRE's purposes, and maybe that's where they don't work, but if you have two differing purposes, unless you can fully appreciate what those two different purposes are, maybe it doesn't work out as well as it could...

...I think there was a feeling that, well, we could get something out of [the EFHIA]. There were probably two rationales for why it would be useful. One is that we could get some, a critique if you like, or some feedback about, through an equity lens, on the strategies that we had proposed. And the second one was that it would perhaps serve a process of helping people who are more engaged in the consultation process.

Interviewee 3

In a way, it was about improving the quality of the document, it was actually quite important to be able to debate some of the issues.

Interviewee 2

These differences may be partly due to variations in the way people involved in the EFHIA understood the purpose of the HIA in general. Those from the Department of Health tended to describe HIA as a process for using evidence to informing decision-making, whereas those from the EFHIA working group tended to describe HIA as a process for quality enhancement and examining unanticipated impacts.

There is an increasing consensus internationally that impact assessment should be understood as a learning activity [16, 8285]. Glasbergen [86] describes three types of learning that can occur through impact assessment:

  • Technical learning, which involves searching for technical solutions to fixed policy objectives;

  • Conceptual learning, which involves redefining policy goals, problem definitions and strategies; and

  • Social learning, which emphasises dialogue and increased interaction between policy actors (this is distinct from the concept of social learning described in the psychology literature [87]).

Some of the differences in this case may be understood as different attitudes to desired learning goals of the EFHIA. Many of those from the Department of Health described the EFHIA as a technical learning activity; those involved in the working group described the EFHIA in terms more consistent with conceptual learning. There was very little discussion of impacts that might be classed as social learning within the impact evaluation interviews. Understanding the different types of learning that may come from an EFHIA, and which one is desired within a specific context, is important as different expectations may serve to create confusion and tension amongst those involved. In our work we would describe HIA as both a technical tool and a process and it is this process that provides the opportunities for conceptual and social learning to build ongoing relationships with other stakeholders.

Resource description findings

Costs and time details are included in this paper to transparently report what resources were used to undertake the rapid EFHIA and to assist future cost utility analyses [10]. The resources invested were estimated by the EFHIA working group and are detailed in Table 4.
Table 4

Estimation of resources invested to undertake the EFHIA

Resource

Project Hours

(If Applicable)

Estimated Cost/Hour

(If Applicable, USD)

Cost Estimate

(USD)

EFHIA Workshop

42 hrs (Includes Participants)

$34.80

$1,461.60

Report Writing

48 hrs

$34.80

$1,670.40

Review and Comment on Report

8 hrs

$34.80

$278.40

Report Formatting, Referencing and Proof Reading

8 hrs

$34.80

$278.40

Travel Costs (1 Airfare)

-

-

$227.00

Catering for Workshop

-

-

$120.00

TOTAL

106

-

US $4,035.80

In Kind Subtotal*

106

-

US $3,688.80

Cash Subtotal

-

-

US $347.00

* In-kind costs refers to people's time that was donated, rather than being paid for directly to undertake the EFHIA

The limited number of papers describing the human resource investments made suggest that between 684 and 3,784 project hours for an HIA are not uncommon [88, 89]. This suggests that the estimated 106 project hours for this EFHIA was little by comparison.

Costs are also rarely described in the literature but 15 HIAs conducted in Europe have been reported to range between US $1,316 and US $190,878 [73], 15 English HIAs included in a cost benefit study ranged between US $1,578 and US $93,006 [74], and the Merseyside Guidelines for Health Impact Assessment reported in 2000 that the mean cost of three HIAs conducted in Liverpool was US $18,033 [90]. This suggests that at US $4,025.80 the EFHIA of the ABHI implantation plan is one of the least costly HIAs that has been documented.

Limitations

This is paper describes a rapid, specific and contextually situated EFHIA. Care should be taken not to over-generalise the findings to other settings especially as it was conducted in response to the needs of a specific decision-making context.

A limitation in how the EFHIA was conducted was that the rapid EFHIA process relied on expert opinion from a relatively small group. Consultation was limited, as was systematic review of the literature, both largely due to time constraints. There was limited reference to increased dialogue or increased interaction between those involved following the EFHIA. This may suggest that social learning [86] through this EFHIA was limited. This could be due to its rapid nature, though other factors that were not identified may also have limited the extent of further collaboration.

There were also limitations in terms of how the EFHIA's impact was evaluated. Firstly, unlike some other HIAs whose impact has been evaluated [72] there was unfortunately no final document against which recommendations from this EFHIA could be checked off. This is often a feature of higher-level government implementation plans that cross a number of portfolio areas. This is a limitation that should be borne in mind when considering the EFHIA's direct impacts on decision-making. Secondly, a relatively small number of people were interviewed (five) for the impact evaluation. This is because this is the number of people who were intimately involved in both the EFHIA and the further development of the policy implementation plan was limited. This was in part due to the EFHIA's rapid nature, partly due to context-specific practices, i.e. who is involved in the development of policy implementation plans in NSW. Thirdly, several of this paper's authors were involved in conducting the EFHIA. The second, third and fourth authors played an active role (see section on Authors' Contributions). Whilst efforts have been made to ensure that the findings are empirically supported, their involvement in the EFHIA process may have influenced the interpretation of findings. Lastly there is a possibility of recall bias, as some of those interviewed may have revisited the recommendations more often than others interviewed [91].

It is important to note that this paper is not solely an impact evaluation of a rapid EFHIA; it also seeks to describe the methods by which the EFHIA was conducted in some detail. This is because there are relatively few examples of EFHIAs reported in the literature to date, something that is required given the World Health Organization's recent calls for the use of health equity impact assessment [4]. Despite the limitations outlined, measures were taken to ensure procedural fidelity to EFHIA guidance [46] and the features of the impact evaluation interviews are outlined in Table 4.

Conclusions

Although it was only a rapid process this EFHIA had an impact on the development of the implementation plan. The EFHIA was well received by the Department of Health and its recommendations were incorporated into the NSW ABHI implementation plan's revision. Those responsible for developing specific sections of the implementation plan were asked to demonstrate how they had addressed issues raised in the EFHIA in their section.

This rapid EFHIA process relied to a large extent on expert opinion from a small group of people. There was little capacity to consult with other stakeholders or to systematically review the literature. Despite this the EFHIA has had an impact on the ways in which the ABHI initiatives were planned. This process also highlighted that in many areas, even if there had been more time for a detailed assessment, there was little direct evidence relating to potential inequities or on effective interventions to prevent or redress them [4]. The ability to adapt existing knowledge to new contexts will be an important skill required in future rapid EFHIAs.

This EFHIA has demonstrated that HIA processes can be used within the political realities and time frames within which policy-makers operate. It demonstrated that EFHIA specifically, and HIA generally, can make a contribution to the implementation of health sector initiatives, not just other's sectors decision-making. It was also highlighted as a example of action towards enhanced capacity for monitoring, research, and intervention in the Final Report of the World Health Organization Commission on the Social Determinants of Health[4].

This process would not have been possible without the support of NSW Department of Health and the willingness of those involved in the development of the health promotion components of the NSW Department of Health Australian Better Health Initiative (ABHI) Implementation Plan to have their work scrutinised by people who largely worked outside the Department. It was also feasible to undertake the assessment within the time constraints due to the involvement of an expert panel with knowledge of the policy area and the ways in which the health system operated. Because this group of people were also experienced in working with government policy processes they were also able to concentrate on how potential problems could be minimised and potential gains enhanced within resource constraints.

It is important that whilst EFHIA can have impacts on decision-making and planning that it not be regarded as a panacea. The evidence that informed the EFHIA was limited and the assessment itself was not comprehensive, though nor did it claim to be. There is a need to be realistic about the extent to which a rapid process can be expected to systematically inform subsequent activities. Given that many policies require considerable time, expertise and resources to develop, however, an investment of four days to ensure that health equity issues have been explicitly considered may be regarded as time well spent.

A major challenge for all HIAs is to be able to respond in flexible and timely ways to the needs of policy-makers who are often developing proposals within brief timeframes and in politicised contexts. A rapid EFHIA process may provide a practical mechanism for looking at the potential health equity impacts of proposed initiatives.

Abbreviations

ABHI: 

Australian Better Health Initiative

CHETRE: 

Centre for Health Equity Training, Research and Evaluation

COAG: 

Council of Australian Governments

EFHIA: 

Equity focused health impact assessment

EqIA: 

Equality impact assessment

HEIA: 

Health equity impact assessment

HIA: 

Health impact assessment

NSW: 

New South Wales.

Declarations

Acknowledgements

This EFHIA was undertaken under the auspices of the NSW HIA Project, which was funded by NSW Health. The authors would like to recognise the contribution of the staff of the Centre for Health Advancement in the NSW Department of Health and the staff of the UNSW Centre for Primary Health Care and Equity for supporting the EFHIA.

Authors’ Affiliations

(1)
Centre for Health Equity Training, Research and Evaluation (CHETRE), part of the UNSW Research Centre for Primary Health Care and Equity, School of Public Health and Community Medicine, University of New South Wales

References

  1. Commission of the European Communities: Together for Health: A Strategic Approach for the EU 2008-2013. White Paper. 2007, Brussels: Commission of the European CommunitiesGoogle Scholar
  2. Thai National Health Commission: National Health Act, B.E.2550. 2007, Bangkok: The Kingdom of ThailandGoogle Scholar
  3. Gagnon F, Turgeon J, Dallaire C: Healthy public policy: A conceptual cognitive framework. Health Policy. 2007, 81 (1): 42-55. 10.1016/j.healthpol.2006.05.012.View ArticlePubMedGoogle Scholar
  4. WHO: Closing the Gap in a Generation: Health equity through action on the social determinants of health. 2008, Geneva: Commission on the Social Determinants of Health, World Health OrganizationGoogle Scholar
  5. WHO: World Health Report 2008: Primary health care - Now more than ever. 2008, Geneva: World Health OrganizationGoogle Scholar
  6. Dahlgren G, Whitehead M: Levelling up (part 2): a discussion paper on European strategies for tackling social inequities in health. 2006, Copenhagen,: WHO Regional Office for EuropeGoogle Scholar
  7. Wismar M, Blau J, Ernst K, Elliott E, Golby A, van Herten L, Lavin T, Stricka M, Williams G: Implementing and Institutionalizing Health Impact Assessment in Europe. Health in All Policies: Prospects and potentials. Edited by: Ståhl T, Wismar M, Ollila E, Lahtinen E, Leppo K. 2006, Helsinki: Ministry of Social Affairs and Health, 231-252.Google Scholar
  8. Equator Principles: The Equator Principles: A financial industry benchmark for determining, assessing and managing social & environmental risk in project financing. 2006, Washington D.C.: Equator Principles Financial InstitutionsGoogle Scholar
  9. IFC: Performance Standards on Social & Environmental Sustainability. 2006, Washington DC: International Finance Corporation, World Bank GroupGoogle Scholar
  10. Kemm J: Health Impact Assessment and Health in All Policies. Health in All Policies: Prospects and potentials. Edited by: Ståhl T, Wismar M, Ollila E, Lahtinen E, Leppo K. 2006, Helsinki: Ministry of Social Affairs and Health, 189-208.Google Scholar
  11. Kemm J, Parry J, Palmer S: Health Impact Assessment. 2004, Oxford, New York: Oxford University PressView ArticleGoogle Scholar
  12. Acheson D: Independent Inquiry into Inequalities in Health. 1998, London: Stationery OfficeGoogle Scholar
  13. Sukkumnoed D: Better Power for Health: Healthy public policy and sustainable energy in the Thai power sector. PhD. 2007, Aalborg: Aalborg UniversityGoogle Scholar
  14. Sukkumnoed D, Nuntavorakarn S, Phoolchaeron W, Sabrum N, Sukkumnoed R: Toward Healthy Society: Healthy public policy and health impact assessment in Thailand. 2005, BangkokGoogle Scholar
  15. Health Impact Assessment in Australia and New Zealand: [http://www.hiaconnect.edu.au/other_australian_initiatives.htm]
  16. Harris-Roxas B, Harris P: Learning by Doing: The value of case studies of health impact assessment. NSW Pub Health Bull. 2007, 18 (9-10): 161-163. 10.1071/NB07110.View ArticleGoogle Scholar
  17. Harris-Roxas B, Simpson S: The New South Wales Health Impact Assessment Project. NSW Pub Health Bull. 2005, 16 (7-8): 120-123.Google Scholar
  18. NPHP: Health Impact Assessment: Legislative and administrative frameworks. 2005, Melbourne: National Public Health PartnershipGoogle Scholar
  19. Morgan RK: Institutionalising Health Impact Assessment: The New Zealand experience. Impact Assessment and Project Appraisal. 2008, 26 (1): 2-16. 10.3152/146155108X300547.View ArticleGoogle Scholar
  20. PHAC: An Idea Whose Time Has Come: New opportunities for health impact assessment in New Zealand public policy and planning. 2007, Wellington: Public Health Advisory CommitteeGoogle Scholar
  21. Signal L, Durham G: Health Impact Assessment in the New Zealand Policy Context. Social Policy Journal of New Zealand. 2000, 15: 11-26.Google Scholar
  22. Dannenberg AL, Bhatia R, Cole BL, Heaton SK, Feldman JD, Rutt CD: Use of Health Impact Assessment in the US: 27 Case Studies, 1999-2007. American Journal of Preventive Medicine. 2008, 34 (3): 241-256. 10.1016/j.amepre.2007.11.015.View ArticlePubMedGoogle Scholar
  23. Cole BL, Shimkhada R, Fielding JE, Kominski G, Morgenstern H: Methodologies for Realizing the Potential of Health Impact Assessment. American Journal of Preventive Medicine. 2005, 28 (4): 382-389. 10.1016/j.amepre.2005.01.010.View ArticlePubMedGoogle Scholar
  24. Rutt C, Pratt M, Dannenberg A, Cole B: Connecting Public Health and Planning Professionals: Health impact assessment. Places: A Forum of Environmental Design. 2005, 17 (1): 86-87.Google Scholar
  25. Dannenberg AL, Bhatia R, Cole BL, Dora C, Fielding JE, Kraft K, McClymont-Peace D, Mindell J, Onyekere C, Roberts JA, Ross CL, Rutt CD, Scott-Samuel A, Tilson HH: Growing the Field of Health Impact Assessment in the United States: An agenda for research and practice. Am J Public Health. 2006, 96 (2): 262-270. 10.2105/AJPH.2005.069880.PubMed CentralView ArticlePubMedGoogle Scholar
  26. Vohra S: International perspective on health impact assessment in urban settings. New South Wales Public Health Bulletin. 2007, 18 (9-10): 152-154. 10.1071/NB07085.View ArticlePubMedGoogle Scholar
  27. ECHP: Gothenburg Consensus Paper on Health Impact Assessment: Main concepts and suggested approach. 1999, Brussels: European Centre for Health Policy, WHO Regional Office for EuropeGoogle Scholar
  28. Banken R: Strategies for Institutionalising HIA. ECHP Health Impact Assessment Discussion Papers Number 1. 2001, Brussels: European Centre for Health Policy, WHO EuropeGoogle Scholar
  29. Wright J, Parry J, Scully E: Institutionalizing Policy-Level Health Impact Assessment in Europe: Is coupling health impact assessment with strategic environmental assessment the next step forward?. Bulletin of the World Health Organization. 2007, 83 (6): 472-477.Google Scholar
  30. Douglas MJ, Scott-Samuel A: Addressing Health Inequalities in Health Impact Assessment. JECH. 2001, 55 (7): 450-451.Google Scholar
  31. Harris-Roxas B, Simpson S, Harris E: Equity Focused Health Impact Assessment: A literature review. 2004, Sydney: CHETRE on behalf of the Australasian Collaboration for Health Equity Impact AssessmentGoogle Scholar
  32. Mackenbach J, Veerman L, Barendregt J, Kunst A: Health Inequalities and HIA. Health Impact Assessment. Edited by: Kemm J, Parry J, Palmer S. 2004, Oxford, New York: Oxford University Press, 25-39.Google Scholar
  33. Parry J, Scully E: Health impact assessment and the consideration of health inequalities. Journal of Public Health Medicine. 2003, 25 (3): 243-245. 10.1093/pubmed/fdg042.View ArticlePubMedGoogle Scholar
  34. Blau J, Ernst K, Wismar M, Baro F, Blenkus MG, von Bremen K, Fehr R, Gulis G, Kauppinen T, Mekel O, Nelimarkka N, Perttilä K, Scagnetti N, Sprenger M, Stegeman I, Welteke R: The use of HIA across Europe. The Effectiveness of Health Impact Assessment: Scope and limitations of supporting decision-making in Europe. Edited by: Wismar M, Blau J, Ernst K, Figueras J. 2007, Copenhagen, Denmark: World Health Organization Regional Office for Europe, on behalf of the European Observatory on Health Systems and Policies, 37-55.Google Scholar
  35. Kemm J: The future challenges for HIA. Environmental Impact Assessment Review. 2005, 25 (7-8): 799-807. 10.1016/j.eiar.2005.07.012.View ArticleGoogle Scholar
  36. Davenport C, Mathers J, Parry J: Use of Health Impact Assessment in Incorporating Health Considerations in Decision Making. JECH. 2006, 60: 196-201.Google Scholar
  37. Krieger N, Northridge M, Gruskin S, Quinn M, Kriebel D, Davey Smith G, Bassett M, Rehkopf DH, Miller C: Assessing Health Impact Assessment: Multidisciplinary and international perspectives. JECH. 2003, 57: 659-662.Google Scholar
  38. Fell G, Haroon S: Learning from a Rapid Health Impact Assessment of a Proposed Maternity Service Reconfiguration in the English NHS. BMC Public Health. 2008, 8 (138):
  39. Ahmad BS: Integrating Health into Impact Assessment: Challenges and opportunities. Impact Assessment & Project Appraisal. 2004, 22 (1): 2-4.View ArticleGoogle Scholar
  40. Birley M: A Review of Trends in Health-Impact Assessment and the Nature of the Evidence Used. Environmental Management and Health. 2002, 13 (1): 21-39. 10.1108/09566160210417804.View ArticleGoogle Scholar
  41. Bond A: Lessons from EIA. Health Impact Assessment. Edited by: Kemm J, Parry J, Palmer S. 2004, Oxford: Oxford University Press, 131-142.Google Scholar
  42. Kemm J: Can Health Impact Assessment Fulfil the Expectations it Raises?. Public Health. 2000, 114: 432-433. 10.1016/S0033-3506(00)00382-6.View ArticleGoogle Scholar
  43. Everett S: The Policy Cycle: Democratic process or rational paradigm revisited?. Australian Journal of Public Administration. 2003, 32 (2): 65-70. 10.1111/1467-8497.00325.View ArticleGoogle Scholar
  44. Stewart-Weeks M: From Control to Networks. Beyond the Policy Cycle: The policy process in Australia. Edited by: Colebatch H. 2006, Sydney: Allen & Unwin, 184-202.Google Scholar
  45. Quigley R, Cave B, Elliston K, Pratt A, Vohra S, Taylor L: Practical lessons for dealing with inequalities in health impact assessment. 2005, London: Health Development AgencyGoogle Scholar
  46. Mahoney M, Simpson S, Harris E, Aldrich R, Stewart Williams J: Equity Focused Health Impact Assessment Framework. 2004, Newcastle: Australasian Collaboration for Health Equity Impact AssessmentGoogle Scholar
  47. Simpson S, Mahoney M, Harris E, Aldrich R, Stewart-Williams J: Equity-Focused Health Impact Assessment: A tool to assist policy makers in addressing health inequalities. Environmental Impact Assessment Review. 2005, 25: 772-782. 10.1016/j.eiar.2005.07.010.View ArticleGoogle Scholar
  48. Harris P, Harris-Roxas B, Kemp L: Health Impact Assessment in Urban Settings (Editorial). NSW Pub Health Bull. 2007, 18: 149-150. 10.1071/NB07104.View ArticleGoogle Scholar
  49. UCL: Strategic Review of Health Inequalities in England Post 2010 (Marmot Review). 2010, London: University College LondonGoogle Scholar
  50. Acheson D: Health Inequalities Impact Assessment. Bulletin of the World Health Organisation. 2000, 75-76. 78Google Scholar
  51. WHO Europe: Health Impact Assessment: From theory to practice. Report on the Leo Kaprio Workshop Gothenburg 28-30 Octiber 1999. 2001, Stockholm: Nordic School of Public HealthGoogle Scholar
  52. Barnes R: Equity and Health Impact Assessment Seminar Report, Manchester 16-17 June. 2000, Liverpool: Liverpool Public Health ObservatoryGoogle Scholar
  53. WHO: The Jakarta Declaration on Leading Health Promotion into the 21st Century. 1997, Geneva: World Health OrganisationGoogle Scholar
  54. WHO: The Bangkok Charter on Health Promotion in a Globalized World. 2006, Geneva: World Health OrganisationGoogle Scholar
  55. Stewart Williams J, Aldrich R, Mahoney M, Harris E, Simpson S: Equity Focused Health Impact Assessment: Working collaboratively for a strategic framework. 2004, Newcastle: Australasian Collaboration for Health Equity Impact AssessmentGoogle Scholar
  56. Harris P, Harris-Roxas B, Harris E, Kemp L: Health Impact Assessment: A practical guide. 2007, Sydney: UNSW Research Centre for Primary Health Care and Equity and NSW HealthGoogle Scholar
  57. Abott T: Media Release: Better Health For All Australians. 2006, Canberra: Australian Government Minister for Health and AgeingGoogle Scholar
  58. Turrell G, Stanley L, de Looper MBO: Health Inequalities in Australia: Morbidity, health behaviours, risk factors and health service use. Health Inequalities Monitoring Series No 2. 2006, Canberra: Queensland University: AIHWGoogle Scholar
  59. AIHW: Chronic Diseases and Associated Risk Factors in Australia. 2006, Canberra: Australian Institute of Health and Welfare, Cat. No PHE, 81Google Scholar
  60. Aldrich R, Kemp L, Williams JS, Harris E, Simpson S, Wilson A, McGill K, Byles J, Lowe J, Jackson T: Using socioeconomic evidence in clinical practice guidelines. BMJ. 2003, 327 (7426): 1283-1285. 10.1136/bmj.327.7426.1283.PubMed CentralView ArticlePubMedGoogle Scholar
  61. Lester C, Temple M: Health Impact Assessment and Community Involvement in Land Remediation Decisions. Public Health. 2006, 120 (10): 915-922. 10.1016/j.puhe.2006.05.011.View ArticlePubMedGoogle Scholar
  62. Lester C, Temple M: Rapid Collaborative Health Impact Assessment: A three-meeting process. Public Health. 2004, 118: 218-224. 10.1016/j.puhe.2003.08.005.View ArticlePubMedGoogle Scholar
  63. PHAC: A Guide to Health Impact Assessment: A policy tool for New Zealand. 2005, Wellington: Public Health Advisory Committee, 2Google Scholar
  64. Signal L, Martin J, Cram F, Robson B: Health Equity Assessment Tool: A user's guide. 2008, Wellington: New Zealand Ministry of HealthGoogle Scholar
  65. Harris E, Harris P, Kemp L: Rapid Equity Focused Health Impact Assessment of the Australia Better Health Initiative: Assessing the NSW components of priorities 1 and 3. 2006, Sydney: Centre for Primary Health Care and Equity, University of New South WalesGoogle Scholar
  66. Quigley R, Taylor L: Evaluating Health Impact Assessment. Public Health. 2004, 118 (8): 544-552. 10.1016/j.puhe.2003.10.012.View ArticlePubMedGoogle Scholar
  67. Parry JM, Kemm JR, Evaluation of Health Impact Assessment W: Criteria for use in the evaluation of health impact assessments. Public Health. 2005, 119 (12): 1122-1129. 10.1016/j.puhe.2005.05.002.View ArticlePubMedGoogle Scholar
  68. Tong A, Sainsbury P, Craig J: Consolidated Criteria for Reporting Qualitative Research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care. 2007, 19 (6): 349-357. 10.1093/intqhc/mzm042.View ArticlePubMedGoogle Scholar
  69. Colaizzi PF: Psychological Research as the Phenomenologist Views It. Existential Phenomenological Alternatives for Psychology. Edited by: Valle RS, King M. 1978, New York: Oxford University Press, 48-71.Google Scholar
  70. Wojnar DM, Swanson KM: Phenomenology: An Exploration. Journal of Holistic Nursing. 2007, 25 (3): 172-180. 10.1177/0898010106295172.View ArticlePubMedGoogle Scholar
  71. NSW Health: NSW Health and Equity Statement: In All Fairness. 2004, Sydney: New South Wales Department of HealthGoogle Scholar
  72. Mathias K, Harris-Roxas B: Process and Impact Evaluation of the Greater Christchurch Urban Development Strategy Health Impact Assessment. BMC Public Health. 2009, 9 (1): 97-10.1186/1471-2458-9-97.PubMed CentralView ArticlePubMedGoogle Scholar
  73. Wismar M, Blau J, Ernst K, Figueras J, (eds.): The Effectiveness of Health Impact Assessment: Scope and limitations of supporting decision-making in Europe. 2007, Copenhagen: European Observatory on Health Systems and Policies, World Health Organization
  74. O'Reilly J, Trueman P, Redmond S, Yi Y, Wright D: Cost Benefit Analysis of Health Impact Assessment. 2006, York: York Health Economics ConsortiumGoogle Scholar
  75. Veerman JL, Bekker MP, Mackenbach JP: Health Impact Assessment and Advocacy: A challenging combination. Sozial-und Praventivmedizin. 2006, 51: 151-152. 10.1007/s00038-006-0034-z.View ArticlePubMedGoogle Scholar
  76. Bekker M: The Politics of Healthy Policies: Redesigning health impact assessment to to integrate health in public policy. 2007, Delft: Eburon Uitgeverij B.VGoogle Scholar
  77. Sadler B: Environmental Assessment in a Changing World: Evaluating to improve performace - Final report of the International Study of the Effectiveness of Environmental Assessment. 1996, Ottawa: Canadian Environmental Assessment Agency and the International Association for Impact AssessmentGoogle Scholar
  78. Steinemann A: Improving alternatives for environmental impact assessment. Environmental Impact Assessment Review. 2001, 21 (1): 3-21. 10.1016/S0195-9255(00)00075-5.View ArticleGoogle Scholar
  79. Benson JF: What is the alternative? Impact assessment tools and sustainable planning. Impact Assessment and Project Appraisal. 2003, 21 (4): 261-280. 10.3152/147154603781766185.View ArticleGoogle Scholar
  80. Kearney M: Walking the Walk? Community Participation in HIA: A qualitative interview study. Environmental Impact Assessment Review. 2004, 24 (2): 217-229. 10.1016/j.eiar.2003.10.012.View ArticleGoogle Scholar
  81. Mahoney M, Potter JL, Marsh R: Community participation in HIA: Discords in teleology and terminology. Critical Public Health. 2007, 17 (3): 229-241. 10.1080/09581590601080953.View ArticleGoogle Scholar
  82. Fiorino DJ: Environmental Policy As Learning: A new view of an old landscape. Public Administration Review. 2001, 61 (3): 322-334. 10.1111/0033-3352.00033.View ArticleGoogle Scholar
  83. Hayward G, Diduck A, Mitchell B: Social Learning Outcomes in the Red River Floodway Environmental Assessment. Environmental Practice. 2007, 9 (4): 239-250. 10.1017/S1466046607070561.View ArticleGoogle Scholar
  84. Muro M, Jeffrey P: A Critical Review of the Theory and Application of Social Learning in Participatory Natural Resource Management Processes. Journal of Environmental Planning and Management. 2008, 51 (3): 325-344. 10.1080/09640560801977190.View ArticleGoogle Scholar
  85. Saarikoski H: Environmental Impact Assessment (EIA) as Collaborative Learning Process. Environmental Impact Assessment Review. 2000, 20 (6): 681-700. 10.1016/S0195-9255(00)00059-7.View ArticleGoogle Scholar
  86. Glasbergen P: Learning to Manage the Environment. Democracy and the Environment: Problems and Prospects. Edited by: Lafferty W, Meadowcroft J. 1999, Cheltenham: Edward Elgar, 175-193.Google Scholar
  87. Bandura A, Walters R: Social Learning and Personality Development. 1963, New York: Holt, Rinehart and WinstonGoogle Scholar
  88. Abdel Aziz M, Radford J, McCabe J: The Finningley Airport HIA: A case study in health impact assessment. Health Impact Assessment. Edited by: Kemm J, Parry J, Palmer S. 2004, Oxford: Oxford University Press, 285-298.Google Scholar
  89. Close N: Alconbury Airfield Development Health Impact Assessment Evaluation Report. 2001, Cambridge: Cambridgeshire Health AuthorityGoogle Scholar
  90. Scott-Samuel A, Birley M, Ardern K: The Merseyside Guidelines for Health Impact Assessment. 2001, Liverpool: International Health Impact Assessment Consortium (IMPACT), 2Google Scholar
  91. Sackett D: Bias in analytic research. Journal of Chronic Disease. 1979, 32: 51-63. 10.1016/0021-9681(79)90012-2.View ArticleGoogle Scholar
  92. QSR: NVivo. 2008, Melbourne: QSR International, 8Google Scholar

Copyright

© Harris-Roxas et al; licensee BioMed Central Ltd. 2011

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.