Design
The Delphi technique was used to identify and prioritize the most important strategies for reducing health inequities among the four aforementioned focus areas: leadership, policy, research, and diversity. The Delphi technique is a group communication process that allows experts to address complex problems without meeting face-to-face [28]. This technique has been used for a variety of purposes, including strategic planning [29] and priority setting [30, 31]. The Delphi technique is appropriate when subjective judgements from experts are useful to address complex issues that lack a clear factual basis for decision making. In addition, the technique allows experts from various disciplines and geographic regions to be convened in a way that reduces the risk of dominance by a small number of vocal participants [28]. For these reasons, the Delphi technique was deemed appropriate for this study. The overall study design is described in detail in the following sections (see Fig. 1 for study design structure). This study was reviewed by the American Academy of Family Physician’s Institutional Review Board and was deemed exempt under 45 CFR 46.101(B) - (HRP-312) Category 2.
Participants: sampling and recruitment
Individuals were recruited to the study based on their ability to provide meaningful insight into one or more of the focus areas (leadership, policy, diversity, and research). Every recruit was employed in a position requiring substantial expertise in principles important to health equity (social justice, engaging disadvantaged populations, advocacy, promoting diversity in medicine, increasing access to care, etc.). Participants came from various types of organizations, including national medical specialty societies and public health associations, medical schools and schools of public health, the Federal government, as well as medical practice and public health service. Every participant held an advanced degree (e.g., Doctor of Medicine, Doctor of Osteopathic Medicine, Doctor of Philosophy, Juris Doctorate, Master of Public Health, or other master’s degree). In addition to these qualifications, participants were recruited for their specific knowledge related to one or more of the focus areas. The leadership focus area included family physicians and individuals with experience collaborating with the health care sector to identify the skills and characteristics family physicians need to be health equity leaders. The policy focus area included individuals involved in policy development and advocacy to identify public policies needed to advance health equity. The research focus group included social epidemiologists, primary care researchers, and individuals responsible for translating evidence into practice to identify health equity research gaps. The diversity focus area included individuals involved in primary and secondary education, as well as medical education to identify policies and practices that could increase diversity in medicine Expertise was ascertained based on a combination of academic preparation, research and publication, service and leadership, as well as job experience.
Participants were recruited by email. Each participant was sent an introductory message that described the purpose and procedures of the study, the focus areas they were asked to respond to (a maximum of two), and a consent form. Participants that completed the consent form were enrolled in the study. The goal was to enroll 10 participants per focus area for each round. This goal was chosen because previous research on the Delphi technique suggests this is a minimum threshold, as well as the relatively rapid needs of the environmental scan and the number of focus areas being explored [32]. Forty-one individuals were recruited in round one and 66 individuals were recruited in rounds two and three to bolster the number of participants.
Data collection and analysis
SurveyMonkey was used to administer the surveys [33]. Data were collected in three rounds for each focus area separately, occurring over a three-month period in the Spring of 2018. Participants were asked to complete the survey in 2 weeks; however, up to 6 weeks were provided to increase the response rate. Three reminder emails were sent per round to increase the response rate. Individuals that completed a round were invited into the next round. Additional individuals were recruited if fewer than 10 individuals participated in a round.
Round one
The purpose of round one was to identify a comprehensive list of strategy statements related to each focus area that would be used in rounds two and three. To accomplish this, participants were asked an open-ended question about which strategies they thought were important to the focus area and to explain why they thought these strategies were important. The specific questions asked in each focus area were:
-
Leadership: What are the essential skills and characteristics family physicians need to become leaders for health equity?
-
Policy: What public policies are the most important for improving health equity?
-
Research: What are the most important areas of research needed to advance health equity?
-
Diversity: What are the most important policies and practices to increase diversity in medicine?
Participants were provided a definition of health equity and health inequities. For the study, health equity was defined as the “highest attainment of health for all people,” and health inequity was defined as the “differences in health that are avoidable, unfair, and unjust, and that are driven by social, economic, and environmental conditions.” These definitions were informed by the American Public Health Association and The Centers for Disease Control and Prevention [1, 34].
The participants’ input was used to develop a list of strategy statements to be used in round two. When participants provided clear and complete thoughts about a strategy, the language was used with only minor edits to develop the strategy statement. When participants provided incomplete or truncated thoughts about a strategy, the language for the strategy statement was informed by a literature review of systematic reviews, research articles, and editorials [12, 35,36,37,38,39]. Eighty-six strategy statements were provided in total (leadership: 17; policy: 34; diversity: 18; and research: 17) (Additional file 1).
Round two
The primary purpose of round two was to begin to prioritize the strategy statements identified in round one. Strategy statements were presented in random order to reduce order effect bias. Participants were asked to rate each strategy statement on a seven-point ordinal scale (1 = Not a Priority, 2 = Low Priority, 3 = Medium-Low Priority, 4 = Medium Priority, 5 = Medium-High Priority, 6 = High Priority, 7 = Essential). The research team suspected that participants may have high approval for every strategy statement and would rank them as all being priorities. This was in fact the case, and all but two of the 86 strategy statements were rated as being a priority (mean rating ≥ 4). To address the lack of variance, we also asked participants to rank the top three most important strategy statements. Bradburn and colleagues suggest using this type of approach when approval is expected to be high and when the list of statements is large [40]. The research team calculated mean and median ratings for all the strategy statements. A rank score was also calculated for all the strategy statements by assigning points to the rankings (three points for first, two points for second, and one point for third) and taking the sum across participants. Strategy statements that were ranked in the top three by any participant were retained for round three.
Round three
The purpose of round three was to finalize the prioritization of the strategy statements. Strategy statements were presented in order of the rank score from most to least important to reinforce the group’s input from round two. Ordinal scales were not used in this round due to the high approval observed in round two. Participants were asked to rank each strategy statement from most to least important in accordance with recommendations from Bradburn and colleagues [40]. Participants were also asked why they thought the statements were important, and what the most important things the AAFP should do for each focus area.
Final analysis
The data were analyzed by integrating quantitative rankings with qualitative data. For each focus area, the average rank and standard deviation for each statement were calculated using results from round three. Statements were ordered from the most to least important. Priority among ties was given to statements with a smaller standard deviation. The interquartile interval was calculated for the average rank to further stratify the statements.
Qualitative data from each round of the study were used to provide context and meaning to the prioritized lists [41, 42]. The research team used a deliberative and iterative process for the final analysis. Quotations were coded to inventory their meaning. Two researchers (KK and CL) worked together to clarify the meaning of the codes and come to agreement on codes. The entire research team further deliberated until consensus was reached. Quotations were then integrated with the prioritized lists by identifying statements raised in the qualitative data, selecting representative quotations, and triangulating this with the prioritized lists.