- Letter to the Editor
- Open Access
Following the legacy of professors Barbara Starfield and Leiyu Shi in Brazil as health policy: the National Health Survey (PNS), led by the Brazilian National Institute of Geography and Statistics (IBGE) and the Primary Care Assessment Tool (PCAT)
International Journal for Equity in Health volume 18, Article number: 176 (2019)
We present to the scientific community the pioneering of Brazilian National Institute of Geography and Statistics (IBGE, the Brazilian Census Bureau) in partnership with the Ministry of Health, the largest fieldwork ever conducted in a single country in the world, using the PCAT in a national household sample survey, visiting more than 100,000 households and 40% of the country’s municipalities. In Brazil, PCAT is being consolidated as an instrument to support public policy for the evaluation of primary health care. We believe that it represents a virtuous example of dialogue between scientific community and health management, following the legacy of Professors Barbara Starfield and Leiyu Shi.
Since the early 2000s, when the team led by professors Barbara Starfield and Leiyu Shi of the Johns Hopkins Bloomberg School of Public Health proposed to the scientific community the Primary Care Assessment Tools (PCAT) [1,2,3] to the evaluation of Primary Health Care (PHC) in the United States, dozens of other studies have been developed over the 20 last years on all continents, using validated versions adapted to the reality of each country, especially in North America, Latin America and most recently with the leadership of Professor Leiyu Shi in Asia [4, 5]. In low and middle-income countries, the access / first contact attribute remains the most difficult to reach when its PCAT score is calculated. Comprehensiveness is what most differentiates cities in each country in which the instrument has been used across continents .
But who could imagine that in almost 20 years later, a country of continental dimensions like Brazil, with huge social inequality, would adopt this instrument as an important part of a public policy, evaluating the Family Health Teams that work in primary health care and, to this end, would initiate the major household sample survey to create a national, regional and local baseline for each of the 27 federation units? Who would have thought that from Oiapoque to Chuí (from north to south of Brazil) the enormous challenge of collecting data with probabilistic household samples could be accomplished across the nation?
In Brazil, PCAT questionnaires using the same Starfield-Shi four-point Likert scale was validated and published in 2006 by Harzheim, Starfield, Rajmil, Álvarez-Dardet and Stein . In 2010, Ministry of Health, with the technical support of Professor Starfield herself published PCAT expanded questionnaires . To adapt it to the Brazilian reality, each original version of the instrument was transformed into an applicable tool by interviewers and went through a process of translation and reverse translation, adaptation, debriefing and validation of content and construct, as well as reliability analysis. From a scientific point of view, PCAT remains the one in which the techniques recommended by the Statistical Sciences for questionnaire validation are followed (multivariate / factorial analysis, correlation analysis, use of item-response theory, validity and reliability by Cronbach’s alpha coefficient) .
Until 2016, the largest sample in the world (n = 6675) performed in a single city occurred in Rio de Janeiro, Brazil, the second largest in the country, with the application of the instrument in expanded versions of adult and child users in family health units , including the most known poor community of Brazil, the neighborhood of Rocinha . In Spain, the Catalonia region included a reduced PCAT version for both adult and children  in a national survey in 2006, with the participation of Professor Barbara Starfield.
Like the United States Census Bureau, Brazilian National Institute of Geography and Statistics (IBGE) is huge, and has a national structure with more than 500 support offices in all five regions of the country. However, the giant heterogeneity of each federation unity and the difficulty of connectivity and linkage of administrative databases do not allow the use of interviews in national surveys by telephone or email. Therefore, a true “war operation” was planned in detail for the five-year National Health Survey (PNS), the main sample household survey of the country that portrays the most diverse facets of Brazilian population.
PNS emerged from breaking down the major growth of elements included in the National Household Sample Survey (PNAD) Special Health Supplements, published every 5 years since 1998 (1998, 2003, 2008, and then for the first time in 2013 as PNS), maintaining the same research aspects (set of questions or Modules) so that data may be compared to previous surveys. This survey is part of the IBGE Integrated System of Household Surveys (SIPD), and uses the “Master Sample” infrastructure. Sampling for PNS-2019 was designed to produce indicators for 80 geographic sections: the country, five regions, 27 states, 21 metropolitan regions and 27 state capitals (this one the maximum level of granularity allowed). It is important to point out the need to disclose estimates and their coefficients of variation.
Encouraged by one of the authors of this text, in August 2019, IBGE in partnership with the Ministry of Health started the survey fieldwork visiting more than 100,000 households in about 2000 municipalities (about 40% of the country’s municipalities) from all over the 27 states and regions of the country, interviewing an adult aged 18 or older in each household, and applying for the first time in its 20 years of Health Surveys, the reduced validated version of the PCAT, as a part of one of the PNS set of questions (Module H) .
To the best of our knowledge, it will be the first time that a country in the world applies the instrument on a nationwide sample scale, albeit in a reduced version so that it can draw a baseline for further comparisons in future national surveys.
Following the legacy of Professors Barbara Starfield and Leiyu Shi, we believe that strong primary care should follow the essential and derivative attributes stated in the 1990s by Shi and Starfield, who later had her work translated into Portuguese by United Nations Educational, Scientific, and Cultural Organization (UNESCO) and launched in Brazil in the same year 2002 . With Brazil’s pioneering health household national experience in 2019, PCAT is being consolidated as an instrument to support public policy for the evaluation of primary health care. It also considers a new federal financial model of pay for performance that will begin in 2020 . This match between scientific community and public health policy therefore represents a virtuous example of success in this partnership. So our tribute to Professors Barbara Starfield and Leiyu Shi is a way to keep the flame burning and the legacy built and under construction to demonstrate that yes, a nationwide scientific survey and primary care services integration is possible with representative random samples, from a statistical point of view, that is, with external validity, and its results can be generalized to the population as a whole, disclosing the scores found and their respective coefficients of variation.
Availability of data and materials
Not applicable. The National Survey of Health (PNS-2019) has just started to collect data from 100,000 household all over the 27 States of Brazil.
The authors chose the section “Letter to the Editor” modality, since the objective is to inform readers of what is considered as something unprecedented in the use of the PCAT instrument, that is, they believe it is the first time that a continental country such as Brazil conducted a national household sample survey with the application of the this evaluation tool in all regions, federation units and capitals of a country.
Cassady CE, Starfield B, Hurtado MP, Berk RA, Nanda JP, Friedenberg LA. Measuring consumer experiences with primary care. Pediatrics. 2000;105(4 Pt 2):998–1003.
Shi L, Starfield B, Xu J. Validating the adult primary care assessment tool. The Journal of Family Practice. 2001;50(2):161–4.
Johns Hopkins Bloomberg School of Public Health. The Johns Hopkins Primary Care Policy Center. Primary Care Assessment Tools [recurso eletrônico]. Baltimore, 2016. http://www.jhsph.edu/research/centers-and-institutes/johns-hopkins-primarycare-policy-center/pca_tools.html. Accessed 05 Sept 2019.
Wang W, Shi L, Yin A, Lai Y, Maitland E, Nicholas S. Development and validation of the Tibetan primary care assessment tool. BioMed Research International, May. 2014;21:308739. https://doi.org/10.1155/2014/308739.
Wei X, Li H, Yang N, Wong SY, Chong MC, Shi L, et al. Changes in the perceived quality of primary care in Shanghai and Shenzhen, China: a difference-indifference analysis. Bulletin of the World Health Organization. 2015;93(6):407–16. https://doi.org/10.2471/BLT.14.139527.
D’Avila OP, Pinto LF, Hauser L, Gonçalves MR, Harzheim E. The use of the Primary Care Assessment Tool (PCAT): an integrative review and proposed update. Ciênc. saúde coletiva, 22(3): 855–865, 2017. http://www.scielo.br/pdf/csc/v22n3/en_1413-8123-csc-22-03-0855.pdf, https://doi.org/10.1590/1413-81232017223.03312016 Accessed 17 Oct 2019.
Harzheim E, Starfield B, Rajmil L, Álvarez-Dardet C, Stein AT. Consistência interna e confiabilidade da versão em português do Instrumento de Avaliação da Atenção Primária (PCATool-Brasil) Para serviços de saúde infantil [internal consistency and reliability of primary care assessment tool (PCATool-Brasil) for child health services]. Cad Saúde Pública. 2006;22(8):1649–59 https://doi.org/10.1590/S0102-311X2006000800013. Accessed 19 Oct 2019.
Brasil. Ministério da Saúde. Secretaria de Atenção em Saúde. Departamento de Atenção Básica. Manual do instrumento de avaliação da atenção primária à saúde: primary care assessment tool pcatool - Brasil / Ministério da Saúde, Secretaria de Atenção em Saúde, Departamento de Atenção Básica. – Brasília : Ministério da Saúde, 2010, 80 p. http://bvsms.saude.gov.br/bvs/publicacoes/manual_avaliacao_pcatool_brasil.pdf Accessed 05 Sept 2019.
Hauser L. Aprimoramento do Instrumento de Avaliação da Atenção Primária à Saúde: PCATool-Brasil. Tese de Doutorado (Doctoral PhD Thesis). Porto Alegre, Rio Grande do Sul: Programa de Pós-Graduação em Epidemiologia, Faculdade de Medicina, Universidade Federal do Rio Grande do Sul (UFRGS), 2016.
Harzheim E, Pinto LF, Hauser L, Soranz D. Assessment of child and adult users of the degree of orientation of Primary Healthcare in the city of Rio de Janeiro, Brazil. Ciênc. saúde coletiva, 21(5): 1399–1408. http://www.scielo.br/pdf/csc/v21n5/en_14138123-csc-21-05-1399.pdf, https://doi.org/10.1590/1413-81232015215.26672015. Accessed 05 Sept 2019.
Pinto LF; Harzheim E; Hauser L; D’Avila OP; Gonçalves MR; Travassos P et al. Primary Health Care quality in Rocinha – Rio de Janeiro, Brazil, from the perspective of children caregivers and adult users Ciênc. saúde coletiva, 22(3): 771–781, 2017. http://www.scielo.br/pdf/csc/v22n3/en_1413-8123-csc-22-03-0771.pdf, https://doi.org/10.1590/1413-81232017223.33132016. Accessed 17 Oct 2019.
Berra S, Rocha KB, Rodríguez-Sanz M, Pasarín MI, Rajmil L, Borrell C, Starfield B. Properties of a short questionnaire for assessing primary care experiences for children in a population survey. BMC Public Health. 2011;11:285. https://doi.org/10.1186/14712458-11-285.
Brazilian National Institute of Geography and Statistics (IBGE). For the first time, IBGE carries out survey on patient check-in in public health system. https://agenciadenoticias.ibge.gov.br/en/agencia-news/2184-newsagency/news/25542-for-the-first-time-ibge-carries-out-survey-on-patient-check-in-inpublic-health-system Accessed 02 Oct 2019.
Starfield B. Atenção primária: equilíbrio entre necessidades de saúde, serviços e tecnologia. Brasil. Ministério da Saúde, 2002.
Conselho Nacional de Secretários Estaduais de Saúde (CONASS) [National Council of State Health Secretaries]. Pautas e Resumo CIT, 2019. https://www.conass.org.br/pautas-e-resumos-cit-2016/ Accessed 20 Oct 2019.
We would like to thank Brazilian National Institute of Geography and Statistics (IBGE) for the opportunity to adapt the former “Medical Care” Question Module (set of questions) which now contains the Adult PCAT questions in their reduced version validated in Brazil, by the same Likert scale used by Professor Barbara Starfield and Professor Leiyu Shi.
The Brazilian Ministry of Health and Brazilian National Institute of Geography and Statistics (IBGE) fund the National Survey of Health (PNS-2019) which has one of its part, addressing questions about Adult PCAT.
Ethics approval and consent to participate
The National Survey of Health (PNS-2019) was approved by the National Health Ethics Committee (CONEP) (process n° 3.529.376).
Consent for publication
By Federal Law, the IBGE, although collecting household data does not disclose data in order to identify the person interviewed, anonymizing the data collected.
EH published one article with Professor Barbara Starfield in 2006. The others have published some articles about PCAT Brazil.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Harzheim, E., Felipe Pinto, L., D’Avila, O.P. et al. Following the legacy of professors Barbara Starfield and Leiyu Shi in Brazil as health policy: the National Health Survey (PNS), led by the Brazilian National Institute of Geography and Statistics (IBGE) and the Primary Care Assessment Tool (PCAT). Int J Equity Health 18, 176 (2019). https://doi.org/10.1186/s12939-019-1083-2
- Primary health care