The Sustainable Development Goals have helped to focus attention on the importance of reducing inequality and ‘leaving no one behind’. Monitoring health inequalities is essential for providing evidence to inform policies, programmes and practices that can close existing gaps and achieve health equity. The Health Equity Assessment Toolkit (HEAT and HEAT Plus) software was developed by the World Health Organization to facilitate the assessment of within-country health inequalities.
HEAT contains a built-in database of disaggregated health data, while HEAT Plus allows users to upload and analyze inequalities using their own datasets. Version 4.0 of the software incorporated enhancements to the toolkit’s capacity for equity assessments. This includes a multilingual interface, interactive and downloadable visualizations, flexibility to analyze inequalities using any dataset of disaggregated data, and the built-in calculation of 19 summary measures of inequality. This paper outlines the improved features and functionalities of the HEAT and HEAT Plus software since their original release, highlighted through an example of how the toolkit can be used to assess inequalities in the COVID-19 pandemic era.
The features of the HEAT and HEAT Plus software make it a valuable tool for analyzing and reporting inequalities related to the COVID-19 pandemic, as well as its indirect impacts on inequalities in other health and non-health areas, providing evidence to inform equity-oriented interventions and strategies.
Monitoring health inequalities, or the measured differences in health across population subgroups, is essential to serve as both a warning system when health differences between population subgroups widen, informing equity-oriented responses, and as an evaluation system, to be able to assess the impact of health policies, programmes and interventions on the most-disadvantaged subgroups. The Sustainable Development Goals (SDGs) have helped to focus attention on the importance of reducing inequality and ‘leaving no one behind’. However, the COVID-19 pandemic has further revealed existing health inequalities, with the poorest and most vulnerable people and communities tending to be disproportionately affected . Moreover, social, political, and economic inequalities have amplified the impacts of the pandemic. Never has monitoring and tackling inequalities been more important. Monitoring inequalities in the COVID-19 pandemic (including inequalities in cases, deaths, vaccines and service access across different population groups), as in any health area, provides evidence to inform equity-oriented policies, programmes and practices.
The World Health Organization (WHO) works closely with national governments, regional offices and global public health actors to build capacity for monitoring inequalities in health, including the interpretation of results for decision-making to target existing gaps. Through this capacity building work, it was recognized that interpretation and priority-setting is greatly enhanced if analysts are equipped with a user-friendly tool that can help visualize disaggregated data as well as calculate summary measures of inequality, in order to better understand the latest situation of inequality and how inequalities have changed over time .
The Health Equity Assessment Toolkit (HEAT and HEAT Plus) software was subsequently developed in collaboration with capacity building efforts in countries and regions, and benefited from the testing and feedback of workshop participants and technical experts. The process of developing the software is described elsewhere in detail [2–4]. HEAT, the Built-in Database Edition, was first launched in 2016 and comes pre-installed with a global database of disaggregated health data . HEAT Plus, the Upload Database Edition, was launched in 2017 and allows users to upload and work with their own databases. HEAT and HEAT Plus are available via the WHO Health Inequality Monitor, which provides evidence, tools, resources and training on health inequality monitoring to advance health equity .
Both editions of the software have undergone several iterations since their launch, each incorporating additional improvements to enhance the toolkit’s capacity for equity assessments. Version 4.0 of HEAT and HEAT Plus was released in April 2021. The aim of this paper is to outline the improved features and functionalities of HEAT and HEAT Plus since their original release, highlighted through an example of how the toolkit can be used to assess inequalities in COVID-19 related indicators.
Overview of new features and improvements
In April 2021, Version 4.0 of HEAT and HEAT Plus was released, following a complete rebuild of the software to improve performance and add new features to enhance usability. The new features and improvements that have been introduced since the toolkit was first launched will be described in more detail in the subsequent sections, but in summary include:
Software and licensing
◦ A complete restructuring of the source code to improve performance.
◦ Restructuring and reorganization of the tool’s components and interface for improved usability.
◦ The introduction of an online version of HEAT Plus with the ability to store and manage uploaded data on a secure server with login access.
◦ Translation of the toolkit from English into French, Spanish and Portuguese.
Design and functionality
◦ Improvements to visualizations in line with best practices in visualizing inequality data and to introduce additional interactive and customizable features.
◦ Within HEAT, visualization of geographical data from Demographic and Health Surveys (DHS) on chloropleth maps.
◦ An increase in the number of summary measures of inequality that are calculated within the toolkit.
◦ An updated and improved set of supporting materials, including user manuals, technical notes, the HEAT Plus Template and Validation Tool, a data repository of disaggregated datasets, and an eLearning course.
The rebuild of the toolkit involved dividing the application into four R packages: heat, heatplus, heatdata, and heatmeasures. These packages handle validating and bundling the built-in database in HEAT and uploaded databases in HEAT Plus (heatdata), generating 19 different measures of inequality (heatmeasures) as well as generating the interactive charts within the application and static charts for download, and contain all the modules used to build and run HEAT and HEAT Plus (heat and heatplus).
HEAT is available as online, while HEAT Plus is available as both an online and desktop version. Both editions are available from the WHO Health Inequality Monitor website (https://www.who.int/data/inequality-monitor/assessment_toolkit). The online version of HEAT is hosted on shinyapps.io. The desktop version of HEAT Plus is downloadable as a .zip file that also includes portable versions of R and Google Chrome, which are required to run the toolkit but do not require any installation. Version 4.0 introduced an online version of HEAT Plus, which uses Microsoft Azure for secure login and storage of datasets. The toolkit is available in four languages: English, French, Spanish and Portuguese.
Design and functionality
HEAT comes pre-installed with a global dataset of reproductive, maternal, newborn and child health (RMNCH) indicators disaggregated by up to six dimensions of inequality (age, economic status, education, place of residence, subnational region and sex). HEAT Plus, on the other hand, allows users to upload their own databases and assess inequalities in a variety of different settings (global, regional, national and subnational), from any data source and period, for any indicator (health-related or not), and any dimension of inequality (including double-disaggregation). The HEAT Plus Template and Validation Tool specifies the required content and format of the dataset to be uploaded to HEAT Plus, and also includes validation checks to ensure that data can be uploaded successfully. Data must be disaggregated before they can be uploaded, which requires some data management skills. To facilitate this, the WHO has produced example statistical codes for commonly used statistical packages (R, SAS, SPSS and Stata) to calculate disaggregated estimates using population-based household surveys, taking into account sampling design complexities (such as clustering, weighting and stratification) (https://www.who.int/data/inequality-monitor/tools-resources/statistical_codes).
The interface has been entirely redesigned to enhance user friendliness and facilitate navigation. The software is organized around two main components, accessible via the main menu at the top of the screen: ‘Explore Inequality’, which allows the user to explore the situation of inequality in one setting of interest, and ‘Compare Inequality’, where the situation of inequality can be compared across settings. Both components allow the user to select either disaggregated data (which shows the situation of inequality by population subgroups) or summary measures of inequality (which quantify the level of inequality across multiple subgroups using a single measure). Within these two subcomponents, users can select a variety of different types of visualizations, including horizontal line plots (also known as equiplots ), bar charts, maps and tables. The structure of HEAT and HEAT Plus and the visualizations available are almost identical, apart from map visualizations which are only available in HEAT, and horizontal bar charts in HEAT are restricted to showing subnational region data (whereas in HEAT Plus they can be used to present data for any inequality dimension) (see Fig. 1).
Each visualization is customizable using the side panel to the left of the visualization. In the Selection menu, setting, dates, data sources, indicators, dimensions of inequality and summary measures of inequality can be selected for visualization. Within the Options menu, users now have access to many more options to modify the visualizations (Fig. 2). The options that are available vary depending on what data and visualization is being viewed, but include the ability to: change titles and axis ranges; show/hide data point labels and 95% confidence intervals; control the size of data labels and choose the number of decimals; sort bars by indicator value or subgroup name; and show reference lines, such as the median or setting average, on the graph. Within the ‘Compare Inequality’ component of the tool there is a Benchmark menu where settings can be selected for benchmarking. Moreover, the Options menu for the scatterplot in this section now includes the option to show data points as settings labels (e.g. country names) in addition to points or ISO 3 labels (for country level data). In ‘horizontal bar graphs’ showing disaggregated data under the ‘Explore inequality’ component, special summary measure calculations are now listed as a separate tab in the side panel for dimensions with a large number of subgroups. Via the Downloads menu, the visualizations and the raw data can both be downloaded as .png/.jpg/.pnf and .csv/.xlsx files, respectively. The selection menus employ a range of different selector types to facilitate user navigation, including drop-down menus, numeric and text input, checkboxes, and buttons.
Visualizations throughout the toolkit have been improved in line with best practices in visualizing inequality data to support interpretation and analysis. This includes plain white backgrounds, new color palettes, and clear titles and legends. Screenshots of the visualizations available are shown in Additional file 1: Figure 3. Tooltips appear when the user hovers over data points in graphs, showing additional information about each data point.
Up to 19 summary measures of inequality are calculated within the toolkit , depending on the data being analyzed (Table 1). The technical notes provide detailed explanation about the calculation and interpretation of all of the summary measures, as well as a guide of which summary measures can be used depending on the characteristics of the data being analyzed.
Both editions come with a range of supporting materials that have been updated and improved, including user manuals, technical notes and an indicator compendium (in the case of HEAT). In addition, training videos are available on the Health Equity Assessment Toolkit website, and an eLearning course hosted on OpenWHO (https://openwho.org/) will be developed to support users in gaining proficiency in using HEAT and HEAT Plus.
The WHO Health Inequality Monitor has a data repository  of disaggregated datasets covering various health topics, which have been pre-formatted to be able to be used with the HEAT Plus software in order to assess health inequalities. This data repository is updated and added to on a regular basis. As at October 2022, it includes disaggregated datasets useful for monitoring inequalities in topics including RMNCH, COVID-19, the WHO Thirteenth General Programme of Work (GPW13), Water, Sanitation and Hygiene (WASH), women’s empowerment, immunization, HIV/AIDS, tuberculosis and malaria (Table 2).
HEAT Plus can be used for monitoring health inequalities at any level (for instance, global, regional, national or subnational), from any data source and period, for any indicator (health-related or not) and any dimension of inequality (including double-disaggregation). The following example illustrates this through the application of HEAT Plus to analyzing global inequalities in COVID-19 related indicators.
COVID-19 case study
The data analyzed comes from the University of Maryland (UMD) Social Data Science Center Global COVID-19 Trends and Impact Survey, in partnership with Facebook (UMD-CTIS) . This survey covers a range of COVID-related indicators, collected daily via Facebook since April 2020. Country-level disaggregated monthly estimates for a selection of indicators are published by UMD . These were downloaded for 109 WHO Member States covering the period between May 2020 and January 2022. Data were formatted as per the HEAT Plus template, and then uploaded to HEAT Plus for analysis.
Here we present data for two indicators related to mental health: the percentage of people worried about catching COVID-19 and the percentage of people reporting feeling depressed most or all of the past 7 days. For each indicator we calculated medians across country-level disaggregated subgroup estimates (globally and by World Bank country income grouping). For each indicator, medians were based only on countries that had sample sizes of at least 100 in each subgroup in all months.
Using the ‘Explore inequality, Disaggregated data’ component of HEAT Plus, we explored inequalities in feeling depressed during the COVID-19 pandemic and in being worried about catching COVID-19, across multiple dimensions of inequality (Additional file 1: Figure 4a). Overall across all countries with data, there were inequalities related to age, education level and sex. While depression was more frequently reported among the youngest age groups and least educated, worry about catching COVID-19 was more common among the older age groups and most educated. Females were more likely than males to report being depressed or worried about catching COVID-19.
Using the ‘Explore inequality, Summary measures’ component allows these levels of inequality to be quantified using a single measure. Using the difference between the two (extreme) subgroups of 18–24 years and 65+ years to assess absolute inequality in being worried about catching COVID-19 (Additional file 1: Figure 4b), we found that age- and sex-related inequality increased over the nine-month period, while education-related inequality remained more-or-less stable. Using ratio, a measure of relative inequality, showed similar results.
HEAT Plus also enables the exploration of double (or multiple) disaggregated data. For example, Additional file 1: Figure 4c shows patterns of age-related inequality among females and males separately in the proportion of people feeling depressed in the past 7 days using the line graph visualization of the ‘Explore inequality, Summary measures’ component. Age-related inequality (measured by the difference between the oldest and youngest age groups) was overall larger among females than males, however the level of inequality remained constant for females but had some fluctuation over time among males.
Using the ‘Compare inequality’ component of HEAT Plus we compared age-related inequalities in being worried about catching COVID-19 across 24 European high income countries. Additional file 1: Figure 4d shows that while older age groups tended to report being more worried than younger age groups across most high income countries, there was substantial variation across countries, as well as variation over time. For example, in July 2021 there was low age-related inequality in the Netherlands, Norway and Portugal, while the gap between the oldest and youngest age groups was very large in Croatia, Poland and Romania. Age-related inequality was substantially larger in November 2021 than in July 2021 in almost all of the 24 countries.
When comparing the level of age-related inequality and national averages (Additional file 1: Figure 4e) in November 2021, we found that there are countries with high inequality and high national averages (such as Italy, Portugal and Spain) and countries with low inequality and low national averages (such as Czech Republic, Croatia, Finland and Poland). Interestingly, there are countries with similar national averages but very different levels of inequality – such as Austria and Croatia, where around 50% of people overall report being worried about catching COVID-19 however there was a gap of 43 percentage points between the oldest and youngest in Croatia whereas in Austria this gap was only 10 percentage points.
In summary, using HEAT Plus facilitates the assessment of inequalities in mental health indicators during the COVID-19 pandemic, including how the situation of inequality differs across dimensions, indicators, over time, and across countries. The dataset used for this case study, which includes data for several COVID-19 related indicators and corresponding metadata, is publicly available in the data repository of the WHO Health Inequality Monitor . Also available in the data repository is age and sex disaggregated data on COVID-19 cases and deaths from the WHO COVID-19 Surveillance Database .
HEAT and HEAT Plus are the only open-source and interactive software applications currently available for exploring and comparing within-country inequalities. The software offers an attractive combination of features that makes it a valuable toolkit for technical staff within country health programmes (for example, in ministries of health), public health professionals, policymakers, researchers, students and others. This includes a free, open source license and source code that can be modified for any use case (for an example of this, see the Health Equity Dataset of the WHO European Health Equity Status Report Initiative (https://worldhealthorg.shinyapps.io/european_health_equity_dataset/). It also includes the ability to analyze inequalities using any dataset of disaggregated data, with built-in calculation of 19 summary measures of inequality (several using complex statistical methods). The interface is multilingual and data visualizations are interactive, customizable, and downloadable for easy use in reporting. Desktop versions enable use without an internet connection, with secure upload, storage and management of datasets.
As with any tool, users should be aware of the limitations of HEAT and HEAT Plus. HEAT and HEAT Plus are not designed to calculate estimates disaggregated by dimensions of inequality from raw datasets; datasets should contain already disaggregated estimates before upload. The toolkit cannot improve the quality of data or impute missing data for population subgroups. The toolkit also cannot calculate some summary measures of inequality and their 95% confidence intervals if some required data are missing. For example, some summary measures can only be calculated if subgroup estimates are available for all subgroups of a dimension. The implications of missing data are outlined in the HEAT Plus Template and Validation Tool. The toolkit handles data visualization, however users need to know how to interpret and report inequality data. The toolkit also cannot explain why inequalities exist; for this, further analysis of quantitative and qualitative information may be required.
A new version of HEAT and HEAT Plus will be released in early 2023 that will include the ability to explore all datasets within the data repository of the Health Inequality Monitor using HEAT. It will also include a mobile-friendly version and the ability to upload and explore data with different time intervals (years, months, weeks, etc.).
Going forward, additional features that will be explored to enhance the utility of the toolkit include:
Visualization of summary measures of inequality on maps.
Adding additional information to tooltips to support interpretation (for instance, subgroup estimates used for the calculation of summary measures).
Integration of the toolkits into DHIS2 software , to embed inequality monitoring as an essential component of health information systems.
While originally designed to facilitate the assessment of health inequalities, the software can be adapted to any topic area. Monitoring inequalities across the SDGs is essential for ensuring that disadvantaged or hard-to-reach populations are not left behind. Therefore, a potential further development for the HEAT toolkit is to expand the software application to enable the assessment of within- and between-country inequalities in all SDG indicators.
HEAT and HEAT Plus is a software application that facilitates the assessment of health inequalities, either by using datasets from the WHO Health Inequality Monitor data repository or using custom datasets. The features of the software make it a valuable tool for analyzing and reporting inequalities related to the COVID-19 pandemic, as well as its indirect impacts on inequalities in other health and non-health areas, providing evidence to inform policies, programmes and practices that can close existing gaps and achieve health equity. To conclude, the toolkit represents a concrete and tangible contribution to the realization of the SDG vision of leaving no one behind.
Availability and requirements
Project name: Health Equity Assessment Toolkit (HEAT and HEAT Plus).
Hosseinpoor AR, Nambiar D, Tawilah J, Schlotheuber A, Briot B, Bateman M, et al. Capacity building for health inequality monitoring in Indonesia: enhancing the equity orientation of country health information system. Glob Health Action. 2018;11(sup1). Available from: https://pubmed.ncbi.nlm.nih.gov/29569528/. [cited 2021 Nov 22].
Hosseinpoor AR, Schlotheuber A, Nambiar D, Ross Z. Health Equity Assessment Toolkit Plus (HEAT Plus): software for exploring and comparing health inequalities using uploaded datasets. Glob Health Action. 2018;11(sup1):20–30. Available from: https://www.tandfonline.com/doi/abs/10.1080/16549716.2018.1440783. [cited 2021 Oct 14].
Schlotheuber A, Hosseinpoor AR. Summary measures of health inequality: a review of existing measures and their application. Int J Environ Res Public Heal. 2022;19(6):3697. Available from: https://www.mdpi.com/1660-4601/19/6/3697/htm. [cited 2022 Mar 22].
Kreuter F, Barkay N, Bilinski A, Bradford A, Chiu S, Eliat R, et al. Partnering with a global platform to inform research and public policy making. Surv Res Methods. 2020;14(2):159–63. Available from: https://ojs.ub.uni-konstanz.de/srm/article/view/7761. [cited 2021 Nov 22].
University of Maryland. The University of Maryland social data science center global COVID-19 trends and impact survey in partnership with Facebook. 2021. Available from: https://covidmap.umd.edu/. [cited 2021 Nov 22].
Allan M, Lièvre M, Laurenson Schaefer H, de Barros S, Jinnai Y, Andrews S, et al. The World Health Organization COVID-19 surveillance database. Int J Equity Health. 2022;21(Suppl 3). https://doi.org/10.1186/s12939-022-01767-5.
Authors are grateful to people who have reviewed HEAT and HEAT Plus and provided valuable feedback and suggestions. The authors would like to thank Nathan Teetor for his contributions to the coding and development of the application.
The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated.
ARH, AS and KK conceptualized the toolkit and coordinated its development, with inputs from CVF. ZR developed the application. ARH, AS, CVF and KK conceptualized the manuscript and case study example. CVF prepared the dataset and HEAT Plus visuals used for the case study. KK drafted the manuscript, which was revised by all authors. CVF is affiliated with the Inter-American Development Bank at the time of publication of the article. All authors read and approved the final manuscript.
The data presented in this paper are published by the University of Maryland (UMD). The main data source (the UMD Global CTIS study) received ethical clearance from the UMD Institutional Review Board (1587016–10). All respondents gave informed consent before participating in the survey. Informed consent was documented in the digital platform by the respondent (no witness required) and this study did not include minors. None of the data that support the findings of this study included any identifiable human data.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Horizontal line graph. Figure 3(b). Vertical bar graph. Figure 3(c). Horizontal bar graph. Figure 3(d). Line graph. Figure 3(e). Scatterplot. Figure 3(f). Map. Figure 4(a). The proportion of people feeling depressed and feeling worried about catching COVID-19 by age, education and sex: global median disaggregated estimates (UMD-CTIS, May 2021–January 2022). Figure 4(b). The proportion of people worried about catching COVID-19 by age, education and sex: difference and ratio between the two (extreme) subgroups (UMD-CTIS, May 2021–January 2022). Figure 4(c). Age-related inequality among females and males in the proportion of people feeling depressed for most or all of the past 7 days: difference between the oldest and youngest age groups (UMD-CTIS, May 2020–January 2022). Figure 4(d). Age-related inequality in the proportion of people worried about catching COVID-19: disaggregated estimates in 24 European high income countries (UMD-CTIS, July 2021 andNovember 2021). Figure 4(e). The proportion of people worried about catching COVID-19: age-related inequality (measured by the difference between the oldest and youngest age groups) compared to the national average in 24 European high income countries (UMD-CTIS, November 2021).
Rights and permissions
This article is distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 IGO License (https://creativecommons.org/licenses/by-nc-sa/3.0/igo/legalcode), which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. If you remix, transform, or build upon this article or a part thereof, you must distribute your contributions under the same license as the original. In any reproduction of this article there should not be any suggestion that World Health Organisation or this article endorse any specific organization or products. The use of the World Health Organisation logo is not permitted. This notice should be preserved along with the article's original URL.
Kirkby, K., Schlotheuber, A., Vidal Fuertes, C. et al. Health Equity Assessment Toolkit (HEAT and HEAT Plus): exploring inequalities in the COVID-19 pandemic era.
Int J Equity Health21
(Suppl 3), 172 (2022). https://doi.org/10.1186/s12939-022-01765-7