This study shows that the ranking of households into wealth groups and the magnitude of poor-rich inequality in under-5 mortality and immunisation coverage are sensitive to the measure of economic status used. The size and direction of change, however, varied per country and alternative index, in some cases ranging up to a 60% change in observed inequality.
Our results seem to contrast to the findings of Filmer and Pritchett, who found that the ranking of households is robust to the items included . However, their conclusions are based on the analysis of only one country (India). Furthermore, they did not analyse the sensitivity of the association of such ranking with a health outcome such as mortality.
Bollen et al.  compared a broad set of proxies for economic status, including a PCA-based consumer goods index, education and occupation for two countries, using fertility as outcome variable. They concluded that the effect of economic status varied with the measure used, and that the PCA-based method was most predictive. Our in-depth comparisons of different PCA-based indices show that even for this specific type of measure, the specific indicators used influence the magnitude of observed inequalities. In addition, our analyses of a broad set of countries showed that the extent and direction of sensitivity varies between countries. Moreover, our findings for both mortality and immunisation demonstrate that sensitivity also can vary with the outcome measures studied.
Which weight should we attribute to the sensitivity observed? While observed inequalities changed for most countries, in many of these cases the order of magnitude remained the same: large inequalities remained large, small inequalities small. Moreover, the confidence intervals of the RII's were large and overlapping. While this can not be interpreted as a lack of statistical significance – the RII's have not been calculated on basis of independent groups – it does indicate that the importance of the sensitivity found should not be overestimated. Furthermore, the reliability of the retrospective surveys used is not such to allow for very precise estimation of poor-rich differences in health. So in many cases, the changes in inequality found when using alternative measures of economic status, are not alarming. However, in a number of cases the measure of economic status used did make an important difference, ranging up to a 60% change in RII. Therefore, it is important to be aware that the measure of economic status used can affect observed poor-rich differences in health and health related outcomes.
We expect also for other developing countries and health outcome measures inequality to be sensitive to the measure of economic status used. The countries included in our study are diverse in terms of region, average mortality levels and pattern of inequality . Yet, since the size and direction of change varied by country, index and health indicators, it is difficult to predict this a priori for specific cases.
An issue that needs to be mentioned is related to the method of PCA for constructing indices. Even though PCA can be a useful measure for constructing composite indices, it may produce odd results when applied to short lists of items as in Index 2 and 3. In Cameroon, for example, the item 'bicycle' got a negative factor score. As a consequence, households owning only a bike, were categorised as poorer compared to households owning nothing. The question arises whether in such cases the asset index is still conceptually valid. While this problem could have influenced the results in such specific cases, it is not likely to have influenced our overall conclusions.
It also needs to be mentioned that the distinction between direct and indirect determinants is not always clear-cut. One could, for example, argue that since the type of stove owned can have a direct effect on respiratory illnesses, it should have been excluded from the alternative indices. The additional exclusion of these items, will, most likely, lead to even larger sensitivity than reported. Generally, in explanatory studies, it should be made explicit, for instance by using a conceptual framework, which factors are considered as direct and which as indirect determinants.
Finally, it must be remembered that in this study we examined poor-rich differences, and their sensitivity to the measure of economic status, in a descriptive way. It was not the purpose of this study to establish whether the wealth and health are causally related. Readers should keep this in mind when interpreting the results.
Explaining the results
Sensitivity of inequality to alternative constructions of the WB index is likely to be related to the low common variance of the items in the WB index. This low common variance can be explained by the fact that the WB includes a broad range of different items, each of which has its own determinants besides economic status. Upon exclusion of items from this index, the common variance increased. The reason is that the new lists were shorter and consisted of more homogeneous sets of items. As a consequence, the categorisation of households into wealth groups changed, leading in its turn to different mortality rates per wealth group.
Even though the observed sensitivity is understandable, it is more puzzling why the use of alternative indices had different effects for different countries as well as for different outcome indicators. Below, we will forward some explanations.
We hypothesised that inequality in under-5 mortality would decrease upon exclusion of water and sanitation items from the WB index and would further decrease upon exclusion of housing items. This is because we expected that part of the relationship between wealth as measured by the WB index and under-5 mortality would be explained by variables that have a direct effect on child mortality, apart from their indirect effect as indicators of economic status . For a number of countries we saw this expected decrease. This supports the hypothesis that for some countries part of inequality in mortality measured using the WB index can be attributed to direct determinants of health rather then to economic status alone.
This hypothesis, however, cannot explain the decrease in inequality in immunisation coverage observed for some countries upon exclusion of direct determinants of health from the index. The reason is that housing characteristics, and water and sanitation facilities only influence immunisation coverage as indicators of economic status, and don't have a 'direct' impact that is comparable to their effect on mortality. An alternative hypothesis would be that water and sanitation facilities and housing characteristics are also indicators for regional development or rural/urban residence. For instance, using the bush as latrine probably indicates rural residence, whereas using a private toilet is probably more related to urban residence. Therefore, the decrease in observed inequality in both mortality and immunisation may also in part be explained by the fact that the WB index captures rural-urban differences in both wealth and the health indicators.
We expected a further decrease in inequality upon subsequent exclusion of electricity from the index. Electricity can be an indicator of community wealth. Regional disparities in the availability of electricity probably run parallel to disparities in access to and quality of health care services and disparities in mortality. When excluding electricity from the asset index, these regional disparities in wealth and mortality as measured through electricity, are given less weight. Doing so, one can expect a decrease in inequality in mortality. We saw that indeed for a number of the countries, inequality in under-5 mortality decreased upon exclusion of electricity from the wealth index. This may indicate that health inequality as measured by the WB index, through electricity, also captures for some countries some of the regional disparities in wealth and health.
The hypotheses above, however, cannot explain why in some cases inequality was not sensitive, and why in one case inequality in under-5 mortality increased, instead of decreased. Additional explanations therefore need to be sought.
Inequality in under-5 mortality was robust to changes in the measure of economic status used for Uganda. This is related to the fact that Uganda was the only country for which items on food sufficiency were included in the WB index. When doing an additional analysis, excluding food sufficiency from all four indices, the RII became slightly sensitive also for this country (the largest change in RII being 14%, from 1.68 when using the WB index to 1.77 when using Index 1). Sensitivity thus may depend on the specific items included in the asset index.
The slight increase in inequality in under-5 mortality in Cameroon upon exclusion of water and sanitation items could not be attributed to the above factors. As already mentioned, household ownership of assets is also determined by other factors besides economic status, such as local availability and preferences. These factors can act as confounders in the relationship between household wealth and child mortality. Apparently, these confounders are in some cases more difficult to disentangle than in others. Multivariate analysis would be needed to gain more insight into these relationships, and thus into the underlying mechanisms linking wealth and health.
Our study shows that researchers and policy makers should be aware that the choice between alternative indicators of economic status often does affect, and in some cases to an important extent, the observed magnitude of poor-rich differences in health and health-related outcome measures. It also shows that it is difficult to predict the size and direction of sensitivity.
This is important in the present context in which monitoring and tackling poor-rich inequalities in developing countries have become increasingly important policy objectives, and in which many studies are being published on this issue. One of the major difficulties this new field is facing, is determining who is rich and who is poor. An index based on household ownership of assets is an often-used way to do so. Different researchers, however, use different sets of asset items. Our study shows that who is defined as poor and who as rich, varies with the asset items included in the index.
Our study implies that we should be extremely careful comparing results of studies using different indicators of economic status, as differences between countries and trends over time may in part be an artefact of the different indicators used. This is important both for monitoring health inequalities, evaluating the effects of policy interventions on these inequalities, and for targeting the poor in health policies. The choice of the measure of economic status should therefore be carefully made.
For descriptive and monitoring purposes we advise to use a comprehensive list of asset items such as used by the World Bank. A good alternative would be a much more extended list of consumer goods. In countries or regions where durable consumer goods are hardly accessible to anyone, or where investments in housing and amenities are given priority, the latter can be important indicators of economic status or wealth. Moreover, surveys such as the DHS only include a limited number of durable consumer goods, whereas items that the poor and inhabitants of rural areas are likely to own (e.g. chair, plastic recipients, animals, farming tools) are not included. In such cases, the inclusion of water, sanitation and housing items facilitates stratification of households at the lower end of the wealth ladder. For these reasons, it would generally be advisable to use a comprehensive list of asset items for descriptive and monitoring studies.
For explanatory studies, though, it can be important to analyse the different sets of asset items separately, and not to combine them into one index. It enables the assessment of the relative importance of different components of material wealth, especially water and sanitation versus housing versus consumer items versus indicators of community wealth. Estimates of the relative importance of these components can contribute to the detection of causal mechanims that are most responsible for high child mortality among poor families. This information is important for intervention purposes, since it addresses questions such as: would it be more effective to invest in income generating projects or in housing, water and sanitation programs; and should development efforts be focussed on the household level or the community level? For such explanatory studies it would be advisable to use multiple regression, path analysis or similar multivariate techniques.
For those designing new surveys intending to measure economic status in developing countries, we advice to also include items that poor households are likely to own and indicators of economic status in rural areas, such as the ownership of land, animals, and farming tools. Also the inclusion of context-specific indicators of economic status, as shown by the example of 'food sufficiency' in Uganda, would be useful when aiming to make a refined stratification along the lines of economic status. The inclusion of 'rural' and context specific items can also be important for making a proper identification of target groups for health policies.