Estimating the cost of implementing a facility and community score card for maternal and newborn care service delivery in a rural district in Uganda

Introduction This paper aimed at estimating the resources required to implement a community Score Card by a typical rural district health team in Uganda, as a mechanism for fostering accountability, utilization and quality of maternal and child healthcare service. Methods This costing analysis was done from the payer’s perspective using the ingredients approach over five quarterly rounds of scoring between 2017 and 2018. Expenditure data was obtained from project records, entered and analyzed in Microsoft excel. Two scale-up scenarios, scenario one (considered cost inputs by the MakSPH research teams) and scenario two (considering cost inputs based on contextual knowledge from district implementing teams), were simulated to better understand the cost implications of integrating the Community Score Card (CSC) into a district health system. Results The total and average cost of implementing CSC for five quarterly rounds over a period of 18 months were USD 59,962 and USD 11,992 per round of scoring, respectively. Considering the six sub-counties (including one Town Council) in Kibuku district that were included in this analysis, the average cost of implementating the CSC in each sub-county was USD 1998 per scoring round. Scaling-up of the intervention across the entire district (included 22 sub-counties) under the first scenario would cost a total of USD 19,003 per scoring round. Under the second scaleup scenario, the cost would be lower at USD 7116. The total annual cost of scaling CSC in the entire district would be USD 76,012 under scenario one compared to USD 28,465 under scenario two. The main cost drivers identified were transportation costs, coordination and supervision costs, and technical support to supplement local implementers. Conclusion Our analysis suggests that it is financially feasible to implement and scale-up the CSC initiative, as an accountability tool for enhancing service delivery. However, the CSC design and approach needs to be embedded within local systems and implemented in collaboration with existing stakeholders so as to optimise costs. A comprehensive economic analysis of the costs associated with transportation, involvement of the district teams in coordination, supervision as well as provision of technical support is necessary to determine the cost-effectiveness of the CSC approach. Supplementary Information The online version contains supplementary material available at 10.1186/s12939-020-01335-9.


Introduction 1
For decades governments in low income countries have failed to provide poor populations with 2 adequate social services to meet their needs (1,2). Social accountability is increasingly being 3 seen as an approach that could augment public sector actions to meet the needs of the poor (2). 4 The community score card (CSC) is one of the social accountability tools that has been 5 employed to monitor the availability, access and quality of social services (3,4). Use of CSCs has 6 contributed to increased expression of community and health provider concerns, improved 7 responsiveness and accessibility of health services in addition to improved accountability, quality 8 as well as improved communication between service providers and users (4-9). However, 9 evidence regarding the effect of prevailing social accountability tools is mixed, with some 10 authors reporting enhanced accountability and others reporting the opposite (1,10-13). The CSCs 11 picture in Uganda is similar to the global picture, where community score cards have been 12 implemented largely as pilot projects without national scale up (14). 13 Scale-up is commonly defined as efforts to increase the impact of the innovations successfully 14 tested in pilots or experimental projects so as to benefit more people and foster policy and 15 programme development on a lasting basis (15). Scalability is defined by Milat et al., 2012 as the 16 ability of a health intervention shown to be efficacious on a small scale and or under controlled 17 conditions to be expanded under real world conditions to reach a greater proportion of the 18 eligible population, while retaining effectiveness (16). However, the accountability literature 19 argues that simply increasing or expanding the scale of doing something may not necessarily 20 achieve the desired objective of increasing accountability (17). Increasing accountability requires 21 that specific action is taken to address the underlying accountability failures through upward 22 vertical integration between various actors at local, sub national and national levels so as to get 23 more leverage over more powerful institutions. This is what Fox refers to as "taking scale into 24 account" (17). "Taking scale into account" then is less about scaling up to more 25 locations/geographies, and more about working through and across different levels of decision-26 making and practice from local to sub national to national (11,17). It should begin in the initial 27 planning and design phases, but can only happen if practitioners look beyond the details of the 28 specific intervention that they are piloting into the broader enabling environment. This facilitates 29 coordinated action among different actors that allow horizontal and vertical coalitions to develop 30 and bring about desired actions that transform the behavior of health system actors so as to 31 promote a culture of accountability and accountability systems (17,18) (19).

32
With growing interest in strengthening social accountability to make progress towards Universal 33 Health Coverage, the issues of scaling up and institutionalizing promising pilot projects, is 34 therefore timely. If we assume that the factors influencing scale-up are dependent on the initial 35 planning and design of the intervention, documenting this process, as well as how the design 36 adapts over time relative to what it is trying to achieve is important, though seldom done.

37
The theory of change in this paper serves as a reference point for iterative and adaptive design 38 during implementation (20). It provides a framework for the analysis of feasibility, 39 embeddedness, ownership and legitimacy and also helps to enrich the understanding about the 40 pathways of change, as outlined by Wild and Harris (21) and documented by Ekirapa-Kiracho et  The purpose of this paper is to document the iterative design and planning undertaken by the 43 MakSPH team to support the implementation of a community score card pilot in five sub

48
The CSC intervention was conducted for five quarterly rounds between the months of June 2017 49 to December 2018. The CSC intervention consisted of eight main stages; as illustrated in Figure   50 1, below (24). More details about the CSC intervention can be obtained from the paper by The planning phase was guided by a theory of change (see figure 2), whose development was 55 guided by previously published scaling up frameworks (15,16,(26)(27)(28)(29), in particular the Expand 56 Net framework (30), the FHS project institutionalization framework (31) as well as project wide 57 discussions.
58 Figure 2 here 59 According to our theory of change, four main factors were central to ensuring that the CSC 60 designed was scalable and sustainable. These included embeddedness (entrenchment into already 61 existing systems or processes or policies at the local or national level), legitimacy (working with 62 persons/structures that are mandated to carry out specific activities), feasibility (low cost, 63 simplicity of tools, acceptability, less human resource intensive) and ownership (high level 64 9 stakeholder participation and acceptance of the CSC). We believed that the inclusion of these 65 components would facilitate the ability of the CSC to stimulate collective action by the 66 community, health providers, health facility managers, sub county and the district leaders. These 67 actions would then act through the six pathways proposed by Wild and Harris to bring about the 68 desired changes at various levels (21). The six pathways include strengthening citizens' demand,  Data collection methods 85 We conducted ten focus group discussions (FGDs); five female and five male and seven key 86 informant interviews (KIIs) as well as one reflection meeting with the project team. Data was 87 10 also abstracted from quarterly project and stakeholder meeting reports.

99
The reflection meeting was conducted once at the end of the fifth round of scoring with 100 researchers from MakSPH. It was guided by a tool adapted from the ExpandNet 20 questions for 101 developing a case study for scaling up (32).

102
Data was also abstracted from notes from the stakeholder and project meeting reports. These  All FGDs and KIIs were transcribed verbatim. During the stakeholder meetings, notes were 109 taken and then later typed. All transcripts were read several times to allow familiarization with 110 the data. We then developed an analytical framework based on key themes; embeddedness, 111 legitimacy, feasibility and ownership guided by the ExpandNet framework as highlighted in the 112 project theory of change. Codes were then developed and applied according to the analytical 113 framework. Any new emerging codes related to the study objectives were also included (33).

115
We present the actions that were taken to design and set up a scalable CSC model that takes scale 116 into account by putting in place features that enhance ownership, embeddedness, legitimacy and 117 feasibility of the CSC. We also present the challenges that were encountered and how they were 118 mitigated.

120
To promote ownership of the CSC process two main actions were taken by the MakSPH team,   The participatory implementation design further promoted stakeholder buy-in, ownership and     However, the technical capacity of CSC facilitators from the existing structures was not optimal 194 in some cases. An initial training was conducted over a five-day period for the core 195 implementation team by MakSPH and the district health team (DHT). Thereafter their ability to 196 facilitate a CSC meeting was assessed and those who were deemed too inept to carry out the 197 required tasks were excluded. This was echoed by one of the MakSPH researchers.  (RMNCAH) score card were also identified and targeted for action.

223
"I think one of the main things we [research team] did was that we co-created the whole idea, we 224 did not come in with our own model(s), we did provide technical guidance on what needed to be 225 done but the structures and the processes were informed by the people [local stakeholders] who 226 were going to implement this. That means we identified the people mandated to do it,…platforms 227 that were supposed to be used,… available tools and or the lack thereof and then we tried to 228 strengthen both the human resource, the platforms and the structures so that whatever we did in 229 terms of timing of these activities is primarily informed by the actual people on the ground who 230 are mandated to do this work." MakSPH staff 2 231 232 However, getting entrenched into existing systems and processes requires adequate time and in some 233 cases negotiation with key players. Although we tried to embed feedback meetings into existing 234 platforms this was not always successful. Some of the platforms were nonfunctional for example some 235 council meetings did not happen when there were no allowances for the councilors.

238
As noted above legitimacy was ensured by aligning CSC implementation within existing 239 systems, policies and processes including; working with personnel who had the mandate to 240 perform different tasks within the CSC process. This was considered important because such 241 structures could potentially continue performing the expected services even after the project exits 242 or continue with minimal additional pay since they (the local personnel) would be performing 243 duties that are within their mandate. These leaders felt that the CSC was enabling them fulfill 244 their mandate and were therefore supportive of the programme and its continuity. In addition, 245 they command the respect that is required from the community, as acknowledged in the 246 quotation below. Moreover, during the design phase, the technical and political leaders cautioned against 253 designing a CSC which operates outside of the district system. They also noted that appointing 254 district staff and assigning them roles outside their mandate results in officials overstepping their 255 roles creating friction within the district.

257
The research team aimed at designing a simple low-cost intervention to enhance the feasibility 258 for scale-up, sustainability, and institutionalization of the CSC. During the implementation of 259 this intervention, several actions were undertaken to lower the associated costs. These included 260 use of locally existing personnel who could be paid government allowance rates which are lower 261 than rates often paid to NGOs, removal of refreshments for the community meetings and 262 allowances for the community and health workers. These low cost implementation approaches 263 were however not always welcomed by stakeholders who were used to receiving allowances 264 from other projects and political leaders as noted in the quotations below.

265
"…the challenge has been that at the start when you [FGD participant]  budgets or how to lobby certain partners to be able to meet these costs." MakSPH staff 1.

292
The participatory design of implementation of CSCs was selected to allow flexibility during 293 implementation. This enabled review of the implementation approach and simplification of 294 aspects that were considered complex during each scoring round. This led to the modification of 295 tools and meeting guides used during the initial scoring meetings making it easier for the 296 facilitators to understand the tasks that they were required to carry out as they facilitated CSC 297 meetings. The number of meetings was also reduced from 45 to 25 hence reducing workload on 298 the facilitators and coordinators. The intervention was further simplified by transferring the 299 responsibility of coordinating CSC meetings from two district coordinators to twelve sub county 300 coordinators. However, some aspects of the intervention remained rather complex and could potentially have 312 hindered scale up for example the initial process of selection of indicators. This activity was 313 difficult for most of the participants and this could affect the potential for scale-up, sustainability 314 and institutionalization of the CSC. Since it was done once, it was not possible to repeat this 315 aspect of the intervention. Another activity that was done once and also noted to be rather 316 complicated for some of the facilitators was the development of action plans.

318
Whereas the scale up literature often puts emphasis on the ability to implement an intervention 319 on a large geographical scale, taking scale into account for social accountability interventions 320 emphasizes the importance of putting in place deliberate actions that encourage strategic 321 partnerships that can enhance accountability by leveraging the influence of more powerful 322 parties/stakeholders (17). In the discussion we reflect on the extent to which we were able to 323 achieve both these aims by using a model that aimed at enhancing embeddedness, feasibility, 324 ownership and legitimacy. 325 We found that by far the most important domains for enabling wide scale implementation within 326 our framework were feasibility and ownership. To make the CSC feasible and scalable, attention 327 should be paid to its design, technical capacity of implementers and the cost of implementation.

328
The design should be simple without overly complicated processes and tools to allow  Information access and citizen voice are often not enough to deliver accountability (17,19,36,37).

355
They need to be accompanied by the support of powerful leaders and building of relationships 356 (1,17). Local ownership and legitimacy were therefore particularly pivotal for taking scale into  Leaders also need to appreciate the benefits of their participation in the CSC to secure their buy-373 in and active participation in holding duty bearers accountable. It is therefore important to ensure 374 that the CSC design allows the CSC to identify and contribute to meeting the local needs. From  This may have biased the responses. However, these interviews were triangulated by considering 391 responses from all the different groups of stakeholders involved. Furthermore, we reported both 392 positive and negative findings. Another limitation was the short implementation period which 393 was inadequate for observing scale up. Furthermore, this design did not allow us to assess the 394 extent to which the community voice was truly realized. We recommend this as an area for 395 further research.