https://orcid.org/0000-0002-0939-7695
Figures
Abstract
Background
Identifying approaches to improve levels of health care provider knowledge in resource-poor settings is critical. We assessed level of provider knowledge for HIV testing and counseling (HTC), prevention of mother-to-child transmission (PMTCT), and voluntary medical male circumcision (VMMC). We also explored the association between HTC, PMTCT, and VMMC provider knowledge and provider and facility characteristics.
Methods
We used data collected in 2012 and 2013. Vignettes were administered to physicians, nurses, and counselors in facilities in Kenya (66), Rwanda (67), South Africa (57), and Zambia (58). The analytic sample consisted of providers of HTC (755), PMTCT (709), and VMMC (332). HTC, PMTCT, and VMMC provider knowledge scores were constructed using item response theory (IRT). We used GLM regressions to examine associations between provider knowledge and provider and facility characteristics focusing on average patient load, provider years in position, provider working in another facility, senior staff in facility, program age, proportion of intervention exclusive staff, person-days of training in facility, and management score. We estimated three models: Model 1 estimated standard errors without clustering, Model 2 estimated robust standard errors, and Model 3 estimated standard errors clustering by facility.
Results
The mean knowledge score was 36 for all three interventions. In Model 1, we found that provider knowledge scores were higher among providers in facilities with senior staff and among providers in facilities with higher proportions of intervention exclusive staff. We also found negative relationships between the outcome and provider years in position, average program age, provider working in another facility, person-days of training, and management score. In Model 3, only the coefficients for provider years in position, average program age, and management score remained statistically significant at conventional levels.
Conclusions
HTC, PMTCT, and VMMC provider knowledge was low in Kenya, Rwanda, South Africa, and Zambia. Our study suggests that unobservable organizational factors may facilitate communication, learning, and knowledge. On the one hand, our study shows that the presence of senior staff and staff dedication may enable knowledge acquisition. On the other hand, our study provides a note of caution on the potential knowledge depreciation correlated with the time staff spend in a position and program age.
Citation: Pineda-Antunez C, Contreras-Loya D, Rodriguez-Atristain A, Opuni M, Bautista-Arredondo S (2021) Characterizing health care provider knowledge: Evidence from HIV services in Kenya, Rwanda, South Africa, and Zambia. PLoS ONE 16(12): e0260571. https://doi.org/10.1371/journal.pone.0260571
Editor: Pablo Brañas-Garza, Universidad Loyola Andalucia Cordoba, SPAIN
Received: November 27, 2020; Accepted: November 12, 2021; Published: December 2, 2021
Copyright: © 2021 Pineda-Antunez et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The data and code used in this study are available from the Harvard dataverse at https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/BD25HZ (doi:10.7910/DVN/BD25HZ).
Funding: Funding was provided by the Bill & Melinda Gates Foundation. The funders played no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: All authors declare no competing interests. Marjorie Opuni is an independent consultant. This does not alter our adherence to PLOS ONE policies on sharing data and materials.
Introduction
Ending the AIDS epidemic by 2030 is a target of the Sustainable Development Goals (SDGs) [1]. Accompanying this objective are the Joint United Nations Programme on HIV/AIDS (UNAIDS) goal of reducing the number of new HIV infections by 90% between 2010 and 2030, and its 95-95-95 target which proposes that by 2030, 95% of people living with HIV should be diagnosed, of whom 95% should be on treatment, of whom 95% should be virally suppressed [2]. The world is not on track to reach these objectives despite remarkable progress in the global response to HIV in recent years [3]. Experts highlight “suboptimal or inappropriate policies, lack of funding, [and] limited or misdirected implementation of available strategies and tools” to explain why progress has been slower than foreseen [4]. According to the World Health Organization (WHO), gaps in service access and quality span the cascade of HIV services, and countries will reach HIV targets only if they substantially improve quality of care [5].
Improving quality of care in low- and middle-income countries (LMICs) has become a global priority encompassing all health care services [6–8], with quality of care defined as “the degree to which health care services for individuals and populations increase the likelihood of desired outcomes and are consistent with current professional knowledge [7].” There is a growing literature describing the substantial deficiencies that exist in quality of care in resource-poor settings (e.g. [9–16]) and an increasing number of studies documenting poor health care provider performance in LMICs [6, 7] and low levels of provider knowledge [17–28]. Several factors have been identified as barriers to provider adherence to clinical guidelines and implementation of evidence-based practices [7]. At the organizational level, these factors include lack of supervision and mentorship, poor communication, information sharing, and collaboration within individual health care facilities [7, 29]. At the provider level, substandard pre-service training and continuing education have been highlighted as significant barriers [6, 29]. In addition, experts have shown that management practices and management capacity in LMIC health care facilities are poor and improving these is key to improving quality of care in these settings [6].
This paper contributes to the body of work on provider knowledge in resource-limited settings. Following Donabedian, we conceptualized provider knowledge as a dimension of structural quality and a necessary but insufficient condition for adequate process quality [30–32]. We assumed that provider knowledge can accumulate over time, but that it can also depreciate [33, 34]. Some explanations for knowledge depreciation include the evolution of knowledge and the time it takes to diffuse new guidelines and evidence [35], lack of motivation among staff to acquire knowledge and skills [36], and staff turnover and organizational inertia in facilities [37]. We also assumed that in addition to learning through education and training, individuals embedded in organizations also learn from knowledge reservoirs (e.g. treatment guidelines), tools, learning by doing, social networks [38], and transactive memory systems that develop when people in groups develop ways of assigning responsibility for information based on shared understanding of one another’s knowledge [39, 40].
In this analysis, we focused on provider knowledge for three outpatient HIV services: HIV testing and counseling (HTC), prevention of mother-to-child transmission (PMTCT), and voluntary medical male circumcision (VMMC). We used vignettes to assess knowledge of HTC, PMTCT, and VMMC guidelines among HIV service providers in health care facilities in four countries in sub-Saharan Africa: Kenya, Rwanda, South Africa, and Zambia. We also explored the association between HTC, PMTCT, and VMMC provider knowledge and provider and facility characteristics drawn from literature in organizational psychology, economics, management, and production research.
Methods
Study design and sample
We used data collected as part of the “Optimizing the Response in Prevention: HIV Efficiency in Africa” (ORPHEA) study, which was a cross-sectional, micro-costing study that collected year-long data in health care facilities in Kenya, Rwanda, South Africa, and Zambia. We collected data in 2012 in Kenya, South Africa, and Zambia and in 2013 in Rwanda. The ethical review boards of the following institutions approved the study: National Institute of Public Health, Mexico; Kenyatta National Hospital and University of Nairobi; Northeastern University; Rwanda Biomedical Center; University of the Witwatersrand; and University of Zambia. We have published elsewhere detailed descriptions of the sampling and data collection methods used [41–44] and briefly summarize them here. In each country, we used multistage sampling to first select sub-national areas and then randomly select facilities providing at least one of the following outpatient HIV services: HTC, PMTCT, and VMMC (see S1 Table for descriptions of the HTC, PMTCT, and VMMC services implemented in Kenya, Rwanda, South Africa, and Zambia at the time of the ORPHEA study). We enumerated facilities in selected sub-national areas along with information on the services of interest stratified by ownership and level of service provision. We randomly sampled facilities within these strata, using probability proportional to size, giving preference to sites providing more than one service of interest. In sampled facilities, we randomly selected up to five providers who had contact with HTC, PMTCT, and VMMC patients to take part in the vignette-based interviews described below. The ORPHEA study assembled the detailed input and output data required to estimate the annual cost of service provision of HTC, PMTCT, and VMMC per patient [41–44]. In addition, the study also collected data on: provider knowledge; provider attributes (e.g. age, cadre, education); and facility characteristics (e.g. type, ownership, management practices). The sub-sample we used in this paper is limited to those providers and facilities with complete information on provider and facility characteristics.
Provider knowledge measure
We administered medical vignettes for the three outpatient HIV services considered in the ORPHEA study: HTC, PMTCT, and VMMC. Previous studies have shown that vignettes reliably measure the medical knowledge of health care providers [18, 26, 45–48]. ORPHEA study vignettes were paper-based hypothetical scenarios describing HTC, PMTCT, and VMMC patients. The HTC vignette described a 32-year-old married woman testing for HIV for the first time; the PMTCT vignette represented a pregnant 26-year-old married woman living with HIV attending her second antenatal care visit; and the VMMC vignette described an HIV-negative 26-year-old man attending pre- and post-operative counselling. We asked respondents a series of questions about the activities they would carry out with the patients in these hypothetical scenarios, reproducing the actions they would undertake with actual patients. Caregivers sampled included physicians, nurses, and counselors interacting with HTC, PMTCT, and VMMC patients. Providers of HTC, PMTCT, and VMMC services were each assessed using the vignette pertaining to their area of work. We evaluated health care workers for each of the three interventions using the same vignette irrespective of professional category. National and World Health Organization HTC, PMTCT, and VMMC guidelines in place at the time of the study were used to assess the responses provided. For this study, we retained only those items that conformed to global guidance and were consistent across the four countries. We evaluated 112 items for HTC, 89 items for PMTCT, and 76 items for VMMC. We scored all items dichotomously, with a value of 1 if the provider mentioned the item and 0 if the provider did not (see S2 Table for average percent of items correctly identified by intervention, provider cadre, and country).
We constructed three provider knowledge indices—one for HTC, one for PMTCT, and another for VMMC—with each index combining the information collected from all HTC, PMTCT, and VMMC providers across the four countries. To construct the three indices, we used item response theory (IRT) [9, 18, 49]. IRT is a collection of models in which an individual’s level of a latent trait (i.e. knowledge) depends on their responses to a series of items and on the characteristics of those items [50, 51]. The IRT models we estimated considered two-item characteristics: item difficulty (level of item difficulty given provider knowledge) and item discrimination (capacity of the item to distinguish between providers able to respond correctly from those who were not). We applied a second order IRT model which allows items to vary by both difficulty and discrimination. To estimate the expected knowledge score for each provider, we summed the probabilities of providing the correct answer for each of the items. We rescaled each knowledge score to range from 0 to 100, where the two extremes represent the lowest and the highest scores observed in our sample. We tested the internal consistency of the knowledge scores using Cronbach’s Alpha (α = 0.95 for HTC, 0.92 for PMTCT, and 0.94 for VMMC). To further validate the estimated HTC, PMTCT, and VMMC knowledge scores, we split the samples into tertiles (low, medium, and high) and compared the proportion of correct answers to the ten most difficult items for each intervention across the three knowledge levels. High knowledge providers were more likely to respond correctly to each of the ten most difficult HTC, PMTCT, and VMMC items, while low knowledge providers were less likely to do so (S1 Fig).
Provider and facility characteristics of interest
In our exploration of factors associated with HTC, PMTCT, and VMMC provider knowledge, we focused on provider and facility characteristics drawn from literature in organizational psychology, economics, management, and production research, while adjusting for factors identified as predictors of health care worker performance in previous work [9, 21, 52–59]. Our main variables of interest were: average patient load (average number of patients seen per week), provider years in position (number of years a provider has been in current position), provider working in another facility (whether the provider also works in another facility), senior staff (whether senior providers work in a facility), program age (number of years that a facility has been implementing HTC, PMTCT, and/or VMMC), proportion of exclusive staff (proportion of staff in a facility that works exclusively on one HIV intervention), training person-days (number of person-days of training provided to staff in a facility), and management score. The concepts that these variables aim to operationalize are shown in Table 1. We constructed both the senior staff variable and the management score. For the senior staff variable, we used principal component analysis (PCA) to construct a seniority score for all providers in the sample using years of education, years of experience, years in position, and cadre. We then grouped the seniority scores for each country into tertiles. Providers in the highest tertile were considered senior staff. The senior staff variable is a facility-level dummy variable, that is 1 if the facility has at least one senior provider and 0 otherwise. We created the management score based on information on facility-level management practices collected in the ORPHEA study through in-person interviews with managers responsible for HTC, PMTCT, and VMMC services. The interview questions were based on the World Management Survey (WMS) developed for manufacturing firms [60] and applied to health care facilities [54–57, 59]. We adapted questions to capture information relevant to health care facilities delivering HIV services [44, 61]. Our approach captured information on management practices in four domains: operations management, performance monitoring, people management, and community engagement. We constructed additive scores for each of these domains based on 68 management practice items (23 items for operations management, 15 for performance monitoring, 19 for people management, and 11 for community engagement) (See S3 Table for the list of management practice items). We then constructed an overall management score, including the 68 items together. We tested the internal consistency of the management scores using Cronbach’s Alpha (α = 0.85). The scores were standardized on a 0 to 100 scale. Multiple imputation was used to impute less than 5% of the management items that had missing values.
Data analysis
We estimated the mean, median, and standard deviation (SD) of the HTC, PMTCT, and VMMC provider knowledge scores and for the provider and facility characteristics. To explore the heterogeneity in HTC, PMTCT, and VMMC provider knowledge scores within countries, we mapped the distributions by country. To assess the heterogeneity across countries, we mapped the country distributions of provider knowledge scores for the three interventions.
We modeled the association between HTC, PMTCT, and VMMC provider knowledge and provider and facility attributes. We used generalized linear model (GLM) estimation—a maximum likelihood generalization of the ordinary linear regression approach. We specified a Poisson family distribution and the identity as the link function using Akaike and Bayes criteria to identify the model with the best goodness of fit.
The models were estimated as:
where yij is the provider knowledge score of provider i in facility j, PLij is the provider patient load in log, YPij is the number of years a provider has been in current position, OFij indicates whether the provider worked in another facility, SSj indicates whether the facility had at least one senior staff member, PAj denotes program age, PEj is the proportion of intervention exclusive staff, TPj is the number of training person-days, MSj is the management score. β and γ are the regression coefficients for provider and facility characteristics respectively. X is the vector of control variables (provider age, provider years of education, provider years of experience, provider cadre, intervention, urbanicity, facility type, facility ownership, number of annual patients, and country) and δ represents the regression coefficients for the control variables (see S4 Table for variable definitions and descriptions). We included interactions for country with cadre, and country with intervention. We report models with robust and clustered standard errors to account for the heteroskedasticity of the error term εij and unobserved intra-facility heterogeneity without explicitly modelling intra- and inter-cluster correlation. The clustered specification assumes that regression model errors are independent across facilities but correlated within facilities [62, 63]. We examined the Variance Inflation Factor (VIF) to assess multicollinearity.
For four of our variables of interest that were continuous (provider years in position, program age, proportion of exclusive staff, and management score), we computed the expected knowledge at different percentiles of the variables of interest (using the model with non-clustered standard errors).
Results
The analytic sample comprised 755 providers of HTC, 709 providers of PMTCT, and 332 providers of VMMC (Table 2). These providers came from 66 facilities in Kenya, 67 facilities in Rwanda, 57 facilities in South Africa, and 58 facilities in Zambia (Table 3). The average knowledge score was 36 for each of the three interventions.
Overall, HTC, PMTCT, and VMMC provider knowledge scores were low in all four countries. Within countries, there was some variation in scores across interventions, with provider knowledge scores for VMMC and PMTCT somewhat higher in Kenya than they were in Rwanda and Zambia (Fig 1). The distributions of scores for the three interventions in South Africa were similar. Across countries, provider knowledge scores for all three interventions were somewhat higher in Kenya and South Africa than they were in Rwanda and Zambia (Fig 2).
Notes: HTC, HIV testing and counseling; PMTCT, Prevention of Mother-to-Child Transmission; VMMC, Voluntary Medical Male Circumcision.
Notes: HTC, HIV testing and counseling; PMTCT, Prevention of Mother-to-Child Transmission; VMMC, Voluntary Medical Male Circumcision.
Table 4 shows the results of the GLM models. The coefficients are expressed as marginal effects. Model 1 estimated standard errors without clustering, Model 2 estimated robust standard errors, and Model 3 estimated standard errors clustering by facility. In Model 1, provider knowledge scores were higher among providers in facilities with senior staff members and among providers in facilities with higher proportions of intervention exclusive staff. We also found negative relationships between the outcome and provider years in position, average program age, provider working in another facility, average number of person-days of training, and management score. The robust standard errors (Model 2) were about double the size of the standard errors in Model 1, while in the model with clustering (Model 3) the standard errors were about three times the size of the errors in Model 1. In Model 3, only the coefficients for provider years in position, average program age, and management score remained statistically significant at conventional levels.
Fig 3 shows the relationship between provider knowledge scores and provider years in position, program age, proportion of exclusive staff, and management score along percentiles of the distribution of these variables. Provider knowledge scores ranged from 40 to 33 (17.5% reduction) between p10 and p90 of the management score, from 38 to 35 (7.9% reduction) when shifting from p10 to p90 of program age, and from 37 to 36 (2.7% reduction) when moving between p10 and p90 of provider years in position. Knowledge scores ranged from 34 to 38 (11.7% increase) between p10 and p90 of the proportion of exclusive staff.
Discussion
As others have argued, “the first step to closing the quality gap is to measure it [13].” As countries work to improve quality of HIV care and reach global HIV targets, measuring provider knowledge and other aspects of quality within individual countries and between countries is important. In this study of HIV service provider knowledge in Kenya, Rwanda, South Africa, and Zambia, we found mean knowledge scores of 36 for HTC, PMTCT, and VMMC. Within each country, some providers performed better, although few providers performed well. Between countries, knowledge scores were higher in Kenya and South Africa than they were in Rwanda and Zambia. The low scores we observed were consistent with prior studies using vignettes to assess provider knowledge in LMICs. Though we are unaware of previous work using vignettes to examine HIV service provider knowledge, this technique has been used to assess provider knowledge in an array of services including maternal and newborn health services [24–26, 28], childhood diarrhea and pneumonia [17–19, 21, 23, 26, 28], malaria [20, 21, 28], tuberculosis [17–19, 22, 26, 28, 64], and noncommunicable diseases [25, 27, 28]. Again and again, this growing body of work has shown that provider knowledge in resource-limited settings is low.
In our examination of the association between HTC, PMTCT, and VMMC provider knowledge and provider and facility characteristics, we observed differences in model results depending on whether we adjusted for facility-level clustering. This underscores the likely importance of organizational factors in individual HIV service provider knowledge. When we adjusted for intra-facility correlation, though the coefficients remained the same as in the models in which we did not adjust for clustering, many of the coefficients lost statistical significance. This suggests the presence of common, unobserved characteristics that affect the knowledge of providers belonging to the same cluster. This finding is in line with the arguments of organizational behavior researchers that knowledge accumulation occurs not just at the individual level, but at the organizational and industry levels as well [65]. Experts in organizational behavior argue that knowledge is a function of organizational experience [66], and that organizations learn through the accumulation of know-how and skill, improvements in workflow and coordination, and advances in technology [67]. Meanwhile, individuals embedded in organizations learn from knowledge reservoirs (e.g. treatment guidelines), tools, routines, social networks [38], and transactive memory systems [39, 40]. These transactive memory systems function like a computer or library for group members who can access information as part of the group that they might not access individually [68]. While these organizational characteristics are unobserved variables in our study, and therefore captured by the residual term of our regression models, our results are consistent with this literature. The potential of social learning and task coordination to increase provider knowledge should become part of future research agendas on service provision in resource-constrained settings.
Regardless of the regression specification model (either 1, 2, or 3), our results are consistent with five main hypotheses. First, individual experience is positively correlated with knowledge. Second, the presence of senior staff in a facility is positively correlated with knowledge. Third, there are diminishing returns of time on knowledge, both at the individual (provider years in position) and organizational (program age) levels. Fourth, staff dedication/specialization is positively associated with knowledge. Fifth, management and training are reactive rather than proactive with regard to improving provider knowledge.
Our finding that provider average patient load was positively associated with provider knowledge scores is consistent with the learning by doing literature that posits that as individuals repeat a task, they learn from direct experience and their knowledge about the task increases [38, 69]. Our hypothesis based on this literature was that more time spent on the job would translate into experience and knowledge as long as the workload was not so high that it overburdened staff. Previous assessments on the relationship between provider knowledge and workload and working time also found positive associations [22, 70].
Our result that provider knowledge scores were higher in facilities with at least one senior staff is supported by the extensive literature on champions as important facilitators in the successful implementation of health care services [71]. These individuals working in facilities with traits including enthusiasm, grit, persuasiveness, and intrinsic motivations and commitment to implementing improvements generally have a lot of influence [71, 72]. Champions tend to be more connected than other staff and this higher level of connectedness can facilitate a positive learning culture [72].
Our findings that provider years in position and average program age were negatively associated with provider knowledge are consistent with the assumption that knowledge capital depreciates over time and that knowledge can be unlearned or lost just as it can be learned or accumulated. Most literature on knowledge accumulation comes from manufacturing and finds sizeable rates of knowledge decay over time (e.g. [34, 73, 74]). Professional service workers like health care providers can be resistant to knowledge depreciation through formal education and training, compliance with standards and protocols, accreditation obligations, and continuous education requirements [75]. However knowledge decay can be considerable for procedural tasks which are dominant in health care [76, 77] and knowledge capital can depreciate when treatment guidelines evolve and take time to diffuse [35], staff do not have the motivation to master knowledge and skills [36], and facilities face staff turnover and organizational inertia [37].
Our findings on providers working in another facility and average proportion of staff in a facility focused exclusively on HTC, PMTCT, or VMMC are consistent with the literature on specialization and teamwork [39, 78, 79]. Providers working in another facility was negatively associated with provider knowledge, while average proportion of staff in a facility focused exclusively on HTC, PMTCT, or VMMC was positively associated with provider knowledge. Researchers have shown that specialization of the individuals that form a work group is an important dimension of the group’s transactive memory and that specialization is associated with more developed transactive memory systems [80, 81].
Our result that provider knowledge scores were higher in facilities with lower management scores was unexpected since several studies have found positive associations between quality of care and management practices [56, 57, 59, 82, 83]. It is true that these studies were conducted in high-resource settings and that there are low levels of adoption of modern management practices in low-resource settings, particularly in the public sector [6, 84]. It is also possible, however, that intensified management was a reaction to lower HIV service provider knowledge. There is evidence that management intensity increases when work processes are suboptimal and managers take actions to ensure that problems are addressed [85, 86]. This reactive feature of management will create a negative relationship when examining cross-sectional data like the ORPHEA study data explored here [87].
Our finding that number of person days of training in a facility was negatively associated with provider knowledge was also unexpected since training programs have been shown to improve provider practices when knowledge deficiencies impede the provision of quality services [88]. However, it is possible that, as with management practices, training of HIV service providers is reactive in our data and that training intensity increases when work processes are inferior.
Several limitations should be kept in mind when considering this analysis. Quality of care comprises multiple elements and our study focused on one dimension. Using vignettes, we measured the knowledge of health care providers. Vignettes measure what providers know and not what they do [18]. There is a growing literature describing the existence in LMICs of both a knowledge gap—a deficiency of knowledge or awareness of guidelines among providers [9, 18, 21, 25, 45] and a “know-do” gap—a gap between what providers know they should do and what they actually do [12, 16, 23, 64, 89–91]. Knowledge is a crucial starting point for quality of care, a necessary though not sufficient condition. Our knowledge scores were also limited in terms of the scope of the services studied, especially for PMTCT and VMMC. The PMTCT vignette focused on one antenatal care visit, while the VMMC vignette focused on pre- and post-operative counselling. We also note that there have been considerable changes in the ways in which HIV testing, prevention, and treatment interventions are implemented since the time of the study. However, our findings provide insights that are not specific to HTC, PMTCT, and VMMC, as implemented in 2012 and 2013. In this paper, we investigated aspects of provider knowledge that are relevant for quality of health care services beyond HIV services. The level of completeness of provider and facility characteristics in the ORPHEA surveys varied and limited our sample and the associations we could examine in this analysis. In addition, the ORPHEA study did not collect information on health systems and other environmental factors that may be associated with provider knowledge including country training and education schemes and health governance and regulations [13]. Our data were cross-sectional, and the results were descriptive. The potential endogeneity caused by reverse causality between provider knowledge and management practices and staff training days highlights the importance of longitudinal data to understand quality of care and its determinants.
Conclusion
The policy recommendations arising from a cross-sectional study on the level of provider knowledge of HIV service providers and the associations between provider and facility characteristics and provider knowledge are limited [19]. Our cross-sectional data do not allow us to establish causality though our rich data are suitable to identify relevant hypotheses for future studies. Our exploration of expected provider knowledge at different percentiles of provider and facility characteristics does highlight the difficulty of raising provider knowledge scores from the low levels we observed. Our study also suggests that unobservable organizational factors at the facility level may facilitate communication, learning and knowledge. On the one hand, our study indicates that the presence of senior staff and staff dedication may enable knowledge acquisition. On the other hand, our study provides a note of caution on the potential knowledge depreciation correlated with the amount of time staff spend in a position and program age. Finally, our study underscores the importance of investing in human capital and improving managerial skills, with a focus on proactive management practices aimed at fostering a culture of reporting errors and continuous improvement. We advocate for gathering more granular and longitudinal data on the quality of HIV services and other health care services in sub-Saharan Africa.
Supporting information
S1 Fig. Proportion of correct answers to most difficult items for HTC, PMTCT, and VMMC by provider knowledge levels.
Notes: HTC, HIV testing and counseling; PMTCT, prevention of mother-to-child transmission; VMMC, voluntary medical male circumcision.
https://doi.org/10.1371/journal.pone.0260571.s001
(TIF)
S1 Table. Description of HTC, PMTCT, and VMMC services examined in the ORPHEA study.
Notes: HTC, HIV testing and counseling; PMTCT, prevention of mother-to-child transmission; VMMC, voluntary medical male circumcision; AZT, Zidovudine; 3TC, lamivudine; EFV, Efavirenz; FTC, Emcitricitabine; TDF, Tenofovir; sd, single dose; NVP, Nevirapine; CTX, Cotrimoxazole.
https://doi.org/10.1371/journal.pone.0260571.s002
(DOCX)
S2 Table. Average percent of items correctly identified by intervention, provider cadre, and by country.
Notes: HTC, HIV testing and counseling; PMTCT, prevention of mother-to-child transmission; VMMC, voluntary medical male circumcision.
https://doi.org/10.1371/journal.pone.0260571.s003
(DOCX)
S3 Table. Management practice items by management domain.
https://doi.org/10.1371/journal.pone.0260571.s004
(DOCX)
S4 Table. Description of variables.
Notes: HTC, HIV testing and counseling; PMTCT, prevention of mother-to-child transmission; VMMC, voluntary medical male circumcision.
https://doi.org/10.1371/journal.pone.0260571.s005
(DOCX)
Acknowledgments
We are grateful to our colleagues Gina La Hera-Fuentes, Andrea Salas-Ortiz and Nerissa Nance for their help in the construction of the management score. We acknowledge the contribution of the ORPHEA study team in the design of the ORPHEA study and data collection: Kenya: Joseph Wang’ombe, Richard Wamai, and Omar Galárraga. Rwanda: Jeanine Condo and Sabin Nsanzimana. South Africa: Neil Martinson. Zambia: Felix Masiye, Abson Chompolola, and Kumbutso Dzekedzeke. Mexico: Sandra Sosa-Rubí, Ada Kwan, and Claire Chaumont.
References
- 1.
United Nations. Resolution adopted by the General Assembly on 25 September 2015. Transforming our world: the 2030 Agenda for Sustainable Development (A/Res/70/1). New York: United Nations General Assembly; 2015.
- 2.
Joint United Nations Programme on HIV/AIDS. Understanding fast-track; accelerating action to end the AIDS epidemic by 2030. Geneva: Joint United Nations Programme on HIV/AIDS, 2015.
- 3.
Joint United Nations Programme on HIV/AIDS. Global AIDS Update 2020. Geneva: Joint United Nations Programme on HIV/AIDS; 2020.
- 4. Ghys PD, Williams BG, Over M, Hallett TB, Godfrey-Faussett P. Epidemiological metrics and benchmarks for a transition in the HIV epidemic. PLoS Med. 2018;15(10):e1002678. Epub 2018/10/26. pmid:30359372
- 5.
World Health Organization. Maintaining and improving quality of care within HIV clinical services. Geneva: World Health Organization, 2019.
- 6. Kruk ME, Gage AD, Arsenault C, Jordan K, Leslie HH, Roder-DeWan S, et al. High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Glob Health. 2018;6(11):e1196–e252. Epub 2018/09/10. pmid:30196093.
- 7.
National Academies of Sciences Engineering and Medicine. Crossing the Global Quality Chasm: Improving Health Care Worldwide. Washington, DC: The National Academies Press, 2018.
- 8.
World Health Organization, Organisation for Economic Co-operation and Development, the World Bank. Delivering quality health services: a global imperative for universal health coverage. Geneva: World Health Organization, 2018.
- 9. Das J, Hammer J, Leonard K. The quality of medical advice in low-income countries. J Econ Perspect. 2008;22(2):93–114. Epub 2008/04/01. pmid:19768841.
- 10. Jha AK, Larizgoitia I, Audera-Lopez C, Prasopa-Plaizier N, Waters H, Bates DW. The global burden of unsafe medical care: analytic modelling of observational studies. BMJ Qual Saf. 2013;22(10):809–15. Epub 2013/09/21. pmid:24048616.
- 11. Das J, Mohpal A. Socioeconomic status and quality of care in rural India: new evidence from provider and household surveys. Health Aff (Millwood). 2016;35(10):1764–73. Epub 2016/10/06. pmid:27702947.
- 12. Daniels B, Dolinger A, Bedoya G, Rogo K, Goicoechea A, Coarasa J, et al. Use of standardised patients to assess quality of healthcare in Nairobi, Kenya: a pilot, cross-sectional study with international comparisons. BMJ Glob Health. 2017;2(2):e000333. Epub 2017/12/12. pmid:29225937
- 13. Kruk ME, Chukwuma A, Mbaruku G, Leslie HH. Variation in quality of primary-care services in Kenya, Malawi, Namibia, Rwanda, Senegal, Uganda and the United Republic of Tanzania. Bull World Health Organ. 2017;95(6):408–18. Epub 2017/06/13. pmid:28603307
- 14. Leslie HH, Spiegelman D, Zhou X, Kruk ME. Service readiness of health facilities in Bangladesh, Haiti, Kenya, Malawi, Namibia, Nepal, Rwanda, Senegal, Uganda and the United Republic of Tanzania. Bull World Health Organ. 2017;95(11):738–48. Epub 2017/11/18. pmid:29147054
- 15. Leslie HH, Sun Z, Kruk ME. Association between infrastructure and observed quality of care in 4 healthcare services: A cross-sectional study of 4,300 facilities in 8 countries. PLoS Med. 2017;14(12):e1002464. Epub 2017/12/13. pmid:29232377
- 16. Kwan A, Daniels B, Saria V, Satyanarayana S, Subbaraman R, McDowell A, et al. Variations in the quality of tuberculosis care in urban India: A cross-sectional, standardized patient study in two cities. PLoS Med. 2018;15(9):e1002653. Epub 2018/09/27. pmid:30252849
- 17. Barber SL, Gertler PJ, Harimurti P. Differences in access to high-quality outpatient care in Indonesia. Health Aff (Millwood). 2007;26(3):w352–66. Epub 2007/03/29. pmid:17389632.
- 18. Das J, Hammer J. Which doctor? Combining vignettes and item response to measure clinical competence. J Dev Econ. 2005;78(2):348–83.
- 19. Das J, Hammer J. Location, location, location: residence, wealth, and the quality of medical care in Delhi, India. Health Aff (Millwood). 2007;26(3):w338–51. Epub 2007/03/29. pmid:17389631.
- 20. Gage AD, Kruk ME, Girma T, Lemango ET. The know-do gap in sick child care in Ethiopia. PLoS One. 2018;13(12):e0208898. Epub 2018/12/13. pmid:30540855
- 21. Leonard KL, Masatu MC. Variations in the quality of care accessible to rural communities in Tanzania. Health Aff (Millwood). 2007;26(3):w380–92. Epub 2007/03/29. pmid:17389635.
- 22. Mohanan M, Goldhaber-Fiebert JD, Giardili S, Vera-Hernandez M. Providers’ knowledge of diagnosis and treatment of tuberculosis using vignettes: evidence from rural Bihar, India. BMJ Glob Health. 2016;1(4):e000155. Epub 2017/06/08. pmid:28588984
- 23. Mohanan M, Vera-Hernandez M, Das V, Giardili S, Goldhaber-Fiebert JD, Rabin TL, et al. The know-do gap in quality of health care for childhood diarrhea and pneumonia in rural India. JAMA Pediatr. 2015;169(4):349–57. Epub 2015/02/17. pmid:25686357
- 24. Murphy GAV, Gathara D, Mwaniki A, Nabea G, Mwachiro J, Abuya N, et al. Nursing knowledge of essential maternal and newborn care in a high-mortality urban African setting: A cross-sectional study. J Clin Nurs. 2019;28(5–6):882–93. Epub 2018/10/26. pmid:30357971
- 25. Peabody JW, DeMaria L, Smith O, Hoth A, Dragoti E, Luck J. Large-scale evaluation of quality of care in 6 countries of Eastern Europe and Central Asia using clinical performance and value vignettes. Glob Health Sci Pract. 2017;5(3):412–29. Epub 2017/10/01. pmid:28963174
- 26. Peabody JW, Liu A. A cross-national comparison of the quality of clinical care using vignettes. Health Policy Plan. 2007;22(5):294–302. Epub 2007/07/31. pmid:17660225.
- 27. Peabody JW, Luck J, DeMaria L, Menon R. Quality of care and health status in Ukraine. BMC Health Serv Res. 2014;14:446. Epub 2014/10/02. pmid:25269470
- 28. Rao KD, Sundararaman T, Bhatnagar A, Gupta G, Kokho P, Jain K. Which doctor for primary health care? Quality of care and non-physician clinicians in India. Soc Sci Med. 2013;84:30–4. Epub 2013/03/23. pmid:23517701.
- 29. Puchalski Ritchie LM, Khan S, Moore JE, Timmings C, van Lettow M, Vogel JP, et al. Low- and middle-income countries face many common barriers to implementation of maternal health evidence products. J Clin Epidemiol. 2016;76:229–37. Epub 2016/03/05. pmid:26931284.
- 30. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8. Epub 1988/09/23. pmid:3045356.
- 31. Donabedian A. The quality of medical care. Science. 1978;200(4344):856–64. Epub 1978/05/26. pmid:417400
- 32. Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966;44(3):Suppl:166–206. Epub 1966/07/01. pmid:5338568.
- 33. Epple D, Argote L, Murphy K. An empirical investigation of the microstructure of knowledge acquisition and transfer through learning by doing. Oper Res. 1996;44(1):77–86.
- 34. Darr ED, Argote L, Epple D. The acquisition, transfer, and depreciation of knowledge in service organizations: Productivity in franchises. Manag Sci. 1995;41(11):1750–62.
- 35. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629. Epub 2004/12/15. pmid:15595944
- 36. Katz D. The motivational basis of organizational behavior. Behav Sci. 1964;9(2):131–46. pmid:5888769
- 37. Hannan MT, Freeman J. Structural inertia and organizational change. Am Sociol Rev. 1984;49:149–64.
- 38. Levitt B, March JG. Organizational learning. Annu Rev Sociol. 1988;14:319–40.
- 39. Brandon DP, Hollingshead AB. Transactive memory systems in organizations: matching tasks, expertise, and people. Organ Sci. 2004;15(6):633–44.
- 40. Walsh JP, Ungson GR. Organizational memory. Acad Manage Rev. 1991;16(1):57–91.
- 41. Bautista-Arredondo S, Sosa-Rubi SG, Opuni M, Contreras-Loya D, Kwan A, Chaumont C, et al. Costs along the service cascades for HIV testing and counselling and prevention of mother-to-child transmission. AIDS. 2016;30(16):2495–504. pmid:27753679
- 42. Bautista-Arredondo S, Sosa-Rubi SG, Opuni M, Contreras-Loya D, La Hera-Fuentes G, Kwan A, et al. Influence of supply-side factors on voluntary medical male circumcision costs in Kenya, Rwanda, South Africa, and Zambia. PLoS One. 2018;13(9):e0203121. Epub 2018/09/14. pmid:30212497
- 43. Bautista-Arredondo S, Sosa-Rubi SG, Opuni M, Kwan A, Chaumont C, Coetzee J, et al. Assessing cost and technical efficiency of HIV prevention interventions in sub-Saharan Africa: the ORPHEA study design and methods. BMC Health Serv Res. 2014;14:599. pmid:25927555
- 44. Salas-Ortiz A, La Hera-Fuentes G, Nance N, Sosa-Rubi SG, Bautista-Arredondo S. The relationship between management practices and the efficiency and quality of voluntary medical male circumcision services in four African countries. PLoS One. 2019;14(10):e0222180. Epub 2019/10/04. pmid:31581192
- 45. Das J, Gertler PJ. Variations in practice quality in five low-income countries: a conceptual overview. Health Aff (Millwood). 2007;26(3):w296–309. Epub 2007/03/29. pmid:17389637.
- 46. Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000;283(13):1715–22. pmid:10755498.
- 47. Peabody JW, Luck J, Glassman P, Jain S, Hansen J, Spell M, et al. Measuring the quality of physician practice by using clinical vignettes: a prospective validation study. Ann Intern Med. 2004;141(10):771–80. pmid:15545677.
- 48. Peabody JW, Tozija F, Munoz JA, Nordyke RJ, Luck J. Using vignettes to compare the quality of clinical care variation in economically divergent countries. Health Serv Res. 2004;39(6 Pt 2):1951–70. Epub 2004/11/17. pmid:15544639
- 49. Leonard KL, Masatu MC, Vialou A. Getting clinicians to do their best: ability, altruism, and incentives. J Hum Resour. 2007;42(3):682–700.
- 50.
Embretson SE, Reise SP. Item response theory. Mahwah: Lawrence Erlbaum Associates, Inc; 2000. https://doi.org/10.1207/S15327906MBR3504_05 pmid:26811203
- 51.
de Ayala RJ. The theory and practice of item response theory. New York: Guilford Press; 2009. https://doi.org/10.3382/ps.2008-00502 pmid:19590082
- 52. Barber SL, Gertler PJ. Empowering women to obtain high quality care: evidence from an evaluation of Mexico’s conditional cash transfer programme. Health Policy Plan. 2009;24(1):18–25. Epub 2008/11/22. pmid:19022854
- 53. Das J, Hammer J. Money for nothing: the dire straits of medical practice in Delhi, India. J Dev Econ. 2007;83(1):1–36.
- 54.
Dorgan S, Layton D, Bloom N, Homkes R, Sadun R, Van Reenen J. Management in healthcare: why good practice really matters. London: McKinsey & Company; 2010.
- 55. McConnell KJ, Chang AM, Maddox TM, Wholey DR, Lindrooth RC. An exploration of management practices in hospitals. Healthc (Amst). 2014;2(2):121–9. Epub 2015/08/08. pmid:26250380.
- 56. McConnell KJ, Lindrooth RC, Wholey DR, Maddox TM, Bloom N. Management practices and the quality of care in cardiac units. JAMA Intern Med. 2013;173(8):684–92. Epub 2013/04/05. pmid:23552986.
- 57. McConnell KJ, Lindrooth RC, Wholey DR, Maddox TM, Bloom N. Modern management practices and hospital admissions. Health Econ. 2016;25(4):470–85. Epub 2015/02/26. pmid:25712429.
- 58. Rowe AK, de Savigny D, Lanata CF, Victora CG. How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet. 2005;366(9490):1026–35. Epub 2005/09/20. pmid:16168785.
- 59. Tsai TC, Jha AK, Gawande AA, Huckman RS, Bloom N, Sadun R. Hospital board and management practices are strongly related to hospital performance on clinical quality metrics. Health Aff (Millwood). 2015;34(8):1304–11. Epub 2015/08/05. pmid:26240243.
- 60. Bloom N, Van Reenen J. Measuring and explaining management practices across firms and countries. Q J Econ. 2007;122(4):1351–408.
- 61. Macarayan EK, Ratcliffe HL, Otupiri E, Hirschhorn LR, Miller K, Lipsitz SR, et al. Facility management associated with improved primary health care outcomes in Ghana. PLoS One. 2019;14(7):e0218662. Epub 2019/07/03. pmid:31265454
- 62.
Abadie A, Athey S, Imbens GW, Wooldridge J. When should you adjust standard errors for clustering? Cambridge, MA: National Bureau of Economic Research, Inc; 2017.
- 63. Cameron AC, Miller DL. A practitioner’s guide to cluster-robust inference. J Hum Resour. 2015;50(2):317–72.
- 64. Das J, Kwan A, Daniels B, Satyanarayana S, Subbaraman R, Bergkvist S, et al. Use of standardised patients to assess quality of tuberculosis care: a pilot, cross-sectional study. Lancet Infect Dis. 2015;15(11):1305–13. Epub 2015/08/14. pmid:26268690
- 65. Harvey RA. Learning in production. The Statistician. 1979;28(1):39–57.
- 66. Argote L. Organizational learning research: Past, present and future. Manag Learn. 2011;42(4):439–46.
- 67.
Argote L. Organizational learning: creating, retaining and transferring knowledge. Berlin: Springer; 1999. https://doi.org/10.1111/j.1552-6909.1999.tb01970.x pmid:9924870
- 68. Liang DW, Moreland R, Argote L. Group versus individual training and group performance: the mediating role of transactive memory. Pers Soc Psychol Bull. 1995;21(4):384–93.
- 69. Kim I, Seo HL. Depreciation and transfer of knowledge: an empirical exploration of a shipbuilding process. Int J Prod Res. 2009;47(7):1857–76.
- 70. Lohela TJ, Nesbitt RC, Manu A, Vesel L, Okyere E, Kirkwood B, et al. Competence of health workers in emergency obstetric care: an assessment using clinical vignettes in Brong Ahafo region, Ghana. BMJ Open. 2016;6(6):e010963. Epub 2016/06/15. pmid:27297010
- 71. Miech EJ, Rattray NA, Flanagan ME, Damschroder L, Schmid AA, Damush TM. Inside help: An integrative review of champions in healthcare-related implementation. SAGE Open Med. 2018;6:2050312118773261. Epub 2018/05/26. pmid:29796266
- 72. Bonawitz K, Wetmore M, Heisler M, Dalton VK, Damschroder LJ, Forman J, et al. Champions in context: which attributes matter for change efforts in healthcare? Implement Sci. 2020;15(1):62. Epub 2020/08/09. pmid:32762726
- 73. Argote L, Epple D. Learning curves in manufacturing. Science. 1990;247(4945):920–4. Epub 1990/02/23. pmid:17776451.
- 74. Epple D, Argote L, Devadas R. Organizational Learning Curves: A Method for Investigating Intra-Plant Transfer of Knowledge Acquired Through Learning by Doing. Organ Sci. 1991;2(1):58–70.
- 75. Boone T, Ganeshan R, Hicks RL. Learning and knowledge depreciation in professional services. Manage Sci. 2008;54(7):1231–6.
- 76. Arzi Y, Shtub A. Learning and forgetting in mental and mechanical tasks: a comparative study. IIE Trans. 1997;29:759–68.
- 77. Bailey CD. Forgetting and the learning curve: A laboratory study. Manage Sci. 1989;35(3):340–52.
- 78. Postrel S. Islands of shared knowledge: specialization and mutual understanding in problem-solving teams. Organ Sci. 2002;13:303–20.
- 79. Powell SG. Specialization, teamwork, and production efficiency. Int J Prod Econ. 2000;67:205–18.
- 80. Lewis K. Measuring transactive memory systems in the field: scale development and validation. J Appl Psychol. 2003;88(4):587–604. Epub 2003/08/28. pmid:12940401.
- 81. Wegner DM. A computer network model of human transactive memory. Soc Cogn. 1995;13(3):319–39.
- 82. Bloom N, Sadun R, Lemos R, Van Reenen J. Healthy business? Managerial education and management in healthcare. Rev Econ Stat. 2020;102(3):506–17.
- 83. Lega F, Prenestini A, Spurgeon P. Is management essential to improving the performance and sustainability of health care systems and organizations? A systematic review and a roadmap for future studies. Value Health. 2013;16(1 Suppl):S46–51. Epub 2013/01/18. pmid:23317645.
- 84.
Dunsch F, Evans DK, Eze-Aioku E, Macis M. Management, supervision, and healthcare: a field experiment. Cambridge, MA: National Bureau of Economic Research, Inc; 2017.
- 85. Larson LL, Bussom RS, Vicars W, Jauch L. Proactive versus reactive manager: is the dichotomy realistic? J Manag Stud. 1986;23(4):385–400.
- 86. Zaleznik A. Managers and leaders. Are they different? 1977. Harv Bus Rev. 2004;82(1):74–81. pmid:14723179
- 87. March JG, Sutton RI. Crossroads—Organizational performance as a dependent variable. Organ Sci. 1997;8(6):698–706.
- 88. Rowe AK, Rowe SY, Peters DH, Holloway KA, Chalker J, Ross-Degnan D. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. Lancet Glob Health. 2018;6(11):e1163–e75. Epub 2018/10/13. pmid:30309799
- 89. Das J, Holla A, Das V, Mohanan M, Tabak D, Chan B. In urban and rural India, a standardized patient study showed low levels of provider training and huge quality gaps. Health Aff (Millwood). 2012;31(12):2774–84. Epub 2012/12/06. pmid:23213162
- 90. Satyanarayana S, Kwan A, Daniels B, Subbaraman R, McDowell A, Bergkvist S, et al. Use of standardised patients to assess antibiotic dispensing for tuberculosis by pharmacies in urban India: a cross-sectional study. Lancet Infect Dis. 2016;16(11):1261–8. Epub 2016/10/30. pmid:27568359
- 91. Sylvia S, Xue H, Zhou C, Shi Y, Yi H, Zhou H, et al. Tuberculosis detection and the challenges of integrated care in rural China: A cross-sectional standardized patient study. PLoS Med. 2017;14(10):e1002405. Epub 2017/10/19. pmid:29040263