https://orcid.org/0000-0003-2457-4183
Figures
Abstract
This paper describes the simultaneous co-development of Oral Health Behavior Social Support (OHBSS) scales in English and Spanish. OHBSS scales assess social support for toothbrushing, flossing, and dental care utilization, which are targets for interpersonal-level interventions to promote oral health among Hispanic/Latino adults. The focus was on Mexican-origin adults, who comprise the largest United States Hispanic/Latino subgroup and experience a high oral disease burden. All participants self-identified as Mexican-origin adults (ages 21–40 years old), living along the California-Arizona-Mexico border. Independent samples were recruited for each study partnering with Federally Qualified Health Centers. First, we conducted semi-structured interviews about social support for oral health behaviors in August to November 2018 (Study 1, N = 72). Interviews were audio recorded, transcribed (in original language, Spanish or English), and qualitative data were coded and analyzed in Dedoose following three topical codebooks; excerpts were used to co-create the large bilingual item data bank (OHBSSv1). The item bank was pre-tested via 39 cognitive interviews between December 2019 to March 2020, reviewed by an expert panel with several bilingual members, reduced to 107 Spanish/109 English items (OHBSSv2), then pilot tested in January to December 2021 (Study 2, N = 309). Pilot survey data were analyzed through Exploratory Factor Analysis and Horn’s parallel analysis, overall and by language, to examine response patterns and inform item selection (OHBSSv3). The scales queried social support for toothbrushing, flossing, and dental care utilization across 39 items from three sources (family, health providers, others/friends), plus up to nine optional dental care-related items (Study 3, conducted April 2022 to February 2023, N = 502). Confirmatory Factor Analysis (CFA) assessed model fit, overall and by language (multiple group CFA). Final OHBSS scales include 37 items, plus seven optional items. Acceptable model fit for three-factor structures for each oral health behavior was found, providing evidence of the scales’ construct validity. Cronbach’s alphas and McDonald’s omegas were tabulated; all were above 0.95, overall and by language, supporting scales’ internal consistency.
Citation: Finlayson TL, Malcarne VL, Ayala GX, Schiaffino MK, Hoeft KS, Garcia-Alcaraz C, et al. (2025) Development of new bilingual oral health behavior social support (OHBSS) scales in English and Spanish. PLoS ONE 20(3): e0317133. https://doi.org/10.1371/journal.pone.0317133
Editor: Samane Shirahmadi, Hamadan University of Medical Sciences School of Dentistry, IRAN, ISLAMIC REPUBLIC OF
Received: February 8, 2024; Accepted: December 22, 2024; Published: March 11, 2025
Copyright: © 2025 Finlayson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: Data cannot be shared publicly because participants were not informed that data would be shared publicly as part of the informed consent process. Data access requests can be directed to Eileen Allen, IBACH Institute Manager, at eallen@sdsu.edu
Funding: Research reported was supported by the National Institute of Dental & Craniofacial Research of the National Institutes of Health (NIH) under Award Number R01DE026742 (Finlayson). Administrative and IT support was provided by the SDSU HealthLINK Endowment from NIH grant S21 MD010690 (Ayala). Data for Study 2 and 3 were collected with UCSF's REDCap Academic system electronic data collection tools that were developed with support from NIH grant U01DE025507 (Gansky). A San Diego County Employees' Charitable Organization to Dr. Finlayson supported the purchase of dental kits for San Diego county participants in Study 3. Several student trainees were supported by the San Diego State University (SDSU) Summer Undergraduate Research Program and SDSU doctoral program matching support awards to Dr. Finlayson. Several staff and trainees were also directly supported by Dr. Finlayson’s SDSU Research Support Fund. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Disparities in oral diseases and access to care persist among racial/ethnic minority adults in the United States (US), particularly among individuals of lower socioeconomic status (SES), on Medicaid, and those living in rural geographic locations [1]. Hispanics/Latinos/as/x/e comprise the largest racial/ethnic group in the US [2]. They are of diverse heritage [3], with the largest subgroup of US Latinos being of Mexican-origin, either US-born or Mexico-born, with many living along the Mexico border region in California (CA) and Arizona (AZ) [4]. Mexican-origin adults are disproportionately disadvantaged with unmet dental needs, and Mexican immigrants had worse oral health status than other Latino heritage groups [5–7]. Mexican-American adults have the highest periodontal disease prevalence compared to non-Hispanic Whites and non-Hispanic Blacks [1,8]. Poor oral hygiene practices are a potentially modifiable behavioral risk factor for worse oral health.
Improving and maintaining daily oral hygiene for Mexican-origin adults can help reduce risk of oral diseases. Hygiene behavior frequency affects clinically assessed oral health outcomes; e.g., dental caries and periodontal disease [9–13]. The American Dental Association (ADA) recommends twice daily toothbrushing and once daily flossing, and annual dental visits [14,15]. Brushing alone does not clean interproximally (i.e., in between teeth) and many do not floss effectively ([9]). Longitudinal clinical trials showed good plaque control effectively ameliorates the rate of periodontal disease progression [16–18]. Good oral hygiene may effectively manage early periodontal disease [19]. Mexican-American adults may face barriers to proper adherence to ADA hygiene recommendations, such as knowing how to properly brush and floss [20]. Understanding how factors like social support could be targeted in future interventions to promote good oral hygiene are needed for Mexican-origin adults.
Dental visits provide opportunities for oral hygiene instruction and education about preventing oral diseases. However, US adult dental care utilization has been declining nationwide (41% to 37% in 2001–2010) [21]. Lack of insurance and high cost are frequently cited barriers to dental care among Hispanic immigrants [22]. There are challenging structural and financial barriers to accessing dental care, but some aspects of the complex health system could be navigated with support from others. There is strong evidence for family system [23], peer support [24] and community health worker interventions [24,25] to successfully promote behavior change, including among US Hispanics/Latinos [26]. Thus, sources and types of social support for oral health-promoting behaviors are important components for behavioral interventions. Yet, no domain-specific social support measure exists for oral health.
Social support refers to a social relationship with another individual from whom they might draw some support (such as a family member, friend, or health provider, for example). Social support has both structural (e.g., number of social ties with others, frequency of contacts) and functional components (satisfaction with emotional, informational, appraisal and instrumental support) [27,28], with the structural component as a necessary antecedent to social support. Perceived availability of structural and functional components is often measured [29–31], and important for health [31]. Greater perceived social support is positively associated with better mental and physical health outcomes [32–37]. There is evidence of health benefits from social support interventions, including those involving Mexican heritage populations in the US [38], however, we do not have a full understanding of mechanisms of action [39]. Social support has been more frequently studied as a psychosocial resource that people can draw on to help them cope with stressors or problem solve barriers [40,41]. Two dominant theories about how social relationships and social support influence health include: a) a direct model through physiological processes (like inflammation) and b) an indirect model affecting behaviors, which then influence the physiological process and health outcomes [41–43].
Research on social support in oral health is growing, though the existing body of research is still mostly descriptive, and conducted using varied brief general social support scales [44–49]. Dahlan and colleagues recently reviewed the relationship between social support and several oral health outcomes among immigrants, and found higher social support positively associated with dental visits and oral hygiene behaviors [44]. Existing validated general social support scales do exist [30,31], and most assess perceived functional support and emotional support. However, general scales may miss social support dimensions that are meaningful for particular diseases or behaviors [50]. For instance, in Sallis and colleagues’ diet and exercise social support measure, general social support measures were not associated with the domain-specific measure developed, and only the domain-specific social support measure was associated with the target heath behaviors [51].
Creating new Oral Health Behavior-specific Social Support (OHBSS) scales will enable more precise and accurate measurement for use in behavioral intervention research. The OHBSS scales were developed for Spanish- and English-speaking Mexican-origin adults, to assess their social support in the context of three oral health behaviors: toothbrushing, flossing, and dental care. For each behavior, we measure social support from three different sources: family, health providers, and others/friends. This paper details the development and refinement of a new tool to measure social support (the OHBSS scales) in English and Spanish. We present model fit statistics (overall and by language) and internal consistency coefficients to demonstrate the structural validity and reliability of the final scales.
Materials and methods
Dual-focus bilingual scale development approach and study design overview
This multi-phased, mixed methods sequential study [52] began with an exploratory qualitative study (Study 1) to co-create a large item bank for the scales in two languages (English and Spanish) simultaneously, employing a dual-focus approach informed by Erkut [53,54]. Study 2 employed qualitative cognitive interviews to pre-test items, then quantitatively analyzed the pilot survey draft scales, to attain a highly focused reduction in the size the scales. Study 3 employed quantitative methods to refine and test the structural validity and internal consistency of the final scales, and comprise the main final scale results [55,56]. In the present manuscript, we report results for the three studies.
We first composed a multi-disciplinary study team of investigators with appropriate expertise and background in oral health, social support, biostatistics, translation, and psychology (specifically in scale development and validation). Some investigators and many study staff were bilingual in English/Spanish, and most of the staff were of Mexican origin. All investigators and several study staff, along with staff from the partner Federally Qualified Health Centers (FQHCs) and community, were part of the expert panel. The panel was knowledgeable about the priority populations, and many were fluent in and had an in-depth understanding of Mexican Spanish. We planned and conducted primary data collection through three studies to develop and refine the OHBSS scales (see Fig 1), outlining the dual focus approach to developing the bilingual OHBSS scales.
Setting
Our community-engaged study was conducted in partnership with three FQHCs along the California (CA) and Arizona (AZ) border with Mexico (MX). Two large migrant FQHCs in CA, Vista Community Clinic (VCC) and Innercare (formerly Clínicas de Salud del Pueblo, Inc., or CSDSP, during Studies 1 and 2), were subcontracted partners involved in all phases of study planning and implementation. The largest FQHC in AZ, El Rio Health, partnered on Study 3.
Priority population
We focused on Mexican-origin men and women, ages 21–40 years. We intentionally focused on the largest racial/ethnic group in the US, Hispanics/Latinos, and further selected to focus on adults of Mexican-origin background (the largest Hispanic/Latino heritage group). Clinical reasons supported our narrowed focus on Mexican-origin adults; they experience a high burden of disease, and poorer oral health than other Hispanic/Latino subgroups and non-Hispanic Whites and Blacks [1,6,8]. Periodontal disease prevalence trends among US adults over age 30 show the increase with age is largely driven by those in the moderate periodontal disease category; those in the mild or severe categories appeared stable over age 40 [57]. The target age range was selected for developmental reasons. “Late adolescence” (ages 18–20) is marked by significant physical, social, emotional growth and transitions in home, school, and work [58]. Age 21–40 was selected as it captured all “young adulthood.” This age range also encompasses a prime window of opportunity for behavioral intervention and preventive efforts that could significantly affect oral health [59]. All data collection activities involved both US- and MX-born young adults living near the southwestern US-Mexico border.
Eligibility criteria.
For all three studies, participants were eligible if they: 1) self-identified as Mexican-origin (defined broadly, included Mexican, Mexican-American, Chicano/a, and other identities), 2) were between 21–40 years old (at eligibility screening), 3) able to speak, read, and provide written informed consent in English or Spanish, and 4) resided in designated areas. Geographic inclusion criteria were expanded with each subsequent study to simplify screening and recruitment, more effectively reach new potential participants and enable enrolling independent samples to meet increasing sample size goals for each study. For Study 1, participants had to reside in northern San Diego County, CA, within specified clinic service areas near VCC’s largest clinic in Vista, or in Imperial County, CA, within Innercare’s clinic service areas. For Study 2, participants could reside anywhere in San Diego or Imperial counties. For Study 3, participants could reside anywhere in San Diego, Riverside, Orange, or Imperial Counties in CA (two CA-MX border counties, plus two counties where VCC and Innercare operated clinic sites); or Pima, Cochise, Santa Cruz, or Yuma Counties in AZ (all four AZ-MX border counties). Recruitment and data collection for Studies 2 and 3 occurred during the COVID-19 pandemic; it was simpler to define and screen residential inclusion eligibility criterion by county.
Exclusion criteria for all studies included: being edentulous or having a health condition requiring pre-medication before dental exams or having a mental or physical impairment and thus unable to provide informed consent. For women, being pregnant was a temporary exclusion criterion, due to pregnancy-related hormone changes that affect women’s oral health [60].
Study design balance characteristics.
For all three studies, each data collection effort recruited independent parallel samples of participants. Given our bilingual scale development goal, we closely monitored and strived to manage enrollment to achieve optimally balanced study samples, especially by language. The goal was for each sample to comprise about 50% Spanish-speakers (top priority), and ideally also 50% male (for sex-stratification analyses), and 50% married/partnered (to account for potential differences in available social support); also, this approach optimized statistical power to examine differences based on such characteristics. Thus, it was possible for an eligible and interested potential participant to not be enrolled, if a particular subgroup had already reached the target goal. In these cases, potential participants were waitlisted and contacted for a later study in this effort to achieve ideal cross-classified balanced samples by language, sex, and marital status in all phases.
Recruitment.
For all three studies, we employed convenience and snowball sampling to reach potential participants. We trained clinic partners’ community health workers (CHWs) to share study information and recruit potential participants. Being registered patients with our FQHC partners was not an eligibility criterion. However, to support recruitment efforts, VCC and Innercare sent text messages and/or called patients appearing to meet inclusion criteria to invite them to contact the study staff to enroll. Clinic CHWs recruited widely, and distributed paper flyers at various events (e.g., food distributions, back to school events, health fairs, dad’s club meetings) and shared electronic versions of flyers and short recruitment videos on their social media platforms (primarily Facebook and Instagram). In rural Imperial County, CA, outreach and recruitment efforts relied more on newspaper and radio ads, public service announcements, phone calls, and door-to-door and group presentations. Participants were also asked to share study information by word-of-mouth and recruit other potentially eligible friends or family members.
Study staff independently recruited, hosted booths at community and cultural events, gave presentations, and shared flyers with other community organization leaders. Leaders’ organizations included county oral health coalitions and health departments, hospital/health systems, language and trade schools for adults, grade schools and after-school programs to reach parents, cultural events (e.g., Día de Los Muertos and tamale festivals), and other settings (e.g., YMCA, laundromats, Mexican restaurants and grocery stores, libraries). For most in-person recruitment events, dental kits or single flossers with study contact information were distributed.
Ethical considerations.
The San Diego State University (SDSU) Institutional Review Board (IRB) reviewed and approved each of the three studies (Study 1: HS-2017-0351; Study 2: HS-2019-0117; Study 3: HS-2021-0201). Written informed consent was obtained in English or Spanish before data collection for each study. A copy of the grant application and scope of work for the University of California San Francisco (UCSF) subcontract was also submitted to the UCSF IRB, which reviewed UCSF co-investigators’ roles, and approved their roles on the study annually between 2018–2021, then deemed that annual UCSF IRB renewal was no longer required after 2021 (IRB#18-24606). All the clinics relied on the SDSU IRB. All participants received a copy of the fully executed document, which included a certificate of confidentiality. Paper consent forms were used for Study 1, while consent signatures were obtained electronically in Studies 2 and 3. Only trained study staff consented and enrolled participants. Study 3 procedures were also reviewed and approved by the National Institute of Dental and Craniofacial Research (NIDCR) and a NIDCR-appointed Clinical Study Oversight Committee (CSOC). A NIDCR-appointed external monitoring contractor reviewed all informed consent forms, regulatory documents, and data from a subset of participants.
Data collection and analysis procedures
Study 1: Qualitative interviews
Investigators conducted a social support and oral health literature review to inform writing a semi-structured interview guide in English to be administered with the target population of Mexican-origin young adults. The literature review was conducted by systematically searching in the following databases: PubMed, CINAHL, Web of Science, and PsycInfo. Relevant references cited in the studies were also reviewed. A recent existing review of social support measures by Lopez and Cooper [30] and a social support theory and measurement text by Lakey and Cohen [29] were also closely reviewed at this phase. The investigators determined that the interviews should solicit information about oral hygiene (all methods of cleaning teeth, mouth, and gums) and all experiences related to seeking all types of dental care (preventive, restorative, and emergency services). The interview guide was translated into Spanish independently by three professional translators, each with appropriate experience and understanding of Mexican Spanish and who were briefed on the study topic and purpose. The three translations were independently reviewed, compared, discussed with the translators, and adjudicated over multiple iterations to ensure conceptual and linguistic equivalence in meaning and tone. The guide was then finalized by one experienced bilingual/bicultural co-investigator, following evidence-based methods to maximizing equivalency across scales [61]. This step was critical, as these translations affected how the primary constructs of interest for the scale were phrased in the interview guides (see S1 Table).
Semi-structured interviews and interviewer-administered brief demographic and health behavior surveys were conducted with 72 participants in August-November 2018 by trained interviewers. Interviews lasted about 1.5 hours, were audio-recorded, and conducted in-person in research or clinic offices, or a private community space (e.g., library room). Participants received $25 cash and a dental kit. Audio recordings of the interviews were transmitted to one of three professional translators/transcribers via secure file transfer protocol (ftp) for transcription. Interviews were transcribed in their original language (English or Spanish). Two of the three transcribers were familiar with the project and involved in the interview guide translation process. Interviewers reviewed their transcripts for accuracy, clarified any inaudible sections (referring to audio files if necessary), de-identified text, then added fieldnotes and other administrative information to the final, clean transcript file.
Study 1 Data Analysis.
Qualitative coding and analysis was facilitated by Dedoose (Version 8.1.8), a secure cloud-based program which allows for layered co-coding (i.e., the coding of two concepts applied to one excerpt) [62]. Coding was guided by three detailed codebooks that captured all domains of interest and were designed to be overlapping for cross-coding analysis (see Table 1 for codebook overview, and full codebooks in S2 Table): 1) oral health behaviors; 2) sources of support; and 3) types of social support (pre-defined codes from the social support literature). Emergent codes were developed for the first two codebooks after reading a subset of interview data. The codebooks and all codes applied were in English. Six bilingual coders completed about fourteen hours of trainings in qualitative methods and interviewing [63], coding in Dedoose, and study-specific coding procedures. Coders were assigned codebooks, and coded the same passages in order to become certified to code independently, after reaching at least 80% agreement across code applications. All coders used a tracking sheet to raise questions about code applications. These were reviewed at least weekly by the project manager (PM) and principal investigator (PI), then discussed at coder meetings, and the code definitions and examples in the codebooks were clarified as needed to ensure all coders coded consistently. All coding and quality checks were done on a rolling basis until completed in June 2019; over 20,000 excerpts were coded. Each coder completed a second review to double check all their final code applications. Then one independent trained coder performed quality checks and reviewed all code applications on all transcripts. There were very few instances of any missing codes or disagreements about codes applied at this final checking stage. When there was a missing or disputed code, that excerpt was reviewed by the PM, discussed with the PI, and the final decision made was documented. It was determined that no additional or new codes were needed, and saturation had been reached.
Code occurrences in text excerpts were tabulated and summarized in reports generated by Dedoose as part of an initial content analysis to understand the qualitative data excerpts and begin to look for meaningful themes and patterns. Next, co-code occurrence reports were generated in Dedoose to examine excerpts with multiple layered codes of interest applied. These reports illustrated layered inter-relationships of interest, showing narratives about different types of social support (Codebook 3) provided by different sources (Codebook 2) for each target oral health behavior (Codebook 1). These co-code reports aided our thematic analysis. We sorted excerpts by Codebook, to identify potential patterns across the layered co-coded excerpt for each behavior. A subgroup of the expert panel (PI, PM, and co-investigators with qualitative expertise) conducted thematic analyses, and led discussions with the full expert panel to guide decisions about the new scale structure and content.
The investigators were interested in all themes, including less frequent themes, to understand the full range of types and sources of social support for the target oral health behaviors. Themes were summarized and informed discussions and decisions about the scale’s intended content and purpose. Critical guiding themes included the time point (current versus past) and people; the panel decided to focus the analysis and scale creation on current patterns of social support provided from people, as opposed to support from other sources. Among people, we separated them into groups of family members, health professionals, and others. For oral hygiene, the panel decided to focus on brushing and flossing behaviors only. We inquired about rinsing and other teeth/mouth cleaning behaviors, but given the lack of ADA guidelines about rinsing, these were not retained as priority target behaviors to include in the new scale. There were three themes identified related to dental utilization: access to care, routine dental care, and major dental treatment. The co-coded reports were sorted by these themes and text excerpts from each group of reports were analyzed and used to construct potential items for the item bank.
Scale structure.
The initial structure and response options for the OHBSS scales are presented in Table 2. This structure came directly from the qualitative interviews, because participants talked about different people being sources of support; thus, it became clear that we needed to ask about sources of support for each oral health behavior separately. Instructions and definitions of target behaviors were written by the investigators, in English and Spanish. Participants are asked to rate each social support item three times, once for each source of support (family, health providers, others/friends). A frequency response scale asked how often support was received from each source (1 = Never to 5 = Always, or 6 = not applicable). A non-oral health example was provided to show that each item would need to be rated three separate times.
Item generation.
An expert panel comprised of all nine investigators, two bilingual study staff, and two bilingual clinic partner staff (thirteen total) met repeatedly in 2019 to generate items. English and Spanish items were generated simultaneously by the panel to create the new scales’ item bank, using interview quotes/excerpts as the source material for item wording. Items were originally drafted in either English or Spanish, then cross-language versions were immediately considered, rather than creating a scale in one language and then translating to the other upon completion. During the panel’s bi-directional item generation group process, items were also checked for equivalence in both languages. The panel purposefully included repetitive items with alternate wording options. The panel reviewed items to ensure wording for each was clear, concise, and did not have a double negative and was not double-barreled. Per the dual-focus approach, panel meetings were conducted bilingually; additionally, the panel held three monolingual Spanish-speaking sessions with clinic CHWs who preferred and/or only spoke Spanish to review the Spanish item bank.
Study 2: Cognitive interviews, expert panel review, and pilot testing
Cognitive interviews are a useful step in developing new scales [64,65], and typically a minimum of ten interviews, or 30 interviews conducted iteratively, are recommended to identify problematic items [66,67]. During the cognitive interview, trained bilingual staff asked a series of probes to prompt respondents to “think aloud” and state in their own words what question items were asking, aiming to have them verbalize their thought process in why they selected a particular response option, and explain what the response meant. This process checked for understanding of each item as intended, assessed acceptability and face validity with the target population, and informed item reduction and revisions. Interviewers documented feedback about problematic items (specifically, needing to repeat a question or possible response options, or when terms were not understood or needed alternate explanation). Overall, we captured diverse opinions in each language (English/Spanish), from both sexes (male/female), and in different sites (urban/rural) through 39 interviews and covered the entire OHBSSv1 item bank. Participants received $20. Interviews were conducted in-person from December 2019 to March 2020; after the COVID-19 pandemic started, interviews shifted to virtual webconferencing platforms.
The expert panel expanded beyond the core group of thirteen at this stage, to include more input from bilingual study staff, clinic staff, and community members in the review of all cognitive interview feedback to inform item reduction and refinement. The panel referred to a list of criteria for evaluating items and documented reasons for the decision to drop or retain items. Literacy scores were considered in the decisions for item retention. Flesch-Kincaid [68] scores were tabulated for English items, and comparable Fernandez-Huerta [69] scores were tabulated for Spanish items to assess approximate reading ease/grade level of each item. Additionally, the panel balanced comprehensive construct coverage (all four types of social support assessed for each of the three oral health behavioral targets in both languages), with efficiency (reducing the number of domains to most essential, and combining similar domains where possible), considerations for application (ensuring that domains are measured that will be needed for future use), and respondent burden (the scale must be user-friendly and not too burdensome in terms of difficulty or time to complete). The panel also ensured that parallel items in the brushing and flossing sections were retained. The refined item bank had three sections, one for each behavior of interest (one section about brushing, one section about flossing, and one section about dental care utilization with different subsections). The OHBSSv2 item bank for pilot testing (not shown) included 107 items in Spanish and 109 items in English (including alternate English wording for two related hygiene items).
A final round of four virtual cognitive interviews (two in English, two in Spanish) was conducted to time administration of the OHBSSv2 scale, assess respondent fatigue and burden, and provide a final check before launching Study 2.
Pilot testing.
The OHBSSv2 scales had 107 Spanish and 109 English items. Due to the COVID-19 pandemic, data collection was carried out online and the OHBSSv2 survey was programmed into REDCap Academic [70,71]. Pilot survey data were collected January-December 2021. Participants received a $30 Amazon gift card for completing the one-time, one-hour survey (n = 340). The minimum sample size determination for this phase was guided by results of a measure development review indicating subject-to-item ratios ≥ 2 [72].
Study 2 data analysis.
Pilot survey data were cleaned and analyzed for panel review and item reduction. Frequency distributions and patterns of responses were explored. The panel dropped participants who did not complete at least 50% of the OHBSS scale items with valid answers; 31 of the 340 replied majority “not applicable” responses and were dropped. However, prior to finalizing the analytic sample, patterns of not applicable responses were reviewed more closely. For example, several reported not flossing and did not answer or selected not applicable for the flossing section but answered the rest. To maximize available data, participants were retained for analyses if they completed selected sections of the OHBSS scales (i.e., participants who answered brushing but not flossing items were retained for the brushing scale analyses). There were 279 participants who answered all OHBSS sections; 298 completed Brushing, 302 completed Flossing, and 287 completed Dental Care sections.
We used a series of exploratory factor analysis (EFA) and maximum likelihood extraction methods with orthogonal varimax rotation to identify scale factors [73,74] and item factor structure. Items with factor loadings ≥ 0.4 were retained for review by the expert panel [73,75–77]. Orthogonal rotation is a recommended approach that facilitates results interpretation [78,79]. Analyses were run separately for each behavior (brushing, flossing, and dental utilization), overall and by language, with alternate sets of items to ascertain if factors loaded more clearly with certain items dropped. Many sets of iterative EFA results (not shown) were reviewed closely by the panel in tandem with item descriptive frequencies, especially for similar items with alternate wording, to identify the best items to keep. From the distributions of descriptive frequencies, it was clear that social support varied by source of support group (family, health providers, others/friends). EFAs were run for each behavior and source of support group combination. The panel examined the English- and Spanish-language EFA results to check for similar patterns in factor loadings to see if items loaded in the same way in both language groups. The panel also considered which items were essential for the new scales in terms of importance for capturing changes in social support in a future intervention and were most relevant for the target behaviors, as well as if the set of items covered all desired types of social support.
We also used Horn’s parallel analysis [80] to confirm the estimated factor structure, overall, and by language. Horn’s parallel analysis results also guided decisions about factor retention by comparing observed eigenvalues to those from a computer-generated set of random variables with the same number of scale items and subjects [77,81–83]. We retained eigenvalues that exceeded the 95th percentile in the parallel analysis [82,83]. Analyses were conducted in SAS (v9.4, SAS Institute, Cary, NC).
Scale refinement and item reduction.
The expert panel met regularly from December 2021 to March 2022 to review results from the iterative EFAs and Horn’s parallel analyses, identify factor structure, and make decisions to further refine the scale and reduce items to create the OHBSSv3. Items were dropped if the item content was redundant, and/or the item had problematic or less preferred phrasing, and/or did not load well (≥0.4). The panel also considered the importance of an item for use in a future behavioral intervention (e.g., will it capture potential changes in social support for oral health behaviors) in deciding to retain or drop items. Overall scale length, respondent burden, and literacy levels of items were considered.
After scale refinement and item reduction, cognitive interviews with the shortened OHBSSv3 scales were administered and timed with four participants from the community (two in English, two in Spanish) as a final check for clarity before launching the Study 3 survey.
Study 3: Finalizing scales
Data collection.
The online Study 3 survey with the 39 required plus nine optional items in the OHBSSv3 scales was conducted between April 2022 and February 2023, following the same recruitment and online survey administration procedures as Study 2. Participants received a $30 Amazon gift card after completing the survey (N = 540). In this phase, we again referred to our prior sample size determination criteria [72], but increased it significantly higher than our Study 2 goals to ensure a much larger final analytic sample size for scale validation purposes.
Study 3 data analysis.
Using SPSS (v29, IBM Corp., Armonk, NY), descriptive statistics were calculated for the OHBSSv3 scales. Descriptive statistics included means, standard deviations, modes, medians, skewness, and kurtosis, or frequencies and percentages, as appropriate. Data were tabulated to examine the distribution of item responses and extent and mechanisms of missingness. Of 540 participants, 502 provided complete responses to the OHBSSv3 scales. Two participants had a few missing OHBSSv3 scale responses and regression imputation was used to maximize available data for analysis.
Using MPLUS version 8, a series of confirmatory factor analysis (CFAs) were conducted to evaluate the structural validity of the nine OHBSSv3 scales (required items only) by behavior-source of social support group combination, overall, and by language group (n = 199 Spanish-speakers, n = 303 English-speakers). While there is no consensus on CFA sample size recommendations, CFAs can adequately be undertaken with a minimum sample of 200 [84]. The first CFA examined the fit of a three-factor structure that represented toothbrushing social support, with the subdomains of social support from family (BF), health providers (BP), and others/friends (BO). The second CFA examined the fit of a three-factor structure that represented flossing social support, with the subdomains of social support from family (FF), health providers (FP), and others/friends (FO). The third CFA examined the fit of a three-factor factor structure that represented dental care social support, with the subdomains of social support from family (DF), health providers (DP), and others/friends (DO).
The maximum likelihood robust estimator was used as the CFA estimation method. Several recommended goodness-of-fit indicators were applied to assess model fit [85]. Statistical model fit was evaluated through the Satorra-Bentler (S-B
) for multivariate non-normal data [86]. S-B
indicates adequate statistical fit if the estimate was non-significant at alpha 0.05. Descriptive fit was also evaluated, considering that S-B
is sensitive to sample size and can falsely reject an adequate model. The comparative fit index (CFI) [87], the root mean square error of approximation (RMSEA) [88]. and the standardized root mean residual (SRMR) [89] were also employed to evaluate descriptive fit. CFI ≥ 0.90 indicates acceptable model fit and CFI ≥ 0.95 indicates good model fit. RMSEA and SRMR values ≤ 0.08 indicate acceptable model fit, with ≤ 0.05 indicating good model fit. Acceptable model fit was indicated if at least two of the three descriptive fit estimates (CFI, RMSEA, SRMR) indicated adequate descriptive fit [89]. Acceptable statistical and/or descriptive fit supported adequate structural validity for the corresponding set of OHBSS scales.
An exploratory phase was planned if a CFA resulted in unacceptable model fit for any of the OHBSS scales. The first step was to inspect the CFA modification fit indices, which suggest changes to the model that could substantially improve model fit. The suggested changes would be applied only if they were theoretically reasonable, practical, and that could help achieve acceptable model fit. However, if modification indices suggested complex, unexpected changes, the planned follow-up step was to conduct EFA to examine the content of suggested factors and loading of items. It was expected that EFA findings would result in changes (e.g., trimming of items) to the model that would improve CFA model fit. The goal was to present final sets of OHBSS scales with adequate structural validity as measured by adequate statistical and/or descriptive model fit.
Using R Studio (v4.0.3 (2020-10-10), Boston, MA), Cronbach’s alphas and McDonald’s omegas were calculated to examine the internal consistency of the OHBSS scales. Bootstrapped 95% confidence intervals (CIs) were also calculated for each estimate of internal consistency. Cronbach’s alpha is the dominant measure of internal consistency, but this estimate has assumptions that are often unlikely to be met [90]. The estimate can also be inflated due to a large sample size and large number of items. McDonald’s omegas were calculated because they are more robust to deviation from assumptions, and can provide a more accurate estimate of internal consistency.
Results
Participant demographics for each study’s independent sample are in Table 3, and summarize distribution across age groups (21–30, 31–40), self-identified biological sex (female, male), marital status (married, single), language preference (Spanish, English), and site (CA or AZ county). Desired balance across the three key design characteristics (language, sex, marital status) was achieved in Studies 1 and 2. Study 3 was not as evenly balanced by sex, and skewed to female. Language stratification was prioritized over the other two study design characteristics (sex, marital status) in Study 3 to meet sample size goals on the study timeline. While there were more English-speakers than Spanish-speakers in Study 3, each language group was large enough to support stratified analyses by language, a critically important feature for scale development and validation in both languages.
Study 1 results
The qualitative phase yielded the working first draft item bank for the scales (OHBSSv1), with a total of 274 English items and 286 Spanish items (see Table 4), to be pre-tested for refinement and item reduction with input from community members prior to broader pilot testing (full item bank not shown). Items were generated in five categories: dental care access, routine dental care (including preventive care and simple procedures), major dental treatment, brushing, and flossing.
OHBSSv1 appeared to have face and content validity. The OHBSSv1 scale structure and content was similar to other validated general social support scales found in the literature and used in health research. The initial OHBSSv1 queried social support from three different potential sources of social support (family, health providers, and others/friends). The OHBSSv1 item bank also included items that appeared to assess the intended social support dimensions of interest (i.e., informational, instrumental, emotional, and appraisal types of social support). Table 5 summarizes the OHBSSv1 item bank content by type of social support.
Study 2 results
Feedback from the cognitive interviews assisted in identifying the best items to retain. The final round of cognitive interviews prior to Study 2 pilot testing suggested face and content validity of the OHBSSv2 scales. The OHBSSv2 scales in the pilot test survey included 107 Spanish items and 109 English items, and had three sections to cover the three target behaviors (Table 6) and four types of social support (Table 7).
Factor structure identification.
Patterns of EFA results were highly suggestive that brushing, flossing, and dental care social support each had a three-factor structure. Initial EFA results ranged from three to seven factors in the Full Sample, though all items primarily loaded on the first three factors, following a clear pattern for groups by three sources of support for family, health providers and others/friends (see S3 Table: Study 2 EFA item loadings). Brushing and dental care behavior results yielded more than three factors. When additional factors were identified, they included very few items that had cross-loaded, but with lower loadings on the fourth-seventh factors. Patterns suggested a three-factor structure, and generally persisted in each language group as well. Generally, consistent patterns emerged across the three behaviors, with three factors for each of the different source of support groups. There did not appear to be separate factors to delineate different types of social support. The panel interpreted these patterns to be supportive of a three-factor structure. Parallel analysis results also showed that items loaded together into a three-factor structure for brushing, flossing and dental care social support, with a factor for each of the three source of support groups. Additional EFAs were run for each behavior with restrictions to three factors, and all items loaded cleanly and grouped as expected across the three source of support groups. The ranges of item loadings for each behavior, overall and by language, are summarized in Table 8. The panel interpreted the three-component results as supportive of brushing, flossing, and dental care social support each having a three-factor structure.
From the EFA and Horn’s parallel analysis results, there was support for distinct factors for each source of support group, leading to structuring separate OHBSS scales separately, for a total of nine scales for each behavior-source of support combination.
Scale refinement.
The panel made one substantial change to the scales’ structure after reviewing the frequency distribution of item responses. There were certain types of barriers related to accessing dental care services that were common for a large group of respondents, but not applicable to everyone. The panel conducted additional sensitivity analyses (not shown) to understand the differences between “never” and “not applicable” responses. Patterns of responses were similar, and the panel decided to collapse the two response categories. The panel recognized “never” is different than “not applicable,” but agreed to drop the “not applicable” option, and updated the instructions to include “not applicable” with the “never” category. Thus, the response options for the OHBSSv3 scales were: 0 = Never (or “not applicable”), 1 = Rarely, 2 = Sometimes, 3 = Often, 4 = Always.
Eleven parallel brushing and flossing items were retained. One general hygiene item was kept, “They remind me to get more dental supplies (for example, toothbrush, toothpaste, floss, etc.)”/ “Me recuerdan de conseguir más productos dentales (por ejemplo, cepillo, pasta, hilo dental, etc.)”. This general dental supply item is only asked once, but is scored with both toothbrushing and flossing, thus, there are 12 items scored for toothbrushing and 12 items scored for flossing. This general hygiene item is only counted once in the item count.
Scales for each behavior are denoted as follows: Brushing (B; 12 unique items), Flossing (F; 11 unique items, plus one general dental supplies item gets scored with both brushing and flossing) and Dental care (D; 16 unique items). There are a total of 39 required items to compute scale scores for each behavior. From among the 39 items, one item assessed support for dental hygiene supplies and was included in the subscale score for both Brushing and Flossing. Each behavioral scale can assess social support from three different sources of support: Family (F), Health Providers (P), and Others/Friends (O). Thus, there are nine OHBSSv3 scales, named with the combination of the oral health behavior and source of support (e.g., “Brushing Social Support from Family,” abbreviated “BF”). All the scales included items for informational, instrumental, emotional and appraisal types of social support, although overall, they skewed informational.
When the panel reviewed the dental care items most frequently rated as “not applicable,” they decided to move nine items into a supplemental, optional set of questions. These items queried relevant topics related to dental care and captured aspects of social support that the panel wanted to assess, but these did not fit into the required scales as constructed. These nine dental care items were deemed important, but not generally applicable to everyone. Short screening questions were added by the panel to identify if the individual needed help for one of four optional topics: 1) language interpretation services (one item), 2) transportation (one item), 3) payment (three items), or 4) relieving dental fears/worries (four items). If applicable, respondents can complete up to nine additional items related to seeking dental care. See S4 Table for a copy of the OHBSSv3 scales, with 39 required items plus up to nine optional items, in English and Spanish. Table 9 summarizes the number of items in the OHBSSv3 scales by behavior, and Table 10 displays the breakdown of the number of items in the OHBSSv3 scales by type of social support.
The OHBSSv3 scales took 17-22 minutes to complete, and all participants indicated the instructions, questions and response options were clear. Feedback from the cognitive interviews suggested the OHBSSv3 scales were clear and understandable, and had acceptable face and content validity.
Study 3 results
Structural validity.
Overall sample.
The OHBSSv3 scales had 39 required items. Three CFAs evaluated the structural validity of the OHBSSv3 scales in the total sample for Study 3. For toothbrushing, flossing, and dental care social support, descriptive fit supported a three-factor structure (see Table 10). Toothbrushing social support had the subdomains of social support from family (BF), health providers (BP), and others/friends (BO) (Fig 2). Flossing social support had the subdomains of social support from family (FF), health providers (FP), and others/friends (FO) (Fig 3). Dental care social support had the subdomains of social support from family (DF), health providers (DP), and others/friends (DO) (Fig 4). Statistical fit was poor in all CFAs. These findings support adequate structural validity for each set of three scales among Mexican-origin adults, even without adequate statistical fit.
By language group.
Follow-up CFAs evaluated the structural validity of the OHBSSv3 scales by Spanish-speaking and English-speaking Mexican-origin adults. Model fit indices are summarized in Table 11. Only the set of flossing social support scales (FF, FP, FO) had adequate structural validity in both language groups. The three-factor structure for flossing social support had adequate descriptive fit but poor statistical fit in both language groups. The flossing social support scales were not revised. Figs 5 and 6 show the flossing social support scales’ factor structures and item correlations, by language.
Revisions were made to the set of brushing scales because of inadequate structural validity across language groups. The three-factor structure for brushing social support initially had acceptable descriptive fit among English-speakers, but poor descriptive fit among Spanish-speakers. The modification indices suggested that the following items should be allowed to correlate to achieve acceptable descriptive fit in both language groups: Brush-Family (BF) items 6 with 7 in the BF scale; and Brush-Others (BO) items 2 with 3, and BO5 with BO6 in the BO scale. These were considered practical and substantive changes to the model; however, these changes did not result in changes to the item structure of the scales, and items from the same subscales should correlate. After these changes were made to the theoretical model, the BF, BP, and BO brushing social support scales had adequate descriptive fit in both language groups. Figs 7 and 8 show the brushing scales’ factor structures and item correlations, by language. The new findings support the structural validity of this set of brushing scales but with a slight change in the latent structure.
Revisions were made to the set of dental care scales because of inadequate structural validity across language groups. The three-factor structure for dental care social support had poor statistical and descriptive fit in both Spanish- and English-speaking Mexican-origin adults. The modification indices suggested complex, unexpected changes to the model to improve model fit (items should be predicted by a different factor or items from different scales should be allowed to correlate). Thus, a follow-up EFA was conducted to examine the content of suggested factors and loading of items.
Findings still supported the three-factor structure of dental care social support, with the subdomains of social support from family, health providers, and others/friends (DF, DP, DO, respectively). However, after closer inspection of factor loadings and item content, two items may have been problematic in the models. First, one item (“They helped me get care”/ “Me ayudan a conseguir cuidado dental”) was similar to and simpler than another item (“They helped me manage challenges or obstacles to getting dental care”/ “Me ayudan a solucionar problemas para poder tener cuidado dental”). The latter item was dropped. The dropped item also had a poor literacy score and was more difficult to understand. Another dental care item (“They talk me through my dental treatment options” / “Me hablan de las opciones que tengo para mi tratamiento dental”) was reviewed and deemed to have idiomatic phrasing in English; it also assumed a need for treatment with different potential options to be discussed that may not be applicable to everyone. This item was dropped. Model fit was improved after these two problematic items were dropped.
A new CFA was conducted to examine the fit of a three-factor structure for dental care for social support without the two problematic items. This model still had poor statistical and descriptive fit in both language groups. However, the modification indices suggested that the following items should be allowed to correlate in the model to achieve acceptable descriptive fit in both language groups: Dental-Family items 1 with 2, and DF4 with DF5 in the DF scale; Dental-Others items 1 with 2, and DO4 with DO5 in the DO scale. These were considered practical and substantive changes to the model. They would not result in new changes to the structure of the scales, and items from the same subscale should correlate. After these changes were made to the theoretical model, the updated model had acceptable descriptive fit in both language groups for the Dental Care social support scales. Figs 9 and 10 show the dental care scales’ factor structures and item correlations, by language. The findings support the structural validity of the revised set of dental care scales.
These changes resulted in trimming two required dental care items from the final OHBSS scales. The updated models had adequate fit. The results suggested that these final versions of the OHBSS scales for brushing social support, flossing social support, and dental care social support have adequate structural validity, in the full sample, and by language.
Final expert panel review and item reductions
The expert panel conducted a final close review of the items and the response distributions for the 37 required and nine optional items, following scale development best practices [55]. The panel noticed an inconsistency with items in the optional section, and decided to drop two of the four items in the optional dental worries section. For one item, the English and Spanish item wording did not exactly match, and it had a poor literacy score, so this item was dropped: “They listen to my fears about going to the dentist./ Escuchan mis preocupaciones sobre ir al dentista.” The panel missed this wording inconsistency before testing OHBSSv3. The expert panel decided to also drop the other optional item with the “dental fear” phrasing (“They ease my fears about dental treatments./ Calman mis miedos sobre los tratamientos dentales.”) and only retain the two items with the “dental worries” phrasing. The distribution of responses was similar across the four dental worries items, so the panel felt dropping these two optional items was worthwhile to create a more parsimonious set of final scales.
Final OHBSS scales.
The final OHBSS scales include 37 required items and up to seven optional items, with each unique item asked three times, once for each of three different sources of social support. The scales collectively cover three behaviors, three sources of social support, in two languages. See Tables in S5 for the final OHBSS scales.
Tables in S6 lists all required and optional items in the final OHBSS scales in English and Spanish side-by-side, along with information about type of social support and literacy level (Flesch-Kincaid scores, Fernandez-Huerta scores, and reading ease grade level ratings), and variable names. Our goal was to keep the final scales below 9th grade reading levels, and ideally lower, closer to 5th or 6th grade levels (which approximates completing elementary school). Flesch-Kincaid scores for the English brushing and flossing scale items ranged from grade 0 (pre-Kindergarten) to 7.5, with most items written at or below a 5th grade level. Fernandez-Huerta scores for the Spanish brushing and flossing scale items ranged from 0 to 6.0, with one item scoring at a more difficult level. Flesch-Kincaid scores for the English dental care items ranged from 0.5 to 7.3 among required items and from 4.1 to 9.6 among optional items. Fernandez-Huerta scores for most Spanish dental care items ranged from 4.0- to 8.0 among required and optional items, with one required item and three optional items scoring at a more difficult level. Scale instructions were at a 7th grade level in both languages.
See Table 12 for a summary of the number of items in the final OHBSS scales by behavior, and Table 13 for the final OHBSS scales by types of social support.
The response distributions of each required and optional item in the final OHBSS scales are presented in S7 Tables, summarizing the descriptive mean, standard deviation, median, mode, skewness and kurtosis. The optional items applied to some Study 3 participants as follows: Help was needed for translation services when seeking dental care by 112 of the 502 participants (22%); transportation help was needed by 69 of the 502 participants (14%); almost half (242 of the 502 participants) identified needing help paying for dental care; and dental worries were reported by 172 of the 502 participants (34%).
The final OHBSS Brushing, Flossing, and Dental Care social support scale scores by source of support are summarized in Table 14. For each behavior-source of support combination, the scale is scored by averaging the required items in that section. Behavioral social support scale scores varied by source of support, with health providers having the highest scores on average, reflecting that they provided more perceived social support for all behaviors. On average, family social support scores were slightly lower than the health provider group for all behaviors. Others/friends had very low average scores, indicating this group rarely provided social support for any oral health behaviors. Social support scores were similar for brushing and dental care, and usually slightly higher than for flossing. Overall, Spanish-speakers had slightly higher average OHBSS scores than English speakers. Significant differences by language were found for the BF, FF, DF and BP and DP social support scales.
Internal consistency.
The final OHBSS scales exhibited adequate internal consistency, reflected by robust Cronbach’s alphas and McDonald’s omegas (which were near identical), in both languages all ≥ 0.953 (lower CI bound ≥ 0.944); English all ≥ 0.954 (lower CI bound ≥ 0.944), and Spanish all ≥ 0.946 (lower CI bound ≥ 0.926) (Table 15). These scores provide evidence in support of the final scales’ reliability, in the full sample, and for both language groups.
Discussion
The dual-focus co-creation approach (i.e., creating two language versions simultaneously) was successful, and the final OHBSS scales in English and Spanish are comprehensive in terms of coverage of key structural and functional dimensions of social support for three important oral health behaviors. The new scales also assess social support from different sources, which has implications for use in future behavioral intervention research and targeting different potential sources of social support. Our approach utilized independent samples for each study, and ensured scale development followed evidence-based guidelines for achieving conceptual, content, and normative equivalence in both languages. Co-creating new scales in two languages simultaneously for behavioral research is relatively unique. The dual-focus approach is feasible, with proper planning and support, and promotes equity. Items were generated bi-directionally, sometimes in Spanish first, sometimes in English first, with items written from the source interviews in Study 1. Beginning with an exploratory qualitative phase and literature review we created a large item bank in both English and Spanish, which is an ideal starting point for scale development.
Some types of social support were easier to capture than others and may be most relevant for our target dental behaviors; this can be observed in the greater number of items on informational and instrumental types of social support, and fewer emotional or appraisal support items. It is possible the latter types of support (e.g., emotional) may not be as relevant for the widely applicable behaviors of toothbrushing, flossing, and dental care, unless individuals have strong dental phobias. The fact that there were differences in OHBSS scale scores across sources of social support was expected, suggesting potential avenues for intervention and the scales’ sensitivity to detect changes in social support from these different sources. A potential application is the ability to compare differences in sources of support and identify where more support may be offered. While the “Others/Friends” group was often an infrequent source of social support, CHWs are part of this group, and they may be trained to promote oral health in various aspects. CHWs have provided oral health education and supported changing behaviors and supporting access to dental services [91].
The optional items also appear to be valuable screening tools and identified additional needs that were applicable to many, but not all. The screening questions in the optional section are brief and have important implications for utility in clinic settings, especially among FQHCs that serve many lower-income populations who struggle with these barriers to care.
The longest existing social support scales have 40 items, according to a 2011 review by Lopez and Cooper [30]. Thus, the 37-required unique item count (and 44-items, if all optional items are included) in the OHBSS scales are not excessively long. The OHBSS scales offer flexibility in assessing social support from three different sources. Structuring the scale in a way to allow rating one item three times (once for each source of support) is efficient and minimizes the respondent burden. Shorter social support scales are less likely to capture multiple dimensions [31], with a potential tradeoff between length and utility.
The final OHBSS scales exhibited structural validity and internal consistency overall, and in both English and Spanish. Very minor revisions were required when exploring model fit indices. There were more English-speakers than Spanish-speakers in Study 3, though there did not appear to be substantial differences by preferred language. Although there were more English-speakers, it is notable that many participants were bilingual, and had the choice of which language they preferred to complete the survey. Future analyses will explore other psychometric properties of the final scales, overall and in each language.
While we focused on one priority population of 21– to 40-year-old Mexican-origin men and women to develop the scales, future work can expand upon this demographic, and test the scales further for generalizability to other Hispanic/Latino heritage groups. There may also be differences among Hispanic/Latino heritage groups living further from the US-MX border or in other geographic regions outside of the Southwest. Our priority population for scale development primarily captured one cohort, millennials (also called “Gen Y”), born between 1981-1996 [92]. In addition to context of the recent global COVID-19 pandemic, there may be other important or unique age-period-cohort effects experienced by our study samples that could have affected our results and final scales. Future multivariable modeling and analyses can account for other sample characteristics.
Strengths and Limitations
Significant study strengths include our rigorous mixed methods and bilingual co-creation approach. We collected rich life course perspectives in our qualitative Study 1, which had a large sample size for qualitative inquiry; these data will be explored in more depth in future analyses. Our dual-focus approach and detailed documentation of our translation and bilingual item bank co-creation steps are a major study strength that can serve as a template for other scale development efforts. Adjudicating the interview guide was a time-intensive step in the formative phase, but was critical and shaped how we asked about the constructs of interest, which informed later phases of scale development. OHBSS scales were co-created in two languages simultaneously, which is not frequently done or described in scale development, and is the first to our knowledge to adopt this approach for an oral health behavior scale. This parallel processing approach appeared superior to other translation methodologies like back-translation or serial translation, and kept one language from dominating the other. While it was challenging to coordinate a large multi-site study team and expert panel for all study phases, we succeeded in bringing together the necessary expertise, including input from community members and clinic partners throughout. We built on a long history of strong collaboration with community partners, which was critical for successful outreach to our priority demographic group. Trust, flexibility, and regular updates and communication (standing meetings with each clinic partner at least once/month) enabled our teams to shift and adapt as needed during the pandemic to still meet study goals.
We achieved successfully balanced samples by languages across all three studies; samples were well-balanced by marital status, and to a lesser degree by sex. Recruiting and enrolling men was very challenging in all three studies, even with spouses/partners and other family members living in the same household being eligible. This is not uncommon in previous research. A study sample of at least 30%-50% men was needed to power meaningful sex-stratified statistical analysis. The rationale for striving for balance across these characteristics was due to documented differences in the sources and types of social support for men and women [31,93]. Notable sex differences have been observed in racially/ethnically diverse adults, with women engaging in more frequent oral hygiene behaviors than men and visiting the dentist more often [94–96].
Data collection for Studies 2 and 3 occurred during the global COVID-19 pandemic, which necessitated shifting to remote outreach, recruitment, and data collection. Conducting online surveys may have introduced some biases in our sampling in favor of respondents who were already more familiar with and had better access to internet/ e-media technology. Study staff offered alternative options for survey completion including having an interviewer administer the entire survey via phone, or mailing a hard copy, but respondents did not request these modalities. Study staff were able to assist participants with the screening and enrollment process via phone call and texts, and in some cases stayed on the phone ensuring participants could access the unique survey links and start the survey or continue where they left off.
Conclusion
This study yielded new, culturally-relevant, oral health behavior-specific social support scales in English and Spanish for oral hygiene behaviors (brushing, flossing) and dental care utilization from family, health providers, and others/friends. The final OHBSS scales exhibited structural validity and internal consistency, overall and by language.
Supporting information
S1 Table. Study 1 interview guides in English and Spanish.
https://doi.org/10.1371/journal.pone.0317133.s001
(PDF)
S7 Table. Final OHBSS item descriptive frequencies.
https://doi.org/10.1371/journal.pone.0317133.s007
(PDF)
References
- 1.
US Department of Health and Human Services, National Institutes of Health, National Institute of Dental and Craniofacial Research. Oral health in America: Advances and challenges. Bethesda, MD. 2021.
- 2. Pew Research Center. [internet] Key facts about U.S. Latinos for National Hispanic Heritage Month. 2023 [cited 22 Jul 2016. ]. Available from: https://www.pewresearch.org/short-reads/2023/09/22/key-facts-about-us-latinos-for-national-hispanic-heritage-month/
- 3.
Rumbaut RG. The making of a people. In: Hispanics and the future of America. Tienda M, Mitchell F, editors. Washington DC: The National Academies Press; 2006. p. 16–65.
- 4.
Pew Research Center. [internet] Mapping the Latino population, by state, county and city. Washington DC. 2013 [cited 22 Jul 2016. ] Available from: http://www.pewhispanic.org/2013/08/29/mapping-the-latino-population-by-state-county-and-city/
- 5. Akinkugbe AA, Raskin SE, Singer RH, Finlayson TL, Youngblood M, Laniado N, et al. Perceived dental care need and actual oral health status in the Hispanic Community Health Study/Study of Latinos (HCHS/SOL). J Public Health Dent. 2020;80(4):259–70. pmid:32662151
- 6. Sanders AE, Campbell SM, Mauriello SM, Beck JD, Jimenez MC, Kaste LM, et al. Heterogeneity in periodontitis prevalence in the Hispanic community health study/study of Latinos. Ann Epidemiol. 2014;24(6):455–62. pmid:24731697
- 7. Spolsky VW, Marcus M, Der-Martirosian C, Coulter ID, Maida CA. Oral health status and the epidemiologic paradox within Latino immigrant groups. BMC Oral Health. 2012;12:39. pmid:22958726
- 8. Eke PI, Dye BA, Wei L, Thornton-Evans GO, Genco RJ. Prevalence of periodontitis in adults in the United States: 2009 and 2010. J Dent Res. 2012;91(10):914–20. pmid:22935673
- 9. Lang WP, Ronis DL, Farghaly MM. Preventive behaviors as correlates of periodontal health status. J Public Health Dent. 1995;55(1):10–7. pmid:7776285
- 10. Lang WP, Farghaly MM, Ronis DL. The relation of preventive dental behaviors to periodontal health status. J Clin Periodontol. 1994;21(3):194–8. pmid:8157773
- 11. Lang NP, Cumming BR, Löe H. Toothbrushing frequency as it relates to plaque development and gingival health. J Periodontol. 1973;44(7):396–405. pmid:4514570
- 12. Tedesco LA, Keffer MA, Davis EL, Christersson LA. Self-efficacy and reasoned action: Predicting oral health status and behaviour at one, three, and six month intervals. Psychol Health. 1993;8(2–3):105–21.
- 13. Tedesco LA, Keffer MA, Davis EL, Christersson LA. Effect of a social cognitive intervention on oral health status, behavior reports, and cognitions. J Periodontol. 1992;63(7):567–75. pmid:1507036
- 14. American Dental Association. Wake up to prevention for the smile of a lifetime. J Am Dent Assoc. 1988;116(5):3G, 6G-–13G. pmid:3164352
- 15. American Dental Association. Home oral care recommendations to reduce the risk of caries and gum disease. 2018. Available from: https://www.ada.org/-/media/project/ada-organization/ada/ada-org/files/resources/research/oral-health-topics/homeoralcare_web_final_july2018.pdf
- 16. Axelsson P, Nyström B, Lindhe J. The long-term effect of a plaque control program on tooth mortality, caries and periodontal disease in adults. Results after 30 years of maintenance. J Clin Periodontol. 2004;31(9):749–57. pmid:15312097
- 17. Suomi JD, Greene JC, Vermillion JR, Doyle J, Change JJ, Leatherwood EC. The effect of controlled oral hygiene procedures on the progression of periodontal disease in adults: results after third and final year. J Periodontol. 1971;42(3):152–60. pmid:29538812
- 18. Lightner LM, O’Leary JT, Drake RB, Crump PP, Allen MF. Preventive periodontic treatment procedures: results over 46 months. J Periodontol. 1971;42(9):555–61. pmid:5285683
- 19. Darby I. Non-surgical management of periodontal disease. Aust Dent J. 2009;54 Suppl 1:S86-95. pmid:19737271
- 20. Aguirre-Zero O, Westerhold C, Goldsworthy R, Maupome G. Identification of barriers and beliefs influencing engagement by adult and teen Mexican-Americans in oral health behaviors. Community Dent Health. 2016;33(1):44–7. pmid:27149773
- 21. Vujicic M, Nasseh K. A decade in dental care utilization among adults and children (2001-2010). Health Serv Res. 2014;49(2):460–80. pmid:24299620
- 22. Pérez-Escamilla R, Garcia J, Song D. Health care access among hispanic immigrants: ¿alguien está escuchando? [is anybody listening?]. NAPA Bull. 2010;34(1):47–67. pmid:21116464
- 23. Knowlden AP, Sharma M. Systematic review of family and home-based interventions targeting paediatric overweight and obesity. Obes Rev. 2012;13(6):499–508. pmid:22221298
- 24. Ramchand R, Ahluwalia SC, Xenakis L, Apaydin E, Raaen L, Grimm G. A systematic review of peer-supported interventions for health promotion and disease prevention. Prev Med. 2017;101:156–70. pmid:28601621
- 25. Jeet G, Thakur JS, Prinja S, Singh M. Community health workers for non-communicable diseases prevention and control in developing countries: Evidence and implications. PLoS One. 2017;12(7):e0180640. pmid:28704405
- 26. Ickes MJ, Sharma M. A systematic review of physical activity interventions in Hispanic adults. J Environ Public Health. 2012;2012:156435. pmid:22496702
- 27. Barrera M. Distinctions between social support concepts, measures, and models. Am J Commun Psychol. 1986;14(4):413–45.
- 28.
Lakey B, Drew JB. A social-cognitive perspective on social support. In: Pierce BL GR, Sarason IG, Sarason BR, editor. Sourcebook of social support and personality. New York: Plenum; 199. p. 107–40.
- 29.
Lakey B, Cohen S. Social support theory and measurement. In: Cohen S, Gottlieb BH, Underwod LG, editor. Social support measurement and intervention: A guide for health and social scientists. New York, NY: Oxford University Press; 2010.
- 30. Lopez M, Cooper L. [internet] Social support measures review-Final Report. National Center for Latino Child and Famliy Research. 2011 [cited 22 Jul 2016. ] Available at: https://www.yumpu.com/en/document/read/15702064/social-support-measures-review-first-5-la/77
- 31.
Wills T, Shinar O. Measuring perceived and received social support. In: Cohen S, Underwood LG, Gottlieb BH, editors. Social support measurement and intervention: A guide for health and social scientists. New York NY: Oxford University Press; 2000. p. 86–135.
- 32.
Berkman L, Breslow L, Wingard D. Social networks and mortality risk1983. p. 112–75.
- 33.
Berkman LF, Glass T. Social integration, social networks, social support and health. In: Kawachi I, editor. Social Epidemiology. Oxford University Press; 2000. p. 137–273.
- 34. Turner RJ, Marino F. Social support and social structure: a descriptive epidemiology. J Health Soc Behav. 1994;35(3):193–212. pmid:7983334
- 35.
House JS. Social support and the quality and quantity of life. Research on the quality of life. USA: Institute of Social Research at the University of Michigan; 1986. p. 253–69.
- 36. Barth J, Schneider S, von Känel R. Lack of social support in the etiology and the prognosis of coronary heart disease: a systematic review and meta-analysis. Psychosom Med. 2010;72(3):229–38. pmid:20223926
- 37. Hanson BS, Isacsson SO, Janzon L, Lindell SE. Social network and social support influence mortality in elderly men. The prospective population study of “Men born in 1914,” Malmö, Sweden. Am J Epidemiol. 1989;130(1):100–11. pmid:2787102
- 38. Ayala GX, Chan JCN, Cherrington AL, Elder J, Fisher EB, Heisler M, et al. Predictors and effects of participation in peer support: a prospective structural equation modeling analysis. Ann Behav Med. 2022;56(9):909–19. pmid:35830356
- 39. Hogan BE, Linden W, Najarian B. Social support interventions: Do they work?. Clin Psychol Rev. 2002;22:381–440.
- 40. Matthews KA, Gallo LC, Taylor SE. Are psychosocial factors mediators of socioeconomic status and health connections? A progress report and blueprint for the future. Ann N Y Acad Sci. 2010;1186:146–73. pmid:20201872
- 41.
Cohen S, Gottlieb B, Underwood LG. Social relationships and health. In: Cohen S, Gottlieb B, Underwood LG, editor. Social support measurement and intervention: A guide for health and social scientists. New York, NY: Oxford University Press; 2010.
- 42. Lakey B, Heller K. Social support from a friend, perceived support, and social problem solving. Am J Comm Psychol. 1988;16(6):811–24. pmid:3223486
- 43. Uchino BN. Social support and health: a review of physiological processes potentially underlying links to disease outcomes. J Behav Med. 2006;29(4):377–87.
- 44. Dahlan R, Ghazal E, Saltaji H, Salami B, Amin M. Impact of social support on oral health among immigrants and ethnic minorities: A systematic review. PLoS One. 2019;14(6):e0218678. pmid:31220165
- 45. Laniado N, Sanders AE, Fazzari MJ, Badner VM, Singer RH, Finlayson TL, et al. Social support and dental caries experience: Findings from the Hispanic Community Health Study/Study of Latinos Sociocultural Ancillary Study. Comm Dent Oral Epidemiol. 2021;49(5):494–502. pmid:33638557
- 46. Laniado N, Cloidt M, Shah P. Social support and oral health among working-age and older adults in the United States. J Public Health Dent. 2023;83(3):247–53. pmid:37303076
- 47. Laniado N, Badner VM, Sanders AE, Singer RH, Finlayson TL, Hua S, et al. Social capital and periodontal disease in Hispanic/Latino adults in the United States: Results from the Hispanic Community Health Study/Study of Latinos. J Clin Periodontol. 2020;47(5):542–51. pmid:31998991
- 48. Dahlan R, Badri P, Saltaji H, Amin M. Impact of acculturation on oral health among immigrants and ethnic minorities: A systematic review. PLoS One. 2019;14(2):e0212891. pmid:30818332
- 49. Brooks CV, Maupomé G. Social support associated with restorative treatment, professionally applied fluoride and flossing: A cross-sectional analysis including recent immigrants from Central America and Mexico in the Midwest USA. Community Dent Oral Epidemiol. 2024;52(2):187–95. pmid:37779340
- 50.
Vaux A. Assessment of social support. In: Vieiel HOF, Baumann U, editors. The meaning and measurement of social support. New York: Hemisphere; 1992. p. 193–216.
- 51. Sallis JF, Grossman RM, Pinski RB, Patterson TL, Nader PR. The development of scales to measure social support for diet and exercise behaviors. Prev Med. 1987;16(6):825–36. pmid:3432232
- 52.
Creswell JW, Klassen AC, Plano Clark VL, Smith KC, with the assistance of a Specially Appointed Working Group. Best practices for mixed methods research in health sciences. Commissioned by the Office of Behavioral and Social Science Research (OBSSR); 2011.
- 53. Erkut S, Alarcón O, García Coll C, Tropp LR, García HAV. The dual-focus approach to creating bilingual measures. J Cross Cult Psychol. 1999;30(2):206–18. pmid:21552397
- 54. Erkut S. Developing Multiple Language Versions of Instruments for Intercultural Research. Child Dev Perspect. 2010;4(1):19–24. pmid:21423824
- 55. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quiñonez HR, Young SL. Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer. Front Public Health. 2018;6:149. pmid:29942800
- 56.
DeVellis RF, Thorpe CT. Scale development: Theory and applications. 5th ed. Thousand Oaks, CA: Sage; 2021. p. 320.
- 57. Eke PI, Borgnakke WS, Genco RJ. Recent epidemiologic trends in periodontitis in the USA. Periodontol 2000. 2020;82(1):257–67. pmid:31850640
- 58.
State Adolescent Health Resources Center [internet]. Developmental tasks and attributes of late adolescence/young adulthood (ages 18-24 years). Konopka Institute, University of Minnesota [cited 22 Jul 2016. ] Available at: https://sahrc.umn.edu/sites/sahrc.umn.edu/files/2021-01/late_adol_stage-eng.pdf
- 59.
National Academy of Sciences, Committee on Improving the Health, Safety and Well-Being of Young Adults; Board on Children, Youth, and Families; Institute of Medicine; National Research Council. In: Bonnie RJ, Stroud C, Breiner H, editors. Investing in the Health and Well-Being of Young Adults. Washington (DC): National Academies Press (US), 2015.
- 60. Loe H. Periodontal changes in pregnancy. J Periodontol (1930). 1965;36:209–17. pmid:14300227
- 61. Tsai T-I, Luck L, Jefferies D, Wilkes L. Challenges in adapting a survey: ensuring cross-cultural equivalence. Nurse Res. 2018;26(1):28–32. pmid:29856168
- 62. SocioCultural Research Consultants MB, CA, USA. Dedoose for Qualitative and Mixed Methods Research. https://www.dedoose.com/
- 63.
Patton MQ. Qualitative evaluation and research methods. 2nd ed. Newbury Park, CA: Sage. 1990.
- 64. Miller K. Conducting cognitive interviews to understand question-response limitations. Am J Health Behav. 2003;27 Suppl 3-S264–72. pmid:14672387
- 65. Knafl K, Deatrick J, Gallo A, Holcombe G, Bakitas M, Dixon J, et al. The analysis and interpretation of cognitive interviews for instrument development. Res Nurs Health. 2007;30(2):224–34. pmid:17380524
- 66. Willis GB, Artino AR Jr. What do our respondents think we’re asking? using cognitive interviewing to improve medical education surveys. J Grad Med Educ. 2013;5(3):353–6. pmid:24404294
- 67.
Willis G. Cognitive interviewing and questionnaire design: A training manual. National Center for Health Statistics Cognitive Methods Staff Working Paper Series. 1994 Mar 7.
- 68. Flesch R. A new readability yardstick. J Appl Psychol. 1948;32(3):221–33.
- 69. Fernández Huerta J. Medidas sencillas de lecturabilidad. Consigna. 1959;214:29–32.
- 70. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81. pmid:18929686
- 71. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O’Neal L, et al. The REDCap consortium: Building an international community of software platform partners. J Biomed Inform. 2019;95103208. pmid:31078660
- 72. Anthoine E, Moret L, Regnault A, Sébille V, Hardouin J-B. Sample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measures. Health Qual Life Outcomes. 2014;12176. pmid:25492701
- 73. Costello A, Osborne J. Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis. Pract Assess Res Eval. 2005;10:1–9.
- 74.
Streiner DL, Norman GR. Health measurement scales: A practical guide to their development and use: A practical guide to their development and use. 4th ed. Oxford: University Press; 2008.
- 75. Guadagnoli E, Velicer WF. Relation of sample size to the stability of component patterns. Psychol Bull. 1988;103(2):265–75. pmid:3363047
- 76. Zwick W, Velicer W. Comparison of five rules for determining the number of components to retain. Psychol Bull. 1986;99(3):432–42.
- 77. Ledesma RD, Valero-Mora P. Determining the number of factors to retain in EFA: an easy-to-use computer program for carrying out Parallel Analysis. Practical Assessment, Research, and Evaluation. 2007;12(2).
- 78.
Gorsuch RL. Factor Analysis. 2nd ed. Psychology Press; 1983.
- 79.
Kim J, Mueller CW. Introduction to factor analysis: what it is and how to do it. Newbury Park: Sage; 1978.
- 80. HORN JL. A rationale and test for the number of factors in factor analysis. Psychometrika. 1965;30:179–85. pmid:14306381
- 81. Velicer WF. Determining the Number of Components from the Matrix of Partial Correlations. Psychometrika. 1976;41(3):321–7.
- 82. Hayton JC, Allen DG, Scarpello V. Factor retention decisions in exploratory factor analysis: a tutorial on parallel analysis. Organizational Research Methods. 2004;7(2):191–205.
- 83.
Franklin S, Gibson D, Robertson P, Pohlmann J, Fralish J. Parallel analysis: A method for determining significant principal components. Feb 1995.
- 84. Kyriazos T. Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology. 2018;9:2207–30.
- 85. Bentler P. On tests and indices for evaluating structural models. Pers Ind Diff. 2007;42(5):825–9.
- 86. Satorra A, Bentler PM. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika. 2001;66(4):507–14.
- 87. Bentler PM. Comparative fit indexes in structural models. Psychol Bull. 1990;107(2):238–46. pmid:2320703
- 88. Steiger JH. Structural model evaluation and modification: an interval estimation approach. Multivariate Behav Res. 1990;25(2):173–80. pmid:26794479
- 89. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struc Equ Model. 1999;6(1):1–55.
- 90. Hayes AF, Coutts JJ. Use omega rather than Cronbach’s alpha for estimating reliability. but…. Comm Methods Measures. 2020;14(1):1–24.
- 91. Garcia DT, Lawson JA, Brody ER, McKernan SC, Raskin SE, Arauz NR, et al. A scoping review of the roles, training, and impact of community health workers in oral health. Comm Dent Health. 2021;38(3):198–208. pmid:34029018
- 92. Pew Research Center [internet]. Defining generations: Where Millennials end and Generation Z begins. 2019 [cited 8 Nov 2023. ] Available from: https://www.pewresearch.org/short-reads/2019/01/17/where-millennials-end-and-generation-z-begins/
- 93. Bird CE, Rieker PP. Gender matters: an integrated model for understanding men’s and women’s health. Soc Sci Med. 1999;48(6):745–55. pmid:10190637
- 94. AlJehani YA. Risk factors of periodontal disease: review of the literature. Int J Dent. 2014;2014:182513. pmid:24963294
- 95. Lipsky MS, Su S, Crespo CJ, Hung M. Men and oral health: a review of sex and gender differences. Am J Mens Health. 2021;15(3):15579883211016361. pmid:33993787
- 96. Su S, Lipsky MS, Licari FW, Hung M. Comparing oral health behaviours of men and women in the United States. J Dent. 2022;122:104157. pmid:35545161