Methods
Study Design
We used a retrospective cohort design.
Study Population
We defined 2 cohorts of females aged 9 to 26 years beginning in March 1, 2007, with at least 1 doctor appointment between March 1, 2007, and January 25, 2010, seen in 2 different community-based family medicine practices.
Study Setting
The exposure, or prompted, cohort comprised patients seen in 5 academic, community-based family medicine practices in the Midwest with a common institution-created EHR and electronic reminder system. The control, or unprompted, cohort comprised patients from another 4 academic, community-based family medicine practices in the Midwest with a common EHR without any electronic prompting or alert system for vaccines. The 2 academic centers do not have overlapping catchment areas. The available characteristics of the clinics from 2007 are summarized in Table 1 by prompted and unprompted cohort. There are a wide range of full-time equivalent faculty physicians, number of patient visits in 2007, and patients per full-time equivalent faculty physicians between clinics within a cohort and between cohorts, as highlighted in Table 1. Among the prompted cohort, 2 of 5 clinics have residents, whereas all clinics within the unprompted cohort have residents. All the clinics have medical students. The research ethics were reviewed by human research committees and approved by an institutional review board.
Outcomes
The primary outcome of interest was the initiation and completion of the HPV vaccine series, including time between each vaccine. The covariates of interest were age, race, and number of visits during the observation period.
Exposure and Intervention
The primary intervention was the exposure of the prompted cohort to HPV vaccine alerts during encounters with their health care providers. At the time of the patient encounter, providers (including physicians, nurse practitioners, physician assistants, and medical assistants) received the HPV vaccine prompt produced by the EHR. These practices had been using such a system since 2000, which included alerts for preventive and chronic care services. Vaccine alerts had been in place for a variety of childhood, adolescent, and adult vaccines. The HPV vaccine alert was started March 2, 2007. The provider alert was a simple list of services needed. For the HPV vaccine, the alert was "HPV" followed by the needed vaccine in the series. As part of this active prompting system, providers were required to respond to the HPV vaccine prompts with 1 of the following options: done, ordered, patient declined, patient not eligible, discussed, or not addressed. If results indicate not addressed or if additional vaccinations are due, the prompt returns at the following visit. Patients and parents received a brief note of "services your provider will recommend for you today." Reminder algorithms were developed using Cielo Clinic (Cielo MedSolutions, Ann Arbor, MI) to prompt providers and patients at all appointments with females who are eligible to initiate or complete HPV vaccination at appropriate intervals.
The unprompted cohort was seen in practices with an EHR without any systematic form of alerts for vaccines or other preventive services.
Analytic Variables
Patient Characteristics. Patient age was based on age at the start of the observation period and was categorized as 9 to 18 years or 19 to 26 years. Patient race was categorized as white, African American, or other. The total number of visits during the observation period was categorized as 1 to 2 visits or at least 3 visits with a clinician including a medical doctor (MD), doctor of osteopathic medicine (DO), nurse practitioner, or physician assistant. Visits made solely for vaccine delivery were not included in the analysis.
HPV Vaccine Initiation and Series Completion. Vaccine initiation was defined as receipt of at least 1 dose of the HPV vaccine during the observation period. Series completion was defined as receipt of all 3 doses of the HPV vaccine during the observation period. Variables were created to indicate opportunities to receive subsequent HPV vaccine doses, based on the lower limits recommended by the ACIP. Patients with at least 30 weeks elapsing after their first HPV vaccine dose were considered to have had the opportunity to complete the vaccine series during the observation period. We defined the time between doses using the date of vaccine from the EHR for each dose.
Statistical Analysis
Descriptive statistics on age, race, and number of vaccines received were calculated and compared between the 2 study cohorts using χ and independent samples t tests and between clinics within cohorts using χ and one-way analysis of variance. Multiple imputation was performed to impute values of race for subjects whose race was unknown. A multinomial logistic regression model, including covariates of age, study cohort, number of visits, and number of HPV vaccines received, was used to estimate race category probabilities for observations with missing race information. These probabilities were used to impute race information for a total of 10 imputed data sets. Analysis including race was performed on each imputed data set and the results were combined using the formula described by Little and Rubin.
Correlates of vaccine initiation were examined using a clustered multivariable logistic regression model predicting receipt of ≥1 HPV vaccine dose versus no vaccination during the observation period. Correlates of vaccine series completion were examined in the subset of study patients who initiated the HPV vaccine series and had the opportunity to complete the series during the observation period, using a clustered multivariable logistic regression model predicting receipt of all 3 HPV vaccine doses versus receipt of any (ie, 1 or 2) doses during the observation period. Both models included the covariates of age, race, number of visits during the observation period, study cohort, and interactions of study cohort with all other covariates. Both models were fit using a generalized estimating equations approach with an exchangeable working correlation structure to account for practice clustering. Results are presented as adjusted odds ratios (ORs) and associated 95% confidence intervals.
Time to subsequent vaccine dose was estimated using the Kaplan-Meier product-limit estimator for time between doses 1 and 2 and doses 1 and 3 (completion) within the subset of subjects who initiated the series and for time between doses 2 and 3 within the subset of subjects who received a second dose during the study period. Differences in survivor functions by study cohort were tested using the log-rank test. Cox proportional hazard models were used to estimate the impact of study cohort on time to vaccination after adjusting for covariates for the same 3 intervals, using the same subsets of subjects as those used for the Kaplan-Meier estimates. Covariates in all 3 models included study cohort, patient age, patient race, and interactions between cohort and both race and age. Standard errors were calculated using a robust variance estimator method to account for practice clustering. Results are presented as hazard ratios with corresponding 95% confidence intervals. All analyses were conducted using SAS version 9.3 (SAS Institute, Inc., Cary, NC).