Validation of Educational Tools for Use in a Human Papillomavirus Intervention Study

Purpose: The aim of this study is to report the process used to validate an educational Human Papillomavirus (HPV) pamphlet and a HPV video to use in a HPV intervention study for women of color to increase HPV knowledge, health beliefs, health behaviors and intention to use the HPV vaccine for themselves or members of the family. Methods: Fifteen women enrolled in a two part, two hour methodological pilot study to validate the appearance, content and readability of the educational pamphlet and video. Quantitative data was determined women completing two 4-pont Likert type questionnaires consisting of 10 items each. The qualitative data was retrieved with the use of a semi-structure interview. Results: Greater than 86% of the participants indicated that both educational tools were acceptable in appearance, content and readability/ understanding. Participants’ responses suggested a change in the pamphlet color and use of more colored illustrations as well as condensing the video narrative with bulleted summary pages after each scene. Conclusion: The tools were well received by the participants. Next steps were identified toward revision. A HPV pamphlet and an HPV video were validated for use in a future intervention study. Future Considerations: More educational pamphlets identifying validation process for use in intervention studies for targeted high risk populations.


Introduction
Human papillomavirus (HPV) is the primary causative factor causing 70% to 90% of all cervical cancer. It is estimated that 12,000 women are diagnosed with cervical cancer annually and in 2014, more than 4,000 women died of invasive cervical cancer. 1,2 Women of color, especially African American women are disproportionately impacted with Human Papillomavirus (HPV), and have a higher morbidity rate of cervical cancer. 3 Lack of knowledge regarding HPV, the need for Pap smears and preventive measurements related to cervical cancer, follow up and adherence to treatment as well as low incidence of receiving the HPV vaccine, and a low incidence for recommending the HPV vaccine to their children are reported as components noted in women of color that attribute to the higher incidence of cervical cancer. 4,5 Death from cervical cancer could be grossly diminished through education, change in health beliefs, health behaviors and access to preventative measures. Papas 6 described the impact of educational intervention among women aged 30 and older who received an educational intervention during the time of their annual gynecologic examination and the High risk (HR)-HPV screening. In this particular study, knowledge concerning HPV, cervical cancer, and cervical cancer screening was significantly improved after the educational intervention. Educational materials are used by healthcare professionals as a gold standard for disseminating health information to consumers of health care. Gender specific and culturally appropriate educational materials that effectively communicates health education messages assist in meeting the specific needs of targeted audiences.

Background and Significance
Educational materials are identified as the single most important component positively impacting the health status of people in the United States. 7 Their use by researchers, educators and health care professionals is equated with an increase in patient health outcomes. 7, 8 However, disparities in the health of people of color, continue to exist. There are limited HPV educational materials, especially pamphlets and videos, used in intervention studies specifically targeted for African American women. Success of educational materials 36 Validation of Educational tools for Use in a Human Papillomavirus Intervention Study is therefore based on the success of the target group to use the data to positively impact their health status. Validity and readability testing prior to the use of the educational materials in research studies becomes a major factor in attributing to their success. Data from the 2003 National Assessment of Adult Literacy and 2012 Program for the International Assessment of Adult Competencies reported only 12% of the adult population show reading proficiency on reading tests. Many of current tools lack clarity, are not gender specific or culturally appropriate, have reading levels higher than 10 th grade. 9, 10 Low literacy levels translate into difficulty understanding educational materials thus contributing to poor health outcomes. 11, 12, 13 Health care organizations mandate that educational materials reading levels be no higher than the an 8 th grade, 13,14,15 while others suggest the 6 th grade level. 16 Therefore, it is important to validate all educational materials so they can be meet the cultural and educational needs of the targeted audience.
In 2014, a HPV pamphlet and HPV video was developed by members of the research team, HPV experts, literature reviews and data received from five focus groups. Information regarding the development of the tools can be located in a previous manuscript. 17 The aim of this study is to report the process used during a pilot study to validate the educational HPV pamphlet and a HPV video. The educational tools will be used in a HPV intervention study for women of color to determine if HPV knowledge, Pap test, follow-up, HPV vaccine data improves knowledge, health belief, health behaviors, and intent to use or having family members receive the HPV vaccination. Validity measurements used in this study include content validity and readability.

Theoretical Framework
The Health Belief Model (HBM consist of six constructs: perceived susceptibility, perceived severity, perceived threats, perceived barriers, cues to action and self-efficacy. The HBM has been used extensively through the literature in health promotion, health prevention and disease prevention in adults. 17,18,19 Figure  1, represents the conceptual-theoretical-empirical model that operationalizes the original concepts of the HBM for this study. The concept "cues to action" is defined as strategies that activate readiness by the patient to change their health behavior.

Content validity
Content validity is defined as essential content that accurately address a certain topic for discussion. 22 Researchers agree that content validity is conducted by experts with experience in the topic for discussion. Furthermore, lay members/ participants also play an important role in the review and critique educational materials for content validity and readability. They assess educational materials for its cultural appropriate, clarity, and understanding. 23, 24, 25, 26 Content validity was measured by relying on the knowledge of participants who are familiar with the content being measure. 27.

Readability
For this study, readability is defined as "characteristics of reading materials that make [educational] material easy to read" and understand by the targeted audiences. 28 Educational tools that are difficult to read have a negative impact on the healthcare needs of persons seeking health information. The literature confirms the gap between educational materials and the patient's ability to read and understand its content. 8, 28, 29, 30 Most Americans read at the 5 th grade or lower level. 31 More than 40 readability formulas were noted in the literature. For this study, the author used the Flesh-Kincaid Grade Level Formula, 32 to print documents for use with the selected targeted audience. Its use in validating educational materials has been addressed by multiple researchers and educators since the early 1990s. 8, 22,33 A 5th grade reading level was selected for the educational materials in this study. This represents one to three levels below recommendation reported in the literature. 34

Methodology
Research design and Sample: This two hour pilot study is a methodological study that identifies the process used to validate two HPV educational tools. There are four components to this study which began in fall 2011. The pilot study was conducted in January 2015 in a large urban Southeastern region of Louisiana to recruit women of color to validate two educational materials for use in the large HPV intervention study. A convenience sample of fifteen women was recruited. Eligibility criteria for the women were as follows: 1) women of color, 2) age 19 years or over, 3) women attending the OB/GYN clinic, 4) women without a history of HPV infection, and 4) women who could who could speak, understand and write in the English language and provide written consent. The exclusion criteria were Caucasian women, incarcerated women and women who could not read, write or understand English. Hertzog recommends a sample size of 10-20% of the intended intervention study. 35. A total of 150 women are intended for the intervention study. Therefore a sample size of 15 was appropriate to validate the tools.
Procedures: Flyers were posted in the Women's Pavilion, one of five hospitals within the Ochsner Medical Center in the Greater New Orleans area. Each participant signed informed consent forms prior to participating in the study. Ochsner Clinical Trials Unit and Clinical Research Coordinators assisted in setting up the meeting site and assisted with the registration process. The study was approved by Dillard University, Louisiana State University Health Sciences Center-New Orleans and Ochsner Foundation Clinic Institutional Review Boards.

Instruments:
After reading and viewing a HPV pamphlet and a HPV video, the women were asked to complete a questionnaire. Guidelines to ensure clarity of printed HPV pamphlet were developed from the work of Davidhizar & Brownson, 36 Doak et al., 22 Kessels, 37 and Weixel. 38 These guidelines included: 1) Use of accurate content, 2) Organization of content in a logical manner, include only need to know information, 3) Keep the content simple and smart (KISS), 4) Limit medical jargon and if used include definitions, 5) Write the materials two to four levels below the targeted audience, 6) Use pictures to increase recall, and 7) Format words for easy reading avoid using unattractive crowded pages. Based on these guidelines, the HPV pamphlet included the following: what is HPV, HPV transmission, how to conduct prepare for a Pap Smear and a Colposcopy, the HPV vaccine, who's eligible, and the importance of the follow-up visit for women with abnormal pap smear results. A ten item 4 point Likert type questionnaire based on the guidelines were used for participants evaluating the HPV pamphlet.
Guidelines to ensure a successful HPV video were based on the work of Fleming et al., 39 Hill, Hooper and Wahl, 40 van Vilet, Hillen, van der Wall, Plum, and Bensing, 41 and Winters et al. 42 Components of the HPV video were the prologue, three scenes and the epilogue. The prologue included information regarding HPV and the objectives that the viewer should achieved when viewing the video. Scene 1: What is HPV and How is it Transmitted; Scene 2: The Pap Smear and Colposcopy, definitions and patient preparation, what occurs during the procedure and patient follow-up instructions after the procedure. In scene 3, a woman inquires about the HPV vaccine for her son or daughter, the ages for pediatric administration, cost, side effects and the follow-up series for injections. The video epilogue included a review of the purpose of the film and the need for HPV awareness. Like the HPV pamphlet, a ten item questionnaire was developed for use by the participants to evaluate the HPV video in the area of communication, culturally appropriate and the ability to answer the film content objectives.
Data Analyses: Descriptive analyses of women of color who participated in the pilot were conducted to obtain demographics and participants responses. Descriptive statistics, analyzed by using the Statistical Package for Social Scientists software (SPSS) version 16.0 were 38 Validation of Educational tools for Use in a Human Papillomavirus Intervention Study calculated for each item. 43 A ten item 4 point Likert type questionnaire based on the guidelines were used for participants evaluating the HPV pamphlet in three areas: appearance, content and readability. The ten item Likert type questionnaire allowed participants to validate the HPV video in the area of communication, culturally appropriate and the ability to answer the film objectives. Frequencies and percentages were tabulated and recorded in the tables and figures listed in this article. Ten open ended questions, correlated with the qualitative questions for the HPV pamphlet and the HPV video were asked by the PI of the study at the completion of the participants completed the quantitative part of the study. Content analysis of qualitative data was conducted independently by the PI and reviewed by an external educational auditor.

Quantitative analyses
Demographic data: Fifteen women enrolled in a pilot study to validate the appearance, content and readability of the educational pamphlet and video. After reading and viewing the tools, the women completed two 4-pont Likert type questionnaires consisting of 10 items each. The majority (73.3%) of the 15 women who participated in the pilot study self-identified as African American/ Black. Ages ranged from 19 through 60, with eight (53.3%) women indicating an age range of 31-40. Most women (80%) were never married, divorced or single. Descriptive characteristics of the women enrolled in the pilot study are summarized in Table 1.

HPV pamphlet
Each participant's response was counted as a point for each item. The legend for the Likert type scale was: Strongly Agree -4 points, Agree -3 point; Disagree = 2 points; and Strongly Disagree = 1 point. Analysis of the items results in points on the Likert scale: Strongly Agree (n=8), Agree (n=55), Disagree (n=13), and Strongly Disagree (n=0). Study findings for the HPV pamphlet are located in Table 2. In analyzing each individual category, the following percentages were obtained: Question 1 addressed appearance: 44.66% strongly agreed, 33.33% agreed, with 20% disagreeing. Scoring for Questions 2 and 3 were identical. This question addressed the pamphlet's appearance and appeal; to identify if the women would take the pamphlet home. Scores for Question 2 and 3 combined were: 40% strongly agreed, 45.66% agreed, while 13.33% disagreed. Questions 4 and 5 addressed content for the Pap smear procedure and content for the Colposcopy procedure. Participants scored: 86.66% strongly agree and 33,33% agree for question 4. There were no scores for disagree. Question 5 Colposcopy, participants scored: 46.66 strongly agree and 53.33 agree, with no scores in the disagree section. Questions 6 and 7 addressed if after reading the pamphlet would the participant have a Pap smear or Colposcopy. In question 6, participants 100% of the participants selected strongly agree or agree. For question 7, 93% of the women agreed they would have a colposcopy. Questions 8, 9 and 10 the pictures used to describe the procedures and the cover of the pamphlet. Total overall scores were: Strongly agree (n=24) with a mean score of 8 (53.33%); agree (n=16) with a mean of 5.3 (35.33%) and disagree (n=5) with a mean of 1.6 (10.66%).  Additional information you feel should be added to the HPV pamphlet.

40
Validation of Educational tools for Use in a Human Papillomavirus Intervention Study

HPV pamphlet
Responses to questions 1, 2, 3 and 10 on the HPV evaluation tool related to appearance of the pamphlet and pictures on the cover of the pamphlet. An overwhelming, 100% of the participants, suggested a change in the appearance of the pamphlet. They did not like the front color of the pamphlet and suggested "an "eye catcher" to "lure women" to not only read [the pamphlet]in the clinic setting but to bring it home". Also, instead of a white colored pamphlet they suggested a more colorful "bright" colored pamphlet. Items 4 and 5 asked about the content for the Pap smear and Colposcopy, comments were "okay" easy to understand, while a few suggested a larger print. The participants commented on the influence of the written text, if the content would motivate you to have the test performed.
All participants had experienced the Pap smear and would continue to have the test performed because they "valued the outcomes and trust the health care provider" as when to have the pap smear or colposcopy. Questions 8 and 9 referred to the pictures used to depict the Pap smear and the Colposcopy, again all women agreed that the colored pictured was clear and accurate while the black and white picture for the colposcopy, while clear and accurate, should be changed to a color photo.

HPV video
Participants' responses to questions to questions 1, 2, 3, 4 and 9 were positive. The comments from women who self-identified as African American/ Black, Muslim or Hispanic agreed that the presentation was culturally appropriate, with a "believable performers", such as the physician whom some had seen in the clinic and the students.
Questions 5, 6, 7 and 8 referred to the video meeting the objectives as defined in the beginning of the film. Comments from the participants conformed that the video answered its objectives. However, two women suggested using bullets at the end of each segment to "summarize and reinforce content" would assist in the video meeting its objectives. Question 10 discusses the video ability to communicate information accurately with clarity to the participants. Comments from the participants to assist in making the video clearer would be to "cut some of the dialogue in the beginning of the film and spread throughout the film" and to "insert a blank page" prior to each scene so that the audience knows what information is coming next.

Discussion
The aim of this study was to reveal the process used to validate two HPV educational tools for use in a HPV educational intervention study for women of color. The socioeconomic status of the participants did not coincide completely with the data; however, the majority of the participants were between 31 and 40 years of age and four indicated they were married. A total of seven indicated an income less than 40,000 a year with three of the seven indicating 20,000 or less. These findings were similar to the other demographics noted in studies related to educational materials. Seventy five percent of the four women who indicated that they were married, self-reported as East Asian or Asian and represented the higher educational levels. This corresponds with a study conducted by Robinson et al. 44 revealed that Asian Americans who were married and educated with high annual incomes reported poor HPV and cervical cancer knowledge with a high incidence of HPV and cervical in their specific countries.
Additionally, Krawczyk et al. 45 recruited two hundred mix-gendered undergraduates students to participate in a HPV intervention study to compare the efficacy of two (HPV) educational interventions on increasing HPV knowledge and vaccination intentions in college students. A total of 38.5 % of the participants self-identified as non-Caucasian. Low HPV knowledge and intention to use HPV vaccination was assess initially in the college students. However, an increased in both HPV knowledge and HPV intention was identified after the group was exposed to HPV educational materials.
In our study, quantitative data results indicated participants responses within strongly agree and agree categories as they validated the HPV pamphlet and video. Comments identified areas that would strengthen the tools. Studies report that gender specific and tailored educational materials are effective for improving HPV knowledge in women of color. Studies have linked literacy as an indicator of positive health outcomes. Sharpe et al. 46 identified literacy and culture as essential components for HPV educational material in women of color. They concurred that American Indian women like other women of color required accurate, up to date information tailored to the need of the target population. This provides the need for further discussion related to HPV knowledge in educated as well as less educated persons.
For the researcher, the HPV pamphlet and the HPV video met the approval of the participants. Greater than 86% of the participants indicated that both educational tools were acceptable in appearance, content and readability. No instance of total disagreement was noted for any question. Alterations to the HPV pamphlet and the HPV booklet were altered based on group consensus. The educational materials were revised and are currently being used in the HPV intervention study with women of color.
This study has a few potential limitations. First, the sample size and the inclusionary criteria for women of color as the targeted audience in southeastern urban Louisiana may not yield the same results as women in rural settings thus limiting the generalizability of the findings. Although the researchers assured the participants on the use of the information as aggregate data and that use of confidentiality and anonymity, biases in self-reports may occur. Also the use of a different site was used to conduct the pilot study, contamination may have occurred secondary to one's knowledge about HPV, HPV pamphlets and videos. 26, 47

Conclusions
Data from the pilot study provided information for validating HPV educational tools. For the researcher, the HPV pamphlet and the HPV video met the informational approval for the intended HPV intervention study. Revisions to the HPV pamphlet and the HPV video were based on comments from the quantitative and qualitative data received from the pilot study participants. The importance of this study is the use of lay persons/ participants in the validation of educational tools. Nurse researchers have the opportunity to enhance or develop educational tools used when conducting research, especially in intervention studies. While studies suggest less educated women require more information regarding HPV, it is also noted that all women require more information regarding HPV knowledge and the HPV vaccine. Finally, validation of content should not only be developed by experts, but should be validated for clarity, readability and cultural appropriateness by targeted participants. Next a step for future researchers is the validation of more HPV educational materials and their use in intervention studies.