Development and validation of the reproductive health literacy questionnaire for Chinese unmarried youth

Objectives This study aimed to develop and validate the reproductive health literacy questionnaire for Chinese unmarried youth aged 15–24. Methods We conducted a validity and reliability study of the questionnaire through a cross-sectional survey and test–retest analysis in four districts in Shanghai between April and June 2017. A total of 1587 participants completed a self-administered questionnaire anonymously on-site and the trained investigators conducted quality check afterwards. Sixty participants among them completed the test–retest assessment with 2 weeks interval. The reliability was determined by internal consistency, spilt-half reliability and test–retest reliability. The construct validity was assessed by confirmatory factor analysis. Results The 58-question reproductive health literacy questionnaire for Chinese unmarried youth demonstrated high internal consistency (Cronbach’s α = 0.919), spilt-half reliability (Guttman splitting coefficient = 0.846) and test–retest reliability (correlation coefficient = 0.720). The confirmatory factor analysis showed that the construct of the questionnaire fitted well with the hypothetical model. The reproductive health literacy scores in unmarried girls aged 15–24 were higher than boys (P < 0.05) and college students who lived in rural areas when they were middle and high school obtained lower score than those living in cities and suburbs (P < 0.01). Conclusion The reproductive health literacy questionnaire for Chinese unmarried youth demonstrated good reliability and validity, which could potentially be used as an effective evaluation instrument to assess reproductive health literacy among Chinese young people.


Background
Youth aged 15-24 years are in a period of rapid matureness in reproductive function and social adaptation. With sexual consciousness awakening, secretion of sex hormones increasing and sexual impulses emerging, they need adequate reproductive health literacy to get through this key period [1]. With rapid development of the global society, youth in all societies are reaching puberty earlier but marrying later, which expands the gap between puberty and marriage [2]. Thus, unmarried youth have urgent need for accessible, quality reproductive health care. However, they are more likely to lack the knowledge, skills and ability to maintain reproductive health in low and middle income countries as they don't have necessary information on sexual behaviour and contraception as well as inadequate understanding of negative effects of abortion, sexually transmitted diseases (STDs) and acquired immune deficiency syndrome (AIDS) [3,4]. Notably, the long-term insufficient sexual education in China and the large population size make the reproductive health of Chinese unmarried youth an essential research focus [5].
Health literacy was first introduced in a discussion of health education at an international conference in 1974 [6]. Sørensen et al. proposed an integrated definition of health literacy in 2012, which is the competencies related to the process of accessing, understanding, appraising and applying health-related information as an asset for improving people's empowerment within the domains of healthcare, disease prevention and health promotion [7]. Improving population health literacy is regarded as a cost-effective health promotion intervention as well as an important step toward reducing health inequities [8]. With the research on health literacy continuously more specific, there is a trend that more studies emerged with aiming at specific population, such as maternal and infant health literacy [9], child health literacy [10], and patients' health literacy [11]. However, the consensus of youth health literacy related to reproductive health measurement is still unreached. More importantly, sexual and reproductive health education in China started and developed later compared with western countries, and still in exploratory stage. Although there are limited research having explored some evaluation structures for adolescent health literacy, most of them are designed for general population and not based on a unified conceptual framework [12,13]. Additionally, there is still a research gap about health literacy evaluation tool specifically for youth sexual and reproductive health.
In the past 27 years since the 1994 International Conference on Population and Development, where to strengthen the promotion of adolescent sexual and reproductive health was proposed, many aspects of adolescent reproductive health have substantially improved, including increasing use of contraceptives, decline of child marriage and female genital mutilation. However, there are still some challenges ahead of us, such as more STDs (including human immunodeficiency virus, HIV) and negative effect of more obesity on reproductive health, which warrants much closer attention looking forward [14]. Furthermore, as part of reproductive health literacy, early detection of sexually transmitted diseases through screening is critical for disease prevention, such as the human papillomavirus (HPV) screening test which is associated with cervical cancer [15]. Given the health literacy can be cultivated and promoted by education, it is necessary to develop a reproductive health literacy measuring tool for unmarried youth. Therefore, we proposed the concept of "reproductive health literacy" and developed it as an evaluation tool for reproductive health literacy of unmarried youth aged 15-24, to measure the reproductive health literacy among youth and to provide ideas for implementing targeted interventions to promote reproduction health.

Instrument development Stage 1: conceptual framework and indicators generation
The conceptual framework developed by Sørensen et al. in 2012 [7] and World Health Organisation (WHO) in 2013 [16], was used in the present study, which consists of three health domains (health care, disease prevention, and health promotion) and four factors of ability to health information in each domain, including accessing, understanding, appraising, and applying. Based on this conceptual framework, a pool of 46 indicators including determinants of reproductive health in the physical and social environment (health promotion), risk factors questionnaire we developed is effective to assess the reproductive health literacy in Chinese unmarried youth aged 15-24, which includes four aspects of literacy ability (accessing, understanding, appraising, and applying) to three health domains (reproductive health determinants, behavioural risk factors, and health service utilisation). The development of this tool can not only measure the reproductive health literacy level, but also be used for long-term monitoring, as well as facilitating effect evaluation of intervention. This questionnaire may also help to develop target interventions to improve reproductive health literacy of Chinese young people.
for reproductive health (disease prevention), health services related to reproductive health problems or diseases (health care) was set up. Then, a two-round Delphi expert consultation process was carried out among 20 multi-disciplinary specialists, including researchers, doctors and teaching staff, and who working in adolescent and school health, maternal and child health, sexual and reproductive health, and health education. They were required to evaluate the content representativeness, health literacy relevance, and evaluation feasibility of each indicator [17]. As a result, 45 reproductive health literacy indicators were identified by consensus, as shown in Table 1.

Stage 2: questionnaire development
The final questionnaire was developed based on the 45 indicators. Firstly, in general, each indicator was directly transformed into one item and some indicators were converted into two items through discussion within the research group. Secondly, feedbacks from 4 experts with extensive professional experience were collected via email to modify the questionnaire. Thirdly, a total of 30 volunteers, including 10 public health professionals and 20 unmarried youth, were invited to complete the questionnaire and provide feedback. Then, based on their feedback, we adjusted the statements and improved the questionnaire language to avoid ambiguity and make the questionnaire more reader-friendly. Lastly, a questionnaire with 58 items on reproductive health literacy for unmarried youth was formed, which covered six topics: physiological and psychological development during adolescence, personal health care during adolescence, heterosexual relationship and sexual behaviour, prevention and response to sexual harassment and sexual abuse, prevention of AIDS and STDs, and prevention and response to unintended pregnancy. The questionnaire was also involved with four aspects of literacy ability (accessing, understanding, appraising, and applying) to reproductive health determinants, behavioural risk factors, and health service utilisation, as shown in Table 1.
The scoring system was as follows: (1) 4 points for the correct answer to the two-category multiple-choice items and 0 point for the wrong answer; (2) In the graded multiple-choice items, 0, 1, 2, 3, and 4 points were assigned to the answers of "I don't know", "very difficult", "difficult", "easy" and "very easy", respectively; (3) When multiplechoice items were scored, each option was treated as a true/false item. If there is "n" options, the correct score for each option was 4/n and the total score of this item was the sum of the scores for each option. If the answer is "don't know", 0 point was assigned directly. According to this rule, the scores of each item were uniformly processed into continuous variables in the range of 0 ~ 4.

Validation of the questionnaire Study participants and data collection
A cross-sectional survey was conducted with multistage sampling method in four districts in Shanghai from April to June 2017, including Huangpu District, Jiading District, Songjiang District and Pudong District of Shanghai. The inclusion criteria were as follows: (1) students of grade one and two in general public high schools; (2) undergraduate students in the first three years from general universities and junior colleges; (3) unmarried youth who have basic Chinese reading skills and been informed consent. The multi-stage sampling method was carried out as follows. Two general public schools in each district and two institutions of higher education (one general university and one junior college) both in Songjiang District and Pudong District were included. Then, using a stratified random sampling method, two classes from each grade of each school/college were chosen, and 30 students (15 girls and 15 boys) from each class were sampled by a systematic sampling method. Finally, a total of 1606 participants were recruited to complete a self-administered questionnaire anonymously on-site and the trained investigators conducted quality check afterwards. In order to evaluate test-retest reliability, 60 students from a high school in Songjiang District were invited to complete the questionnaire again two weeks later. Data on demographics and reproductive health status were also collected from the participants, including age, gender, place of household registration, parents' education levels, family income, etc. This study was approved by the Ethics Committee of School of Public Health, Fudan University (IRB#2016-11-0604).

Item analysis
The measuring indicators used for item analysis included degree of difficulty, degree of discrimination, and internal consistency analysis. Items that were too simple, undistinguished and had no contribution to internal consistency were deleted. Accordingly, if the item met the following three conditions at the same time: degree of difficulty > 0.8; discrimination < 0.2; internal consistency became better after deleting the item; then the item was deleted.
After item analysis, the degree of difficulty for items ranged 0.27-0.89, and the degree of discrimination ranged 0.155-0.572, and the correlation coefficients were statistically significant. Among them, 25.86% of items had the degree of difficulty > 0.7, that was, about one quarter of the items were relatively not difficult. As a result, no items were deleted.

Reliability and validity test
All the items in the reproductive health literacy questionnaire were divided into three parts for reliability and validity evaluation according to their first-level indicators: Health Care, Disease Prevention and Health Promotion [18]. Reliability was measured by the internal consistency, split-half reliability and test-retest reliability [19]. The internal consistency was measured with Cronbach's α coefficient, and the split-half reliability was measured with the Guttman half-coefficient between odd and even items. Additionally, test-retest reliability was measured with the correlation coefficient between the twice results completed by the 60 high school students with a twoweek interval. Each of the values greater than 0.70 indicated acceptable reliability [19,20].
Validity was measured by the content validity and structural validity. The content validity was ensured by the repeated revisions from the international advanced health literacy conceptual model, the consultation of the experts, and practical experience of the questionnaire. The confirmatory factor analysis (CFA) was used to verify the construct validity [21]. The analysis was conducted separately for each domain of the questionnaire (health control, disease prevention, health promotion) to match the four domains of "acquisition", "understanding", "evaluation" and "application" in the conceptual framework. The model fit was evaluated by the key indicators of root mean square error of approximation (RMSEA), goodness-of-fit index (GFI) and adjusted goodness-of-fit index (AGFI), with which < 0.08, > 0.9, > 0.9, respectively, indicated "relatively good" fit [18,22].

Statistical analysis
The returned questionnaires were entered into database by using Epidata 3.1, and the invalid questionnaires (the missing data in a questionnaire more than 20%) were deleted. The scores of each item in the reproductive health literacy questionnaire were calculated in Excel 2016 while reliability analyses and other parametric tests were computed in SPSS 21.0. Descriptive statistics of the participants' characteristics, correlation analysis, t-test and one-way analysis of variance (ANOVA) were conducted to describe the sociodemographic characteristics and reproductive health literacy scores, to understand the relationship among various domains of health literacy, and to estimate the potential risk factors related to health literacy. Additionally, CFA was conducted with maximum likelihood estimation by using AMOS 21.0. The significance level was set at P < 0.05.
When scoring each item for reproductive health literacy, the indicator score was the average score of corresponding items if one indicator corresponded to multiple items. Then the total score was the indicator score multiplied by its corresponding weight which was assessed during Delphi consultation. The final standardised total score (out of 100) was calculated as follows: final total score = ∑ (each indicator score × corresponding weight) × 100/4. A higher score indicated higher reproductive health literacy.

Results
Results of the validation study of the 58-item reproductive health literacy questionnaire for Chinese unmarried youth using a cross-sectional survey are presented below.

Sociodemographic characteristics of participants and descriptive results for the questionnaire
A total of 1587 valid questionnaires were collected from schools in four districts in Shanghai. As shown in Table 2, gender distribution of the participants was balanced, and the age ranged from 16 to 25 years old, with 94.9% in the 16-21 age group. Nearly 90% of the high school students' household registration was in Shanghai, with the non-Shanghai households of college students accounted for more than 50%. Similarly, more than 70% of high school students and only about 40% of college students lived in large cities during junior and high school age. In addition, the monthly income per capita of most core family was 4500 ~ 7500 RMB, accounting for 35.0%, and 28.1% of the families got 7500 ~ 12,500 RMB monthly income per capita.
After comparing the average reproductive health literacy score in different socio-demographic groups, score differences in sex were significant both in high school students and college students. Among 960 high school students, girls average scored 3.28 higher than boys (U = 10.661, P = 0.001), and among 627 college students, girls average scored 4.16 higher than boys (U = 6.122, P = 0.013). Additionally, the living area during junior and high school age was associated with the scores of reproductive health literacy of college students (F = 5.023, P < 0.01). Further analysis between either two groups showed the score of the students who lived in rural areas during junior and high school age was lower than that of students who lived in the large cities (P = 0.001), small and medium-sized cities (P < 0.001), and rural suburbs (P = 0.002).
Overall, the P 50 (P 25 , P 75 ) of the total score of the reproductive health literacy among 1587 participants was 64.67 (53. 30, 72.19). As shown in Table 3, of three domains, total score in disease prevention was highest, with median of 73.77, whereas total in health promotion was lowest, with median of 57.42. On the other hand, compared with other 3 ability aspects, the ability of applying was highest. Additionally, the results showed significant differences across scores in three domains (χ 2 = 409.8, P < 0.01). After compared in pairs, these scores indicated significant differences between either two of them, with the highest score in disease prevention and the lowest score in health promotion. Table 2 Demographic and socio-economic characteristics of participants and average scores of reproductive health literacy a Non-parametric tests (Mann-Whitney U test or Kruskal-Wallis test). *Family socio-economic status was determined by Green score [23], which was calculated based on maternal and paternal occupation and education. Green score = (paternal education × 0.7 + paternal occupation × 0.4 + maternal education × 0.7 + maternal occupation × 0.4)/2

Reliability
As shown in Table 4, generally, the Cronbach' α coefficient of total questionnaire and the three domains (health control, disease prevention, health promotion) ranged 0.493-0.919, and the Guttman splitting coefficient ranged 0.469-0.846, indicating the good internal consistency of the questionnaire. However, the values for the health promotion were relatively lower, which was due to fewer items in this domain. The correlation coefficients of the questionnaire completed by 60 participants' twice in two weeks ranged 0.495-0.772, which indicated good test-retest reliability.

Validity
Content validity was ensured by the scientific developing process of the reproductive health literacy questionnaire. Firstly, an extensive literature review was conducted to identify the main reproductive health problems in the 15 ~ 24 age group, followed by an on-site discussion within 20 experts to develop 45 indicators based on WHO's health literacy framework. Secondly, 20 professionals were invited to give feedback repeatedly to improve the indicators and revised into 46. Thirdly, an original version of the questionnaire was developed based on the indicators, and experts and volunteers were invited to improve it through professional suggestions and language changes.

Construct validity
The results of the confirmatory factor analysis (CFA) showed a relatively good fit of all the four ability factors within three domains of reproductive health literacy (Table 5).

Discussion
The 58-question reproductive health literacy questionnaire for Chinese unmarried youth was developed for evaluating reproductive health literacy among Chinese young people. The validation study was carried out among 1587 students from 12 high schools and colleges in four districts of Shanghai, which was a representative sample of Shanghai. The range of the reproductive health score is between 0 and 100, with median of 64.67, and a higher score indicates higher reproductive health literacy level. Psychometric analysis results indicated that it has good reliability and validity and could be a useful instrument for assessing reproductive health literacy for unmarried youth in the Chinese context. The development of reproductive health literacy questionnaire for Chinese unmarried youth is in line with the opinion from Nutbeam who proposed that the measurement of health literacy would be best achieved where content and context were well defined [24]. This study was based on the conceptual framework of health literacy [7], that is to say, reproductive health literacy integrated the content of medical services and public health, formed by three domains (medical services, disease prevention and health promotion) and four abilities (acquisition, understanding, evaluation and application). The application of this conceptual framework provides a theoretical basis for the development of instruments for assessing reproductive health literacy, with which we developed and applied the first measurement in China for assessing reproductive health literacy among young people.
The psychometric evaluation of the reproductive health literacy questionnaire for Chinese unmarried youth produced plausible results. The overall 58-question questionnaire was reliable, demonstrated by high internal consistency, spilt-half reliability and test-retest reliability (all coefficients > 0.7). For the three domains,  all reliability coefficients were over 0.6 which was considered as acceptable reliability [25], except for the health promotion. It may be due to the less amount of questions in this domain, therefore, we recommend that the questionnaire should be administered as a whole when applying in the future studies. The results of confirmatory factor analysis suggested that the construct of the questionnaire fitted well with the theoretical model, represented an acceptable fit [22,26]. On the other hand, we applied various methods to ensure the content validity of the questionnaire, including literature review, professional consultation and exiting evidence reference, as well as application of the health literacy integration conceptual framework [7]. In the meantime, both experts and unmarried youth were asked to provide feedback to help us improve the questionnaire's content validity as well as language adjustments to make the questionnaire more reader-friendly. The total reproductive health literacy score among youth in our study was around 60, indicating that the overall reproductive health literacy of the unmarried youth in Shanghai is at an intermediate level. Our study found that the unmarried youth obtained the highest score in "disease prevention", and the lowest in "health promotion", which suggests that unmarried youth in China have a poor understanding of the social determinants of reproductive health. The reasons may be that they do not recognise the importance of reproductive health or they cannot effectively identify the social determinants of reproductive health in daily life, which reminds that the social environment and policy system for the reproductive health of unmarried youth need to be further optimised and improved [27]. Additionally, we found that youth scored lower in the competences of accessing, understanding and appraising, compared with the competence of applying, indicating that a comprehensive health literacy intervention is needed to empower youth to access, understand and appraise health information [4]. Rather than the "knowledge, attitude and practice" model, health literacy covers the acquisition and evaluation of health information. Also, we found that youth scored lower in these two ability factors, indicating young people still lack access to health information and authorities should establish more professional platforms to reduce barriers in acquiring reproductive health information and provide professional and accessible information to the public via audience-friendly ways [28].
The main social determinants that affect the scores of high school and college students' reproductive health literacy were gender and the living area in junior and high school. We found that unmarried girls aged 15-24 had higher scores than boys and the difference is statistically significant, which is consistent with other studies on health literacy among Chinese young people [29][30][31][32]. It may be due to the fact that girls are sexually mature earlier than boys and are more concerned about their own health. Another explanation could be that society often places higher moral expectations on females due to the differences in gender norms, causing girls to receive more information about reproductive health at a younger age than boys [33]. Additionally, the living area during junior and high school was associated with the scores of reproductive health literacy among college students. Specifically, college students who lived in rural areas when they were at middle and high school age scored lower than those living in large cities, small and medium-sized cities and suburbs. This may be because of the fact that middle and high school stage is a critical phase for sexual development, with relatively high plasticity and health awareness [34]. Furthermore, rural areas have a more conservative culture, a lower level of economic development and a relative lack of health resources as compared to cities, all of which may contribute to the difference in scores between rural and urban youth. However, we did not find significant effects of family social status (parents' occupation and education) and family income per capita on the reproductive health score among unmarried youth, suggesting that reproductive health literacy level in the 15-24 age group is hardly influenced by the family, which is inconsistent with the results from Xu et al. [35]. It may be that reproductive health is different from other health areas, and it is still a private and sensitive topic in the Chinese family environment [5]. The acquisition and exchange of reproductive health information may come more from school and peer education than family [36,37]. Thus, future reproductive health promotion interventions should be considered for implementation in the school setting.
The development and validation of an appropriate measurement is fundamental and essential for youth reproductive health literacy research. To the best of our knowledge, this is the first study proposing the concept of reproductive health literacy and also the first time developing and evaluating reproductive health literacy questionnaire for unmarried youth in China. Additionally, this study suggests that improving reproductive health literacy can be a good way to promote reproductive health. Improving the reproductive health literacy of unmarried youth will help reduce and prevent reproductive health problems in this age group, improve the utilisation rate of reproductive health services and compliance of reproductive health related diseases treatment, and thereby promote the reproductive health and decrease social burden of related disease [4].
However, some limitations of this study should be concerned. First, the reproductive health literacy presented in this study only included unmarried youth in schools and colleges but not the same age population outside schools. Second, since the participants were all from Shanghai, which are not representative enough for the whole country, further studies are needed to generalise this measurement in other regions and settings of China. Third, response bias cannot be avoided as the questionnaire is based on self-reporting.

Conclusion
This is the first time developing the concept and evaluating domains of reproductive health literacy based on the WHO health literacy framework. The reproductive health literacy questionnaire has demonstrated good reliability and validity among Chinese youth aged 15-24. The development of this tool can not only measure the reproductive health literacy level, but also be used for long-term monitoring, as well as facilitating effect evaluation of intervention. This questionnaire may also help to develop target interventions to improve reproductive health literacy of Chinese young people.