Intercultural Adaptation and Validity Study: Universal Science Literacy Scale (USLS)

The purpose of the current study is to adapt ‘The Universal Science Literacy Scale’ developed by Mun, Shin, Lee, Kim, Choi, Choi and Krajcik into Turkish. The study group of the current research is comprised of a total of 645 pre-service science teachers from 6 different universities of Turkey. In the first stage of the adaptation study, the items of the original scale were translated into Turkish by ELT field experts. Then, a linguistic equivalence test was conducted for the linguistic validity of the scale. The construct validity of the scale was tested with Confirmatory Factor Analysis (CFA). The obtained findings showed that the adapted scale has the same factor structure as the original scale. In addition to this, the results of CFA revealed that the goodness-of-fit indices of the scale are quite good. Cronbach alpha internal consistency (reliability) coefficient of the scale was found to be 0.91. As a result of the analyses conducted, it was concluded that the Universal Science Literacy Scale is a valid scale that can make contributions to the relevant literature in Turkey.


Introduction
The question "What is science?" has remained as a question for many years that scientists have had difficulty in reaching a common decision on. Lack of agreement on a common definition stems from the fact that science is constantly evolving and has a multi-faceted and dynamic structure with no limitations on the subjects covered by it [6,19]. Science, while living its heydays in the Egyptian, Mesopotamian, Babylonian and Chinese civilizations in the East, spread over time to the ancient Greek world and then to Europe. Science began to rise in the Islamic world since the 8 th century when it lost its effectiveness in Europe. This circulation, going between the East and the West, has come up to now through cumulative progression of science [67]. The rapid development of science and technology in the 19th century, which is called the information age, brought about many changes in the structure of the societies and forced education to undergo significant changes to keep pace with this rapid development. Especially after the 1950s, many governments saw scientific knowledge as the key to the future economic race and struggled to raise the scientific knowledge and skills of the individuals who constituted the society [58]. In the 21st century, when knowledge is increasing at a rapid pace, people feel the need to follow rapid developments in science and technology and keep up with these developments. Information societies have attached a great importance to these developments for their own future. Countries that find this important want to educate their youth as individuals who are in constant change and development, who follow science, who are aware of innovations and developments, who think how they can contribute to these developments and who can put their thoughts into action [52]. Each country has set up a student profile and identified and targeted specific criteria for educating citizens in accordance with their own cultural and sociological structure, in line with the country's policy. Students with this profile are called literate individuals. Scientists recognizing the importance of educating individuals in the society as literate over time have conducted studies focusing on different aspects of literacy. In this respect, they focused on information literacy [17,54,55], science literacy [1,3,34,53], scientific literacy [35,68], technological literacy [8,13], science and technological literacy [18,20,74], math literacy [21,56], environmental literacy [5,37].
Among these types of literacy, science literacy is thought to have a very important place in terms of improving the level of development of the society by making individuals more able to understand their environment, to establish a relationship between science and everyday life, to make their lives more meaningful, to contribute to scientific culture and to take an active role in society. Because of this, the education of science literate individuals through science curricula has become the priority of many countries in the world [69]. Especially since the 1980s, international associations of science education and science education institutions have tried to define and shape science literacy in science programs (Bybee, 1997) cited in [69], [15,44,46,49,61]. Therefore, determination of the target population's literacy level, science literacy perception, opinions about science literacy and characteristics to be possessed by a science literature individual is of great importance in the definition and shaping of science literacy. In this connection, in the literature there are many studies focusing on different aspects of science literacy such as those attempting to develop scales to measure science literacy levels [24,27,39,48], proposing theoretical basis on which science literacy can be built [9,11,29], offering suggestions to enhance students' science literacy perceptions [40,41,50], measuring individuals' science literacy levels [4,39,47,48,53,72,73], exploring the effect of different variables on science literacy level [2,25,32,59,63], evaluating science curriculums and textbooks in terms of science literacy [7,10,23,75,76] and investigating historical and conceptual evolution of science literacy [9,16,30,38,45,69]. The scale developed in one of these studies by Laugksch and Spargo [39]) "The Basic Science Literacy Scale" has been observed to be one of the most widely used scales in the determination of individuals' science literacy [22,31,70,74]. Since this scale was developed in the 1990s, behaviors expected from science-literate individuals have changed. In addition to this, the renewal of the science curriculum in 2013 required the existence of a current scale to represent the renewed understanding of science literacy. In this connection, the scale to be developed through an adaptation study in the current research is believed to meet this emerging need in the relevant literature in Turkey.

Significance of the Study
The Ministry of National Education (MEB) Head Council of Education and Morality made some changes on the Science and Technology course curriculum in 2013. The reasons for these changes are expressed as follows:  Changing basic education system from 2012-2013 onwards,  Each unit in the Science and Technology course curriculum has too many objectives,  The objectives of the 2004 Science and Technology curriculum are predominantly in the field of cognitive learning, which is why students need objectives to strengthen their affective learning,  The need felt for putting a greater emphasis on the 21 st century skills (analytical thinking, communication and team work, decision making and entrepreneurship),  Instruction should be conducted not only according to 5E model but also according to all learning models [12] in which students are active.
Based on these reasons, the new science curriculum consists of 4 learning areas; knowledge, skill, affective and Science-Technology-Society-Environment (STCE) ( Table  1).
From these learning areas, the learning area of skill consists of the following sub-learning areas; analytical thinking, decision making, creative thinking, entrepreneurship, communication and team work. From among these sub-learning areas, communication and team work sub-learning area corresponds to USLS's communication and cooperation factor and the analytical learning sub-learning area corresponds to USLS's systematic thinking/information management factor.
The affective learning area of the science curriculum consists of the following sub-learning areas; positive attitude towards science, motivation, values and responsibility. These sub-learning areas are believed to correspond to USLS's characters and values dimension.
STSE learning area consists of the following sub-learning areas; socio-scientific issues, nature of science, science and technology relationship, contribution of science to society, awareness of sustainable development and awareness of science and career [12,43]. USLS's socio-scientific responsibility factor seems to correspond to the sub-learning area of socio-scientific issues in the curriculum; characteristics of scientific knowledge seems to correspond to the sub-learning area of nature of science; science and society and ecological world view factor seems to correspond to the sub-learning areas of science and technology relationship and awareness of sustainable development. Thus, it seems that "The Universal Science Literacy Scale" developed by Mun et al. [48] matches up with the vision of the current science curriculum and the learning areas in the curriculum; therefore, it can be argued that the scale covers the dimensions of science literacy in the renewed program. Introduction of such a scale covering the up-to-date dimensions of science literacy in the literature will be helpful to academicians, practitioners and teachers working in this field.
Thus, the purpose of the current study is to adapt "The Universal Science Literacy Scale" developed by Mun et al., [48] into Turkish. To this end, the current study seeks answers to the following sub-problems:

1.
Is there a significant correlation between the scores taken from the original form and Turkish version of USLS?

2.
Is there a significant difference between the scores taken from the original form and Turkish version of USLS? 3. Does the factor structure obtained from the Turkish version of USLS comply with the factor structure of the original scale? 4. Are the test results of USLS reliable?

Study Group
Throughout the study, applications were conducted on two different study groups. In order to ensure the linguistic equivalence of the scale translated from English to Turkish, a total of 44 engineering faculty students having competence both in English and Turkish made up the first sampling for linguistic equivalence. The second sampling was selected to test whether the adapted scale has a structure similar to the factor structure of the original scale. This second sampling is comprised of 1 st , 2 nd , 3 rd and 4 th year 645 pre-service science teachers. This sampling size seems to be quite adequate to conduct confirmatory factor analysis [51, 57, 66].

Data Collection Tool
The Universal Science Literacy Scale (USLS) was developed in 2015 by Mun et al. with the purpose of contributing to the education of universal individuals having social characteristics, behaviors and values of the 21 st century and having mastered socio-scientific issues. The scale has a total of 48 items. The items and dimensions making up the original English USLS were combined with the scale developed by Laugsch and Spargo [39] including the subjects of Science-Technology-Society Relationship and Nature of Science and the scale developed by Manhart [42] including the subjects of societal perspective, science through human effort and the place of basic natural sciences in daily life and enhanced with the 21 st century skills and competences [48]. Moreover, the concept of universality added to science literacy entails viewing every citizen in the society as a universal citizen. Built on this theoretical framework, USLS has 4 dimensions and 8 factors. These dimensions are:

1)
Habits of mind 2) Character and values 3) Science as human endeavor 4) Metacognition and self-direction The information about the extent to which the dimensions making up the theoretical framework of the 21 st century universal science literacy and the factors making up these dimensions are represented in USLS and with how many items they are represented can be seen in Table 2.  When Table 2 is examined, it is seen that of the items making up USLS, 9 (18 Mun et al. [48] obtained a four-dimensional structure explaining 50.22% of the total variance as a result of the exploratory factor analysis (EFA). First of these dimensions consists of 13 items (items 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 and 13). The factor loadings of the items in this dimension vary between 0.48 and 0.67 and explain 6.76% of the total variance in the scale. The second dimension consists of 9 items (items 14,15,16,17,18,19,20,21,22) with factor loadings varying between 0.46 and .76. This dimension explains 5.5% of the total variance in the scale. The third factor of the scale includes a total of 13 items (items 23,24,25,26,27,28,29,30,31,32,33,34 and 35) whose factor loadings vary between 0.41 and 0.65. This dimension explains 27.78% of the total variance. The fourth dimension has 13 items (items 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47 and 48) whose factor loadings vary between 0.42 and 0.71. This dimension explains 10.18% of the total variance in the scale. Findings of the EFA conducted by Mun et al. [48] are given in Table 3.

Application
In order to adapt USLS to Turkish, we contacted Hyunju Lee, one of the developers of the scale. As a result of the correspondence, necessary permissions were obtained.
Following the granting of permissions, translation of the original scale into Turkish was performed by an English language expert. After the first translation, the suitability of the translated items in terms of science terminology was checked by two experts in the field of science.
In order to increase the quality of the translation and make its complete adaptation to the Turkish culture, the opinions of a science educator and an ELT field expert who had never seen the items throughout the translation process were separately gathered by using an Expert Evaluation Form (EEF). Final form of the scale was given in light of the data obtained from EEF and became ready for the linguistic equivalence study. Linguistic equivalence was tested to see whether the original form and the Turkish version have the same meaning. In this respect, the application was conducted on 44 engineering faculty students having competence in both Turkish and English. The reason for conducting the application to ensure linguistic equivalence on the engineering students is that students studying in departments related to science and math are familiar with positive sciences, science literacy and experimental works. The original English form of USLS copied to be numbered from 1 to 44 was administered to the students. Three weeks after this first administration, the Turkish version of USLS was administered to the students. In both of these administrations, the same person was given the English and Turkish forms with the same number to be filled in because the correlation coefficient and significance level between the two applications would yield information about similarity in meaning.
The validity of the scale was established on 645 pre-service science teachers. The data collected from the pilot study was tested with confirmatory factor analysis. As a result of the analysis, the goodness-of-fit indices showed that in the adaptation of the scale to Turkish, validity was established. The applicable form of the scale is given in App. 1.

Data Analysis
The analysis of the collected data was conducted by using SPSS 21 and LISREL 8.70 program packages. In the interpretation of the analysis results, the upper limit of margin of error was accepted to be .05. On the other hand, in the linguistic equivalence, the significance level was set to be .01. In order to ensure the linguistic equivalence, the relationship between the responses given to the Turkish and English versions of the form was tested with Pearson-product Moment Correlation Coefficient. On the other hand, independent samples t-test was conducted to analyze the significance between the participants' responses to the two forms of the scale.
USLS is a scale consisting of 8 factors and 4 dimensions. For the establishment of the same factor structure in the adaptation to Turkish, LISREL 8.70 program package was used. As a result of this analysis, whether the factor structures of the original form of USLS and its Turkish version are identical was tested through confirmatory factor analysis (CFA). On the other hand, the USLS's reliability was tested with Cronbach Alpha internal consistency coefficient calculated on the Turkish version of the scale.

USLS's Linguistic Equivalence
In order to establish the linguistic equivalence of the translated scale items in the target culture, an answer to the question "Is there a significant correlation and a significant difference between the scores obtained from the administrations of USLS's original form and Turkish version?". In order to test the linguistic equivalence of the scale, the correlation between the scores obtained from the original form and Turkish version of the scale was calculated by using Pearson Product-moment Correlation Coefficient. The findings derived from this analysis are given in Table 4. As can be seen in Table 4, the correlation coefficient between the responses given to the original form and Turkish version of the scale was calculated to be 0.83 (r=.83, p<.01).
On the other hand, whether there is a significant difference between the scores obtained from the administrations of the original scale and Turkish version was analyzed through independent samples t-test. The findings obtained from this analysis are given in Table 5. As can be seen in Table 5, while the mean score of the responses given to the original English scale is 3.94, the mean score of the responses given to the Turkish version is 3.96. Moreover, no significant difference was found between the participants' responses given to the original English form and its Turkish version [t(43)= -.433, p>0,05].

Construct Validity of USLS
The data gathered from 645 pre-service science teachers within the context of the pilot study were analyzed with LISREL 8.70 program package, the results of CFA are given in Figure 1.
As can be seen in Figure 1, the goodness-of-fit indices of the Universal Science Literacy Scale consisting of 48 items and 8 factors were found to be significant (X 2 =2132.16, sd=1050, p=.00). Moreover, as the sampling size directly affects chi-square value, X 2 /sd ratio that is not affected from the sampling size should be examined [14]. Thus, this ratio was calculated to be 2132.16/1050=2.03 in the current study.
Moreover, in figure 1, error variance for each item of the model can be seen. When the error variance for each item was examined, it was found to be lower than 0.90.
On the other hand, in the model obtained from CFA, the item loading values between the observed variables and the latent variables were found to be ranging from 0.34 to 0.82 and t-value was found to be significant. The factor loading value shows the degree of the correlation between the item and its relevant factor. Therefore, it is desirable and expected that the value of the item over the relevant factor is high. However, different arguments have been raised about which of the values of factor loadings are high, which are low and which are acceptable or which are unacceptable.
As a result of the CFA, many goodness-of-fit indices are calculated which show the correspondence between the theoretical model and the data. Some of these indices are Chi-Square Goodness (χ 2 ), Normed Fit Index (NFI), Comparative Fit Index (CFI), Non-Normed Fit Index (NNFI), Goodness of Fit Index (GFI), Adjusted Goodness of Fit Index (AGFI) and Root Mean Square Error of Approximation (RMSEA).
These goodness-of-fit indices are one of the criteria that should be taken into consideration to establish the construct validity of a scale. However, which of these fit indices will be used depends on the researcher. Yet, from these fit indices, the most commonly used ones in the literature are CFI and RMSEA [65]. The fit indices calculated for the original scale and the ones calculated in the current study are shown in Table 7.

USLS's Reliability
The third sub-problem of the research requires testing whether the Turkish version of the scale is a reliable scale. To this end, Cronbach alpha internal consistency coefficient was calculated on the data collected in the pilot study. Cronbach alpha coefficients calculated for each dimension of the scale are shown in Table 6.  As can be seen in Table 7, Cronbach alpha internal consistency coefficients calculated for the Turkish version of USLS are as follows; 0.81 for the dimension of habits of mind, 0.76 for the dimension of character and values, 0.79 for the dimension of science as human endeavor and 0.85 for the dimension of metacognition and self-direction. On the other hand, for the whole scale, Cronbach alpha internal consistency coefficient was calculated to be 0.91.

Discussion
In this study, it was aimed to adapt the "Universal Science Literacy Scale" developed by Mun, Shin, Lee, Kim, Choi, Choi and Krajcik [48] into Turkish. For this purpose, the linguistic equivalence study was conducted and the results of this linguistic equivalence study were examined and then the correlation coefficient calculated to test the correlation between the participants' responses to the items of the original English scale and to the items of the Turkish version revealed a highly positive and significant correlation. Thus, it can be said that the meaning derived from the original scale by the participants is considerably similar to the meaning derived from the Turkish version. This shows that the linguistic equivalence of the translated items was ensured. Both the correlation coefficient and significance test results show that the translation of USLS is in a high correspondence with the original one and the linguistic equivalence of the items was achieved.
Some analysis needs to be done to determine whether the scale required to be developed in one culture and adapted to another target culture serves its purpose in the target culture. This analysis, called the Confirmatory Factor Analysis (CFA), is conducted to obtain information on the validity of the measuring tool, to reveal the factor design, or to determine whether the theoretical structure of a previously developed scale provides the same factor structure in another culture [28].
When construct validity is tested, there are more than one methods for exposing the theoretical construct. The method that best matches up with the purpose of the study should be chosen in line with the information the researcher wishes to elicit [14]. In the case of the intercultural scale adaptation studies, if the factor design of the measurement tool is elicited in the original culture by means of qualitative and quantitative research methods and experimental evidence is presented as regards the construct validity, the factor structure of the given tool is tested through confirmatory factor analysis [14]. As for the scale used in the current study, the theoretical basis, factor design and reliability and validity background have already been provided by Mun et al. [14], the testing process of validity was continued with confirmatory factor analysis.
When X 2 /sd ratio calculated as a result of CFA is lower than 3, it indicates a perfect match between the data and the original model [36,65]. On the other hand, Sümer [62] states that when this ratio is lower than 3, it indicates a good fit and when it is between 3 and 5, it indicates an acceptable fit. In the current study, X 2 /sd ratio obtained as a result of CFA seems to be highly adequate.
According to Kline [36], loading values of 0.60 or more are considered to be high and loading values between 0.30 and 0.59 are considered to be medium. Tabachnick and Fidell [65] argue that the item loading value of 0.32 and above should be regarded as a basic acceptance. Şencan [64] considered the factor loading value to be at least 0.30 as a sufficient acceptance for the item to be included in the scale. However, it was noted that sample size should be taken into account when deciding on this value.
Kim-Yin (2004 cited in [64]); while determining the lower limit of the factor loading value, stated that for 0.30 loading value, there should be at least 350 participants; for 0.40 loading value, there should be at least 200 participants; for 0.50 loading value, there should be at least 120 participants and for 0.60 loading value, there should be at least 85 participants. In light of the values given above; given that the pilot study was conducted on 645 participants, inclusion of the items with factor loading of 0.30 or above won't yield any statistical problems.
According to Table 6, the Normed Fit Index (NFI) is one of the good fit indices which are based on the comparison of the chi-square value of the relevant model with the chi-square value of the original model and influenced by the sample size. When this value is higher than 0.90, it indicates a good fit [65,60]. In the current study, it is seen that the NFI value calculated as 0.94 is larger than the 0.90 value which is considered to be the lower limit of good fit, and even it is larger than NFI value possessed by the original scale. Thus, it can be argued that NFI value indicates a good fit. As it has already been noted, NFI value is affected from the sampling size. In this case, the NNFI value that is not affected by sample size is also taken into account. When the relevant literature is examined, it is seen that NNFI value approaching 1 increases the degree of a model's exhibiting a perfect fit [65,62]. In the current study, NNFI value was calculated to be 0.97. Therefore, it can be said that the NNFI value of the adapted scale has a very high value.
The GFI value is a fit index that gives information about the extent to which the model measures the covariance matrix in the sample. Unlike the Chi-square (X 2 ) value, it is a descriptive fit index that is not affected by the sample size. Moreover, another fit index is AGFI. Both the GFI value and the AGFI value are between 0 and 1 [65,62]. However, when the AGFI and GFI values are higher than 0.95, they show a perfect fit, while values between 0.90 and 0.95 show a good fit. However, when the literature is examined, it is seen that for AGFI fit index, the value between 0.85 and 0.90 is regarded to be acceptable [71]. Thus, it can be argued that in the current study GFI value calculated as 0.91 indicating a good fit and AGFI value was calculated as 0.89 indicating an acceptable fit.
Another goodness-of-fit index is CFI. This index also affected by the sample size can take a value ranging from 0 to 1. However, it is desirable to be higher than 0.90 in order to be able to say that it is a good fit, [65] or even higher than 0.95 in order to be able to speak of a perfect fit [62]. The CFI value calculated as 0.97 in the current study shows a perfect fit.
RMSEA tests for covariance differences between the universe and the sample. Contrary to goodness-of-fit indices, it is expected that the value obtained in this index type will be close to 0. The value close to 0 indicates that there is no difference between the sample and the universe covariances. When the relevant literature is examined, it is seen that when RMSEA value is lower than 0.05, it indicates a perfect fit [60,62]; when it is between 0.05 and 0.08, it indicates a good fit [33,62]. In this connection, the calculated RMSEA value for the current study is smaller than the value of 0.05 which is considered as the perfect fit limit. Thus, it can be said that there is no covariance difference between the universe and the sample of the present study.
CFA analysis was conducted to establish the validity of the Turkish adaptation of USLS. Fit indices obtained on the basis of CFA results indicated the same model as the original scale; thus, it can be argued that the Turkish version of the scale is valid. Therefore, this scale can be reliably used to contribute to the relevant literature in Turkey. The Cronbach alpha internal consistency coefficients, which were calculated for both the subscales of the scale and the whole scale, show that the Turkish version of EFOÖÖ has a reliable and consistent structure [26]. The values of the fit indices calculated for the current study are summarized in Table 8 together with the evaluation criteria.

Conclusions and Suggestion
Following the translation of the Universal Science Literacy Scale into Turkish, linguistic equivalence application was conducted. Both the original English form of the scale and its Turkish version were administered to the students during the linguistic equivalence application and the correlation coefficient between the students' responses given to the original form of the scale and the Turkish version was found to be considerably high. Moreover, significance test was carried out revealing that there is no significant difference between the two scales. The results of the two analyses showed that the meaning derived from the items in the original scale by the participants is considerably similar to the meaning derived from the items in the Turkish version.
Following the linguistic equivalence, a pilot study was conducted to determine the factor structure of the scale. The sample of the piloting is composed of pre-service science teachers who are studying at different levels of education from the first year to the fourth year. Confirmatory factor analysis (CFA) was performed on the data obtained from the pilot application to test whether the translated scale has a similar factor structure with that of the original scale. The fit indices calculated as a result of CFA showed very good values. It was therefore determined that the original factor structure of the Universal Science Literacy Scale and the factor structure of the adapted scale fit with each other. For this reason, it has been determined that the Universal Science Literacy Scale is a valid scale for use in the literature.
The internal consistency coefficients calculated for both the sub-scales and the whole scale revealed that the scale has a consistent and reliable structure at a desired level.
USLS whose adaptation process is successfully completed is suitable for the cognitive and affective development of all students at university level. Therefore, all the academicians, teachers and field specialists who will conduct research to determine the literacy level of students can use the Universal Science Literacy Scale.