Introducing the Thought-Listing Technique to Measure Affective Factors Influencing Attitudes toward Science

Until now, most of tests for attitudes toward science have been conducted through structured questionnaires. The test with structured questionnaire is difficult to exclude the intention of researcher. This study focuses on the need for a more detailed assessment of the sub-elements of attitudes toward science and unstructured open tests. For the purpose of this research, the attitudes toward science were divided into cognitive, behavioral, and affective factors, with a specific focus on affective factors. An unstructured, open, thought-listing test was conducted to measure affective factors influencing attitudes toward science. In the first test, 522 students participated in a thought-listing test on the open topic of “science,” and the correlation between participants’ test scores and science achievements were analyzed. The results revealed that among the categories inside, science area, science learning, science teacher, scientist, the score for science learning had the highest correlation with science achievement. As a result, the second thought-listing test, conducted with 102 middle school students, was limited to the topic of “science learning in the classroom.” The resulting correlation between the test score and science achievement was .257 of Pearson correlation coefficient, which was a higher correlation than that of the first test. This value is similar to or slightly higher than the relationship between science achievement and attitudes toward science found in previous studies. These findings suggest that the thought-listing technique has advantages over structured questionnaires, as it is easier to conduct, excludes researcher bias, and is a valid method to measure affective factors influencing attitudes toward science.


Introduction
In an era when national competitiveness depends heavily on scientific and technological power, there is a growing emphasis on the importance of science education. In addition, the science community has increasingly emphasized the significance of scientific literacy among the general public (Brossard & Shanahan, 2006;DeBoer, 2000;Jarman & McClune, Lee, 2014;Mende et al., 2012;Nisbet & Scheufele, 2009;Norris & Phillips, 1994;NRC, 1996;van Eijck & Roth, 2010). However, low admission rates to university science and engineering courses is becoming a growing problem around the world. In this regard, attitudes toward science are widely regarded as the most important factor influencing scientific achievement, entrance into science and engineering courses, employment in science-related fields, and the creation of effective science policy (Fraser, 1978;Freedman, 1997;Simpson & Oliver, 1990). In particular, the relationship between attitudes toward science and science achievement has long been of concern to scholars (Bloom, 1979; Kan & Akbas, 2006;Willson, 1983). A number of studies have revealed a significant correlation between attitudes toward science and science achievement. However, most of these studies have a low correlation value of between 0.1 and 0.3, limiting their application to further discussion. Papanastasiou and Zembylas (2002) argue that these limitations were due to a lack of a theoretical framework and uncertainty regarding the ability of the research methods employed to reveal attitudes toward science. The aim of this study is to examine the possibility of introducing a new method for measuring attitudes toward science. Existing methods reveal the following two weaknesses: The first is that the attitude toward science has been treated comprehensively and the second is that the use of structured questionnaires generally includes some researcher bias. Therefore, this study sought to find a method that could exclude researcher bias and focus on the sub-elements of attitudes toward science. Anderson (1988) classifies attitudes toward science into cognitive, behavioral, and affective factors. Of these, affective factors have been known to have the highest degree of correlation with attitude toward science score (Lee, 1998). Kan and Akbas (2006) emphasized that affective factors, which refer to an individual's feelings or affections, are the most important variables among the elements influencing science achievement. Therefore, the research presented here explored the affective factors influencing attitudes toward science.
The thought-listing technique is an open, self-confessional test, used to explore the structure of human cognition (Cacioppo & Petty, 1981). This technique was created based on the premise that the psychological importance of how one thinks and feels can be reviewed by analyzing the content of thoughts, ideas, images, and feelings reported by individuals. Therefore, it is an appropriate tool for assessing the rapidly changing thoughts, ideas, and emotions of individuals. In addition, numerous studies have verified the validity of this test (Heimberg et al., 1990;Huber & Altmaier, 1983;Blankstein & Flett, 1990;Kolligian & Sternberg, 1995). Because it is an unstructured open test, it has the advantage of avoiding researcher bias. Therefore, the purpose of this study was to explore the feasibility of using the thought-listing technique as a method to test the affective factors influencing attitudes toward science. To this end, this study aims to examine the most important affective factors and verify their correlation with science achievement. Specifically, it has been checked that which of the positive and negative affection is highly correlated with science achievement, and also what kind of thought associated with science is highly correlated with science achievement.

Study #1
The following steps were taken to conduct study 1: the test instrument was developed, the test was implemented, categories of student responses were set, coding schemes were determined, responses were coded, and results were analyzed.

Test Instrument and Participants
For the test instrument, Cacioppo and Petty's (1981) thought-listing test was utilized. Students were instructed to write down 10 sentences containing ideas that came to mind when they thought about "science." They were asked to write these ideas in sequence, starting with the first box. Although they were encouraged to fully contemplate or write as many ideas and sentences as possible, writing all 10 was not mandatory. Participants consisted of 522 students from Daegu, South Korea, 120 of whom were elementary students, 206 middle school students, and 196 high school students.
Each student's end of semester science subject score was regarded as their science achievement. Because raw scores differed depending on school and educational level, standardized scores were calculated for further analysis. Table 1 shows the raw science achievement scores. The standardized mean score of each group was 50 and the standard deviation (SD) was between 9.85 and 9.87.

Analysis
Student responses were categorized based on the theme of each idea (Table 2). First, they were classified based on whether the theme of thinking related to respondents'' internal or external factors. In cases where the theme was external, the idea was further divided into science area, science learning, science teaching, and scientists. After the categorization was complete, each response was further divided into a "positive (+)" case or "negative (−)" case. For example, a respondent's internal feelings with positive content was coded as I+, while a science teacher theme with negative content was coded as T−. The following are some coding examples: I am good at science: (I+) Science makes our life convenient: (S+) I hate science teacher: (T−) Two experts were consulted to provide judgments on the positive and negative coding decisions. If their judgments differed, they were asked to conduct another review for adjustment. Once coding was complete, students' responses were counted in each category, and the counted frequency of each category was regarded as the category score. Positive and negative responses were scored separately for each category. Finally, the correlation coefficient was calculated for students' category scores and science achievement.

Results
To examine the relationship between affective attitude toward science and science achievement, the correlation coefficient was calculated. First, the influence of positive affection and negative affection on science achievement was examined, and a significant correlation was found between positive affection toward science and science achievement. No such correlation was found for negative affection (Table 3). In Table 3, the total number of positive responses is the sum of every positively coded response in all categories and the total number of negative responses is the sum of every negatively coded response in all categories. This result is contrary to the common psychological phenomenon in which negative affection has a more critical influence on attitude or behavior than positive affection. Therefore, it can be interpreted that positive affection has a higher impact on science achievement than negative affection. The results in Table 3 suggest that investing classroom time and effort in encouraging students to have positive attitudes toward science is more effective in enhancing science achievement than investing in eliminating negative affections. Next, the correlations between the detailed scores of each category in Table 2 and science achievement were analyzed. This showed a significant positive correlation between the number of internal affection responses and science achievement (Table 4). In the detailed analysis of positive internal responses and negative internal responses, a significant correlation was only found between positive internal responses and science achievement; in contrast, no such correlation was found between negative internal responses, such as "I hate science" and "I am not good at science," and science achievement. This differs from having a positive feeling for science and voluntarily expressing it in an open test situation. Even if a student believed that he or she did well in science, this belief might not be expressed in the thought-listing test. Writing positively about themselves in the thought-listing test not only meant that a student had positive emotions about science but also felt a strong internal connection with science. A significant positive correlation was also found between positive affections toward science learning, including science as a subject or class and science achievement. In contrast, no correlation was found between negative affections toward science learning and science achievement. The results for each school grade group (elementary, middle, and high school) were found to be similar, with only minor differences in correlation coefficients, i.e., .144, .153, and .147. Meanwhile, differences in the responses for each school-level were analyzed, and the result revealed that the higher the grade, the higher the number of responses to science learning, and the lower the grade, the higher the number of responses to science teachers and scientists. However, this did not influence the correlation with scientific achievement.
In addition, the study also found that thoughts and affections as regards science or science teachers and scientists had no significant correlation with science achievement. These results confirm that affections toward respondents' internal thoughts and science learning have a significant impact on science achievement, whereas thoughts and affection toward science teachers and scientists do not have a significant impact on science achievement.

Study #2
This additional test was conducted only for those situations related to the category of science learning to more thoroughly examine the effects of respondents' internal thoughts and science learning on science achievement.

Test Instrument
Study 2 was a slightly modified version of study 1. While study 1 presented only the "science" situation and did not limit the thoughts of students, study 2 presented students with a particular situation, as shown in Figure 1 to limit thoughts toward science learning in the classroom. As with study 1, participants were asked to freely list their thoughts.

Participants
Participants in study 2 consisted of 102 middle school students from Daegu, South Korea, comprising 49 male students and 53 female students, with a science achievement distribution of 33 high achievers, 38 middle achievers, and 31 low achievers.

Analysis
The participants' responses were coded to positive and negative sentences. Affective verbs were extracted from the sentences according to the list suggested in Shaver et al. (1987). Next, verbs extracted as affective expressions were divided into positive and negative categories. Similar to study 1, two experts were asked to record their judgments and if these judgments differed, they were asked to conduct a second review to make the appropriate adjustments.

Results
The students' responses were divided into positive and negative affections, and the number of positive and negative responses was scored by each individual. In addition, the number of affective verbs was separately scored in instances where several affective verbs were included in the same sentence. Scores were awarded for the number of positive sentences, negative sentences, neutral sentences, positive affective verbs, and negative affective verbs, and their correlation with scientific achievement was analyzed. The results revealed that the number of positive affective verbs and positive sentences positively is correlated with science achievement (Table 5). Similar to the results from study 1, neither the number of negative affective verbs nor negative sentences is correlated with science achievement.

Conclusions
In this study, the affective factor influencing attitudes toward science was measured using the thought-listing technique. The correlation between affective attitudes toward science and science achievement was also examined. The results showed that having a positive affection toward one's inner self in relation to science and science classroom learning is positively correlated with science achievement. In addition, the correlation between students' affections toward science and science achievement is slightly higher at Pearson R.257, when the test is limited to a focus on science classroom learning.
The correlation coefficients do not exceed those of the previous studies, ranging from .1 to .3 for the correlation between attitudes toward science and science achievement. However, it is noteworthy that this study's results for affective attitudes toward science alone are similar to those of previous studies. These results suggest a high probability that affective factors influence attitudes toward science. They also show that the thought-listing test is a more reliable method than the development of large-scale questionnaire surveys for measuring students' attitudes toward science. The thought-listing technique has some advantages over other attitude tests as it requires less time and fewer resources than developing a valid and reliable questionnaire. Furthermore, participants are only required to write down naturally occurring ideas, thus avoiding the issues related to reading, understanding, and responding to a large number of posed questions. This study clearly confirms that thought-listing is a valid alternative to questionnaire surveys for research of this kind.