Assessment of Scientific Literacy of Estonian Gymnasium Students during the Operation of a Competence-based Science Curriculum

Sustainable societies need a workforce with a very creative and flexible nature to be able cope with rapidly changing situations making creative and reasonable decisions. The only suggested way to obtain this is through education that is oriented to competences. The goal of this research is to determine the influence of a new competence-based science curriculum on gymnasium students’ scientific creativity skills and socio-scientific reasoning in the context of studying lactose intolerance. In order to investigate the development of these skills at the gymnasium level, a scientific literacy test is administered to 1116 10 grade students (16 –17 year old) and 802 12 grade students (17–18 year old), totally 1918 students in. 44 randomly selected schools from different regions of Estonia, both urban and rural, to the scientific literacy test on a new Estonian competence-based curriculum is conducted in the fall of 2011 with the 10 grade students and the test is repeated with the same students in the spring of 2013 before they graduated 12 grade. The data are gathered with an 8-item test assessing students’ scientific literacy components including scientific creativity and socio-scientific reasoning. The test situation is initiated by a scenario describing an expected visit of a family (mother and triplets) and their food provision problems, which arise because the mother and one of the boys suffered from lactose intolerance. One of the tasks is designed to measure scientific creativity skills and one task to assess socio-scientific reasoning skills. Responses are scored from 0 – 3 points per task. The results of the study show that the new competence-based science curriculum has not yet affected significantly the students’ learning outcomes in enhancing competences associated with scientific creativity and socio-scientific reasoning skills. Studies during gymnasium level have not supported the students’ ability to transfer biological and chemical knowledge into everyday situation effectively.


Introduction
During the past few decades there has been much international discussion about the need to make changes in secondary school science. Two main arguments have been put forward to justify change, namely, the decrease in student interest in science [1] and the need to broaden the curriculum with more emphasis on the competencies and transferable skills rather than academic knowledge [2].
In addition, there is a need for a scientifically literate workforce and for citizens who are provided with the appropriate skills and knowledge [3,4,5].
In 2011, Estonia introduced a new competence-based curriculum, intended to initiate a paradigm shift from memorization of knowledge to competences and transferable skills [6]. The goals for science education were specified as fostering scientific literacy through: problem solving, decision making, reasoning and creative thinking skills. This study is focused on the development of socio-scientific reasoning and scientific creativity skills.
The role of science education in promoting scientific literacy is to offer support for the ideas that innovation plays the central role in national success and fostering innovation requires a supportive social environment [7]. According to Kurt Heller [8] the hypothetical construct 'scientific ability' can be defined as scientific thinking potential, or as a special talent for excellence in the (natural) sciences. Similar to this definition, 'scientific creativity' can be conceptualized as individual and social capacities for solving complex scientific problems in an innovative and productive way. Recently suggested ideas for increasing the interest towards learning science, including biology, is through making use of socio-scientific issues [9]. Socio-scientific issues (SSI) have been proved to be effective in developing students' creative problem solving, decision making and socio-scientific reasoning skills [10,11]. The use of socio-scientific issues (SSI) has been effective both for engagement of students learning science [12,13] and also for development and measurement of 21 st century skills [14] needed for future workforce [15].
Although scientific creativity is considered to be crucial in solving complex scientific problems, on the other hand the most powerful long-range predictors of professional success in science are apparently domain specific problem-solving abilities, motivation and social leadership competences, while general intelligence and creativity tests have had the lowest prognostic value [8].
Scientific reasoning is an important component within cognitive 21st century skills and is highly emphasized in the new science education standards [16]. The addition of a social component to the reasoning increases the relevance of decision making for students as well as fostering their divergent approach to socio-scientific decision making and its reasoning that includes social aspects.
With the emergence of increased attention to competences required by citizens for a knowledge-based society, schools and educational systems around the world were called upon to make changes to their curricula [17]. In Estonia, the reform in science curriculum was implemented in 2011 and one of its important goals was to develop students' scientific literacy, including creative problem solving and socio-scientific reasoning of decisions.
In 2011, Estonia introduced a new competence-based science curriculum, intended to initiate a paradigm shift from memorization of knowledge to the development of competences. The goal for the current research was to assess the possible change of students' scientific literacy components in terms of scientific creativity and socio-scientific reasoning skills during the operation of a competence-based science curriculum. According to this goal the following research questions were put forward: 1. Is the created assessment test generalizable for determining the levels of scientific creativity and socio-scientific reasoning of Estonian gymnasium students? 2. Has the competence-based science curriculum been effective in increasing the students' scientific creativity and socio-scientific skills? This is the first and only study in Estonia to evaluate the change of scientific literacy among the students throughout the gymnasium studies.

Study design
This was a longitudinal comparative study for investigating the possibility to assess the influence of a new Estonian competence-based science curriculum on gymnasium level students' scientific literacy, measured in the form of scientific creativity and socio-scientific reasoning skills. The scientific literacy test was conducted with 1116 10 th grade students in the year 2011 when the Estonian new competence-based curriculum was implemented and the same test was carried out again with 802 students after three years in 2013 when the same 44 schools' students had reached grade 12.

Sample
The research was conducted on a regional representative sample of 44 different Estonian schools with 1116 10 th grade (boys 519 and girls 597; 16 -17 years old) and 802 12 th grade (boys 386 and girls 416; 17-18 years old) students (N=1918). The study was designed on the basis of schools and from different regional lists of schools, every 4 th was selected to get a random representative sample.

Instrument
For measuring the level of scientific literacy of gymnasium students, a complex interdisciplinary assessment test was designed in the context of lactose intolerance and the two above-mentioned skills of scientific literacy identified in our framework were a focus of the current paper.
The part of the instrument used for this study was based on a socio-scientific contextual situation of lactose intolerance that consisted of 8 tasks which presumed the abilities of transferring the biology knowledge into a new everyday situation. The situation was based on an expected visitation of a family (mother and triplets -two boys and a girl) and their food provision problems arising because one of the boys (and the mother?) suffered from lactose intolerance. Two of the tasks were designed to measure the following aspects of divergent thinking: (1) scientific creativity and (2) socio-scientific reasoning skills. The task of scientific creativity asked for possible privileges of lactose tolerance for human beings during evolution and the different answers were coded as follows: (1) no answer or biologically wrong -0 points; (2) one possible adequate privilege -1 point; (3) two possible adequate privileges -2 points; and (4) three or more possible adequate privileges of lactose tolerance -3 points.
Similar coding was used for the task of socio-scientific reasoning, where students were asked to reason their decision made about creating or not creating special menus for lactose intolerant people in private restaurant: (1) no answer -0 point; (2) just describing the decision -1 point; (3) reasoning the decision from one aspect -2 points and (4) reasoning the decision from two or more aspects -3 points.

Data Analysis
The data analysis was performed with Microsoft Excel and IBM SPSS Statistics 22. In addition to descriptive statistics (average, percentage), the nonparametric Wilcoxon Signed Ranks Test was used to assess the statistical significance of differences between groups. In case the difference was statistically significant, Cohen's D for T-Test (Independent Samples) was calculated, using the online Calculator from the Social Science Statistics homepage [18] to assess the magnitude of this difference.
Assessment of Scientific Literacy of Estonian Gymnasium Students during the Operation of a Competence-based Science Curriculum

Results and Analysis
The increase of mean results of students' scientific creativity skills was statistically significant (Z = -7,39, p = 0,000) and as the size effect was 0,34,it can be considered a real increase which could be meaningful in a pedagogical sense. The results of the socio-scientific reasoning task increased also during the three years of gymnasium studies, but this increase was not statistically significant (Z = -1,03, p = 0,301). As the mean results were lower than half of the maximum points available to achieve, it was obvious that the level of scientific creativity and socio-scientific reasoning skills were, for majority of students, very low, even by the end of gymnasium school.
The mean results (Table 1) showed that in the 10th grade the students' scientific creativity skills were considerably lower than the skills of socio-scientific reasoning but during the gymnasium studies, with the new curriculum, the scientific creativity and socio-scientific reasoning skills levelled off indicating that the new curriculum was effective only in one investigated aspect (scientific creativity).
In comparison of male and female students' scientific creativity and socio-scientific reasoning skills, the girls outperformed the boys in both skills within both grades and the differences were statistically significantly, although the differences had no crucial meaning, as effect sizes (Table 3) did not show enough magnitude of this effect. The girls' results of both scientific creativity and socio-scientific reasoning skills increased in both grades, while the 12 th grade boys' socio-scientific reasoning skills did not increase statistically, or meaningfully. 1,49 (0,71) 0,23 -9,56 0,000** 0,29 *Significant difference between the groups at the 0,05 level of confidence ** Significant difference between the groups at the 0,01 level of confidence  *Significant difference between the groups at the 0,05 level of confidence ** Significant difference between the groups at the 0,01 level of confidence   The current study gave a preliminary overview of the levels of students' scientific literacy in two aspects. Table 4 illustrated that a large number (729) of students stayed at lower levels (1 and 2), both in scientific creativity and socio-scientific reasoning scores. At the same time, a considerably smaller number (410) of students had achieved higher scores (2 and 3) in both scientific creativity and socio-scientific reasoning tasks. Only 21 students (1,1 %) out of 1918 in both components of scientific literacy attained maximum results (3 points) and such a result was certainly unsatisfactory.
The Figures 1 and 2 illustrate the detailed changes of students' competences, showing that the distribution of students test results followed different patterns: the results of the scientific creativity skills distribution moved more significantly in the direction of higher scores. The results for Assessment of Scientific Literacy of Estonian Gymnasium Students during the Operation of a Competence-based Science Curriculum the distribution of socio-scientific reasoning skills maintained the general pattern, except for the changing proportion of the lowest and highest scores.

Conclusions
Creating and using the assessment instrument, based on lactose intolerance, in order to map students' learning outputs from science studies was shown to be suitable for measuring students' scientific creativity and socio-scientific reasoning skills and their changes during gymnasium science studies. This outcomes was similar to that obtained by Zhou et al. [16] who described the effectiveness of task context in assessing reasoning skills The results of our study indicate that in a similar fashion to other previous studies investigating the scientific creativity and socio-scientific reasoning skills of students [14], there are problems in students' achievement. Although the new competence-based curriculum is intended to improve students' so called 21 st century skills, it is a slow process [17].
This study showed gender differences in achievements of important components of scientific literacy were better achieved and developed by female students. Although there was a statistical increase, both in scientific creativity and socio-scientific reasoning skills of both genders and both grades (except 12 th grade male students' socio-scientific reasoning skills), it cannot be considered as a meaningful change due to the low magnitude of the statistic.
As a conclusion, it can be said that the instrument was suitable for assessment of scientific creativity and socio-scientific reasoning skills. The estimated low levels of investigated skills show, like in previous studies [10], which fostering scientific creativity and socio-scientific reasoning skills are still undervalued in science classes. The new curriculum has only been implemented for several years and many teachers are not yet prepared to foster these skills by purposefully assessing them, even though the curriculum has been updated for this purpose.
The new Estonian curriculum [6], focusing on the development of students' scientific literacy skills measured in the aspects of divergent thinking (scientific creativity), had a significant effect in scientific creativity skills but not a considerable effect on achievement on students' socio-scientific reasoning skills.
The mean results of the test at the beginning of the gymnasium studies showed that the students had obtained a moderate level of socio-scientific reasoning skills already at lower secondary school and did not evolve further during three years of gymnasium level studies. At the same time, the scientific creativity skills developed significantly during gymnasium studies, as the base from previous studies was very low. That's why it is possible to conclude that the focus of the new competence-based curriculum had favored the field of scientific creativity which was not purposefully developed during the earlier science lessons. Although the creativity tests had the lowest prognostic value in comparison with other components of scientific literacy (e.g. problem solving, etc.), according to literature [8], it was still very important in the context of 21 st century skills needed in future careers [14]. Unfortunately there was almost no positive increase in the pedagogical sense during gymnasium studies, in the field of socio-scientific reasoning this being more crucial and more highly emphasized in the new science education standards [16]. The test results illustrate that during three years of gymnasium studies developing of scientific creativity skills is more successful than the enhancement of socio-scientific reasoning skills but as the 12 th grade students' average score for scientific creativity is 1,43 and 1,52 for socio-scientific reasoning skills (out of 3) it cannot be considered a satisfactory result.
The outcomes of the study will be used for developing the new external assessment tool in Estonian science education.