Some Criticisms against Exam Questions: To What Extent Can Mythic, Romantic and Philosophical Questions be Used?

In the present study, some of the pre-service teachers’ criticisms against their exams were investigated. Moreover, as an alternative, to what extent philosophical, romantic and mythic questions could be used was also looked at. The study group consists of 117 pre-service teachers from the classroom teacher education. In the study, it was investigated what criticisms directed to exams by the pre-service teachers are the most important ones. Moreover, a form consisting of 20 items was administered to the study group to test mythic, romantic and philosophical understanding. The aim of this form is to elicit the pre-service teachers’ opinions about the questions testing mythic, romantic and philosophical understanding. The present study employed a mixed method as it used both quantitative and qualitative content analysis techniques. Incorporation of romantic, mythic and philosophical questions into both exams and teaching process may serve the function of trying “unusual” questions. In addition, use of such questions may have contributions to the consideration of affective components as well as cognitive components of teaching.


Introduction
The main purpose of education in early years of the Republic was to train individuals comprehending the philosophy of the Republic and approving reform movements so that the Republic could be developed. Moreover, with the adaptation of the Latin alphabet in 1928, other primary aims of the education were to increase the number of literate people and raise the awareness of the public. In this period, high majority of people were living in the countryside, low levels of literacy and shortage of teachers resulted in an emphasis on quantity rather than quality. In the Republic era, various means of training qualified teachers was tried. Particularly after 1946, political instability was experienced and quality was sacrificed in favor of quantity in teacher training. The main emphasis of teacher training programs was on the quantity rather than improving the system and creation of policies directed to the quality of the teacher. In this respect, many reformation attempts focused on which courses should be included in the program and the balance between theoretical and practical courses.
In 2006-2007 academic year, teacher education programs were redesigned. In the "new" program, the aim is to train intellectual teachers solving problems and learning to learn instead of training technician teachers [1]. Moreover, as a prerequisite of the constructive philosophy of new primary school programs, while the programs are being implemented, it is of great importance that first students should be provided with experiments and experiences and based on these they learn concepts and definitions. On the other hand, another factor that should be taken into consideration is to associate the subject topics with the content of the curriculum developed by the Ministry of National Education for the given grade level and they should be enriched with the examples from the daily life. In the faculties where teachers are educated in Turkey, courses and course contents complying with the structure required by the constructivist approach have been incorporated into the curriculums of these faculties. However, it can be argued that most of the prospective teachers are educated within the applications that are in line with pedagogy than can be called "behaviorist" in their own learning experiences [2].
In recent years it has been desired for the programs in the education system in Turkey to move from the behavioural linear approach to the more flexible constructivist approach. When the old programs and program applications are examined, the critiques on the dominance of the teacher-centred and behavioural approach generate the basis for preparing a new program [3].
Teachers in Turkey are recruited mostly in state schools. Teachers working in state schools spend most of their time on the cognitive development of students. They also spend some time voluntarily after classes and on their students' academic development. Exams, selection and appointment have an important place in the Turkish education system [4]. As for teacher selection and appointment in Turkey, there are two institutions training teachers. One of them is teachers' college. The students having a diploma from a teachers' college can work as a teacher in any part of the country if they pass the general teacher appointment exam. Each year the Turkish Ministry of National Education (MONE) appoints certain numbers of teachers from different branches. In addition, graduates of the other faculties -mostly-science and letters-can be appointed as teachers if they complete an initial teacher-training program. This teacher formation education is given through courses organized by teachers' colleges to a certain number of faculties. Pre-service teachers need to take the Public Personnel Selection Exam (PPSC) after graduation to be appointed as teachers. When the exams taken by pre-service teachers within the PPSC are examined, it is seen that different fields such as general teaching methods, assessment and evaluation, education psychology and classroom management are tested through multiple-choice questions. However, in 2013, in addition to these fields, subject exams were also added (English, Chemistry etc.). In the Turkish education system, there is a period for internship but the appointments of the teachers in Turkey are made through a central exam formed of multiple-choice items.
It is seen that there are many classifications of alternative questions in the related literature. Tofade, Elsner and Haines [5], classified questions as convergent, divergent, focal and funnel. Convergent questions are questions having original answers and not providing many options. Divergent questions, on the other hand, allow the expression of different viewpoints and reactions. Focal questions allow students to make choices and they are usually constructed as "Do suggest? Why? If not, why?" to encourage students to brainstorm so that different viewpoints can be elicited. Funnel questions start with asking more than one general question and lead to more focused investigation. In such questions, few successive questions are followed by a focus on more specific questions. Pearson and Johnson [6], classified questions under three headings as textually explicit (TE), textually implicit (TI) questions and scripturally implicit (SI) questions, which are questions that cannot be answered independent of the reader's opinions and beliefs [7].

Preferences for Questions and Some Results of These Preferences
Types of the questions asked naturally affect the learning and teaching process and outcomes of this process. Preferences for the types of questions to be asked can be viewed as preferences for teaching process and outcomes.
These preferences at the same time are our preferences for learning and teaching process. In this regard, rather than the types of questions, the types of outcomes to be obtained from exam questions should be investigated. Our questions determine what types of outcomes to be obtained. H.Kissinger says that "Does anyone have any questions for my answers?" Colin Powell states that "Always remember: They get to ask the questions, but you get to give the answers" [8]. Through questions, it is possible to overcome problems [9].
Can similar considerations be met with unusual cognitive tools (mythic, philosophical and romantic understanding) as stated by Egan. Egan states that [10] in general most of the educators believe that making connections with the prior knowledge while presenting a new material is the best way of teaching. Though Egan [11], does not reject the importance of students' prior knowledge and experiences, he sees their dreams as a better starting point in learning. Egan [12], argues that making clear-cut distinction between daily experiences and world of imagination should be avoided as they complement and support each other when necessary. Egan is against a view only emphasizing experiences. He thinks that when experiences are excessively emphasized, the meaning may get lost. After shortly summarizing what mythic, romantic and philosophical understanding is, some suggestions will be made about possible questions to be asked about these kinds of understanding.

Mythic, Romantic and Philosophical Understanding
Mythic understanding, according to Egan [13], is most effective during a period starting when the language is developing grammatically at the age of 2-3 and extending up to ages of 6-7-8. In mythic understanding, there are efforts for understanding by internalizing information. In these efforts, tools such as binary oppositions (public-private, male-female etc.), fantasy (rearranging the rules of the world), metaphor (simile, meanings derived from a very different thing about a thing), rhythmic narration, images (internalization of the content by visualizing in mind) are used.
Romantic understanding is a means of understanding the world through exotic, strange and mysterious experiences [10]. Knowledge is not only associated with the prior knowledge, it is also connected with fears, concerns, hopes and expectations [10]. In learning and teaching process, they should also be taken into consideration. Hadzigeorgiou argues that romantic understanding requires students to use their imagination and questions can be a means of challenging. He thinks that students' romantic understanding of the world can answer questions such as what is significant, what is meaningful [14].
Philosophical understanding, covers the late years of secondary education and higher education period. Following criteria are shown to be related to philosophical understanding [13]:  Making abstractions,  Dramatic presentations of human, theories and ideas,  Grasping the dramatic nature of theories,  Explaining theories clearly and strikingly,  Clarification of ironic suspicion and clues,  Philosophical principle, researching general laws operating among phenomena… In the content of the philosophical understanding, there is the opening the doors of the world of ideas and writing essays. Moreover, unbiased works, independent mind games are seen as the outcomes of this understanding in philosophical understanding and in this way, rhetorical skills are improved.
According to Egan [15], after getting through mythic and romantic levels, other levels can be reached. Egan points out that dreams are important from historical point of view. Egan [15], suggests evaluation forms for mythic and romantic understanding. These are story form in mythic understanding and romantic form in romantic understanding. In romantic understanding, students grasp more details than they do in mythic understanding. Egan emphasizes the importance of mythic and romantic understanding: "… In history programs, after mythic level comes romantic level. History should teach great stories in human struggles such as repression versus freedom, treat versus security and ignorance versus wisdom. On romantic level, more details are obtained through special examples. The aim should not be only to get information about Alexander the Great but to inspire admiration for their courage and energy. While getting introduced to heroes, students find opportunities to learn about their struggles, adventures and conflicts they are confronted with and thus feel themselves more like cultural beings" [15].
What can contribute to students' understanding? Narrative texts are more conducive to students' understanding than expository texts [16].

Different Approaches -Tools-for Mythic, Romantic and Philosophical Understanding
There is no single tool to evaluate students' understanding. The key to successful teaching is focusing on students' dreams. In his study about deep learning, Egan [17], explains an event reported by Gardner. In this event, it is stated that students from the physics department of Harvard experience difficulties with questions related to basic principles. When the questions are asked in different contexts, students have difficulties. Same problem is observed in other disciplines. If students are not involved in the topic, such problems can be experienced. In addition to traditional ways of assessment, alternative teaching settings and assessment tools should be used in such a way as to arrange exam questions to make students engrossed in the topic during learning and teaching process.
Developing technologies have been changing the ways of assessment. With the advancement of new technologies, strong multimedia format such as video, voice and virtual reality will be able to be integrated into new ways of assessment [18]. Romantic understanding is related to logical explanation of an issue and represents the talent of a person in relation to this issue. Moreover, through romantic understanding, television, radio and video games can be used together with books and course notes as means of thinking [19]. Fettes [19], suggests following criteria in relation to romantic understanding: The criteria set in relation to mythic, romantic and philosophical understanding are as follows: Mythic understanding is constituted by attempts to understand such as revelation of conflicts, formation of cognitive image, play, drama and use of metaphors. Romantic understanding includes attempts to understand such as creating relationships with heroes, changing the content, extreme experiences and arousing curiosity etc. Philosophical understanding is related to general ideas and anomalies, search for authority and truth. Some sample questions relating mythic, romantic and philosophical understanding are given below:

Mythic
Why do people talk of "going to the country" on a holiday (especially if they live in the city)? What kinds of things does one find in rural versus urban areas? [20].

Romantic
Is there an ultimate, indivisible unit of matter? [21]. Did you know?? -Water is the only substance that occurs naturally on Earth in all three physical states of matter -solid, liquid, and gas [22].

Philosophical
Questions/thoughts-What is James Heartfield describing when he describes work as "an expression of your own self"? What does it mean to have creativity present, but blocked off? State whether you agree or disagree with these ideas (above) and why? [22].
Through technology, imaginations can be used in teaching [23]. Imaginations are important in mathematics teaching [24]. Egan [25], proposes to look at the history through mythic thinking and argues that imaginations can play a role conducive to the understanding of the meaning of past events. From this viewpoint, following questions including fantasy, play, revolt, idealism and comedy can be included in the process: In the present study, the pre-service teachers' most important criticisms against these questions were investigated. Moreover, the pre-service teachers' opinions about the applicability of the criteria developed in relation to these questions targeting mythic, romantic and philosophical understanding were sought. Divergent, textually implicit and scripturally implicit questions can be given as examples of questions different from the traditional ones. Designing of learning setting, planning of learning process and our desired learning materials are among the factors affecting our exam preferences. It is commonly agreed that learning and teaching process should be designed according to students' past experiences. In this regard, what matters most in terms of the process and product is information and experience. Though in the formation of learning product, prior experiences are of great importance, some importance should also be attached to the consideration of students' excitements, enthusiasm and longings, in short, affective characteristics that make students human should be taken into consideration while designing learning settings.
In the current study looking at pre-service teachers' attitudes towards another classification of questions as mythic, romantic and philosophical, the aim was to determine the extent to which mythic, romantic and philosophical questions can be incorporated into teaching and assessment processes. For this purpose, some criterion statements were constructed by reviving the works of Egan written about mythic, romantic and philosophical understanding. Based on these criteria, a 20-item questionnaire was developed. The goal of these items was to elicit the pre-service teachers' opinions about the extent to which these types of questions can be used in teaching and assessment processes. In this regard, answers to the following questions have been sought for in the research: 1. Can questions related to romantic, mythic and philosophical understanding be used during teaching process and as exam questions?
2. What types of questions do teacher opt for?

Method
The present study employed a mixed method as it used both quantitative and qualitative content analysis techniques. "At its basic elements, mixed methods research involves collecting both quantitative and qualitative data (in response to quantitative and qualitative research questions), the merging, linking, or combining of the two sources of data, and then conducting research as a single study or a longitudinal project with multiple phases [26]." It is found essential to include the factors of asking questions such as; deriving questions from drama activities, deriving questions from stories, asking similarities and differences through puzzles and mind games, curiosity-raising questions, asking odd and interesting questions, asking questions from life stories, asking questions attaching importance to ideas and opinions Asking questions attaching importance to ideas and opinions. The findings from the literature, as given in Table 1, show that criteria of the studies on Egan's mythic, romantic and philosophical understanding.

Study Group
The study group of the current research consists of 107 pre-service teachers from the department of classroom teacher education of an education faculty. The participants of the present study are 3rd and 4th-year students from the department of classroom teacher education. Within two or three years, they will be able to work as teachers at elementary level. The pre-service classroom teachers were selected as the study group because these students will be teachers who will teach mathematical information, language skills, reading, writing and social skills and they will not be restricted to specific fields. In the present study random sampling method was employed and the pre-service teachers' opinions were elicited by asking them to what extent they think that the criteria developed by the researcher in relation to mythic, romantic and philosophical understanding could be used in exams and/or learning and teaching process. Moreover, the participants were asked to develop some criteria in relation to exam questions asked to them and they were also asked open-ended questions to determine the priorities in the content of these questions. In this way, their opinions were solicited and analyzed.

Research Process
One of the aims of this study was to reveal what the pre-service teachers understand from the content of a good question and what their criticisms against the questions asked to them are. In fact, what do the pre-service teachers understand from a good question and how do they describe the questions they encounter? Moreover, what their opinions are about the quest for alternative questions (i.e. mythic, romantic and philosophical understanding) was investigated. The students were given a form developed by the researcher concerning mythic, romantic and philosophical understanding. While preparing these questions, the relevant literature was revived and on the basis of this literature review, items were formed. Moreover, split-half reliability was tested for the questions related to romantic mythic and mythic understanding and the reliability value was found to be .86. Moreover, the current article was examined by Egan considered to be one of the authorities in the field. Furthermore, they were asked to describe what the questions they frequently encounter are and what the content of a good question should be. The pre-service teachers responded to the questions on a voluntary basis. The responses of 107 pre-service teachers given to the open-ended questions were analyzed by putting them under themes. Throughout this process, all of the responses of the pre-service teachers given to the open-ended questions were read and analyzed. Then, their opinions about the questions directed to them and the content of a good question were put under themes. Whether the opinions were put under the correct themes was asked to some colleagues. They were also asked under which themes some of the criticisms against the questions randomly selected should be collected. In this way, inter-rater consistency was checked and high consistency was found.

Results and Discussions
1. Can questions related to romantic, mythic and philosophical understanding be used during teaching process and as exam questions?
The pre-service teachers' reactions to some criteria related to romantic, mythic and philosophical questions are presented in Table 1. The data presented in Table 1 show that the highest percentage of pre-service teachers (80%) prefer the inclusion of visuals in questions directed to mythic, romantic and philosophical questions. The pre-service teachers think that the questions asked for the discovery of the curiosity of scientists and interesting, surprising and fascinating questions should be asked during the teaching process in the class rather than in exams. They think that the questions interrogating experience (65%), allowing active interaction with the world (63%), attaching importance to opinions and ideas (66%), opening the door of the world of ideas (70%) and allowing flexible answers (63%) should be asked both during teaching process and in exams.
All of the pre-service teachers preferring mythic, romantic and philosophical questions to be asked in both teaching process and exams think that such questions are better to be asked in teaching process. Moreover, they prefer interesting, odd and curiosity-raising questions in exams. The ratio of the pre-service teachers thinking that mythic, romantic and philosophical questions should not be used in both teaching process and exams ranges from 5.6% to 15.9%. This shows that is not easy to give up old habits and we overestimate objective evaluation. During the non-systematic interviews conducted with the pre-service teachers, when they were asked why they do not prefer alternative questions in the instructional process; that is, why alternative questions are less preferred as exam questions, they provided the following responses: Not each student can put up with this; for example, introvert students may not be encouraged enough to participate in drama or role-play activities. Thus, for such students learning can be more difficult and difficulties can be experienced in the evaluation of students. The students usually give positive responses to the question "Can questions be asked from the stories?".

What types of questions do teacher opt for?
According to pre-service teachers, the questions are conducive to rote learning but not to long-term retention. Another problem emphasized by the pre service teachers is that the exams cannot differentiate the knowledgeable students from those not knowledgeable, the questions are conducive to rote memorization and kill creativity and exam questions are subjective. Types of exams used hinder objective evaluation and make students upset due to great differences seen among the students' achievement levels. Some of the pre-service teachers' opinions about this are given below: …the aim of the questions is not to evaluate creativity, they only require students to show what they have memorized. Hence, the students are directed to learning through memorization and their creativity is killed … (Student teacher D.Ş.) …the system we are exposed to mostly aims to evaluate memorized knowledge. Therefore, we try to memorize our notes before taking an exam and the next day, we forget everything … (Student teacher Ç.K.) … questions are memorization-based questions. Including applied courses, the questions asked in the courses require memorization. Usually the questions asked can be answered without much thinking about them by just remembering. This is the same for applied courses … (Student teacher N. E.) …students only study mid-term and final exams only one or fee days before the exam and memorization is usually enough to pass these exams and within a short time after the exam all the information is forgotten … (Student teacher E. B.) …the exams are not to evaluate knowledge. For many exams, we need to memorize the information given in books. In most of the exams, questions are asked to evaluate unimportant details … (Student teacher E. K.) …Can everybody be sure about the correct answer to the question? In the question, some redundant phrases are used and even if you do not know the subject, you can correctly answer the question …(Student teacher T.I.) … I think that the reliability and fault coverage validity of multiple-choice exams are low. We feel concerned about the possible mistakes to be committed while the teacher is evaluating classic exams. Hence, we prefer multiple-choice questions. On the other hand, reliability and validity of multiple-choice tests can be quite low but if they are designed carefully, they may be discriminating…(Student teacher F.Ü.) … There are some important objectivity problems in exams. As a result, you may get low marks from exams in which you think you have performed well. The teacher evaluates your exam paper according to how he remembers you not according to what you have written … (Student teacher E. K.) … The teacher gives high marks to students giving answers in compliance with the teacher's opinions and gives low marks to those writing things different from what the teacher believes correct. For instance, the teacher asks us to write a composition about a word he/she has selected but then assigns low marks when there are deviations between the student's opinions and his/her opinions… (Student teacher Y. A.) Ermurat, Gumuş, Kurt and Feyatörbay [29], analyzed 1923 questions asked by 14 Science and Technology teachers teaching 6th, 7th and 8th graders and found that 97% of the teachers prefer to ask low level questions mostly at the knowledge level. In another study [4] found that all of the teachers in their sampling use written exams or multiple-choice tests. As they are easy and objective means of evaluation and same types of questions are asked in university entrance exams, teachers seem to prefer multiple-choice questions. Çinici and Demir [31], analyzed totally 970 questions asked in biology course at secondary school level. They found that 555 (57.2%) of these questions are at knowledge level, 259 (26.7%) of them are at comprehension level, 117 (12%) of them are at application level, 38 (4%) are at analysis level and 1 (0.1%) are at synthesis level. The effect of behavioristic approach on Turkish education system is apparent. This effect of behavioristic approach naturally finds reflections on evaluation and exam questions.

Conclusions
All of the pre-service teachers preferring mythic, romantic and philosophical questions to be asked in both teaching process and exams think that such questions are better to be asked in teaching process. Pre-service teachers think that mythic, romantic and philosophical questions should not be used in both teaching process and exams. This shows that is not easy to give up old habits and we overestimate objective evaluation. It is seen that traditional tests are widely used in learning-teaching process and exams in Turkey. Such responses indicate a tendency towards result-oriented and objective evaluations. Criticisms of the pre-service teachers against exam questions were collected through their responses to the open-ended questions.
The pre-service teachers in the study group directed important criticisms against exam questions. These criticisms can be summarized as follows: questions do not comply with question writing rules, there are some problems related to validity and reliability of exams, questions require rote memorization and they do not contribute to the retention of information, they hinder creativity, they are not objectively evaluated, they do not differentiate those knowing from those not knowing. These criticisms can be seen as the products of traditional evaluation and assessment approaches used in their classes.
In literature, it is reported that the questions asked in exams are usually lower level questions; hence, there is a need to make quest about what the content of our questions should be. Despite the subjectivity of mythic, romantic and philosophical questions, they should be included in teaching-learning process and evaluation process. Which one should we opt for; objective elements or subjective elements activating deep understanding and affective factors? The approach assuming that learning is based on prior information has become a classical approach. Though this approach seems to be cognitively-based approach, it ignores the fact that excitements, enthusiasms, fears, worries and hopes of individuals should be somehow included in learning process. Our questions in teaching process should not mostly focus on cognitive pedagogies. "Within new quests, to what extent criteria related to mythic, romantic and philosophical understanding can be included?" Pre-service teachers mostly prefer the inclusion of visuals in questions. In addition to this, they think that the questions asked for the discovery of the curiosity of scientists and interesting, surprising and fascinating questions should be asked during the teaching process in the class rather than in exams. Moreover, they prefer interesting, odd and curiosity-arousing questions to be used in teaching process. These findings may stem from the pre-service teachers' concerns about objective evaluation of the exam questions. Pre-service teachers think that the questions leading to investigation of experiences, active interaction with the world, attaching importance to the revelation of the opinions and ideas, opening the door of the world of ideas and allowing flexible responding should be used in both teaching process and exams. Pre-service teachers think that mythic, romantic and philosophical questions should not be used in both teaching process and exams. Using questions having mythic, romantic and philosophical contents may have some contributions to learning-teaching process [6,11,13]. Rhythmic narrations and images may serve the retention of information, They can contribute to learners' understanding of the world through exotic, odd and mysterious experiences and through fascinations and surprises, writing essays, mind-games, puzzles and development of rhetoric skills aiming to open the world of ideas may contribute to more profound understanding and internalization of information, Moreover, students will be able to see themselves as more cultural beings as a result of encounters with heroes and learning about their struggles and adventures. 'Alternative questions', may help students to be engrossed in the subject. Moreover, they may develop more humanitarian perspective of the program, teaching and exams.
Questions related to mythic, romantic and philosophical understanding may increase the variety in teaching and exams. In this way, they may indirectly contribute to the use of unusual teaching methods and activities during teaching process. This contribution may enhance students' quest for meaning and concepts. Egan [7], states, "determination of how successful students are in exams may not be a good way of determining their expectations and hopes from their future jobs". Moreover, academic success may not be enough to perform professions particularly those like teaching including many affective dimensions. New search for alternative questions may serve as a springboard for conducting program developments works. There are some limitations of the present study. Conducting research on a more comprehensive sampling including teachers and university instructors and investigation of the settings where traditional questions and alternative questions (directed to mythic, romantic and philosophical understanding) are used together may reduce these limitations.