Mathematics Teachers' Challenges in Implementing Reasoning and Proof Assessment: A Case of Indonesian Teachers

The assessment in mathematics proof and reasoning is one of the crucial aspects of mathematics education. It aims to elicit the development and the level of students' mathematics ability. Nevertheless, several challenges arise from implementing proof and reasoning assessment. This study investigated the challenges experienced by the high school teachers in implementing assessment in proof and reasoning. The study sample comprises three groups with different teaching experiences consisting of ten mathematics teachers in each group. The data were collected through a questionnaire, a depth interview, and a focus group discussion. Bogdan and Biklen's qualitative descriptive analysis is used to analyses the obtained data to elicit a particular theme. The significant challenges included dealing with the implementation of reasoning and proof evaluation. The results indicate that (1) in developing item tests, teacher faced challenges in arranging test framework, determining the suitable mathematics topics, and developing scoring framework; (2) during the test implementation, the teachers have difficulties on the time allocation issue; (3) in analyzing of the assessment result, the teachers inadequately describe the students' mathematical reasoning and proof ability. A collaboration involving mathematics teacher’s organization with mathematics education experts is needed to counter the challenges. In detail, this collaboration should focus on the assessment of mathematical reasoning and proof implementation.


Introduction
The ability to reasoning and proof has already been a research focus in the past decade [1][2][3]. These studies suggest that reasoning ability is crucial for developing better mathematical skills and supporting other mathematical knowledge. Furthermore, reasoning ability is a mathematical process standard that must be mastered by the students not only for urgent order but also high order mathematical thinking [4]. For mathematics teachers, reasoning ability plays an essential role in mathematics teaching since all mathematical knowledge is involved in conducting reasoning activity, including high-order thinking, mathematical communication, and meaningful learning [5].
The importance of the development and improvement in students' mathematical reasoning and proof ability is contained in the competency standard of mathematics teaching and learning curriculum [6,7]. The curriculum includes the educational aims, learning topics, learning activities, learning materials, and evaluation instruments used. The role of the curriculum is significant toward the way of students' learning, students' cognitive development, and, especially, students' 21st-century skills [8,9]. Cai & Cirillo [7] suggest that, from the analysis of the curriculum in their country, the ability of reasoning and proof as one of high order thinking skills become the main focus in the mathematics curriculum. It is undeniable that reasoning ability also becomes a fundamental ability in students' mathematics competency in Indonesian Curriculum.
Based on Curriculum 2013, the new official curriculum in Indonesia, the description of mathematical ability that must be mastered by the high school students is that the students can conduct reasoning, processing, and presenting mathematics concept effectively, creatively, productively, critically, independently, collaboratively, communicatively, and solutive [10]. In detail, the students are required to learn how to solve problems, justify and explain solutions, and elaborate on the mathematical concepts also with the ability to relate the mathematical ideas to other subjects [11,12]. The aforementioned teaching and learning mathematics objectives could be achieved by implementing reasoning and proof ability integrated activities in the class rather than rote learning activities [13]. In detail, the use of proof and reasoning mathematics tasks may foster the students' reasoning and proof ability [14][15][16].
High school mathematics assessment involving proof tasks analyze abstract mathematics domain. The analysis is conducted based on high school students' cognitive level, which already in the formal operational stage where the students have already been able to think logically and abstractly. In this stage, the students start to develop their higher-order thinking skills in the form of proof and reasoning ability deductively and inductively. This process occurs while the teachers are solving mathematics problems. The ability of deductive and inductive thinking not only enable the students to understand the formal process in developing consistent logical argument based on axioms, definitions, and traditional theorems, but also in every activity related to mathematical facts discoveries, conjecture determination, and truth development [17]. In detail, these activities involve exploration, generalization, reasoning, argumentation, and validation.
In reality, the students tend to solve mathematical problems using algorithmically and remembering the given formula only. These students' preferences are the symptoms of the fact that the students experience difficulties in using their reasoning to construct mathematics proof [18]. The mathematics learning guidebooks that the students use have already implemented tasks that facilitate mathematical proof and reasoning abilities [19,20]. These problems appear since the inability of the teachers about reasoning and proof related to the tasks and inability to integrate proof and reasoning tasks in the classroom. Most of the teachers do not understand the various characteristics of mathematical thinking cultures and belief that those proof and reasoning tasks are too complicated for their students [21].
Furthermore, mathematics teachers only consider the final proof on evaluating whether their students can prove a mathematics statement without further analysis of the students' reasoning. Besides the fact that the pre-service teachers are mostly unable to understand the new curriculum, their perception of evaluating instruments that only consider inductive thinking is affected [22].
In improving the quality of mathematics assessments for high school mathematics, this study would discuss the difficulties the high school teachers faced in implementing proof and reasoning ability assessments, which consisted of developing test items, conducting evaluation, and analyzing assessment results. In the developing test items, the quality of developed test items will be analyzed by the teachers based on the question of whether the test items have already considered the hindrances faced by the teachers. In conducting assessment phases, teachers' difficulties in doing evaluation will be analyzed. Lastly, in the analysis phase, teachers' difficulties in analyzing evaluation will be described. Therefore, the description of the high school teachers' difficulties in evaluating the proof and reasoning ability would be an essential reference for the government in developing the improvement of education quality in mathematics education.
This study aims to describe the profiles of high school teachers' difficulties in implementing students' mathematical proof and reasoning assessments. This description elicits the teachers' difficulties starting from developing, conducting, and analyzing the result of students' assessment.

Design
This research is a qualitative phenomenological. The use of phenomenology aims to uncover and to understand a phenomenon, including its unique context experienced by individuals to the extends of personal belief [23]. The type of this research is descriptive-exploratory research. The phenomenon described in this study depicted teachers' challenges in implementing mathematics proof and reasoning assessment. The reasoning and proof assessment implementation challenges were examined from several points, including preparation in developing test items, carrying out the test, and analyzing the result. The selected research subjects are 20 high school mathematics teachers (SMA/MA) in Indonesia who have already implemented the proof and reasoning assessments in their mathematics classrooms. The categorization of the research subject based on the teaching experience are classified into two groups.
Moreover, supporting data that are relevant to the research context are elicited from the written document. The document is an example of mathematical problems and questions, along with the evaluation criteria developed by the subject. This written document was used to provide additional information to support the claims on the teachers' challenges.

Data Collection and Analysis
The technique of data collection used in this study is giving a questionnaire, Focus Group Discussion (FGD), and followed by an interview. This questionnaire included several items related to the teacher's difficulties while implementing the mathematics reasoning assessment. The participants' responses to the questionnaire were then discussed in the FGD. Meanwhile, the interview technique is chosen to genuinely obtain information and validation on the collected data of teachers' difficulty in implementing mathematics reasoning and proof ability from FGD. After the data collection phase, analyzing data is required in order to acquire systematics and well-described data. The data of the teachers' difficulties in implementing mathematics proof and reasoning assessment are analyzed from FGD and depth interview results. Afterward, the data were analyzed using a model developed by Bogdan & Biklen [24] to determine the inter-theme relations and gain a more detailed understanding. The results of this analysis were used to understand the phenomenon of challenges and strategies in implementing the reasoning and proof assessment. To assure the credibility of this research, the method used in this research is data triangulation, by which the data validity is checked by using different data resources, which are the written documents.

Results
This study aimed to describe the high school teachers' difficulties profiles in evaluating the students' mathematical proof and reasoning. The teachers' problems were elicited in this description. The profiles were starting from developing, conducting, and analyzing the result of students' assessment. Based on FGD and depth interview records conducted with 20 high school mathematics teachers, three critical themes that refer to problems in implementing students' proof and reasoning ability assessment were drawn. Three critical aspects were described as (1) preparation for developing reasoning and proof test, (2) carrying out the assessment, and (3) analyzing the assessment results. In detail, from the three themes mentioned above, more detailed sub-themes were elaborated from the summary of the core of the teachers' difficulties.

Preparation for Developing Reasoning and Proof Test
The initial process conducted by the teacher in assessing students' ability was preparing to develop test items. The mapping of challenges in this stage found in this research is presented in Table 1.
The challenges related to the preparation should be appropriately administered before conducting reasoning and proof assessment. Workshops for teachers in developing an assessment instrument for measuring student's reasoning and proof ability had already been held based on standard evaluation on reasoning in the revised Curriculum 2013. The workshops were carried out along with local mathematics subject teacher committees. Teachers had already understood the concept of proof and reasoning assessments. However, the main problem in this stage was that the teachers unable to develop the frameworks for the test items well and providing score guides. The teacher should have better preparation before conducting reasoning and proof assessment in order to get precise results on the students' ability in reasoning and proof from the assessment.
The unavailability of a particular workshop that focuses on the mathematics assessment on higher-order thinking problems The teacher's confusion in determining the topics related to the reasoning and proof assessment Only induction topic that can be used by the teachers concerning mathematics reasoning and proof The references are a bit difficult to be found; not every problem is in proof tasks.

Difficult to find the test item resource
It is too difficult to arrange the problems by ourselves, and we have no time Problems Book of mathematics Olympiad can be used for alternative problem sources, but several adjustments should be made according to the reasoning and proof characteristic.
The teachers usually use the reasoning and proof problems from resources that have incomplete solutions, making the teachers difficulties in scoring the solution.
The inability of the teachers to provide the key answers and scoring guides. There are documents of the students' proof task evaluation result that does not have the scoring guide.
The first constraint was the difficulties in developing the item test framework, and several teachers experienced difficulties in the development of the test items, especially in creating the assessment frameworks, especially for mathematics teachers who have taught for ten years. They were unable to decide firmly regarding the content of mathematics should be tested. The teachers' understanding of the possible topics in mathematics that could be considered in reasoning and proof tests were limited only at mathematical induction, which was a chapter in Curriculum 2013 textbook. Besides, it was found that some the teachers only consider mathematics as procedural algorithmic calculation, so promoting reasoning and proof tests would be impossible. They also considered that only complex mathematical problems that can be used for the tests. Consequently, the teachers tend only to be able to develop reasoning and proof instruments using induction topics. The second constraint was the difficulties in finding the test item resource; another problem relating to these issues was the selection of books as references. The use of reference books was considered as resources for developing mathematics proof related problems that could be used in evaluating student's proof and reasoning ability. Several teachers stated that they tend to use Mathematics Olympiad problems in which mathematical proof tasks can be found, although the contexts were usually irrelevant to the high school topics. Some other teachers who experienced difficulties in developing the tests used student's handbook as a reference because by using them cost the teachers less time than finding books from the bookstore. In developing test items, Figure 1 showed that the problems obtained from student's handbook were usually only proving math formula in which a complicated proof was needed. Consequently, the teachers felt uncertain about the problems since sometimes, they were also unable to solve those proof tasks. Other problems faced by the teachers were related to the key answers and scoring guides. The difficulty in this issue resulted since the problems obtained from references were usually provided incomplete proof. This made the teachers have to prove the tasks by themselves in which, sometime, the teachers unable to do so. These problems were experienced by the teacher who had less than ten years of teaching experience. Furthermore, this finding could also be found in the students' scoring result on Table 2, where the main focus in the evaluation was only the final result of the proof. Consequently, the assessment did not consider any process on the student's answer as the scoring criteria.

Implementation of Mathematics Reasoning and Proof Assessment
After designing the test items, the stage of implementation of the assessment based on the topics selected by the teachers was conducted. The teachers' difficulty in this stage was that the test items were not clear enough to be understood by the students. The students were often unable to understand the flow in solving proving tasks and failed to understand clearly about the problems which mostly happened in the students with adequate to low mathematical abilities. This issue was considered by one of the teachers who had already taught for ten years. Therefore, it was required for students to be taught about mathematical proof techniques in the separated topics in the curriculum. The topics should focus not only on inductive proofs but also on deductive proofs.
Other difficulties occur in the test implementation were the insufficient time allocation for the problem solving and the incomplete solutions given by the students. These problems were experienced by the teachers who still have taught less than five years. The students were unable to solve because they are unable to understand the problem that makes the class quite noisy. Besides, most of them ask for more time to solve the problem. Furthermore, most of them unable to complete the answers. Consequently, most of the students submit the given reasoning and proof task lately. The steps in solving the tasks were not in the right order that made it difficult to evaluate. The written systematics on the students' answers were not suitable for the answer keys.

Analysis of the Result of Mathematical Reasoning and Proof Ability Test
The test result showed that students tend to perform a low ability in reasoning. Unable to describe Students' Reasoning and Proof Ability Students' reasoning was quite low. The students' reasoning tended to be inductively developed because the tasks of proving content were only studied in mathematics induction topics.
The stage of analyzing the test results of the reasoning and proof tests was done to elicit the achievement of the students' abilities in solving reasoning and proof tasks, in details, the test analysis was mean for determining whether the test items successfully assess the students' abilities in reasoning and proof, providing feedback for the students based on their level of reasoning and proof abilities, and giving suggestions for the upcoming lesson to the teachers about the students' difficulties in solving reasoning and proof tasks. The difficulties faced by the teachers in analyzing the result of reasoning and proof tests were when they analyze the students' responses and describe the students' reasoning and proof activities.
One of the problems faced by the teachers was that, in analyzing students' answers, the solution given by the student was different from the available vital answers. This situation made one of them had difficulties in scoring the student's answers. Besides, the teachers' difficulties in analyzing the students' answers were unclear directions on the students' proof. The students' responses were mostly arranged inconsistently, which made it challenging to score on scoring guides.
In analyzing the results of the tests, eliciting information on the students' abilities on reasoning and proof based on the grade was not an easy task. It is agreeable that, when the grade of students' achievement was above the criteria of minimum completeness score, the students could be considered as having a high reasoning ability. However, when the teachers were asked to determine which dominant types of students' mathematics reasoning, they were unable to describe clearly. The teachers' inability to describe the reasoning and proof abilities were caused by the low teachers' understanding of the reasoning and proof ability. The teachers tended to only consider the mathematical induction as a reasoning and proof activities applied in the school, even though it only represents inductive proof. They were unable to develop students' abilities in deductive reasoning.

Discussions
In developing a good, valid, and reliable assessments, a careful test item arrangement is required. However, from the result of FGD and interviews, the fact that the teachers had difficulties in developing the evaluation is undeniable. The difficulties vary as the development of arranging test frameworks, using resources as references, and arranging students' scoring guide for the reasoning and proof assessments. Designing the instrument framework is the initial stage in implementing evaluation [25]. Moreover, the scoring framework is a profile of the tested evaluation domain and content.
Nevertheless, the teachers experience difficulties in determining the suitable material used in the evaluation. Most of the teachers only included the topic of mathematics induction from which the students' abilities evaluated in the test is involving inductive proof only. This finding is supported by İmamo\uglu & To\ugrol [22] with pre-service mathematics teachers as the subjects stating that there is a tendency of inductive thinking on the pre-service teachers' reasoning and proof ability.
Selecting and determining the resources for the test reference became one problem for the teachers in designing the test instruments. This issue results from the teachers' lack of knowledge in evaluating students' reasoning and proof abilities. This finding is in line with what McCrory & Stylianides [20] found that the topics of reasoning and proof are not available as the full content of assessments in the curriculum for the pre-service teacher students. Even there are several courses relating to the reasoning and proof, the concept of reasoning and proof evaluation is still rare to be discussed. Therefore, when the pre-service teacher teaches in the real class in the future, they were unable to design and implement a sound reasoning and proof assessment test well.
Profound mathematical content knowledge of the teachers results in another problem in the arrangement of test item stages. Therefore, the teachers only provide limited alternative solutions for the problems that make the teachers' difficulties in grading the students' answers. This idea is also stated by Harel & Sowder [26] that one factor inhibiting the students' capability of reasoning and proof is the teachers' mathematical content knowledge in developing the students' cognitive development and bridging various solutions made by the students.
The test's implementation is one of the critical stages in evaluating the students' reasoning and proof ability using the test items designed before. The teachers experience difficulties in this stage because of students' responses due to their lack of understanding of reasoning and proof tasks. During the test, the students tend to ask frequently to the teachers about the technique in solving the tasks or the meaning of the context of the problems. Another problem resulting from this issue is that the time allocation is not enough for the students to complete all of the tasks. This finding is supported by Stylianides et al. [21], who stated that the challenges and difficulties of mathematics teachers in the teaching and assessing their students are that the teachers' tendency on the procedural and algorithmic skill only. This situation made the students unable to solve a test that involves reasoning and proof abilities. They will just solve the problem using the algorithmic solution without any reasoning behind it.
In analyzing the test result, the teachers also experienced difficulties in interpreting the students' solutions. This problem resulted from the fact that the students unable to write their answers systematically. Besides, the students tend to answer whatever they know, even using procedural answers that made the teachers unable to grade objectively. The teachers experience difficulties in grading the students' responses according to the students' unexpected solutions, and they are unable to understand the students' answers [27] entirely.
In this analysis stage, the teachers also have difficulties in describing to what extent the students' abilities on reasoning and proof developed. This idea is essential for the improvement in teaching by the teachers in the future. The teacher only classifies the students' ability in reasoning and proof as high and low based on the criteria of completeness score. This finding is in line with Campbell & Zelkowski [28] finding that teachers are unable to reflect on the students' thinking in the teaching and learning process, especially about the reasoning and proof abilities. Another finding also supported by the fact that the teachers are not accustomed to writing a full description of the evaluation report [29]. They only write a note that states that the students are included remedial or not.

Conclusions
Based on the finding of teachers' difficulties in implementing mathematical reasoning and proof assessment, it can be concluded that there are three fundamental problems experienced by the teacher in developing reasoning and proof assessment. They are categorized as (1) the difficulties in developing the test, (2) the difficulties in implementing the test in the classroom, and (3) the difficulties in analyzing the result of the test. In detail, the teachers' difficulties in developing the reasoning and proof test occur on the teachers who still have 0-5 years of teaching experience. They have difficulties in defining suitable mathematics topics, selecting references, and arranging the scoring test framework. Meanwhile, the difficulties in implementing the reasoning and proof test are faced by the teachers who have 5-10 teaching experience. They are unable to design and estimate the adequate time allocation for working with the test. A poor decision on the time allocation would make the students unable to answer the test based on their real proof and reasoning ability. Moreover, for the teacher in 5-10 years of teaching experience, they are also unable to analyze the result of proof and reasoning test. Most of the time, they have difficulty describing the students' mathematical proof ability and reasoning whether the answer can be categorized as visual, spatial, algebraic, or quantitative reasoning and proof.
Finding our research may be useful in directing future research related to teacher challenges in conducting reasoning and proof assessment and how to deal with it. Therefore, other researchers can develop a strategy in teaching and learning to overcome its challenges and provide a guideline for teachers in assessing reasoning and proof test.