The Effect of Marker-Based Augmented Reality (MBAR) Applications on Academic Achievement and Permanence

The aim of this study is to determine the effect of learning objects developed in marker-based augmented reality environment on academic achievement and permanence of learning. In the study, experimental design with pre-test and post-test control group has been used. The study has been carried out in the spring term of 2016-2017 academic year with 45 students studying in the 7th grade of a state secondary school in Sincan district of Ankara. At the beginning of the study, “Academic achievement scale” has been applied to the experimental and control group students. The subject has been discussed by using augmented reality applications for the experimental group, the traditional method (narration, presentation on a model or board and question and answer) for the control group. At the end of the experiment, the same scale has been carried out as post-test and permanence test four weeks after the end date of experimental procedure to each group. According to the result of study, it has been seen that augmented reality application is effective in terms of the academic achievement of students and permanence of their learning.


Introduction
When the literature is looked over, it has been determined that, "Augmented Reality" has many different equivalents in Turkish as "Artırılmış Gerçeklik (AR) ", "Zenginleştirilmiş Gerçeklik", "Genişletilmiş Gerçeklik", "Harmanlanmış Gerçeklik". It is known that the first experience with Augmented Reality (AR) has been started in 1960s. Real environment has been enriched with head-mounted display and mechanical/ultrasonic monitors.
Only very basic images could be got with the limited processor of computers in that time. (Johnson et al., 2010).
There are different classifications about augmented reality application. If digital objects generated in real world use a geographical location as reference point, it is defined as location-based (based on location) or as geography-centered augmented reality. If generated objects use a marker as reference point, it is defined as vision-based/marker-based augmented reality (MBAR). Location-based reference point of AR systems are generated through data obtained from GPS or/and compass.
Vision-based (image-based) AR systems are realized by detecting the reference pointer in the real environment with the camera or various imaging devices, processing the position-size-angle information by the computer processor, and displaying the visual objects produced by the computer on the display device as if they were part of the real environment. Briefly, MBAR systems are a combination of real-world environments with computer-generated elements and presentation of them (Azuma, 1997). Nowadays, thanks to developing technological opportunities, besides the combination of images, it is possible to present them by enriching through multimedia such as audio, video, animation and to add interaction at high level (Lee, 2012). AR applications have become widespread in many areas such as tourism, architecture, entertainment and education. The reasons behind becoming widespread of these applications, especially in education, are based on the fact that the two-dimensional visuals are more effective than words in learning, these images are much more effective when supported in three-dimensional form, and the effect is getting increased when the real world is transformed into three-dimensional images carrying the individual (Coimbra et al., 2015). The number of studies on the use of AR applications in the process of education and teaching continue increasing. It is seen that AR applications support student's curiously research-based learning and they continue persistently without giving up researching and questioning in teaching (Fleck et al., 2015).
Serious initiatives and studies in that area are being carried out for using technology in education in Turkey like other countries all over the world in recent years. Projects being carried out by MEB and aiming at equipping schools with technological systems and being most effectively used of teaching ware by teachers and students are put into practice. Academic studies need to become varied to support and make widespread all these initiatives. In this context, it is seen that the number of studies on especially AR applications in literature has started to increase. In the study by Akkuş and Özhan (2017) it is stated that the studies on AR concentrate on especially teaching of math and geometry and these studies are marker-based augmented reality. There are many studies on using digital technology as content in mathematic teaching and their impact on academic achievement (Özerbaş and Erdoğan, 2016). The striking point here is that AR applications are being frequently seen in educational studies in the fields of mathematic and geometry while being used in areas of expertise such as chemistry, biology, health, engineering, architecture; however, in teaching of these, it is noteworthy that there are a limited number of academic studies on education applications in different components. For these reasons, it is expected that the study will guide future studies regarding the use of AR systems by examining the effect of AR systems consisting of advanced technology imaging and image processing units on achievement of student in teaching Science. It is also important in terms of identifying the problems that may be encountered during the use of these systems. It has been preferred because of the reason that the subject of absorption of light in the unit of interaction of light with matter is suitable for MBAR applications and another reason is that it has been observed that there are not so many studies on it. For all these reasons, the aim of this study is to determine the effect of MBAR applications that is improved on the subject of absorption of light in the unit of interaction of light with matter in Science on the academic achievement and permanence of students. In order to reach this aim, two basic questions given below have been sought. 1. Is there a significant difference between the pre-test and post-test academic achievement scores of the experimental group (students studying with MBAR) and the pre-test post-test academic achievement scores of the control group (students studying without MBAR)? 2. Is there any difference on their permanence scores between the experiment group (the students receiving education through MBAR) and the control group (the students receiving education without MBAR)?

Method
In this study, experimental pre-test post-test control group design has been used; it is tried to determine the effect of augmented reality applications on academic achievement and permanence level of students.

Study Group
The study group of this study consists of 45 students studying in the 7th grade of a state secondary school in Sincan district of Ankara in the term 2016-2017 academic year and 24 of them are experiment and 21 of them are control group.

Data Collection Tools
In this study, achievement test including the unit of "Interaction of Light with Matter" with 25 items, which is developed by researcher in the guidance of two experts has been used to determine academic achievement of students before and after the application. With this purpose, the content developed by the MoNE (Ministry of National Education/MEB) was utilized (MEB, 2017). Prepared academic achievement test based on multiple choice test technic has been applied to 310 students in 3 different schools as pilot and the data have been analyzed with SPSS program. The matter distinctiveness and matter difficulty index of each matter has been calculated. In the selection of suitable substances, if the item discrimination index value is 0.40, the item is considered to be very good, if it is between 0.30 and 0.39, it can be kept on the scale without correcting the item, and items less than 0.20 can be developed or removed from the test. (Büyüköztürk et al., 2012). Then, matters aiming at the same attainment have been categorized, and matters having the most suitable difficulty and distinctiveness have been chosen from question pool. The mean difficulty index of matters extend of testing has been calculated as 0.38. Depending on this result, it can be said that the difficulty level of the majority of the matters selected in the scope of the test are clustered at the medium level. The KR-20 reliability formula was used because the values of the questions in the measurement instruments were determined with equal weight (homogeneous measurement) and only 1 items were answered correctly and 0 items were answered incorrectly or never answered (Kuder-Richardson). As a result of pilot scheme, KR-20 reliability (pre and post-test) has been calculated as 0.87. According to the results of item analysis conducted on the items of the achievement test of the Interaction of Light with Matter, the average discrimination of the test was calculated as 0.50.

Implementation Process
Achievement test has been carried out as pre-test to determine readiness level of students before learning and as post-test to determine their achievement about subject after learning. The same test has been used as permanence test three weeks after the end of experimental process. Before starting implementation, the achievement test has been carried out as pre-test to each group. Then, to the experimental group the information about augmented reality technology, model applications and videos have been presented in two hours lesson and it has been shown how to install developed software to phone. The implementation has lasted 24 hours in total for 4 lesson hours in a week during 6 weeks. While the experimental group was shown augmented reality applications developed, the control group was informed about the Interaction of Light with Matter in the usual way by means of expression, demonstration and question and answer without ITAG. Markers related to ITAG applications were distributed to the students in the experimental group each week. Experimental group students, who demonstrated the augmented reality applications, tried the augmented reality applications related to the subjects individually or in groups for 6 weeks. At the end of the application, retention test was applied to the experimental and control group students as a post-test four weeks later.

Development of Instructional Materials
Within the scope of the study, augmented reality applications on the unit of Interaction of Light with Matter have been developed. There are many different types of augmented reality technology in literature. In this study, Marker-Based Augmented Reality (MBAR) applications have been developed. Thirty-two MBAR applications have been developed on absorption of light, refraction of light, mirrors and lenses in the unit of Interaction of Light with Matter in Science lesson in 7th grade. In the Figure 1, there are applications related to subjects in the unit of Interaction of Light with Matter when MBAR applications are combined. In the step of development of applications, Unity 3D program, Vuforia and OpenCV (Open Source Computer Vision) that is MBAR plug-in of this program open-source image processing library have been used. To use developed MBAR application the students have been supplied to install it from Google Play to their cell phones and tablets. During the implementation, students have studied individually and/or in cooperation. Firstly, theoretical information has been given to students by course teacher and then related augmented reality application has been shown. Then, papers with theoretical information and markers have been handed out to students. Students have been supplied to use applications shown by the teacher. The students individually applied ITAG applications with their personal phones or tablets, and in the meantime, they were allowed to look at the applications in collaboration, thus enabling them to interact more.

Analysis of Data
SPSS (Statistical Package for Social Sciences) have been used for analysis of the data obtained in the study. In quantitative studies, both parametric and non-parametric statistical methods are used for data analysis. In order to be able to use parametric analysis methods in study, quantitative data obtained from all tests and scales during the study requires showing normal distribution. For this reason, data obtained from achievement and permanence test has been analyzed and since it has been found to show normal distribution, parametric statistical techniques have been used. Independent group t test has been used to determine whether there is a significant difference between score means of achievement pre-test of students in experiment and control group. Two Way ANOVA test has been used for data analysis of mixed measurements regarding achievement pre and post-test and permanence test. At the same time, Bonferroni test has been used for pairwise comparison of point means that experiment and control groups get from achievement pre and post-test and permanence test. The significance level has been taken as p <0.05 in all analyses.

Results
Descriptive data set, which included the number of subjects, arithmetic mean and standard deviation scores, related to the study is presented in Table 1. When the table showing the descriptive statistics of achievement pre-test, achievement post-test and permanence test score means of groups is analyzed, it is found that the achievement pre-test score means of subjects in experiment group (x̄= 37.58) is close to achievement pre-test score means of subjects in control group and the mean of pre-test scores in the two groups are relatively low (test score range: 0-55). When the achievement post-test and permanence test score means of groups are analyzed, it is found that both achievement post-test score means (x̄= 56.16) and permanence test score means of subjects in experiment group (x̄= 42.70) are higher than achievement test score means of subjects (x̄= 48.61) and permanence test score means of subjects (x̄= 29.04) in control groups. As it is seen in Table 1, there is an increase in the achievement of subjects in both groups from pre-measurements before application to post-measurements after application but there is more increase in the experiment group. The permanence test scores show that the permanence test score means of subjects in the experiment group are higher than the permanence test score means of subjects in the control group. When data in Table 2 are analyzed, there is no significant difference between achievement score means of subjects before implementation (experiment group x̄= 37.58, control group x̄= 39.80) (p > .05, t = -.842). The fact that there is no significant difference between achievement scores of subjects before implementation is appropriate for the purpose of determining the effectiveness of applied teaching method. There is no significant difference between pre-test results of subjects in experiment and control group and Two Way ANOVA For Mixed Measures has been used in data analysis related to achievement pre-test, achievement post-test and permanence test of experiment and control groups.
According to Table 3, as a result of variance analysis on score means of participants in experiment and control groups obtained from academic achievement pre-test, post-test and tracking measurement, it has been found that the group effect is significant (F(1-44)=78,045;p<.05). According to this, it can be said that there is a significant difference between score means without making any exception between pre-test, post-test and tracking measurements of experiment and control groups. The difference between the score means of subjects obtained from pre-test, post-test and tracking measurements is also significant (F (1-86) = 73, 34; p<.05). This finding shows that the Science academic achievement levels of the participants vary depending on the experimental procedure when the group is not discriminated. In addition, it is seen that the value obtained as a result of examining the common effect (group * measurement effect) which is important for this research is significant (F (1-86) = 52, 14; p<.05). In this study, Bonferroni Test has been used for pairwise comparison of score means of pre-test, post-test and permanence test under the measurement variable in order to find which test causes this significant difference. The reason for choosing this test as post-hoc is that it is a widely used multiple comparison test and it does not require number of equal sample principle.

1930
The Effect of Marker-Based Augmented Reality (MBAR) Applications on Academic Achievement and Permanence The source of the difference between pre-test, post-test and permanence test mean scores of the groups has been investigated with Bonferroni test. When the results in Table 4. are analyzed, a significant difference is found between pre-test and post-test mean scores and post-test and permanence test mean scores of experimental group (p <.05). These differences are in favor of the final test in both. While there is a difference of 5.12 in favor of permanence between pre-test and permanence, this difference is not significant (p> .05). Again, there is a significant difference between the pre-test and post-test and post-test and permanence tests of the control group in favor of the post test. It is seen that there is also a significant difference between the mean scores of pre-test and permanence test of the control group. (p <.05) However, this difference is in favor of the pre-test with 10.76.

Discussion and Conclusions
In order to determine the effectiveness of the MBAR applications used in the study, the subjects have been applied to pre-test at the beginning of the application, and thereafter they have been applied to achievement and permanence test as a post-test. Two sub-problems were determined in order to determine the scores of the subjects in experimental and control groups from these tests before and after the study and to reveal whether these scores change depending on the effectiveness of the teaching method applied during the course and the findings obtained from the data related to these sub-problems given in tables in the previous section.
According to the results of the normality test, the findings that are obtained from the results of analysis using parametric statistical methods are presented below to meet all the sub-problems.
The findings related to the first question of the study show that there is no significant difference between the score means of pre-test achievement of the two groups (p> .05, t =,-842). The fact that there is no statistically significant difference between the mean scores of the two groups before starting the study is very important both to determine the effectiveness of the applied teaching method and to make selection regarding the statistical method to be applied to the post-test and permanence test scores. The second sub-problem of the study and the achievement post-test mean scores of the subjects in the experimental and control groups, and whether there is a significant difference between the permanence test mean scores of both groups are investigated.
The findings presented in Table 3 show that there is a significant difference in favor of the experimental group between the mean scores of both the post-test and permanence test of the groups[F (1, 44) = 78.045, p < .005]. In the experimental process, it is concluded that in terms of their achievement in science there is a significant difference between the pre-test and post-test scores of the experimental group students in which the augmented reality applications have been shown. This finding shows that the students in the experimental group in which the augmented reality applications have been shown, increased their achievement scores at the end of the experimental procedure.
According to another finding of the study, it is concluded that in terms of their achievement in science there is a significant difference between the pre-test and post-test scores of the control group students in which MBAR is not applied. This finding shows that students in the control group in which MBAR is not applied, increased their achievement scores at the end of the experimental procedure. It is concluded that in terms of their achievement in science there is a significant difference between the posttest scores of students in the control group to which MBAR is not applied and students in the experimental group to which MBAR is applied. This finding is evaluated as augmented reality applications that are effective in increase of academic achievement towards science. However, despite the significant difference in favor of the MBAR in both the post -test and permanence score means, the permanence scores decreased in both groups as expected, and this situation shows that the recall of previously acquired knowledge is reduced by dropping behind pre-test score means in control group. This finding shows that the experimental group students' learning for science is not relatively permanent, and that the achievement scores of the science fall for the permanence test. According to the results relating to academic achievement mentioned above, it is seen that MBAR has a significant effect on students' achievement. This situation has been interpreted as MBAR applied in experimental group has increased the achievement and supported permanence of the knowledge.
When the literature is reviewed, studies show that the augmented reality technology has positive effects on academic achievement and also supports the outcome of this study.
In an experimental study by Ibanez, Serio, Villaran and Kloos (2014), students have understood the subject of electromagnetism with augmented reality applications. And in a study by Cai, Chiang, Sun and Lee (2017), augmented reality applications have increased the achievement of the students and in an experimental study by Dünser, Walker, Horner, Bentall (2012), augmented reality books are useful in learning complex abstract concepts are concluded. Similarly, in the study conducted by Sırakaya and Çakmak (2018), it is seen that augmented reality applications have a positive effect on students' academic achievement. Coimbra and others (2015) have concluded that the effect of AR that is applied to 13 students enrolled in the evening program of the electrotechnics engineering department of a higher education institution in Portugal on mathematics learning is high. In this study, they emphasized the importance of AR contributing to both use of knowledge and access of knowledge in digital technology and development of knowledge. At the same time, an important factor in the increase of academic achievement and permanence is that objects containing MBAR applications draw student's attention and this stimulation contributes to the learning process (Gün and Atasoy, 2017).
In the light of all these information, MBAR applications that elaborated on the teaching designs in accordance with the educational scenarios in technical terms have a positive impact on many teaching areas such as science, geography, history, mathematics, etc. and it is believed that its implementation should be supported by spreading. Concentrating on detailed design of these works will contribute to the field. In spite of all these, more detailed effects of MBAR applications are needed to be determined through long-term monitoring studies.