Are MOOCs in Blended Learning More Effective than Traditional Classrooms for Undergraduate Learners?

Massive open online courses (MOOCs) have become one of the most prominent trends in higher education. In contrast to a traditional classroom where knowledge is delivered through face-to-face interaction, MOOCs offer open-access, video-based instructional content, and forums which are released through an online platform to unlimited number of learners. Despite impressive notable features, their effectiveness has yet to be proven. Intrigued with this issue, the objective of this study is to examine the effectiveness of learning via MOOCs in blended learning and traditional classrooms among undergraduate learners. An experimental study which employed non-equivalent pre-test and post-test control group design among two hundred and five undergraduate accounting students was conducted in one of the universities in Malaysia. The students were divided into two groups; experimental and control. The experimental group was taught via MOOCs in blended learning, while the control group was taught in traditional classrooms. The results of paired sample T-test indicated that there were significant differences in academic achievements between the pre-test and post-test groups (experimental and control groups). The magnitude of effect of the former was higher than the latter. The results of MANOVA indicated that there were significant differences in academic achievements between groups in post-test, but not in the pre-test. Hence, MOOCs are more likely to be more effective than traditional classroom in teaching and learning today’s generation which consists of digital savvy natives. The discoveries provide significant insights for educational fraternities on technological and pedagogical aspects of how teaching and learning can best be delivered and acquired.


Introduction
The acceleration of Information Communication and Technology (ICT) has dynamically changed the landscape of tertiary education. It replaces a classroom from face-to-face teaching to online teaching using the internet, causing the classroom to lose its monopoly as the place of teaching and learning (Nguyen, 2015). As seen today, the quality and capacity of online delivery has tremendously evolved and emerged in varieties of forms (Weller, 2014). Many researchers and educators are interested on online delivery due to its varieties of attributes that can be applied to make instructions more interesting to learners. Massive open online courses (MOOCs) are a recent innovation of online delivery or pedagogical tool which has drawn great attention from both public and academic fraternities.
Initially, MOOCs were opened to everyone with web accessibility without charging any fee. However, in early 2019, minimal fees have been imposed to cover the maintenance cost. These open online courses are set up by third party as independent online platforms, and educators are encouraged to develop and upload their teaching materials through MOOCs. The courses support lifelong learning and are able to cater for high volume of learners. They offer incomparable classroom experience by developing an effective learning engagement (Baturay 2014). This is done by displaying the multimedia materials extensively on an interface screen and the teaching is conducted through interactive mode. The presence of these digital courses continues to grow in higher education, as many learners are eager and excited to learn the technology. To date, thousands of courses have been invented via MOOCs that have attracted millions of learners all over the world.
According to Olazabalaga et al. (2016), MOOCs have prevalently become one of the most prominent trends in higher education since their development in 2008. The demand for online educational platforms coupled with the ubiquitous medium of information delivery means has made MOOCs as one of the popular innovative and viable pedagogical tool applications on the web. Although MOOCs have been supported and favoured by all sectors of the society, including in Malaysia, many still dispel and wary of their effectiveness in delivering educational content because in reality, there are various challenges and threats that would hamper their success (Zheng & Yang, 2017).
To date, abundance of research has been conducted to examine the effectiveness of online learning over traditional classroom on student's learning outcomes. While some studies found significant positive effects (Bralic & Dvjak;Bowen & Ithaka 2012;Deterding, et al. 2011;Lack 2013;Kapp, 2012;Israel, 2015), others reported no significant effects (McCutheon, et al., 2015;Means, et al. 2010) while some indicated mixed or negative effects (Figlio, et al. 2010;Xu & Jaggers 2013). However, it should be noted that most of the afore-mentioned studies have investigated the effectiveness of MOOCs as one of online pedagogical and technological tools over traditional classroom teaching in relation to learning outcomes. Scarce studies were found examining MOOCs with regard to academic achievements.
Academic achievements indicate how well students perform in relation to a course or program, whilst learning outcomes are knowledge and skills that students acquire after completion of course or program. Both academic achievements and learning outcomes are of equally important pedagogical tools to assess the effectiveness of student's learning. As abundance of studies have focussed on learning outcomes, this study is motivated to bridge the gap in which its objective is to examine the effectiveness of two modes of learning; MOOCs in blended learning and traditional classrooms in relation to academic achievements.
The following section will review past literature in relation to the variables that form the bases for the hypothesis development of the current study. It will then be followed by the description of the methodology used in data collection before the sections on discussions, theoretical and managerial implications are presented. Finally, the paper will highlight the limitations of the study before it ends with a summary of the study.

Massive Open Online Courses (MOOCs) and Blended Learning
Blended learning is a mode of learning that combines online learning and traditional face-to-face interactions, whereby the online learning is taught in a classroom. MOOCs can be delivered either entirely over the internet or through blended learning. Blended and purely online learning have always been classified in a similar mode by past studies that are online learning (Nguyen 2015). According to Bilington and Fronmeller (2014), MOOCs present greater challenges of high-quality interaction compared to other learning modes. These modes of learning are preferable due to their effectiveness in educating learners, enhancing professional development and providing world class education to everyone with internet connection (Koller et al., 2014;Lorenzetti 2013). Traditional classrooms emphasize face-to-face interactions, whereby teaching and learning are centered on educators. Online learning, on the other hand, provides opportunity for learners to learn in a virtual classroom, causing the role of teaching and learning to be shifted from educators to learners. Although online learning offers a more learner-friendly approach, the learners must struggle on their own without educator's guidance (Zawacki-Richter et al., 2018). Blended learning on the other hand, takes the best features of those two modes with the use of technology. In short, apart from embracing technology, blended learning upholds human connections.

Traditional Classrooms
Traditional classrooms refer to physical "brick and mortar" classrooms, in which knowledge is delivered through face-to-face interactions between an educator and learners. In traditional classrooms, marker pens and whiteboards are important teaching tools used to impart knowledge. However, due to the advent of knowledge economy, the traditional classrooms have started to lose their monopoly as the place of learning. To date, the internet has made significant changes to almost all spheres of teaching and learning (Nguyen, 2015).
To date, many studies have been conducted to examine the effects of online learning over traditional classrooms in relation to learning outcomes. The results were found inconclusive due to their mixed findings (Bowen & Ithaka 2012;Deterding et al., 2011;Feeley & Parris 2012;Figlio et al., 2010;Kapp, 2012;Lack, 2013;McCutheon et al., 2015;Means et al. 2010;Nguyen, 2015;Xu & Jaggers, 2013). McCuthcheon et al. (2015) for one, compared the effects of a clinical skill course on learning outcomes among undergraduate nursing students from teaching delivered online and in traditional classrooms. They found that both modes did not have any significant effects on learning outcomes.
In a similar research, Nguyen (2015) did a meta-analysis on 92 studies to examine the effectiveness of online teaching over traditional classrooms in relation to learning outcomes. He found that 92% of the studies indicated that online learnings were more effective than traditional classrooms while only 3% of the compiled studies showed the reverse, i.e. traditional classrooms were more effective than online learning. The remaining 4% indicated mixed findings.
Prior to that, Lack (2013) also conducted a meta-analysis on 30 studies to investigate the effect of online learnings over traditional classrooms on learning outcomes. The results however contradicted with Nguyen's (2015) as the former reported mixed findings. Some studies revealed that students who learned via online learning were more likely to perform better while others indicated the reversed results. There were also studies that showed no significant effects between the two learning modes. Based on her analysis, Lack (2013) reported that there was no conclusive result as to whether online learning was more effective than traditional classrooms with regard to learning outcomes. Means et al. (2010) conducted a meta-analysis research from 1996 to 2008 on more than a thousand empirical studies of online learning. They compared online learning with traditional classrooms in relation to students' learning outcomes. Based on analyses, they found that students who learned through online learning were more likely to perform modestly better than those in traditional classrooms. Moreover, the analyses indicated that the difference in learning outcomes was higher in studies where online learning was blended with face-to-face interactions.
On the effects of online learning on motivation, Harandi's (2015) study, involving 140 students in one of Tehran universities reported that online learnings were more likely to motivate students to learn as they tend to be more engaged with technology and successfully achieved the learning outcomes. Bowen and Ithaka (2012) conducted an experimental study by dividing students into two groups. The first group, identified as the experimental group, was taught via online through blended learning that took place in a classroom. The students in this group met once a week and they did most of the work online. The second group was assigned to a traditional classroom and identified as the control group. This group learned in a classroom through face-to-face interactions. The study found that the students who learned through blended learning were more likely to achieve better learning outcomes than the students who learned in the traditional classroom. Feeley (2012) examined the effect of an online pedagogical tool namely PeerWise, over a traditional classroom in relation to learning outcomes. The study was conducted on a course taken by undergraduate political science students using a mixed-method approach. They found that students who used the online tool were more likely to have better learning outcomes than those who learned in the traditional classroom. Moreover, the students in the former mode were more motivated and enthusiastic to learn as opposed to the students in the latter mode. Figlio et al. (2010) investigated the effects of online learning versus traditional classrooms in relation to learning outcomes through an experimental study by grouping students into two groups namely experimental and control groups. Students in the experimental group were taught with on-line learning, while those in the second group were taught in the traditional classroom. Students of both groups were given similar supplemental materials and instructions. From the study, Figlio et al. (2010) discovered that only the traditional classroom was more likely to have positive effects on learning outcomes. Additionally, they found one possible and very likely significant internal validity threat which was treatment diffusion for the 'live-only" students since everybody could look at the online lectures using a friend's account. Educators also could not monitor whether the students focussed and paid attention on teaching deliveries as they were conducted in a virtual classroom.
Despite the many research findings that showed online learning was more effective than traditional classrooms on learning outcomes (Bowen & Ithaka, 2012;Deterding et al., 2011;Feeley, 2012;Kapp, 2012;Lack, 2013;Nguyen, 2015), there were also studies that indicated no significant effects between those variables (McCutheon et al., 2015;Means et al. 2010). Yet, there were studies that indicated negative or mixed findings; either traditional classroom were more effective than online learning or both learning modes had significant effects on learning outcomes (Figlio et al., 2010;Rush & Yin, 2010;Xu & Jaggers, 2013).
In conclusion, the findings from previous studies provided inconclusive outcomes but intertwining paths that researchers and educators can ponder at this juncture as related to academic achievements. Whether online learning, traditional classrooms or both have significant effects on academic achievement are yet to be discovered. After reviewing the afore-mentioned literatures, the following four hypotheses are formulated.
H1: There is a significant effect between the pre-test and post-test of experimental group's academic achievements (who learned via MOOC in blended learning).
H2: There is a significant effect between the pre-test and post-test of control group's academic achievements (who learned in a traditional classroom).
H3: There is no significant effect on the pre-test academic achievements between experimental and control groups.
H4: There is a significant effect on the post-test academic achievements between experimental and control groups.

Methodology
The study employed an experimental method with non-equivalent pre-test and post-test control group design in which the hypotheses were tested by using inferential statistics. Respondents of this study were 205 five final year accounting students from one public university in Malaysia. The selection of students in any of the two groups was done by the system. This was to ensure that there was no selection bias in grouping the respondents. Approximately about 102 students were placed in the first group known as experimental control. While the remaining 103 students were placed in the second group referred as experimental group.
Integrated case study course was chosen in examining the effectiveness of learning via MOOC in blended learning over traditional classrooms because it was the only final course for degree in accounting program that can be learned via MOOC. This course was developed and uploaded online by educators in 2018. The course emphasises on complex case studies by integrating knowledge from various disciplines. Students are required to solve and analyse the cases by applying higher order thinking skills. For the purposes of this research, the course was delivered by the educator to learners in two learning modes. The experimental group learned via MOOC in blended learning (teaching via MOOCs in a traditional classroom), while the control group learned in a traditional classroom in which learning heavily relies on face-to-face interactions.
Two instruments were used for this study namely pre-test and post-test. Both groups of students were required to sit for a pre-test (O) prior to treatment process. The respondents were given a test on a palm oil and rainforest case study of 4 pages-lengths with 4 main questions. The maximum and minimum scores were 50 and 0 respectively, whereby marks would be awarded on appropriate answers given by the respondents. After completing the pre-test within an allocated time of an hour, both groups were given treatments. The experimental group was treated with MOOC (X 1 ) in blended learning while the control group was treated with traditional classroom teaching (TC) (X 2 ). To ensure that the teaching delivery adopted a purely MOOC in blended learning or traditional classroom, the researcher explained to the respondents on the meaning of MOOC in blended learning and traditional classroom modes, methods, scopes, schedules and duration of sessions before the teaching delivery started.
The Integrated Case Study course was delivered to the Experimental Group (EG) and Control Group (CG) in the same scopes, schedules and durations. Both modes had the same lesson plans and learning outcomes for each session. To ensure that MOOC in blended learning and traditional classroom were properly performed, the sessions were monitored by a checklist. The checklist contains 16 items that describe the situation or behaviour of an educator when handling the teaching and learning sessions. The researcher was required to indicate either "yes" or "no" to each item based on the situation or what was usually done by the educator. This was important to ensure that there was no bias against MOOC in blended learning or traditional classrooms. The sessions were conducted for 3 hours in a week for a period of 14 weeks, totalling to 42 hours of delivering sessions.
After completing the treatment of X 1 and X 2 in 14 weeks period, both groups were required to sit for a post-test (O1). They were given the same questions with similar assessments. Data or scores obtained by the students were compared to examine any significant effects between pre-test and post-test.
The data were analysed by multivariate of MANOVA and paired sample t-test. These two tests were carried out to examine whether there was a significant effect on academic achievements (pre-test and post-test) before and after the treatment was given for EG and CG. In the context of this study, the results or scores obtained by the students would represent their academic achievements. During the test sessions, all respondents were in good health and fit to answer. They were placed in a comfortable and conducive surrounding with no disruptions. Table 1 depicts the research design. The scores for pre-test and post-test of a case study were analysed using paired samples T-test and multivariate analysis of variance (MANOVA). The paired samples T-test was used to test whether there were significant effects in the scores obtained by the experimental or control group. On the other hand, the MANOVA was used to test whether there were significant effects between marks obtained by the experimental and control groups. Hypotheses 1 and 2 would be tested by paired samples T-test while MANOVA test was employed to test hypotheses 3 and 4.

Normality Test
To ensure that data were within normal distribution, a Shapiro-Wilk test was carried out on pre-test and post-test for experimental group (EG) and control group (CG). The results showed that pre-test and post-test scores of respondent's academic achievements for EG were 0.26 and 0.34 respectively (the academic achievements were based on the integrated case study scores obtained by every student) while pre-test and post-test scores for CG were 0.33 and 0.34 respectively, indicating that the data were within normal distribution.

Reliability Analysis
A reliability test is conducted to examine whether a measuring instrument consistently represents the items it is measuring (Sekaran and Bougie 2010). The results of Kuder Richardson (KR 20) of pre-test and post-test reliability Cronbach alpha indicated that EG obtained 0.88 and 0.75 respectively, while CG attained 0.73 and 0.70 respectively. All values were greater than 0.70 as recommended by Sekaran and Bougie (2010) in which the instruments were consistent in representing the measured items. According to Nunally (1978), the values indicate that the data are reliable for further analysis.

Effects of MOOCs and Traditional Classrooms on
Academic Achievements The first hypothesis states that there is a significant effect between the pre-test and post-test of experimental group (EG)'s academic achievements (who learned via MOOCs in blended learning). The EG's mean scores for pre-test and post-test were 16.18 and 40.37 respectively, while the standard deviations for pre-test and post-test were 1.88 and 1.96 respectively. The results of paired samples T-test supported hypothesis 1 which indicated that there was a significant effect on the academic achievement scores for the EG between the pre-test and post-test (t = 64.19, p < 0.05). Table 2 depicts the results of paired samples T-test for experimental group who was treated with MOOCs in blended learning. The second hypothesis states that there is a significant effect between pre-test and post-test of control group's academic achievements (who learned in a traditional classroom). The results indicated that the academic achievement mean scores for pre-test and post-test were 15.95 and 28.31 respectively, while the standard deviations for pre-test and post-test were 1.98 and 1.93 respectively. The results of paired samples T-test supported hypothesis two and indicated that there was a significant effect on the academic achievement mean scores obtained by the CG between pre-test and post-test (t= 40.01, p < 0.05). Table 3 depicts the paired samples T-test for the control group who learned in a traditional classroom. The third hypothesis proposes that there is no significant effect on the pre-test academic achievements between experimental and control groups (before being treated with MOOC in blended learning and traditional classrooms). Multivariate analysis of variance (MANOVA) was adopted to test the hypothesis by comparing the academic achievement mean scores of experimental and control groups, followed by testing whether the scores were significantly different from one to another.
The results of MANOVA indicated that the academic achievement scores obtained by EG and CG were 20.33 and 21.37 respectively. The results further indicated that the t-value was 0.07, (p > 0.05), indicating that there was no significant difference in the academic achievement mean scores between the groups; thus, hypothesis 3 is supported. This showed that both groups consisted of students with equal level of academic achievements before the treatment was given, which is a pre-requisite for experimental study criteria. Table 4 depicts the MANOVA results for pre-test academic achievements for EG and CG before being treated with MOOC in blended learning and traditional classrooms. p<0.05, t value is greater than 1.96 **p<0.01, t value is greater than 2.33 *** p<0.00, t value is greater than 2.58 The fourth hypothesis postulates that there is a significant effect on the post-test academic achievements between EG and CG. The results showed that the mean scores for EG and CG were 42.27 and 31.23, respectively. The results supported hypothesis 4 that there is a significant difference in the academic achievement mean score between the two groups (t = 508.85, p < 0.05). Table  5 depicts the results of MANOVA on post-test academic achievements between groups. p<0.05, t value is greater than 1.96 **p<0.01, t value is greater than 2.33 *** p<0.00, t value is greater than 2.58

Discussion
The first research objective is to examine whether there is a significant effect between the pre-test and post-test of experimental group's academic achievements who learns via MOOC in blended learning. The results indicated that there was a significant effect on pre-test and post-test mean scores of the academic achievements within the experimental group (t = 64.19, p < 0.05, n=103). The results are in tandem with prior studies (Bowen & Ithaka, 2012;Deterding et al., 2011;Feeley, 2012;Kapp, 2012;Lack, 2013;Nguyen, 2015) that examined them in relation to learning outcomes. These studies reported that online blended learning had a significant effect on academic achievements.
The second research objective is to examine whether there is a significant effect between the pre-test and post-test of control group's academic achievements (students who learn in traditional classrooms). The results indicated that there was a significant effect between pre-test and post-test mean scores for the control group's academic achievements (t = 40.01, p < 0.05, n = 102). The results were likely to be consistent with the work of Xu andJaggers (2013), andFiglio et al., (2010) which indicated that traditional classrooms do have a significant effect on students' learning outcomes.
The third research objective is to examine whether there is no significant difference in the pre-test mean scores of the academic achievements between experimental and control groups. The results supported the hypothesis in which it indicated that there was no significant difference in the mean scores of the academic achievements between the two groups (t = 0.084, p < 0.05). In other words, the results indicated that both groups (before being treated via MOOC in blended learning and traditional classrooms) had equal level of academic achievements. The results fulfilled the experimental study criteria of the respondents' homogeneity in which members in the two groups should possess an equal level of knowledge before they were given treatments.
The fourth research objective is to examine whether there is a significant effect on the post-test academic achievements between experimental and control groups. The results showed that the post-test mean scores for experimental and control groups were 40.37 and 28.31 respectively. The results supported hypothesis 4 in which it indicated that there is significant effect on the academic achievement mean scores between groups (t = 52.31, p < 0.05). The results concurred with the prior studies (Bowen & Ithaka, 2013;Feeley, 2010;Kapp, 2012;Lack, 2013) which indicated that the learning outcomes in the group of online blended learning were significantly different from those who learned in the traditional classrooms.

Implications to Academics
This study contributes to the existing body of pedagogical literature on the importance of blending MOOCs in a traditional taught course. Although totally relying on online learning is cost effective, it impedes social interactions between educators, learners and peers. A combination of online learning and a traditional classroom is the best pedagogical approach not only to accelerate academic achievements, but also to harness soft skills. The skills can only be acquired through personal interactions which are essential for one to succeed in the workplace.
This study also provides an implication that in blended learning, the burden of learning is shifted from educators to learners. MOOCs provide a platform to facilitate communication between the educators and the learners, and enable them to share learning materials. Learners have opportunities to learn in virtual classrooms. They are motivated to be engaged and take responsibility for what they are learning and possibility to learn at their own pace. These would increase confidence as they are able to learn to use new tools and handle more complex tasks through supports and capabilities provided by technology.
The study also implicates that blended learning requires both academic fraternities namely educators and learners to be computer literacy and technology acceptance. Indeed, technology is important to increase knowledge retention, reduce stress related to adopting big amount of content and assist learners to face the challenges in the advent of digital and technology era.

Implication to Management
Particularly to management, this study provides the following implications. First, the management should aggressively support MOOCs through blended learning in various forms. Blended learning is a good fit in academic setting as it takes the best out of the two worlds. Online learning offers open-access, video-based instructional content, problem sets and forums and cater unlimited number of learners, while traditional classrooms provide face-to-face interactions. On one hand, it accommodates different learning needs and a well-designed delivery styles and preferences, while on the other hand, face-to-face deliveries are becoming useful in solving complex and personal issues that are unable to be resolved by technology.
Secondly, the management is expected to increase investments in human and technology resources by formulating policy, strategy and action plans for the diffusion, implementation and sustainability of online courses. Academicians who agree to be involved in the development of online courses should be given incentives by recognizing their efforts. This can be done in the form of alleviation of other duties such as lecture time, marking, research and administrative duties amongst others.

Limitations of the Study
A few limitations are acknowledged after the study was carried out. The first limitation identified is that some of the respondents requested others to complete their online assignments. As a result, they may not be able to follow all the self-guidance teaching materials which could affect their understanding of the course. This would affect their scores on the post-test. However, the number of the respondents was minimal and did not affect the validity of the whole results.
Another limitation identified is that this study was conducted in a quantitative mode, whereby the participants were required to answer a case study in two tests namely pre-test and post-test. They could not freely express their views and highlight the problems encountered while learning via MOOCs in blended learning and traditional classrooms. This limits the in-depth understanding of the problem that should be unleashed. To overcome this methodological issue, future studies are suggested to be conducted in a qualitative mode. The qualitative analysis enables the researchers to explore the problems from the students, parents, teachers and policy makers' point of views. Hence, the issues can be analysed in a more comprehensive and holistic manner that may contribute for the betterment of the learners.
The research also used small samples. Future research is recommended to employ a large scale of samples, augmenting more data on the effectiveness of integrating MOOCs in undergraduate classrooms. This would provide more comprehensive results and increase their consensus and validity.

Conclusions
The study found that both modes namely learning via MOOCs in blended learning and traditional classrooms had positive effects on academic achievements. However, the magnitude of effects of the former was higher than the latter. MOOCs are pedagogical innovations that will change traditional learning idea and provide a new way for acquiring knowledge to meet the competency demands of a digital and knowledge driven society. Despite being highly preferred by those who are technology driven and offers learners the opportunity to learn in virtual environment, online learning hinders social interactions between people. The interactions are essential to harness soft skills needed to connect with people and make them successful in the real world. This contrasts with a traditional classroom in which the delivery heavily relies on face-to-face interactions. Blending MOOCs with traditional classrooms in a taught course would give the best of these two pedagogical approaches in which it impetuses learners' interests in learning, increases motivation and self-esteem, accelerates understanding as well as connects people; individual and worldwide. Hence, it is hoped that MOOCs through blended learning would prepare students for the future as they become more agile to meet the challenges in relation to the vibration of technologies and thus elevate them to achieve the highest academic achievements.