An Empirical Evaluation of Critical Factors Influencing Learner Satisfaction in Blended Learning: A Pilot Study

Blended learning, a convergence of e-learning approach and face-to-face learning, has been regarded as a new paradigm in modern education. The degree of learners’ satisfaction with blended learning played a crucial role in evaluating the effectiveness of blended learning adoption. Therefore, this study examined the primary factors affecting learners’ satisfaction in blended learning environment. A comprehensive questionnaire was used to assess the impact of six dimensions such as learner, instructor, course, technology, design and environment, on perceived e-learner satisfaction specifically on the e-learning component within the blended learning environment. The respondents for this study were generally young with an average age of 20 years old. Results from univariate regression analysis revealed all six dimensions to be positively associated with perceived e-learner satisfaction. Subsequent multivariate regression analysis evidenced only the design dimension to be the critical factor influencing perceived e-learner satisfaction. This study concluded younger generation prioritized the design dimension to be the most critical factor in their satisfaction towards e-learning component in blended learning environment. Therefore, it may be more strategic for institutions to emphasize on the design dimension in their e-learning implementations within the blended learning environment specifically for younger learners.


Background/ Objectives and Goals
The advancement of Information and Communication Technologies (ICTs) has enabled the younger generations to be well equipped with technological devices such as smartphones and tablets. The impact of this adoption has been enormous since there were no economic and technical barriers for the younger generations to create share and distribute digital contents across a broad range of devices [1].
In view of this technological shift, the education sector has integrated the application of ICTs in education resulting in the establishment of e-learning platforms [2]. Competitive global institutions of higher education were under significant pressure to expand their delivery of courses through e-learning platforms such as Moodle or Blackboard. Past studies have shown the adaptation of e-learning to be effective in connecting people and resources, facilitating active learning, deepen understanding, enhancing critical thinking skills as well as promoting creative communication [3][4]. However, major drawbacks in integrating e-learning into education included high initial costs for preparing content materials, substantial costs for system maintenance as well as students' feeling of depression and isolation in virtual environment [5]. In addition, the low completion rates of e-learning courses, and the important of instructor-student and student-student interactions in classrooms have subsequently suggested that e-learning alone was unlikely to be the most effective strategy for teaching and learning [6][7]. Hence, questions related to learners' satisfaction and effectiveness in e-learning approach were raised.
In recent years, blended learning, a convergence of e-learning and face-to-face learning, has emerged to be a promising alternative learning approach compared to conventional e-learning approach [5]. Graham [8] argued the blended learning to be the "new traditional approach" in education because it maximized the best advantages of face-to-face and e-learning approaches. Nevertheless, learners' intentions to continue to use blended learning were strongly influenced by their satisfaction [9]. In order to evaluate the potential success of educational design based on novel technologies, it was essential to understand the learners' attitudes, perceptions as well as their level of acceptance and satisfaction [10].
The adoption of blended learning in supporting learning has made it significant to probe the vital determinants that would entice learner to use blended learning and enhance their learning satisfaction. The degree of learners' satisfaction with blended learning played a crucial role in evaluating the effectiveness of blended learning adoption. Therefore, comprehending the essentials of what determined learners' satisfaction provided insight into developing effective strategies that benefit learners. Hence, this study utilized research model proposed by Sun et al. [11] to examine the primary factors affecting learners' satisfaction toward e-learning component in a blended learning environment.

Theoretical Framework
Sun et al. [11] argued past studies have identified important variables that contributed significantly to the success of e-learning. Unfortunately, these factors tended to focus primarily on technology. A more comprehensive framework was subsequently developed and proposed by Sun el al. [11]. This integrated framework enabled the examination of the impact of vital variables or dimensions such as learner, instructor, course, technology, design and environment on learners' satisfaction toward e-learning component in blended learning environment.
Under these six dimensions, thirteen factors were identified. The learner dimension comprised learner attitude toward computers, computer anxiety and internet self-efficacy. The instructor response timeliness and attitude toward e-learning were clustered in the instructor dimension. Course flexibility and course quality were assessed through the course dimension. The technology dimension focused on technology quality and internet quality. Perceived usefulness and perceived easy to use were categorized in the design dimension. The environment dimension consisted of diversity in assessment as well as learner perceived interaction with others (see Figure 1).

Study Design and Procedure
Blended learning approach was implemented in a General Studies course during the summer semester, which spanned from October 2015 to February 2016, in Monash University Malaysia campus. Both face-to-face lectures and e-learning supports were incorporated in the course design. A total of one hundred students were initially targeted for this study. Potential students were invited for participation through emails. They were well informed that the participation into this study was voluntary and it was their right to withdraw from the study at any time. In addition, students were informed about the background of this study. Online written consent was obtained from students before they embarked into online questionnaires. Despite numerous reminders and follow-up emails, only twenty students completed the online questionnaires.

Study Questionnaire
A combination of questionnaires in evaluating the thirteen factors from six dimensions on perceived e-learner satisfaction was adopted from Sun et al. [11]. A five-point Likert scale ranging from 1 as strongly disagree to 5 as strongly agree was used for the evaluation. Apart from the above questionnaires, the respondents' demographic background information was also collected.

Ethical Approval
All students participated in this study were based upon voluntary participation. This study was approved by Monash University Human Research Ethics Committee (CF15/4617-2015001988).

Statistical analysis
The reliability of the questionnaires was examined using Cronbach's alpha. An alpha value of more than 0.70 was considered desirable [12]. Pearson correlation was applied to examine the relationship between six dimensions, while multicollinearity among the dimensions was investigated using the variance inflation factor (VIF). By convention, a VIF value of more than 10 was regarded as an indication of high multicollinearity [13]. Multiple regression analysis was subsequently performed to assess the relationship of six dimensions on the perceived e-learner satisfaction. All missing responses were excluded from analysis using StataIC version 13.0. Statistical significance was set at 0.05.

Results
The average age of the respondents was found to be approximately 20 years old with a minimum age of 19 years old and a maximum age of 22 years old. Majority of the respondents were female (64%), from the School of Science (36%), in their first semester of undergraduate studies (64%) and majority had minimal prior experiences in e-learning courses (refer to Table 1).

Reliability analysis
As presented in Table 2, the overall Conbach's alpha value for the questionnaires was 0.91, while the Cronbach's alpha value for all six dimensions ranged from 0.88 to 0.94. These values were well above the desirable level recommended by Haywood et al. [12]. Table 2 revealed the Pearson correlation coefficients for all six dimensions ranged from 0.09 to 0.93. The highly correlated dimensions were course and design (ρ=0.93) as well as environmental and technology ((ρ=0.92). Apart from the above, all six dimensions were positively affecting the perceived e-learner satisfaction. The design dimension had the strongest positive relationship with perceived e-learner satisfaction (ρ=0.84), while the weakest positive relationship with perceived e-learner satisfaction was the learner dimension (ρ=0.05).

Multicollinearity
The variance inflation factor (VIF) values for course, design, environmental and technology dimensions were above 10. These findings were consistent with the outcomes from the Pearson correlation. The VIF for Instructor dimension was the lowest (VIF=2.46) followed by the learner dimension (VIF=5.84) (refer to Table 2).

Regression analysis
Through univariate regression analyses, all demographic variables had insignificant relationship with the perceived e-leaner satisfaction. Further analyses revealed four out of the six dimensions were significantly affecting the e-leaner satisfaction. These dimensions were course dimension (p-value=0.01), technology dimension (p-value=0.02), design dimension (p-value=0.01) and environmental dimension (p-value=0.01). Multivariate regression analyses evidenced the design dimension (p-value=0.01) to be the only significant factor for perceived e-learner satisfaction.

Discussion
E-learning to be integrated in addition to face-to-face learning in blended learning environment has been regarded as more effective alternative to face-to-face alone education or e-learning alone. Many institutions implemented e-learning as part of the blended learning to meet learners' need. Therefore, it is equally essential to capture the learners' perceived satisfaction to determine the effectiveness of this learning approach.
This study revealed only the design dimension, which included the factors of perceived usefulness and ease of use, to be the critical factors for perceived e-learner satisfaction. This finding was consistent with past studies conducted by Liaw et al. [14], Calisir et al. [15], Chen et al. [16], Tarhini et al. [17] and Motaghian et al. [18]. The main function of an e-learning system was to provide useful content and to help prepare learners for their studies. Therefore, the higher the perceived usefulness of an e-learning system, the more satisfaction learners had.
Learner's notion of ease of use was an important antecedent to perceptions of satisfaction. An e-learning system's ease of use encouraged learners to devote their attention to learning the content instead of spending effort learning the system. Consequently, a higher learning satisfaction should be expected. This finding was consistent with Islam [19].
It was also evidenced from this study that younger generation prioritized the design dimension to be the most critical factor in their satisfaction toward e-learning within the blended learning environment. Therefore, this study recommended institutions to emphasize more on the design dimension to capture more satisfaction toward e-learning within the blended learning environment from young learners. An unsatisfactory perception will demotivate learners to continue with their studies.

Limitations
Although this research was conducted under careful and systematic efforts, it was not without limitations. First, the research proposed an integrated model covering a diverse range of factors influencing e-learners' satisfaction, it might not be comprehensive due to limitations of time and resources in collecting data. Second, the response rate for this study was low due to limited response period. Third, this research only considered perceived e-learner satisfaction to be the only dependent variable. It may be interesting to include learning performance and student scores as additional dependent variables.

Future Research
Although blended learning has successful been adopted, the degree of learners' satisfaction with blended learning played a crucial role in evaluating the effectiveness of blended learning adoption. Future research with larger number of respondents is warranted to further ascertain whether the design dimension remains as the sole contributing factor in the effectiveness of blended learning adoption for younger generation.

Conclusions
This study concluded the younger generation regarded the design dimension, which included the factors of perceived usefulness and perceived ease of use, to be the most vital factor in affecting their satisfaction toward the e-learning component within the blended learning environment. This finding has further suggested institutions to emphasize more on the design dimension if they would like to target students from the younger generation.