Face-to-Face vs On-line: An analysis of Profile, Learning, Performance and Satisfaction among Post Graduate Students

The aim of this study is to explore the differences between face-to-face and on-line students in a post graduate education program. The variables considered are Post Graduate Student’s profile, competences and learning outcomes, academic performance and satisfaction. The sample was composed by 47 students (64% face-to-face). Analysis of variance (ANOVA) and student’s t utilizing SPPS Statistics 22.0 were performed. Results showed differences in all variables: (i) Regarding student profile, face-to-face students were younger and from a broader range of nationalities; (ii) Both students’ profiles showed positive and significant differences between their pre-post competences, learning outcomes and self-evaluation scores in several of the programs courses. Moreover, there were significant differences when considering specific courses and profiles; (iii) Face-to-face students obtained better grades in 4 out of 7 courses of the post graduate program; (iv) Finally, face-to-face students reported higher satisfaction and a more positive perception of the teaching methodologies utilized than on-line students. Theoretical and practical implications are discussed to improve specific teaching methodologies for on-line students.


Introduction
Nowadays, distance education is consolidated within university setting. This learning methodology allows access to higher education to a great diversity of people, addressing relevant issues, such as mobility, work-compatibility and conciliation of studies and personal life. In addition, distance education has become a powerful strategy for making higher education available for more people, regardless their economic and social conditions. Thus, actually more and more people access postgraduate and undergraduate studies than ever before. However, this is an incipient area that still needs improvement. The use of the latest developments of information and communication technologies (ICT) could be a useful tool to improve distance education [1].
The information and telecommunications revolution emerged around twenty years ago. Since then many proposals have been developed for using ICT in the distance education's field. Hence, the concept of "distance education" has evolved to the idea of "on-line education" and new teaching and learning strategies, methods and environments have been created [2]. Many on-line programs have emerged worldwide and the number of postgraduate studies has broadened. This new educational reality requests a pedagogical adaptation [3].
Considering this new reality, scholars and lecturers need to question the utility and the efficacy of the classical pedagogical tools within a new learning context [3]. According to academic satisfaction literature, teaching methodology well-fitted to students' characteristics is a relevant factor to improve quality in the learning process [4]. Thus, a good adaptation to student's characteristics has been proposed as a key strategy to improve students' satisfaction, which in turn would determine the success or failure of an e-Learning process [5]. In addition, there is evidence that academic satisfaction is related to academic performance in a higher education setting [6].
From an evidence-based perspective, it is necessary to evaluate the characteristics of the students to design a good individualized academic program. The main objective of this study is to assess the differences in personal characteristics, satisfaction, learning and performance between face-to-face and on-line students of a Master in Work and Organizations Psychology and Human Resources (WOPHR). Results will be useful for academic program curriculum development, as well as to assess the inclusion of methodologies aimed at increasing the quality of teaching-learning process in the on-line program.

Postgraduate in Work and Organizations Psychology and Human Resources
Improving technical skills is a significant strategy for professional development as it could be useful in enhancing employability [8]. In the Spanish and European context, practitioners and former university students (i.e., Human Resources workers, consultors, organizational psychologists, and managers) are currently interested in enrolling in a Master degree program in WOPHR. Recent graduates need to improve their knowledge and professional skills, but practitioners also need to update and reinforce their knowledge.
Online postgraduate program facilitate access to high quality educational programs regardless geographical position. This is particularly important for workers who seek to improve their knowledge and skills through higher education and or graduate specialization, but have limited time for face-to face lessons. Research highlight the different motives, personal and social characteristics students have to start an online university program, when compared to those who prefer face-to face instruction [7]. These differences are crucial for the program success, since they affect students' satisfaction and quality of learning. [5].
Taking into account the proposals and the information exposed above, this study explores the differences and similarities between both kinds of students in a specific Spanish postgraduate in WOPHR. We are interested in evaluating not only differences in personal characteristics but also in satisfaction and learning levels. This information will be useful to improve the on-line methodology, the most required in the last few years.

Overview to the Present Study
The present study was conducted at the Master degree program in Work and Organizations Psychology and Human Resources at the Universitat Jaume I (UJI), Spain (now in its 11 th edition). Students were enrolled in two different learning methods: face-to-face and on-line. Both methods shared learning objectives and technical content. The same instructors taught both methods. However, teaching methodology was slightly different. Face-to-face students participated on classroom lessons while on-line students took lessons through live streaming. Additionally, a virtual learning environment was used to support both face-to-face and on-line students. Both kind of students have access to all the class recordings and could communicate with teachers and peers through virtual forums. Tutoring through chat was offered to on-line students while face-to-face tutoring was conducted in traditional in-person sessions.
UJI is committed to improve programs' quality of learning. The present study belongs to the University strategy for Educational Innovation Teaching. A wide variety of students and professionals are enrolled in UJI Master in Work and Organizations Psychology and Human Resources. The objective of this paper is to analyze the differences between students of both modalities. Results will serve as empirical evidence for the introduction of methodologies and educational innovation in the on-line mode. These changes will be aimed at increasing the quality of teaching up to face-to-face modality levels.

Participants and Procedure
The total number of students enrolled in the master program was 52 (36.5% on-line). As five students (9%) didn't fill the questionnaires, the study sample was 47 students (66% women), 64% (n=30) of them were face-to-face students and 34% (n=17) were on-line students. The average age was 30.7 years old in the face-to-face group and 36.4 in the on-line group. Besides, 30% of the face-to-face students and 32% of the on-line students came from careers other than Psychology. Finally, 24% of the face-to-face students and 14% of the on-line students were not from Spain.
At the beginning of the academic year 2014-15 the students were invited to participate in a research project organized by the University. This study focused on supporting Educational Innovation initiatives developed by the team of teachers in charge of the WOPHR postgraduate program. The students signed a participation agreement and received an individual report about their learning outcomes at the end of the academic year. Furthermore, this information helped them to elaborate a part of their Master Final Project. Learning outcomes were measured using their perception of proficiency and development of specific competences both at the beginning and the end of each course. The researchers distributed the questionnaire to face-to-face students at the first class of each course (pre-test) and after the course's final exam (post-test). On-line students also completed the questionnaires at the beginning and at the end of each course, through the on-line teaching platform. The questionnaire was the same for both face-to-face and on-line students. It took approximately 5 minutes to complete each questionnaire. They consisted of self-evaluation of the specific competences and learning outcomes described in each course's official syllabus. The students were asked to create their own individual identification code, based on personal information; formed by four letters and two numbers.
Confidentiality of the responses was guaranteed. By following this method, the research team ensured strict compliance with applicable regulations, especially regarding the utmost confidentiality in handling specific data of academic performance measured by student scores in each course.

Student Profile
This information was collected by analyzing the application forms of students. We looked for basic sociodemographic antecedents, such as age, nationality, and employment.

Competences and Learning Outcomes Questionnaire
These were assessed by an instrument constructed by the research team, based on two dimensions: competences and learning outcomes. The scale's items were specific for every course. They take into consideration the specific competences and learning outcomes proposed in each course's syllabus. (i) Competences were evaluated through three items, depending on the courses' content, with a 4-point Likert-type scale, based on the European Certificate in Psychology qualification standards (EuroPsy) [9], ranging from 1 (insufficiently developed) to 4 (completely developed). An example item for a course entitled Human Resources Management Techniques is: 'indicate the degree of development that you possess of the following competences to this day: -Interpreting the different theories and development processes of Occupational Health Psychology and Ergonomics-'. (ii) Learning Outcomes were evaluated through an 11-point Likert-type scale, ranging from 0 (Incapable) to 10 (Capable). The items were formulated based on the self-efficacy evaluation instruments by Bandura [10]. An example item from the course Human Resources Management Techniques is: 'I'm able to evaluate different psychosocial risks among different kinds of jobs'.
The total number of courses included in this study was seven. The selection criterion was that all the students from both conditions participated in the course. The courses' code, name, and Cronbach Alpha [11] of the Learning Outcomes Scale for each one can be seen on Table 1.

Academic Performance
Academic Performance was measured by students' scores in each one of the courses. The standard scoring scale utilized in Spain ranges from 1 (Rejected) to 10 (Approved with Honors). The global mean score for all courses was 7.95. The global mean score for on-line students was 8.0. The global mean score for face-to-face students was 7.9. Finally, the global minim score was 6.9 and the maximum score was 8.35.

Satisfaction
Student satisfaction was evaluated by analyzing two different qualitative information sources: a final Master's satisfaction survey and the complaints and suggestions mailbox. Students' participation in these actions was voluntary, so data collected was not extensive.

Analytic Strategy
To achieve the objective of this study, descriptive analysis, mean score differences using t-student test and differences between groups mean scores using ANOVA where conducted with SPSS 23.0 statistical program. Table 2 shows the results for total and per group mean score differences in competences and learning outcomes. For total scores, mean differences between pre and post measurement points were statistically significant for all the courses.

Competences, Learning Outcomes and Performance
When comparing both groups, the face-to-face students reported lower levels of competences and knowledge previous to the Master (see pre-mean cells in table 2). Furthermore, they had significant lower mean scores than the on-line students at the pre-measurement point of competences in one course (SBE004) and knowledge in four courses (SBE001, SBE002, SBE004 and SBE006). The same pattern of results was found in the measurements after the courses. The on-line students reported higher levels of competences and knowledge (see post-media cells in table 2). They showed significantly higher scores at the post measurement for courses SBE001 and SBE006 regarding competences and for course SBE003 regarding learning outcomes.
Beside, increase in levels of competences and learning after the courses was different in both groups. The on-line students' competences increased by 1.36 between pre and post measures whilst the face-to-face students reported an increase of 1.27. In addition, the perception of contents knowledge after the courses was higher in 4 out of 7 courses in the on-line group. However, learning outcome measures showed a higher increase for the face-to-face students. The face-to-face learning increased by 4.15 between pre and post measures. The on-line students reported an increase of 3.24. In addition, the perception of learning increase was higher in 5 out of 7 courses for the face-to-face group (table  2).
Finally, the comparison of grade descriptive mean scores shows that the face-to-face students had higher scores than the on-line students in 4 out of 7 courses. Further details are shown in Figure 1.

Students' Profile and Satisfaction
The analysis of the student's application forms showed differences in age, nationality and employment. Generally, the on-line students were older than the face-to-face. They were living in different cities (Spain and Latin-American countries) and most of them had a full time job. On the other hand, most of the face-to-face students were recent graduates.
Moreover, the face-to-face students reported higher satisfaction and a more positive perception of teaching methodologies utilized than their on-line counterparts. Furthermore, the on-line students reported difficulties to keep up with courses work and to attend to live streaming lessons or to watch the full video about each session. Finally, some of the on-line students reported problems when they watched the live stream lessons or the full video classes. They could not understand some group dynamics that took place in the face-to-face sessions.

Conclusions
The aim of this study was to explore the differences and similarities between face-to-face and on-line students in a Spanish postgraduate in WOPHR. Results showed differences between both type of students in profile, competences, learning outcomes, academic performance and satisfaction.
Both groups had a different student profile. As follows, on-line students were older than face-to-face ones. They were living in different cities and countries. Most of them had a full time job or usually did not have plenty of time. Unlike the WOPHR Master's face-to-face students, on-line students did not fit a 'recent graduates' profile. They tended to be practitioners and people without too much time to spend on long activities, such as watching live stream lessons, watching full video classes, or reading an amount of papers and book chapters.
Moreover, face-to-face students valued more positively and were more satisfied with the methodology used, than their on-line counterparts. We consider that lower satisfaction levels in on-line students, could be related to their special characteristics. It had been proposed that a mismatch between teaching methodology and on-line student's profile could be harmful to student's satisfaction, engagement and psychological well-being [5]. We suggest that this mismatch could be affecting the satisfaction of on-line students of the postgraduate in WOPHR. Furthermore, according to literature, we suggest that the relationship between a non-profile-fitted methodology and low levels of satisfaction could negatively affect academic outcomes, such us performance, competences development and perception of learning [5,6].
In this sense, considering the outcomes in both groups, our results showed some differences when comparing the two modalities. Both face-to-face and on-line students perceived that they have acquired new knowledge and they have developed their competences. However they didn't start from the same point of departure and they didn't reach the same finish line. on-line students reported higher levels of knowledge before the Master. As per our understanding, these results are related with the differences in students' profiles. Most of the on-line students were practitioners or not "recent graduates" students. Hence, they were students with some working experience in WOPHR. In this sense, we can assume that some prior knowledge is necessary to work in the WOPHR field, which could explain the differences found between on-line and face-to-face students' perceptions of WOPHR knowledge at the beginning of the postgraduate.
On the one hand, the higher scores found in the on-line group at pre-measurement point, were found again at post-measurement point. On-line students reported higher levels of competences after the courses and their levels increased more than their face-to-face counterparts during the courses. On the other hand, they also reported higher levels of learning outcomes after the courses but, in this case, the increase was lower than within face-to-face students. We consider the effects of experience-based learning as an explanation for these results. It has been proposed that professional practice helps students to learn through experience, and this learning is possible due to the capacity for effective reflection involved in the process [12]. According to the literature and considering the on-line students' profile showed in this study, we suggest that WOPHR Master's on-line students had a higher capacity for self-analysis than the face-to-face students. This capacity might be related to higher levels in work self-efficacy beliefs (competences) [13]. Nevertheless, we found a mismatch between on-line teaching methodology and on-line student's profile. According to the literature, a non-profile-fitted methodology could negatively affect the educational outcomes [5,6]. Thus, face-to-face students obtained higher levels of performance (mean grade scores) than their on-line counterparts.

Contribution and Further Research
This study shows differences in students' profile and learning outcomes between face-to-face and on-line students of a specific Spanish postgraduate in WOPHR. Regarding on-line students, results shows a non-profile-fitted methodology and low levels of satisfaction and learning. According to on-line education literature, this information should serve as empirical evidence for the introduction of methodologies and educational innovation in on-line models in order to improve the on-line methodology quality and to reach the face-to face modality levels. Additionally directions for future research include testing effectiveness of the new strategies implanted.
This study has several limitations, which highlight important avenues for future research. First, our results are based on a sample from the same year, and sample size shall need to be increased. Thus, it may be interesting to repeat this study in subsequent years. Indeed, similar studies have already been designed for future editions of the Master in WOPHR. Second, students' participation in the satisfaction survey was voluntary and data collected was not extensive. However to increase data, we analyze another source related to satisfaction: the complaints and suggestions mailbox, where students' participation was as well voluntary. Strategies to increase participation should be developed for future research. Third, we found mean differences between face-to-face and on-line students, but we did not find a high level of statistical significant differences in these comparisons. As we exposed above, this Master in WOPHR is a high-recognized postgraduate degree, which possess high quality requirement from both students and teachers. Therefore, learning outcomes use to be very satisfactory (see figure 1) making it difficult to obtain significant differences when comparing both groups. Finally, this study shows some evidence about the relationship between e-learning methodology and educational outcomes. However, further research is necessary to explore the quality of teaching strategies as a decisive factor in the on-line student performance and satisfaction.
A great diversity of students and professionals are enrolled every year in the on-line modality of UJI Master in Work and Organizations Psychology and Human Resources. In addition, it is the most required modality in recent years. To meet the needs of this new social reality, it is important to improve the on-line program's quality using a strategy for Educational Innovation Teaching. Recent pedagogical theories, models and methodologies point to Positive Education [14] and affective e-learning [5], as the scaffold of knowledge in which to build actions aimed at improving this modality.