The Evaluation of Learning Services during the COVID-19 Pandemic

This study aims to evaluate the planning, processes and learning outcomes during the COVID-19 pandemic. This is an evaluative study with a qualitative approach. This research was conducted in the Department of Primary School Education, Faculty of Education, Universitas Negeri Yogyakarta. Research time is in Academic Year 2019/2020 specifically in the COVID-19 pandemic. The evaluation model used is Countenance Stake Model, which contains an evaluation of antecedents, transactions, and outcomes. Subjects in this study were students who took part in lectures and lecturers who did the learning during the COVID-19 pandemic. Data collection is done by documentation, interviews and questionnaire and then analyzed using a qualitative approach. In addition to ensuring the validity of the data of this study, triangulation of sources was carried out. The results showed that the learning service on the antecedent aspect has been running well. This is evidenced by the well-planned lecture services during the pandemic. Another thing obtained in the aspect of transaction that still needs to be improved. This is proven by the fact that effective lecturing methods have not yet been applied during the pandemic. The same thing also happened in the aspect of outcomes that was still not optimal. This is evidenced by the low student satisfaction in learning services during the pandemic.


Introduction
The COVID-19 pandemic has made significant changes, imposing many challenges in the global education community. The education sector is the first few sectors to discuss the rapidity of its activities. Thousands of educational institutions as well as students defected due to the COVID-19 pandemic. Temporarily halting operations is an effort in the education sector to prevent this pandemic [1,2,3,4]. Work from home (WFH) is the best choice today, especially for service organizations. Each service organization is expected to follow the government's advice, even academic institutions must be temporarily closed. This certainly has an impact on academic services. This requires every education service must find new alternatives to academic services, and virtual classes are an effort to adjust work from home [5].
The COVID-19 pandemic has an impact on shock waves throughout the higher education system on a global scale. When the virus spreads, lock orders in all cities, regions and all countries were enforced throughout the world. It is very important to stay at home, maintaining a physical distance that prohibits larger meetings. In addition, the decision to cancel, postpone, or move classes directly online is made on a daily basis in most countries [6]. This certainly demands a sudden change in various academic services such as learning program services. Such an urgent situation requires effective thinking in designing, implementing and monitoring the continuation of the remaining learning so that curriculum goals can still be achieved.
Lecturers have a very important role in determining the quality of lectures carried out. Therefore, lecturers must think and make careful planning in increasing learning opportunities for students and improving the quality of teaching. This requires changes in organizing the class, the use of teaching methods, teaching and learning strategies, as well as the attitudes and characteristics of lecturers in managing the teaching and learning process. The lecturer acts as a manager of the teaching and learning process, acts as a facilitator who seeks to create conditions for effective teaching and learning, thus enabling the teaching and learning process, developing learning materials properly, and increasing the ability of students to listen to lessons and master the learning goals they must achieve. To fulfill this, lecturers are required to be able to manage the teaching and learning process that provides stimulation to students, so students want to learn because students are the main subject in learning. Students' self-regulated learning became a popular learning method during the COVID-19 Pandemic. Through students' self-regulated learning, students can set their own learning goals, determine content and progress, choose skills and methods, monitor the whole process, and conduct self-assessments [7].
The phenomenon felt in the Department of Primary School Education during the COVID-19 era showed that the change in learning services for students suddenly had quite a surprising effect on all parties both lecturers and students. The lack of preparation in welcoming these service changes has impacted on various obstacles to learning services in the field. The constraints in question such as the difficulty of adjusting online learning designed by lecturers, the lack of lecturer preparation for online learning, constraints on learning facilities that are not fully ready and complete for students and technical obstacles such as the internet network that many students complain about during online learning.
The gap felt during this pandemic is the emergence of various complaints from students regarding the implementation of learning during the pandemic. Complaints that arise are quite varied ranging from difficulty in accessing, economic capacity that is also complained to attend online lectures every day to policies that they feel are quite burdensome. This is felt to have an impact on students' motivation to learn so that they are less responsive in attending online lectures, lack of encouragement to complete the assignments to impact on attendance in online classes.
This phenomenon is an attraction for researchers to conduct more in-depth studies related to learning services during the COVID-19 pandemic. The purpose of this study is to have an overview of the evaluation of learning services through the application of the Stake Model.

Methods
This is an evaluation study, for the evaluation of learning services during the COVID-19 pandemic. This research uses qualitative. Associated with research on research involving the process carried out through a study of the activities or activities of the researchers involved in it. This research was conducted in the Department of Primary School Education, Faculty of Education, Universitas Negeri Yogyakarta. Research time in Academic Year 2019/2020 specifically in the COVID-19 pandemic.
In accordance with this type of research is an evaluation study using the Countenance Stake Model, which contains an evaluation of antecedents, transactions, and outcomes. In the antecedent stage an evaluation is made related to the suitability between lecture plans made by lecturers during the COVID-19 pandemic. At this stage of the transaction an evaluation is made regarding the suitability between the lectures conducted by the lecturer during the COVID-19 pandemic. On the results of the evaluation conducted on aspects of student learning products in learning during the COVID-19 pandemic.
Subjects in this study were students who took part in lectures and lecturers who did the learning during the COVID-19 pandemic. Information from students was obtained by involving four classes with a total of 160 students. From these research subjects, the informants were chosen again purposively according to the research questions to be revealed.
To obtain evaluation data on this learning service, interviews, documentation and questionnaire are conducted with the distribution of components, indicators and sources as follows. Based on the research data collection distribution table, information was obtained that from the three evaluated components scattered into seven indicators with data sources from lecturers and students. Data obtained from lecturers for indicators in the antecedents group were obtained through lecturer interviews related to lecture planning. In addition to interviews, collecting supporting documents is also carried out to verify the data obtained from interviews with lecturers. Furthermore, data obtained from the indicators in the transaction group were obtained from students and lecturers through interviews related to 5928 The Evaluation of Learning Services during the COVID-19 Pandemic the ongoing learning process including the application of learning methods and assessment of ongoing learning. The outcomes component consisting of indicators of learning outcomes and student satisfaction is obtained through interviews with students and lecturers as well as gathering related documents to verify data. To obtain broad student satisfaction data, a student satisfaction questionnaire was used.
The analysis technique used in this study is qualitative analysis. In this analysis model, the three components of the analysis are data reduction, data display, and drawing conclusions or verification [8]. In this process the researcher's activities move between the components of analysis and data collection during this process. In addition to ensuring the validity of the data of this study, triangulation of sources was carried out. Source triangulation is done by using various sources of information to obtain valid data. In addition, the use of various data collection methods is also used to verify data obtained from interviews with data obtained from related documents.

Result
This research was conducted in the Department of Elementary School Education, Faculty of Education, Universitas Negeri Yogyakarta. This evaluation was carried out in the second semester of Academic Year 2019/2020. This study is an evaluation research, to evaluate learning services during the COVID- 19 Pandemic. Considering that this research is a study that focuses more on the process and quality aspects, the type of research used is evaluative research through a qualitative approach. The evaluation model in this study is the Countenance Stake Model, which includes the evaluation of antecedents, transactions, and outcomes. At this stage of antecedents, an evaluation is made related to the planning of lecture programs made by lecturers. The transactions phase focuses on the lecture process during the COVID- 19 Pandemic. The outcome phase is an evaluation of the results of lectures that have been carried out during the COVID- 19 Pandemic. The results of this study are described based on the stages of the Countenance Stake Evaluation Model. The results of the study were presented in accordance with the sequence of phases carried out.
One component analyzed from learning services is the planning component. At this stage an evaluation is carried out related to arranging lecture schedules and preparation of lesson plans. Evaluation results based on aspects of arranging lecture schedules indicate that the preparation of lecture schedules during the pandemic is basically still in accordance with the lecture schedules that were arranged before the WFH came into force. However, as long as the WFH takes place the lecture schedule is more flexible, more manageable through the agreement of lecturers and students because the implementation of lectures is not bound by the use of space or lecture halls which are usually a limiting factor for scheduling for classical classifications as usual. Problems changing class schedules can be arranged and adjusted to the terms of agreement between students and lecturers and do not interfere with the course material. However, in this aspect, it was also found that the problem was when some lecturers wanted to arrange an online meeting which might coincide with other lecturers. For this reason, good organization and communication between students and lecturers is needed if they want to move the lecture schedule.
Next is the aspect of preparation of lesson plans. Of course this is a little change because there are technical changes from offline meetings to online meetings. That has an impact on the preparation of lesson plans. In the pandemic, lesson plans are prepared to adjust various factors such as the type of material to be delivered, what platforms will be used in delivering the material or other factors. Based on the findings in the field, it shows that the adjustment of lesson plans is in the syntax of lectures, adjusting the duration of time used and the preparation of other plans that can support the achievement of the objectives of lectures properly. The core of this aspect is the syntax of lectures in lesson plans adapted to online meetings without changing the indicators and objectives of the lectures or the material achievements expected at the meeting. Based on the evaluation of the planning component (antecedents) in general it can be concluded that the planning aspects of the lecture program during the COVID-19 Pandemic were well implemented. Although it is realized that there are challenges faced, it does not really interfere with the course of planning to conduct lectures properly.
The Evaluations on application of learning methods indicators have a focus on how lecturers present lecture material during a pandemic. Evaluation results on this indicator show that generally the presentation of material made by lecturers during online learning is still not as expected. Based on the findings in the field shows that students complain that most lecturers only give online assignments not online lectures. Lectures only go one way, interaction is less because lectures are only focused on giving assignments so there is no opportunity for students to freely give questions and receive feedback from lecturers. Assignments that accumulate from one lecturer and another lecturer make students overwhelmed in completing assignments every week. Besides the condition of the internet network that does not always support making its own challenges for students. While students' expectations are that lecturers can use a lecture method that can realize maximum interaction between lecturers and students so that the material and purpose of lectures can be well understood by students.
Based on the evaluation results on the transaction component, it can be explained that this component has not run optimally. The indicators of the lecture have indeed been carried out in accordance with the specified time and all the time changes have been agreed between the lecturer and students. But in practice, in particular the use of lecture methods is not optimal. The lecture method used is generally still one-way and the opportunity for interaction between students and lecturers is still lacking so that during lectures online the lecturer to student feedback is still not optimal. Therefore this needs to be improved in the future.
The third component in this evaluation is the outcome component. Basically evaluation of this component has a focus on the results of lectures. In this component there are two indicators that are evaluated, namely learning outcomes and student satisfaction. Evaluation results based on learning outcomes indicators indicate that basically students are of the opinion that the value obtained from lecturers during online lectures is quite high. The findings obtained in the field indicate that the average student score during online lectures is high. Data obtained in the f ield shows that students feel that sometimes subjects tend to be random in their assessment. Based on data from one of the class leaders that sometimes there are students who are not maximal in collecting assignments but get high grades and conversely there are students who are maximum in collecting assignments but the scores obtained are low. Even though students realize that giving full value is the right of lecturers and they still accept it.
Another thing is found in student satisfaction indicators. The results of the evaluation of this indicator show that although the scores of students during online learning are high, students have a tendency that is not satisfied with online lectures during the pandemic. Students are of the opinion that online lecture implementation is not optimal, especially in the use of lecture methods that are deemed necessary to improve. An internet network that is not fully equitable for students is also felt as an obstacle that often arises. Although obtaining high grades, students still consider the understanding obtained from lecture material to be less than optimal. But besides that, students still give a positive appreciation to the policy efforts of university leaders who have provided services and facilities during online lectures. The mentioned facilities are like free internet packages during online lectures from home.
Based on the results of the evaluation on the outcomes component, it can be explained that this component is still not optimal. The learning outcomes indicator shows that the students' grades are basically in the high category, but even though they have obtained a high enough score, they still have various complaints and feel less satisfied with the online lecture during the pandemic. Students feel the use of lecture methods needs to be reviewed and adjusted to pandemic conditions. In addition to students hoping that if the pandemic continues and lectures in the next semester will still be based online, the lecture design is expected to be improved and adapted to online learning conditions. Besides that, the students' expectation on positive things has been maintained such as free internet services if lectures will still be held online.

Discussion
The findings of this evaluation indicate that the planning component (antecedent) in general can be concluded that planning in the lecture program during the COVID- 19 Pandemic was carried out well. Regarding the challenges incurred in the lecture planning process but not a significant problem. It is still in the agreement of lecturers and students in any changes that shift the planning of lectures. During a pandemic, educational institutions must quickly change their curriculum to digital teaching. These changes have broad impacts including learning planning. The concept of a classic class must be modified into an online setting to suit the situation [9]. This shows the relevance of the findings of this study about changes in the design of learning plans that are directed to online learning approaches. COVID-19 has a strong impact on society, the traditional economy and even on the educational aspect. Learning from home with Online or Distance learning without lecturer-student interaction is not good for the long term even though it can improve students' digitizing abilities. Blended learning with the Internet of Things model is the most suitable for all stakeholders including faculty, educators, teachers, administrators, staff, and students to keep all interactions maintained during this pandemic [10].
As found in the results of this evaluation that so far, learning planning is directed at distance learning to avoid 5930 The Evaluation of Learning Services during the COVID-19 Pandemic physical interaction on campus. This is the main choice at this time and will continue to be evaluated by the leadership along with the development of the situation. Distance learning through management systems and learning platforms that are different from usual can replace face-to-face learning, and extraordinary efforts are offered by all faculties to continue teaching and assessing their study programs [11]. Face-to-face teaching is part of a common element in distance learning, open learning and resource-based learning. Through this thinking all subjects and disciplines can be taught from a distance [12]. This is important not only for lecturers to ensure that they continue to get to know their students, but also for students to practice various types of knowledge use. The application of social distance and social isolation shows that they are mediated online [13]. Programs that utilize virtual learning environments are still very likely to be continued through various levels of blended learning [14].
Based on these reviews, the planning aspects of lectures for the coming semester are still very possible for the application of distance learning both virtual and others that are still within the scope of online learning. This is very important because the current conditions are different from the conditions in the previous semester where the pandemic came suddenly without any prior planning. The current condition is that we will start the semester by preparing careful planning to achieve maximum goals.
Evaluation results on the transaction component show that this component has not been running optimally. The selection and application of lecture methods is the most highlighted in this component. The lecture method used is generally still one-way and the opportunity for interaction between students and lecturers still needs to be improved so that during lecturing online lecturer-to-student feedback is still not optimal. The main point is not where students and lecturers are, but how to create an effective learning process while maintaining interaction during the lecture even though the lecture is conducted online [13]. Online learning cannot replace the need for face-to-face learning, but it can complement existing traditional classroom-based learning methods. If well designed, online distance learning is the right alternative during the pandemic [15]. Even the use of online learning modules can be a good consideration [16].
Online learning has become a growing trend in the use of educational technology [17,18]. Moreover, during the current pandemic, online learning is the first choice in efforts to support health protocols. Online learning is influenced by the constructivist teaching approach [19,20,21]. Asynchronous distance learning is used to overcome time and location limitations for learning activities and the techno-pedagogical ability of systems for learning administration [22]. Through online learning lecturers must carefully develop lecture content while preparing an activity room for students [23]. This view shows that online learning methods need to be well designed.
Innovation and creativity of lecturers are needed in creating interaction with students so that the lecture process does not go in one direction but can be a fun alternative method and encourage students' motivation to learn and follow the material.
Another interesting thing is about the outcome component. Although the value of student learning outcomes is generally good and quite high, but in this component it is found that students' satisfaction with lecture services is still low. Students' satisfaction is one thing that must be measured in online learning [24]. Generally students complained about the implementation of lectures which they said were not online lectures but only online assignments. This has relevance to previous research that there are various dimensions of factors that influence student satisfaction in online learning such as course structure, learning activities, curriculum, instructor knowledge and facilitation, instructor's presence, and instructor feedback [25,26]. This shows that if the instructor is passive in online learning then this has the most potential to reduce students' satisfaction in lectures.
Other findings are various student complaints that lectures that rely heavily on assignments make students overwhelmed completing assignments for all courses every week. This needs to get attention in the implementation of the next semester lectures. The interaction between lecturers and students is a challenge for online learning methods that must be well thought out and prepared [23]. The results of other study explained that there was no significant difference in student satisfaction between learning done face-to-face and learning conducted online [27]. This shows that it is not a matter of face-to-face or online, but there are things that are more influential in increasing student satisfaction in learning. So many factors have the potential to increase student satisfaction in learning. If related to the findings of this study, it turns out that the learning method applied by many lecturers is complained about and discussed during lectures online. The evaluation results also support that students need interaction in online learning, they need two-way discussion, they need feedback on every lecture activity carried out and not just online assignments. These findings have relevance to previous research that learner-content interaction is the most powerful predictor in increasing student satisfaction in learning [28,29,30].
The success of online courses always leads to student satisfaction [31]. Course content design, selection of appropriate learning methods, understanding student characteristics are important in the success of online learning. The course design should be user friendly, easy to navigate, have clear guidelines and rubrics and must provide clear feedback. What is also important is that the course design must open the interaction between instructors and students [32]. This is an effort to realize student satisfaction in attending online courses. This should be a serious concern in the future.
Program evaluation is a process that is carried out independently when the program has been established as an effort to increase accountability, deeper understanding and improvement of phenomena [33]. The findings of this study indicate that the transition of learning services from synchronous to asynchronous is not an easy process. This is due to the many aspects involved in it. The case in this study is an example that the implementation of lectures from face to face turned into online lectures encounters a variety of challenges that trigger less optimal learning services. For example, the habit factor is one of the triggers for the challenge that face-to-face lectures have become a routine that is inherent both to students and lecturers so that the presence of a pandemic turns everything into unusual lecture activities.
In addition, another factor is the planning factor. This is certainly very influential because the COVID-19 pandemic comes in the middle of an ongoing academic program. Though all the plans that have been prepared previously are designs that assume that normal conditions will continue until the end of the academic year. Of course this needs a sudden shift in the middle of the program so that it also triggers the quality of learning services provided to students.
The most highlighted finding was the application of learning methods during asynchronous lectures. This is very prominent in the results of the study that most students complained about the selection of learning methods in lectures. The challenges of planning and habit factors have strong relevance with the not yet applying methods that are deemed suitable for presenting asynchronous lectures.
Something similar was found in the application of learning assessment. Any changes in the learning process must certainly be followed by adjustments to the learning assessment. The assessment approach that used to be more dominantly focused on learning outcomes must be adjusted to the assessment that is more focused on the process of student activity. So the distance learning process makes the assessment approach more on the application of authentic assignments than by simply answering the quiz questions so that the value determination process still follows the assessment principles that are objective and accountable.
Interesting findings were also obtained from the evaluation of outcomes components. The results showed that the student learning outcomes were basically in the good category. Based on the assessment documents show that the value of students during distance learning is quite high. But this is not in line with student satisfaction, especially in the satisfaction of the learning process. The findings show that 71.8% of informants were not satisfied with the learning process at the time of the pandemic. This dissatisfaction generally arises in the application of learning methods during the pandemic. This is certainly very interesting because it shows the gap between achievement and satisfaction.
The need for a study of learning methods with nuanced distance learning is highly recommended in the findings of this study. This is very important because the current conditions still signal that the pandemic is not over and it still has the potential to continue until the next academic year. Responding to the findings of this evaluation, surely better planning must be done, especially related to the selection of online lecture methods that can arouse the enthusiasm and motivation of students in learning. This is considered appropriate for further research studies or other studies.

Conclusions
Based on the results of research and discussion, it can be concluded that the learning service on the antecedent aspect has been running well. This is evidenced by the well-planned lecture services during the pandemic. Another thing obtained in the aspect of transaction that still needs to be improved. This is proven by the fact that effective lecturing methods have not yet been applied during the pandemic. The same thing also happened in the aspect of outcomes that was still not optimal. This is evidenced by the low student satisfaction in learning services during the pandemic.
Based on the findings of this study it is suggested that there is need to improve learning services in the future or at least in the coming semester. The improvement of this service is highlighted more in the online lecture process. The need for more effective lecture methods that can improve interaction in learning and can improve student communication with lecturers in lectures. This has implications for the need for training activities on online learning for lecturers. This is very important so that the implementation of lectures from home (distance learning) can be more effective given the pandemic period that has not fully recovered.