Evaluation of Principal Partnership Programs in the Directorate of Education Management-The Application of Kirkpartick and Countenance Stake Evaluation Model

This research aims to examine and explore the phenomenon of the principal's partnership program developed by the Directorate of Education Personnel in order to increase the professionalism of the school principal. The principal's partnership program is one of the most important and strategic programs in improving the quality of learning in schools, especially in the dimensions of learning, learning supervision, managerial and entrepreneurship as mandated in Permendiknas No. 13 of 2007 concerning competency standards of school principals. This study uses a combination of the "Kirckpatrick and Countenance Stake Evaluation Model" in evaluating the effectiveness of the principal's partnership program in 2018. The participants of the principal's partnership program involved were 295 people consisting of SD 130 school principals, SMP 60; SMA / SMK 89 and SLB 16. This study employs "mixed methods" that is the combination of quantitative and qualitative procedures in answering research questions. The results of the study show that from the antecedent aspect of the Model Countenance Stake, partnership programs are beneficial for improving the quality of school principals and equitable distribution of educational quality. The results of the study on level 1 of the Kirckpatrick model show participants’ satisfaction with the implementation of partnership programs. At the level 2 learning, it has shown the effectiveness of partnership programs especially in increasing participants' knowledge in aspects of learning, managerial, supervision, and entrepreneurship. At Kirckpatrick's level 3, the behavioral dimension shows the level of discipline and skills in supervising learning at school. In the Kirckpatrick level 4 model, it shows a fairly good impact on the attitudes of teachers, students and the implementation of the 2013 curriculum and the development of RPP. Overall, the two evaluation models, "Stake's Kirckpatrick and Countenance have found the benefits of the principal's partnership program results.


Introduction
The principal's partnership program has been developed and implemented by the Ministry of Education and Culture's Directorate (Kinanti, 2016) of Educational staff from 2015 to 2018 with various modifications programs according to the experience and results of the monitoring and evaluation that have been implemented so far (Waylen et al., 2019). Partnership programs among principals whose schools are more qualified and schools that are less qualified in various provinces, districts / cities in Indonesia, are developed to provide a medium for transformation of the quality culture between schools (Liu, 2018). However, the problem of disparity in the quality of schools between provinces, districts / cities within the province and between schools within districts / cities becomes the main challenge that needs to be addressed through the principal's partnership program (Seeley, Witbeck, & Mulholland, 2018). School quality is one indicator of the principal's performance. This is in line with Permendiknas No. 13  ) .The principal's partnership program is essential and strategic as an effort to minimize the quality culture gap between advanced and disadvantaged regional schools. This empirical question then encourages the importance of conducting research in this dissertation. The principal's partnership program is part of the effort to improve the distribution of education quality and school quality through (Kaplan, deBlois, Dominguez, & Walsh, 2016) the top leadership of school organizations so that it can achieve a transformation of quality culture (Waylen et al., 2019) not only in schools but also in close partnership schools. Based on the consideration that the partnership program is a government program that has a noble goal to reduce the quality gap between schools, is widely implemented in various regions, has lasted several years, and there has never been a comprehensive research in this field; thus the authors are interested to focus her research on the principal's partnership program (Lau & Ng, 2019), namely the exchange of good practices and mentoring from more qualified schools to less qualified schools.

Research Question
The research problems that need to be answered in this study are: 1. How is the descriptive description of various (Weiss-Cohen, Konstantinidis, Speekenbrink, & Harvey, 2018) aspects of the principal's partnership program (the basic needs of the partnership program; the targeting of the partnership program; the selection of partnership participants; the facilitator, and the determination of the contents of the principal partnership program) to meet the principal's partnership program objectives? 2. How is the effectiveness of the principal's partnership program in terms of the dimensions of the Kirckpatrick Evaluation Model and the Countenance Stake model: Level 1 reaction (reaction); Level 2 (learning); Level 3 (Behavior); and level 4 results (outcomes)? (Kirkpatrick Partners, 2000)

Principal Partnership
The Ministry of Education and Culture partnership program which began in 2015 was carried out through (Roberts, 2018) the mechanism of exchanging partner principals with the impacted school principals and was named the Principal Leadership Comparative Study Program (Taylor, Goeke

Evaluation Model of Kirkpatrick and Stake
This evaluation study uses a mixed model between the Evaluation Model by Kirkpatrick (2006) and the Countenance Model by Stake. Kirkpatrick's model is used to evaluate the implementation of training or In-Service Learning as well as Countenance Stake to evaluate the overall program. In this model, evaluation is carried out through four levels or stages, namely stages: 1) Reaction; this stage aims to determine the level of satisfaction of participants in the implementation of a training, 2) Learning; this stage aims to determine the level of change in knowledge, skills and changes in the attitude of participants after training, 3) behavior; this stage aims to determine the level of change in work behavior of trainees after returning to their work environment and 4) result is an evaluation to determine the impact of changes in the work behavior of trainees on the level of productivity of the organization or agency.
Countenance Stake's model, according to Fernandes, has two main things, namely: (1) description and (2) judgments, as well as distinguishing the existence of three stages in program evaluation, namely: (1) antecedent (context), (2) transactions (process), and (3) output (outcomes) (Worthen & Sanders, 1973). Countenance Evaluation developed by Stake collects a series of information that is descriptive and contains elements of consideration from an object.
The use of Kirkpatrick's Model and Stake's Model in this study is showed in the following research design diagram. This study uses a mixed approach that is the combination of quantitative and qualitative methods. Cresswell (2012) describes "a mixed methods research design as a procedure for collecting, analyzing, and" mixing " both the quantitative and qualitative studies of a series of studies to understand research problems" (Cresswell.W.John,2012). Furthermore, Cresswell.J.W (2002) states that "mixed methods" is a form of research design that is suitable for use when researchers intend to build the strength of both quantitative and qualitative data and complement in their analysis. This is supported by Miles and Huberman who stated that "when one combines quantitative and qualitative data, we have a very powerful mix" (Miles & Huberman, 1994; Cresswell.J.W., 2012). By analyzing both the results of quantitative research and quality, complex and intact images can be developed from the phenomenon under study.
To answer the research question number one about how the antecedent aspect (context) of the principal's partnership program, it requires qualitative data in the form of documents and words of the interview results and analysis according to the qualitative approach. However, to examine how the effectiveness of the principal's partnership program in terms of the dimensions of the Kirckpatarick level 1, 2, 3 and 4 models used data and quantitative analysis such as descriptive statistics and t-test to test the increase in knowledge of the learning process results. In this context, of course, interview procedures, in-depth interviews and various documents can be clarified and deepened in understanding the interplay between phenomena.
According to Cresswell.W.J., (2012), there are generally four mixed methods designs: (1) the convergent parallel design; (2) the explanatory sequential design; (3) the exploratory sequential design and (4) the embedded design. In accordance with the four types of "mixed method" research designs, this dissertation employs "The Explanatory Sequential Design" because the researcher collects the data and analyzes the quantitative and qualitative data sequentially. Sequential stages are carried out by collecting data and quantitative analysis first and continuing with qualitative data collection and analysis. The design of the study using the "mixed methods" approach is described below.

Population and Samples
The principal's partnership program in 2018 was attended by 495 school principals, consisting of 200 partner school principals who came from better quality schools and 295 affected school principals from poor schools in terms of its quality. The population of the study was the principal program held by the Directorate of Education Staff that there were about 295 affected school principals consisting of elementary school principals of The whole population of affected school principals was taken as a sample so that the total sample of this study was 295 principals. The whole schools come from several schools in the provinces which have the lowest national examinations in Indonesia established in the principal's partnership program.

Data Collection Technique
In answering research question particularly number one about the antecedent aspects of the principal's partnership program (Model Countenance Stake) which include sthe context of the basic needs of the partnership program, targeting partnership programs, selection of partnership principals, and the contents of the principal partnership program are carried out through: Firstly, document Study Technique. Study procedures, document and policy studies are conducted to explore aspects of the partnership program development that cover policy background, targets to be achieved in partnership programs, basis for determining partnership program participants, mechanisms and procedures for implementing programs and the effectiveness of partnership programs in order to increase the professionalism of school principals .
Secondly, in-depth Interview Technique. That was conducted to explore more deeply and comprehensively of the document study so that it can produce more accurate views and various information or data obtained from the document review above. Interviews were conducted with stakeholders who are knowledgeable and involved directly or indirectly on partnership programs. Interviews were also conducted with decision-making officials and implementing program partners so as to obtain comprehensive information about the context of partnership policies and programs in order to improve the professionalism of school principals. The study of the context of the partnership program is an aspect that is emphasized in the Stake Countenance Model which is used as an evaluation model used in this dissertation.
Thirdly, Questionnaire Technique. That is to answer research question number two about the effectiveness of the principal's partnership program in terms of the "mixed Kirckpatrick model and Coutenance Stake model", and a questionnaire was used to measure the perceptions and impressions of participants in education and training on the implementation of the principal's partnership program as an indicator of the principal's partnership program effectiveness. Measurement of effectiveness according to the two models reflects the evaluation of the level 1 reaction (Kirckpatrick) of the principal program participant and the reaction is a manifestation of the process of interaction between participants, facilitators, services, facilities and environment during the partnership program.
Fourthly, Pre-test and Post-test Techniques. Learning dimensions obtained data from the results of pre-test and post-test for partnership material which included aspects of learning, managerial, academic supervision and entrepreneurship provided to participants of partnership programs consisting of elementary, junior high schools, vocational and high school principals. The learning dimension is level 2 in the Kirckpatrcik model which uses tests on the knowledge of participants in the four aspects of learning, managerial, supervision, and entrepreneurship which consists of 50 questions developed specifically for the principal's partnership program. Lastly, Assessment and Observation Techniques. Data collection for the behavioral dimension which is Kirkpatrick's Model 3 level and the results of the interaction process in the Model Countenance Stake, and the results of observations and assessments of the disability behavior of principals during the partner and impact schools in the first and second periods were used. Furthermore, the behavioral dimension uses an assessment of the implementation of academic supervision obtained by participants in participating in the partnership program. The application of skills and knowledge in academic supervision is an indicator of change including disciplinary behavior as presented by the Kirckpatrick Level 3 model and the realization of the interaction of the learning process according to Stake's Model Countenance.

Results and Findings
In accordance with the research questions to be answered in this study, there are four variables used as a measure in assessing the effectiveness of education and training according to Kirkpatrick's Model (1998) as follows.
a. The reaction variable includes several indicators about participants' perceptions of the service of the organizing committee and its indicators; readiness of supporting facilities and indicators; readiness and suitability of teaching materials and indicators; learning atmosphere and indicators; and presentation of material by the facilitator and the indicators. Some of the indicators are set out in 5 Likert scale choices as stated in the questionnaire.
b. Learning variable is to determine the extent to which participants gain knowledge, improve their knowledge in the fields of learning, managerial, academic supervision and entrepreneurship. The principal partnership participants were given 50 tests that measured the mastery of the principal's knowledge in the fields of learning, managerial, academic supervision and entrepreneurship. Through the measurement of the pre-test and post-test, it was intended to determine the absorption of the knowledge of participants towards the four fields of knowledge.
c. Behavior variables are to determine how many partners in the principal program partnership have behavior changes after they return to their respective schools. Thus the behavioral dimension is more concerned with the question of how the principal's behavior in applying the knowledge and skills in their respective schools for the progress and development of the school. In this context the aspects of disciplinary behavior and indicators and the behavior of the application of learning supervision are indicators that are measured.
d. Result Variables are the overall results or impacts (outcomes) of the implementation of education and training programs. For the purposes of this study, the indicators measured are from various aspects of teachers' perceptions about curriculum 13, student interest, development of lesson plans, teacher professional skills in learning, and pedagogical abilities of teachers in learning.
Summary of the overall level 1 reaction (level) variables, level 2 aspects of learning (learning), level 3 about aspects of behavior (behavior) and level 4 about results (results) or the impact of the principal's partnership program, according to the Kirckpatrick and Antecedence models from the Countenance Model Stake is presented in the following variable, instrument and indicator.

Data Analysis
Analysis of data to answer research questions about reactions as variables or dimensions of training effectiveness, descriptive statistical analysis was employed. Meanwhile, data analysis to answer research questions about learning variables employs the Non Parametric Statistical Test, the Wilcoxon Test. To answer the research question about the variable or behavioral dimension (descriptive behavior), descriptive statistical analysis is used in the form of a percentage or frequency of distribution that describes aspects of discipline and the behavior of the implementation of academic supervision carried out by the principals in each school environment. Furthermore, to answer the research questions about the results variable, statistical descriptive techniques are used in the form of distribution frequency and on average to find out the impact of the partnership program on teachers, students and the atmosphere of learning in the schools of each principal participating in the partnership program.

Conclusions
Program evaluation at any time is seen as a process that is held independently when the program is established and has been operated so that it can make an improvement, increase accountability, and a deeper understanding of the phenomenon.