Identifying Instruments to Measure Programme Educational Objectives (PEO) Achievement in Malaysia

Outcome Based Education (OBE) is required to be implemented in all academic programmes in Malaysia in order to fulfil the standards specified by the Malaysian Qualification Agency (MQA). One part of the OBE that needs to be measured is Programme Educational Objectives (PEOs). PEOs are broad statements that describe the expected achievement of graduates within three to five years of graduation. All institutions in Malaysia need to measure the PEOs. However, the PEOs of community colleges are not measured yet after five years of programme's implementation. Therefore, this paper systematically reviews the current instruments applied by community colleges in Malaysia to evaluate the programmes and identify the problems in the instruments. This survey study adopted triangulation approach where the data were collected from semi-structured interviews and document analysis. Semi-structured interviews were conducted with five officials from different divisions in the Department of Polytechnic and Community Colleges Education (JPPKK). These samples were purposively selected because their scope of the task was directly involved with the process of development and evaluation of the programme outcomes. Analysis of PEOs was also conducted to compare the PEOs of community college with eight local universities which are UKM, USM, UPM, UTM, UTHM, UTeM, UniMAP and UiTM. The analysis is based on PEOs of engineering or technology program from each institution. The results of this study identified four problems from the community college’s instruments which are: (1) no specific measurement of PEOs, (2) unsuitable timeframe of data collection, (3) respondents’ feedback is distracted by too many questions, and (4) lack of stakeholders’ point of view. These results indicate that community colleges need the right instrument to measure the achievement of PEOs. Therefore, further research is suggested to study the instrument to measure the achievement of community college’s PEOs.


Introduction
In a rapidly developing country, higher education providers, including technical and vocational education and training (TVET) institutions, play a crucial role in driving the engine of change to increase national productivity in economic and social status. Upgrading the teaching and presentation content in vocational training will consequently contribute to the development of human capital thus leading to producing skilled workforce and Malaysia having the upper hand to compete economically at a global stage. Through continuous upgrades and skills taught in the education system, technical colleges become a significant contributor to the production of creative and innovative learners who are able to contribute more effectively to the national economic development and prosperity (Lam & Hassan, 2018).
To contribute effectively, TVET graduates, especially from polytechnics and community colleges, need to be more competent and skilled. The Department of Polytechnic and Community College Education (JPPKK) is committed to ensure that the programmes offered meet the standards set so that the graduates produced have relevant skills that fit the employer's wishes. This aforementioned department aspires to fulfil the aim to be the leading TVET institutions that provide quality education and training required by industry and the community in the region by the year 2025.
Early in 2009, all academic programmes offered by the public and private higher learning institutions in Malaysia were required to fulfill the standards specified by the 136 Identifying Instruments to Measure Programme Educational Objectives (PEO) Achievement in Malaysia Malaysian Qualification Agency (MQA) as directed by the Ministry of Education to ensure that the programme is accredited. Community colleges are one of the public higher institutions that are included to be accredited by the MQA. In addition to that, all programmes that apply for accreditation need to implement the Outcome-Based Education (OBE) in their respective curriculum design and delivery.
Each Higher Education Provider (HEP) is also advised to implement various forms of assessment to ensure the quality of programmes offered complies with the requirements of MQA and its stakeholders. One of the elements that need to be measured is the Programme Educational Objectives (PEO). PEOs are broad statements that describe the expected achievement of graduates within three to five years of graduation. All institutions in Malaysia need to measure the PEOs. However, the PEOs of community colleges are not measured yet after five years of programme's implementation.
Due to the lack of the PEOs evaluation, this paper aims to identify issues related to the current instrument used by community colleges to assess and evaluate the achievement of the PEOs. The objectives of the research were to: a. identify current instrument adopted by the JPPKK to measure PEOs b. identify issues with the current instrument c. determine suggestions to improve the current instrument.

Literature Review
The quality education system requires all academic programmes offered by the Higher Education Provider (HEP) to adopt the Outcome-Based Education (OBE) concept that is in line with the aspirations of the Ministry of Education Malaysia and the Malaysian Qualifications Agency (MQA). The OBE curriculum design starts with the aim of the programme which outlines the philosophy, rationale, and purpose of the programme. The design details on the outcomes of learning that students need to achieve when engaging in the academic programmes. These intended learning outcomes ensure the availability of graduates (MQA, 2014b). These learning outcomes also form the basis of the formation of five clusters within the Malaysian Qualifications Framework (2017) namely:

Knowledge and Understanding
Knowledge and understanding refers to a systematic understanding of facts, ideas, information, principles, concepts, theories, technical knowledge, regulations, numeracy, practical skills, tools to use, processes and systems. Knowledge and understanding enables the learners to relate their prior knowledge on the course learning with their line of work. Knowledge provides the basis for applications of the learning outcomes.

Cognitive Skills
Cognitive skills are skills related to thinking or intellectual capabilities to apply learned knowledge and skills. The capacity to develop levels of intellectual skills progressively begins from understanding, critical or creative thinking, assessment, and applying, analyzing, problem-solving, as well as synthesizing to create new ideas, solutions, strategies or new practices. Such intellectual skills enable learners to search and comprehend new information from different fields of knowledge and practices.

Functional Work Skills with a Focus On:
a) Practical skills These are generally work skills and operational skills that are applicable in familiar employment environment such as planning, organizational skills, selection of tools, material, technology methods, and procedures, while in the theoretical context, the practical skills include study skills and preparations, undertaking procedures, scientific skills, designs, research and so forth. They also include specialized skills which are set by a specific subject, discipline, technical or occupation-related work skills, and professional practice which helps with the enhancement of professional competence. Practical skills should include safe and sustainable practices.

b) Interpersonal skills
Interpersonal skills refer to a range of skills which, amongst others, include interactive communications, relationships and collaborative skills in managing relationships in teams and within the organizations, networking with people of different cultures, as well as social skills or etiquette.

c) Communication skills
Communication skills generally refer to the ability to communicate or convey information, ideas, or reports cogently and professionally with the use of appropriate language. The communication skills must be valid and in appropriate forms, in various medium, to a range of audiences and different situations. The ability to communicate in more than one languages is encouraged.

d) Digital skills
Digital skills generally refer to the ability to use information or digital technologies to support work and studies. The skills include sourcing and storing information, processing data, using applications for problem-solving and communication, as well as practicing ethics in applying digital skills.

e) Numerical skills
These are the quantitative skills that require learners to acquire higher levels of numerical abilities. It is acknowledged as an essential skill to survive and remain relevant in the study, work, and daily life. Within the MQF levels, this learning outcome may not be specifically mentioned for every level but it is expected that numerical skills are required as the outcome which ought to be indicated for every specific programme. f) Leadership, autonomy, and responsibility This cluster of skills refers to an individual's ability to build relationships and work with teams made up of peers or in managerial capacities with varying degrees of autonomy, to make decisions or set goals at organizational, unit, or team levels, to take responsibilities and provide accountability, to be confident, knowledgeable, articulate, honest, professional, concerned, resilient, a risk taker and possess other intrapersonal skills including working in, and leading teams.

Personal Skills and Entrepreneurship
Personal skills are life skills that learners are expected to use in daily life. They are generally portrayed through enthusiasm for independent learning and intellectual and self-development. Personal skills are portrayed through demonstration of confidence, self-control, social skills and proper etiquette, and commitment to professionalism in the workplace. They also include the capability to plan for career development or higher education. Aspects of character such as honesty, punctuality, time management, and keeping to and maintaining deadlines that are important in a work environment are also essential personal skills.
Entrepreneurial skills require relevant knowledge, skills, and expertise in critical areas of an enterprise. Outstanding personal qualities include creativity, grit, and drive. The learning outcomes describe the incremental development of these skills. The drive to be an entrepreneur is not only set as personal skills but entrepreneurship also requires the requisite of relevant knowledge, cognitive and functional skills.

Ethics and Professionalism
Ethics and values are essential in personal, organizational, societal or community, and global settings as they guide personal actions and interactions at work and within the community at large. Awareness or understanding and respect of ethical, social, and cultural differences and issues are critical in the exercise of professional skills and responsibilities, integrity, professional conduct (professionalism), and standards of conduct such as upholding regulations, laws, and codes of good practices or code of professional conduct. A sensitive approach in dealing with other cultures adds value to this learning domain.
Hence, the implementation of OBE requires every programme to have the Programme Educational Objectives (PEOs) which include the five clusters mentioned above. PEOs are a statement that describes the achievement of graduates within three to five years of graduation by measuring the outcome of the education that has been in the course of their study. Also, PEOs represent the judgment of the relevance and success of a programme. PEOs are provided in conjunction with the involvement of stakeholders in the programmes offered so the requirements and standard set are met.
Besides that, PEOs have a direct link with learning outcomes and curriculum design. Curriculum and learning outcomes prepare students to achieve PEOs (MQA, 2014b). As a result, if students do not achieve the PEOs after they have graduated, there are probably issues with the curriculum and learning outcomes of the program (Abbadeni, Ghoneim, & AlGhamdi, 2013). Therefore, curriculum developers must carefully identify PEOs and make sure the curriculum and the outcomes prepare students for the achievements of the aforementioned PEOs.
However, PEOs' achievement for community college programs has never been measured after five years of the program's implementation. In the context of a dynamic environment, PEOs need to be monitored and systematically reviewed to ensure that they are aligned with the strategic directions, changing needs, and education's good practice. Program monitoring and review is a standard to be followed in Area 7 based on the Code of Practices Program Accreditation (COPPA) (MQA, 2014a). This standard covers the process of monitoring, reviewing and evaluating institutional structures and processes (administrative structures, leadership and governance, planning and review mechanisms), curriculum components (syllabus, teaching methods, learning outcomes), as well as graduates' progress, performance, and employability.
The purpose of monitoring and reviewing programme is to identify and evaluate whether the institution has reached the strategic plan targets and whether the aforementioned plan is still relevant and suitable (MQA, 2014a). Curry (2018) defined monitoring as a continuous function that provides intervention with early indications of progress in the results achieved. Programme monitoring identifies the issues and shortcomings in maintaining academic standards so that necessary action can be taken to improve the academic standards. Program monitoring improves the quality of students' experience by providing continuous review, identifying areas for improvement, and taking appropriate and timely actions (MQA, 2014a).
At this stage, the institution needs to measure and analyse the achievement of the predefined targets. Abdullah et al. (2008) stated that the absence of a PEO measurement mechanism will affect the success of an accredited program by MQA. This is reinforced by the statement of the full accreditation compliance requirements set out on the community college program where the program should have a mechanism to measure the Program Educational Objectives (PEO), Program Learning Outcomes (PLO), and Course Learning Outcomes (CLO). Failure of the institution to comply with this requirement may result in the withdrawal of accreditation certificate (MQA, 2016). Unaccredited programs will affect the graduates when they want to pursue studies or apply for jobs in the public sector. Therefore, in order to meet the accreditation requirements, various forms of assessment need to be undertaken to determine the quality of the program implemented in compliance with the requirements of the Malaysian Qualifications Agency (MQA) and various stakeholders. Assessment, in general, serves the purposes to (MQA, 2014b): a. promote learning; b. measure performance, by awarding grades which indicate whether and how well a particular student has attained the stated learning outcomes; c. determine whether a particular student is sufficiently well prepared in a subject area to proceed to the next level of instruction; d. give feedback to students which indicates levels of attainment and diagnose misunderstandings and learning difficulties; and e. give feedback to teaching staff to identify and diagnose ineffective teaching methods or techniques.
To comply with the demand for assessment that leads to continuous improvement, institutions are advised to create a systematic process that identifies specific learning outcomes, developmental objectives, and ways to measure the student's achievement (Merhout, Benamati, Rajkumar, Anderson, & Marado, 2008). Abd Rahman, Ab-Rahman, and Mohd Yusoff (2016) suggested that two methods of assessment can be applied to measure the achievement of graduates, namely, direct and indirect measurement methods. Direct measurement is the tool used to assess the achievement of alumni, usually via a survey. This will provide evidence that are collected and analyzed to demonstrate that actual learning has taken place. The survey is helpful to inform students, teachers, and other stakeholders of the depth, breadth, and performance quality of how well that students have gained regarding relevant knowledge, understanding, skills, habits of minds, and ways of responding to people and situations (MQA, 2014b).
Meanwhile, the indirect method can be implemented by mapping the attainment of PLO with the PEO. Indirect assessment is essential in predicting PEOs' attainment and therefore justifying the assumption that the students to be already competent upon completion of the programme (Abd Rahman et al., 2016). The PEOs can be assessed by using these instruments or methods which are alumni surveys, alumni interviews, employer surveys, employer interviews, job offers (starting salaries), admission to graduate school, parents survey, academic staff perception survey, and reports with industrial advisory panel ( To ensure that the graduates meet the employability market and industry demands, the quality of graduates must be aligned with the programme educational objectives (PEOs) as well as gain the information from graduate's work life (Ibrahim, 2014). Low quality of graduates becomes a factor in the unemployment issue (Nooriah Yusof & Jamaluddin, 2015). Therefore, the quality of students' learning including employability capabilities needs to be assessed. This attention to feedback from those who employ or educate graduates, and from graduates themselves, is an important element of efforts to improve education in higher education provider (HEP). This assessment is a complex endeavour and it needs to be carefully done as higher education providers will bear the implications of ineffective assessment (Rowe & Zegwaard, 2017).
Despite all the instruments used to assess and evaluate programmes offered in the public and private universities, PEOs' achievement for community colleges is still unknown due to the PEO assessment that is part of the Continuous Quality Improvement (CQI) cycle being incomplete. At the institutional level, CQI is a systematic and structured mechanism that enables the HEP to achieve its vision, mission, and educational goals (MQA, 2014a). CQI practice is important to measure the achievement of vision, mission, educational goals, and learning outcomes. The goal is to keep the current PEOs by adopting a process which identifies the needs of the programme's various constituencies, critically assesses the attainment of graduates, and reassesses and updates the objectives. This is a cyclical process. Although elements of this process are continuous and will vary among programmes, each institution has the responsibility for ensuring that every programme goes through a documented cycle of activity such as PEO that is evaluated at least every three years to resolve potential issues while fulfilling as many needs and as practical as possible (Mamat et al., 2014). The MQA accreditation's report for community colleges stated that one of the requirements for accreditation is a mechanism to measure Programme Educational Objectives (PEOs), Learning Outcome Programme (PLO) and Course Learning Outcomes (CLO) to meet the CQI process. Failure to practice CQI will affect the quality of the graduates produced that do not meet the competence characteristics required by potential employers. Thus, to consider the likes of graduates being seen incompetent and unfit to work in the fields they studied in, the PEOs and curriculum content need to be enhanced and aligned with the needs of future potential employers. Graduates of community colleges will have better prospects of marketability.

Methodology
To identify issues related to the current instrument used by community colleges to assess and evaluate the achievement of the PEOs, this research adopted a triangulation approach to determine the validity of data through cross verification from several sources. Triangulation is an attempt to check the validity of the data or the information obtained by researchers from different angles through reducing bias in the data collection and analysis process (Yahaya et al. 2017). This survey study adopted qualitative approach where the data were collected from semi-structured interviews to identify the problems in the instruments and document analysis to support, complement, and strengthen the findings from interview sessions.
Semi-structured interviews were conducted with five officials from different divisions in the Department of Polytechnic and Community Colleges Education (JPPKK). They represented: (1) one official of quality assurance unit, (2) one official of graduate employability unit, (3) one official of curriculum unit, and (4) two officials of programme planning unit. In general, their working experience at JPPKK ranges from 1 to 7 years. These samples were purposively selected because their scope of the task was directly involved with the process of development and evaluation of the programme outcomes. The samples of this study were asked six open-ended questions in the semi-structured interview. The questions are on: (1) the current instruments used to measure the success of the programme outcomes, (2) methods to utilise the instruments, (3) issues with the utilisation of the instruments, (4) suggestions to assess PEO, (5) the needs for PEO to be assessed and evaluated with a specific instrument, and (6) issues that may arise if PEO is not assessed and evaluated.
In order to further validate the findings of the data, this research also included document analysis on the evaluation and assessment of PEOs that have already been developed and implemented in universities in Malaysia. Information obtained from the documents is used to support and complement evidence while verifying the information obtained from interviews and observation (Creswell, 2012). There were eight instruments of PEOs achievement that were analysed based on four research universities (UKM, USM, UPM, and UTM), four Malaysian Technical University Network (UTHM, UTeM and UNIMAP), and one comprehensive university (UiTM). These universities were chosen because they provide engineering or technology programmes. Besides that, the universities also shared the instruments used to measure the PEOs. The instruments used by these universities are employer surveys and alumni surveys. These instruments of JPPKK and universities were compared to identify if there are any similarities or differences between them. The analysis was done using NVivo 12 Plus.

Results and Discussion
In order to identify the instruments and issues related to them, analysis of interview session, documents, and PEOs was done to identify the instruments and issues related to them being utilised in the universities abovementioned.

Interview Session Analysis
Based on the interviews, there were four instruments used by the JPPKK to measure the outcome of the programme as summarized in Table 1  An internal audit to identify any strengths and weaknesses of the programme Download the Excel-based instrument from MQA portal and fill in the form by programme and institution. The instrument will view the result summary of programme status. The result will be sent to MQA for evaluation.

Polytechnic Programme Alignment
Review program offers based on student enrolment, student applications, graduate employability in field, and the median range of graduate's salary.
Fill up the Excel-based instrument. The data are gathered from various divisions in JPPKK.

Polytechnic
Employer Survey Identify employer's needs and perception about graduates Distribute the survey to employers during industrial visits.
Five themes were identified regarding the issues with the instruments and they are explained as below: (1) The Graduate Employability (GE) survey is too general and it is not specified for each respective programme (2) The time duration of data collection is not suitable (too short or too long?) (3) Too many items in survey form distract the respondent's response (4) Lack of stakeholder's point of view (5) Internal audit is too broad and not specific to measure PEO Based on the data collected and analysed, while the Graduate Employability (GE) survey is effective in assessing and evaluating the programme outcomes, the items in the survey are too general, are not specified for each area, and do not reflect the primary objective for each, specific programme. This problem was raised by one of the interviewees who said: "The questions are broad. No specific question to the specific programme area." [Interviewee 1 | Theme: Issue] To ensure data collected are valid, the objectives set for this survey should be specified by cooperating with the survey's relevant users and its stakeholders (Brancato et al., 2006). In assessing PEOs, the set of items asked must include the important criteria in the PEOs (Zulkifley, Misran, Abd. Aziz, & Hussain, 2008). A thorough and accurate assessment ensures that the theories and application of knowledge identified will truly direct students towards achieving the intended outcomes (Yarbrough, Donald, Shulha, Hopson, & Caruthers, 2010).
The data collected also showed that the current GE survey is inadequate in the sense that it assesses and evaluates the outcome six months after graduates complete their studies; this contradicts the statement made by the MQA that PEOs must be measured after three to five years of graduation (MQA, 2014b). One of the interviewees was worried that the result might not be precise because of the time allocated for data collection.
"We collected the graduates' data before convocation day. Some of them may just work for a few months so their work experience is still not enough to be evaluated." [Interviewee 1 | Theme: Issue] Different time allocation for data collection will give different results depending on the respondent's experience (Danielle & William, 2013). Therefore, utilising GE to assess and evaluate PEOs may be irrelevant and obsolete due to the two aforementioned issues discussed. Besides GE survey, another instrument which is employer survey is also distributed during industrial visits by industrial training officials. The interviewees shared their views that there were cases in which employers did the assessment based on the industrial trainees instead of their real employees. This reflects on the issue of the suitable time for data collection.
"Usually the employer survey was distributed during industrial visits. The employer may wrongly assess the industrial trainees instead of the real employees. In my opinion, the employer survey should be distributed in time other than industrial visits." [Interviewee 2 | Theme: Issue] Interviewees also expressed their concerns during the interview sessions about the instruments. They were worried that there too many items in 18 pages of the GE survey and how those would affect the validity of data. One of the respondents shared: "You know our students; they answer this survey only for the sake of attending their convocation. Some of them, not even read. A research done by Khairuddin (2013) revealed that Malaysians in general and students particularly have poor reading habits as reading is perceived as displeasing, complicated, and many avoid it. Rolstad, Adler, and Ryden (2011) have identified one of the factors that affect the response burden is the length of the questionnaire and this burden is reflected in the response rate. The longer the questionnaire, the less likely potential participants will be motivated to volunteer for the study or to complete all the items (Worthington & Whittaker, 2016). Therefore, it is suggested that the total length of the survey should be as short as possible and it has to be directly related to the study's purpose.
Besides that, community colleges and polytechnics have conducted self-review audit, which is also known as an internal quality audit, in order to comply with the continuous quality improvement process. However, the audit is conducted internally by the institution without taking in views from its stakeholders. MQA has outlined guidelines for PEOs assessment to gather feedbacks from stakeholders such as alumni and employers (MQA, 2014b). The interviewees frequently cited that the self-review audit is too broad as it includes many general constructs which are: (1) programme delivery, (2) student assessment, (3) student support and services, (4) academic staff, (5) educational resources, (6) programme management (7) and monitoring as well as (8)  This response showed that the respondent disagreed that using the self-review audit to measure PEOs due to the items in the self-review audit were not purposely to measure the PEOs. PEOs measurement should have been done via more suitable mechanisms such as advisory board meeting, focus group survey, alumni survey, and employer survey (Abbadeni et al., 2013).
Finally, JPPKK has also developed an Excel-based instrument to review programme outcomes in order to ensure they are aligned with the programmes offered. The review is based on student enrolment, students' applications, graduate employability in field, and the median range of graduate's salary. In the interview, the interviewees claimed that: This response showed that the stakeholders' view is very crucial. According to Abbadeni et al. (2013), industrial and societal demands play an important role in forming and building the programme outcomes. Institution has the responsibilities for meeting the needs, interests, and viewpoints of the stakeholders (Asiyai, 2014). Hence, feedback from experts in the industry is important as it keeps the curriculum up-to-date and relevant to the field (Norhafezah Yusof et al., 2018).
In summary, these findings show that specific instrument with suitable and convenient time for data collection should be considered as they will affect the objective of instrument implementation. Also, feedbacks from alumni, employers, and industries are critical in ensuring the sustainability of the programme. A suitable and specific instrument is needed to measure the PEOs.

Documents Analysis
Document analysis was also adopted as part of data collection and analysis of this study to identify the usability and effectiveness of various instruments utilised by different universities to measure PEOs. Each of the instruments utilised by these respective universities is compared with the current instrument utilised by JPPKK. The summary of the comparison is tabulated and presented in Table 2

1)
To identify the graduates' status: employed, unemployed or furthering studies.

2)
To identify the learning experience offered at an institution 3) To identify the effectiveness of programme To evaluate the quality of graduates as employees in the organisation / / Based on Table 2, all the universities use alumni survey and employer survey to measure their PEOs while community colleges and polytechnics use various outcome-based instruments but not specifically to measure PEOs. Several themes were identified and they are stated as below: (1) The items are not specific and tailored according to the programmes offered (2) There are too many items (3) The time allocated for data collection is unsuitable The current instruments and survey utilised do not evaluate the programmes' achievement holistically as they are conducted by different divisions at JPPKK. Since each instrument aims to achieve different and respective objectives, the outcomes become too general. Survey works best with clear targets to gather the right information and important factors (Denscombe, 2010).Survey should be specific enough to cover in the areas researched (Mason, 2014). Therefore, a survey has to be specific to answer questions raised, solve problems posted or observed, assess needs and set goals, and determine whether or not the specific objectives set are met (Glasow, 2005).
MQA also suggested that evidence should be to attain the PEOs using indirect assessment tools like alumni survey, employer survey, stakeholder survey, alumni interviews, job offers, starting salaries, and admission to graduate programmes or professional qualification. According to Merhout et al. (2008), while indirect assessment provides information about students' perception of their level of learning and abilities related to the programme or course learning outcomes direct assessment should also be included. Using only one method to gather information about the programme will only reflect a part of students' achievement. In addition, it is best to utilise at least two assessment tools to reduce bias and validate inferences obtained from the data collected (MQA, 2014b& Merhout, et al., 2008. Most of the universities' instruments consist of three to four constructs with a total number of at most 32 items whereas the Graduate Employability (GE) survey that is currently used by the JPPKK has 170 items. Glasow (2005) emphasized that the length of a questionnaire should not be onerous and the questions should not double negatives and long questions as they may lead to respondents losing interest in answering. Rolstad et al. (2011) highlighted their concern about respondents' burden and they say that can be eased by reducing the number of items in the questionnaire. Respondents would be more motivated to complete the survey if it is shorter and takes less time to answer (Bista, 2017;Rolstad et al., 2011;Worthington & Whittaker, 2016).
Besides that, the GE survey is also conducted within six months of graduation and the timeframe is not adequate to evaluate PEOs achievement as described in the previous interview session. MQA recommended that PEOs achievement need to be evaluated within three to five years of graduation instead of within six months after graduation (MQA, 2014b). Glasow (2005) stated his concern about changing time references whereas he proposed that the survey that is distributed at different times will lead to seasonal differences of responses. Therefore, the survey has to be distributed and completed at a specific point in time and it is more effective in providing information about the current state of affairs (Denscombe, 2010).
Overall, these findings indicate that a proper survey should be short and specific, in addition being distributed at a more convenient time. Two assessment tools which are alumni survey and employer survey are good enough to measure the PEOs achievement as implemented by the universities in Malaysia. Also, both direct and indirect assessment should be implemented simultaneoulsy so as to get a more precise result.

The Programme Educational Objectives (PEO) Analysis
Analysis of PEOs was also conducted to compare the PEOs at universities with the PEOs at community colleges. This analysis is made based on PEOs of engineering or technology programme from each institution. The result is presented and tabulated in Table 3 below. Table 3, PEOs of community college have many similarities with the PEOs of universities. However, considering that the universities' roles are to produce engineers, the universities aim for their graduates becoming professional engineers or attaining the equivalent professional qualification. The bachelor degrees and a number of diploma qualifications, which are recognized by the various statutory professional bodies in Malaysia, enable graduates to be registered as professionals to practice their professions (MQA, 2017). Also, as bachelor programme providers, universities emphasise on pursuing lifelong learning in which the alumni are encouraged to enrol as postgraduates and contribute to the research fieldwork. As for research universities (UKM, USM, and UPM) the graduates must be able to investigate complex problems using the appropriate research methods. These skills are not necessary for Malaysian Technical University Network (MTUN) such as UTHM, UTeM, and UniMAP.

Based on
On the other hand, community colleges as skilled and semi-skilled workforce providers and one of the leading edges of TVET institutions focus more on the aspects of technical competence that fulfils the demands of the industries and communities. This aids community colleges to become a renowned institution for education and on-the-job training as preparation before joining the workforce. Community colleges were first designed to produce a holistic human capital that is capable of facing any challenges and provide a highly trained workforce to meet the country's needs in the job market (Rasul, Mohamed Ashari, & Azman, 2015). Besides that, community colleges provide training for a specific job while universities put more emphasis on scientific and systematic knowledge (M. J. Ahmad, Jalani, & Hasmori, 2015). The generic learning outcomes' clusters that apply to all TVET qualifications include knowledge and cognitive skills, functional and work skills, and specific industry-appropriate competencies. The differences in the various programmes and levels include specialised technical and vocational knowledge as well as agreed occupational standards of specific industries. At least 60% of the programmes are made up of practical components that emphasise psychomotor skills that are strengthened further through industrial exposure and training as part of the curriculum (MQA, 2017).
The graduates should also be able to demonstrate entrepreneurship skills in line with the government's intention to develop entrepreneurship characteristics among graduates and become job creators at the same time (MOE, 2015). Surprisingly, UKM, a research university, and UTHM, an MTUN, aspire to produce engineers with entrepreneurial qualities. This is probably led by the Malaysian government initiative that has made entrepreneurship courses compulsory for all public universities (Buchanan & Ahmad 2015). Apart from that, further attention is also given to the commercialisation of research universities to enrich entrepreneurial activities ( While there are many similarities in universities and community colleges, they both have different profound roles as a learning hub. Community colleges are designed as lifelong learning hubs that offer opportunities for better job prospects by providing flexible courses to Malaysian citizen through TVET, upskilling programmes, and entrepreneurship education programmes. Due to this difference, community college cannot adopt the universities' instrument to measure the PEOs because these two institutions have different aims and objectives.

Conclusions
Based on the data collected and analysed from the interviews and document analysis, it was found that there were several issues with the current instruments adopted by the JPPKK to measure the PEOs. This study has found that generally the current instrument applied by JPPKK was too broad and not specific to any program. The time allocated for data collection is unsuitable and inconvenient as it does not help the universities and community colleges to evaluate and measure if their respective PEOs are achieved by the graduates. There are also too many items or questions in the survey and they distract the graduates' focus to complete the survey. Lack of stakeholders' point of view has also become an issue that affects the evaluation process. Hence, it is best to adhere to MQA's suggestion by using at least two assessment tools to collect the information; most universities choose to implement alumni survey and employer survey as their tools. In addition to that, the survey should be tailored to follow the suggestions made; the survey should be short, specific, and be distributed at the right time, which is 3 to 5 years after graduates have completed their studies. This study focuses solely on the shortcomings of the current survey instruments used to collect data about graduates. Therefore, further research is suggested to be conducted in studying the instrument to measure the achievement of community college's PEOs.