Investigating Factors that Promote Higher Order Thinking Skills in Information and Communication Technology (ICT) Integrated Classes

The current industry environment demands employees to have higher order thinking skills (HOTS). Thus, higher learning institutions should play an important role to inculcate HOTS among learners. Integration of ICT is believed to promote HOTS among learners effectively. However, existing literature in the Malaysian context shows that not many academicians understand the concepts of HOTS. Furthermore, academicians are not integrating ICT in their classrooms to promote HOTS among learners. Hence, this study aims to investigate academicians’ awareness and knowledge level of HOTS, and the factors that influence the use of ICT by academicians in promoting HOTS among learners. This study used the survey method to collect data. Thirty-five academicians from a local public university participated in the study. Data were collected via questionnaire. Descriptive and thematic analyses were carried out to analyse the data. The results showed that the majority of respondents have low awareness level and knowledge about HOTS. Meanwhile, it is also noted that most of the academicians did not integrate HOTS elements in their courses. All the respondents integrated ICT in their classrooms and believed that ICT could promote HOTS. The findings also showed that learner factor such as attitude towards technology, self-efficacy and instructor factor such as technology competency strongly influenced the use of ICT by the respondents to promote HOTS among learners. This study could contribute in terms of expanding readers’ understanding on the academicians’ awareness and knowledge level of HOTS and the factors that can contribute to enhancing HOTS among learners in ICT integrated classes.


Abstract
The current industry environment demands employees to have higher order thinking skills (HOTS). Thus, higher learning institutions should play an important role to inculcate HOTS among learners. Integration of ICT is believed to promote HOTS among learners effectively. However, existing literature in the Malaysian context shows that not many academicians understand the concepts of HOTS. Furthermore, academicians are not integrating ICT in their classrooms to promote HOTS among learners. Hence, this study aims to investigate academicians' awareness and knowledge level of HOTS, and the factors that influence the use of ICT by academicians in promoting HOTS among learners. This study used the survey method to collect data. Thirty-five academicians from a local public university participated in the study. Data were collected via questionnaire. Descriptive and thematic analyses were carried out to analyse the data. The results showed that the majority of respondents have low awareness level and knowledge about HOTS. Meanwhile, it is also noted that most of the academicians did not integrate HOTS elements in their courses. All the respondents integrated ICT in their classrooms and believed that ICT could promote HOTS. The findings also showed that learner factor such as attitude towards technology, self-efficacy and instructor factor such as technology competency strongly influenced the use of ICT by the respondents to promote HOTS among learners. This study could contribute in terms of expanding readers' understanding on the academicians' awareness and knowledge level of HOTS and the factors that can

Introduction
The current economic climate demands employees who can think critically and solve problems creatively. Employers normally value employees who have the ability to generate ideas and think creatively to expand their business into the global market. Consequently, employers often highlight the importance of graduates to be equipped with higher order thinking skills (HOTS) [1,2]. Thus, higher learning institutions should play an important role in cultivating students with the desired skills demanded by the employers. Higher learning institution management should plan a curriculum that includes HOTS elements and train their academicians to teach HOTS. According to Bloom [3], HOTS are reflected by the top three levels in Bloom's Taxonomy: analysis, synthesis and evaluation. The analysis stage refers to the ability to break down ideas into smaller parts to understand its organisational structure. Synthesis involves the ability to put parts together as a whole to produce a unique outcome. Evaluation involves giving justification to any statement derived. Meanwhile, Anderson and Krathwohl [4] further refined HOTS with elements such as analyse, evaluate and create. Lewis and Smith [5] defined HOTS as critical thinking as well as problem solving, creative thinking and decision making and use them in new situations. Hopson, Simms, and Knezek [6] defined HOTS as the cognitive skills that allow learners to execute Bloom's Taxonomy at the analysis, synthesis and evaluation levels. Despite the varied descriptions of HOTS, a common criterion of HOTS is the involvement of multiple levels of intellectual process to produce a unique outcome that has values.
The goal of promoting higher learning institution students' HOTS is emphasised in many educational models from various countries [7]. Basically, HOTS play an important role in producing graduates that have critical thinking and problem-solving skills [8]. These skills assist the graduates to solve problems in unpredictable situations which are deemed necessary in this competitive world [8]. Zohar and Dori [9] pointed out that HOTS are a significant determinant for success in both academia and workplace. Thus, higher learning institutions should play an important role in promoting HOTS among learners.
It is understood that instructional strategies play an important role in imparting HOTS among students. Academicians are encouraged to find new ways of teaching and create learning activities that start with shaping the knowledge and slowly expanding into the creation of HOTS [10]. One such intervention involves integrating technology into teaching and learning environments, an approach known as Technology-Enhanced Learning (TEL) or Technology-Enhanced Learning & Teaching (TEL&T). TEL or TEL&T have been defined as "integrating the use of digital technology into the learning and teaching processes to improve the quality of learning" [11]. Researchers agree that using appropriate digital tools and resources may help learners "use higher-order thinking skills to plan and conduct research, manage projects, solve problems and make informed decisions" [12].
However, the scenario in Malaysian higher learning institutions shows that the most difficult skills or attributes to cultivate in a classroom environment are the ability of the students to think beyond traditional learning measures. These students can solve problems that were taught by the educators in the classrooms. However, if the educator introduces a minor/major twist to the questions by incorporating HOTS elements, the majority of students faced difficulties in solving the given problem. Thus, it is believed that the integration of technology in classrooms by academicians can promote HOTS among learners. Researchers reported that integration of ICT in teaching and learning activities that have HOTS elements helped [10] the learners to visualise and understand the problem better and consequently find the appropriate solution to the given problem [12]. However, not all academicians integrate ICT in their classrooms. The lack of awareness and insufficient evidence that shows that technology integration is capable of promoting HOTS among higher learning institution students could be the reasons that hinder the academicians to integrate ICT in their teaching and learning activities. This shows that few researches have been done on the integration of technology in classroom in higher learning institutions in Malaysia to promote HOTS among students. Hence, the objectives of this study are twofold. First, it investigates the level of awareness and understanding of academicians towards HOTS concepts. Secondly, it determines the factors that will influence the use of ICT by academicians in promoting HOTS among learners.

Literature Review
A past study has reported that academicians are aware of HOTS concepts [14]. However, many academicians lack the skills in transforming their HOTS knowledge into practice appropriately. Academicians are reportedly lacking the skills in explaining when, why and how the thinking patterns should be integrated in the students' learning activities [15]. Insufficient preparedness from the academicians was highlighted as the main contributor for this problem [16]. Meanwhile, Yusoff and Selman [17] reported that academicians were unable to explain and teach HOTS concepts appropriately to students. Furthermore, academicians are also reported to have very basic knowledge about HOTS and not skilful in designing HOTS questions [17].
Literature also reported that academicians in developing countries are adopting a teaching approach that is less student-centred, thus hindering the integration of HOTS in the classroom [18]. Budu [18] also emphasised and encouraged academicians to adopt appropriate teaching methods that can cultivate HOTS among learners.
Academicians could promote HOTS among learners using many methods. One such method is integrating ICT in classrooms [12]. Van der Rijst, Baggen and Sjoer [19] reported that academicians showed a higher level of readiness and have greater self-confidence in integrating a variety of ICT in their teaching and learning activities. Similarly, Yemothy [20] asserted that all the academicians in that study had integrated ICT in their classrooms. Ghavifekr, Kunjappan, Ramasamy and Anthony [21] reported that academicians integrated a variety of technology tools such as word processing software, spreadsheet software, presentation software, database software, desktop publishing software, social media and 5818 Investigating Factors that Promote Higher Order Thinking Skills in Information and Communication Technology (ICT) Integrated Classes e-learning portals in their teaching and learning activities. Some of the academicians also highlighted that they are still facing problems in integrating ICT in classrooms such as limited ICT infrastructure, connectivity problems, lack of technical support and training, and lack of competency in handling the ICT [21]. It is understood that several factors were identified as the contributing factors that influence academicians to integrate ICT in their classroom to promote HOTS among learners. A past study reported that learner factors that define attitude of learners such as passion and interest towards the ICT could determine the integration of ICT in the classroom to promote HOTS [6]. According to Hopson, Simms and Knezek [6] and Meerza and Beauchamp [22], attitude towards technology plays a vital role in determining the learning approach and HOTS of the learners in a TEL environment. A past study regarding attitude towards technology showed that technology assisted the learners to engage in active learning and actual learning outcomes depend on learner's attitude towards technology. Hopson, Simms and Knezek [6] also reported that attitude towards technology can impact learners' engagement in learning, which in turn can influence their learning approach and learning output. Lee and Choi [12] pointed out that little is known about the relationship between attitudes towards learning and promotion of HOTS in a TEL environment.
Self-efficacy of the students is also influencing the integration of ICT by academicians to promote HOTS. According to Chen [23], higher self-efficacy among the students increased their confidence level and ability in operating ICT. Thus, it increases students' interest towards the use of ICT and eases the integration process in classrooms.
Besides, the academicians' competence in operating the technology is also found to influence the integration of ICT in classrooms. This is because higher technology competency enabled the academicians to use ICT without anxiety and fear [24]. This consequently will increase the academicians' confidence level to control the ICT and allowed them to pay more attention to designing activities that can promote HOTS. Wei, Piaw, Kannan and Moulod [25] stated that there is a strong correlation between academicians' ICT competency and use of ICT for academic activities.

Research Design
This study used the cross-sectional survey method to collect data. Cross-sectional survey method was used because the study wanted to collect opinions about the integration of ICT to promote HOTS from a small sample of respondents in a large organisation. It used both quantitative and qualitative approaches.

Respondents
Thirty-five academicians from a local public university participated in the study. All the respondents are junior academicians and have experience in integrating ICT in their classrooms. Data were collected from respondents who attended an academic workshop. Junior academicians were selected as the respondents because the output from young academicians would be fruitful for the university management to administer the appropriate plan to motivate them to integrate ICT to promote HOTS among learners in the future.

Data Collection Procedure
Data of this study were collected via a questionnaire which was adapted from Bahr [23] and Savi et al. [24]. The questionnaire is divided into closed and open-ended questions. The closed ended questions have two sections. The first section is to collect the demographic details of the respondents. The second section consisted of questions regarding HOTS and reflection on the problems that respondents faced when using ICT, factors that will influence the use of ICT in promoting HOTS among the students and other ways that can promote HOTS among the students effectively besides using ICT. The open-ended questions were designed to obtain more in-depth data regarding reflection on the problems that the respondents faced when using ICT, factors that will influence the use of ICT in promoting HOTS among the students and other ways that can promote HOTS among the students effectively besides using ICT.

Data Analysis
Descriptive and thematic analyses were carried out to analyse the data. Data from closed-ended questions were analysed in terms of descriptive analysis mainly in the form of percentages, while, open-ended questions were analysed in terms of thematic analysis.

Demographic Details
60% of the respondents were female and 40% of the respondents were male. Respondents' age ranged from 30 to 39 years old with an average age of 35.5 years old. The respondents were from the School of Science, Engineering and Arts. All respondents had PhD in their respective fields and were junior academicians with less than five years of experience in the academic sector. Figure 1 shows that the majority of the respondents (82.9%) have never heard of HOTS. This signifies that most of the junior academicians are not aware of HOTS prior to the survey. The findings clearly indicate that the junior academicians were not exposed to HOTS elements during their teaching and learning activities by the university management. Furthermore, the junior academicians have also never taken any initiatives to know more and practise the HOTS elements in their teaching and learning activities.  Figure 2 shows that only 17.1% of the respondents know what HOTS is. This further signifies that most of the junior academicians do not know the meaning of HOTS. This again shows that most of the junior academicians have never been exposed and trained to implement HOTS elements in their teaching and learning activities. Meanwhile, these academicians are also not aware of the importance of HOTS to the students' future career.

Awareness and Knowledge about HOTS Concept
When the respondents were asked whether their school management ever discussed HOTS openly in academic meeting, training or workshop, only 28.6% of the respondents said "Yes". This clearly depicts that the school management takes little initiative to bring awareness about HOTS to the respondents. This also indicates that the school management is not taking much effort to bring in experts to explain and guide the respondents to integrate HOTS in their classrooms. School management should be aware that the employability of the graduates will be affected if graduates are not equipped with HOTS.
The respondents who reported that their school management did discuss openly about HOTS stated that the management normally discussed the concept and elements of HOTS (34.3%), ways to integrate HOTS in teaching (22.9%), how to further improve integration of HOTS in teaching (22.9%), how technology could be used to promote HOTS among students (8.6%) and provide encouragement to promote HOTS in teaching (17.1%). The results are summarised in Figure 3. This signifies that university management is taking some initiatives to promote HOTS to the young academicians. However, the initiatives are not exhaustive enough to promote the elements of HOTS. Furthermore, university management should ensure that all academicians are involved in HOTS workshop organised in their organisation. This will ensure that all academicians are aware of the significance of HOTS and integrate it in their teaching and learning activities.

Integration of HOTS in Teaching and Learning Activities
As shown in Figure 4, the majority of respondents (62.9%) stated that HOTS are not embedded in the classroom teaching and learning activities. This depicts that not many junior academicians are aware of the importance of HOTS to the learners' career path. This could be because the majority of the respondents do not know the appropriate method of integrating HOTS in their classroom. The respondents who have embedded HOTS in their teaching and learning activities reported that they integrated HOTS in courses such as organisation behaviour, design, research methodology, nuclear medicine, nuclear medicine physics, analytical chemistry, biochemistry, mine planning design and organic chemistry. This shows that the majority of respondents from the pure science courses embedded HOTS in their teaching and learning activities. All respondents used ICT in their teaching and learning activities where a majority of them used ICT such as presentation software and Web 2.0 applications. This shows that ICT facilitated the respondents in their teaching and learning activities. This also indicates that the respondents are up-to-date in incorporating current technology tools in their teaching and learning activities. Table 1 shows the applications used by the respondents. Figure 5 shows that most of the respondents used the ICT either always (42.9%) or every time in class (34.3%). This shows that junior academicians frequently integrate ICT tools in their classrooms. This could be because the respondents realise the importance of integrating ICT tools in their classrooms as ICT tools could ease communication with students and provide flexible platform to disseminate teaching resources and feedback to students. Meanwhile, 97.1% of the respondents agreed that integration of ICT in teaching and learning activities could promote HOTS among learners. This shows that academicians have a positive view on integrating ICT to promote HOTS among their students. This also shows that the respondents felt comfortable using ICT as an important instrument to instil HOTS among the students.

Challenges in Using ICT
The findings from open-ended questions show that the respondents faced many challenges when integrating ICT in the classroom. The respondents admitted that slow internet speed or poor internet access are usually the main problem that they encountered when using ICT. Besides, some of the respondents also reported that they have limited hardware skill. Besides, the respondents found difficulties in terms of security issues, technical issues and obsolete ICT facilities. All these challenges create frustration among the respondents to integrate technology in teaching and learning activities. This may also lead to ignorance among academicians to integrate ICT. Thus, it can affect the initiatives of promoting HOTS using technology tools.

Factors that Influence Use of ICT to Promote HOTS among Learners
The result shows that learner factors that include attitude Universal Journal of Educational Research 8(11B): 5816-5823, 2020 5821 towards technology influenced the respondents' decision in integrating technology in classrooms. This is because attitude towards technology can impact learners' engagement in learning, which in turn can influence their learning approach and learning output. Besides, the respondents stated that learners' interest towards the ICT integrated classroom environment is vital because it will determine the continuous use of the ICT by both learners and academicians to obtain better learning outcome. Besides, the respondents also stated that learners' self-efficacy towards ICT determines their decisions to adopt technology to promote HOTS. This is because self-efficacy of learners will affect learners' confidence level and decision to adopt ICT tools. The result of this study shows that learners' confidence in handling ICT is an important determinant as it could increase the learners' motivation to use ICT in their teaching and learning activities.
Furthermore, the respondents stated that their competency and knowledge in operating ICT determine its use to promote HOTS. The respondents' competency level in operating the ICT tools gives confidence to them to control the technology and focus on activities that can promote HOTS. ICT is always evolving, and respondents need to update their technical skills and competence to operate the relevant ICT.

Other Methods to Promote HOTS Effectively
The results showed that the respondents agreed that besides ICT, there are other approaches that could be used to promote HOTS among learners. The respondents stated that variety of learning tasks such as doing role play in the class, conducting forums and discussions in the class, experimental works, tutorial discussions, peer discussions, outdoor activities, creative interaction group activities and one-to-one teaching could promote HOTS. Besides, the respondents also admitted that the quality of the learning material and resources could also increase HOTS. Furthermore, adopting different kinds of pedagogical approach such as practical-based learning, experiential learning and flipped classroom could increase learners' HOTS. This shows that diversifying the teaching approach and activities and increasing the quality of the learning resources could provide a promising platform to the respondents to promote HOTS among the learners.

Discussion
The results showed that majority of the junior academicians have low awareness level and knowledge about HOTS. This implies that the junior academicians are unaware of the agenda of the Malaysia Education Blueprint 2013-2025 that aimed to produce graduate with creative and critical thinking to fulfill the employers' demand. This could be because the management of higher learning institution did not openly discuss HOTS. This result is in line with the findings of Yusoff and Selman [17] who stated that academicians only have very basic knowledge about HOTS elements. It is suggested that the respective higher learning institution management to conduct more workshops and training programmes to increase the awareness level and provide more exposure towards how to carry out activities related to HOTS in ICT integrated classes.
Meanwhile, it is also noted that most of the junior academicians did not integrate HOTS elements in their courses. This is because many of the junior academicians are still not aware or know about the concept of HOTS and its significance to the learners' career path. Meanwhile, it is obvious from the result that the junior academicians who had integrated HOTS in their teaching and learning activities are from the pure science department. This is because the Ministry of Education in Malaysia has given more emphasis to integrate HOTS elements in STEM education to produce more eligible workforce. The findings support the findings of Baharin, Kamarudin and Manaf [28] who reported that the Malaysian government took more initiatives to integrate HOTS in STEM education.
Furthermore, the current study also reported that all junior academicians always integrated any kind of ICT in their teaching and learning activities. This implies that junior academicians relied on current ICT to conduct their academic work effectively. They integrated technologies such as presentation software, Web 2.0 applications and e-learning portal in their teaching and learning activities to disseminate information to their students. Similarly, Yemothy [20] and Ghavifekr el al. [21] also stated that all academicians prefer to integrate ICT in their classroom to conduct the lecture and distribute meaningful information to the students. However, as stated by Ghavifekr et al. [21], the junior academicians in this study reported that they faced challenges such as technical competency, unreliable connectivity, security issues and obsolete ICT facility. This problem could be overcome if both the junior academicians and the top personnel in the organisation work together closely to find the appropriate solutions.
The findings of this study also showed that learners' attitude towards ICT influenced the use of ICT for learning activities to promote HOTS. Learners' positive attitude towards ICT determines the effective use of ICT in their learning activities. Positive attitudes towards ICT will increase learners' interest and passion towards the subject content and this may lead to the promotion of HOTS. The findings are in congruent with the findings of Meerza and Beauchamp [22] who stated that positive attitudes towards ICT influenced the successful integration of ICT in teaching and learning activities to promote HOTS. Furthermore, Meerza and Beauchamp [22] also stated that in order to create positive attitudes towards ICT among Meanwhile, the findings emphasised that learners' self-efficacy towards ICT determines its adoption by junior academicians to promote HOTS. Learners' higher self-efficacy increases their confidence level and motivates them to use ICT regularly in their learning activities. Besides, it enables the learners to pay more attention to the subject content and leads to the promotion of HOTS. Similarly, Chen [23] highlighted that higher self-efficacy of learners is one of the important determinants that influence the use of ICT among learners to promote HOTS.
Furthermore, the finding reported that ICT competency of junior academicians is another important factor that determines the integration of ICT in teaching and learning activities to promote HOTS among learners. This is because if the junior academicians have higher ICT competency, they can integrate the technology confidently in their teaching activities and this may lead them to introduce more interesting teaching pedagogy to enhance HOTS among learners. The finding is in agreement with the findings of Wei at al. [25] who highlighted that ICT competency of the academicians determines the effective integration of ICT in their teaching and learning activities to produce excellent academic outcome.
This study reported that junior academicians believe that ICT plays an important role in promoting HOTS among learners. At the same time, they also suggested other approaches such as interactive teaching and learning activities, quality resources and variety of pedagogy approaches to promote HOTS among learners besides using ICT. Similarly, Jerome, Lee and Ting [26] stated that innovative pedagogical approach and interactive teaching and learning task could promote HOTS among learners.

Conclusions
The aim of this study is to investigate junior academicians' awareness and knowledge on HOTS. Besides, it determines the factors that influence integration of ICT in teaching and learning activities to promote HOTS among learners.
It is recommended that the related university top management should take appropriate steps to promote the awareness on HOTS and its integration in teaching and learning activities among academicians. The university management should upgrade the ICT facilities at their organisation to motivate the academicians to use them regularly and effectively in their teaching and learning activities. Furthermore, the academicians should take their own initiatives to expand their knowledge about HOTS and its significance to students' career path.
This study contributes in terms of expanding readers' understanding on the background information of ICT usage by junior academicians and the factors that can contribute to enhancing HOTS among learners in ICT integrated classes. Since this is a pilot study, only a small number of respondents participated. Future research will include more respondents from more higher learning institutions to collect a richer set of data.