Engineering Students' Performances in Mathematics through Project-based Learning

Project-based learning is an example of active learning and is student-driven, interdisciplinary, collaborative and technology-based. To test the hypothesis that project-based learning maximises course performance, we analysed a difficulty index of examination scores or failure rates and compared between 422 students in the 2016/2017 session, who took the Vector Calculus course and project-based learning with 342 students from the 2015/2016 session without project-based learning. The analysis of the difficulty index is used to investigate the achievement of the course outcome and the analysis on the correlation between the project-based learning scores and the final exam scores are identified using Pearson's product-moment correlation. The effect sizes indicate that on average examination scores improved by about 12% with project-based learning and students in classes with project-based learning were 3.4 times more likely to get as than students in classes without project-based learning. It is observed that the difficulty index for all course outcomes are achieved and distributed between a good range of 0.3– 0.8. It is also proven that the students find it easier to answer the exam questions after the project-based learning is implemented based on the results of their mid and final semester exams.


Introduction
Engineers are increasingly being challenged and confronted with various situations in their profession that require them to think critically, creatively, and outside the box. Apart from meeting the needs of clients, the government, the environment, and the general public, an engineer at the same time has to be smart in consolidating the humanity in the course of employment (Deutsch, 2020;Li et al., 2020). The rapid development of technology and changes in organisational infrastructure are among the challenges that engineers nowadays have to face, hence they are required to have good generic skills such as communication and teamwork skills in complement their disciplinary expertise (Mills & Treagust, 2003;Harpe et al., 2000;Male, 2010). In this scenario, institutions of higher learning play their roles in producing quality engineers with a broad perspective of their field of work. In line with the government's desire to produce more graduates in this field, science, technology, engineering, and mathematics education need to be strengthened in today's teaching and learning process (Bybee, 2010, Han et al., 2012. To ensure these goals are achieved, institutions of higher education should take steps by changing and innovating their approach in teaching and learning. The Learning Pyramid illustrated by Rate (2010) shows that passive learning, such as attending lectures, reading, audio visual, and demonstration, is seen to have less effect on teaching and learning outcomes compared to active learning, in which students are directly involved in the process. In active learning, for example through group discussions, by conducting training and by teaching their own friends, students were more intrinsically motivated and had higher conceptual learning (Lalley & Miller, 2007;Wood, 2004;Benware & Deci, 1984). The outcome is different if the student is actively engaged in any learning methods used, either active or passive. The teaching and learning paradigm of 'what has been taught?' should be shifted to 'what has been learnt'? The lecturers should be the facilitators and co-constructors of knowledge in the process of developing student minds while the students themselves direct their own learning.
Project-based learning is an example of active learning and is a current instructional strategy that is student-driven, interdisciplinary, collaborative and technology-based (Wurdinger et al., 2020). Similar to cooperative learning where the learning process is student-centric (Saad, A., 2020), the students have to work in groups to solve challenging problems based on the curriculum. They have to gather information from variety of sources and synthesise, analyse and derive knowledge from them. This process is able to develop teamwork skills, information analysis skills, skills to teach friends, decision-making skills from data analysis and reflection skills on the ongoing learning process (Krajcik & Blumenfeld, 2006;Carlson & Sullivan, 1999). Adderly (1975) defined project-based learning as follows: 1. Involves the solution of a problem; often, though not necessarily, set by the student himself/herself. 2. Involves initiative by the student or group of students, and necessitates a variety of educational activities. 3. Commonly results in an end product (e.g. thesis, report, design plans, computer programme, and model). 4. Work often goes on for a considerable length of time. 5. Teaching staff are involved in an advisory, rather than authoritarian, role at any or all of the stagesinitiation, conduct, and conclusion.
Aspect (3) is crucial and distinguishes project-based learning and problem-based learning (Mills & Treagust, 2003). Three general models of project work for educational purposes are listed below: 1. Project Exercise: Students should apply knowledge and techniques already acquired to an academic issue in a subject area already familiar to them. This represents the most traditional kind of project-based learning. 2. Project Component: Related to real world issues, interdisciplinary in nature and has a broader and larger scope. Develops problem-solving abilities and a capacity for independent work. Often, traditionally taught courses are studied in parallel with the project course. 3. Project Orientation: Denotes the entire curriculum philosophy of a programme of study; the project that students complete from the entire basis of their university education, while instructional teaching is provided only to supplement the requirement of the project topics.
Project exercise is the capstone event designed to integrate the subject material learnt during a specific course and is typically a teacher-centred project. This type of project-based learning is typically conducted during the final year of study. In contrast, project component and project orientation tend to leave more scope for student-centredness. In promoting active learning in higher education, the Department of Engineering Education (DEEd), took initiative by implementing project-based learning, specifically the project component model, as one of the teaching and learning methods in the Engineering Mathematics 1 course (Vector Calculus) for the 2016/2017 academic session. This course is the first mathematics course that engineering students in the Faculty of Engineering and Built Environment, Universiti Kebangsaan Malaysia has to take. Based on previous research, it is observed that students find it difficult to understand engineering mathematics courses and the failure rate is high (Tang, Voon & Julaihi, 2009;Othman et al., 2012). The purpose of this study is to investigate the course outcome achievement and the performance of the students in terms of their ability to answer the exam questions before and after the implementation of project-based learning.

Methodology
A total of 422 students in session 2016/2017 who took the Vector Calculus course were involved in this study and 342 students from session 2015/2016 were used as control items in determining the effectiveness of the project in overall performance. All students are from four different departments i.e. the department of civil engineering, the department of mechanical and structural engineering, the department of chemical and process engineering and the department of electric and electrical engineering. There are one lecturer and one tutor involved for each department in teaching this course making a total of 4 lecturers and 4 tutors for each session. The method of measurement was carried out by: i) Measuring the difficulty index:  Before and after project-based learning was implemented for students in session 2016/2017.  Between students from session 2015/2016 who did not participate in project-based learning and students in session 2016/2017.
ii) Measuring the passing rate/failure rate between students from sessions 2016/2017 and 2015/2016 and thereby measuring the correlation i.e. whether there is a relationship between the performance in final exams with a project. This measurement was carried out using Pearson's product-moment correlation coefficient.
There are six Course Outcomes (COs) for this course. CO1 is the most important element in which students should understand the basics of the surface in spaces such as a sphere, ellipsoid, paraboloid, hyperboloid and others. CO2 and CO3 focus on partial derivatives and their applications while CO4 and CO5 cover the topics of integration and their applications. CO6 is the final outcome and is an introduction to the differentiation and integration of complex functions. Table 1 shows the Course Outcome matrix for this subject. Understand the basics of surfaces in space CO2 Able to apply the basic concepts of partial derivatives

CO3
Understand and able to apply the concepts of vector function, vector field, scalar field, gradient, divergence and curl.

CO4
Able to apply the concepts of line integral, double integral and triple integral in solving engineering problems.

CO5
Able to apply Green's Theorem, Stokes' Theorem, and Gauss' Theorem in solving engineering problems.

CO6
Understand basic concepts of differentiation and integration of complex functions. The lesson plans for each topic along with the Learning Outcomes are mapped in Table 2. Project-based learning is conducted after the mid-semester exam and covers topics 1 to 11. Students are initially required to construct models using the equations for surfaces in space. The model then becomes the basis for understanding the next topics such as determining the curvature of the model, calculating the volume of the built model by using integration, determining the equations for the tangent and normal planes to the model and justifying the theorems learned. The student then presents the project that was implemented in the revision week (week 15).
For both sessions, there are three questions that cover CO1 and CO2 in the mid-semester exam questions. The final exam questions are made up of Parts A and B and consist of CO1 until CO6 as in Table 3 and Table 4. In terms of assessment division, the 2015/2016 session includes 15% quizzes, 5% e-learning, 10% cooperative learning, 20% mid-semester exams and 50% final semester exams. For the 2016/2017 session, assessments for the mid and final exams are the same as the previous session except quizzes and cooperative learning with 10% each, as well as project and e-learning with 5% each.  The difficulty index is an instrument used to test the difficulty level of a question. The level can be categorised as easy, moderate and difficult (Chatterjee et al., 2020) and is shown in Table 5. Good questions are at moderate levels ranging from 0.3 to 0.8. Moderate questions can determine whether the Learning Outcomes are achieved or not. Difficult questions with index values approaching 0 can be used to determine high-achieving and outstanding students. The formula for calculating the difficulty index is as follows: where where

Pearson Product-Moment Correlation Coefficient Test
For Pearson's product correlation coefficient, r test is used to examine, measure and visualise the existence and strength relationship between two continuous variable data i.e. in this case the scores of projects obtained with the final exam scores and also between the marks of the mid-semester exams and final exam scores.
If all the assumptions are met, the Pearson product moment correlation coefficient, r can be found by using two hypotheses:  Figure 1 shows the difficulty index value for each CO calculated for students from the 2016/2017 session. The comparison is done between the mid and final exam results. The average value of the difficulty index is calculated for questions with the same CO. The index value before the project was implemented is taken based on the mid-semester exam score while the index value after the project was done is calculated based on the final exam score. Only the difficulty index from two COs is taken since they are both tested in mid and final exam questions. Based on the figure, it can be seen that the index values for final exam questions (0.587 and 0.509) are bigger and lie at the top of the graph compared to the index values for the mid-semester exam (0.512 and 0.466). This result indicates that the students found it easier to answer final exam questions compared to mid-semester exam questions.  Figure 2 shows a comparison of the difficulty index values based on the mid-semester exam score between the students from 2015/2016 who did not perform the project-based learning and students from the 2016/2017 session. It can be seen that all questions that include CO1 and CO2 are in the good range between 0.4-0.7. In terms of students' ability to answer, it is observed that students from the 2015/2016 session feel that these questions are easier to answer than students from the 2016/2017 session. This can be seen based on the position of the graph that lies above and approaching the index value of 1.

Results and Discussion
In Figure 3,   In Figure 4, we present the overall percentage including continuous assessments for both sessions. It is observed that the overall percentage of students with good grades increased for the 2016/2017 session compared to the 2015/2016 session especially for A where 17% of students obtained this grade in the 2016/2017 session compared to 5% in the previous year. Likewise, for grades A-, B + and B, where each grade recorded an increase of 5%, 4%, and 1%, respectively. The percentages of students with poor grades from B to D are lower in the 2016/2017 session compared to the previous session. Grade D recorded a significant drop where 0% of the 2016/2017 session students gained this grade compared to 10% in the previous session. However, the percentage of students earning E for the 2016/2017 session slightly increased to 9% compared to 6% from the previous session.
In terms of the correlation between the final score and project-based learning score, it can be seen that the relation is weak (Pearson's 0.123 correlation) and indicates that the linear assumptions are not met. This means that the project scores do not have a big impact on the final score with an r 2 value of only 1.5% of project score contributing to the final exam scores. The correlation of the final and mid-semester scores is however moderately positive with a value of 0.658 and 43.2% of the final score contributed by the results of the mid-semester exam results. This can be seen in Figures 5 and 6 and summarised in Table 6.

Conclusions
Based on the results of this study, we can conclude that the students have achieved all the course outcomes based on the result of the difficulty index that lies in the acceptable range. The performance of students implementing project-based learning is better after using this method in teaching and learning sessions. This can be clearly seen based on the difficulty index value before and after the project was implemented. Comparisons were also made among students in the previous session. The effect sizes indicate that on average examination scores improved by about 12% with project-based learning and students in classes with project-based learning were 3.4 times more likely to get As than students in classes without project-based learning. Although the comparisons were made between two different sets of students and have different backgrounds, the results give some positive feedback on the improvements that have been made. The number of teaching staffs for both sessions is the same. However, project-based learning is student-centric (active learning) and therefore the role of the teachers are as a facilitator when compared to the previous session where there is no project-based learning and the role of teacher is to give lecture (passive learning). The findings show that students in the 2016/2017 session were able to master the project-based learning related questions better than previous students. The increase in the percentage of students getting A grades in the 2016/2017 session is large, but project-based learning scores are seen as not the major contributors to the overall scores. Suggestions for raising the percentage of project scores are encouraged to increase the impact on overall scores, thereby enhancing student performance and reducing the percentage of student failures.