Predicting Student's STEM Subject Performance by Using the Malay Version of S-STEM

The study employed the Malay version of Student Attitude towards STEM (S-STEM) to predict STEM subject performance. The original S-STEM was translated from English to the Malay language and tested on a sample of 144 first year undergraduates. The instrument consisted of four constructs on attitudes: mathematics, science, engineering/technology, and 21st century skills. Confirmatory factor analysis confirmed the four factors contributed 53.2% to the explained variance. Structural Equation Modelling was employed to test the measurement and structural model. Item loadings below 0.6 were removed from the instrument. Results show that all constructs fulfilled most of the reliability criteria. Fit indices presented CFI = 0.925 and RMSEA= 0.069 which abide the rule of thumb with GFI = 0.836, AGFI = 0.789, NFI = 0.837 readings near the level of acceptance. When the Malay version of S-STEM was administered on 111 students in the following year, result shows that mathematics attitudes was the best factor to predict STEM subject performance followed by science attitudes, and then the 21st century skill. Impact of engineering/technology was not statistically significant at p <.05. Future researchers are encouraged to employ the Malay version S-STEM on other STEM subjects to generate more conclusive results.


Introduction
STEM is more than just an acronym for Science, Technology, Engineering, and Mathematics. It is an approach to learning and development that integrates the areas of science, technology, engineering and mathematics [1]. STEM education is essential for developing a highly skilled and adaptable workforce that is capable of meeting productivity demands and changing technologies in the globalized world [2]. Concern for improving STEM education in many nations continues to grow as demand for STEM skills to meet economic challenges is becoming more acute [3,4].
Continual advances in technology has indeed gradually replacing manual labour with automation and creating new jobs that require technological skills each day. As technology advances, it also changes the way students learn, connect and interact every day. The Department of Education [1] in Western Australia predicted that in coming years approximately 75 percent of all new jobs will require qualifications and skills in the areas of science, technology, engineering and mathematics. Per se, employer's demand for STEM qualifications and skills will continue to increase in the future.
Although STEM education has been contemplated worldwide, numerous educational studies have indicated a common global phenomenon that is students' interest and motivation towards STEM learning has declined especially in the western countries and in more prosperous Asian nations [4]. Wang [5] pointed out that while global demand for students to enter postsecondary STEM fields is increasing, students' interest and readiness in pursuing STEM majors are not encouraging. A good example is found in the developed country like the United States who is currently facing a shortage of STEM majors and graduates [6].

Objectives of the Study
The aim of the study in this paper was to generate and employ a Malay version of STEM questionnaire to predict students' performance in the STEM subject. The objectives of the study were: 1) To translate the existing S-STEM questionnaire [7] from English to Malay language; 2) To validate the new Malay version of S-STEM; and 3) To identify the factors (i.e. mathematics, science, engineering/technology, and 21 st century skill) that best predict STEM subject performance using the Malay version of S-STEM.

Factors Affecting STEM Choice
Wang [5] states that a student's decision to pursue a STEM major is a longitudinal process that develops during secondary education until postsecondary studies. She found that college student's decision to major in STEM was directly related to 12th grade mathematics achievement, exposure to mathematics and science courses, and mathematics self-efficacy. Reamer, Ivy, Vila-Parrish and Young [8] claim that a student's success in higher level science and mathematics coursework in high school is built on a firm foundation in mathematics in early education.
Past studies indicated that there are several factors that affect students' STEM choice. For in-stance, Anaya, Stafford and Zamarro's study [9] shows that mathematics achievement, perceived mathematics ability, and parental occupation in a science field, are significant predictors of students' inclination to majoring in science in college. Kundu and Ghose [10] claim that a healthy attitude towards mathematics can augur good performance in mathematics examination. Unfried et al. [7] highlighted that attitudes towards 21st century skills is an important variable that predicts student participation in STEM related careers. Blotnicky, Franz-Odendaal, French and Joy [11] found that students with higher mathematics self-efficacy (MSE) were more knowledgeable about STEM career requirements. Furthermore, students with higher MSE and STEM career knowledge were more likely to choose a STEM career. Students with greater interest in technical and scientific skills were also more likely to consider a STEM career. Researchers have also pointed out that students who have high MSE [5] or science [12] are more likely to pursue postsecondary schooling in STEM fields.
The National Research Council of the United States (2011, as cited in [8] argued that mathematics as a pillar of STEM education plays the most important role in permeating in science, technology, and engineering disciplines because its concepts and methodologies. Based on the importance of mathematics achievement, this study focused on score in mathematics examination score as a variable for STEM subject performance.

Social Cognitive Career Theory (SCCT)
Lent, Sheu, Gloster and Wilkins [13] attributed STEM attitudes and career interests as two key components in a larger theory of career development called social cognitive career theory. The theory claims that people "tend to become interested in activities they believe they are good at and at which they expect to receive positive outcomes" [14, p. 576].

Student Attitudes towards STEM (S-STEM)
Unfried et al. [7] developed an S-STEM model to measure students' attitudes towards STEM major based on SCCT which focuses on self-efficacy, out-come expectancy at the individual level and, goals. They express attitude as an attribute that comprises both self-efficacy and expectancy value beliefs. They define self-efficacy as is "the belief in one's ability to complete tasks or influence events that have an impact on one's life" and state that "expectancy value theories posit that individuals regularly assess the likelihood of attaining specific goals and appraise the value gained or lost from such attainment." (p. 623). Some examples of the items in S-STEM are such as "I would consider choosing a career that uses math"; "Knowing science will help me earn a living", "Designing products or structures will be important for my future work", "I am confident I can produce high quality work", etc.
Researchers commonly applied SCCT to study STEM career pathways at the post-secondary level with few focusing at the elementary, middle, or high school levels. Unfried et al. [7] presented S-STEM to explore Upper Elementary (4th-and 5th-grade) and Middle/High (6ththrough 12th-grade) student attitudes towards STEM. In the model they adopt a four-factor structure (science attitudes, mathematics attitudes, engineering/technology attitudes, and 21st century skills attitudes) to measure student attitudes towards STEM. Unfried et al.'s S-STEM unfortunately, is only available in the English language despite reaching to elementary students from different ethnicities. This study fills the gap by supplementing the current S-STEM with a Malay version of the instrument. The goal is to provide an alternative for researchers to collect data using Malay language for expanding the understanding of student participation and persistence in STEM career pathways.

The Translation Process
This study adopted S-STEM for Middle/High students [7]. The original questionnaire consisted of 43 items. The study began with the forward translation whereby two instructors who are fluent in reading and writing English and Malay were engaged to translate separately the 43-item S-STEM from English to Malay. These two instructors then worked together to compile their translations into one Malay version. The resulting Malay version was inspected by a third instructor who is also fluent in both languages.
The back-translation from Malay to English was done by an English language instructor who had not seen the original S-STEM version of the survey instrument. The final stage was to compare the content of original English version with the back-translated English version by the researchers.

Validating the Malay Version of S-STEM
A total of 144 undergraduates from the 2017 cohort aged between 17-18 years old were selected to participate in the study using cluster sampling to collect data for Stage 1of the study: the psychometric testing and validation of the Malay instrument.

Predicting STEM Subject Performance by Using the Malay Version of S-STEM
A total 111 undergraduates from the 2018 (the following year) cohort aged between 17-18 years old were selected to participate in Stage 2 of the study: to examine the attitudes towards STEM and hence to predict students' STEM subject performance. Table 1 below shows the demographic profile of the participants.

Data Analysis
Data analysis included descriptive and inferential techniques where SPSS version 24 and AMOS 24 were used to analyze the data. Exploratory factor analysis (EFA) was run using the SPSS while Amos was used to run Confirmatory Factor Analysis (CFA) to determine the model fitness of the underlying variables that described the attitudes towards STEM. The analysis was conducted in two stages. Stage 1 focused on the psychometric properties of the instrument. Stage 2 tested the hypotheses of the study. The four factors consisting of science attitudes, mathematics attitudes, engineering/technology attitudes, and 21st century skills attitudes were explored using EFA. Table 2 shows the breakdown of the number of items according to the factor. Result shows that Kaiser Criterion value was 0.826 which is more than 0.7 and at significant level <.0001 indicating there are variances between the items. The total variance explained was 53.2%. Table 2 shows the reliabilities of the four constructs. EFA shows that all translated original 43 items loaded significantly above 0.3 into four factors namely math, science, engineering/technology, and 21st century skills. The four first-order factors explained 53.2% of the extracted variance. Reliability for each factor were above 0.8 (α math = 0.899 [10 items], α science = 0.873[9 items], α ET = 0.891 [11 items], and α 21CS = 0.921 [13 items]). These readings are very close to the original model [7] which reported the reliability of αmath = 0.90, αscience = 0.89, αET = 0.90, and α21CS = 0.92.

Results and
After re-specifying and rerunning the model using SEM, a final measurement model was achieved. From the original 43 items, the number of items was reduced to 20. The items were removed one at a time until all items loadings were above 0.6.

The Measurement Model
The refined 20-item S-STEM measurement model was assessed using a sample size of 144 respondents (n = 144). Reliability of the instrument was assessed according to the following criteria: a) Internal reliability is achieved when Cronbach's alpha ≥ 0.7 or higher; b) Construct reliability (CR) measures reliability and internal consistency of the items representing the latent construct. Reliability is achieved when CR ≥ 0.6; and c) Average Variance Extracted (AVE) is the average percentage of variation explained by the items in a construct. Reliability is achieved if AVE ≥ 0.5.

2040
Predicting Student's STEM Subject Performance by Using the Malay Version of S-STEM   Table 3 displays the result of the reliability. The factor loadings of all the four factors consisting of 20 items were above 0.6 as recommended by Hair, Black, Babin and Anderson et al. [15]. The criteria of Cronbach's alpha ≥ 0.7, AVE ≥ 0.5 and CR ≥ 0.6 were achieved for all constructs. Overall results indicated sufficient reliability for the measurement model.
Three types of validity were calculated for the measurement model 3 used in this study. 1) Convergent validity: The validity was achieved since all items are significant at p < 0.01. 2) Discriminant validity: The presence of two redundant items was identified from the report in the Modification Indices (M.I.). Item 12 and Item 13 in the "21st Century Skills" were set as "free parameter estimate" when the model was re-specified and ran a few times.
Correlation between each pair of latent exogenous construct was checked to make sure that none of the correlation coefficient is more than 0.85 (see Table 4). Secondly, discriminant validity was established (see Table 5) since the square-root of the average variance extracted (AVE) is greater than its correlations with all other constructs [16].  Table 6 shows the results of fit index for the measurement model. The fit indices showed that GFI = 0.836, AGFI = 0.789, CFI = 0.925, NFI = 0.837 and RMSEA= 0.069 abide the rule of thumb. The Chisq/df is 1.683 is less than the baseline value of 0.3 (χ2/df = 1.683, DF = 93, N = 144, p =.000) in the CFA analysis. The result also shows that is an index that fits well in each category (absolute fit, incremental fit, and parsimonious fit). For instance, the absolute fit RMSEA = 0.069 (< 0.08); incremental fit CFI = 0.925 (> 0.9), and parsimonious fit Chisq/df = 1.683 (< 3.0). On these ground it is argued that ade-quate model fit for the Malay version S-STEM measurement model was achieved.  In examining the theoretical implication of the overall model, results show that there is a significance influence from each construct (Math, Science, and 21st Century Skills) towards mathematics test scores except for the engineering and technology skills. Table 6 shows the details of the finding. Three out of four paths were significant with p-values less than 0.05.

Predicting the Impact of the Malay Version of S-STEM on Students' STEM Subject Performance
In the following year, the Malay version of S-STEM was again tested on another group of 111 students (Refer to Table 1 for demographic details). The data were analyzed using linear multiple regression to predict the impact each factor on final MAT112 exam which is a business mathematics paper common for all students. This subject is a common STEM subject sat by all students from three different programs of study (Computer Science, Banking, and Policy Management).  Table 7 shows that the four factors significantly [F (4, 105) = 10.23, p < .0001] contribute 28% of the explained variance (R2 = .280) on mathematics performance (see Table 8 and Table 9).  Table 9 indicates that mathematics, science, and 21st century skills predicted students' MAT112 marks. Among these three, mathematics attitudes ( = .464, p < .0001) contributes the most impact on the STEM subject performance followed by science attitudes ( = .232, p < .05). This is undeniable as MAT112 is a mathematics paper. However, the 21st century skills attitudes also shows some influence towards the achievement ( = .196, p < .05). Lastly, the impact from the engineering/technology was not statistically significant at p <.05.

Conclusions
The primary goal of this study was to translate Unfried et al.'s [7] S-STEM questionnaire into the Malay language and then use it to predict STEM subject performance. The translated version went through a series of psychometric tests to confirm the factor structure, reliability, and construct validity. Additional analyses were conducted to establish convergent and discriminant validity, and criterion-related validity and the goodness of fit.
After re-specifying and rerunning the model using SEM, a final measurement model and structural was achieved. Results show that after retaining items with item loadings above 0.6, all four constructs indicated sufficient reliability for the measurement model with fit indices abiding the rule of thumb. However, GFI = 0.836, AGFI = 0.789, NFI = 0.837 were near but below the level of acceptance.
Results of using the Malay version of S-STEM to test the STEM subject performance show that mathematics attitudes ( = .464, p < .0001) best predicts STEM subject performance followed by science attitudes ( = .232, p < .05) and the 21st century skill attitudes ( = .196, p < .05). However, the engineering/technology attitudes has no statistically significant impact on STEM subject performance at p <.05. Hence, more studies on attitudes towards engineering and technology and its impact on STEM subject achievement are needed.
The current results can be further enhanced if future researchers employ the Malay version instrument to predict other STEM subject performances to generate more conclusive results. It is also hope that the Malay version of S-STEM can be undertaken by secondary schools to examine students' intention to pursue STEM courses. Early detection of STEM interest is in line with the aspiration of the Malaysian Education Blueprint 2013-2025 that is to make better plans to ensure that Malaysia has a sufficient number of qualified STEM graduates.