Effectiveness of an Alternate Assessment Pedagogy in Business Programs with Non-native English Speakers

There is a lot of emphasis on teaching and assessment pedagogy in academia considering the significance of learning in this era of knowledge economies. Students all over the world are being predominantly tested using conventional testing methods, such as written examinations in English. Written examinations enable the students to express their knowledge freely, but they may also be a challenge for students who are non-native English speakers and learners. As a result, when these students enter higher education institutions (HEIs), they are faced with severe setbacks in terms of answering their exams. Hence, inclusion of other assessment methods, such as oral exams, is needed. These alternative methods help in fostering learner autonomy by promoting critical evaluation and reflective thinking among the students. This paper uses quantitative research to assess the effectiveness of using alternative methods including oral exams as an assessment instrument to supplement conventional written assessments. The findings reveal that oral exams are independently related to student knowledge and learning in the two economics courses considered. However, their contribution increases substantially when used in conjunction with written exams.


Introduction
In this era of knowledge economies, in academia there is a lot of emphasis on teaching and assessment pedagogy. Today student populations are diverse, each student has unique learning abilities and aptitudes especially in his/her oral and written skills. This triggers the need to incorporate different teaching and testing methods in the educational field, especially when it comes to non-native speakers of English. The majority of the educational institutions all over the world predominantly use conventional methods of testing -written examinations. Although written examinations have been a popular assessment technique to test student learning, they may challenge students who have studied in public or government schools where written and spoken English is not given much importance. On entering higher education institutions (HEIs), these students face severe setbacks in terms of writing their exams, thus resulting in poor academic performance. This issue has kindled the need for exploring other methods of testing students' knowledge and grasp of the subject matter.
This paper aims to provide an insight into the use of oral exams as an alternative method of testing students' knowledge and ascertain its effectiveness as a method of evaluating students' performance. As faculty, we are used to conducting viva-voces for senior level students (undergraduates) but to introduce a simplified version of this evaluation technique at the Freshman and Sophomore level and analyze its effectiveness is what forms the crux of this research paper. This is a study in this direction, and its purpose is to establish the relevance and importance of the use of oral exams through empirical investigation. The 6186 Effectiveness of an Alternate Assessment Pedagogy in Business Programs with Non-native English Speakers longitudinal nature of the research helps to ascertain the reliability of the findings.

Literature Review
Written examinations can be a difficult task for students who do not have a good command of the English language and have received their elementary school education in public schools. As a result, students are unable to score well in written examinations as they lack the necessary language skills, including spelling and grammar. In such cases, oral exams often happen to be a boon as they focus on the communication skills and the knowledge content of the student [1]. Students can give, at times, examples to prove their viewpoint and the examiner has the freedom to probe the examinees to obtain the required answers. Hence, oral exams can be used to complement conventional written examinations so as to foster in-depth learning of a course.
The Center for Enhancement and Teaching and Learning (CETL of HKU) [2] has identified Oral Exams to be a form of assessment whereby the students are questioned on the basis of the Learning Outcomes of the Course. According to Prof. of Philosophy, Marianne Janack of Hamilton University, Oral exams motivate students to think "out of the box" in terms of their topic and, at the same time, make them more well-versed with their subject [3]. A study that ventured into the benefits of Oral exams for Introductory Biology [4] has indicated that verbal exams are an effective means to promote higher level student learning and also enable the instructors to assess and aid the students, particularly in Introductory Courses. Oral exams also seem to have an added advantage in that they tend to bridge the gap between students with learning disabilities and normal students as students with learning disabilities have faced the issues of grammatical errors, punctuation, spelling mistakes and so on. [5]. A number of authors have also propounded that some students face less difficulties while answering an Oral Exam as compared to written assessments [6]. Another major benefit for Oral Exams is that they tend to be very interactive for both the assessor and the student [7]. Hasmik Gharibyan [8], in his article, stated that students who take courses such as Computer Science would benefit more from Oral exams, and this substantiates the need for different assessment techniques for different courses. One of the main benefits of Oral exams is that they help in evading the problem of academic dishonesty, which is very prevalent in written examinations [9], [10].
However, the methodology of conducting Oral exams poses a challenge in terms of reliability and validity since, in certain cases, the marks awarded for Oral exams seemed to moderately correlate with overall academic performance [11] It has also been noted that many students feel anxious and nervous when it comes to taking Oral exams as they lack initial experience in such exams [12]. Oral Exams can also be time-consuming when it comes to classes with more than 25 students as allotting sufficient time for each student can be quite challenging given that some students take more time to answer questions. Some students become very stressed out as Oral exams require them to give an answer "on the spot", which can be quite challenging to those students who face problems in spoken English. However, these hurdles can be overcome by proper planning and preparation. Today's corporate world requires a great deal of analytical and communication skills, and oral exams could present a good opportunity to eliminate such anxiety and instill confidence in the students.

Research questions
Here, we are trying to ascertain the validity of Oral Exams as a supplementary assessment tool for analyzing students' performance. Our research questions are as follows:  Whether oral exam marks are related to student performance in a Microeconomics course  Whether oral exam marks are related to student performance in a Macroeconomics course

Research tool
We have used oral exams (a simplified version of the viva-voce) as a component of testing methodology to assess students' learning. The oral test required each student to explain a concept and provide examples to show its application, or a case is provided by the faculty and the student is required to identify the concept involved. The exam involved the same amount of time for each student, using predetermined question sets of almost the same difficulty level and administered with the same wordings and order.
The topics included in the sets, which spanned across 5-6 chapters, were shared in the class with the students in different sections to ensure uniformity. Five sets of questions were prepared from the topics announced and met the learning outcomes of the course. The students could choose one out of five sets randomly. The students were then given an aggregated score between 1 and 10 based on their performance.
Quantitative analysis has been used to validate whether this pedagogy helps in reflecting students' learning. The period of analysis was from 2015 to 2018.

Methodology
The Interactive Compensatory Model of Learning (ICML) by Brooks, Schraw, and Crippen [13] proposes a model of learning and suggests methods to improve students' learning. The research, originally based on learning science, identifies five components that contribute to learning: cognitive ability, knowledge base, learning strategy, metacognition, and motivation. To improve students' problem-solving and metacognitive skills, the research suggests the use of strategic instruction and frequent opportunities for practice and timely feedback. Oral exams help to integrate many of the components of the ICML model by providing an opportunity for students to receive immediate, performance-related feedback [12]. Though the model proposed was in learning Science, it has been considered for the present study since our aim is to improve students' learning and enhance their academic and work performance. Using this premise, this study attempts to establish that oral exams could be a vital tool in contributing to this goal.
Oral exams which require students to think 'aloud' make them think carefully, thereby developing their cognitive ability. This extra measure of care is evident when students correct their initial response even before finishing their statement, indicating that speaking out loud makes them work logically. This process of checking and clarifying their established knowledge base steers them towards the correct answer, and gradually helps them to apply it to subsequent problems. Students start formulating their learning strategy and become aware of the most efficient and effective ways to express their learning. Assessment scenarios (such as final exams) further provide opportunity and motivation to test out their learnings.
This study proposes oral exams as a useful supplement to the conventional assessment system used in business courses. Oral exams (a simplified version of viva-voce) are used here as a component of testing methodology to capture students' learning. We have used quantitative analysis to validate whether this pedagogy helps in reflecting this.

Sample
The data collection was done in a medium-sized private educational institution in Muscat, Sultanate of Oman, with a total student population of 1500 across different programs. English is mandated as the language of instruction for all undergraduate study in Oman, and students come from both public (state) and private schools. The oral exam scores of multiple sections of students in two introductory courses -Microeconomics and Macroeconomics-were taken across several semesters and involved a census method. For the Microeconomics course, the sample consisted of six sections from Spring 2015 semester till Fall 2018, totaling 455 students. For Macroeconomics, the data spanned the same period, except for Fall 2016, totaling 256 students.
The two courses are mandatory for all the students in all the business programs of the college. Almost all are non-native speakers as they are from Oman with very few expatriates. These expatriates are from the MENA, the GCC, or the Asian subcontinent and hence are also non-native speakers. The class size for each section ranges between 18 and 28 students.
Apart from oral exams (10%), the other assessment items used in the course include three one-hour written tests (30%), one research project (10%), class participation (10%) and a final written exam (40%). The total available marks for each course are 100, which is considered as the "overall" marks for each student, and an indication of their overall knowledge acquisition from the course.

Data collection
The two introductory courses in Economics -Microeconomics and Macroeconomics -for which the oral exam was used are taken by "freshmen" level students. The number and type of questions depended on the course, class size and the purpose of the assessment [12]. The oral exam for the two courses were designed accordingly.
The grading rubric for the oral exam was given at the beginning of the semester, along with those for the other assessments for the course. The components of the criteria included: a). Comprehension of concepts of the course; for instance, knowing the definitions of basic economic terms such as demand, supply, equilibrium, GDP, inflation, recession and so on, and how they are inter-related. b). Demonstration of learning by providing relevant examples to explain how the different concepts impact our day-to-day activities, including demand for goods, components of GDP, and effects of inflation. c). Articulation of knowledge; for instance, analyzing the current economic scenario, identification of causation factors for recession in the local context, and so on.
For the Macroeconomics course, the rubric placed an emphasis on application whereas for Microeconomics, it reflected the need for understanding concepts.
The criteria helped to develop qualities such as critical thinking, soft skills, mastery of subject knowledge, and its application, all of which are highly valued in the contemporary workplace and need to be acquired by the students since they eventually aim to be employed. The students were given marks between 1 and 10 on each criterion. These were then aggregated to arrive at the oral exam score averaged to 10. Microeconomics and Macroeconomics are some of the first courses taken by students early in their undergraduate program. Use of oral exams in these early years can be challenging to students. Though oral exams are perceived to contribute significantly to the development of graduate attributes, they can induce a high level of stress. Therefore, adequate measures were taken to alleviate the anxiety in the form of clear standard oral instructions about the process, the content, the grading rubric, the time provided, and a sample of questions had been given beforehand. 6188 Effectiveness of an Alternate Assessment Pedagogy in Business Programs with Non-native English Speakers Students were also informed that a follow-up question or notes made by the teacher did not mean that they had given an incorrect answer. Instead, they were meant to clarify students' logic or understanding.
In terms of reliability, the use of data involving only one teacher is considered to increase reliability as this ensures consistency in the style of prompting [14]. Apart from design, the quality of the test administration process is critical to establish reliability. The exams were conducted in a student-faculty meeting room which was booked for the event and students were informed since there were multiple sections of the course taught by one faculty member during the semester. Students were given instructions to leave their mobile phones and other belongings outside the room. Due to the large class size, 15 minutes were allotted to each student for the oral exam followed by a break of 5 minutes to grade the performance of the students according to the rubrics. The oral exam weightage towards the students' total score in the semester was 10%. The oral exam was conducted a few weeks before the final exam so that it served as a self-assessment for students to improve their scores in the major written assessment.

Data analysis
Multiple linear regression was performed involving the Overall marks of the student as dependent variable, and Oral exam marks and Final exam marks as independent variables. The Final Exam is a comprehensive written exam in that it is supposed to cover the contents of the entire course and capture all the learning outcomes. The oral exam included all the topics and reflected the course objectives. Following are the null hypotheses that were studied: 0 1: Oral exam marks are not related to the overall performance of the students in the Microeconomics course The regression model proposed is as follows: Where Ov is the Overall Marks; O is the Oral exam score and W is the Written Final exam score for the Microeconomics course 0 2: Oral exam marks are not related to the overall performance of the students in the Macroeconomics course The regression model is proposed as follows: Where Ov is the Overall Marks; O ′ is the Oral exam score and ′ is the Written Final exam score for the Macroeconomics course The multiple linear regression was performed on the data for Microeconomics and Macroeconomics. Before conducting the multiple linear regression, a test of normality was conducted for the model to ensure that it was suitable for multiple regression analysis.

Results and Discussion
The Kolmogorov-Smirnov test for normality gave values which were more than 0.005 for both the models indicating that the values were normally distributed.
The regression models proposed and tested are: For the Microeconomics course: For the Macroeconomics course: The results of the multiple regression analysis indicate the following: ANOVA indicates the combined effect of both the predictors (Oral and Written exams) on the dependent variable. ANOVA provides high value for the two courses (F value (Microeconomics): 1459.5 and F value (Macroeconomics): 601, p< .00) with a high level of significance. This shows that there is a good fit for the models, that is to say that the two models proposed are valid and significant.
For both models, 2 and Adjusted 2 values have been found to be very high. For Microeconomics, it is at 0.870 and 0.869 respectively and highly significant (p < .00) and for Macroeconomics, 0.830 and 0.828 respectively and with a high level of significance (p < .00). The high value for adjusted 2 indicates that the predictor variables are significantly contributing to the variance in the predicted variables. We can infer that performance of students in both oral and written exams are found to be a good predictor of their overall performance for both the courses. Therefore, there is strong support for the alternative hypotheses -oral exams are related to the overall performance of students in both the Microeconomics and Macroeconomics courses. Consequently, Oral exams are valuable as a separate and meaningful assessment tool reflecting student performance.
In the attempt to understand the contribution of the two independent variables to the dependent variable, the analysis of the unstandardized coefficients was undertaken. The unstandardized coefficients for oral marks and written exams were found to be 1.278 and 1.636 respectively for the Microeconomics course, and 1.772 and 1.556 for the Macroeconomics course. It indicates that for every 1 mark increase in the oral exam, the overall performance in the course improves by 1.278 and 1.772 respectively for the two courses when the other independent variable is kept constant. It can also be inferred from the values that the contribution of the oral exam to the prediction of overall performance is higher in Macroeconomics compared to Microeconomics. This might indicate that oral exams are more meaningful for courses which need more application of concepts, such as in Macroeconomics. Since the performance of the students in the oral exam is reflected in their overall performance, Oral exams are found to contribute to student learning In order to establish the individual contribution of each of the chosen assessments and their combined effect on the variance in Overall marks, the analysis of 'Part' correlation was undertaken. The 'Part' correlation values indicated that the unique variance contributed to the Overall marks (which indicates student learning) by Oral and Written exams are 3.8% and 50.7% at p < .00 respectively for Microeconomics. However, their combined effect on the Overall marks is 32.5%, which is substantial. Therefore, we can infer that though the independent effect of oral exams on the Overall marks i.e. the overall student learning might be only 3.8% but when combined with written assessment, it contributes almost 32.5% to students' learning. For the Macroeconomics course, the unique contribution of Oral and written assessments was found to be 7.3% and 45.7% at p < .00 respectively. Both the assessments together contribute to 30% variance in the Overall mark, thus to students' learning. This establishes the importance of oral exams as a supplementary assessment method along with written tests to improve students' learning.
The unique contribution of Oral exams in the Macroeconomics course (7.3%) is almost twice that of the Microeconomics course (3.8%). This is in line with the findings from the analysis of unstandardized coefficients. However, it is important to note that when Oral assessments are combined with Written assessments, they can contribute substantially to students' learning in both the courses.

Implications of the Study
Oral communication is an important graduate attribute, especially for business graduates, who would predominantly be expected to communicate and persuade their stakeholders. However, this does not find its deserving place among the assessments used in the different courses and curriculum. The literature reveals that few studies have explored this topic for the business curriculum, which is a significant gap. Though several studies in the area of pharmacy, sports education, medicine, nutrition and so on have explored and established the importance of this aspect, research has been largely missing in business education. This study is a step in this direction and the purpose is to establish the relevance and importance of the use of oral exams through empirical investigation. The longitudinal nature of the research helps to ascertain the reliability of the findings. The reliability and validity are further enhanced using cross-sectional nature of this study which involves the two introductory economics courses.
The results of the study reveal that:  Oral exams independently contributed to students' performance and learning in both the courses;  Oral exams along with Written exams more significantly contributed to students' performance and learning in both the courses.
Improved scores in the oral exams contributed to the students' overall performance in that as the scores of the students' increased, their overall marks for the course were also found to increase. The results reiterate the validity and usefulness of oral exams as a supplementary assessment method. Furthermore, students' knowledge and learning are found to be enhanced with the use of oral exams. The finding supports Brooks, Schraw and Crippen's (2005), statement that, through performance-related feedback, the knowledge base of students is refined during the examination thereby enhancing their cognitive ability.
Both oral and written exams uniquely contributed to the overall grades. However, it is interesting to note that Oral exams are more effective when combined with Written exams. This strongly justifies the use of both the formats together for better outcomes.
The individual contribution of the Oral exam score to the overall grades was higher for the Macroeconomics course than for Microeconomics. This could be attributed to the nature of the courses. Microeconomics is more of a conceptual course while Macroeconomics is characterized by application of the concepts of Microeconomics. This finding strongly supports the use of oral exams for application-oriented courses.
Almost one-third of the variance in the Overall marks, and hence students' learning in both the courses, is through the combined effect of Written and Oral exams, thereby strongly supporting the use of both these assessment methods together for better students' performance and learning.
The one-to-one interaction offered by oral exams also serves as an alternative method to check the understanding of students who face difficulties due to language issues or comprehension of content. This type of assessment provides an opportunity to observe each student individually and helps gather insights in order to tailor the materials and approaches to achieve optimum learning.
Oral exams emerge as an important and essential testing method for non-native speakers who have challenges with having written exams in English. Oral assessments would be a valuable tool to provide the right opportunities to exhibit learning by such students. The "played back" strategy suggested by Kovacs-Boerger [15], which involves a teacher reiterating a student's answer using the student's own words, helps them to instantaneously identify where they might have erred. Discovery of error by themselves increases students' knowledge as learners and contributes to their motivation.
The findings, therefore, support the use of oral exams as a supplementary assessment tool to embed concepts and learning, and hence aid in the achievement of the learning outcomes of business programs.

Implications to Various Stakeholders
Oral exams might provide great promise towards improvement of students' academic performance as well as the interests of various stakeholders. However, the success would depend on how the assessment is administered. Standardizing the process by providing the same amount of time, pre-determined questions using the same wordings for all students, having multiple sets to have adequate randomness and so on would be important. To improve the scoring reliability, pre-determined evaluation criteria and rubrics should be administered to all the students [16].
The anxiety quotient of the oral assessments needs to be managed using appropriate strategies, including giving due attention to the process beforehand to reassure students regarding what to expect. Sharing the experience of students in the previous semesters also vicariously prepares them for performing better in the Oral exams. The students benefit immensely from the experience since it mimics real-life situations in the workplace. Apart from the initial "anxiety" factor, most of the students are comfortable with oral exams as they feel they can express themselves more clearly verbally than on paper. Their confidence and cognitive ability seem to receive a significant boost.
For instructors, feedback from oral exams can prove very useful to identify weaknesses in instruction and to personally grow as facilitators of learning. Non-native speakers have difficulty in understanding instruction in English especially when faced with the complex vocabulary of the subject. In fact, the instructors who are from different contexts or countries are unaware of the many challenges faced by the students in the initial years of the undergraduate program. Oral exams provide the opportunity for teachers to assess the level of the students and adapt their instruction accordingly, helping to grow mutually.
Today's teachers are expected to be facilitators for students in the process of learning. To be able to meaningfully contribute to this goal, the instructors should be able to understand the barriers that interfere with students' learning. However, many of the barriers are context-, country-and course-specific and there have not been many studies in this area. Oral exams provide help to uncover those barriers and clearly see students' weaknesses, further providing opportunities for individualized mentoring and facilitating learning.
Though oral exams can consume a great deal of time for faculty, they can save on the time taken in grading or evaluation. However, this is applicable only for small to medium-sized classes. Larger class sizes may put a substantial strain on the instructor unless teaching assistants are available.
The industry desires the business graduates to have good interpersonal skills. Oral assessments can be considered vital in the development of soft skills and thereby contribute towards this goal. The students should be made aware of which graduate attributes are valued by the industry and the approach used to embed them in the assessment formats. The next step in the process is to make it reliable by using the right design and implementation procedures.
Private higher education institutions operate not only for the corporate goal of profits but also for making meaningful contributions to developing the young minds of those who are the architects of the future. Ensuring better learning and attributes in these college graduates can guarantee a sustainable future for the nation and the economy, which is essentially becoming knowledge based.

Conclusions
Academia can be a challenging field as we need to safeguard the fact that we are responsible for producing "industry ready" graduates. Oral exams help in reinforcing different desirable attributes in students because they imitate real life interview sessions, where students face the interviewer alone and have to be spontaneous with their answers. It helps in improving the communication skills of students to a great extent. Instructors can also benefit from oral exams as they are able to grade the student on the spot (based on rubrics). It also enables the instructors to know in which areas students face difficulties.
This study attempts to explore whether Oral Exams contribute to the knowledge and learning of non-native English-speaking students in Microeconomics and Macroeconomics courses at the undergraduate level. The study reveals that Oral exams do contribute independently to improve the performance of students in both courses and are a valuable assessment tool. However, the effectiveness of Oral exams towards overall learning increases substantially when combined with Written exams.
Oral exams can serve as a beneficial complementary testing method to the already existing assessment techniques in many business subjects as they help to widen the knowledge base of students and improve their communication skills, which are highly valued in today's world.

Limitations of Research
The study was done only in one private college in the Sultanate of Oman and only in two introductory economics courses. Considering the nature and content of the courses in the business programs, the findings are quite reliable. However, replication of the study in other courses in the business program or in other countries with non-native speakers would help in establishing the robustness of the findings.
Use of one rater could also be considered a limitation but considering the large classes, student anxiety, limited time during the semester and other types of assessments that are required to be used, multiple rater or test-retest approach was not feasible.