The Application of a Textbook Evaluation Checklist: A Research Study of English as a Foreign Language Textbook

Locating the appropriate teaching and learning material for students and learners is an essential task for leaders at colleges and universities. However, without an effective textbook evaluation checklist, department heads may not have any directions on how to score the textbook materials. Therefore, the development of an effective textbook evaluation checklist and the related application is needed. The purpose of this research study has two directions. First, the researcher employed a widely used Textbook Evaluation Checklist to measure the effectiveness and usefulness of an English as a Foreign Language textbook. Second, the researcher employed the textbook evaluation checklist and produced a result of this targeted English as a Foreign Language textbook. The surveys have been collected from 323 college and university students who completed their first-year or second-year English as a Foreign Language course in South Korea. The results indicated that the textbook evaluation checklist was effective, useful, and user-friendly. Also, based on the written feedback from the students, the researchers, department heads, publishers, school leaders, and policymakers could take the feedback to reform their process of textbook selection.


Research Background
English for Academic Purpose (EAP) has been established and taught for nearly half of a century in many locations and schools across both English-speaking countries and non-English speaking countries internationally [1]. In fact, the English language is one of the most popular languages in both academic areas and business fields due to international coverage [2]. Also, due to the reasons of globalisations and international interactions, a standardised language should be established for the purpose of communication [3]. The English language was selected as the one which allowed people from different parts of the world to exchange and share their understanding and idea. As a result, many schools, colleges, and universities always required their students to complete compulsory English language courses for both personal development and career development [4].
For more than three decades, the English language has become one of the most popular foreign languages in South Korea [5][6][7][8]. Due to the rapid development of the East Asian region, a large number of visitors decided to come to the East Asian region for the purpose of leisure. As a result, the professional workers in the field of service management, hospitality and tourism management, and business management, are required to have upper-level of the English language proficiency in order to conduct trading and commercial activities with visitors from different parts of the world [9]. Although researcher [10] has indicated that Chinese Mandarin, Spanish, Japanese, French, German, and Portuguese are some of the popular foreign languages for learners to master, the English language is considered as the top choice for many South Korean learners.

The Purpose and Significance of the Research
Many colleges and universities in non-English speaking countries usually required their students to have at least an upper-intermediate level of foreign language proficiency in order to graduate from the colleges and universities. In the case of South Korea, although the government did not officially indicate colleges and universities graduates must have certain levels of the English language proficiency in order to earn their undergraduate degree, many colleges and universities required students to take different types of English language examinations and tests before graduation [5]. However, locating the appropriate textbooks and related materials, such as videos, audio-recording, exercises, and even teachers' teaching materials, is the most important process as both teachers and students rely on the applications and instructions [11][12][13][14][15]. As a result, there are four purposes for this study.
First, unlike some literature, liberal arts, religious studies, and philosophy courses with stable teaching and learning materials, foreign language teaching and learning always require continual evaluations as the needs, demands, and social-cultural developments change from time to time [12,16]. For example, due to the development of technologically assisted teaching and learning materials, foreign language teaching and learning materials should be updated with the latest technology.
Second, due to the demand of different industries, English language learners should have different aims for English language learning and purposes. [17][18][19][20][21]. Therefore, a single textbook and teaching and learning material cannot meet the needs of multiple learners and departments.
Third, the teaching and learning behaviours and strategies of different age-groups are not the same. Although colleges and universities tended to have a larger number of traditional-age students' enrolment, many colleges and universities always welcome non-traditional, returning, evening, and adult students for college and university education [22]. Additionally, many postgraduate students want to upgrade their English language proficiency due to the demands of their industry leaders.
Fourth, the themes and series of each textbook and related material could be different due to the styles of authors, book publishers, geographic regions, ethical background, and even religious understanding [23,24]. Therefore, although some textbooks and related materials received excellent ranking and feedback from one region, the materials and details may not meet the needs of all learners internationally.
Based on the abovementioned purposes and significances, the researcher decided to conduct a research study about an English for Academic Purpose textbook evaluation and the application of a textbook evaluation checklist. The results of this research study outlined two directions. First, the application and the effectiveness of the textbook evaluation checklist for textbook evaluation. Second, the appropriateness of an English language as Foreign Language textbook materials for college and university students in the South Korean educational environment.

Literature Reviews
Teachers and educators should conduct material-based measurements and evaluations. In fact, textbooks and related materials were designed to meet the needs of both students and teachers. Although the materials were well-designed and established, some parts of the materials may not meet the needs of the learners, background, and the social-cultural context of the community. Without the measurements and evaluations, teachers may select inappropriate materials to learners [11]. Several scholars indicated that such evaluations and checklists could be categorised as pre-use, in-use, and post-use evaluation forms [25,26]. Also, the material measurement and evaluation checklists always help educators and teachers to understand the advantages and disadvantages from the perspectives of the material users (i.e. teachers and students). Although pre-use, in-use, and post-use checklist may have minor differences due to the nature of the applications, teachers and educators may edit the items based on the needs of the situation.
Most textbooks and materials may be evaluated with the checklists and evaluation forms, regardless of the subject matters. For example, several previous studies had employed the checklists to understand the appropriateness and effectiveness of foreign language textbooks internationally [11,13,15,27,28]. These studies employed the checklist as the tool to evaluate the appropriateness and applications of the textbooks and related materials from the perspectives of students, teachers, department heads and school leaders with the connection of the industry leaders. The quantitative scales (i.e. Likert-scale) with ten semi-structured questions allowed the users to provide directed answers to understand the effectiveness of the materials.
Some scholars [29] argued that the checklist and measurement should be picked based on the subject matters and social-cultural context in order to meet the requirements and needs of the materials. However, Ur (1996) [30] argued that although checklists could be 3866 The Application of a Textbook Evaluation Checklist: A Research Study of English as a Foreign Language Textbook different due to the subject matters, teachers and educators should edit the checklists with the context and content based on the needs of the learners. Teachers and educators, however, should not edit the checklist (i.e. majorly) based on their background as the developments of the checklist should have its own purposes and advantages for measurement and evaluation.

Materials and Methods
The current research study employed a well-established textbook evaluation checklist to test the effectiveness of a textbook material in the field of English as a Foreign Language. This textbook evaluation checklist was first developed in for a Japanese as a Foreign Language textbook in the United States. Over the years, the textbook evaluation checklist was employed by a large number of research studies in similar fields and directions.

Participants and Sites
In order to collect some wider data information from different learners at three South Korean universities with various departments and faculties, the researcher invited 323 college and university students (i.e. first-year and second-year students) for the survey.

The Textbook Material
The teaching and learning material to be evaluated was the Stretch Level 1: Student's Book with Online Practice by Stempleski (2014) [31]. The book was published by the Oxford University Press with the American standardised English language as the teaching and learning tool.
The Stretch is a textbook series and system with four different levels, which are Stretch Starter, Stretch Level 1, Stretch Level 2, and Stretch Level 3. The current teaching and learning material was the Level 1 series. Based on the level, the teaching and learning material was designed for students with at least one year of English language proficiency. Based on the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFRL), the current teaching and learning material was at the A2 level.
The Stretch Level 1: Student's Book with Online Practice [31] has 12 Units with eight major sections, as the following table. The eight major sections were 1) Vocabulary; 2) Speaking; 3) Grammar; 4) Reading and Writing; 5) Viewing (BBS Worldwide Learning); 6) Presenting; 7) Culture Talk; and 8) Learning Outcomes. Table 1 refers to the Unit arrangements of the teaching and learning material. Besides the written practices and exercises, the teaching and learning materials also involved audio listening practice, online practice, and watching component.

The Data Collection Procedure
First, the nature of this survey is to collect meaningful data information from students who have completed the English language course with the targeted teaching and learning material. In order to provide meaningful data information, the students must complete the course before finishing the survey, regardless of their grades [26,27].
Second, the researcher contributed the survey to the students (i.e. who completed their courses with this teaching and learning materials). The survey was given to each student during the last day of the course. All students understood the completion and answers of the survey would not impact their grades and outcomes of the course.
Third, in order to collect some wider data information, the researcher also invited other students from other colleges and universities who had completed their English language course with this targeted teaching and learning material. The students followed the same procedures and progressions.
Due to the language barriers and understanding, the survey was written in both English language and Korean language. Therefore, students should understand the meaning, purpose, detail, risk, and outcome of this survey. Students had the rights to complete or not complete the survey as this was not compulsory for the English language course. However, no students refused the survey [34].

The Data Analysis Procedure
The researcher collected 323 physical surveys with meaningful data information and evaluation of this teaching and learning materials. With the inductive approach [35], the researcher merged the numbers and short answers based on the opinions and feedback. All students completed the Likert-scale numerical answers. Less than one-fifth students provided effective short answers. However, the researcher merged and gathered some effective short answers for reporting [36].
It is worth noting that the researcher did not categorise the location (e.g. which school), classrooms (e.g. which English course), instruction (e.g. which teacher), and name (e.g. which student). In other words, all surveys were analysed based on the data information. No identities and personal information were marked.

Results and Discussions
The Likert-Scale textbook evaluation checklist regarding the teaching and learning material for this research study was designed to explore students' perspective of the effectiveness of a group of first-year and second-year students. They needed to complete their general English language requirements for their undergraduate degree. The responses provided to the participants were rated as of the following: Strongly Agree / SA (5); Agree / A (4); Neutral / N (3); Disagree / D (2); and Strongly Disagree / SD (1). In addition to the Likert-scale scores, each survey question came with an optional comment box as well as the overall comment box for the whole teaching and learning material. Table 2 refers to the Summary of the Teaching and Learning Material textbook evaluation checklist.

Price Consideration
The first textbook evaluation checklist question was: The textbook is easy to access, particularly the price. Over 90% of the students advocated that the current teaching and learning material was affordable based on the South Korea standard. Based on the major South Korean-based bookstores and online retailers, the current price for this teaching and learning material is 18.000KRW ($15 USD). The current price of this teaching and learning material was considered affordable in the East Asian region (e.g. Japan, South Korea, Taiwan, Hong Kong, and Macau) based on the current standard.
Based on the concerns of the price, several students indicated that the current teaching and learning material for their English language course is cheaper than all of the textbooks and materials from other courses, said, "this English book is cheaper than all my other courses' textbooks. I like the price of this book." (Student #301). Another student also shared a similar idea about the price of this teaching and learning material, said, "This textbook is the cheapest among all. I wish the publisher can apply this price to all other textbooks." (Student #22).
In short, almost all students agreed that the current teaching and learning material was affordable based on the current South Korean living standard. Unlike other textbook materials and teaching and learning material with higher copyrights fees, the current teaching and learning material was six years old already. Therefore, the contemporary demands were lower than other textbooks and teaching and learning materials in the market. As a result, almost all students could afford the price for their English as a Foreign Language course at the college and university level. The second textbook evaluation checklist question was: The textbook materials and exercises are contemporary and applicable. Over 90% of the students advocated that the current teaching and learning material, and the related exercises are contemporary and applicable to their workplace, society, community, and social-cultural environment. In fact, many of the English as a Foreign Language teaching and learning materials were designed for students at all colleges and universities internationally. In other words, unless targeted designs, most of the English as a Foreign Language materials followed the universal standard. Students with various backgrounds and understanding should be beneficial to the learning guidelines of the materials.
With the direction of general education requirements and general English course requirements for first-year and second-year students, most of the students understood that these English courses were designed for general foreign language proficiency. Therefore, based on the general learning direction and perspective, over 90% advocated that the general direction and universal standard of the current teaching and learning material were applicable to their knowledge, one said, "I can learn the daily English conversation and reading from the textbook." (Student #33). Besides the daily uses of the teaching and learning material, a few also believed that "the reading allowed me to read knowledge from people in different countries. I learnt some general knowledge from the reading." (Student #89).
In short, due to the nature of the relationship between general education and the English language course, most of the students believed the textbook materials and exercises are contemporary and applicable. Also, many advocated that the universal standardised reading and materials always allowed them to gain some overall performance and understanding internationally.

Relationship between Exercise and Application
The third textbook evaluation checklist question was: The students' exercise is applicable and useful. Nearly 90% ranked Strongly Agree, and nearly 10% ranked Agree for the relationship between the exercise and application of the teaching and learning material. Echoing the second textbook evaluation checklist question above, most of the students were first-year and second-year students for the general education requirements. Therefore, most expected that the current English language course and the teaching and learning material were designed for the general purpose and perspective. Therefore, unlike the courses for their major and specialised expectations, almost all agreed that the current teaching and learning material and related exercises applied to their knowledge, one said, "the exercise and word banks are very interesting. It allowed me to increase my vocabulary and grammar banks." (Student #1).
In short, as most of the students understood the nature of their general education requirement and the application of the general foreign language teaching and learning direction, almost all advocated that the exercises of the teaching and learning materials were applicable and useful to their language learning knowledge.

Pronunciation Training
The fourth textbook evaluation checklist question was: The textbook provides practices about essential pronunciation training. Over 95% of the students indicated they strongly agreement about this teaching and learning material. First, several students indicated that the teaching and learning material was equipped with audio-recording, which allowed them to listen to the conversations and materials during leisure time, said, "I feel good that I can listen to the unit and reading at home." (Student #67). Another sharing was captured, said, "The textbook has an online website, I can download the audio recording online. So, I can practice." (Student #33).
Second, some students indicated that they were able to find some related speaking exercises online. Therefore, besides the traditional online teaching and learning platform from the publisher, many advocated that the materials were equipped with effective audio materials online, as one said, "I can find some speaking exercise and oral speaking online. I think I can access these materials at home." In short, both online teaching and learning platform and online open-access teaching and learning materials always increased the effectiveness and interests for the speaking training and exercise. Unlike the traditional teaching and learning materials which required in-class and face-to-face training, the technologically assisted materials allowed students to access different materials at home and during leisure time.

Well Organisation
The fifth textbook evaluation checklist question was: The textbook is well organised. Unlike other teaching and learning materials categorised the focuses with different series, this teaching and learning material combined all information, skills, and training into one material. As all information was combined, the organisation is vital. Therefore, the current teaching and learning material categorised the learning sections into eight major sections, including 1) Vocabulary; 2) Speaking; 3) Grammar; 4) Reading and Writing; 5) Viewing (BBS Worldwide Learning); 6) Presenting; 7) Culture Talk; and 8) Learning Outcomes. As a result, almost 100% of the students agreed on the effective organisation of this teaching and learning material.
Some students shared a number of effective and meaningful feedback about this teaching and learning material. First, many indicated that the cohesive connections between different sections allowed them to engage the knowledge and exercises as a unit, as said, "I like each section has the connection." (Student #45). Another student also indicated similar comment, said, "the exercise connected the previous knowledge…" (Student #21).
Besides the cohesive connection between section, several also believed the four skills training were engaged and connected between units and unit groups. In other words, units were engaged with similar and cohesive themes as a teaching and learning cohort. It increased students' knowledge bases and learning themes as a whole, one said, "unit 1, 2, and 3 are connected. I can apply the previous unit's Vocabulary into the unit the current unit." (Student #104).
In short, many advocated that the organisation of a teaching and learning material always increased their interests and learning experience. In this case, this teaching and learning material matched the expectation. Also, the textbook evaluation checklist allowed the students to express their feedback.

Well Balance between Activities and Lessons
The sixth textbook evaluation checklist question was: The textbook offers a balance between activities and lessons. Although teachers and instructors have the management to arrange the balance between activities and lessons, the teaching and learning materials could suggest the recommended management. Based on the score from the students, over 85% indicated that balance and management were positive for this teaching and learning material. For example, some students advocated that there were related exercises after each major section, said, "I like the arrangement of the section. Each section is accompanied by related exercises." (Student #89).
Besides individual activities, many students said the teaching and learning material designed positive and engaging pair works and interactions between students and groups. In fact, foreign language teaching and learning should not be isolated. Therefore, appropriate engagements and activities were positive, said, "I like the book…allowed us to have the pair works with my classmates…" (Student #91).
In short, foreign language teaching and learning highly relied on the interactions between the activities and lessons. In this case, this teaching and learning material allowed students to balance their energy and focus in between.

Encouragement of Learning
The seventh textbook evaluation checklist question was: The activities encourage learners to learn, communicate and participate. Surprisingly, only 62.8% of the students strongly agreed the teaching and learning material encouraged learners to learn, communicate and participate. At the same time, 31% of the students agreed with the statement. Although most agreed on the supportive encouragement, not all strongly agreed. The researcher captured some sharing from the students. For example, several students (i.e. ranked agree) indicated that the activities were traditional. In other words, many youths disliked the traditional discussion and written exercises as their learning pathways, said, "the exercises are all written tasks. Very boring…" (Student #221).
Several students indicated that the contemporary generations and youths prefer technologically assisted and internet-based teaching and learning materials and visual teaching and learning materials as their major tools of learning. For example, several indicated that they want to watch videos and visual-oriented materials as the tools of learning, said, "Very boring, I want to watch more movies and videos." (Student #294). Similar feedback was captured, said, "I like to watch some moving pictures, not the drew pictures on the book." (Student #163).
In short, unlike the past generations, the current generation highly relied on the visual-oriented teaching and learning approach [31][32][33]. Therefore, teachers, instructors, authors, and publishers may use this textbook evaluation checklist as the tool to understand the feedback from the users.

Grammar and Vocabulary Connection
The eighth textbook evaluation checklist question was: The Grammar and Vocabulary are introduced in realistic contexts. Both grammar and vocabulary teaching and learning were important for second language learners, particularly for students with lower-level proficiency. As a result, over 90% of the students advocated that grammar and vocabulary applications were positive for their learning. Specifically, several students shared positive feedback based on the unit arrangement and cohesive connections between the exercise and Vocabulary, as one said, "I like the Vocabulary from Unit 1. I learnt the name of my job." (Student #307).
In addition to the occupation-oriented Vocabulary, the teaching and learning material also connected the Vocabulary with grammar and environmental arrangements. Some students shared their ideas about the connection, said, "We learnt some adjectives and some cities…We can connect the Vocabulary together…to describe the cities…" (Student #256).
In short, the connections between grammar and vocabulary are the key for many lower-level students as they need to have the knowledge banks for communication. Based on the feedback and opinions collected from the textbook evaluation checklist, the researcher advocated that this teaching and learning materials encouraged the knowledge bank of the learners.

Four Skills Balancing
The ninth textbook evaluation checklist question was: The Textbook offers a balance of four skills. Over 90% of students strongly agreed the textbook provided an excellent balance between the four skills. Four skills mean reading, 3870 The Application of a Textbook Evaluation Checklist: A Research Study of English as a Foreign Language Textbook listening, speaking, and writing. Although some teaching and learning materials categorised different skills into different books and materials, the current teaching and learning materials combined all four skills into one textbook. However, almost all of the students strongly agreed the combinations were effective for their learning experience, as said, "I really enjoy. I can learn all skills all-in-one at once." (Student #202).
Second, due to the cohesive connections between each section and unit, many students found that the connections allowed them to engage different skills from unit to unit and section to section, as said, "Unit 5 on the weekend and Unit 6 downtown. I can connect many vocabularies and speaking skills between units and sections." (Student #3).
In short, four-skill training is essential for most of the foreign language teaching and learning materials. For this teaching and learning material, the publisher combined all four-skill together as a whole for the teaching and learning purposes. However, the balances and inter-connections matched the expectations and needs for both students and teachers. Future publishers and teachers could follow the recommendations of this teaching and learning materials for reform and polishing.

Appropriate Level
The tenth textbook evaluation checklist question was: The language usage is at the appropriate level of students. Over 95% of the students advocated that the language level matched their language level and expectation. In order to complete this survey, students must be first-year or second-year college and university who completed this English as a Foreign Language course as their general education requirement. Based on the South Korean educational system, students should complete their high school diploma and took the South Korean College Scholastic Ability Test (CAST) in order to enrol in one of the undergraduate education programmes. One of the requirements of this CAST exam was the English language test. Therefore, students should have at least lower-intermediate English language proficiency for college and university education.
However, due to language proficiency, most of the first-year and second-year college and university students in South Korea only have an A2 level of English language proficiency. In this case, over 95% strongly agreed this current teaching and learning material satisfied their needs and expectations, said, "I liked the level of English of this book." (Student #50).
Some students found that they have learnt some knowledge and information from the current teaching and learning material. However, many would like to enhance their previous knowledge and information from this lesson and this teaching and learning material, said, "I like to refresh my knowledge, vocabulary, and grammar in this course." (Student #111). Another student also echoed similar feedback, said, "I can practice my speaking skills. I have learnt this skill and some vocabulary in high school. But I can refresh this again." (Student #113).

Conclusions and Future Research Direction
The completion of this research study concluded three key points for the current literature. First, many colleges and universities always locate the appropriate teaching and learning materials for their students and teachers. However, without an effective textbook evaluation checklist, it is hard to understand the effectiveness and usefulness of the materials. As a result, the inappropriate teaching and learning materials matched the inappropriate students and teachers. The results and outcomes of this research study provided the ideas about textbook evaluation checklist. They increased the interests about how textbook evaluation checklist should be employed to school leaders, department heads, teachers, students, employers, and even publishers to reform, polish, and upgrade their teaching and learning materials.
Second, the current textbook evaluation checklist was relatively new in the current literature review. Therefore, further testing and research studies are needed. This research study re-tested this textbook evaluation checklist for its effectiveness and usefulness. The result indicated that the textbook evaluation checklist was effective. Further research studies could employ teaching and learning materials in other subject matters. In fact, not only foreign language teaching and learning should conduct detailed evaluation but also other subject matters.
Third, the outcomes of this research study indicated a useful teaching and learning material for English as a Foreign Language learners with lower-intermediate English language speakers. Although the research study was conducted in South Korea, students with a similar background and language proficiency could employ this teaching and learning material to their language learners. Further research studies may employ this textbook evaluation checklist to test their current teaching and learning materials for the effectiveness of their textbook.