Turkish Metalinguistic Awareness Scale: A Validity and Reliability Study

The aim of this study is to develop a useful, valid and reliable measurement tool that will help teacher candidates determine their Turkish metalinguistic awareness. During the development of the scale, a pool of items was created by scanning the relevant literature and examining other awareness scales. The materials prepared were re-examined according to the opinions of two lecturers working in the field of Turkish language education and two working in educational science and a draft scale was prepared. The items in the draft scale were evaluated by pre-application, items with a weak measurement relation were removed from the scale and the scale was prepared for factor analysis. The scale, designed as a 5-point Likert type scale, was applied to a total of 480 students. The α value of the scale formulated by factor analysis (Principal Component Analysis) was found to be 0.87. As a result of the analyses made, it was determined that the scale had 6 sub-factors and factor loadings ranging from 0.499 to 0.841 were found in these factors. These findings show that the scale can be used to determine Turkish metalinguistic awareness and is a valid and reliable measurement tool.


Introduction
A nation can maintain its existence by developing a national consciousness and a common language. Language is an important structure which is commonly referred to by the adjective derived from the nation's name and which carries within it all the elements that make up a sense of national identity, enabling the nation to take up a permanent place in history. The elements of national culture are transmitted to and by individuals through language. The nation and the culture to which individuals feel themselves to belong are embodied in language. Furthermore, the ways in which societies think and express themselves also develop and are communicated through language. From this perspective, language can be thought of as one of the deepest bonds connecting one individual to another and to their shared culture. As Aksan [5] points out, language is the element that most strongly reflects the unconscious culture that underpins the relation of people to society.
A nation's conscious development of a common language is also directly related to how that language is used. Using a language is a broad process involving actions practiced by all speakers of the language as much as the individual. As shared patterns of communication and interaction develop through the use of a language, a common language and cultural heritage emerge. Maintaining this heritage also helps to preserve the bond between the past and the future and increases individuals' respect for and loyalty towards their own languages and cultures. The conscious use of a language is the clearest indication of this respect and loyalty.
Language consciousness, language motivation, language awareness and language attitude are important factors affecting language preference and language usage. In this framework, language awareness can be defined as the specific knowledge that an individual has about the language they speak. It can also be described as individual's consciousness and sensitivity towards language teaching, language learning and language usage and as the conscious attention paid to the relationship between culture and language [24]. The fact that an individual is intricately involved with language and culture in a social context enables them to acquire use of language as a natural process. As a result of making language and culture a part of their own lives, all the psychological behaviours, feelings and thoughts individuals develop with regard to language help determine the kinds of language they prefer to use. In this respect, language awareness is also one of the cognitive factors affecting language usage.
Language awareness is an important element that contributes to linguistic unity and promotes consensus and understanding between individuals speaking the same language. If the individual has an awareness of the language they are using, a sense of ownership of the language and sensitivity in using it develops. Language awareness consists of the totality of language sensitivity, consciousness and perception. It is general cognition about language. In this respect, language should be assessed as it is, and in its entirety, without being judged, rejected or suppressed.

Importance and Purpose of the Study
When language awareness is examined in the context of literature, it is seen that issues regarding awareness tend to be handled individually and are mostly focused on certain types of awareness, while other topics are less researched and the sample groups studied mainly consist of children. In all these studies, no specific research has been carried out regarding measurement and evaluation tools for Turkish metalinguistic awareness and the measurement instruments used in the research tend to lag behind the subjects studied.
This study is therefore needed to fill the gap in this area of research, to inspire other researchers to conduct similar research, and to develop a valid, reliable and useful scale for measuring Turkish metalinguistic awareness.
The specific aim of this study was to develop a useful, valid and reliable scale that will help determine the metalinguistic awareness of Turkish teacher candidates.

Study Model
The research model used in this study is the screening model. It was used in order to provide a suitable model for large sample group surveys [51]. It is a widely used research model and since this is a scale development study based on determining awareness it was suitable for the research.

Participants
In this study, data were collected from 480 teacher candidates studying at Gaziantep University, selected by the easily accessible sampling technique (N=78 for the preliminary application and N=402 for the application). All individuals participated on a voluntary basis.
The descriptive characteristics of the participants are summarized in Table 1: Tavşancıl [65], who gave recommendations about sample size and how many people should participate, states that the number of participants should be at least 5 times higher than the number of items to be analysed with a factor of not less than 100 persons. In this regard, it can be said that the number of participants contributing to the research was sufficient.

Study Process
The process of the study consisted of the following stages: literature survey, item pool creation, validity of scope, pilot study and selection of materials, main implementation for factor analysis, structure validity, reliability calculation, factor analysis, naming and reporting of factor sub-dimensions.
The literature review examined scales developed in the field of language teaching and research on awareness, language consciousness, language use, language awareness, teacher language awareness and pragmatics. Teacher candidates were asked various questions in order to determine the level of their awareness about the topic, and the answers received were found to be inadequate, inconsistent or incomplete. There are many studies on the subject worldwide. In the Turkish literature, however, besides there being only a limited number of studies, it was seen that language awareness is mostly handled in terms of phonetics, morphology, syntax and semantics. It was also observed that studies related to reading and writing language skills are included within this field. It could thus be argued that a comprehensive scale should be developed which includes not only the development of Turkish metalinguistic awareness but also the communicative and cultural dimensions of language.
Based on the studies examined, 15 items were determined for each of the themes of phonological awareness, morphological awareness, syntactic awareness, semantic awareness, communicative awareness and cultural awareness, making a total of 90 items. After the items were identified, they were presented to four experts, two of whom were working in the field of Turkish language education and two in the field of educational science. Items changed following feedback from the experts were rewritten in a form suitable for the 5-point Likert type scale, which would be bi-directional (positive-negative).
The validity of the scope was examined in order to check whether the items in the draft scale to be used in the study were adequate in terms of quantity and quality. Expert opinions were received to determine the validity of scope [23]. Expert opinions were also consulted to determine the intelligibility of the items and whether the characteristics to be measured were reflected in the items in the scale. Some items were corrected according to the feedback from the experts, some were removed from the scale (n=26) and the final form was then given to the draft scale. After this reorganization, there were 64 items on the draft scale. The scale was pre-applied to 78 students to determine whether the 64 items on the draft scale were appropriate for the sample group and whether they represented the behaviours intended to be measured. According to the data obtained from this process, the Cronbach's alpha (α) value of the draft scale was calculated as 0.79. After this stage the scale was applied to a sample group of 402 students.
Factor analysis (basic components) was performed to determine the structure validity of the scale. Factor analysis is used for cases in which items in similar categories can be grouped according to similar structures and qualities [46]. Kaiser Meyer Olkin (KMO) and Bartlett's tests, cofactor variance values, eigenvalue, principal component analysis and the varimax rotation technique were used in order to demonstrate the validity of the structure. The reliability of the scale was examined by calculating the Cronbach's Alpha (α) reliability coefficient and item total correlation. The α value is a measure of the internal consistency of the scale and values above 0.70 are considered sufficient for reliability. The item total correlation value is used to explain the correlation between the item score and the total score of the test. The positive values for the item total correlation and its high score indicated that the scale had internal consistency [23].

Validity of the Structure
In order to determine whether the data used in the factor analysis were obtained from a suitable sample group, the KMO test was performed. The Barlett test was also performed to show whether the correlation between the factors in the test was adequate [66]. The significance rate of the Bartlett test and a KMO value that is larger than 0.50 are considered to be necessary for the suitability of the sample. The KMO rates are evaluated as follows: 0.60 is moderate; 0.70 is good; 0.80 is very good; 0.90 is excellent [61]. The Barlett test result and the KMO value of this study are presented in Table 2: According to Table 2, the KMO value was 0.837 and the results of the Barlett test were found to be significant. These results showed that the sample size was very good and that the scale items were suitable for factor analysis. Finding a p value that is smaller than p<0.05 significance level showed that there was a sufficient correlation between the variables to perform factor analysis. In order to determine the validity of the structure of the scale, covariance included in the scale were used. The items with a load factor value of 0.45 and above were selected to be included in the process of analysis. According to [23], the value of 0.45 is sufficient at the minimum level. In this study, it was determined that the load factor values of 64 items in the scale ranged from 0.335 to 0.841. A total of 23 items with a load factor less than 0.500 were removed from the process of analysis and the remaining 41 items were evaluated. The load factor values of the scale consisting of the 41 items remaining are shown in Table 3: When the data in Table 3 were examined, the load factor values of 41 items were found to be appropriate. For this reason, analyses were performed to determine the subscales of the scale.

Factor Analysis
An eigenvalue and a line graph are used to show the number of correlations between the items and to determine the number of factors effectively [23]. The line graph obtained for the 41 items is shown in Figure 1: The line graph obtained by combining the eigenvalues of the items shows the rapid declines that could be seen between the items. The fracture points on the graph were also used to determine the number of factors [23]. In Figure 1, the rapid declines in the line graph start from item 6 and take a horizontal view. According to these values, it can be seen that there were six factors which were more significant and whose eigenvalue was higher than 1: The first factor accounted for 6.151% of the total variance, the second factor accounted for 4.077% of the total variance, the third factor accounted for 3.436% of the total variance, the fourth factor accounted for 4.344% of the total variance, the fifth factor accounted for 3.336% of the total variance and the sixth factor accounted for 3.144% of the total variance. The cumulative variance for the eigenvalues revealed 70.162% of the total variance. Tavşancıl [65] points out that values between 40% and 60% indicate that the factor structure is strong. In this case, it was seen that the total variance ratio of the scale developed is sufficient.
The correlation values were examined to determine the distribution of the factors of the scale items. The strongest correlation of variables with which factor was determined by varimax analysis. The factor correlation values of the obtained items are presented in Table 5:

Reliability Analysis
As a result of the reliability analysis for 41 items, the Cronbach's alpha reliability coefficient was calculated as 0.87. According to Büyüköztürk [23], scales that have a lower limit of reliability coefficients than 0.70 are considered sufficient. In this study, the reliability coefficient, which is determined on the scale, also indicated that there was an expected level of reliability. Other findings for reliability analyses are shown in Table 7. In order to determine the discrimination between the items on the scale, the difference between the items was examined and the ANOVA test was performed. A significant difference was obtained between the items. The findings are presented in Table 8.  The Hotelling t 2 statistic was used to determine the difficulty level of the scale items. The findings are shown in Table 9: According to the values in Table 9, the findings regarding the discrimination of the items were found to be significant at a significance level of p<0.05.

Result
This study aimed to develop a scale to determine the Turkish metalinguistic awareness in a sample of teacher candidates. According to the results of the validity and reliability analyses, it was determined that the scale developed was appropriate for determining teacher candidates' metalinguistic awareness. The sub-dimensions of this scale, which has a six-dimensional structure, are as follows: • phonological awareness, • morphological awareness, • semantic awareness, • syntactic awareness, • communicative awareness, • cultural awareness The general conclusion reached in the study was that the scale is a useful, valid and reliable tool for determining the Turkish metalinguistic awareness of teacher candidates.

Suggestions
Based on the results of this study, the following suggestions can be made: 1. New measurement tools for each sub-dimension of the scale could be developed. 2. New educational measurement tools could be developed by using research findings from disciplines such as psychology, sociology and linguistics to increase language use, language awareness and language ability.