Construction of Validity and Reliability of an Observational Instrument to Assess the Technical Execution in Lead Climbing

Background: The speed world record (WR) performance is an achievement achieved by athletes in the competitive rock-climbing category, where the athlete must complete the climbing route on a standard artificial climbing wall with a height of 15m in the shortest possible time. Purpose: This study aimed to construct and validate an observational instrument to assess technical execution in lead climbing. The instrument developed can be used to assess progress to achieve the important aspects of climbing movement methods. Design / methodology / approach: Importantly, the construction of its validity and reliability was conducted in four stages. The first stage was a literature review on the construction of the observational instrument. In the second stage, the instrument’s construction was established. The content validity coefficient was determined in the third stage using Aiken's V and 7 experts with various qualifications, including the level of understanding, appropriateness of the words, and the relevance of the statement with a grading scale of 0-10. The instrument’s internal validity was calculated using 20 climbers and was quantitatively tested using the product-moment correlation. In the fourth stage, instrument reliability was determined using Cronbach's Alpha reliability test. Results showed that the observational instrument has content validity with optimal internal value as well as satisfactory reliability. Conclusion: Observational instruments were valid and reliable; therefore, they can be used to assess technical execution for novice climbers in lead climbing. It allows instructors, coaches, and teachers to assess the technical execution of novice climbers specifically in lead climbing.

Abstract Background: The speed world record (WR) performance is an achievement achieved by athletes in the competitive rock-climbing category, where the athlete must complete the climbing route on a standard artificial climbing wall with a height of 15m in the shortest possible time. Purpose: This study aimed to construct and validate an observational instrument to assess technical execution in lead climbing. The instrument developed can be used to assess progress to achieve the important aspects of climbing movement methods. Design / methodology / approach: Importantly, the construction of its validity and reliability was conducted in four stages. The first stage was a literature review on the construction of the observational instrument. In the second stage, the instrument's construction was established. The content validity coefficient was determined in the third stage using Aiken's V and 7 experts with various qualifications, including the level of understanding, appropriateness of the words, and the relevance of the statement with a grading scale of 0-10. The instrument's internal validity was calculated using 20 climbers and was quantitatively tested using the product-moment correlation. In the fourth stage, instrument reliability was determined using Cronbach's Alpha reliability test. Results showed that the observational instrument has content validity with optimal internal value as well as satisfactory reliability. Conclusion: Observational instruments were valid and reliable; therefore, they can be used to assess technical execution for novice climbers in lead climbing. It allows

Introduction
Within few recent decades of years, there have been a sizeable number of people exhibiting extreme interests in physical sports, like jogging, running, climbing, and so forth with gulf discrepancy in preference. Sport climbing was classified among the most recurring sports opted for by many [1]. By using hands and footholds as the paramount tools in the climbing practice, the main purpose of the sport climbing is to climb on a climbing wall with additional and indispensable equipment serving merely for protection [2]. In the wake of sport climbing, some distinctive climbing types are acknowledged; that is speed climbing; aiming to reach the higher top of the climbing lane in the shortest possible amount of time. While the lead climbing broadly envisaging climbers to get to the top of the difficult path with rope protection, in bouldering activity climbers are required to escalading large rocks without bearing rope protection [2]. Additionally, sport climbing is also executed outdoors with the aid of rigid natural rocks with or without rope protection usage [3]. Explicitly, rock climbing is another type of sport or activity of climbing rock faces, especially with the aid of ropes and special materials designed accordingly. According to [3] rock climbing is comprised of numerous distinctive disciplines, including, but not delineated to top-rope climbing, traditional climbing, sport climbing, and bouldering. If taken as a physical education course, rock climbing is considered to be an attractive physical activity for college students since it is one of the competitive sports events that serve the needs or interests of these scholars [4]. Despite being a leisure, physical activity especially for youths has also become an international competitive sport [5]. For instance, IOC as the International Olympic Committee in his 129 th plenary session in Brazil, Rio De Janeiro, has officially notified that competitive climbing to be accepted as an official game in 2020 Tokyo Olympic [6]. In short, if compared to other standard sports like soccer, basketball, Ping-Pong or table tennis, and others, all disciplines of rock climbing are different from one another in terms of activities performed by any engaged individuals especially in which youth traditionally participate.
As such, rock climbing has been very popular among children, adolescents, and adults as a recreational and competitive sport [7]- [11]. As mentioned in the preceding section, It mostly leads to an increase in related disciplines, such as sport climbing, mountaineering, bouldering, big wall climbing, traditional, ice, and mixed climbing, etc. [12]. Physical and physiological factors, such as endurance and strength, determine optimal climbing performance, starting from the base level to the elite. However, optimum performance is achieved only in case one train continuously and with a lot of focus [7], [13], [14]. Besides, other factors such as psychological and mental strategy also affect the performance of rock climbing [15], [16]. Evidence shows that rock climbing performance is generally determined by the physical aspect [17], [6], in which endurance and muscle strength are influenced [18]- [23]. However, as a part of physical performance, other aspects were highlighted that prove the need to improve the performance of rock climbing (physiology, psychology, and strategy) [15]. Additionally, rock climbing performance is determined by the food consumed by athletes. Some focus on macronutrients that need to be adequate, such as protein and fat (energy sources mostly for the young generation), as well as other micronutrients in form of minerals and vitamins [24]. Conversely, the Body mass index (henceforth BMI) has found no significant impact on climbing ability [25]. Some studies have focused on the technical execution of climbing ability level [26], [27]. Technical execution focuses on the extremities of the body's limbs, including the arms and legs, using the key aspects of the movement with stages during the climb [28]. Recently a similar study was conducted on the form of instruments assessing the technical execution with top-roping [29]. Results showed that instruments designed present optimal levels of reliability and validity even though some experts suggested removing three aspects of them, namely the displacement of the hips, interaction segments between hands and feet, and the crossed force application.
Lead climbing is attributed to multiple combinations of moves and is accomplished by a climber on higher walls of over 10m [30]. Some common factors are involved in boulder and lead climbing routes such as hold type (size and shape) and the gradient of the wall that determine the adversities on the path [31], [32]. To use sufficiently the force generated from the back of the climber, shoulders, and limb muscles to the holding spots during moves, strength generated in fingers are required, thereby finger flexor strength is staple factors for expecting a good performance in climbing or during climbing in specific tests [33].
Even so, the increasing participation of rock climbing in the community is gradually gaining popularity either as a recreational sport or an accomplishment. Apart from that, given the limitations of research related to instruments to assess techniques in rock climbing, namely theoretically and empirically, lead climbing is considered to be an issue to deal with through accurate instrumentations. Therefore, this research aims to create an instrument that can measure the technique of executing rock climbing paths in the lead climbing category. This study will be useful for sports coaches and teachers to assess climbing techniques for both athletes and students. By determining the validity and reliability of this product, it can further be used by coaches, instructors, and physical education teachers in assessing the technical execution of lead climbing and its relation to the achievement of the main aspects of climbing movement.
Literature such as handbooks and also other methods for novice climbers, as well as the theories on basic techniques for the climbing movement, were used as the basis of suggesting and constructing the instruments [34]- [41]. The literature selected shows that the technique involves the completion of the wall climbing, arm position, weight distribution between the arms and legs, the position of the grip and foot placement, as well as body position closer to wall climbing.

Materials and Methods
To begin with, this study used 7 expert judges with different levels of expertise. In respect to this, they were then divided into three groups based on their educational fields, profile, level, and experience. For instance, among the 7 individuals consulting in instrument validation, three of them have a degree in physical education with at least five years of experience in teaching and training sports and recreation, simply known as climbing. One other was a chief coach of the national rock climbing in Indonesia. The rest three individuals have degrees in sports science with 7 years of career as instructors, coaching, and route setters of climbing sport. All these 7 individuals were pointed out and became the source of information since they played a crucial role through the agreement and consensus process during the content validation of the instruments. However, these experts selected have no different impacts on the results expected but they put their hands together during instrument validation to help strengthen the accuracy of the items in order to measure what is needed to be measured and correctly. After the instrument's appraisal, they were then applied to a sample of 20 novice climbing athletes willing to be participants. Participant consent in the form of a letter was also signed by research subjects.
To our better understanding, we acknowledge that there are different types of instrument validation, like content validity, face validity, construct validity, and so forth [42]. Validity is explained by [29] as the degree or the extent to which a test measures what is presumed to measure. In this study, content validity was tested through the agreement and consensus means among seven expert judges. They are in charge of analyzing qualitatively the degree of instrument understanding, grammatical arrangement of words in items, and the relevance of questions constructed. As an instrument is concerned, the observational instrument employed was expected to enable assessing the technical execution in lead climbing for the novice climber. But, the achievement of such assessment depends merely upon the key features of the climbing technique. On the side of the observer, he/she served as an indicator whose main role is to decide with a yes or no decision. The ''Yes'' approval means that the beginning climbers married up with the conditions defined for each staple aspect. Whereas with the ''No'' decision explains that the subject failed to meet the conditions recapped for each key aspect. Then, the research in hands employed main aspects assessed by three main levels scale (from 1 up to 3 level or 1, 2, 3 levels).
On the aspect of procedures, the study was designed to be conducted in 4 stages [42]. To this end, the first stage aims to suggest and design a proposal during the elaboration of the observational instrument. In connection with this stage, emphasis was put on revealing evidence from diverse databases in form of assessing some articles chiefly selected from international journals with credibility reputation/indexation. Some of which were with the quartile ranks of Q4 Scopus indexed journals, Sports Discuss, Google Scholar, Web of Science (shortened WOS) the ISI Thomson Reuters, Latindex, Scopus, Scielo, Dialnet. While selecting articles and other relevant materials for the study, some keywords have guided and led us to the successful search; namely climbing activity, performance analysis, physical fitness, assessment techniques in sport, and specifically technical execution in lead climbing. Articles mostly published from 2000 up to 2019 were included in this study but those adhering to the following criteria, 1) should have examined the effect of rock-climbing interventions, 2) research subjects as young students frequently involved in rock climbing, 3) studies that have original and 4) study with complete information. Studies were excluded if they have been repeatedly published by the same author(s) or if the data are incomplete. After synthesizing all materials used, it was viewed that an extensive number of authors shortly defined climbing movements based on four dimensions; namely arm position, weight distribution between limbs (lower and upper limbs), the link resides between the vertical axis and the wall of the body, and the feet location toward the grip [29]. Given these four climbing movements latterly highlighted, the rock-climbing movement in this study was also analyzed following the previous studies by focusing on the four positions mostly acknowledged in climbing movements and as well as installing a safety position [32], [34], [36]. While developing the instrument, consultation and discourse upon the crude instruments were executed with experts respectively. Each expert gave scores on the draft instrument based on his/her expertise through a validation sheet administrated [42]. To seek expert agreement, the Aikens V Formula was used to analyze the data [43].
The second stage was the design concept of the observational instrument. The role of experts was to evaluate different aspects of the observational instrument. In the third stage, content validity was established by 7 experts with the Delphi technique [43]- [45] through mix method approach (qualitative and quantitative) [46]. To this, all aspects observed and criticized by the expert judges were removed or rectified. This was meant to seek the agreement of experts using the Delphi technique [47], [48]. The content validity was calculated using Aikens's V coefficient [49], while product moments are used to test the internal validity. During the fourth stage, the reliability of the observational instrument was calculated using Cronbach's Alpha statistical methods [50], with the Statistical Package for Social Sciences (SPSS, Version 23) software. Then, after going through all these stages audited above, the researcher came up with the observational instrument which is ready to be employed.
For statistical analysis, the analysis data of content validity was calculated using Aikens's V coefficient [49]. Product moment statistical methods were used to determine the internal validity. Additionally, the reliability of the instrument was calculated through Cronbach's Alpha statistical methods [50]. All of the data were calculated using Microsoft Excel and Statistical Package for Social Sciences (SPSS, version 23) software.

Results
As stated in the preceding section, this study was governed by the review of literature as the assessment of International journals relevant and comprehensive, like Q4 Scopus journal, Sport Discus, Google Scholar, Web of Science (WOS) de ISI Thomson Reuters, Latindex, Scopus, Scielo, Dialnet, and Google Books). Rock climbing movement was grouped into various positions, including the arm, the weight distribution between the arm and leg, foot position on the grip, posture with rock climbing board, install a safety position, and the position falls [35], [37]- [39]. The result of this stage in table 3 below (appendix 1).

a) Content Validity
The result of the assessment by experts (n=7) based on Aiken's formula is shown in Table 1 below.

b) Internal validity
The result of the applied instrument by novice climber (n=20) was further analyzed using product-moment correlation outcome is shown in Table 4 below (appendix 2).

c) Result of Reliability Inter Item
Reliability test results to assess the observational instrument using Cronbach's Alpha are shown in Table 2 below. Based on the reliability test results in Table 3 above, the value of Cronbach's Alpha on the observation instrument was 0.71. Therefore, the instrument ratings have sufficient inter-rater reliability.

Discussion
We recall here that the main aim of this present study describes the construction of validity and reliability of an observational instrument to assess the technical execution in lead climbing. The reason behind this is to provide a hint in form of an instrument that can be used to assess progress to achieve the important aspects of climbing movement methods in young athletes. As retrieved from different resources, it has been found that most of authors in physical fitness as rock climbing indeed, divided the climbing movement into four staple positions; namely limbs position, weight distribution between limbs (lower and upper limbs), feet location to the grip, body location to the vertical axis and the wall [51]. However, [29] manifested his dissatisfaction by saying that climbing does not allow isolation/specificity of a set of muscles, but instead, it mobilizes large muscular groups.
The observational instrument was examined in several stages and required expert and pilot study [43]- [45], [47], [48]. This research constructed an observational instrument to assess technical execution in lead climbing. In this way, a valid and reliable instrument has been constructed for this sport. A total number of 7 expert judges have been the source of information by participating in the design and validation of the observational instrument. In this respect, it should be noticed that the particularity of the subject matter and experience restricts the existence of the experts in this area of knowledge. The number of expert judges participating in this study was the same or higher than in other studies conducted in singles physical sports such as Judo, table tennis, or beach volleyball [52], [53], [54]. The inclusion criteria for expert judges as having high qualifications in sport fields, such as awarded with a master's degree in physical activity and sport sciences, being a coach with a diploma, and some years of training experience have been considered. This level of experience and expertise has served the theoretical along with their rock climbing experience, providing salient information to the researchers [55], [56]. Overall, their quantitative and qualitative contribution was crucial for the suggestion and construction of the observational tool. With this respect, thanks to them, the instrument has been approved and proceeded to be used. As such, the instrument can be used by instructors and physical education teachers to assess technical execution in lead climbing. Additionally, these instruments focus on assessing the novice climber in lead climbing. In term of statistics, the selected variables provide an appropriated content validity to the observational instrument, being valid to assess the technical execution in lead climbing, since the involvement of Aiken's V formula in calculations, its outputs showed a positive evaluation of the different variables in both sides tested (pertinence and definition) [57].
According to [58] who stated that the validity of any kind of instrument developed, can be accepted if the critical level calculated is equal to or over 0.50 in value. However, in some cases, most of the values of the variables are found higher than the critical level of 0.70 [59]. Hence, the content and internal, as well as the inter-rater reliability were tested. The results showed that using the formula Aiken's V, the observational instrument has an optimal validity value not less than 0.90. Furthermore, the internal validity test using product-moment correlation also indicates that these instruments have a high validity of each item. The results also show a sufficient level of reliability, which is the value of Cronbach's Alpha (7.10). This means that the instrument can be used to observe technical execution in lead climbing for the novice climber. Overall, it can be concluded that the results yielded especially based on both the quantitative and qualitative inputs of experts' judges, the value of Aiken's and Cronbach's Alpha, reflect that the design or construction of validity and reliability of an observational instrument to assess the technical execution in lead climbing have indicators of content validity and reliability since their values proposed by experts are all above the exact critical level.

Conclusion
This study concludes that the observational instrument assessing technical execution in lead climbing has a high value of content and internal validity, apart from sufficient reliability. This means the observational instrument can be considered as a useful hint, which could be used in similar circumstances by instructors or teachers to novice climbers. As limitation reported, sample sizes should be greater, as said by Louis Cohen, Lawrence Manion & Keith Morrison (2011) in their book "Research Methods in Education" that the greater is the sample the better. Yet, it could be also an additional asset if this instrument developed is applied on other surfaces like natural rock distribution, and to construct an instrument set that assesses others skills like the leading climber's skills. 408 Construction of Validity and Reliability of an Observational Instrument to Assess the Technical Execution in Lead Climbing Appendix 1 Table 3. An overview of the key aspects in the design of the instrument used to assess technical execution in lead climbing.

The observational instrument to assess the technical execution of lead climbing
Key Aspects Explanation 1) Doing orientation on the climbing route The climbers giving attention and learn how to execute the route of lead climbing during orientation time 2) Using the three supporting points The climbers use at least 3 supporting points (griping or footing) during the climb.
(Yes) The climbers use at least 3 supporting points.
(No) The climber does not use at least 3 supporting points. They only use hands or and one foot or remain on two hands and two feet.

3) Balanced position
The climbers can adjust the balance. The center of gravity is located on two legs or one leg.
(Yes) The climber can adjust the balance. The center of gravity is located on two legs or one leg.
(No) The climber cannot control the balance. The center of gravity outside the area of the foot.

4) The movement of arms and legs
The climbers use the arm as a counterweight to the body during climbing. Positioning the feet to hold The burden of weight and make it function during climbing instead of the arm.
(Yes) The climber uses the arm as a counterweight. Foot preferably be the focus of weight.
(No) The climber more often uses the arm instead of the foot. The foot is not the focus of weight during climbing.

5) Fluency in climbing
The climbers climb smoothly without stopping at one point in time.
(Yes) Climbing is smooth. No stopping at one point in time.
(No) Climbing is irregular. The Climber spent a lot of time on the same handholds and footholds twice.

6) Orient the supporting points
Climbers try to find the right grip and selecting before the next move during the climb.
(Yes) The climber always looks and selects the proper grip before the next move. The climber will continue to see further grip to complete the climbing route.
(No) The climber does not look and select the proper grip before the next move. The climber only touchpoints the grip without completing the route climbing.

7) Stretch out his arms, at the rest position
The Climbers attempt to stretch their hands for climbing or at rest when climbing.
(Yes) The climber tries to stretch out their arms during climbing and resting.
(No) The climber does not try to stretch their arms during climbing and resting.

8) Installation of anchor
The climbers put up the rope on anchor with one hand stretched.
(Yes) The climber put up the rope in safety with one hand stretched.
(No) The climber does not put up the rope in safety with one hand stretched.

9) Grip technique
The observer looks at the point held and indicated the level of the techniques used in the image below.

10) Supporting footpoints
The observer examines the use of the climber's feet to increase height, indicating the level of the techniques used in the image below.

11) Instructions climbing
Climbers communicate with the belayer regularly during climbing using the words "take" or"on belayer!" (Yes) Climber communicates with the belayer regularly using the words: "take" or "on belay!" (No) The climber does not communicate with the belayer regularly using the words "take" or "on belay!" Legend: Observer indicates "yes" or "no" when a climber found the conditions in each aspect. A key aspect is reached when it has at least 70% of the above items.