The Survey for Nursing Management Behavior of Diabetes Liaison Nurses in Guangdong Province

Aim: To develop a specific nursing work assessment scale for diabetes liaison nurses to standardize their practice. Background: Currently, there are no validated tools to assess the special practices of diabetes liaison nurses; therefore, the efficacy of these nurses cannot be evaluated properly. Design: The Delphi method, using 12 nurse experts from 8 tertiary hospitals in China, was used to construct the diabetic liaison nurse work assessment scale based on the Hamric Model of Advanced Practice Nurses. The scale was then tested on 21 diabetic liaison nurses. Method: An expert panel participated in 2 rounds of Delphi exercises to develop the assessment scale for diabetic liaison nurses by consensus using a Likert scale to indicate whether a dimension or item should be included in the assessment scale. The initial scale was then revised based on the suggestions from the experts’ importance ratings. The final version of the scale was formed after 2 Delphi rounds and was tested for applicability in a preliminary test with 21 diabetic liaison nurses. A convenience and cluster sample of 426 diabetes liaison nurses from 25 hospitals in Guangdong province was invited to participate and complete the scale. Results: Experts reached consensus on a scale with 4 dimensions and 13 items. The item-level content validity index was 1 for 10 items and ≥ 0.75 for 3 items, and the scale-level content validity index was 0.96. The reliability was high; Cronbach's alpha was 0.851, and the retest reliability was 0.822. Construct validity was supported by the results of factor analysis; the Kaiser-Meyer-Olkin measure of sampling adequacy was 0.822, and Bartlett's Test of Sphericity was statistically significant (P < 0.0001). Four components with eigenvalues > 1 accounted for 69.93% of the total variance. The results of the Rotated Component Matrix demonstrated that the factor loading of 12 items was > 0.6. Conclusion: The content validity of the scale was high; The specific nursing work assessment scale for diabetes liaison nurses is a promising tool that can be used to evaluate the specific nursing practices of diabetes liaison nurses. Relevance to clinical practice: Diabetes liaison nurses have been appointed at many hospitals to improve diabetes management of hospitalized patients. The specific nursing work assessment scale for diabetes liaison nurses is a validated tool that can be used to assess the special practices of diabetes liaison nurses.


Introduction and Background
Large-scale randomized trials have demonstrated that tight glycaemic control in patients with type I and II diabetes reduces microvascular and macrovascular complications [1][2][3][4], indicating that glycaemic control is crucial for patients with diabetes. Diabetes specialist nurses play an important role in managing patients with diabetes to improve glycaemic control in primary care and hospital settings [5][6]; however, patients with diabetes are admitted to various departments in a hospital other than the department of endocrinology. A study that investigated 175 hospitalized patients with diabetes found that two-thirds of patients are admitted to non-endocrinology wards [7]. The workload of medical staff in the department of endocrinology does not allow them to be responsible for the health education and diabetes management of all hospitalized patients with diabetes. Additionally, many nurses assigned to non-endocrinology wards possess inadequate knowledge of current diabetes management standards [8]. To address this problem, diabetes liaison nurses with specialized knowledge of diabetes have been appointed at many hospitals worldwide to improve the diabetes management of hospitalized patients [9][10][11].
In China, in addition to routine nursing tasks in the base ward, diabetes liaison nurses work part-time to participate in training programmes and conferences concerning diabetes management; the nurses maintain the blood glucose monitoring records of patients in various departments, teach diabetes self-management to patients, educate other nursing staff, and regularly report diabetes management work to diabetes nurse specialists [12]. Xu et al [13 ]reported that within 1 year of the appointment of a diabetic liaison nurse 310 The Survey for Nursing Management Behavior of Diabetes Liaison Nurses in Guangdong Province team at their hospital, no diabetes-related nursing errors or complaints occurred in non-endocrinology departments, and patient compliance with diabetes regimens improved. However, although diabetes liaison nurses can reduce the burden of diabetes specialist nurses, they cannot significantly improve the quality of diabetes care due to poor administrative organization and management as well as inadequate diabetes-related knowledge and skills [12]. Currently, reports on the specialized nursing practices of diabetes liaison nurses are rare, and there are no validated tools to assess the special practices of diabetes liaison nurses. This study aimed to develop a scale with acceptable content validity that assesses the specialized nursing practices of diabetes liaison nurses, and a preliminary test was conducted to determine whether the application of the newly developed scale is feasible.

Theoretical Framework and Design of the Scale
This study was approved by the Institutional Review Board of the First Affiliated Hospital of Jinan University. The Hamric Model of Advanced Nursing Practice [14] has proposed that advanced practice nurses should have seven core competencies: direct clinical practice, expert coaching and advice, consultation, research skills, clinical and professional leadership, collaboration, and ethical decision-making. The specific nursing work assessment scale for diabetes liaison nurses was designed using the Hamric Model of Advanced Nursing Practice as a theoretical framework. Beginning with the job description of diabetes liaison nurses that was defined in a previous report [12], we completed a literature review and conducted an expert interview to propose the initial five dimensions (health education; hypoglycaemia management; blood glucose monitoring and insulin injection management; nurse training and patient referral). Hypoglycaemia management, blood glucose monitoring, and insulin injection management are direct clinical practice skills. The health education dimension may reflect consultation competencies, whereas the nursing training and patient referral dimension may represent the competencies of leadership, collaboration, and ethical decision-making. The initial scale has eighteen items and one open-ended question, as follows: items 1-3 belong to the health education dimension; items 4-6 belong to the hypoglycaemia management dimension; items 7-11 belong to the blood glucose monitoring and insulin injection management dimension; items 12-14 belong to the nurse education dimension; and items 15-18 belong to the patient referral dimension. The open-ended question queries comments or suggestions concerning diabetes liaison nurse management. A 5-point Likert scale was used to collect responses to each item from diabetes liaison nurses, with 5 representing "always," 4 representing "often," 3 representing "sometimes," 2 representing "occasionally," and 1 representing "never." A higher score indicates better specific nursing practices of diabetes liaison nurses.

Delphi Rounds
In the 1950s, the Douglas Company collaborated with the RAND Corporation in the United States to study a technique widely used in science and technology: the effective feedback method of expert opinion known as the Delphi Method, named after the Oracle of Delphi. The core concept of this method is to solicit opinions from experts in several iterations without attribution; when experts reach basic agreement, consulting cases can be ended. The traditional Delphi method generally involves 3 ~ 4 rounds of consultation; when using a modified Delphi method [15], the indexes for the content are provided to the experts at the beginning of the first round, which reduces the number of times experts contribute content feedback. Even two rounds can yield forecast results.
Based on the modified Delphi method and consensus building [15], two rounds of Delphi exercises were conducted to determine the dimensions and items that should be included in the scale. The Delphi rounds were conducted within an expert panel. The inclusion criteria of experts were as follows: > 10 years of related work experience and familiarity with the current status of diabetes nursing practices and specific nursing practices of diabetes liaison nurses; ≥ 5 years of related experience in nurse management; a title of primary nurse or higher and a minimum education level of a bachelor's degree; an interest in the development of a specific nursing work assessment scale for diabetes liaison nurses; and a willingness to participate in the Delphi rounds. Fourteen experts were included in the expert panel.
The scale evaluation questionnaire distributed to experts during the Delphi rounds consisted of two parts. The first part used a 5-point Likert scale to evaluate the opinions of experts on both the importance of the dimensions and the importance and relevance of the items, with 1 representing "very unimportant" or "very irrelevant" and 5 representing "very important" or "very relevant." The second part investigated the experts' basis for judging, the degree of influence, and the perceived degree of expertise related to each dimension and item. The options for the basis for judging included "practical experience," "theoretical analysis," "literature review and peer exchange," and "intuition." The degree of influence included three levels: "strong," "medium," and "small." The perceived degree of expertise was categorized into the following six levels: "very high," "high," "moderate-to-high," "moderate," "low," and "very low." The experts were permitted to provide comments on content and clarity of wording or to recommend any dimensions or items other than those that already existed. Before the first Delphi round, a pretest that included three experts was administered to evaluate the structure of the scale evaluation questionnaire as well as the conceptualization and content of items. The scale evaluation questionnaire was then revised and distributed to experts Universal Journal of Public Health 4(6): 309-318, 2016 311 along with the scale for the first Delphi round. Questionnaires were delivered to experts in person or via email. All of the experts were informed of the objective of the assessment scale, the theoretical framework, the Delphi method, and the criteria for selecting dimensions and items (an average importance rating > 3.50 and a coefficient of variation < 0.30). Experts were asked to return the scale evaluation questionnaire within 3 weeks. In the first Delphi round, the scale was revised according to both the dimension and item selecting criteria and experts' suggestions. If two or more experts had the same comments on an item, that item would be revised based on the experts' comments. In the second Delphi round, experts received the revised scale and the scale evaluation questionnaire. The scale was revised again based on the experts' responses to each dimension and item to form the final version of the specific nursing work assessment scale for diabetes liaison nurses. The content validity of the scale was supported on a theoretical basis, by expert consensus, and based on the results of Delphi rounds.

Preliminary Test
The feasibility of the final version of the specific nursing work assessment scale for diabetes liaison nurses was tested in a preliminary study. As noted in the literature [15], preliminary experiments require 20~30 samples; this study selected 21 liaison nurses. A convenience sample of diabetes liaison nurses at the Red Cross Hospital of Guangzhou was invited to participate and complete the scale. The test was repeated using the same participant group after 2 weeks.

Survey for Nursing Management Behavior of Diabetes Liaison Nurses in Guangdong province
A convenience and cluster sample of diabetes liaison nurses from 25 hospitals in Guangdong province was invited to participate and complete the scale; 426 ultimately participated. 426 cases of diabetes link nurses were cross-sectional surveyed by application of the management behavior scale. Investigator provides and recycles questionnaire on the spot, researcher herself (live in Guangzhou city) retrieves recycling questionnaire in Guangzhou, or investigators (who is not live in Guangzhou) sent the questionnaire back to the researcher.

Statistical Analysis
All analyses were performed using SPSS 13.0 (SPSS, Inc., Chicago, IL, USA). For the Delphi exercises, experts' questionnaire response rates were calculated. The average importance and relevance rating of experts to each dimension and item was also calculated. The experts' basis for judging and the perceived degree of expertise were scored (Tables 1 and 2). The degree of authoritativeness was calculated as the average score of the basis for judging and the perceived degree of expertise, which ranged between 0 and 1, with a larger number indicating a higher degree of authoritativeness of the expert panel on the subject [16]. Both the coefficient of variation and Kendall's coefficient of concordance [17] were calculated to indicate the consensus of experts. For each dimension and item, the coefficient of variation is the ratio of the standard deviation of the responses to the average rating of the responses. Kendall's coefficient of concordance ranges between 0 and 1; a larger Kendall's coefficient of concordance indicates better consensus of the experts. Two content validity indices, the item-level CVI (I-CVI) and the scale-level CVI (S-CVI), were calculated. The rating scale measuring responses to the relevance of the item to the dimension was a five-point scale: 0 = "not relevant;" 1 = "weakly relevant;" 2= "somewhat relevant;" 3 = "quite relevant;" and 4 = "highly relevant." I-CVI represented the number of experts who had a rating of 3 or 4 divided by the total number of experts (18Polit et al. 2007). S-CVI was the average of all I-CVIs [18].

Expert Panel
The fourteen experts were associated with 8 tertiary hospitals in Guangdong Province, and 12 experts participated in Delphi rounds and responded to questionnaires. Two experts did not respond within 1 month of the first Delphi round and were thus excluded. The ages of the 12 experts ranged from 35 to 48 years, with an average of 41.2 years. Their work experience ranged from 14-30 years, with an average of 22.1 years. The characteristics of the experts are listed in Table 3.

Delphi Rounds
During the 2 Delphi rounds, the expert response rates to questionnaires were 86% and 100%, respectively. Furthermore, 83% and 33% of the experts made comments or suggestions in the first and second rounds of the Delphi exercises, respectively.
The average scores on the "basis for judging" in the first and second rounds of the Delphi exercises were 0.893 (range = 0.870 ～ 0.910) and 0.889 (range = 0.858 ～ 0.908), respectively. The average scores of the "perceived degree of expertise" were 0.858 (range = 0.840～0.880) and 0.865 (range = 0.850 ～ 0.883) in the first and second rounds, respectively. The degree of authoritativeness was 0.876 (range = 0.855～0.890) in the first round and 0.877 (range = 0.854～0.892) in the second round.
In the first round of the Delphi exercises, the importance ratings of the experts on the 5 dimensions were 3.08 to 4.25, and the importance and relevance ratings of the 18 items were 2.83 to 4.50 and 2.75 to 4.42, respectively. The dimension "hypoglycaemia management" received an importance rating < 3.50 and a coefficient of variation > 0.30; this dimension was thus deleted. Four experts suggested combining some content of the "hypoglycaemia management" dimension with the "health education" dimension and recommended "patient management on the wards" as the name of that dimension. According to the experts' suggestions, the "nurse training" dimension was renamed "special guidance for nurses on the wards," and the "patient referral" dimension was renamed "exchange with a diabetes nurse specialist." Items C3 (health education for patients with diabetes covering the entire content of "five carriages") and C6 (regularly review patient's care record during hypoglycaemic events) had a coefficient of variation > 0.30; thus, they were deleted. According to the experts' comments, item C10 (you are attentive to levels of insulin-related knowledge in patients undergoing insulin injection), C11 (you are attentive to meal preparation and patient's eating before and after insulin injection), and C2 (you are attentive to patients' levels of diabetes knowledge after health education) had repetitive content; therefore, items C10 and C11 were deleted. Three experts suggested deleting item C15 (you can refer patients with diabetes in your ward to the diabetes education program) and putting items C16-C18 in the "exchange with diabetes nurse specialist" dimension, and their suggestions were implemented. Consequently, the revised scale had 4 dimensions and 13 items, and it was used for the second round of the Delphi exercises.
In the second round of the Delphi exercises, the importance ratings of experts on the 4 dimensions were 3.67-4.42, and the importance and relevance ratings of the 13 items were 4.20-4.60 and 4.09-4.73, respectively. Two experts suggested revising the name of the "exchange with a diabetes nurse specialist" dimension to "teamwork and feedback." The wording of items C9 and C12 was revised according to the experts' suggestions. At the end of the second Delphi round, the final version of the specific nursing work assessment scale for diabetes liaison nurses was formed and included 4 dimensions and 13 close-ended items (Appendix 1).

Preliminary Test
All diabetes liaison nurses included in the preliminary test (n = 21) were female, and their ages ranged from 23 to 48 years. The median age was 37 years. Their work experience ranged from 1 to 27 years, and the median amount of work experience was 16 years. The technical title was nurse for 2 participants, primary nurse for 12 participants, and nurse-in-charge for 7 participants. The educational level was professional training school for 1 participant, junior college for 11 participants, and bachelor's degree for 9 participants.

Reliability of the Scale
Cronbach's alpha [19] was computed to indicate the reliability of the scale. Cronbach's alpha > 0.80 and > 0.90 indicate very good and excellent reliability, respectively [20]. The retest reliability was also calculated [21]. The results were interpreted using the following criteria: < 0.2 is poor; 0.21-040 is fair; 0.41-0.60 is moderate; 0.61-0.80 is good; and 0.81-1 is perfect [22].
During the preliminary test, Cronbach's alpha was 0.851, indicating a very good reliability of the scale. Cronbach's alpha values of each dimension were 0.710, 0.797, 0.853, and 0.851, respectively. The retest reliability was 0.822, which was nearly perfect. The KMO was 0.822, and Bartlett's Test of Sphericity was statistically significant (P < 0.0001). The total variance explained is listed in Table 5. Four components with eigenvalues > 1 were rotated, and they accounted for 69.93% of the total variance. The results of the Rotated Component Matrix demonstrated that the factor loadings of 12 items were > 0.6, with the exception of item C4, for which the factor loading was 0.380. The first component reflects special guidance for nurses on the wards, the second component reflects teamwork and feedback, the third component reflects blood glucose monitoring and insulin injection management, and the fourth component reflects patient management on the wards.

Validity of the Scale
In the first and second rounds of the Delphi exercises, the average coefficient of variation for the importance rating of items was 0.18 and 0.14, respectively, indicating improved consensus of the experts. The coefficient of variation for the importance rating of each item is listed in Table 4. Kendall's coefficients of concordance were 0.25 and 0.34 in the two rounds of the Delphi exercises, respectively, indicating improved consensus of experts in the second round. The I-CVIs of items C3, C4, and C13 were 0.75, 0.83, and 0.92, respectively. The I-CVIs of the remaining items were 1 and the S-CVI was 0.96.
The Kaiser-Meyer-Olkin measure of sampling adequacy (KMO), Bartlett's Test of Sphericity, and factor analysis were performed. A KMO ≥ 0.50 indicates the appropriateness of factor analysis [23]. For Bartlett's Test of Sphericity, the null hypothesis that there is no correlation among the variables in the correlation matrix should be rejected to indicate that the factor model is appropriate [24].

Confirmatory Factor Analysis
A convenience and cluster sample of diabetes liaison nurses from 25 hospitals in Guangdong province was invited to participate and complete the scale; 426 ultimately participated. Using the Amos 17.0 program, confirmatory factor analysis was performed on the scale. The maximum likelihood estimation was used to test the degree of fit of the four factors. In the practical application of the covariance model [25][26], the common index usually comprises the Comparative Fit Index (CFI) and Tacker-Lewis index (TLI) as well as the root mean square error of approximation (RSMEA); among them, CFI and TLI values above 0.95 indicate goodness of fit in the model. The closer the goodness of fit index (GFI) and adjusted goodness of fit index (AGFI ) are to 1, the better the fitness of the model is, at least when they are greater than 0.8. Each item on the first-order load factor was > 0.4, except items 3 and 4, for which the reliability coefficients were less than 0.30. The rest of the items' indexes of reliability ranges were between 0.45 and 0.80, which demonstrated good reliability and validity of the model structure. When Chi-square values/degrees of freedom are close to 2, the results are significant. However, when the sample size is large, a ratio expanding to 5 is also acceptable. The confirmatory factor analysis results of this study were as follows: chi-square value 136.79, degrees of freedom59, and ratio 2.319 (P = 0.001). CFI 0.966, TLI 0.955, GFI 0.953, AGFI 0.928, and RMSEA 0.056 demonstrate that both the assumed theoretical model and the observed data exhibited a high degree of fit. Figure 1 is a structural equation model based on the results.

Discussion
In this study, a specific nursing work assessment scale for diabetes liaison nurses was constructed through two rounds of Delphi exercises. At the end of the two Delphi rounds, the final version of the scale contained 4 domains and 13 items. The four domains that characterize the specific nursing practices of diabetes liaison nurses are "patient management on the wards," "blood glucose monitoring and insulin injection management," "special guidance for nurses on the wards," and "teamwork and feedback." Both the reliability and validity of the scale were further tested in a preliminary study.
The strength of this study was that the scale was developed based on a theoretical framework [14]. Another particular strength of this study was that we set strict inclusion criteria for the expert panel. The expert panel consisted of 12 experts specializing in nursing management, clinical care, and nursing health education and the experts had > 22 years of average work experience. All experts were familiar with the work status of diabetes liaison nurses, and their average perceived degree of expertise was very high in the second Delphi round. The degree of authoritativeness was high in the 2 rounds of Delphi exercises (0.876 and 0.877, respectively), which was strong evidence that our expert panel was authoritative [16]. Experts had high response rates to questionnaires (86% in the first Delphi round and 100% in the second Delphi round), indicating a high representativeness of the study results [27].
In a Delphi round, a coefficient of variation < 0.5 indicates good consensus of experts [28]. At the end of the second Delphi round, the coefficients of variation of all items were < 0.25. Kendall's coefficient of concordance increased in the second round, indicating improved consensus of experts. An importance/relevance rating of an item > 3.0 indicates high item importance and relevance [29]. In our study, the importance and relevance ratings of all items were high (> 4.0) at the end of the second round, suggesting great item importance and relevance.
Content validity refers to the extent to which an instrument measures a given construct [30]. In this study, the high content validity of the scale was supported by the theoretical basis of the scale, expert consensus, and the results of Delphi rounds, such as importance/relevance ratings and I-and S-CVIs. In particular, an I-CVI > 0.78 and an S-CVI > 0.9 indicate high content validity [18,29]. All items of our scale had an I-CVI ranging from 0.83 to 1.0 with the exception of only 1 item (item C3), for which the I-CVI was 0.75 and very close to 0.78. The S-CVI was 0.96. All of these results suggest that our scale had a very high content validity.
Generally, a Cronbach's alpha > 0.85 indicates adequate internal consistency and high reliability of the instrument [20]. In such a circumstance, the instrument will be suitable for group and individual comparisons [20]. When Cronbach's alpha is between 0.7 and 0.85, the instrument will only be suitable for group comparisons [20]. The Cronbach's alpha of our scale was 0.851, suggesting adequate internal consistency. The Cronbach's alpha of all four dimensions was > 0.7, and the retest reliability was close to perfect, indicating that the scale would be appropriate for group comparisons.
Factor analysis is a common method to evaluate the construct validity of an instrument [31]. In this study, the results of both KMO and Bartlett's Test of Sphericity demonstrated that the factor analysis was appropriate for the scale. Further factor analysis revealed that 4 components with eigenvalues > 1 accounted for 69.93% of the total variance. Guadagnoli and Velicer [32] proposed that a factor is reliable, regardless of the sample size, if it has four or more loadings > 0.6. Our results of the Rotated Component Matrix demonstrated that the factor loadings in 12 of 13 items were > 0.6, which was very high. In general, our results suggest that the scale has high reliability and validity, and the scale is a promising and simple self-evaluation tool for assessing the specific nursing work of diabetes liaison nurses.
There were some limitations to this study. A disadvantage of the Delphi technique is that the expert consensus is based on the experts' current knowledge level. With time, knowledge will substantially expand in depth and complexity. Therefore, this scale has an inevitable limitation. A convenience sample was included in the preliminary test, which was another limitation of this study. Currently, there is no authoritative scale for assessing the specific nursing work of diabetes liaison nurses. Hence, the criterion validity of our scale cannot be evaluated. In the field of nursing care evaluation, peer review has been identified as a valuable approach for ensuring the accountability of evaluation results and fostering professional growth [33][34]. Additionally, the combination of qualitative and quantitative approaches has been considered the complete method for investigating a specific phenomenon in nursing [35]. Our scale is only a qualitative self-evaluation tool; thus, it has limitations. In the future, quantitative evaluation items and peer assessment content can be added to make the scale more complete and valid.

Conclusions
In the present study, we successfully developed a specific nursing work assessment scale for diabetes liaison nurses using the Delphi technique. The scale is a qualitative self-evaluation tool. Experts reached a consensus after two rounds of Delphi exercises, and the results of the Delphi rounds and the preliminary test demonstrated high content validity, construct validity, and reliability of the scale. Therefore, the scale can be used to evaluate the specific nursing practices of diabetes liaison nurses.

Relevance to Clinical Practice
In patients with diabetes were increasing year by year, so