Closing the Gap: An Evaluation of Community-Based Projects to Reduce Diabetes Disparities among Minority and Underrepresented Groups

Diabetes poses severe disease and economic burden in the United States of America as well as in Florida State. The study aimed at evaluating eight community-based diabetes disparities projects funded by FDOH ‘Closing the Gap’ initiatives. This three-phase evaluation included evaluability assessment, process, immediate outcome and impact evaluation using qualitative assessment, case study approach, survey, document analysis, and face-to-face interviews. In 2006-2007, 2007-2008, and 2008-2009 total 3,764; 3,280; 1,691 individuals participated in the programs respectively. In 2006-07 to 2007-08, the proportion of participants referred for the wellness program decreased 8.64%, and participants enrolled in the wellness program decreased 3.4%. On the other hand, DSME and wellness program referrals, enrolment, and completions increased significantly from the 2007-08 to 2008-09 program year (12.6+8.64=21%, 3.4+36.1=39.5%, and 31.8-2.2=29.6%, respectively). The largest increase happened in the DSME with a change of (51.2-8.7= 42.5%). From the 2006-2007 to 2007-2008 funding year, with the largest increase in pre and post-test score, HbA1c, eye-exam etc. participation and in 2008-2009 participation of females in the diabetes program increased significantly. Overall, the diabetes program successfully recruited minority populations and had immediate positive outcomes and impact on minorities, hard-to-reach populations, and women which are good indicators of the program’s success.


Introduction
Diabetes poses severe disease burden in the United States According to the Center for Disease Control (CDC), an estimated 48 million people are expected to have diagnosed diabetes in 2050 2 . In 2012, a total of 245 billion costs incurred to diagnosed diabetes, out of which 176 billion direct medical costs and 69 billion in reduced productivity 3 . After adjusting for population age and sex differences, average medical expenditures among people with diagnosed diabetes were 2.3 times higher than what expenditures would be in the absence of diabetes 3 .
Diabetes is a systemic disease and affects vital systems of the body and considered as major cause of heart disease, stroke, kidney failure, blindness, leg and foot amputation unrelated to injury. In 2004, heart disease was noted on 68% of diabetes-related death certificates among people aged 65 years or older and stroke was noted on 16% of diabetes-related death certificates among people aged 65 years or older. Adults with diabetes have heart disease death rates about 2 to 4 times higher than adults without diabetes. The risk for stroke is 2 to 4 times higher among people with diabetes. From 1997 to 2011, the number of people aged 35 years or older with diabetes and with self-reported heart disease or stroke increased from 4.2 million to 7.6 million. In 2011, among people with diabetes aged 35 years and older and with self-reported heart disease or stroke, 5.0 million reported having coronary heart disease, 3.7 million reported having other heart disease or condition, and 2.1 million reported having stroke 2 . These national trends are also reflected in the population in the state of Florida.
Diabetes mellitus was the sixth leading cause of death in Florida in 2008 4  In 2007, there were 5,092 deaths from diabetes, representing 3% of the total deaths in Florida. Also, approximately 2 million Floridians (9.5% of the adult population) were diagnosed with diabetes 4 . In 2008, the prevalence of diabetes was highest among non-Hispanic Blacks (14.6%) followed by non-Hispanic Whites (9.0%) and Hispanics (7.9%).
Diabetes and its complications occur among Floridians of all ages and racial/ethnic groups, but those who are 65 years of age or older, African-Americans, or Hispanics are disproportionately affected by the disease. Obesity and type 2 diabetes are increasing in children; from 1994 to 1998, type II diabetes increased from 9% to 19% among children (based on a study of data from three Florida university-based diabetes centers). Primary, secondary, and tertiary prevention of diabetes has paramount importance in reducing the overall burden of disease, as well as the burden associated with related social and economic issues. However, most of the community-based diabetes prevention program did not collect data at individual level and it is hard to establish evaluation measures based on aggregated data.
In 2000, the "Reducing Racial and Ethnic Health Disparities: Closing the Gap Act" was enacted in an effort to mobilize underserved communities in Florida to address and eliminate health outcome disparities and promote disease prevention activities. After enactment of this Closing the Gap Act, this program has been providing underserved populations such as African-American, Black Caribbean, Hispanic, Asian and Haitian men, women, infants, and children with much needed preventive care and health services. In 2006-2009 funding years, total eight community based programs were funded (Table 1). This current study aimed at conducting an evaluability assessment and evaluation of Closing the Gap to establish the indicators for immediate impact and outcome of the project based on data collected by the projects, which will lead to establishing the indicators enacted by Florida Statute in reducing racial and ethnic health disparities 6 . The study also aimed at developing a recommendation based on current findings as well as evidence-based practices in diabetes prevention and control.

Materials and Methods
The evaluation was done in three phases; first phase involved qualitative assessment and a case study approach to evaluate each project's goals, objectives, plans, resources, achievements, needs and other topics that are contextual to each project. Content analysis, telephone interviews and surveys are used in this step to establish process measures 7 . Second phase included an evaluability assessment for clarifying program designs, exploring program reality, and, if necessary, helping redesign programs to ensure that they meet the criteria such as a) well-defined program goals, objectives, and performance measures, b) plausible program goals and objectives, c) ability to obtain relevant performance data (or use alternative methods to collect data, d) agreement on the evaluation; The objectives of evaluability assessment were twofold: too delineate whether a program can be meaningfully evaluated and whether evaluation is likely to contribute to improve program performance 8 . Both qualitative and quantitative methods are used for an evaluability assessment. Documents analysis, interview, and face-to-face meetings with the program manager, data manager, and previous evaluator were the primary methods to establish evaluability of the program. Third phase included immediate outcome and impact evaluation and actual immediate outcome and impact measures.
Due to the lack of individual-level data, the following methodology was followed to establish immediate outcomes and impact of the diabetes program. As indicated in Table 3, the immediate outcome and impact indicators are as follows: number of participants referred, enrolled, and completed the Diabetes Self-management Education (DSME) program; number of participants referred, enrolled, and completed Wellness Education Program; number of participants tested HbA1c (% of glycosylated hemoglobin) past 6 months/2x 12 months; number of participants who had an eye examination, foot care, or influenza shot in the past 12 months. In this evaluation, the proportion of change is calculated in the total number for 2006-2007 and 2008-2009. In the empirical studies, "effect size" was considered as mean differences, standardized mean differences, correlation coefficient, odds ratio, or reduction of relative risk. However, in quasi-experimental designs or in a program that does not have pre-planned design, changes of the proportion in relevant factors may be used. A review of the literature shows that the effect size may vary from 10% to 90% depending on the outcome measures [11][12][13] . In this study, 10% change was considered an arbitrary effect size, and a hypothesis was tested as to whether the proportion of change from 10% was statistically significant or not. This study also looked at the direction of change, such as positive or negative from the base or zero.

Process Measures
The Office of Minority Health (OMH) of the United States Department of Health and Human Services is mandated by Congress to coordinate federal agency efforts to improve racial/ethnic minority health status and eliminate health disparities. Uniform Data Set (UDS) efforts were undertaken to create a basis for evaluating program implementation and impact and, in doing so, to improve the evaluation and quality of the programs9. However, many state-level programs did not implement uniform data set efforts.
In the previous evaluation for the years of 2006-2008, a thorough review was done for all program areas and several process indicators were established. In this evaluation, both qualitative and quantitative methods were used7. However, data needed to establish immediate outcome measures and intermediate outcome measures were not collected in a format that would be suitable for analyses. This situation is not unique for this particular program; rather, it is a common challenge encountered in health disparity-related community-based research10 and we conceptualized aggregated data-based measures.  For racial identity, a significant number of participants identified themselves as Latino; however, the total number of all races is more than the number of participants because Latino identity has been cross referred in other racial groups. For detailed demographics please see Table 2:

Immediate Outcome and Impact Measures
To meet the challenges due to lack of individual level data the study incorporated 'proportion change' as a measure to indicate whether programs have created positive impact or not. For example diabetes wellness program referral was  In this analysis it is found that from 2006-07 to 2007-08, the proportion of participants referred for the wellness program decreased 8.64%, and participants enrolled in the wellness program decreased 3.4%. On the other hand, diabetes self-management education and wellness program referrals, enrollment, and completions increased significantly from the 2007-08 to 2008-09 program year (12.6+8.64=21%, 3.4+36.1=39.5%, and 31.8-2.2=29.6%, respectively). The greatest increase happened in the DSME program, with a change of (51.2-8.7= 42.5%) and increased women's participation (Figure 3). These increased referrals indicate the programs' performance improved which may have lead to increased completions in later years.

Conclusion
A great deal of qualitative information was generated on the programs' cultural sensitivity, staff communication, community health workers recruitment, different types of barriers, and facilitators of success in the past evaluation 7 . Along with these factors, health literacy and access to services must be examined. Furthermore, a separate study 14 found that travel impediments may influence access to health care, particularly in cancer screening. Future programs should consider incorporating these factors in providing services.
A large number of factors contribute to racial and ethnic disparities in health status. Health disparity programs have tremendous opportunities not only to provide services to the affected population but also to generate data that could be utilized to draw scientific conclusions about factors responsible for disparities or factors most affecting the reduction of health inequalities. In the CTG project, most of the programs provided services to the target population. However, individual level, disaggregated data on services and interventions were not collected, which means that comparisons of pre-and post-data, intergroup data could not be made, and impacts and outcomes could not be measured using standard indicators.

Discussion and Implications
The overall objectives of this evaluation were to establish the immediate impacts and some outcomes of these projects quantitatively. Due to the lack of individual level disaggregated data, a unique methodology was developed to quantify some of these immediate impacts and outcomes. This method was based on many recommendations designed to overcome observed challenges in health disparity research. However, since this methodology used an aggregated level of data, conclusions on individual levels may lead to ecological fallacies The following recommendations are offered so that in the future, these programs not only provide data to establish and measure changes in health disparity outcomes but also provide data to examine factors that could be addressed to obtain more cost-effective results.
First, program evaluation should be incorporated from the beginning of the projects, with clear indicators of process measures, immediate impact measures, and outcome measures. The Office of Minority Health may consider a full-time program evaluator or consultant who can oversee these critical evaluation components. Second, there should be a clear monitoring and feedback mechanism in the programs and their evaluation. For example, depending on the findings of process evaluation, further adjustments in delivery of service or intervention, data collection, staff and program personnel training, and other areas should be made. From 2006-2009, several changes occurred in the programs' data collection. However, some of those changes actually negatively impacted the overall evaluation of the programs (e.g., discontinuation of data collection on participants pre-/post-knowledge score, HbA1c in diabetes program). Third, all the programs should have a standard data collection instrument for each program area, specific to the health topic (e.g., diabetes) and should be used in consistent manner unless specific recommendations for changed are made for valid reasons. There should be a central database linked to these programs. Data should be collected at the individual level in a disaggregated manner. This process will allow the programs to collect meaningful data and allow researchers to examine a cohort of participants receiving services through Closing the Gap projects, as well as different social, economic, and environmental factors related to the health topics. Fourth, numerous research studies have documented the link between lower socioeconomic status and higher mortality 15,16 . Higher mortality among Blacks or African-Americans relative to Whites has been connected to their lower socioeconomic profile 17,18 and residential segregation in urban areas 19,20 . Closer consideration of the role of poverty, residential segregation, and low-income environments on the mortality of Black or African-American residents of Florida is warranted. The project may consider including some details of these parameters in the data collection process. Fifth, critical outcome areas such as monitoring diabetes, self-management, reduction in complications, and increased preventive health lifestyle have paramount importance in reducing the overall burden of disease. However, meaningful data need to be collected in order to pinpoint success for different indicators. For example, the number of diabetics who monitored and self-managed their diabetes before and after the intervention should be considered. Sixth and finally success of any program should be considered from a cost-effectiveness standpoint because reducing health inequalities is cost-effective 21 . Since resources are limited, further evaluation on economic and policy change, especially 70 Closing the Gap: An Evaluation of Community-Based Projects to Reduce Diabetes Disparities among Minority and Underrepresented Groups cost-effectiveness analysis of the programs or components of the programs, should be done. The cost-effectiveness component should be integrated from the beginning of the projects so that relevant data are collected. The program may consider a consultant to develop and implement cost-effectiveness components from the beginning of the projects.
The unique methodology clearly indicates that even though community based programs fail to collect data that could be analyzed in meaningful ways, evaluator can quantify some changes and based on those changes further recommendations could be made for the improvements of the projects. Though many factors may have contributed to the proportion changes, however, the changes clearly indicate whether the programs are going to the expected direction or not.