Locating Spaces for Standards and Competencies in Care: Evaluating Child Care and Protection Services

In most evaluations of projects, including programs for child care and protection, the focus is most likely to be on assessing the organisation’s outputs as proposed by the donor rather than on what it means to the child. This undermines the very purpose of making a significant difference to the children, the recipients of the program, due to misplaced priorities. A standards and competencies in care framework were used to assess the impact the program had on children in nine child care and protection homes in three cities. The evaluation used a qualitative inquiry that appreciated the strengths and the shortcomings from the prism of the child. Participatory processes that had a strong grounding on elucidating the strengths and competencies through the eyes of the service provider and the children were used. The evaluation findings explored spaces, and provided suggestions, for improvising child care and protection across the chain of homes in existence and those planned to be set in the future. Having carried out the evaluation within the early years of the establishment of the services, the findings provided the impetus for the organisation to revisit their strategies.


Introduction
It is universally recognised and accepted that residential care is never the best option for alternative care of children needing protection. A review of studies on institutional care for children have shown that even the best residential care has psychological/emotional effect on children (North American Council on Adoptable Children, no date; Jeon H, Moulson M C., Fox N, Zeanah C, & Nelson III C. A., 2010). The Residential Care Guidelines and Standards (UNICEF, & Department of Social Welfare, 2006) brings out that children in any form of residential care are extremely vulnerable, because they are totally dependent on the staff and are isolated from their family and community. Those running or working in a residential care facility have considerable power over children. The transition to independent living after long periods in institution is noted by many as a particularly sensitive period of change for the young person, when high quality, individualized preparation and on-going support is crucial for the individual to become independent. Notwithstanding the overriding evidence of negative outcomes for children with care experience, there are few longitudinal studies to show the circumstances under which successful outcomes can be achieved for such children. For example, the Finnish country analysis notes the results of a study that followed children who had grown up in an SOS Children's Village. Adults between 22 and 51 were interviewed and their life situation in relation to their education, employment and health, was not dissimilar from that of the rest of the population.
Nearly every fifth child in the world lives in India. There are about 43 crore children in the age group of 0-18 years attention (Ministry of Women and Child Development, no date). It is estimated that about 40 per cent of children are in difficult circumstances or vulnerable which include like children without family support, children forced into labour, abused/trafficked children, children on the streets, vulnerable children, children affected by substance abuse, by armed conflict/civil unrest/natural calamity etc. Survival, growth, development and protection of these very large numbers therefore needs a priority focus and attention (Government of India Ministry of Women and Child Development, 2014). The sub-group report on Child Protection in the Eleventh Five Year Plan in India (Government of India Ministry of Women and Child Development, 2011) has expressed its concern regarding the increasing number of children entering the institutional system. It states that these children are not necessarily orphans but destitute and they do have a family somewhere, however, once these children enter the institutional system there are very limited opportunities for them to go back to their families.
Many such separated children often end up in institutions for residential care that rarely provide the ISSN: 2332-6840 (Online) 2332-6832 (Print) Copyright © 2014 Horizon Research Publishing International Journal of Social Work and Human Services Practice 257 Vol.2. No.6 Dec, 2014, pp. 256-263 environment that children, especially young children, need for healthy development. Many children are also removed from their families, often against the wishes of the family, in the belief that this is the best or the only option because of the family's poverty, the mother's unwed status, a child's disability, a parent's positive HIV status, or the lack of educational opportunities for the child. Poverty is often the driving force behind the vulnerability of single orphans (those who have lost one parent). The channelling of significant resources into residential care institutions for children, rather than in support programs to assist single parents, relatives and foster carers promotes a situation where those same parents and families, together with communities and government officials, turn to institutional facilities as a first response. As a consequence, resources for family-based and community-based alternatives for vulnerable children decreases further as key donors construct new institutions and direct funding into existing ones (International Social Service and UNICEF, 2004).

Standards and Competencies in Care
Standards can be seen as part of the continuum of care; they refer to a range of services that can be offered to vulnerable children based on individual need (Juvenile Justice (Care and Protection of Children) Act (2000, amended in 2006), UNICEF & Department of Social Welfare, 2006;Government of Ireland, 2004). Such services would begin at one end of the continuum with the least intrusive and disruptive intervention, such as supporting the child within the family context, and would move incrementally towards those services at the other end of the continuum which are most intrusive, restrictive and disruptive, such as a secure residential care centre for sentenced children. The purpose is to build competencies in the children to achieve a full and meaningful life. The nature of a continuum also allows for the possibility that two or more services can run parallel to each other and be delivered by the same or different organisations working in partnership. For example, an organisation offering short-term residential care could, at the same time, support and strengthen the community and families. They apply to, both, the temporary or long-term reception of children with the purpose of providing care and protection. The standards follow a developmental approach and the minimum standards are meant to facilitate a process toward good practice and at the same time, ensure the protection and well-being of children. The standards provide the minimum goals to be reached and maintained by the practitioners, programs and residential care centres, with the vision that over time and after capacity building, practices should actually reflect a standard beyond the minimum (Government of Ireland, 2004).
A number of documents related to standards of care in several countries including European countries, Africa, Australia and Asia are available. A lot of attention is paid to residential child care standards in the African countries, which have a large number of orphans on account of the high prevalence of HIV/AIDS. The standards required have to be good particularly when we integrate children having the disease with those not having it. Even in these countries, child protection policies have emphasized family and community based care (Mann, Long, Delap and Connell 2012).
The United Nations Convention on the Rights of the Child (UNCRC) (1989) is the backbone of all instruments for the rights of children and has been ratified by all State Governments apart from the United States of America and Somalia. Most of the standards worked out by the various countries follow these rights. Eurochild (2010) opines that children or young people should not be taken into care because they are poor and social excluded. At the same time young people should not leave alternative care to plummet into a situation of poverty and social exclusion. Quality standards in the care system should be in place and regularly monitored. The principles highlighted in the UN Alternate Care Guidelines (United Nations, 2010) note that institutionalization of a child shall be the last resort after reasonable inquiry and that too for the minimum possible duration. The Council of Australian Governments (2010) has developed a list of standards that encompasses actions at entering care, Case management and Care provision, participation of stakeholders and transition planning. The 'Looking After Children' project has been implemented in a number of jurisdictions to guide the assessment of the developmental needs of children in care and to determine whether services are meeting the needs of children. Seven developmental dimensions have been identified, encompassing both physical and emotional well-being. These are: Health, Education, Identity, Family and Social Relationships, Social Presentation, Emotional and Behavioural Development and Self-care skills. Save the Children UK (2005) has provided clear cut standards with regard to each of the above, with standard, indicators, contra indicators, basis and practice implications in Central Africa.
In India, the Integrated Child Protection Scheme (Government of India Integrated Child Protection Scheme, Government of India, 2009) emphasizes the expansion of family and community based programs as against institutional ones. At the same time, there is a lot of concern expressed about standards of care in residential homes through the Juvenile Justice (Care and Protection of Children) Act (2000, amended in 2006), in view of the fact that children in extremely difficult situations do need residential care with due consideration for their future. The burgeoning interest in safeguarding standards of care and protection and affirming rights of children in institutional settings ( Save the Children, UNICEF and Better Care Network, 2009) has prompted increased attention on monitoring and evaluation. The Child Welfare Committee is one such an instrument put in place through the government systems to oversee children's institutions while donors have considered periodic monitoring and evaluation. It has been observed that residential care, when monitored with good standards of care can benefit children to a great extent, creating opportunities for healthy growth and development, which their own family may never be able to provide (North American Council on Adoptable Children, no date; Jeon, Moulson, Fox, Zeanah, & Nelson III, 2010). However, this can be a missed opportunity when focus of such initiatives and instruments is often arbitrary and limited to assessing the organisation's outputs as deemed fit by the policy makers and power holders. This undermines the very purpose of making a significant difference to the children, the recipients of the program, due to misplaced priorities. Recognising the methodological and practical constraints in conducting complex interventions, evaluation programs are required to be amenable for scientific evidence generation using standards of care and competencies.

Rainbow Homes
Rainbow Homes 1 , a Residential Care facility for vulnerable girls in the 'on and of the streets' category with 30 homes across the country under the banner Rainbow Federation India, intends to provide protection and care within an entry point of education for street children. The focus is on improving conditions and lives of the street children by providing mainstreamed education in regular schools and preparing them to compete on an equal platform with the general population children. Partnership Foundation 2 , the ongoing donor as well as grant facilitator, through business partnerships, combines emotional engagement with a business through a down-to-earth approach through innovative and structural solutions. The program is based on voluntary corporate funding where the core proposition for sponsoring is: 'Fund your own Rainbow Home'. The Foundation program, using tools of education and residential care for the most needy, gives street children an opportunity to lead an independent and fulfilling life, independent of charity and to grow up to be responsible citizens who can and want to make a contribution to society. Children selected based on a criterion are provided continued support for completion of schooling and other services required for all round development; if necessary, the support is extended for longer periods until the girl is able to stand independently on her own abilities. Partnership Foundation positions the Rainbow Homes as one of the several programs in the spectrum of service providers for the betterment of the street child. Partnership Foundation intends to support 100 Rainbow homes reaching out to 10,000 street children and escort them to adulthood through these homes by 2018 with an approximate investment of 500 Euros per child per year. Partnership Foundation intends to draw on the support of other programs for reintegration of the Rainbow children and for referring needy children to the Homes. The existing school building is modified for use by the 100 and 200 Rainbow girls. With minimal redesigning, the buildings are equipped to house a kitchen, showers, toilets and lockers. This would be cost effective as there is no infrastructural cost; it would incur only a fraction of the expense of building a home. The Sarva Shiksha Abhiyan 3 Framework for Implementation published in February 2011 has recommended the Rainbow Homes approach for educating street children in cities.
Rainbow Homes and Partnership Foundation on their pursuit to sustain high standard of care approached the Tata Institute of Social Sciences to evaluate the outcomes of the Rainbow Homes initiative on children in comparison with three other non-Rainbow Programs to inform on practices that have scope for scale. This is a mid-program evaluation study carried out at six Rainbow Homes in three cities and three other Non-Rainbow Home interventions for street children in the same three cities over a one year period using an ethnographic approach for evaluation. This paper is based on the methodology used in evaluating the Rainbow Homes.

Assessing Standards of Care
The current paper explores the process and approach used to assess standards of care and competencies at the residential homes using the tool specifically developed for the purpose. The evaluation framework was used to evaluate the two Rainbow Home and one non-Rainbow Home child care institutional service one each in three cities namely, Kolkata, Hyderabad and Delhi. While the philosophy of the homes is the same, the ideologies of the each of the NGOs managing the Rainbow Homes influenced the running of the homes and thus transcended the child's growth. NGOs who partnered on the Rainbow Home project stemmed from varied ideologies such a child rights from protection perspectives, women's empowerment and education as a child right to name a few. Vol.2. No.6 Dec, 2014, pp. 256-263 The framework that was to be developed intended to assess the merits and demerits of the various intervention methods, strategies and programs for street children, provide for in-depth insights and lessons on the implementation of successful large scale interventions and strategies for street girls and suggest desirable standards of care for street girls residing in the Rainbow Homes.
After reviewing all the available frameworks, the project team deliberated and decided to consider the Standards of Care Assessment frameworks provided by United Nations Alternate Care guidelines and the Juvenile Justice (Care and Protection of Children) Act (2000, amended in 2006) as the basic instruments for development of the tools. It is also noteworthy to mention that India has been progressive and has laid guidelines for making operational the standards of care in Residential care as a sequel to the Juvenile Justice (Care and Protection of Children) Act (2000, amended in 2006). It makes a pitch towards establishing care standards to ensure the quality and conditions that are conducive to the child's development, such as individualized and small-group care. The standards provide a good framework in general to understanding the minimum requirements. There is tremendous scope for going much beyond this 'minimum'. However, the challenge was on how to use the framework as a comparative tool to assess the Rainbow Homes and Non-Rainbow Child care centres considered for the study. It would be pertinent to iterate that the nuances of a child friendly environment cannot be quantified fully, and the overall existence of a positive environment is paramount even when Homes and Centres may fall short of the prescribed space or facility. As Rainbow Homes share common resources with schools, they are included as resources available and at their disposal for use by children.
Several key questions worked out in collaboration with Partnership Foundation, Rainbow Homes and interactions with children formed the basis for the evaluation framework construction. o Food and Nutrition: Food/nutrition schedule, quality of food served, nutrition adequacy, eating arrangements and way of serving, feeding during illness, cooking arrangements.
o Education: Mainstreaming into regular schools, provision of developmental opportunities, extracurricular activities, development of child's sense of self, space and comfort, additional coaching beyond school.
• Psychosocial bonding and support: Connectedness to significant adults/staff/peers, social connectedness and bonding, social exposure mechanisms, structured daily work plans.
• Children's involvement and decision making: Staff's awareness of the child rights framework, involvement of children in the formulation of activities, and in planning for their future and existence of a Child Rights framework for the organisation and mechanisms for children to influence and understand home rules and regulations.
• Child Plan and Focus on reintegration into family and society: Child Development Plan, reintegration plan, adoption linkages and fostering arrangements.
• Staffing: Staff structure, staff selection, staff comfort Anita Rego et al.
Horizon Research Publishing with children, staff professional diversity, capacity building, arrangements to review staff and child progress/issues, supervisory mechanisms for quality enhancement, staff remuneration and motivation processes, awareness of staff on policy guidelines on child care and volunteers.
• Policy frameworks: Organisational child care policy, program alignment with national guidelines, availability and access of policy documents to staff members and CWC-Home coordination mechanism.
While we used the framework of the Juvenile Justice Act, there were some terminology and sections that were not relevant to the kind of children admitted to Rainbow Homes, for e.g. the term Superintendent was associated with an underlying power dynamic and Rainbow Homes do not have such a position. The terminology 'institutions' was replaced by 'homes'. Certain procedures pertaining to 'action related to child abuse' needed to be viewed keeping in mind that the first step of response is at the Home level followed by a complete response chain that encompasses the involvement of Senior Management. We needed to factor the admission procedures to make it more organization based. The section on 'Family study of children where families have relinquished care to the organization permanently' was not applicable as Rainbow Homes cannot take such children and implementing adoption service did not apply as they are not a licensed agency. Components related to conflict with law were not considered as the Home was less likely to have taken in these children. The Management committee followed a different structure from what was given in the rules and was more local.
The data gathering on the tool was primarily qualitative. Both individual and group methods of data collection were used, including case studies. The study used an ethnographic approach which included daily observations, several discussions and in-depth interviews with the management (two per home 18 in total), staff ( a minimum of three staff of each home 29 in total), children of the homes ( 3 older children in each home 32 in total) and key informants (7 only for Rainbow Home). Eighteen focus group discussions were conducted, one for children and other for staff, to understand major policy issues, their impact on the children and the lessons learnt in the process. The data collectors developed daily notes on their discussions with children and staff, observations at the home and prepared reports for the FDGs and in-depth interviews. Much of the information on indicators in this study depended on self-reports of the staff and children and observations made by research investigators.
Most of the children approached the service after experiencing a difficult life circumstance. The organization consent and the children's assent were taken. Consent forms were developed and verbal consent was taken from children and staff members. Confidentiality of information shared was assured. Secondary sources of data maintained by the Rainbow Homes and Non Rainbow Homes as part of regular monitoring such as child reports, newsletters and the organisation's policy documents and quantitative data was used to substantiate the qualitative data. Participant observations were carried out with the data collectors spending around 15 days at each of the Rainbow Homes over a three month period. This was a learning experience, and it gave them a better perspective. They were able to identify what was working well and what not so well. Data collection and analysis were carried out simultaneously. The analysis carried out using Altas.ti and the report was developed. A theme-based analysis was carried out based. The themes were drawn from the evaluation framework. By reading and selecting text passages, we created a data corpus for a theme by assigning codes. We arrived at concentrates of the several ideas on the said theme across the many documents. We compiled a written report describing each of thematic areas.
We believe that evaluation is not a stand-alone process. We maintained the approach as flexible and adaptable to the needs of the intervention. What we desired as an end product was an evidence-based framework that would help to improve the quality of evaluation and provide information for the development of relevant policies. The framework that we used served as an instrument to come to a judgement on the value of an activity. It supported us to assess whether the programme has achieved its objectives, and how different processes contributed to achieving these objectives. For us, the ultimate purpose of the framework was to provide information to managers for decision making; make recommendations for improvements to programs, policies, or procedures; and identifying where administrative action may be necessary.

The Experience and the Learnings
Our study observed that the Rainbow Homes project has great potential to intervene at the right time in the lives of the vulnerable girls and change their future to a great extent. It helped prevent all possible dangers of dropping out of school, getting married before 18 years and/or being victims of various exploitative forces such as trafficking, child labour etc. A well-managed Rainbow Home offered all the inputs for the protection, education and overall development of vulnerable girls 'on' or 'of' the street. Our experiences on the standards of care and competencies as a guide for managing the evaluation process collaborated with our position that a systematic approach using the tool and our methodology was robust to capture the unique dimensions of children's progress at the home.
The inquiry process affirmed and appreciated the strengths and the shortcomings of the home from the prism of the child. Participatory evaluation processes had a strong grounding on elucidating the strengths and competencies through the eyes of the service provider and the children. The methodology we used allowed opportunity for the Vol.2. No.6 Dec, 2014, pp. 256-263 stakeholders to be consulted in laying down the criteria for assessment and describe stakeholders' participation.
The evaluation findings explored spaces, and provided suggestions, for improvising child care and protection across the chain of homes in existence and those planned to be set in the future. Having carried the evaluation within the early years of the establishment of the services, the findings provided an impetus to the organisation to revisit their strategies.
The explorative journey provided us the answers for the following five critical areas: • Potential benefit of the various programs for vulnerable and street girls: Physically, psychologically, socially, mentally, educationally and otherwise • Aspects of each intervention model that worked well, can be improved and suggestions to do so and the lessons learnt from the various models • Quality, effectiveness and variations in care strategies and processes across the various intervention models that have value and contribute to differences in results • Elements with potential for scale and indicative directions for policy development within the Indian Government's street children protection assertions • Linkages, networks and cooperation with local NGOs initiatives for street children The tool provided a comprehensive perspective on the various dimensions that are important for the child in care and protection. Furthermore, it captured comprehensively the various considerations that are important and need to be weighed in formulating an evaluation design.
The crux of any evaluation is on the essential question "What will be evaluated?" This emerges from the fundamental enquiry "what is the program" and in what context does it exist? This leads us to the selection of the appropriate standards (i.e., type or level of performance) that must be reached for the program to be considered successful and the evidence will be used to indicate how the program has performed. The use of Country mandated standards for child care and protection helped us to derive what aspects of the program should be considered when judging program performance to assess effectiveness, efficiency, relevance, impact and sustainability of the intervention within the Country contexts. We were using a legal instrument that was developed with the Government Homes in the background and hence several elements did not apply or had to be modified to suit those girls who were neither convicted by the court nor were without a guardian or family. The UN Alternate Guidelines for Care were assertions and we had to convert them into indicators for assessing standards. The next task was to compare the Rainbow Homes with non-Rainbow programs. They have different purposes and are non-comparable on several aspects. Quantifying the outcomes was something we aspired for, but as most organisations did not have completeness, the data available was not very meaningful. However, the qualitative data was rich and provided information that helped us capture the commonalities, the differences and the lessons learnt, and make them available for those who wished to enter into partnership with the Rainbow Home model or improve the quality of care in existing Rainbow Homes.
The exploration followed clearly from the evaluation questions and led the direction of the analysis of data. The indicators followed the SMART framework (specific, measurable, attainable, relevant and time bound). It helped us to remain focused and broad-based to ensure that the evaluation served the information needs of intended users: Partnership Foundation, Rainbow Homes and Children. By being realistic, prudent, diplomatic, and frugal and at the same time conduct the evaluation legally and ethically, we assured that due regard was paid for the welfare of those involved in the evaluation, as well as those affected by its results. The evaluators ensured that the evaluation conveyed technically adequate information about the features that determine the worth or merit of the programme being evaluated. Any discrepancies between the planned and actual implementation of the object being evaluated are explained.
The use of standards allowed scope to consider softer areas of change such as strategies used, gender equality, protection and human rights. It had elements that described the socio-political instruments such as that committees within which the intervention take place and how they influence the outcome and impact on children. The results provided direction on laying down policy considerations relevant to the child's developmental intervention, the development agencies and partners' policy documents, objectives and strategies.
The analysis was challenging. We had to understand the whole range of effects and how they vary, for example, among recipients or between sites. The key question was on how the intervention worked or in other words, what are the active ingredients and how are they exerting their effect? Answers to this kind of question are essential as there was intent to take lessons from this design and apply them appropriately other Rainbow Homes that are intended to be set in other locations. Our tool was able to bring to the table areas of success and what defines the success, where are the areas where the project had strengths and places where they need to tweak to make it more effective.
Comparisons no doubt were difficult; there are differences in approaches across the models studied. We have extensive information and data on the Rainbow Homes, however, the information that is available for the other interventions do not have similar depth. Variability in implementation pre-planned or otherwise, made it important Anita Rego et al.

Horizon Research Publishing
for us to report both the process and the outcome giving a clear description of the intervention is provided to enable replication and synthesis of evidence. At times, access to similar information especially in the Non-Rainbow Homes was a challenge at the ground level even though we had the needed permissions; accessing the information within the short time available was difficult given the priorities of the partnering organisations. We captured what we had access to and observed. This has been a weakness of the reporting of complex intervention studies in the past and can be seen also in our evaluation. Maintaining fidelity across the evaluation was not straightforward and strict standardisation in our approach was not possible. We had to make judgement on how controls should be put in and limits.

Conclusion
The evaluation framework drawn from a standards and competence perspective provides a structure to answers to the questions that we had proposed in our pursuit of understanding outcomes on children and comparing models of interventions. We appreciate and understand that evaluation should not be a stand-alone activity, nor should it occur only at the end of a project. It should be considered an integral part of the overall project planning, ideally beginning before a project is implemented and closely linked to setting the project's objectives. The tool is an instrument in the hands of the evaluator and may not necessarily be an end in itself. A lot depends on the one who is using it and how the analysis is carried out. Nevertheless, a tool is done without a thorough thinking process can leave a lot to be desired when actions are to be taken based on the evaluation learnings.