Measuring Fidelity to Improve Health and Human Services Outcomes
In human services and behavioral health research, policy makers and stakeholders are oftentimes focused on outcome evaluations.
- Is this therapy improving the safety of this person by reducing the amount of suicidal idealizations?
- Is this school-based support group improving the student’s wellbeing by contributing to their psychosocial development?
- Are we reducing the amount of out-of-home placements by implementing wraparound services?
Ultimately in outcome evaluation, we want to know, is this product or service working? Is anyone better off after receiving these services?
However, there is an additional component: a service that is working and providing outcomes that are improving people’s well-being is a positive thing; but how did we get there? This is where fidelity measures become so imperative. Fidelity is “the adherence of actual treatment delivery to the protocol originally developed”1.
The importance of fidelity is highlighted when utilizing evidence-based approaches in health and human services, most often in the field of mental health. These treatments, therapeutic processes, and models have undergone strict measures that are reliable and valid to determine their effectiveness2. Measuring fidelity refers to measuring the adherence to evidence-based practices.
If a program’s clients/consumers are achieving positive results, the first step is to determine if adherence to fidelity was met. If so, great! If not, what interventions were taking place that were, in fact, working? If we are seeing negative outcomes from a population, such as increased psychiatric hospitalizations or foster care placements, what treatment protocols and intervention standards are not being met and impacting client outcomes?
It is important to measure fidelity because we need to know if interventions are working when they are being implemented. Regardless of the type of service provided, be it prevention focused, or maintenance counseling, providers and their supervisors, as well as stakeholders, need to know if the way they are conducting their practice will result in positive outcomes and reach predetermined goals.
By observing and measuring fidelity, health and human service agencies are able to note how the community responds to interventions that are administered equally, following their appropriate implementation plans. If the community changes the way they respond to an intervention that is being implemented with fidelity, a needs assessment can be conducted to determine what other interventions may be more effective, if needed.
The bottom line: Programs can learn, adapt and innovate to meet new needs of the community only if fidelity is measured.
1 Mowbray, C. T., Holter, M. C., Teague, G. B., Bybee, D. (2003). Fidelity criteria: development, measurement, and validation. American Journal of Evaluation, 24(3), 315-340.
2 Orwin, R. G. (2000). Assessing program fidelity in substance abuse health services research. Addiction 95,(Suppl. 3), S309-27.
Carroll, C., Patterson, M., Wood, S. et al. A conceptual framework for implementation fidelity. Implementation Sci 2, 40 (2007). https://doi.org/10.1186/1748-5908-2-40