Evaluation & Impact

The TAC pilot and all replications of the TAC have been subject to ongoing, rigorous evaluation designed to assess training delivery, outcomes, and effectiveness:

• Training delivery is assessed using fidelity observations and feedback on each of the modules from participants and trainers.
• Training outcomes are assessed using pre- and post-training self-assessments of adoption competency and surveys reporting changes in clinical practices at the midpoint and conclusion of training.
• Training effectiveness is assessed using a pre- and post-test administered to training participants and to a comparison group of comparably qualified clinicians. In addition, interviews are conducted with TAC-trained clinicians six to eight months after training completion to identify and explore the longer-term impact(s) of training on clinical practices.

Approach

The evaluation implementation employs a team approach with PolicyWorks, Ltd. providing leadership for evaluation across replication sites.  Data from trainees and from trainers are collected online using technologies provided by PolicyWorks, Ltd. to which replication sites have access.

Findings Highlights

Training Outcome: Changes in Clinical Practices

TAC participants are asked at the midpoint and conclusion of training to identify and comment on the aspects of practice influenced by information or insights gained from the training. Findings to date are based on 2,191 responses that contained 10,637 separate narrative responses describing ways practices were influenced by the training and most important learning. At the conclusion of training,

• all TAC participants to date report change in at least two of the six defined aspects of practice;
• 61.65 percent report change in all five aspects at the individual clinician level; and
• 50.41 percent report change in procedures, programming and/or services at the organizational level.

The TAC is making a significant impact on the quality of mental health services for members of the adoption kinship network.

TAC-trained clinicians point to their more effective use of quality assessment tools in their work with children, youth and families while also having stronger skills in using evidence-based and evidence-informed clinical approaches in their work with members of the adoption kinship network. TAC-trained clinicians also report changes at the organizational level – in procedures, services, and programming – that represent the integration of adoption competency knowledge, skills and values into the organization.

Clinicians Trained to Date

TAC participants to date report an average of 8.7 years clinical experience and an average 13.5 years professional experience. Demographic characteristics of TAC participants are summarized in the table below. Participants are 90.74 percent female and 9.26 percent male, a gender representation pattern similar to that found among clinical practitioners. Race/ethnicity representation among participants varies
widely across states and reflects general race/ethnicity representation in individual states. However, across all TAC sites participants have been 76 percent White/Caucasian, 14 percent Black/African American, 3 percent Hispanic/Latino, 2 percent Multi-race, 2 percent Asian, less than 2 percent American Indian, and less than one percent Other. Almost 12 percent of participants have completed or are near completing requirements for doctoral degrees, 85 percent hold masters degrees and 3.5 percent hold degrees at the Bachelors level. A remarkable 72 percent report a personal connection to adoption, many as adoptive parents but also as adopted persons or having family and/or close friends touched by adoption.