Newsletter Sign UpThe TAC pilot and all replications of the TAC have been subject to ongoing, rigorous evaluation designed to assess training delivery, outcomes, and effectiveness:
- Training delivery is assessed using fidelity observations and feedback on each of the modules from participants and trainers.
- Training outcomes are assessed using pre- and post-training self-assessments of adoption competency and surveys reporting changes in clinical practices at the midpoint and conclusion of training.
- Training effectiveness is assessed using a pre- and post-test administered to training participants and to a comparison group of comparably qualified clinicians. In addition, interviews are conducted with TAC-trained clinicians six to eight months after training completion to identify and explore the longer-term impact(s) of training on clinical practices.
Approach
The evaluation implementation employs a team approach with PolicyWorks, Ltd. providing leadership for evaluation across replication sites. Data from trainees and from trainers are collected online using technologies provided by PolicyWorks, Ltd. to which replication sites have access.
Findings Highlights
Training Effectiveness. TAC participants from 33 cohorts scored an average 43 points higher on post-tests than control groups of comparably qualified professionals not enrolled in the training. Further, participants experienced an average gain in pre- to post-scores of 45.75 points while those in the control group experienced a gain of only 1.50 points.
Training Outcome: Changes in Clinical Practices. TAC participants are asked at the midpoint and conclusion of training to identify and comment on the aspects of practice influenced by information or insights gained from the training. Findings to date are based on 617 responses that contained 2,992 separate narrative responses describing ways practices were influenced by the training and most important learning. At the conclusion of training,
- all TAC participants to date report change in at least two of the six defined aspects of practice;
- an average 60.85 percent report change in all five aspects at the individual clinician level; and
- 51.90 percent report change in programming and services at the organizational level.
Clinicians Trained to Date
TAC participants to date have had an average of 7.6 years clinical experience and an average 13.4 years professional experience. Fifty-five percent are social workers, 20 percent are counselors, 13 percent are marriage and family therapists, and almost 12 percent are psychologists. Their work settings include public and private mental health agencies (31.6%) and private practices (18.6%), adoption specialty organizations (18.9%), child welfare agencies (14.6%), family services organizations (14%), residential treatment facilities (9.1%) and other settings (8%). About ten percent of TAC participants reported more than one work setting.
