Development and evaluation of an Acceptance and Commitment Therapy online competency assessment: A contextual behavioral building block approach

Acceptance and Commitment Therapy (ACT) is an empirically based psychosocial intervention that targets psychological flexibility processes and has demonstrated benefits in the treatment of a variety of mental health conditions. A faster pace of training research is needed in order to inform empirically based training guidelines and this requires the development of new measures of clinical skill. The ability to identify psychological flexibility processes as they occur in therapy sessions and to discriminate between effective and ineffective interventions may be one foundational skill for ACT. This dissertation developed and evaluated a web-based discriminative ability assessment focused on this skill, called the ACT Online Competency Assessment (ACT OCA)***. Simulated therapy sessions in the Learning ACT manual’s companion DVD (Luoma, Hayes, & Walser, 2007) were adapted into a video coding task wherein participant ratings of ACT-consistent and ACT-inconsistent interventions along with the psychological flexibility processes involved were compared with expert ratings, such that higher scores indicated greater agreement with experts across three subscales. The ACT OCA was distributed in an online survey to 189 undergraduates recruited from psychology classes at the University of Nevada in Reno who had no training in ACT. This same survey was distributed to 209 participants in intensive experiential ACT workshops conducted by expert trainers. 62 undergraduates provided sufficient data and were compared with 108 therapists who provided sufficient data. Changes associated with ACT training were examined amongst 64 workshop participants who provided Pre- and Post-workshop survey responses. Therapists demonstrated superior ACT OCA performance and also reported higher levels of acceptance and mindfulness relative to undergraduates. Amongst workshop participants superior ACT OCA performance was correlated with superior performance in an ACT Knowledge Questionnaire and was correlated on some subscales with: ACT books read, ACT workshops attended, hours of simulated ACT sessions observed, ACBS membership status, and years spent in an ACT supervision group. Comparisons of Pre- and Post-workshop surveys indicated improvements in the ACT OCA as well as in psychological flexibility. Greater psychological flexibility at Pre predicted greater improvements in the ACT OCA at Post. However, psychological flexibility was also associated with lower ACT OCA scores at Pre. Limitations included the lack of a randomized control condition and high rates of drop-out. This study demonstrated an assessment development strategy with broad relevance to dissemination research and with particular importance to ACT’s functional contextual conceptualization of fidelity.