Journal of Contextual Behavioral Science (JCBS)
Volume 29, July 2023, Pages 86-97
Authors
Ian Hussey
Abstract
Following recent calls to examine the replicability of behavioral research, I examined two key determinants of replicability, namely sample sizes and statistical power, in research using the Implicit Relational Assessment Procedure (IRAP). A systematic review was used to gather all published studies employing the IRAP and extract their designs and sample sizes. The use of Null Hypothesis Significance Testing was found to be nearly ubiquitous, justifying the examination of statistical power. Using an established method, median sample sizes were used to estimate the statistical power to detect the average published effect size in psychological research (r = .20) in each year. Sample sizes and the statistical power they imply were found to be very low in IRAP studies (in 2022: median N = 64, power = 0.34). At the current rate of growth, the median study will only reach the recommended minimum power of at least 0.80 by 2080. The IRAP literature was directly compared with the Social and Personality psychology literature using an existing dataset. Median sample sizes and their implied statistical power were lower in the IRAP literature in all years than they were in Social and Personality psychology at the beginning of the Replication Crisis in 2011, and indeed in all subsequent years. Improvements in sample sizes and statistical power in the Social and Personality psychology literature were significantly and substantially larger than in the IRAP literature. Direct tests of the reproducibility and replicability of claims in the IRAP literature are needed.