Background Low replication rates are a concern in most, if not all, scientific disciplines. In psychiatric genetics specifically, targeting intermediate brain phenotypes, which are more closely associated with putative genetic effects, was touted as a strategy leading to increased power and replicability. In the current study, we attempted to replicate previously published associations between single nucleotide polymorphisms and threat-related amygdala reactivity, which represents a robust brain phenotype not only implicated in the pathophysiology of multiple disorders, but also used as a biomarker of future risk.
Methods We conducted a literature search for published associations between single nucleotide polymorphisms and threat-related amygdala reactivity and found 37 unique findings. Our replication sample consisted of 1117 young adult volunteers (629 women, mean age 19.72 1.25 years) for whom both genetic and functional magnetic resonance imaging data were available.
Results Of the 37 unique associations identified, only three replicated as previously reported. When exploratory analyses were conducted with different model parameters compared to the original findings, significant associations were identified for 28 additional studies: eight of these were for a different contrast/laterality; five for a different gender and/or race/ethnicity; and 15 in the opposite direction and for a different contrast, laterality, gender, and/or race/ethnicity. No significant associations, regardless of model parameters, were detected for six studies. Notably, none of the significant associations survived correction for multiple comparisons.
Conclusions We discuss these patterns of poor replication with regard to the general strategy of targeting intermediate brain phenotypes in genetic association studies and the growing importance of advancing the replicability of imaging genetics findings.
Not surprising, but also still troubling in certain ways. ---
Not quite. The brain being really hard is the excuse for having relatively soft theory.
It doesn't explain why previous results do not have a high replication rate. What does explain it are things like average low statistical power (small sample in relation to true effect size) (leading to poor estimations), scientific culture pushing researchers to commit questionable research practices: things like p-hacking and the like, and methodological differences between replication studies and original studies. Most, if not all, of these things are really hard to correct within the science community, but there's been an on-going push to work towards solutions. ---