I will argue that published results demonstrate the likelihood that new insights into human brain function may be obscured by poor and/or limited choices in the data-processing pipeline, and review the work on performance metrics for
optimizing processing pipelines: prediction, reproducibility and related empirical Receiver Operating Characteristic (ROC) curve metrics. Using the NPAIRS split-half resampling framework for estimating prediction and reproducibility metrics (Strother et al., NI, 2002; LaConte et al., NI, 2003; Shaw et al., NI, 2003), I
will illustrate their use by testing the relative importance of selected pipeline components (interpolation, in-plane spatial smoothing, temporal detrending and between-subject alignment) in the group analysis of BOLD-fMRI scans from 16 subjects performing a block-design, parametric-static-force task. Both prediction and
reproducibility metrics were required for optimizing the pipeline and gave somewhat different results. In addition, the parameter
settings of components in the pipeline interact so that the current practice of reporting the optimization of components tested in relative isolation is unlikely to lead to fully-optimized processing pipelines.
Copyright. All Rights Reserved.