| AJNMMI Copyright © 2011-present, All rights reserved. Published by e-Century Publishing Corporation, Madison, WI 53711, USA |
Am J Nucl Med Mol Imaging 2013;3(2):175-181
Original Article
In situ study of the impact of inter- and intra-reader variability on re-gion of
interest (ROI) analysis in preclinical molecular imaging
Frezghi Habte, Shradha Budhiraja, Shay Keren, Timothy C Doyle, Craig S Levin, David S Paik
Molecular Imaging Program at Stanford (MIPS), Department of Radiology, Department of Pediatrics, Stanford University, Stanford, CA,
USA. Current Address: Adobe Systems India Private Limited, City Center, Sector 25-A, Noida 20130, India. Current Address: Nofim
School, Haifa, Israel.
Received December 24, 2012; Accepted January 28, 2013; Epub March 8, 2013; Published March 18, 2013
Abstract: We estimated reader-dependent variability of region of interest (ROI) analysis and evaluated its impact on preclinical
quantitative molecular imaging. To estimate reader variability, we used five independent image datasets acquired each using
microPET and multispectral fluorescence imaging (MSFI). We also selected ten experienced researchers who utilize molecular
imaging in the same environment that they typically perform their own studies. Nine investigators blinded to the data type completed
the ROI analysis by drawing ROIs manually that delineate the tumor regions to the best of their knowledge and repeated the
measurements three times, non-consecutively. Extracted mean intensities of voxels within each ROI are used to compute the
coefficient of variation (CV) and characterize the inter- and intra-reader variability. The impact of variability was assessed through
random samples iterated from normal distributions for control and experimental groups on hypothesis testing and computing
statistical power by varying subject size, measured difference between groups and CV. The results indicate that inter-reader
variability was 22.5% for microPET and 72.2% for MSFI. Additionally, mean intra-reader variability was 10.1% for microPET and 26.4%
for MSFI. Repeated statistical testing showed that a total variability of CV < 50% may be needed to detect differences < 50% between
experimental and control groups when six subjects (n = 6) or more are used and statistical power is adequate (80%). Surprisingly
high variability has been observed mainly due to differences in the ROI placement and geometry drawn between readers, which may
adversely affect statistical power and erroneously lead to negative study outcomes. (ajnmmi1212005)
Keywords: Molecular imaging, preclinical, region of interest analysis, variability, microPET, multispectral fluorescence imaging
Address correspondence to: Dr. Frezghi Habte, Molecular Imaging Program at Stanford (MIPS), Department of Radiology, Stanford
University, Stanford, CA, USA. E-mail: fhabte@stanford.edu
Original Article
In situ study of the impact of inter- and intra-reader variability on re-gion of
interest (ROI) analysis in preclinical molecular imaging
Frezghi Habte, Shradha Budhiraja, Shay Keren, Timothy C Doyle, Craig S Levin, David S Paik
Molecular Imaging Program at Stanford (MIPS), Department of Radiology, Department of Pediatrics, Stanford University, Stanford, CA,
USA. Current Address: Adobe Systems India Private Limited, City Center, Sector 25-A, Noida 20130, India. Current Address: Nofim
School, Haifa, Israel.
Received December 24, 2012; Accepted January 28, 2013; Epub March 8, 2013; Published March 18, 2013
Abstract: We estimated reader-dependent variability of region of interest (ROI) analysis and evaluated its impact on preclinical
quantitative molecular imaging. To estimate reader variability, we used five independent image datasets acquired each using
microPET and multispectral fluorescence imaging (MSFI). We also selected ten experienced researchers who utilize molecular
imaging in the same environment that they typically perform their own studies. Nine investigators blinded to the data type completed
the ROI analysis by drawing ROIs manually that delineate the tumor regions to the best of their knowledge and repeated the
measurements three times, non-consecutively. Extracted mean intensities of voxels within each ROI are used to compute the
coefficient of variation (CV) and characterize the inter- and intra-reader variability. The impact of variability was assessed through
random samples iterated from normal distributions for control and experimental groups on hypothesis testing and computing
statistical power by varying subject size, measured difference between groups and CV. The results indicate that inter-reader
variability was 22.5% for microPET and 72.2% for MSFI. Additionally, mean intra-reader variability was 10.1% for microPET and 26.4%
for MSFI. Repeated statistical testing showed that a total variability of CV < 50% may be needed to detect differences < 50% between
experimental and control groups when six subjects (n = 6) or more are used and statistical power is adequate (80%). Surprisingly
high variability has been observed mainly due to differences in the ROI placement and geometry drawn between readers, which may
adversely affect statistical power and erroneously lead to negative study outcomes. (ajnmmi1212005)
Keywords: Molecular imaging, preclinical, region of interest analysis, variability, microPET, multispectral fluorescence imaging
Address correspondence to: Dr. Frezghi Habte, Molecular Imaging Program at Stanford (MIPS), Department of Radiology, Stanford
University, Stanford, CA, USA. E-mail: fhabte@stanford.edu
 |  |  |  |  |  |  |