The heterogeneity statistic I2 can be biased in small meta-analyses

Research article Open access Published: 14 April 2015 Paul T von Hippel1 BMC Medical Research Methodology volume15, Articlenumber:35 (2015) Cite this article AbstractBackgroundEstimated effects vary across studies, partly because of random sampling error and partly because of heterogeneity. In meta-analysis, the fraction of variance that is due to heterogeneity is estimated by the statistic I2. We calculate the bias of I2, focusing on the situation where the number of studies in the meta-analysis is small. Small meta-analyses are common; in the Cochrane Library, the median number of studies per meta-analysis is 7 or fewer.MethodsWe use Mathematica software to calculate the expectation and bias of I2.ResultsI2 has a substantial bias when the number of studies is small. The bias is positive when the true fraction of heterogeneity is small, but the bias is typically negative when the true fraction of heterogeneity is large. For example, with 7 studies and no true heterogeneity, I2 will ov...