This book shows how to conduct statistical power analysis when the underlying research data are unreliable and explores how sample design effects, measurement error, and equating error impact power analysis. As such, it covers unreliability in test scores and in the measurement scale, and incorporates both the sources of sampling error, such as design effects in survey sampling, and measurement error, such as unreliability and equating error in psychometrics, in power calculations. Five widely used statistics are treated, including the mean for one group, mean difference for two independent groups, mean difference for two dependent groups, proportion, and correlation. Each chapter develops procedures for a simple and complex random sample of true scores or units, and a complex random sample of observed scores or units without and with equating error. Analogously to design effects due to complex random sampling, the reader is introduced to indices of measurement effects, which reflect the impact of measurement error and equating error on the standard error of the statistic. The book includes numerous worked out examples and is aimed at practitioners, researchers and students in psychometrics and other social and behavioral sciences.

Gary W. Phillips holds a PhD in applied statistics and psychometrics and is the Vice President for Psychometrics at Cambium Assessment in Washington, DC, which is a testing company that conducts statewide testing in over 30 states in the United States. Previously, he was the Deputy and Acting Commissioner at the National Center for Education Statistics (NCES).

Tao Jiang holds a PhD in mathematical and computational statistics and is an Institute Fellow for Psychometrics at Cambium Assessment in Washington, DC, USA.