We provide a tutorial on differential item functioning (DIF) analysis, an analytic method useful for identifying potentially biased items in assessments. After explaining a number of methodological approaches, we test for gender bias in two scenarios that demonstrate why DIF analysis is crucial for developing assessments, particularly because simply comparing two groups' total scores can lead to incorrect conclusions about test fairness. First, a significant difference between groups on total scores can exist even when items are not biased, as we illustrate with data collected during the validation of the Homeostasis Concept Inventory. Second, item bias can exist even when the two groups have exactly the same distribution of total scores, as we illustrate with a simulated data set. We also present a brief overview of how DIF analysis has been used in the biology education literature to illustrate the way DIF items need to be reevaluated by content experts to determine whether they should be revised or removed from the assessment. Finally, we conclude by arguing that DIF analysis should be used routinely to evaluate items in developing conceptual assessments. These steps will ensure more equitable-and therefore more valid-scores from conceptual assessments.

Biology Department Edmonds Community College Lynnwood WA 98036

Center for Educational Measurement University of Oslo Oslo 0318 Norway

College of Education University of Washington Seattle WA 98195

Institute of Computer Science Czech Academy of Sciences Praha 182 07 Czech Republic

Institute of Computer Science Czech Academy of Sciences Praha 182 07 Czech Republic Department of Probability and Mathematical Statistics Faculty of Mathematics and Physics Charles University Praha 186 75 Czech Republic

School of Interdisciplinary Arts and Sciences University of Washington Bothell Bothell WA 98011

References provided by Crossref.org