We present the Homeostasis Concept Inventory (HCI), a 20-item multiple-choice instrument that assesses how well undergraduates understand this critical physiological concept. We used an iterative process to develop a set of questions based on elements in the Homeostasis Concept Framework. This process involved faculty experts and undergraduate students from associate's colleges, primarily undergraduate institutions, regional and research-intensive universities, and professional schools. Statistical results provided strong evidence for the validity and reliability of the HCI. We found that graduate students performed better than undergraduates, biology majors performed better than nonmajors, and students performed better after receiving instruction about homeostasis. We used differential item analysis to assess whether students from different genders, races/ethnicities, and English language status performed differently on individual items of the HCI. We found no evidence of differential item functioning, suggesting that the items do not incorporate cultural or gender biases that would impact students' performance on the test. Instructors can use the HCI to guide their teaching and student learning of homeostasis, a core concept of physiology.

**Physiology Educational Research Consortium Seattle WA 98115

Biology Department Edmonds Community College Lynnwood WA 98036

Department of Biology Canisius College Buffalo NY 14208

Department of Biology Niagara University Niagara NY 14109

Department of Biology University of Washington Seattle WA 98195

Department of Molecular Biophysics and Physiology Rush Medical College Chicago IL 60612

Institute of Computer Science Czech Academy of Sciences 18207 Prague Czech Republic

School of Interdisciplinary Arts and Sciences University of Washington Bothell Bothell WA 98011

See more in PubMed

Abraham J. K., Perez K. E., Price R. M. The dominance concept inventory: A tool for assessing undergraduate student alternative conceptions about dominance in Mendelian and population genetics. CBE—Life Sciences Education. 2014;13:349–358. PubMed PMC

Allen M. J., Yen W. M. Introduction to measurement theory. Monterey, CA: Brooks/Cole; 1979.

American Association for the Advancement of Science. Vision and change in undergraduate biology education: A call to action. Washington, DC: 2011.

Ames A. J., Penfield R. D. An NCME instructional module on item-fit statistics for item response theory models. Educational Management: Issues and Practice. 2015;34:39–48. 10.1111/emip.12067.

Association of American Medical Colleges and Howard Hughes Medical Institutes. Scientific foundations for future physicians. Washington, DC: 2009.

Bates D., Maechle M., Bolker B., Walker S. Fitting linear mixed-­effects models using lme4. Journal of Statistical Software. 2015;67:1–48. 10.18637/jss.v067.i01.

Benjamini Y., Hochberg Y. Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B, Statistical Methodology. 1995;57:289–300.

Boone W. J. Rasch analysis for instrument development: Why, when, and how. CBE—Life Sciences Education. 2016;15:rm4. 10.1187/cbe.16-04-0148. PubMed PMC

Carnegie Classification of Institutions of Higher Education. n.d. About Carnegie Classification. Retrieved January 26, 2017, from http://carnegieclassifications.iu.edu.

Chalmers R. P. mirt: A multidimensional item response theory package for the R environment. Journal Statistical Software. 2012;28:1–29.

Cooper S. J. From Claude Bernard to Walter Cannon. Emergence of the concept of homeostasis. Appetite. 2008;51:419–427. PubMed

Cronbach L. J. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334. 10.1007/bf02310555.

Crowther G. J., Price R. M. Re: Misconceptions are “so yesterday!”. CBE—Life Sciences Education. 2014;13:3–5. 10.1187/cbe.13-11-0226. PubMed PMC

D’Avanzo C. Biology concept inventories: Overview, status, and next steps. BioScience. 2008;58:1079–1085.

Deane T., Nomme K., Jeffery E., Pollock C., Birol G. Development of the Statistical Reasoning in Biology Concept Inventory (SRBCI) CBE—Life Sciences Education. 2016;15:ar5. 10.1187/cbe.15-06-0131. PubMed PMC

De Ayala R. J. The theory and practice of item response theory. New York: Guilford; 2008.

Dirks C., Wenderoth M. P., Withers M. Assessment in the college science classroom. New York: Freeman; 2014.

Drabinova A., Martinková P., Zvara K. 2016. difNLR: Detection of Dichotomous Differential Item Functioning (DIF) by Non-Linear Regression Function R Package, version 1.0.0. Retrieved May 11, 2017, from https://cran.r-project.org/package=difNLR/

Fletcher T. D. 2010. psychometric: Applied Psychometric Theory, R Package Version 2.2. Retrieved May 11, 2017, from https://CRAN.R-project.org/package=psychometric.

Haladyna T. M., Downing S. M. A taxonomy of multiple-choice item-writing rules. Applied Measurement Education. 1989;2:37–50. 10.1207/s15324818ame0201_3.

Hastie T., Tibshirani R., Friedman J. The elements of statistical learning: Data mining, Inference and prediction. New York: Springer; 2009.

Heckler A. F. 2010. Concrete vs. abstract problem formats: A disadvantage of prior knowledge. Paper presented at 9th International Conference of the Learning Sciences (Chicago, IL)

Hu L., Bentler P. M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling. 1999;6:1–55.

Jorion N., Gane B. D., James K., Schroeder L., DiBello L. V., Pellegrino J. W. An analytic framework for evaluating the validity of concept inventory claims. Journal of Engineering Education. 2015;104:454–496. 10.1002/jee.20104.

Kalinowski S. T., Leonard M. J., Taper M. L. Development and validation of the Conceptual Assessment of Natural Selection (CANS) CBE—Life Sciences Education. 2016;15:ar64. 10.1187/cbe.15-06-0134. PubMed PMC

Kaminski J. A., Sloutsky V. M., Heckler A. F. The cost of concreteness: the effect of nonessential information on analogical transfer. Journal of Experimental Psychology: Applied. 2013;19:14–29. PubMed

Knight J. K., Wood W. B. W. Teaching more by lecturing less. Cell Biology Education. 2005;4:298–310. 10.1187/05-06-0082. PubMed PMC

Kuznetsova A., Brockhoff P. B., Christensen R. H. B. 2016. lmerTest: Tests in Linear Mixed Effects Models, R Package Version 2.0–30. Retrieved May 11, 2017, from https://CRAN.R-project.org/package=lmerTest.

Leonard M. J., Kalinowski S. T., Andrews T. C. Misconceptions—yesterday, today, and tomorrow. CBE—Life Sciences Education. 2014;13:179–186. 10.1187/cbe.13-12-0244. PubMed PMC

Libarkin J. 2008. Concept inventories in higher education science. Paper presented at National Research Council Promising Practices in Undergraduate STEM Education Workshop 2 (Washington, DC)

Livingston S. A. Item analysis. In: Downing S. M., Haladyna T. M., editors. Handbook of test development. London: Erlbaum; 2006. (pp. 421–441)

Magis D., Beland S., Raiche G. 2015. difR: Collection of Methods to Detect Dichotomous Differential Item Functioning (DIF), R Package, version 4.7. Retrieved May 11, 2017, from https://cran.r-project.org/package=difR/ PubMed

Maloney E. A., Sattizahn J. R., Beilock S. L. Anxiety and cognition. WIREs Cognitive Science. 2014;5:403–411. 10.1002/wcs.1299. PubMed

Martinková P., Drabinova A., Leder O., Houdek J. 2017a. ShinyItemAnalysis: Test and Item Analysis via Shiny, R Package. Retrieved May 11, 2017, from https://CRAN.R-project.org/package=ShinyItemAnalysis.

Martinková P., Drabinovà A., Liaw Y.-L., Sanders E. A., McFarland J. L., Price R. M. Checking equity: Why differential item functioning analysis should be a routine part of developing conceptual assessments. CBE—Life Sciences Education. 2017b;16:rm2. PubMed PMC

McFarland J., Wenderoth M. P., Michael J., Cliff W., Wright A., Modell H. A conceptual framework for homeostasis: development and validation. Advances in Physiology Education. 2016;40:213–222. 10.1152/advan.00103.2015. PubMed PMC

McNeil N. M., Uttal D. H., Jarvin L., Sternberg R. J. Should you show me the money? Concrete objects both hurt and help performance on mathematics problems. Learning and Instruction. 2009;19:171–184.

Michael J. In pursuit of meaningful learning. Advances in Physiology Education. 2001;25:145–158. PubMed

Michael J. Conceptual assessment in the biological sciences: a National Science Foundation-sponsored workshop. Advances in Physiology Education. 2007;31:389–391. 10.1152/advan.00047.2007. PubMed

Michael J., McFarland J. The core principles (“big ideas”) of physiology: results of faculty surveys. Advances in Physiology Education. 2011;25:336–341. 10.1152/advan.00004.2011. PubMed

Michael J. A., Modell H. I. Active learning in secondary and college science classrooms: a working model for helping the learner to learn. Mahwah, NJ: Erlbaum; 2003.

Michael J., Modell H., McFarland J., Cliff W. The “core principles” of physiology: What should students understand. Advances in Physiology Education. 2009;33:10–15. 10.1152/advan.90139.2008. PubMed

Modell H. I. How to help students understand physiology? Emphasize general models. Advances in Physiology Education. 2000;23:101–107. PubMed

Modell H., Cliff W., Michael J., McFarland J., Wenderoth M. P., Wright A. A physiologist’s view of homeostasis. Advances in Physiology Education. 2015;39:259–266. 10.1152/advan.00107.2015. PubMed PMC

Momsen J. L., Long T. M., Wyse S. A., Ebert-May D. Just the facts? Introductory undergraduate biology courses focus on low-level cognitive skills. CBE Life Sciences Education. 2010;9:435–440. 10.1187/cbe.10-01-0001. PubMed PMC

National Research Council. How people learn: brain, mind, experience, and school (expanded ed.) Washington, DC: National Academies Press; 2000.

National Research Council. A New biology for the 21st century: Ensuring the United States leads the coming biology revolution. Washington, DC: National Academies Press; 2009. PubMed

National Research Council. A framework for K–12 science education. Washington, DC: National Academies Press; 2012.

Nehm R. H., Beggrow E. P., Opfer J. E., Ha M. S. Reasoning about natural selection: Diagnosing contextual competency using the ACORNS instrument. American Biology Teacher. 2012a;74:92–98.

Nehm R. H., Ha M., Mayfield E. Transforming biology assessment with machine learning: Automated scoring of written evolutionary explanations. Journal of Science Education and Technology. 2012b;21:183–196. 10. 1007/S10956-01 1-9300-9.

Neumann I., Neumann K., Nehm R. Evaluating instrument quality in science education: Rasch-based analyses of a nature of science test. International Journal of Science Education. 2011;33:1373–1405. 10.1080/09500693.2010.511297.

Newman D. L., Snyder C. W., Fisk J. N., Wright L. K. Development of the Central Dogma Concept Inventory (CDCI) assessment tool. CBE—Life Sciences Education. 2016;15:ar9. 10.1187/cbe.15-06-0124. PubMed PMC

Nunnally J., Bernstein I. Psychometric theory. New York: McGraw-Hill; 1994.

Orlando M., Thissen D. Likelihood-based item-fit indices for dichotomous item response theory models. Applied Psychological Measurement. 2000;24:50–64.

Pollitt A., Ahmed A., Baird J.-A., Tognolini J., Davidson M. Improving the quality of GCSE assessment. Coventry, UK: Qualifications and Curriculum Authority; 2008.

Prevost L. B., Knight J., Smith M., Urban-Lurain M. 2013. Student writing reveals their heterogeneous thinking about the origin of genetic variation in populations. Talk presented at the Annual Meeting of the National Association for Research in Science Teaching, April 6–9, Rio Grande, PR, www.narst.org/annualconference/2013_NARST_Abstracts.pdf.

Price R. M., Andrews T. C., McElhinny T. L., Mead L. S., Abraham J. K., Thanukos A., Perez K. E. The Genetic Drift Inventory: A tool for measuring what advanced undergraduates have mastered about genetic drift. CBE—Life Sciences Education. 2014;13:65–75. PubMed PMC

R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2016.

Reeves T. D., Marbach-Ad G. Contemporary test validity in theory and practice: A primer for discipline-based education researchers. CBE—Life Sciences Education. 2016;15:1–9. 10.1187/cbe.15-08-0183. PubMed PMC

Revelle W. 2015. psych: Procedures for Personality and Psychological Research, Northwestern University, Evanston, Illinois, Retrieved May 11, 2017, from http://CRAN.R-project.org/package=psych.

Rizopoulos D. ltm: an R package for latent variable modelling and item response theory analyses. Journal of Statistical Software. 2006;17:1–25.

Samejima F. Estimation of reliability coefficients using the test information function and its modifications. Applied Psychological Measurement. 1994;18:229–244.

Schwarz G. Estimating the dimension of a model. Annals of Statistics. 1978;6:461–464.

Shapiro J. R., Neuberg S. L. From stereotype threat to stereotype threats: Implications of a multi-threat framework for causes, moderators, mediators, consequences, and interventions. Personality and Social Psychology Review. 2007;11:107–130. 10.1177/1088868306294790. PubMed

Shapiro J. R., Williams A. M. The role of stereotype threats in undermining girls’ and women’s performance and interest in STEM fields. Sex Roles. 2012;66:175–183. 10.1007/s11199-011-0051-0.

Smith M. K., Knight J. K. Using the Genetics Concept Assessment to document persistent conceptual difficulties in undergraduate genetics courses. Genetics. 2012;191:21–32. 10.1534/genetics.111.137810. PubMed PMC

Steele C. M., Spencer S. J., Aronson J. Contending with group image: the psychology of stereotype and social identity threat. Advances in Experimental Social Psychology. 2002;34:379–440.

Stenlund T., Eklöf H., Lyrén P. E. Group differences in test-taking behaviour: An example from a high-stakes testing program. Assessment in Education Principles, Policy, & Practice. 2017;24:4–20. 10.1080/0969594X.2016.1142935.

Torres Irribarra D., Freund R. 2014. Wright Map: IRT Item-Person Map with ConQuest Integration. Retrieved May 11, 2017, from http://github.com/david-ti/wrightmap.

Valverde G. A., Schmidt W. H. Refocusing US math and science education. Issues in Science and Technology. 1997;14:60–66.

Wei R., Simko V. 2010. corrplot: Visualization of a Correlation Matrix, R Package Version 0.77. Retrieved May 11, 2017, from https://CRAN.R-project.org/package=corrplot.

Weston M., Haudek K. C., Prevost L., Urban-Lurain M., Merrill J. Examining the impact of question surface features on students’ answers to constructed-response questions on photosynthesis. CBE—Life Sciences Education. 2015;14:ar19. PubMed PMC

Wickham H. ggplot2: elegant graphics for data analysis. New York: Springer; 2009.

Wright A., McFarland J., Cliff W., Michael J., Modell H., Wenderoth M. P. Preliminary results on the prevalence of physiology students’ homeostatic misconceptions. FASEB Journal. 2013;27:739.735.

Wright A., McFarland J., Wenderoth M. P., Michael J., Modell H., Cliff W. Knowing common misconceptions about homeostasis helps students’ learning. FASEB Journal. 2015;29:541.534.

Wright C. D., Eddy S. L., Wenderoth M. P., Abshire E., Blankenbiller M., Brownell S. E. Cognitive difficulty and format of exams predicts gender and socioeconomic gaps in exam performance of students in introductory biology courses. CBE—Life Sciences Education. 2016;15:ar23. 10.1187/cbe.15-12-0246. PubMed PMC

Zheng A. Y., Lawhorn J. K., Lumley T., Freeman S. Application of Bloom’s taxonomy debunks the “MCAT myth.”. Science. 2008;319:414. 10.1126/science.1147852. PubMed

Zumbo B. D. A handbook on the theory and methods of differential item functioning (DIF): logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense; 1999.

Checking Equity: Why Differential Item Functioning Analysis Should Be a Routine Part of Developing Conceptual Assessments