Bridging Big Data: Procedures for Combining Non-equivalent Cognitive Measures from the ENIGMA Consortium
Status PubMed-not-MEDLINE Jazyk angličtina Země Spojené státy americké Médium electronic
Typ dokumentu preprinty, časopisecké články
Grantová podpora
I01 RX003444
RRD VA - United States
T32 MH019535
NIMH NIH HHS - United States
R01 EB006841
NIBIB NIH HHS - United States
R25 MH080663
NIMH NIH HHS - United States
U54 EB020403
NIBIB NIH HHS - United States
M01 RR001066
NCRR NIH HHS - United States
R01 EB000840
NIBIB NIH HHS - United States
R56 AG058854
NIA NIH HHS - United States
R01 MH121246
NIMH NIH HHS - United States
P50 MH115846
NIMH NIH HHS - United States
RC1 MH089257
NIMH NIH HHS - United States
K08 MH068540
NIMH NIH HHS - United States
UL1 RR033176
NCRR NIH HHS - United States
P41 EB015922
NIBIB NIH HHS - United States
P20 GM103472
NIGMS NIH HHS - United States
IK2 RX002922
RRD VA - United States
R01 NS100973
NINDS NIH HHS - United States
R01 HD061504
NICHD NIH HHS - United States
R01 AG064955
NIA NIH HHS - United States
R01 EB005846
NIBIB NIH HHS - United States
R01 MH094524
NIMH NIH HHS - United States
R01 NS122827
NINDS NIH HHS - United States
R01 MH111671
NIMH NIH HHS - United States
P01 AG055367
NIA NIH HHS - United States
R01 MH116147
NIMH NIH HHS - United States
UL1 TR000124
NCATS NIH HHS - United States
P41 RR014075
NCRR NIH HHS - United States
K23 MH095661
NIMH NIH HHS - United States
R01 MH083553
NIMH NIH HHS - United States
R01 MH113827
NIMH NIH HHS - United States
R01 NS122184
NINDS NIH HHS - United States
R01 AG060470
NIA NIH HHS - United States
R01 NS124585
NINDS NIH HHS - United States
R01 AG076838
NIA NIH HHS - United States
U24 RR021992
NCRR NIH HHS - United States
R61 NS120249
NINDS NIH HHS - United States
F32 NS119285
NINDS NIH HHS - United States
U01 MH124639
NIMH NIH HHS - United States
R01 AG050595
NIA NIH HHS - United States
PubMed
36712107
PubMed Central
PMC9882238
DOI
10.1101/2023.01.16.524331
PII: 2023.01.16.524331
Knihovny.cz E-zdroje
- Klíčová slova
- Harmonization, Mega analysis, Tools, Verbal learning,
- Publikační typ
- časopisecké články MeSH
- preprinty MeSH
Investigators in neuroscience have turned to Big Data to address replication and reliability issues by increasing sample sizes, statistical power, and representativeness of data. These efforts unveil new questions about integrating data arising from distinct sources and instruments. We focus on the most frequently assessed cognitive domain - memory testing - and demonstrate a process for reliable data harmonization across three common measures. We aggregated global raw data from 53 studies totaling N = 10,505 individuals. A mega-analysis was conducted using empirical bayes harmonization to remove site effects, followed by linear models adjusting for common covariates. A continuous item response theory (IRT) model estimated each individual's latent verbal learning ability while accounting for item difficulties. Harmonization significantly reduced inter-site variance while preserving covariate effects, and our conversion tool is freely available online. This demonstrates that large-scale data sharing and harmonization initiatives can address reproducibility and integration challenges across the behavioral sciences.
3rd Faculty of Medicine Charles University Prague Czech Republic
Brain Imaging and Analysis Center Duke University Durham NC
Brain Research Institute UCLA Los Angeles CA USA
British Columbia Mental Health and Substance Use Services Research Institute Vancouver BC Canada
Center for Autism Research Kessler Foundation East Hanover NJ USA
Center for Behavior Genetics of Aging University of California San Diego La Jolla CA USA
Center for Multimodal Imaging and Genetics UC San Diego San Diego CA USA
Center for Traumatic Brain Injury Kessler Foundation East Hanover NJ
Center of Brain Behaviour and Metabolism University of Lübeck Lübeck Germany
Centro Investigación Biomédica en Red Salud Mental Madrid Spain
Cognitive Health Initiative Central Clinical School Monash University Melbourne VIC Australia
Cognitive Neuroscience Unit School of Psychology Deakin University Geelong Australia
Department of Biological and Medical Psychology University of Bergen Bergen Norway
Department of Biomedicine University of Bergen Bergen Norway
Department of Electrical and Electronics Engineering Antalya Bilim University Antalya Turkey
Department of Medicine Royal Melbourne Hospital Melbourne VIC Australia
Department of Medicine University of Otago Christchurch New Zealand
Department of Neurobiology and Anatomy Wake Forest School of Medicine Winston Salem NC
Department of Neurology Alfred Health Melbourne Australia
Department of Neurology Hershey Medical Center State College PA 16801
Department of Neurology Icahn School of Medicine at Mount Sinai New York NY
Department of Neurology Te Whatu Ora Health New Zealand Waitaha Canterbury Christchurch New Zealand
Department of Neurology Uniformed Services University Bethesda MD
Department of Neurology University of Utah School of Medicine Salt Lake City UT 84132
Department of Neuropsychiatry Seoul National University Hospital Seoul South Korea
Department of Neuroscience Monash University Melbourne Australia
Department of Neuroscience Uniformed Services University of the Health Sciences Bethesda MD USA
Department of Neurosurgery David Geffen School of Medicine at UCLA Los Angeles CA USA
Department of Paediatrics The University of Melbourne Parkville VIC Australia
Department of Pediatrics Division of Neurology UCLA Mattel Children's Hospital Los Angeles CA USA
Department of Physical Medicine and Rehabilitation Rutgers New Jersey Medical School Newark NJ USA
Department of Physical Medicine and Rehabilitation Virginia Commonwealth University Richmond VA
Department of Psychiatry and Psychotherapy University of Marburg Marburg Germany
Department of Psychiatry Dalhousie University Halifax Canada
Department of Psychiatry Seoul National University College of Medicine Seoul South Korea
Department of Psychiatry University of British Columbia Vancouver BC Canada
Department of Psychiatry University of California San Diego La Jolla CA USA
Department of Psychiatry University of Pittsburgh Pittsburgh PA USA
Department of Psychology Georgia State University Atlanta GA
Department of Psychology Norwegian University of Science and Technology Trondheim Norway
Department of Psychology Penn State University State College PA 16801
Department of Psychology Phoenix VA Health Care System Phoenix AZ USA
Department of Psychology UCLA Los Angeles CA USA
Department of Psychology University of Oregon Eugene OR USA
Department of Psychology University of Oslo Oslo Norway
Department of Radiation Medicine and Applied Sciences and Psychiatry UC San Diego La Jolla CA USA
Division of Endocrinology Emory University School of Medicine Atlanta GA USA
Division of Epidemiology University of Utah Salt Lake City UT 84132
Division of Psychiatry Haukeland University Hospital Bergen Norway
Faculty of Computer Science Dalhousie University Halifax NS Canada
FIDMAG Research Foundation Barcelona Spain
George E Wahlen Veterans Affairs Medical Center Salt Lake City UT 84132
H Ben Taub Department of Physical Medicine and Rehabilitation Baylor College of Medicine Houston TX
Hospital Universitari Institut Pere Mata Tarragona Spain
Institut d'Investiació Sanitària Pere Virgili CERCA Universitat Rovira i Virgili Tarragona Spain
Institute D'Or for Research and Education São Paulo Brazil
Institute for Translational Neuroscience University of Münster Münster Germany
Institute for Translational Psychiatry University of Münster Germany
Kessler Foundation East Hanover NJ USA
KG Jebsen Center for Neurodevelopmental Disorders University of Oslo Oslo Norway
Laboratory of Neuropsychiatry Santa Lucia Foundation IRCCS Rome Italy
Michael E DeBakey Veterans Affairs Medical Center Houston TX
Minneapolis VA Health Care System Minneapolis MN
National Institute of Mental Health Klecany Czech Republic
National Intrepid Center of Excellence Walter Reed National Military Medical Center Bethesda MD
Neuroscience Institute and Department of Psychology Georgia State University Atlanta GA USA
Neurosurgery Stanford School of Medicine Stanford CA USA
New Zealand Brain Research Institute Christchurch New Zealand
NorHEAD Norwegian Centre for Headache Research Trondheim Norway
NORMENT Division of Mental Health and Addiction Oslo University Hospital Oslo Norway
Psychiatry and Behavioral Health Ohio State Wexner Medical Center Columbus OH USA
Rehabilitation Medicine Department National Institutes of Health Clinical Center Bethesda MD
Richmond Veterans Affairs Medical Center Central Virginia VA Health Care System Richmond VA
Rocco Ortenzio Neuroimaging Center Kessler Foundation East Hanover NJ USA
School of Psychology Speech and Hearing University of Canterbury Christchurch New Zealand
Section of Pediatrics The Ohio State University College of Medicine Columbus Ohio USA
Social Life and Engineering Science Imaging Center Penn State University State College PA 16801
Sunnaas Rehabilitation Hospital Nesodden Norway
The Florey Institute of Neuroscience and Mental Health Melbourne Australia
UCLA Steve Tisch BrainSPORT Program Los Angeles CA USA
University of Basel Department of Psychiatry Basel Switzerland
VA Mid Atlantic Mental Illness Research Education and Clinical Center Durham NC
VISN 6 MIRECC Durham VA Durham NC
W G Hefner VA Medical Center Salisbury NC
WRIISC WOMEN and Rehabilitation Department VA Palo Alto Palo Alto CA USA
Zobrazit více v PubMed
Thompson P. M., Jahanshad N., Ching C. R. K., Salminen L. E., Thomopoulos S. I., Bright J., Baune B. T., Bertolín S., Bralten J., Bruin W. B., Bülow R., Chen J., Chye Y., Dannlowski U., de Kovel C. G. F., Donohoe G., Eyler L. T., Faraone S. V., Favre P., Filippi C. A., Frodl T., Garijo D., Gil Y., Grabe H. J., Grasby K. L., Hajek T., Han L. K. M., Hatton S. N., Hilbert K., Ho T. C., Holleran L., Homuth G., Hosten N., Houenou J., Ivanov I., Jia T., Kelly S., Klein M., Kwon J. S., Laansma M. A., Leerssen J., Lueken U., Nunes A., Neill J. O., Opel N., Piras F., Piras F., Postema M. C., Pozzi E., Shatokhina N., Soriano-Mas C., Spalletta G., Sun D., Teumer A., Tilot A. K., Tozzi L., van der Merwe C., Van Someren E. J. W., van Wingen G. A., Völzke H., Walton E., Wang L., Winkler A. M., Wittfeld K., Wright M. J., Yun J.-Y., Zhang G., Zhang-James Y., Adhikari B. M., Agartz I., Aghajani M., Aleman A., Althoff R. R., Altmann A., Andreassen O. A., Baron D. A., Bartnik-Olson B. L., Marie Bas-Hoogendam J., Baskin-Sommers A. R., Bearden C. E., Berner L. A., Boedhoe P. S. W., Brouwer R. M., Buitelaar J. K., Caeyenberghs K., Cecil C. A. M., Cohen R. A., Cole J. H., Conrod P. J., De Brito S. A., de Zwarte S. M. C., Dennis E. L., Desrivieres S., Dima D., Ehrlich S., Esopenko C., Fairchild G., Fisher S. E., Fouche J.-P., Francks C., Frangou S., Franke B., Garavan H. P., Glahn D. C., Groenewold N. A., Gurholt T. P., Gutman B. A., Hahn T., Harding I. H., Hernaus D., Hibar D. P., Hillary F. G., Hoogman M., Hulshoff Pol H. E., Jalbrzikowski M., Karkashadze G. A., Klapwijk E. T., Knickmeyer R. C., Kochunov P., Koerte I. K., Kong X.-Z., Liew S.-L., Lin A. P., Logue M. W., Luders E., Macciardi F., Mackey S., Mayer A. R., McDonald C. R., McMahon A. B., Medland S. E., Modinos G., Morey R. A., Mueller S. C., Mukherjee P., Namazova-Baranova L., Nir T. M., Olsen A., Paschou P., Pine D. S., Pizzagalli F., Rentería M. E., Rohrer J. D., Sämann P. G., Schmaal L., Schumann G., Shiroishi M. S., Sisodiya S. M., Smit D. J. A., Sønderby I. E., Stein D. J., Stein J. L., Tahmasian M., Tate D. F., Turner J. A., van den Heuvel O. A., van der Wee N. J. A., van der Werf Y. D., van Erp T. G. M., van Haren N. E. M., van Rooij D., van Velzen L. S., Veer I. M., Veltman D. J., Villalon-Reina J. E., Walter H., Whelan C. D., Wilde E. A., Zarei M., Zelman V., ENIGMA Consortium, ENIGMA and global neuroscience: A decade of large-scale studies of the brain in health and disease across more than 40 countries. Transl. Psychiatry. 10, 100 (2020). PubMed PMC
Nagaraj A., Shears E., de Vaan M., Improving data access democratizes and diversifies science. Proc. Natl. Acad. Sci. U. S. A. 117, 23490–23498 (2020). PubMed PMC
Rajtmajer S. M., Errington T. M., Hillary F. G., How failure to falsify in high-volume science contributes to the replication crisis. Elife. 11, e78830 (2022). PubMed PMC
Kennedy E., Dennis E. L., Lindsey H. M., deRoon-Cassini T., Du Plessis S., Fani N., Kaufman M. L., Koen N., Larson C. L., Laskowitz S., Lebois L. A. M., Morey R. A., Newsome M. R., Palermo C., Pastorek N. J., Powers A., Scheibel R., Seedat S., Seligowski A., Stein D., Stevens J., Sun D., Thompson P., Troyanskaya M., van Rooij S. J. H., Watts A., Weis C. N., Williams W., Hillary F. G., Pugh M. J., Wilde E. A., Tate D. F., Harmonizing PTSD Severity Scales Across Instruments and Sites. Neuropsychology. Accepted. PubMed PMC
Pan F.-F., Huang L., Chen K.-L., Zhao Q.-H., Guo Q.-H., A comparative study on the validations of three cognitive screening tests in identifying subtle cognitive decline. BMC Neurol. 20, 78 (2020). PubMed PMC
Roalf D. R., Moore T. M., Mechanic-Hamilton D., Wolk D. A., Arnold S. E., Weintraub D. A., Moberg P. J., Bridging cognitive screening tests in neurologic disorders: A crosswalk between the short Montreal Cognitive Assessment and Mini-Mental State Examination. Alzheimers. Dement. 13, 947–952 (2017). PubMed PMC
Woods S., Delis D., Scott J., Kramer J., Holdnack J., The California Verbal Learning Test – second edition: Test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Archives of Clinical Neuropsychology. 21 (2006), pp. 413–420. PubMed
Guilmette T. J., Rasile D., Sensitivity, specificity, and diagnostic accuracy of three verbal memory measures in the assessment of mild brain injury. Neuropsychology. 9, 338–344 (1995).
Spreen O., Strauss E., A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary (Oxford University Press, USA, 1998).
Cardenas S. A., Kassem L., Brotman M. A., Leibenluft E., McMahon F. J., Neurocognitive functioning in euthymic patients with bipolar disorder and unaffected relatives: A review of the literature. Neurosci. Biobehav. Rev. 69, 193–215 (2016). PubMed PMC
Jennett B., Teasdale G., Wechsler D. A standardized memory scale for clinical use. J. Psychol.
Thiruselvam I., Hoelzle J. B., Refined measurement of verbal learning and memory: Application of item response theory to California Verbal Learning Test - Second Edition (CVLT-II) learning trials. Arch. Clin. Neuropsychol. 35, 90–104 (2020). PubMed
Pomponio R., Erus G., Habes M., Doshi J., Srinivasan D., Mamourian E., Bashyam V., Nasrallah I. M., Satterthwaite T. D., Fan Y., Launer L. J., Masters C. L., Maruff P., Zhuo C., Völzke H., Johnson S. C., Fripp J., Koutsouleris N., Wolf D. H., Gur R., Gur R., Morris J., Albert M. S., Grabe H. J., Resnick S. M., Bryan R. N., Wolk D. A., Shinohara R. T., Shou H., Davatzikos C., Harmonization of large MRI datasets for the analysis of brain imaging patterns throughout the lifespan. Neuroimage. 208, 116450 (2020). PubMed PMC
Radua J., Vieta E., Shinohara R., Kochunov P., Quidé Y., Green M. J., Weickert C. S., Weickert T., Bruggemann J., Kircher T., Nenadić I., Cairns M. J., Seal M., Schall U., Henskens F., Fullerton J. M., Mowry B., Pantelis C., Lenroot R., Cropley V., Loughland C., Scott R., Wolf D., Satterthwaite T. D., Tan Y., Sim K., Piras F., Spalletta G., Banaj N., Pomarol-Clotet E., Solanes A., Albajes-Eizagirre A., Canales-Rodríguez E. J., Sarro S., Di Giorgio A., Bertolino A., Stäblein M., Oertel V., Knöchel C., Borgwardt S., du Plessis S., Yun J.-Y., Kwon J. S., Dannlowski U., Hahn T., Grotegerd D., Alloza C., Arango C., Janssen J., Díaz-Caneja C., Jiang W., Calhoun V., Ehrlich S., Yang K., Cascella N. G., Takayanagi Y., Sawa A., Tomyshev A., Lebedeva I., Kaleda V., Kirschner M., Hoschl C., Tomecek D., Skoch A., van Amelsvoort T., Bakker G., James A., Preda A., Weideman A., Stein D. J., Howells F., Uhlmann A., Temmingh H., López-Jaramillo C., Díaz-Zuluaga A., Fortea L., Martinez-Heras E., Solana E., Llufriu S., Jahanshad N., Thompson P., Turner J., van Erp T., ENIGMA Consortium collaborators, Increased power by harmonizing structural MRI site differences with the ComBat batch adjustment method in ENIGMA. Neuroimage. 218, 116956 (2020). PubMed PMC
Wilde E. A., Dennis E. L., Tate D. F., The ENIGMA Brain Injury Working Group: Approach, Challenges, and Potential Benefits. Brain Imaging Behav. Under Review (2019). PubMed PMC
Coryn C., Hobson K., McCowen R., Meta-analysis as a method of multi-site evaluation: An example from international development. Evaluation Journal of Australasia. 15, 4–14 (2015).
Cifu D. X., Dixon K. J., Chronic effects of neurotrauma consortium. Brain Inj. 30, 1397–1398 (2016). PubMed
Schoenberg M. R., Dawson K. A., Duff K., Patton D., Scott J. G., Adams R. L., Test performance and classification statistics for the Rey Auditory Verbal Learning Test in selected clinical samples. Arch. Clin. Neuropsychol. 21, 693–703 (2006). PubMed
Benedict R. H. B., Schretlen D., Groninger L., Brandt J., Hopkins Verbal Learning Test – Revised: Normative Data and Analysis of Inter-Form and Test-Retest Reliability. The Clinical Neuropsychologist. 12 (1998), pp. 43–55.
Priestley D. R., Staph J., Koneru S. D., Rajtmajer S. M., Cwiek A., Vervoordt S., Hillary F. G., Establishing ground truth in the traumatic brain injury literature: if replication is the answer, then what are the questions? Brain Commun. 5, fcac322 (2023). PubMed PMC
Naim M., Katkov M., Romani S., Tsodyks M., Fundamental Law of Memory Recall. Phys. Rev. Lett. 124, 018101 (2020). PubMed
Barwegen K., Resch Z. J., Ovsiew G. P., Jennette K. J., Soble J. R., A-232 Head-To-Head Comparison of the Rey Auditory Verbal Learning Test Effort Score and Forced Choice Embedded Performance Validity Indicators Among Patients with and without Verbal Memory Impairment. Arch. Clin. Neuropsychol. 37, 1388–1388 (2022).
Wang T., Zeng L., Item Parameter Estimation for a Continuous Response Model Using an EM Algorithm. Appl. Psychol. Meas. 22, 333–344 (1998).
Hastings J., Frishkoff G. A., Smith B., Jensen M., Poldrack R. A., Lomax J., Bandrowski A., Imam F., Turner J. A., Martone M. E., Interdisciplinary perspectives on the development, integration, and application of cognitive ontologies. Front. Neuroinform. 8, 62 (2014). PubMed PMC
Pugh M. J., Kennedy E., Prager E. M., Humpherys J., Dams-O’Connor K., Hack D., McCafferty M. K., Wolfe J., Yaffe K., McCrea M., Ferguson A. R., Lancashire L., Ghajar J., Lumba-Brown A., Phenotyping the Spectrum of Traumatic Brain Injury: A Review and Pathway to Standardization. J. Neurotrauma. 38, 3222–3234 (2021). PubMed PMC
Shapiro A. M., Benedict R. H., Schretlen D., Brandt J., Construct and concurrent validity of the Hopkins Verbal Learning Test-revised. Clin. Neuropsychol. 13, 348–358 (1999). PubMed
Toga A. W., Crawford K. L., The Alzheimer’s Disease Neuroimaging Initiative informatics core: A decade in review. Alzheimers. Dement. 11, 832–839 (2015). PubMed PMC
Schmidt A., Crossley N. A., Harrisberger F., Smieskova R., Lenz C., Riecher-Rössler A., Lang U. E., McGuire P., Fusar-Poli P., Borgwardt S., Structural Network Disorganization in Subjects at Clinical High Risk for Psychosis. Schizophrenia Bulletin (2016), p. sbw110. PubMed PMC
Lundervold A. J., Halleland H. B., Brevik E. J., Haavik J., Sørensen L., Verbal Memory Function in Intellectually Well-Functioning Adults With ADHD: Relations to Working Memory and Response Inhibition. Journal of Attention Disorders. 23 (2019), pp. 1188–1198. PubMed
Petrovsky N., Quednow B. B., Ettinger U., Schmechtig A., Mössner R., Collier D. A., Kühn K.-U., Maier W., Wagner M., Kumari V., Sensorimotor Gating is Associated with CHRNA3 Polymorphisms in Schizophrenia and Healthy Volunteers. Neuropsychopharmacology. 35 (2010), pp. 1429–1439. PubMed PMC
Broglio S. P., McAllister T., Katz B. P., LaPradd M., Zhou W., McCrea M. A., CARE Consortium Investigators, The Natural History of Sport-Related Concussion in Collegiate Athletes: Findings from the NCAA-DoD CARE Consortium. Sports Med. 52, 403–415 (2022). PubMed
Logue M. W., van Rooij S. J. H., Dennis E. L., Davis S. L., Hayes J. P., Stevens J. S., Densmore M., Haswell C. C., Ipser J., Koch S. B. J., Korgaonkar M., Lebois L. A. M., Peverill M., Baker J. T., Boedhoe P. S. W., Frijling J. L., Gruber S. A., Harpaz-Rotem I., Jahanshad N., Koopowitz S., Levy I., Nawijn L., O’Connor L., Olff M., Salat D. H., Sheridan M. A., Spielberg J. M., van Zuiden M., Winternitz S. R., Wolff J. D., Wolf E. J., Wang X., Wrocklage K., Abdallah C. G., Bryant R. A., Geuze E., Jovanovic T., Kaufman M. L., King A. P., Krystal J. H., Lagopoulos J., Bennett M., Lanius R., Liberzon I., McGlinchey R. E., McLaughlin K. A., Milberg W. P., Miller M. W., Ressler K. J., Veltman D. J., Stein D. J., Thomaes K., Thompson P. M., Morey R. A., Smaller Hippocampal Volume in Posttraumatic Stress Disorder: A Multisite ENIGMA-PGC Study: Subcortical Volumetry Results From Posttraumatic Stress Disorder Consortia. Biol. Psychiatry. 83, 244–253 (2018). PubMed PMC
Kennedy E., Panahi S., Stewart I. J., Tate D. F., Wilde E. A., Kenney K., Werner J. K., Gill J., Diaz-Arrastia R., Amuan M., Van Cott A. C., Pugh M. J., Traumatic Brain Injury and Early Onset Dementia in Post 9–11 Veterans. Brain Inj. 36, 620–627 (2022). PubMed PMC
Nasir N. S., Hand V. M., Exploring Sociocultural Perspectives on Race, Culture, and Learning. Rev. Educ. Res. 76, 449–475 (2006).
Zopluoglu C., EstCRM: An R package for Samejima’s continuous IRT model. Appl. Psychol. Meas. 36, 149 (2012).
Shojima K., A noniterative item parameter solution in each EM cycle of the continuous response model. Educ. Technol. Res. Dev. 28, 11–22 (2005).
Samejima F., Homogeneous case of the continuous response model. Psychometrika. 38, 203–219 (1973).