There have recently been considerable advances in our understanding of the neuronal mechanisms underlying multitasking, but the role of multimodal integration for this faculty has remained rather unclear. We examined this issue by comparing different modality combinations in a multitasking (stop-change) paradigm. In-depth neurophysiological analyses of event-related potentials (ERPs) were conducted to complement the obtained behavioral data. Specifically, we applied signal decomposition using second order blind identification (SOBI) to the multi-subject ERP data and source localization. We found that both general multimodal information integration and modality-specific aspects (potentially related to task difficulty) modulate behavioral performance and associated neurophysiological correlates. Simultaneous multimodal input generally increased early attentional processing of visual stimuli (i.e. P1 and N1 amplitudes) as well as measures of cognitive effort and conflict (i.e. central P3 amplitudes). Yet, tactile-visual input caused larger impairments in multitasking than audio-visual input. General aspects of multimodal information integration modulated the activity in the premotor cortex (BA 6) as well as different visual association areas concerned with the integration of visual information with input from other modalities (BA 19, BA 21, BA 37). On top of this, differences in the specific combination of modalities also affected performance and measures of conflict/effort originating in prefrontal regions (BA 6).

Cognitive Neurophysiology Department of Child and Adolescent Psychiatry Carl Gustav Carus Faculty of Medicine TU Dresden Germany

Department of Psychology University of Oslo Oslo Norway

Experimental Neurobiology National Institute of Mental Health Klecany Czech Republic

Zobrazit více v PubMed

Beste C, Yildiz A, Meissner TW, Wolf OT. Stress improves task processing efficiency in dual-tasks. Behav. Brain Res. 2013;252C:260–265. doi: 10.1016/j.bbr.2013.06.013. PubMed DOI

Wu C, Liu Y. Queuing network modeling of the psychological refractory period (PRP) Psychol. Rev. 2008;115:913–954. doi: 10.1037/a0013123. PubMed DOI

Yildiz A, Beste C. Parallel and serial processing in dual-tasking differentially involves mechanisms in the striatum and the lateral prefrontal cortex. Brain Struct. Funct. 2015;220:3131–3142. doi: 10.1007/s00429-014-0847-0. PubMed DOI

Gohil K, Stock A-K, Beste C. The importance of sensory integration processes for action cascading. Sci. Rep. 2015;5:9485. doi: 10.1038/srep09485. PubMed DOI PMC

Mückschel M, Stock A-K, Beste C. Psychophysiological mechanisms of interindividual differences in goal activation modes during action cascading. Cereb. Cortex N. Y. N 1991. 2014;24:2120–2129. PubMed

Gohil K, Dippel G, Beste C. Questioning the role of the frontopolar cortex in multi-component behavior–a TMS/EEG study. Sci. Rep. 2016;6:22317. doi: 10.1038/srep22317. PubMed DOI PMC

Gohil K, Hahne A, Beste C. Improvements of sensorimotor processes during action cascading associated with changes in sensory processing architecture-insights from sensory deprivation. Sci. Rep. 2016;6:28259. doi: 10.1038/srep28259. PubMed DOI PMC

Göthe K, Oberauer K, Kliegl R. Eliminating dual-task costs by minimizing crosstalk between tasks: The role of modality and feature pairings. Cognition. 2016;150:92–108. doi: 10.1016/j.cognition.2016.02.003. PubMed DOI

Stock A-K, Gohil K, Beste C. Age-related differences in task goal processing strategies during action cascading. Brain Struct. Funct. 2015 PubMed

Hazeltine E, Ruthruff E. Modality pairing effects and the response selection bottleneck. Psychol. Res. 2006;70:504–513. doi: 10.1007/s00426-005-0017-3. PubMed DOI

Stelzel C, Schumacher EH, Schubert T, D’Esposito M. The neural effect of stimulus-response modality compatibility on dual-task performance: an fMRI study. Psychol. Res. 2006;70:514–525. doi: 10.1007/s00426-005-0013-7. PubMed DOI

Stelzel C, Schubert T. Interference effects of stimulus-response modality pairings in dual tasks and their robustness. Psychol. Res. 2011;75:476–490. doi: 10.1007/s00426-011-0368-x. PubMed DOI

Luck SJ, Woodman GF, Vogel EK. Event-related potential studies of attention. Trends Cogn. Sci. 2000;4:432–440. doi: 10.1016/S1364-6613(00)01545-X. PubMed DOI

Stock A-K, Arning L, Epplen JT, Beste C. DRD1 and DRD2 Genotypes Modulate Processing Modes of Goal Activation Processes during Action Cascading. J. Neurosci. 2014;34:5335–5341. doi: 10.1523/JNEUROSCI.5140-13.2014. PubMed DOI PMC

Verleger R, Jaśkowski P, Wascher E. Evidence for an integrative role of P3b in linking reaction to perception. J. Psychophysiol. 2005;19:165–181. doi: 10.1027/0269-8803.19.3.165. DOI

Huster RJ, Plis SM, Calhoun VD. Group-level component analyses of EEG: validation and evaluation. Front. Neurosci. 2015;9:254. doi: 10.3389/fnins.2015.00254. PubMed DOI PMC

Nunez PL, et al. EEG coherency. I: Statistics, reference electrode, volume conduction, Laplacians, cortical imaging, and interpretation at multiple scales. Electroencephalogr. Clin. Neurophysiol. 1997;103:499–515. doi: 10.1016/S0013-4694(97)00066-7. PubMed DOI

Winter WR, Nunez PL, Ding J, Srinivasan R. Comparison of the effect of volume conduction on EEG coherence with the effect of field spread on MEG coherence. Stat. Med. 2007;26:3946–3957. doi: 10.1002/sim.2978. PubMed DOI

Eichele T, Rachakonda S, Brakedal B, Eikeland R, Calhoun VD. EEGIFT: group independent component analysis for event-related EEG data. Comput. Intell. Neurosci. 2011;2011:129365. doi: 10.1155/2011/129365. PubMed DOI PMC

Verbruggen F, Schneider DW, Logan GD. How to stop and change a response: the role of goal activation in multitasking. J. Exp. Psychol. Hum. Percept. Perform. 2008;34:1212–1228. doi: 10.1037/0096-1523.34.5.1212. PubMed DOI

Musacchia G, Arum L, Nicol T, Garstecki D, Kraus N. Audiovisual deficits in older adults with hearing loss: biological evidence. Ear Hear. 2009;30:505–514. doi: 10.1097/AUD.0b013e3181a7f5b7. PubMed DOI

Tomita H, Fujiwara K. Effects of allocation of visuo-spatial attention to visual stimuli triggering unilateral arm abduction on anticipatory postural control. Clin. Neurophysiol. 2008;119:2086–2097. doi: 10.1016/j.clinph.2008.05.001. PubMed DOI

Yaguchi C, Fujiwara K. Effects of attentional dispersion on sensory-motor processing of anticipatory postural control during unilateral arm abduction. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 2012;123:1361–1370. doi: 10.1016/j.clinph.2011.10.045. PubMed DOI

Talsma D, Senkowski D, Woldorff MG. Intermodal attention affects the processing of the temporal alignment of audiovisual stimuli. Exp. Brain Res. 2009;198:313–328. doi: 10.1007/s00221-009-1858-6. PubMed DOI PMC

Catani, M. & Thiebaut de Schotten, M. Atlas of human brain connections. (Oxford University Press, 2012).

Keil J, Pomper U, Senkowski D. Distinct patterns of local oscillatory activity and functional connectivity underlie intersensory attention and temporal prediction. Cortex J. Devoted Study Nerv. Syst. Behav. 2016;74:277–288. doi: 10.1016/j.cortex.2015.10.023. PubMed DOI

Mesulam MM. From sensation to cognition. Brain J. Neurol. 1998;121(Pt 6):1013–1052. doi: 10.1093/brain/121.6.1013. PubMed DOI

Gohel B, Jeong Y. Sensory modality-specific spatio-temporal dynamics in response to counting tasks. Neurosci. Lett. 2014;581:20–25. doi: 10.1016/j.neulet.2014.08.015. PubMed DOI

Hidaka S, Teramoto W, Sugita Y. Spatiotemporal Processing in Crossmodal Interactions for Perception of the External World: A Review. Front. Integr. Neurosci. 2015;9:62. doi: 10.3389/fnint.2015.00062. PubMed DOI PMC

Hanson JVM, Whitaker D, Heron J. Preferential processing of tactile events under conditions of divided attention. Neuroreport. 2009;20:1392–1396. doi: 10.1097/WNR.0b013e3283319e25. PubMed DOI PMC

Dippel G, Chmielewski W, Mückschel M, Beste C . Response mode-dependent differences in neurofunctional networks during response inhibition: an EEG-beamforming study. Brain Struct. Funct. 2016;221:4091–4101. doi: 10.1007/s00429-015-1148-y. PubMed DOI

Schneider D, Beste C, Wascher E. Attentional capture by irrelevant transients leads to perceptual errors in a competitive change detection task. Front. Psychol. 2012;3:164. doi: 10.3389/fpsyg.2012.00164. PubMed DOI PMC

Schneider D, Beste C, Wascher E. On the time course of bottom-up and top-down processes in selective visual attention: an EEG study. Psychophysiology. 2012;49:1492–1503. doi: 10.1111/j.1469-8986.2012.01462.x. PubMed DOI

Wascher E, Schneider D, Hoffmann S, Beste C, Sänger J. When compensation fails: attentional deficits in healthy ageing caused by visual distraction. Neuropsychologia. 2012;50:3185–3192. doi: 10.1016/j.neuropsychologia.2012.09.033. PubMed DOI

Botvinick MM. Conflict monitoring and decision making: reconciling two perspectives on anterior cingulate function. Cogn. Affect. Behav. Neurosci. 2007;7:356–366. doi: 10.3758/CABN.7.4.356. PubMed DOI

Botvinick MM, Cohen JD, Carter CS. Conflict monitoring and anterior cingulate cortex: an update. Trends Cogn. Sci. 2004;8:539–546. doi: 10.1016/j.tics.2004.10.003. PubMed DOI

Mesulam MM. Large-scale neurocognitive networks and distributed processing for attention, language, and memory. Ann. Neurol. 1990;28:597–613. doi: 10.1002/ana.410280502. PubMed DOI

Ffytche DH, Blom JD, Catani M. Disorders of visual perception. J. Neurol. Neurosurg. Psychiatry. 2010;81:1280–1287. doi: 10.1136/jnnp.2008.171348. PubMed DOI

Quinn BT, et al. Intracranial cortical responses during visual-tactile integration in humans. J. Neurosci. Off. J. Soc. Neurosci. 2014;34:171–181. doi: 10.1523/JNEUROSCI.0532-13.2014. PubMed DOI PMC

ffytche DH. Neural codes for conscious vision. Trends Cogn. Sci. 2002;6:493–495. doi: 10.1016/S1364-6613(02)02029-6. PubMed DOI

Büchel C, Price C, Friston K. A multimodal language region in the ventral visual pathway. Nature. 1998;394:274–277. doi: 10.1038/28389. PubMed DOI

Meredith MA. On the neuronal basis for multisensory convergence: a brief overview. Brain Res. Cogn. Brain Res. 2002;14:31–40. doi: 10.1016/S0926-6410(02)00059-9. PubMed DOI

Bluschke A, Roessner V, Beste C. Specific cognitive-neurophysiological processes predict impulsivity in the childhood attention-deficit/hyperactivity disorder combined subtype. Psychol. Med. 2016;46:1277–1287. doi: 10.1017/S0033291715002822. PubMed DOI

Cornoldi C, et al. Working memory interference control deficit in children referred by teachers for ADHD symptoms. Child Neuropsychol. J. Norm. Abnorm. Dev. Child. Adolesc. 2001;7:230–240. PubMed

Carmona S, et al. Sensation-to-cognition cortical streams in attention-deficit/hyperactivity disorder. Hum. Brain Mapp. 2015;36:2544–2557. doi: 10.1002/hbm.22790. PubMed DOI PMC

Bluschke A, Chmielewski WX, Roessner V, Beste C. Intact Context-Dependent Modulation of Conflict Monitoring in Childhood ADHD. J. Atten. Disord. 2016 PubMed

de Jong JJ, Hodiamont PPG, de Gelder B. Modality-specific attention and multisensory integration of emotions in schizophrenia: reduced regulatory effects. Schizophr. Res. 2010;122:136–143. doi: 10.1016/j.schres.2010.04.010. PubMed DOI

Mayer AR, et al. An fMRI study of multimodal selective attention in schizophrenia. Br. J. Psychiatry J. Ment. Sci. 2015;207:420–428. doi: 10.1192/bjp.bp.114.155499. PubMed DOI PMC

Logan GD, Cowan WB. On the ability to inhibit thought and action: A theory of an act of control. Psychol. Rev. 1984;91:295–327. doi: 10.1037/0033-295X.91.3.295. PubMed DOI