Figure-ground organisation is a perceptual grouping mechanism for detecting objects and boundaries, essential for an agent interacting with the environment. Current figure-ground segmentation methods rely on classical computer vision or deep learning, requiring extensive computational resources, especially during training. Inspired by the primate visual system, we developed a bio-inspired perception system for the neuromorphic robot iCub. The model uses a hierarchical, biologically plausible architecture and event-driven vision to distinguish foreground objects from the background. Unlike classical approaches, event-driven cameras reduce data redundancy and computation. The system has been qualitatively and quantitatively assessed in simulations and with event-driven cameras on iCub in various scenarios. It successfully segments items in diverse real-world settings, showing comparable results to its frame-based version on simple stimuli and the Berkeley Segmentation dataset. This model enhances hybrid systems, complementing conventional deep learning models by processing only relevant data in Regions of Interest (ROI), enabling low-latency autonomous robotic applications.

Istituto Italiano di Tecnologia Event Driven Perception for Robotics Genoa Italy

Johns Hopkins University Mind Brain Institute Baltimore Maryland USA

MSCA Fellow at The Czech Technical University Prague Prague Czechia

Zobrazit více v PubMed

Craft, E., Schütze, H., Niebur, E. & von der Heydt, R. A neural model of figure–ground organization. PubMed DOI

Bregman, A. S. et al.

Pawluk, D., Kitada, R., Abramowicz, A., Hamilton, C. & Lederman, S. J. Figure/ground segmentation via a haptic glance: attributing initial finger contacts to objects or their supporting surfaces. PubMed DOI

Hu, B., Kane-Jackson, R. & Niebur, E. A proto-object based saliency model in three-dimensional space. PubMed DOI PMC

Wagemans, J. et al. A century of gestalt psychology in visual perception: I. perceptual grouping and figure–ground organization. PubMed DOI PMC

Zhou, H., Friedman, H. S. & Von Der Heydt, R. Coding of border ownership in monkey visual cortex. PubMed DOI PMC

Posch, C., Matolin, D. & Wohlgenannt, R. A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. In

Guo, Z., Hou, Y., Xiao, R., Li, C. & Li, W. Motion saliency based hierarchical attention network for action recognition. DOI

Zong, M., Wang, R., Ma, Y. & Ji, W. Spatial and temporal saliency based four-stream network with multi-task learning for action recognition. DOI

Monforte, M., Arriandiaga, A., Glover, A. & Bartolozzi, C.

Schnider, Y. et al.

Vitale, A., Renner, A., Nauer, C., Scaramuzza, D. & Sandamirskaya, Y.

Paulun, L., Wendt, A. & Kasabov, N. A retinotopic spiking neural network system for accurate recognition of moving objects using neucube and dynamic vision sensors. PubMed DOI PMC

VanRullen, R. & Thorpe, S. J. Spatial attention in asynchronous neural networks. DOI

Eklundh, J.-O., Uhlin, T., Nordlund, P. & Maki, A.

Minaee, S. et al. Image segmentation using deep learning: A survey. PubMed

Arsenio, A. M. Figure/ground segregation from human cues. In

Comaniciu, D. & Meer, P. Robust analysis of feature spaces: Color image segmentation. In

Mishra, A., Aloimonos, Y. & Fermuller, C. Active segmentation for robotics. In

Censi, A. & Scaramuzza, D. Low-latency event-based visual odometry. In

Hu, B., von der Heydt, R. & Niebur, E. Figure-ground organization in natural scenes: Performance of a recurrent neural model compared with neurons of area v2. PubMed PMC

Iacono, M. et al. Proto-object based saliency for event-driven cameras. In

Bartolozzi, C. et al. Embedded neuromorphic vision for humanoid robots. In

Fowlkes, C. C., Martin, D. R. & Malik, J. Local figure–ground cues are valid for natural images. PubMed DOI

Martin, D., Fowlkes, C., Tal, D. & Malik, J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In

Martin, D., Fowlkes, C., Tal, D. & Malik, J. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In

Russell, A. F., Mihalaş, S., von der Heydt, R., Niebur, E. & Etienne-Cummings, R. A model of proto-object based saliency. PubMed DOI PMC

Rensink, R. A. The dynamic representation of scenes. DOI

Driver, J., Davis, G., Russell, C., Turatto, M. & Freeman, E. Segmentation, attention and phenomenal visual objects. PubMed DOI

Nakayama, K., Shimojo, S. & Silverman, G. H. Stereoscopic depth: its relation to image segmentation, grouping, and the recognition of occluded objects. PubMed DOI

Nakayama, K., He, Z. J. & Shimojo, S.

Rubin, E. et al.

Koffka, K. & Cabral, Á.

Franken, T. P. & Reynolds, J. H. Columnar processing of border ownership in primate visual cortex. PubMed DOI PMC

Martin, A. B. & von der Heydt, R. Spike synchrony reveals emergence of proto-objects in visual cortex. PubMed DOI PMC

Wagatsuma, N., von der Heydt, R. & Niebur, E. Spike synchrony generated by modulatory common input through NMDA-type synapses. PubMed DOI PMC

Wagatsuma, N., Hu, B., von der Heydt, R. & Niebur, E. Analysis of spiking synchrony in visual cortex reveals distinct types of top-down modulation signals for spatial and object-based attention. PubMed DOI PMC

Herault, L. & Horaud, R. Figure-ground discrimination: a combinatorial optimization approach. DOI

Stricker, M. & Leonardis, A.

Pham, V.-Q., Takahashi, K. & Naemura, T.

Yang, W., Cai, J., Zheng, J. & Luo, J. User-friendly interactive image segmentation through unified combinatorial user inputs. PubMed DOI

Lempitsky, V., Kohli, P., Rother, C. & Sharp, T.

Hsiao, Y.-M. & Chang, L.-W.

Liang, Y., Zhang, M. & Browne, W. N.

Naha, S. & Wang, Y.

Nishimura, H. & Sakai, K. Determination of border-ownership based on the surround context of contrast. DOI

Ramenahalli, S., Mihalas, S. & Niebur, E. Local spectral anisotropy is a valid cue for figure-ground organization in natural scenes. PubMed DOI PMC

Hu, B. & Niebur, E. A recurrent neural model for proto-object based contour integration and figure-ground segregation. PubMed DOI PMC

Sun, Y. & Fisher, R. Hierarchical selectivity for object-based visual attention. In

Grossberg, S. & Mingolla, E. PubMed

Itti, L., Koch, C. & Niebur, E. A model of saliency-based visual attention for rapid scene analysis. DOI

Ghosh, S. et al. Event-driven proto-object based saliency in 3d space to attract a robot’s attention. PubMed DOI PMC

D’Angelo, G., Perrett, A., Iacono, M., Furber, S. & Bartolozzi, C. Event driven bio-inspired attentive system for the icub humanoid robot on spinnaker. DOI

Iacono, M., Weber, S., Glover, A. & Bartolozzi, C. Towards event-driven object detection with off-the-shelf deep learning. In

Rebecq, H., Gehrig, D. & Scaramuzza, D. Esim: an open event camera simulator. In

Szeliski, R.

Jiang, Y. et al.

Sorbaro, M., Liu, Q., Bortone, M. & Sheik, S. Optimizing the energy consumption of spiking neural networks for neuromorphic applications. PubMed DOI PMC

Liu, Q. et al. Live demonstration: face recognition on an ultra-low power event-driven convolutional neural network ASIC. In

Davies, M. et al. Loihi: A neuromorphic manycore processor with on-chip learning. DOI

Furber, S. B., Galluppi, F., Temple, S. & Plana, L. A. The spinnaker project. DOI

Moradi, S., Qiao, N., Stefanini, F. & Indiveri, G. A scalable multicore architecture with heterogeneous memory structures for dynamic neuromorphic asynchronous processors (dynaps). PubMed DOI

Azzopardi, G., Rodríguez-Sánchez, A., Piater, J. & Petkov, N. A push-pull corf model of a simple cell with antiphase inhibition improves SNR and contour detection. PubMed DOI PMC

Martinez-Conde, S. & Macknik, S. L. Fixational eye movements across vertebrates: comparative dynamics, physiology, and perception. PubMed DOI