| Citation: | Xuanyu Shan, Ya Lin, Zhongqiang Wang, Xiaoning Zhao, Ye Tao, Haiyang Xu, Yichun Liu. Emerging multimodal memristors for biorealistic neuromorphic applications[J]. Materials Futures, 2024, 3(1): 012701. doi: 10.1088/2752-5724/ad119e
|
The integration of sensory information from different modalities, such as touch and vision, is essential for organisms to perform behavioral functions such as decision-making, learning, and memory. Artificial implementation of human multi-sensory perception using electronic supports is of great significance for achieving efficient human–machine interaction. Thanks to their structural and functional similarity with biological synapses, memristors are emerging as promising nanodevices for developing artificial neuromorphic perception. Memristive devices can sense multidimensional signals including light, pressure, and sound. Their in-sensor computing architecture represents an ideal platform for efficient multimodal perception. We review recent progress in multimodal memristive technology and its application to neuromorphic perception of complex stimuli carrying visual, olfactory, auditory, and tactile information. At the device level, the operation model and undergoing mechanism have also been introduced. Finally, we discuss the challenges and prospects associated with this rapidly progressing field of research.
| [1] |
Shi Q, Dong B, He T, Sun Z, Zhu J, Zhang Z and Lee C 2020 Progress in wearable electronics/photonics—moving toward the era of artificial intelligence and internet of things InfoMat 2 1131–62
|
| [2] |
Yan Z et al 2021 Flexible high-resolution triboelectric sensor array based on patterned laser-induced graphene for self-powered real-time tactile sensing Adv. Funct. Mater. 31 2100709
|
| [3] |
Liang S J, Li Y, Cheng B and Miao F 2022 Emerging low-dimensional heterostructure devices for neuromorphic computing Small Struct. 3 2200064
|
| [4] |
Du J et al 2021 A robust neuromorphic vision sensor with optical control of ferroelectric switching Nano Energy 89 106439
|
| [5] |
Wang Y et al 2023 A three-dimensional neuromorphic photosensor array for nonvolatile in-sensor computing Nano Lett. 23 4523–32
|
| [6] |
Leo F, Bolognini N, Passamonti C, Stein B E and Ladavas E 2008 Cross-modal localization in hemianopia: new insights on multisensory integration Brain 131 855–65
|
| [7] |
Frégnac Y and Bathellier B 2015 Cortical correlates of low-level perception: from neural circuits to percepts Neuron 88 110–26
|
| [8] |
Tan H, Zhou Y, Tao Q, Rosen J and van Dijken S 2021 Bioinspired multisensory neural network with crossmodal integration and recognition Nat. Commun. 12 1120
|
| [9] |
Albustanji R N, Elmanaseer S and Alkhatib A A A 2023 Robotics: five senses plus one—an overview Robotics 12 68
|
| [10] |
Wan C, Cai P, Guo X, Wang M, Matsuhisa N, Yang L, Lv Z, Luo Y, Loh X J and Chen X 2020 An artificial sensory neuron with visual-haptic fusion Nat. Commun. 11 4602
|
| [11] |
Zhao B, Zhao X, Li Q, Xun X, Ouyang T, Zhang Z, Kang Z, Liao Q and Zhang Y 2022 Reproducible and low-power multistate bio-memristor from interpenetrating network electrolyte design InfoMat 4 e12350
|
| [12] |
Lin Y, Meng F, Zeng T, Zhang Q, Wang Z, Cheng Y, Zhao X, Gu L, Xu H and Liu Y 2023 Direct observation of oxygen ion dynamics in a WO3-x based second-order memristor with dendritic integration functions Adv. Funct. Mater. 33 2302787
|
| [13] |
Yan X, Zhao J, Liu S, Zhou Z, Liu Q, Chen J and Liu X Y 2018 Memristor with Ag-cluster-doped TiO2 films as artificial synapse for neuroinspired computing Adv. Funct. Mater. 28 1705320
|
| [14] |
Xu L, Yuan R, Zhu Z, Liu K, Jing Z, Cai Y, Wang Y, Yang Y and Huang R 2019 Memristor-based efficient in-memory logic for cryptologic and arithmetic applications Adv. Mater. Technol. 4 1900212
|
| [15] |
Tan H, Liu G, Yang H, Yi X, Pan L, Shang J, Long S, Liu M, Wu Y and Li R-W 2017 Light-gated memristor with integrated logic and memory functions ACS Nano 11 11298–305
|
| [16] |
Jo S H, Chang T, Ebong I, Bhadviya B B, Mazumder P and Lu W 2010 Nanoscale memristor device as synapse in neuromorphic systems Nano Lett. 10 1297–301
|
| [17] |
Wang Z Q, Xu H Y, Li X H, Yu H, Liu Y C and Zhu X J 2012 Synaptic learning and memory functions achieved using oxygen ion migration/diffusion in an amorphous InGaZnO memristor Adv. Funct. Mater. 22 2759–65
|
| [18] |
Lin H, Wang C, Hong Q and Sun Y 2020 A multi-stable memristor and its application in a neural network IEEE Trans. Circuits Syst. II 67 3472–6
|
| [19] |
Choi S, Jang S, Moon J-H, Kim J C, Jeong H Y, Jang P, Lee K-J and Wang G 2018 A self-rectifying TaOy/nanoporous TaOx memristor synaptic array for learning and energy-efficient neuromorphic systems NPG Asia Mater. 10 1097–106
|
| [20] |
Kim S, Kim H, Hwang S, Kim M-H, Chang Y-F and Park B-G 2017 Analog synaptic behavior of a silicon nitride memristor ACS Appl. Mater. Interfaces 9 40420–7
|
| [21] |
Zhang C et al 2019 Bioinspired artificial sensory nerve based on nafion memristor Adv. Funct. Mater. 29 1808783
|
| [22] |
Lee D, Chae M, Jung J and Kim H D 2023 Correlation between sensing accuracy and read margin of a memristor-based NO gas sensor array estimated by neural network analysis ACS Sens. 8 2105–14
|
| [23] |
Zhou Y et al 2020 Black phosphorus based multicolor light-modulated transparent memristor with enhanced resistive switching performance ACS Appl. Mater. Interfaces 12 25108–14
|
| [24] |
Yan B, Kuang D, Wang W, Wang Y, Sun B and Zhou G 2022 Investigation of multi-photoconductance state induced by light-sensitive defect in TiOx-based memristor Appl. Phys. Lett. 120 253506
|
| [25] |
Park J, Lee S, Lee J and Yong K 2013 A light incident angle switchable ZnO nanorod memristor: reversible switching behavior between two non-volatile memory devices Adv. Mater. 25 6423–9
|
| [26] |
Chen S, Lou Z, Chen D and Shen G 2018 An artificial flexible visual memory system based on an UV-motivated memristor Adv. Mater. 30 1705400
|
| [27] |
Wu X, Li E, Liu Y, Lin W, Yu R, Chen G, Hu Y, Chen H and Guo T 2021 Artificial multisensory integration nervous system with haptic and iconic perception behaviors Nano Energy 85 106000
|
| [28] |
Duan Q, Zhang T, Liu C, Yuan R, Li G, Jun Tiw P, Yang K, Ge C, Yang Y and Huang R 2022 Artificial multisensory neurons with fused haptic and temperature perception for multimodal in-sensor computing Adv. Intell. Syst. 4 2200039
|
| [29] |
Pei Y et al 2022 A multifunctional and efficient artificial visual perception nervous system with Sb2Se3/CdS-Core/Shell (SC) nanorod arrays optoelectronic memristor Adv. Funct. Mater. 32 2203454
|
| [30] |
Wang T-Y, Meng J-L, Li Q-X, He Z-Y, Zhu H, Ji L, Sun Q-Q, Chen L and Zhang D W 2021 Reconfigurable optoelectronic memristor for in-sensor computing applications Nano Energy 89 106291
|
| [31] |
Sun L, Wang Z, Jiang J, Kim Y, Joo B, Zheng S, Lee S, Yu W J, Kong B-S and Yang H 2021 In-sensor reservoir computing for language learning via two-dimensional memristors Sci. Adv. 7 eabg1455
|
| [32] |
Bae B, Park M, Lee D, Sim I and Lee K 2023 Hetero-integrated InGaAs photodiode and oxide memristor-based artificial optical nerve for in-sensor NIR image processing Adv. Opt. Mater. 11 2201905
|
| [33] |
Sun J et al 2022 A dual-mode organic memristor for coordinated visual perceptive computing Fundam. Res. (https://doi.org/10.1016/j.fmre.2022.06.022)
|
| [34] |
Dang B et al 2020 Physically transient optic-neural synapse for secure in-sensor computing IEEE Electron Device Lett. 41 1641–4
|
| [35] |
Li Z, Tang W, Zhang B, Yang R and Miao X 2023 Emerging memristive neurons for neuromorphic computing and sensing Sci. Technol. Adv. Mater. 24 2188878
|
| [36] |
Liu H et al 2023 Artificial neuronal devices based on emerging materials: neuronal dynamics and applications Adv. Mater. 35 2205047
|
| [37] |
Wang M et al 2022 Tactile near-sensor analogue computing for ultrafast responsive artificial skin Adv. Mater. 34 2201962
|
| [38] |
Cho S W, Kwon S M, Kim Y-H and Park S K 2021 Recent progress in transistor-based optoelectronic synapses: from neuromorphic computing to artificial sensory system Adv. Intell. Syst. 3 2000162
|
| [39] |
Yuan Y et al 2023 Artificial leaky integrate-and-fire sensory neuron for in-sensor computing neuromorphic perception at the edge ACS Sens. 8 2646–55
|
| [40] |
Yuan R et al 2022 A calibratable sensory neuron based on epitaxial VO2 for spike-based neuromorphic multisensory system Nat. Commun. 13 3973
|
| [41] |
John R A et al 2021 Diffusive and drift halide perovskite memristive barristors as nociceptive and synaptic emulators for neuromorphic computing Adv. Mater. 33 2007851
|
| [42] |
Seo S et al 2018 Artificial optic-neural synapse for colored and color-mixed pattern recognition Nat. Commun. 9 5106
|
| [43] |
Zhou F et al 2019 Optoelectronic resistive random access memory for neuromorphic vision sensors Nat. Nanotechnol. 14 776–82
|
| [44] |
Sun L, Zhang Y, Hwang G, Jiang J, Kim D, Eshete Y A, Zhao R and Yang H 2018 Synaptic computation enabled by Joule heating of single-layered semiconductors for sound localization Nano Lett. 18 3229–34
|
| [45] |
Zeng T, Wang Z, Lin Y, Cheng Y, Shan X, Tao Y, Zhao X, Xu H and Liu Y 2023 Doppler frequency-shift information processing in WO2-based memristive synapse for auditory motion perception Adv. Sci. 10 2300030
|
| [46] |
Wang T, Huang H M, Wang X X and Guo X 2021 An artificial olfactory inference system based on memristive devices InfoMat 3 804–13
|
| [47] |
Qian C, Choi Y, Choi Y J, Kim S, Choi Y Y, Roe D G, Kang M S, Sun J and Cho J H 2020 Oxygen-detecting synaptic device for realization of artificial autonomic nervous system for maintaining oxygen homeostasis Adv. Mater. 32 2002653
|
| [48] |
Jiang C, Tan D, Sun N, Huang J, Ji R, Li Q, Bi S, Guo Q, Wang X and Song J 2021 60 nm Pixel-size pressure piezo-memory system as ultrahigh-resolution neuromorphic tactile sensor for in-chip computing Nano Energy 87 106190
|
| [49] |
Ayyad A, Halwani M, Swart D, Muthusamy R, Almaskari F and Zweiri Y 2023 Neuromorphic vision based control for the precise positioning of robotic drilling systems Robot. Comput.-Integr. Manuf. 79 102419
|
| [50] |
Cho S W, Jo C, Kim Y H and Park S K 2022 Progress of materials and devices for neuromorphic vision sensors Nano-Micro Lett. 14 203
|
| [51] |
Kumar D, Li H, Das U K, Syed A M and El-Atab N 2023 Flexible solution processable black phosphorus based optoelectronic memristive synapse for neuromorphic computing and artificial visual perception applications Adv. Mater. 35 2300446
|
| [52] |
Chen W, Zhang Z and Liu G 2022 Retinomorphic optoelectronic devices for intelligent machine vision Iscience 25 103729
|
| [53] |
Subin P S, Midhun P S, Antony A, Saji K J and Jayaraj M K 2022 Optoelectronic synaptic plasticity mimicked in ZnO-based artificial synapse for neuromorphic image sensing application Mater. Today Commun. 33 104232
|
| [54] |
Ai L, Pei Y, Song Z, Yong X, Song H, Liu G, Nie M, Waterhouse G I N, Yan X and Lu S 2023 Ligand-triggered self-assembly of flexible carbon dot nanoribbons for optoelectronic memristor devices and neuromorphic computing Adv. Sci. 10 2207688
|
| [55] |
Kong N S P and Ibrahim H 2008 Color image enhancement using brightness preserving dynamic histogram equalization IEEE Trans. Consum. Electron. 54 1962–8
|
| [56] |
Tsuruoka T, Terabe K, Hasegawa T, Valov I, Waser R and Aono M 2012 Effects of moisture on the switching characteristics of oxide-based, gapless-type atomic switches Adv. Funct. Mater. 22 70–77
|
| [57] |
Ahmed T et al 2019 Multifunctional optoelectronics via harnessing defects in layered black phosphorus Adv. Funct. Mater. 29 1901991
|
| [58] |
Li H, Jiang X, Ye W, Zhang H, Zhou L, Zhang F, She D, Zhou Y and Han S-T 2019 Fully photon modulated heterostructure for neuromorphic computing Nano Energy 65 104000
|
| [59] |
Ahmed T et al 2021 Fully light-controlled memory and neuromorphic computation in layered black phosphorus Adv. Mater. 33 2004207
|
| [60] |
Hu L, Yang J, Wang J, Cheng P, Chua L O and Zhuge F 2021 All-optically controlled memristor for optoelectronic neuromorphic computing Adv. Funct. Mater. 31 2005582
|
| [61] |
Shan X et al 2022 Plasmonic optoelectronic memristor enabling fully light-modulated synaptic plasticity for neuromorphic vision Adv. Sci. 9 2104632
|
| [62] |
Qiao Q, Zhang X, Lu Z, Wang L, Liu Y, Zhu X and Li J 2009 Formation of holographic fringes on photochromic Ag/TiO2 nanocomposite films Appl. Phys. Lett. 94 074104
|
| [63] |
Xie D, Gao G, Tian B, Shu Z, Duan H, Zhao -W-W, He J and Jiang J 2023 Porous metal–organic framework/ReS2 heterojunction phototransistor for polarization-sensitive visual adaptation emulation Adv. Mater. 35 2212118
|
| [64] |
Xie D, Yin K, Yang Z, Huang H, Li X, Shu Z, Duan H, He J and Jiang J 2022 Polarization-perceptual anisotropic two-dimensional ReS2 neuro-transistor with reconfigurable neuromorphic vision Mater. Horiz. 9 1448–59
|
| [65] |
Liu Q, Wei Q, Ren H, Zhou L, Zhou Y, Wang P, Wang C, Yin J and Li M 2023 Circular polarization-resolved ultraviolet photonic artificial synapse based on chiral perovskite Nat. Commun. 14 7179
|
| [66] |
Lv Z, Hu Q, Xu Z, Wang J, Chen Z, Wang Y, Chen M, Zhou K, Zhou Y and Han S-T 2019 Organic memristor utilizing copper phthalocyanine nanowires with infrared response and cation regulating properties Adv. Electron. Mater. 5 1800793
|
| [67] |
Kim T, Choi C, Hur J, Ha D, Kuh B J, Kim Y, Cho M H, Kim S and Jeong J K 2022 Progress, challenges, and opportunities in oxide semiconductor devices: a key building block for applications ranging from display backplanes to 3D integrated semiconductor chips Adv. Mater. 35 2204663
|
| [68] |
Zhu J, Zhang X, Wang R, Wang M, Chen P, Cheng L, Wu Z, Wang Y, Liu Q and Liu M 2022 A heterogeneously integrated spiking neuron array for multimode-fused perception and object classification Adv. Mater. 34 2200481
|
| [69] |
Brattoli M, De Gennaro G, De Pinto V, Demarinis Loiotile A, Lovascio S and Penza M 2011 Odour detection methods: olfactometry and chemical sensors Sensors 11 5290–322
|
| [70] |
Plutowska B and Wardencki W 2008 Application of gas chromatography–olfactometry (GC–O) in analysis and quality assessment of alcoholic beverages–a review Food Chem. 107 449–63
|
| [71] |
Baldovini N and Chaintreau A 2020 Identification of key odorants in complex mixtures occurring in nature Nat. Prod. Rep. 37 1589–626
|
| [72] |
Song Z, Tong Y, Zhao X, Ren H, Tang Q and Liu Y 2019 A flexible conformable artificial organ-damage memory system towards hazardous gas leakage based on a single organic transistor Mater. Horiz. 6 717–26
|
| [73] |
Kolarik A J, Moore B C J, Zahorik P, Cirstea S and Pardhan S 2016 Auditory distance perception in humans: a review of cues, development, neuronal bases, and effects of sensory loss Atten. Percept. Psychophys. 78 373–95
|
| [74] |
Wang W, Pedretti G, Milo V, Carboni R, Calderoni A, Ramaswamy N, Spinelli A S and Ielmini D 2018 Learning of spatiotemporal patterns in a spiking neural network with resistive switching synapses Sci. Adv. 4 eaat4752
|
| [75] |
Ji X, Zhao X, Tan M and Zhao R 2020 Artificial perception built on memristive system: visual, auditory, and tactile sensations Adv. Intell. Syst. 2 1900118
|
| [76] |
Yu J, Zeng F, Wan Q, Lu Z and Pan F 2023 Emulation of auditory senses depending on chaotic dynamics of threshold switching memristor InfoMat 5 e12458
|
| [77] |
Coen M H 2001 Multimodal integration-a biological view Int. Joint Conf. on Artificial Intelligence vol 17 (Lawrence Erlbaum Associates Ltd) pp 1417–24
|
| [78] |
Navarro-Guerrero N, Toprak S, Josifovski J and Jamone L 2023 Visuo-haptic object perception for robots: an overview Auton. Robots 47 377–403
|
| [79] |
Steil J J, Röthling F, Haschke R and Ritter H 2004 Situated robot learning for multi-modal instruction and imitation of grasping Robot. Auton. Syst. 47 129–41
|
| [80] |
Yu J, Yang X, Gao G, Xiong Y, Wang Y, Han J, Chen Y, Zhang H, Sun Q and Wang Z L 2021 Bioinspired mechano-photonic artificial synapse based on graphene/MoS2 heterostructure Sci. Adv. 7 eabd9117
|
| [81] |
Wang Y et al 2021 MXene-ZnO memristor for multimodal in-sensor computing Adv. Funct. Mater. 31 2100144
|
Figures(1)
Copyright © 2021 Materials Futures Editorial Office
Supported by:
Beijing Renhe Information Technology Co., Ltd.
DownLoad: