Spike‐Timing‐Dependent Plasticity in Memristors Spike ‐ Timing ‐ Dependent Plasticity in Memristors

The spike‐timing‐dependent plasticity (STDP) characteristic of the memristor plays an important role in the development of neuromorphic network computing in the future. The STDP characteristics were observed in different memristors based on different kinds of materials. The investigation regarding the influences of device hysteresis character ‐ istic, the initial conductance of the memristors, and the waveform of the voltage pulses applied to the memristor as preneuron voltage spike and postneuron voltage spike on the STDP behavior of memristors are reviewed.


Introduction
The state-of-the-art artificial intelligence based on traditional von Neumann computation paradigm has shown remarkable learning and thinking abilities, for instance, AlphaGo created by the Google-owned company Deep Mind beat the top Go player Lee Sedol by 4:1 recently [1]. However, the information processing through the digital von Neumann computation paradigm is much less efficient as compared to human brains, which is the major bottleneck of von Neumann computation paradigm. Synapse plays the key role in learning, thinking, and memorizing for a human being, and there are approximately 10 14 synapses in a human's brain [2]. A synapse is formed between two neuron cells [3], and the synapse weight can be precisely tuned by the ionic flowing through them. It is well known that the adaptation of the synapse weight between two neurons it connects with makes the biological systems functional [4]. In order to build up a system that behaves in a much more efficient way like a human brain, people have never stopped searching for an electrical element that mimics the basic function of a synapse until "the miss memristor found [5]." Similar to a biological synapse, memristor is a two-terminal device whose conductance can be changed by the input pulses or by controlling the charge through it [4,6] and in such a way, a memristor works as an artificial electronic synapse. Electronic synapses based on memristor devices are around three orders of magnitude smaller than a prominent CMOS design [2]; thus, the memristor has a great potential for scalability as compared to the electronic synapse made by traditional complex circuits [7].
Synapses have different kinds of plasticity, which have been realized and investigated in different memristors [8]. And the research on the application of memristors with the common synaptic plasticity in some kind of neural networks has also been conducted. For instance, HfO 2 -based memristors were used in a Hopfield neural network to implement associative memory [9]. The relationship between the resistance of the memristor and the synaptic weight was defined. And the resistances of the memristors were tuned to the target resistances through the application of the voltage pulses on the memristors as the training process [9]. Prezioso et al. realized pattern classification by using the neural network based on memristors with synaptic plasticity [10]. The 12 × 12 crossbar with Pt/ Al 2 O 3 /TiO 2−x /Ti/Pt memristors at each cross point was fabricated, which is illustrated in Figure 1(a). Sixty memristors among them were used to realize the function. The relationship between synaptic weight and conductance of the memristors is shown in Eq. (1). The synaptic weight was changed by applying fixed voltage pulses with the amplitude of ±1.3 V on the memristors, and the change of conductance under different voltage pulses is shown in Figure 1(c).

STDP in memristors
In the common synaptic plasticity mentioned above, the change of the conductance (weight) is only related to one voltage pulse applied on the memristors. Another kind of plasticity of the synapses is spike-timing-dependent plasticity (STDP). STDP is one of the most important synaptic characteristics. STDP modulates synapse weight based on the activities of the so-called pre-and postsynaptic neurons [11]. The spikes from both preneuron and postneuron arrive at the synapse occasionally in the opposite direction [7]. In STDP, the change of the synaptic weight is the function of relative neuron spike timing ∆t (∆t = t pre − t post ), where t pre is the time when the presynaptic neuron spike arrives and t post is the time when the postsynaptic neuron spike arrives [4]. In a typical STDP behavior, if postsynaptic neuron spike arrives after presynaptic neuron spike (∆t < 0), the synaptic weight increases. If postsynaptic neuron spike arrives before presynaptic neuron spike (∆t > 0), the synaptic weight decreases. In electronic synapse based on memristor, voltage spikes or pulses are applied on the memristor through the two electrodes, which modulates the conductance of the memristor, and the change of conductance is related to the relative timing of voltage spikes or pulses. Memristors can realize STDP function which is similar with that of biological synaptic systems, which is shown in Figure 2 [4]. The relationship between the change in excitatory postsynaptic current (EPSC) of rat hippocampal neurons after repetitive correlated spiking (60 pulses at 1 Hz) and relative spike timing. The figure was reconstructed with permission from Ref. [8,12]. Inset (b) is the phase contrast image of a hippocampal neuron, which was adapted with permission from Ref. [4,13,26].
STDP have been intensively investigated in the different memristors with different materials. The memristors are usually composed of two electrodes and memristive materials sandwiched between two electrodes. Metals such as Au, Pt, Ag, Cu, conductive nitrides such as TiN, and conductive oxides such as ITO are usually used as the materials of electrodes. The memristive materials can be grouped into binary oxides, ternary and more complex oxides, polymer, and other kind of materials.
The STDP behavior was also observed in polymer such as poly(3,4-ethylenedioxythiophene): poly(styrenesulfonate) (PEDOT:PSS) [21], EV(ClO 4 ) 2 /BTPA-F [22], and so on. Li et al. imitated the STDP of Ag/PEDOT:PSS/Ta structure [23]. A pair of temporally correlated voltage pulses with amplitudes V + /V − = 2 V/−2 V was used as presynaptic spikes and postsynaptic spikes, which was applied to the memristors, respectively. The change of the synaptic weights related to the precise timing between pre-and postsynaptic spikes is shown in Figure 8(c).
In addition, the investigations on the STDP of the memristors based on other kind of materials such as Si/Ag mixture [4], polycrystal CH 3 NH 3 PbI 3 [24], have also been conducted.
Some factors in the STDP measurements can change some characteristics of the STDP, for example, the waveform of voltage spikes used to imitate the presynaptic neuron spike and postsynaptic neuron spike influences the STDP behavior significantly. It has been reported that the STDP function can be strongly influenced by the shape of the input voltage spikes [25]. The shape of voltage spike generated from presynaptic neuron is the same with that generated from postsynaptic neuron. Zamarreño-Ramos et al. investigated the influence of the shape of the voltage spikes (spk(t)) on STDP learning function ξ (∆T). The results are shown in Figure 9. The results reveals that the voltage spikes with a narrow short positive pulse of large amplitude and a longer relaxing slowly decreasing negative tail are needed in order to obtain the STDP function similar with the behavior of the biological synapses [25].  Figure 10. The influence of switching characteristics on the operating region used for STDP was discussed. A smooth switching characteristics leads to a much wider operation region, and a steep switching characteristics leads to a much narrower operation region [26].
Du et al. reported that the learning time constant can be adjusted through changing the duration of the voltage spikes. The scheme of the voltage spikes is shown in Figure 11, and pulse width (t p ) is one of the parameters of the voltage spikes. The range of the delay time ∆t where the normalized current is larger than 50% is called learning window. As shown in Figure 12, learning window decreases from 25 ms to 125 μs with the decrease of pulse width (t p ) from 10 ms to 50 μs. In addition, energy consumption of the memristors was also discussed in this work, the authors showed that energy consumption of the Au/BFO/ Pt/Ti memristor is 4.7 pJ. A method to reduce the energy consumption was proposed and tested, and the results indicate by decreasing the pulse width (t p ) energy consumption can be reduced to 4.5 pJ.
Xiao et al. reported the STDP characteristics of the memristor with the structure of Au/polycrystal CH 3 NH 3 PbI 3 /ITO/PEDOT:PSS. Different waveforms were used as presynaptic neuron voltage spike and postsynaptic neuron voltage spike, which are shown in Figure 13(b-e). Four different kinds of STDP characteristics, including asymmetric Hebbian rule, asymmetric anti-Hebbian rule, symmetric Hebbian rule, and symmetric anti-Hebbian rule, were obtained corresponding to four different waveforms applied to the memristor as shown in Figure 13(f-i).
And the four kinds of STDP behaviors were fit by different equations [24].

ΔW = A exp (−
Prezioso et al. investigated the STDP characteristics of the memristor with the structure of Pt/Al 2 O 3 /TiO 2−x /Ti/Pt. Three pairs of preneuron spike and postneuron spike with different waveforms, which are shown in Figure 14(a-c), were applied on the memristor. Three different STDP behaviors were observed, which are illustrated in Figure 14(g-i). The results demonstrated the dependence of STDP window on the waveform of preneuron spike and postneuron spike. The investigation regarding the influence of the initial conductance (G 0 ) on the STDP behavior was also conducted. In this set of tests, the waveform shown in Figure 14(a) was used. The STDP functions for different initial conductance G 0 = 25, 50, 75, and 100 μS were measured and compared. The results shown in Figure 15 indicate the influence of the switching dynamics' saturation of the memristors on the STDP property.
All the memristors have their own dynamic range of the conductance. When G 0 is close to its maximum value, the increase of the conductance is very low. And when G 0 is close to its minimum value, the decrease of the conductance is very low [14].

Conclusions
In summary, the STDP characteristics have been observed in different memristors based on different kinds of materials, which make memristors become promising in the bio-inspired neuromorphic application. Great efforts have also been made in the investigation on the influence factors of the STDP characteristics such as device hysteresis characteristic and the waveform of the voltage pulses applied to the memristor as preneuron voltage spike and postneuron voltage spike. Different kinds of waveform were used, and different kinds of STDP characteristics were observed. Figure 15. The experimentally measured STDP window function with several initial values G 0 = 25, 50, 75, and 100 μs together with the results of its fitting with equations (dash-dot lines) [14].