Memristor Synapses for Neuromorphic Computing

Neuromorphic computing, which imitates the principle behind biological synapses with a high degree of parallelism, has recently emerged as a promising candidate for novel and sustainable computing technologies. The first step toward realizing a massively parallel neuromorphic system is to develop an artificial synapse capable of emulating synapse functionality, such as analog modulation, with ultralow power consumption and robust controllability. We begin this chapter with a simple description of neuromorphic systems and memristor synapses. Further, we introduce and evaluate the state-of-the-art neuromorphic hardware technology in terms of novel functional materials and device architectures toward the implementation of fully neuromorphic computers, which have been extensively explored in recent years. Finally, we briefly describe artificial neural networks based on memristor synapse in forms of crossbar arrays.


Introduction
Modern computers and electronics, such as smartphones and supercomputers, have been developed in accordance with Moore's law [1], which implies improvement in cost, speed, and power consumption by scaling down devices. However, the fundamental physical limits and increased fabrication costs pose a hindrance to sustainable development of computing technology [2,3]. Moreover, with the advent of the big data era, unstructured data and data complexity explosively increases, imposing constraints on the conventional computing technology owing to the von Neumann bottleneck [4,5]. Neuromorphic systems [6,7], which mimic the nervous system in the brain, have recently become known as strong candidates to overcome these technical and economic limitations owing to their proficiency in cognitive and data-intensive tasks, together with their low power consumption. To successfully implement these neuromorphic systems, it is of utmost importance to research and develop artificial synapses capable of synapse functions, high reliability, low energy consumption, etc. [8,9]. In the plethora of possible devices, memristors have gained the spotlight because of their desirable characteristics as artificial synapses [10][11][12], including device speed [13], footprint [14], low energy consumption [15], and analog switching [16,17].
In this chapter, we introduce the basic concepts of neuromorphic systems and memristor synapses. We also describe diverse examples for state-of-the-art artificial synapses in terms of novel functional materials and device architecture. We then briefly review the implemented neuromorphic systems based on memristor synapses.

Memristor synapses
Memristors that consist of a storage layer inserted between the top and bottom electrodes can undergo dynamic reconfiguration within the storage layer with the application of electrical stimuli, resulting in resistance modulation referred to as memory effect [16,17]. The changed resistance state can be retained even after electrical inputs are removed, and memristors are based on the history of applied electrical stimuli. These capabilities lead to analog switching, which resembles biological synapses where the strength (or synaptic weight) can increase or decrease depending on the applied action potential [25,26]. When neuromorphic architecture is implemented on the conventional computing architecture, the synaptic weights are stored in the memory unit and are continuously read into the processor unit to transfer information to post-neurons. In other words, practically, the von Neumann bottleneck still remains challenged. However, in case of memristor synapse-based neuromorphic systems, the synapses can not only store a specific weight but also naturally transmit information into post-neurons, overcoming the von Neumann bottleneck and improving system efficiency [8,9]. In addition to analog switching, memristors have exhibited desirable device properties, including nanoscale footprint [14], long endurance and retention [17,27], nanosecond switching speed [13,15], and low power consumption [15]. Owing to these characteristics, memristors have emerged as promising candidates for artificial synapses. However, it should be noted that no specific material/device system has shown all-encompassed characteristics so far.

Switching mechanisms
Depending on their storage layer and electrode, memristors can be broadly classified into two categories: cation-based devices and anion-based devices. It is widely believed that cation-based devices are based on migration of metallic cations (see Figure 2a) [17,28]. They employ electrochemically active materials such as Ag or Cu as an electrode [29][30][31][32]. The counter electrode is usually an electrochemically inert material, such as Pt, Au, or W, and the storage layer consists of a solidelectrolyte like Ta 2 O 5 , SiO 2 , or Cu 2 S. For example, when a positive voltage is applied to an Ag top electrode, the atoms from this electrode are electrochemically oxidized to Ag + cations because of anodic reaction, which are then dissolved into a solidelectrolyte layer. The Ag + cations migrate across the solid-electrolyte layer toward the counter electrode (e.g., Pt) depending on electric field. At the Pt electrode, the Ag + cations are electrochemically reduced to Ag atoms because of cathodic reaction and are deposited on its surface. Thus, conductive filaments grow toward the Ag top electrode, and eventually the filaments bridge the anode and the cathode, indicating that the device switches into ON state (low resistance state) as shown in Figure 2a. In contrast, when a negative voltage is applied to the Ag top electrode, the Ag filament begins to dissolve anodically, starting from the interface of the Ag top electrode/Ag filament, which results in OFF state (high resistance state). Owing to this process, cation-based devices are referred to as electrochemical metallization memories and conductive bridging random access memories. It should be noted that the initial formation of conductive filaments is called the electroforming process, which needs a voltage higher than a switching voltage.
Anion-based devices usually require the initial electroforming process and are switched depending on the O 2− anions (or positively charged oxygen vacancy V) induced into the storage layer by soft-breakdown (see Figure 2b). These devices consist of a sub-stoichiometric storage layer made of HfO x [33,34], TaO x [35,36], WO x [37,38], etc. When a positive forming voltage is applied to the top electrode, the induced O 2− ions migrate toward it. This anion motion causes a change in the valence state of the cation to keep the charge neutral; hence, these devices are also referred to as valance change memories. Throughout the process, the oxygen vacancies continue to form conductive filaments in the storage layer. When the filaments bridge the top and bottom electrodes, current flows through the filaments, with the result that the device switches to ON state. Contrastingly, when a negative voltage is applied to the top electrode, the O 2− ions either recombine with oxygen vacancies present in the filaments or oxidize the cation precipitates, with the result that the device switches to OFF state. Thus, memristors could be understood to some extent based on cation-and anion-based mechanisms. However, identifying the precise mechanism of a specific device is a challenge because of the presence of mingled mechanisms and different driving forces or locations. Therefore, further studies are necessary for a deeper understanding of the switching mechanism.

Desirable properties of memristor synapses
Various properties of memristor synapses that affect the performance of neuromorphic computing need to be discussed in detail. Among them, representative characteristics such as the linearity in weight update, multilevel states, dynamic range (ON/OFF ratio), variation, retention, endurance, and footprint will be addressed in this section as they can substantially affect computing achievements [8,35]. The linearity of the weight update indicates the linear relationship between synaptic weight change (∆w) and programming pulse. In other words, the conductance of the memristor synapse changes linearly in accordance with the number of programming pulses, which is associated with the mapping of weight in the algorithms for conductance in memristor synapses. Hence, the linearity of weight update affects the performance (e.g., accuracy). Notably, most memristor synapses  show a nonlinear weight update, where the conductance change gradually saturates, as shown in Figure 3. Hence, the nonlinearity of weight update should be improved to achieve highly efficient computing.
The resolution capability of storage is influenced by multilevel states and dynamic ranges because numerous conductance states can distinguishably store individual pixels of input patterns. Moreover, variations, including cycle-to-cycle and device-to-device variations, could degrade neuromorphic computing, particularly in large-scale systems. However, considering that neuromorphic computing exhibits the fault-tolerant property, neuromorphic architectures could be immune to the variation to some extent, and this is supported by several papers [8,35,52]. In addition, memristor synapses are repeatedly updated during the training process and should retain the trained weights (i.e., final conductance). Subsequently, the larger the endurance cycles and retention time, the better are the achievements of the neuromorphic network. Last but not least, it is desirable that device's footprint is below sub-10 nm because high density leads to more synaptic devices that store learned information under a specific area [8].
Furthermore, it is efficient to improve the characteristics of memristor synapses depending on individual neuromorphic networks, because a desirable memristor synapse capable of being employed into neuromorphic systems is yet to be reported. Supervised learning-based networks [35,[40][41][42][43][44], for example, are less vulnerable to cycle-to-cycle and device-to-device variations. This is because memristor synapses are updated according to calculated errors under known target values. By contrast, the networks based on unsupervised learning [39,[45][46][47] are directly affected by the variation owing to unknown target values. Therefore, memristor synapses need to be designed or selected depending on individual neuromorphic networks.

Artificial synapses in terms of device architecture and novel functional materials
Memristors for synaptic devices with two-terminal (e.g., vertical/planar-type and gap-type) and three-terminal (e.g., field-effect transistor and lateral coupling type) structures are manufactured by well-established processing technologies [7-12, 35, 39-55].
In the case of a two-terminal structure, when different voltages are applied to each of the two electrodes, resulting in current flow through the insulator, varying the conductance of the device enables emulation of biological synapse functions such as synapse plasticity [10-12, 16, 35, 48]. In particular, the crossbar array of two-terminal devices has received attention because of its characteristics relevant to synaptic devices, such as scalability for high density, simple fabrication process, low cost of fabrication, parallel connection structure, low power, fault-tolerance, and compactness. Thus, they are expected to provide an appropriate structure to support synaptic electronics. The type of two-terminal memristors that are being reffered to as the artificial synapses includes resistive random-access memory, phase change memory, conductive bridge memory, and spin-based memory. Although two-terminal devices are attracting much attention because of their ease of implementation of crossbar arrays, a two-terminal device, as a matter of fact, requires a select device to eliminate the sneak path that occurs in a crossbar array configuration. Additionally, it is difficult to imitate complex synaptic functions such as hetero-synaptic plasticity (e.g., modulatory input-dependent plasticity).
Three-terminal structures (e.g., field effect transistor memory and floating/gate transistor memory) with tunable conductance of channels between the source and the drain are also considered as synaptic devices [49][50][51]. The gate electrode acts as the pre-synapse, transferring the stimulus to the insulating layer, indicating the cleft of the synapse, and modulates the conductance of the channel representing the synaptic strength. Although the three-terminal structure is more complicated than the two-terminal structure and is disadvantageous in terms of density, the terminal for the signal transmission process and the learning terminal are separated such that simultaneous signal processing is possible, and complex synapse functions such as hetero-synaptic plasticity can be mimicked. Moreover, they do not require an additional selector device to reduce sneak current in an integrated array architecture.
Recently, going beyond simply implementing a synapse function, researchers have demonstrated advanced concepts of synapse device functions, including selfrectification, photo-assisted synaptic plasticity and neuromodulation to achieve more delicate imitation of the human brain and learning-and energy-efficiency in neurocomputing.
In [35], Choi et al. fabricated a self-rectifying memristor synapse through a twoterminal structure (Pt/TaO y /nanoporous TaO x /Ta), which is capable of suppressing unwanted leakage pathways and then a 16 x 16 crossbar array using only the devices without an additional selector (see Figure 4a and b). The mechanism of memristive switching and synaptic functions, including long-term potentiation (LTP), STDP (spike-timing dependent plasticity), and long-term depression (LTD) were caused by the migration of O 2− ions with oxygen vacancies V by applied electric field in the TaO x . In addition, the asymmetric interface contacts of Pt/TaO y and TaO x /Ta prevent the undesired signal by performing the self-rectification function without the selector.
In [51], Huh et al. reported a synapse device that performs the neuromodulator function of a barristor structure using 2D material as shown in Figure 4c. The three-terminal device consisted of a vertically integrated monolithic tungsten oxide memristor, and a variable-barrier tungsten selenide/graphene Schottky diode, termed as a "synaptic barrister." This synaptic barristor could implement fundamental synaptic functions, including short-term plasticity (STP), paired pulse facilitation (PPF), LTP, and LTD, with external gate controllability, termed as a neuromodulator in bio-synapse. This architecture potentially offers considerable power-saving benefits while significantly tuning the synaptic weights and intrinsically modifying the synaptic plasticity, in comparison with conventional two-neuronal-based synaptic architectures.
In [52], Ham et al. fabricated an organo-lead halide perovskite (OHP)-based photonic synapse in which the synaptic plasticity is modified by both electrical pulses and light illumination. The switching mechanism originates from the presence of a conductive filament by iodine-vacancy mediator, with its switching states controlled by electric-field domination (see Figure 4d). Using diverse electrical stimuli and DOI: http://dx.doi.org /10.5772/intechopen.85301 relative timing between the input pulses, essential synaptic functionalities such as STP, LTP, and LTD were successfully demonstrated. In addition, owing to the accelerated migration of the iodine vacancy inherently existing in the coated OHP film under light illumination, the OHP synaptic device exhibits light-tunable synaptic functionalities with very low programming inputs (≈0.1 V) as shown in Figure 4d. The ability of high-order tuning of the photo-assisted synaptic plasticity in an artificial synapse can offer significant improvements in the processing time, low-power recognition, and learning capability in a neuro-inspired computing system (Figure 4e).
In [12], Wang et al. designed a diffusive memristor for STP synapses and threshold neurons. The devices contain a switching layer doped with Ag nanoclusters (MgO x :Ag, SiO x N y :Ag, and HfO x :Ag) using the co-sputtering method. The switching mechanism is based on the growth and relaxation of Ag nanoclusters depending on whether the voltage pulse is applied, which was experimentally verified by in-situ high-resolution transmission electron microscopy (HRTEM). The designed device mimicked STP under PPF and PPD. Moreover, the device was used as a threshold neuron along with drift memristor synapse based on TaO x to emulate STDP learning rule. Because the conductance of the device gradually increases according to applied voltage and then abruptly decreases under no applied voltage, the device can be used as a threshold neuron. The results give a potential application for simple artificial neurons as compared with CMOS artificial neurons [53,54].

Neuromorphic systems based on crossbar array of memristor synapses
Prezioso et al. experimentally demonstrated neuromorphic networks based on memristor synapses (see [55]). In their paper, Al 2 O 3 /TiO 2−x memristor was used to fabricate a 12 × 12 crossbar array to implement a single-layer network [56]. The single-layer network architecture was schematically described as shown in Figure 5a, where 10 input neurons and 3 output neurons are fully linked by 10 × 3 = 30 synaptic weights (W i,j ). Notably, this ANN architecture naturally corresponds to a crossbar array [9,35]. Input voltages (V i = 1…9 ) assigned from pixels of the 3 × 3 input images (see Figure 5b) were applied to each input neuron. After being applied into the network, the input voltages were individually weighted depending on each synaptic weight. Note that V 10 is a bias voltage to control the degree of activation of the output neurons. The output neurons received each weighted voltage through linked weights and then integrated the weighted voltages (∑W i,j V j ), where j and i represent the input (j = 1-9) and output (i = 1-3) neurons respectively. The output neurons converted each integrated voltage into output (f i ) ranging from −1 to 1 according to the nonlinear activation function: f i = tanh(βI i ), where β adjusts the nonlinearity of the activation function and I i = ∑W i,j V j . The activation function can be considered as the threshold firing function in a biological neuron. The synaptic weights were represented by a pair of adjacent memristors (W i,j = G i,j + − G i,j − ) for the effectiveness of weight update. The number of selected memristor synapses in 12 × 12 array were 30 × 2 = 60, due to a pair of memristors (Figure 5c). When the network was under the training process, as shown in Figure 5d and e, memristor synapses between input and output neurons were updated based on the Manhattan update rule, which is classified as supervised learning:  where η is the learning rate, t i (n) is the target value, f i (n) is the output value, and n is the n th input image. After the training process was complete, the memristor synapses retained their final conductance, and the test process was performed without weight update (see Figure 5d). From the test process, the neuromorphic network exhibited perfect classification for the first time in 21 epochs (note that one epoch indicates one training process). Although simple and few input images were used to train/test the neuromorphic network, this work greatly contributed to neuromorphic systems based on memristor synapses in terms of experimental demonstration using crossbar arrays.
It should be noted that the circuit that acquires sgn could be implemented by a virtual ground circuit and a differential amplifier [43,57]. Then, the output value is compared with the target value by circuits using a comparator. According to calculated ∆W i,j , programming memristors of the array, for example, could be performed as shown in Figure 6 [39]. The test board contains four digital-to-analog converters (DACs) providing voltage pulses through the DACs. The DACs 1-4 represent the chosen bottom line, the unchosen bottom line, the chosen top line, and the unchosen top line, respectively. Using matrix switches (Switch 1 and 2), individual memristor is assigned to the corresponding DAC. The multiplexer (MUX) is operated to obtain currents that flow through memristors in the array by delivering the currents into the analog-to-digital converter (ADC). The ADC obtains the applied voltage of the resistor (1 kΩ), and the voltage is changed into the current. The arrows of Figure 6 represent the current flowing through a chosen memristor in case of write, erase, and read processes. Notably, there are non-idealities such as sneak currents and wire resistance in array-level, which could degrade the performance of neuromorphic computing [35,44,[58][59][60]. The sneak currents affect learning accuracy and epochs because of undesired information, especially large-scale array. In Figure 6, in order to avoid sneak currents during read process, unchosen rows and columns are grounded [39]. Moreover, wire resistance consumes input voltages, so that memristors far from points of input voltage could be applied by smaller voltage than input voltage.  This influences output currents, leading to degradation of learning performance. The non-idealities in array-level could be overcome by device functions [35,44], operational scheme [39,[58][59][60], or learning algorithms [35,[40][41][42][43][44] to some degree.

Conclusion
Neuromorphic systems are one of the most promising candidates to deal with the von Neumann bottleneck caused by the memory wall between memory and process units. Using memristor synapses simply classified into cation-and anionbased devices can resolve this bottleneck owing to their storage and transmittance capabilities. To obtain higher performance of neuromorphic systems, representative characteristics, including the linearity of weight update, large multilevel states and dynamic range (ON/OFF ratio), variation and endurance, and retention need to be improved. In this context, different memristor synapses based on novel materials and device structures were introduced. Finally, we have briefly explained neuromorphic networks based on crossbar arrays of memristor synapses, and the network demonstrated perfect classification after 21 epochs. We believe that this chapter offers a deep understanding of the field of memristor synapses.