## 1. Introduction

Liquid crystals (LC) are characterized by the physical properties intermediate between ordinary isotropic fluids and solids [1]. LC flow like liquids but also exhibit some properties of crystals [1, 2]. The various phases in which such materials can exist are called mesophases [1, 2]. The LC molecules are large, anisotropic, and complex [2]. Dielectric constants, elastic constants, viscosities, absorption spectra, transition temperatures, anisotropies, and optical nonlinearities of LC are determined by the structure of these molecules [1, 2]. There exist three different types of LC: lyotropic, polymeric, and thermotropic [1, 2]. Lyotropic LC are obtained when an appropriate concentration of a material is dissolved in a solvent [2]. They can demonstrate a one-, two-, or three-dimensional positional order [2]. Liquid crystalline polymers are built up by the joining together the rigid mesogenic monomers [2]. Thermotropic LC exhibit different mesophases depending on temperature [1, 2]. Typically, they consist of organic molecules elongated in one direction and represented as rigid rods [2]. There are two types of LC sample orientation with respect to the boundary: (i) a homeotropic orientation when the long molecular axes are perpendicular to the boundary and (ii) a planar orientation when the long molecular axes are parallel to the boundary [1].

In this work, we consider only thermotropic LC, which are divided into three groups according to their symmetry: nematic LC (NLC), cholesteric LC (CLC), and smectic LC (SLC) [1, 2]. NLC are characterized by some long-range order in the direction of the molecular long axes, while the centers of gravity of the molecules do not have any long range order [1, 2]. The general direction of the molecules is defined by a unit vector function
^{*} LC consisting of the chiral molecules and possessing the spontaneous polarization; (v) different exotic smectic mesophases [1]. In this work, we consider only SALC. The SALC layered structure can be described by the one-dimensional mass density wave characterized by the complex order parameter. The modulus of this order parameter describes the mass density and its phase is related to smectic layer displacement

LC are highly nonlinear optical materials due to their complex physical structures, and their temperature, molecular orientation, mass density, electronic structure can be easily perturbed by an external optical field [2, 3, 4]. Almost all known nonlinear optical phenomena have been observed in LC in time scale range from picoseconds to hours, involving laser powers from 10^{6} Watt to 10^{−9} Watt, in different configurations such as bulk media, optical waveguides, optical resonators and cavities, and spatial light modulators [3]. For instance, a typical LC slab optical waveguide is a thin film of LC with a thickness of about

We investigated theoretically the nonlinear optical phenomena in SALC related to the specific mechanism of the cubic nonlinearity, which is determined by the smectic layer normal displacement

The theoretical analysis of the nonlinear optical phenomena in SALC related to the layer displacement was based on the simultaneous solution of the Maxwell equations for the optical waves propagating in SALC and the equation of motion for the SALC layers in the electric field of these waves. We used the slowly varying amplitude approximation (SVAA) [14]. We investigated the following nonlinear optical effects in SALC based on the layer displacement nonlinearity: self-focusing and self-trapping, SLS, and four-wave mixing (FWM) [5, 6, 7, 8, 9, 10]. We applied the developed theory of the nonlinear optical phenomena in SALC to the SPP interactions in SALC [11, 12, 13]. The SPP stimulated scattering in SALC and the metal/insulator/metal (MIM) plasmonic waveguide with the SALC core are theoretically studied [11, 12, 13]. The detailed calculations and complicated explicit analytical expressions can be found in Refs [5, 6, 7, 8, 9, 10, 11, 12, 13]. In this chapter, we describe the general approach to the theoretical analysis of the nonlinear optical phenomena in SALC and present the main results.

The chapter is constructed as follows. The equation of motion for the smectic layer normal displacement

## 2. The smectic layer equation of motion

The structure of the homeotropically oriented SALC in an external electric field
**Figure 1**.

The hydrodynamics of SALC is described by the following system of Eq. [1]

Here,

where

Here,
_{,} the purely orientational second term in the free energy density

Here,

Generally, for an arbitrary direction of the wave vector

Here,

## 3. Self-focusing and self-trapping of optical beams in SALC

We first consider the self-action effects of the optical waves propagating in an anisotropic inhomogeneous nonlinear medium. The light beam propagation through a nonlinear medium is accompanied by the intensity-dependent phase shift on the wavefront of the beam [2]. Self-focusing of light results from the wavefront distortion inflicted on the beam by itself while propagating in a nonlinear medium [14]. In such a case, the field-induced refractive change

Here,

The extraordinary wave is polarized in the

Here,
**Figure 2**.

The corresponding linear electric induction vectors

In the linear approximation, substituting Eqs. (14)–(16) into the wave Eq. (13) we obtain the dispersion relations for the ordinary and extraordinary waves, respectively [7, 18]

It should be noted that the ordinary and extraordinary beams in the uniaxial medium propagate in different directions and the vectors
_{,} which is determined by the angle

We consider separately the self-focusing and self-trapping of the ordinary and extraordinary beams [7]. We start with the analysis of the slab-shaped ordinary beam with the dimension in the

Expression (18) shows that the nonlinearity related to the smectic layer normal strain is the Kerr-type nonlinearity [14]. We introduce now the coordinates

Here,

We are interested in the spatially localized solutions with the following boundary conditions [7, 9]

Then, substituting expressions (14), (18), (19) and the first ones of Eq. (16), (17) into Eq. (13) and taking into account the SVAA conditions (20), we obtain the truncated equation for the SVA
_{,} which has the form [7]

Eq. (22) is the nonlinear Schrodinger equation (NSE) [19]. The coefficient of the last term in the left-hand side (LHS) of Eq. (22) is positive definite
_{,} which corresponds to the stationary two-dimensional self-focusing of the light beam. The solution of Eq. (22) with the boundary conditions (21) has the form [7]

The self-trapped beam (23) is the so-called spatial soliton with the width

The self-trapped ordinary beam normalized intensity spatial distribution is shown in **Figure 3**.

The self-trapping of the extraordinary beam (15) can be realized only when the anisotropy angle

Here,

For the typical values of the SALC parameters [1, 2], the optical beam electric field

The optical wave self-trapping can occur also at the interface between the linear medium in the region

The cubic susceptibility of SALC

## 4. Stimulated light scattering (SLS) in SALC

SLS is a process of parametric coupling between light waves and the material excitations of the medium [14]. We consider the SLS in SALC related to the smectic layer normal displacement and SS excited by the interfering optical waves [5, 6, 8, 9, 10]. We have taken into account the combined effect of SALC layered structure and anisotropy. It should be noted that SS propagates in SALC without the change of the mass density in such a way that the SS wave and the ordinary sound wave are decoupled [1].

In general case when the coupled optical waves have arbitrary polarizations and propagation directions, each optical wave in SALC
**Figure 4**. The XZ plane is chosen to be the propagation plane of the waves

Here,

Here,

The parametric amplification of the fundamental optical waves

Combining Eqs. (27)–(30), we obtain the nonlinear part of the electric induction, or the nonlinear polarization

We start with the analysis of the parametric coupling among the waves (27). Substituting expressions (27)–(30) and the phase-matched part of

(31) |

(32) |

(33) |

(34) |

Here,

Eqs. (31) and (32) describe the parametric energy exchange between the fundamental waves, Eqs. (33) and (34) describe the cross-phase modulation (XPM) of these waves [6]. Combining Eqs. (31) and (32), we obtain the Manley-Rowe relation, which expresses the conservation of the total photon number [14]. In our case, it has the form [6]

The solution of the system of Eqs. (31)–(34) can be written in the integral form [6, 10]

Here, the dimensionless variables are given by [6]:

Comparison of Eq. (36), (37), and (40) shows that for
_{,} which can be satisfied for

In general case, the exact analytical solution of Eqs. (31)–(34) is hardly possible. However, the explicit expressions for the coupled wave SVA have been obtained when both waves are propagating in the same XZ plane [5, 6]. For instance, assume that the pumping extraordinary wave with the frequency

Here,
**Figure 5**. The numerical estimations show that for the typical values of SALC parameters [1, 2, 3] in the resonant case the coupling constant per unit optical intensity
_{,} the SLS gain
_{,} which is at least an order of magnitude larger than the gain at Brillouin SLS in isotropic organic liquids [14]. Such optical intensities are feasible [20, 21].

The explicit expressions of the small component intensities

It is easy to see from Eq. (43) that for

The evaluation of the phases

The Brillouin-like SLS also results in the generation of six Stokes small harmonics with the frequency

## 5. The nondegenerate FWM in SALC

Consider now the nondegenerate FWM in SALC [8, 9, 10]. Assume that four coupled fundamental optical waves have different close frequencies

The interfering waves (44) create a dynamic grating of the smectic layer normal displacement of the type (28), but this time each harmonic has a different frequency

Here, the factors

In the special case when some ordinary optical waves (44) have perpendicular polarizations vectors
_{,} then FWM is divided in two separate two-wave mixing processes between the waves

In the important case when the pumping wave
_{,} the approximate solution can be obtained similarly to the solution (42) and (43) in the case of SLS [8, 9, 10]. It has been shown that this solution is stable in the case of FWM [8].

In the particular case when the fundamental waves (44) are counter-propagating, the phase conjugation is possible as a result of the nondegenerate FWM in SALC [8, 9, 10]. Optical phase conjugation (OPC) is the wavefront reversion property of a backward propagating optical wave with respect to a forward propagating wave [22]. The optical waves are phase conjugated to each other if their complex amplitudes are conjugated with respect to their phase factors [22]. Typically, OPC results from nonlinear optical processes such as FWM and SLS [20]. LC are commonly used for FWM and OPC [22].

Suppose that the waves

(46) |

Here,
_{,} we obtain the following solution for the probe wave and the phase-conjugate wave SVA

Analysis of the truncated equations for
^{−2} [9], which is feasible [20, 21]. OPC in the homeotropically oriented SALC film with the thickness of 250 μm had been demonstrated experimentally [20].

The components of the nonlinear electric induction
_{,} which are not phase matched to the fundamental waves (46) give rise to 12 doubly degenerate combination harmonics of the type.

## 6. Nonlinear interaction of surface plasmon polaritons (SPP) in SALC

Integration of strongly nonlinear LC with plasmonic structures and metamaterials would enable active switching and tuning operations with low threshold [4]. LC may be also used in reconfigurable metamaterials for tuning the resonant frequency, the transmission/ reflection coefficient, and the refractive index [23]. Combination of metamaterials and active plasmonic structures with NLC has been investigated [4, 23]. In this section, we discuss the nonlinear optical effects caused by the SPP mixing in SALC, which is characterized by low losses and a strong nonlinearity related to the smectic layer normal displacement without the change of the mass density [11, 12, 13]. Consider the interface
**Figure 6** [11, 12]. The SALC optical Z axis and the X axis are chosen to be perpendicular and parallel to the interface

SPP from the metal penetrate into SALC. The permittivity of the metal

The SPP are polarized as transverse magnetic (TM) waves with the electric field components
_{1,2} wave vectors are practically equal [11, 12]. They have the form [11, 12]

Numerical estimations show that for the typical values of
_{,} the SPP propagation length

Substituting the SPP fields (48) into the smectic layer equation of motion (9), we obtain the dynamic grating

Here,

(52) |

Unlike the dynamic grating (28) created by the interfering optical waves, the grating (50) caused by SPP is spatially localized both in the X and in the Z directions [11, 12]. The localized grating (50) can be characterized as an enhanced Rayleigh wave of SS [26]. Analysis of

We substitute the localized layer displacement
_{,} we obtain the following truncated equations for the normalized SPP intensities

Here,

Here,

It is easy to see from Eq. (55) that

Expressions (55) show that the energy exchange between SPP takes place. In the limiting case
_{,} we obtain:
**Figure 7**. The phases

It is easy to see from Eqs. (56) and (57) that for
**Figure 7**. It is much faster than the thermal response time

Structures consisting of alternative conducting and dielectric thin films are capable of guiding SPP light waves [24, 25]. Each single interface can sustain bound SPP. When the distance between adjacent interfaces is comparable or smaller than the SPP localization length

We consider the nonlinear optical processes in an MIM waveguide with the SALC core [13]. The structure of such a waveguide is shown in **Figure 8**. SPP propagating in the metal claddings and in SALC core are TM waves [24, 25]. The SPP electric and magnetic fields in the metallic claddings

The complex wave number

Numerical estimations show that for the typical values of the SPP frequency

The distribution of the TM even mode normalized intensity
**Figure 9**. It is seen from **Figure 9** that the intensity is filling the MIM waveguide core due to the overlapping of SPP inserted from the metallic claddings

The nonlinear polarization in the SALC core caused by the smectic layer strain (64) has the form [13]

Substituting the SPP electric field (63) and nonlinear polarization (65) into Eq. (13) and separating linear and nonlinear parts, we obtain the truncated equation for SVA

(66) |

At the distances
_{,} the SVA dependence on

Here,

(68) |

The solution of Eq. (67) has the form

Eq. (69) shows that the strong SPM of the even SPP mode in the MIM wave guide occurs. It is enhanced by a large geometric factor
_{,} which can achieve a value of

## 7. Conclusions

In SALC, there exists a specific mechanism of the optical nonlinearity related to the normal displacement

We derived the equation of motion (9) of the smectic layer displacement

In an optically uniaxial SALC, an ordinary wave and an extraordinary one can propagate. Both the ordinary and extraordinary optical beams propagating in SALC undergo self-focusing and self-trapping and form spatial solitons. The optical wave self-trapping can occur at the interface between the linear medium and SALC. We obtained the analytical solutions for the SVA of the self-trapped beams.

SLS of two arbitrary polarized optical waves in SALC transforms into the partially frequency degenerate FWM because each optical wave splits into the ordinary and extraordinary waves. The coupled optical waves create a dynamic grating of the smectic layer normal displacement

The nondegenerate FWM in SALC results in the amplification of the signal optical wave with the lowest frequency and depletion of three other waves with higher frequencies. The polarization-decoupled FWM may take place when the polarizations of some optical waves are perpendicular to one another. If the coupled optical waves are counter propagating and their frequencies satisfy the balance conditions typical for OPC process then BEFWM takes place accompanied by the amplification of the phase-conjugate wave. The spectrum of the scattered harmonics consists of 24 Stokes and anti-Stokes terms with combination frequencies and wave vectors.

LC applications in nanophotonics, plasmonics, and metamaterials attracted a wide interest due to the combination of LC large nonlinearity and strong localized electric fields of SPP. Until now, NLC applications in nanophotonics and plasmonics have been investigated. We studied theoretically the nonlinear optical processes at the interface of a metal and a homeotropically oriented SALC. In such a case, SPP penetrating into SALC create the spatially localized surface dynamic grating of smectic layer normal displacement. We have shown that for optical frequencies of about 10^{15} s^{−1} and coupled SPP frequency difference of about 10^{8} s^{−1}, the SALC-metal system cubic susceptibility may be one to two orders of magnitude larger than the cubic susceptibility of isotropic organic liquids. We solved the wave Eq. (13) for the counter-propagating SPP in SALC with the spatially localized nonlinear polarization and obtained the explicit expressions (55)–(57) for the magnitudes and phases of the coupled SPP SVA. It has been shown that the Rayleigh stimulated scattering of SPP on the surface smectic layer oscillations occurs. The rise time of the amplified SPP of about 10 ns can be achieved, which is much faster than the Brillouin relaxation constant in NLC.

The plasmonic waveguides with NLC for nanophotonic and plasmonic have been theoretically investigated. We proposed an MIM waveguide with an SALC core. We evaluated the electric field of the strongly localized SPP even mode, the smectic layer normal strain and the nonlinear polarization in the MIM core. The evaluation of the SPP SVA shows that the strong SPM process takes place. The SPM is enhanced by the geometric factor caused by the strong SPP localization in the MIM core.