## Abstract

We analyze the modulation instability induced by cross-phase modulation of two co-propagating optical beams in nonlinear fiber with the effect of higher-order dispersion and septic nonlinearity. We investigate in detail the effect of relaxation nonlinear response to the gain spectrum both in normal group velocity dispersion (GVD) and anomalous dispersion regime. We show that the walk-off, the relaxation nonlinear response time as well as the higher-order process particularly influence the generation of the modulation instability gain. Our results shows that the emerging Raman peaks is observable both in the case of weak dispersion and in a higher-order dispersion for mixed GVD regime with slow response time. These Raman peaks are shifted toward higher frequencies with the decrease of their magnitude, when the walk-off increases.

### Keywords

- septic nonlinearity
- higher-order dispersions
- walk-off effects
- delay response time
- cross-phase modulation

## 1. Introduction

The generation of a wave train is a preoccupying subject in the realm of nonlinear science. This is mainly due to two effects: nonlinearity and dispersion. These two notions are essential in the propagation of the wave over long distances and the optical pulse resulting from this interaction gives rise to an optical soliton. The dynamic evolution of nonlinear pulses in nonlinear optical systems can be modeled by the well known nonlinear Schrödinger (NLS) equation which represents the lowest-order nontrivial condition describing the propagation process [1]. The co-propagation of two nonlinear waves in nonlinear optical Kerr media under a slowly varying amplitude approximation is made by using extensions of the NLS equation, whose analytical results provide the dispersion relation, the unstable conditions, as well as the gain spectra. This extension of NLS equation can take into account a large variety of physical properties such as higher-orders dispersion (like third-order dispersion (TOD) and fourth-order dispersion (FOD)) [2, 3, 4, 5, 6, 7, 8, 9]; multiple optical beams [10]; negative index material [11]; saturable nonlinearity [12]; and non-instantaneous nonlinear response [13]. Third-order dispersion is used to describe the proprieties of ultrashort pulses in the subpicosecond to femtosecond domain. Usually in nonlinear optic, Kerr nonlinearity is used to compensate the dispersion effects leading to the formation of soliton. Due to this property, focusing Kerr-type media promote the stable propagation of bright solitons [14]. Despite this, the diffraction effect is not enough to balance the self-focusing in focusing quintic nonlinear media and therefore the pulse undergoes critical collapse [15]. Thus, higher-order nonlinearities (HON) play an important role in the description and the propagation of the pulses in Kerr media. Consequently, the extension of NLS equation can also take into account the effect of HON. Reyna et al. [16, 17, 18] have experimentally and numerically investigated the nonlinearity management and spatial modulation instability for cubic, quintic, and septic nonlinearity for optical beams propagation in metal-dielectric nanocomposites.

The study of the propagation of an intense optical beams through a nonlinear and dispersive media may lead to fascinating effects such as exponential growth of amplitude due to modulation in amplitude and frequency. This effect is called modulation instability (MI). Modulation instability is a fundamental phenomenon firstly detected in hydrodynamic systems [19] and appears in most nonlinear wave systems. In nonlinear Kerr media such as optical fiber, MI results from the interaction between the nonlinear and dispersive effects [20] characterized by the instability undergone by a continuous wave (cw) or quasi-cw when it propagates inside a nonlinear dispersive system with low noise [21]. Modulation instability has been studied for waves in fluids dynamics [22], plasmas physic [23], dielectric media [24], electrodynamics [25], and atomic Bose-Einstein condensates [26, 27] and was first analyzed theoretically in glass fiber by Hasegawa and Brinkman [28] in 1984 to study the generation of ultrashort optical beams. This phenomenon is generally studied in the anomalous dispersion regime, but it can also be observed in normal dispersion regime when the pumping is carried out close to the zero dispersion subject to certain conditions on the higher order dispersion (HOD) coefficients [29, 30]. Cavalcanti et al. [31] predicted the possibility of MI to occur even in normal group velocity dispersion regime due to the negative values of fourth-order dispersion (FOD). The effects of FOD was also been investigated by many authors who show their inclusion leads to generation of new spectral window. Tchofo et al. have analytically and numerically investigated the behavior of MI under the combined effects of HOD and delayed Raman response [5, 6]. In Ref. [7], the authors shown that FOD shifts the MI peak gain to the higher frequency side and also increases the instability region. Nithyanandan et al. [8] analyzed that the cumulative effect of HOD and walk-off brings new characteristic spectral bands at a definite frequency window.

Wang et al. have shown in their work that the distribution of speed of system flow and the speed of sound determining the occurrence of the sonic horizon is in agreement with the corresponding quantities obtained from a pure numerical evaluation for quantum system incorporating septic nonlinearity modeled by NLS equation [32]. An essential manifestation of the intensity dependence of the refractive index in nonlinear optical media rises through self-phase modulation (SPM) [33], which leads to spectral spreading of optical pulses. It is well known that the co-propagation of two optical waves in nonlinear Kerr media is coupled due to refractive index of the media through the nonlinear phenomenon called cross-phase modulation (XPM) [29, 34].

In this paper, we study the MI in non-instantaneous Kerr media with cubic-quintic-septic nonlinearity, described by a system of two-coupled NLS equation. We mainly focus on revealing the contribution of group velocity mismatch

## 2. Model equations and linear stability analysis

The optical electromagnetic field propagations are described, under a slowly varying amplitude approximation by a coupled nonlinear Schrödinger equation (CNLSE) with higher order dispersion [8] in a single-mode optical Kerr media where higher-order nonlinearities [16] are taken into account. These equations result from two optical beams at different frequencies and the same polarizations where time-dependent nonlinear response is incorporated in the system. The governing systems are read as:

The nonlinear Kerr parameters

Reyna et al. [16] have proposed this class of HON to study the spatial phase modulation induced by quintic and septic nonlinearities for optical beams propagation in metal colloids, where septic nonlinearity arises from the development up to seventh-order susceptibility of theoretical treatment of Maxwell-Garnett model. Quintic nonlinearity has been considered by Kumar [35] to show the influence of spatial delay on the modulational instability in a composite system with a controllable nonlinearity.

We extend the previous works on MI taking into account the CNLS with higher-order effects and using the linear stability approach. Therefore, we study the stability of the steady-state solution of the above dynamical equations against the small harmonic perturbations; the steady-state solution of the continuous wave (cw) is obtained by setting the time derivative in Eqs. (1)–(4) to zero. This leads the exacts solution in the form:

and

where

with

where

where

where the parameters of this equation are defined by the following relation:

With the parameters

Hence, for usual Kerr approach

The condition so that MI can occur read as:

This condition shows that MI depends on the sign of

By considering the effects of relaxation time

## 3. Effects of weak dispersion on XPM-induced modulation instability

We investigate the effect of HON on MI gain in the case of low dispersion regime (

### 3.1 Roles of walk-off and relaxation of nonlinear response on modulation instability gain spectra

First, let us briefly consider the case of instantaneous response time

Now, considering the effect of delay response time by setting GVM null, Eq. (15) leads to four unstable modes which turns to two as given in Figure 1(b). We note from this figure that, for ultra-fast response time, the behavior is as for fast response time for cubic nonlinearity, and the rest of the curves is as described in quintic nonlinearity [39]. As already mentioned, the non-instantaneous response time brings new bands attributed to Raman effect involves in the fibers.

### 3.2 Combined effects of walk-off and delay response time on modulation instability gain spectra

Taking into account the combined effects of delay response time and GVM, Eq. (15) leads to four unstable modes that turns to two for response time reaching certain value

Now considering that GVM varies with fixed nonlinear response time, we have observed that when the GVM increases, the behavior of the MI gain spectra changes drastically as given in Figure 3 for mixed GVD. In this figure, the four unstable modes coalesce into one at high frequencies.

However, for fast response time gives in Figure 3(a), we observe that for conventional and Raman band, the magnitude increases when

In the case of anomalous dispersion regime, the four unstable mode turns to two modes as given in Figure 4. One can observe from this figure that, for conventional spectral band, as GVM increases, the MI gain decreases, while for Raman band (second spectral band), it is inverted. However, the behavior of the MI gain spectra is different from these panels and we can see that as nonlinear response time increases, the magnitude of the gain decreases. In panel (a), we note that before extended in all frequencies, the curves are neutralized at a given frequency and move slowly toward high frequencies while in panel (b), they move rapidly as GVM increases.

In the case of normal dispersion regime, for slow response time [panel (a)], the two unstable modes turn to one for lower frequencies and to two at characteristic frequency until colliding to one at high frequencies. The previous description in anomalous case remains the same. Meanwhile for fast response time [panel (b)], the situation is slightly different where one can observe two unstable modes that collid to one as GVM increases. Figure 5 portrays this feature. The Raman and sporadic peaks observed previously in Figure 3 disappeared in these figures due probably to the sign of SOD (they have the same sign).

## 4. Impact of higher-order dispersion and septic nonlinearity on XPM-induced modulation instability

It has been shown that HOD effects such as FOD and TOD are most essential and give rise to a wide diversity of information concerning the MI dynamics.

### 4.1 Effect of fourth-order dispersion on modulation instability

The inclusion of FOD brings a great change to the MI spectrum and we obtain for each GVM value four gain spectra. In the case of mixed dispersion regime, from Figure 6 we note that, the spectrum presents two regions of instability which are connected to each other. The first one at the pump frequency is separated to the second one due to the effect of FOD. Thus in the case of fast delay response time gives in panel (a), one can observe that for the first instability region, the primary band remains unchanged for all GVM values; while for the Raman band as GVM increases, the MI gain spectra increase. For the second instability region, the situation remains the same as in the mixed case for weak dispersion regime (Figure 3(a)). In panel (b), our work unveiled that for first instability region, the conventional spectral MI gain decreases as GVM increases, while it is inverted for Raman bands. According to the second instability region, we revealed the presence of Raman and sporadic peaks that magnitude and width decrease as GVM increases and they move toward high frequencies. The fourth unstable mode turns to one at high frequencies. In addition at around

Taking into account the propagation with anomalous GVM qualitatively changes the MI spectrum, like shape as in Figure 7 that for the primary spectral band as GVM increases, the MI gain spectra decrease, contrarily to the Raman bands which increase as GVM increases. In addition, when increasing the delay response time, the magnitude and the width of MI gain spectra decrease considerably with a singularity around

### 4.2 Effect of third-order dispersion on modulation instability

We now shift our analysis to the effect of walk-off and delay response time in virtue of TOD. In the case of mixed GVD as seen in Figure 9, the spectra hardly change for

In the case of anomalous dispersion regime, the four obtained solution leads to four unstable modes that coalesce to one at higher frequencies. Figure 10 depicts the MI gain spectrum for some GVM value. One observed two singularities due certainly by HOD effects. In the case of fast response time, for first spectral region in panel (a), the primary and Raman bands decrease as GVM increases while for spectra obtained at low MI gain in this region the situation is inverted. In the second region, the inclusion of TOD brings Raman and sporadic peaks that shifted toward higher frequencies due to GVM effect. For slow response time given in panel (b), the situation is almost different as earlier. The sporadic and Raman peaks get shrink and still move toward higher frequencies. It is to explain the behavior of the obtained modes at lower frequencies coalesce to two at middle frequencies and to one at high frequencies. Contrarily to what observed in the case of quintic nonlinearity, in these figures (Figure 10), HOD acts in the case of septic nonlinearity.

We now turn our attention to the case of normal dispersion regime as given in Figure 11. We note that, the second spectral region remains almost the same as described in previous figure. However, in the first spectral region, the four solution leads to two unstable modes in panel (a) and to one in panel (b) for lower frequencies. The effect of FOD vanishes due probably to the dominance of SOD and HON. Obviously, we note that as response time increases, the Raman and sporadic peaks get shrink.

Figure 12 depicts the evolution of MI gain spectra in the case of “total mixed” GVD. It is obvious to see from panel (a) (fast response time) that as GVM increases, the conventional bands decrease while it is inverted for Raman bands. The effects of TOD is visible around

We generally note from these different figures that, the inclusion of TOD and FOD increases considerably the MI gain spectrum and their width. In addition, the formation of Raman and sporadic peaks observed is due on the one hand to the combined effect of SOD and relaxation time and on the other hand due to the combined effect of TOD. They move toward high frequencies due to the effect of GVM. From fast to slow response time, we note that the magnitude of MI gain spectra decreases considerably. The MI occurs in the case of HOD and septic nonlinearity only when the interactions between beams are less (gives here by walk-off parameter

### 4.3 Effect of septic nonlinearity parameter κ 3 on modulation instability gain

To better appreciate the effect of septic nonlinearity on MI gain, we portrayed the latter versus

We have plotted the behavior of the MI gain in Figure 13 for normal GVD. We observe from this figure that MI appears on both side of

In the case of anomalous dispersion regime plotted in Figure 14, we notice that the side bands brought by FOD are suppressed due to their opposite sign with the sign of SOD, the latter dominates and leads to the observed spectrum. Apart from this difference, the information obtained in the normal case remains almost the same. We now turns our attention in the case of mixed dispersion regime gives in Figure 15. Our results show that contrarily to what are observed in previous figures, the bandwidth is clearly visible for

## 5. Conclusion

In this work, we have investigated analytically and numerically the MI process in nonlinear Kerr media with delayed Kerr nonlinearity and subjected to XPM, described by a system of two coupled NLS equation with cubic-quintic-septic nonlinearities, experiencing diverse GVD regime. The system is considered by incorporating the Debye relaxational time-dependent nonlinearities in NLS equation with combined high-order effects modeling the propagation of ultrashort optical pulses in highly nonlinear Kerr media. By considering a small harmonic perturbations to the stationary solutions, we derive the exact dispersion relation for the components of the perturbation fields that include both the XPM and relaxation effects. Our study revealed the effects of septic nonlinearity in the case of the pulses co-propagating in diverse GVD regime. Obviously from our work, the incorporation of septic nonlinearity increases considerably the MI gain spectrum with the emergence of new instabilities frequencies contrarily to that was observed in Ref. [8, 9, 38, 39], while the increase of delay response time decreases the magnitude and the width of the MI gain.

Our study unveiled in the case of weak dispersion regime that, in instantaneous Kerr media, the typical frequency converges to finite value for mixed GVD regime with two gain maxima for certain GVM, meanwhile the MI gain continuously grows with the GVM until it reaches certain values before decays. In non-instantaneous Kerr media, we note the appearance of Raman band with the emergence of Raman peaks for slow response time

Now considering the effect of HOD, we observed that their incorporation hardly change the spectrum. The inclusion of FOD does’t affect the MI gain in normal dispersion regime contrarily to the works done in the case of cubic and quintic nonlinearities [8, 9, 39] where the effect of FOD vanishes in the case of anomalous GVD. The effects of TOD vanish in the case of mixed dispersion regime. However, in the case of anomalous and normal dispersion regime, we note the appearance of Raman peaks that width and magnitude decrease as GVM increases and get shrink as response time increases. In addition, the four unstable modes observed in fast response time leads to two unstable modes for slow response time. For “total mixed” dispersion regime, we also note the appearance of Raman peaks that slightly shifted toward higher frequencies due to the effects of GVM. The effects of septic nonlinearity was also portrayed and our results unveil the appearance of instability in both side of septic nonlinearity parameter. This aspect shows that MI can be investigated as well as in focusing nonlinearity as in defocusing nonlinearity.

Due to the fact that the MI gain increases when taking into account the effect of HON, we can conclude that the system is more stable against harmonic perturbations. In addition, the incorporation of higher order effects lead to the emergence of Raman peaks on the Raman bands. This means that the incident particles after collision with the molecules of the medium, diffuse with a new energy thus creating the Raman peaks observed resulting to the generation of new frequencies range. The study of MI gain has aroused great enthusiasm within the scientific community, motivated in large part by their potential in innovative applications in information technology and telecommunications.