A Set of Proposals to Overcome the Crisis in Atmospheric Remote Sensing Technologies

Abstract

This chapter summarizes proposals to expand the capabilities of lidar measurements of the Earth’s atmosphere as a structural stratified medium with complex thermodynamics. The generalized structure of the revealed, statistically stable horizontal optical inhomogeneities of the surface layer and their dynamics are taken as an informative indicator of the thermodynamic stability of the atmosphere in the zone even beyond the limits of measurements. A special statistical method for analyzing the two identified structures is used. It is based on the separate fixation and statistical averaging of the intervals between local maxima and minima of the lidar backscattering signal and the analysis of the time dynamics of such an interval component of the signal during the night, under anticyclone conditions. It assumed that this approach makes it possible to identify certain statistically stable structures of atmospheric inhomogeneities of varying connectivity, depending on the overall thermodynamic stability of the atmosphere. In addition, a specific method is tested to search for emerging and temporarily stable anomalies of such a structure of atmospheric inhomogeneities, but associated with a violation of the thermodynamic stability of the atmosphere. Such a filtration method (identification of a certain anomaly against the background of an extended organized structure of inhomogeneities) showed high efficiency in model experiments compared to standard filtration methods.

1. Introduction

Classical lidar measurements of the atmosphere assumed obtaining information about the properties of the medium by recording and special processing of the backscattering signal, usually in the photon counting mode. The received signal assumes its digitization and sampling with a given constant resolution. As a result, we get a certain sequence of numeric values that are used in subsequent processing to restore the properties of the medium based on such a signal [1, 2]. This approach to the organization of measurements grew out of laboratory experiments conducted in an environment with stable properties and stable thermodynamics. This standard technology for measuring and processing the received signal leads to an exponential increase in technical problems when trying to develop such an approach. These difficulties led to a reorientation of lidar systems to create models and maps of objects based on a signal directly reflected from the underlying surface. The purpose of the research is to find ways to circumvent the identified problems for the further development of lidar measurements of atmospheric properties directly, to increase their effectiveness by revising the temporary organization of sampling and digitization of the received signal and reorientation to more effective and informative research directions. The complex of ideas was formed by 1989.

One of such directions is the study of the processes of formation of the structure of optical inhomogeneities in the atmosphere. Under certain conditions, the atmosphere is a stratified medium in which the so-called internal gravitational waves arise and propagate [3, 4, 5]. The sources are: flows over an inhomogeneous surface, local jumps in the thermodynamic characteristics of the medium (on medium seals, on atmospheric fronts, on sources of turbulence in the boundary layer above the surface, during convection under the temperature inversion layer. Favorable conditions for the formation of such structures arise in the surface layer, especially at night, in anticyclone conditions.

In the presence of surface temperature inversion and extended fields of polluting urban aerosol, at the boundaries of stratification layers, the amplitude of internal waves increases, becoming longer than the wavelength, which leads to loss of stability and collapse of waves. The potential energy of the wave is used to mix the medium in the collapse zone, which leads to the formation of turbulent spots of relative uniformity in density. Moreover, they have sharp boundaries and a long scattering time (scattering velocity ~ 10 cm/s). Stable stratification of the medium helps to suppress turbulence in the flow, but in such spots it exists for a long time. This helps to retain a relatively coarse aerosol in such spots, which usually settles under the influence of gravity, which leads to intense light scattering on such structures. The degree of scattering increases with increasing power of the turbulent structure and with the concentration of the polluting aerosol, which serves as a kind of “developer” of the structure of inhomogeneities.

Indeed, visual [6], radar [7, 8], radiometric [9], lidar [10, 11, 12, 13], and radar [14, 15] measurements indicate the presence of similar ordered structures in the atmosphere. In some cases, their length in the horizontal plane reaches tens or even hundreds of kilometers. The time of their existence varies from seconds (for small-sized structures), minutes, to several hours. The size of the registered formations for the surface layer varies from meters to hundreds of meters, and their contrast varies accordingly from percent to tens of percent. A certain condition for the stability of these ordered structures is the stability of the vertical stratification of the medium, which in this case can be maintained in time intervals from several minutes to several days. However, the existing data only allow us to state the existence of a connection, without specifying the conditions of its existence, due to the low efficiency and insufficient information content of the measurements carried out. Local measurements at a limited number of points do not allow us to specify the nature and features of the relationship of the structure of inhomogeneities of atmospheric formations with the meteorological situation and its stability. The assumption that variations in the power of the scattering signal are determined by fluctuations in the attenuation coefficient of backscattered radiation associated with fluctuations in aerosol density in turbulent structures is confirmed by a whole set of data [5, 6, 16, 17]. This suggests the prospects of using lidar systems for this type of tasks, with a certain agreement between the properties of the equipment and the medium under study.

2. Formulation of the problems

By now, the energy growth of lidar systems is achieved with a significant increase in weight, dimensions and a clear violation of environmental safety. With the radiation energy unchanged, a decrease in the measurement error is accompanied by a deterioration in the spatial resolution along the sounding path. Increasing the accuracy of measurements has limitations due to the temporal variability of the medium itself. Under these conditions, attempts to reduce the measurement error of the signal by increasing the signal registration gate or by accumulating the signal in a series of measurements are blocked by a concomitant deterioration in spatial and temporal resolution. Such limitations are caused by a number of implicit contradictions present in the technologies of remote measurement and processing of the obtained results, which have escaped the attention of researchers.

2.1 Conflict 1

The confrontation between the zone of ongoing large-scale processes under study and the desire for hardware sampling of the incoming signal with increasing resolution. This excludes the assumption of the medium as a coherent whole formation, with obviously complex thermodynamics, and reduces the entire analysis of the properties of a specific, virtually isolated point of the medium (in the extreme case).

2.2 Conflict 2

The confrontation between the requirement to increase the required energy of the pulsed laser used, to increase both the measurement range and the spatiotemporal resolution of the recorded signal while reducing the environmental safety of the measurements. The harmful effects on natural processes are increasing and the danger of irreversible effects on humans and wildlife is increasing.

2.3 Conflict 3

The confrontation between the orientation to measuring the properties of the medium at a point and volumetric thermodynamic processes that affect the overall organization and stability of the spatial and temporal structure of the inhomogeneities of the medium.

2.4 Conflict 4

The confrontation between the need to smooth out signal changes caused by the structure of its inhomogeneities, background noise, and the need to amplify signal changes caused by the appearance of abnormal inhomogeneity in the medium (not peculiar to its equilibrium state).

3. Overcoming technologies

3.1 Technology 1

Overcoming the first contradiction involves changing the sampling algorithm of the incoming backscattering signal and its subsequent digitization. Here, the concept of sampling with variable duration is introduced, when the minimum duration of the reference corresponds to the time when the radiation pulse passes through half of the probing path in the zone of single scattering, when two sets of samples are formed [18]. The first set is formed with a constant initial boundary and the growth of each subsequent sample by the required sampling step for the far half of the route. The second set is formed accordingly with a constant final boundary for the near half of the route. Similarly, the second sequence of samples from the far half of the sensing path is organized accordingly by forming an initial sample from the scattering signal of the probing signal from the far half of the sensing path (Figure 1).

Moreover, the length of the route corresponds to the zone of single signal scattering and provides a signal-to-noise ratio of at least the selected one in each sample, and the noise consists of external illumination and internal noise of the equipment itself. Similarly, the second sequence of counts of the far half of the route is formed by forming an initial count from the scattering signal of the probing signal from the far half of the route (Figure 1).

Figure 1.

The principle of the temporary organization of double (counter) discretization of samples of the backscattered lidar signal.

The rationale for the chosen discretization is based on numerical simulations based on real measurements with a constant sampling step carried out in the laboratory of condensed matter at the Belarusian State University [3, 4, 5]. The results of such a comparative experiment [6] are shown in the Figure 2.

Figure 2.

The dependence of the normalized measurement error on the length of the measurement zone for registration gates or pulses of probing radiation commensurate with the length of the measurement zone (new system) in relation to the conventional, standard system (old system).

The presented graph is illustrative. Eight measurement error values are given, normalized for the error at the end of the selected sounding route for the old system, with a characteristic atmospheric attenuation coefficient of 0.3 km^-1 and a uniform step along the route. The specific quantitative characteristics of such modeling are presented in detail in Refs. [17, 18].

For the proposed sampling technology of the incoming signal, the error varies slightly from strobe to strobe over most of the route, which makes it possible to reduce hardware requirements due to the small dynamic range of the recorded signal.

The restoration of the characteristics of the medium with the required resolution is achieved by sequentially comparing neighboring pairs of samples having a constant initial (final) boundary of each sample and differing in the final boundaries for the selected sampling step (see Figure 1).

The simulation results showed that the proposed measurement scheme makes it possible to carry out measurements at a high level of daytime illumination conditions, while maintaining an error in the range of ~1 … 10% [19]. In addition, the approach used makes it possible to shift the task of improving measurement accuracy from an area related to the environment to an area related only to hardware capabilities, to an area more subject to control.

3.2 Technology 2

Overcoming the second contradiction involves the transition from using a high-power pulsed laser to using a continuous radiation laser of significantly lower power. For this purpose, it is proposed to use the zone of a single interruption of radiation for a time close to the time of radiation of a pulsed laser of a conventional lidar system as a probing “dark” pulse [20, 21]. Parallel modeling of measurements for such a system has shown that the above conclusions on the resolution of the first contradiction are applicable in this case. Continuous 0.5 watt laser radiation can provide the necessary sensing ranges, measurement accuracy and resolution along the sensing path. Combining both technologies for both the radiating part of the equipment and its recording part will allow for even better results.

3.3 Technology 3

Overcoming the third contradiction involves shifting the focus of attention from classical studies of properties at each point to the medium as an extended object with a certain stratification, the properties of which depend on the stability of its thermodynamic state. As such a criterion, the generalized value of the difference between the given Richardson numbers taken, which was calculated according to a certain method based on meteorological parameters obtained from the nearest weather station [18]. The search for signs of organization in the structure of the recorded signal and its correlation with the structure of the real environment, its variability and restructuring associated with a violation of the stability of the thermodynamic state not only in the measurement zone but also beyond it, comes to the fore.

Lidar studies of the atmosphere at night in the surface layer under conditions of a fairly stable anticyclone [19] have shown that the structure of horizontal optical inhomogeneities is quite regular [22, 23]. In some cases, this is clearly observed as the collapse of so-called internal waves (such as Kelvin-Helmholtz waves). A clear example of this situation is presented below (Figure 3).

Figure 3.

Clouds along the horizon in Birmingham, Alabama, December 16, 2011 (image credit: ABC 33/40 in Birmingham).

Lidar sensing makes it possible to register a backscattering signal from the atmosphere as the radiation pulse passes through the selected sensing path. Subsequent processing of such a signal using a special technique allows us to uniquely correlate the structure of the transformed signal with the structure of atmospheric inhomogeneities [24]. The complex nature of such structures and their dynamics forced us to abandon the initial harmonic analysis, exclude the amplitude characteristic of the signal from consideration and switch to an independent analysis of the intervals between local maxima or minima of the transformed signal using statistical analysis. Previously, a model experiment was conducted in which the harmonic signal was distorted in a certain way. In harmonic analysis, such distortions appear as a weak expansion of the peak of the corresponding correlation function (when compared with an accurate sine wave). In our case, such distortion manifests itself as a clear additional peak, which indicates a high probability of detecting such distortions in a fairly simple way (Figure 4).

Figure 4.

Normalized cross-correlation function of power spectra or time interval distributions between signal extremes for accurate and distorted sinusoids, respectively (numerical simulation).

In the case of processing real lidar signals, the ultimate goal was to obtain a model in the form of a generalized interval structure of inhomogeneities [11, 12]. The mathematical expectations of the intervals between local maxima (M1), local minima (M2), maxima and minima (M3), as well as their variance were taken as characteristics of the identified structures (Figure 5).

Figure 5.

The results of determining the generalized interval between the elements of the structure of different types (M1, M2, M3), for the optical heterogeneities of the atmosphere in the five selected sensing areas.

As can be seen, there is a changing difference between the “positive” (M1) and “negative” (M2) structure of inhomogeneities and the intervals between them (M3). In addition, such an analysis allows us to relate the features of the detected structures to the thermodynamic stability of the atmosphere. To do this, we used data on meteorological parameters from the nearest weather station. The generalized Richardson number [13] was used as a criterion for the thermodynamic stability of the medium, which was calculated on the basis of the mentioned meteorological parameters. It turned out that the structure of inhomogeneities becomes more complicated with a decrease in the stability of the atmosphere. There is a transition from a simple regular structure of inhomogeneities, the appearance of long-wave component, through an intermediate stage of growth of the dispersion of the initial structure [24, 25, 26, 27]. Moreover, the statistical relationship between the “positive” and “negative” structure decreased with decreasing thermodynamic stability of the atmosphere. They became more and more independent, which is obviously not revealed by standard Fourier analysis methods.

3.4 Technology 4

Overcoming the fourth contradiction involves the use of a nonlinear signal filtering method, when, after sampling and digitization, its periodic inhomogeneities are significantly smoothed out, and one-time anomalies are amplified. Numerical modeling was carried out using an interactive system of spectral statistical processing of a noisy model signal. An exponentially decreasing signal with a drop in the center of the sample of 50 samples was used [28, 29]. The effectiveness of such filtration has been verified in a numerical experiment, when compared with linear and median filtration. An example of such filtering of a model signal is shown in Figure 6.

Figure 6.

A model signal with an introduced difference at the noise level and an imposed structure (upper graph) and filtering results revealing such a drop (lower graph), where the ordinate axis is the magnitude of the signal, and the abscissa axis is the number of samples.

Filtration allows you to clearly identify the boundary of the difference. The signal after the drop exceeds the original value. In this case, linear and median filtering are not effective and do not allow identifying the introduced drop in the model signal (Figure 7).

Figure 7.

Results of comparison of the proposed processing of the incoming scattering signal (nonlinear filtering—GRAN) with linear and median filtering methods (LINF, MEDF). DISP1, DISP2—dispersion before and after the introduced signal difference, SIG1, 2, 3—signals at the ratio between the maximum value, drop and noise: 10:1:1; 10:1:5; 10:1:10, respectively.

It should be expected that the transition from an anticyclone-type situation to a cyclone will cause a significant transformation of the structure of heterogeneities, even before the appearance of obvious thermodynamic signs of such a transition. The conducted studies suggest that the occurrence of thermodynamic instability of the atmosphere, even outside the observation zone, leads to a proactive sharp violation of the stability of the structure of optical heterogeneities of the atmosphere, long before a visible significant change in the situation in the observation zone. Predictors of such a change may be some kind of soliton-type anomalies and can be clearly identified by the described filtering.

3.5 Technology 5

One of the visual options for representing a meteorological situation may be the creation of a generalized model structure of inhomogeneities (based on the results of a large-scale array of real measurements) as a basis for creating a visual image of a developing nonlinear process in a continuous environment, as an integral object.

A structure such as the surface of a model material (similar to the structure of a hologram) can be used as a holographic visual image of the process. The final stage of the proposed technology may be the use of information about the unit cell of the structure of inhomogeneities to create a kind of model of the surface of the material with a structure similar to that of a hologram. The dynamics of the generated holographic images of the structure of inhomogeneities will allow us to visualize the process of changing the stability of the situation. The more catastrophic the change, the more complex the recorded structure and the formed holographic image acquires greater clarity. You can even use the symbolic name of such an image - “Smile of the Devil”. His “facial expressions” will allow, without going into complex mathematical operations, to quickly fix the degree of catastrophism of developing processes. One of the visual options for representing the meteorological situation may be the creation of a generalized model structure of heterogeneities (based on the results of a volumetric array of real measurements) as a basis for creating a visual image of a developing nonlinear process in a continuous environment, as an integral entity.

4. Conclusion

The proposed set of technologies for measuring, recording, and processing remote measurements makes it possible to create lidar sensors for monitoring the stability of a meteorological situation and use them in a network of standard weather stations.

Such technologies are applicable for a wide range of tasks for processing noise-like signals from various continuous media, where it is possible to relate the complexity of the structure of heterogeneities to the “thermodynamic” stability of the situation, for example, in the banking sector or sociological research.