MST radar signal processing using iterative adaptive approach

Abstract

Power spectrum is the considerable aspect in the atmospheric radar data processing to estimate wind parameters. Due to the poor resolution and high sidelobe level problems of the existing algorithms, there is a requisite for the novel data-dependent approaches. A non-parametric and hyperparameter-free iterative adaptive approach (IAA) is presented for the power spectral density estimation. This approach is able to work with single snapshot and is obtained by minimizing the weighted least square fitting criterion. The IAA method provides the accurate amplitude and frequency estimation for the simulated data. The data for the above study is collected from Indian MST (mesosphere, stratosphere, and troposphere) radar. The power spectrum and Doppler frequency are estimated using IAA. In this paper, zonal (U), meridional (V), windspeed (W) are also calculated and validated using Global Positioning System Sonde data. The effectiveness of the spectral estimation performance showed by IAA is demonstrated and assessed.

Background

Indian MST radar provides information on wind data above 3.5 km with a resolution of 150 m. The three wind components U, V and W are determined by the Doppler beam swinging (DBS) method of the MST radar. The radar collects the data using multiple beam positions with 16 μs coded pulse and with an inter pulse period (IPP) of 100 μs. The online calculation of Doppler power spectra for each range bin can be obtained by subjecting the complex time series data to the process of fast Fourier transform (FFT). The DC removal, average noise level estimation, interference removal and incoherent integration are the steps that are involved in offline data processing. The 0th, 1st and 2nd moments denotes the signal strength, mean Doppler shift and half width parameters of the spectrum are computed, respectively.

The accurate estimation of the Doppler frequency is the crucial one in the detection and estimation of the wind speed by the MST radar. A package for processing the radar data has been developed by the National Atmospheric Research Laboratory (NARL), Gadanki, Andhra Pradesh, India. It is known as the atmospheric data processor (ADP) (Anandan 2002). The Doppler frequencies can be accurately estimated by the ADP up to certain heights. Since the signals are highly corrupted with noise at higher altitudes, the ADP is unable to estimate the Doppler frequencies and thus the wind speed. It can be seen in the literature that several algorithms have been put to use to accurately estimate the Doppler frequencies from MST radar data.

Bispectral estimation algorithm (Anandan et al. 2001) is applied to radar at a high computational cost. The multitaper-based spectral estimation (Anandan et al. 2004) has the advantage of reduction in variance and has been applied to the radar data. However, it has spectral peak broadening. An adaptive estimation technique has been presented to estimate the Doppler spectra, with certain parameters to adaptively track the signal in the range of Doppler spectral frame (Anandan et al. 2005). Several methods like wavelet-based denoising (Thatiparthi et al. 2009) and cepstral thresholding (Reddy and Reddy 2010a, b) have also been used for spectrum cleaning and then Doppler spectrum estimation leading to the calculation of wind velocities. According to (Reddy and Reddy 2010a, b), a polyphase approach was employed for spectrum estimation using uniform filter banks. Recent studies applied principal component analysis (PCA) on the radar data, prior to periodogram estimation using Blackman–Tukey, and minimum variance methods (Uma Maheswara Rao et al. 2014). All the existing methods used in spectral estimation for atmospheric radar data falls under two estimation methods either parametric or nonparametric. However, parametric method requires the prior knowledge of some parameters and nonparametric method has global leakage (peaks at unwanted frequencies) and local leakage (main beam limits). In the present study, the iterative adaptive approach (IAA) (Yardibi et al. 2010) is put into use to estimate the spectrum of MST radar data collected from NARL. The IAA algorithm is based on weighted least squares minimization and is found to give excellent results for simulated data and real-time data.

In this paper, lowercase and uppercase boldface letters represent vectors and matrices, respectively. Normal letters are used to indicate scalar quantities. Notations $| \cdot |$, $\left\| \cdot \right\|$, $( \cdot )^{*}$, $( \cdot )^{T}$, and $E( \cdot )$ denote the modulus, the Frobenius norm, the complex conjugate transpose (Hermitian transpose), the transpose and the expectation operator. Subscript $[ \cdot ]_{k}$ denotes the kth element of a vector, and $\varvec{I}_{N}$ represents an identity matrix of size N.

Data model

The MST radar data collected from the NARL is a uniformly spaced complex baseband signal consisting of in-phase (I) and quadrature (Q) phase components.

Let $\left\{ {y_{n} } \right\}_{n = 1}^{N}$ be the complex data obtained by weighted combination of C complex exponentials with frequencies $\left\{ {\varOmega_{r} } \right\}_{r = 1}^{N} { \in }\left[ {0,\varOmega_{max} } \right]$

$$y_{n} = \mathop \sum \limits_{r = 1}^{C} q_{r} e^{{j\omega_{r} t_{n} }} + e_{n}$$

(1)

where C is a small number, $\left\{ {t_{n} } \right\}_{n = 1}^{N}$ denotes the sampling time instants which can be non-uniformly spaced. q_r is the magnitude associated with r-th frequency component Ω_r, e_n is the additive white Gaussian noise component corresponding to the n-th sampling time.

Consequently, the complex data signal can be modelled as

$$y_{n} = \mathop \sum \limits_{r = 1}^{R} q_{r} e^{{j\omega_{r} t_{n} }} + e_{n}$$

(2)

The expanded version of the above equation is

$$\left[ {\begin{array}{*{20}c} {y_{1} } \\ \vdots \\ {y_{N} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {e^{{j\omega_{1} t_{1} }} } & \cdots & {e^{{j\omega_{R} t_{1} }} } \\ \vdots & \ldots & \vdots \\ {e^{{j\omega_{1} t_{N} }} } & \cdots & {e^{{j\omega_{R} t_{N} }} } \\ \end{array} } \right] \left[ {\begin{array}{*{20}c} {q_{1} } \\ \vdots \\ {q_{R} } \\ \end{array} } \right] + \left[ {\begin{array}{*{20}c} {e_{1} } \\ \vdots \\ {e_{N} } \\ \end{array} } \right]$$

(3)

Those values of r for which $\omega_{r} \in \left\{ {\varOmega_{r} } \right\}_{r = 1}^{N}$, the corresponding q_r values will be non-zero. The equation can vectorially be represented in the following form:

$$\varvec{y} = {\varvec{Dq}} + \varvec{e}$$

(4)

where $\varvec{y} = \left[ {y_{1} ,y_{2} , \ldots , y_{N} } \right]^{T} ,\,\varvec{D} = \left[ {\varvec{d}\left( {\omega_{1} } \right),\varvec{d}\left( {\omega_{2} } \right), \ldots ,\varvec{d}\left( {\omega_{N} } \right)} \right]^{T}$ with

$$\varvec{d}\left( {\omega_r} \right) = \left[ {e^{{j\omega_{1} t_{1} }} , \ldots , e^{{j\omega_{r} t_{N} }} } \right] = \varvec{ d}_{\varvec{r}}$$

(5)

$\varvec{q} = \left[ {q_{1} ,q_{2} , \ldots , q_{R} } \right]^{T}$, |q_r|² is the power value associated with the r-th frequency component that has to be estimated.

Iterative adaptive approach

Iterative adaptive approach is a weighted least square-based data-dependent, non parametric algorithm. It can be used for the single data sequence or the multiple data snapshots spectral estimation. Here, we assume the single snapshot case.

The covariance matrix of the received signal is given by the following model

$$\varvec{R} = E\left( {{\varvec{yy}}^{\varvec{*}} } \right) = \varvec{ }\mathop \sum \limits_{r = 1}^{R} \left| {q_{r} } \right|^{2} \varvec{d}_{r} \varvec{d}_{r}^{\varvec{*}} + E\left( {{\varvec{ee}}^{\varvec{*}} } \right) = {\varvec{DPD}}^{\varvec{*}}$$

(6)

where

$$E\left( {{\varvec{ee}}^{\varvec{*}} } \right) = \text{diag}\left( {\sigma_{1} ,\sigma_{2} , \ldots ,\sigma_{N} } \right)$$

(7)

$$\varvec{P} = {\text{diag}}\left( {\left| {q_{1} } \right|^{2} ,\left| {q_{2} } \right|^{2} , \ldots ,\left| {q_{R} } \right|^{2} ,\sigma_{1} ,\sigma_{2} , \ldots ,\sigma_{N} } \right) \triangleq {\text{diag}}\left( {m_{1} ,m_{2} , \ldots ,m_{R + N} } \right)$$

(8)

where $\left\{ {m_{i} } \right\}_{i = 1}^{R + N}$ is a replacement for $\left\{ {\left| {q_{i} } \right|^{ 2} } \right\}_{i = 1}^{R + N}$. The first R diagonal elements of the matrix $\varvec{P}$ denote the power values that are to be estimated and the remaining N diagonal elements denote the noise variance values.

The IAA algorithm is based on the minimization of weighted least squares fitting criterion,

$$f = \left\| {\varvec{y} - \varvec{d}_{r} m_{r} } \right\|_{{\varvec{O}_{r}^{ - 1} }}^{2} , \quad r = 1,2, \ldots ,R + N$$

(9)

where $\left\| \varvec{X} \right\|_{{\varvec{O}_{r}^{ - 1} }}^{2} \triangleq \varvec{X}^{H} \varvec{O}_{r}^{ - 1} \varvec{X}.$ $\varvec{O}_{\varvec{r}}$ is the interference covariance matrix written as

$$\varvec{O}_{\varvec{r}} = E\left\{ {\left( {\varvec{y} - \varvec{d}_{r} {\text{m}}_{r} } \right)\left( {\varvec{y} - \varvec{d}_{r} {\text{m}}_{r} } \right)^{H} } \right\} = \varvec{R} - {\text{m}}_{r} \varvec{d}_{r} \varvec{d}_{r}^{*}$$

(10)

Let m_r(i) denotes the estimate of m_r at ith iteration, and let $\varvec{R}\left( i \right)$ be the covariance matrix $\varvec{R}$ derived from$\left\{ {m_{i} } \right\}_{i = 1}^{R + N}$.

Minimizing the Eq. (9) with respect to ${\text{m}}_{r} , \,r = 1,2, \ldots ,R + N,$ yields

$$\hat{m}_{r} \left( {i + 1} \right) = \frac{{\varvec{d}_{r}^{*} \varvec{O}_{r}^{ - 1} \varvec{y}}}{{\varvec{d}_{r}^{*} \varvec{O}_{r}^{ - 1} \varvec{d}_{r} }}, \quad r = 1,2, \ldots ,R + N$$

(11)

Making use of the Woodbury matrix identity of

$$\varvec{d}_{r}^{*} \varvec{O}_{r}^{ - 1} = \frac{{\varvec{d}_{r }^{*} \varvec{R}^{ - 1} \left( i \right)}}{{1 - \left| {m_{r} \left( i \right)} \right|\varvec{d}_{r}^{*} \varvec{R}^{ - 1} \left( i \right)\varvec{d}_{r} }}$$

(12)

The estimation of the power by means of the following iterative process

$$\hat{m}_{r} \left( {i + 1} \right) = \frac{{\varvec{d}_{r}^{*} \varvec{R}^{ - 1} \left( i \right)\varvec{y}}}{{\varvec{d}_{r}^{*} \varvec{R}^{ - 1} \left( i \right)\varvec{d}_{r} }}, \quad r = 1,2, \ldots , R + N$$

(13)

The initial estimate m_r(0) can be obtained using the single frequency least-squares (SFLS) method (which is known as periodogram).

$$m_{r} \left( 0 \right) = \frac{{\left| {\varvec{d}_{r}^{*} \varvec{y}} \right|^{2} }}{{\left\| {\varvec{d}_{r} } \right\|^{4} }},\quad r = 1,2, \ldots , R + N.$$

(14)

The algorithm is summarized in Table 1.

Table 1 Summary of the IAA algorithm

Results

Simulation

The simulation results for the complex data explained in Data Model are displayed. With N = 200 and C = 3, the data samples are generated from a signal consisting of three exponentials at 0.3100, 0.3150, 0.1450 Hz frequencies having the amplitudes $q_{1} = 10e^{{j\varphi_{1} }} ,$ $q_{2} = 10e^{{j\varphi_{2} }} ,$ and $q_{3} = 10e^{{j\varphi_{3} }}$ with a sampling interval of 1 s. The phase values $\left\{ {\varphi_{r} } \right\}_{r = 1}^{3}$ are distributed in uniform and independent manner [0, 2π]. The noise component is introduced in the ɛ term, which is the normal white noise with zero mean and σ variance. The signal-to-noise ratio (SNR) in dB for the data model is defined as ${\text{SNR}} = 10\log \left( {\frac{100}{\sigma }} \right).$

The power spectrum of the original test signal generated using the data model before adding noise is depicted in Fig. 1. The spectrum of the signal for SNR = 0 dB using periodogram and IAA are shown in Fig. 2. The periodogram is obtained by padding the 200 element time-series data with 312 zeros and performing the 512-point FFT. For IAA the number of processing points R is taken as 512.

Fig. 1

Spectrum of the original signal

Fig. 2

Spectrum of the test signal for SNR = 0 dB a periodogram b IAA

The root mean square error (RMSE) of the frequency estimation and amplitude estimation versus SNR plots are shown in Fig. 3a, b, respectively. All the performance curves are obtained via 100 Monte Carlo simulations. We change the value of σ² to obtain different signal-to-noise ratio conditions. From Fig. 3a, b, it can be observed that both the RMSE of the frequency and the amplitude estimates decreased with the SNR as expected. It can also be observed that the IAA method has better variance characteristics than the Periodogram method. It is evident from the above simulations, that even the signal is completely buried in noise the IAA method is able to retrieve the parameters well.

Fig. 3

a RMSE of normalized frequency versus SNR. b Amplitude RMSE versus SNR plot

MST radar data

The radar data collected from the Indian MST radar being operated at the NARL, Gadanki, Andhra Pradesh is taken for the present study. The MST radar data is one of the e formats of 15 scans with each scan having signal information from six beam directions (East, West, Zenith-X, Zenith-Y, North and South). Each beam consists of 147 height range bins with a resolution of 150 m, starting from 3.6 km and reaching to a height of 25.6 km. Each range bin contains complex time-series data with 512 samples. The spectrum of the radar signal is calculated using IAA. Since, the echoes are usually corrupted by interference, clutter, etc., it has to be cleaned before analysis. Maximum peak detection method (Anandan et al. 1996) is used for the estimation of Doppler profile after performing spectrum cleaning of the radar signal. The Doppler frequencies are calculated from the Doppler profiles.

Once the Doppler frequencies are computed, the Doppler velocities are found out by multiplying each of the frequencies with c/2f_c where c is the velocity of light and f_c is the operating frequency of the Doppler radar. Both Doppler frequencies and the velocities are calculated for all 6 beams and 147 range bins. Using the Doppler velocities for the 6 beams denoted as $v_{E} ,\,v_{W} ,\,v_{ZX} ,\,v_{ZY} ,\, v_{N} ,\,v_{S}$ where the subscripts represents the corresponding beams, the three wind velocities components are evaluated as follows

$$\left[ {\begin{array}{*{20}c} {v_{x} } \\ {v_{y} } \\ {v_{z} } \\ \end{array} } \right] = \left[ {\begin{array}{*{20}c} {0.603} & 0 & 0 \\ 0 & {0.603} & 0 \\ 0 & 0 & {0.603} \\ \end{array} } \right]^{ - 1} *\left[ {\begin{array}{*{20}c} {0.1736\left( {v_{E} - v_{W} } \right)} \\ {0.1736\left( {v_{N} - v_{S} } \right)} \\ {0.1736\left( {v_{ZX} - v_{ZY} } \right)} \\ \end{array} } \right]$$

(15)

where v_x, v_y, v_z are the zonal U, meridional V, and the vertical Z velocity components. The Zenith-X and Zenith-Y beams are in the vertical direction and do not play a role in the determination of the wind velocity.The Wind speed W is computed as

$$W = \left( {v_{x}^{2} + v_{y}^{2} } \right)^{1/2}$$

(16)

The wind speed thus obtained is then compared with the corresponding wind speed collected from the global positioning system (GPS) radiosonde (Jagannadha Rao et al. 2003).

The power spectrum of the collected data is determined using IAA method. The basic method of periodogram is employed for estimating the power spectrum when complex time-series data is subjected to the ADP.

Figure 4a, b shows the improvement of output SNR estimated from power spectrum using periodogram and IAA for the east and south beams, respectively, for the MST radar data collected on February 9, 2015. The output SNR is obtained using the method (Hildebrand and Sekhon 1974). The comparison of average SNR values in dB for six beams on February 9 and 10, 2015 for the periodogram and IAA algorithms is given in Table 2. From Table 2, it is seen that the IAA gives the better improvement in SNR values for all the six beams.

Fig. 4

Height profiles of SNR estimated using periodogram and IAA. a East beam; b south beam for data collected on February 9, 2015

Table 2 Comparison of average SNR for periodogram and IAA algorithm

The Doppler height profiles for four scans of the east beam attained using ADP and IAA are shown in Fig. 5a, b, respectively, for the radar data collected on February 9, 2015. The compared mean Doppler profiles and standard deviations are shown in Fig. 5c, d respectively. The observed significant difference between ADP and IAA is that the standard deviation for IAA lied very close to zero.

Fig. 5

Doppler height profiles for four scans of the east beam using a ADP, b IAA. c Mean Doppler height profile of the east beam. d Standard deviation for the east beam for February 9, 2015 data

The Zonal, Meridional and Wind Speed components calculated using the ADP, IAA and GPS radiosonde are depicted in Figs. 6 and 7 for the data collected on June 9, 2006 and February 9, 2015, respectively. It is revealed that the IAA is following the GPS. The minor deviation from 15 to 17.5 km can be attributed to the fact that the data used in IAA are collected from the reflected echoes from the layers of the atmosphere in the vertical direction without any drift in the horizontal direction, whereas in the data collected through GPS, there can be horizontal drift of the balloon due to high wind speeds. The wind speeds using ADP, IAA and GPS radiosonde for real-time radar data collected on two different dates namely July 2, 2014 and February 9, 2015 is represented in Fig. 8.

Fig. 6

Zonal, Meridional and Wind Speed for June 9, 2006 data using ADP, IAA and GPS radiosonde

Fig. 7

Zonal, Meridional and Wind Speed for February 9, 2015 data using ADP, IAA and GPS radiosonde

Fig. 8

Wind speed using ADP, IAA and GPS radiosonde for radar data collected on a July 2, 2014 b February 9, 2015—Comparison

The consistency of the algorithm is checked by calculating the correlation between GPS radiosonde data and IAA wind speeds for the radar data collected during 4th–8th October, 2006 and 9th–12th February, 2015. A significant correlation coefficient of 0.8996 and 0.9246 is obtained between the GPS and IAA, 0.843 and 0.85406 between GPS and ADP, respectively. The high correlation factor acquired is indicating the relative accuracy of the wind speed calculated using IAA method confirming its efficiency and effectiveness. Figures 9 and 10 shows the correlation between IAA and GPS wind speeds.

Fig. 9

Correlation between IAA and GPS wind speeds for data during 4th–8th October 2006

Fig. 10

Correlation between IAA and GPS wind speeds for data during 9th–12th February 2015

Conclusion

An iterative weighted least square-based estimation (IAA) is presented and applied to the atmospheric radar data in the current study. The reduction in the deviation of mean Doppler profiles and the standard deviation is illustrated in IAA compared to the existing ADP. This proves that IAA functions superior compared to existing algorithms which have failed to perform well especially in this height range. Significant enhancement in SNR at higher altitudes is achieved with IAA demonstrating its efficiency and effectiveness. The obtained wind velocities from the IAA algorithm are validated using the GPS sonde values. The correlation between wind speeds calculated using GPS and IAA has been also presented for the radar data collected in the month of October 2006 and February 2015 which attained a correlation factor of 0.8996 and 0.9246. The IAA method gives better results with high computational complexity. This is due to the fact that it is an iterative process involving matrix inversion. Henceforth, this can be carried through efficient methods which results in consuming less computational time.