Theory of Stochastic Signals/Stochastic System Theory

1. Theory of Stochastic Signals
2. Filtering of Stochastic Signals
3. Stochastic System Theory

• Previous page
• Next page

• Previous page
• Next page

# OVERVIEW OF THE FIFTH MAIN CHAPTER #

his chapter describes the influence of a filter on the auto-correlation function (ACF) and the power-spectral density (PSD) of stochastic signals.

In detail, it covers:

• the calculation of ACF and PSD at the filter output (Stochastic System Theory ),
• the structure and representation of digital filters  (non-recursive and recursive),
• the dimensioning  of filter coefficients for a given ACF,
• the meaning of the 'matched filter  communication systems (SNR maximization),
• the properties of the Wiener-Kolmogorow filter  for signal reconstruction.

Problem

Filter influence on spectrum and power-spectral density (PSD)

As in the book  Linear and Time Invariant Systems,  we consider the setup sketched on the right, where the system

• characterized both by the impulse response  $h(t)$
• as well as by its frequency response  $H(f)$

is described unambiguously.  The relationship between these descriptive quantities in the time and frequency domain is given by the  Fourier transformation. 
If we apply the signal  $x(t)$  to the input and denote the output signal by  $y(t)$, classical system theory provides the following statements:

• The output signal  $y(t)$  results from the  convolution  between the input signal  $x(t)$  and the impulse response  $h(t)$.  The following equation is equally valid for deterministic and stochastic signals:

$$y(t) = x(t) \ast h(t) = \int_{-\infty}^{+\infty} x(\tau)\cdot h ( t - \tau) \,\,{\rm d}\tau.$$

• For deterministic signals, one usually takes a roundabout route using the spectral functions.  The spectrum  $X(f)$  is the Fourier transform of  $x(t)$.  The multiplication with the frequency response  $H(f)$  leads to the output spectrum  $Y(f)$.  From this, the signal  $y(t)$  can be obtained by Fourier inverse transformation.
• In the case of stochastic signals this procedure fails, because then the time functions  $x(t)$  and  $y(t)$  are not predictable for all times  from ­$–∞$  to  $+∞$  and thus the corresponding amplitude spectra  $X(f)$  and  $Y(f)$  do not exist at all.
• In this case, we have to switch to the  power-spectral densities  defined in the last chapter.

Amplitude and power-spectral density

We consider an ergodic random process  $\{x(t)\}$, whose auto-correlation function  $φ_x(τ)$  is assumed to be known.  The power-spectral density  ${\it Φ}_x(f)$  is then also uniquely determined via the Fourier transform and the following statements hold:

For the ACF and PSD calculation of a random signal

• The power-spectral density  ${\it Φ}_x(f)$  can be given – as well as the auto-correlation function  $φ_x(τ)$ – for each individual pattern function of the stationary and ergodic random process  $\{x(t)\}$,  even if the specific course of  $x(t)$  is explicitly unknown.
• The  amplitude spectrum  $X(f)$,  on the other hand, is undefined because if the spectral function  $X(f)$  is known, the entire time function  $x(t)$  from  $–∞$  to  $+∞$  would also have to be known via the Fourier inverse transform, which clearly cannot be the case for a stochastic signal.
• If a time section of the finite time duration  $T_{\rm M}$  is known according to the sketch on the left, the Fourier transform can of course be applied to it again.

$\text{Theorem:}$  The following relationship exists between the power-spectral density  ${\it Φ}_x(f)$  of the infinite time random signal  $x(t)$  and the amplitude spectrum  $X_{\rm T}(f)$  of the finite time section  $x_{\rm T}(t)$: 

$${ {\it \Phi}_x(f)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \vert X_{\rm T}(f)\vert ^2.$$

$\text{Proof:}$  Previously, the  auto-correlation function  of an ergodic process with the sample function  $x(t)$  was given as follows:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot x(t + \tau)\hspace{0.1cm} \rm d \it t.$$

• It is permissible to replace the function  $x(t)$,  which is unbounded in time, by the function  $x_{\rm T}(t)$,  which is bounded on the time range  $-T_{\rm M}/2$  to  $+T_{\rm M}/2$.    $x_{\rm T}(t)$  corresponds to the spectrum  $X_{\rm T}(f)$, and by applying the  first Fourier integral  and the  shifting theorem:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x_{\rm T}(t)\cdot \int^{+\infty}_{-\infty}X_{\rm T}(f)\cdot {\rm e}^{ {\rm j}2 \pi f ( t + \tau) } \hspace{0.1cm} \rm d \it f \hspace{0.1cm} \rm d \it t.$$

• After splitting the exponent and swapping the time and frequency integrals, we get:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \int^{+\infty}_{-\infty}X_{\rm T}(f)\cdot \left[ \int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x_{\rm T}(t)\cdot {\rm e}^{ {\rm j}2 \pi f t } \hspace{0.1cm} \rm d \it t \right] \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f.$$

• The inner integral describes the conjugate-complex spectrum  $X_{\rm T}^{\star}(f)$.  It further follows that:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \int^{+\infty}_{-\infty}\vert X_{\rm T}(f)\vert^2 \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f.$$

• A comparison with  Wiener  and  Chintchin's theorem which is always valid in ergodicity,

$${ {\it \varphi}_x(\tau)} = \int^{+\infty}_{-\infty}{\it \Phi}_x(f) \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f ,$$

shows the validity of the above relation:

$${ {\it \Phi}_x(f)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \vert X_{\rm T}(f)\vert^2.$$

q.e.d.

Power-spectral density of the filter output signal

Combining the statements made in the last two sections, we arrive at the following important result:

$\text{Theorem:}$  The power-spectral density (PSD) at the output of a linear time-invariant system with frequency response  $H(f)$  is obtained as the product of the input PSD  ${\it Φ}_x(f)$  and the "power transfer function"  $\vert H(f)\vert ^2$.

$${ {\it \Phi}_y(f)} = { {\it \Phi}_x(f)} \cdot \vert H(f)\vert ^2.$$

$\text{Proof:}$  Starting from the three relations already derived before:

$${ {\it \Phi}_x(f)} =\hspace{-0.1cm} \lim_{T_{\rm M}\to\infty}\hspace{0.01cm} \frac{1}{ T_{\rm M} }\hspace{-0.05cm}\cdot\hspace{-0.05cm} \vert X_{\rm T}(f)\vert^2, \hspace{0.5cm} { {\it \Phi}_y(f)} =\hspace{-0.1cm} \lim_{T_{\rm M}\to\infty}\hspace{0.01cm} \frac{1}{ T_{\rm M} }\hspace{-0.05cm}\cdot\hspace{-0.05cm}\vert Y_{\rm T}(f)\vert^2, \hspace{0.5cm} Y_{\rm T}(f) = X_{\rm T}(f) \hspace{-0.05cm}\cdot\hspace{-0.05cm} H(f).$$

Substituting these equations into each other, we get the above result.

q.e.d.

The following example illustrates the relationship with white noise.

Filter influence in the frequency domain

$\text{Example 1:}$  At the input of a Gaussian low-pass filter with the frequency response

$$H(f) = {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}$$

white noise  $x(t)$  with noise power density  ${ {\it \Phi}_x(f)} =N_0/2$  is present   ⇒   two-sided representation.  Then, the following holds for the power-spectral density of the output signal:

$${ {\it \Phi}_y(f)} = \frac {N_0}{2} \cdot {\rm e}^{- 2 \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}.$$

The diagram shows the signals and power-spectral densities at the filter input and output.

Notes:

1.   The signal  $x(t)$  – strictly speaking – cannot be plotted at all, since it has an infinite power   ⇒   integral over  ${\it Φ}_x(f)$  from  $-\infty$  to  $+\infty$.
2.   The output signal  $y(t)$  has a lower frequency than  $x(t)$  and a finite power corresponding to the integral over  ${\it Φ}_y(f)$.
3.   In one-sided representation, (only) for  $f>0$ would hold:  ${ {\it \Phi}_x(f)} =N_0$.  he statements  (1)  and  (2)  would also apply here in the same way.

The auto-correlation function of the filter output signal

The calculated power-spectral density (PSD) can also be written as follows:

$${{\it \Phi}_y(f)} = {{\it \Phi}_x(f)} \cdot H(f) \cdot H^{\star}(f)$$

$\text{Theorem:}$  The corresponding auto-correlation function (ACF) is then obtained according to the  regularities of the Fourier transform  and by applying the  convolution theorem:

$${ {\it \varphi}_y(\tau)} = { {\it \varphi}_x(\tau)} \ast h(\tau)\ast h(- \tau).$$

In the transition from the spectral to the time domain, note:

• The Fourier retransforms are to be inserted in each case, namely

$${{\it \varphi}_y(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\bullet\,{{\it \Phi}_y(f)}, \hspace{0.5cm}{{\it \varphi}_x(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\!\bullet\,{{\it \Phi}_x(f)}, \hspace{0.5cm}{h(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\bullet\,{H(f)}, \hspace{0.5cm}{h(-\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\!\bullet\,{H^{\star}(f)}$$

• Moreover, each multiplication becomes a convolution operation.

Filter influence in the time domain

$\text{Example 2:}$  We consider again the same scenario as  in $\text{Example 1}$, but this time in the time domain:

• white noise  ${ {\it \Phi}_x(f)} =N_0/2$,

• Gaussian filter:   $H(f) = {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}\hspace{0.3cm}\Rightarrow \hspace{0.3cm} h(t) = \Delta f \cdot {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(\Delta f \hspace{0.03cm}\cdot \hspace{0.03cm}t)^2}.$

One can see from this diagram:

1.   The ACF of the input signal is now a Dirac function with weight  $N_0/2$.
2.   By convolution twice with the (here also Gaussian) impulse response  $h(t)$  or  $h(–t)$  one obtains the ACF  $φ_y(τ)$  of the output signal.
3.   Thus, the ACF  $φ_y(τ)$  of the output signal is also Gaussian.
4.   The ACF value at  $τ = 0$  is identical to the area of the power-spectral density  ${\it Φ}_y(f)$  and characterizes the signal power (variance)  $σ_y^2$.
5.   In contrast, the area at  $φ_y(τ)$  gives the PSD value  ${\it Φ}_y(f = \rm 0)$, i.e.,  $N_0/2$.

Cross correlation function between input and output signal

Calculating the cross correlation function

We again consider a filter with the frequency response  $H(f)$  and the impulse response  $h(t)$.  Further applies:

• The stochastic input signal  $x(t)$  is a sample function of the ergodic random process  $\{x(t)\}$.
• The corresponding auto-correlation function (ACF) at the filter input is thus  $φ_x(τ)$, while the power-spectral density (PSD) is denoted by  ${\it Φ}_x(f)$. 
• The corresponding descriptors of the ergodic random process  $\{y(t)\}$  at the filter output are the sample function  $y(t)$, the auto-correlation function  $φ_y(τ)$  and the conductance density spectrum  ${\it Φ}_y(f)$.

$\text{Theorem:}$  For the  cross correlation function  (CCF) between the input and the output signal holds:

$${ {\it \varphi}_{xy}(\tau)} = h(\tau)\ast { {\it \varphi}_x(\tau)} .$$

Here,  $h(τ)$ denotes the impulse response of the filter  $($with the time variable  $τ$  instead of  $t)$  and  ${ {\it \varphi}_{x}(\tau)}$  denotes the ACF of the input signal.

$\text{Proof:}$  In general, for the cross-correlation function between two signals  $x(t)$  and  $y(t)$:

$${ {\it \varphi}_{xy}(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot y(t + \tau)\hspace{0.1cm} \rm d \it t.$$

• With the generally valid relation  $y(t) = h(t) \ast x(t)$  and the formal integration variable  $θ$,  we can also write for this:

$${ {\it \varphi}_{xy}(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot \int^{+\infty}_{-\infty} h(\theta) \cdot x(t + \tau - \theta)\hspace{0.1cm}{\rm d}\theta\hspace{0.1cm}{\rm d} \it t.$$

• By interchanging the two integrals and subtracting the limit into the integral, we obtain:

$${ {\it \varphi}_{xy}(\tau)} = \int^{+\infty}_{-\infty} h(\theta) \cdot \left[ \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} } \cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot x(t + \tau - \theta)\hspace{0.1cm} \hspace{0.1cm} {\rm d} t \right]{\rm d}\theta.$$

• The expression in the square brackets gives the ACF value at the input at time  $τ - θ$:

$${ {\it \varphi}_{xy}(\tau)} = \int^{+\infty}_{-\infty}h(\theta) \cdot \varphi_x(\tau - \theta)\hspace{0.1cm}\hspace{0.1cm} {\rm d}\theta = h(\tau)\ast { {\it \varphi}_x(\tau)} .$$

• However, the remaining integral describes the convolution operation in detailed notation.

q.e.d.

$\text{Conclusion:}$  In the frequency domain, the corresponding equation is:

$${ {\it \Phi}_{xy}(f)} = H(f)\cdot{ {\it \Phi}_x(f)} \hspace{0.3cm} \Rightarrow \hspace{0.3cm} H(f) = \frac{ {\it \Phi}_{xy}(f)}{ {\it \Phi}_{x}(f)}.$$

This equation shows that the filter frequency response  $H(f)$  from a measurement with stochastic excitation can be calculated completely – i.e., both magnitude and phase – if the following descriptive quantities are determined:

• the statistical characteristics at the input, either the ACF  $φ_x(τ)$  or the  PSD ${\it Φ}_x(f)$,
• as well as the cross correlation function  $φ_{xy}(τ)$  or its Fourier transform  ${\it Φ}_{xy}(f)$.

Exercises for the chapter

Exercise 5.1: Gaussian ACF and Gaussian Low-Pass

Exercise 5.1Z: $\cos^2$ Noise Limitation

Exercise 5.2: Determination of the Frequency Response

Exercise 5.2Z: Two-Way Channel