Theory of Stochastic Signals/Stochastic System Theory

1. Theory of Stochastic Signals
2. Filtering of Stochastic Signals
3. Stochastic System Theory

• Previous page
• Next page

• Previous page
• Next page

# OVERVIEW OF THE FIFTH MAIN CHAPTER #

This chapter describes the influence of a filter on the  »auto-correlation function«  $\rm (ACF)$  and  »the power-spectral density  $\rm (PSD)$«  of stochastic signals.

In detail,  this chapter covers:

• the  »calculation of ACF and PSD«  at the filter output  ("Stochastic System Theory"),
• the structure and representation of  »digital filters«  (non-recursive and recursive),
• the  »dimensioning«  of the filter coefficients for a given ACF,
• the meaning of the  »matched filter«  for the SNR maximization of communication systems,
• the properties of the  »Wiener-Kolmogorow filter«  for the signal reconstruction.

System model and problem definition

As in the book  Linear and Time Invariant Systems,  we consider the setup sketched on the right, where the system characterized both

• by the impulse response  $h(t)$
• as well as by its frequency response  $H(f)$

is described unambiguously.  The relationship between these descriptive quantities in the time and frequency domain is given by the  Fourier transformation. 

Filter influence on  "spectrum"  and 
"power-spectral density"

If we apply the signal  $x(t)$  to the input and denote the output signal by  $y(t)$,  the classical system theory provides the following statements:

• The output signal  $y(t)$  results from the  convolution  between the input signal  $x(t)$  and the impulse response  $h(t)$.  The following equation is equally valid for deterministic and stochastic signals:

$$y(t) = x(t) \ast h(t) = \int_{-\infty}^{+\infty} x(\tau)\cdot h ( t - \tau) \,\,{\rm d}\tau.$$

• For deterministic signals,  one usually takes a roundabout route using the spectral functions.  The spectrum  $X(f)$  is the Fourier transform of  $x(t)$.  The multiplication with the frequency response  $H(f)$  leads to the output spectrum  $Y(f)$.  From this,  the signal  $y(t)$  can be obtained by the Fourier inverse transformation.
• In the case of stochastic signals this procedure fails,  because then the time functions  $x(t)$  and  $y(t)$  are not predictable for all times  from ­$–∞$  to  $+∞$  and thus,  the corresponding amplitude spectra  $X(f)$  and  $Y(f)$  do not exist at all.  In this case,  we have to switch to the  power-spectral densities  defined in the last chapter.

Amplitude spectrum and power-spectral density

We consider an ergodic random process  $\{x(t)\}$,  whose auto-correlation function  $φ_x(τ)$  is assumed to be known.  The power-spectral density  ${\it Φ}_x(f)$  is then also uniquely determined via the Fourier transform and the following statements hold:

For the ACF and PSD calculation of a random signal

1. The  power-spectral density  ${\it Φ}_x(f)$  can be given – as well as the auto-correlation function  $φ_x(τ)$ – for each individual pattern function of the stationary and ergodic random process  $\{x(t)\}$,  even if the specific course of  $x(t)$  is explicitly unknown.
   
   
2. The  amplitude spectrum  $X(f)$,  on the other hand,  is undefined because if the spectral function  $X(f)$  is known, the entire time function  $x(t)$  from  $–∞$  to  $+∞$  would also have to be known via the Fourier inverse transform,  which clearly cannot be the case for a stochastic signal.
   
   
3. If a time section of finite time duration  $T_{\rm M}$  is known according to the sketch on the right,  the Fourier transform can of course be applied to it again.

$\text{Theorem:}$  The following relationship exists between the power-spectral density  ${\it Φ}_x(f)$  of the infinite time random signal  $x(t)$  and the amplitude spectrum  $X_{\rm T}(f)$  of the finite time section  $x_{\rm T}(t)$: 

$${ {\it \Phi}_x(f)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \vert X_{\rm T}(f)\vert ^2.$$

$\text{Proof:}$  Previously,  the  auto-correlation function  of an ergodic process with the sample function  $x(t)$  was given as follows:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot x(t + \tau)\hspace{0.1cm} \rm d \it t.$$

• It is permissible to replace the function  $x(t)$,  which is unbounded in time, by the function  $x_{\rm T}(t)$,  which is bounded on the time range  $-T_{\rm M}/2$  to  $+T_{\rm M}/2$.    $x_{\rm T}(t)$  corresponds to the spectrum  $X_{\rm T}(f)$,  and by applying the  first Fourier integral  and the  shifting theorem:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x_{\rm T}(t)\cdot \int^{+\infty}_{-\infty}X_{\rm T}(f)\cdot {\rm e}^{ {\rm j}2 \pi f ( t + \tau) } \hspace{0.1cm} \rm d \it f \hspace{0.1cm} \rm d \it t.$$

• After splitting the exponent and swapping the time and frequency integrals,  we get:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \int^{+\infty}_{-\infty}X_{\rm T}(f)\cdot \left[ \int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x_{\rm T}(t)\cdot {\rm e}^{ {\rm j}2 \pi f t } \hspace{0.1cm} \rm d \it t \right] \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f.$$

• The inner integral describes the conjugate-complex spectrum  ${X_{\rm T} }^{\star}(f)$.  It further follows that:

$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \int^{+\infty}_{-\infty}\vert X_{\rm T}(f)\vert^2 \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f.$$

• A comparison with  Wiener  and  Chintchin's theorem which is always valid in ergodicity,

$${ {\it \varphi}_x(\tau)} = \int^{+\infty}_{-\infty}{\it \Phi}_x(f) \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f ,$$

shows the validity of the above relation:

$${ {\it \Phi}_x(f)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} }\cdot \vert X_{\rm T}(f)\vert^2.$$

q.e.d.

Power-spectral density of the filter output signal

Combining the statements made in the last two sections, we arrive at the following important result:

$\text{Theorem:}$  The power-spectral density (PSD) at the output of a linear time-invariant system with frequency response  $H(f)$  is obtained as the product of the input PSD  ${\it Φ}_x(f)$  and the "power transfer function"  $\vert H(f)\vert ^2$.

$${ {\it \Phi}_y(f)} = { {\it \Phi}_x(f)} \cdot \vert H(f)\vert ^2.$$

$\text{Proof:}$  Starting from the three relations already derived before:

$${ {\it \Phi}_x(f)} =\hspace{-0.1cm} \lim_{T_{\rm M}\to\infty}\hspace{0.01cm} \frac{1}{ T_{\rm M} }\hspace{-0.05cm}\cdot\hspace{-0.05cm} \vert X_{\rm T}(f)\vert^2, \hspace{0.5cm} { {\it \Phi}_y(f)} =\hspace{-0.1cm} \lim_{T_{\rm M}\to\infty}\hspace{0.01cm} \frac{1}{ T_{\rm M} }\hspace{-0.05cm}\cdot\hspace{-0.05cm}\vert Y_{\rm T}(f)\vert^2, \hspace{0.5cm} Y_{\rm T}(f) = X_{\rm T}(f) \hspace{-0.05cm}\cdot\hspace{-0.05cm} H(f).$$

Substituting these equations into each other, we get the above result.

q.e.d.

The following example illustrates the relationship with white noise.

Filter influence in the frequency domain

$\text{Example 1:}$  At the input of a Gaussian low-pass filter with the frequency response

$$H(f) = {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}$$

white noise  $x(t)$  with noise power density  ${ {\it \Phi}_x(f)} =N_0/2$  is present   ⇒   two-sided representation.  Then, the following holds for the power-spectral density of the output signal:

$${ {\it \Phi}_y(f)} = \frac {N_0}{2} \cdot {\rm e}^{- 2 \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}.$$

The diagram shows the signals and power-spectral densities at the filter input and output.

Notes:

1.   The signal  $x(t)$  – strictly speaking – cannot be plotted at all, since it has an infinite power   ⇒   integral over  ${\it Φ}_x(f)$  from  $-\infty$  to  $+\infty$.
2.   The output signal  $y(t)$  has a lower frequency than  $x(t)$  and a finite power corresponding to the integral over  ${\it Φ}_y(f)$.
3.   In one-sided representation, (only) for  $f>0$ would hold:  ${ {\it \Phi}_x(f)} =N_0$.  he statements  (1)  and  (2)  would also apply here in the same way.

The auto-correlation function of the filter output signal

The calculated power-spectral density (PSD) can also be written as follows:

$${{\it \Phi}_y(f)} = {{\it \Phi}_x(f)} \cdot H(f) \cdot H^{\star}(f)$$

$\text{Theorem:}$  The corresponding auto-correlation function (ACF) is then obtained according to the  regularities of the Fourier transform  and by applying the  convolution theorem:

$${ {\it \varphi}_y(\tau)} = { {\it \varphi}_x(\tau)} \ast h(\tau)\ast h(- \tau).$$

In the transition from the spectral to the time domain, note:

• The Fourier retransforms are to be inserted in each case, namely

$${{\it \varphi}_y(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\bullet\,{{\it \Phi}_y(f)}, \hspace{0.5cm}{{\it \varphi}_x(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\!\bullet\,{{\it \Phi}_x(f)}, \hspace{0.5cm}{h(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\bullet\,{H(f)}, \hspace{0.5cm}{h(-\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\!\bullet\,{H^{\star}(f)}$$

• Moreover, each multiplication becomes a convolution operation.

Filter influence in the time domain

$\text{Example 2:}$  We consider again the same scenario as  in $\text{Example 1}$, but this time in the time domain:

• white noise  ${ {\it \Phi}_x(f)} =N_0/2$,

• Gaussian filter:   $H(f) = {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}\hspace{0.3cm}\Rightarrow \hspace{0.3cm} h(t) = \Delta f \cdot {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(\Delta f \hspace{0.03cm}\cdot \hspace{0.03cm}t)^2}.$

One can see from this diagram:

1.   The ACF of the input signal is now a Dirac function with weight  $N_0/2$.
2.   By convolution twice with the (here also Gaussian) impulse response  $h(t)$  or  $h(–t)$  one obtains the ACF  $φ_y(τ)$  of the output signal.
3.   Thus, the ACF  $φ_y(τ)$  of the output signal is also Gaussian.
4.   The ACF value at  $τ = 0$  is identical to the area of the power-spectral density  ${\it Φ}_y(f)$  and characterizes the signal power (variance)  $σ_y^2$.
5.   In contrast, the area at  $φ_y(τ)$  gives the PSD value  ${\it Φ}_y(f = \rm 0)$, i.e.,  $N_0/2$.

Cross correlation function between input and output signal

Calculating the cross correlation function

We again consider a filter with the frequency response  $H(f)$  and the impulse response  $h(t)$.  Further applies:

• The stochastic input signal  $x(t)$  is a sample function of the ergodic random process  $\{x(t)\}$.
• The corresponding auto-correlation function (ACF) at the filter input is thus  $φ_x(τ)$, while the power-spectral density (PSD) is denoted by  ${\it Φ}_x(f)$. 
• The corresponding descriptors of the ergodic random process  $\{y(t)\}$  at the filter output are the sample function  $y(t)$, the auto-correlation function  $φ_y(τ)$  and the conductance density spectrum  ${\it Φ}_y(f)$.

$\text{Theorem:}$  For the  cross correlation function  (CCF) between the input and the output signal holds:

$${ {\it \varphi}_{xy}(\tau)} = h(\tau)\ast { {\it \varphi}_x(\tau)} .$$

Here,  $h(τ)$ denotes the impulse response of the filter  $($with the time variable  $τ$  instead of  $t)$  and  ${ {\it \varphi}_{x}(\tau)}$  denotes the ACF of the input signal.

$\text{Proof:}$  In general, for the cross-correlation function between two signals  $x(t)$  and  $y(t)$:

$${ {\it \varphi}_{xy}(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot y(t + \tau)\hspace{0.1cm} \rm d \it t.$$

• With the generally valid relation  $y(t) = h(t) \ast x(t)$  and the formal integration variable  $θ$,  we can also write for this:

$${ {\it \varphi}_{xy}(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot \int^{+\infty}_{-\infty} h(\theta) \cdot x(t + \tau - \theta)\hspace{0.1cm}{\rm d}\theta\hspace{0.1cm}{\rm d} \it t.$$

• By interchanging the two integrals and subtracting the limit into the integral, we obtain:

$${ {\it \varphi}_{xy}(\tau)} = \int^{+\infty}_{-\infty} h(\theta) \cdot \left[ \lim_{T_{\rm M}\to\infty}\hspace{0.2cm} \frac{1}{ T_{\rm M} } \cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot x(t + \tau - \theta)\hspace{0.1cm} \hspace{0.1cm} {\rm d} t \right]{\rm d}\theta.$$

• The expression in the square brackets gives the ACF value at the input at time  $τ - θ$:

$${ {\it \varphi}_{xy}(\tau)} = \int^{+\infty}_{-\infty}h(\theta) \cdot \varphi_x(\tau - \theta)\hspace{0.1cm}\hspace{0.1cm} {\rm d}\theta = h(\tau)\ast { {\it \varphi}_x(\tau)} .$$

• However, the remaining integral describes the convolution operation in detailed notation.

q.e.d.

$\text{Conclusion:}$  In the frequency domain, the corresponding equation is:

$${ {\it \Phi}_{xy}(f)} = H(f)\cdot{ {\it \Phi}_x(f)} \hspace{0.3cm} \Rightarrow \hspace{0.3cm} H(f) = \frac{ {\it \Phi}_{xy}(f)}{ {\it \Phi}_{x}(f)}.$$

This equation shows that the filter frequency response  $H(f)$  from a measurement with stochastic excitation can be calculated completely – i.e., both magnitude and phase – if the following descriptive quantities are determined:

• the statistical characteristics at the input, either the ACF  $φ_x(τ)$  or the  PSD ${\it Φ}_x(f)$,
• as well as the cross correlation function  $φ_{xy}(τ)$  or its Fourier transform  ${\it Φ}_{xy}(f)$.

Exercises for the chapter

Exercise 5.1: Gaussian ACF and Gaussian Low-Pass

Exercise 5.1Z: $\cos^2$ Noise Limitation

Exercise 5.2: Determination of the Frequency Response

Exercise 5.2Z: Two-Way Channel