View source for Theory of Stochastic Signals/Stochastic System Theory

 
{{Header
|Untermenü=Filtering of Stochastic Signals
|Vorherige Seite= Verallgemeinerung auf N-dimensionale Zufallsgrößen
|Nächste Seite=Digitale Filter
}}

== # OVERVIEW OF THE FIFTH MAIN CHAPTER # ==
<br>
This chapter describes the influence of a filter on the&nbsp; &raquo;auto-correlation function&laquo;&nbsp; $\rm (ACF)$&nbsp; and&nbsp; &raquo;the power-spectral density&nbsp; $\rm (PSD)$&laquo;&nbsp; of stochastic signals.

In detail,&nbsp; this chapter covers:

*the&nbsp; &raquo;calculation of ACF and PSD&laquo;&nbsp; at the filter output&nbsp; ("Stochastic System Theory"),
*the structure and representation of&nbsp; &raquo;digital filters&laquo;&nbsp; (non-recursive and recursive),
*the&nbsp; &raquo;dimensioning&laquo;&nbsp; of the filter coefficients for a given ACF,
*the meaning of the&nbsp; &raquo;matched filter&laquo;&nbsp; for the SNR maximization of communication systems,
*the properties of the&nbsp; &raquo;Wiener-Kolmogorow filter&laquo;&nbsp; for the signal reconstruction.





==System model and problem definition==
<br>
As in the book&nbsp; [[Linear_and_Time_Invariant_Systems|"Linear and Time Invariant Systems"]],&nbsp; we consider the setup sketched on the right, where the system characterized both  
*by the impulse response&nbsp; $h(t)$ 
*as well as by its frequency response&nbsp; $H(f)$ 


is described unambiguously.&nbsp; The relationship between these descriptive quantities in the time and frequency domain is given by the&nbsp; [[Signal_Representation/Fourier_Transform_and_its_Inverse#Properties_of_aperiodic_signals|$\text{Fourier transformation}$]].&nbsp; 
[[File:EN_Sto_T_5_1_S1.png |right| 300px|frame|Filter influence on&nbsp; "spectrum"&nbsp; and&nbsp; <br>"power-spectral density"]]

<br>If we apply the signal&nbsp; $x(t)$&nbsp; to the input and denote the output signal by&nbsp; $y(t)$,&nbsp; the classical system theory provides the following statements:
*The output signal&nbsp; $y(t)$&nbsp; results from the&nbsp; [[Signal_Representation/The_Convolution_Theorem_and_Operation|$\text{convolution}$]]&nbsp; between the input signal&nbsp; $x(t)$&nbsp; and the impulse response&nbsp; $h(t)$.&nbsp; The following equation is equally valid for deterministic and stochastic signals:
:$$y(t) = x(t) \ast h(t) = \int_{-\infty}^{+\infty} x(\tau)\cdot h ( t - \tau) \,\,{\rm d}\tau.$$

*For deterministic signals,&nbsp; one usually takes a roundabout route using the spectral functions.&nbsp; The spectrum&nbsp; $X(f)$&nbsp; is the Fourier transform of&nbsp; $x(t)$.&nbsp; The multiplication with the frequency response&nbsp; $H(f)$&nbsp; leads to the output spectrum&nbsp; $Y(f)$.&nbsp; From this,&nbsp; the signal&nbsp; $y(t)$&nbsp; can be obtained by the Fourier inverse transformation.
*In the case of stochastic signals this procedure fails,&nbsp; because then the time functions&nbsp; $x(t)$&nbsp; and&nbsp; $y(t)$&nbsp; are not predictable for all times&nbsp; from $–∞$&nbsp; to&nbsp; $+∞$&nbsp; and thus,&nbsp; the corresponding amplitude spectra&nbsp; $X(f)$&nbsp; and&nbsp; $Y(f)$&nbsp; do not exist at all.&nbsp; In this case,&nbsp; we have to switch to the&nbsp; [[Theory_of_Stochastic_Signals/Power-Spectral_Density|$\text{power-spectral density}$]]&nbsp; defined in the last chapter.

==Amplitude spectrum and power-spectral density==
<br>
We consider an ergodic random process&nbsp;  $\{x(t)\}$,&nbsp; whose auto-correlation function&nbsp; $φ_x(τ)$&nbsp; is assumed to be known.&nbsp; The power-spectral density&nbsp; ${\it Φ}_x(f)$&nbsp; is then also uniquely determined via the Fourier transform and the following statements hold:
:[[File:P_ID467__Sto_T_5_1_S2_neu.png|right| |frame| For the ACF and PSD calculation of a random signal]]
<br>
#The&nbsp; [[Theory_of_Stochastic_Signals/Power-Spectral_Density|$\text{power-spectral density}$]]&nbsp; ${\it Φ}_x(f)$&nbsp; can be given – as well as the auto-correlation function&nbsp; $φ_x(τ)$ – for each individual pattern function of the stationary and ergodic random process&nbsp; $\{x(t)\}$,&nbsp; even if the specific course of&nbsp; $x(t)$&nbsp; is explicitly unknown.<br><br>
#The&nbsp; [[Signal_Representation/Fourier_Transform_and_its_Inverse#The_first_Fourier_integral|$\text{amplitude spectrum}$]]&nbsp; $X(f)$,&nbsp; on the other hand,&nbsp; is undefined because if the spectral function&nbsp; $X(f)$&nbsp; is known, the entire time function&nbsp; $x(t)$&nbsp; from&nbsp; $–∞$&nbsp; to&nbsp; $+∞$&nbsp; would also have to be known via the Fourier inverse transform,&nbsp; which clearly cannot be the case for a stochastic signal.<br><br>
#If a time section of finite time duration&nbsp; $T_{\rm M}$&nbsp; is known according to the sketch on the right,&nbsp; the Fourier transform can of course be applied to it again.
<br clear=all>
{{BlaueBox|TEXT=  
$\text{Theorem:}$&nbsp; The following relationship exists between the power-spectral density&nbsp; ${\it Φ}_x(f)$&nbsp; of the infinite time random signal&nbsp; $x(t)$&nbsp; and the amplitude spectrum&nbsp; $X_{\rm T}(f)$&nbsp; of the finite time section&nbsp; $x_{\rm T}(t)$:&nbsp; 
:$${ {\it \Phi}_x(f)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} }\cdot \vert X_{\rm T}(f)\vert ^2.$$}}


{{BlaueBox|TEXT=  
$\text{Proof:}$&nbsp; Previously,&nbsp; the&nbsp; 
[[Theory_of_Stochastic_Signals/Auto-Correlation_Function_(ACF)|$\text{auto-correlation function}$]]&nbsp; of an ergodic process with the random signal&nbsp; $x(t)$&nbsp; was given as follows: 
:$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm
M}/2}x(t)\cdot x(t + \tau)\hspace{0.1cm} \rm d \it t.$$
*It is permissible to replace the function&nbsp; $x(t)$,&nbsp; which is unbounded in time, by the function&nbsp; $x_{\rm T}(t)$,&nbsp; which is bounded on the time range&nbsp; $-T_{\rm M}/2$&nbsp; to&nbsp; $+T_{\rm M}/2$.&nbsp; &nbsp; $x_{\rm T}(t)$&nbsp; corresponds to the spectrum&nbsp; $X_{\rm T}(f)$,&nbsp; and by applying the&nbsp; [[Signal_Representation/Fourier_Transform_and_its_Inverse#The_first_Fourier_integral|$\text{first Fourier integral}$]]&nbsp; and the&nbsp; [[Signal_Representation/Fourier_Transform_Theorems#Shifting_Theorem|$\text{shifting theorem}$]]: 
:$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} }\cdot \int^{+T_{\rm M}/2}_{-T_{\rm
M}/2}x_{\rm T}(t)\cdot \int^{+\infty}_{-\infty}X_{\rm
T}(f)\cdot {\rm e}^{ {\rm j}2 \pi f ( t + \tau) } \hspace{0.1cm}
\rm d \it f \hspace{0.1cm} \rm d \it t.$$
*After splitting the exponent and swapping the time and frequency integrals,&nbsp; we get:
:$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} }\cdot  \int^{+\infty}_{-\infty}X_{\rm
T}(f)\cdot \left[ \int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x_{\rm
T}(t)\cdot  {\rm e}^{ {\rm j}2 \pi f  t  } \hspace{0.1cm} \rm d \it
t \right] \cdot {\rm e}^{ {\rm j}2 \pi f  \tau} \hspace{0.1cm} \rm d \it f.$$
*The inner integral describes the conjugate-complex spectrum&nbsp; ${X_{\rm T} }^{\star}(f)$.&nbsp; It further follows that: 
:$${ {\it \varphi}_x(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} }\cdot  \int^{+\infty}_{-\infty}\vert X_{\rm
T}(f)\vert^2 \cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d
\it f.$$
*A comparison with the theorem from&nbsp; [https://en.wikipedia.org/wiki/Norbert_Wiener $\text{Wiener}$]&nbsp; and&nbsp; [https://en.wikipedia.org/wiki/Aleksandr_Khinchin $\text{Chintchin}$]&nbsp; which is always valid in ergodicity,
:$${ {\it \varphi}_x(\tau)} = \int^{+\infty}_{-\infty}{\it \Phi}_x(f)
\cdot {\rm e}^{ {\rm j}2 \pi f \tau} \hspace{0.1cm} \rm d \it f ,$$
:shows the validity of the above relation:
:$${ {\it \Phi}_x(f)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} }\cdot \vert X_{\rm T}(f)\vert^2.$$
<div align="right">'''q.e.d.'''</div>}}

==Power-spectral density of the filter output signal==
<br>
Combining the statements made in the last two sections,&nbsp; we arrive at the following important result:

{{BlaueBox|TEXT=  
$\text{Theorem:}$&nbsp; The power-spectral density&nbsp; $\rm (PSD)$&nbsp; at the output of a linear time-invariant system with frequency response&nbsp; $H(f)$&nbsp; is obtained as the product  
*of the&nbsp; "input power-spectral density"&nbsp; ${\it Φ}_x(f)$&nbsp; 
*and the&nbsp; "power transfer function"&nbsp; $\vert H(f)\vert ^2$. 
:$${ {\it \Phi}_y(f)} = { {\it \Phi}_x(f)} \cdot \vert H(f)\vert ^2.$$}}


{{BlaueBox|TEXT=  
$\text{Proof:}$&nbsp; Starting from the three relations already derived before:
:$${ {\it \Phi}_x(f)} =\hspace{-0.1cm} \lim_{T_{\rm M}\to\infty}\hspace{0.01cm}
\frac{1}{ T_{\rm M} }\hspace{-0.05cm}\cdot\hspace{-0.05cm} \vert X_{\rm T}(f)\vert^2,$$
:$$ { {\it \Phi}_y(f)} =\hspace{-0.1cm} \lim_{T_{\rm M}\to\infty}\hspace{0.01cm}
\frac{1}{ T_{\rm M} }\hspace{-0.05cm}\cdot\hspace{-0.05cm}\vert Y_{\rm T}(f)\vert^2, $$
:$$Y_{\rm T}(f) = X_{\rm T}(f) \hspace{-0.05cm}\cdot\hspace{-0.05cm} H(f).$$

Substituting these equations into each other,&nbsp; we get the above result.
<div align="right">'''q.e.d.'''</div>}}


The following example illustrates the relationship with white noise.

{{GraueBox|TEXT=  
$\text{Example 1:}$&nbsp; 
At the input of a Gaussian low-pass filter with the frequency response
:$$H(f) = {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}$$
[[File:P_ID468__Sto_T_5_1_S3_neu.png |right|frame| Filter influence in the frequency domain]]
white noise&nbsp; $x(t)$&nbsp; with noise power density&nbsp; ${ {\it \Phi}_x(f)} =N_0/2$&nbsp; is present &nbsp; &rArr; &nbsp; two-sided representation.&nbsp; Then, the following holds for the power-spectral density of the output signal:
:$${ {\it \Phi}_y(f)} = \frac {N_0}{2} \cdot {\rm e}^{- 2 \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta
f)^2}.$$
The diagram shows the signals and power-spectral densities at the filter input and output. 

Notes:
#The signal&nbsp; $x(t)$&nbsp; – strictly speaking – cannot be plotted at all,&nbsp; since it has an infinite power &nbsp; &rArr; &nbsp; integral over&nbsp; ${\it Φ}_x(f)$&nbsp; from&nbsp; $-\infty$&nbsp; to&nbsp; $+\infty$. 
#The output signal&nbsp; $y(t)$&nbsp; has a lower frequency than&nbsp; $x(t)$&nbsp; and a finite power corresponding to the integral over&nbsp; ${\it Φ}_y(f)$.
#In one-sided representation,&nbsp; (only) for&nbsp; $f>0$&nbsp; would hold:&nbsp; ${ {\it \Phi}_x(f)} =N_0$.&nbsp; The statements&nbsp; (1)&nbsp; and&nbsp; (2)&nbsp; would also apply here in the same way.}} 

==The auto-correlation function of the filter output signal==
<br>
The calculated power-spectral density&nbsp; $\rm (PSD)$&nbsp; can also be written as follows: 
:$${{\it \Phi}_y(f)} = {{\it \Phi}_x(f)} \cdot H(f) \cdot H^{\star}(f).$$

{{BlaueBox|TEXT=  
$\text{Theorem:}$&nbsp;  The corresponding auto-correlation function&nbsp; $\rm (ACF)$&nbsp; is then obtained according to the&nbsp; [[Signal_Representation/Fourier_Transform_Laws|$\text{Fourier transform laws}$]]&nbsp; and by applying the&nbsp; [[Signal_Representation/The_Convolution_Theorem_and_Operation#Convolution_in_the_time_domain|$\text{convolution theorem}$]]: 
:$${ {\it \varphi}_y(\tau)} = { {\it \varphi}_x(\tau)} \ast h(\tau)\ast h(-
\tau).$$}}

 
In the transition from the spectral to the time domain, note:
* The Fourier retransforms are to be inserted in each case, namely
:$${{\it \varphi}_y(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\bullet\,{{\it \Phi}_y(f)}, \hspace{0.5cm}{{\it \varphi}_x(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\!\bullet\,{{\it \Phi}_x(f)}, \hspace{0.5cm}{h(\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\bullet\,{H(f)}, \hspace{0.5cm}{h(-\tau)} \circ\hspace{0.05cm}\!\!\!-\!\!\!-\!\!\!-\!\!\!\bullet\,{H^{\star}(f)}.$$
*Moreover,&nbsp; each multiplication becomes a convolution operation.


{{GraueBox|TEXT=  
$\text{Example 2:}$&nbsp; 
We consider again the same scenario as&nbsp; in&nbsp; [[Theory_of_Stochastic_Signals/Stochastic_System_Theory#Power-spectral_density_of_the_filter_output_signal| $\text{Example 1}$]],&nbsp; but this time in the time domain.&nbsp; It holds:
[[File:P_ID591__Sto_T_5_1_S4_neu.png |right|frame| Filter influence in the time domain]]
*Two-sided white noise power density:&nbsp; ${ {\it \Phi}_x(f)} =N_0/2$,
*Gaussian filter: &nbsp; $H(f) = {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(f/\Delta f)^2}\hspace{0.3cm}\Rightarrow \hspace{0.3cm}
 h(t) = \Delta f \cdot {\rm e}^{- \pi \hspace{0.03cm}\cdot \hspace{0.03cm}(\Delta f \hspace{0.03cm}\cdot \hspace{0.03cm}t)^2}.$


One can see from this diagram:
#The ACF of the input signal is now a Dirac delta function with weight&nbsp; $N_0/2$. 
#By convolution twice with the&nbsp;  (here also Gaussian)&nbsp;  impulse response&nbsp; $h(t)$&nbsp; or&nbsp; $h(–t)$&nbsp; one obtains the ACF&nbsp; $φ_y(τ)$&nbsp; of the output signal.
#Thus,&nbsp;  the ACF&nbsp; $φ_y(τ)$&nbsp; of the output signal is also Gaussian.
#The ACF value at&nbsp; $τ = 0$&nbsp; is identical to the area of the power-spectral density&nbsp; ${\it Φ}_y(f)$&nbsp; and characterizes the signal power (variance)&nbsp; $σ_y^2$. 
#In contrast, the area at&nbsp; $φ_y(τ)$&nbsp; gives the PSD value:&nbsp; ${\it Φ}_y(f = \rm 0)=N_0/2$. }}

==Cross-correlation function between input and output signal==
<br>
[[File:EN_Sto_T_5_1_S5.png |frame| Calculating the cross-correlation function |right]]
We again consider a filter with the frequency response&nbsp; $H(f)$&nbsp; and the impulse response&nbsp; $h(t)$.&nbsp; Further applies:
 
#The stochastic input signal&nbsp; $x(t)$&nbsp; is a sample function of the ergodic random process&nbsp;  $\{x(t)\}$.<br><br> 
#The corresponding auto-correlation function&nbsp; $\rm  (ACF)$&nbsp;  at the filter input is thus&nbsp; $φ_x(τ)$,&nbsp; while the power-spectral density&nbsp; $\rm  (PSD)$&nbsp;  is denoted by&nbsp;  ${\it Φ}_x(f)$.&nbsp;<br><br> 
#The corresponding descriptors of the ergodic random process&nbsp;  $\{y(t)\}$&nbsp; at the filter output are
 
::*the random output signal&nbsp; $y(t)$, 
::*the auto-correlation function&nbsp; $φ_y(τ)$&nbsp; 
::*and the conductance power-spectral density&nbsp;  ${\it Φ}_y(f)$.
<br clear=all>
{{BlaueBox|TEXT=  
$\text{Theorem:}$&nbsp;  For the&nbsp; &raquo;'''cross-correlation function'''&laquo;&nbsp; $\rm (CCF)$&nbsp; between the input&nbsp; and the output signal holds:
:$${ {\it \varphi}_{xy}(\tau)} = h(\tau)\ast { {\it \varphi}_x(\tau)}  .$$ 
Here,&nbsp;  $h(τ)$&nbsp; denotes the impulse response of the filter&nbsp; $($with the time variable&nbsp; $τ$&nbsp; instead of&nbsp; $t)$&nbsp; and&nbsp; ${ {\it \varphi}_{x}(\tau)}$&nbsp; denotes the ACF of the input signal.}}


{{BlaueBox|TEXT=  
$\text{Proof:}$&nbsp;  In general,&nbsp; for the cross-correlation function between two signals&nbsp; $x(t)$&nbsp; and&nbsp; $y(t)$:
:$${ {\it \varphi}_{xy}(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot y(t + \tau)\hspace{0.1cm} \rm d \it t.$$
*With the generally valid relation&nbsp; $y(t) = h(t) \ast x(t)$&nbsp; and the formal integration variable&nbsp; $θ$,&nbsp; we can also write for this:
:$${ {\it \varphi}_{xy}(\tau)} = \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}\frac{1}{ T_{\rm M} }\cdot\int^{+T_{\rm M}/2}_{-T_{\rm M}/2}x(t)\cdot \int^{+\infty}_{-\infty} h(\theta) \cdot x(t + \tau - \theta)\hspace{0.1cm}{\rm d}\theta\hspace{0.1cm}{\rm d} \it t.$$
*By interchanging the two integrals and subtracting the limit into the integral, we obtain:
:$${ {\it \varphi}_{xy}(\tau)} = \int^{+\infty}_{-\infty}
h(\theta) \cdot \left[ \lim_{T_{\rm M}\to\infty}\hspace{0.2cm}
\frac{1}{ T_{\rm M} } \cdot\int^{+T_{\rm M}/2}_{-T_{\rm
M}/2}x(t)\cdot x(t + \tau - \theta)\hspace{0.1cm}
\hspace{0.1cm} {\rm d} t \right]{\rm d}\theta.$$
*The expression in the square brackets gives the ACF value at the input at time&nbsp; $τ - θ$:
:$${ {\it \varphi}_{xy}(\tau)} = \int^{+\infty}_{-\infty}h(\theta) \cdot  \varphi_x(\tau - \theta)\hspace{0.1cm}\hspace{0.1cm}  {\rm d}\theta = h(\tau)\ast { {\it \varphi}_x(\tau)}  .$$ 
*However,&nbsp; the remaining integral describes the convolution operation in detailed notation.
<div align="right">'''q.e.d.'''</div>}}


{{BlaueBox|TEXT=  
$\text{Conclusion:}$&nbsp; In the frequency domain,&nbsp; the corresponding equation is:
:$${ {\it \Phi}_{xy}(f)} =   H(f)\cdot{ {\it \Phi}_x(f)} \hspace{0.3cm} \Rightarrow \hspace{0.3cm} H(f) = \frac{ {\it \Phi}_{xy}(f)}{ {\it \Phi}_{x}(f)}.$$

This equation shows that the filter frequency response&nbsp; $H(f)$&nbsp; from a measurement with stochastic excitation can be calculated completely&nbsp; &ndash; i.e., both magnitude and phase &ndash;&nbsp; if the following descriptive quantities are determined:
*the statistical characteristics at the input, either the ACF&nbsp; $φ_x(τ)$&nbsp; or the &nbsp; PSD ${\it Φ}_x(f)$, 
*as well as the cross-correlation function&nbsp; $φ_{xy}(τ)$&nbsp; or its Fourier transform&nbsp; ${\it Φ}_{xy}(f)$. }}

==Exercises for the chapter==
<br>
[[Aufgaben:Exercise_5.1:_Gaussian_ACF_and_Gaussian_Low-Pass|Exercise 5.1: Gaussian ACF and Gaussian Low-Pass]]

[[Aufgaben:Exercise_5.1Z:_Cosine_Square_Noise_Limitation|Exercise 5.1Z: Cosine Square Noise Limitation]]

[[Aufgaben:Aufgabe_5.2:_Bestimmung_des_Frequenzgangs|Exercise 5.2: Determination of the Frequency Response]]

[[Aufgaben:Exercise_5.2Z:_Two-Way_Channel|Exercise 5.2Z: Two-Way Channel]]



{{Display}}