Theory of Stochastic Signals/Expected Values and Moments

1. Theory of Stochastic Signals
2. Continuous Random Variables
3. Expected Values and Moments

• Previous page
• Next page

• Previous page
• Next page

Moment calculation as ensemble average

The probability density function (PDF), like the distribution function (CDF), provides very extensive information about the random variable under consideration.  Less, but more compact information is provided by the so-called  expected values  and  moments.

• Their calculation possibilities have already been given for discrete random variables in the chapter  moments of a discrete random variable .
• Now these integrative descriptive quantities "expected value" and "moment" are considered in the context of the probability density function (PDF) of continuous random variables and thus formulated more generally.

$\text{Definition:}$  The  expected value  with respect to any weighting function $g(x)$ can be calculated with the PDF  $f_{\rm x}(x)$  in the following way:

$${\rm E}\big[g (x ) \big] = \int_{-\infty}^{+\infty} g(x)\cdot f_{x}(x) \,{\rm d}x.$$

Substituting into this equation for  $g(x) = x^k$  we get the  moment of $k$-th order:

$$m_k = {\rm E}\big[x^k \big] = \int_{-\infty}^{+\infty} x^k\cdot f_{x} (x ) \, {\rm d}x.$$

From this equation follows.

• with  $k = 1$  for the  linear mean:

$$m_1 = {\rm E}\big[x \big] = \int_{-\infty}^{ \rm +\infty} x\cdot f_{x} (x ) \,{\rm d}x,$$

• with  $k = 2$  for the  root mean square:

$$m_2 = {\rm E}\big[x^{\rm 2} \big] = \int_{-\infty}^{ \rm +\infty} x^{ 2}\cdot f_{ x} (x) \,{\rm d}x.$$

For a discrete, $M$–-level random variable, the formulas given here again yield the equations already given in the second chapter (calculation as a ensemble average):

$$m_1 = \sum\limits_{\mu=1}^{ M}\hspace{0.15cm}p_\mu\cdot x_\mu,$$

$$m_2 = \sum\limits_{\mu= 1}^{ M}\hspace{0.15cm}p_\mu\cdot x_\mu^2.$$

Here it is taken into account that the integral over the Dirac function  $δ(x)$  is equal  $1$ .

In connection with signals, the following terms are also common:

• $m_1$  indicates the DC component ,
• $m_2$  corresponds to the  (referred to the unit resistance  $1 \ Ω$ )  signal power.

For example, if  $x$  denotes a voltage, then according to these equations 

• $m_1$  has the unit  ${\rm V}$  and 
• $m_2$  the unit  ${\rm V}^2$. 

If one wants to indicate the power in "Watt"  $\rm (W)$ , then  $m_2$  must still be divided by the resistance value  $R$ .

Central moments

$\text{Definition:}$  Especially important in statistics are the  central moments, which, in contrast to the conventional moments, are each related to the mean  $m_1$  :

$$\mu_k = {\rm E}\big[(x-m_{\rm 1})^k\big] = \int_{-\infty}^{+\infty} (x-m_{\rm 1})^k\cdot f_x(x) \,\rm d \it x.$$

The noncentered moments  $m_k$  can be directly converted to the centered moments  $\mu_k$  :

$$\mu_k = \sum\limits_{\kappa= 0}^{k} \left( \begin{array}{*{2}{c}} k \ \kappa \ \end{array} \right)\cdot m_\kappa \cdot (-m_1)^{k-\kappa}.$$

According to the general equations of  last page  the formal quantities  $m_0 = 1$  and  $\mu_0 = 1$ result. For the first order central moment, according to the above definition, always  $\mu_1 = 0$ holds.

In the opposite direction, the following equations hold for  $k = 1$,  $k = 2$,  and so on:

$$m_k = \sum\limits_{\kappa= 0}^{k} \left( \begin{array}{*{2}{c}} k \ \kappa \ \end{array} \right)\cdot \mu_\kappa \cdot {m_1}^{k-\kappa}.$$

$\text{Example 1:}$  All moments of a binary random variable with probabilities  ${\rm Pr}(0) = 1 - p$   and  ${\rm Pr}(1) = p$  are of equal value:

$$m_1 = m_2 = m_3 = m_4 = \hspace{0.05cm}\text{...} \hspace{0.05cm}= p.$$

Using the above equations, we then obtain for the first three central moments:

$$\mu_2 = m_2 - m_1^2 = p -p^2, $$

$$\mu_3 = m_3 - 3 \cdot m_2 \cdot m_1 + 2 \cdot m_1^3 = p - 3 \cdot p^2 + 2 \cdot p^3, $$

$$ \mu_4 = m_4 - 4 \cdot m_3 \cdot m_1 + 6 \cdot m_2 \cdot m_1^2 - 3 \cdot m_1^4 = p - 4 \cdot p^2 + 6 \cdot p^3- 3 \cdot p^4. $$

Some common central moments

From the  last definition  the following additional characteristics can be derived:

$\text{Definition:}$  The  variance  $σ^2$  of the considered random variable is the second order central moment   ⇒   $\mu_2.$

• The variance  $σ^2$  physically corresponds to the alternating power  and the dispersion  $σ$  gives the rms value  .
• From the linear and the quadratic mean, the variance can be calculated according to Steiner's theorem  in the following way:

$$\sigma^{2} = m_2 - m_1^{2}.$$

$\text{Definition:}$  The  Charlier's skewness'  $S$ denotes the third central moment related to  $σ^3$  .

• For symmetric density function, this parameter is always  $S=0$.
• The larger  $S = \mu_3/σ^3$  is, the more asymmetric is the WDF around the mean  $m_1$.
• For example, for the  exponential distribution  the (positive) skewness $S =2$, and this is independent of the distribution parameter  $λ$.
• For positive skewness  $(S > 0)$  one speaks of "a right-skewed or left-sloping distribution";  this slopes flatter on the right side than on the left.
• When the skewness is negative  $(S < 0)$  there is a "left-skewed or right-steep distribution";  such a distribution falls flatter on the left side than on the right

. $\text{Definition:}$  The fourth-order central moment is also used for statistical analyses;  The quotient  $K = \mu_4/σ^4$ is called  kurtosis  .

• For a Gaussian distributed random variable  this always yields the value  $K = 3$.
• Using also the so-called  excess  $\gamma = K - 3$ ,  also known under the term "overkurtosis".
• This parameter can be used, for example, to check whether a random variable at hand is approximately Gaussian:  $\gamma \approx 0$.

$\text{Example 2:}$ 

• If the PDF has fewer offshoots than the Gaussian distribution, the kurtosis  $K < 3$.  For example, for the uniformly distributed  $K = 1.8$  ⇒   $\gamma = - 1.2$.
• In contrast,  $K > 3$  indicates that the spurs are more pronounced than for the Gaussian distribution. For example, for the  exponential distribution   $K = 9$.
• For the  Laplace distribution  ⇒   two-sided exponential distribution results in a slightly smaller kurtosis  $K = 6$  and the excess $\gamma = 3$.

Moment calculation as time average

The expected value calculation according to the previous equations of this section corresponds to a  ensemble averaging, that is, averaging over all possible values  $x_\mu$.

However, the moments  $m_k$  can also be determined as  time averages  if the stochastic process generating the random variable is stationary and ergodic:

• The exact definition for such a stationary and ergodic random process can be found in  Chapter 4.4.
• A time-averaging is always denoted by a sweeping line in the following.
• For discrete time, the random signal  $x(t)$  is replaced by the random sequence  $〈x_ν〉$ .
• For finite sequence length, these time averages are with  $ν = 1, 2,\hspace{0.05cm}\text{...}\hspace{0.05cm} , N$:

$$m_k=\overline{x_{\nu}^{k}}=\frac{1}{N} \cdot \sum\limits_{\nu=1}^{N}x_{\nu}^{k},$$

$$m_1=\overline{x_{\nu}}=\frac{1}{N} \cdot \sum\limits_{\nu=1}^{N}x_{\nu},$$

$$m_2=\overline{x_{\nu}^{2}}=\frac{1}{N} \cdot \sum\limits_{\nu=1}^{N}x_{\nu}^{2}.$$

If the moments (or expected values) are to be determined by simulation, in practice this is usually done by time averaging.  The corresponding computational algorithm differs only mariginally for discrete and continuous random variables.

The topic of this chapter is illustrated with examples in the (German language) learning video  Momentenberechnung bei diskreten Zufallsgrößen  $\Rightarrow$ Moment calculation for discrete random variables.

Charakteristische Funktion

$\text{Definition:}$  Ein weiterer Sonderfall eines Erwartungswertes ist die  charakteristische Funktion, wobei hier für die Bewertungsfunktion  $g(x) = {\rm e}^{\hspace{0.03cm}{\rm j} \hspace{0.03cm}{\it Ω}\hspace{0.05cm}x}$  zu setzen ist:

$$C_x({\it \Omega}) = {\rm E}\big[{\rm e}^{ {\rm j} \hspace{0.05cm} {\it \Omega} \hspace{0.05cm} x}\big] = \int_{-\infty}^{+\infty} {\rm e}^{ {\rm j} \hspace{0.05cm} {\it \Omega} \hspace{0.05cm} x}\cdot f_{\rm x}(x) \hspace{0.1cm}{\rm d}x.$$

Ein Vergleich mit dem Kapitel  Fouriertransformation und Fourierrücktransformation  im Buch "Signaldarstellung" zeigt, dass die charakteristische Funktion als die Fourierrücktransformierte der Wahrscheinlichkeitsdichtefunktion interpretiert werden kann:

$$C_x ({\it \Omega}) \hspace{0.3cm} \circ \!\!-\!\!\!-\!\!\!-\!\! \bullet \hspace{0.3cm} f_{x}(x).$$

Ist die Zufallsgröße  $x$  dimensionslos, so ist auch das Argument  $\it Ω$  der charakteristischen Funktion ohne Einheit.

• Das Symbol  $\it Ω$  wurde gewählt, da das Argument hier einen gewissen Bezug zur Kreisfrequenz beim zweiten Fourierintegral aufweist  (gegenüber der Darstellung im  $f$–Bereich fehlt allerdings der Faktor  $2\pi$  im Exponenten).
• Es wird aber nochmals eindringlich darauf hingewiesen, dass – wenn man einen Bezug zur Systemtheorie herstellen will – $C_x({\it Ω})$  der „Zeitfunktion” und  $f_{x}(x)$  der „Spektralfunktion” entsprechen würde.

$\text{Berechnungsmöglichkeit:}$  Entwickelt man die komplexe Funktion  ${\rm e}^{\hspace{0.03cm}{\rm j} \hspace{0.03cm}{\it Ω}\hspace{0.05cm}x}$  in eine Potenzreihe  und vertauscht Erwartungswertbildung und Summation, so folgt die Reihendarstellung der charakteristischen Funktion:

$$C_x ( {\it \Omega}) = 1 + \sum_{k=1}^{\infty}\hspace{0.2cm}\frac{m_k}{k!} \cdot ({\rm j} \hspace{0.01cm}{\it \Omega})^k .$$

Die  Aufgabe 3.4  zeigt weitere Eigenschaften der charakteristischen Funktion auf.

$\text{Beispiel 3:}$ 

• Bei einer symmetrischen binären (zweipunktverteilten) Zufallsgröße  $x ∈ \{\pm1\}$  mit den Wahrscheinlichkeiten  ${\rm Pr}(–1) = {\rm Pr}(+1) = 1/2$  verläuft die charakteristische Funktion cosinusförmig.
• Das Analogon in der Systemtheorie ist, dass das Spektrum eines Cosinussignals mit der Kreisfrequenz  ${\it Ω}_{\hspace{0.03cm}0}$  aus zwei Diracfunktionen bei  $±{\it Ω}_{\hspace{0.03cm}0}$  besteht.

$\text{Beispiel 4:}$ 

• Eine Gleichverteilung zwischen  $±y_0$  besitzt nach den  Gesetzen der Fouriertransformation  folgende charakteristische Funktion:

$$C_y({\it \Omega}) = \frac{1}{2 y_0} \cdot \int_{-y_0}^{+y_0} {\rm e}^{ {\rm j} \hspace{0.05cm} {\it \Omega} \hspace{0.05cm} y} \,{\rm d}y = \frac{ {\rm e}^{ {\rm j} \hspace{0.05cm} y_0 \hspace{0.05cm}{\it \Omega} } - {\rm e}^{ - {\rm j} \hspace{0.05cm} y_0 \hspace{0.05cm} {\it \Omega} } }{2 {\rm j} \cdot y_0 \cdot {\it \Omega} } = \frac{ {\rm sin}(y_0 \cdot {\it \Omega})}{ y_0 \cdot {\it \Omega} } = {\rm si}(y_0 \cdot {\it \Omega}). $$

• Die Funktion  ${\rm si}(x) = \sin(x)/x$  kennen wir bereits aus dem Buch  Signaldarstellung.
• Sie ist auch unter dem Namen  Spaltfunktion  bekannt.

Aufgaben zum Kapitel

Aufgabe 3.3: Momente bei $\cos^2$–WDF

Aufgabe 3.3Z: Momente bei Dreieck–WDF

Aufgabe 3.4: Charakteristische Funktion