Theory of Stochastic Signals/Moments of a Discrete Random Variable

1. Theory of Stochastic Signals
2. Discrete Random Variables
3. Moments of a Discrete Random Variable

• Previous page
• Next page

• Previous page
• Next page

Calculation as ensemble average or time average

The probabilities and the relative frequencies provide extensive information about a discrete random variable.  Reduced information is obtained by the so-called  »moments«   $m_k$,  where  $k$  represents a natural number.

$\text{Two alternative ways of calculation:}$ 

Under the condition  »$\text{ergodicity}$«  implicitly assumed here,  there are two different calculation possibilities for the  $k$-th order moment:

$\rm (A)$  the  »ensemble averaging«  or  »expected value formation«   ⇒  averaging over all possible values  $\{ z_\mu\}$  with index  $\mu = 1 , \hspace{0.1cm}\text{ ...} \hspace{0.1cm} , M$:

$$m_k = {\rm E} \big[z^k \big] = \sum_{\mu = 1}^{M}p_\mu \cdot z_\mu^k \hspace{2cm} \rm with \hspace{0.1cm} {\rm E\big[\text{ ...} \big]\hspace{-0.1cm}:} \hspace{0.3cm} \rm expected\hspace{0.1cm}value ;$$

$\rm (B)$  the  »time averaging«  over the random sequence  $\langle z_ν\rangle$  with index  $ν = 1 , \hspace{0.1cm}\text{ ...} \hspace{0.1cm} , N$:

$$m_k=\overline{z_\nu^k}=\hspace{0.01cm}\lim_{N\to\infty}\frac{1}{N}\sum_{\nu=\rm 1}^{\it N}z_\nu^k\hspace{1.7cm}\rm with\hspace{0.1cm}horizontal\hspace{0.1cm}line\hspace{-0.1cm}:\hspace{0.1cm}time\hspace{0.1cm}average.$$

Note:

1. Both types of calculations lead to the same asymptotic result for sufficiently large values of  $N$.
2. For finite  $N$,  a comparable error results as when the probability is approximated by the relative frequency.

First order moment – linear mean – DC component

$\text{Definition:}$  With  $k = 1$  we obtain from the general equation the first order moment   ⇒   the  »linear mean«:

$$m_1 =\sum_{\mu=1}^{M}p_\mu\cdot z_\mu =\lim_{N\to\infty}\frac{1}{N}\sum_{\nu=1}^{N}z_\nu.$$

• The left part of this equation describes the ensemble averaging  $($over all possible values$)$.

• The right equation gives the determination as time average.

• In the context of signals,  this quantity is also referred to as the  »$\text{direct current}$«  $\rm (DC)$  component.

DC component  $m_1$  of a binary signal

$\text{Example 1:}$  A binary signal  $x(t)$  with the two possible values

• $1\hspace{0.03cm}\rm V$  $($for symbol  $\rm L)$,
• $3\hspace{0.03cm}\rm V$  $($for symbol  $\rm H)$

as well as the occurrence probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  has the linear mean  $($»DC component«$)$

$$m_1 = 0.2 \cdot 1\,{\rm V}+ 0.8 \cdot 3\,{\rm V}= 2.6 \,{\rm V}. $$

This is drawn as a red line in the graph.

If we determine this parameter by time averaging over the displayed  $N = 12$  signal values,  we obtain a slightly smaller value:

$$m_1\hspace{0.01cm}' = 4/12 \cdot 1\,{\rm V}+ 8/12 \cdot 3\,{\rm V}= 2.33 \,{\rm V}. $$

1. Here,  the probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  were replaced by the corresponding frequencies  $h_{\rm L} = 4/12$  and  $h_{\rm H} = 8/12$  respectively.
2. In this example the relative error due to insufficient sequence length  $N$  is greater than  $10\%$.

$\text{Note about our (admittedly somewhat unusual) nomenclature:}$

We denote binary symbols here as in circuit theory with  $\rm L$  $($Low$)$  and  $\rm H$  $($High$)$  to avoid confusion.

• In coding theory,  it is useful to map  $\{ \text{L, H}\}$  to  $\{0, 1\}$  to take advantage of the possibilities of modulo algebra.

• In contrast,  to describe modulation with bipolar  $($antipodal$)$  signals,  one better chooses the mapping  $\{ \text{L, H}\}$ ⇔ $ \{-1, +1\}$.

Second order moment – power – variance – standard deviation

$\text{Definitions:}$ 

$\rm (A)$  Analogous to the linear mean,   $k = 2$  obtains the  »second order moment«:

$$m_2 =\sum_{\mu=\rm 1}^{\it M}p_\mu\cdot z_\mu^2 =\lim_{N\to\infty}\frac{\rm 1}{\it N}\sum_{\nu=\rm 1}^{\it N}z_\nu^2.$$

$\rm (B)$  Together with the DC component  $m_1$  the  »variance«  $σ^2$  can be determined from this as a further parameter  $($»Steiner's theorem«$)$:

$$\sigma^2=m_2-m_1^2.$$

$\rm (C)$  The  »standard deviation«  $σ$  is the square root of the variance:

$$\sigma=\sqrt{m_2-m_1^2}.$$

$\text{Notes on units:}$

1. For a random signal  $x(t)$   ⇒   the second moment  $m_2$  gives the total power  $($DC power plus AC power$)$  related to the resistance  $1 \hspace{0.03cm} Ω$.
2. If  $x(t)$  describes a voltage,  accordingly  $m_2$  has the unit  ${\rm V}^2$  and the rms value  $($»root mean square«$)$  $x_{\rm eff}=\sqrt{m_2}$   has the unit  ${\rm V}$. 
3. The total power for any reference resistance  $R$  is calculated to   $P=m_2/R$  and accordingly  has the unit  $\rm V^2/(V/A) = W$.
4. If  $x(t)$  describes a current waveform,  then  $m_2$  has the unit  ${\rm A}^2$  and the rms value  $x_{\rm eff}=\sqrt{m_2}$  has the unit  ${\rm A}$.  
5. The total power for any reference resistance  $R$  is calculated to   $P=m_2\cdot R$  and accordingly  has the unit  $\rm A^2 \cdot(V/A) = W$.
6. Only in the special case  $m_1=0$   ⇒   the variance is  $σ^2=m_2$.  Then the standard deviation   $σ$  coincides also with the rms value  $x_{\rm eff}$ .

⇒   The following  $($German language$)$  learning video illustrates the defined quantities using the example of a digital signal:
            »Momentenberechnung bei diskreten Zufallsgrößen»   ⇒   »Moment Calculation for Discrete Random Variables».

"Standard deviation"  of a binary signal

$\text{Example 2:}$  For a binary signal  $x(t)$  with the amplitude values

• $1\hspace{0.03cm}\rm V$  $($for symbol  $\rm L)$,
• $3\hspace{0.03cm}\rm V$  $($for symbol  $\rm H)$

and the probabilities of occurrence  $p_{\rm L} = 0.2$  resp.  $p_{\rm H} = 0.8$  results for the second moment:

$$m_2 = 0.2 \cdot (1\,{\rm V})^2+ 0.8 \cdot (3\,{\rm V})^2 = 7.4 \hspace{0.1cm}{\rm V}^2,$$

1. The rms value  $x_{\rm eff}=\sqrt{m_2}=2.72\,{\rm V}$  is independent of the reference resistance  $R$  unlike the total power.
2. For the latter, with  $R=1 \hspace{0.1cm} Ω$  the value  $P=m_2/R=7.4 \hspace{0.1cm}{\rm W}$,  with  $R=50 \hspace{0.1cm} Ω$  on the other hand, only  $P=0.148 \hspace{0.1cm}{\rm W}$.

With the DC component  $m_1 = 2.6 \hspace{0.05cm}\rm V$  $($see  $\text{Example 1})$  it follows for

• the variance  $ σ^2 = 7.4 \hspace{0.05cm}{\rm V}^2 - \big [2.6 \hspace{0.05cm}\rm V\big ]^2 = 0.64\hspace{0.05cm} {\rm V}^2$,

• the standard deviation  $σ = 0.8 \hspace{0.05cm} \rm V$.

The same variance  $ σ^2 = 0.64\hspace{0.05cm} {\rm V}^2$ and the same standard deviation  $σ = 0.8 \hspace{0.05cm} \rm V$  result for the amplitudes  $0\hspace{0.05cm}\rm V$  $($for symbol  $\rm L)$  and $2\hspace{0.05cm}\rm V$  $($for symbol  $\rm H)$,  provided the occurrence probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  remain the same.  Only the DC component and the total power change:

$$m_1 = 1.6 \hspace{0.05cm}{\rm V}, $$

$$P = {m_1}^2 +\sigma^2 = 3.2 \hspace{0.05cm}{\rm V}^2.$$

Exercises for the chapter

Exercise 2.2: Multi-Level Signals

Exercise 2.2Z: Discrete Random Variables