Theory of Stochastic Signals/Moments of a Discrete Random Variable

1. Theory of Stochastic Signals
2. Discrete Random Variables
3. Moments of a Discrete Random Variable

• Previous page
• Next page

• Previous page
• Next page

Calculation as ensemble average or time average

The probabilities and the relative frequencies provide extensive information about a discrete random variable. 

Reduced information is obtained by the so-called moments  $m_k$,  where  $k$  represents a natural number.

$\text{Two alternative ways of calculation:}$ 

Under the condition  "Ergodicity"  implicitly assumed here,  there are two different calculation possibilities for the  $k$-th order moment:

• the  ensemble averaging  or  "expected value formation"   ⇒  averaging over all possible values  $\{ z_\mu\}$  with the index  $\mu = 1 , \hspace{0.1cm}\text{ ...} \hspace{0.1cm} , M$:

$$m_k = {\rm E} \big[z^k \big] = \sum_{\mu = 1}^{M}p_\mu \cdot z_\mu^k \hspace{2cm} \rm with \hspace{0.1cm} {\rm E\big[\text{ ...} \big]\hspace{-0.1cm}:} \hspace{0.3cm} \rm expected\hspace{0.1cm}value ;$$

• the  time averaging  over the random sequence  $\langle z_ν\rangle$  with the index  $ν = 1 , \hspace{0.1cm}\text{ ...} \hspace{0.1cm} , N$:

$$m_k=\overline{z_\nu^k}=\hspace{0.01cm}\lim_{N\to\infty}\frac{1}{N}\sum_{\nu=\rm 1}^{\it N}z_\nu^k\hspace{1.7cm}\rm with\hspace{0.1cm}horizontal\hspace{0.1cm}line\hspace{-0.1cm}:\hspace{0.1cm}time\hspace{0.1cm}average.$$

Note:

• Both types of calculations lead to the same asymptotic result for sufficiently large values of  $N$.
• For finite  $N$,  a comparable error results as when the probability is approximated by the relative frequency.

Linear mean - DC component

$\text{Definition:}$  With  $k = 1$  we obtain from the general equation for moments the  linear mean:

$$m_1 =\sum_{\mu=1}^{M}p_\mu\cdot z_\mu =\lim_{N\to\infty}\frac{1}{N}\sum_{\nu=1}^{N}z_\nu.$$

• The left part of this equation describes the ensemble averaging  (over all possible values),

while the right equation gives the determination as time average.

• In the context of signals,  this quantity is also referred to as the  "direct current"  $\rm (DC)$  component.

DC component  $m_1$  of a binary signal

$\text{Example 1:}$  A binary signal  $x(t)$  with the two possible values

• $1\hspace{0.03cm}\rm V$  $($for the symbol  $\rm L)$,
• $3\hspace{0.03cm}\rm V$  $($for the symbol  $\rm H)$

as well as the occurrence probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  has the linear mean  ("DC component")

$$m_1 = 0.2 \cdot 1\,{\rm V}+ 0.8 \cdot 3\,{\rm V}= 2.6 \,{\rm V}. $$

This is drawn as a red line in the graph.

If we determine this parameter by time averaging over the displayed  $N = 12$  signal values,  we obtain a slightly smaller value:

$$m_1\hspace{0.01cm}' = 4/12 \cdot 1\,{\rm V}+ 8/12 \cdot 3\,{\rm V}= 2.33 \,{\rm V}. $$

• Here,  the probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  were replaced by the corresponding frequencies  $h_{\rm L} = 4/12$  and  $h_{\rm H} = 8/12$  respectively.
• In this example the relative error due to insufficient sequence length  $N$  is greater than  $10\%$.

$\text{Note about our (admittedly somewhat unusual) nomenclature:}$

We denote binary symbols here as in circuit theory with  $\rm L$  ("Low")  and  $\rm H$  ("High")  to avoid confusion.

• In coding theory,  it is useful to map  $\{ \text{L, H}\}$  to  $\{0, 1\}$  to take advantage of the possibilities of modulo algebra.
• In contrast,  to describe modulation with bipolar  (antipodal)  signals,  one better chooses the mapping  $\{ \text{L, H}\}$ ⇔ $ \{-1, +1\}$.

Quadratic mean – variance – standard deviation

$\text{Definitions:}$ 

• Analogous to the linear mean,  $k = 2$  is obtained for the  root mean square  (short:  "rms"):

$$m_2 =\sum_{\mu=\rm 1}^{\it M}p_\mu\cdot z_\mu^2 =\lim_{N\to\infty}\frac{\rm 1}{\it N}\sum_{\nu=\rm 1}^{\it N}z_\nu^2.$$

• Together with the DC component  $m_1$,  the  variance  $σ^2$  can be determined from this as a further parameter ("Steiner's theorem"):

$$\sigma^2=m_2-m_1^2.$$

• The square root  $σ$  of the variance is called  "root mean square"   ⇒   rms value (sometimes this quantity is also called the  "standard deviation"):

$$\sigma=\sqrt{m_2-m_1^2}.$$

$\text{Notes on units:}$

• For message signals,  $m_2$  indicates the  "(average) power"  of a random signal,  referenced to  $1 \hspace{0.03cm} Ω$ resistance.
• If  $z$  describes a voltage,  $m_2$  accordingly has the unit  ${\rm V}^2$.
• The variance  $σ^2$ of a random signal corresponds physically to the  "alternating power".
• These definitions are based on the reference resistance  $1 \hspace{0.03cm} Ω$.

The following  (German language)  learning video illustrates the defined quantities using the example of a digital signal:
    Momentenberechnung bei diskreten Zufallsgrößen   ⇒   "Moment Calculation for Discrete Random Variables".

Standard deviation ("rms value"  of a binary signal

$\text{Example 2:}$  A binary signal  $x(t)$  with the two possible values

• $1\hspace{0.03cm}\rm V$  $($for the symbol  $\rm L)$,
• $3\hspace{0.03cm}\rm V$  $($for the symbol  $\rm H)$

as well as the occurrence probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  has the total signal power

$$P_{\rm total} = 0.2 \cdot (1\,{\rm V})^2+ 0.8 \cdot (3\,{\rm V})^2 = 7.4 \hspace{0.05cm}{\rm V}^2,$$

if one assumes the reference resistance  $R = 1 \hspace{0.05cm} Ω$ .

With the DC component  $m_1 = 2.6 \hspace{0.05cm}\rm V$  $($see  $\text{Example 1})$  it follows for

• the variance  $σ^2 = 7.4 \hspace{0.05cm}{\rm V}^2 - \big [2.6 \hspace{0.05cm}\rm V\big ]^2 = 0.64\hspace{0.05cm} {\rm V}^2$,
• the alternating power  $P_{\rm alter} = 0.64\hspace{0.05cm} {\rm W}$   ⇒   same numerical value as  $σ^2$,  but different unit,
• the rms value  $s_{\rm eff} = σ = 0.8 \hspace{0.05cm} \rm V$.

Insertion:   With other reference resistance   ⇒   $R \ne 1 \hspace{0.1cm} Ω$,  not all these calculations apply.  For example,  with  $R = 50 \hspace{0.1cm} Ω$,  the power  $P_{\rm total} $,  the alternating power  $P_{\rm alter}$,  and the rms value  $s_{\rm eff}$  have the following physical values:

$$P_{\rm total} \hspace{-0.05cm}= \hspace{-0.05cm} \frac{m_2}{R} \hspace{-0.05cm}= \hspace{-0.05cm} \frac{7.4\,{\rm V}^2}{50\,{\rm \Omega} } \hspace{-0.05cm}= \hspace{-0.05cm}0.148\,{\rm W},\hspace{0.5cm} P_{\rm alter} \hspace{-0.05cm} = \hspace{-0.05cm} \frac{\sigma^2}{R} \hspace{-0.05cm}= \hspace{-0.05cm}12.8\,{\rm mW} \hspace{0.05cm},\hspace{0.5cm} s_{\rm eff} \hspace{-0.05cm} = \hspace{-0.05cm}\sqrt{R \cdot P_{\rm W} } \hspace{-0.05cm}= \hspace{-0.05cm} \sigma \hspace{-0.05cm}= \hspace{-0.05cm} 0.8\,{\rm V}.$$

The same variance  $σ^2 = 0.64\hspace{0.05cm} {\rm V}^2$ and the same rms value  $s_{\rm eff}= 0.8 \hspace{0.05cm} \rm V$  are obtained for amplitudes  $0\hspace{0.05cm}\rm V$  $($for symbol  $\rm L)$  and $2\hspace{0.05cm}\rm V$  $($for symbol  $\rm H)$,  provided that the probabilities  $p_{\rm L} = 0.2$  and  $p_{\rm H} = 0.8$  remain the same.  Only the DC component and the total power change:

$$m_1 = 1.6 \hspace{0.05cm}{\rm V}, \hspace{0.5cm}P_{\rm total} = {m_1}^2 +\sigma^2 = 3.2 \hspace{0.05cm}{\rm V}^2.$$

Exercises for the chapter

Exercise 2.2: Multi-Level Signals

Exercise 2.2Z: Discrete Random Variables