View source for Theory of Stochastic Signals/Moments of a Discrete Random Variable

 
{{Header
|Untermenü=Discrete Random Variables
|Vorherige Seite=From Random Experiment to Random Variable
|Nächste Seite=Binomial Distribution
}}
==Calculation as ensemble average or time average==
<br>
The probabilities and the relative frequencies provide extensive information about a discrete random variable.&nbsp; Reduced information is obtained by the so-called&nbsp; &raquo;'''moments'''&laquo; &nbsp; $m_k$,&nbsp; where&nbsp; $k$&nbsp; represents a natural number. 

{{BlueBox|TEXT=  
$\text{Two alternative ways of calculation:}$&nbsp;

Under the  condition&nbsp; [[Theory_of_Stochastic_Signals/Auto-Correlation_Function#Ergodic_random_processes|&raquo;$\text{ergodicity}$&laquo;]]&nbsp; implicitly assumed here,&nbsp; there are two different calculation possibilities for the&nbsp; $k$-th order moment:
 
$\rm (A)$&nbsp; the&nbsp; &raquo;'''ensemble averaging'''&laquo;&nbsp; or&nbsp; &raquo;expected value formation&laquo; &nbsp; &rArr; &nbsp;averaging over all possible values&nbsp; $\{ z_\mu\}$&nbsp; with index&nbsp; $\mu = 1 , \hspace{0.1cm}\text{ ...}  \hspace{0.1cm} , M$: 
:$$m_k = {\rm E} \big[z^k \big] = \sum_{\mu = 1}^{M}p_\mu \cdot z_\mu^k \hspace{2cm} \rm with \hspace{0.1cm} {\rm E\big[\text{ ...} \big]\hspace{-0.1cm}:} \hspace{0.3cm} \rm expected\hspace{0.1cm}value ;$$
$\rm (B)$&nbsp; the&nbsp; &raquo;'''time averaging'''&laquo;&nbsp; over the random sequence&nbsp; $\langle z_ν\rangle$&nbsp; with index&nbsp; $ν = 1 , \hspace{0.1cm}\text{ ...}  \hspace{0.1cm} , N$: 
:$$m_k=\overline{z_\nu^k}=\hspace{0.01cm}\lim_{N\to\infty}\frac{1}{N}\sum_{\nu=\rm 1}^{\it N}z_\nu^k\hspace{1.7cm}\rm with\hspace{0.1cm}horizontal\hspace{0.1cm}line\hspace{-0.1cm}:\hspace{0.1cm}time\hspace{0.1cm}average.$$}}


<u>Note:</u>
#Both types of calculations lead to the same asymptotic result for sufficiently large values of&nbsp; $N$. 
#For finite&nbsp; $N$,&nbsp; a comparable error results as when the probability is approximated by the relative frequency. 

==First order moment &ndash; linear mean &ndash; DC component==
<br>
{{BlueBox|TEXT= 
$\text{Definition:}$&nbsp; With&nbsp; $k = 1$&nbsp; we obtain from the general equation the first order moment &nbsp; &rArr; &nbsp; the&nbsp; &raquo;'''linear mean'''&laquo;: 
:$$m_1 =\sum_{\mu=1}^{M}p_\mu\cdot z_\mu =\lim_{N\to\infty}\frac{1}{N}\sum_{\nu=1}^{N}z_\nu.$$
*The left part of this equation describes the ensemble averaging&nbsp; $($over all possible values$)$.
 
* The right equation gives the determination as time average.
 
*In the context of signals,&nbsp; this quantity is also referred to as the&nbsp; [[Signal_Representation/Direct_Current_Signal_-_Limit_Case_of_a_Periodic_Signal|&raquo;$\text{direct current}$&laquo;]]&nbsp; $\rm (DC)$&nbsp; component.}}


{{GraueBox|TEXT=
[[File:P_ID49__Sto_T_2_2_S2_neu.png|right|frame|DC component&nbsp; $m_1$&nbsp; of a binary signal]]
$\text{Example 1:}$&nbsp; A binary signal&nbsp; $x(t)$&nbsp; with the two possible values 
*$1\hspace{0.03cm}\rm V$&nbsp; $($for symbol&nbsp; $\rm L)$, 
*$3\hspace{0.03cm}\rm V$&nbsp; $($for symbol&nbsp; $\rm H)$ 


as well as the occurrence probabilities&nbsp; $p_{\rm L} = 0.2$&nbsp; and&nbsp; $p_{\rm H} = 0.8$&nbsp; has the linear mean&nbsp; $($&raquo;DC component&laquo;$)$
:$$m_1 = 0.2 \cdot 1\,{\rm V}+ 0.8 \cdot 3\,{\rm V}= 2.6 \,{\rm V}. $$
This is drawn as a red line in the graph.

If we determine this parameter by time averaging over the displayed&nbsp; $N = 12$&nbsp; signal values,&nbsp; we obtain a slightly smaller value: 
:$$m_1\hspace{0.01cm}' = 4/12 \cdot 1\,{\rm V}+ 8/12 \cdot 3\,{\rm V}= 2.33 \,{\rm V}. $$
#Here,&nbsp; the probabilities&nbsp; $p_{\rm L} = 0.2$&nbsp; and&nbsp; $p_{\rm H} = 0.8$&nbsp; were replaced by the corresponding frequencies&nbsp; $h_{\rm L} = 4/12$&nbsp; and&nbsp; $h_{\rm H} = 8/12$&nbsp; respectively. 
#In this example the relative error due to insufficient sequence length&nbsp; $N$&nbsp; is greater than&nbsp; $10\%$. 


$\text{Note about our (admittedly somewhat unusual) nomenclature:}$

We denote binary symbols here as in circuit theory with&nbsp; $\rm L$&nbsp; $($Low$)$&nbsp; and&nbsp; $\rm H$&nbsp; $($High$)$&nbsp; to avoid confusion. 
*In coding theory,&nbsp; it is useful to map&nbsp; $\{ \text{L,  H}\}$&nbsp; to&nbsp; $\{0, 1\}$&nbsp; to take advantage of the possibilities of modulo algebra. 

*In contrast,&nbsp; to describe modulation with bipolar&nbsp; $($antipodal$)$&nbsp; signals,&nbsp; one better chooses the mapping&nbsp; $\{ \text{L, H}\}$ ⇔ $ \{-1, +1\}$.
}}


==Second order moment &ndash; power   &ndash; variance &ndash; standard deviation ==
<br>
{{BlaueBox|TEXT=
$\text{Definitions:}$&nbsp;

$\rm (A)$&nbsp; Analogous to the linear mean, &nbsp; $k = 2$&nbsp; obtains the&nbsp; &raquo;'''second order moment'''&laquo;:
:$$m_2 =\sum_{\mu=\rm 1}^{\it M}p_\mu\cdot z_\mu^2 =\lim_{N\to\infty}\frac{\rm 1}{\it N}\sum_{\nu=\rm 1}^{\it N}z_\nu^2.$$

$\rm (B)$&nbsp; Together with the DC component&nbsp; $m_1$&nbsp; the&nbsp; &raquo;'''variance'''&laquo;&nbsp; $σ^2$&nbsp; can be determined from this as a further parameter&nbsp; $($&raquo;Steiner's theorem&laquo;$)$: 
:$$\sigma^2=m_2-m_1^2.$$
$\rm (C)$&nbsp; The&nbsp; &raquo;'''standard deviation'''&laquo;&nbsp; $σ$&nbsp; is the square root of the variance:
:$$\sigma=\sqrt{m_2-m_1^2}.$$}}


$\text{Notes on units:}$

#For a random signal&nbsp; $x(t)$ &nbsp; &rArr; &nbsp; the second moment&nbsp; $m_2$&nbsp; gives the total power&nbsp; $($DC power plus AC power$)$&nbsp; related to the resistance&nbsp; $1 \hspace{0.03cm} Ω$. 
#If&nbsp; $x(t)$&nbsp; describes a voltage,&nbsp; accordingly&nbsp; $m_2$&nbsp; has the unit&nbsp; ${\rm V}^2$&nbsp; and the rms value&nbsp; $($&raquo;root mean square&laquo;$)$&nbsp; $x_{\rm eff}=\sqrt{m_2}$ &nbsp; has the unit&nbsp; ${\rm V}$.&nbsp; 
#The total power for any reference resistance&nbsp; $R$&nbsp; is calculated to &nbsp; $P=m_2/R$&nbsp; and accordingly&nbsp; has the unit&nbsp; $\rm V^2/(V/A) = W$.
#If&nbsp; $x(t)$&nbsp; describes a current waveform,&nbsp; then&nbsp; $m_2$&nbsp; has the unit&nbsp; ${\rm A}^2$&nbsp; and the rms value&nbsp; $x_{\rm eff}=\sqrt{m_2}$&nbsp; has the unit&nbsp; ${\rm A}$. &nbsp; 
#The total power for any reference resistance&nbsp; $R$&nbsp; is calculated to &nbsp; $P=m_2\cdot R$&nbsp; and accordingly&nbsp; has the unit&nbsp; $\rm A^2 \cdot(V/A) = W$.
#Only in the special case&nbsp; $m_1=0$ &nbsp; &rArr; &nbsp; the variance is&nbsp; $σ^2=m_2$.&nbsp; Then the standard deviation &nbsp; $σ$&nbsp; coincides also with the rms value&nbsp; $x_{\rm eff}$&nbsp;.


&rArr; &nbsp; The following&nbsp; $($German language$)$&nbsp; learning video illustrates the defined quantities using the example of a digital signal: <br> &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; [[Momentenberechnung bei diskreten Zufallsgrößen (Lernvideo)|&raquo;Momentenberechnung bei diskreten Zufallsgrößen&raquo;]] &nbsp; &rArr;  &nbsp; &raquo;Moment Calculation for Discrete Random Variables&raquo;.

{{GraueBox|TEXT=
[[File:P_ID456__Sto_T_2_2_S3_neu.png | right|frame|"Standard deviation"&nbsp; of a binary signal]]
  
$\text{Example 2:}$&nbsp;
For a binary signal&nbsp; $x(t)$&nbsp; with the amplitude values 
*$1\hspace{0.03cm}\rm V$&nbsp; $($for symbol&nbsp; $\rm L)$, 
*$3\hspace{0.03cm}\rm V$&nbsp; $($for symbol&nbsp; $\rm H)$ 


and the probabilities of occurrence&nbsp; $p_{\rm L} = 0.2$&nbsp; resp.&nbsp; $p_{\rm H} = 0.8$&nbsp; results for the second moment:  
:$$m_2 = 0.2 \cdot (1\,{\rm V})^2+ 0.8 \cdot (3\,{\rm V})^2 = 7.4 \hspace{0.1cm}{\rm V}^2,$$

#The rms value&nbsp; $x_{\rm eff}=\sqrt{m_2}=2.72\,{\rm V}$&nbsp; is independent of the reference resistance&nbsp; $R$&nbsp; unlike the total power. 
#For the latter, with&nbsp; $R=1 \hspace{0.1cm} Ω$&nbsp; the value&nbsp; $P=m_2/R=7.4 \hspace{0.1cm}{\rm W}$,&nbsp; with&nbsp; $R=50 \hspace{0.1cm} Ω$&nbsp; on the other hand, only&nbsp; $P=0.148 \hspace{0.1cm}{\rm W}$. 
 
   
With the DC component&nbsp; $m_1 = 2.6 \hspace{0.05cm}\rm V$&nbsp; $($see&nbsp; [[Theory_of_Stochastic_Signals/Moments_of_a_Discrete_Random_Variable#First_order_moment_.E2.80.93_linear_mean_.E2.80.93_DC_component|$\text{Example 1})$]]&nbsp; it follows for 
*the variance&nbsp; $ σ^2 = 7.4 \hspace{0.05cm}{\rm V}^2 - \big [2.6 \hspace{0.05cm}\rm V\big ]^2 = 0.64\hspace{0.05cm} {\rm V}^2$,

*the standard deviation&nbsp; $σ = 0.8 \hspace{0.05cm} \rm V$. 


The same variance&nbsp; $ σ^2 = 0.64\hspace{0.05cm} {\rm V}^2$ and the same standard deviation&nbsp; $σ = 0.8 \hspace{0.05cm} \rm V$&nbsp; result for the amplitudes&nbsp; $0\hspace{0.05cm}\rm V$&nbsp; $($for symbol&nbsp; $\rm L)$&nbsp; and $2\hspace{0.05cm}\rm V$&nbsp; $($for  symbol&nbsp; $\rm H)$,&nbsp; provided the occurrence probabilities&nbsp; $p_{\rm L} = 0.2$&nbsp; and&nbsp; $p_{\rm H} = 0.8$&nbsp; remain the same.&nbsp; Only the DC component and the total power change: 
:$$m_1 = 1.6 \hspace{0.05cm}{\rm V}, $$
:$$P = {m_1}^2 +\sigma^2 = 3.2 \hspace{0.05cm}{\rm V}^2.$$}}


==Exercises for the chapter==
<br>
[[Aufgaben:Exercise_2.2:_Multi-Level_Signals|Exercise 2.2: Multi-Level Signals]]

[[Aufgaben:Exercise_2.2Z:_Discrete_Random_Variables|Exercise 2.2Z: Discrete Random Variables]]


{{Display}}