View source for Theory of Stochastic Signals/Probability Density Function

 
 
{{Header
|Untermenü=Continuous Random Variables
|Vorherige Seite=Generation of Discrete Random Variables
|Nächste Seite=Cumulative Distribution Function 
}}

== # OVERVIEW OF THE THIRD MAIN CHAPTER # ==
<br>
We consider here&nbsp; &raquo;'''continuous random variables'''&laquo;,&nbsp; i.e.,&nbsp; random variables which can assume infinitely many different values,&nbsp; at least in certain ranges of real numbers.&nbsp; 
*Their applications in information and communication technology are manifold.
 
*They are used,&nbsp; among other things,&nbsp; for the simulation of noise signals and for the description of fading effects.


We restrict ourselves at first to the statistical description of the&nbsp; &raquo;'''amplitude distribution'''&laquo;.&nbsp; In detail,&nbsp; the following are treated:

*The relationship between&nbsp; &raquo;probability density function&laquo;&nbsp; $\rm (PDF)$&nbsp; and&nbsp; &raquo;cumulative distribution function&laquo;&nbsp; $\rm (CDF)$;
*the calculation of&nbsp; &raquo;expected values&nbsp; and&nbsp; moments&laquo;;
*some&nbsp; &raquo;special cases&laquo;&nbsp; of continuous-value distributions: 
#uniform distributed random variables,&nbsp; 
#Gaussian distributed random variables,&nbsp; 
#exponential distributed random variables,&nbsp; 
#Laplace distributed random variables,&nbsp; 
#Rayleigh distributed random variables,&nbsp; 
#Rice distributed random variables,&nbsp; 
#Cauchy distributed random variables;
*the&nbsp; &raquo;generation of continuous random variables&laquo;&nbsp; on a computer.


&raquo;'''Inner statistical dependencies'''&laquo;&nbsp; of the underlying processes&nbsp; '''are not considered here'''.&nbsp; For this,&nbsp; we refer to the following main chapters&nbsp; $4$&nbsp; and&nbsp; $5$. 


==Properties of continuous random variables==
<br>
In the second chapter it was shown that the amplitude distribution of a discrete random variable is completely determined by its&nbsp; $M$&nbsp; occurrence probabilities,&nbsp; where the level number&nbsp; $M$&nbsp; usually has a finite value. 

{{BlaueBox|TEXT=  
$\text{Definition:}$&nbsp; By a&nbsp; &raquo;'''value-continuous random variable'''&laquo;&nbsp; is meant a random variable whose possible numerical values are uncountable &nbsp; &rArr; &nbsp; $M \to \infty$.&nbsp; In the following,&nbsp; we will often use the short form&nbsp; &raquo;continuous random variable&laquo;.}}


Further it shall hold: 
#We&nbsp; (mostly)&nbsp; denote value-continuous random variables with&nbsp; $x$&nbsp; in contrast to the value-discrete random variables,&nbsp; which are denoted with&nbsp; $z$&nbsp; as before. 
#No statement is made here about a possible time discretization,&nbsp; i.e.,&nbsp; value-continuous random variables can be discrete in time. 
#We assume for this chapter that there are no statistical bindings between the individual samples&nbsp; $x_ν$,&nbsp; or at least leave them out of consideration. 


{{GraueBox|TEXT=
$\text{Example 1:}$
The graphic shows a section of a stochastic noise signal&nbsp; $x(t)$&nbsp; whose instantaneous value can be taken as a continuous random variable&nbsp; $x$. 
[[File: P_ID41__Sto_T_3_1_S1_neu.png |right|frame| Signal and PDF of a Gaussian noise signal]]

*From the&nbsp; &raquo;probability density function&raquo; &nbsp; $\rm (PDF)$&nbsp; shown on the right,&nbsp; it can be seen that instantaneous values around the mean&nbsp; $m_1$&nbsp; occur most frequently for any exemplary signal. 


*Since there are no statistical dependencies between the samples $x_ν$,&nbsp; such a signal is often also referred to as&nbsp; &raquo;'''white noise'''&laquo;.}}


==Definition of the probability density function==
<br>
For a value-continuous random variable,&nbsp; the probabilities that it takes on quite specific values are zero.&nbsp; Therefore,&nbsp; to describe a value-continuous random variable,&nbsp; we must always refer to the&nbsp; &raquo;probability density function&laquo;&nbsp; $\rm (PDF)$. 

{{BlaueBox|TEXT=
$\text{Definition:}$ &nbsp; 
The value of the&nbsp; &raquo;'''probability density function'''&laquo;&nbsp; $f_{x}(x)$&nbsp; at location&nbsp; $x_\mu$&nbsp; is equal to the probability that the instantaneous value of the random variable&nbsp; $x$&nbsp; lies in an&nbsp; $($infinitesimally small$)$&nbsp; interval of width&nbsp; $Δx$&nbsp; around&nbsp; $x_\mu$,&nbsp; divided by&nbsp; $Δx$:
:$$f_x(x=x_\mu) = \lim_{\rm \Delta \it x \hspace{0.05cm}\to \hspace{0.05cm}\rm 0}\frac{\rm Pr \{\it x_\mu-\rm \Delta \it x/\rm 2 \le \it x \le x_\mu \rm +\rm \Delta \it x/\rm 2\} }{\rm \Delta \it  x}.$$}}


This extremely important descriptive variable has the following properties:

*Although from the time course in&nbsp; [[Theory_of_Stochastic_Signals/Probability_Density_Function_(PDF)#Properties_of_continuous_random_variables|$\text{Example 1}$]]&nbsp; it can be seen&nbsp; that the most frequent signal components lie at&nbsp; $x = m_1$&nbsp; and the PDF has its largest value here,&nbsp; for a value-continuous random variable the probability&nbsp; ${\rm Pr}(x = m_1)$,&nbsp; that the instantaneous value is exactly equal to the mean&nbsp; $m_1$,&nbsp; is identically zero.

*The probability that the random variable lies in the range between&nbsp; $x_{\rm u}$&nbsp; and&nbsp; $x_{\rm o}$:
:$${\rm Pr}(x_{\rm u} \le  x \le x_{\rm o})= \int_{x_{\rm u} }^{x_{\rm o} }f_{x}(x) \,{\rm d}x.$$

*As an important normalization property,&nbsp; this yields for the area under the PDF with the boundary transitions&nbsp; $x_{\rm u} → \hspace{0.05cm} - \hspace{0.05cm} ∞$&nbsp; and&nbsp; $x_{\rm o} → +∞:$
:$$\int_{-\infty}^{+\infty} f_{x}(x) \,{\rm d}x = \rm 1.$$

*The corresponding equation for value-discrete,&nbsp; $M$-level random variables states that the sum over the&nbsp; $M$&nbsp; occurrence probabilities gives the value&nbsp; $1$. 


{{BlaueBox|TEXT=
$\text{Note on nomenclature:}$&nbsp; 

In the literature,&nbsp; a distinction is often made between the random variable&nbsp; $X$&nbsp; and its realizations&nbsp; $x ∈ X$.&nbsp; Thus,&nbsp; the above definition equation is 
:$$f_{X}(X=x) = \lim_{ {\rm \Delta} x \hspace{0.05cm}\to \hspace{0.05cm} 0}\frac{ {\rm Pr} \{ x - {\rm \Delta} x/2 \le X \le x +{\rm \Delta} x/ 2\} }{ {\rm \Delta} x}.$$

We have largely dispensed with this more precise nomenclature in our&nbsp; $\rm LNTwww$&nbsp; so as not to use up two letters for one quantity. 
#Lowercase letters&nbsp; $($as&nbsp; $x)$&nbsp; often denote signals and uppercase letters&nbsp; $($as&nbsp; $X)$&nbsp; the associated spectra in our case. 
#Nevertheless,&nbsp; today (2017)&nbsp; we have to honestly admit that the 2001 decision was not entirely happy.}}

==PDF definition for discrete random variables==
For reasons of a uniform representation of all random variables&nbsp; (both discrete-value and continuous-value),&nbsp; it is convenient to define the probability density function also for discrete random variables. 

{{BlaueBox|TEXT=
$\text{Definition:}$ &nbsp;
Applying the definition equation of the last section to discrete random variables,&nbsp; the PDF takes infinitely large values at some points&nbsp; $x_\mu$&nbsp; due to the nonvanishingly small probability value and the limit transition&nbsp; $Δx → 0$. 

Thus,&nbsp; the PDF results in a sum of&nbsp; [[Signal_Representation/Direct_Current_Signal_-_Limit_Case_of_a_Periodic_Signal#Dirac_.28delta.29_function_in_frequency_domain|$\text{Dirac delta functions}$]] &nbsp; &rArr; &nbsp; "distributions": 
:$$f_{x}(x)=\sum_{\mu=1}^{M}p_\mu\cdot {\rm \delta}( x-x_\mu).$$

The weights of these Dirac delta functions are equal to the probabilities&nbsp; $p_\mu = {\rm Pr}(x = x_\mu$).}} 


Here is another note to help classify the different descriptive quantities for discrete and continuous random variables: &nbsp; Probability and probability density function are related in a similar way as in the book&nbsp; [[Signal_Representation|"Signal Representation"]]
*a discrete spectral component of a harmonic oscillation ⇒ line spectrum,&nbsp; and 
*a continuous spectrum of an energy-limited&nbsp; (pulse-shaped)&nbsp; signal.


[[File:P_ID40__Sto_T_3_1_S3_NEU.png|right|frame|Signal and PDF of a ternary signal]]
{{GraueBox|TEXT=
$\text{Example 2:}$&nbsp; Below is a section 
*of a rectangular signal with three possible values, 
*where the signal value&nbsp; $0 \ \rm V$&nbsp; occurs twice as often as the outer signal values&nbsp; $(\pm 1 \ \rm V)$. 



Thus,&nbsp; the corresponding&nbsp; PDF&nbsp; (values from top to bottom)&nbsp; is: 
:$$f_{x}(x) = 0.25 \cdot \delta(x - {\rm 1 V})+ 0.5\cdot \delta(x) + 0.25\cdot \delta (x + 1\rm V).$$}}


For a more in-depth look at the topic covered here,&nbsp; we recommend the following&nbsp; (German language)&nbsp; learning video:

:[[Wahrscheinlichkeit_und_WDF_(Lernvideo)|"Wahrscheinlichkeit und WDF"]] &nbsp; &rArr; &nbsp; "Probability and probability density function" 

==Numerical determination of the PDF==
<br>
You can see here a scheme for the numerical determination of the probability density function:&nbsp; 

Assuming that the random variable&nbsp; $x$&nbsp; at hand has negligible values outside the range from&nbsp; $x_{\rm min} = -4.02$&nbsp; to&nbsp; $x_{\rm max} = +4.02$,&nbsp; proceed as follows: 
[[File:EN_Sto_T_3_1_S4_neu.png |right|frame| For numerical determination of the PDF]]
#Divide the range of&nbsp; $x$-values into&nbsp; $I$&nbsp; intervals of equal width&nbsp; $Δx$&nbsp; and define a field&nbsp; $\text{PDF}[0 : I-1]$.&nbsp; In the sketch&nbsp; $I = 201$&nbsp; and accordingly&nbsp; $Δx = 0.04$&nbsp; is chosen. 
#The random variable&nbsp; $x$&nbsp; is now called&nbsp; $N$&nbsp; times in succession,&nbsp; each time checking to which interval&nbsp; $i_{\rm act}$&nbsp; the current random variable&nbsp; $x_{\rm act}$ belongs: <br> &nbsp; &nbsp; $i_{\rm act} = ({\rm int})((x + x_{\rm max})/Δx).$ 
#The corresponding field element PDF( $i_{\rm act}$) is then incremented by&nbsp; $1$.&nbsp; 
#After $N$ iterations, $\text{PDF}[i_{\rm act}]$ then contains the number of random numbers belonging to the interval $i_{\rm act}$. 
#The actual PDF values are obtained if,&nbsp; at the end,&nbsp; all field elements&nbsp; $\text{PDF}[i]$&nbsp; with&nbsp; $0 ≤ i ≤ I-1$&nbsp; are still divided by&nbsp; $N \cdot Δx$. 
<br clear=all>
{{GraueBox|TEXT=
$\text{Example 3:}$&nbsp; 
From the drawn green arrows in the graph above,&nbsp;  one can see: 
*The value&nbsp; $x_{\rm act} = 0.07$&nbsp; leads to the result&nbsp; $i_{\rm act} =$ (int) ((0.07 + 4.02)/0.04) = (int) $102.25$.
* Here&nbsp; "(int)"&nbsp; means an integer conversion after float division &nbsp; ⇒ &nbsp; $i_{\rm act} = 102$. 
*The same interval&nbsp; $i_{\rm act} = 102$&nbsp; results for&nbsp; $0.06 < x_{\rm act} < 0.10$,&nbsp; so for example also for&nbsp; $x_{\rm act} = 0.09$. }}

==Exercises for the chapter==
<br>
[[Aufgaben:Exercise_3.1:_cos²-PDF_and_PDF_with_Dirac_Functions|Exercise 3.1: Cosine-square PDF and PDF with Dirac Functions]]

[[Aufgaben:Exercise_3.1Z:_Triangular_PDF|Exercise 3.1Z: Triangular PDF]]


{{Display}}