Difference between revisions of "Theory of Stochastic Signals/Exponentially Distributed Random Variables"

From LNTwww
 
(20 intermediate revisions by 4 users not shown)
Line 9: Line 9:
 
{{BlaueBox|TEXT=   
 
{{BlaueBox|TEXT=   
 
$\text{Definition:}$   
 
$\text{Definition:}$   
A continuous random variable  $x$  is called (one-sided)  '''exponentially distributed''' if it can take only non–negative values and the PDF for  $x>0$  has the following shape:  
+
A continuous random variable  $x$  is called  (one-sided)  »'''exponentially distributed'''«  if it can take only non–negative values and the probability density function has the following shape  for  $x>0$:  
 
:$$f_x(x)=\it \lambda\cdot\rm e^{\it -\lambda \hspace{0.05cm}\cdot \hspace{0.03cm} x}.$$}}
 
:$$f_x(x)=\it \lambda\cdot\rm e^{\it -\lambda \hspace{0.05cm}\cdot \hspace{0.03cm} x}.$$}}
  
Line 15: Line 15:
 
[[File: P_ID72__Sto_T_3_6_S1_neu.png |right|frame| PDF and CDF of an exponentially distributed random variable]]
 
[[File: P_ID72__Sto_T_3_6_S1_neu.png |right|frame| PDF and CDF of an exponentially distributed random variable]]
  
The left image shows the  ''probability density function''  (PDF) of such an exponentially distributed random variable  $x$.  Highlight:   
+
The left sketch shows the  "probability density function"  $\rm (PDF)$  of such an exponentially distributed random variable  $x$.  To be emphasized:   
*The larger the distribution parameter  $λ$  is, the steeper the decay occurs.
+
#The larger the distribution parameter  $λ$,    the steeper the decay.
*By definition  $f_{x}(0) = λ/2$, i.e. the mean of left-hand limit  $(0)$  and right-hand limit  $(\lambda)$.
+
#By definition  $f_{x}(0) = λ/2$,  i.e. the mean of left-hand limit  $(0)$  and right-hand limit  $(\lambda)$.
  
 
+
*For the  "cumulative distribution function"  $\rm (CDF)$,  we obtain for  $r > 0$  by integration over the PDF   (right graph):  
For the  ''cumulative distribution function''  (right graph), we obtain for  $r > 0$  by integration over the PDF:  
 
 
:$$F_{x}(r)=1-\rm e^{\it -\lambda\hspace{0.05cm}\cdot \hspace{0.03cm} r}.$$
 
:$$F_{x}(r)=1-\rm e^{\it -\lambda\hspace{0.05cm}\cdot \hspace{0.03cm} r}.$$
  
The  ''moments''  of the one-sided exponential distribution are generally equal to  
+
*The  "moments"  of the one-sided exponential distribution are generally equal to  
 
:$$m_k = k!/λ^k.$$
 
:$$m_k = k!/λ^k.$$
From this and from Steiner's theorem, we get for the mean and the dispersion:  
+
*From this and from Steiner's theorem, we get for the "mean" and the "standard deviation":  
 
:$$m_1={1}/{\lambda},$$
 
:$$m_1={1}/{\lambda},$$
 
:$$\sigma=\sqrt{m_2-m_1^2}=\sqrt{\frac{2}{\lambda^2}-\frac{1}{\lambda^2}}={1}/{\lambda}.$$
 
:$$\sigma=\sqrt{m_2-m_1^2}=\sqrt{\frac{2}{\lambda^2}-\frac{1}{\lambda^2}}={1}/{\lambda}.$$
  
 
{{GraueBox|TEXT=   
 
{{GraueBox|TEXT=   
$\text{Example 1:}$  The exponential distribution has great importance for reliability studies, and the term "lifetime distribution" is also commonly used in this context.  
+
$\text{Example 1:}$  The exponential distribution has great importance for reliability studies,  and the term  "lifetime distribution"  is also commonly used in this context.  
*In these applications, the random variable is often the time  $t$ that elapses before a component fails.  
+
#In these applications,  the random variable is often the time  $t$  that elapses before a component fails.  
*Furthermore, it should be noted that the exponential distribution is closely related to the  [[Theory_of_Stochastic_Signals/Poisson_Distribution|Poisson distribution]] . }}
+
#Furthermore,  it should be noted that the exponential distribution is closely related to the  [[Theory_of_Stochastic_Signals/Poisson_Distribution|$\text{Poisson distribution}$]]. }}
  
 
==Transformation of random variables==
 
==Transformation of random variables==
 
<br>
 
<br>
To generate such an exponentially distributed random variable on a digital computer, for example, a&nbsp; '''nonlinear transformation'''&nbsp; The underlying principle is first stated here in general terms.  
+
To generate such an exponentially distributed random variable on a digital computer,&nbsp; you can use e.g. a&nbsp; &raquo;'''nonlinear transformation'''&laquo;.&nbsp; The underlying principle is first stated here in general terms.  
  
 
{{BlaueBox|TEXT=   
 
{{BlaueBox|TEXT=   
$\text{Procedure:}$&nbsp; If a continuous random variable&nbsp; $u$&nbsp; possesses the PDF&nbsp; $f_{u}(u)$, then the probability density function of the random variable transformed at the nonlinear characteristic&nbsp; $x = g(u)$&nbsp; $x$ holds:  
+
$\text{Procedure:}$&nbsp; If a continuous-valued random variable&nbsp; $u$&nbsp; possesses the PDF&nbsp; $f_{u}(u)$,&nbsp; then the probability density function of the random variable transformed at the nonlinear characteristic&nbsp; $x = g(u)$&nbsp; holds:  
 
:$$f_{x}(x)=\frac{f_u(u)}{\mid g\hspace{0.05cm}'(u)\mid}\Bigg \vert_{\hspace{0.1cm} u=h(x)}.$$
 
:$$f_{x}(x)=\frac{f_u(u)}{\mid g\hspace{0.05cm}'(u)\mid}\Bigg \vert_{\hspace{0.1cm} u=h(x)}.$$
  
Here&nbsp; $g\hspace{0.05cm}'(u)$&nbsp; denotes the derivative of the characteristic curve&nbsp; $g(u)$&nbsp; and&nbsp; $h(x)$&nbsp; gives the inverse function to&nbsp; $g(u)$&nbsp; . }}
+
Here,&nbsp; $g\hspace{0.05cm}'(u)$&nbsp; denotes the derivative of the characteristic curve&nbsp; $g(u)$&nbsp; and&nbsp; $h(x)$&nbsp; gives the inverse function to&nbsp; $g(u)$&nbsp; . }}
  
  
*The above equation is valid, however, only under the condition that the derivative&nbsp; $g\hspace{0.03cm}'(u) \ne 0$&nbsp; .  
+
*However,&nbsp; the above equation is only valid under the condition that the derivative&nbsp; $g\hspace{0.03cm}'(u) \ne 0$.  
*For a characteristic with horizontal sections&nbsp; $(g\hspace{0.05cm}'(u) = 0)$&nbsp; additional Dirac functions appear in the PDF if the input quantity has components in the range.  
+
*For a characteristic with horizontal sections&nbsp; $(g\hspace{0.05cm}'(u) = 0)$:&nbsp; Additional Dirac delta functions appear in the PDF if the input variable has components in these ranges.  
*The weights of these Dirac functions are equal to the probabilities that the input quantity lies in these domains.  
+
*The weights of these Dirac delta functions are equal to the probabilities that the input variable lies in these ranges.  
  
 
   
 
   
[[File:P_ID76__Sto_T_3_6_S2_neu.png |frame| To transform random variables | right]]
+
{{GraueBox|TEXT=
{{GraueBox|TEXT=  
+
[[File:P_ID76__Sto_T_3_6_S2_neu.png |frame| To transform random variables | right]]   
 
$\text{Example 2:}$&nbsp;  
 
$\text{Example 2:}$&nbsp;  
Given a random variable distributed between&nbsp; $-2$&nbsp; and&nbsp; $+2$&nbsp; triangularly&nbsp; $u$&nbsp; on a nonlinearity with characteristic&nbsp; $x = g(u)$,  
+
Given a random variable&nbsp; $u$&nbsp; triangularly distributed between&nbsp; $-2$&nbsp; and&nbsp; $+2$&nbsp; on a nonlinearity with characteristic&nbsp; $x = g(u)$,  
*which, in the range&nbsp; $\vert u \vert ≤ 1$&nbsp; triples the input values,&nbsp; and  
+
*which,&nbsp; in the range&nbsp; $\vert u \vert ≤ 1$&nbsp; triples the input values,&nbsp; and  
*mapping all values&nbsp; $\vert u \vert > 1$&nbsp; to&nbsp; $x = \pm 3$&nbsp; depending on the sign,  
+
*mapping all values&nbsp; $\vert u \vert > 1$&nbsp; to &nbsp; $x = \pm 3$ &nbsp; depending on the sign,  
  
  
then the PDF $f_{x}(x)$ sketched on the right is obtained.  
+
then the PDF&nbsp; $f_{x}(x)$&nbsp; sketched on the right is obtained.  
  
  
 
Please note:  
 
Please note:  
  
'''(1)''' &nbsp; Due to the amplification by a factor of&nbsp; $3$&nbsp; $f_{x}(x)$&nbsp; is wider and lower than $f_{u}(u) by this factor.$
+
# Due to the amplification by a factor of&nbsp; $3$ &nbsp; &rArr; &nbsp; $f_{x}(x)$&nbsp; is wider and lower than $f_{u}(u)$ by this factor.  
 
+
# The two horizontal limits of the characteristic at &nbsp; $u = ±1$ &nbsp; lead to two Dirac delta functions at&nbsp; $x = ±3$,&nbsp; each with weight&nbsp; $1/8$.
'''(2)''' &nbsp; The two horizontal limits of the characteristic at&nbsp; $u = ±1$&nbsp; lead to the two Dirac functions at&nbsp; $x = ±3$, each with weight&nbsp; $1/8$.
+
# The weight&nbsp; $1/8$&nbsp; corresponds to the green areas in the PDF&nbsp; $f_{u}(u).$}}  
 
 
'''(3)''' &nbsp; The weight&nbsp; $1/8$&nbsp; corresponds to the green areas in the PDF $f_{u}(u).$}}  
 
  
 
==Generation of an exponentially distributed random variable==
 
==Generation of an exponentially distributed random variable==
Line 73: Line 70:
 
{{BlaueBox|TEXT=  
 
{{BlaueBox|TEXT=  
 
$\text{Procedure:}$&nbsp;  
 
$\text{Procedure:}$&nbsp;  
Now we assume that the random variable to be transformed&nbsp; $u$&nbsp; is uniformly distributed between&nbsp; $0$&nbsp; (inclusive) and&nbsp; $1$&nbsp; (exclusive).&nbsp; Moreover, we consider the monotonically increasing characteristic curve  
+
Now we assume that the random variable&nbsp; $u$&nbsp; to be transformed is uniformly distributed between&nbsp; $0$&nbsp; (inclusive) and&nbsp; $1$&nbsp; (exclusive).&nbsp;  
 +
 
 +
*Moreover,&nbsp; we consider the monotonically increasing characteristic curve  
 
:$$x=g_1(u) =\frac{1}{\lambda}\cdot \rm ln \ (\frac{1}{1-\it u}).$$
 
:$$x=g_1(u) =\frac{1}{\lambda}\cdot \rm ln \ (\frac{1}{1-\it u}).$$
  
It can be shown that by this characteristic&nbsp; $x=g_1(u)$&nbsp; a one-sided exponentially distributed random variable&nbsp; $x$&nbsp; with the following PDF arises&nbsp; <br>(derivation see [[Theory_of_Stochastic_Signals/Exponentially_Distributed_Random_Variables/Derivation_of_the_corresponding_transformation_characteristic|next page]]):  
+
*It can be shown that by this characteristic&nbsp; $x=g_1(u)$&nbsp; a one-sided exponentially distributed random variable&nbsp; $x$&nbsp; with the following PDF arises&nbsp; <br>(derivation see [[Theory_of_Stochastic_Signals/Exponentially_Distributed_Random_Variables#Derivation_of_the_corresponding_transformation_characteristic|"next section"]]):  
 
:$$f_{x}(x)=\lambda\cdot\rm e^{\it -\lambda \hspace{0.05cm}\cdot \hspace{0.03cm} x}\hspace{0.2cm}{\rm for}\hspace{0.2cm} {\it x}>0.$$
 
:$$f_{x}(x)=\lambda\cdot\rm e^{\it -\lambda \hspace{0.05cm}\cdot \hspace{0.03cm} x}\hspace{0.2cm}{\rm for}\hspace{0.2cm} {\it x}>0.$$
*For&nbsp; $x = 0$&nbsp; the PDF value is half&nbsp; $(\lambda/2)$.
+
*Note:
* Negative&nbsp; $x$ values do not occur because for&nbsp; $0 ≤ u < 1$&nbsp; the argument of the (natural) logarithm function does not become smaller than&nbsp; $1$.}}
+
#For&nbsp; $x = 0$&nbsp; the PDF value is half&nbsp; $(\lambda/2)$.
 +
# Negative&nbsp; $x$ values do not occur because for&nbsp; $0 ≤ u < 1$&nbsp; the argument of the (natural) logarithm function does not become smaller than&nbsp; $1$.}}
 
   
 
   
  
By the way, the same PDF is obtained with the monotonically decreasing characteristic curve  
+
By the way,&nbsp; the same PDF is obtained with the monotonically decreasing characteristic curve  
 
:$$x=g_2(u)=\frac{1}{\lambda}\cdot \rm ln \ (\frac{1}{\it u})=-\frac{1}{\lambda}\cdot \rm ln(\it u \rm ).$$
 
:$$x=g_2(u)=\frac{1}{\lambda}\cdot \rm ln \ (\frac{1}{\it u})=-\frac{1}{\lambda}\cdot \rm ln(\it u \rm ).$$
  
 
Please note:
 
Please note:
*When using a computer implementation corresponding to the first transformation characteristic&nbsp; $x=g_1(u)$&nbsp; the value&nbsp; $u = 1$&nbsp; must be excluded.
+
*When using a computer implementation corresponding to the first transformation characteristic&nbsp; $x=g_1(u)$ &nbsp; &rArr; &nbsp; the value&nbsp; $u = 1$&nbsp; must be excluded.
*If one uses the second transformation characteristic&nbsp; $x=g_2(u)$, on the other hand, the value&nbsp; $u =0$&nbsp; must be excluded.  
+
*On the other hand,&nbsp; if one uses the second transformation characteristic&nbsp; $x=g_2(u)$ &nbsp; &rArr; &nbsp;  the value&nbsp; $u =0$&nbsp; must be excluded.  
  
  
The (German) learning video&nbsp; [[Erzeugung_einer_Exponentialverteilung_(Lernvideo)|Erzeugung einer Exponentialverteilung]]&nbsp; $\Rightarrow$ Generation of an exponential distribution, shall clarify the transformations derived here.
+
The following&nbsp; (German language)&nbsp; learning video shall clarify the transformations derived here: <br> &nbsp; &nbsp; &nbsp;[[Erzeugung_einer_Exponentialverteilung_(Lernvideo)|"Erzeugung einer Exponentialverteilung"]] &nbsp; $\Rightarrow$ &nbsp; "Generation of an exponential distribution".
  
 
==Derivation of the corresponding transformation characteristic==
 
==Derivation of the corresponding transformation characteristic==
 
<br>
 
<br>
 
{{BlaueBox|TEXT=
 
{{BlaueBox|TEXT=
$\text{Exercise:}$&nbsp;  
+
$\text{Task:}$&nbsp;
Now derive the transformation characteristic&nbsp; $x = g_1(u)= g(u)$&nbsp; already used on the last page, which is derived from a random variable&nbsp; equally distributed between&nbsp; $0$&nbsp; and&nbsp; $1$&nbsp; ; $u$&nbsp; with the probability density function (PDF)&nbsp; $f_{u}(u)$&nbsp; forms a one-sided exponentially distributed random variable&nbsp; $x$&nbsp; with the PDF&nbsp; $f_{x}(x)$&nbsp; :
+
#&nbsp; Now the transformation characteristic&nbsp; $x = g_1(u)= g(u)$&nbsp; already used in the last section is derived.
 
+
#&nbsp; This forms from the uniformly distributed random variable&nbsp; $u$&nbsp; with PDF&nbsp; $f_{u}(u)$&nbsp; a one-sided exponentially distributed random variable&nbsp; $x$&nbsp; with PDF&nbsp; $f_{x}(x)$:
:$$f_{u}(u)= \left\{ \begin{array}{*{2}{c} } 1 & \rm if\hspace{0.3cm} 0 < {\it u} < 1,\\ 0.5 & \rm if\hspace{0.3cm} {\it u} = 0, {\it u} = 1,\ 0 & \rm else, \end{array}    \right. \hspace{0.5cm}\rightarrow \hspace{0.5cm}
+
  f_{x}(x)= \left\{ \begin{array}{*{2}{c} }        \lambda\cdot\rm e^{\it -\lambda\hspace{0.03cm} \cdot \hspace{0.03cm} x} & \rm if\hspace{0.3cm} {\it x} > 0,\ \lambda/2 & \rm if\hspace{0.3cm} {\it x} = 0 ,\ 0 & \rm if\hspace{0.3cm} {\it x} < 0. \ \end{array}    \right.$$}}
+
::$$f_{u}(u)= \left\{         \begin{array}{*{2}{c} }         1 & \rm if\hspace{0.3cm} 0 < {\it u} < 1,\\         0.5 & \rm if\hspace{0.3cm}   {\it u} = 0, {\it u} = 1,\\          0 & \rm else, \\              \end{array}    \right. \hspace{0.5cm}\Rightarrow \hspace{0.5cm}
 +
  f_{x}(x)= \left\{         \begin{array}{*{2}{c} }        \lambda\cdot\rm e^{\it -\lambda\hspace{0.03cm} \cdot \hspace{0.03cm} x} & \rm if\hspace{0.3cm}   {\it x} > 0,\\        \lambda/2 & \rm if\hspace{0.3cm} {\it x} = 0 ,\\          0 & \rm else\hspace{0.3cm} {\it x} < 0. \\              \end{array}    \right.$$}}
  
  
Line 105: Line 106:
 
$\text{Solution:}$&nbsp;  
 
$\text{Solution:}$&nbsp;  
  
'''(1)'''&nbsp; Starting from the general transformation equation.
+
'''(1)'''&nbsp; Starting from the general transformation equation  
:$$f_{x}(x)=\frac{f_{u}(u)}{\mid g\hspace{0.05cm}'(u) \mid }\Bigg \vert _{\hspace{0.1cm} u=h(x)}$$
+
::$$f_{x}(x)=\frac{f_{u}(u)}{\mid g\hspace{0.05cm}'(u) \mid }\Bigg \vert _{\hspace{0.1cm} u=h(x)}$$
is obtained by converting and substituting the given PDF $f_{ x}(x):$
+
:is obtained by converting and substituting the given PDF&nbsp; $f_{ x}(x):$
:$$\mid g\hspace{0.05cm}'(u)\mid\hspace{0.1cm}=\frac{f_{u}(u)}{f_{x}(x)}\Bigg \vert _{\hspace{0.1cm} x=g(u)}= {1}/{\lambda} \cdot {\rm e}^{\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}g(u)}.$$  
+
::$$\mid g\hspace{0.05cm}'(u)\mid\hspace{0.1cm}=\frac{f_{u}(u)}{f_{x}(x)}\Bigg \vert _{\hspace{0.1cm} x=g(u)}= {1}/{\lambda} \cdot {\rm e}^{\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}g(u)}.$$  
Here&nbsp; $x = g\hspace{0.05cm}'(u)$&nbsp; gives the derivative of the characteristic curve, which we assume to be monotonically increasing.  
+
:Here&nbsp; $x = g\hspace{0.05cm}'(u)$&nbsp; gives the derivative of the characteristic curve,&nbsp; which we assume to be monotonically increasing.  
  
'''(2)'''&nbsp; With this assumption we get &nbsp;$\vert g\hspace{0.05cm}'(u)\vert = g\hspace{0.05cm}'(u) = {\rm d}x/{\rm d}u$&nbsp; and the differential equation &nbsp;${\rm d}u =  \lambda\  \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}  x}\, {\rm d}x$&nbsp; with solution &nbsp;$u = K - {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}.$
+
'''(2)'''&nbsp; With this assumption we get &nbsp; $\vert g\hspace{0.05cm}'(u)\vert = g\hspace{0.05cm}'(u) = {\rm d}x/{\rm d}u$ &nbsp; and the differential equation &nbsp; ${\rm d}u =  \lambda\  \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}  x}\, {\rm d}x$ &nbsp; with solution &nbsp; $u = K - {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}.$
  
'''(3)'''&nbsp; From the condition that the input quantity &nbsp;$u =0$&nbsp; should lead to the output value &nbsp;$x =0$&nbsp;, we obtain for the constant&nbsp; $K =1$&nbsp; and thus &nbsp;$u = 1- {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}.$
+
'''(3)'''&nbsp; From the condition that the input variable &nbsp;$u =0$&nbsp; should lead to the output value &nbsp;$x =0$,&nbsp; we obtain for the constant&nbsp; $K =1$&nbsp; and thus &nbsp; $u = 1- {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}.$
  
 
'''(4)'''&nbsp; Solving this equation for&nbsp; $x$&nbsp; yields the equation given in front:  
 
'''(4)'''&nbsp; Solving this equation for&nbsp; $x$&nbsp; yields the equation given in front:  
:$$x = g_1(u)= \frac{1}{\lambda} \cdot {\rm ln} \left(\frac{1}{1 - u} \right) .$$
+
::$$x = g_1(u)= \frac{1}{\lambda} \cdot {\rm ln} \left(\frac{1}{1 - u} \right) .$$
  
*In a computer implementation, however, ensure that the critical value&nbsp; $1$&nbsp; is excluded for the equally distributed input variable&nbsp; $u$&nbsp; &nbsp;  
+
*In a computer implementation,&nbsp; however,&nbsp; ensure that the critical value&nbsp; $1$&nbsp; is excluded for the uniformly distributed input variable&nbsp; $u$.&nbsp;
*This, however, has (almost) no effect on the final result. }}
+
*This,&nbsp; however,&nbsp; has (almost) no effect on the final result. }}
  
  
 
==Two-sided exponential distribution - Laplace distribution==
 
==Two-sided exponential distribution - Laplace distribution==
 
<br>
 
<br>
Closely related to the exponential distribution is the so-called&nbsp; [https://en.wikipedia.org/wiki/Laplace_distribution Laplace distrubtion]&nbsp; with the probability density function  
+
Closely related to the exponential distribution is the&nbsp; [https://en.wikipedia.org/wiki/Laplace_distribution $\text{Laplace distribution}$]&nbsp; with the probability density function  
 
:$$f_{x}(x)=\frac{\lambda}{2}\cdot\rm e^{\it -\lambda \hspace{0.05cm} \cdot \hspace{0.05cm} | x|}.$$
 
:$$f_{x}(x)=\frac{\lambda}{2}\cdot\rm e^{\it -\lambda \hspace{0.05cm} \cdot \hspace{0.05cm} | x|}.$$
  
The Laplace distribution is a&nbsp; ''two-sided exponential distribution'' that approximates sufficiently well, in particular, the amplitude distribution of speech&ndash; and music signals.  
+
The Laplace distribution is a&nbsp; "two-sided exponential distribution"&nbsp; that approximates sufficiently well the amplitude distribution of speech and music signals.  
* The moments&nbsp; $k$&ndash;th order &nbsp; &rArr; &nbsp; $m_k$&nbsp; of the Laplace distribution agree with those of the exponential distribution for even&nbsp; $k$&nbsp; .
+
* The&nbsp; $k$&ndash;th order moments&nbsp; $m_k$&nbsp; of the Laplace distribution agree with those of the exponential distribution for even&nbsp; $k$.
* For odd&nbsp; $k$&nbsp; on the other hand, the (symmetric) Laplace distribution always yields&nbsp; $m_k= 0$.
+
* For odd&nbsp; $k$,&nbsp; the&nbsp; (symmetric)&nbsp; Laplace distribution always yields&nbsp; $m_k= 0$.
 
+
*For generation of the Laplace distribution,&nbsp; one uses a between&nbsp; $±1$&nbsp; uniformly distributed random variable&nbsp; $v$&nbsp; $($where&nbsp; $v = 0$&nbsp; must be excluded$)$&nbsp; and the transformation characteristic curve
 
 
For generation one uses a between&nbsp; $±1$&nbsp; equally distributed random variable&nbsp; $v$&nbsp; (where&nbsp; $v = 0$&nbsp; must be excluded)&nbsp; and the transformation characteristic curve
 
 
:$$x=\frac{{\rm sign}(v)}{\lambda}\cdot \rm ln(\it v \rm ).$$
 
:$$x=\frac{{\rm sign}(v)}{\lambda}\cdot \rm ln(\it v \rm ).$$
 
  
 
Further notes:
 
Further notes:
*From the&nbsp; [[Aufgaben:Exercise_3.8:_Amplification_and_Limitation| Exercise 3.8]]&nbsp; one can see further properties of the Laplace distribution.
+
*From the&nbsp; [[Aufgaben:Exercise_3.8:_Amplification_and_Limitation|"Exercise 3.8"]]&nbsp; one can see further properties of the Laplace distribution.
 
+
*With the HTML 5/JavaScript applet&nbsp; [[Applets:PDF,_CDF_and_Moments_of_Special_Distributions|"PDF, CDF and Moments of Special Distributions"]]&nbsp; you can display the characteristics of the exponential and the Laplace distribution.
*In the (German) learning video&nbsp; [[Wahrscheinlichkeit_und_WDF_(Lernvideo)|Wahrscheinlichkeit und WDF]]&nbsp; $\Rightarrow$ Probability and PDF, it is shown which meaning the Laplace distribution has for the description of speech&ndash; and music signals.
+
*In the&nbsp; (German language)&nbsp; learning video&nbsp; [[Wahrscheinlichkeit_und_WDF_(Lernvideo)|"Wahrscheinlichkeit und WDF"]] &nbsp; $\Rightarrow$ &nbsp; "Probability and PDF",&nbsp; it is shown which meaning the Laplace distribution has for the description of speech and music signals.
*With the applet&nbsp; [[Applets:PDF,_CDF_and_Moments_of_Special_Distributions|PDF, CDF and Moments]]&nbsp; you can display the characteristics&nbsp; $($PDF, CDF, Moments$)$&nbsp; of exponential and Laplace distributions.
+
*We also refer you to the&nbsp; (German language)&nbsp; HTML 5/JavaScript applet&nbsp; [[Applets:Zweidimensionale_Laplace-Zufallsgrößen_(Applet)|"Zweidimensionale Laplace-Zufallsgrößen"]] &nbsp; &rArr; &nbsp; "Two-dimensional Laplace random variables".
*We also refer you to the applet&nbsp; [[Applets:Zweidimensionale_Laplace-Zufallsgrößen_(Applet)|Two-dimensional Laplace random quantities]]&nbsp;.
 
 
    
 
    
  
 
==Exercises for the chapter==
 
==Exercises for the chapter==
 
<br>
 
<br>
[[Aufgaben:3.8 Verstärkung und Begrenzung|Aufgabe 3.8: Verstärkung und Begrenzung]]
+
[[Aufgaben:Exercise_3.8:_Amplification_and_Limitation|Exercise 3.8: Amplification and Limitation]]
  
[[Aufgaben:Aufgabe_3.8Z:_Kreis(ring)fläche|Aufgabe 3.8Z: Kreis(ring)fläche]]
+
[[Aufgaben:Exercise_3.8Z:_Circle_(Ring)_Area|Exercise 3.8Z: Circle (Ring) Area]]
  
[[Aufgaben:3.9 Kennlinie für Cosinus-WDF|Aufgabe 3.9: Kennlinie für Cosinus-WDF]]
+
[[Aufgaben:Exercise_3.9:_Characteristic_Curve_for_Cosine_PDF|Exercise 3.9: Characteristic Curve for Cosine PDF]]
  
[[Aufgaben:3.9Z Sinustransformation|Aufgabe 3.9Z: Sinustransformation]]
+
[[Aufgaben:Exercise_3.9Z:_Sine_Transformation|Exercise 3.9Z: Sine Transformation]]
  
  
  
 
{{Display}}
 
{{Display}}

Latest revision as of 21:22, 20 December 2022

One-sided exponential distribution


$\text{Definition:}$  A continuous random variable  $x$  is called  (one-sided)  »exponentially distributed«  if it can take only non–negative values and the probability density function has the following shape for  $x>0$:

$$f_x(x)=\it \lambda\cdot\rm e^{\it -\lambda \hspace{0.05cm}\cdot \hspace{0.03cm} x}.$$


PDF and CDF of an exponentially distributed random variable

The left sketch shows the  "probability density function"  $\rm (PDF)$  of such an exponentially distributed random variable  $x$.  To be emphasized:

  1. The larger the distribution parameter  $λ$,    the steeper the decay.
  2. By definition  $f_{x}(0) = λ/2$,  i.e. the mean of left-hand limit  $(0)$  and right-hand limit  $(\lambda)$.
  • For the  "cumulative distribution function"  $\rm (CDF)$,  we obtain for  $r > 0$  by integration over the PDF  (right graph):
$$F_{x}(r)=1-\rm e^{\it -\lambda\hspace{0.05cm}\cdot \hspace{0.03cm} r}.$$
  • The  "moments"  of the one-sided exponential distribution are generally equal to  
$$m_k = k!/λ^k.$$
  • From this and from Steiner's theorem, we get for the "mean" and the "standard deviation":
$$m_1={1}/{\lambda},$$
$$\sigma=\sqrt{m_2-m_1^2}=\sqrt{\frac{2}{\lambda^2}-\frac{1}{\lambda^2}}={1}/{\lambda}.$$

$\text{Example 1:}$  The exponential distribution has great importance for reliability studies,  and the term  "lifetime distribution"  is also commonly used in this context.

  1. In these applications,  the random variable is often the time  $t$  that elapses before a component fails.
  2. Furthermore,  it should be noted that the exponential distribution is closely related to the  $\text{Poisson distribution}$.

Transformation of random variables


To generate such an exponentially distributed random variable on a digital computer,  you can use e.g. a  »nonlinear transformation«.  The underlying principle is first stated here in general terms.

$\text{Procedure:}$  If a continuous-valued random variable  $u$  possesses the PDF  $f_{u}(u)$,  then the probability density function of the random variable transformed at the nonlinear characteristic  $x = g(u)$  holds:

$$f_{x}(x)=\frac{f_u(u)}{\mid g\hspace{0.05cm}'(u)\mid}\Bigg \vert_{\hspace{0.1cm} u=h(x)}.$$

Here,  $g\hspace{0.05cm}'(u)$  denotes the derivative of the characteristic curve  $g(u)$  and  $h(x)$  gives the inverse function to  $g(u)$  .


  • However,  the above equation is only valid under the condition that the derivative  $g\hspace{0.03cm}'(u) \ne 0$.
  • For a characteristic with horizontal sections  $(g\hspace{0.05cm}'(u) = 0)$:  Additional Dirac delta functions appear in the PDF if the input variable has components in these ranges.
  • The weights of these Dirac delta functions are equal to the probabilities that the input variable lies in these ranges.


To transform random variables

$\text{Example 2:}$  Given a random variable  $u$  triangularly distributed between  $-2$  and  $+2$  on a nonlinearity with characteristic  $x = g(u)$,

  • which,  in the range  $\vert u \vert ≤ 1$  triples the input values,  and
  • mapping all values  $\vert u \vert > 1$  to   $x = \pm 3$   depending on the sign,


then the PDF  $f_{x}(x)$  sketched on the right is obtained.


Please note:

  1. Due to the amplification by a factor of  $3$   ⇒   $f_{x}(x)$  is wider and lower than $f_{u}(u)$ by this factor.
  2. The two horizontal limits of the characteristic at   $u = ±1$   lead to two Dirac delta functions at  $x = ±3$,  each with weight  $1/8$.
  3. The weight  $1/8$  corresponds to the green areas in the PDF  $f_{u}(u).$

Generation of an exponentially distributed random variable


$\text{Procedure:}$  Now we assume that the random variable  $u$  to be transformed is uniformly distributed between  $0$  (inclusive) and  $1$  (exclusive). 

  • Moreover,  we consider the monotonically increasing characteristic curve
$$x=g_1(u) =\frac{1}{\lambda}\cdot \rm ln \ (\frac{1}{1-\it u}).$$
  • It can be shown that by this characteristic  $x=g_1(u)$  a one-sided exponentially distributed random variable  $x$  with the following PDF arises 
    (derivation see "next section"):
$$f_{x}(x)=\lambda\cdot\rm e^{\it -\lambda \hspace{0.05cm}\cdot \hspace{0.03cm} x}\hspace{0.2cm}{\rm for}\hspace{0.2cm} {\it x}>0.$$
  • Note:
  1. For  $x = 0$  the PDF value is half  $(\lambda/2)$.
  2. Negative  $x$ values do not occur because for  $0 ≤ u < 1$  the argument of the (natural) logarithm function does not become smaller than  $1$.


By the way,  the same PDF is obtained with the monotonically decreasing characteristic curve

$$x=g_2(u)=\frac{1}{\lambda}\cdot \rm ln \ (\frac{1}{\it u})=-\frac{1}{\lambda}\cdot \rm ln(\it u \rm ).$$

Please note:

  • When using a computer implementation corresponding to the first transformation characteristic  $x=g_1(u)$   ⇒   the value  $u = 1$  must be excluded.
  • On the other hand,  if one uses the second transformation characteristic  $x=g_2(u)$   ⇒   the value  $u =0$  must be excluded.


The following  (German language)  learning video shall clarify the transformations derived here:
     "Erzeugung einer Exponentialverteilung"   $\Rightarrow$   "Generation of an exponential distribution".

Derivation of the corresponding transformation characteristic


$\text{Task:}$ 

  1.   Now the transformation characteristic  $x = g_1(u)= g(u)$  already used in the last section is derived.
  2.   This forms from the uniformly distributed random variable  $u$  with PDF  $f_{u}(u)$  a one-sided exponentially distributed random variable  $x$  with PDF  $f_{x}(x)$:
$$f_{u}(u)= \left\{ \begin{array}{*{2}{c} } 1 & \rm if\hspace{0.3cm} 0 < {\it u} < 1,\\ 0.5 & \rm if\hspace{0.3cm} {\it u} = 0, {\it u} = 1,\\ 0 & \rm else, \\ \end{array} \right. \hspace{0.5cm}\Rightarrow \hspace{0.5cm} f_{x}(x)= \left\{ \begin{array}{*{2}{c} } \lambda\cdot\rm e^{\it -\lambda\hspace{0.03cm} \cdot \hspace{0.03cm} x} & \rm if\hspace{0.3cm} {\it x} > 0,\\ \lambda/2 & \rm if\hspace{0.3cm} {\it x} = 0 ,\\ 0 & \rm else\hspace{0.3cm} {\it x} < 0. \\ \end{array} \right.$$


$\text{Solution:}$ 

(1)  Starting from the general transformation equation

$$f_{x}(x)=\frac{f_{u}(u)}{\mid g\hspace{0.05cm}'(u) \mid }\Bigg \vert _{\hspace{0.1cm} u=h(x)}$$
is obtained by converting and substituting the given PDF  $f_{ x}(x):$
$$\mid g\hspace{0.05cm}'(u)\mid\hspace{0.1cm}=\frac{f_{u}(u)}{f_{x}(x)}\Bigg \vert _{\hspace{0.1cm} x=g(u)}= {1}/{\lambda} \cdot {\rm e}^{\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}g(u)}.$$
Here  $x = g\hspace{0.05cm}'(u)$  gives the derivative of the characteristic curve,  which we assume to be monotonically increasing.

(2)  With this assumption we get   $\vert g\hspace{0.05cm}'(u)\vert = g\hspace{0.05cm}'(u) = {\rm d}x/{\rm d}u$   and the differential equation   ${\rm d}u = \lambda\ \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}\, {\rm d}x$   with solution   $u = K - {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}.$

(3)  From the condition that the input variable  $u =0$  should lead to the output value  $x =0$,  we obtain for the constant  $K =1$  and thus   $u = 1- {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm} x}.$

(4)  Solving this equation for  $x$  yields the equation given in front:

$$x = g_1(u)= \frac{1}{\lambda} \cdot {\rm ln} \left(\frac{1}{1 - u} \right) .$$
  • In a computer implementation,  however,  ensure that the critical value  $1$  is excluded for the uniformly distributed input variable  $u$. 
  • This,  however,  has (almost) no effect on the final result.


Two-sided exponential distribution - Laplace distribution


Closely related to the exponential distribution is the  $\text{Laplace distribution}$  with the probability density function

$$f_{x}(x)=\frac{\lambda}{2}\cdot\rm e^{\it -\lambda \hspace{0.05cm} \cdot \hspace{0.05cm} | x|}.$$

The Laplace distribution is a  "two-sided exponential distribution"  that approximates sufficiently well the amplitude distribution of speech and music signals.

  • The  $k$–th order moments  $m_k$  of the Laplace distribution agree with those of the exponential distribution for even  $k$.
  • For odd  $k$,  the  (symmetric)  Laplace distribution always yields  $m_k= 0$.
  • For generation of the Laplace distribution,  one uses a between  $±1$  uniformly distributed random variable  $v$  $($where  $v = 0$  must be excluded$)$  and the transformation characteristic curve
$$x=\frac{{\rm sign}(v)}{\lambda}\cdot \rm ln(\it v \rm ).$$

Further notes:


Exercises for the chapter


Exercise 3.8: Amplification and Limitation

Exercise 3.8Z: Circle (Ring) Area

Exercise 3.9: Characteristic Curve for Cosine PDF

Exercise 3.9Z: Sine Transformation