Aufgaben:Exercise 4.3Z: Exponential and Laplace Distribution - Revision history

Javier: Text replacement - "value-continuous" to "continuous"

2021-10-11T08:27:12Z

Text replacement - "value-continuous" to "continuous"

Guenter at 14:33, 1 October 2021

2021-10-01T14:33:11Z

Guenter at 13:06, 28 September 2021

2021-09-28T13:06:16Z

Guenter at 12:53, 28 September 2021

2021-09-28T12:53:03Z

Guenter at 12:49, 28 September 2021

2021-09-28T12:49:52Z

Guenter at 15:19, 27 September 2021

2021-09-27T15:19:43Z

Noah at 11:53, 27 September 2021

2021-09-27T11:53:21Z

Show changes

Noah at 10:38, 27 September 2021

2021-09-27T10:38:30Z

Guenter at 09:17, 16 August 2021

2021-08-16T09:17:51Z

Guenter: Guenter moved page Aufgaben:Aufgabe 4.3Z: Exponential– und Laplaceverteilung to Aufgaben:Exercise 4.3Z: Exponential and Laplace Distribution

2021-08-16T09:12:43Z

Guenter moved page Aufgabe 4.3Z: Exponential– und Laplaceverteilung to Exercise 4.3Z: Exponential and Laplace Distribution

@@ Line 4: / Line 4: @@
 [[File:EN_Inf_Z_4_3.png|right|frame|Exponential PDF (above) d<br>Laplace PDF (below)]]
-We consider here the probability density functions&nbsp; $\rm (PDF)$&nbsp; of two value-continuous random variables:
+We consider here the probability density functions&nbsp; $\rm (PDF)$&nbsp; of two continuous random variables:
 *The random variable &nbsp; $X$&nbsp; is exponentially distributed (see top plot): &nbsp; For&nbsp; $x<0$&nbsp;  &nbsp; &rArr; &nbsp; $f_X(x) = 0$,&nbsp; and for positive $x$&ndash;values:
 :$$f_X(x) =  \lambda \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}x}\hspace{0.05cm}. $$

@@ Line 3: / Line 3: @@
 }}
-[[File:EN_Inf_Z_4_3.png|right|frame|PDF of exponential distribution and Laplace distribution (below)]]
+[[File:EN_Inf_Z_4_3.png|right|frame|Exponential PDF (above) d<br>Laplace PDF (below)]]
 We consider here the probability density functions&nbsp; $\rm (PDF)$&nbsp; of two value-continuous random variables:
 *The random variable &nbsp; $X$&nbsp; is exponentially distributed (see top plot): &nbsp; For&nbsp; $x<0$&nbsp;  &nbsp; &rArr; &nbsp; $f_X(x) = 0$,&nbsp; and for positive $x$&ndash;values:

@@ Line 66: / Line 66: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; Although in this exercise the result should be given in "bit" , we use the natural logarithm for derivation.
+'''(1)'''&nbsp; Although in this exercise the result should be given in&nbsp; "bit",&nbsp; we use the natural logarithm for derivation.
 *Then the differential entropy is:
@@ Line 72: / Line 72: @@
 \hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm}  f_X(x) \cdot {\rm ln} \hspace{0.1cm} \big [f_X(x)\big] \hspace{0.1cm}{\rm d}x
 \hspace{0.05cm}.$$
-*For the exponential distribution, the integration limits are&nbsp; $0$&nbsp; and&nbsp; $+&#8734;$&nbsp;.&nbsp; In this range, the PDF&nbsp; $f_X(x)$&nbsp; is used according to the specification sheet:
+*For the exponential distribution,&nbsp; the integration limits are&nbsp; $0$&nbsp; and&nbsp; $+&#8734;$.&nbsp; In this range, the PDF&nbsp; $f_X(x)$&nbsp; according to the specification sheet  is used:
 :$$h(X) =-  \int_{0}^{\infty} \hspace{-0.15cm}
 \lambda \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}x}
@@ Line 84: / Line 84: @@
 We can see:
 * The first integrand is identical to the PDF&nbsp; $f_X(x)$ considered here.&nbsp;  Thus, the integral over the entire integration domain yields&nbsp; $1$.
-* The second integral corresponds exactly to the definition of the mean value&nbsp; $m_1$&nbsp; (moment of first order).&nbsp;  For the exponential distribution,&nbsp; $m_1 = 1/&lambda;$ holds.&nbsp; From this follows:
+* The second integral corresponds exactly to the definition of the mean value&nbsp; $m_1$&nbsp; (moment of first order).&nbsp;  For the exponential PDF,&nbsp; $m_1 = 1/&lambda;$ holds.&nbsp; From this follows:
 :$$h(X) = - \hspace{0.05cm} {\rm ln} \hspace{0.1cm} (\lambda) + 1 =
 - \hspace{0.05cm} {\rm ln} \hspace{0.1cm} (\lambda) + \hspace{0.05cm} {\rm ln} \hspace{0.1cm} ({\rm e}) = {\rm ln} \hspace{0.1cm} ({\rm e}/\lambda)
 \hspace{0.05cm}.$$
-*This result is to be given the additional unit "nat".&nbsp; Using&nbsp; $\log_2$&nbsp;  instead of&nbsp; $\ln$&nbsp;, we obtain the differential entropy in "bit":
+*This result is to be given the additional unit&nbsp; "nat".&nbsp; Using&nbsp; $\log_2$&nbsp;  instead of&nbsp; $\ln$,&nbsp; we obtain the differential entropy in&nbsp; "bit":
 :$$h(X) =  {\rm log}_2 \hspace{0.1cm} ({\rm e}/\lambda)
 \hspace{0.3cm} \Rightarrow \hspace{0.3cm} \lambda = 1{\rm :}
@@ Line 110: / Line 110: @@
 '''(3)'''&nbsp; For the Laplace distribution, we divide the integration domain into two subdomains:
-* $Y$&nbsp; negative &nbsp; &#8658; &nbsp; Anteil&nbsp; $h_{\rm neg}(Y)$,
+* $Y$&nbsp; negative &nbsp; &#8658; &nbsp; proportion&nbsp; $h_{\rm neg}(Y)$,
-* $Y$&nbsp; positive &nbsp; &#8658; &nbsp; Anteil&nbsp; $h_{\rm pos}(Y)$.
+* $Y$&nbsp; positive &nbsp; &#8658; &nbsp; proportion&nbsp; $h_{\rm pos}(Y)$.
@@ Line 129: / Line 129: @@
 - \hspace{0.05cm} {\rm ln} \hspace{0.1cm} (\lambda/2) + \hspace{0.05cm} {\rm ln} \hspace{0.1cm} ({\rm e}) = {\rm ln} \hspace{0.1cm} (2{\rm e}/\lambda)
 \hspace{0.05cm}.$$
-*Since the result is required in "bit" , we still need to replace&nbsp; $\ln$&nbsp; by&nbsp; $\log_2$&nbsp;:
+*Since the result is required in&nbsp; "bit",&nbsp; we still need to replace&nbsp; "$\ln$"&nbsp; by&nbsp; "$\log_2$":
 :$$h(Y) =  {\rm log}_2 \hspace{0.1cm} (2{\rm e}/\lambda)
 \hspace{0.3cm} \Rightarrow \hspace{0.3cm} \lambda = 1{\rm :}
@@ Line 145: / Line 145: @@
 \hspace{0.05cm}.$$
 *Consequently, the&nbsp; ${\it \Gamma}_{{\hspace{-0.05cm} \rm L}}$ value is twice as large for the Laplace distribution as for the exponential distribution.
-*Thus, the Laplace distribution is better than the exponential distribution in terms of differential entropy when power-limited signals are assumed.
+*Thus, the Laplace PDF is better than the exponential PDF in terms of differential entropy when power-limited signals are assumed.
-*Under the constraint of peak limiting, both the exponential and Laplace distributions are completely unsuitable, as is the Gaussian distribution.  These all extend to infinity.
+*Under the constraint of peak limitation,&nbsp; both the exponential and Laplace PDF are completely unsuitable, as is the Gaussian PDF.&nbsp;  These all extend to infinity.
 {{ML-Fuß}}

@@ Line 33: / Line 33: @@
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Differentielle_Entropie|Differential Entropy]].
 *Useful hints for solving this task can be found in particular on the page&nbsp;  [[Information_Theory/Differentielle_Entropie#Differential_entropy_of_some_power-constrained_random_variables|Differential entropy of some power-constrained random variables]].
-*For the variance of the exponentially distributed random variable&nbsp; $X$&nbsp; holds, as derived in&nbsp;  [[Aufgaben:4.01Z_Momentenberechnung|Exercise 4.1Z]]: &nbsp; $\sigma^2 = 1/\lambda^2$.
+*For the variance of the exponentially distributed random variable&nbsp; $X$&nbsp; holds, as derived in&nbsp;  [[Aufgaben:Exercise_4.1Z:_Calculation_of_Moments|Exercise 4.1Z]]: &nbsp; $\sigma^2 = 1/\lambda^2$.
 *The variance of the Laplace distributed random variable&nbsp; $Y$&nbsp; is twice as large for the same&nbsp; $\it \lambda$: &nbsp; $\sigma^2 = 2/\lambda^2$.

@@ Line 4: / Line 4: @@
 [[File:EN_Inf_Z_4_3.png|right|frame|PDF of exponential distribution and Laplace distribution (below)]]
-We consider here the probability density functions (WDF) of two continuous-value random variables:
+We consider here the probability density functions&nbsp; $\rm (PDF)$&nbsp; of two value-continuous random variables:
-*The random variable &nbsp; $X$&nbsp; is exponentially distributed (see top plot): &nbsp; For&nbsp; $x<0$&nbsp;  for&nbsp; $f_X(x) = 0$,&nbsp; and for positive $x$&ndash;values:
+*The random variable &nbsp; $X$&nbsp; is exponentially distributed (see top plot): &nbsp; For&nbsp; $x<0$&nbsp;  &nbsp; &rArr; &nbsp; $f_X(x) = 0$,&nbsp; and for positive $x$&ndash;values:
 :$$f_X(x) =  \lambda \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}x}\hspace{0.05cm}. $$
-* On the other hand, for the Laplace distributed random variable&nbsp; $Y$&nbsp; in the whole range&nbsp; $ - \infty < y < + \infty$&nbsp;  (lower sketch) holds:
+* On the other hand, for the Laplace distributed random variable&nbsp; $Y$&nbsp; in the whole range&nbsp; $ - \infty < y < + \infty$&nbsp;  holds (lower sketch):
 :$$f_Y(y) =  \lambda/2 \cdot {\rm e}^{-\lambda \hspace{0.05cm}\cdot \hspace{0.05cm}|\hspace{0.05cm}y\hspace{0.05cm}|}\hspace{0.05cm}.$$
-To be calculated are the differential entropies&nbsp; $h(X)$&nbsp; and&nbsp; $h(Y)$&nbsp; adepending on the PDF parameter&nbsp; $\it \lambda$.&nbsp; For example:
+To be calculated are the differential entropies&nbsp; $h(X)$&nbsp; and&nbsp; $h(Y)$&nbsp; depending on the PDF parameter&nbsp; $\it \lambda$.&nbsp; For example:
 :$$h(X) = -\hspace{-0.7cm}  \int\limits_{x \hspace{0.05cm}\in \hspace{0.05cm}{\rm supp}
 \hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.55cm}  f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [f_X(x) \big ] \hspace{0.1cm}{\rm d}x
 \hspace{0.05cm}.$$
-If&nbsp; $\log_2$&nbsp; is used, add the pseudo-unit "bit".
+If&nbsp; $\log_2$&nbsp; is used, add the pseudo-unit&nbsp; "bit".
 In subtasks&nbsp; '''(2)'''&nbsp; and&nbsp; '''(4)'''&nbsp; specify the differential entropy in the following form:
-:$$h(X) = {1}/{2} \cdot {\rm log} \hspace{0.1cm} ({\it \Gamma}_{{\hspace{-0.01cm} \rm L}}^{\hspace{0.08cm}(X)}  \cdot \sigma^2)
+:$$h(X) = {1}/{2} \cdot {\rm log} \hspace{0.1cm} ({\it \Gamma}_{{\hspace{-0.01cm} \rm L}}^{\hspace{0.08cm}(X)}  \cdot \sigma^2),
-\hspace{0.5cm}{\rm bzw.} \hspace{0.5cm}h(Y) = {1}/{2} \cdot {\rm log} \hspace{0.1cm} ({\it \Gamma}_{{\hspace{-0.05cm} \rm L}}^{\hspace{0.08cm}(Y)}  \cdot \sigma^2)
+\hspace{0.8cm}h(Y) = {1}/{2} \cdot {\rm log} \hspace{0.1cm} ({\it \Gamma}_{{\hspace{-0.05cm} \rm L}}^{\hspace{0.08cm}(Y)}  \cdot \sigma^2)
 \hspace{0.05cm}.$$
-Determine by which factor&nbsp; ${\it \Gamma}_{{\hspace{-0.05cm} \rm L}}^{\hspace{0.08cm}(X)}$&nbsp; the exponential distribution is characterized and which factor&nbsp; ${\it \Gamma}_{{\hspace{-0.01cm} \rm L}}^{\hspace{0.08cm}(Y)}$&nbsp; results for the Laplace distribution.
+Determine by which factor&nbsp; ${\it \Gamma}_{{\hspace{-0.05cm} \rm L}}^{\hspace{0.08cm}(X)}$&nbsp; the exponential PDF is characterized and which factor&nbsp; ${\it \Gamma}_{{\hspace{-0.01cm} \rm L}}^{\hspace{0.08cm}(Y)}$&nbsp; results for the Laplace PDF.
@@ Line 33: / Line 33: @@
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Differentielle_Entropie|Differential Entropy]].
 *Useful hints for solving this task can be found in particular on the page&nbsp;  [[Information_Theory/Differentielle_Entropie#Differential_entropy_of_some_power-constrained_random_variables|Differential entropy of some power-constrained random variables]].
-*For the variance of the exponentially distributed random variable $X$, as derived in  [[Aufgaben:4.01Z_Momentenberechnung|Exercise 4.1Z]] holds. &nbsp; $\sigma^2 = 1/\lambda^2$.
+*For the variance of the exponentially distributed random variable&nbsp; $X$&nbsp; holds, as derived in&nbsp;  [[Aufgaben:4.01Z_Momentenberechnung|Exercise 4.1Z]]: &nbsp; $\sigma^2 = 1/\lambda^2$.
-*Die Varianz der laplaceverteiten Zufallsgröße $Y$ ist bei gleichem $\it \lambda$ doppelt so groß: &nbsp; $\sigma^2 = 2/\lambda^2$.
+*The variance of the Laplace distributed random variable&nbsp; $Y$&nbsp; is twice as large for the same&nbsp; $\it \lambda$: &nbsp; $\sigma^2 = 2/\lambda^2$.

← Older revision		Revision as of 10:38, 27 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Differentielle Entropie~~	+	{{quiz-Header\|Buchseite=Information_Theory/Differential_Entropy
	}}		}}

← Older revision		Revision as of 09:17, 16 August 2021
Line 152:		Line 152:


−	[[Category:Information Theory: Exercises\|^4.1 ~~Differentielle Entropie~~^]]	+	[[Category:Information Theory: Exercises\|^4.1 Differential Entropy^]]

← Older revision	Revision as of 09:12, 16 August 2021
(No difference)