Aufgaben:Exercise 4.2Z: Mixed Random Variables - Revision history

Guenter at 13:57, 1 October 2021

2021-10-01T13:57:57Z

Guenter at 13:48, 1 October 2021

2021-10-01T13:48:47Z

Guenter at 13:48, 1 October 2021

2021-10-01T13:48:09Z

Guenter at 13:42, 1 October 2021

2021-10-01T13:42:28Z

Noah at 10:45, 29 September 2021

2021-09-29T10:45:31Z

Show changes

Noah at 10:38, 27 September 2021

2021-09-27T10:38:16Z

Noah at 09:16, 24 September 2021

2021-09-24T09:16:05Z

Guenter at 09:16, 16 August 2021

2021-08-16T09:16:08Z

Guenter: Guenter moved page Aufgaben:Aufgabe 4.2Z: Gemischte Zufallsgrößen to Aufgaben:Exercise 4.2Z: Mixed Random Variables

2021-08-16T09:03:36Z

Guenter moved page Aufgabe 4.2Z: Gemischte Zufallsgrößen to Exercise 4.2Z: Mixed Random Variables

Javier: Text replacement - "„" to """

2021-05-28T14:38:32Z

Text replacement - "„" to """

@@ Line 64: / Line 64: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; <u>Proposed solution 2</u> is correct because the integral over the PDF&nbsp; $1$&nbsp; must yield:
+'''(1)'''&nbsp; <u>Proposed solution 2</u>&nbsp; is correct because the integral&nbsp; $1$&nbsp; over the PDF must yield:
 :$$f_X(x) \hspace{0.1cm}{\rm d}x =
 .25 \cdot 2 + (A - 0.25) \cdot \varepsilon \stackrel{!}{=} 1 \hspace{0.3cm}
@@ Line 98: / Line 98: @@
-'''(3)'''&nbsp; <u>All the proposed solutions</u> are correct:
+'''(3)'''&nbsp; <u>All the proposed solutions</u>&nbsp; $1$&nbsp; are correct:
 *After the boundary transition &nbsp; $\varepsilon &#8594; 0$ &nbsp; we obtain for the differential entropy
 :$$h(X) = \lim\limits_{\varepsilon \hspace{0.05cm}\rightarrow \hspace{0.05cm} 0} \hspace{0.1cm}\big[(2-\varepsilon) - (0.5 + 0.25 \cdot \varepsilon) \cdot {\rm log}_2 \hspace{0.1cm}(0.5/\varepsilon +0.25)\big]
@@ Line 108: / Line 108: @@
 :$$f_X(x) = \left\{ \begin{array}{c} 0.25 + 0.5 \cdot \delta (x-1) \\  0 \\  \end{array} \right. \begin{array}{*{20}c}   {\rm{f\ddot{u}r}} \hspace{0.1cm} 0 \le x \le 2, \\     {\rm sonst} \\ \end{array}
 \hspace{0.05cm}.$$
-Consequently, it is a "mixed" random variable with
+Consequently, it is a&nbsp; "mixed"&nbsp; random variable with
 * a stochastic, uniformly distributed part in the range&nbsp; $0 \le x \le 2$, and
 * a discrete component at&nbsp; $x = 1$&nbsp; with probability&nbsp; $0.5$.
-The graph shows the PDF &nbsp;$f_X(x)$&nbsp; on the left and the distribution function &nbsp;$F_X(x)$ on the right.
+The graph shows the PDF &nbsp;$f_X(x)$&nbsp; on the left and the CDF &nbsp;$F_X(x)$ on the right.
 <br clear=all>
-'''(4)'''&nbsp; <u>The correct solutions are 2, 3 and 5</u>.
+'''(4)'''&nbsp; <u>The correct solutions are 2, 3 and 5</u>.&nbsp;
 The lower graph shows the PDF and the CDF of the random variable&nbsp; $Y$.&nbsp; You can see:
 [[File:P_ID2872__Inf_Z_4_2d_neu.png|right|frame|PDF and CDF of the mixed random variable $Y$]]
@@ Line 122: / Line 122: @@
 * Since &nbsp;$F_Y(y)= {\rm Pr}(Y \le y)$&nbsp; holds, the right-hand side limit is:
 :$$F_Y(y = 1) = 0.55.$$
-* The continuous component is not uniformly distributed;&nbsp; rather, there is a triangular distribution.
+* The continuous component is not uniformly distributed;&nbsp; rather, there is a triangular PDF.
 *The last proposition is also correct: &nbsp; $h(Y) = h(X) = - \infty$.
 <br clear=all>
-Because: &nbsp; '''For every random quantity with a discrete part &ndash; and it is also so small, the differential entropy is equal minus infinity.'''.
+Because: &nbsp; '''For every random quantity with a discrete part &ndash; and it is also extremely small, the differential entropy is equal minus infinity.'''.
 {{ML-Fuß}}

@@ Line 23: / Line 23: @@
 Hints:
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Differentielle_Entropie|Differential Entropy]].
-*Further information on mixed random variables can be found in the chapter &nbsp;  [[Theory_of_Stochastic_Signals/Verteilungsfunktion|Cumulative Distribution Function]]&nbsp; of the book "Theory of Stochastic Signals".
+*Further information on mixed random variables can be found in the chapter &nbsp;  [[Theory_of_Stochastic_Signals/Cumulative_Distribution_Function_(CDF)|Cumulative Distribution Function]]&nbsp; of the book "Theory of Stochastic Signals".

@@ Line 6: / Line 6: @@
 One speaks of a&nbsp; "mixed random variable",&nbsp; if the random variable contains discrete components in addition to a continuous component.
-*For example, the random variable&nbsp; $Y$&nbsp; with&nbsp; [[Theory_of_Stochastic_Signals/Verteilungsfunktion|cumulative distribution function]]&nbsp; $F_Y(y)$&nbsp;  as shown in the sketch below has both a continuous and a discrete component.
+*For example, the random variable&nbsp; $Y$&nbsp; with&nbsp; [[Theory_of_Stochastic_Signals/Cumulative_Distribution_Function_(CDF)|cumulative distribution function]]&nbsp; $F_Y(y)$&nbsp;  as shown in the sketch below has both a continuous and a discrete component.
 *The&nbsp; [[Theory_of_Stochastic_Signals/Wahrscheinlichkeitsdichtefunktion|probability density function]]&nbsp; $f_Y(y)$&nbsp; is obtained from&nbsp; $F_Y(y)$&nbsp; by differentiation.
 *The jump at&nbsp; $y= 1$&nbsp; in the CDF thus becomes a "Dirac" in the probability density function.

@@ Line 4: / Line 4: @@
 [[File:P_ID2868__Inf_Z_4_2_neu.png|right|frame|PDF of&nbsp; $X$&nbsp; (top),&nbsp; and <br>CDF of&nbsp; $Y$&nbsp; (bottom)]]
-One speaks of a "mixed random variable", if the random variable contains discrete components in addition to a continuous component.
+One speaks of a&nbsp; "mixed random variable",&nbsp; if the random variable contains discrete components in addition to a continuous component.
 *For example, the random variable&nbsp; $Y$&nbsp; with&nbsp; [[Theory_of_Stochastic_Signals/Verteilungsfunktion|cumulative distribution function]]&nbsp; $F_Y(y)$&nbsp;  as shown in the sketch below has both a continuous and a discrete component.
@@ Line 10: / Line 10: @@
 *The jump at&nbsp; $y= 1$&nbsp; in the CDF thus becomes a "Dirac" in the probability density function.
-*In subtask&nbsp; '''(4)'''&nbsp; the differential entropy&nbsp; $h(Y)$&nbsp; of the random variable&nbsp; $Y$&nbsp;  is to be determined (in bit), assuming the following equation:
+*In subtask&nbsp; '''(4)'''&nbsp; the differential entropy&nbsp; $h(Y)$&nbsp; of&nbsp; $Y$&nbsp;  is to be determined (in bit), assuming the following equation:
 :$$h(Y) =
 \hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_Y)} \hspace{-0.35cm}  f_Y(y) \cdot {\rm log}_2 \hspace{0.1cm} \big[ f_Y(y) \big] \hspace{0.1cm}{\rm d}y
 \hspace{0.05cm}.$$
-*In subtask&nbsp; '''(2)'''&nbsp;, calculate the differential entropy&nbsp;  $h(X)$&nbsp; of the random variable&nbsp; $X$&nbsp; whose PDF&nbsp; $f_X(x)$&nbsp; is sketched above.&nbsp; If one performs a suitable boundary transition, the random variable&nbsp; $X$&nbsp; also becomes a mixed random variable.
+*In subtask&nbsp; '''(2)''',&nbsp; calculate the differential entropy&nbsp;  $h(X)$&nbsp; of the random variable&nbsp; $X$&nbsp; whose PDF&nbsp; $f_X(x)$&nbsp; is sketched above.&nbsp; If one performs a suitable boundary transition, the random variable&nbsp; $X$&nbsp; also becomes a mixed random variable.
@@ Line 26: / Line 23: @@
 Hints:
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Differentielle_Entropie|Differential Entropy]].
-*Further information on mixed random variables can be found in the chapter &nbsp;  [[Theory_of_Stochastic_Signals/Verteilungsfunktion|Cumulative Distribution Function]] of the book "Theory of Stochastic Signals".
+*Further information on mixed random variables can be found in the chapter &nbsp;  [[Theory_of_Stochastic_Signals/Verteilungsfunktion|Cumulative Distribution Function]]&nbsp; of the book "Theory of Stochastic Signals".
@@ Line 34: / Line 31: @@
 <quiz display=simple>
-{What is the WDF height&nbsp; $A$&nbsp; of &nbsp;$f_X(x)$&nbsp; around &nbsp;$x = 1$?
+{What is the PDF height&nbsp; $A$&nbsp; of &nbsp;$f_X(x)$&nbsp; around &nbsp;$x = 1$?
 |type="[]"}
 - $A = 0.5/\varepsilon$,
@@ Line 41: / Line 38: @@
 {Calculate the differential entropy for different &nbsp;$\varepsilon$&ndash;values.
 $ε = 10^{-1}\text{:} \ \    h(X) \ = \ $ { 0.644 3% } $\ \rm bit$
 $ε = 10^{-2}\text{:} \ \    h(X) \ = \ $ { -0.87--0.83 } $\ \rm bit$
@@ Line 52: / Line 50: @@
-{Which statements are true for the random variable&nbsp; $Y$&nbsp; zu?
+{Which statements are true for the random variable&nbsp; $Y$?
 |type="[]"}
 - The CDF value at the point &nbsp;$y = 1$&nbsp; is&nbsp; $0.5$.
 + $Y$&nbsp; contains a discrete and a continuous component.
-+ The discrete portion at&nbsp; &nbsp;$Y = 1$&nbsp;  occurs with &nbsp;$10\%$&nbsp; probability.
++ The discrete component at&nbsp; &nbsp;$Y = 1$&nbsp;  occurs with &nbsp;$10\%$&nbsp; probability.
-- The continuous component o&nbsp; $Y$&nbsp; is uniformly distributed.
+- The continuous component of&nbsp; $Y$&nbsp; is uniformly distributed.
 + The differential entropies of&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; are equal.

← Older revision		Revision as of 10:38, 27 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Differentielle Entropie~~	+	{{quiz-Header\|Buchseite=Information_Theory/Differential_Entropy
	}}		}}

← Older revision		Revision as of 09:16, 16 August 2021
Line 135:		Line 135:


−	[[Category:Information Theory: Exercises\|^4.1 ~~Differentielle Entropie~~^]]	+	[[Category:Information Theory: Exercises\|^4.1 Differential Entropy^]]

← Older revision	Revision as of 09:03, 16 August 2021
(No difference)

@@ Line 4: / Line 4: @@
 [[File:P_ID2868__Inf_Z_4_2_neu.png|right|frame|WDF von&nbsp; $X$&nbsp; (oben),&nbsp; und <br>VTF von&nbsp; $Y$&nbsp; (unten)]]
-Man spricht von einer &bdquo;gemischten Zufallsgröße", wenn die Zufallsgröße neben einem kontinuierlichen Anteil auch noch diskrete Anteile beinhaltet.
+Man spricht von einer "gemischten Zufallsgröße", wenn die Zufallsgröße neben einem kontinuierlichen Anteil auch noch diskrete Anteile beinhaltet.
 *Die Zufallsgröße&nbsp; $Y$&nbsp; mit der&nbsp; [[Theory_of_Stochastic_Signals/Verteilungsfunktion|Verteilungsfunktion]]&nbsp; $F_Y(y)$&nbsp; gemäß der unteren Skizze besitzt beispielsweise sowohl einen kontinuierlichen als auch einen diskreten Anteil.
 *Die&nbsp; [[Theory_of_Stochastic_Signals/Wahrscheinlichkeitsdichtefunktion|Wahrscheinlichkeitsdichtefunktion]]&nbsp; $f_Y(y)$&nbsp; erhält man aus&nbsp; $F_Y(y)$&nbsp; durch Differentiation.
-*Aus dem Sprung bei&nbsp; $y= 1$&nbsp; in der Verteilungsfunktion (VTF) wird somit ein &bdquo;Dirac" in der Wahrscheinlichkeitsdichtefunktion (WDF).
+*Aus dem Sprung bei&nbsp; $y= 1$&nbsp; in der Verteilungsfunktion (VTF) wird somit ein "Dirac" in der Wahrscheinlichkeitsdichtefunktion (WDF).
 *In der Teilaufgabe&nbsp; '''(4)'''&nbsp; soll die differentielle Entropie&nbsp; $h(Y)$&nbsp; der Zufallsgröße&nbsp; $Y$&nbsp; ermittelt werden (in bit), wobei von folgender Gleichung auszugehen ist:
@@ Line 26: / Line 26: @@
 ''Hinweise:''
 *Die Aufgabe gehört zum  Kapitel&nbsp; [[Information_Theory/Differentielle_Entropie|Differentielle Entropie]].
-*Weitere Informationen zu gemischten Zufallsgrößen finden Sie im Kapitel&nbsp;  [[Theory_of_Stochastic_Signals/Verteilungsfunktion|Verteilungsfunktion]] des Buches &bdquo;Stochastische Signaltheorie".
+*Weitere Informationen zu gemischten Zufallsgrößen finden Sie im Kapitel&nbsp;  [[Theory_of_Stochastic_Signals/Verteilungsfunktion|Verteilungsfunktion]] des Buches "Stochastische Signaltheorie".
@@ Line 75: / Line 75: @@
-'''(2)'''&nbsp; Die differentielle Entropie (in &bdquo;bit") ist wie folgt gegeben:
+'''(2)'''&nbsp; Die differentielle Entropie (in "bit") ist wie folgt gegeben:
 :$$h(X) =
 \hspace{0.1cm}  \hspace{-0.45cm} \int\limits_{{\rm supp}(f_X)} \hspace{-0.35cm}  f_X(x) \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{f_X(x)} \hspace{0.1cm}{\rm d}x
@@ Line 111: / Line 111: @@
 :$$f_X(x) = \left\{ \begin{array}{c} 0.25 + 0.5 \cdot \delta (x-1) \\  0 \\  \end{array} \right. \begin{array}{*{20}c}   {\rm{f\ddot{u}r}} \hspace{0.1cm} 0 \le x \le 2, \\     {\rm sonst} \\ \end{array}
 \hspace{0.05cm}.$$
-Es handelt sich demzufolge um eine &bdquo;gemischte" Zufallsgröße mit
+Es handelt sich demzufolge um eine "gemischte" Zufallsgröße mit
 * einem stochastischen, gleichverteilten Anteil im Bereich&nbsp; $0 \le x \le 2$, und
 * einem diskreten Anteil bei&nbsp; $x = 1$&nbsp; mit der Wahrscheinlichkeit&nbsp; $0.5$.