Aufgaben:Exercise 4.4: Conventional Entropy and Differential Entropy - Revision history

Hwang at 12:44, 17 November 2022

2022-11-17T12:44:23Z

Javier: Text replacement - "value-discrete" to "discrete"

2021-10-11T08:27:33Z

Text replacement - "value-discrete" to "discrete"

Javier: Text replacement - "value-continuous" to "continuous"

2021-10-11T08:27:23Z

Text replacement - "value-continuous" to "continuous"

Javier: Text replacement - "continuous-valued" to "continuous"

2021-10-11T08:19:47Z

Text replacement - "continuous-valued" to "continuous"

Guenter at 14:45, 1 October 2021

2021-10-01T14:45:55Z

Guenter at 14:38, 1 October 2021

2021-10-01T14:38:47Z

Noah at 10:54, 1 October 2021

2021-10-01T10:54:44Z

Guenter at 15:12, 28 September 2021

2021-09-28T15:12:32Z

Guenter at 15:11, 28 September 2021

2021-09-28T15:11:48Z

Guenter at 15:08, 28 September 2021

2021-09-28T15:08:42Z

@@ Line 16: / Line 16: @@
 The probability density functions&nbsp; $\rm (PDF)$&nbsp; of these discrete random variables are each composed of&nbsp; $M$&nbsp; Dirac functions whose momentum weights are given by the interval areas of the associated continuous random variables.
-From this, the entropies&nbsp; $H(Z_{X,\hspace{0.05cm}M})$&nbsp; and&nbsp; $H(Z_{Y,\hspace{0.05cm}M})$&nbsp; can be determined in the conventional way according to the page&nbsp; [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Probability_mass_function_and_entropy|Probability mass function and entropy]]&nbsp;.
+From this, the entropies&nbsp; $H(Z_{X,\hspace{0.05cm}M})$&nbsp; and&nbsp; $H(Z_{Y,\hspace{0.05cm}M})$&nbsp; can be determined in the conventional way according to the section&nbsp; [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Probability_mass_function_and_entropy|Probability mass function and entropy]]&nbsp;.
 In the section&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_continuous_random_variables_after_quantization|Entropy of value-ontinuous random variables after quantization]],&nbsp; an approximation was also given.&nbsp; For example:
@@ Line 33: / Line 33: @@
 Hints:
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Differentielle_Entropie|Differential Entropy]].
-*Useful hints for solving this task can be found in particular on the page&nbsp;  [[Information_Theory/Differentielle_Entropie#Entropy_of_continuous_random_variables_after_quantization|Entropy of continuous random variables after quantization]].
+*Useful hints for solving this task can be found in particular in the section&nbsp;  [[Information_Theory/Differentielle_Entropie#Entropy_of_continuous_random_variables_after_quantization|Entropy of continuous random variables after quantization]].
@@ Line 72: / Line 72: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; According to the corresponding theory page, with &nbsp;$x_{\rm min} = 0$&nbsp; and &nbsp;$x_{\rm max} = 1/2$:
+'''(1)'''&nbsp; According to the corresponding theory section, with &nbsp;$x_{\rm min} = 0$&nbsp; and &nbsp;$x_{\rm max} = 1/2$:
 :$$h(X) = {\rm log}_2 \hspace{0.1cm} (x_{\rm max} - x_{\rm min}) = {\rm log}_2 \hspace{0.1cm} (1/2) \hspace{0.15cm}\underline{= - 1\,{\rm bit}}\hspace{0.05cm}.$$

@@ Line 9: / Line 9: @@
-We also consider two value-discrete random variables:
+We also consider two discrete random variables:
 *The variable&nbsp; $Z_{X,\hspace{0.05cm}M}$&nbsp; is obtained by (suitably) quantizing the random quantity&nbsp; $X$&nbsp;with the quantization level number&nbsp; $M$  <br>&#8658; &nbsp; quantization interval width&nbsp; ${\it \Delta} = 0.5/M$.
 * The variable&nbsp; $Z_{Y,\hspace{0.05cm}M}$&nbsp; is obtained after quantization of the random quantity&nbsp; $Y$&nbsp; with the quantization level number&nbsp; $M$ &nbsp; <br>&#8658; &nbsp; quantization interval width&nbsp; ${\it \Delta} = 2/M$.
@@ Line 49: / Line 49: @@
 $ h(Y) \ = \ $  { 1 3% } $\ \rm bit$
-{Calculate the entropy of the value-discrete  random variables&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; using the direct method</u>.
+{Calculate the entropy of the discrete  random variables&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; using the direct method</u>.
 |type="{}"}
 $H(Z_{X,\hspace{0.05cm}M=4})\ = \ $ { 2 3% } $\ \rm bit$
-{Calculate the entropy of the value-discrete random variables&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; using the given approximation</u>.
+{Calculate the entropy of the discrete random variables&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; using the given approximation</u>.
 |type="{}"}
 $H(Z_{X,\hspace{0.05cm}M=4})\ = \ $ { 2 3% } $\ \rm bit$
-{Calculate the entropy of the value-discrete random variable&nbsp; $Z_{Y,\hspace{0.05cm}M=8}$&nbsp; with the given approximation</u>.
+{Calculate the entropy of the discrete random variable&nbsp; $Z_{Y,\hspace{0.05cm}M=8}$&nbsp; with the given approximation</u>.
 |type="{}"}
 $H(Z_{Y,\hspace{0.05cm}M=8})\ = \ $ { 3 3% } $\ \rm bit$
@@ Line 115: / Line 115: @@
 '''(6)'''&nbsp; Only <u>statement 1</u>&nbsp; is correct:
-* The entropy&nbsp; $H(Z)$&nbsp; of a value-discrete random variable&nbsp; $Z = \{z_1, \ \text{...} \ , z_M\}$&nbsp; is never negative.
+* The entropy&nbsp; $H(Z)$&nbsp; of a discrete random variable&nbsp; $Z = \{z_1, \ \text{...} \ , z_M\}$&nbsp; is never negative.
 *For example, the limiting case&nbsp; $H(Z) = 0$&nbsp; results for &nbsp;${\rm Pr}(Z = z_1) = 1$&nbsp; and &nbsp;${\rm Pr}(Z = z_\mu) = 0$&nbsp; for &nbsp;$2 \le \mu \le M$.

@@ Line 14: / Line 14: @@
-The probability density functions&nbsp; $\rm (PDF)$&nbsp; of these discrete random variables are each composed of&nbsp; $M$&nbsp; Dirac functions whose momentum weights are given by the interval areas of the associated value-continuous random variables.
+The probability density functions&nbsp; $\rm (PDF)$&nbsp; of these discrete random variables are each composed of&nbsp; $M$&nbsp; Dirac functions whose momentum weights are given by the interval areas of the associated continuous random variables.
 From this, the entropies&nbsp; $H(Z_{X,\hspace{0.05cm}M})$&nbsp; and&nbsp; $H(Z_{Y,\hspace{0.05cm}M})$&nbsp; can be determined in the conventional way according to the page&nbsp; [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Probability_mass_function_and_entropy|Probability mass function and entropy]]&nbsp;.
-In the section&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|Entropy of value-ontinuous random variables after quantization]],&nbsp; an approximation was also given.&nbsp; For example:
+In the section&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_continuous_random_variables_after_quantization|Entropy of value-ontinuous random variables after quantization]],&nbsp; an approximation was also given.&nbsp; For example:
 :$$H(Z_{X, \hspace{0.05cm}M}) \approx  -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(X)\hspace{0.05cm}. $$
 *In the course of the task it will be shown that in the case of rectangular PDF  &nbsp; &#8658; &nbsp; uniform distribution this&nbsp; "approximation"&nbsp; gives the same result as the direct calculation.
-*But in the general case &ndash; so in&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|$\text{Example 2}$]]&nbsp; with triangular PDF &ndash; this equation is in fact only an approximation, which agrees with the actual entropy&nbsp;  $H(Z_{X,\hspace{0.05cm}M})$&nbsp; only in the limiting case &nbsp; ${\it \Delta} \to 0$&nbsp;.
+*But in the general case &ndash; so in&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_continuous_random_variables_after_quantization|$\text{Example 2}$]]&nbsp; with triangular PDF &ndash; this equation is in fact only an approximation, which agrees with the actual entropy&nbsp;  $H(Z_{X,\hspace{0.05cm}M})$&nbsp; only in the limiting case &nbsp; ${\it \Delta} \to 0$&nbsp;.
@@ Line 33: / Line 33: @@
 Hints:
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Differentielle_Entropie|Differential Entropy]].
-*Useful hints for solving this task can be found in particular on the page&nbsp;  [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|Entropy of value-continuous random variables after quantization]].
+*Useful hints for solving this task can be found in particular on the page&nbsp;  [[Information_Theory/Differentielle_Entropie#Entropy_of_continuous_random_variables_after_quantization|Entropy of continuous random variables after quantization]].

@@ Line 4: / Line 4: @@
 [[File:P_ID2878__Inf_A_4_4.png|right|frame|Two uniform distributions]]
-We consider the two continuous-valued random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; with probability density functions $f_X(x)$&nbsp; and $f_Y(y)$.&nbsp; For these random variables one can
+We consider the two continuous random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; with probability density functions $f_X(x)$&nbsp; and $f_Y(y)$.&nbsp; For these random variables one can
 * not specify the conventional entropies&nbsp; $H(X)$&nbsp; and&nbsp; $H(Y)$&nbsp;, respectively,
 * but the differential entropies&nbsp; $h(X)$&nbsp; and&nbsp; $h(Y)$.

@@ Line 85: / Line 85: @@
 '''(3)'''&nbsp; The adjacent graph illustrates the best possible quantization of random variable&nbsp; $X$&nbsp; with quantization level number&nbsp; $M = 4$&nbsp; &nbsp; &#8658; &nbsp; random variable&nbsp; $Z_{X, \ M = 4}$:
 *The interval width here is equal to &nbsp;${\it \Delta} = 0.5/4 = 1/8$.
-*The possible values&nbsp; (at the center of the interval, respectively)&nbsp; are &nbsp;$z \in \{0.0625,\ 0.1875,\ 0.3125,\ 0.4375\}$.
+*The possible values&nbsp; (at the center of the interval,&nbsp; respectively)&nbsp; are &nbsp;$z \in \{0.0625,\ 0.1875,\ 0.3125,\ 0.4375\}$.
-Using the probability function, the <u>direct entropy calculation</u> gives $P_Z(Z) = \big [1/4,\ \text{...} , \ 1/4 \big]$:
+Using the probability mass function, the <u>direct entropy calculation</u> gives $P_Z(Z) = \big [1/4,\ \text{...} , \ 1/4 \big]$:
 :$$H(Z_{X, \ M = 4}) = {\rm log}_2 \hspace{0.1cm} (4) \hspace{0.15cm}\underline{= 2\,{\rm bit}}
 \hspace{0.05cm}.$$
@@ Line 102: / Line 102: @@
 * The quantization parameter is now &nbsp;${\it \Delta}  = 2/4 = 1/2$.
 * The possible values are now &nbsp;$z \in \{\pm 0.75,\ \pm 0.25\}$.
-* Thus, here the "approximation"&nbsp; (as well as the direct calculation)&nbsp; gives the result:
+* Thus, here the&nbsp; "approximation"&nbsp; (as well as the direct calculation)&nbsp; gives the result:
 :$$H(Z_{Y,\hspace{0.05cm} M = 4})   \approx    -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(Y) =
 \,{\rm bit} + 1\,{\rm bit}\hspace{0.15cm}\underline{= 2\,{\rm bit}}\hspace{0.05cm}.$$
@@ Line 108: / Line 108: @@
 [[File:P_ID2881__Inf_A_4_4e.png|right|frame|Quantized random variable&nbsp;  $Z_{Y, \ M = 8}$]]
-'''(5)'''&nbsp; In contrast to subtask&nbsp; '''(5)'''&nbsp; &nbsp;${\it \Delta}  = 1/4$ is now valid.&nbsp; From this follows for the "approximation":
+'''(5)'''&nbsp; In contrast to subtask&nbsp; '''(5)''':&nbsp; &nbsp;${\it \Delta}  = 1/4$&nbsp; is now valid.&nbsp; From this follows for the "approximation":
 :$$H(Z_{Y,\hspace{0.05cm} M = 8})   \approx    -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(Y) =
 \,{\rm bit} + 1\,{\rm bit}\hspace{0.15cm}\underline{= 3\,{\rm bit}}\hspace{0.05cm}.$$
 Again, one gets the same result as in the direct calculation.
-<br clear=all>
 '''(6)'''&nbsp; Only <u>statement 1</u> is correct:
-* The entropy&nbsp; $H(Z)$&nbsp; of a discrete random variable&nbsp; $Z = \{z_1, \ \text{...} \ , z_M\}$&nbsp; is never negative.
+'''(6)'''&nbsp; Only <u>statement 1</u>&nbsp; is correct:
 *For example, the limiting case&nbsp; $H(Z) = 0$&nbsp; results for &nbsp;${\rm Pr}(Z = z_1) = 1$&nbsp; and &nbsp;${\rm Pr}(Z = z_\mu) = 0$&nbsp; for &nbsp;$2 \le \mu \le M$.

@@ Line 37: / Line 37: @@
-===Fragebogen===
+===Questions===
 <quiz display=simple>
-{Berechnen Sie die differentielle Entropie&nbsp; $h(X)$.
+{Calculate the differential entropy&nbsp; $h(X)$.
 |type="{}"}
 $ h(X) \ = \ $ { -1.03--0.97 } $\ \rm bit$
-{Berechnen Sie die differentielle Entropie $h(Y)$.
+{Calculate the differential entropy $h(Y)$.
 |type="{}"}
 $ h(Y) \ = \ $  { 1 3% } $\ \rm bit$
-{Berechnen Sie die Entropie der wertdiskreten Zufallsgrößen&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; nach der direkten Methode</u>.
+{Calculate the entropy of the discrete value random variables&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; using the direct method</u>.
 |type="{}"}
 $H(Z_{X,\hspace{0.05cm}M=4})\ = \ $ { 2 3% } $\ \rm bit$
-{Berechnen Sie die Entropie der wertdiskreten Zufallsgrößen&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; mit der angegebenen Näherung</u>.
+{Calculate the entropy of the discrete value random variables&nbsp; $Z_{X,\hspace{0.05cm}M=4}$&nbsp; using the given approximation</u>.
 |type="{}"}
 $H(Z_{X,\hspace{0.05cm}M=4})\ = \ $ { 2 3% } $\ \rm bit$
-{Berechnen Sie die Entropie der wertdiskreten Zufallsgröße&nbsp; $Z_{Y,\hspace{0.05cm}M=8}$&nbsp; mit der angegebenen Näherung</u>.
+{Calculate the entropy of the discrete value random variable&nbsp; $Z_{Y,\hspace{0.05cm}M=8}$&nbsp; with the given approximation</u>.
 |type="{}"}
 $H(Z_{Y,\hspace{0.05cm}M=8})\ = \ $ { 3 3% } $\ \rm bit$
-{Welche der folgenden Aussagen sind zutreffend?
+{Which of the following statements is true?
 |type="[]"}
-+ Die Entropie einer wertdiskreten Zufallsgröße&nbsp; $Z$&nbsp; ist stets&nbsp; $H(Z) \ge 0$.
++ The entropy of a discrete value random variable&nbsp; $Z$&nbsp; is always&nbsp; $H(Z) \ge 0$.
-- Die differenzielle Entropie einer wertkontinuierlichen Zufallsgröße&nbsp; $X$&nbsp; ist stets&nbsp; $h(X) \ge 0$.
+- The differential entropy of a continuous value random variable&nbsp; $X$&nbsp; is always&nbsp; $h(X) \ge 0$.
@@ Line 70: / Line 70: @@
 </quiz>
-===Musterlösung===
+===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; Gemäß der entsprechenden Theorieseite gilt mit &nbsp;$x_{\rm min} = 0$&nbsp; und &nbsp;$x_{\rm max} = 1/2$:
+'''(1)'''&nbsp; According to the corresponding theory page, with &nbsp;$x_{\rm min} = 0$&nbsp; and &nbsp;$x_{\rm max} = 1/2$:
 :$$h(X) = {\rm log}_2 \hspace{0.1cm} (x_{\rm max} - x_{\rm min}) = {\rm log}_2 \hspace{0.1cm} (1/2) \hspace{0.15cm}\underline{= - 1\,{\rm bit}}\hspace{0.05cm}.$$
-'''(2)'''&nbsp; Mit &nbsp;$y_{\rm min} = -1$&nbsp; und &nbsp;$y_{\rm max} = +1$&nbsp; ergibt sich dagegen für die differentielle Entropie der Zufallsgröße&nbsp; $Y$:
+'''(2)'''&nbsp; On the other hand, with &nbsp;$y_{\rm min} = -1$&nbsp; and &nbsp;$y_{\rm max} = +1$&nbsp; the differential entropy of the random variable&nbsp; $Y$ is given by:
 :$$h(Y) = {\rm log}_2 \hspace{0.1cm} (y_{\rm max} - y_{\rm min}) = {\rm log}_2 \hspace{0.1cm} (2) \hspace{0.15cm}\underline{= + 1\,{\rm bit}}\hspace{0.05cm}. $$
-[[File:P_ID2879__Inf_A_4_4c.png|right|frame|Quantisierte Zufallsgröße&nbsp;  $Z_{X, \ M = 4}$]]
+[[File:P_ID2879__Inf_A_4_4c.png|right|frame|Quantized random variable&nbsp;  $Z_{X, \ M = 4}$]]
-'''(3)'''&nbsp; Die nebenstehende Grafik verdeutlicht die bestmögliche Quantisierung der Zufallsgröße&nbsp; $X$&nbsp; mit der Quantisierungsstufenzahl&nbsp; $M = 4$&nbsp; &nbsp; &#8658; &nbsp; Zufallsgröße&nbsp; $Z_{X, \ M = 4}$:
+'''(3)'''&nbsp; The adjacent graph illustrates the best possible quantization of random variable&nbsp; $X$&nbsp; with quantization level number&nbsp; $M = 4$&nbsp; &nbsp; &#8658; &nbsp; random variable&nbsp; $Z_{X, \ M = 4}$:
-*Die Intervallbreite ist hier gleich &nbsp;${\it \Delta} = 0.5/4 = 1/8$.
+*The interval width here is equal to &nbsp;${\it \Delta} = 0.5/4 = 1/8$.
-*Die möglichen Werte&nbsp; (jeweils in der Intervallmitte)&nbsp; sind &nbsp;$z \in \{0.0625,\ 0.1875,\ 0.3125,\ 0.4375\}$.
+*The possible values&nbsp; (at the center of the interval, respectively)&nbsp; are &nbsp;$z \in \{0.0625,\ 0.1875,\ 0.3125,\ 0.4375\}$.
-Die <u>direkte Entropieberechnung</u> ergibt mit der Wahrscheinlichkeitsfunktion $P_Z(Z) = \big [1/4,\ \text{...} , \ 1/4 \big]$:
+Using the probability function, the <u>direct entropy calculation</u> gives $P_Z(Z) = \big [1/4,\ \text{...} , \ 1/4 \big]$:
 :$$H(Z_{X, \ M = 4}) = {\rm log}_2 \hspace{0.1cm} (4) \hspace{0.15cm}\underline{= 2\,{\rm bit}}
 \hspace{0.05cm}.$$
-Mit der <u>Näherung</u> erhält man unter Berücksichtigung des Ergebnisses von&nbsp; '''(1)''':
+With the <u>approximation</u>, considering the result of&nbsp; '''(1)''', we obtain:
 :$$H(Z_{X,\hspace{0.05cm} M = 4}) \approx  -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(X) =
 \,{\rm bit} +(- 1\,{\rm bit})\hspace{0.15cm}\underline{= 2\,{\rm bit}}\hspace{0.05cm}. $$
-<i>Hinweis:</i>&nbsp; Nur bei der Gleichverteilung liefert die Näherung genau das gleiche Ergebnis wie die direkte Berechnung, also die tatsächliche Entropie.
+<i>Note:</i>&nbsp; Only in the case of uniform distribution, the approximation gives exactly the same result as the direct calculation, i.e. the actual entropy.
-[[File:P_ID2880__Inf_A_4_4d.png|right|frame|Quantisierte Zufallsgröße  $Z_{Y, \ M = 4}$]]
+[[File:P_ID2880__Inf_A_4_4d.png|right|frame|Quantized random variable  $Z_{Y, \ M = 4}$]]
 <br>
-'''(4)'''&nbsp; Aus der zweiten Grafik erkennt man die Gemeinsamkeiten / Unterschiede zur Teilaufgabe&nbsp; '''(3)''':
+'''(4)'''&nbsp; From the second graph, one can see the similarities / differences to subtask&nbsp; '''(3)''':
-* Der Quantisierungsparameter ist nun &nbsp;${\it \Delta}  = 2/4 = 1/2$.
+* The quantization parameter is now &nbsp;${\it \Delta}  = 2/4 = 1/2$.
-* Die möglichen Werte sind nun &nbsp;$z \in \{\pm 0.75,\ \pm 0.25\}$.
+* The possible values are now &nbsp;$z \in \{\pm 0.75,\ \pm 0.25\}$.
-* Somit liefert hier die "Näherung"&nbsp; (ebenso wie die direkte Berechnung)&nbsp; das Ergebnis:
+* Thus, here the "approximation"&nbsp; (as well as the direct calculation)&nbsp; gives the result:
 :$$H(Z_{Y,\hspace{0.05cm} M = 4})   \approx    -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(Y) =
 \,{\rm bit} + 1\,{\rm bit}\hspace{0.15cm}\underline{= 2\,{\rm bit}}\hspace{0.05cm}.$$
-[[File:P_ID2881__Inf_A_4_4e.png|right|frame|Quantisierte Zufallsgröße&nbsp;  $Z_{Y, \ M = 8}$]]
+[[File:P_ID2881__Inf_A_4_4e.png|right|frame|Quantized random variable&nbsp;  $Z_{Y, \ M = 8}$]]
-'''(5)'''&nbsp; Im Gegensatz zur Teilaufgabe&nbsp; '''(5)'''&nbsp; gilt nun &nbsp;${\it \Delta}  = 1/4$.&nbsp; Daraus folgt für die "Näherung":
+'''(5)'''&nbsp; In contrast to subtask&nbsp; '''(5)'''&nbsp; &nbsp;${\it \Delta}  = 1/4$ is now valid.&nbsp; From this follows for the "approximation":
 :$$H(Z_{Y,\hspace{0.05cm} M = 8})   \approx    -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(Y) =
 \,{\rm bit} + 1\,{\rm bit}\hspace{0.15cm}\underline{= 3\,{\rm bit}}\hspace{0.05cm}.$$
-Man erhält wieder das gleiche  Ergebnis wie bei der direkten Berechnung.
+Again, one gets the same result as in the direct calculation.
 <br clear=all>
-'''(6)'''&nbsp; Richtig ist nur die <u>Aussage 1</u>:
+'''(6)'''&nbsp; Only <u>statement 1</u> is correct:
-* Die Entropie&nbsp; $H(Z)$&nbsp; einer diskreten Zufallsgröße&nbsp; $Z = \{z_1, \ \text{...} \ , z_M\}$&nbsp; ist nie negativ.
+* The entropy&nbsp; $H(Z)$&nbsp; of a discrete random variable&nbsp; $Z = \{z_1, \ \text{...} \ , z_M\}$&nbsp; is never negative.
-*Der Grenzfall&nbsp; $H(Z) = 0$&nbsp; ergibt sich zum Beispiel für &nbsp;${\rm Pr}(Z = z_1) = 1$&nbsp; und &nbsp;${\rm Pr}(Z = z_\mu) = 0$&nbsp; für &nbsp;$2 \le \mu \le M$.
+*For example, the limiting case&nbsp; $H(Z) = 0$&nbsp; results for &nbsp;${\rm Pr}(Z = z_1) = 1$&nbsp; and &nbsp;${\rm Pr}(Z = z_\mu) = 0$&nbsp; for &nbsp;$2 \le \mu \le M$.
-* Dagegen kann die differentielle Entropie&nbsp; $h(X)$&nbsp; einer wertkontinuierlichen Zufallsgröße&nbsp; $X$&nbsp; wie folgt sein:
+* In contrast, the differential entropy&nbsp; $h(X)$&nbsp; of a continuous value random variable&nbsp; $X$&nbsp; can be as follows:
-** $h(X) < 0$&nbsp; $($Teilaufgabe 1$)$,
+** $h(X) < 0$&nbsp; $($subtask 1$)$,
-** $h(X) > 0$&nbsp; $($Teilaufgabe 2$)$, oder auch
+** $h(X) > 0$&nbsp; $($subtask 2$)$, or even
-**$h(X) = 0$&nbsp;  $($zum Beispiel für &nbsp;$x_{\rm min} = 0$&nbsp; und  &nbsp;$x_{\rm max} = 1)$.
+**$h(X) = 0$&nbsp;  $($for example fo &nbsp;$x_{\rm min} = 0$&nbsp; and  &nbsp;$x_{\rm max} = 1)$.
 {{ML-Fuß}}

@@ Line 9: / Line 9: @@
-We also consider two discrete value random variables:
+We also consider two value-discrete random variables:
-*The random variable&nbsp; $Z_{X,\hspace{0.05cm}M}$&nbsp; is obtained by (suitably) quantizing the random variable&nbsp; $X$&nbsp;with the quantization step number&nbsp; $M$  <br>&#8658; &nbsp; quantization interval width&nbsp; ${\it \Delta} = 0.5/M$.
+*The variable&nbsp; $Z_{X,\hspace{0.05cm}M}$&nbsp; is obtained by (suitably) quantizing the random quantity&nbsp; $X$&nbsp;with the quantization level number&nbsp; $M$  <br>&#8658; &nbsp; quantization interval width&nbsp; ${\it \Delta} = 0.5/M$.
-* The random quantity&nbsp; $Z_{Y,\hspace{0.05cm}M}$&nbsp; is obtained after quantization of the random quantity&nbsp; $Y$&nbsp; with the quantization level number&nbsp; $M$ &nbsp; <br>&#8658; &nbsp; quantization interval width&nbsp; ${\it \Delta} = 2/M$.
+* The variable&nbsp; $Z_{Y,\hspace{0.05cm}M}$&nbsp; is obtained after quantization of the random quantity&nbsp; $Y$&nbsp; with the quantization level number&nbsp; $M$ &nbsp; <br>&#8658; &nbsp; quantization interval width&nbsp; ${\it \Delta} = 2/M$.
-The probability density functions of these discrete random variables are each composed of&nbsp; $M$&nbsp; Dirac functions whose momentum weights are given by the interval areas of the associated continuous-value random variables.
+The probability density functions&nbsp; $\rm (PDF)$&nbsp; of these discrete random variables are each composed of&nbsp; $M$&nbsp; Dirac functions whose momentum weights are given by the interval areas of the associated value-continuous random variables.
 From this, the entropies&nbsp; $H(Z_{X,\hspace{0.05cm}M})$&nbsp; and&nbsp; $H(Z_{Y,\hspace{0.05cm}M})$&nbsp; can be determined in the conventional way according to the chapter&nbsp; [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Probability_mass_function_and_entropy|Probability Mass Function and Entropy]]&nbsp;.
-In the section&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|Entropy of Continuous-Value Random Variables after Quantization]]&nbsp; , an approximation was also given.&nbsp; For example:
+In the section&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|Entropy of Continuous-Value Random Variables after Quantization]],&nbsp; an approximation was also given.&nbsp; For example:
 :$$H(Z_{X, \hspace{0.05cm}M}) \approx  -{\rm log}_2 \hspace{0.1cm} ({\it \Delta}) + h(X)\hspace{0.05cm}. $$
-*In the course of the task it will be shown that in the case of rectangular PDF  &nbsp; &#8658; &nbsp; uniform distribution this "approximation" gives the same result as the direct calculation.
+*In the course of the task it will be shown that in the case of rectangular PDF  &nbsp; &#8658; &nbsp; uniform distribution this&nbsp; "approximation"&nbsp; gives the same result as the direct calculation.
-*But in the general case &ndash; so in&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|$\text{Example 2}$]]&nbsp; with triangular PDF &ndash; above equation is in fact only an approximation, which agrees with the actual entropy&nbsp;  $H(Z_{X,\hspace{0.05cm}M})$&nbsp; only in the limiting case &nbsp; ${\it \Delta} \to 0$&nbsp;.
+*But in the general case &ndash; so in&nbsp; [[Information_Theory/Differentielle_Entropie#Entropy_of_value-continuous_random_variables_after_quantization|$\text{Example 2}$]]&nbsp; with triangular PDF &ndash; this equation is in fact only an approximation, which agrees with the actual entropy&nbsp;  $H(Z_{X,\hspace{0.05cm}M})$&nbsp; only in the limiting case &nbsp; ${\it \Delta} \to 0$&nbsp;.