Aufgaben:Exercise 3.7: Some Entropy Calculations - Revision history

Noah at 08:15, 24 September 2021

2021-09-24T08:15:56Z

Noah at 10:09, 14 September 2021

2021-09-14T10:09:30Z

Guenter at 10:13, 1 September 2021

2021-09-01T10:13:02Z

Guenter at 10:11, 1 September 2021

2021-09-01T10:11:02Z

Guenter at 10:02, 1 September 2021

2021-09-01T10:02:54Z

Guenter at 10:00, 1 September 2021

2021-09-01T10:00:25Z

Guenter at 09:59, 1 September 2021

2021-09-01T09:59:13Z

Noah at 19:49, 26 August 2021

2021-08-26T19:49:44Z

Show changes

Guenter at 16:49, 20 July 2021

2021-07-20T16:49:44Z

Guenter: Guenter moved page Aufgaben:Aufgabe 3.7: Einige Entropieberechnungen to Aufgaben:Exercise 3.7: Some Entropy Calculations

2021-07-20T16:42:56Z

Guenter moved page Aufgabe 3.7: Einige Entropieberechnungen to Exercise 3.7: Some Entropy Calculations

← Older revision		Revision as of 08:15, 24 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Verschiedene Entropien zweidimensionaler Zufallsgrößen~~	+	{{quiz-Header\|Buchseite=Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables
	}}		}}

@@ Line 8: / Line 8: @@
 :$$P_{UV}(U, V) \hspace{0.05cm}= \begin{pmatrix} 0.068 & 0.132\\ 0.272 & 0.528 \end{pmatrix}\hspace{0.05cm}$$
-For the random variable&nbsp; $XY$&nbsp; the following are to be calculated in this task:
+For the random variable&nbsp; $XY$&nbsp; the following are to be calculated in this exercise:
 * the joint entropy:
 :$$H(XY) = -{\rm E}\big [\log_2  P_{ XY }( X,Y) \big ],$$

@@ Line 106: / Line 106: @@
-'''(5)'''&nbsp; According to the diagram on the right, statements 1, 2 and 4 are correct:
+'''(5)'''&nbsp; According to the diagram on the right, <br>statements 1, 2 and 4 are correct:
 *One recognises the validity of&nbsp; $P_{ UV } = P_U  · P_V$  &nbsp; &rArr; &nbsp;   mutual information $I(U; V) = 0$&nbsp; by the fact that the second row of the&nbsp; $P_{ UV }$ matrix differs from the first row only by a constant factor&nbsp; $(4)$&nbsp;.
-*This results in the same one-dimensional probability functions as for the random variable&nbsp; $XY$ &nbsp; &rArr; &nbsp;  $P_U(U) = \big [0.2, \ 0.8 \big ]$ &nbsp;and&nbsp;  $P_V(V) = \big [0.34, \ 0.66 \big ]$.
+*This results in the same one-dimensional probability mass functions as for the random variable&nbsp; $XY$ &nbsp; &rArr; &nbsp;  $P_U(U) = \big [0.2, \ 0.8 \big ]$ &nbsp;and&nbsp;  $P_V(V) = \big [0.34, \ 0.66 \big ]$.
 *Therefore&nbsp; $H(U) = H(X) = 0.722\ \rm  bit$ &nbsp;and&nbsp; $H(V) = H(Y) = 0.925 \ \rm bit$.
 *Here, however, the following now applies for the joint entropy: &nbsp; $H(UV) = H(U) + H(V) ≠ H(XY)$.

@@ Line 81: / Line 81: @@
-'''(2)'''&nbsp; The 1D probability functions are&nbsp; $P_X(X) = \big [0.2, \ 0.8 \big ]$&nbsp; and&nbsp; $P_Y(Y) = \big [0.34, \ 0.66 \big ]$.&nbsp; From this follows:
+'''(2)'''&nbsp; The one-dimensional probability functions are&nbsp; $P_X(X) = \big [0.2, \ 0.8 \big ]$&nbsp; and&nbsp; $P_Y(Y) = \big [0.34, \ 0.66 \big ]$.&nbsp; From this follows:
 :$$H(X)  = 0.2 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.2} + 0.8\cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.8}\hspace{0.15cm} \underline {= 0.722\,{\rm (bit)}} \hspace{0.05cm},$$
 :$$H(Y) =0.34 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.34} + 0.66\cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.66}\hspace{0.15cm} \underline {= 0.925\,{\rm (bit)}} \hspace{0.05cm}.$$
@@ Line 98: / Line 98: @@
 [[File:P_ID2767__Inf_A_3_6d.png|right|frame|Entropy values for the random variables $XY$ and $UV$]]
-*The graph on the left summarises the results of subtasks&nbsp; '''(1)''', ... , &nbsp;'''(4)'''&nbsp; true to scale.
+*The  left diagram summarises the results of subtasks&nbsp; '''(1)''', ... , &nbsp;'''(4)'''&nbsp; true to scale.
 *The joint entropy is highlighted in grey and the mutual information in yellow.
-*A red background refers to the random variable&nbsp; $X$, a green one to&nbsp; $Y$.&nbsp; Hatched fields indicate conditional entropy.
+*A red background refers to the random variable&nbsp; $X$,&nbsp; and a green one to&nbsp; $Y$.&nbsp; Hatched fields indicate a  conditional entropy.
@@ Line 106: / Line 106: @@
-'''(5)'''&nbsp; According to the diagram, statements 1, 2 and 4 are correct:
+'''(5)'''&nbsp; According to the diagram on the right, statements 1, 2 and 4 are correct:
 *One recognises the validity of&nbsp; $P_{ UV } = P_U  · P_V$  &nbsp; &rArr; &nbsp;   mutual information $I(U; V) = 0$&nbsp; by the fact that the second row of the&nbsp; $P_{ UV }$ matrix differs from the first row only by a constant factor&nbsp; $(4)$&nbsp;.
-*This results in the same 1D probability functions as for the random variable&nbsp; $XY$ &nbsp; &rArr; &nbsp;  $P_U(U) = \big [0.2, \ 0.8 \big ]$ &nbsp;and&nbsp;  $P_V(V) = \big [0.34, \ 0.66 \big ]$.
+*This results in the same one-dimensional probability functions as for the random variable&nbsp; $XY$ &nbsp; &rArr; &nbsp;  $P_U(U) = \big [0.2, \ 0.8 \big ]$ &nbsp;and&nbsp;  $P_V(V) = \big [0.34, \ 0.66 \big ]$.
-*Therefore&nbsp; $H(U) = H(X) = 0.722\ \rm  bit$ &nbsp;und&nbsp; $H(V) = H(Y) = 0.925 \ \rm bit$.
+*Therefore&nbsp; $H(U) = H(X) = 0.722\ \rm  bit$ &nbsp;and&nbsp; $H(V) = H(Y) = 0.925 \ \rm bit$.
 *Here, however, the following now applies for the joint entropy: &nbsp; $H(UV) = H(U) + H(V) ≠ H(XY)$.

@@ Line 43: / Line 43: @@
 $H(XY) \ = \ $  { 1.393 3% } $\ \rm bit$
-{What are the entropies of the 1D random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp;?
+{What are the entropies of the one-dimensional random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp;?
 |type="{}"}
 $H(X) \ = \ $ { 0.722 3% } $\ \rm bit$
 $H(Y) \ = \ $ { 0.925 3% } $\ \rm bit$
-{How large is the transinformation between the random variables&nbsp; $X$&nbsp; and&nbsp; $Y$?
+{How large is the mutual information between the random variables&nbsp; $X$&nbsp; and&nbsp; $Y$?
 |type="{}"}
 $I(X; Y) \ = \ $ { 0.254 3% } $\ \rm bit$
@@ Line 58: / Line 58: @@
-{Which of the following statements are true for the 2D random variable $UV$?
+{Which of the following statements are true for the two-dimensional random variable $UV$?
 |type="[]"}
-+ The 1D random variables&nbsp; $U$&nbsp; and&nbsp; $V$&nbsp; are statistically independent.
++ The one-dimensional random variables&nbsp; $U$&nbsp; and&nbsp; $V$&nbsp; are statistically independent.
 + The mutual information of&nbsp; $U$&nbsp; and&nbsp; $V$&nbsp;   is&nbsp;  $I(U; V) = 0$.
 - For the compound entropy&nbsp; $H(UV) = H(XY)$ holds.

@@ Line 31: / Line 31: @@
 Hints:
-*The exercise belongs to the chapter&nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables|Different entropies of two-dimensional random variables]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables|Different entropy measures of two-dimensional random variables]].
 *In particular, reference is made to the pages&nbsp; <br> &nbsp; &nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Conditional_probability_and_conditional_entropy|Conditional probability and conditional entropy]] &nbsp; as well as <br> &nbsp; &nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Mutual_information_between_two_random_variables|Mutual information between two random variables]].

@@ Line 4: / Line 4: @@
 [[File:P_ID2766__Inf_A_3_6.png|right|frame|Diagram: Entropies and information]]
-We consider the two random variables&nbsp; $XY$&nbsp; and&nbsp; $UV$&nbsp; with the following 2D probability functions:
+We consider the two random variables&nbsp; $XY$&nbsp; and&nbsp; $UV$&nbsp; with the following two-dimensional probability mass functions:
 :$$P_{XY}(X, Y) = \begin{pmatrix} 0.18 & 0.16\\ 0.02 & 0.64 \end{pmatrix}\hspace{0.05cm} \hspace{0.05cm}$$
 :$$P_{UV}(U, V) \hspace{0.05cm}= \begin{pmatrix} 0.068 & 0.132\\ 0.272 & 0.528 \end{pmatrix}\hspace{0.05cm}$$
@@ Line 31: / Line 31: @@
 Hints:
-*The task belongs to the chapter&nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen|Different entropies of two-dimensional random variables]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables|Different entropies of two-dimensional random variables]].
-*In particular, reference is made to the pages&nbsp; <br> &nbsp; &nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen#Conditional_probability_and_conditional_entropy|Conditional probability and conditional entropy]] &nbsp; as well as <br> &nbsp; &nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen#Mutual_information_between_two_random_variables|Mutual information between two random variables]].
+*In particular, reference is made to the pages&nbsp; <br> &nbsp; &nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Conditional_probability_and_conditional_entropy|Conditional probability and conditional entropy]] &nbsp; as well as <br> &nbsp; &nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Mutual_information_between_two_random_variables|Mutual information between two random variables]].

← Older revision		Revision as of 16:49, 20 July 2021
Line 116:		Line 116:


−	[[Category:Information Theory: Exercises\|^3.2 ~~Entropien von~~ 2D~~-Zufallsgrößen~~^]]	+	[[Category:Information Theory: Exercises\|^3.2 Entropies of 2D Random Variables^]]

← Older revision	Revision as of 16:42, 20 July 2021
(No difference)