Aufgaben:Exercise 3.8Z: Tuples from Ternary Random Variables - Revision history

Noah at 08:16, 24 September 2021

2021-09-24T08:16:14Z

Guenter at 13:19, 21 September 2021

2021-09-21T13:19:53Z

Guenter at 13:06, 21 September 2021

2021-09-21T13:06:57Z

Noah at 10:04, 14 September 2021

2021-09-14T10:04:25Z

Noah at 10:49, 13 September 2021

2021-09-13T10:49:59Z

Show changes

Guenter at 16:50, 20 July 2021

2021-07-20T16:50:47Z

Guenter: Guenter moved page Aufgaben:Aufgabe 3.8Z: Tupel aus ternären Zufallsgrößen to Aufgaben:Exercise 3.8Z: Tuples from Ternary Random Variables

2021-07-20T16:45:47Z

Guenter moved page Aufgabe 3.8Z: Tupel aus ternären Zufallsgrößen to Exercise 3.8Z: Tuples from Ternary Random Variables

Javier: Text replacement - "”" to """

2021-05-28T14:31:27Z

Text replacement - "”" to """

Javier: Text replacement - "„" to """

2021-05-28T14:30:13Z

Text replacement - "„" to """

Javier: Text replacement - "Category:Aufgaben zu Informationstheorie" to "Category:Information Theory: Exercises"

2021-03-23T12:48:00Z

Text replacement - "Category:Aufgaben zu Informationstheorie" to "Category:Information Theory: Exercises"

← Older revision		Revision as of 08:16, 24 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Verschiedene Entropien zweidimensionaler Zufallsgrößen~~	+	{{quiz-Header\|Buchseite=Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables

	}}		}}

@@ Line 62: / Line 62: @@
 \hspace{0.15cm}\underline{= 1.585\,{\rm (bit)}}\hspace{0.05cm}.$$
-*The 2D random variable&nbsp; $XY = \{00,\ 01,\ 02,\ 10,\ 11,\ 12,\ 20,\ 21,\ 22\}$  &nbsp; &rArr; &nbsp;  $|XY| = |Z| = 9$&nbsp; also has equal probabilities:
+*The two-dimensional random variable&nbsp; $XY = \{00,\ 01,\ 02,\ 10,\ 11,\ 12,\ 20,\ 21,\ 22\}$  &nbsp; &rArr; &nbsp;  $|XY| = |Z| = 9$&nbsp; has also equal probabilities:
 :$$p_{ 00 } = p_{ 01 } =\text{...} = p_{ 22 } = 1/9.$$
 *From this follows:
@@ Line 71: / Line 71: @@
 '''(2)'''&nbsp;  The random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; are statistically independent because of&nbsp; $P_{ XY }(⋅) = P_X(⋅) · P_Y(⋅)$&nbsp;.
 *From this follows&nbsp;  $I(X, Y)\hspace{0.15cm}\underline{ = 0}$.
-*The same result is obtained by the equation&nbsp; $I(X; Y) = H(X) + H(Y) – H(XY)$.
+*The same result is obtained by the equation&nbsp; $I(X; Y) = H(X) + H(Y) - H(XY)$.
-'''(3)'''&nbsp;  If one interprets&nbsp; $I(X; Z)$&nbsp; as the remaining uncertainty with regard to the tuple&nbsp; $Z$,when the first component&nbsp; $X$&nbsp; is known, then the following obviously applies
+'''(3)'''&nbsp;  If one interprets&nbsp; $I(X; Z)$&nbsp; as the remaining uncertainty with regard to the tuple&nbsp; $Z$,&nbsp; when the first component&nbsp; $X$&nbsp; is known,&nbsp; then the following obviously applies:
 :$$ I(X; Z) = H(Y)\hspace{0.15cm}\underline{  = 1.585 \ \rm bit}.$$
 In purely formal terms, this task can also be solved as follows:
-* The entropy&nbsp; $H(Z)$&nbsp; is equal to the joint entropy&nbsp; $H(XY) = 3.170 \ \rm bit$.
+* The entropy&nbsp; $H(Z)$&nbsp; is equal to the joint entropy&nbsp; $H(XY) = 3.17 \ \rm bit$.
 * The joint probability&nbsp; $P_{ XZ }(X, Z)$&nbsp; contains nine elements of probability&nbsp; $1/9$,&nbsp; all others are occupied by zeros &nbsp; &rArr; &nbsp;   $H(XZ) = \log_2 (9) = 3.170 \ \rm bit $.
-* Thus, the following applies to the mutual information&nbsp; $($mutual information of the random variables&nbsp; $X$&nbsp; and&nbsp; $Z)$:
+* Thus, the following applies to the mutual information of the random variables&nbsp; $X$&nbsp; and&nbsp; $Z$:
 :$$I(X;Z) = H(X) + H(Z) - H(XZ) = 1.585 + 3.170- 3.170\hspace{0.15cm} \underline {= 1.585\,{\rm (bit)}} \hspace{0.05cm}.$$
-[[File:P_ID2773__Inf_Z_3_7d.png|right|frame|Entropies of the 2D random variable&nbsp; $XZ$]]
+[[File:P_ID2773__Inf_Z_3_7d.png|right|frame|Entropies of the 2D variable&nbsp; $XZ$]]
 '''(4)'''&nbsp;  According to the second graph:
 :$$H(Z \hspace{-0.1cm}\mid \hspace{-0.1cm} X) = H(XZ) - H(X) = 3.170-1.585\hspace{0.15cm} \underline {=1.585\ {\rm (bit)}} \hspace{0.05cm},$$
 :$$H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Z)  = H(XZ) - H(Z) = 3.170-3.170\hspace{0.15cm} \underline {=0\ {\rm (bit)}}  \hspace{0.05cm}.$$
-* $H(Z|X)$&nbsp; gives the residual uncertainty with respect to the tuple&nbsp; $Z$ when the first componen&nbsp; $X$&nbsp; is known.
+* $H(Z|X)$&nbsp; gives the residual uncertainty with respect to the tuple&nbsp; $Z$,&nbsp; when the first componen&nbsp; $X$&nbsp; is known.
-* The uncertainty regarding the tuple&nbsp; $Z$&nbsp; is $H(Z) = 2 · \log_2 (3) \ \rm bit$.
+* The uncertainty regarding the tuple&nbsp; $Z$&nbsp; is&nbsp; $H(Z) = 2 · \log_2 (3) \ \rm bit$.
 * When the component&nbsp; $X$&nbsp; is known, the uncertainty is halved to&nbsp; $H(Z|X) = \log_2 (3)\ \rm  bit$.
-* $H(X|Z)$&nbsp; gives the remaining uncertainty with respect to component&nbsp; $X$&nbsp; when the tuple&nbsp; $Z = (X, Y)$&nbsp; is known.&nbsp;
+* $H(X|Z)$&nbsp; gives the remaining uncertainty with respect to component&nbsp; $X$,&nbsp; when the tuple&nbsp; $Z = (X, Y)$&nbsp; is known.&nbsp;
 * This uncertainty is of course zero: &nbsp; If one knows&nbsp; $Z$, one also knows&nbsp; $X$.

@@ Line 4: / Line 4: @@
 }}
-[[File:P_ID2771__Inf_Z_3_7.png|right|frame|"Probabilities" of the random variable&nbsp; $XY$]]
+[[File:P_ID2771__Inf_Z_3_7.png|right|2D random variable&nbsp; $XY$]]
-We consider the tuple&nbsp; $Z = (X, Y)$,&nbsp; where the individual components&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; each represent ternary random variables &nbsp; &rArr; &nbsp;  symbolic range&nbsp; $|X| = |Y| = 3$.&nbsp; The joint probability function&nbsp; $P_{ XY }(X, Y)$&nbsp; is sketched on the right.
+We consider the tuple&nbsp; $Z = (X, Y)$,&nbsp; where the individual components&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; each represent ternary random variables &nbsp; &rArr; &nbsp;  symbol set size&nbsp; $|X| = |Y| = 3$.&nbsp; The joint probability function&nbsp; $P_{ XY }(X, Y)$&nbsp; is sketched on the right.
-In this exercise, the following are to be calculated:
+In this exercise, the following entropies are to be calculated:
-* the ''joint entropy''&nbsp; $H(XY)$&nbsp; and the ''mutual information''&nbsp; $I(X; Y)$,
+* the&nbsp; "joint entropy"&nbsp; $H(XY)$&nbsp; and the&nbsp; "mutual information"&nbsp; $I(X; Y)$,
-* the ''joint entropy''&nbsp; $H(XZ)$&nbsp; and the ''mutual information''&nbsp; $I(X; Z)$,
+* the&nbsp; "joint entropy"&nbsp; $H(XZ)$&nbsp; and the&nbsp; "mutual information"&nbsp; $I(X; Z)$,
-* the two ''conditional entropies''&nbsp; $H(Z|X)$&nbsp; and&nbsp; $H(X|Z)$.
+* the two&nbsp; "conditional entropies"&nbsp; $H(Z|X)$&nbsp; and&nbsp; $H(X|Z)$.

@@ Line 7: / Line 7: @@
 We consider the tuple&nbsp; $Z = (X, Y)$,&nbsp; where the individual components&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; each represent ternary random variables &nbsp; &rArr; &nbsp;  symbolic range&nbsp; $|X| = |Y| = 3$.&nbsp; The joint probability function&nbsp; $P_{ XY }(X, Y)$&nbsp; is sketched on the right.
-In this task, the following are to be calculated:
+In this exercise, the following are to be calculated:
 * the ''joint entropy''&nbsp; $H(XY)$&nbsp; and the ''mutual information''&nbsp; $I(X; Y)$,
 * the ''joint entropy''&nbsp; $H(XZ)$&nbsp; and the ''mutual information''&nbsp; $I(X; Z)$,
@@ Line 20: / Line 20: @@
 Hints:
-*The task belongs to the chapter&nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen|Different entropies of two-dimensional random variables]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen|Different entropies of two-dimensional random variables]].
 *In particular, reference is made to the pages&nbsp; <br> &nbsp; &nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen#Conditional_probability_and_conditional_entropy|Conditional probability and conditional entropy]] &nbsp; as well as <br> &nbsp; &nbsp;[[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen#Mutual_information_between_two_random_variables|Mutual information between two random variables]].

← Older revision		Revision as of 16:50, 20 July 2021
Line 103:		Line 103:


−	[[Category:Information Theory: Exercises\|^3.2 ~~Entropien von~~ 2D~~-Zufallsgrößen~~^]]	+	[[Category:Information Theory: Exercises\|^3.2 Entropies of 2D Random Variables^]]

← Older revision	Revision as of 16:45, 20 July 2021
(No difference)

@@ Line 4: / Line 4: @@
 }}
-[[File:P_ID2771__Inf_Z_3_7.png|right|frame|"Wahrscheinlichkeiten&rdquo; der Zufallsgröße&nbsp; $XY$]]
+[[File:P_ID2771__Inf_Z_3_7.png|right|frame|"Wahrscheinlichkeiten" der Zufallsgröße&nbsp; $XY$]]
 Wir betrachten das Tupel&nbsp; $Z = (X, Y)$,&nbsp; wobei die Einzelkomponenten&nbsp; $X$&nbsp; und&nbsp; $Y$&nbsp; jeweils ternäre Zufallsgrößen darstellen &nbsp; &rArr; &nbsp;  Symbolumfang&nbsp; $|X| = |Y| = 3$.&nbsp; Die gemeinsame Wahrscheinlichkeitsfunktion&nbsp; $P_{ XY }(X, Y)$&nbsp; ist rechts skizziert.

← Older revision		Revision as of 12:48, 23 March 2021
Line 103:		Line 103:


−	[[Category:~~Aufgaben zu Informationstheorie~~\|^3.2 Entropien von 2D-Zufallsgrößen^]]	+	[[Category:Information Theory: Exercises\|^3.2 Entropien von 2D-Zufallsgrößen^]]