Aufgaben:Exercise 3.2: Expected Value Calculations - Revision history

Noah at 08:11, 24 September 2021

2021-09-24T08:11:36Z

Guenter at 09:35, 17 August 2021

2021-08-17T09:35:12Z

Guenter at 09:23, 17 August 2021

2021-08-17T09:23:37Z

Noah at 20:42, 10 August 2021

2021-08-10T20:42:31Z

Noah at 20:42, 10 August 2021

2021-08-10T20:42:09Z

Noah at 20:41, 10 August 2021

2021-08-10T20:41:37Z

Show changes

Guenter at 13:31, 16 July 2021

2021-07-16T13:31:59Z

Guenter: Guenter moved page Aufgaben:Aufgabe 3.2: Erwartungswertberechnungen to Aufgaben:Exercise 3.2: Expected Value Calculations

2021-07-16T13:09:44Z

Guenter moved page Aufgabe 3.2: Erwartungswertberechnungen to Exercise 3.2: Expected Value Calculations

Javier: Text replacement - "”" to """

2021-05-28T14:29:41Z

Text replacement - "”" to """

Javier: Text replacement - "„" to """

2021-05-28T14:28:57Z

Text replacement - "„" to """

← Older revision		Revision as of 08:11, 24 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Einige Vorbemerkungen zu zweidimensionalen Zufallsgrößen~~	+	{{quiz-Header\|Buchseite=Information_Theory/Some_Preliminary_Remarks_on_Two-Dimensional_Random_Variables
	}}		}}

@@ Line 88: / Line 88: @@
 :$${\rm supp} (P_X)  = \{ 0\hspace{0.05cm}, 1\hspace{0.05cm}, 3 \}  \hspace{0.05cm}.$$
 *With&nbsp; $F(X) = 1/P_X(X)$&nbsp; one further obtains:
-:$${\rm E} \big [ 1/P_X(X)\big ] = \hspace{-0.4cm} \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm}, 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.4cm} P_{X}(x) \cdot {1}/{P_X(x)}
+:$${\rm E} \big [ 1/P_X(X)\big ] = \hspace{-0.4cm} \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.4cm} P_{X}(x) \cdot {1}/{P_X(x)}
-= \hspace{-0.4cm}  \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm}, 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.3cm} 1
+= \hspace{-0.4cm}  \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.3cm} 1
 \hspace{0.15cm}\underline{ = 3} \hspace{0.05cm}.$$
 *The second expected value gives the same result with&nbsp; ${\rm supp} (P_Y)  = \{ 0\hspace{0.05cm}, 1\hspace{0.05cm}, 2 \} $&nbsp;:
@@ Line 96: / Line 96: @@
-'''(2)'''&nbsp; In both cases, the index of the probability function is identical with the random variable&nbsp; $(X$&nbsp; or &nbsp; $Y)$&nbsp; and we obtain
+'''(2)'''&nbsp; In both cases, the index of the probability mass function is identical with the random variable&nbsp; $(X$&nbsp; or &nbsp; $Y)$&nbsp; and we obtain
-:$${\rm E} \big [ P_X(X)\big ] =  \hspace{-0.3cm}   \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm}, 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.3cm}  P_{X}(x) \cdot {P_X(x)}
+:$${\rm E} \big [ P_X(X)\big ] =  \hspace{-0.3cm}   \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.3cm}  P_{X}(x) \cdot {P_X(x)}
 = (1/2)^2 + (1/8)^2 + (3/8)^2 = 13/32
 \hspace{0.15cm}\underline{ \approx 0.406} \hspace{0.05cm},$$
-:$${\rm E} \big [ P_Y(Y)\big ] = \hspace{-0.3cm}  \sum_{y \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm}, 1\hspace{0.05cm},\hspace{0.05cm} 2 \}}  \hspace{-0.3cm}  P_Y(y) \cdot P_Y(y) = (1/2)^2 + (1/4)^2 + (1/4)^2
+:$${\rm E} \big [ P_Y(Y)\big ] = \hspace{-0.3cm}  \sum_{y \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 2 \}}  \hspace{-0.3cm}  P_Y(y) \cdot P_Y(y) = (1/2)^2 + (1/4)^2 + (1/4)^2
 \hspace{0.15cm}\underline{ = 0.375} \hspace{0.05cm}.$$
@@ Line 106: / Line 106: @@
 '''(3)'''&nbsp; The following equations apply here:
-:$${\rm E} \big [ P_Y(X)\big ] = \hspace{-0.3cm}  \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm}, 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.3cm}  P_{X}(x) \cdot {P_Y(x)} = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{8} \cdot \frac{1}{4} + \frac{3}{8} \cdot 0 = 9/32
+:$${\rm E} \big [ P_Y(X)\big ] = \hspace{-0.3cm}  \sum_{x \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 3 \}}  \hspace{-0.3cm}  P_{X}(x) \cdot {P_Y(x)} = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{8} \cdot \frac{1}{4} + \frac{3}{8} \cdot 0 = 9/32
 \hspace{0.15cm}\underline{ \approx 0.281} \hspace{0.05cm},$$
 *The expected value formation here refers to&nbsp; $P_X(&middot;)$, i.e. to the random variable&nbsp; $X$.
 *$P_Y(&middot;)$&nbsp;is the formal function without (direct) reference to the random variable&nbsp; $Y$.
+*The same numerical value is obtained for the second expected value&nbsp; (this does not have to be the case in general):
-*The same numerical value is obtained for the second expected value&nbsp; (this does not have to be the case):
+:$${\rm E} \big [ P_X(Y)\big ] = \hspace{-0.3cm}  \sum_{y \hspace{0.05cm}\in \hspace{0.05cm} \{ 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 2 \}}  \hspace{-0.3cm}  P_{Y}(y) \cdot {P_X(y)} = \frac{1}{2} \cdot \frac{1}{2} + \frac{1}{4} \cdot \frac{1}{8} + \frac{1}{4} \cdot 0 = 9/32 \hspace{0.15cm}\underline{ \approx 0.281} \hspace{0.05cm}.$$

@@ Line 3: / Line 3: @@
 }}
-[[File:P_ID2751__Inf_A_3_2.png|right|frame|2D probability function]]
+[[File:P_ID2751__Inf_A_3_2.png|right|frame|Two-dimensional <br>probability mass function]]
-We consider the following probability functions:
+We consider the following probability mass functions&nbsp; $\rm (PMF)$:
 :$$P_X(X) = \big[1/2,\ 1/8,\ 0,\ 3/8 \big],$$
@@ Line 15: / Line 15: @@
 : $X= \{0,\ 1,\ 2,\ 3\}$, &nbsp; &nbsp;  $Y= \{0,\ 1,\ 2,\ 3\}$,&nbsp; &nbsp; $U = \{0,\ 1\}$, &nbsp; &nbsp; $V = \{0, 1\}$.
-Often, for such discrete random variables, one must have different expected values of the form
+Often, for such discrete random variables, one must have to calculate different expected values of the form
 :$${\rm E} \big [ F(X)\big ] =\hspace{-0.3cm}  \sum_{x \hspace{0.05cm}\in \hspace{0.05cm}\hspace{-0.03cm}  {\rm supp} (P_X)}  \hspace{-0.1cm}
-  P_{X}(x) \cdot F(x) $$
+  P_{X}(x) \cdot F(x). $$
-calculate. Here, denote:
+Here, denote:
-* $P_X(X)$&nbsp; denotes the <i>probability function</i> of the discrete random variable &nbsp; $X$.
+* $P_X(X)$&nbsp; denotes the probability mass function of the discrete random variable &nbsp; $X$.
-* The&nbsp; <i>support</i>&nbsp; of&nbsp; $P_X$&nbsp; includes all those realisations&nbsp; $x$&nbsp; of the random variable&nbsp; $X$&nbsp; with non-vanishing probability.
+* The&nbsp; "support"&nbsp; of&nbsp; $P_X$&nbsp; includes all those realisations&nbsp; $x$&nbsp; of the random variable&nbsp; $X$&nbsp; with non-vanishing probability.
-*Formally, this can be written
+*Formally, this can be written as
 :$${\rm supp} (P_X)  = \{ x: \hspace{0.25cm}x \in X \hspace{0.15cm}\underline{\rm and} \hspace{0.15cm} P_X(x) \ne 0 \} \hspace{0.05cm}.$$
 * $F(X)$&nbsp; is an (arbitrary) real-valued function that can be specified in the entire domain of definition of the random variable&nbsp; $X$&nbsp;.
@@ Line 30: / Line 30: @@
 In the task, the expected values for various functions&nbsp; $F(X)$&nbsp; are to be calculated, among others for
-: $F(X)= 1/P_X(X)$,
+#&nbsp; $F(X)= 1/P_X(X)$,
-: $F(X)= P_X(X)$,
+#&nbsp; $F(X)= P_X(X)$,
-: $F(X)= - \log_2 \ P_X(X)$.
+#&nbsp; $F(X)= - \log_2 \ P_X(X)$.
@@ Line 42: / Line 42: @@
 Hints:
-*The exercise belongs to the chapter&nbsp; [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen|Some preliminary remarks on 2D random variables]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen|Some preliminary remarks on two-dimensional  random variables]].
-* The two 1D probability functions&nbsp; $P_X(X)$&nbsp; and&nbsp; $P_Y(Y)$&nbsp; result from the presented 2D&ndash;PMF&nbsp; $P_{XY}(X,\ Y)$, as will be shown in&nbsp; [[Aufgaben:3.2Z_2D–Wahrscheinlichkeitsfunktion|Exercise 3.2Z]]&nbsp;.
+* The two one-dimensional  probability mass functions&nbsp; $P_X(X)$&nbsp; and&nbsp; $P_Y(Y)$&nbsp; result from the presented 2D&ndash;PMF&nbsp; $P_{XY}(X,\ Y)$,&nbsp; as will be shown in&nbsp; [[Aufgaben:3.2Z_2D–Wahrscheinlichkeitsfunktion|Exercise 3.2Z]].
-* The binary probability functions&nbsp; $P_U(U)$&nbsp; and&nbsp; $P_V(V)$&nbsp; are obtained according to the modulo operations&nbsp; $U = X \hspace{0.1cm}\text{mod} \hspace{0.1cm}2$&nbsp; and&nbsp; $V = Y \hspace{0.1cm}\text{mod} \hspace{0.1cm} 2$.
+* The binary probability mass functions&nbsp; $P_U(U)$&nbsp; and&nbsp; $P_V(V)$&nbsp; are obtained according to the modulo operations&nbsp; $U = X \hspace{0.1cm}\text{mod} \hspace{0.1cm}2$&nbsp; and&nbsp; $V = Y \hspace{0.1cm}\text{mod} \hspace{0.1cm} 2$.

@@ Line 24: / Line 24: @@
 * The&nbsp; <i>support</i>&nbsp; of&nbsp; $P_X$&nbsp; includes all those realisations&nbsp; $x$&nbsp; of the random variable&nbsp; $X$&nbsp; with non-vanishing probability.
 *Formally, this can be written
-:$${\rm supp} (P_X)  = \{ x: \hspace{0.25cm}x \in X \hspace{0.15cm}\underline{\rm und} \hspace{0.15cm} P_X(x) \ne 0 \} \hspace{0.05cm}.$$
+:$${\rm supp} (P_X)  = \{ x: \hspace{0.25cm}x \in X \hspace{0.15cm}\underline{\rm and} \hspace{0.15cm} P_X(x) \ne 0 \} \hspace{0.05cm}.$$
 * $F(X)$&nbsp; is an (arbitrary) real-valued function that can be specified in the entire domain of definition of the random variable&nbsp; $X$&nbsp;.

@@ Line 126: / Line 126: @@
 * The entropy&nbsp; $H(U) = 1$&nbsp; bit&nbsp; can be calculated according to the first equation.&nbsp; It applies to the binary random variable&nbsp; $U$&nbsp; with equal probabilities.
 * The entropy&nbsp; $H(V) = 0.811$&nbsp; bit&nbsp; is calculated according to the second equation.&nbsp; Due to the probabilities&nbsp; $3/4$&nbsp; and&nbsp; $1/4$&nbsp;, the entropy (uncertainty) is smaller here than for the random variable&nbsp; $U$.
-* The third expected value cannot indicate the entropy of a binary random variable, which is always limited to&nbsp; $1$&nbsp; (bit)&nbsp, simply because of the result &nbsp; $(1.208$&nbsp; bit$)$&nbsp;.
+* The third expected value cannot indicate the entropy of a binary random variable, which is always limited to&nbsp; $1$&nbsp; (bit)&nbsp;, simply because of the result &nbsp; $(1.208$&nbsp; bit$)$&nbsp;.
 {{ML-Fuß}}

← Older revision		Revision as of 13:31, 16 July 2021
Line 131:		Line 131:


−	[[Category:Information Theory: Exercises\|^3.1 ~~Allgemeines zu~~ 2D~~-Zufallsgrößen~~^]]	+	[[Category:Information Theory: Exercises\|^3.1 General Information on 2D Random Variables^]]

← Older revision	Revision as of 13:09, 16 July 2021
(No difference)

@@ Line 85: / Line 85: @@
   P_{X}(x) \cdot F(x)  \hspace{0.05cm}.$$
 Im vorliegenden Beispiel gilt dabei&nbsp; $X = \{0,\ 1,\ 2,\ 3\}$&nbsp; und&nbsp; $P_X(X) = \big [1/2, \ 1/8, \ 0, \ 3/8\big ]$.
-*Wegen&nbsp; $P_X(X = 2) = 0$&nbsp; ergibt sich somit für die zu berücksichtigende Menge&nbsp; (dem "Support&rdquo;)&nbsp; in obiger Summation:
+*Wegen&nbsp; $P_X(X = 2) = 0$&nbsp; ergibt sich somit für die zu berücksichtigende Menge&nbsp; (dem "Support")&nbsp; in obiger Summation:
 :$${\rm supp} (P_X)  = \{ 0\hspace{0.05cm}, 1\hspace{0.05cm}, 3 \}  \hspace{0.05cm}.$$
 *Mit&nbsp; $F(X) = 1/P_X(X)$&nbsp;  erhält man weiter: