Aufgaben:Exercise 3.10: Mutual Information at the BSC - Revision history

Hwang at 05:05, 18 September 2022

2022-09-18T05:05:58Z

Guenter at 11:55, 24 September 2021

2021-09-24T11:55:10Z

Noah at 08:17, 24 September 2021

2021-09-24T08:17:54Z

Guenter at 15:07, 21 September 2021

2021-09-21T15:07:42Z

Guenter at 14:55, 21 September 2021

2021-09-21T14:55:30Z

Noah at 12:04, 16 September 2021

2021-09-16T12:04:51Z

Noah at 11:48, 16 September 2021

2021-09-16T11:48:46Z

Noah at 10:01, 14 September 2021

2021-09-14T10:01:42Z

Noah at 09:31, 14 September 2021

2021-09-14T09:31:40Z

Noah at 09:31, 14 September 2021

2021-09-14T09:31:17Z

@@ Line 129: / Line 129: @@
 '''(6)'''&nbsp; Here, too, the <u>second proposed solution</u> is correct:
-*Because of&nbsp; $I(X;Y) = H(X) - H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Y) = H(Y) - H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X)$&nbsp;,&nbsp; $H(Y|X)$&nbsp; is greater than&nbsp; $H(X|Y)$ by the same amount that&nbsp; $H(Y)$&nbsp; is greater than&nbsp; $H(X)$:
+*Because of&nbsp; $I(X;Y) = H(X) - H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Y) = H(Y) - H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X)$&nbsp;,&nbsp; $H(Y|X)$&nbsp; is greater than&nbsp; $H(X|Y)$ by the same magnitude that&nbsp; $H(Y)$&nbsp; is greater than&nbsp; $H(X)$:
 :$$H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X) = H(Y) -I(X;Y) = 0.8268 - 0.3578 ={ 0.469\,{\rm bit}} \hspace{0.05cm}$$
 *Direct calculation gives the same result&nbsp; $H(Y|X) = 0.469\ \rm  bit$:

← Older revision		Revision as of 11:55, 24 September 2021
Line 138:		Line 138:


−	[[Category:Information Theory: Exercises\|^3.3 ~~Anwendung auf DSÜ-Kanäle~~^]]	+	[[Category:Information Theory: Exercises\|^3.3 Application to Digital Signal Transmission^]]

← Older revision		Revision as of 08:17, 24 September 2021
Line 1:		Line 1:
−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Anwendung auf DSÜ-Kanäle~~	+	{{quiz-Header\|Buchseite=Information_Theory/Application_to_Digital_Signal_Transmission
	}}		}}

@@ Line 116: / Line 116: @@
 :$$H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Y) = {\rm E} \hspace{0.02cm} \left [ \hspace{0.05cm} {\rm log}_2 \hspace{0.1cm} \frac{1}{P_{\hspace{0.03cm}X \mid \hspace{0.03cm} Y} (X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y)} \hspace{0.05cm}\right ] = {\rm E} \hspace{0.02cm} \left [ \hspace{0.05cm} {\rm log}_2 \hspace{0.1cm} \frac{P_Y(Y)}{P_{XY} (X, Y)} \hspace{0.05cm} \right ] \hspace{0.05cm}$$
-*In the example, the same result&nbsp; $H(X|Y) = 0.3642 \ \rm bit$ is also obtained according to this calculation rule:
+*In the example, the same result&nbsp; $H(X|Y) = 0.3642 \ \rm bit$&nbsp; is also obtained according to this calculation rule:
 :$$H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Y) = 0.18 \cdot {\rm log}_2 \hspace{0.1cm} \frac{0.26}{0.18} + 0.02 \cdot {\rm log}_2 \hspace{0.1cm} \frac{0.74}{0.02} + 0.08 \cdot {\rm log}_2 \hspace{0.1cm} \frac{0.26}{0.08} + 0.72 \cdot {\rm log}_2 \hspace{0.1cm} \frac{0.74}{0.72} \hspace{0.05cm}.$$
@@ Line 124: / Line 124: @@
 *In the case of disturbed transmission&nbsp; $(ε > 0)$&nbsp; the uncertainty regarding the sink is always greater than the uncertainty regarding the source.&nbsp; One obtains here as a numerical value:
 :$$H(Y) = H_{\rm bin}(0.26)={ 0.8268\,{\rm bit}} \hspace{0.05cm}.$$
-*With error-free transmission&nbsp; $(ε = 0)$&nbsp;, on the other hand,&nbsp; $P_Y(⋅) = P_X(⋅)$&nbsp; and&nbsp; $H(Y) = H(X)$&nbsp; would apply.
+*With error-free transmission&nbsp; $(ε = 0)$,&nbsp; on the other hand,&nbsp; $P_Y(⋅) = P_X(⋅)$&nbsp; and&nbsp; $H(Y) = H(X)$&nbsp; would apply.
 '''(6)'''&nbsp; Here, too, the <u>second proposed solution</u> is correct:
-*Because of&nbsp; $I(X;Y) = H(X) - H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Y) = H(Y) - H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X)$&nbsp;,&nbsp; $H(Y|X)$&nbsp; by the same amount that&nbsp; $H(X|Y)$, um den&nbsp; $H(Y)$&nbsp; is greater than&nbsp; $H(X)$:
+*Because of&nbsp; $I(X;Y) = H(X) - H(X \hspace{-0.1cm}\mid \hspace{-0.1cm} Y) = H(Y) - H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X)$&nbsp;,&nbsp; $H(Y|X)$&nbsp; is greater than&nbsp; $H(X|Y)$ by the same amount that&nbsp; $H(Y)$&nbsp; is greater than&nbsp; $H(X)$:
-:$$H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X) = H(Y) -I(X;Y) = 0.8268 - 0.3578 ={ 0.4690\,{\rm bit}} \hspace{0.05cm}$$
+:$$H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X) = H(Y) -I(X;Y) = 0.8268 - 0.3578 ={ 0.469\,{\rm bit}} \hspace{0.05cm}$$
-*Direct calculation gives the same result&nbsp; $H(Y|X) = 0.4690\ \rm  bit$:
+*Direct calculation gives the same result&nbsp; $H(Y|X) = 0.469\ \rm  bit$:
 :$$H(Y \hspace{-0.1cm}\mid \hspace{-0.1cm} X) = {\rm E} \hspace{0.02cm} \left [ \hspace{0.02cm} {\rm log}_2 \hspace{0.1cm} \frac{1}{P_{\hspace{0.03cm}Y \mid \hspace{0.03cm} X} (Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X)} \right ] = 0.18 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.9} + 0.02 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.1} + 0.08 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.1} + 0.72 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{0.9} \hspace{0.05cm}.$$

@@ Line 2: / Line 2: @@
 }}
-[[File:P_ID2787__Inf_A_3_9.png|right|frame|BSC Model Considered]]
+[[File:P_ID2787__Inf_A_3_9.png|right|frame|BSC model considered]]
 We consider the&nbsp; [[Channel_Coding/Kanalmodelle_und_Entscheiderstrukturen#Binary_Symmetric_Channel_.E2.80.93_BSC|Binary Symmetric Channel]]&nbsp; $\rm (BSC)$. The parameter values are valid for the whole exercise:
 * Crossover probability: &nbsp; $\varepsilon = 0.1$,
-* Probability for $0$: &nbsp;  $p_0 = 0.2$,
+* Probability for&nbsp; $0$: &nbsp;  $p_0 = 0.2$,
-* Probability for $1$: &nbsp;  $p_1 = 0.8$.
+* Probability for&nbsp; $1$: &nbsp;  $p_1 = 0.8$.
-Thus the probability function of the source is: &nbsp; $P_X(X)= (0.2 , \ 0.8)$&nbsp; and for the source entropy applies:
+Thus the probability mass function of the source is: &nbsp; $P_X(X)= (0.2 , \ 0.8)$&nbsp; and for the source entropy applies:
 :$$H(X) = p_0 \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p_0} + p_1\cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p_1} = H_{\rm bin}(0.2)={ 0.7219\,{\rm bit}} \hspace{0.05cm}.$$
@@ Line 37: / Line 37: @@
 Hints:
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung|Application to Digital Signal Transmission]].
-*Bezug genommen wird insbesondere auf die Seite&nbsp;    [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung#Calculation_of_the_mutual_information_for_the_binary_channel|Mutual information calculation for the binary channel]].
+*Reference is made in particular to the page&nbsp;    [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung#Calculation_of_the_mutual_information_for_the_binary_channel|Mutual information calculation for the binary channel]].
 *In&nbsp; [[Aufgaben:Aufgabe_3.10Z:_BSC–Kanalkapazität|Exercise 3.10Z]]&nbsp; the channel capacity&nbsp; $C_{\rm BSC }$&nbsp; of the BSC model is calculated.
-*This results as the maximum mutual information&nbsp; $I(X;\ Y)$&nbsp;  by maximisation with respect to the probabilities&nbsp; $p_0$&nbsp; or&nbsp; $p_1 = 1 - p_0$.
+*This results as the maximum mutual information&nbsp; $I(X;\ Y)$&nbsp;  by maximization with respect to the probabilities&nbsp; $p_0$&nbsp; or&nbsp; $p_1 = 1 - p_0$.
@@ Line 54: / Line 54: @@
 $P_{ XY }(1, 1) \ = \ $ { 0.72 3% }
-{What is the probability function&nbsp; $P_Y(Y)$&nbsp; of the sink?
+{What is the probability mass function&nbsp; $P_Y(Y)$&nbsp; of the sink?
 |type="{}"}
 $P_Y(0)\ = \ $ { 0.26 3% }

@@ Line 3: / Line 3: @@
 [[File:P_ID2787__Inf_A_3_9.png|right|frame|BSC Model Considered]]
-We consider the&nbsp; [[Channel_Coding/Kanalmodelle_und_EntscheiderstrukturenBinary Symmetric Channel|Binary Symmetric Channel]]&nbsp; $\rm (BSC)$. The parameter values are valid for the whole exercise:
+We consider the&nbsp; [[Channel_Coding/Kanalmodelle_und_Entscheiderstrukturen#Binary Symmetric Channel|Binary Symmetric Channel]]&nbsp; $\rm (BSC)$. The parameter values are valid for the whole exercise:
 * Probability of falsification: &nbsp; $\varepsilon = 0.1$,
 * Probability for $0$: &nbsp;  $p_0 = 0.2$,