Aufgaben:Exercise 4.5Z: Again Mutual Information - Revision history

Javier: Text replacement - "value-discrete" to "discrete"

2021-10-11T08:27:33Z

Text replacement - "value-discrete" to "discrete"

Javier: Text replacement - "value-continuous" to "continuous"

2021-10-11T08:27:12Z

Text replacement - "value-continuous" to "continuous"

Guenter at 13:07, 7 October 2021

2021-10-07T13:07:43Z

Guenter at 12:22, 2 October 2021

2021-10-02T12:22:55Z

Guenter at 11:53, 2 October 2021

2021-10-02T11:53:46Z

Guenter at 11:52, 2 October 2021

2021-10-02T11:52:59Z

Noah at 12:31, 1 October 2021

2021-10-01T12:31:33Z

Noah at 12:25, 1 October 2021

2021-10-01T12:25:45Z

Show changes

Guenter at 09:27, 16 August 2021

2021-08-16T09:27:41Z

Guenter: Guenter moved page Aufgaben:Aufgabe 4.5Z: Nochmals Transinformation to Aufgaben:Exercise 4.5Z: Again Mutual Information

2021-08-16T09:21:28Z

Guenter moved page Aufgabe 4.5Z: Nochmals Transinformation to Exercise 4.5Z: Again Mutual Information

@@ Line 65: / Line 65: @@
 {Which of the following quantities is never negative?
 |type="[]"}
-+ Both &nbsp;$H(X)$&nbsp; and &nbsp;$H(Y)$&nbsp; in the value-discrete case.
++ Both &nbsp;$H(X)$&nbsp; and &nbsp;$H(Y)$&nbsp; in the discrete case.
-+ The mutual information &nbsp;$I(X; Y)$&nbsp; in the value-discrete case.
++ The mutual information &nbsp;$I(X; Y)$&nbsp; in the discrete case.
 + The mutual information &nbsp;$I(X; Y)$&nbsp; in the continuous case.
 - Both &nbsp;$h(X)$&nbsp; and &nbsp;$h(Y)$&nbsp;  in the continuous case.
@@ Line 125: / Line 125: @@
 Again for clarification:
 * For the mutual information &nbsp;$I(X;Y) \ge 0$ always holds.
-* In the  value-discrete case there is no negative entropy, but in the  continuous case there is.
+* In the  discrete case there is no negative entropy, but in the  continuous case there is.
 {{ML-Fuß}}

@@ Line 24: / Line 24: @@
 Hints:
-*The exercise belongs to the chapter&nbsp; [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang|AWGN channel capacity with value-continuous input]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang|AWGN channel capacity with continuous input]].
 *If the results are to be given in "nat", this is achieved with "log" &nbsp;&#8658;&nbsp; "ln".
 *If the results are to be given in "bit", this is achieved with "log" &nbsp;&#8658;&nbsp; "log<sub>2</sub>".
@@ Line 67: / Line 67: @@
 + Both &nbsp;$H(X)$&nbsp; and &nbsp;$H(Y)$&nbsp; in the value-discrete case.
 + The mutual information &nbsp;$I(X; Y)$&nbsp; in the value-discrete case.
-+ The mutual information &nbsp;$I(X; Y)$&nbsp; in the value-continuous case.
++ The mutual information &nbsp;$I(X; Y)$&nbsp; in the continuous case.
-- Both &nbsp;$h(X)$&nbsp; and &nbsp;$h(Y)$&nbsp;  in the value-continuous case.
+- Both &nbsp;$h(X)$&nbsp; and &nbsp;$h(Y)$&nbsp;  in the continuous case.
-- Both &nbsp;$h(X|Y)$&nbsp; and &nbsp;$h(Y|X)$&nbsp; in the value-continuous case.
+- Both &nbsp;$h(X|Y)$&nbsp; and &nbsp;$h(Y|X)$&nbsp; in the continuous case.
-- The joint entropy &nbsp;$h(XY)$&nbsp; in the value-continuous case.
+- The joint entropy &nbsp;$h(XY)$&nbsp; in the continuous case.
@@ Line 125: / Line 125: @@
 Again for clarification:
 * For the mutual information &nbsp;$I(X;Y) \ge 0$ always holds.
-* In the  value-discrete case there is no negative entropy, but in the  value-continuous case there is.
+* In the  value-discrete case there is no negative entropy, but in the  continuous case there is.
 {{ML-Fuß}}

@@ Line 3: / Line 3: @@
 }}
-[[File:P_ID2893__Inf_Z_4_5.png|right|frame|Given joint PDF and <br>graph of differential entropies]]
+[[File:EN_Inf_Z_4_5.png|right|frame|Given joint PDF and <br>graph of differential entropies]]
 The graph above shows the joint PDF&nbsp; $f_{XY}(x, y)$&nbsp; to be considered in this task,&nbsp; which is identical to the "green" constellation in&nbsp; [[Aufgaben:Exercise_4.5:_Mutual_Information_from_2D-PDF|Exercise 4.5]].
 * In this sketch&nbsp; $f_{XY}(x, y)$&nbsp; is enlarged by a factor of&nbsp; $3$&nbsp; in &nbsp; $y$&ndash;direction.

@@ Line 77: / Line 77: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; Since the results are required in "nat", it is convenient to use the natural logarithm:
+'''(1)'''&nbsp; Since the results are required in&nbsp; "nat",&nbsp; it is convenient to use the natural logarithm:
 *The random variable&nbsp; $X$&nbsp; is uniformly distributed between&nbsp; $0$&nbsp; and&nbsp; $1/{\rm e}^2={\rm e}^{-2}$:
 :$$h(X) =  {\rm ln} \hspace{0.1cm} (\hspace{0.05cm}{\rm e}^{-2}\hspace{0.05cm})
@@ Line 88: / Line 88: @@
 * The area of the parallelogram is given by
 :$$F = A \cdot B = {\rm e}^{-2} \cdot {\rm e}^{0.5} = {\rm e}^{-1.5}\hspace{0.05cm}.$$
-*Thus, the 2D WDF in the area highlighted in green has constant height&nbsp; $C = 1/F ={\rm e}^{1.5}$&nbsp; and we obtain for the joint entropy:
+*Thus, the 2D-PDF in the area highlighted in green has constant height&nbsp; $C = 1/F ={\rm e}^{1.5}$&nbsp; and we obtain for the joint entropy:
 :$$h(XY) =  {\rm ln} \hspace{0.1cm} (F)
 =  {\rm ln} \hspace{0.1cm} (\hspace{0.05cm}{\rm e}^{-1.5}\hspace{0.05cm})
@@ Line 107: / Line 107: @@
-'''(3)'''&nbsp; The mutual information can also be written in the form &nbsp;$I(X;Y) = h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) - h(Y)$&nbsp;:
+'''(3)'''&nbsp; The mutual information can also be written in the form &nbsp;$I(X;Y) = h(Y)-h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) $&nbsp;:
 :$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 1 \,{\rm nat} - 0.5 \,{\rm nat} \hspace{0.15cm}\underline{= 0.5\,{\rm nat}= 0.721\,{\rm bit}}\hspace{0.05cm}.$$
@@ Line 121: / Line 121: @@
-'''(5)'''&nbsp; Correct are the <u>proposed solutions 1 to 3</u>.
+'''(5)'''&nbsp; Correct are the&nbsp; <u>proposed solutions 1 to 3</u>.
 Again for clarification:
 * For the mutual information &nbsp;$I(X;Y) \ge 0$ always holds.
-* In the discrete value case there is no negative entropy, but in the continuous value case there is.
+* In the  value-discrete case there is no negative entropy, but in the  value-continuous case there is.
 {{ML-Fuß}}

@@ Line 4: / Line 4: @@
 [[File:P_ID2893__Inf_Z_4_5.png|right|frame|Given joint PDF and <br>graph of differential entropies]]
-The graph above shows the joint PDF&nbsp; $f_{XY}(x, y)$&nbsp; to be considered in this task,&nbsp; which is identical to the "green" constellation in&nbsp; [[Aufgaben:Aufgabe_4.5:_Transinformation_aus_2D-WDF|Exercise 4.5]].
+The graph above shows the joint PDF&nbsp; $f_{XY}(x, y)$&nbsp; to be considered in this task,&nbsp; which is identical to the "green" constellation in&nbsp; [[Aufgaben:Exercise_4.5:_Mutual_Information_from_2D-PDF|Exercise 4.5]].
 * In this sketch&nbsp; $f_{XY}(x, y)$&nbsp; is enlarged by a factor of&nbsp; $3$&nbsp; in &nbsp; $y$&ndash;direction.
 *In the definition area highlighted in green, the joint PDF is constant equal to&nbsp; $C  = 1/F$,&nbsp; where&nbsp; $F$&nbsp; indicates the area of the parallelogram.

@@ Line 3: / Line 3: @@
 }}
-[[File:P_ID2893__Inf_Z_4_5.png|right|frame|Given joint PDF and graph of differential entropies.]]
+[[File:P_ID2893__Inf_Z_4_5.png|right|frame|Given joint PDF and <br>graph of differential entropies]]
-The graph above shows the joint PDF&nbsp; $f_{XY}(x, y)$, to be considered in this task, which is identical to the "green" constellation in&nbsp; [[Aufgaben:Aufgabe_4.5:_Transinformation_aus_2D-WDF|Exercise 4.5]].
+The graph above shows the joint PDF&nbsp; $f_{XY}(x, y)$&nbsp; to be considered in this task,&nbsp; which is identical to the "green" constellation in&nbsp; [[Aufgaben:Aufgabe_4.5:_Transinformation_aus_2D-WDF|Exercise 4.5]].
-*  $f_{XY}(x, y)$&nbsp; is enlarged by a factor of&nbsp; $3$&nbsp; in the &nbsp; $y$&ndash;direction.
+* In this sketch&nbsp; $f_{XY}(x, y)$&nbsp; is enlarged by a factor of&nbsp; $3$&nbsp; in &nbsp; $y$&ndash;direction.
-*In the definition area highlighted in green, the joint PDF is constant equal to&nbsp; $C  = 1/F$, where&nbsp; $F$&nbsp; indicates the area of the parallelogram.
+*In the definition area highlighted in green, the joint PDF is constant equal to&nbsp; $C  = 1/F$,&nbsp; where&nbsp; $F$&nbsp; indicates the area of the parallelogram.
@@ Line 24: / Line 24: @@
 Hints:
-*The exercise belongs to the chapter&nbsp; [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang|AWGN channel capacity with continuous value input]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang|AWGN channel capacity with value-continuous input]].
 *If the results are to be given in "nat", this is achieved with "log" &nbsp;&#8658;&nbsp; "ln".
 *If the results are to be given in "bit", this is achieved with "log" &nbsp;&#8658;&nbsp; "log<sub>2</sub>".
@@ Line 35: / Line 35: @@
 <quiz display=simple>
-{State the following information theoretic quantities in "nat":
+{State the following information theoretic quantities&nbsp;  "nat":
 |type="{}"}
 $h(X) \ = \ $  { -2.06--1.94 } $\ \rm nat$
@@ Line 43: / Line 43: @@
-{What are the same quantities with the pseudo&ndash;unit "bit"?
+{What are the same quantities with the pseudo&ndash;unit&nbsp; "bit"?
 |type="{}"}
 $h(X) \ = \ $ { -2.986--2.786  } $\ \rm bit$
@@ Line 65: / Line 65: @@
 {Which of the following quantities is never negative?
 |type="[]"}
-+ Both &nbsp;$H(X)$&nbsp; and &nbsp;$H(Y)$&nbsp; in the discrete value case.
++ Both &nbsp;$H(X)$&nbsp; and &nbsp;$H(Y)$&nbsp; in the value-discrete case.
-+ The mutual information &nbsp;$I(X; Y)$&nbsp; in the discrete value case.
++ The mutual information &nbsp;$I(X; Y)$&nbsp; in the value-discrete case.
-+ The mutual information &nbsp;$I(X; Y)$&nbsp; in the continuous value case.
++ The mutual information &nbsp;$I(X; Y)$&nbsp; in the value-continuous case.
-- Both &nbsp;$h(X)$&nbsp; and &nbsp;$h(Y)$&nbsp;  in the continuous value case.
+- Both &nbsp;$h(X)$&nbsp; and &nbsp;$h(Y)$&nbsp;  in the value-continuous case.
-- Both &nbsp;$h(X|Y)$&nbsp; and &nbsp;$h(Y|X)$&nbsp; in the continuous value case.
+- Both &nbsp;$h(X|Y)$&nbsp; and &nbsp;$h(Y|X)$&nbsp; in the value-continuous case.
-- The joint entropy &nbsp;$h(XY)$&nbsp; in the continuous value case.
+- The joint entropy &nbsp;$h(XY)$&nbsp; in the value-continuous case.

← Older revision		Revision as of 12:31, 1 October 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~AWGN–Kanalkapazität bei wertkontinuierlichem Eingang~~	+	{{quiz-Header\|Buchseite=Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input
	}}		}}

← Older revision		Revision as of 09:27, 16 August 2021
Line 131:		Line 131:


−	[[Category:Information Theory: Exercises\|^4.2 AWGN ~~& kontinuierlicher Eingang~~^]]	+	[[Category:Information Theory: Exercises\|^4.2 AWGN and Value-Continuous Input^]]