Aufgaben:Exercise 4.5: Mutual Information from 2D-PDF - Revision history

Javier: Text replacement - "value-continuous" to "continuous"

2021-10-11T08:27:01Z

Text replacement - "value-continuous" to "continuous"

Guenter at 16:12, 1 October 2021

2021-10-01T16:12:44Z

Guenter at 15:50, 1 October 2021

2021-10-01T15:50:25Z

Guenter at 15:49, 1 October 2021

2021-10-01T15:49:58Z

Guenter at 15:47, 1 October 2021

2021-10-01T15:47:55Z

Guenter at 15:45, 1 October 2021

2021-10-01T15:45:31Z

Noah at 12:31, 1 October 2021

2021-10-01T12:31:23Z

Noah at 11:11, 29 September 2021

2021-09-29T11:11:44Z

Show changes

Guenter: Guenter moved page Aufgaben:Exercise 4.5: Mutual Information from 2D-WDF to Aufgaben:Exercise 4.5: Mutual Information from 2D-PDF

2021-08-20T14:26:25Z

Guenter moved page Exercise 4.5: Mutual Information from 2D-WDF to Exercise 4.5: Mutual Information from 2D-PDF

Guenter: Guenter moved page Aufgaben:Exercise 4.5: Mutual Information from 2D WDF to Aufgaben:Exercise 4.5: Mutual Information from 2D-WDF

2021-08-20T14:23:58Z

Guenter moved page Exercise 4.5: Mutual Information from 2D WDF to Exercise 4.5: Mutual Information from 2D-WDF

@@ Line 12: / Line 12: @@
 respectively.&nbsp; Within each of the regions shown, let&nbsp; $f_{XY}(x, y) = C = \rm const.$
-For example, the mutual information between the value-continuous random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; can be calculated as follows:
+For example, the mutual information between the continuous random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; can be calculated as follows:
 :$$I(X;Y) = h(X) + h(Y) - h(XY)\hspace{0.05cm}.$$
@@ Line 36: / Line 36: @@
 Hints:
-*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_value-continuous_random_variables|Mutual information with value-continuous input]].
+*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_continuous_random_variables|Mutual information with continuous input]].
 *Let the following differential entropies also be given:

@@ Line 78: / Line 78: @@
 ===Solution===
 {{ML-Kopf}}
-[[File:P_ID2887__Inf_A_4_5a.png|right|frame|„Red” PDFs]]
+[[File:P_ID2887__Inf_A_4_5a.png|right|frame|"Red"&nbsp; probability density functions; <br>'''!''' Note:&nbsp; Ordinate of &nbsp;$f_{Y}(y)$&nbsp; is directed to the left '''!''']]
-'''(1)'''&nbsp; For the rectangular joint PDF &nbsp;$f_{XY}(x, y)$&nbsp; there are no statistical ties between&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; &nbsp; &#8658; &nbsp; $\underline{I(X;Y) = 0}$.
+'''(1)'''&nbsp; For the rectangular two-dimensional PDF &nbsp;$f_{XY}(x, y)$&nbsp; there are no statistical dependences between&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; &nbsp; &#8658; &nbsp; $\underline{I(X;Y) = 0}$.
 Formally, this result can be proved with the following equation:
 :$$I(X;Y) = h(X) \hspace{-0.05cm}+\hspace{-0.05cm} h(Y) \hspace{-0.05cm}- \hspace{-0.05cm}h(XY)\hspace{0.02cm}.$$
-*The red area 2D-WDF &nbsp;$f_{XY}(x, y)$&nbsp; is&nbsp; $F = 4$.&nbsp; Since &nbsp;$f_{XY}(x, y)$&nbsp; is constant in this area and the volume under &nbsp;$f_{XY}(x, y)$&nbsp; must be equal to&nbsp; $1$&nbsp;, the height is &nbsp; $C = 1/F = 1/4$.
+*The red area of the two-dimensional PDF &nbsp;$f_{XY}(x, y)$&nbsp; is&nbsp; $F = 4$.&nbsp;
-*From this follows for the differential joint entropy in "bit":
+*Since &nbsp;$f_{XY}(x, y)$&nbsp; is constant in this area and the volume under &nbsp;$f_{XY}(x, y)$&nbsp; must be equal to&nbsp; $1$,&nbsp; the height is&nbsp; $C = 1/F = 1/4$.
 :$$h(XY) \  =  \  \hspace{0.1cm}-\hspace{0.2cm} \int \hspace{-0.9cm} \int\limits_{\hspace{-0.5cm}(x, y) \hspace{0.1cm}\in \hspace{0.1cm}{\rm supp} \hspace{0.03cm}(\hspace{-0.03cm}f_{XY}\hspace{-0.08cm})}
   \hspace{-0.6cm} f_{XY}(x, y) \cdot {\rm log}_2 \hspace{0.1cm} [ f_{XY}(x, y) ]
@@ Line 91: / Line 92: @@
   \hspace{-0.6cm} f_{XY}(x, y)
   \hspace{0.15cm}{\rm d}x\hspace{0.15cm}{\rm d}y = 2 \,{\rm bit}\hspace{0.05cm}.$$
-*It is considered that the double integral is equal to&nbsp; $1$&nbsp;.&nbsp; The pseudo-unit "bit" corresponds to the <i>binary logarithm</i> &nbsp;&#8658;&nbsp; "log<sub>2</sub>".
+*It is considered that the double integral is equal to&nbsp; $1$&nbsp;.&nbsp; The pseudo-unit&nbsp; "bit"&nbsp; corresponds to the&nbsp; "binary logarithm" &nbsp;&#8658;&nbsp; "log<sub>2</sub>".
@@ Line 98: / Line 99: @@
 * The marginal probability density functions &nbsp;$f_{X}(x)$&nbsp; and  &nbsp;$f_{Y}(y)$&nbsp; are rectangular &nbsp; &#8658; &nbsp; uniform distribution between&nbsp; $0$&nbsp; and&nbsp; $2$:
 :$$h(X) = h(Y) = {\rm log}_2 \hspace{0.1cm} (2) = 1 \,{\rm bit}\hspace{0.05cm}.$$
 * Substituting these results into the above equation, we obtain:
 :$$I(X;Y) = h(X) + h(Y) - h(XY) = 1 \,{\rm bit} + 1 \,{\rm bit} - 2 \,{\rm bit} = 0 \,{\rm (bit)}
@@ Line 104: / Line 108: @@
 '''(2)'''&nbsp; Also for this parallelogram we get&nbsp; $F = 4, \ C = 1/4$&nbsp; as well as&nbsp; $h(XY) = 2$ bit.
-* Here, as in subtask&nbsp; '''(1)'''&nbsp;, the random variable&nbsp; $Y$&nbsp; is uniformly distributed between&nbsp; $0$&nbsp; and&nbsp; $2$&nbsp;&nbsp; &rArr; &nbsp; $h(Y) = 1$ bit.
+* Here, as in subtask&nbsp; '''(1)'''&nbsp;, the variable&nbsp; $Y$&nbsp; is uniformly distributed between&nbsp; $0$&nbsp; and&nbsp; $2$&nbsp;&nbsp; &rArr; &nbsp; $h(Y) = 1$ bit.
 *In contrast,&nbsp; $X$&nbsp; is triangularly distributed between&nbsp; $0$&nbsp; and&nbsp; $4$&nbsp; $($with maximum at $2)$.&nbsp;
@@ Line 115: / Line 118: @@
 \hspace{0.05cm}.$$
 <br clear=all>
-[[File:P_ID2889__Inf_A_4_5c_neu.png|right|frame|„Green” probability density functions]]
+[[File:P_ID2889__Inf_A_4_5c_neu.png|right|frame|"„Green"&nbsp; probability density functions]]
 '''(3)'''&nbsp; The following properties are obtained for the green conditions:
 :$$F = A \cdot B \hspace{0.3cm}  \Rightarrow \hspace{0.3cm} C = \frac{1}{A \cdot B}
@@ Line 121: / Line 124: @@
 \Rightarrow \hspace{0.3cm} h(XY)  =   {\rm log}_2 \hspace{0.1cm} (A \cdot B)
 \hspace{0.05cm}.$$
-*The random variable&nbsp; $Y$&nbsp; is now uniformly distributed between&nbsp; $0$&nbsp; and&nbsp; $A$&nbsp; and the random variable&nbsp; $X$&nbsp; is triangularly distributed between&nbsp; $0$&nbsp; and&nbsp; $B$&nbsp;:
+*The random variable&nbsp; $Y$&nbsp; is now uniformly distributed between&nbsp; $0$&nbsp; and&nbsp; $A$&nbsp; and the random variable&nbsp; $X$&nbsp; is triangularly distributed between&nbsp; $0$&nbsp; and&nbsp; $2B$&nbsp; $($with maximum at&nbsp; $B)$:
 :$$h(X)   \ =  \   {\rm log}_2 \hspace{0.1cm} (B \cdot \sqrt{ e})
 \hspace{0.05cm},$$ $$
@@ Line 130: / Line 133: @@
 \hspace{0.05cm}.$$
 [[File: P_ID2890__Inf_A_4_5d.png |right|frame|Other examples of 2D PDF&nbsp; $f_{XY}(x, y)$]]
-*$I(X;Y)$&nbsp; thus independent of WDF parameters&nbsp; $A$&nbsp; and&nbsp; $B$.
+*$I(X;Y)$&nbsp; thus independent of th PDF parameters&nbsp; $A$&nbsp; and&nbsp; $B$.
 {{ML-Fuß}}

@@ Line 36: / Line 36: @@
 Hints:
-*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_value-continuous_random_variables|Mutual_information with value-continuous input]].
+*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_value-continuous_random_variables|Mutual information with value-continuous input]].
 *Let the following differential entropies also be given:

@@ Line 36: / Line 36: @@
 Hints:
-*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_value-continuous_random_variables|AWGN channel capacitance with value-continuous input]].
+*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_value-continuous_random_variables|Mutual_information with value-continuous input]].
 *Let the following differential entropies also be given:

@@ Line 36: / Line 36: @@
 Hints:
-*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang|AWGN channel capacitance with continuous-value input]].
+*The exercise belongs to the chapter&nbsp;  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input#Mutual_information_between_value-continuous_random_variables|AWGN channel capacitance with value-continuous input]].
 *Let the following differential entropies also be given:

@@ Line 4: / Line 4: @@
 [[File:P_ID2886__Inf_A_4_5_neu.png|right|frame|Given joint PDF]]
-Given here are the three different 2D regions&nbsp; $f_{XY}(x, y)$, which in the task are identified by their fill colors with
+Given here are the three different two-dimensional regions&nbsp; $f_{XY}(x, y)$,&nbsp; which in the task are identified by their fill colors with
 * red  joint PDF,
-* blue joint PDF,&nbsp; und
+* blue joint PDF,
-* green joint PDF
+* green joint PDF,
 respectively.&nbsp; Within each of the regions shown, let&nbsp; $f_{XY}(x, y) = C = \rm const.$
-For example, the transinformation between the continuous-value random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; can be calculated as follows:
+For example, the mutual information between the value-continuous random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; can be calculated as follows:
 :$$I(X;Y) = h(X) + h(Y) - h(XY)\hspace{0.05cm}.$$
@@ Line 39: / Line 39: @@
 *Let the following differential entropies also be given:
-:* If&nbsp; $X$&nbsp; is triangularly distributed between&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$, then:
+:* If&nbsp; $X$&nbsp; is triangularly distributed between&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$,&nbsp; then:
 ::$$h(X) = {\rm log} \hspace{0.1cm} [\hspace{0.05cm}\sqrt{ e} \cdot (x_{\rm max} - x_{\rm min})/2\hspace{0.05cm}]\hspace{0.05cm}.$$
-:* If&nbsp; $Y$&nbsp; is equally distributed between&nbsp; $y_{\rm min}$&nbsp; and&nbsp; $y_{\rm max}$, then holds:
+:* If&nbsp; $Y$&nbsp; is equally distributed between&nbsp; $y_{\rm min}$&nbsp; and&nbsp; $y_{\rm max}$,&nbsp; then holds:
 ::$$h(Y) = {\rm log} \hspace{0.1cm} \big [\hspace{0.05cm}y_{\rm max} - y_{\rm min}\hspace{0.05cm}\big ]\hspace{0.05cm}.$$
-*All results should be expressed in "bit".&nbsp; This is achieved with &nbsp; $\log$ &nbsp;&#8658;&nbsp; $\log_2$.
+*All results should be expressed in&nbsp; "bit". &nbsp; This is achieved with &nbsp; $\log$ &nbsp;&#8658;&nbsp; $\log_2$.
@@ Line 66: / Line 66: @@
 $I(X; Y) \ = \ $ { 0.721 3% } $\ \rm bit$
-{What conditions must the random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; satisfy simultaneously for &nbsp;$I(X;Y)  = 1/2 \cdot \log (\rm e)$&nbsp; to hold in gener
+{What conditions must the random variables&nbsp; $X$&nbsp; and&nbsp; $Y$&nbsp; satisfy simultaneously for &nbsp;$I(X;Y)  = 1/2 \cdot \log (\rm e)$&nbsp; to hold in general?
 |type="[]"}
-+ The joint WDF &nbsp;$f_{XY}(x, y)$&nbsp; results in a parallelogram.
++ The two-dimensional PDF &nbsp;$f_{XY}(x, y)$&nbsp; results in a parallelogram.
 + One of the random variables&nbsp; $(X$ &nbsp;or&nbsp; $Y)$&nbsp; is uniformly distributed.
 + The other random variable&nbsp; $(Y$&nbsp; or&nbsp; $X)$&nbsp; is triangularly distributed.

← Older revision		Revision as of 12:31, 1 October 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~AWGN–Kanalkapazität bei wertkontinuierlichem Eingang~~	+	{{quiz-Header\|Buchseite=Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input
	}}		}}

← Older revision	Revision as of 14:26, 20 August 2021
(No difference)