Aufgaben:Exercise 3.15: Data Processing Theorem - Revision history

Guenter at 12:22, 24 September 2021

2021-09-24T12:22:45Z

Guenter at 12:09, 24 September 2021

2021-09-24T12:09:09Z

Guenter at 12:08, 24 September 2021

2021-09-24T12:08:38Z

Guenter at 12:05, 24 September 2021

2021-09-24T12:05:06Z

Guenter at 11:58, 24 September 2021

2021-09-24T11:58:24Z

Noah at 09:12, 24 September 2021

2021-09-24T09:12:03Z

Noah at 12:41, 23 September 2021

2021-09-23T12:41:39Z

Show changes

Guenter: Guenter moved page Aufgaben:Aufgabe 3.15: Data Processing Theorem to Aufgaben:Exercise 3.15: Data Processing Theorem

2021-07-22T14:58:55Z

Guenter moved page Aufgabe 3.15: Data Processing Theorem to Exercise 3.15: Data Processing Theorem

Javier: Text replacement - "”" to """

2021-05-28T14:37:08Z

Text replacement - "”" to """

Javier: Text replacement - "„" to """

2021-05-28T14:34:50Z

Text replacement - "„" to """

@@ Line 73: / Line 73: @@
 :$$ I(X;Y) = 1 + p \cdot {\rm log}_2 \hspace{0.1cm} (p) + (1-p) \cdot {\rm log}_2 \hspace{0.1cm} (1-p) = 1 - H_{\rm bin}(p)\hspace{0.05cm},
 \hspace{1.0cm}H_{\rm bin}(p)= p \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p}+ (1-p) \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{1-p}\hspace{0.05cm}.$$
-*However, the result holds only for&nbsp; $P_X(X) = (0.5, \ 0.5)$ &nbsp; &rArr; &nbsp; maximale mutual information &nbsp; &rArr; &nbsp; channel capacity.
+*However, the result holds only for&nbsp; $P_X(X) = (0.5, \ 0.5)$ &nbsp; &rArr; &nbsp; maximum mutual information &nbsp; &rArr; &nbsp; channel capacity.
 *Otherwise,&nbsp; $I(X; Y)$&nbsp; is smaller.&nbsp; &nbsp; &nbsp;For example, for $P_X(X) = (1, \ 0)$: &nbsp;   $H(X) = 0$  &nbsp; &rArr; &nbsp;   $I(X; Y) = 0.$
-*The binary entropy function is concave, but this property was not used in the derivation &nbsp; &rArr; &nbsp;  Antwort 2 is incorrect.
+*The binary entropy function is concave, but this property was not used in the derivation &nbsp; &rArr; &nbsp;  answer 2 is incorrect.
@@ Line 94: / Line 94: @@
-'''(5)'''&nbsp; The answer is&nbsp; <u>YES</u>, of course&nbsp; ''Data Processing Theorem'' assumes exactly the conditions given here.
+'''(5)'''&nbsp; The answer is&nbsp; <u>YES</u>,&nbsp; because the&nbsp; "Data Processing Theorem" assumes exactly the conditions given here.
 However, we want to evaluate some additional numerical results:
-* If&nbsp; $0 ≤ p < 0.5$ &nbsp;and&nbsp; $0 ≤ q < 0.5$, hold, we obtain:
+* If&nbsp; $0 ≤ p < 0.5$ &nbsp;and&nbsp; $0 ≤ q < 0.5$&nbsp; hold, we obtain:
 :$$\varepsilon = p + q \cdot (1-2p) > p \hspace{0.3cm}\Rightarrow \hspace{0.3cm} I(X;Z) < I(X;Y) \hspace{0.05cm},$$
 :$$\varepsilon = q + p \cdot (1-2q) > q \hspace{0.3cm}\Rightarrow \hspace{0.3cm} I(X;Z) < I(Y;Z) \hspace{0.05cm}.$$
-*For &nbsp;$p = 0.5$&nbsp; holds independently of &nbsp;$q$, since&nbsp; $I(X; Z)$&nbsp; cannot be greater than&nbsp; $I(X; Y)$:
+*For &nbsp;$p = 0.5$&nbsp; holds independently of &nbsp;$q$,&nbsp; since&nbsp; $I(X; Z)$&nbsp; cannot be greater than&nbsp; $I(X; Y)$:
 :$$\varepsilon = 0.5 + q \cdot (1-1) = 0.5 \hspace{0.3cm}\Rightarrow \hspace{0.3cm} I(X;Y) = 0 \hspace{0.3cm}\Rightarrow \hspace{0.3cm} I(X;Z) = 0 \hspace{0.05cm}.$$
 *Similarly, with &nbsp;$q = 0.5$&nbsp; independent of &nbsp;$p$, we obtain:

@@ Line 30: / Line 30: @@
 Hints:
 *The task belongs to the chapter&nbsp; [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung| Application to Digital Signal Transmission]].
-*Reference is also made to the page&nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Chain_rule_of_the_mutual_information|Chain rule of the_mutual_information]]nbsp; in the previous chapter]].
+*Reference is also made to the page&nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Chain_rule_of_the_mutual_information|Chain rule of the mutual information]]&nbsp; in the previous chapter]].

@@ Line 30: / Line 30: @@
 Hints:
 *The task belongs to the chapter&nbsp; [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung| Application to Digital Signal Transmission]].
-*Reference is also made to the page&nbsp; [[Information_Theory/Verschiedene_Entropien_zweidimensionaler_Zufallsgrößen#Chain_rule_of_the_mutual_information|Chain Rule of Transinformation in the previous chapter]].
+*Reference is also made to the page&nbsp; [[Information_Theory/Different_Entropy_Measures_of_Two-Dimensional_Random_Variables#Chain_rule_of_the_mutual_information|Chain rule of the_mutual_information]]nbsp; in the previous chapter]].

@@ Line 5: / Line 5: @@
 [[File:P_ID2818__Inf_A_3_14.png|right|frame|"Data Processing Theorem"]]
 We consider the following data processing chain:
-* Binary input data&nbsp; $X$&nbsp; is processed by processor&nbsp; $1$&nbsp; (top half in the graph) which is describable by conditional probabilities &nbsp; &rArr; &nbsp; $P_{Y\hspace{0.03cm}|\hspace{0.03cm}X}(\cdot)$&nbsp;.&nbsp; Its output variable is&nbsp; $Y$.
+* Binary input data&nbsp; $X$&nbsp; is processed by processor &nbsp; $1$&nbsp; (top half in the graph)&nbsp; which is describable by conditional probabilities &nbsp; &rArr; &nbsp; $P_{Y\hspace{0.03cm}|\hspace{0.03cm}X}(\cdot)$&nbsp;.&nbsp; Its output variable is&nbsp; $Y$.
-* A second processor with random variable&nbsp; $Y$&nbsp; at input and random variable&nbsp; $Z$&nbsp; at output is given by&nbsp; $P_{Z\hspace{0.03cm}|\hspace{0.03cm}Y}(\cdot)$&nbsp; gegeben&nbsp;  (lower half in the graph).&nbsp; $Z$&nbsp; depends on&nbsp; $Y$&nbsp; alone&nbsp; (deterministic or stochastic)&nbsp; and is independent of&nbsp; $X$:
+* A second processor with random variable&nbsp; $Y$&nbsp; at input and random variable&nbsp; $Z$&nbsp; at output is given by&nbsp; $P_{Z\hspace{0.03cm}|\hspace{0.03cm}Y}(\cdot)$&nbsp;   (lower half in the graph).&nbsp; $Z$&nbsp; depends on&nbsp; $Y$&nbsp; alone&nbsp; (deterministic or stochastic)&nbsp; and is independent of&nbsp; $X$:
 :$$P_{Z\hspace{0.05cm}|\hspace{0.03cm} XY\hspace{-0.03cm}}(z\hspace{0.03cm}|\hspace{0.03cm} x, y) =P_{Z\hspace{0.05cm}|\hspace{0.03cm} Y\hspace{-0.03cm}}(z\hspace{0.03cm}|\hspace{0.03cm} y) \hspace{0.05cm}.$$
 The following nomenclature was used for this description:
@@ Line 14: / Line 14: @@
 This also means:
-$X → Y → Z$&nbsp; form a&nbsp; [[Theory_of_Stochastic_Signals/Markovketten|Markov chain]].&nbsp; For such a one, the&nbsp; ''Data Processing Theorem''&nbsp; holds with the following consequence:
+$X → Y → Z$&nbsp; form a&nbsp; [[Theory_of_Stochastic_Signals/Markovketten|Markov chain]].&nbsp; For such a one, the&nbsp; "Data Processing Theorem"&nbsp; holds with the following consequence:
 :$$I(X;Z)   \le  I(X;Y ) \hspace{0.05cm}, $$
 :$$I(X;Z)   \le   I(Y;Z ) \hspace{0.05cm}.$$
 The theorem thus states:
-:* One cannot gain additional information about input&nbsp; $X$&nbsp; by manipulating &nbsp; (''processing'')&nbsp; data&nbsp; $Y$&nbsp;.
+:* One cannot gain additional information about input&nbsp; $X$&nbsp; by manipulating &nbsp; ("processing")&nbsp; data&nbsp; $Y$&nbsp;.
 :* Data processing&nbsp; (by processor &nbsp; $1$)&nbsp; serves only the purpose of making the information contained in&nbsp; $X$&nbsp; more visible.
@@ Line 42: / Line 42: @@
 + The derivation is based on the properties of a strictly symmetric channel.
 - It is exploited that &nbsp;$H_{\rm bin}(p)$&nbsp; is a concave function.
-- The result is valid for any probability function &nbsp;$P_X(X)$.
+- The result is valid for any probability mass function &nbsp;$P_X(X)$.

← Older revision		Revision as of 11:58, 24 September 2021
Line 112:		Line 112:


−	[[Category:Information Theory: Exercises\|^3.3 ~~Anwendung auf DSÜ-Kanäle~~^]]	+	[[Category:Information Theory: Exercises\|^3.3 Application to Digital Signal Transmission^]]

← Older revision		Revision as of 09:12, 24 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Anwendung auf die Digitalsignalübertragung~~	+	{{quiz-Header\|Buchseite=Information_Theory/Application_to_Digital_Signal_Transmission
	}}		}}

← Older revision	Revision as of 14:58, 22 July 2021
(No difference)

@@ Line 3: / Line 3: @@
 }}
-[[File:P_ID2818__Inf_A_3_14.png|right|frame|"Data Processing Theorem&rdquo;]]
+[[File:P_ID2818__Inf_A_3_14.png|right|frame|"Data Processing Theorem"]]
 Wir betrachten die folgende Datenverarbeitungskette:
 * Binäre Eingangsdaten&nbsp; $X$&nbsp; werden durch den Prozessor&nbsp; $1$&nbsp; (obere Hälfte in der Grafik) verarbeitet, der durch bedingte Wahrscheinlichkeiten &nbsp; &rArr; &nbsp; $P_{Y\hspace{0.03cm}|\hspace{0.03cm}X}(\cdot)$&nbsp; beschreibbar ist.&nbsp; Dessen Ausgangsgröße ist&nbsp; $Y$.
@@ Line 57: / Line 57: @@
 $I(X; Z) \ = \ $ { 0.173 3% } $\ \rm bit$
-{Erfüllt dieses Beispiel das&nbsp; "Data Processing Theorem&rdquo;?
+{Erfüllt dieses Beispiel das&nbsp; "Data Processing Theorem"?
 |type="()"}
 + Ja,