Aufgaben:Exercise 3.14: Channel Coding Theorem - Revision history

Hwang at 12:50, 17 November 2022

2022-11-17T12:50:28Z

Guenter at 11:57, 24 September 2021

2021-09-24T11:57:58Z

Guenter at 10:46, 24 September 2021

2021-09-24T10:46:06Z

Guenter at 10:36, 24 September 2021

2021-09-24T10:36:00Z

Noah at 09:11, 24 September 2021

2021-09-24T09:11:23Z

Noah at 12:28, 23 September 2021

2021-09-23T12:28:38Z

Noah at 13:20, 20 September 2021

2021-09-20T13:20:15Z

Show changes

Guenter at 15:39, 30 August 2021

2021-08-30T15:39:37Z

Guenter: Guenter moved page Aufgaben:Aufgabe 3.14: Kanalcodierungstheorem to Aufgaben:Exercise 3.14: Channel Coding Theorem

2021-07-22T14:58:21Z

Guenter moved page Aufgabe 3.14: Kanalcodierungstheorem to Exercise 3.14: Channel Coding Theorem

Javier: Text replacement - "„" to """

2021-05-28T14:38:43Z

Text replacement - "„" to """

@@ Line 29: / Line 29: @@
 Hints:
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung|Application to Digital Signal Transmission]].
-*Reference is made in particular to the page&nbsp;    [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung#Definition_and_meaning_of_channel_capacity|Definition and meaning of channel capacity]].
+*Reference is made in particular to the section&nbsp;    [[Information_Theory/Anwendung_auf_die_Digitalsignalübertragung#Definition_and_meaning_of_channel_capacity|Definition and meaning of channel capacity]].
@@ Line 94: / Line 94: @@
 '''(3)''' &nbsp; <u>Proposed solution 1</u>&nbsp; is correct:
-*From the graph on the information page,&nbsp; it can be read for the capacities of the two channels:
+*From the graph in the information section,&nbsp; it can be read for the capacities of the two channels:
 :$$C_{\rm BSC} = 0.1887 \ \rm {bit/use}, \hspace{0.5cm}C_{rm EUC} = 0.3219 \ \rm {bit/use}.$$
 *According to the channel coding theorem,&nbsp; a special channel coding can be found at&nbsp; $R ≤ C$&nbsp; with which the error probability can be made zero.

← Older revision		Revision as of 11:57, 24 September 2021
Line 114:		Line 114:


−	[[Category:Information Theory: Exercises\|^3.3 ~~Anwendung auf DSÜ-Kanäle~~^]]	+	[[Category:Information Theory: Exercises\|^3.3 Application to Digital Signal Transmission^]]

@@ Line 74: / Line 74: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)''' &nbsp; <u>Proposed solution 3</u> is correct:
+'''(1)''' &nbsp; <u>Proposed solution 3</u>&nbsp; is correct:
-*The BSC error probability is&nbsp; &rArr; &nbsp; $R = 1$ with $p_0 = p_1 = 0.5$&nbsp; for uncoded transmission: &nbsp;
+*The BSC error probability is with $p_0 = p_1 = 0.5$&nbsp; for uncoded transmission&nbsp; &rArr; &nbsp; $R = 1$: &nbsp;
 :$$ p_{\rm B} = 0.5 \cdot 0.25 + 0.5 \cdot 0.25=0.25 \hspace{0.05cm}.$$
 *Correspondingly, with the same boundary conditions for the EUC model::
@@ Line 82: / Line 82: @@
-'''(2)''' &nbsp; <u>Proposed solution 3</u> is correct:
+'''(2)''' &nbsp; <u>Proposed solution 3</u>&nbsp; is correct:
-*In the BSC model with the distortion probability&nbsp; $ε = 0.25$&nbsp;, for uncoded transmission &nbsp; &rArr; &nbsp; $R = 1$ &nbsp;, the bit error probability is equal to&nbsp; $p_{\rm B} = 0.25$. regardless of&nbsp; $p_0$&nbsp; and&nbsp; $p_1$
+*In the BSC model with error probability&nbsp; $ε = 0.25$,&nbsp; for uncoded transmission &nbsp; &rArr; &nbsp; $R = 1$&nbsp; the bit error probability is&nbsp; $p_{\rm B} = 0.25$,&nbsp; regardless of&nbsp; $p_0$&nbsp; and&nbsp; $p_1$
 *In contrast, with the EUC model, for example, a smaller bit error probability is obtained with&nbsp; $p_0 = 0.6$&nbsp; and&nbsp; $p_1 = 0.4$&nbsp;:
 :$$p_{\rm B} = 0.6 \cdot 0 + 0.4 \cdot 0.5=0.2 \hspace{0.05cm}.$$
 *Note, however, that now the source entropy is no longer&nbsp; $H(X) = 1\ \rm  (bit)$&nbsp; but only&nbsp; $H(X) = H_{bin} (0.6) = 0.971 \ \rm   (bit)$.
-*In the limiting case&nbsp; $p_0 = 1$&nbsp;, only zeros are transmitted and&nbsp; $H(X) = 0$.&nbsp; For the bit error probability, however, the following then actually applies:
+*In the limiting case&nbsp; $p_0 = 1$,&nbsp; only zeros are transmitted and&nbsp; $H(X) = 0$.&nbsp; For the bit error probability, however, the following then actually applies:
 :$$ p_{\rm B} = 1 \cdot 0 + 0 \cdot 0.5=0 \hspace{0.05cm}.$$
 :So no information is transmitted, but it is transmitted with the bit error probability "zero".
@@ Line 93: / Line 93: @@
-'''(3)''' &nbsp; <u>Proposed solution 1</u> is correct:
+'''(3)''' &nbsp; <u>Proposed solution 1</u>&nbsp; is correct:
-*From the graph on the information page, it can be read for the capacities of the two channels:
+*From the graph on the information page,&nbsp; it can be read for the capacities of the two channels:
 :$$C_{\rm BSC} = 0.1887 \ \rm {bit/use}, \hspace{0.5cm}C_{rm EUC} = 0.3219 \ \rm {bit/use}.$$
-*According to the channel coding theorem, a channel coding can be found at&nbsp; $R ≤ C$&nbsp; with which the error probability can be made zero.
+*According to the channel coding theorem,&nbsp; a special channel coding can be found at&nbsp; $R ≤ C$&nbsp; with which the error probability can be made zero.
 *For both channels this condition is true with rate&nbsp; $R = 0.16$&nbsp;.
-'''(4)''' &nbsp; <u>Proposed solution 3</u> is correct:
+'''(4)''' &nbsp; <u>Proposed solution 3</u> &nbsp;is correct:
-*With the EUC model, the necessary condition &nbsp; $R ≤ C$&nbsp; for an error-free transmission is fulfilled with &nbsp;$R = 0.32$ &nbsp;and&nbsp; $C = 0.3219$&nbsp;
+*With the EUC model, the necessary condition &nbsp; $R ≤ C$&nbsp; for an error-free transmission is fulfilled with &nbsp;$R = 0.32$ &nbsp;and&nbsp; $C = 0.3219$.&nbsp;
 *However, the prerequisite for this is the probability function &nbsp; $P_X(X) = (0.6,\ 0.4).$
-*In contrast, for equally probable symbols &nbsp; &rArr; &nbsp; $P_X(X) = (0.5,\ 0.5)$&nbsp; the mutual information&nbsp; $I(X; Y) = 0.3113$ would result, <br>i.e. a smaller value than for the channel capacity&nbsp; $C$, and&nbsp; $I(X; Y) < R$ also applies.
+*In contrast, for equally probable symbols &nbsp; &rArr; &nbsp; $P_X(X) = (0.5,\ 0.5)$&nbsp; the mutual information&nbsp; $I(X; Y) = 0.3113$&nbsp; would result, <br>i.e. a smaller value than for the channel capacity&nbsp; $C$, and&nbsp; $I(X; Y) < R$&nbsp; also applies.
-*It can be seen that the EUC model offers more potential for the application of channel coding than the BSC model.&nbsp; Here, for example, it can be exploited in the code that a transmitted "0" is always transmitted without errors.
+*It can be seen that the EUC model offers more potential for the application of channel coding than the BSC model.&nbsp; Here, for example, it can be exploited in the code that a transmitted&nbsp; "0"&nbsp; is always transmitted without errors.
-'''(5)''' &nbsp; The comments on subtasks&nbsp; '''(3)'''&nbsp; and&nbsp; '''(4)'''&nbsp; show that the<u>proposed solution 4</u> applies.
+'''(5)''' &nbsp; The comments on subtasks&nbsp; '''(3)'''&nbsp; and&nbsp; '''(4)'''&nbsp; show that the&nbsp; <u>proposed solution 4</u>&nbsp; applies.
 {{ML-Fuß}}

@@ Line 4: / Line 4: @@
 [[File:EN_Inf_A_3_14.png|right|frame|Information-theoretical quantities of the&nbsp; $\rm BSC$ &nbsp; and&nbsp; $\rm EUC models$]]
-[https://en.wikipedia.org/wiki/Claude_Shannon Shannon's]&nbsp; channel coding theorem states that an error-free transmission can be made over a ''discrete memoryless channel'' (DMC) with the code rate&nbsp; $R$&nbsp; as long as&nbsp; $R$&nbsp; is not greater than the channel capacity
+[https://en.wikipedia.org/wiki/Claude_Shannon Shannon's]&nbsp; channel coding theorem states that an error-free transmission can be made over a&nbsp; "discrete memoryless channel"&nbsp; $\rm(DMC)$&nbsp; with the code rate&nbsp; $R$&nbsp; as long as&nbsp; $R$&nbsp; is not greater than the channel capacity
 :$$C = \max_{P_X(X)} \hspace{0.15cm} I(X;Y) \hspace{0.05cm}.$$
-The channel coding theorem is to be evaluated numerically in this task, whereby two typical channel models are to be considered:
+The&nbsp; "Channel Coding Theorem"&nbsp; is to be evaluated numerically in this task, whereby two typical channel models are to be considered:
-* the&nbsp; $\rm BSC$ model&nbsp; (''Binary Symmetric Channel'')&nbsp; with distortion probability&nbsp;$ε = 0.25$&nbsp; and channel capacity &nbsp;$C = 1 - H_{\rm bin}(ε),$
+* The&nbsp; $\rm BSC$&nbsp; model&nbsp; ("Binary Symmetric Channel")&nbsp; with error probability&nbsp;$ε = 0.25$&nbsp; and channel capacity &nbsp;$C = 1 - H_{\rm bin}(ε),$
-* the&nbsp; $\rm EUC$ model&nbsp; (from&nbsp; ''Extremely Unsymmetric Channel'', this designation originates from us and is not common) according to)&nbsp; [[Aufgaben:Aufgabe_3.11Z:_Extrem_unsymmetrischer_Kanal|Exercise 3.11Z]].
+* the&nbsp; $\rm EUC$&nbsp; model&nbsp; (from&nbsp; "Extremely Unsymmetric Channel";&nbsp; this designation originates from us and is not common) according to&nbsp; [[Aufgaben:Aufgabe_3.11Z:_Extrem_unsymmetrischer_Kanal|Exercise 3.11Z]].
-The graphs show the numerical values of the information-theoretical quantities for the two channels&nbsp; $\rm BSC$&nbsp; and&nbsp; $\rm EUC$:
+The graphs show the numerical values of the information-theoretical quantities for the two models&nbsp; $\rm BSC$&nbsp; and&nbsp; $\rm EUC$:
-* the source entropy&nbsp; $H(X),$
+* The source entropy&nbsp; $H(X),$
 * the equivocation&nbsp; $H(X|Y),$
 * the mutual information&nbsp; $I(X; Y),$
-* the irrelevance&nbsp; $H(Y|X),$&nbsp; and
+* the irrelevance&nbsp; $H(Y|X),$&nbsp;
 * the sink entropy&nbsp; $H(Y).$
@@ Line 37: / Line 37: @@
 <quiz display=simple>
-{Which statements are true for uncoded transmission &nbsp; &rArr; &nbsp;$\underline{R = 1}$ assuming&nbsp;$p_0 = p_1 = 0.5$&nbsp;?
+{Which statements are true for uncoded transmission &nbsp; &rArr; &nbsp;$\underline{R = 1}$ assuming&nbsp; $p_0 = p_1 = 0.5$&nbsp;?
 |type="()"}
 - BSC results in a smaller bit error probability.
@@ Line 43: / Line 43: @@
 + Both models lead to the same bit error probability.
-{With &nbsp;$\underline{R = 1}$&nbsp; can the result be (formally) improved by other values of&nbsp; $p_0$ &nbsp;or&nbsp; $p_1&nbsp;?
+{With &nbsp;$\underline{R = 1}$&nbsp; the result can be (formally) improved by other values of&nbsp; $p_0$ &nbsp;or&nbsp; $p_1&nbsp;?
 |type="()"}
-- For both channels.
+- With both channels.
 - With the BSC model.
 + With the EUC model.
@@ Line 52: / Line 52: @@
 {Over which channel can error-free transmission be achieved with the&nbsp; $\underline{R = 0.16}$&nbsp;?
 |type="()"}
-+ For both channels.
++ With both channels.
-- For the BSC model.
+- With the BSC model.
 - With the EUC model.
 - With none of the models.
@@ Line 60: / Line 60: @@
 |type="()"}
 - Over both channels.
-- For the BSC model.
+- With the BSC model.
 + With the EUC model.
-- With no model.
+- With none of the models.
 {Over which channel can error-free transmission be achieved with the rate&nbsp; $\underline{R = 0.48}$&nbsp;?
 |type="()"}
-- For both channels.
+- With both channels.
-- For the BSC model.
+- With the BSC model.
 - With the EUC model.
 + With none of the models.

← Older revision		Revision as of 09:11, 24 September 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Anwendung auf die Digitalsignalübertragung~~	+	{{quiz-Header\|Buchseite=Information_Theory/Application_to_Digital_Signal_Transmission
	}}		}}

@@ Line 3: / Line 3: @@
 }}
-[[File:P_ID2817__Inf_A_3_13.png|right|frame|Informationstheoretische Größen von&nbsp; $\rm BSC$–&nbsp; und&nbsp; $\rm EUC–Modell$]]
+[[File:EN_Inf_A_3_14.png|right|frame|Informationstheoretische Größen von&nbsp; $\rm BSC$–&nbsp; und&nbsp; $\rm EUC–Modell$]]
 [https://de.wikipedia.org/wiki/Claude_Shannon Shannons]&nbsp; Kanalcodierungstheorem besagt, dass über einen&nbsp; ''diskreten gedächtnislosen Kanal''&nbsp; (englisch:&nbsp; ''Discrete Memoryless Channel'',&nbsp; $\rm DMC)$&nbsp; mit der Coderate&nbsp; $R$&nbsp; fehlerfrei übertragen werden kann, so lange&nbsp; $R$&nbsp; nicht größer ist als die Kanalkapazität
 :$$C = \max_{P_X(X)} \hspace{0.15cm} I(X;Y) \hspace{0.05cm}.$$

← Older revision	Revision as of 14:58, 22 July 2021
(No difference)

@@ Line 90: / Line 90: @@
 *Im Grenzfall&nbsp; $p_0 = 1$&nbsp; werden nur noch Nullen übertragen und es gilt&nbsp; $H(X) = 0$.&nbsp; Für die Bitfehlerwahrscheinlichkeit gilt dann aber tatsächlich:
 :$$ p_{\rm B} = 1 \cdot 0 + 0 \cdot 0.5=0 \hspace{0.05cm}.$$
-:Man überträgt also keinerlei Information, diese aber mit der Bitfehlerwahrscheinlichkeit &bdquo;Null".
+:Man überträgt also keinerlei Information, diese aber mit der Bitfehlerwahrscheinlichkeit "Null".