Channel Coding/Information Theoretical Limits of Channel Coding - Revision history

Guenter at 10:12, 22 November 2022

2022-11-22T10:12:08Z

Guenter at 10:03, 22 November 2022

2022-11-22T10:03:16Z

Guenter at 10:00, 22 November 2022

2022-11-22T10:00:26Z

Hwang at 15:28, 18 November 2022

2022-11-18T15:28:08Z

Hwang at 15:27, 18 November 2022

2022-11-18T15:27:48Z

Hwang at 15:27, 18 November 2022

2022-11-18T15:27:19Z

Hwang at 15:18, 18 November 2022

2022-11-18T15:18:34Z

Hwang at 15:16, 18 November 2022

2022-11-18T15:16:01Z

Hwang at 15:15, 18 November 2022

2022-11-18T15:15:02Z

Hwang at 15:11, 18 November 2022

2022-11-18T15:11:47Z

@@ Line 194: / Line 194: @@
 #For all binary codes holds per se&nbsp; $0 < R &#8804; 1$. Only with non-binary codes&nbsp; &rArr; &nbsp; rates&nbsp; $R > 1$&nbsp; are possible.&nbsp; For example,&nbsp; the maximum possible code rate of a quaternary code:&nbsp; $R = \log_2 \, M_y = \log_2 \, 4 = 2$.<br>
 #All one-dimensional modulation types &ndash; i.e., those methods that use only the in-phase&ndash; or only the quadrature component such as&nbsp; [[Digital_Signal_Transmission/Carrier_Frequency_Systems_with_Coherent_Demodulation#On.E2.80.93off_keying_.282.E2.80.93ASK.29|$\text{2&ndash;ASK}$]],&nbsp;  [[Digital_Signal_Transmission/Carrier_Frequency_Systems_with_Coherent_Demodulation#Binary_phase_shift_keying_.28BPSK.29 |$\text{BPSK}$]]&nbsp; and&nbsp; [[Digital_Signal_Transmission/Carrier_Frequency_Systems_with_Non-Coherent_Demodulation#Non-coherent_demodulation_of_binary_FSK_.282.E2.80.93FSK.29|$\text{2&ndash;FSK}$]]&nbsp; must be in the blue area of the present graphic.<br>
-#As shown in the chapter&nbsp; [[Information_Theory/AWGN_Channel_Capacity_for_Discrete_Input#Maximum_code_rate_for_QAM_structures|$\text{Maximum code rate for QAM structures}$]],&nbsp;  there is a&nbsp; "friendlier"&nbsp; limit curve for two-dimensional modulation types such as the&nbsp; [[Modulation_Methods/Quadrature_Amplitude_Modulation|$\text{Quadrature Amplitude Modulation}$]].
+#As shown in the chapter&nbsp; [[Information_Theory/AWGN_Channel_Capacity_for_Discrete_Input#Maximum_code_rate_for_QAM_structures|"Maximum code rate for QAM structures"]],&nbsp;  there is a&nbsp; "friendlier"&nbsp; limit curve for two-dimensional modulation types such as the&nbsp; [[Modulation_Methods/Quadrature_Amplitude_Modulation|"Quadrature Amplitude Modulation"]].
 == AWGN channel capacity for binary input signals ==
@@ Line 275: / Line 275: @@
 #These codes,&nbsp; invented by&nbsp; [http://perso.telecom-bretagne.eu/claudeberrou/ $\text{Claude Berrou}$]&nbsp; around 1990,&nbsp; can be decoded iteratively.&nbsp; The&nbsp; (red)&nbsp; marks are each less than&nbsp; $1 \, \rm dB$&nbsp; from the Shannon bound.<br>
 #Similar behavior is shown by&nbsp; [[Channel_Coding/The_Basics_of_Low-Density_Parity_Check_Codes|$\text{LDPC codes}$]]&nbsp; ("Low Density Parity&ndash;check Codes")&nbsp; marked by white rectangles,&nbsp; which have been used since in 2006 on&nbsp; [https://en.wikipedia.org/wiki/DVB-S2 $\text{DVB&ndash;S(2)}$]&nbsp; ("Digital Video Broadcast over Satellite").
-#These are well suited for iterative decoding using&nbsp; "factor&ndash;graphs"&nbsp; &nbsp; and &nbsp; "exit charts"&nbsp; due to the sparse occupancy of the parity-check matrix&nbsp; $\boldsymbol {\rm H}$&nbsp; (with "ones").&nbsp; See  [Hag02]<ref name='Hag02'>Hagenauer, J.: The Turbo Principle in Mobile Communications.&nbsp; In: Int. Symp. on Information Theory and Its Applications, Oct.2010,  [http://wwwmayr.in.tum.de/konferenzen/Jass05/courses/4/papers/prof_hagenauer.pdf PDF document.]</ref><br>
+#These are well suited for iterative decoding using&nbsp; "factor&ndash;graphs"&nbsp; &nbsp; and &nbsp; "exit charts"&nbsp; due to the sparse occupancy of the parity-check matrix&nbsp; $\boldsymbol {\rm H}$&nbsp; (with "ones").&nbsp; See  [Hag02]<ref name='Hag02'>Hagenauer, J.: The Turbo Principle in Mobile Communications.&nbsp; In: Int. Symp. on Information Theory and Its Applications, Oct.2010,&nbsp;  [http://wwwmayr.in.tum.de/konferenzen/Jass05/courses/4/papers/prof_hagenauer.pdf $\text{PDF document}$.]</ref><br>
 #The black dots mark the CCSDS specified&nbsp; [[Channel_Coding/The_Basics_of_Low-Density_Parity_Check_Codes|$\text{LDPC codes}$]],&nbsp; all of which assume a constant number of information bits&nbsp; $(k = 16384)$.&nbsp;
 #In contrast,&nbsp; for all white rectangles,&nbsp; the code length&nbsp; $n = 64800$&nbsp; is constant,&nbsp; while the number&nbsp; $k$&nbsp; of information bits changes according to the rate&nbsp; $R = k/n$.<br>

@@ Line 45: / Line 45: @@
 {{BlaueBox|TEXT=
-$\text{Definition:}$&nbsp;  The&nbsp; &raquo;'''channel capacity'''&laquo;&nbsp; introduced by Shannon gives the maximum mutual information&nbsp; $I(x; y)$&nbsp; between the input variable&nbsp; $x$&nbsp; and the output variable&nbsp; $y$:
+$\text{Definition:}$&nbsp;  The&nbsp; &raquo;'''channel capacity'''&laquo;&nbsp; introduced by Shannon gives the maximum mutual information&nbsp; $I(x; y)$&nbsp; between the input variable&nbsp; $x$&nbsp; and output variable&nbsp; $y$:
 ::<math>C = \max_{{{\rm Pr}(x_i)}} \hspace{0.1cm} I(X; Y) \hspace{0.05cm}.</math>
-The pseudo&ndash;unit&nbsp; "bit/channel use"&nbsp; must be added.}}<br>
+*The pseudo&ndash;unit&nbsp; "bit/channel use"&nbsp; must be added.}}<br>
 :Since the maximization of the mutual information&nbsp; $I(x; y)$&nbsp; must be done over all possible&nbsp; (discrete)&nbsp; input distributions&nbsp; ${\rm Pr}(x_i)$,

@@ Line 21: / Line 21: @@
 Notes:&nbsp;
-*The statement&nbsp; "The error probability approaches zero"&nbsp; is not identical with the statement&nbsp; "The transmission is error-free".&nbsp; Example: &nbsp; For an infinitely long sequence,&nbsp; finitely many symbols are corrupted '''KORREKTUR: falsified'''.
+*The statement&nbsp; "The error probability approaches zero"&nbsp; is not identical with the statement&nbsp; "The transmission is error-free".&nbsp; Example: &nbsp; For an infinitely long sequence,&nbsp; finitely many symbols are falsified.
 *For some channels,&nbsp; even with&nbsp; $R=C$&nbsp; the error probability still goes towards zero&nbsp; (but not for all).
@@ Line 144: / Line 144: @@
 ::<math>P_X  = \frac{E_{\rm S}}{T_{\rm S} } \hspace{0.05cm}, \hspace{0.4cm} P_N  = \frac{N_0}{2T_{\rm S} }\hspace{0.05cm}. </math>
-*Thus, the AWGN channel capacity can also be expressed by the&nbsp; &raquo;'''transmit '''KORREKTUR: transmitted?''' energy per symbol'''&laquo;&nbsp; $(E_{\rm S})$&nbsp; and the&nbsp; &raquo;'''noise power density'''&laquo;&nbsp; $(N_0)$:
+*Thus, the AWGN channel capacity can also be expressed by the&nbsp; &raquo;transmitted energy per symbol'''&laquo;&nbsp; $(E_{\rm S})$&nbsp; and the&nbsp; &raquo;'''noise power density'''&laquo;&nbsp; $(N_0)$:
 ::<math>C =  {1}/{2 } \cdot {\rm log_2 } \hspace{0.05cm}\left ( 1 + {2 E_{\rm S}}/{N_0 } \right )\hspace{0.05cm}, \hspace{1.9cm}
@@ Line 161: / Line 161: @@
-*However,&nbsp; a comparison of different coding '''KORREKTUR: encoding''' methods at constant&nbsp; "energy per transmitted symbol" &nbsp; &rArr; &nbsp; $E_{\rm S}$&nbsp; is not fair.&nbsp;
+*However,&nbsp; a comparison of different encoding methods at constant&nbsp; "energy per transmitted symbol" &nbsp; &rArr; &nbsp; $E_{\rm S}$&nbsp; is not fair.&nbsp;
 *Rather,&nbsp; for this comparison,&nbsp; the&nbsp; "energy per source bit" &nbsp; &rArr; &nbsp; $E_{\rm B}$&nbsp; should be fixed.&nbsp; The following relationships apply:
@@ Line 284: / Line 284: @@
 *For two-dimensional methods such as QAM and multilevel PSK, on the other hand, the "Gaussian (complex)" limit curve applies, which is drawn here as a dashed line and always lies to the left of the solid curve.
 *For more details, see the&nbsp; [[Information_Theory/AWGN_Channel_Capacity_for_Discrete_Input#Maximum_code_rate_for_QAM_structures|$\text{Maximum code rate for QAM structures}$]]&nbsp; section of the "Information Theory" book.<br>
-*Also, this limit curve has now been nearly reached with QAM procedures '''KORREKTUR: methods''' and very long channel codes, without ever being exceeded.}}<br>
+*Also, this limit curve has now been nearly reached with QAM  methods and very long channel codes, without ever being exceeded.}}<br>
 == Exercises for the chapter ==

@@ Line 275: / Line 275: @@
 #These codes,&nbsp; invented by&nbsp; [http://perso.telecom-bretagne.eu/claudeberrou/ $\text{Claude Berrou}$]&nbsp; around 1990,&nbsp; can be decoded iteratively.&nbsp; The&nbsp; (red)&nbsp; marks are each less than&nbsp; $1 \, \rm dB$&nbsp; from the Shannon bound.<br>
 #Similar behavior is shown by&nbsp; [[Channel_Coding/The_Basics_of_Low-Density_Parity_Check_Codes|$\text{LDPC codes}$]]&nbsp; ("Low Density Parity&ndash;check Codes")&nbsp; marked by white rectangles,&nbsp; which have been used since in 2006 on&nbsp; [https://en.wikipedia.org/wiki/DVB-S2 $\text{DVB&ndash;S(2)}$]&nbsp; ("Digital Video Broadcast over Satellite").
-#These are well suited for iterative decoding using&nbsp; "factor&ndash;graphs"&nbsp; &nbsp; and &nbsp; "exit charts"&nbsp; due to the sparse occupancy of the parity-check matrix&nbsp; $\boldsymbol {\rm H}$&nbsp; (with "ones").&nbsp; See  [Hag02]<ref name='Hag02'>Hagenauer, J.: The Turbo Principle in Mobile Communications.&nbsp; In: Int. Symp. on Information Theory and Its Applications, Oct.2010,  [http://wwwmayr.in.tum.de/konferenzen/Jass05/courses/4/papers/prof_hagenauer.pdf PDF dokument.]</ref><br>
+#These are well suited for iterative decoding using&nbsp; "factor&ndash;graphs"&nbsp; &nbsp; and &nbsp; "exit charts"&nbsp; due to the sparse occupancy of the parity-check matrix&nbsp; $\boldsymbol {\rm H}$&nbsp; (with "ones").&nbsp; See  [Hag02]<ref name='Hag02'>Hagenauer, J.: The Turbo Principle in Mobile Communications.&nbsp; In: Int. Symp. on Information Theory and Its Applications, Oct.2010,  [http://wwwmayr.in.tum.de/konferenzen/Jass05/courses/4/papers/prof_hagenauer.pdf PDF document.]</ref><br>
 #The black dots mark the CCSDS specified&nbsp; [[Channel_Coding/The_Basics_of_Low-Density_Parity_Check_Codes|$\text{LDPC codes}$]],&nbsp; all of which assume a constant number of information bits&nbsp; $(k = 16384)$.&nbsp;
 #In contrast,&nbsp; for all white rectangles,&nbsp; the code length&nbsp; $n = 64800$&nbsp; is constant,&nbsp; while the number&nbsp; $k$&nbsp; of information bits changes according to the rate&nbsp; $R = k/n$.<br>

@@ Line 275: / Line 275: @@
 #These codes,&nbsp; invented by&nbsp; [http://perso.telecom-bretagne.eu/claudeberrou/ $\text{Claude Berrou}$]&nbsp; around 1990,&nbsp; can be decoded iteratively.&nbsp; The&nbsp; (red)&nbsp; marks are each less than&nbsp; $1 \, \rm dB$&nbsp; from the Shannon bound.<br>
 #Similar behavior is shown by&nbsp; [[Channel_Coding/The_Basics_of_Low-Density_Parity_Check_Codes|$\text{LDPC codes}$]]&nbsp; ("Low Density Parity&ndash;check Codes")&nbsp; marked by white rectangles,&nbsp; which have been used since in 2006 on&nbsp; [https://en.wikipedia.org/wiki/DVB-S2 $\text{DVB&ndash;S(2)}$]&nbsp; ("Digital Video Broadcast over Satellite").
-#These are well suited for iterative decoding using&nbsp; "factor&ndash;graphs"&nbsp; &nbsp; and &nbsp; "exit charts"&nbsp; due to the sparse occupancy of the parity-check matrix&nbsp; $\boldsymbol {\rm H}$&nbsp; (with "ones").&nbsp; See  [Hag02]<ref name='Hag02'>Hagenauer, J.: The Turbo Principle in Mobile Communications.&nbsp; In: Int. Symp. on Information Theory and Its Applications, Oct.2010,  [http://wwwmayr.in.tum.de/konferenzen/Jass05/courses/4/papers/prof_hagenauer.pdf PDF&ndash;Dokument.]</ref><br>
+#These are well suited for iterative decoding using&nbsp; "factor&ndash;graphs"&nbsp; &nbsp; and &nbsp; "exit charts"&nbsp; due to the sparse occupancy of the parity-check matrix&nbsp; $\boldsymbol {\rm H}$&nbsp; (with "ones").&nbsp; See  [Hag02]<ref name='Hag02'>Hagenauer, J.: The Turbo Principle in Mobile Communications.&nbsp; In: Int. Symp. on Information Theory and Its Applications, Oct.2010,  [http://wwwmayr.in.tum.de/konferenzen/Jass05/courses/4/papers/prof_hagenauer.pdf PDF dokument.]</ref><br>
 #The black dots mark the CCSDS specified&nbsp; [[Channel_Coding/The_Basics_of_Low-Density_Parity_Check_Codes|$\text{LDPC codes}$]],&nbsp; all of which assume a constant number of information bits&nbsp; $(k = 16384)$.&nbsp;
 #In contrast,&nbsp; for all white rectangles,&nbsp; the code length&nbsp; $n = 64800$&nbsp; is constant,&nbsp; while the number&nbsp; $k$&nbsp; of information bits changes according to the rate&nbsp; $R = k/n$.<br>

← Older revision		Revision as of 15:16, 18 November 2022
Line 161:		Line 161:


−	*However,  a comparison of different coding methods at constant  "energy per transmitted symbol"   ⇒   $E_{\rm S}$  is not fair.	+	*However,  a comparison of different coding '''KORREKTUR: encoding''' methods at constant  "energy per transmitted symbol"   ⇒   $E_{\rm S}$  is not fair.

	*Rather,  for this comparison,  the  "energy per source bit"   ⇒   $E_{\rm B}$  should be fixed.  The following relationships apply:		*Rather,  for this comparison,  the  "energy per source bit"   ⇒   $E_{\rm B}$  should be fixed.  The following relationships apply:

@@ Line 130: / Line 130: @@
 *The input quantity&nbsp; $x$&nbsp; optimized with respect to maximum mutual information will certainly be continuous-valued,&nbsp; that is,&nbsp; for the AWGN channel in addition to &nbsp; $M_y \to\infty$ &nbsp; also holds &nbsp; $M_x \to\infty$.
-*While for discrete-valued input optimization is to be done over all probabilities&nbsp; ${\rm Pr}(x_i)$&nbsp; now optimization is done using the&nbsp; [[Theory_of_Stochastic_Signals/Gaussian_Distributed_Random_Variables#Probability_density_function_. E2.80.93_Cumulative_density_function|$\text{probability density funcion&nbsp; $\rm (PDF)$}$]]&nbsp; $f_x(x)$&nbsp; of the input signal under the constraint&nbsp; [[Digital_Signal_Transmission/Optimization_of_Baseband_Transmission_Systems#Power_and_peak_limitation|$\text{power limitation}$]]:
+*While for discrete-valued input optimization is to be done over all probabilities&nbsp; ${\rm Pr}(x_i)$&nbsp; now optimization is done using the&nbsp; [[Theory_of_Stochastic_Signals/Gaussian_Distributed_Random_Variables#Probability_density_function_.E2.80.93_Cumulative_density_function|$\text{probability density function&nbsp; $\rm (PDF)$}$]]&nbsp; $f_x(x)$&nbsp; of the input signal under the constraint&nbsp; [[Digital_Signal_Transmission/Optimization_of_Baseband_Transmission_Systems#Power_and_peak_limitation|$\text{power limitation}$]]:
 ::<math>C = \max_{f_x(x)} \hspace{0.1cm} I(x; y)\hspace{0.05cm},\hspace{0.3cm}{\rm where \hspace{0.15cm} must \hspace{0.15cm} apply} \text{:}\hspace{0.15cm} {\rm E} \left [ x^2 \right ] \le P_x   \hspace{0.05cm}.</math>