Information Theory/AWGN Channel Capacity for Discrete-Valued Input - Revision history

Guenter at 14:28, 28 February 2023

2023-02-28T14:28:56Z

Guenter at 14:27, 28 February 2023

2023-02-28T14:27:32Z

Guenter at 14:25, 28 February 2023

2023-02-28T14:25:58Z

Guenter at 14:24, 28 February 2023

2023-02-28T14:24:25Z

Guenter at 14:03, 26 February 2023

2023-02-26T14:03:29Z

Guenter at 16:58, 25 February 2023

2023-02-25T16:58:47Z

Show changes

Guenter at 16:12, 25 February 2023

2023-02-25T16:12:35Z

Show changes

Guenter: Guenter moved page Information Theory/AWGN Channel Capacity for Discrete Input to Information Theory/AWGN Channel Capacity for Discrete-Valued Input

2023-02-25T15:13:21Z

Guenter moved page Information Theory/AWGN Channel Capacity for Discrete Input to Information Theory/AWGN Channel Capacity for Discrete-Valued Input

Guenter at 15:12, 25 February 2023

2023-02-25T15:12:55Z

Guenter at 15:08, 25 February 2023

2023-02-25T15:08:23Z

@@ Line 377: / Line 377: @@
 ==Maximum code rate for QAM structures==
 <br>
-[[File:EN_Inf_T_4_3_S8b.png|right|frame|Channel capacity of&nbsp; $\rm BPSK$&nbsp; and&nbsp; $M\text{–QAM}$ '''KORREKTUR: capacity''']]
+[[File:EN_Inf_T_4_3_S8a_neu.png|right|frame|Channel capacity of&nbsp; $\rm BPSK$&nbsp; and&nbsp; $M\text{–QAM}$]]
 The graph shows the capacity of the complex AWGN channel as a red curve:
 :$$C_{\text{ Gaussian, complex} }= {\rm log}_2 \hspace{0.1cm} ( 1 + \rm SNR)

@@ Line 312: / Line 312: @@
 The early channel codes mentioned in&nbsp; $\text{Example 4}$&nbsp;  are still relatively far away from the channel capacity curve. &nbsp; This was probably also a reason why the author of this learning tutorial was unaware of the also great practical significance of information theory when he became acquainted with it in his studies in the early 1970s.&nbsp; The view changed significantly when very long channel codes together with iterative decoding appeared in the 1990s.&nbsp; The new marker points are much closer to the capacity limit curve.
 <br clear=all>
-[[File:EN_Inf_T_4_3_S6b.png|right|frame|Rates and required &nbsp;$E_{\rm B}/{N_0}$&nbsp;  of iterative coding methods '''KORREKTUR: Shannon boundary/limit?''']]
+[[File:EN_Inf_T_4_3_S6b_v2.png|right|frame|Rates and required &nbsp;$E_{\rm B}/{N_0}$&nbsp;  of iterative coding methods]]
 Here are a few more explanations of this graph:
 #Red crosses mark the so-called&nbsp; [https://en.wikipedia.org/wiki/Turbo_code "turbo codes"]&nbsp; according&nbsp; $\rm CCSDS$&nbsp; $($"Consultative Committee for Space Data Systems"$)$&nbsp; with&nbsp; $k = 6920$&nbsp; information bits each and different code lengths&nbsp; $n = k/R$.&nbsp;

@@ Line 209: / Line 209: @@
 <br>
 In the previous sections of this chapter we always assumed a Gaussian distributed,nbsp;  i.e. a continuous-valued AWGN input&nbsp; $X$&nbsp; according to Shannon theory.&nbsp;
-[[File:EN_Inf_T_4_3_S5a.png|right|frame|Calculation of the AWGN channel capacity for BPSK '''KORREKTUR: encoder''']]
+[[File:EN_Inf_T_4_3_S5a_v2.png|right|frame|Calculation of the AWGN channel capacity for BPSK]]
 [[File:EN_Inf_T_4_3_S5b.png|right|frame|Conditional PDF for&nbsp; $X=-1$&nbsp; $($red$)$&nbsp; and&nbsp; $X=+1$&nbsp; $($blue$)$ ]]

@@ Line 39: / Line 39: @@
   \delta(t- \nu \cdot T_{\rm A}
   )\hspace{0.05cm}.$$
-[[File:EN_Inf_T_4_3_S1b.png|right|frame| AWGN model considering time discretization and band-limitation  '''KORREKTUR: Abtastung''']]
+[[File:EN_Inf_T_4_3_S1b_v2.png|right|frame| AWGN model considering time discretization and band-limitation]]
 The spacing of all&nbsp; $($weighted$)$&nbsp; "Dirac delta functions"&nbsp; is uniform&nbsp; $T_{\rm A}$.&nbsp; Through the interpolation filter with the impulse response&nbsp; $h(t)$&nbsp; as well as the frequency response&nbsp; $H(f)$, where

@@ Line 335: / Line 335: @@
 == Channel capacity of the complex AWGN channel==
 <br>
-Higher-level modulation methods can each be represented by an in-phase and a quadrature component.&nbsp; These include, for example
+Higher-level modulation methods can each be represented by an&nbsp; "in-phase"&nbsp; and a&nbsp; "quadrature component".&nbsp; These include,&nbsp; for example
-*the&nbsp; [[Modulation_Methods/Quadrature_Amplitude_Modulation#Quadratic_QAM_signal_space_constellations|$\text{M–QAM}$]]&nbsp; &nbsp; ⇒  &nbsp; quadrature amplitude modulation;&nbsp; $M ≥ 4$&nbsp; quadrature signal space points,
+*the&nbsp; [[Modulation_Methods/Quadrature_Amplitude_Modulation#Quadratic_QAM_signal_space_constellations|$\text{M–QAM}$]]&nbsp; &nbsp; ⇒  &nbsp; quadrature amplitude modulation;&nbsp; $M ≥ 4$ &nbsp; quadrature signal space points,
-The two components can also be described in the&nbsp; [[Signal_Representation/Equivalent_Low-Pass_Signal_and_its_Spectral_Function|$\text{equivalent low-pass range}$]]&nbsp; as the&nbsp; "real part" &nbsp;and "imaginary part" &nbsp;of a complex noise term&nbsp; $N$,&nbsp; respectively.
-*All the above methods are two-dimensional.
+The two components can also be described in the&nbsp; [[Signal_Representation/Equivalent_Low-Pass_Signal_and_its_Spectral_Function|"equivalent low-pass range"]]&nbsp; as the&nbsp; "real part" &nbsp;and the&nbsp; "imaginary part" &nbsp;of a complex noise term&nbsp; $N$,&nbsp; respectively.
-*The&nbsp; (complex)&nbsp; AWGN channel thus provides &nbsp;$K = 2$&nbsp; independent Gaussian channels.
+*All the above methods are two-dimensional.
 *According to the section&nbsp;  [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang#Parallel_Gaussian_channels|"Parallel  Gaussian channels"]],&nbsp; the capacity of such a channel is therefore given by:
-[[File:P_ID2955__Inf_T_4_3_S7.png|right|frame|2D PDF of complex Gaussian noise]]
+[[File:P_ID2955__Inf_T_4_3_S7.png|right|frame|2D&ndash;PDF of complex Gaussian noise]]
 :$$C_{\text{ Gaussian, complex} }= C_{\rm total} ( K=2)
 =  {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_X/2}{\sigma^2})
 \hspace{0.05cm}.$$
-The sketch shows the two-dimensional PDF&nbsp; $f_N(n)$&nbsp; of the Gaussian noise process&nbsp; $N$&nbsp; over the two axes:
+* $N_{\rm Q}$&nbsp; $($quadrature component,&nbsp; imaginary part$)$.
-Darker areas of the rotationally symmetrical PDF&nbsp; $f_N(n)$&nbsp; around the zero point indicate more noise components.&nbsp; The following relationship applies to the variance of the complex Gaussian noise&nbsp; $N$&nbsp; due to the rotational invariance&nbsp; $(σ_{\rm I} = σ_{\rm Q})$&nbsp;:
+Darker areas of the rotationally symmetrical PDF&nbsp; $f_N(n)$&nbsp; around the zero point indicate more noise components.&nbsp; The following relationship applies to the variance of the complex Gaussian noise&nbsp; $N$&nbsp; due to the rotational invariance&nbsp; $(σ_{\rm I} = σ_{\rm Q})$:
 :$$\sigma_N^2 = \sigma_{\rm I}^2 + \sigma_{\rm Q}^2 = 2\cdot \sigma^2
 \hspace{0.05cm}.$$
-Thus, the channel capacity can also be expressed as follows:
+Thus,&nbsp; the channel capacity can also be expressed as follows:
 :$$C_{\text{ Gaussian, complex} }= {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_X}{\sigma_N^2})  = {\rm log}_2 \hspace{0.1cm} ( 1 + SNR)
 \hspace{0.05cm}.$$
-The equation is evaluated numerically in the next section.&nbsp; However, we can already say that for the signal-to-noise power ratio it will be:
+The equation is evaluated numerically in the next section.&nbsp; However,&nbsp; we can already say that for the signal-to-noise power ratio  will be:
-:$$SNR = {P_X}/{\sigma_N^2}
+:$$\rm SNR = {P_X}/{\sigma_N^2}
 \hspace{0.05cm}.$$
@@ Line 375: / Line 379: @@
 [[File:EN_Inf_T_4_3_S8b.png|right|frame|Channel capacity of&nbsp; $\rm BPSK$&nbsp; and&nbsp; $M\text{–QAM}$ '''KORREKTUR: capacity''']]
 The graph shows the capacity of the complex AWGN channel as a red curve:
-:$$C_{\text{ Gaussian, complex} }= {\rm log}_2 \hspace{0.1cm} ( 1 + SNR)
+:$$C_{\text{ Gaussian, complex} }= {\rm log}_2 \hspace{0.1cm} ( 1 + \rm SNR)
 \hspace{0.05cm}.$$
-*The unit of this channel capacity is "bit/channel use" or "bit/source symbol".
+*The unit of this channel capacity is&nbsp; "bit/channel use"&nbsp; or&nbsp; "bit/source symbol".
 *The abscissa denotes the signal-to-interference power ratio&nbsp; $10 · \lg (SNR)$&nbsp; with&nbsp; ${SNR} = P_X/σ_N^2$.
-*The graph was taken from&nbsp; [Göb10]<ref name='Göb10'>Göbel, B.:&nbsp; Information–Theoretic Aspects of Fiber–Optic Communication Channels'. Dissertation. TU München. <br>Verlag Dr. Hut, Reihe Informationstechnik, ISBN 978-3-86853-713-0, 2010.</ref>.&nbsp; &nbsp; We would like to thank our former colleague&nbsp; [[Biographies_and_Bibliographies/Beteiligte_der_Professur_Leitungsgebundene_Übertragungstechnik#Dr.-Ing._Bernhard_Göbel_(at_LÜT_from_2004-2010)|$\text{Bernhard Göbel}$]]&nbsp; for his permission to use this figure and for his great support of our learning tutorial during his entire active time.
+*The abscissa denotes the signal-to-interference power ratio&nbsp; $10 · \lg \rm (SNR)$&nbsp; with&nbsp; $\rm {SNR} = P_X/σ_N^2$.
 One can see from this representation:
-*All curves&nbsp; (BPSK and&nbsp; ''M''–QAM)&nbsp; lie to the right of the red Shannon boundary '''KORREKTUR: limit''' curve.&nbsp; At low&nbsp; $SNR$,&nbsp; however, all curves are almost indistinguishable from the red curve.
+#All curves&nbsp; $($BPSK and&nbsp; M–QAM$)$&nbsp; lie right of the red&nbsp; "Shannon limit"&nbsp; curve.&nbsp;
-*The final value of all curves for discrete-valued input signals is&nbsp; $K = \log_2 (M)$.&nbsp; For example, for&nbsp; $SNR  \to ∞$:&nbsp; $C_{\rm BPSK} = 1$&nbsp; (bit/use),&nbsp; $C_{\rm 4-QAM} = C_{\rm QPSK} = 2$.
+#At low&nbsp; SNR,&nbsp; however,&nbsp; all curves are almost indistinguishable from the red curve.
-*The blue markings show that a&nbsp; $\rm 2^{10}–QAM$&nbsp; with &nbsp;$10 · \lg (SNR) ≈ 27 \ \rm dB$&nbsp; allows a code rate of &nbsp;$R ≈ 8.2$.&nbsp; Distance to the Shannon curve:&nbsp; $1.53\ \rm dB$.
+#The final value of all curves for discrete-valued input signals is&nbsp; $K = \log_2 (M)$.&nbsp; For example,&nbsp; for&nbsp; $\rm SNR  \to ∞$:&nbsp; $C_{\rm BPSK} = 1$&nbsp; $($bit/use$)$,&nbsp; $C_{\rm 4-QAM} = C_{\rm QPSK} = 2$.
-*Therefor the&nbsp; "Shaping Gain"&nbsp; is&nbsp; $10 · \lg (π \cdot {\rm e}/6) = 1.53 \ \rm dB$.&nbsp; This improvement can be achieved by changing the position of the&nbsp; $2^{10} = 32^2$&nbsp; square signal space points to give a Gaussian-like input PDF &nbsp; ⇒  &nbsp;"Signal Shaping".
+#The blue markings show that a&nbsp; $\rm 2^{10}–QAM$&nbsp; with &nbsp;$10 · \lg \rm (SNR) ≈ 27 \ \rm dB$&nbsp; allows a code rate of &nbsp;$R ≈ 8.2$.&nbsp; Distance to the Shannon curve:&nbsp; $1.53\ \rm dB$.
@@ Line 396: / Line 408: @@
 $\text{Conclusion:}$&nbsp;
 &nbsp;  [[Aufgaben:Exercise_4.Ten:_QPSK_Channel_Capacity|"Exercise 4.10"]]&nbsp; discusses the AWGN capacity curves of BPSK and QPSK:
-*Starting from the abscissa&nbsp;  $10 · \lg (E_{\rm B}/N_0)$&nbsp; with&nbsp; $E_{\rm B}$&nbsp; (energy per information <u>bit</u>)&nbsp; one arrives at the QPSK curve by doubling the BPSK curve:
+*Starting from the abscissa &nbsp;  $10 · \lg (E_{\rm B}/N_0)$ &nbsp; with &nbsp; $E_{\rm B}$&nbsp; $($energy per information <u>bit</u>$)$&nbsp; one arrives at the QPSK curve by doubling the BPSK curve:
 :$$C_{\rm QPSK}\big [10 \cdot {\rm lg} \hspace{0.1cm}(E_{\rm B}/{N_0})\big ]
+=
 \cdot C_{\rm BPSK}\big [10 \cdot {\rm lg} \hspace{0.1cm}(E_{\rm B}/{N_0}) \big ] .$$
-*However, if we compare BPSK and QPSK at the same energy&nbsp; $E_{\rm S}$&nbsp; per information <u>symbol</u>,&nbsp; then:
+*However,&nbsp; if we compare BPSK and QPSK at the same energy&nbsp; $E_{\rm S}$&nbsp; per information <u>symbol</u>,&nbsp; then:
 :$$C_{\rm QPSK}[10 \cdot {\rm lg} \hspace{0.1cm}(E_{\rm S}/{N_0})]
+=

← Older revision	Revision as of 15:13, 25 February 2023
(No difference)

@@ Line 208: / Line 208: @@
 ==AWGN channel capacity for binary input signals ==
 <br>
 [[File:EN_Inf_T_4_3_S5a.png|right|frame|Calculation of the AWGN channel capacity for BPSK '''KORREKTUR: encoder''']]
 Now we consider the binary case and thus only now do justice to the title of the chapter "AWGN channel capacity for discrete  input”.
+The graph shows the underlying block diagram for  &nbsp;[[Digital_Signal_Transmission/Linear_Digital_Modulation_-_Coherent_Demodulation#Common_block_diagram_for_ASK_and_BPSK|"Binary Phase Shift Keying"]]&nbsp; $\rm (BPSK)$nbsp;  with binary input&nbsp; $U$&nbsp; and binary output&nbsp; $V$.&nbsp;
 *The best possible coding is to achieve that the bit error probability &nbsp;$\text{Pr}(V ≠ U)$&nbsp; becomes vanishingly small.
 *The encoder output is characterized by the binary random variable&nbsp; $X \hspace{0.03cm}' = \{0, 1\}$ &nbsp; ⇒ &nbsp; $M_{X'} = 2$,&nbsp; while the output&nbsp; $Y$&nbsp; of the AWGN channel remains continuous: &nbsp; $M_Y → ∞$.

@@ Line 197: / Line 197: @@
 &rArr; &nbsp; The detailed&nbsp;  $C(E_{\rm B}/N_0)$&nbsp; calculation can be found in&nbsp; [[Aufgaben:Exercise_4.8:_Numerical_Analysis_of_the_AWGN_Channel_Capacity|$\text{Exercise 4.8}$]].
 <br clear=all>
-In the following, we interpret the green &nbsp;$C(E_{\rm B}/N_0)$&nbsp; result in comparison to the red &nbsp;$C(E_{\rm S}/N_0)$&nbsp; curve:
+In the following,&nbsp; we interpret the green &nbsp;$C(E_{\rm B}/N_0)$&nbsp; result in comparison to the red &nbsp;$C(E_{\rm S}/N_0)$&nbsp; curve:
-*Because of &nbsp;$E_{\rm S} = R · E_{\rm B}$,&nbsp; the Intersection of both curves is at &nbsp;$C (= R) = 1$&nbsp; bit/symbol.&nbsp; <br>Required are  &nbsp;$10 · \lg (E_{\rm S}/N_0) = 1.76$&nbsp; dB &nbsp;or&nbsp; $10 · \lg (E_{\rm B}/N_0) = 1.76$  &nbsp; dB equally.
+#Because of &nbsp;$E_{\rm S} = R · E_{\rm B}$,&nbsp; the Intersection of both curves is at &nbsp;$C (= R) = 1$&nbsp; bit/symbol.&nbsp; Required are  &nbsp;$10 · \lg (E_{\rm S}/N_0) =10 · \lg (E_{\rm B}/N_0) = 1.76$&nbsp; dB.<br>
-*In the range &nbsp;$C > 1$&nbsp; the green curve always lies above the red curve.&nbsp; For example, for &nbsp;$10 · \lg (E_{\rm B}/N_0) = 20$&nbsp; dB&nbsp; the channel capacity is &nbsp;$C ≈ 5$, for &nbsp;$10 · \lg (E_{\rm S}/N_0) = 20$&nbsp; dB &nbsp;only&nbsp; $C = 3.83$.
+#For &nbsp;$C > 1$&nbsp; the green curve always lies above the red curve,&nbsp; e.g. for &nbsp;$10 · \lg (E_{\rm B}/N_0) = 20$&nbsp; dB &nbsp; &rArr;  &nbsp; $C ≈ 5$;&nbsp; for &nbsp;$10 · \lg (E_{\rm S}/N_0) = 20$&nbsp; dB &nbsp; &rArr; &nbsp; $C = 3.83$.<br>
-*A comparison in horizontal direction shows that the channel capacity &nbsp;$C = 3$&nbsp; bit/symbol is already achievable with &nbsp;$10 · \lg (E_{\rm B}/N_0) \approx 10$&nbsp; dB, but one needs &nbsp;$10 · \lg (E_{\rm S}/N_0) \approx 15$&nbsp; dB.
+#Comparison in horizontal direction: &nbsp;$C = 3$&nbsp; bit/symbol is already achievable with &nbsp;$10 · \lg (E_{\rm B}/N_0) \approx 10$&nbsp; dB,&nbsp;  but one needs &nbsp;$10 · \lg (E_{\rm S}/N_0) \approx 15$&nbsp; dB.
-*In the range &nbsp;$C < 1$&nbsp;, the red curve is always above the green one.&nbsp; For any &nbsp;$E_{\rm S}/N_0 > 0$&nbsp; &rArr; &nbsp;$C > 0$.&nbsp; Thus, for logarithmic abscissa as in the present plot, the red curve extends to&nbsp; "minus infinity".
+#For &nbsp;$C < 1$&nbsp;, the red curve is always above the green one.&nbsp; For any &nbsp;$E_{\rm S}/N_0 > 0$&nbsp; &rArr; &nbsp;$C > 0$.&nbsp;
-*In contrast, the green curve ends at &nbsp;$E_{\rm B}/N_0 = \ln (2) = 0.693$ &nbsp; ⇒ &nbsp; $10 · \lg (E_{\rm B}/N_0)= -1.59$&nbsp; dB  &nbsp; ⇒ &nbsp;   absolute limit for (error-free) transmission over the AWGN channel.}}
+#Thus,&nbsp; for logarithmic abscissa as in the present plot, the red curve extends to&nbsp; "minus infinity".