Information Theory/AWGN Channel Capacity for Discrete-Valued Input - Revision history

Maintenance script: Join 1 split formula blocks

2026-03-16T14:15:28Z

Join 1 split formula blocks

← Older revision		Revision as of 16:15, 16 March 2026
Line 8:		Line 8:
	==AWGN model for discrete-time band-limited signals==		==AWGN model for discrete-time band-limited signals==
	<br>		<br>
	At the end of the  [[Information_Theory/~~AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang~~#Channel_capacity_of_the_AWGN_channel\|"last chapter"]],  the AWGN model was used according to the left graph,  characterized by the two random variables  $X$  and  $Y$  at the input and output and the stochastic noise  $N$  as the result of a mean-free Gaussian random process   ⇒   "white noise" with variance  $σ_N^2$.  The noise power  $P_N$  is also equal to  $σ_N^2$.		At the end of the  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous-Valued_Input#Channel_capacity_of_the_AWGN_channel\|"last chapter"]],  the AWGN model was used according to the left graph,  characterized by the two random variables  $X$  and  $Y$  at the input and output and the stochastic noise  $N$  as the result of a mean-free Gaussian random process   ⇒   "white noise" with variance  $σ_N^2$.  The noise power  $P_N$  is also equal to  $σ_N^2$.

	[[File:EN_Inf_T_4_3_S1a.png\|right\|frame\|Two largely equivalent models for the AWGN channel<br><br><br><br>]]		[[File:EN_Inf_T_4_3_S1a.png\|right\|frame\|Two largely equivalent models for the AWGN channel<br><br><br><br>]]
Line 61:		Line 61:
	It follows,  according to the insights of the book  [[Theory_of_Stochastic_Signals/Stochastic_System_Theory#System_model_and_problem_definition\|"Theory of Stochastic Signals"]]:		It follows,  according to the insights of the book  [[Theory_of_Stochastic_Signals/Stochastic_System_Theory#System_model_and_problem_definition\|"Theory of Stochastic Signals"]]:

	:$$P_N =		:$$P_N =\int_{-\infty}^{+\infty}\hspace{-0.3cm} {\it \Phi}_N (f) \cdot \vert H_{\rm E}(f)\vert^2\hspace{0.15cm}{\rm d}f = \int_{-B}^{+B}\hspace{-0.3cm} {\it \Phi}_N (f)\hspace{0.15cm}{\rm d}f = \frac{N_0}{2} \cdot 2B = N_0 \cdot B\hspace{0.05cm}.$$}}
	\int_{-\infty}^{+\infty}
	\hspace{-0.3cm} {\it \Phi}_N (f) \cdot \vert H_{\rm E}(f)\vert^2
	\hspace{0.15cm}{\rm d}f = \int_{-B}^{+B}
	\hspace{-0.3cm} {\it \Phi}_N (f)
	\hspace{0.15cm}{\rm d}f = \frac{N_0}{2} \cdot 2B = N_0 \cdot B
	\hspace{0.05cm}.$$}}


Line 124:		Line 118:
	==System model for the interpretation of the AWGN channel capacity==		==System model for the interpretation of the AWGN channel capacity==
	<br>		<br>
	In order to discuss the  [[Information_Theory/~~Anwendung_auf_die_Digitalsignalübertragung~~#Definition_and_meaning_of_channel_capacity\|"Channel Coding Theorem"]]  in the context of the AWGN channel,		In order to discuss the  [[Information_Theory/Application_to_Digital_Signal_Transmission#Definition_and_meaning_of_channel_capacity\|"Channel Coding Theorem"]]  in the context of the AWGN channel,
	[[File:EN_Inf_T_4_3_S3.png\|right\|frame\|Model for the interpretation of the AWGN channel capacity]]		[[File:EN_Inf_T_4_3_S3.png\|right\|frame\|Model for the interpretation of the AWGN channel capacity]]

Line 211:		Line 205:
	:$${\rm Pr}(X =+1) = {\rm Pr}(X =-1) = 0.5 $$		:$${\rm Pr}(X =+1) = {\rm Pr}(X =-1) = 0.5 $$

	will lead to the optimum.  According to the section  [[Information_Theory/~~AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang~~#Calculation_of_mutual_information_with_additive_noise\|"Calculation of mutual information with additive noise"]],  there are several calculation possibilities:		will lead to the optimum.  According to the section  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous-Valued_Input#Calculation_of_mutual_information_with_additive_noise\|"Calculation of mutual information with additive noise"]],  there are several calculation possibilities:

	*$ C_{\rm BPSK} = h(X) + h(Y) - h(XY)\hspace{0.05cm},$		*$ C_{\rm BPSK} = h(X) + h(Y) - h(XY)\hspace{0.05cm},$
Line 319:		Line 313:
	*The  $($complex$)$  AWGN channel thus provides  $K = 2$  independent Gaussian channels.		*The  $($complex$)$  AWGN channel thus provides  $K = 2$  independent Gaussian channels.

	*According to the section  [[Information_Theory/~~AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang~~#Parallel_Gaussian_channels\|"Parallel Gaussian channels"]],  the capacity of such a channel is therefore given by:		*According to the section  [[Information_Theory/AWGN_Channel_Capacity_for_Continuous-Valued_Input#Parallel_Gaussian_channels\|"Parallel Gaussian channels"]],  the capacity of such a channel is therefore given by:
	[[File:P_ID2955__Inf_T_4_3_S7.png\|right\|frame\|2D–PDF of complex Gaussian noise]]		[[File:P_ID2955__Inf_T_4_3_S7.png\|right\|frame\|2D–PDF of complex Gaussian noise]]
	:$$C_{\text{ Gaussian, complex} }= C_{\rm total} ( K=2)= {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_X/2}{\sigma^2})\hspace{0.05cm}.$$		:$$C_{\text{ Gaussian, complex} }= C_{\rm total} ( K=2)= {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_X/2}{\sigma^2})\hspace{0.05cm}.$$
Line 352:		Line 346:
	*The abscissa denotes the signal-to-interference power ratio  $10 · \lg \rm (SNR)$  with  $\rm {SNR} = P_X/σ_N^2$.		*The abscissa denotes the signal-to-interference power ratio  $10 · \lg \rm (SNR)$  with  $\rm {SNR} = P_X/σ_N^2$.

	*The graph was taken from  [Göb10]<ref name='Göb10'>Göbel, B.:  Information–Theoretic Aspects of Fiber–Optic Communication Channels. Dissertation. TU München. <br>Verlag Dr. Hut, Reihe Informationstechnik, ISBN 978-3-86853-713-0, 2010.</ref>.    We would like to thank our former colleague  [[Biographies_and_Bibliographies/~~Beteiligte_der_Professur_Leitungsgebundene_Übertragungstechnik~~#Dr.-Ing._Bernhard_Göbel_(at_LÜT_from_2004-2010)\|$\text{Bernhard Göbel}$]]  for his permission to use this figure and for his great support of our learning tutorial during his entire active time.		*The graph was taken from  [Göb10]<ref name='Göb10'>Göbel, B.:  Information–Theoretic Aspects of Fiber–Optic Communication Channels. Dissertation. TU München. <br>Verlag Dr. Hut, Reihe Informationstechnik, ISBN 978-3-86853-713-0, 2010.</ref>.    We would like to thank our former colleague  [[Biographies_and_Bibliographies/LNTwww_members_from_LÜT#Dr.-Ing._Bernhard_Göbel_(at_LÜT_from_2004-2010)\|$\text{Bernhard Göbel}$]]  for his permission to use this figure and for his great support of our learning tutorial during his entire active time.

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:46:56Z

Join split $$ formula lines for correct rendering

Show changes

Maintenance script: Add German interlanguage link

2026-03-16T12:27:22Z

Add German interlanguage link

← Older revision		Revision as of 14:27, 16 March 2026
Line 436:		Line 436:

	{{Display}}		{{Display}}
			[[de:Informationstheorie/AWGN–Kanalkapazität_bei_wertdiskretem_Eingang]]

Guenter at 14:28, 28 February 2023

2023-02-28T14:28:56Z

← Older revision		Revision as of 16:28, 28 February 2023
Line 377:		Line 377:
	==Maximum code rate for QAM structures==		==Maximum code rate for QAM structures==
	<br>		<br>
	[[File:~~EN_Inf_T_4_3_S8b~~.png\|right\|frame\|Channel capacity of  $\rm BPSK$  and  $M\text{–QAM}$ ~~'''KORREKTUR: capacity'''~~]]		[[File:EN_Inf_T_4_3_S8a_neu.png\|right\|frame\|Channel capacity of  $\rm BPSK$  and  $M\text{–QAM}$]]
	The graph shows the capacity of the complex AWGN channel as a red curve:		The graph shows the capacity of the complex AWGN channel as a red curve:
	:$$C_{\text{ Gaussian, complex} }= {\rm log}_2 \hspace{0.1cm} ( 1 + \rm SNR)		:$$C_{\text{ Gaussian, complex} }= {\rm log}_2 \hspace{0.1cm} ( 1 + \rm SNR)

Guenter at 14:27, 28 February 2023

2023-02-28T14:27:32Z

← Older revision		Revision as of 16:27, 28 February 2023
Line 312:		Line 312:
	The early channel codes mentioned in  $\text{Example 4}$  are still relatively far away from the channel capacity curve.   This was probably also a reason why the author of this learning tutorial was unaware of the also great practical significance of information theory when he became acquainted with it in his studies in the early 1970s.  The view changed significantly when very long channel codes together with iterative decoding appeared in the 1990s.  The new marker points are much closer to the capacity limit curve.		The early channel codes mentioned in  $\text{Example 4}$  are still relatively far away from the channel capacity curve.   This was probably also a reason why the author of this learning tutorial was unaware of the also great practical significance of information theory when he became acquainted with it in his studies in the early 1970s.  The view changed significantly when very long channel codes together with iterative decoding appeared in the 1990s.  The new marker points are much closer to the capacity limit curve.
	<br clear=all>		<br clear=all>
	[[File:~~EN_Inf_T_4_3_S6b~~.png\|right\|frame\|Rates and required  $E_{\rm B}/{N_0}$  of iterative coding methods ~~'''KORREKTUR: Shannon boundary/limit?'''~~]]		[[File:EN_Inf_T_4_3_S6b_v2.png\|right\|frame\|Rates and required  $E_{\rm B}/{N_0}$  of iterative coding methods]]
	Here are a few more explanations of this graph:		Here are a few more explanations of this graph:
	#Red crosses mark the so-called  [https://en.wikipedia.org/wiki/Turbo_code "turbo codes"]  according  $\rm CCSDS$  $($"Consultative Committee for Space Data Systems"$)$  with  $k = 6920$  information bits each and different code lengths  $n = k/R$.		#Red crosses mark the so-called  [https://en.wikipedia.org/wiki/Turbo_code "turbo codes"]  according  $\rm CCSDS$  $($"Consultative Committee for Space Data Systems"$)$  with  $k = 6920$  information bits each and different code lengths  $n = k/R$.

Guenter at 14:25, 28 February 2023

2023-02-28T14:25:58Z

← Older revision		Revision as of 16:25, 28 February 2023
Line 209:		Line 209:
	<br>		<br>
	In the previous sections of this chapter we always assumed a Gaussian distributed,nbsp; i.e. a continuous-valued AWGN input  $X$  according to Shannon theory.		In the previous sections of this chapter we always assumed a Gaussian distributed,nbsp; i.e. a continuous-valued AWGN input  $X$  according to Shannon theory.
	[[File:~~EN_Inf_T_4_3_S5a~~.png\|right\|frame\|Calculation of the AWGN channel capacity for BPSK ~~'''KORREKTUR: encoder'''~~]]		[[File:EN_Inf_T_4_3_S5a_v2.png\|right\|frame\|Calculation of the AWGN channel capacity for BPSK]]
	[[File:EN_Inf_T_4_3_S5b.png\|right\|frame\|Conditional PDF for  $X=-1$  $($red$)$  and  $X=+1$  $($blue$)$ ]]		[[File:EN_Inf_T_4_3_S5b.png\|right\|frame\|Conditional PDF for  $X=-1$  $($red$)$  and  $X=+1$  $($blue$)$ ]]

Guenter at 14:24, 28 February 2023

2023-02-28T14:24:25Z

← Older revision		Revision as of 16:24, 28 February 2023
Line 39:		Line 39:
	\delta(t- \nu \cdot T_{\rm A}		\delta(t- \nu \cdot T_{\rm A}
	)\hspace{0.05cm}.$$		)\hspace{0.05cm}.$$
	[[File:~~EN_Inf_T_4_3_S1b~~.png\|right\|frame\| AWGN model considering time discretization and band-limitation ~~'''KORREKTUR: Abtastung'''~~]]		[[File:EN_Inf_T_4_3_S1b_v2.png\|right\|frame\| AWGN model considering time discretization and band-limitation]]

	The spacing of all  $($weighted$)$  "Dirac delta functions"  is uniform  $T_{\rm A}$.  Through the interpolation filter with the impulse response  $h(t)$  as well as the frequency response  $H(f)$, where		The spacing of all  $($weighted$)$  "Dirac delta functions"  is uniform  $T_{\rm A}$.  Through the interpolation filter with the impulse response  $h(t)$  as well as the frequency response  $H(f)$, where

Guenter at 14:03, 26 February 2023

2023-02-26T14:03:29Z

Show changes

Guenter at 16:58, 25 February 2023

2023-02-25T16:58:47Z

Show changes

Guenter at 16:12, 25 February 2023

2023-02-25T16:12:35Z

Show changes