Aufgaben:Exercise 4.6: AWGN Channel Capacity - Revision history

Maintenance script: Fix interlanguage link: resolve redirect chain

2026-03-16T15:58:13Z

Fix interlanguage link: resolve redirect chain

← Older revision		Revision as of 17:58, 16 March 2026
Line 4:		Line 4:

	[[File:P_ID2899__Inf_A_4_6.png\|right\|frame\|Flowchart of the information]]		[[File:P_ID2899__Inf_A_4_6.png\|right\|frame\|Flowchart of the information]]
	We start from the   [[Information_Theory/~~AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang~~#Channel_capacity_of_the_AWGN_channel\| AWGN channel model]] :		We start from the   [[Information_Theory/AWGN_Channel_Capacity_for_Continuous-Valued_Input#Channel_capacity_of_the_AWGN_channel\| AWGN channel model]] :
	* $X$  denotes the input (transmitter).		* $X$  denotes the input (transmitter).
	* $N$  stands for a Gaussian distributed noise.		* $N$  stands for a Gaussian distributed noise.
Line 145:		Line 145:

	[[Category:Information Theory: Exercises\|^4.2 AWGN and Value-Continuous Input^]]		[[Category:Information Theory: Exercises\|^4.2 AWGN and Value-Continuous Input^]]
	[[de:Aufgaben:4.~~06_AWGN–Kanalkapazität~~]]		[[de:Aufgaben:Aufgabe 4.6: AWGN–Kanalkapazität]]

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:48:41Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:48, 16 March 2026
Line 11:		Line 11:

	For the probability density function  $\rm (PDF)$  of the noise,  let hold:		For the probability density function  $\rm (PDF)$  of the noise,  let hold:
	:$$f_N(n) = \frac{1}{\sqrt{2\pi \hspace{0.03cm}\sigma_{\hspace{-0.05cm}N}^2}} \cdot {\rm e}^{		:$$f_N(n) = \frac{1}{\sqrt{2\pi \hspace{0.03cm}\sigma_{\hspace{-0.05cm}N}^2}} \cdot {\rm e}^{- \hspace{0.05cm}{n^2}\hspace{-0.05cm}/{(2 \hspace{0.03cm} \sigma_{\hspace{-0.05cm}N}^2) }} \hspace{0.05cm}.$$
	- \hspace{0.05cm}{n^2}\hspace{-0.05cm}/{(2 \hspace{0.03cm} \sigma_{\hspace{-0.05cm}N}^2) }} \hspace{0.05cm}.$$
	Since the random variable  $N$  is zero mean   ⇒   $m_{N} = 0$,  we can equate the variance $\sigma_{\hspace{-0.05cm}N}^2$  with the power  $P_N$ .  In this case, the differential entropy of the random variable  $N$  is specifiable  (with the pseudo–unit "bit")  as follows:		Since the random variable  $N$  is zero mean   ⇒   $m_{N} = 0$,  we can equate the variance $\sigma_{\hspace{-0.05cm}N}^2$  with the power  $P_N$ .  In this case, the differential entropy of the random variable  $N$  is specifiable  (with the pseudo–unit "bit")  as follows:
	:$$h(N) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot P_N \right )\hspace{0.05cm}.$$		:$$h(N) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot P_N \right )\hspace{0.05cm}.$$
Line 27:		Line 26:

	* The channel capacity is defined as the maximum mutual information between input  $X$  and output  $Y$  with the best possible input distribution:		* The channel capacity is defined as the maximum mutual information between input  $X$  and output  $Y$  with the best possible input distribution:
	:$$C = \max_{\hspace{-0.15cm}f_X:\hspace{0.05cm} {\rm E}[X^2] \le P_X} \hspace{-0.2cm} I(X;Y)		:$$C = \max_{\hspace{-0.15cm}f_X:\hspace{0.05cm} {\rm E}[X^2] \le P_X} \hspace{-0.2cm} I(X;Y)\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$

	*The channel capacity of the AWGN channel is:		*The channel capacity of the AWGN channel is:
	:$$C_{\rm AWGN} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{P_X}{P_N} \right )		:$$C_{\rm AWGN} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{P_X}{P_N} \right )= {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{P_{\hspace{-0.05cm}X}\hspace{0.01cm}'}{P_{\hspace{-0.05cm}N}\hspace{0.01cm}'} \right )\hspace{0.05cm}.$$
	= {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{P_{\hspace{-0.05cm}X}\hspace{0.01cm}'}{P_{\hspace{-0.05cm}N}\hspace{0.01cm}'} \right )\hspace{0.05cm}.$$
	:It can be seen:  The channel capacity  $C$  and also the mutual information  $I(X; Y)$  are independent of the above normalization, in contrast to the differential entropies.		:It can be seen:  The channel capacity  $C$  and also the mutual information  $I(X; Y)$  are independent of the above normalization, in contrast to the differential entropies.

Line 102:		Line 99:
	:$$C_{\rm bit} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.05cm}.$$		:$$C_{\rm bit} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.05cm}.$$
	:With  $C_{\rm bit} = 2$  this results in:		:With  $C_{\rm bit} = 2$  this results in:
	:$$4 \stackrel{!}{=} {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )		:$$4 \stackrel{!}{=} {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.3cm}\Rightarrow \hspace{0.3cm} 1 + {P_X}/{P_N} \stackrel {!}{=} 2^4 = 16\hspace{0.3cm}\Rightarrow \hspace{0.3cm} P_X = 15 \cdot P_N\hspace{0.15cm}\underline{= 15\,{\rm mW}}\hspace{0.05cm}. $$
	\hspace{0.3cm}\Rightarrow \hspace{0.3cm} 1 + {P_X}/{P_N} \stackrel {!}{=} 2^4 = 16
	\hspace{0.3cm}\Rightarrow \hspace{0.3cm} P_X = 15 \cdot P_N
	\hspace{0.15cm}\underline{= 15\,{\rm mW}}
	\hspace{0.05cm}. $$


Line 119:		Line 112:
	[[File: P_ID2901__Inf_A_4_6c.png \|right\|frame\|Information-theoretical values with the AWGN channel]]		[[File: P_ID2901__Inf_A_4_6c.png \|right\|frame\|Information-theoretical values with the AWGN channel]]
	* For  $P_N = 1 \ \rm mW$   ⇒   $P_N\hspace{0.05cm}' = 1$:		* For  $P_N = 1 \ \rm mW$   ⇒   $P_N\hspace{0.05cm}' = 1$:
	:$$h(N) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 1 \right )		:$$h(N) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 1 \right )= \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 17.08 \right )\hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm},$$
	= \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 17.08 \right )
	\hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm},$$
	* For  $P_X = 15 \ \rm mW$   ⇒   $P_X\hspace{0.01cm}' = 15$:		* For  $P_X = 15 \ \rm mW$   ⇒   $P_X\hspace{0.01cm}' = 15$:
	:$$h(X) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 15 \right ) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \right ) +		:$$h(X) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 15 \right ) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \right ) +{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left (15 \right )\hspace{0.15cm}\underline{= 4.000\,{\rm bit}}\hspace{0.05cm}, $$
	{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left (15 \right )
	\hspace{0.15cm}\underline{= 4.000\,{\rm bit}}\hspace{0.05cm}, $$
	* For  $P_Y = P_X + P_N = 16 \ \rm mW$   ⇒  $P_Y\hspace{0.01cm}' = 16$:		* For  $P_Y = P_X + P_N = 16 \ \rm mW$   ⇒  $P_Y\hspace{0.01cm}' = 16$:
	:$$h(Y) = 2.047\,{\rm bit} + 2.000\,{\rm bit}		:$$h(Y) = 2.047\,{\rm bit} + 2.000\,{\rm bit}\hspace{0.15cm}\underline{= 4.047\,{\rm bit}}\hspace{0.05cm}.$$
	\hspace{0.15cm}\underline{= 4.047\,{\rm bit}}\hspace{0.05cm}.$$


Line 147:		Line 135:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) \ = \ h(N) \hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm},$$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) \ = \ h(N) \hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm},$$
	:$$h(Y) \ = \ h(X) \hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$		:$$h(Y) \ = \ h(X) \hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$
	:$$I(X;Y) \ = \ h(Y) - h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X)\hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$ $$		:$$I(X;Y) \ = \ h(Y) - h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X)\hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$ $$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) \ = \ h(X) - I(X;Y)\hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm}.$$
	h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) \ = \ h(X) - I(X;Y)\hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm}.$$

	*The graph shows these quantities in a flowchart.  The same diagram would result in the discrete value case with  $M = 16$  equally probable symbols   ⇒   $H(X)= 4.000 \,{\rm bit}$.		*The graph shows these quantities in a flowchart.  The same diagram would result in the discrete value case with  $M = 16$  equally probable symbols   ⇒   $H(X)= 4.000 \,{\rm bit}$.

Maintenance script: Add German interlanguage link

2026-03-16T12:32:53Z

Add German interlanguage link

← Older revision		Revision as of 14:32, 16 March 2026
Line 158:		Line 158:

	[[Category:Information Theory: Exercises\|^4.2 AWGN and Value-Continuous Input^]]		[[Category:Information Theory: Exercises\|^4.2 AWGN and Value-Continuous Input^]]
			[[de:Aufgaben:4.06_AWGN–Kanalkapazität]]

Guenter at 12:18, 3 November 2021

2021-11-03T12:18:56Z

← Older revision		Revision as of 14:18, 3 November 2021
Line 136:		Line 136:
	*However,  according to the adjacent graph,  also holds:		*However,  according to the adjacent graph,  also holds:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}. $$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}. $$
	~~[[File: P_ID2900__Inf_A_4_6e.png \|right\|frame\|Information-theoretical values with the ideal channel]]~~
	*From this,  the differential equivocation can be calculated as follows:		*From this,  the differential equivocation can be calculated as follows:
	:$$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) = h(X) - I(X;Y) = 4.000 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.000\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) = h(X) - I(X;Y) = 4.000 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.000\,{\rm bit}}\hspace{0.05cm}.$$
			[[File: P_ID2900__Inf_A_4_6e.png \|right\|frame\|Information-theoretical values with the ideal channel]]
	*Finally,  the differential composite entropy is also given,  which cannot be read directly from the above diagram:		*Finally,  the differential composite entropy is also given,  which cannot be read directly from the above diagram:
	:$$h(XY) = h(X) + h(Y) - I(X;Y) = 4.000 \,{\rm bit} + 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 6.047\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(XY) = h(X) + h(Y) - I(X;Y) = 4.000 \,{\rm bit} + 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 6.047\,{\rm bit}}\hspace{0.05cm}.$$

Guenter at 12:17, 3 November 2021

2021-11-03T12:17:55Z

← Older revision		Revision as of 14:17, 3 November 2021
Line 101:		Line 101:
	'''(1)'''  The equation for the AWGN channel capacity in  "bit"  is:		'''(1)'''  The equation for the AWGN channel capacity in  "bit"  is:
	:$$C_{\rm bit} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.05cm}.$$		:$$C_{\rm bit} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.05cm}.$$
	With  $C_{\rm bit} = 2$  this results in:		:With  $C_{\rm bit} = 2$  this results in:
	:$$4 \stackrel{!}{=} {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )		:$$4 \stackrel{!}{=} {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )
	\hspace{0.3cm}\Rightarrow \hspace{0.3cm} 1 + {P_X}/{P_N} \stackrel {!}{=} 2^4 = 16		\hspace{0.3cm}\Rightarrow \hspace{0.3cm} 1 + {P_X}/{P_N} \stackrel {!}{=} 2^4 = 16
Line 113:		Line 113:
	* With  $P_X = 15 \ \rm mW$  the maximum mutual information  $I(X; Y) = 2$  bit is only achievable if the input quantity  $X$  is Gaussian distributed.  <br>The output quantity  $Y$  is then also Gaussian distributed.		* With  $P_X = 15 \ \rm mW$  the maximum mutual information  $I(X; Y) = 2$  bit is only achievable if the input quantity  $X$  is Gaussian distributed.  <br>The output quantity  $Y$  is then also Gaussian distributed.
	* If the random variable  $X$  has a constant proportion  $m_X$  then the variance  $\sigma_X^2 = P_X - m_X^2 $  for given  $P_X$  is smaller, and it holds   <br>$I(X; Y) = 1/2 · \log_2 \ (1 + \sigma_X^2/P_N) < 2$  bit.		* If the random variable  $X$  has a constant proportion  $m_X$  then the variance  $\sigma_X^2 = P_X - m_X^2 $  for given  $P_X$  is smaller, and it holds   <br>$I(X; Y) = 1/2 · \log_2 \ (1 + \sigma_X^2/P_N) < 2$  bit.
	* The precondition for the given channel capacity equation is that  $X$  and  $N$  are uncorrelated.  On the other hand, if the random variables  $X$  and  $N$  were uncorrelated, then  $I(X; Y) = 0$ would result.		* The precondition for the given channel capacity equation is that  $X$  and  $N$  are uncorrelated.  On the other hand, if the random variables  $X$  and  $N$  were uncorrelated, then  $I(X; Y) = 0$  would result.


	~~[[File: P_ID2901__Inf_A_4_6c.png \|right\|frame\|Information-theoretical quantities for the AWGN channel]]~~
	'''(3)'''  The given equation for differential entropy makes sense only for dimensionless power.  With the proposed normalization, one obtains:		'''(3)'''  The given equation for differential entropy makes sense only for dimensionless power.  With the proposed normalization, one obtains:
			[[File: P_ID2901__Inf_A_4_6c.png \|right\|frame\|Information-theoretical values with the AWGN channel]]
	* For  $P_N = 1 \ \rm mW$   ⇒   $P_N\hspace{0.05cm}' = 1$:		* For  $P_N = 1 \ \rm mW$   ⇒   $P_N\hspace{0.05cm}' = 1$:
	:$$h(N) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 1 \right )		:$$h(N) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 1 \right )
Line 126:		Line 126:
	{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left (15 \right )		{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left (15 \right )
	\hspace{0.15cm}\underline{= 4.000\,{\rm bit}}\hspace{0.05cm}, $$		\hspace{0.15cm}\underline{= 4.000\,{\rm bit}}\hspace{0.05cm}, $$
	* ~~Für~~  $P_Y = P_X + P_N = 16 \ \rm mW$   ⇒  $P_Y\hspace{0.01cm}' = 16$:		* For  $P_Y = P_X + P_N = 16 \ \rm mW$   ⇒  $P_Y\hspace{0.01cm}' = 16$:
	:$$h(Y) = 2.047\,{\rm bit} + 2.000\,{\rm bit}		:$$h(Y) = 2.047\,{\rm bit} + 2.000\,{\rm bit}
	\hspace{0.15cm}\underline{= 4.047\,{\rm bit}}\hspace{0.05cm}.$$		\hspace{0.15cm}\underline{= 4.047\,{\rm bit}}\hspace{0.05cm}.$$
Line 132:		Line 132:


	'''(4)'''  ~~For~~ differential irrelevance, for the AWGN channel:		'''(4)'''  The differential irrelevance for the AWGN channel:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(N) \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(N) \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}.$$
	*However, according to the adjacent graph, also holds:		*However,  according to the adjacent graph,  also holds:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}. $$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}. $$
	*From this, the differential equivocation can be calculated as follows:		[[File: P_ID2900__Inf_A_4_6e.png \|right\|frame\|Information-theoretical values with the ideal channel]]
			*From this,  the differential equivocation can be calculated as follows:
	:$$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) = h(X) - I(X;Y) = 4.000 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.000\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) = h(X) - I(X;Y) = 4.000 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.000\,{\rm bit}}\hspace{0.05cm}.$$
			*Finally,  the differential composite entropy is also given,  which cannot be read directly from the above diagram:
	*Finally, the differential composite entropy is also given, which cannot be read directly from the above diagram:
	:$$h(XY) = h(X) + h(Y) - I(X;Y) = 4.000 \,{\rm bit} + 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 6.047\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(XY) = h(X) + h(Y) - I(X;Y) = 4.000 \,{\rm bit} + 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 6.047\,{\rm bit}}\hspace{0.05cm}.$$



	~~[[File: P_ID2900__Inf_A_4_6e.png \|right\|frame\|Information-theoretical quantities with the ideal channel]]~~		'''(5)'''  For the ideal channel with  $h(X)\hspace{0.15cm}\underline{= 4.000 \,{\rm bit}}$:
	'''(5)'''  In the ideal channel, $h(X)\hspace{0.15cm}\underline{= 4.000 \,{\rm bit}}$ ~~is obtained:~~:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) \ = \ h(N) \hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm},$$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) \ = \ h(N) \hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm},$$
	:$$h(Y) \ = \ h(X) \hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$		:$$h(Y) \ = \ h(X) \hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$
Line 151:		Line 150:
	h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) \ = \ h(X) - I(X;Y)\hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm}.$$		h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) \ = \ h(X) - I(X;Y)\hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm}.$$

	*The graph shows these quantities in a flowchart.		*The graph shows these quantities in a flowchart.  The same diagram would result in the discrete value case with  $M = 16$  equally probable symbols   ⇒   $H(X)= 4.000 \,{\rm bit}$.
			*One only would have to replace each  $h$  by an  $H$.
	*The same diagram would result in the discrete value case with  $M = 16$  equally probable symbols   ⇒   $H(X)= 4.000 \,{\rm bit}$.
	*One would ~~only~~ have to replace each  $h$  by an  $H$~~ ~~.

	{{ML-Fuß}}		{{ML-Fuß}}

Noah at 11:49, 1 November 2021

2021-11-01T11:49:36Z

← Older revision		Revision as of 13:49, 1 November 2021
Line 116:		Line 116:


	[[File: P_ID2901__Inf_A_4_6c.png \|right\|frame\|~~Informationstheoretische Größen für den~~ AWGN~~-Kanal~~]]		[[File: P_ID2901__Inf_A_4_6c.png \|right\|frame\|Information-theoretical quantities for the AWGN channel]]
	'''(3)'''  ~~Die angegebene Gleichung für die differentielle Entropie macht nur bei dimensionsloser Leistung Sinn~~.  ~~Mit der vorgeschlagenen Normierung erhält man~~:		'''(3)'''  The given equation for differential entropy makes sense only for dimensionless power.  With the proposed normalization, one obtains:
	* ~~Für~~  $P_N = 1 \ \rm mW$   ⇒   $P_N\hspace{0.05cm}' = 1$:		* For  $P_N = 1 \ \rm mW$   ⇒   $P_N\hspace{0.05cm}' = 1$:
	:$$h(N) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 1 \right )		:$$h(N) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 1 \right )
	= \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 17.08 \right )		= \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 17.08 \right )
	\hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm},$$		\hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm},$$
	* ~~Für~~  $P_X = 15 \ \rm mW$   ⇒   $P_X\hspace{0.01cm}' = 15$:		* For  $P_X = 15 \ \rm mW$   ⇒   $P_X\hspace{0.01cm}' = 15$:
	:$$h(X) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 15 \right ) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \right ) +		:$$h(X) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \cdot 15 \right ) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 2\pi {\rm e} \right ) +
	{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left (15 \right )		{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left (15 \right )
Line 132:		Line 132:


	'''(4)'''  ~~Für die differentielle Irrelevanz gilt beim~~ AWGN~~–Kanal~~:		'''(4)'''  For differential irrelevance, for the AWGN channel:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(N) \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(N) \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}.$$
	*~~Entsprechend nebenstehender Grafik gilt aber auch~~:		*However, according to the adjacent graph, also holds:
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}. $$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) = h(Y) - I(X;Y) = 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.047\,{\rm bit}}\hspace{0.05cm}. $$
	*~~Daraus kann die differentielle Äquivokation wie folgt berechnet werden~~:		*From this, the differential equivocation can be calculated as follows:
	:$$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) = h(X) - I(X;Y) = 4.000 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.000\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) = h(X) - I(X;Y) = 4.000 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 2.000\,{\rm bit}}\hspace{0.05cm}.$$

	*~~Abschließend wird auch noch die differentielle Verbundentropie angegeben~~, ~~die aus obigem Schaubild nicht direkt ablesbar ist~~:		*Finally, the differential composite entropy is also given, which cannot be read directly from the above diagram:
	:$$h(XY) = h(X) + h(Y) - I(X;Y) = 4.000 \,{\rm bit} + 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 6.047\,{\rm bit}}\hspace{0.05cm}.$$		:$$h(XY) = h(X) + h(Y) - I(X;Y) = 4.000 \,{\rm bit} + 4.047 \,{\rm bit} - 2 \,{\rm bit} \hspace{0.15cm}\underline{= 6.047\,{\rm bit}}\hspace{0.05cm}.$$



	[[File: P_ID2900__Inf_A_4_6e.png \|right\|frame\|~~Informationstheoretische Größen beim idealen Kanal~~]]		[[File: P_ID2900__Inf_A_4_6e.png \|right\|frame\|Information-theoretical quantities with the ideal channel]]
	'''(5)'''  ~~Beim idealen Kanal erhält man mit~~ $h(X)\hspace{0.15cm}\underline{= 4.000 \,{\rm bit}}$:		'''(5)'''  In the ideal channel, $h(X)\hspace{0.15cm}\underline{= 4.000 \,{\rm bit}}$ is obtained::
	:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) \ = \ h(N) \hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm},$$		:$$h(Y \hspace{-0.05cm}\mid \hspace{-0.05cm} X) \ = \ h(N) \hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm},$$
	:$$h(Y) \ = \ h(X) \hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$		:$$h(Y) \ = \ h(X) \hspace{0.15cm}\underline{= 4\,{\rm bit}}\hspace{0.05cm},$$
Line 151:		Line 151:
	h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) \ = \ h(X) - I(X;Y)\hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm}.$$		h(X \hspace{-0.05cm}\mid \hspace{-0.05cm} Y) \ = \ h(X) - I(X;Y)\hspace{0.15cm}\underline{= 0\,{\rm (bit)}}\hspace{0.05cm}.$$

	*~~Die Grafik zeigt diese Größen~~ in ~~einem Flussdiagramm~~.		*The graph shows these quantities in a flowchart.

	*~~Das gleiche Diagramm ergäbe sich auch im wertdiskreten Fall mit~~  $M = 16$  ~~gleichwahrscheinlichen Symbolen~~   ⇒   $H(X)= 4.000 \,{\rm bit}$.		*The same diagram would result in the discrete value case with  $M = 16$  equally probable symbols   ⇒   $H(X)= 4.000 \,{\rm bit}$.
	*~~Man müsste nur jedes~~  $h$  ~~durch ein~~  $H$  ~~ersetzen~~.		*One would only have to replace each  $h$  by an  $H$ .

	{{ML-Fuß}}		{{ML-Fuß}}

Javier: Text replacement - "continuous-value" to "continuous"

2021-10-11T08:28:22Z

Text replacement - "continuous-value" to "continuous"

← Older revision		Revision as of 10:28, 11 October 2021
Line 47:		Line 47:

	Hints:		Hints:
	*The exercise belongs to the chapter  [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang\|AWGN channel capacity with continuous~~-value~~ input]].		*The exercise belongs to the chapter  [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang\|AWGN channel capacity with continuous input]].
	*Since the results are to be given in  "bit",  "log"  ⇒  "log<sub>2</sub>"  is used in the equations.		*Since the results are to be given in  "bit",  "log"  ⇒  "log<sub>2</sub>"  is used in the equations.

Guenter at 15:57, 9 October 2021

2021-10-09T15:57:00Z

← Older revision		Revision as of 17:57, 9 October 2021
Line 109:		Line 109:


			'''(2)'''  Correct are  <u>proposed solutions 1 through 4</u>.  Justification:
	'''(2)'''  ~~Richtig sind die~~ <u>~~Lösungsvorschläge~~ 1 ~~bis~~ 4</u>. ~~Begründung~~:		* For  $P_X < 15 \ \rm mW$  the mutual information  $I(X; Y)$  will always be less than  $2$  bit,  regardless of all other conditions.
	* ~~Für~~  $P_X < 15 \ \rm mW$  ~~wird die Transinformation~~  $I(X; Y)$  ~~stets kleiner als~~  $2$  bit ~~sein~~, ~~unabhängig von allen anderen Gegebenheiten~~.		* With  $P_X = 15 \ \rm mW$  the maximum mutual information  $I(X; Y) = 2$  bit is only achievable if the input quantity  $X$  is Gaussian distributed.  <br>The output quantity  $Y$  is then also Gaussian distributed.
	* ~~Mit~~  $P_X = 15 \ \rm mW$  ~~ist die maximale Transinformation~~  $I(X; Y) = 2$  bit ~~nur erreichbar, wenn die Eingangsgröße~~  $X$  ~~gaußverteilt ist~~.  <br>~~Die Ausgangsgröße~~  $Y$  ~~ist dann ebenfalls gaußverteilt~~.		* If the random variable  $X$  has a constant proportion  $m_X$  then the variance  $\sigma_X^2 = P_X - m_X^2 $  for given  $P_X$  is smaller, and it holds   <br>$I(X; Y) = 1/2 · \log_2 \ (1 + \sigma_X^2/P_N) < 2$  bit.
	* ~~Weist die Zufallsgröße~~  $X$  ~~einen Gleichanteil~~  $m_X$  ~~auf, so ist die Varianz~~  $\sigma_X^2 = P_X - m_X^2 $  ~~bei gegebenem~~  $P_X$  ~~kleiner~~, ~~und es gilt~~  $I(X; Y) = 1/2 · \log_2 \ (1 + \sigma_X^2/P_N) < 2$  bit.		* The precondition for the given channel capacity equation is that  $X$  and  $N$  are uncorrelated.  On the other hand, if the random variables  $X$  and  $N$  were uncorrelated, then  $I(X; Y) = 0$ would result.
	* ~~Voraussetzung für die gegebene Kanalkapazitätsgleichung ist, dass~~  $X$  ~~und~~  $N$  ~~unkorreliert sind~~.  ~~Wären dagegen die Zufallsgrößen~~  $X$  ~~und~~  $N$  ~~unkorreliert~~, ~~so ergäbe sich~~  $I(X; Y) = 0$.

Guenter at 10:08, 4 October 2021

2021-10-04T10:08:57Z

← Older revision		Revision as of 12:08, 4 October 2021
Line 99:		Line 99:
	===Solution===		===Solution===
	{{ML-Kopf}}		{{ML-Kopf}}
	'''(1)'''  The equation for the AWGN channel capacity in "bit" is:		'''(1)'''  The equation for the AWGN channel capacity in  "bit"  is:
	:$$C_{\rm bit} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.05cm}.$$		:$$C_{\rm bit} = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + {P_X}/{P_N} \right )\hspace{0.05cm}.$$
	With  $C_{\rm bit} = 2$  this results in:		With  $C_{\rm bit} = 2$  this results in:

Guenter at 09:13, 4 October 2021

2021-10-04T09:13:39Z

← Older revision		Revision as of 11:13, 4 October 2021
Line 48:		Line 48:
	Hints:		Hints:
	*The exercise belongs to the chapter  [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang\|AWGN channel capacity with continuous-value input]].		*The exercise belongs to the chapter  [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang\|AWGN channel capacity with continuous-value input]].
	*Since the results are to be given in "bit", "log"  ⇒  "log<sub>2</sub>" is used in the equations.		*Since the results are to be given in  "bit",  "log"  ⇒  "log<sub>2</sub>"  is used in the equations.


Line 56:		Line 56:
	<quiz display=simple>		<quiz display=simple>

	{What ~~transmit~~ power is required for  $C = 2 \ \rm bit$?		{What transmission power is required for  $C = 2 \ \rm bit$?
	\|type="{}"}		\|type="{}"}
	$P_X \ = \ $ { 15 3% } $\ \rm mW$		$P_X \ = \ $ { 15 3% } $\ \rm mW$