Difference between revisions of "Aufgaben:Exercise 4.7Z: About the Water Filling Algorithm"

From LNTwww
 
(23 intermediate revisions by 5 users not shown)
Line 1: Line 1:
  
{{quiz-Header|Buchseite=Informationstheorie/AWGN–Kanalkapazität bei wertkontinuierlichem Eingang
+
{{quiz-Header|Buchseite=Information_Theory/AWGN_Channel_Capacity_for_Continuous_Input
 
}}
 
}}
  
[[File:P_ID2903__Inf_T_4_2_S4d.png|right|]]
+
[[File:EN_Inf_Z_4_7.png|right|frame|Water-filling principle  $(K = 4)$]]
Wir betrachten <i>K</i> parallele Gaußsche Kanäle (AWGN) mit unterschiedlichen Störleistungen <i>&sigma;<sub>k</sub></i><sup>2</sup> (1 &#8804; <i>k</i> &#8804; <i>K</i>), wie in der nebenstehenden Grafik am Beispiel <i>K</i> = 4 verdeutlicht ist. Die Sendeleistung in den einzelnen Kanälen wird mit <i>P<sub>k</sub></i> bezeichnet, deren Summe den vorgegebenen Wert <i>P<sub>X</sub></i> nicht überschreiten darf:
+
We consider&nbsp; $K$&nbsp; parallel Gaussian channels&nbsp; $\rm (AWGN)$&nbsp; with different interference powers&nbsp; $\sigma_k^2$,&nbsp; where&nbsp;  $1 \le k \le K$&nbsp;.&nbsp; The graph illustrates this constellation using&nbsp; $K = 4$&nbsp; as an example.  
$$P_1 + ... \hspace{0.05cm}+ P_K = \hspace{0.1cm} \sum_{k= 1}^K  
+
 
 +
The transmission power in the individual channels is denoted by &nbsp;$P_k$,&nbsp; the sum of which must not exceed the specified value&nbsp;$P_X$&nbsp;:
 +
:$$P_1 +\text{...}\hspace{0.05cm}+ P_K = \hspace{0.1cm} \sum_{k= 1}^K  
 
  \hspace{0.1cm}{\rm E} \left [ X_k^2\right ] \le P_{X} \hspace{0.05cm}.$$
 
  \hspace{0.1cm}{\rm E} \left [ X_k^2\right ] \le P_{X} \hspace{0.05cm}.$$
Sind die Zufallsgrößen <i>X</i><sub>1</sub>, ..., <i>X<sub>k</sub></i> gaußisch, so kann für die (gesamte) Transinformation zwischen dem Eingang <i>X</i> und dem Ausgang <i>Y</i> geschrieben werden:
+
If the random variables &nbsp;$X_1$, ... , $X_K$&nbsp; are Gaussian, then for the (total) mutual information between the input &nbsp;$X$&nbsp; and the output &nbsp;$Y$&nbsp; can be written:
$$I(X_1, ... \hspace{0.05cm}, X_K\hspace{0.05cm};\hspace{0.05cm}Y_1, ... \hspace{0.05cm}, Y_K)  
+
:$$I(X_1,\text{...} \hspace{0.05cm}, X_K\hspace{0.05cm};\hspace{0.05cm}Y_1, \text{...}\hspace{0.05cm}, Y_K)  
 
=  1/2 \cdot \sum_{k= 1}^K  \hspace{0.1cm} {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_k}{\sigma_k^2})\hspace{0.05cm},\hspace{0.5cm}
 
=  1/2 \cdot \sum_{k= 1}^K  \hspace{0.1cm} {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_k}{\sigma_k^2})\hspace{0.05cm},\hspace{0.5cm}
{\rm Ergebnis\hspace{0.15cm} in \hspace{0.15cm} bit}
+
{\rm Result\hspace{0.15cm} in \hspace{0.15cm} \text{"bit"} }
 
\hspace{0.05cm}.$$
 
\hspace{0.05cm}.$$
Das Maximum hierfür ist die Kanalkapazität des Gesamtsystems, wobei sich die Maximierung auf die Aufteilung der Gesamtleistung <i>P<sub>X</sub></i> auf die einzelnen Kanäle bezieht.
 
$$C_K(P_X) = \max_{P_k\hspace{0.05cm},\hspace{0.15cm}{\rm mit} \hspace{0.15cm}P_1 + ... \hspace{0.05cm}+ P_K = P_X} \hspace{-0.5cm} I(X_1, ... \hspace{0.05cm}, X_K\hspace{0.05cm};\hspace{0.05cm}Y_1, ... \hspace{0.05cm}, Y_K) \hspace{0.05cm}.$$
 
Diese Maximierung kann mit dem Water&ndash;Filling&ndash;Algorithmus geschehen, der in obiger Grafik für <i>K</i> = 4 dargestellt ist. Eine genaue Beschreibung finden Sie im [http://en.lntwww.de/Informationstheorie/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang#Parallele_Gau.C3.9Fsche_Kan.C3.A4le '''Theorieteil''']
 
In der vorliegenden Aufgabe soll dieser Algorithmus angewendet werden, wobei von folgenden Voraussetzungen auszugehen ist:
 
:* Zwei parallele Gaußkanäle &nbsp;&#8658;&nbsp; <i>K</i> = 2,
 
:* Normierte Störleistungen <i>&sigma;</i><sub>1</sub><sup>2</sup> = 1 und <i>&sigma;</i><sub>2</sub><sup>2</sup> = 4, 
 
:*Normierte Sendeleistungen <i>P<sub>X</sub></i> = 10 bzw. <i>P<sub>X</sub></i> = 3.
 
  
<b>Hinweis:</b> Die Aufgabe bezieht sich auf das Themengebiet von [http://en.lntwww.de/Informationstheorie/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang '''Kapitel 4.2.''']
+
The maximum for this is the&nbsp; '''channel capacity of the total system''',&nbsp; where the maximization refers to the division of the total power &nbsp;$P_X$&nbsp; among the individual channels:
 +
:$$C_K(P_X) = \max_{P_k\hspace{0.05cm},\hspace{0.15cm}{\rm with} \hspace{0.15cm}P_1 + ... \hspace{0.05cm}+ P_K = P_X} \hspace{-0.5cm} I(X_1, \text{...} \hspace{0.05cm}, X_K\hspace{0.05cm};\hspace{0.05cm}Y_1, \text{...}\hspace{0.05cm}, Y_K) \hspace{0.05cm}.$$
 +
This maximization can be done with the&nbsp; "water&ndash;filling algorithm"&nbsp; shown in the above graph for&nbsp; $K = 4$.
  
===Fragebogen===
+
In the present exercise, this algorithm is to be applied, assuming the following:
 +
* Two parallel Gaussian channels &nbsp; &#8658; &nbsp; $K = 2$,
 +
* normalized noise powers &nbsp; $\sigma_1^2 = 1$ &nbsp; and &nbsp; $\sigma_2^2 = 4$, 
 +
* normalized transmission powers &nbsp; $P_X = 10$ &nbsp; and &nbsp; $P_X = 3$ respectively.
 +
 
 +
 
 +
 
 +
 
 +
 
 +
Hints:
 +
*The exercise belongs to the chapter&nbsp; [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang|AWGN channel capacity with continuous value input]].
 +
*Reference is made in particular to the page&nbsp; [[Information_Theory/AWGN–Kanalkapazität_bei_wertkontinuierlichem_Eingang#Parallel_Gaussian_channels|Parallel Gaussian Channels]].
 +
*Since the results are to be given in&nbsp; "bit",&nbsp; the logarithm to base&nbsp; $2$&nbsp; is used in the equations: &nbsp; $\log_2$.
 +
 +
 
 +
 
 +
===Questions===
  
 
<quiz display=simple>
 
<quiz display=simple>
{Welche Strategien der Leistungszuteilung sind sinnvoll?
+
{Which power allocation strategies are useful?
 
|type="[]"}
 
|type="[]"}
- Einem stark gestörten Kanal <i>k</i> (mit großer Störleistung <i>&sigma;<sub>k</sub></i><sup>2</sup>) sollte eine große Nutzleistung <i>P<sub>k</sub></i> zugewiesen werden.
+
- A very noisy channel&nbsp; $k$&nbsp; $($with large noise power&nbsp; $\sigma_k^2)$&nbsp; should be allocated a large effective power&nbsp; $P_k$.
+ Einem stark gestörten Kanal <i>k</i> (mit großer Störleistung <i>&sigma;<sub>k</sub></i><sup>2</sup>) sollte nur eine kleine Nutzleistung <i>P<sub>k</sub></i> zugewiesen werden.
+
+ A very noisy channel&nbsp; $k$&nbsp; $($with large noise power&nbsp; $\sigma_k^2)$&nbsp;  should be assigned only a small useful power&nbsp; $P_k$.
+ Bei <i>K</i> gleich guten Kanälen &nbsp;&#8658;&nbsp; <i>&sigma;</i><sub>1</sub><sup>2</sup> = ... = <i>&sigma;<sub>K</sub></i><sup>2</sup> =  <i>&sigma;<sub>N</sub></i><sup>2</sup> sollte die Leistung gleichmäßig verteilt werden.    
+
+ For&nbsp; $K$&nbsp; equally good channels &nbsp; &#8658; &nbsp; $\sigma_1^2 = \text{...} = \sigma_K^2 = \sigma_N^2$&nbsp; the power&nbsp; $P_k$&nbsp; should be evenly distributed.
  
  
{Welche Transinformation <i>I</i> <i>I</i>(<i>X</i><sub>1</sub>, <i>X</i><sub>2</sub></i>; <i>Y</i><sub>1</sub>, <i>Y</i><sub>2</sub>) ergibt sich, wenn man die Sendeleistung <i>P<sub>X</sub></i> = 10 gleichmäßig auf beide Kanäle verteilt?
+
{What is the mutual information obtained by distributing the transmission power &nbsp;$P_X = 10$&nbsp; equally to both channels &nbsp; $(P_1= P_2 = 5)$?
 
|type="{}"}
 
|type="{}"}
$P1 = P2 = 5:  I$ = { 1.877 3% }
+
$I(X_1, X_2; Y_1, Y_2) \ = \ $ { 1.877 3% } $\ \rm bit$
  
  
{Es gelte weiter <i>P<sub>X</sub></i> = 10. Welche optimale Leistungsaufteilung ergibt sich nach dem Water&ndash;Filling&ndash;Algorithmus?
+
{Let&nbsp; $P_X = P_1 + P_2 = 10$&nbsp; be further valid.&nbsp; Which optimal power distribution results according to the&nbsp; "water&ndash;filling algorithm"?
 
|type="{}"}
 
|type="{}"}
$PX = 10:  P1$ = { 6.5 3% }
+
$P_1  \ = \ $ { 6.5 3% }
$P1 = P2 = 5:  I$ = { 3.5 3% }
+
$P_2  \ = \ $ { 3.5 3% }
  
  
  
{Wie groß ist die Kanalkapazität für <i>K</i> = 2 und <i>P<sub>X</sub></i> = 10?
+
{What is the channel capacity for&nbsp; $\underline{K = 2}$&nbsp; and&nbsp; $\underline{P_X = 10}$?
 
|type="{}"}
 
|type="{}"}
$C2(PX = 10)$ = { 1.907 3% }
+
$C  \ = \ $ { 1.907 3% } $\ \rm bit$
 +
 
  
 +
{What mutual information results if the transmit power&nbsp; $P_X = 3$&nbsp; is distributed equally to both channels &nbsp; $(P_1= P_2 = 1.5)$?
 +
|type="{}"}
 +
$I(X_1, X_2; Y_1, Y_2) \ = \ $ { 0.891 3% }$\ \rm bit$
  
{Welche Ergebnisse erhält man mit <i>K</i> = 2 und <i>P<sub>X</sub></i> = 3?
+
{What is the channel capacity for&nbsp; $\underline{K = 2}$&nbsp; and&nbsp; $\underline{P_X = 3}$?
 
|type="{}"}
 
|type="{}"}
$P1 = P2 = 1.5:  I$ = { 0.891 3% }
+
$C  \ = \ $ { 1 3% } $\ \rm bit$
$C2(PX = 3)$ = { 1 3% }
 
  
  
 
</quiz>
 
</quiz>
  
===Musterlösung===
+
===Solution===
 
{{ML-Kopf}}
 
{{ML-Kopf}}
'''1.'''
+
'''(1)'''&nbsp;  <u>Proposed solutions 2 and 3</u>&nbsp; are correct:
'''2.'''
+
*According to the explanations in the theory section&nbsp; "Water&ndash;Filling" &nbsp; &#8658; &nbsp; <u>Proposal 2</u>&nbsp; is to be applied when unequal conditions exist.
'''3.'''
+
* However,&nbsp; <u>solution proposal 3</u>&nbsp; is also correct: &nbsp; If the channels are equally good, there is nothing to prevent all&nbsp; $K$&nbsp; channels from being filled with the same power &nbsp; &#8658; &nbsp; &nbsp;$P_1 = P_2 =$&nbsp; ...&nbsp; $= P_K = P_X/K$.
'''4.'''
+
 
'''5.'''
+
 
'''6.'''
+
 
'''7.'''
+
'''(2)'''&nbsp;  For the mutual information, with equal power distribution, the following applies:
 +
:$$I(X_1, X_2\hspace{0.05cm};\hspace{0.05cm}Y_1,  Y_2) \ =  \ {1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{5}{1} \right )
 +
+{1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{5}{4} \right )=1.292\,{\rm bit}+ 0.585\,{\rm bit}
 +
\hspace{0.15cm}\underline{= 1.877\,{\rm bit}}
 +
\hspace{0.05cm}.$$
 +
 
 +
 
 +
 
 +
[[File:EN_Inf_Z_4_7b.png|right|frame|Best possible distribution of transmit power&nbsp; $P_X = 10$]]
 +
'''(3)'''&nbsp;  According to the adjacent sketch, the following must apply:
 +
:$$P_2 = P_1 - (\sigma_2^2 - \sigma_1^2) = P_1 -3\hspace{0.3cm}\text{wobei }\hspace{0.3cm}P_1 + P_2 =  P_X = 10$$
 +
:$$\Rightarrow \hspace{0.3cm}
 +
P_1 + (P_1 -3) = 10\hspace{0.3cm}\Rightarrow \hspace{0.3cm}
 +
2 \cdot P_1 = 13 \hspace{0.3cm}
 +
\Rightarrow \hspace{0.3cm}
 +
\underline{P_1 = 6.5}\hspace{0.05cm},
 +
\hspace{0.3cm}\underline{P_2 = 3.5}\hspace{0.05cm}.$$
 +
For the&nbsp; "water level height"&nbsp; here holds:&nbsp;
 +
:$$H= P_1 + \sigma_1^2= P_2 + \sigma_2^2 = 7.5 = 6.5+1 = 3.5+4.$$
 +
 
 +
 
 +
 
 +
'''(4)'''&nbsp;  The channel capacity indicates the maximum mutual information. &nbsp; The maximum is already fixed by the best possible power sharing according to subtask '''(3)'''.&nbsp;For&nbsp; $P_X = 10$&nbsp; holds:
 +
:$$C={1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{6.5}{1} \right )
 +
+{1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{3.5}{4} \right )$$
 +
:$$\Rightarrow\hspace{0.3cm} C=1.453\,{\rm bit}+ 0.453\,{\rm bit}
 +
\hspace{0.15cm}\underline{= 1.906\,{\rm bit}}
 +
\hspace{0.05cm}.$$
 +
 
 +
 
 +
 
 +
'''(5)'''&nbsp;  For&nbsp; $P_X = 3$, &nbsp; with the same power splitting&nbsp; $(P_1 = P_2 =1.5)$,&nbsp; we obtain:
 +
:$$I(X_1, X_2\hspace{0.05cm};\hspace{0.05cm}Y_1,  Y_2) ={1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{1.5}{1} \right )
 +
+{1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{1.5}{4} \right )
 +
\hspace{0.15cm}\underline{= 0.891\,{\rm bit}}
 +
\hspace{0.05cm}.$$
 +
 
 +
 
 +
 
 +
[[File:EN_Inf_Z_4_7e.png|right|frame|Best possible distribution of the transmit power&nbsp; $P_X = 3$]]
 +
'''(6)'''&nbsp;  According to the water&ndash;filling algorithm, the total  transmission power&nbsp; $P_X = 3$&nbsp; is now fully allocated to the first channel:
 +
:$${P_1 = 3}\hspace{0.05cm},
 +
\hspace{0.3cm}{P_2 = 0}\hspace{0.05cm}.$$
 +
 
 +
*So here for the&nbsp; "water level height":&nbsp;
 +
:$$H= 4= P_1 + \sigma_1^2= P_2 + \sigma_2^2= 3+1=0+4.$$
 +
 
 +
*This gives for the channel capacity:
 +
:$$C ={1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{3}{1} \right )
 +
+{1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{0}{4} \right )=1\,{\rm bit}+ 0\,{\rm bit}
 +
\hspace{0.15cm}\underline{= 1\,{\rm bit}}
 +
\hspace{0.05cm}.$$
 +
 
 +
Further notes:
 +
*While for&nbsp; $P_X = 10$&nbsp; the difference between even and best power splitting was only&nbsp; $0.03$&nbsp; bit, for&nbsp; $P_X = 3$&nbsp; the difference is larger:&nbsp;  $0.109$&nbsp; bit.
 +
*For even larger&nbsp; $P_X > 10$&nbsp;, the difference between even and best power splitting becomes even smaller.
 +
 
 +
 
 +
For example, for&nbsp; $P_X = 100$&nbsp; the difference is only &nbsp; $0.001$&nbsp; bit as the following calculation shows:
 +
*For&nbsp; $P_1 = P_2 = 50$&nbsp; one obtains:
 +
:$$I = I(X_1, X_2\hspace{0.05cm};\hspace{0.05cm}Y_1,  Y_2) = {1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{50}{1} \right )
 +
+{1}/{2}\cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{50}{4} \right )= 2.836\,{\rm bit}+ 1.877\,{\rm bit}
 +
\hspace{0.15cm}\underline{= 4.713\,{\rm bit}}
 +
\hspace{0.05cm}.$$
 +
*In contrast, the best possible split gives &nbsp; &#8658; &nbsp; $P_1 = 51.5, \  P_2 = 48.5$:
 +
:$$C = {1}/{2} \cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{51.5}{1} \right )
 +
+{1}/{2}\cdot  {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{48.5}{4} \right )= 2.857\,{\rm bit}+ 1.857\,{\rm bit}
 +
\hspace{0.15cm}\underline{= 4.714\,{\rm bit}}
 +
\hspace{0.05cm}.$$
 +
 
 
{{ML-Fuß}}
 
{{ML-Fuß}}
  
  
[[Category:Aufgaben zu Informationstheorie|^4.2 AWGN & kontinuierlicher Eingang^]]
+
[[Category:Information Theory: Exercises|^4.2 AWGN and Value-Continuous Input^]]

Latest revision as of 14:10, 7 October 2021

Water-filling principle  $(K = 4)$

We consider  $K$  parallel Gaussian channels  $\rm (AWGN)$  with different interference powers  $\sigma_k^2$,  where  $1 \le k \le K$ .  The graph illustrates this constellation using  $K = 4$  as an example.

The transmission power in the individual channels is denoted by  $P_k$,  the sum of which must not exceed the specified value $P_X$ :

$$P_1 +\text{...}\hspace{0.05cm}+ P_K = \hspace{0.1cm} \sum_{k= 1}^K \hspace{0.1cm}{\rm E} \left [ X_k^2\right ] \le P_{X} \hspace{0.05cm}.$$

If the random variables  $X_1$, ... , $X_K$  are Gaussian, then for the (total) mutual information between the input  $X$  and the output  $Y$  can be written:

$$I(X_1,\text{...} \hspace{0.05cm}, X_K\hspace{0.05cm};\hspace{0.05cm}Y_1, \text{...}\hspace{0.05cm}, Y_K) = 1/2 \cdot \sum_{k= 1}^K \hspace{0.1cm} {\rm log}_2 \hspace{0.1cm} ( 1 + \frac{P_k}{\sigma_k^2})\hspace{0.05cm},\hspace{0.5cm} {\rm Result\hspace{0.15cm} in \hspace{0.15cm} \text{"bit"} } \hspace{0.05cm}.$$

The maximum for this is the  channel capacity of the total system,  where the maximization refers to the division of the total power  $P_X$  among the individual channels:

$$C_K(P_X) = \max_{P_k\hspace{0.05cm},\hspace{0.15cm}{\rm with} \hspace{0.15cm}P_1 + ... \hspace{0.05cm}+ P_K = P_X} \hspace{-0.5cm} I(X_1, \text{...} \hspace{0.05cm}, X_K\hspace{0.05cm};\hspace{0.05cm}Y_1, \text{...}\hspace{0.05cm}, Y_K) \hspace{0.05cm}.$$

This maximization can be done with the  "water–filling algorithm"  shown in the above graph for  $K = 4$.

In the present exercise, this algorithm is to be applied, assuming the following:

  • Two parallel Gaussian channels   ⇒   $K = 2$,
  • normalized noise powers   $\sigma_1^2 = 1$   and   $\sigma_2^2 = 4$,
  • normalized transmission powers   $P_X = 10$   and   $P_X = 3$ respectively.



Hints:


Questions

1

Which power allocation strategies are useful?

A very noisy channel  $k$  $($with large noise power  $\sigma_k^2)$  should be allocated a large effective power  $P_k$.
A very noisy channel  $k$  $($with large noise power  $\sigma_k^2)$  should be assigned only a small useful power  $P_k$.
For  $K$  equally good channels   ⇒   $\sigma_1^2 = \text{...} = \sigma_K^2 = \sigma_N^2$  the power  $P_k$  should be evenly distributed.

2

What is the mutual information obtained by distributing the transmission power  $P_X = 10$  equally to both channels   $(P_1= P_2 = 5)$?

$I(X_1, X_2; Y_1, Y_2) \ = \ $

$\ \rm bit$

3

Let  $P_X = P_1 + P_2 = 10$  be further valid.  Which optimal power distribution results according to the  "water–filling algorithm"?

$P_1 \ = \ $

$P_2 \ = \ $

4

What is the channel capacity for  $\underline{K = 2}$  and  $\underline{P_X = 10}$?

$C \ = \ $

$\ \rm bit$

5

What mutual information results if the transmit power  $P_X = 3$  is distributed equally to both channels   $(P_1= P_2 = 1.5)$?

$I(X_1, X_2; Y_1, Y_2) \ = \ $

$\ \rm bit$

6

What is the channel capacity for  $\underline{K = 2}$  and  $\underline{P_X = 3}$?

$C \ = \ $

$\ \rm bit$


Solution

(1)  Proposed solutions 2 and 3  are correct:

  • According to the explanations in the theory section  "Water–Filling"   ⇒   Proposal 2  is to be applied when unequal conditions exist.
  • However,  solution proposal 3  is also correct:   If the channels are equally good, there is nothing to prevent all  $K$  channels from being filled with the same power   ⇒    $P_1 = P_2 =$  ...  $= P_K = P_X/K$.


(2)  For the mutual information, with equal power distribution, the following applies:

$$I(X_1, X_2\hspace{0.05cm};\hspace{0.05cm}Y_1, Y_2) \ = \ {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{5}{1} \right ) +{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{5}{4} \right )=1.292\,{\rm bit}+ 0.585\,{\rm bit} \hspace{0.15cm}\underline{= 1.877\,{\rm bit}} \hspace{0.05cm}.$$


Best possible distribution of transmit power  $P_X = 10$

(3)  According to the adjacent sketch, the following must apply:

$$P_2 = P_1 - (\sigma_2^2 - \sigma_1^2) = P_1 -3\hspace{0.3cm}\text{wobei }\hspace{0.3cm}P_1 + P_2 = P_X = 10$$
$$\Rightarrow \hspace{0.3cm} P_1 + (P_1 -3) = 10\hspace{0.3cm}\Rightarrow \hspace{0.3cm} 2 \cdot P_1 = 13 \hspace{0.3cm} \Rightarrow \hspace{0.3cm} \underline{P_1 = 6.5}\hspace{0.05cm}, \hspace{0.3cm}\underline{P_2 = 3.5}\hspace{0.05cm}.$$

For the  "water level height"  here holds: 

$$H= P_1 + \sigma_1^2= P_2 + \sigma_2^2 = 7.5 = 6.5+1 = 3.5+4.$$


(4)  The channel capacity indicates the maximum mutual information.   The maximum is already fixed by the best possible power sharing according to subtask (3). For  $P_X = 10$  holds:

$$C={1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{6.5}{1} \right ) +{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{3.5}{4} \right )$$
$$\Rightarrow\hspace{0.3cm} C=1.453\,{\rm bit}+ 0.453\,{\rm bit} \hspace{0.15cm}\underline{= 1.906\,{\rm bit}} \hspace{0.05cm}.$$


(5)  For  $P_X = 3$,   with the same power splitting  $(P_1 = P_2 =1.5)$,  we obtain:

$$I(X_1, X_2\hspace{0.05cm};\hspace{0.05cm}Y_1, Y_2) ={1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{1.5}{1} \right ) +{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{1.5}{4} \right ) \hspace{0.15cm}\underline{= 0.891\,{\rm bit}} \hspace{0.05cm}.$$


Best possible distribution of the transmit power  $P_X = 3$

(6)  According to the water–filling algorithm, the total transmission power  $P_X = 3$  is now fully allocated to the first channel:

$${P_1 = 3}\hspace{0.05cm}, \hspace{0.3cm}{P_2 = 0}\hspace{0.05cm}.$$
  • So here for the  "water level height": 
$$H= 4= P_1 + \sigma_1^2= P_2 + \sigma_2^2= 3+1=0+4.$$
  • This gives for the channel capacity:
$$C ={1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{3}{1} \right ) +{1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{0}{4} \right )=1\,{\rm bit}+ 0\,{\rm bit} \hspace{0.15cm}\underline{= 1\,{\rm bit}} \hspace{0.05cm}.$$

Further notes:

  • While for  $P_X = 10$  the difference between even and best power splitting was only  $0.03$  bit, for  $P_X = 3$  the difference is larger:  $0.109$  bit.
  • For even larger  $P_X > 10$ , the difference between even and best power splitting becomes even smaller.


For example, for  $P_X = 100$  the difference is only   $0.001$  bit as the following calculation shows:

  • For  $P_1 = P_2 = 50$  one obtains:
$$I = I(X_1, X_2\hspace{0.05cm};\hspace{0.05cm}Y_1, Y_2) = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{50}{1} \right ) +{1}/{2}\cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{50}{4} \right )= 2.836\,{\rm bit}+ 1.877\,{\rm bit} \hspace{0.15cm}\underline{= 4.713\,{\rm bit}} \hspace{0.05cm}.$$
  • In contrast, the best possible split gives   ⇒   $P_1 = 51.5, \ P_2 = 48.5$:
$$C = {1}/{2} \cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{51.5}{1} \right ) +{1}/{2}\cdot {\rm log}_2\hspace{0.05cm}\left ( 1 + \frac{48.5}{4} \right )= 2.857\,{\rm bit}+ 1.857\,{\rm bit} \hspace{0.15cm}\underline{= 4.714\,{\rm bit}} \hspace{0.05cm}.$$