# Exercise 4.5: Irrelevance Theorem

Considered optimal system with  "detector"  and  "decision"

The communication system given by the graph is to be investigated.  The binary message  $m ∈ \{m_0, m_1\}$  with equal occurrence probabilities

$${\rm Pr} (m_0 ) = {\rm Pr} (m_1 ) = 0.5$$

is represented by the two signals

$$s_0 = \sqrt{E_s} \hspace{0.05cm},\hspace{0.2cm}s_1 = -\sqrt{E_s}$$

where the assignments  $m_0 ⇔ s_0$  and  $m_1 ⇔ s_1$  are one-to-one.

The detector  $($highlighted in green in the figure$)$  provides two decision values

$$r_1 \hspace{-0.1cm} \ = \ \hspace{-0.1cm} s + n_1\hspace{0.05cm},$$
$$r_2 \hspace{-0.1cm} \ = \ \hspace{-0.1cm} n_1 + n_2\hspace{0.05cm},$$

from which the decision forms the estimated values  $\mu ∈ \{m_0,\ m_1\}$  for the transmitted message  $m$.  The decision includes

• two weighting factors  $K_1$  and  $K_2$,
• a summation point,  and
• a threshold decision with the threshold at  $0$.

Three evaluations are considered in this exercises:

1. Decision based on  $r_1$  $(K_1 ≠ 0,\ K_2 = 0)$,
2. decision based on  $r_2$  $(K_1 = 0,\ K_2 ≠ 0)$,
3. joint evaluation of  $r_1$  und  $r_2$  $(K_1 ≠ 0,\ K_2 ≠ 0)$.

Notes:

• For the error probability of a system  $r = s + n$  $($because of  $N = 1$  here  $s,\ n,\ r$  are scalars$)$  is valid:
$$p_{\rm S} = {\rm Pr} ({\rm symbol\ \ error} ) = {\rm Q} \left ( \sqrt{{2 E_s}/{N_0}}\right ) \hspace{0.05cm},$$
where a binary message signal  $s ∈ \{s_0,\ s_1\}$  with  $s_0 = \sqrt{E_s} \hspace{0.05cm},\hspace{0.2cm}s_1 = -\sqrt{E_s}$  is assumed.
• Let the two noise sources  $n_1$  and  $n_2$  be independent of each other and also independent of the transmitted signal  $s ∈ \{s_0,\ s_1\}$.
• $n_1$  and  $n_2$  can each be modeled by AWGN noise sources  $($white,  Gaussian distributed,  mean-free,  variance  $\sigma^2 = N_0/2)$.
• For numerical calculations, use the values
$$E_s = 8 \cdot 10^{-6}\,{\rm Ws}\hspace{0.05cm},\hspace{0.2cm}N_0 = 10^{-6}\,{\rm W/Hz} \hspace{0.05cm}.$$
$${\rm Q}(0) \hspace{-0.1cm} \ = \ \hspace{-0.1cm} 0.5\hspace{0.05cm},\hspace{1.35cm}{\rm Q}(2^{0.5}) = 0.786 \cdot 10^{-1}\hspace{0.05cm},\hspace{1.1cm}{\rm Q}(2) = 0.227 \cdot 10^{-1}\hspace{0.05cm},$$
$${\rm Q}(2 \cdot 2^{0.5}) \hspace{-0.1cm} \ = \ \hspace{-0.1cm} 0.234 \cdot 10^{-2}\hspace{0.05cm},\hspace{0.2cm}{\rm Q}(4) = 0.317 \cdot 10^{-4} \hspace{0.05cm},\hspace{0.2cm}{\rm Q}(4 \cdot 2^{0.5}) = 0.771 \cdot 10^{-8}\hspace{0.05cm}.$$

### Questions

1

What statements apply here regarding the receiver?

2

What is the error probability with  $K_2 = 0$?

 ${\rm Pr(symbol\hspace{0.15cm} error)}\ = \$ $\ \%$

3

What is the error probability with  $K_1 = 0$?

 ${\rm Pr(symbol\hspace{0.15cm}error)}\ = \$ $\ \%$

4

Can an improvement be achieved by using  $r_1$  and  $r_2$?

 Yes. No.

5

What are the equations for the estimated value  $(\mu)$  for AWGN noise?

 $\mu = {\rm arg \ min} \, \big[(\rho_1 + \rho_2) \cdot s_i \big]$, $\mu = {\rm arg \ min} \, \big[(\rho_2 \, - 2 \rho_1) \cdot s_i \big]$, $\mu = {\rm arg \ max} \, \big[(\rho_1 - \rho_2/2) \cdot s_i \big]$.

6

How can this rule be implemented exactly with the given decision  (threshold at zero)?  Let  $K_1 = 1$.

 $K_2 \ = \$

7

What is the  (minimum)  error probability with the realization according to subtask  (6)?

 ${\rm Minimum \ \big[Pr(symbol\hspace{0.15cm}error)\big]} \ = \$ $\ \cdot 10^{\rm -8}$

### Solution

#### Solution

(1)  The  last alternative solution  is correct:

• In general,  the MAP receiver leads to a smaller error probability.
• However,  if the occurrence probabilities  ${\rm Pr}(m = m_0) = {\rm Pr}(m = m_1) = 0.5$  are equal,  both receivers yield the same result.

(2)  With  $K_2 = 0$  and  $K_1 = 1$  the result is

$$r = r_1 = s + n_1\hspace{0.05cm}.$$
• With bipolar  (antipodal)  transmitted signal and AWGN noise,  the error probability of the optimal receiver  (whether implemented as a correlation or matched filter receiver)  is equal to
$$p_{\rm S} = {\rm Pr} ({\rm symbol\hspace{0.15cm} error} ) = {\rm Q} \left ( \sqrt{ E_s /{\sigma}}\right ) = {\rm Q} \left ( \sqrt{2 E_s /N_0}\right ) \hspace{0.05cm}.$$
• With   $E_s = 8 \cdot 10^{\rm –6} \ \rm Ws$ and $N_0 = 10^{\rm –6} \ \rm W/Hz$,  we further obtain:
$$p_{\rm S} = {\rm Pr} ({\rm symbol\hspace{0.15cm}error} ) = {\rm Q} \left ( \sqrt{\frac{2 \cdot 8 \cdot 10^{-6}\,{\rm Ws}}{10^{-6}\,{\rm W/Hz} }}\right ) = {\rm Q} (4) \hspace{0.05cm}\hspace{0.15cm}\underline {= 0.00317 \%}\hspace{0.05cm}.$$
• This result is independent of  $K_1$,  since amplification or attenuation changes the useful power in the same way as the noise power.

(3)  With  $K_1 = 0$  and  $K_2 = 1$,  the decision variable is:

$$r = r_2 = n_1 + n_2\hspace{0.05cm}.$$
• This contains no information about the useful signal,  only noise,  and it holds independently of  $K_2$:
$$p_{\rm S} = {\rm Pr} ({\rm symbol\hspace{0.15cm}error} ) = {\rm Q} (0) \hspace{0.15cm}\underline {= 50\%} \hspace{0.05cm}.$$

(4)  Because of  ${\rm Pr}(m = m_0) = {\rm Pr}(m = m_1)$,  the decision rule of the optimal receiver  (whether realized as MAP or as ML)  is:

$$\hat{m} \hspace{-0.1cm} \ = \ \hspace{-0.1cm} {\rm arg} \max_i \hspace{0.1cm} [ p_{m \hspace{0.05cm}|\hspace{0.05cm}r_1, \hspace{0.05cm}r_2 } \hspace{0.05cm} (m_i \hspace{0.05cm}|\hspace{0.05cm}\rho_1, \hspace{0.05cm}\rho_2 ) ] = {\rm arg} \max_i \hspace{0.1cm} [ p_{r_1, \hspace{0.05cm}r_2 \hspace{0.05cm}|\hspace{0.05cm} m} \hspace{0.05cm} ( \rho_1, \hspace{0.05cm}\rho_2 \hspace{0.05cm}|\hspace{0.05cm} m_i ) \cdot {\rm Pr} (m = m_i)] = {\rm arg} \max_i \hspace{0.1cm} [ p_{r_1, \hspace{0.05cm}r_2 \hspace{0.05cm}|\hspace{0.05cm} s} \hspace{0.05cm} ( \rho_1, \hspace{0.05cm}\rho_2 \hspace{0.05cm}|\hspace{0.05cm} s_i ) ] \hspace{0.05cm}.$$
• This composite probability density can be rewritten as follows:
$$\hat{m} ={\rm arg} \max_i \hspace{0.1cm} [ p_{r_1 \hspace{0.05cm}|\hspace{0.05cm} s} \hspace{0.05cm} ( \rho_1 \hspace{0.05cm}|\hspace{0.05cm} s_i ) \cdot p_{r_2 \hspace{0.05cm}|\hspace{0.05cm} r_1, \hspace{0.05cm}s} \hspace{0.05cm} ( \rho_2 \hspace{0.05cm}|\hspace{0.05cm} \rho_1, \hspace{0.05cm}s_i ) ] \hspace{0.05cm}.$$
• Now,  since the second multiplicand also depends on the message  ($s_i$),  $r_2$  should definitely be included in the decision process.  Thus,  the correct answer is:  YES.

(5)  For AWGN noise with variance  $\sigma^2$,  the two composite densities introduced in  (4)  together with their product  $P$  give:

$$p_{r_1\hspace{0.05cm}|\hspace{0.05cm} s}(\rho_1\hspace{0.05cm}|\hspace{0.05cm}s_i) \hspace{-0.1cm} \ = \ \hspace{-0.1cm} \frac{1}{\sqrt{2\pi} \cdot \sigma}\cdot {\rm exp} \left [ - \frac{(\rho_1 - s_i)^2}{2 \sigma^2}\right ]\hspace{0.05cm},\hspace{1cm} p_{r_2\hspace{0.05cm}|\hspace{0.05cm}r_1,\hspace{0.05cm} s}(\rho_2\hspace{0.05cm}|\hspace{0.05cm}\rho_1, \hspace{0.05cm}s_i) \hspace{-0.1cm} \ = \ \hspace{-0.1cm} \frac{1}{\sqrt{2\pi} \cdot \sigma}\cdot {\rm exp} \left [ - \frac{(\rho_2 - (\rho_1 - s_i))^2}{2 \sigma^2}\right ]\hspace{0.05cm}$$
$$\Rightarrow \hspace{0.3cm} P \hspace{-0.1cm} \ = \ \hspace{-0.1cm} \frac{1}{{2\pi} \cdot \sigma^2}\cdot {\rm exp} \left [ - \frac{1}{2 \sigma^2} \cdot \left \{ (\rho_1 - s_i)^2 + (\rho_2 - (\rho_1 - s_i))^2\right \}\right ]\hspace{0.05cm}.$$
• We are looking for the argument that maximizes this product  $P$,  which at the same time means that the expression in the curly brackets should take the smallest possible value:
$$\mu = \hat{m} \hspace{-0.1cm} \ = \ \hspace{-0.1cm}{\rm arg} \max_i \hspace{0.1cm} P = {\rm arg} \min _i \hspace{0.1cm} \left \{ (\rho_1 - s_i)^2 + (\rho_2 - (\rho_1 - s_i))^2\right \}$$
$$\Rightarrow \hspace{0.3cm} \mu = \hat{m} \hspace{-0.1cm} \ = \ \hspace{-0.1cm}{\rm arg} \min _i \hspace{0.1cm}\left \{ \rho_1^2 - 2\rho_1 s_i + s_i^2 + \rho_2^2 - 2\rho_1 \rho_2 + 2\rho_2 s_i+ \rho_1^2 - 2\rho_1 s_i + s_i^2\right \} \hspace{0.05cm}.$$
• Here  $\mu$  denotes the estimated value of the message.  In this minimization,  all terms that do not depend on the message  $s_i$  can now be omitted.  Likewise,  the terms  $s_i^2$  are disregarded,  since  $s_0^2 = s_1^2$  holds.  Thus,  the much simpler decision rule is obtained:
$$\mu = {\rm arg} \min _i \hspace{0.1cm}\left \{ - 4\rho_1 s_i + 2\rho_2 s_i \right \}={\rm arg} \min _i \hspace{0.1cm}\left \{ (\rho_2 - 2\rho_1) \cdot s_i \right \} \hspace{0.05cm}.$$
• So,  correct is already the proposed solution 2.  But after multiplication by  $–1/2$,  we also get the last mentioned decision rule:
$$\mu = {\rm arg} \max_i \hspace{0.1cm}\left \{ (\rho_1 - \rho_2/2) \cdot s_i \right \} \hspace{0.05cm}.$$
• Thus,  the solutions 2 and 3  are correct.

(6)  Setting $K_1 = 1$  and  $\underline {K_2 = \, -0.5}$,  the optimal decision rule with realization  $\rho = \rho_1 \, – \rho_2/2$ is:

$$\mu = \left\{ \begin{array}{c} m_0 \\ m_1 \end{array} \right.\quad \begin{array}{*{1}c} {\rm f{or}} \hspace{0.15cm} \rho > 0 \hspace{0.05cm}, \\ {\rm f{or}} \hspace{0.15cm} \rho < 0 \hspace{0.05cm}.\\ \end{array}$$
• Since  $\rho = 0$  only occurs with probability  $0$,  it does not matter in the sense of probability theory whether one assigns the message  $\mu = m_0$  or  $\mu = m_1$  to this event  "$\rho = 0$".

(7)  With  $K_2 = \, -0.5$  one obtains for the input value of the decision:

$$r = r_1 - r_2/2 = s + n_1 - (n_1 + n_2)/2 = s + n \hspace{0.3cm} \Rightarrow \hspace{0.3cm} n = \frac{n_1 - n_2}{2}\hspace{0.05cm}.$$
• The variance of this random variable is
$$\sigma_n^2 = {1}/{4} \cdot \left [ \sigma^2 + \sigma^2 \right ] = {\sigma^2}/{2}= {N_0}/{4}\hspace{0.05cm}.$$
• From this,  the error probability is analogous to subtask  (2):
$${\rm Pr} ({\rm symbol\hspace{0.15cm}error} ) = {\rm Q} \left ( \sqrt{\frac{E_s}{N_0/4}}\right ) = {\rm Q} \left ( 4 \cdot \sqrt{2}\right ) = {1}/{2} \cdot {\rm erfc}(4) \hspace{0.05cm}\hspace{0.15cm}\underline {= 0.771 \cdot 10^{-8}} \hspace{0.05cm}.$$
• Thus,  by taking  $r_2$  into account,  the error probability can be lowered from  $0.317 \cdot 10^{\rm –4}$  to the much smaller value of  $0.771 \cdot 10^{-8}$,  although the decision component  $r_2$  contains only noise.  However,  this noise  $r_2$  allows an estimate of the noise component  $n_1$  of  $r_1$.
• Halving the transmit energy from  $8 \cdot 10^{\rm –6} \ \rm Ws$  to  $4 \cdot 10^{\rm –6} \ \rm Ws$,  we still get the error probability  $0.317 \cdot 10^{\rm –4}$  here,  as calculated in subtask  (2).  When evaluating  $r_1$  alone,  on the other hand,  the error probability would be  $0.234 \cdot 10^{\rm –2}$.