View source for Aufgaben:Exercise 4.1Z: Log Likelihood Ratio at the BEC Model

{{quiz-Header|Buchseite=Channel_Coding/Soft-in_Soft-Out_Decoder}}

[[File:P_ID2978__KC_Z_4_1.png|right|frame|BEC channel model]]
We consider the so-called&nbsp; [[Channel_Coding/Channel_Models_and_Decision_Structures#Binary_Erasure_Channel_.E2.80.93_BEC| $\text{BEC channel }$]]&nbsp; ("binary erasure channel")&nbsp; with
* the input variable&nbsp; $x &#8712; \{+1, \, -1\}$,

* the output variable&nbsp; $y &#8712; \{+1, \, -1, \, {\rm E}\}$,&nbsp; and

* the erasure probability&nbsp; $\lambda$.


Here&nbsp; $y = {\rm E}$&nbsp; ("erasure")&nbsp; means that the initial value&nbsp; $y$&nbsp; could neither be decided as &nbsp;"$+1$"&nbsp; nor as &nbsp;"$-1$".

Also known are the input probabilities
:$${\rm Pr}(x = +1) = 3/4\hspace{0.05cm}, \hspace{0.5cm}{\rm Pr}(x = -1) = 1/4\hspace{0.05cm}.$$

The log likelihood ratio of the binary random variable&nbsp; $x$&nbsp; is given by bipolar approach as follows:
:$$L(x)={\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(x = +1)}{{\rm Pr}(x = -1)}\hspace{0.05cm}.$$

Correspondingly,&nbsp; for the conditional log likelihood ratio in forward direction for all&nbsp; $y &#8712; \{+1, \, -1, \, {\rm E}\}$:
:$$L(y\hspace{0.05cm}|\hspace{0.05cm}x) =
{\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(y\hspace{0.05cm}|\hspace{0.05cm}x = +1)}{{\rm Pr}(y\hspace{0.05cm}|\hspace{0.05cm}x = -1)} \hspace{0.05cm}. $$





<u>Hints:</u>
* This exercise belongs to the chapter&nbsp; [[Channel_Coding/Soft-in_Soft-Out_Decoder| "Soft&ndash;in Soft&ndash;out Decoder"]].

* Reference is made in particular to the sections&nbsp; 
:*[[Channel_Coding/Soft-in_Soft-Out_Decoder#Reliability_information_-_Log_Likelihood_Ratio| "Reliability Information &ndash; Log Likelihood Ratio"]],&nbsp; 
:*[[Channel_Coding/Channel_Models_and_Decision_Structures#Binary_Erasure_Channel_.E2.80.93_BEC|"Binary Erasure Channel"]].





===Questions===
<quiz display=simple>
{What is the log likelihood ratio of the input variable&nbsp; $x$?
|type="{}"}
$L(x) \ = \ ${ 1.099 3% } 

{What probability&nbsp; ${\rm Pr}(x = \, -1)$&nbsp; corresponds to&nbsp; $L(x) = \, -2$?
|type="{}"}
${\rm Pr}(x = \, -1) \ = \ ${ 0.881 3% }

{Calculate the conditional&nbsp; L&ndash;value&nbsp; $L(y = {\rm E}\hspace{0.05cm} |\hspace{0.05cm} x)$&nbsp; in the forward direction.
|type="{}"}
$L(y = {\rm E} \hspace{0.05cm} |\hspace{0.05cm} x) \ = \ ${ 0. }

{Which statements are true for the other two conditional log likelihood ratios?
|type="[]"}
+ $L(y = +1 \hspace{0.05cm} |\hspace{0.05cm} x)$&nbsp; is positive and infinite in magnitude.
+ $L(y = \, -1 \hspace{0.05cm} |\hspace{0.05cm} x)$&nbsp; is negative and infinite in magnitude.
- It holds&nbsp; $L(y = +1 \hspace{0.05cm} |\hspace{0.05cm} x) = L(y = \, -1 \hspace{0.05cm} |\hspace{0.05cm} x) = 0$.

{Under what conditions do the results from subtasks&nbsp; '''(3)'''&nbsp; and&nbsp; '''(4)'''&nbsp; hold?
|type="()"}
- For&nbsp; $0 &#8804; \lambda &#8804; 1$.
- For&nbsp; $0 < \lambda &#8804; 1$.
- For&nbsp; $0 &#8804; \lambda < 1$.
+ For&nbsp; $0 < \lambda < 1$.
</quiz>

===Solution===
{{ML-Kopf}}
'''(1)'''&nbsp; With the given symbol probabilities &nbsp; ${\rm Pr}(x = +1) = 3/4$ &nbsp; and &nbsp; ${\rm Pr}(x = -1) = 1/4$,&nbsp; we obtain:
:$$L(x)={\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(x = +1)}{{\rm Pr}(x = -1)}
={\rm ln} \hspace{0.15cm} \frac{3/4}{1/4}\hspace{0.15cm}\underline{= 1.099}\hspace{0.05cm}.$$


'''(2)'''&nbsp; According to the definition
:$$L(x)={\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(x = +1)}{{\rm Pr}(x = -1)}$$

yields the following equation for&nbsp; $L(x) = \, -2$:
:$$\hspace{0.15cm} \frac{{\rm Pr}(x = +1)}{1-{\rm Pr}(x = +1)} \stackrel{!}{=}{\rm e}^{-2} \approx 0.135 \hspace{0.25cm}\Rightarrow \hspace{0.25cm}
1.135 \cdot {\rm Pr}(x = +1)\stackrel{!}{=}0.135\hspace{0.3cm}
\Rightarrow \hspace{0.3cm}
{\rm Pr}(x = +1) = 0.119\hspace{0.05cm},\hspace{0.4cm}{\rm Pr}(x = -1)
\hspace{0.15cm}\underline{= 0.881}\hspace{0.05cm}. $$


'''(3)'''&nbsp; For the conditional log likelihood ratio &nbsp; $L(y = {\rm E} \hspace{0.05cm} |\hspace{0.05cm} x)$ &nbsp; in the forward direction,&nbsp; valid for the BEC model:
:$$L(y = {\rm E}\hspace{0.05cm}|\hspace{0.05cm}x) =
{\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(y= {\rm E}\hspace{0.05cm}|\hspace{0.05cm}x = +1)}{{\rm Pr}(y= {\rm E}\hspace{0.05cm}|\hspace{0.05cm}x = -1)} 
= {\rm ln} \hspace{0.15cm} \frac{\lambda}{\lambda}\hspace{0.15cm}\underline{= 0}\hspace{0.05cm}.$$


'''(4)'''&nbsp; Analogous to the sample solution of subtask&nbsp; '''(3)''',&nbsp; we obtain for&nbsp; $y = &plusmn;1$:
:$$L(y = +1\hspace{0.05cm}|\hspace{0.05cm}x) \hspace{-0.15cm} \ = \ \hspace{-0.15cm}
{\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(y= +1\hspace{0.05cm}|\hspace{0.05cm}x = +1)}{{\rm Pr}(y= +1\hspace{0.05cm}|\hspace{0.05cm}x = -1)} 
= {\rm ln} \hspace{0.15cm} \frac{1-\lambda}{0}\hspace{0.15cm}\underline{ \hspace{0.05cm}\Rightarrow \hspace{0.15cm}+\infty }\hspace{0.05cm},$$
:$$L(y = -1\hspace{0.05cm}|\hspace{0.05cm}x) \hspace{-0.15cm} \ = \ \hspace{-0.15cm}
{\rm ln} \hspace{0.15cm} \frac{{\rm Pr}(y= -1\hspace{0.05cm}|\hspace{0.05cm}x = +1)}{{\rm Pr}(y= -1\hspace{0.05cm}|\hspace{0.05cm}x = -1)} 
= {\rm ln} \hspace{0.15cm} \frac{0}{1-\lambda}\hspace{0.15cm}\underline{ \hspace{0.05cm}\Rightarrow \hspace{0.15cm}-\infty }\hspace{0.05cm}.  $$

*Accordingly,&nbsp; the&nbsp; <u>proposed solutions 1 and 2</u>&nbsp; are correct.


'''(5)'''&nbsp; Correct is the&nbsp; <u>last proposed solution</u>:
* For&nbsp; $\lambda = 0$&nbsp; $($"ideal channel"$)$ &nbsp; &rArr; &nbsp; $L(y = {\rm E} \hspace{0.05cm} |\hspace{0.05cm} x) = \ln {(0/0)}$ &nbsp; &#8658; &nbsp; indefinite result.

* For&nbsp; $\lambda = 1$&nbsp; $($complete erasure, &nbsp; $y &equiv; {\rm E})$ &nbsp; &rArr; &nbsp; $L(y = +1 \hspace{0.05cm} |\hspace{0.05cm} x)$&nbsp; and &nbsp; $L(y = \, -1 \hspace{0.05cm} |\hspace{0.05cm} x)$ &nbsp; are undefined.
{{ML-Fuß}}



[[Category:Channel Coding: Exercises|^4.1 Soft–in Soft–out Decoder^]]