Difference between revisions of "Channel Coding/The Basics of Turbo Codes"

From LNTwww
 
(32 intermediate revisions by 4 users not shown)
Line 1: Line 1:
 
   
 
   
 
{{Header
 
{{Header
|Untermenü=Iterative Decodierverfahren
+
|Untermenü=Iterative Decoding Methods
|Vorherige Seite=Grundlegendes zu den Produktcodes
+
|Vorherige Seite=The Basics of Product Codes
|Nächste Seite=Grundlegendes zu den Low–density Parity–check Codes
+
|Nächste Seite=The Basics of Low-Density Parity Check Codes
 
}}
 
}}
  
== Grundstruktur eines Turbocodes ==
+
== Basic structure of a turbo code ==
 
<br>
 
<br>
Alle heute (2017) aktuellen Kommunikationssysteme wie&nbsp; [[Examples_of_Communication_Systems/Allgemeine_Beschreibung_von_UMTS#.23_.C3.9CBERBLICK_ZUM_VIERTEN_HAUPTKAPITEL_.23|UMTS]]&nbsp; (<i>Universal Mobile Telecommunications System</i> &nbsp; &rArr; &nbsp;   3. Mobilfunkgeneration) und&nbsp; [[Mobile_Communications/Allgemeines_zum_Mobilfunkstandard_LTE#.23_.C3.9CBERSICHT_ZUM_VIERTEN_HAUPTKAPITEL_.23|LTE]]&nbsp; (<i>Long Term Evolution</i>&nbsp; &rArr; &nbsp; 4. Mobilfunkgeneration) verwenden das Konzept der&nbsp;  [[Channel_Coding/Soft–in_Soft–out_Decoder#Symbolweise_Soft.E2.80.93in_Soft.E2.80.93out_Decodierung| symbolweisen iterativen Decodierung]]. Dass dies so ist, steht unmittelbar mit der Erfindung der&nbsp; ''Turbocodes''&nbsp; im Jahre 1993 durch&nbsp; [https://de.wikipedia.org/wiki/Claude_Berrou Claude Berrou],&nbsp; [https://de.wikipedia.org/wiki/Alain_Glavieux Alain Glavieux]&nbsp; und&nbsp; [https://scholar.google.com/citations?user=-UZolIAAAAAJ Punya Thitimajshima]&nbsp; in Zusammenhang, denn erst mit diesen Codes konnte man sich der Shannon&ndash;Grenze mit vertretbarem Decodieraufwand annähern.<br>
+
All communications systems current today&nbsp; $($2017$)$,&nbsp; such as&nbsp;
 +
* [[Examples_of_Communication_Systems/General_Description_of_UMTS#.23_OVERVIEW_OF_THE_FOURTH_MAIN_CHAPTER_.23|$\rm UMTS$]]&nbsp; $($"Universal Mobile Telecommunications System" &nbsp; &rArr; &nbsp; 3rd generation mobile communications$)$ and&nbsp;
 +
*[[Mobile_Communications/General_Information_on_the_LTE_Mobile_Communications_Standard#.23_OVERVIEW_OF_THE_FOURTH_MAIN_CHAPTER_.23|$\rm LTE$]]&nbsp; $($"Long Term Evolution"&nbsp; &rArr; &nbsp; 4th generation mobile communications$)$
  
Turbocodes ergeben sich durch die parallele oder serielle Verkettung von Faltungscodes. Die Grafik zeigt die parallele Verkettung zweier Codes, jeweils mit den Parametern&nbsp; $k = 1, \ n = 2$ &nbsp; &#8658; &nbsp; Rate $R = 1/2$.<br>
 
  
[[File:P ID3034 KC T 4 3 S1a v1.png|center|frame|Parallele Verkettung zweier Rate–$1/2$–Codes|class=fit]]
+
use the concept of&nbsp; [[Channel_Coding/Soft-in_Soft-Out_Decoder#Bit-wise_soft-in_soft-out_decoding|$\text{symbol-wise iterative decoding}$]].&nbsp; This  is directly related to the invention of&nbsp; &raquo;'''turbo codes'''&laquo;&nbsp; in 1993 by&nbsp; [https://en.wikipedia.org/wiki/Claude_Berrou $\text{Claude Berrou}$],&nbsp; [https://en.wikipedia.org/wiki/Alain_Glavieux $\text{Alain Glavieux}$]&nbsp; and&nbsp; [https://scholar.google.com/citations?user=-UZolIAAAAAJ $\text{Punya Thitimajshima}$]&nbsp; because it was only with these codes that the Shannon bound could be approached with reasonable decoding effort.<br>
  
In dieser Darstellung bezeichnet:
+
Turbo codes result from the parallel or serial concatenation of convolutional codes.&nbsp;
* $u$&nbsp; das aktuell betrachtete Bit der Informationssequenz&nbsp; $\underline{u}$,<br>
+
[[File:EN_KC_T_4_3_S1a_v1.png|right|frame|Parallel concatenation of two rate&nbsp; $1/2$&nbsp; codes|class=fit]]
 +
[[File:EN_KC_T_4_3_S1b_v1.png|right|frame|Rate&nbsp; $1/3$&nbsp; turbo encoder&nbsp; $($parallel concatenation of two rate&nbsp; $1/2$&nbsp; convolutional codes$)$ |class=fit]]
 +
The graphic shows the parallel concatenation of two codes,&nbsp; each with the parameters&nbsp; $k = 1, \ n = 2$ &nbsp; &#8658; &nbsp; code rate $R = 1/2$.<br>
  
* $x_{i,\hspace{0.03cm}j}$&nbsp; das aktuell betrachtete Bit am Ausgang&nbsp; $j$&nbsp; von Coder&nbsp; $i$&nbsp; $($mit&nbsp; $1 &#8804; i &#8804; 2, \ 1 &#8804; j &#8804; 2)$,<br>
+
In this representation is:
 +
# $u$&nbsp; the currently considered bit of the information sequence&nbsp; $\underline{u}$,<br>
 +
# $x_{i,\hspace{0.03cm}j}$&nbsp; the currently considered bit at the output&nbsp; $j$&nbsp; of encoder&nbsp; $i$&nbsp; <br>$($with&nbsp; $1 &#8804; i &#8804; 2, \hspace{0.2cm} 1 &#8804; j &#8804; 2)$,<br>
 +
# $\underline{X} = (x_{1,\hspace{0.03cm}1}, \ x_{1,\hspace{0.03cm}2}, \ x_{2,\hspace{0.03cm}1}, \ x_{2,\hspace{0.03cm}2})$&nbsp; the code word for the current information bit&nbsp; $u$.<br><br>
  
* $\underline{X} = (x_{1,\hspace{0.03cm}1}, \ x_{1,\hspace{0.03cm}2}, \ x_{2,\hspace{0.03cm}1}, \ x_{2,\hspace{0.03cm}2})$&nbsp; das Codewort für das aktuelle Informationsbit&nbsp; $u$.<br><br>
+
The resulting rate of the concatenated coding system is thus&nbsp; $R = 1/4$.&nbsp;  
  
Die resultierende Rate des verketteten Codiersystems ist somit&nbsp; $R = 1/4$.<br>
+
If systematic component codes are used,&nbsp; the second shown model results.&nbsp; The modifications from the top graph to the graph below can be justified as follows:
 +
*For systematic codes&nbsp; $C_1$&nbsp; and&nbsp; $C_2$,&nbsp; both&nbsp; $x_{1,\hspace{0.03cm}1} = u$&nbsp; and&nbsp; $x_{2,\hspace{0.03cm}1} = u$.&nbsp; Therefore,&nbsp; one can dispense with the transmission of a redundant bit&nbsp; $($e.g.&nbsp; $x_{2,\hspace{0.03cm}2})$.
  
Verwendet man systematische Komponentencodes, so ergibt sich das folgende Modell:<br>
+
*With this reduction,&nbsp; the result is a rate&nbsp; $1/3$&nbsp; turbo code with&nbsp; $k = 1$&nbsp; and&nbsp; $n = 3$.&nbsp; The code word with the parity bits is&nbsp; $p_1$&nbsp; $($encoder 1$)$&nbsp; and&nbsp; $p_2$&nbsp; $($encoder 2$)$:  
 
 
[[File:P ID3035 KC T 4 3 S1b v1.png|center|frame|Rate–1/3–Turbocodierer (parallele Verkettung zweier Rate–$1/2$–Faltungscodes) |class=fit]]
 
 
 
Die Modifikationen gegenüber der oberen Grafik lassen sich wie folgt begründen:
 
*Bei systematischen Codes&nbsp; $C_1$&nbsp; und&nbsp; $C_2$&nbsp; ist sowohl&nbsp; $x_{1,\hspace{0.03cm}1} = u$&nbsp; als auch&nbsp; $x_{2,\hspace{0.03cm}1} = u$. Deshalb kann man auf die Übertragung eines redundanten Bits&nbsp; $($zum Beispiel $x_{2,\hspace{0.03cm}2})$&nbsp; verzichten.
 
 
 
*Mit dieser Reduktion ergibt sich ein Rate&ndash;$1/3$&ndash;Turbocode mit&nbsp; $k = 1$&nbsp; und&nbsp; $n = 3$. Das Codewort lautet mit den Paritybits&nbsp; $p_1$&nbsp; (Coder 1) und&nbsp; $p_2$&nbsp; (Coder 2):  
 
 
:$$\underline{X} = \left (u, \ p_1, \ p_2 \right )\hspace{0.05cm}.$$
 
:$$\underline{X} = \left (u, \ p_1, \ p_2 \right )\hspace{0.05cm}.$$
  
  
== Weitere Modifizierung der Turbocode&ndash;Grundstruktur ==
+
== Further modification of the basic structure of the turbo code==
 
<br>
 
<br>
Im Folgenden gehen wir stets von einem noch etwas weiter modifizierten Turbocoder&ndash;Modell aus:
+
In the following we always assume a still somewhat further modified turbo encoder model:
*Wie es für die Beschreibung von Faltungscodes erforderlich ist,  liegt nun am Eingang anstelle des isolierten Informationsbits&nbsp; $u$&nbsp; die Informationssequenz&nbsp; $\underline{u} = (u_1, \ u_2, \ \text{...}\hspace{0.05cm} , \ u_i , \ \text{...}\hspace{0.05cm} )$&nbsp; an.<br>
+
[[File:EN_KC_T_4_3_S1c_v2.png|right|frame|Rate&nbsp; $1/3$&nbsp; turbo encoder with interleaver|class=fit]]
  
*Die Codewortsequenz wird mit&nbsp; $\underline{x} = (\underline{X}_1, \underline{X}_2, \ \text{...}\hspace{0.05cm}  , \ \underline{X}_i, \ \text{...}\hspace{0.05cm} )$&nbsp; bezeichnet. Um Verwechslungen zu vermeiden, wurden vorne die Codeworte&nbsp; $\underline{X}_i = (u, \ p_1, \ p_2)$&nbsp; mit Großbuchstaben eingeführt.<br>
+
#As required for the description of convolutional codes,&nbsp; at the input  is the information sequence&nbsp; $\underline{u} = (u_1, \ u_2, \ \text{...}\hspace{0.05cm} , \ u_i , \ \text{...}\hspace{0.05cm} )$&nbsp; instead of the isolated information bit&nbsp; $u$&nbsp;.<br>
 +
#The code word sequence&nbsp; $\underline{x} = (\underline{X}_1, \underline{X}_2, \ \text{...}\hspace{0.05cm}  , \ \underline{X}_i, \ \text{...}\hspace{0.05cm} )$&nbsp; is generated.&nbsp;  To avoid confusion,&nbsp; the code words&nbsp; $\underline{X}_i = (u, \ p_1, \ p_2)$&nbsp; with capital letters were introduced in the last section.<br>
 +
#The encoders&nbsp; $\mathcal{C}_1$&nbsp; and&nbsp; $\mathcal{C}_2$&nbsp; are conceived&nbsp; $($at least in thought$)$&nbsp; as&nbsp; [[Theory_of_Stochastic_Signals/Digital_Filters| $\text{digital filters}$]]&nbsp; and are thus characterized by the&nbsp; [[Channel_Coding/Algebraic_and_Polynomial_Description#Application_of_the_D.E2.80.93transform_to_rate_.7F.27.22.60UNIQ-MathJax158-QINU.60.22.27.7F_convolution_encoders| $\text{transfer functions}$]]&nbsp; $G_1(D)$&nbsp; and&nbsp; $G_2(D)$.<br>
 +
#For various reasons &nbsp; &#8658; &nbsp; see&nbsp; [[Channel_Coding/The_Basics_of_Turbo_Codes#Second_requirement_for_turbo_codes:_Interleaving| "two sections ahead"]]&nbsp; the input sequence of the second encoder &nbsp; &#8658; &nbsp; $\underline{u}_{\pi}$&nbsp; should be scrambled with respect to the sequence&nbsp; $\underline{u}$&nbsp; by an interleaver&nbsp; $(\Pi)$.<br>
 +
#Thereby there is nothing against choosing both encoders the same:&nbsp; $G_1(D) = G_2(D) = G(D)$.&nbsp; Without interleaver the correction capability would be extremely limited.<br><br>
  
[[File:P ID3036 KC T 4 3 S1c v1.png|center|frame|Rate–$1/3$–Turbocodierer mit Interleaver|class=fit]]
+
{{GraueBox|TEXT=
 +
[[File:EN_KC_T_4_3_S1d_v2.png|right|frame|Example sequences at the rate&nbsp; $1/3$&nbsp; turbo encoder|class=fit]]  
 +
$\text{Example 1:}$&nbsp; The graph shows the different sequences in matched colors. To note:
 +
#For&nbsp; $\underline{u}_{\Pi}$&nbsp; a&nbsp; $3×4$&nbsp; interleaver matrix is considered according to&nbsp; [[Aufgaben:Exercise_4.08Z:_Basics_about_Interleaving|"Exercise 4.8Z"]].<br><br>
 +
#The parity sequences are obtained according to&nbsp; $G_1(D) = G_2(D) = 1 + D^2$ &nbsp; &#8658; &nbsp; see&nbsp; [[Aufgaben:Exercise_4.08:_Repetition_to_the_Convolutional_Codes|"Exercise 4.8"]].}}<br>
  
*Die Coder&nbsp; $\mathcal{C}_1$&nbsp; und&nbsp; $\mathcal{C}_2$&nbsp; werden (zumindest gedanklich) als&nbsp; [[Theory_of_Stochastic_Signals/Digitale_Filter| Digitale Filter]]&nbsp; konzipiert und sind somit durch die&nbsp; [[Channel_Coding/Algebraische_und_polynomische_Beschreibung#Anwendung_der_D.E2.80.93Transformation_auf_Rate.E2.80.931.2Fn.E2.80.93Faltungscoder| Übertragungsfunktionen]]&nbsp; $G_1(D)$&nbsp; und&nbsp; $G_2(D)$&nbsp; darstellbar.<br>
+
== First requirement for turbo codes: Recursive component codes ==
 +
<br>
 +
Non-recursive transfer functions for generating the parity sequences cause a turbo code with insufficiently small minimum distance.&nbsp; The reason for this inadequacy is the finite impulse response&nbsp; $\underline{g} = (1, \ g_2, \ \text{...}\hspace{0.05cm}  , \ g_m, \ 0, \ 0, \ \text{...}\hspace{0.05cm} )$&nbsp; with&nbsp; $g_2, \ \text{...}\hspace{0.05cm}  , \ g_m &#8712; \{0, 1\}$.&nbsp; Here&nbsp; $m$&nbsp; denotes the&nbsp;  "memory".<br>
 +
[[File:EN_KC_T_4_3_S2a_v1.png|right|frame|Non-recursive systematic turbo code and state transition diagram|class=fit]]
  
*Aus verschiedenen Gründen &nbsp; &#8658; &nbsp; siehe&nbsp; [[Channel_Coding/Grundlegendes_zu_den_Turbocodes#Zweite_Voraussetzung_f.C3.BCr_Turbocodes:_Interleaving| übernächste Seite]]&nbsp; sollte die Eingangssequenz des zweiten Coders &nbsp; &#8658; &nbsp; $\underline{u}_{\pi}$&nbsp; gegenüber der Sequenz&nbsp; $\underline{u}$&nbsp; durch einen Interleaver&nbsp; $(\Pi)$&nbsp; verwürfelt sein.<br>
+
<br><br>The graph shows the state transition diagram for the example&nbsp; $\mathbf{G}(D) = \big [1, \ 1 + D^2 \big ]$. The transitions are labeled "$u_i\hspace{0.05cm}|\hspace{0.05cm}u_i p_i$".
 +
*The sequence&nbsp; $S_0 &#8594; S_1 &#8594; S_2 &#8594; S_0 &#8594; S_0 &#8594; \ \text{...}\hspace{0.05cm} \ $&nbsp; leads with respect to the input to the information sequence&nbsp; $\underline{u} = (1, 0, 0, 0, 0, \ \text{...}\hspace{0.05cm})$,&nbsp; and
  
*Somit spricht nichts dagegen, beide Coder gleich zu wählen: &nbsp; $G_1(D) = G_2(D) = G(D)$. Ohne Interleaver würde die Korrekturfähigkeit extrem eingeschränkt sein.<br><br>
+
*with respect to the second encoded symbol to the sequence &nbsp; $\underline{p} = (1, 0, 1, 0, 0, \ \text{...}\hspace{0.05cm})$&nbsp; &nbsp; &#8658; &nbsp; because of&nbsp; $\underline{u} = (1, 0, 0, 0, 0, \ \text{...}\hspace{0.05cm})$&nbsp; identical to the&nbsp; "impulse response"&nbsp; $\underline{g}$ &nbsp; &#8658; &nbsp; memory&nbsp; $m = 2$.<br>
 +
<br clear=all>
 +
The lower graph applies to a so-called&nbsp; &raquo;'''RSC code'''&laquo;&nbsp; $($"Recursive Systematic Convolutional"$)$&nbsp; correspondingly&nbsp;
 +
[[File:EN_KC_T_4_3_S2b_v1.png|right|frame|Non-recursive systematic turbo code and state transition diagram|class=fit]]
 +
:$$\mathbf{G}(D) = \big [1, \ (1+ D^2)/(1 + D + D^2)\big ].$$
 +
*Here the sequence&nbsp;
 +
:$$S_0 &#8594; S_1 &#8594; S_3 &#8594; S_2 &#8594; S_1 &#8594; S_3 &#8594; S_2 &#8594;  \text{...}$$
 +
:leads to the impulse response&nbsp;
 +
:$$\underline{g} = (1, 1, 1, 0, 1, 1, 0, 1, 1, \ \text{...}\hspace{0.05cm}).$$
 +
*This impulse response continues to infinity due to the loop&nbsp; $S_1 &#8594; S_3 &#8594; S_2 &#8594; S_1$.&nbsp; This enables or facilitates the iterative decoding.<br>
  
{{GraueBox|TEXT= 
 
$\text{Beispiel 1:}$&nbsp; Die Grafik zeigt die verschiedenen Sequenzen in angepassten Farben. Anzumerken ist:
 
#&nbsp;&nbsp; Für&nbsp; $\underline{u}_{\pi}$&nbsp;  ist eine&nbsp; $3×4$&ndash;Interleaver&ndash;Matrix entsprechend&nbsp; [[Aufgaben:Aufgabe_4.08Z:_Grundlegendes_zum_Interleaving|Aufgabe 4.8Z]]&nbsp; berücksichtigt.
 
# &nbsp;&nbsp;<sub>&nbsp;</sub>Die Paritysequenzen ergeben sich gemäß&nbsp; $G_1(D) = G_2(D) = 1 + D^2$ &nbsp; &#8658; &nbsp; siehe&nbsp; [[Aufgaben:Aufgabe_4.08:_Wiederholung_zu_den_Faltungscodes|Aufgabe 4.8]].<br>
 
  
[[File:P ID3037 KC T 4 3 S1d v2.png|center|frame|Beispielhafte Sequenzen beim Rate–$1/3$–Turbocodierer|class=fit]]}}
+
More details on the examples in this section can be found in the&nbsp; [[Aufgaben:Exercise_4.08Z:_Basics_about_Interleaving|"Exercise 4.8"]]&nbsp; and the&nbsp; [[Aufgaben:Exercise_4.09:_Recursive_Systematic_Convolutional_Codes|"Exercise 4.9"]].<br>
  
== Erste Voraussetzung für Turbocodes: Rekursive Komponentencodes ==
+
== Second requirement for turbo codes: Interleaving ==
 
<br>
 
<br>
Nichtrekursive Übertragungsfunktionen zur Erzeugung der Paritysequenzen bewirken einen Turbocode mit unzureichend kleiner Minimaldistanz. Grund für diese Unzulänglichkeit ist die endliche Impulsantwort&nbsp; $\underline{g} = (1, \ g_2, \ \text{...}\hspace{0.05cm}  , \ g_m, \ 0, \ 0, \ \text{...}\hspace{0.05cm} )$&nbsp; mit&nbsp; $g_2, \ \text{...}\hspace{0.05cm}  , \ g_m &#8712; \{0, 1\}$. Hierbei  bezeichnet&nbsp; $m$&nbsp; das Codegedächtnis.<br>
+
It is obvious that for&nbsp; $G_1(D) = G_2(D)$&nbsp; an interleaver&nbsp; $(\Pi)$&nbsp; is essential.
 +
*Another reason is that the a-priori information is assumed to be independent.  
  
Die Grafik zeigt das Zustandsübergangsdiagramm für das Beispiel&nbsp; $\mathbf{G}(D) = \big [1, \ 1 + D^2 \big ]$. Die Übergänge sind mit &bdquo;$u_i\hspace{0.05cm}|\hspace{0.05cm}u_i p_i$&rdquo; beschriftet.
+
*Thus,&nbsp; adjacent&nbsp; $($and thus possibly strongly dependent$)$&nbsp; bits for the other sub&ndash;code should be far apart.<br>
*Die Abfolge&nbsp; $S_0 &#8594; S_1 &#8594; S_2 &#8594; S_0 &#8594; S_0 &#8594; S_0 &#8594; \ \text{...}\hspace{0.05cm} \ $&nbsp; führt bezüglich des Eingangs zur Informationssequenz&nbsp; $\underline{u} = (1, 0, 0, 0, 0, \ \text{...}\hspace{0.05cm})$&nbsp;
 
*und bezüglich des jeweils zweiten Codesymbols zur Paritysequenz &nbsp; $\underline{p} = (1, 0, 1, 0, 0, \ \text{...}\hspace{0.05cm})$&nbsp; &nbsp; &#8658; &nbsp;  identisch mit der Impulsantwort&nbsp; $\underline{g}$ &nbsp; &#8658; &nbsp; Gedächtnis $m = 2$.<br>
 
  
  
[[File:P ID3044 KC T 4 3 S2a v2.png|center|frame|Nichtrekursiver systematischer Turbocode und Zustandsübergangsdiagramm|class=fit]]
+
Indeed,&nbsp; for any RSC code &nbsp; &#8658; &nbsp; infinite impulse response&nbsp; $\underline{g}$ &nbsp; &#8658; &nbsp; fractional&ndash;rational transfer function&nbsp; $G(D)$:&nbsp; There are certain input sequences&nbsp; $\underline{u}$,&nbsp; which lead to very short parity sequences &nbsp; $\underline{p} = \underline{u} &#8727; \underline{g}$ &nbsp; with low Hamming weight&nbsp; $w_{\rm H}(\underline{p})$.
  
Die untere Grafik gilt für einen so genannten&nbsp; '''RSC&ndash;Code'''&nbsp; (<i>Recursive Systematic Convolutional</i>) entsprechend&nbsp; $\mathbf{G}(D) = \big [1, \ (1+ D^2)/(1 + D + D^2)\big ]$.
+
For example,&nbsp; such a sequence is given in the second graphic of the&nbsp; [[Channel_Coding/The_Basics_of_Turbo_Codes#First_requirement_for_turbo_codes:_Recursive_component_codes| "last section"]]&nbsp;: &nbsp;  
*Hier führt die Folge&nbsp; $S_0 &#8594; S_1 &#8594; S_3 &#8594; S_2 &#8594; S_1 &#8594; S_3 &#8594; S_2 &#8594; \ \text{...}\hspace{0.05cm} \ $&nbsp; zur Impulsantwort&nbsp; $\underline{g} = (1, 1, 1, 0, 1, 1, 0, 1, 1, \ \text{...}\hspace{0.05cm})$.  
+
:$$\underline{u} = (1, 1, 1, 0, 0, \ \text{...}\hspace{0.05cm}).$$  
*Diese Impulsantwort setzt sich aufgrund der Schleife&nbsp; $S_1 &#8594; S_3 &#8594; S_2 &#8594; S_1$ bis ins Unendliche fort. Dies ermöglicht bzw. erleichtert die iterative Decodierung.<br>
+
Then for the output sequence holds:
  
 +
::<math>P(D) = U(D) \cdot G(D) = (1+D+D^2) \cdot \frac{1+D^2}{1+D+D^2}= 1+D^2\hspace{0.3cm}\Rightarrow\hspace{0.3cm}  \underline{p}= (\hspace{0.05cm}1\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} \text{...}\hspace{0.05cm}\hspace{0.05cm})\hspace{0.05cm}. </math>
  
[[File:P ID3045 KC T 4 3 S2b v2.png|center|frame|Nichtrekursiver systematischer Turbocode und Zustandsübergangsdiagramm|class=fit]]
+
{{BlaueBox|TEXT= 
 +
$\text{Meaning and purpose:}$&nbsp;
 +
By&nbsp; $\rm interleaving$,&nbsp; it is now ensured with high probability that this sequence&nbsp; $\underline{u} = (1, 1, 1, 0, 0, \ \text{...}\hspace{0.05cm})$&nbsp; is converted into a sequence&nbsp; $\underline{u}_{\pi}$&nbsp;
 +
*which also contains only three&nbsp; "ones",<br>
  
Mehr Details zu den Beispielen auf dieser Seite finden Sie in der&nbsp; [[Aufgaben:Aufgabe_4.08Z:_Grundlegendes_zum_Interleaving|Aufgabe 4.8]]&nbsp; und der&nbsp; [[Aufgaben:Aufgabe_4.09:_Recursive_Systematic_Convolutional_Codes|Aufgabe 4.9]].<br>
+
*but whose output sequence is characterized by a large Hamming weight&nbsp; $w_{\rm H}(\underline{p})$.<br>
  
== Zweite Voraussetzung für Turbocodes: Interleaving ==
 
<br>
 
Es ist offensichtlich, dass bei&nbsp; $G_1(D) = G_2(D)$&nbsp; ein Interleaver&nbsp; $(\Pi)$&nbsp; unerlässlich ist. Ein weiterer Grund ist, dass die Apriori&ndash;Information als unabhängig vorausgesetzt wird. Somit sollten benachbarte (und somit möglicherweise stark abhängige) Bits für den jeweils anderen Teilcode weit auseinander liegen.<br>
 
  
Für jeden RSC&ndash;Code &nbsp; &#8658; &nbsp; unendliche Impulsantwort&nbsp; $\underline{g}$ &nbsp; &#8658; &nbsp; gebrochen&ndash;rationale Übertragungsfunktion&nbsp; $G(D)$&nbsp; gibt es nämlich gewisse Eingangssequenzen&nbsp; $\underline{u}$, die zu sehr kurzen Paritysequenzen&nbsp; $\underline{p} = \underline{u} &#8727; \underline{g}$&nbsp; mit geringem Hamming&ndash;Gewicht&nbsp; $w_{\rm H}(\underline{p})$&nbsp; führen. 
+
Thus,&nbsp; the decoder succeeds in resolving such&nbsp; "problem sequences"&nbsp; iteratively.}}<br>
  
Eine solche Sequenz ist beispielsweise in der unteren Grafik auf der&nbsp; [[Channel_Coding/Grundlegendes_zu_den_Turbocodes#Erste_Voraussetzung_f.C3.BCr_Turbocodes:_Rekursive_Komponentencodes| letzten Seite]]&nbsp; angegeben: &nbsp; $\underline{u} = (1, 1, 1, 0, 0, \ \text{...}\hspace{0.05cm})$. Dann gilt für die Ausgangssequenz:
+
For the following description of the interleavers we use the assignment&nbsp; $I_{\rm In} &#8594; I_{\rm Out}$.&nbsp; These labels stand for the indices of output and input sequence,&nbsp; respectively. The interleaver variable is named&nbsp; $I_{\rm max}$&nbsp;.<br>
 +
[[File:P ID3048 KC T 4 3 S3a v5.png|right|frame|Clarification of block interleaving]]
  
::<math>P(D) = U(D) \cdot G(D) = (1+D+D^2) \cdot \frac{1+D^2}{1+D+D^2}= 1+D^2\hspace{0.3cm}\Rightarrow\hspace{0.3cm}  \underline{p}= (\hspace{0.05cm}1\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} \text{...}\hspace{0.05cm}\hspace{0.05cm})\hspace{0.05cm}. </math>
+
There are several,&nbsp; fundamentally different interleaver concepts:<br>
  
{{BlaueBox|TEXT= 
+
&rArr; &nbsp; In a&nbsp; &raquo;<b>block interleaver</b>&laquo;&nbsp; one fills a matrix with&nbsp; $N_{\rm C}$&nbsp; columns and&nbsp; $N_{\rm R}$&nbsp; rows column-by-column and reads the matrix row-by-row.&nbsp; Thus an information block with&nbsp; $I_{\rm max} = N_{\rm C} \cdot N_{\rm R}$&nbsp; bit is deterministically scrambled.<br>
$\text{Sinn und Zweck:}$&nbsp;  
 
Durch&nbsp; $\rm Interleaving$&nbsp; (deutsch: &nbsp; <i>Verwürfelung</i>) wird nun mit großer Wahrscheinlichkeit sichergestellt, dass diese Sequenz&nbsp; $\underline{u} = (1, 1, 1, 0, 0, \ \text{...}\hspace{0.05cm})$&nbsp; in eine Sequenz&nbsp; $\underline{u}_{\pi}$&nbsp; gewandelt wird,
 
*die zwar ebenfalls nur drei Einsen beinhaltet,<br>
 
  
*deren Ausgangssequenz aber durch ein großes Hamming&ndash;Gewicht&nbsp; $w_{\rm H}(\underline{p})$&nbsp; gekennzeichnet ist.<br>
+
The right graph illustrates the principle for&nbsp; $I_{\rm max} = 64$ &nbsp; &#8658; &nbsp; $1 &#8804; I_{\rm In} \le 64$&nbsp; and&nbsp; $1 &#8804; I_{\rm Out} \le 64$.&nbsp; The order of the output bits is then: &nbsp;  
 +
:$$1, 9, 17, 25, 33, 41, 49, 57, 2, 10, 18, \ \text{...}\hspace{0.05cm} , 48, 56, 64.$$
  
Somit gelingt es dem Decoder, solche &bdquo;Problemsequenzen&rdquo; iterativ aufzulösen.}}<br>
+
More information on block interleaving is available in the&nbsp; [[Aufgaben:Exercise_4.08Z:_Basics_about_Interleaving|"Exercise 4.8Z"]].<br>
  
[[File:P ID3048 KC T 4 3 S3a v5.png|right|frame|Zur Verdeutlichung von Block–Interleaving]]
+
[[File:P ID3050 KC T 4 3 S3b v5.png|left|frame|Clarification of $S$&ndash;random interleaving]]
 +
<br><br><br><br>
 +
&rArr; &nbsp; Turbo codes often use the&nbsp; &raquo;'''$S$&ndash;random interleaver'''&laquo;.  This pseudo random algorithm with the parameter&nbsp; "$S$"&nbsp; guarantees
 +
*that two indices less than&nbsp; $S$&nbsp; apart at the input
  
Zur folgenden Beschreibung der Interleaver verwendet wir die Zuordnung&nbsp; $I_{\rm In} &#8594; I_{\rm Out}$. Diese Bezeichnungen stehen für die Indizes von Ausgangs&ndash; bzw. Eingangsfolge. Die Interleavergröße wird mit&nbsp; $I_{\rm max}$&nbsp; benannt.<br>
+
*occur at least at the distance&nbsp; $S + 1$&nbsp; at the output.  
  
Es gibt mehrere, grundsätzlich verschiedene Interleaver&ndash;Konzepte:<br>
 
  
Bei einem&nbsp; <b>Block&ndash;Interleaver</b>&nbsp; füllt man eine Matrix mit&nbsp; $S$&nbsp; Spalten und&nbsp; $Z$&nbsp; Zeilen spaltenweise und liest die Matrix zeilenweise aus. Damit wird ein Informationsblock mit&nbsp; $I_{\rm max} = S \cdot Z$&nbsp; Bit deterministisch verwürfelt.<br>
+
The left graph shows the&nbsp; $S$&ndash;random characteristic&nbsp; $I_{\rm Out}(I_{\rm In})$&nbsp; for&nbsp; $I_{\rm max} = 640$.<br>
  
Die rechte Grafik verdeutlicht das Prinzip für&nbsp; $I_{\rm max} = 64$ &nbsp; &#8658; &nbsp; $1 &#8804; I_{\rm In} \le 64$&nbsp; und&nbsp; $1 &#8804; I_{\rm Out} \le 64$. Die Reihenfolge der Ausgangsbits lautet dann: &nbsp; $1, 9, 17, 25, 33, 41, 49, 57, 2, 10, 18, \ \text{...}\hspace{0.05cm} , 48, 56, 64.$
+
#This algorithm is also deterministic,&nbsp; and one can undo the scrambling in the decoder &nbsp; &#8658; &nbsp; "De&ndash;interleaving".
 +
#The distribution still seems&nbsp; "more random"&nbsp; than with block interleaving.
 +
<br clear=all>
  
Mehr Informationen zum Block&ndash;Interleaving gibt es in der&nbsp; [[Aufgaben:Aufgabe_4.08Z:_Grundlegendes_zum_Interleaving| Aufgabe 4.8Z]].<br>
+
== Bit-wise iterative decoding of a turbo code ==
 
+
<br>
[[File:P ID3050 KC T 4 3 S3b v5.png|left|frame|Zur Verdeutlichung von&nbsp; $S$–Random–Interleaving]]
+
The decoding of a turbo code is basically done as described in section&nbsp; [[Channel_Coding/Soft-in_Soft-Out_Decoder#Bit-wise_soft-in_soft-out_decoding|"Bit-wise Soft&ndash;in Soft&ndash;out Decoding"]].&nbsp; From the following graphic,&nbsp; however,&nbsp; you can also see some special features that apply only to the turbo decoder.<br>
<br><br><br><br>Turbocodes verwenden oft den&nbsp; '''$S$&ndash;Random&ndash;Interleaver'''. Dieser pseudozufällige Algorithmus mit dem Parameter &bdquo;$S$&rdquo; garantiert, dass zwei am Eingang weniger als&nbsp; $S$&nbsp; auseinander liegende Indizes am Ausgang mindestens im Abstand&nbsp; $S + 1$&nbsp; auftreten. Die linke Grafik zeigt die&nbsp; $S$&ndash;Random&ndash;Kennlinie&nbsp; $I_{\rm Out}(I_{\rm In})$&nbsp; für&nbsp; $I_{\rm max} = 640$.<br>
 
  
*Auch dieser Algorithmus ist deterministisch, und man kann die Verwürfelung im Decoder rückgängig machen &#8658; <i>De&ndash;Interleaving</i>.  
+
Assumed is a rate&nbsp; $1/3$&nbsp; turbo code according to the description in the&nbsp; [[Channel_Coding/The_Basics_of_Turbo_Codes#Basic_structure_of_a_turbo_code| "first section of this page"]].&nbsp; Also,&nbsp; the color scheme for the information sequence&nbsp; $\underline{u}$&nbsp; and the two parity sequences&nbsp; $\underline{p}_1$&nbsp; and&nbsp; $\underline{p}_2$&nbsp; are adapted from the earlier graphics.&nbsp; Further,&nbsp; it should be noted:
*Die Verteilung wirkt trotzdem &bdquo;zufälliger&rdquo; als bei Block&ndash;Interleaving.
+
[[File:EN_KC_T_4_3_S4a_v2.png|right|frame|Iterative turbo decoder for rate&nbsp; $R = 1/3$ |class=fit]]
<br clear=all>
 
  
== Symbolweise iterative Decodierung eines Turbocodes ==
+
*The received vectors&nbsp; $\underline{y}_0,\hspace{0.15cm} \underline{y}_1,\hspace{0.15cm} \underline{y}_2$&nbsp; are real-valued and provide the respective soft information with respect to the information sequence&nbsp; $\underline{u}$&nbsp; and the sequences&nbsp; $\underline{p}_1$&nbsp; $($parity for encoder 1$)$&nbsp; and&nbsp; $\underline{p}_2$&nbsp; $($parity for encoder 2$)$.<br>
<br>
 
Die Decodierung eines Turbocodes geschieht grundsätzlich wie im Abschnitt&nbsp; [[Channel_Coding/Soft–in_Soft–out_Decoder#Symbolweise_Soft.E2.80.93in_Soft.E2.80.93out_Decodierung|Symbolweise Soft&ndash;in Soft&ndash;out_Decodierung]]&nbsp; beschrieben. Aus der folgenden Grafik erkennt man aber auch einige nur für den Turbodecoder zutreffende Besonderheiten.<br>
 
  
[[File:P ID3049 KC T 4 3 S4a v2.png|center|frame|Iterativer Turbodecoder für die Rate&nbsp; $R = 1/3$|class=fit]]
+
*The decoder 1 receives the required intrinsic information in the form of the&nbsp; $L$ values $L_{\rm K,\hspace{0.03cm} 0}$&nbsp; $($out&nbsp; $\underline{y}_0)$&nbsp; and&nbsp; $L_{\rm K,\hspace{0.03cm}1}$ $($out&nbsp; $\underline{y}_1)$&nbsp; over each bit of the sequences&nbsp; $\underline{u}$&nbsp; and&nbsp; $\underline{p}_1$.
  
Vorausgesetzt ist ein Rate&ndash;$1/3$&ndash;Turbocode entsprechend der Beschreibung auf der&nbsp; [[Channel_Coding/Grundlegendes_zu_den_Turbocodes#Grundstruktur_eines_Turbocodes| ersten Seite  dieses Abschnitts]]. Auch die Farbgebung für die Informationssequenz&nbsp; $\underline{u}$&nbsp; und die beiden Paritysequenzen&nbsp; $\underline{p}_1$&nbsp; und&nbsp; $\underline{p}_2$&nbsp; sind an die früheren Grafiken angepasst. Weiter ist zu bemerken:
+
*In the second decoder,&nbsp; the scrambling of the information sequence&nbsp; $\underline{u}$&nbsp; must be taken into account.&nbsp; Thus,&nbsp; the&nbsp; $L$ values to be processed are&nbsp; $\pi(L_{\rm K, \hspace{0.03cm}0})$&nbsp; and&nbsp; $L_{\rm K, \hspace{0.03cm}2}$.<br>
*Die Empfangsvektoren&nbsp; $\underline{y}_0, \underline{y}_1$&nbsp; und&nbsp; $\underline{y}_2$&nbsp; sind reellwertig und liefern die jeweilige Soft&ndash;Information bezüglich der Informationssequenz&nbsp; $\underline{u}$&nbsp; sowie der Sequenzen&nbsp; $\underline{p}_1$&nbsp; (Parity für Coder 1) und&nbsp; $\underline{p}_2$&nbsp; (Parity für Coder 2).<br>
 
  
*Der Decoder 1 erhält die erforderliche intrinsische Information in Form der&nbsp; $L$&ndash;Werte $L_{\rm K,\hspace{0.03cm} 0}$&nbsp; $($aus&nbsp; $\underline{y}_0)$&nbsp; und&nbsp; $L_{\rm K,\hspace{0.03cm}1}$ $($aus&nbsp; $\underline{y}_1)$&nbsp; über jedes einzelne Bit der Sequenzen&nbsp; $\underline{u}$&nbsp; und&nbsp; $\underline{p}_1$. Beim zweiten Decoder ist die Verwürfelung der Informationssequenz&nbsp; $\underline{u}$&nbsp; zu berücksichtigen. Die zu verarbeitenden&nbsp; $L$&ndash;Werte sind somit&nbsp; $\pi(L_{\rm K, \hspace{0.03cm}0})$&nbsp; und&nbsp; $L_{\rm K, \hspace{0.03cm}2}$.<br>
+
*In the general&nbsp; [[Channel_Coding/Soft-in_Soft-Out_Decoder#Basic_structure_of_concatenated_coding_systems|$\text{SISO decoder}$]]&nbsp; the information exchange between the two component decoders was controlled with&nbsp; $\underline{L}_{\rm A, \hspace{0.03cm}2} = \underline{L}_{\rm E, \hspace{0.03cm}1}$&nbsp; and&nbsp; $\underline{L}_{\rm A, \hspace{0.03cm}1} = \underline{L}_{\rm E, \hspace{0.03cm}2}$.  
  
*Beim allgemeinen&nbsp; [[Channel_Coding/Soft–in_Soft–out_Decoder#Grundstruktur_von_verketteten_Codiersystemen| SISO&ndash;Decoder]]&nbsp; wurde der Informationsaustausch zwischen den beiden Komponentendecodern mit&nbsp; $\underline{L}_{\rm A, \hspace{0.03cm}2} = \underline{L}_{\rm E, \hspace{0.03cm}1}$&nbsp; sowie&nbsp; $\underline{L}_{\rm A, \hspace{0.03cm}1} = \underline{L}_{\rm E, \hspace{0.03cm}2}$&nbsp; angegeben. Ausgeschrieben auf Bitebene bedeuten diese Gleichungen mit&nbsp; $1 &#8804; i &#8804; n$:
+
*Written out at the bit level,&nbsp; these equations are denoted by&nbsp; $(1 &#8804; i &#8804; n)$:
  
 
::<math>L_{\rm A, \hspace{0.03cm}2}(i) = L_{\rm E, \hspace{0.03cm}1}(i)  
 
::<math>L_{\rm A, \hspace{0.03cm}2}(i) = L_{\rm E, \hspace{0.03cm}1}(i)  
\hspace{0.5cm}{\rm bzw.}\hspace{0.5cm}
+
\hspace{0.5cm}{\rm resp.}\hspace{0.5cm}
 
L_{\rm A, \hspace{0.03cm}1}(i) = L_{\rm E, \hspace{0.03cm}2}(i) \hspace{0.03cm}.</math>
 
L_{\rm A, \hspace{0.03cm}1}(i) = L_{\rm E, \hspace{0.03cm}2}(i) \hspace{0.03cm}.</math>
  
*Beim Turbodecoder muss bei diesem Informationsaustausch auch der Interleaver berücksichtigt werden. Dann gilt für&nbsp; $i = 1, \ \text{...}\hspace{0.05cm} , \ n$:
+
*In the case of the turbo decoder,&nbsp; the interleaver must also be taken into account in this information exchange.&nbsp; Then for&nbsp; $i = 1, \ \text{...}\hspace{0.05cm} , \ n$:
  
 
::<math>L_{\rm A, \hspace{0.03cm}2}\left ({\rm \pi}(i) \right ) = L_{\rm E, \hspace{0.03cm}1}(i)  
 
::<math>L_{\rm A, \hspace{0.03cm}2}\left ({\rm \pi}(i) \right ) = L_{\rm E, \hspace{0.03cm}1}(i)  
Line 137: Line 152:
 
L_{\rm A, \hspace{0.03cm}1}(i) = L_{\rm E, \hspace{0.03cm}2}\left ({\rm \pi}(i) \right ) \hspace{0.05cm}.</math>
 
L_{\rm A, \hspace{0.03cm}1}(i) = L_{\rm E, \hspace{0.03cm}2}\left ({\rm \pi}(i) \right ) \hspace{0.05cm}.</math>
  
*Der Aposteriori&ndash;$L$&ndash;Wert wird in obigem Modell (willkürlich) vom Decoder 1 abgegeben. Dies lässt sich damit begründen, dass eine Iteration für einen zweifachen Informationsaustausch steht.<br>
+
*The a-posteriori&nbsp; $L$ value is&nbsp; $($arbitrarily$)$&nbsp; given by decoder 1 in the above model.&nbsp; This can be justified by the fact that one iteration stands for a twofold information exchange.<br>
  
== Leistungsfähigkeit der Turbocodes ==
+
== Performance of the turbo codes ==
 
<br>
 
<br>
[[File:P ID3053 KC T 4 3 S5a v7.png|right|frame|Bit– und Blockfehlerwahrscheinlichkeit von Turbocodes beim AWGN&ndash;Kanal]]
+
[[File:EN_KC_T_4_3_S5b_v2.png|right|frame|Bit and block error probability of turbo codes at AWGN channel]]
Wir betrachten wie auf den letzten Seiten den Rate&ndash;$1/$3&ndash;Turbocode
+
We consider,&nbsp; as in the last sections,&nbsp; the rate&nbsp; $1/3$&nbsp; turbo code
 
 
*mit gleichen Filterfunktionen&nbsp; $G_1(D) = G_2(D) = (1 + D^2)/(1 + D + D^2)$ &nbsp; &#8658; &nbsp; Gedächtnis&nbsp; $m = 2$,<br>
 
 
 
*mit der Interleavergröße&nbsp; $K$; zunächst gelte&nbsp; $K = 10000,$<br>
 
 
 
*eine ausreichende Anzahl an Iterationen&nbsp; $(I = 20)$, vom Ergebnis her nahezu gleichzusetzen mit &bdquo;$I &#8594; &#8734;$&rdquo;.<br><br>
 
 
 
Die beiden RSC&ndash;Komponentencodes sind jeweils auf&nbsp; $K$&nbsp; Bit terminiert. Deshalb gruppieren wir
 
 
 
* die Informationssequenz&nbsp; $\underline{u}$&nbsp; zu Blöcken mit je&nbsp; $K$&nbsp; Informationsbits, und<br>
 
 
 
*die Codesequenz $\underline{x}$ zu Blöcken mit je&nbsp; $N = 3 \cdot K$&nbsp; Codebits.<br><br>
 
  
Alle Ergebnisse gelten für den&nbsp; [[Channel_Coding/Signal_classification#AWGN.E2.80.93Kanal_bei_bin.C3.A4rem_Eingang| AWGN&ndash;Kanal]]. Die Daten entstammen dem Vorlesungsskript&nbsp; [Liv15]<ref>Liva, G.: ''Channels Codes for Iterative Decoding.'' Vorlesungsmanuskript, Lehrstuhl für Nachrichtentechnik, TU München und DLR Oberpfaffenhofen, 2015.</ref>
+
*with equal filter functions &nbsp; $G_1(D) = G_2(D) = (1 + D^2)/(1 + D + D^2)$ &nbsp; &#8658; &nbsp; memory&nbsp; $m = 2$,<br>
  
Die Grafik zeigt als grüne Kurve die&nbsp; '''Blockfehlerwahrscheinlichkeit''' &nbsp; &#8658; &nbsp; ${\rm Pr(Blockfehler)}$&nbsp; in doppelt&ndash;logarithmischer Darstellung  in Abhängigkeit der AWGN&ndash;Kenngröße&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0)$. Man erkennt:
+
*with the interleaver size&nbsp; $K$; first apply&nbsp; $K = 10000,$&nbsp; and<br>
  
*Die mit Kreuzen markierten Punkte ergaben sich aus den Gewichtsfunktionen des Turbocodes mit Hilfe der&nbsp; [[Channel_Coding/Schranken_f%C3%BCr_die_Blockfehlerwahrscheinlichkeit#Union_Bound_der_Blockfehlerwahrscheinlichkeit| &bdquo;Union Bound&rdquo;]]. Die Simulationsergebnisse &ndash; in der Grafik durch Kreise markiert &ndash; sind nahezu deckungsgleich mit den analytisch berechneten Werten.<br>
+
*a sufficient large number of iterations&nbsp; $(I = 20)$,&nbsp; almost equivalent in result&nbsp; to "$I &#8594; &#8734;$".<br><br>
  
*Die &bdquo;Union Bound&rdquo; ist nur eine obere Schranke, basierend auf Maximum&ndash;Likelihood&ndash;Decodierung (&bdquo;ML&rdquo;). Der iterative Decoder ist suboptimal (also schlecher als &bdquo;ML&rdquo;). Diese beiden Effekte heben sich scheinbar nahezu auf.<br>
+
The two RSC component codes are each terminated on&nbsp; $K$&nbsp; bits.&nbsp; Therefore we group
  
*Etwa bei&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) = 1 \ \rm dB$&nbsp; ist ein Knick im (grünen) Kurvenverlauf festzustellen, der mit der Steigungsänderung von&nbsp; ${\rm Pr(Bitfehler)}$ &nbsp; &#8658; &nbsp; blaue Kurve korrespondiert. <br><br>
+
* the information sequence&nbsp; $\underline{u}$&nbsp; into blocks of&nbsp; $K$&nbsp; information bits each,&nbsp; and<br>
  
Die blauen Kreuze (Berechnung) und die blauen Kreise (Simulation) bezeichnen die&nbsp; '''Bitfehlerwahrscheinlichkeit'''&nbsp; für die Interleavergröße&nbsp; $K = 10000$ . Als Vergleichskurve eingezeichnet ist die (strichpunktierte) Kurve für uncodierte Übertragung. Zu diesen blauen Kurven ist anzumerken:
+
*the encoded sequence&nbsp; $\underline{x}$&nbsp; to blocks with each&nbsp; $N = 3 \cdot K$&nbsp; encoded bits.<br><br>
*Bei kleinen Abszissenwerten ist der Kurvenabfall in der gewählten Darstellung nahezu linear und ausreichend steil. Zum Beispiel benötigt man für&nbsp; ${\rm Pr(Bitfehler)} = 10^{-5}$&nbsp; mindestens&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx \, 0.8 \ \rm dB$.<br>
 
  
*Im Vergleich zur&nbsp; [[Channel_Coding/Informationstheoretische_Grenzen_der_Kanalcodierung#Kanalkapazit.C3.A4t_des_AWGN.E2.80.93Modells_.282.29| Shannon&ndash;Grenze]], die sich für die Coderate&nbsp; $R = 1/3$&nbsp; zu&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx \, &ndash;0.55 \ \rm dB$&nbsp; ergibt, liegt unser Standard&ndash;Turbocode&nbsp; $($mit Gedächtnis&nbsp; $m = 2)$&nbsp; nur etwa&nbsp; $1.35 \ \rm dB$&nbsp; entfernt.<br>
+
All results apply to the&nbsp; [[Channel_Coding/Channel_Models_and_Decision_Structures#AWGN_channel_at_binary_input|$\text{AWGN channel}$]]. Data are taken from the lecture notes&nbsp; [Liv15]<ref>Liva, G.:&nbsp; Channels Codes for Iterative Decoding.&nbsp; Lecture notes, Department of Communications Engineering, TU Munich and DLR Oberpfaffenhofen, 2015.</ref>
  
*Ab&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx 0.5 \ \rm dB$&nbsp; verläuft die Kurve flacher. Ab ca.&nbsp; $1.5 \ \rm dB$&nbsp; ist der Verlauf wieder (nahezu) linear mit geringerer Steigung. Für&nbsp; ${\rm Pr(Bitfehler)} = 10^{-7}$&nbsp; benötigt man etwa&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) = 3 \ \rm dB$.<br><br>
+
The graph shows as a green curve the&nbsp; &raquo;'''block error probability'''&laquo; &nbsp; &#8658; &nbsp; ${\rm Pr(block\:error)}$&nbsp; in double logarithmic representation depending on the AWGN characteristic&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0)$.&nbsp; It can be seen:
  
Wir versuchen nun, den flacheren Abfall der Bitfehlerwahrscheinlichkeit bei größerem&nbsp; $E_{\rm B}/N_0$&nbsp; zu erklären. Man spricht von einem&nbsp; $\text{Error Floor}$:
+
# The points marked with crosses resulted from the weight functions of the turbo code using the&nbsp; [[Channel_Coding/Bounds_for_Block_Error_Probability#Union_Bound_of_the_block_error_probability|$\text{Union Bound}$]].&nbsp; The simulation results &nbsp; &ndash;  in the graph marked by circles &ndash; &nbsp; are almost congruent with the analytically calculated values.<br>
 +
# The&nbsp; "Union Bound"&nbsp; is only an upper bound based on maximum likelihood decoding&nbsp; $\rm (ML)$.&nbsp; The iterative decoder is suboptimal&nbsp; (i.e.,&nbsp; worse than "maximum likelihood").&nbsp; These two effects seem to almost cancel each other out.<br>
 +
# At&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) = 1 \ \rm dB$&nbsp; there is a kink in the&nbsp; (green)&nbsp; curve,&nbsp; corresponding to the slope change of&nbsp; ${\rm Pr(bit\:error)}$ &nbsp; &#8658; &nbsp; blue curve. <br><br>
  
*Der Grund für dieses asymptotisch schlechtere Verhalten bei besserem Kanal&nbsp; $($im Beispiel: &nbsp; ab&nbsp; $10 \cdot {\rm lg} \, E_{\rm B}/N_0 \approx 2 \ \rm dB)$&nbsp; ist die Periode&nbsp; $P$&nbsp; der Coderimpulsantwort&nbsp; $\underline{g}$, wie auf der Seite&nbsp; [[Channel_Coding/Grundlegendes_zu_den_Turbocodes#Zweite_Voraussetzung_f.C3.BCr_Turbocodes:_Interleaving|Interleaving]]&nbsp; nachgewiesen und in der&nbsp; [[Aufgaben:Aufgabe_4.10:_Turbocoder_für_UMTS_und_LTE|Aufgabe 4.10]]&nbsp; an Beispielen erläutert. <br>
+
The blue crosses&nbsp; $($"calculation"$)$&nbsp; and the blue circles&nbsp; $($"simulation"$)$&nbsp; denote the&nbsp; &raquo;'''bit error probability'''&laquo;&nbsp; for the interleaver size&nbsp; $K = 10000$.&nbsp; The&nbsp; (dash-dotted)&nbsp; curve for uncoded transmission is drawn as a comparison curve.&nbsp;  
  
*Für&nbsp; $m = 2$&nbsp; ist die Periode $P = 2^m -1 = 3$. Dadurch ist für $\underline{u} = (1, 1, 1) &#8658; w_{\rm H}(\underline{u}) = 3$&nbsp; trotz unbegrenzter Impulsantwort&nbsp; $\underline{g}$&nbsp; die Paritysequenz begrenzt: &nbsp; $\underline{p} = (1, 0, 1)$ &nbsp; &#8658; &nbsp; $w_{\rm H}(\underline{p}) = 2$.<br>
+
&rArr; &nbsp; To these blue curves is to be noted:
 +
*For small abscissa values,&nbsp; the curve slope in the selected plot is nearly linear and sufficiently steep.&nbsp; For example,&nbsp; for &nbsp; ${\rm Pr(bit\, error)} = 10^{-5}$ &nbsp; one needs at least &nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx \, 0.8 \ \rm dB$.<br>
  
*Die Sequenz&nbsp; $\underline{u} = (0, \ \text{...}\hspace{0.05cm} , \ 0, \ 1, \ 0, \ 0, \ 1, \ 0, \ \text{...}\hspace{0.05cm})$ &nbsp; &#8658; &nbsp; $U(D) = D^x \cdot (1 + D^P)$&nbsp; führt ebenfalls zu einem kleinen Hamming&ndash;Gewicht&nbsp; $w_{\rm H}(\underline{p})$&nbsp; am Ausgang, was den iterativen Decodierprozess erschwert.<br>
+
*Compared to the&nbsp; [[Channel_Coding/Information_Theoretical_Limits_of_Channel_Coding#Channel_capacity_of_the_AWGN_model| $\text{Shannon bound}$]],&nbsp; which results for code rate&nbsp; $R = 1/3$&nbsp; to&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx \, &ndash;0.55 \ \rm dB$,&nbsp; our standard turbo code &nbsp; $($with memory&nbsp; $m = 2)$ &nbsp; is only about&nbsp; $1.35 \ \rm dB$&nbsp; away.<br>
  
*Eine gewisse Abhilfe schafft der Interleaver, der dafür sorgt, dass nicht die beiden Sequenzen&nbsp; $\underline{p}_1$&nbsp; und&nbsp; $\underline{p}_2$&nbsp; gleichzeitig durch sehr kleine Hamming&ndash;Gewichte&nbsp; $w_{\rm H}(\underline{p}_1)$&nbsp; und&nbsp; $w_{\rm H}(\underline{p}_2)$&nbsp; belastet sind.<br>
+
*From&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx 0.5 \ \rm dB$&nbsp; the curve runs flatter.&nbsp; From&nbsp; $\approx 1.5 \ \rm dB$&nbsp; the curve is again&nbsp; (nearly)&nbsp; linear with lower slope.&nbsp; For&nbsp; ${\rm Pr(bit\:error)} = 10^{-7}$&nbsp; one needs about&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) = 3 \ \rm dB$.<br><br>
  
*Aus der Grafik erkennt man auch, dass die Bitfehlerwahrscheinlichkeit umgekehrt proportional zur Interleavergröße&nbsp; $K$&nbsp; ist. Das heißt: &nbsp; Bei großem&nbsp; $K$&nbsp; funktioniert die Entspreizung ungünstiger Eingangssequenzen besser.<br>
+
&rArr; &nbsp; We now try to explain the flatter drop in bit error probability with larger&nbsp; $E_{\rm B}/N_0$.&nbsp; This is called an&nbsp; &raquo;$\text{error floor}$&laquo;:
  
*Allerdings gilt die Näherung&nbsp; $K \cdot {\rm Pr(Bitfehler) = const.}$&nbsp; nur für größere&nbsp; $E_{\rm B}/N_0$&ndash;Werte &nbsp; &#8658; &nbsp; kleine Bitfehlerwahrscheinlichkeiten. Der  beschriebene Effekt tritt zwar auch bei kleinerem&nbsp; $E_{\rm B}/N_0$&nbsp; auf, doch sind dann die Auswirkungen auf die Bitfehlerwahrscheinlichkeit geringer.<br><br>
+
#The reason for this asymptotically worse behavior with better channel&nbsp; $($in the example: &nbsp; from&nbsp; $10 \cdot {\rm lg} \, E_{\rm B}/N_0 \ge 2 \ \rm dB)$&nbsp; is the period&nbsp; $P$&nbsp; of the encoder impulse response&nbsp; $\underline{g}$,&nbsp; as demonstrated in the section&nbsp; [[Channel_Coding/The_Basics_of_Turbo_Codes#Second_requirement_for_turbo_codes: _Interleaving|$\rm Interleaving$]],&nbsp;  and explained  in the&nbsp; [[Aufgaben:Exercise_4.10:_Turbo_Encoder_for_UMTS_and_LTE|"Exercise 4.10"]]&nbsp; with examples. <br>
 +
#For&nbsp; $m = 2$,&nbsp; the period is&nbsp; $P = 2^m -1 = 3$.&nbsp; Thus,&nbsp; for&nbsp; $\underline{u} = (1, 1, 1) &nbsp; &#8658; &nbsp; w_{\rm H}(\underline{u}) = 3$&nbsp;  the parity sequence is bounded: &nbsp; $\underline{p} = (1, 0, 1)$ &nbsp; &#8658; &nbsp; $w_{\rm H}(\underline{p}) = 2$ &nbsp; despite unbounded impulse response&nbsp; $\underline{g}$.<br>
 +
#The sequence &nbsp; $\underline{u} = (0, \ \text{...}\hspace{0.05cm} , \ 0, \ 1, \ 0, \ 0, \ 1, \ 0, \ \text{...}\hspace{0.05cm})$ &nbsp; &#8658; &nbsp; $U(D) = D^x \cdot (1 + D^P)$&nbsp; also leads to a small Hamming weight&nbsp; $w_{\rm H}(\underline{p})$&nbsp; at the output,&nbsp; which complicates the iterative decoding process.<br>
 +
#Some workaround is provided by the interleaver,&nbsp; which ensures that both sequences&nbsp; $\underline{p}_1$&nbsp; and&nbsp; $\underline{p}_2$&nbsp; are not simultaneously loaded by very small Hamming weights&nbsp; $w_{\rm H}(\underline{p}_1)$&nbsp; and&nbsp; $w_{\rm H}(\underline{p}_2)$.<br>
 +
#From the graph you can also see that the bit error probability is inversely proportional to the interleaver size&nbsp; $K$.&nbsp; That means: &nbsp; With large&nbsp; $K$&nbsp; the despreading of unfavorable input sequences works better.<br>
 +
#However,&nbsp; the approximation &nbsp; $K \cdot {\rm Pr(bit\:error) = const.}$ &nbsp; is valid only for larger&nbsp; $E_{\rm B}/N_0$&nbsp; values &nbsp; &#8658; &nbsp; small bit error probabilities.&nbsp; The described effect also occurs for smaller&nbsp; $E_{\rm B}/N_0$&nbsp; values, but then the effects on the bit error probability are smaller.<br>
 +
#The flatter shape of the block error probability&nbsp; $($green curve$)$&nbsp; holds largely independent of the interleaver size&nbsp; $K$,&nbsp; i.e., for&nbsp; $K = 1000$&nbsp; as well as for&nbsp; $K = 10000$.&nbsp; In the range&nbsp; $10 \cdot {\rm lg} \, E_{\rm B}/N_0 > 2 \ \rm dB$&nbsp; namely single errors dominate,&nbsp; so that here the approximation&nbsp; ${\rm Pr(block\: error)} \approx {\rm Pr(bit\:error)} \cdot K$&nbsp; is valid.<br>
  
Dagegen gilt der flachere Verlauf der Blockfehlerwahrscheinlichkeit (grüne Kurve) weitgehend unabhängig von der Interleavergröße&nbsp; $K$, also sowohl für&nbsp; $K = 1000$&nbsp; als auch für&nbsp; $K = 10000$. Im Bereich ab&nbsp; $10 \cdot {\rm lg} \, E_{\rm B}/N_0 > 2 \ \rm dB$&nbsp; dominieren nämlich Einzelfehler, so dass hier die Näherung&nbsp; ${\rm Pr(Blockfehler)} \approx {\rm Pr(Bitfehler)} \cdot K$&nbsp; gültig ist.<br>
 
  
 
{{BlaueBox|TEXT=   
 
{{BlaueBox|TEXT=   
$\text{Fazit:}$&nbsp; Die beispielhaft gezeigten Kurven für Bitfehlerwahrscheinlichkeit und Blockfehlerwahrscheinlichkeit gelten qualitativ auch für&nbsp; $m > 2$, zum Beispiel für den Turbocode von UMTS und LTE&nbsp; $($jeweils mit&nbsp; $m = 3)$, der in der&nbsp; [[Aufgaben:Aufgabe_4.10:_Turbocoder_für_UMTS_und_LTE|Aufgabe 4.10]]&nbsp; analysiert wird. Es ergeben sich aber einige quantitative Unterschiede:
+
$\text{Conclusions:}$&nbsp; The exemplary shown bit error probability and block error probability curves also apply qualitatively for&nbsp; $m > 2$,&nbsp; e.g.&nbsp; for the  UMTS and LTE turbo codes&nbsp; $($each with&nbsp; $m = 3)$,&nbsp; which is analyzed in&nbsp; [[Aufgaben:Exercise_4.10:_Turbo_Encoder_for_UMTS_and_LTE|"Exercise 4.10"]].&nbsp; However,&nbsp; some quantitative differences emerge:
*Die Kurve verläuft bei  kleinem&nbsp; $E_{\rm B}/N_0$&nbsp; steiler und der Abstand von der Shannongrenze ist etwas geringer als im hier gezeigten Beispiel für&nbsp; $m = 2$.<br>
+
*The curve is steeper for small&nbsp; $E_{\rm B}/N_0$&nbsp; and the distance from the Shannon bound is slightly less than in the example shown here for&nbsp; $m = 2$.<br>
  
*Auch für größeres&nbsp; $m$&nbsp; gibt es einen <i>Error Floor</i>. Der Knick in den dargestellten Kurven erfolgt dann aber später, also bei kleineren Fehlerwahrscheinlichkeiten.}}<br>
+
*Also for larger&nbsp; $m$&nbsp; there is an&nbsp; "error floor".&nbsp; However,&nbsp; the kink in the displayed curves then occurs later,&nbsp; i.e. at smaller error probabilities.}}<br>
  
== Seriell verkettete Turbocodes – SCCC ==
+
== Serial concatenated turbo codes – SCCC ==
 
<br>
 
<br>
Die bisher betrachteten Turbocodes werden manchmal auch als&nbsp; <i>Parallel Concatenated Convolutional Codes</i>&nbsp; (PCCC) bezeichnet.  
+
The turbo codes considered so far are sometimes referred to as&nbsp; "parallel concatenated convolutional codes"&nbsp; $\rm (PCCC)$.  
  
Einige Jahre nach Berrou's Erfindung wurden von anderen Autoren auch&nbsp; <i>Serial Concatenated Convolutional Codes</i>&nbsp; (SCCC) entsprechend der folgenden Grafik vorgeschlagen.
+
Some years after Berrou's invention,&nbsp; "serial concatenated convolutional codes"&nbsp; $\rm (SCCC)$&nbsp; were also proposed by other authors according to the following diagram.
*Die Informationssequenz&nbsp; $\underline{u}$&nbsp; liegt am äußeren Faltungscoder&nbsp; $\mathcal{C}_1$&nbsp; an. Dessen Ausgangssequenz sei&nbsp; $\underline{c}$.<br>
+
[[File:EN_KC_T_4_3_S7a_v2.png|right|frame|Serial concatenated convolutional codes:&nbsp; Encoder and decoder<br><br><br><br><br> |class=fit]]
  
*Nach dem Interleaver&nbsp; $(\Pi)$&nbsp; folgt der innere Faltungscoder&nbsp; $\mathcal{C}_2$. Die Codesequenz wird hier&nbsp; $\underline{x}$&nbsp; genannt.<br>
+
*The information sequence&nbsp; $\underline{u}$&nbsp; is located at the outer convolutional encoder&nbsp; $\mathcal{C}_1$.&nbsp; Let its output sequence be&nbsp; $\underline{c}$. <br>
  
*Die resultierende Coderate ist&nbsp; $R = R_1 \cdot R_2$. Bei Rate&ndash;$1/2$&ndash;Komponentencodes ist&nbsp; $R = 1/4$.<br><br>
+
*After the interleaver&nbsp; $(\Pi)$&nbsp; follows the inner convolutional encoder&nbsp; $\mathcal{C}_2$.&nbsp; The encoded sequence is called&nbsp; $\underline{x}$&nbsp;.<br>
  
[[File:P ID3058 KC T 4 3 S7a v1.png|center|frame|<i>Serial Concatenated Convolutional Codes</i> (SCCC): Coder und Decoder |class=fit]]
+
*The resulting code rate is&nbsp; $R = R_1 \cdot R_2$.&nbsp; For rate&nbsp; $1/2$&nbsp; component codes: &nbsp; $R = 1/4$.<br><br>
  
Die untere Grafik zeigt den SCCC&ndash;Decoder und verdeutlicht die Verarbeitung der&nbsp; $L$&ndash;Werte und den Austausch der extrinsischen Information zwischen den beiden Komponentencoder:
+
The bottom diagram shows the SCCC decoder and illustrates the processing of&nbsp; $L$&ndash;values and the exchange of extrinsic information between the two component decoders:
*Der innere Decoder&nbsp; (für den Code&nbsp; $\mathcal{C}_2)$&nbsp; erhält vom Kanal die intrinsische Information&nbsp; $\underline{L}_{\rm K}(\underline{x})$&nbsp; und vom äußeren Decoder (nach Interleaving) die Apriori&ndash;Information&nbsp; $\underline{L}_{\rm A}(\underline{w})$&nbsp; mit&nbsp; $\underline{w} = \pi(\underline{c})$. An den äußeren Decoder wird die extrinsische Information&nbsp; $\underline{L}_{\rm E}(\underline{w})$&nbsp; abgegeben.<br>
+
*The inner decoder&nbsp; $($code&nbsp; $\mathcal{C}_2)$&nbsp; receives the intrinsic information&nbsp; $\underline{L}_{\rm K}(\underline{x})$&nbsp; from the channel and  a-priori information&nbsp; $\underline{L}_{\rm A}(\underline{w})$&nbsp; with&nbsp; $\underline{w} = \pi(\underline{c})$&nbsp; from the outer decoder&nbsp; $($after interleaving$)$&nbsp; and delivers the extrinsic information&nbsp; $\underline{L}_{\rm E}(\underline{w})$&nbsp; to the outer decoder.<br>
  
*Der äußere Decoder $($für&nbsp; $\mathcal{C}_1)$&nbsp; verarbeitet die Apriori&ndash;Information&nbsp; $\underline{L}_{\rm A}(\underline{c})$, also die extrinsische Information&nbsp; $\underline{L}_{\rm E}(\underline{w})$&nbsp; nach dem De&ndash;Interleaving. Er liefert die extrinsische Information&nbsp; $\underline{L}_{\rm E}(\underline{c})$.<br>
+
*The outer decoder&nbsp; $(\mathcal{C}_1)$&nbsp; processes the a-priori information&nbsp; $\underline{L}_{\rm A}(\underline{c})$, i.e. the extrinsic information&nbsp; $\underline{L}_{\rm E}(\underline{w})$&nbsp; after de&ndash;interleaving.&nbsp; It provides the extrinsic information&nbsp; $\underline{L}_{\rm E}(\underline{c})$.<br>
  
*Nach hinreichend vielen Iterationen ergibt sich das das gewünschte Decodierergebnis in Form der Aposteriori&ndash;$L$&ndash;Werte&nbsp; $\underline{L}_{\rm APP}(\underline{u})$&nbsp; der Informationssequenz&nbsp; $\underline{u}$.<br><br>
+
*After a sufficient number of iterations,&nbsp; the desired decoding result is obtained in the form of the a-posteriori&nbsp; $L$&ndash;values&nbsp; $\underline{L}_{\rm APP}(\underline{u})$&nbsp; of the information sequence&nbsp; $\underline{u}$.<br><br>
  
 
{{BlaueBox|TEXT=   
 
{{BlaueBox|TEXT=   
$\text{Fazit:}$&nbsp; Wichtig für <i>Serial Concatenated Convolutional Codes</i> (SCCC) ist, dass der innere Code&nbsp; $\mathcal{C}_2$&nbsp; rekursiv ist (also ein RSC&ndash;Code). Der äußere Code&nbsp; $\mathcal{C}_1$&nbsp; kann auch nichtrekursiv sein.  
+
$\text{Conclusions:}$&nbsp; Important for serial concatenated convolutional codes&nbsp;  $\rm (SCCC)$&nbsp;  is that the inner code&nbsp; $\mathcal{C}_2$&nbsp; is recursive&nbsp;  $($i.e. a RSC code$)$.&nbsp; The outer code&nbsp; $\mathcal{C}_1$&nbsp; may also be non-recursive.  
  
Zur Leistungsfähigkeit solcher Codes ist anzumerken:
+
Regarding the performance of such codes,&nbsp;  it should be noted:
*Ein SCCC ist bei großem&nbsp; $E_{\rm B}/N_0$&nbsp; oft besser als ein PCCC &nbsp;&#8658;&nbsp; niedrigerer <i>Error Floor</i>. Die Aussage gilt schon für SCCC&ndash;Komponentencodes mit Gedächtnis&nbsp; $m = 2$&nbsp; (nur vier Trelliszustände), während bei PCCC das Gedächtnis&nbsp; $m = 3$&nbsp; bzw.&nbsp; $m = 4$&nbsp; (acht bzw. sechzehn Trelliszustände) sein sollte.<br>
+
*An SCCC is often better than a PCCC &nbsp; &#8658; &nbsp; lower error floor for large&nbsp; $E_{\rm B}/N_0$.&nbsp; The statement is already true for SCCC component codes with memory&nbsp; $m = 2$&nbsp; $($only four trellis states$)$,&nbsp;  while for PCCC the memory&nbsp; $m = 3$&nbsp; and&nbsp; $m = 4$&nbsp; $($eight and sixteen trellis states,&nbsp;  respectively$)$&nbsp;  should be.<br>
  
*Im unteren Bereich $($kleines &nbsp;$E_{\rm B}/N_0)$&nbsp; ist dagegen der beste seriell verkettete Faltungscode (SCCC) um einige Zehntel Dezibel schlechter als der vergleichbare Turbocode gemäß Berrou (PCCC). Entsprechend größer ist auch der Abstand von der Shannongrenze.}}<br><br>
+
*In the lower range&nbsp;  $($small &nbsp;$E_{\rm B}/N_0)$&nbsp; on the other hand,&nbsp;  the best serial concatenated convolutional code is several tenths of a decibel worse than the comparable turbo code according to Berrou&nbsp;  $\rm (PCCC)$.&nbsp;  The distance from the Shannon bound is correspondingly larger.}}<br><br>
  
== Einige Anwendungsgebiete für Turbocodes ==
+
== Some application areas for turbo codes ==
 
<br>
 
<br>
[[File:P ID3059 KC T 4 3 S7b v2.png|right|frame|Einige standardisierte Turbocodes im Vergleich zur Shannon–Grenze]]
+
[[File:EN_KC_T_4_3_S7b_v2.png|right|frame|Some standardized turbo codes compared to the Shannon bound]]
In fast allen neueren Kommunikationssystemen werden Turbocodes eingesetzt. Die Grafik zeigt deren Leistungsfähigkeit beim AWGN&ndash;Kanal im Vergleich zur&nbsp; [[Channel_Coding/Informationstheoretische_Grenzen_der_Kanalcodierung#Kanalkapazit.C3.A4t_des_AWGN.E2.80.93Modells| Shannonschen Kanalkapazität]]&nbsp; (blaue Kurve).<br>
+
Turbo codes are used in almost all newer communication systems.&nbsp; The graph shows their performance with the AWGN channel compared to&nbsp; [[Channel_Coding/Information_Theoretical_Limits_of_Channel_Coding#Channel_capacity_of_the_AWGN_model|$\text{Shannon's channel capacity}$]]&nbsp; $($blue curve$)$.<br>
  
Der  grün hinterlegte Bereich &bdquo;BPSK&rdquo; gibt die Shannongrenze für Nachrichtensysteme mit binärem Eingang an, mit der nach dem&nbsp; [[Channel_Coding/Informationstheoretische_Grenzen_der_Kanalcodierung#Kanalcodierungstheorem_und_Kanalkapazit.C3.A4t| Kanalcodierungstheorem]]&nbsp; eine fehlerfreie Übertragung gerade noch möglich ist.<br>
+
The green highlighted area&nbsp; "BPSK"&nbsp; indicates the Shannon bound for digital systems with binary input,&nbsp;  with which according to the&nbsp; [[Channel_Coding/Information_Theoretical_Limits_of_Channel_Coding#Channel_coding_theorem_and_channel_capacity|$\text{channel coding theorem}$]]&nbsp; an error-free transmission is just possible.<br>
  
Anzumerken ist, dass hier für die eingezeichneten Kanalcodes von standardisierten Systemen die Fehlerrate&nbsp; $10^{-5}$&nbsp; zugrunde liegt, während die informationstheoretischen Kapazitätskurven (Shannon, BPSK) für die Fehlerwahrscheinlichkeit&nbsp; $0$&nbsp; gelten.
+
It should be noted that the bit error rate&nbsp; $\rm BER= 10^{-5}$&nbsp; is the basis here for the channel codes of standardized systems which are drawn in,&nbsp;  while the information-theoretical capacity curves&nbsp;  $($Shannon,&nbsp;  BPSK$)$&nbsp;  apply to the error probability&nbsp; $0$.
*Die blauen Rechtecke markieren die Turbocodes für UMTS. Diese sind Rate&ndash;$1/3$&ndash;Codes mit Gedächtnis&nbsp; $m = 3$. Die Leistungsfähigkeit hängt stark von der Interleavergröße ab. Mit&nbsp; $K = 6144$&nbsp; liegt dieser Code nur etwa&nbsp; $1 \ \rm dB$&nbsp; rechts von der Shannon&ndash;Grenze. LTE verwendet die gleichen Turbocodes. Geringfügige Unterschiede ergeben sich aufgrund des unterschiedlichen Interleavers.<br>
+
*The blue rectangles mark the turbo codes for UMTS.&nbsp; These are rate&nbsp;  $1/3$&nbsp; codes with memory&nbsp; $m = 3$.&nbsp;  The performance depends strongly on the interleaver size.&nbsp;  With&nbsp; $K = 6144$&nbsp; this code is only about&nbsp; $1 \rm dB$&nbsp; to the right of the Shannon bound.&nbsp; LTE uses the same turbo codes.&nbsp;  Minor differences occur due to the different interleaver.<br>
  
*Die roten Kreuze markieren die Turbocodes nach CCSDS (<i>Consultative Comittee for Space Data Systems</i>), entwickelt für den Einsatz bei Weltraummissionen. Diese Klasse geht von der festen Interleavergröße&nbsp; $K = 6920$&nbsp; aus und stellt Codes der Rate&nbsp; $1/6$,&nbsp; $1/4$,&nbsp; $1/3$&nbsp; und&nbsp; $1/2$&nbsp; bereit. Die niedrigsten Coderaten erlauben einen Betrieb mit&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx 0 \ \rm dB$.<br>
+
*The red crosses mark the turbo codes according to&nbsp;  $\rm CCSDS$&nbsp;  $($"Consultative Committee for Space Data Systems"$)$,&nbsp;  developed for use in space missions.&nbsp;  This class assumes the fixed interleaver size&nbsp; $K = 6920$&nbsp; and provides codes of rate&nbsp; $1/6$,&nbsp; $1/4$,&nbsp; $1/3$&nbsp; and&nbsp; $1/2$.&nbsp; The lowest code rates allow operation at&nbsp; $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx 0 \ \rm dB$.<br>
  
*Der grüne Kreis steht für einen sehr einfachen&nbsp; <i>Repeat&ndash;Accumulate</i>&nbsp; (RA) Code, einem seriell&ndash;verketteten Turbocode. Nachfolgend ist dessen Struktur skizziert: &nbsp; Der äußere Decoder verwendet einen&nbsp; [[Channel_Coding/Beispiele_bin%C3%A4rer_Blockcodes#Wiederholungscodes_.281.29| Wiederholungscode]]&nbsp; (englisch:&nbsp;  <i>Repetition Code</i>), im gezeichneten Beispiel mit der Rate&nbsp; $R = 1/3$. Nach dem Interleaver folgt ein RSC&ndash;Code mit&nbsp; $G(D) = 1/(1 + D)$ &nbsp; &#8658; &nbsp; Gedächtnis&nbsp; $m = 1$. Bei systematischer Ausführung ist die Gesamtcoderate&nbsp; $R = 1/4$&nbsp; (zu jedem Informationsbit werden noch drei Paritybit hinzugefügt).  
+
*The green circle represents a very simple&nbsp; "Repeat Accumulate"&nbsp; $\rm (RA)$&nbsp;  code,&nbsp; a serial concatenated turbo code.&nbsp;  The following is an outline of its structure: &nbsp; The outer decoder uses a&nbsp; [[Channel_Coding/Examples_of_Binary_Block_Codes#Repetition_Codes|$\text{repetition code}$]],&nbsp; in the drawn example with rate&nbsp; $R = 1/3$.&nbsp; The interleaver is followed by an RSC code with&nbsp; $G(D) = 1/(1 + D)$ &nbsp; &#8658; &nbsp; memory&nbsp; $m = 1$.&nbsp;  When executed systematically,&nbsp;  the total code rate is&nbsp; $R = 1/4$&nbsp; $($three parity bits are added to each bit of information$)$.  
  
[[File:P ID3064 KC T 4 3 S7c v1.png|center|frame|<i>Repeat Accumulate</i> (RA) Code der Rate $1/4$|class=fit]]
 
  
Aus der Grafik rechts oben erkennt man, dass dieser einfache RA&ndash;Code überraschend gut ist. Mit der Interleavergröße&nbsp; $K = 300000$&nbsp; beträgt der Abstand von der Shannon&ndash;Grenze lediglich etwa &nbsp;$1.5 \ \rm dB$&nbsp; (grüner Punkt).
+
[[File:EN_KC_T_4_3_S7c_v2.png|left|frame|Repeat Accumulate&nbsp; $\rm (RA)$&nbsp; code with rate&nbsp; $1/4$|class=fit]]
  
Ähnliche <i>Repeat Accumulate Codes</i> sind für den Standard <i>DVB Return Channel Terrestrial</i> (RCS) sowie für den WiMax&ndash;Standard (IEEE 802.16) vorgesehen.
 
  
 +
*From the graph on the top right,&nbsp; one can see that this simple RA code is surprisingly good.
  
 +
*With the interleaver size&nbsp; $K = 300000$&nbsp; the distance from the Shannon bound is only about &nbsp;$1.5 \ \rm dB$&nbsp; $($green dot$)$.
  
 
+
*Similar repeat accumulate codes are provided for the&nbsp; "DVB Return Channel Terrestrial"&nbsp; $\rm (RCS)$&nbsp; standard and for the WiMax standard&nbsp; $\rm (IEEE 802.16)$.
== Aufgaben zum Kapitel ==
+
<br clear=all>
 +
== Exercises for the chapter ==
 
<br>
 
<br>
[[Aufgaben:Aufgabe_4.08:_Wiederholung_zu_den_Faltungscodes|Aufgabe 4.8: Wiederholung zu den Faltungscodes]]
+
[[Aufgaben:Exercise_4.08:_Repetition_to_the_Convolutional_Codes|Exercise 4.08: Repetition to the Convolutional Codes]]
  
[[Aufgaben:Aufgabe_4.08Z:_Grundlegendes_zum_Interleaving|Aufgabe 4.8Z: Grundlegendes zum Interleaving]]
+
[[Aufgaben:Exercise_4.08Z:_Basics_about_Interleaving|Exercise 4.08Z: Basics about Interleaving]]
  
[[Aufgaben:Aufgabe_4.09:_Recursive_Systematic_Convolutional_Codes|Aufgabe 4.9: Recursive Systematic Convolutional Codes]]
+
[[Aufgaben:Exercise_4.09:_Recursive_Systematic_Convolutional_Codes|Exercise 4.09: Recursive Systematic Convolutional Codes]]
  
[[Aufgaben:Aufgabe_4.10:_Turbocoder_für_UMTS_und_LTE|Aufgabe 4.10: Turbocoder für UMTS und LTE]]
+
[[Aufgaben:Exercise_4.10:_Turbo_Encoder_for_UMTS_and_LTE|Exercise 4.10: Turbo Enccoder for UMTS and LTE]]
  
==Quellenverzeichnis==
+
==References==
 
<references/>
 
<references/>
  
 
{{Display}}
 
{{Display}}

Latest revision as of 15:58, 23 January 2023

Basic structure of a turbo code


All communications systems current today  $($2017$)$,  such as 

  • $\rm UMTS$  $($"Universal Mobile Telecommunications System"   ⇒   3rd generation mobile communications$)$ and 
  • $\rm LTE$  $($"Long Term Evolution"  ⇒   4th generation mobile communications$)$


use the concept of  $\text{symbol-wise iterative decoding}$.  This is directly related to the invention of  »turbo codes«  in 1993 by  $\text{Claude Berrou}$$\text{Alain Glavieux}$  and  $\text{Punya Thitimajshima}$  because it was only with these codes that the Shannon bound could be approached with reasonable decoding effort.

Turbo codes result from the parallel or serial concatenation of convolutional codes. 

Parallel concatenation of two rate  $1/2$  codes
Rate  $1/3$  turbo encoder  $($parallel concatenation of two rate  $1/2$  convolutional codes$)$

The graphic shows the parallel concatenation of two codes,  each with the parameters  $k = 1, \ n = 2$   ⇒   code rate $R = 1/2$.

In this representation is:

  1. $u$  the currently considered bit of the information sequence  $\underline{u}$,
  2. $x_{i,\hspace{0.03cm}j}$  the currently considered bit at the output  $j$  of encoder  $i$ 
    $($with  $1 ≤ i ≤ 2, \hspace{0.2cm} 1 ≤ j ≤ 2)$,
  3. $\underline{X} = (x_{1,\hspace{0.03cm}1}, \ x_{1,\hspace{0.03cm}2}, \ x_{2,\hspace{0.03cm}1}, \ x_{2,\hspace{0.03cm}2})$  the code word for the current information bit  $u$.

The resulting rate of the concatenated coding system is thus  $R = 1/4$. 

If systematic component codes are used,  the second shown model results.  The modifications from the top graph to the graph below can be justified as follows:

  • For systematic codes  $C_1$  and  $C_2$,  both  $x_{1,\hspace{0.03cm}1} = u$  and  $x_{2,\hspace{0.03cm}1} = u$.  Therefore,  one can dispense with the transmission of a redundant bit  $($e.g.  $x_{2,\hspace{0.03cm}2})$.
  • With this reduction,  the result is a rate  $1/3$  turbo code with  $k = 1$  and  $n = 3$.  The code word with the parity bits is  $p_1$  $($encoder 1$)$  and  $p_2$  $($encoder 2$)$:
$$\underline{X} = \left (u, \ p_1, \ p_2 \right )\hspace{0.05cm}.$$


Further modification of the basic structure of the turbo code


In the following we always assume a still somewhat further modified turbo encoder model:

Rate  $1/3$  turbo encoder with interleaver
  1. As required for the description of convolutional codes,  at the input is the information sequence  $\underline{u} = (u_1, \ u_2, \ \text{...}\hspace{0.05cm} , \ u_i , \ \text{...}\hspace{0.05cm} )$  instead of the isolated information bit  $u$ .
  2. The code word sequence  $\underline{x} = (\underline{X}_1, \underline{X}_2, \ \text{...}\hspace{0.05cm} , \ \underline{X}_i, \ \text{...}\hspace{0.05cm} )$  is generated.  To avoid confusion,  the code words  $\underline{X}_i = (u, \ p_1, \ p_2)$  with capital letters were introduced in the last section.
  3. The encoders  $\mathcal{C}_1$  and  $\mathcal{C}_2$  are conceived  $($at least in thought$)$  as  $\text{digital filters}$  and are thus characterized by the  $\text{transfer functions}$  $G_1(D)$  and  $G_2(D)$.
  4. For various reasons   ⇒   see  "two sections ahead"  the input sequence of the second encoder   ⇒   $\underline{u}_{\pi}$  should be scrambled with respect to the sequence  $\underline{u}$  by an interleaver  $(\Pi)$.
  5. Thereby there is nothing against choosing both encoders the same:  $G_1(D) = G_2(D) = G(D)$.  Without interleaver the correction capability would be extremely limited.

Example sequences at the rate  $1/3$  turbo encoder

$\text{Example 1:}$  The graph shows the different sequences in matched colors. To note:

  1. For  $\underline{u}_{\Pi}$  a  $3×4$  interleaver matrix is considered according to  "Exercise 4.8Z".

  2. The parity sequences are obtained according to  $G_1(D) = G_2(D) = 1 + D^2$   ⇒   see  "Exercise 4.8".


First requirement for turbo codes: Recursive component codes


Non-recursive transfer functions for generating the parity sequences cause a turbo code with insufficiently small minimum distance.  The reason for this inadequacy is the finite impulse response  $\underline{g} = (1, \ g_2, \ \text{...}\hspace{0.05cm} , \ g_m, \ 0, \ 0, \ \text{...}\hspace{0.05cm} )$  with  $g_2, \ \text{...}\hspace{0.05cm} , \ g_m ∈ \{0, 1\}$.  Here  $m$  denotes the  "memory".

Non-recursive systematic turbo code and state transition diagram



The graph shows the state transition diagram for the example  $\mathbf{G}(D) = \big [1, \ 1 + D^2 \big ]$. The transitions are labeled "$u_i\hspace{0.05cm}|\hspace{0.05cm}u_i p_i$".

  • The sequence  $S_0 → S_1 → S_2 → S_0 → S_0 → \ \text{...}\hspace{0.05cm} \ $  leads with respect to the input to the information sequence  $\underline{u} = (1, 0, 0, 0, 0, \ \text{...}\hspace{0.05cm})$,  and
  • with respect to the second encoded symbol to the sequence   $\underline{p} = (1, 0, 1, 0, 0, \ \text{...}\hspace{0.05cm})$    ⇒   because of  $\underline{u} = (1, 0, 0, 0, 0, \ \text{...}\hspace{0.05cm})$  identical to the  "impulse response"  $\underline{g}$   ⇒   memory  $m = 2$.


The lower graph applies to a so-called  »RSC code«  $($"Recursive Systematic Convolutional"$)$  correspondingly 

Non-recursive systematic turbo code and state transition diagram
$$\mathbf{G}(D) = \big [1, \ (1+ D^2)/(1 + D + D^2)\big ].$$
  • Here the sequence 
$$S_0 → S_1 → S_3 → S_2 → S_1 → S_3 → S_2 → \text{...}$$
leads to the impulse response 
$$\underline{g} = (1, 1, 1, 0, 1, 1, 0, 1, 1, \ \text{...}\hspace{0.05cm}).$$
  • This impulse response continues to infinity due to the loop  $S_1 → S_3 → S_2 → S_1$.  This enables or facilitates the iterative decoding.


More details on the examples in this section can be found in the  "Exercise 4.8"  and the  "Exercise 4.9".

Second requirement for turbo codes: Interleaving


It is obvious that for  $G_1(D) = G_2(D)$  an interleaver  $(\Pi)$  is essential.

  • Another reason is that the a-priori information is assumed to be independent.
  • Thus,  adjacent  $($and thus possibly strongly dependent$)$  bits for the other sub–code should be far apart.


Indeed,  for any RSC code   ⇒   infinite impulse response  $\underline{g}$   ⇒   fractional–rational transfer function  $G(D)$:  There are certain input sequences  $\underline{u}$,  which lead to very short parity sequences   $\underline{p} = \underline{u} ∗ \underline{g}$   with low Hamming weight  $w_{\rm H}(\underline{p})$.

For example,  such a sequence is given in the second graphic of the  "last section" :  

$$\underline{u} = (1, 1, 1, 0, 0, \ \text{...}\hspace{0.05cm}).$$

Then for the output sequence holds:

\[P(D) = U(D) \cdot G(D) = (1+D+D^2) \cdot \frac{1+D^2}{1+D+D^2}= 1+D^2\hspace{0.3cm}\Rightarrow\hspace{0.3cm} \underline{p}= (\hspace{0.05cm}1\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} 1\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} 0\hspace{0.05cm},\hspace{0.05cm} \text{...}\hspace{0.05cm}\hspace{0.05cm})\hspace{0.05cm}. \]

$\text{Meaning and purpose:}$  By  $\rm interleaving$,  it is now ensured with high probability that this sequence  $\underline{u} = (1, 1, 1, 0, 0, \ \text{...}\hspace{0.05cm})$  is converted into a sequence  $\underline{u}_{\pi}$ 

  • which also contains only three  "ones",
  • but whose output sequence is characterized by a large Hamming weight  $w_{\rm H}(\underline{p})$.


Thus,  the decoder succeeds in resolving such  "problem sequences"  iteratively.


For the following description of the interleavers we use the assignment  $I_{\rm In} → I_{\rm Out}$.  These labels stand for the indices of output and input sequence,  respectively. The interleaver variable is named  $I_{\rm max}$ .

Clarification of block interleaving

There are several,  fundamentally different interleaver concepts:

⇒   In a  »block interleaver«  one fills a matrix with  $N_{\rm C}$  columns and  $N_{\rm R}$  rows column-by-column and reads the matrix row-by-row.  Thus an information block with  $I_{\rm max} = N_{\rm C} \cdot N_{\rm R}$  bit is deterministically scrambled.

The right graph illustrates the principle for  $I_{\rm max} = 64$   ⇒   $1 ≤ I_{\rm In} \le 64$  and  $1 ≤ I_{\rm Out} \le 64$.  The order of the output bits is then:  

$$1, 9, 17, 25, 33, 41, 49, 57, 2, 10, 18, \ \text{...}\hspace{0.05cm} , 48, 56, 64.$$

More information on block interleaving is available in the  "Exercise 4.8Z".

Clarification of $S$–random interleaving





⇒   Turbo codes often use the  »$S$–random interleaver«. This pseudo random algorithm with the parameter  "$S$"  guarantees

  • that two indices less than  $S$  apart at the input
  • occur at least at the distance  $S + 1$  at the output.


The left graph shows the  $S$–random characteristic  $I_{\rm Out}(I_{\rm In})$  for  $I_{\rm max} = 640$.

  1. This algorithm is also deterministic,  and one can undo the scrambling in the decoder   ⇒   "De–interleaving".
  2. The distribution still seems  "more random"  than with block interleaving.


Bit-wise iterative decoding of a turbo code


The decoding of a turbo code is basically done as described in section  "Bit-wise Soft–in Soft–out Decoding".  From the following graphic,  however,  you can also see some special features that apply only to the turbo decoder.

Assumed is a rate  $1/3$  turbo code according to the description in the  "first section of this page".  Also,  the color scheme for the information sequence  $\underline{u}$  and the two parity sequences  $\underline{p}_1$  and  $\underline{p}_2$  are adapted from the earlier graphics.  Further,  it should be noted:

Iterative turbo decoder for rate  $R = 1/3$
  • The received vectors  $\underline{y}_0,\hspace{0.15cm} \underline{y}_1,\hspace{0.15cm} \underline{y}_2$  are real-valued and provide the respective soft information with respect to the information sequence  $\underline{u}$  and the sequences  $\underline{p}_1$  $($parity for encoder 1$)$  and  $\underline{p}_2$  $($parity for encoder 2$)$.
  • The decoder 1 receives the required intrinsic information in the form of the  $L$ values $L_{\rm K,\hspace{0.03cm} 0}$  $($out  $\underline{y}_0)$  and  $L_{\rm K,\hspace{0.03cm}1}$ $($out  $\underline{y}_1)$  over each bit of the sequences  $\underline{u}$  and  $\underline{p}_1$.
  • In the second decoder,  the scrambling of the information sequence  $\underline{u}$  must be taken into account.  Thus,  the  $L$ values to be processed are  $\pi(L_{\rm K, \hspace{0.03cm}0})$  and  $L_{\rm K, \hspace{0.03cm}2}$.
  • In the general  $\text{SISO decoder}$  the information exchange between the two component decoders was controlled with  $\underline{L}_{\rm A, \hspace{0.03cm}2} = \underline{L}_{\rm E, \hspace{0.03cm}1}$  and  $\underline{L}_{\rm A, \hspace{0.03cm}1} = \underline{L}_{\rm E, \hspace{0.03cm}2}$.
  • Written out at the bit level,  these equations are denoted by  $(1 ≤ i ≤ n)$:
\[L_{\rm A, \hspace{0.03cm}2}(i) = L_{\rm E, \hspace{0.03cm}1}(i) \hspace{0.5cm}{\rm resp.}\hspace{0.5cm} L_{\rm A, \hspace{0.03cm}1}(i) = L_{\rm E, \hspace{0.03cm}2}(i) \hspace{0.03cm}.\]
  • In the case of the turbo decoder,  the interleaver must also be taken into account in this information exchange.  Then for  $i = 1, \ \text{...}\hspace{0.05cm} , \ n$:
\[L_{\rm A, \hspace{0.03cm}2}\left ({\rm \pi}(i) \right ) = L_{\rm E, \hspace{0.03cm}1}(i) \hspace{0.5cm}{\rm bzw.}\hspace{0.5cm} L_{\rm A, \hspace{0.03cm}1}(i) = L_{\rm E, \hspace{0.03cm}2}\left ({\rm \pi}(i) \right ) \hspace{0.05cm}.\]
  • The a-posteriori  $L$ value is  $($arbitrarily$)$  given by decoder 1 in the above model.  This can be justified by the fact that one iteration stands for a twofold information exchange.

Performance of the turbo codes


Bit and block error probability of turbo codes at AWGN channel

We consider,  as in the last sections,  the rate  $1/3$  turbo code

  • with equal filter functions   $G_1(D) = G_2(D) = (1 + D^2)/(1 + D + D^2)$   ⇒   memory  $m = 2$,
  • with the interleaver size  $K$; first apply  $K = 10000,$  and
  • a sufficient large number of iterations  $(I = 20)$,  almost equivalent in result  to "$I → ∞$".

The two RSC component codes are each terminated on  $K$  bits.  Therefore we group

  • the information sequence  $\underline{u}$  into blocks of  $K$  information bits each,  and
  • the encoded sequence  $\underline{x}$  to blocks with each  $N = 3 \cdot K$  encoded bits.

All results apply to the  $\text{AWGN channel}$. Data are taken from the lecture notes  [Liv15][1].

The graph shows as a green curve the  »block error probability«   ⇒   ${\rm Pr(block\:error)}$  in double logarithmic representation depending on the AWGN characteristic  $10 \cdot {\rm lg} \, (E_{\rm B}/N_0)$.  It can be seen:

  1. The points marked with crosses resulted from the weight functions of the turbo code using the  $\text{Union Bound}$.  The simulation results   – in the graph marked by circles –   are almost congruent with the analytically calculated values.
  2. The  "Union Bound"  is only an upper bound based on maximum likelihood decoding  $\rm (ML)$.  The iterative decoder is suboptimal  (i.e.,  worse than "maximum likelihood").  These two effects seem to almost cancel each other out.
  3. At  $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) = 1 \ \rm dB$  there is a kink in the  (green)  curve,  corresponding to the slope change of  ${\rm Pr(bit\:error)}$   ⇒   blue curve.

The blue crosses  $($"calculation"$)$  and the blue circles  $($"simulation"$)$  denote the  »bit error probability«  for the interleaver size  $K = 10000$.  The  (dash-dotted)  curve for uncoded transmission is drawn as a comparison curve. 

⇒   To these blue curves is to be noted:

  • For small abscissa values,  the curve slope in the selected plot is nearly linear and sufficiently steep.  For example,  for   ${\rm Pr(bit\, error)} = 10^{-5}$   one needs at least   $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx \, 0.8 \ \rm dB$.
  • Compared to the  $\text{Shannon bound}$,  which results for code rate  $R = 1/3$  to  $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx \, –0.55 \ \rm dB$,  our standard turbo code   $($with memory  $m = 2)$   is only about  $1.35 \ \rm dB$  away.
  • From  $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx 0.5 \ \rm dB$  the curve runs flatter.  From  $\approx 1.5 \ \rm dB$  the curve is again  (nearly)  linear with lower slope.  For  ${\rm Pr(bit\:error)} = 10^{-7}$  one needs about  $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) = 3 \ \rm dB$.

⇒   We now try to explain the flatter drop in bit error probability with larger  $E_{\rm B}/N_0$.  This is called an  »$\text{error floor}$«:

  1. The reason for this asymptotically worse behavior with better channel  $($in the example:   from  $10 \cdot {\rm lg} \, E_{\rm B}/N_0 \ge 2 \ \rm dB)$  is the period  $P$  of the encoder impulse response  $\underline{g}$,  as demonstrated in the section  $\rm Interleaving$,  and explained in the  "Exercise 4.10"  with examples.
  2. For  $m = 2$,  the period is  $P = 2^m -1 = 3$.  Thus,  for  $\underline{u} = (1, 1, 1)   ⇒   w_{\rm H}(\underline{u}) = 3$  the parity sequence is bounded:   $\underline{p} = (1, 0, 1)$   ⇒   $w_{\rm H}(\underline{p}) = 2$   despite unbounded impulse response  $\underline{g}$.
  3. The sequence   $\underline{u} = (0, \ \text{...}\hspace{0.05cm} , \ 0, \ 1, \ 0, \ 0, \ 1, \ 0, \ \text{...}\hspace{0.05cm})$   ⇒   $U(D) = D^x \cdot (1 + D^P)$  also leads to a small Hamming weight  $w_{\rm H}(\underline{p})$  at the output,  which complicates the iterative decoding process.
  4. Some workaround is provided by the interleaver,  which ensures that both sequences  $\underline{p}_1$  and  $\underline{p}_2$  are not simultaneously loaded by very small Hamming weights  $w_{\rm H}(\underline{p}_1)$  and  $w_{\rm H}(\underline{p}_2)$.
  5. From the graph you can also see that the bit error probability is inversely proportional to the interleaver size  $K$.  That means:   With large  $K$  the despreading of unfavorable input sequences works better.
  6. However,  the approximation   $K \cdot {\rm Pr(bit\:error) = const.}$   is valid only for larger  $E_{\rm B}/N_0$  values   ⇒   small bit error probabilities.  The described effect also occurs for smaller  $E_{\rm B}/N_0$  values, but then the effects on the bit error probability are smaller.
  7. The flatter shape of the block error probability  $($green curve$)$  holds largely independent of the interleaver size  $K$,  i.e., for  $K = 1000$  as well as for  $K = 10000$.  In the range  $10 \cdot {\rm lg} \, E_{\rm B}/N_0 > 2 \ \rm dB$  namely single errors dominate,  so that here the approximation  ${\rm Pr(block\: error)} \approx {\rm Pr(bit\:error)} \cdot K$  is valid.


$\text{Conclusions:}$  The exemplary shown bit error probability and block error probability curves also apply qualitatively for  $m > 2$,  e.g.  for the UMTS and LTE turbo codes  $($each with  $m = 3)$,  which is analyzed in  "Exercise 4.10".  However,  some quantitative differences emerge:

  • The curve is steeper for small  $E_{\rm B}/N_0$  and the distance from the Shannon bound is slightly less than in the example shown here for  $m = 2$.
  • Also for larger  $m$  there is an  "error floor".  However,  the kink in the displayed curves then occurs later,  i.e. at smaller error probabilities.


Serial concatenated turbo codes – SCCC


The turbo codes considered so far are sometimes referred to as  "parallel concatenated convolutional codes"  $\rm (PCCC)$.

Some years after Berrou's invention,  "serial concatenated convolutional codes"  $\rm (SCCC)$  were also proposed by other authors according to the following diagram.

Serial concatenated convolutional codes:  Encoder and decoder




  • The information sequence  $\underline{u}$  is located at the outer convolutional encoder  $\mathcal{C}_1$.  Let its output sequence be  $\underline{c}$.
  • After the interleaver  $(\Pi)$  follows the inner convolutional encoder  $\mathcal{C}_2$.  The encoded sequence is called  $\underline{x}$ .
  • The resulting code rate is  $R = R_1 \cdot R_2$.  For rate  $1/2$  component codes:   $R = 1/4$.

The bottom diagram shows the SCCC decoder and illustrates the processing of  $L$–values and the exchange of extrinsic information between the two component decoders:

  • The inner decoder  $($code  $\mathcal{C}_2)$  receives the intrinsic information  $\underline{L}_{\rm K}(\underline{x})$  from the channel and a-priori information  $\underline{L}_{\rm A}(\underline{w})$  with  $\underline{w} = \pi(\underline{c})$  from the outer decoder  $($after interleaving$)$  and delivers the extrinsic information  $\underline{L}_{\rm E}(\underline{w})$  to the outer decoder.
  • The outer decoder  $(\mathcal{C}_1)$  processes the a-priori information  $\underline{L}_{\rm A}(\underline{c})$, i.e. the extrinsic information  $\underline{L}_{\rm E}(\underline{w})$  after de–interleaving.  It provides the extrinsic information  $\underline{L}_{\rm E}(\underline{c})$.
  • After a sufficient number of iterations,  the desired decoding result is obtained in the form of the a-posteriori  $L$–values  $\underline{L}_{\rm APP}(\underline{u})$  of the information sequence  $\underline{u}$.

$\text{Conclusions:}$  Important for serial concatenated convolutional codes  $\rm (SCCC)$  is that the inner code  $\mathcal{C}_2$  is recursive  $($i.e. a RSC code$)$.  The outer code  $\mathcal{C}_1$  may also be non-recursive.

Regarding the performance of such codes,  it should be noted:

  • An SCCC is often better than a PCCC   ⇒   lower error floor for large  $E_{\rm B}/N_0$.  The statement is already true for SCCC component codes with memory  $m = 2$  $($only four trellis states$)$,  while for PCCC the memory  $m = 3$  and  $m = 4$  $($eight and sixteen trellis states,  respectively$)$  should be.
  • In the lower range  $($small  $E_{\rm B}/N_0)$  on the other hand,  the best serial concatenated convolutional code is several tenths of a decibel worse than the comparable turbo code according to Berrou  $\rm (PCCC)$.  The distance from the Shannon bound is correspondingly larger.



Some application areas for turbo codes


Some standardized turbo codes compared to the Shannon bound

Turbo codes are used in almost all newer communication systems.  The graph shows their performance with the AWGN channel compared to  $\text{Shannon's channel capacity}$  $($blue curve$)$.

The green highlighted area  "BPSK"  indicates the Shannon bound for digital systems with binary input,  with which according to the  $\text{channel coding theorem}$  an error-free transmission is just possible.

It should be noted that the bit error rate  $\rm BER= 10^{-5}$  is the basis here for the channel codes of standardized systems which are drawn in,  while the information-theoretical capacity curves  $($Shannon,  BPSK$)$  apply to the error probability  $0$.

  • The blue rectangles mark the turbo codes for UMTS.  These are rate  $1/3$  codes with memory  $m = 3$.  The performance depends strongly on the interleaver size.  With  $K = 6144$  this code is only about  $1 \rm dB$  to the right of the Shannon bound.  LTE uses the same turbo codes.  Minor differences occur due to the different interleaver.
  • The red crosses mark the turbo codes according to  $\rm CCSDS$  $($"Consultative Committee for Space Data Systems"$)$,  developed for use in space missions.  This class assumes the fixed interleaver size  $K = 6920$  and provides codes of rate  $1/6$,  $1/4$,  $1/3$  and  $1/2$.  The lowest code rates allow operation at  $10 \cdot {\rm lg} \, (E_{\rm B}/N_0) \approx 0 \ \rm dB$.
  • The green circle represents a very simple  "Repeat Accumulate"  $\rm (RA)$  code,  a serial concatenated turbo code.  The following is an outline of its structure:   The outer decoder uses a  $\text{repetition code}$,  in the drawn example with rate  $R = 1/3$.  The interleaver is followed by an RSC code with  $G(D) = 1/(1 + D)$   ⇒   memory  $m = 1$.  When executed systematically,  the total code rate is  $R = 1/4$  $($three parity bits are added to each bit of information$)$.


Repeat Accumulate  $\rm (RA)$  code with rate  $1/4$


  • From the graph on the top right,  one can see that this simple RA code is surprisingly good.
  • With the interleaver size  $K = 300000$  the distance from the Shannon bound is only about  $1.5 \ \rm dB$  $($green dot$)$.
  • Similar repeat accumulate codes are provided for the  "DVB Return Channel Terrestrial"  $\rm (RCS)$  standard and for the WiMax standard  $\rm (IEEE 802.16)$.


Exercises for the chapter


Exercise 4.08: Repetition to the Convolutional Codes

Exercise 4.08Z: Basics about Interleaving

Exercise 4.09: Recursive Systematic Convolutional Codes

Exercise 4.10: Turbo Enccoder for UMTS and LTE

References

  1. Liva, G.:  Channels Codes for Iterative Decoding.  Lecture notes, Department of Communications Engineering, TU Munich and DLR Oberpfaffenhofen, 2015.