Aufgaben:Exercise 1.4: Entropy Approximations for the AMI Code - Revision history

Javier: Text replacement - "code symbol sequence" to "encoded sequence"

2021-08-16T12:08:08Z

Text replacement - "code symbol sequence" to "encoded sequence"

Guenter at 12:02, 10 August 2021

2021-08-10T12:02:43Z

Guenter at 14:01, 19 June 2021

2021-06-19T14:01:18Z

Guenter at 13:50, 19 June 2021

2021-06-19T13:50:54Z

Guenter at 13:50, 19 June 2021

2021-06-19T13:50:02Z

Guenter at 13:47, 19 June 2021

2021-06-19T13:47:14Z

Guenter: Guenter moved page Aufgaben:Exercise 1.4: Entropy Approximations for the AMI code to Aufgaben:Exercise 1.4: Entropy Approximations for the AMI Code over redirect

2021-06-19T12:24:55Z

Guenter moved page Exercise 1.4: Entropy Approximations for the AMI code to Exercise 1.4: Entropy Approximations for the AMI Code over redirect

Guenter: Guenter moved page Aufgaben:Exercise 1.4: Entropy Approximations for the AMI Code to Aufgaben:Exercise 1.4: Entropy Approximations for the AMI code

2021-06-19T12:16:37Z

Guenter moved page Exercise 1.4: Entropy Approximations for the AMI Code to Exercise 1.4: Entropy Approximations for the AMI code

Javier: Text replacement - "”" to """

2021-05-28T14:42:05Z

Text replacement - "”" to """

Javier: Text replacement - "„" to """

2021-05-28T14:41:44Z

Text replacement - "„" to """

@@ Line 31: / Line 31: @@
 *This task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Discrete Sources with Memory]].
 *Reference is made in particular to the page&nbsp;  [[Information_Theory/Discrete_Sources_with_Memory#The_entropy_of_the_AMI_code|The Entropy of the AMI code]].
-*In&nbsp;  [[Aufgaben:Exercise_1.4Z:_Entropy_of_the_AMI_Code|Exercise 1.4Z]]&nbsp; the actual entropy of the code symbol sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$.
+*In&nbsp;  [[Aufgaben:Exercise_1.4Z:_Entropy_of_the_AMI_Code|Exercise 1.4Z]]&nbsp; the actual entropy of the encoded sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$.
 *The following relations between the units are to be expected: &nbsp; $H \le$ ...$ \le H_3 \le H_2 \le H_1 \le H_0
   \hspace{0.05cm}.$

← Older revision		Revision as of 12:02, 10 August 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Nachrichtenquellen mit Gedächtnis~~	+	{{quiz-Header\|Buchseite=Information_Theory/Discrete_Sources_with_Memory
	}}		}}

@@ Line 72: / Line 72: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; The symbol size is&nbsp; $M = 3$.&nbsp; This gives the decision content with the&nbsp; <i>binary logarithm</i>&nbsp; to the base $2$ &nbsp; &rArr; &nbsp; $\log_2$ bzw $\rm ld$:
+'''(1)'''&nbsp; The symbol set size is&nbsp; $M = 3$.&nbsp; This gives the decision content with the&nbsp; logarithm&nbsp; to the base $2$ &nbsp; &rArr; &nbsp; $\log_2$:
 :$$H_0  = {\rm log}_2\hspace{0.1cm} M = {\rm log}_2\hspace{0.1cm} (3)   \hspace{0.15cm} \underline { = 1.585 \,{\rm bit/symbol}}
   \hspace{0.05cm}.$$
@@ Line 78: / Line 78: @@
-'''(2)'''&nbsp; The first-order entropy approximation takes into account only the symbol probabilities&nbsp; $p_{\rm P}$,&nbsp; $p_{\rm N}$&nbsp; and&nbsp; $p_{\rm M}$&nbsp; and not the statistical bonds within the code sequence&nbsp; $\langle c_\nu \rangle$.&nbsp; Thus one obtains:
+'''(2)'''&nbsp; The first-order entropy approximation takes into account only the symbol probabilities&nbsp; $p_{\rm P}$,&nbsp; $p_{\rm N}$&nbsp; and&nbsp; $p_{\rm M}$&nbsp; <br>and not the statistical bindings within the code sequence&nbsp; $\langle c_\nu \rangle$.&nbsp; Thus one obtains:
 :$$p_{\rm N} = p_{\rm L} = 1/2\hspace{0.05cm},\hspace{0.2cm}p_{\rm P} = p_{\rm M} = p_{\rm H}/2 = 1/4 \hspace{0.3cm}
 \Rightarrow\hspace{0.3cm} H_1  = \frac{1}{2} \cdot {\rm log}_2\hspace{0.1cm} (2) +
@@ Line 96: / Line 96: @@
 :$$p_{\rm PM} \hspace{0.1cm} =  \hspace{0.1cm} {\rm Pr}( c_1 = \mathbf{P}) \cdot {\rm Pr}(c_2 = \mathbf{M}\hspace{0.05cm} | c_1 = \mathbf{P}) = 1/4 \cdot 1/2 = 1/8 \hspace{0.05cm},$$
 :$$ p_{\rm MP} \hspace{0.1cm} =  \hspace{0.1cm} {\rm Pr}( c_1 = \mathbf{M}) \cdot {\rm Pr}(c_2 = \mathbf{P}\hspace{0.05cm} | c_1 = \mathbf{M}) = 1/4 \cdot 1/2 = 1/8 \hspace{0.05cm}.$$
-* For the remaining probabilities, it must also be taken into account whether the binary symbol&nbsp; $\rm H$&nbsp; with&nbsp; $\rm P$&nbsp; or with&nbsp; $\rm M$&nbsp; last time &nbsp;&#8658;&nbsp; further factor&nbsp; $1/2$:
+* For the remaining probabilities, it must also be taken into account whether the binary symbol&nbsp; $\rm H$&nbsp; was encoded with&nbsp; $\rm P$&nbsp; or with&nbsp; $\rm M$&nbsp; last time &nbsp;&#8658;&nbsp; further factor&nbsp; $1/2$:
 :$$p_{\rm NM} \hspace{0.1cm} =  \hspace{0.1cm} {\rm Pr}( c_1 = \mathbf{N}) \cdot {\rm Pr}(c_2 = \mathbf{M}\hspace{0.05cm} | c_1 = \mathbf{N}) = 1/2 \cdot 1/2 \cdot 1/2= 1/8 \hspace{0.05cm},$$
 :$$ p_{\rm NP} \hspace{0.1cm} =  \hspace{0.1cm} {\rm Pr}( c_1 = \mathbf{N}) \cdot {\rm Pr}(c_2 = \mathbf{P}\hspace{0.05cm} | c_1 = \mathbf{N}) = 1/2 \cdot 1/2 \cdot 1/2 = 1/8 \hspace{0.05cm}.$$
@@ Line 102: / Line 102: @@
 Thus the entropy&nbsp; $H_2'$&nbsp; of a two-tuple or its entropy&nbsp; $H_2$&nbsp; per code symbol:
 :$$H_2\hspace{0.01cm}'  = \frac{1}{4} \cdot {\rm log}_2\hspace{0.1cm} (4) +
-\cdot \frac{1}{8} \cdot {\rm log}_2\hspace{0.1cm}(8)  \hspace{0.15cm} {= 2.75 \,{\rm bit/two-tuple}}\hspace{0.3cm}
+\cdot \frac{1}{8} \cdot {\rm log}_2\hspace{0.1cm}(8)  \hspace{0.15cm} {= 2.75 \,{\text{bit/two-tuple} }}\hspace{0.3cm}
-\Rightarrow\hspace{0.3cm} H_2  = \frac{H_2\hspace{0.01cm}'}{2}   \hspace{0.15cm} \underline {= 1.375 \,{\rm bit/Symbol}}
+\Rightarrow\hspace{0.3cm} H_2  = \frac{H_2\hspace{0.01cm}'}{2}   \hspace{0.15cm} \underline {= 1.375 \,{\rm bit/symbol}}
   \hspace{0.05cm}.$$
 '''(4)'''&nbsp; The calculation of&nbsp; $H_3$&nbsp; is similar to the last subtask for&nbsp; $H_2$, except that now&nbsp; $3^3 = 27$&nbsp; composite probabilities must be determined:
-:$$p_{\rm NNN} = 1/8\hspace{0.4cm}{\rm (nur \hspace{0.15cm}einmal)}
+:$$p_{\rm NNN} = 1/8\hspace{0.4cm}{\rm (nur \hspace{0.15cm}only once)}
   \hspace{0.05cm},$$
-:$$p_{\rm NMM} = p_{\rm NPP} = p_{\rm MNM}  = ... = 0 \hspace{0.4cm}{\rm (ingesamt \hspace{0.15cm}12)}
+:$$p_{\rm NMM} = p_{\rm NPP} = p_{\rm MNM}  = ... = 0 \hspace{0.4cm}{\rm (total \hspace{0.15cm}12)}
   \hspace{0.05cm},$$
-:$$p_{\rm NNM} = p_{\rm NNP} = p_{\rm PMP}  = ... = 1/16 \hspace{0.4cm}{\rm (ingesamt \hspace{0.15cm}14)}$$
+:$$p_{\rm NNM} = p_{\rm NNP} = p_{\rm PMP}  = ... = 1/16 \hspace{0.4cm}{\rm (total \hspace{0.15cm}14)}$$
 :$$\Rightarrow\hspace{0.3cm} H_3  = \frac{1}{3} \cdot \left [
   \frac{1}{8} \cdot {\rm log}_2\hspace{0.1cm} (8) +

@@ Line 29: / Line 29: @@
 ''Hints:''
-*This task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Sources with Memory]].
+*This task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Discrete Sources with Memory]].
 *Reference is made in particular to the page&nbsp;  [[Information_Theory/Discrete_Sources_with_Memory#The_entropy_of_the_AMI_code|The Entropy of the AMI code]].
 *In&nbsp;  [[Aufgaben:Exercise_1.4Z:_Entropy_of_the_AMI_Code|Exercise 1.4Z]]&nbsp; the actual entropy of the code symbol sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$.

@@ Line 30: / Line 30: @@
 ''Hints:''
 *This task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Sources with Memory]].
-*Reference is made in particular to the page&nbsp;  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis#The_entropy_of_the_AMI_code|The Entropy of the AMI-code]].
+*Reference is made in particular to the page&nbsp;  [[Information_Theory/Discrete_Sources_with_Memory#The_entropy_of_the_AMI_code|The Entropy of the AMI code]].
-*In&nbsp;  [[Aufgaben:1.4Z_Entropie_der_AMI-Codierung|Exercise 1.4Z]]&nbsp; the actual entropy of the code symbol sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$.
+*In&nbsp;  [[Aufgaben:Exercise_1.4Z:_Entropy_of_the_AMI_Code|Exercise 1.4Z]]&nbsp; the actual entropy of the code symbol sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$.
 *The following relations between the units are to be expected: &nbsp; $H \le$ ...$ \le H_3 \le H_2 \le H_1 \le H_0
   \hspace{0.05cm}.$

@@ Line 6: / Line 6: @@
 The graph above shows the binary source signal&nbsp; $q(t)$, which can also be described by the symbol sequence&nbsp; $\langle q_\nu \rangle$&nbsp;  with&nbsp; $q_\nu \in \{ {\rm L}, {\rm H} \}$&nbsp;.&nbsp; In the entire task,&nbsp; $p_{\rm L} = p_{\rm H} =0.5$ applies.
-The coded signal&nbsp; $c(t)$&nbsp; and the corresponding symbol sequence&nbsp; $\langle c_\nu \rangle$&nbsp;  with&nbsp; $c_\nu \in \{{\rm P}, {\rm N}, {\rm M}  \}$&nbsp; results from the AMI coding (<i>Alternate Mark Inversion</i>) according to the following rule:
+The coded signal&nbsp; $c(t)$&nbsp; and the corresponding symbol sequence&nbsp; $\langle c_\nu \rangle$&nbsp;  with&nbsp; $c_\nu \in \{{\rm P}, {\rm N}, {\rm M}  \}$&nbsp; results from the AMI coding ("Alternate Mark Inversion") according to the following rule:
-* The binary symbol&nbsp; $\rm L$ &nbsp;&rArr;&nbsp; <i>Low</i> is always represented by the ternary symbol&nbsp; $\rm N$ &nbsp;&rArr;&nbsp; <i>Null</i>&nbsp;.
+* The binary symbol&nbsp; $\rm L$ &nbsp;&rArr;&nbsp; "Low"is always represented by the ternary symbol&nbsp; $\rm N$ &nbsp;&rArr;&nbsp; "German: Null"&nbsp;&rArr;&nbsp;"Zero".
-* The binary symbol&nbsp; $\rm H$ &nbsp;&rArr;&nbsp; <i>High</i>&nbsp; is also coded deterministically but alternately (hence the name "AMI") by the symbols&nbsp; $\rm P$ &nbsp;&rArr;&nbsp; <i>Plus</i>&nbsp; and&nbsp; $\rm M$ &nbsp;&rArr;&nbsp; <i>Minus</i>&nbsp;.
+* The binary symbol&nbsp; $\rm H$ &nbsp;&rArr;&nbsp; "High"&nbsp; is also encoded deterministically but alternately (hence the name "Alternate Mark Inversion") by the symbols&nbsp; $\rm P$ &nbsp;&rArr;&nbsp; "Plus"&nbsp; and&nbsp; $\rm M$ &nbsp;&rArr;&nbsp; "Minus".
-In this task, the entropy approximations for the AMI&ndash;coded signal are to be calculated:
+In this task, the entropy approximations for the AMI coded signal are to be calculated:
 * Approximation&nbsp; $H_1$&nbsp; refers only to the symbol probabilities&nbsp; $p_{\rm P}$,&nbsp; $p_{\rm N}$&nbsp; and&nbsp; $p_{\rm M}$.
-* The&nbsp; $k$&ndash;th entropy approximation&nbsp; $(k = 2, 3, \text{...} \ )$&nbsp; can be determined according to the following equation:
+* The&nbsp; $k$&ndash;th is the entropy approximation&nbsp; $(k = 2, 3, \text{...} \ )$&nbsp; can be determined according to the following equation:
 :$$H_k = \frac{1}{k} \cdot \sum_{i=1}^{3^k} p_i^{(k)} \cdot {\rm log}_2\hspace{0.1cm}\frac {1}{p_i^{(k)}} \hspace{0.5cm}({\rm Einheit\hspace{-0.1cm}: \hspace{0.1cm}bit/Symbol})
   \hspace{0.05cm}.$$
@@ Line 29: / Line 29: @@
 ''Hints:''
-*This task belongs to the chapter&nbsp; [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis|Sources with Memory]].
+*This task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Sources with Memory]].
 *Reference is made in particular to the page&nbsp;  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis#The_entropy_of_the_AMI_code|The Entropy of the AMI-code]].
-*In task&nbsp;  [[Aufgaben:1.4Z_Entropie_der_AMI-Codierung|Aufgabe 1.4Z]]&nbsp; the actual entropy of the code symbol sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$&nbsp;.
+*In&nbsp;  [[Aufgaben:1.4Z_Entropie_der_AMI-Codierung|Exercise 1.4Z]]&nbsp; the actual entropy of the code symbol sequence&nbsp;  $\langle c_\nu \rangle$&nbsp; is calculated to&nbsp; $H = 1 \; \rm bit/symbol$.
 *The following relations between the units are to be expected: &nbsp; $H \le$ ...$ \le H_3 \le H_2 \le H_1 \le H_0
   \hspace{0.05cm}.$

← Older revision	Revision as of 12:16, 19 June 2021
(No difference)

← Older revision		Revision as of 14:42, 28 May 2021
Line 9:		Line 9:

	* The binary symbol  $\rm L$  ⇒  <i>Low</i> is always represented by the ternary symbol  $\rm N$  ⇒  <i>Null</i> .		* The binary symbol  $\rm L$  ⇒  <i>Low</i> is always represented by the ternary symbol  $\rm N$  ⇒  <i>Null</i> .
−	* The binary symbol  $\rm H$  ⇒  <i>High</i>  is also coded deterministically but alternately (hence the name "AMI~~”~~) by the symbols  $\rm P$  ⇒  <i>Plus</i>  and  $\rm M$  ⇒  <i>Minus</i> .	+	* The binary symbol  $\rm H$  ⇒  <i>High</i>  is also coded deterministically but alternately (hence the name "AMI") by the symbols  $\rm P$  ⇒  <i>Plus</i>  and  $\rm M$  ⇒  <i>Minus</i> .

← Older revision		Revision as of 14:41, 28 May 2021
Line 9:		Line 9:

	* The binary symbol  $\rm L$  ⇒  <i>Low</i> is always represented by the ternary symbol  $\rm N$  ⇒  <i>Null</i> .		* The binary symbol  $\rm L$  ⇒  <i>Low</i> is always represented by the ternary symbol  $\rm N$  ⇒  <i>Null</i> .
−	* The binary symbol  $\rm H$  ⇒  <i>High</i>  is also coded deterministically but alternately (hence the name ~~&bdquo;~~AMI”) by the symbols  $\rm P$  ⇒  <i>Plus</i>  and  $\rm M$  ⇒  <i>Minus</i> .	+	* The binary symbol  $\rm H$  ⇒  <i>High</i>  is also coded deterministically but alternately (hence the name "AMI”) by the symbols  $\rm P$  ⇒  <i>Plus</i>  and  $\rm M$  ⇒  <i>Minus</i> .