Aufgaben:Exercise 1.6Z: Ternary Markov Source - Revision history

Guenter at 12:05, 10 August 2021

2021-08-10T12:05:35Z

Guenter at 14:40, 21 June 2021

2021-06-21T14:40:22Z

Guenter at 14:33, 21 June 2021

2021-06-21T14:33:43Z

Guenter at 14:31, 21 June 2021

2021-06-21T14:31:46Z

Guenter at 14:30, 21 June 2021

2021-06-21T14:30:27Z

Noah at 15:14, 2 June 2021

2021-06-02T15:14:59Z

Show changes

Javier: Text replacement - "„" to """

2021-05-28T14:46:08Z

Text replacement - "„" to """

Javier: Text replacement - "”" to """

2021-05-28T14:42:26Z

Text replacement - "”" to """

Guenter at 16:40, 25 May 2021

2021-05-25T16:40:08Z

Guenter: Guenter moved page Aufgaben:Aufgabe 1.6Z: Ternäre Markovquelle to Aufgaben:Exercise 1.6Z: Ternary Markov Source

2021-05-25T16:36:41Z

Guenter moved page Aufgabe 1.6Z: Ternäre Markovquelle to Exercise 1.6Z: Ternary Markov Source

← Older revision		Revision as of 12:05, 10 August 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Nachrichtenquellen mit Gedächtnis~~	+	{{quiz-Header\|Buchseite=Information_Theory/Discrete_Sources_with_Memory
	}}		}}

@@ Line 68: / Line 68: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; The maximum entropy results when the symbols&nbsp; $\rm A$,&nbsp; $\rm B$&nbsp; and&nbsp; $\rm C$&nbsp; are equally probable and the symbols within the sequence are statistically independent of each other. Then the following must apply:
+'''(1)'''&nbsp; The maximum entropy results when the symbols&nbsp; $\rm A$,&nbsp; $\rm B$&nbsp; and&nbsp; $\rm C$&nbsp; are equally probable and the symbols within the sequence are statistically independent of each other.&nbsp; Then the following must apply:
 [[File:Inf_Z_1_6_vers2.png|right|frame|Transition diagram for&nbsp; $p = 1/4$,&nbsp; $q = 1$]]
 * $p_{\rm A} = p_{\rm A|A} =  p_{\rm A|B} = p_{\rm A|C}  = 1/3$,
@@ Line 100: / Line 100: @@
 :$$p_{\rm AA} = p_{\rm BB}= p_{\rm CC}= p_{\rm AC}=p_{\rm BA}=p_{\rm CB}=1/12  \hspace{0.05cm},\hspace{0.5cm}
 p_{\rm AB} = p_{\rm BC}=p_{\rm CA}=1/6$$
-:$$\Rightarrow \hspace{0.2cm} H_2  = \frac{1}{2} \cdot \left [ 6 \cdot \frac{1}{12} \cdot {\rm ld}\hspace{0.1cm} 12  +
+:$$\Rightarrow \hspace{0.2cm} H_2  = \frac{1}{2} \cdot \left [ 6 \cdot \frac{1}{12} \cdot {\rm log_2}\hspace{0.1cm} 12  +
 \cdot \frac{1}{6} \cdot {\rm log_2}\hspace{0.1cm} 6 \right ] =   \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 4  + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3 +   \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 2 +   \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3
   = \frac{3}{4} + \frac{{\rm log_2}\hspace{0.1cm} 3}{2} \hspace{0.15cm} \underline {= 1.5425 \,{\rm bit/symbol}}
@@ Line 107: / Line 107: @@
-'''(4)'''&nbsp; Due to the Markov property of the source, the following holds true
+'''(4)'''&nbsp; Due to the Markov property of the source, the following holds true:
-:$$H = 2 \cdot H_2 - H_1 = \big [ {3}/{2} + {\rm log_2}\hspace{0.1cm} 3 \big ] -  {\rm log_2}\hspace{0.1cm} 3\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}
+:$$H = H_{k \to \infty}= 2 \cdot H_2 - H_1 = \big [ {3}/{2} + {\rm log_2}\hspace{0.1cm} 3 \big ] -  {\rm log_2}\hspace{0.1cm} 3\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}
   \hspace{0.05cm}.$$
 *One would arrive at the same result according to the following calculation:

@@ Line 53: / Line 53: @@
-{What is the actual source entropy with &nbsp;$p = 1/4$ &nbsp;and&nbsp;  $q = 1$?
+{What is the actual source entropy&nbsp; $H= H_{k \to \infty}$&nbsp; with &nbsp;$p = 1/4$ &nbsp;and&nbsp;  $q = 1$?
 |type="{}"}
 $H = \  \ $ { 1.5 1% } $\ \rm bit/symbol$
-{What is the actual source entropy with  &nbsp;$p = 1/2$ &nbsp;and&nbsp;  $q = 0$?
+{What is the actual source entropy&nbsp; $H= H_{k \to \infty}$&nbsp; with  &nbsp;$p = 1/2$ &nbsp;and&nbsp;  $q = 0$?
 |type="{}"}
 $H = \  \ $ { 0.667 1% } $\ \rm bit/symbol$

@@ Line 26: / Line 26: @@
 ''Hint:''
 *The exercise belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Discrete Sources with Memory]].
-*Reference is made in particular to the page&nbsp; [[Information_Theory/Discrete_Sources_with_Memory#Non-binary_Markov_sources|Non-binary Markov Sources]].
+*Reference is made in particular to the page&nbsp; [[Information_Theory/Discrete_Sources_with_Memory#Non-binary_Markov_sources|Non-binary Markov sources]].
 *For all entropies, add the pseudo-unit&nbsp; "bit/symbol".
-''Hints:''

@@ Line 3: / Line 3: @@
 }}
-[[File:Inf_Z_1_6_vers2.png|right|frame|Ternary Markov Source]]
+[[File:Inf_Z_1_6_vers2.png|right|frame|Ternary Markov source]]
-The diagram shows a Markov source with&nbsp; $M = 3$&nbsp; states&nbsp; $\rm A$,&nbsp; $\rm B$&nbsp; and&nbsp; $\rm C$.
+The diagram on the right shows a Markov source with&nbsp; $M = 3$&nbsp; states&nbsp; $\rm A$,&nbsp; $\rm B$&nbsp; and&nbsp; $\rm C$.
 Let the two parameters of this Markov process be:
@@ Line 10: / Line 10: @@
 Due to the Markov property of this source, the entropy can be determined in different ways:
-*One calculates <i>the first two entropy approximations</i>&nbsp; $H_1$&nbsp; and&nbsp; $H_2$.&nbsp; Then the following applies to the actual entropy:
+*One calculates the first two entropy approximations&nbsp; $H_1$&nbsp; and&nbsp; $H_2$.&nbsp; Then the following applies to the actual entropy:
-:$$H  = 2 \cdot H_{\rm 2} - H_{\rm 1}   \hspace{0.05cm}.$$
+:$$H= H_{k \to \infty}   = 2 \cdot H_{\rm 2} - H_{\rm 1}   \hspace{0.05cm}.$$
-*However, according to the <i>direct calculation method,</i> the entropy can also be written as follows&nbsp; (nine terms in total):
+*However, according to the&nbsp; "direct calculation method", the entropy can also be written as follows&nbsp; (nine terms in total):
 :$$H = p_{\rm AA}  \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}|\hspace{0.01cm}A}} + p_{\rm AB}   \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}|\hspace{0.01cm}A}} +  \ \text{...}
   \hspace{0.05cm}, \
@@ Line 24: / Line 24: @@
+''Hint:''
 ''Hints:''

@@ Line 29: / Line 29: @@
 *Die Aufgabe gehört zum  Kapitel&nbsp; [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis|Nachrichtenquellen mit Gedächtnis]].
 *Bezug genommen wird insbesondere auf die Seite&nbsp;  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis#Nichtbin.C3.A4re_Markovquellen|Nichtbinäre Markovquellen]].
-*Bei allen Entropien ist die Pseudoeinheit &bdquo;bit/Symbol" hinzuzufügen.
+*Bei allen Entropien ist die Pseudoeinheit "bit/Symbol" hinzuzufügen.

← Older revision		Revision as of 16:40, 25 May 2021
Line 139:		Line 139:


−	[[Category:Information Theory: Exercises\|^1.2 ~~Nachrichtenquellen mit Gedächtnis~~^]]	+	[[Category:Information Theory: Exercises\|^1.2 Sources with Memory^]]