Aufgaben:Exercise 1.6Z: Ternary Markov Source - Revision history

Maintenance script: Fix interlanguage link: resolve redirect chain

2026-03-16T15:57:26Z

Fix interlanguage link: resolve redirect chain

← Older revision		Revision as of 17:57, 16 March 2026
Line 121:		Line 121:

	[[Category:Information Theory: Exercises\|^1.2 Sources with Memory^]]		[[Category:Information Theory: Exercises\|^1.2 Sources with Memory^]]
	[[de:~~Zusatzaufgaben~~:1.~~6_Ternäre_Markovquelle~~]]		[[de:Aufgaben:Aufgabe 1.6Z: Ternäre Markovquelle]]

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:30:29Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:30, 16 March 2026
Line 14:		Line 14:

	*However, according to the  "direct calculation method", the entropy can also be written as follows  (nine terms in total):		*However, according to the  "direct calculation method", the entropy can also be written as follows  (nine terms in total):
	:$$H = p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} + p_{\rm AB} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A}} + \ \text{...}		:$$H = p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} + p_{\rm AB} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A}} + \ \text{...}\hspace{0.05cm}, \\text{wobei} \ p_{\rm AA} = p_{\rm A} \cdot p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A} \hspace{0.05cm},\hspace{0.2cm}p_{\rm AB} = p_{\rm A} \cdot p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A} \hspace{0.05cm}, \ \text{...}$$
	\hspace{0.05cm}, \
	\text{wobei} \ p_{\rm AA} = p_{\rm A} \cdot p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A} \hspace{0.05cm},\hspace{0.2cm}
	p_{\rm AB} = p_{\rm A} \cdot p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A} \hspace{0.05cm}, \ \text{...}$$


Line 90:		Line 87:
	It is obvious that the symbol probabilities are all equal:		It is obvious that the symbol probabilities are all equal:

	:$$p_{\rm A} = p_{\rm B} = p_{\rm C} = 1/3 \hspace{0.3cm}		:$$p_{\rm A} = p_{\rm B} = p_{\rm C} = 1/3 \hspace{0.3cm}\Rightarrow \hspace{0.3cm} H_1 = {\rm log_2}\hspace{0.1cm} 3 \hspace{0.15cm} \underline {= 1.585 \,{\rm bit/symbol}}\hspace{0.05cm}.$$
	\Rightarrow \hspace{0.3cm} H_1 = {\rm log_2}\hspace{0.1cm} 3 \hspace{0.15cm} \underline {= 1.585 \,{\rm bit/symbol}}
	\hspace{0.05cm}.$$


Line 98:		Line 93:
	'''(3)'''  For the second entropy approximation one needs  $3^2 = 9$  joint probabilities.		'''(3)'''  For the second entropy approximation one needs  $3^2 = 9$  joint probabilities.
	*Using the result from  '''(2)''' , one obtains for this:		*Using the result from  '''(2)''' , one obtains for this:
	:$$p_{\rm AA} = p_{\rm BB}= p_{\rm CC}= p_{\rm AC}=p_{\rm BA}=p_{\rm CB}=1/12 \hspace{0.05cm},\hspace{0.5cm}		:$$p_{\rm AA} = p_{\rm BB}= p_{\rm CC}= p_{\rm AC}=p_{\rm BA}=p_{\rm CB}=1/12 \hspace{0.05cm},\hspace{0.5cm}p_{\rm AB} = p_{\rm BC}=p_{\rm CA}=1/6$$
	p_{\rm AB} = p_{\rm BC}=p_{\rm CA}=1/6$$		:$$\Rightarrow \hspace{0.2cm} H_2 = \frac{1}{2} \cdot \left [ 6 \cdot \frac{1}{12} \cdot {\rm log_2}\hspace{0.1cm} 12 +3 \cdot \frac{1}{6} \cdot {\rm log_2}\hspace{0.1cm} 6 \right ] = \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 4 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 2 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3= \frac{3}{4} + \frac{{\rm log_2}\hspace{0.1cm} 3}{2} \hspace{0.15cm} \underline {= 1.5425 \,{\rm bit/symbol}}\hspace{0.05cm}.$$
	:$$\Rightarrow \hspace{0.2cm} H_2 = \frac{1}{2} \cdot \left [ 6 \cdot \frac{1}{12} \cdot {\rm log_2}\hspace{0.1cm} 12 +
	3 \cdot \frac{1}{6} \cdot {\rm log_2}\hspace{0.1cm} 6 \right ] = \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 4 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 2 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3
	= \frac{3}{4} + \frac{{\rm log_2}\hspace{0.1cm} 3}{2} \hspace{0.15cm} \underline {= 1.5425 \,{\rm bit/symbol}}
	\hspace{0.05cm}.$$



	'''(4)'''  Due to the Markov property of the source, the following holds true:		'''(4)'''  Due to the Markov property of the source, the following holds true:
	:$$H = H_{k \to \infty}= 2 \cdot H_2 - H_1 = \big [ {3}/{2} + {\rm log_2}\hspace{0.1cm} 3 \big ] - {\rm log_2}\hspace{0.1cm} 3\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}		:$$H = H_{k \to \infty}= 2 \cdot H_2 - H_1 = \big [ {3}/{2} + {\rm log_2}\hspace{0.1cm} 3 \big ] - {\rm log_2}\hspace{0.1cm} 3\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$
	*One would arrive at the same result according to the following calculation:		*One would arrive at the same result according to the following calculation:
	:$$H= p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} + p_{\rm AB} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A}} + ... \hspace{0.1cm}= 6 \cdot \frac{1}{12} \cdot {\rm log_2}\hspace{0.1cm} 4 + 3 \cdot \frac{1}{16} \cdot {\rm log_2}\hspace{0.1cm} 2		:$$H= p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} + p_{\rm AB} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A}} + ... \hspace{0.1cm}= 6 \cdot \frac{1}{12} \cdot {\rm log_2}\hspace{0.1cm} 4 + 3 \cdot \frac{1}{16} \cdot {\rm log_2}\hspace{0.1cm} 2\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}\hspace{0.05cm}.$$
	\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}
	\hspace{0.05cm}.$$


Line 120:		Line 108:
	*So there is a binary Markov chain with the symbols  $\rm A$  and  $\rm C$ .		*So there is a binary Markov chain with the symbols  $\rm A$  and  $\rm C$ .
	*The symbol probabilities are therefore given by:		*The symbol probabilities are therefore given by:
	:$$p_{\rm A} = 0.5 \cdot p_{\rm C} \hspace{0.05cm}, \hspace{0.2cm}p_{\rm A} + p_{\rm C} = 1 \hspace{0.3cm}		:$$p_{\rm A} = 0.5 \cdot p_{\rm C} \hspace{0.05cm}, \hspace{0.2cm}p_{\rm A} + p_{\rm C} = 1 \hspace{0.3cm}\Rightarrow \hspace{0.3cm} p_{\rm A} = 1/3 \hspace{0.05cm}, \hspace{0.2cm} p_{\rm C} = 2/3\hspace{0.05cm}. $$
	\Rightarrow \hspace{0.3cm} p_{\rm A} = 1/3 \hspace{0.05cm}, \hspace{0.2cm} p_{\rm C} = 2/3\hspace{0.05cm}. $$

	*This gives the following probabilities:		*This gives the following probabilities:
	:$$p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A} \hspace{0.1cm} = \hspace{0.1cm}0\hspace{0.7cm} \Rightarrow \hspace{0.3cm} p_{\rm AA} = 0 \hspace{0.05cm},$$		:$$p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A} \hspace{0.1cm} = \hspace{0.1cm}0\hspace{0.7cm} \Rightarrow \hspace{0.3cm} p_{\rm AA} = 0 \hspace{0.05cm},$$
	:$$ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C} =1/2\hspace{0.3cm} \Rightarrow \hspace{0.3cm} p_{\rm CA} =		:$$ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C} =1/2\hspace{0.3cm} \Rightarrow \hspace{0.3cm} p_{\rm CA} =p_{\rm C} \cdot p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C} = 2/3 \cdot 1/2 = 1/3 \hspace{0.05cm},\hspace{0.2cm}{\rm log_2}\hspace{0.1cm}(1/p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C} )= 1\hspace{0.05cm},$$
	p_{\rm C} \cdot p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C} = 2/3 \cdot 1/2 = 1/3 \hspace{0.05cm},\hspace{0.2cm}{\rm log_2}\hspace{0.1cm}(1/p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C} )= 1\hspace{0.05cm},$$		:$$ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A} =1\hspace{0.7cm} \Rightarrow \hspace{0.3cm} p_{\rm AC} =p_{\rm A} \cdot p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A} = 1/3 \cdot 1 = 1/3 \hspace{0.05cm},\hspace{0.61cm}{\rm log_2}\hspace{0.1cm}(1/p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A} )= 0\hspace{0.05cm},$$
	:$$ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A} =1\hspace{0.7cm} \Rightarrow \hspace{0.3cm} p_{\rm AC} =		:$$ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C} =1/2\hspace{0.3cm} \Rightarrow \hspace{0.3cm} p_{\rm CC} =p_{\rm C} \cdot p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C} = 2/3 \cdot 1/2 = 1/3\hspace{0.05cm},\hspace{0.2cm}{\rm log_2}\hspace{0.1cm}(1/p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C} )= 1 $$
	p_{\rm A} \cdot p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A} = 1/3 \cdot 1 = 1/3 \hspace{0.05cm},\hspace{0.61cm}{\rm log_2}\hspace{0.1cm}(1/p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A} )= 0\hspace{0.05cm},$$		:$$\Rightarrow \hspace{0.25cm} H = p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} +p_{\rm CA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C}}+ p_{\rm AC} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A}} +p_{\rm CC} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C}}=0 + 1/3 \cdot 1 + 1/3 \cdot 0 + 1/3 \cdot 1\hspace{0.15cm} \underline {= 0.667 \,{\rm bit/symbol}}\hspace{0.05cm}.$$
	:$$ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C} =1/2\hspace{0.3cm} \Rightarrow \hspace{0.3cm} p_{\rm CC} =
	p_{\rm C} \cdot p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C} = 2/3 \cdot 1/2 = 1/3\hspace{0.05cm},\hspace{0.2cm}{\rm log_2}\hspace{0.1cm}(1/p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C} )= 1 $$
	:$$\Rightarrow \hspace{0.25cm} H = p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} +p_{\rm CA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}C}}+ p_{\rm AC} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}A}} +
	p_{\rm CC} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm C\hspace{0.01cm}\|\hspace{0.01cm}C}}=
	0 + 1/3 \cdot 1 + 1/3 \cdot 0 + 1/3 \cdot 1
	\hspace{0.15cm} \underline {= 0.667 \,{\rm bit/symbol}}
	\hspace{0.05cm}.$$
	{{ML-Fuß}}		{{ML-Fuß}}

Maintenance script: Add German interlanguage link

2026-03-16T12:29:06Z

Add German interlanguage link

← Older revision		Revision as of 14:29, 16 March 2026
Line 141:		Line 141:

	[[Category:Information Theory: Exercises\|^1.2 Sources with Memory^]]		[[Category:Information Theory: Exercises\|^1.2 Sources with Memory^]]
			[[de:Zusatzaufgaben:1.6_Ternäre_Markovquelle]]

Guenter at 12:05, 10 August 2021

2021-08-10T12:05:35Z

← Older revision		Revision as of 14:05, 10 August 2021
Line 1:		Line 1:

	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Nachrichtenquellen mit Gedächtnis~~		{{quiz-Header\|Buchseite=Information_Theory/Discrete_Sources_with_Memory
	}}		}}

Guenter at 14:40, 21 June 2021

2021-06-21T14:40:22Z

Guenter at 14:33, 21 June 2021

2021-06-21T14:33:43Z

← Older revision		Revision as of 16:33, 21 June 2021
Line 53:		Line 53:


	{What is the actual source entropy with  $p = 1/4$  and  $q = 1$?		{What is the actual source entropy  $H= H_{k \to \infty}$  with  $p = 1/4$  and  $q = 1$?
	\|type="{}"}		\|type="{}"}
	$H = \ \ $ { 1.5 1% } $\ \rm bit/symbol$		$H = \ \ $ { 1.5 1% } $\ \rm bit/symbol$


	{What is the actual source entropy with  $p = 1/2$  and  $q = 0$?		{What is the actual source entropy  $H= H_{k \to \infty}$  with  $p = 1/2$  and  $q = 0$?
	\|type="{}"}		\|type="{}"}
	$H = \ \ $ { 0.667 1% } $\ \rm bit/symbol$		$H = \ \ $ { 0.667 1% } $\ \rm bit/symbol$

Guenter at 14:31, 21 June 2021

2021-06-21T14:31:46Z

← Older revision		Revision as of 16:31, 21 June 2021
Line 26:		Line 26:
	''Hint:''		''Hint:''
	*The exercise belongs to the chapter  [[Information_Theory/Discrete_Sources_with_Memory\|Discrete Sources with Memory]].		*The exercise belongs to the chapter  [[Information_Theory/Discrete_Sources_with_Memory\|Discrete Sources with Memory]].
	*Reference is made in particular to the page  [[Information_Theory/Discrete_Sources_with_Memory#Non-binary_Markov_sources\|Non-binary Markov ~~Sources~~]].		*Reference is made in particular to the page  [[Information_Theory/Discrete_Sources_with_Memory#Non-binary_Markov_sources\|Non-binary Markov sources]].
	*For all entropies, add the pseudo-unit  "bit/symbol".		*For all entropies, add the pseudo-unit  "bit/symbol".

	~~''Hints:''~~
	*The task belongs to the chapter  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis\|Sources with Memory]].
	*Reference is made in particular to the page  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis#Non-binary_Markov_sources\|Non-binary Markov Sources]].
	*For all entropies, add the pseudo unit "bit/symbol".

Guenter at 14:30, 21 June 2021

2021-06-21T14:30:27Z

← Older revision		Revision as of 16:30, 21 June 2021
Line 3:		Line 3:
	}}		}}

	[[File:Inf_Z_1_6_vers2.png\|right\|frame\|Ternary Markov ~~Source~~]]		[[File:Inf_Z_1_6_vers2.png\|right\|frame\|Ternary Markov source]]
	The diagram shows a Markov source with  $M = 3$  states  $\rm A$,  $\rm B$  and  $\rm C$.		The diagram on the right shows a Markov source with  $M = 3$  states  $\rm A$,  $\rm B$  and  $\rm C$.

	Let the two parameters of this Markov process be:		Let the two parameters of this Markov process be:
Line 10:		Line 10:
	Due to the Markov property of this source, the entropy can be determined in different ways:		Due to the Markov property of this source, the entropy can be determined in different ways:

	*One calculates ~~<i>~~the first two entropy approximations~~</i>~~  $H_1$  and  $H_2$.  Then the following applies to the actual entropy:		*One calculates the first two entropy approximations  $H_1$  and  $H_2$.  Then the following applies to the actual entropy:
	:$$H = 2 \cdot H_{\rm 2} - H_{\rm 1} \hspace{0.05cm}.$$		:$$H= H_{k \to \infty} = 2 \cdot H_{\rm 2} - H_{\rm 1} \hspace{0.05cm}.$$

	*However, according to the ~~<i>~~direct calculation method,~~</i>~~ the entropy can also be written as follows  (nine terms in total):		*However, according to the  "direct calculation method", the entropy can also be written as follows  (nine terms in total):
	:$$H = p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} + p_{\rm AB} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A}} + \ \text{...}		:$$H = p_{\rm AA} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm A\hspace{0.01cm}\|\hspace{0.01cm}A}} + p_{\rm AB} \cdot {\rm log_2}\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}\|\hspace{0.01cm}A}} + \ \text{...}
	\hspace{0.05cm}, \		\hspace{0.05cm}, \
Line 24:		Line 24:


			''Hint:''
			*The exercise belongs to the chapter  [[Information_Theory/Discrete_Sources_with_Memory\|Discrete Sources with Memory]].
			*Reference is made in particular to the page  [[Information_Theory/Discrete_Sources_with_Memory#Non-binary_Markov_sources\|Non-binary Markov Sources]].
			*For all entropies, add the pseudo-unit  "bit/symbol".

	''Hints:''		''Hints:''

Noah at 15:14, 2 June 2021

2021-06-02T15:14:59Z

Show changes

Javier: Text replacement - "„" to """

2021-05-28T14:46:08Z

Text replacement - "„" to """

← Older revision		Revision as of 16:46, 28 May 2021
Line 29:		Line 29:
	*Die Aufgabe gehört zum Kapitel  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis\|Nachrichtenquellen mit Gedächtnis]].		*Die Aufgabe gehört zum Kapitel  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis\|Nachrichtenquellen mit Gedächtnis]].
	*Bezug genommen wird insbesondere auf die Seite  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis#Nichtbin.C3.A4re_Markovquellen\|Nichtbinäre Markovquellen]].		*Bezug genommen wird insbesondere auf die Seite  [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis#Nichtbin.C3.A4re_Markovquellen\|Nichtbinäre Markovquellen]].
	*Bei allen Entropien ist die Pseudoeinheit ~~&bdquo;~~bit/Symbol" hinzuzufügen.		*Bei allen Entropien ist die Pseudoeinheit "bit/Symbol" hinzuzufügen.

← Older revision		Revision as of 16:40, 21 June 2021
Line 68:		Line 68:
	===Solution===		===Solution===
	{{ML-Kopf}}		{{ML-Kopf}}
	'''(1)'''  The maximum entropy results when the symbols  $\rm A$,  $\rm B$  and  $\rm C$  are equally probable and the symbols within the sequence are statistically independent of each other. Then the following must apply:		'''(1)'''  The maximum entropy results when the symbols  $\rm A$,  $\rm B$  and  $\rm C$  are equally probable and the symbols within the sequence are statistically independent of each other.  Then the following must apply:
	[[File:Inf_Z_1_6_vers2.png\|right\|frame\|Transition diagram for  $p = 1/4$,  $q = 1$]]		[[File:Inf_Z_1_6_vers2.png\|right\|frame\|Transition diagram for  $p = 1/4$,  $q = 1$]]
	* $p_{\rm A} = p_{\rm A\|A} = p_{\rm A\|B} = p_{\rm A\|C} = 1/3$,		* $p_{\rm A} = p_{\rm A\|A} = p_{\rm A\|B} = p_{\rm A\|C} = 1/3$,
Line 100:		Line 100:
	:$$p_{\rm AA} = p_{\rm BB}= p_{\rm CC}= p_{\rm AC}=p_{\rm BA}=p_{\rm CB}=1/12 \hspace{0.05cm},\hspace{0.5cm}		:$$p_{\rm AA} = p_{\rm BB}= p_{\rm CC}= p_{\rm AC}=p_{\rm BA}=p_{\rm CB}=1/12 \hspace{0.05cm},\hspace{0.5cm}
	p_{\rm AB} = p_{\rm BC}=p_{\rm CA}=1/6$$		p_{\rm AB} = p_{\rm BC}=p_{\rm CA}=1/6$$
	:$$\Rightarrow \hspace{0.2cm} H_2 = \frac{1}{2} \cdot \left [ 6 \cdot \frac{1}{12} \cdot {\rm ld}\hspace{0.1cm} 12 +		:$$\Rightarrow \hspace{0.2cm} H_2 = \frac{1}{2} \cdot \left [ 6 \cdot \frac{1}{12} \cdot {\rm log_2}\hspace{0.1cm} 12 +
	3 \cdot \frac{1}{6} \cdot {\rm log_2}\hspace{0.1cm} 6 \right ] = \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 4 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 2 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3		3 \cdot \frac{1}{6} \cdot {\rm log_2}\hspace{0.1cm} 6 \right ] = \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 4 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 2 + \frac{1}{4} \cdot {\rm log_2}\hspace{0.1cm} 3
	= \frac{3}{4} + \frac{{\rm log_2}\hspace{0.1cm} 3}{2} \hspace{0.15cm} \underline {= 1.5425 \,{\rm bit/symbol}}		= \frac{3}{4} + \frac{{\rm log_2}\hspace{0.1cm} 3}{2} \hspace{0.15cm} \underline {= 1.5425 \,{\rm bit/symbol}}
Line 107:		Line 107:


	'''(4)'''  Due to the Markov property of the source, the following holds true		'''(4)'''  Due to the Markov property of the source, the following holds true:
	:$$H = 2 \cdot H_2 - H_1 = \big [ {3}/{2} + {\rm log_2}\hspace{0.1cm} 3 \big ] - {\rm log_2}\hspace{0.1cm} 3\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}		:$$H = H_{k \to \infty}= 2 \cdot H_2 - H_1 = \big [ {3}/{2} + {\rm log_2}\hspace{0.1cm} 3 \big ] - {\rm log_2}\hspace{0.1cm} 3\hspace{0.15cm} \underline {= 1.5 \,{\rm bit/symbol}}
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$
	*One would arrive at the same result according to the following calculation:		*One would arrive at the same result according to the following calculation: