Aufgaben:Exercise 1.5: Binary Markov Source - Revision history

Guenter at 12:03, 10 August 2021

2021-08-10T12:03:55Z

Guenter at 12:23, 21 June 2021

2021-06-21T12:23:30Z

Guenter at 08:15, 21 June 2021

2021-06-21T08:15:56Z

Guenter at 18:17, 20 June 2021

2021-06-20T18:17:11Z

Guenter at 18:16, 20 June 2021

2021-06-20T18:16:16Z

Guenter at 18:14, 20 June 2021

2021-06-20T18:14:18Z

Guenter at 17:58, 20 June 2021

2021-06-20T17:58:58Z

Guenter at 16:05, 20 June 2021

2021-06-20T16:05:23Z

Guenter at 16:04, 20 June 2021

2021-06-20T16:04:27Z

Guenter at 16:00, 20 June 2021

2021-06-20T16:00:03Z

← Older revision		Revision as of 12:03, 10 August 2021
Line 1:		Line 1:

−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Nachrichtenquellen mit Gedächtnis~~	+	{{quiz-Header\|Buchseite=Information_Theory/Discrete_Sources_with_Memory
	}}		}}

@@ Line 87: / Line 87: @@
 ===Solution===
 {{ML-Kopf}}
-[[File:Inf_A_1_5a_vers2.png|right|frame|Markov diagram for the subtasks&nbsp; '''(1)''', ... ,&nbsp; '''(5)''']]
+[[File:Inf_A_1_5a_vers2.png|right|frame|Markov diagram for tasks&nbsp; '''(1)''', ... ,&nbsp; '''(5)''']]
 After&nbsp; $\rm A$&nbsp;, &nbsp; $\rm A$&nbsp; and&nbsp; $\rm B$&nbsp; are equally probable.&nbsp; After&nbsp; $\rm B$&nbsp;, &nbsp; $\rm B$&nbsp; occurs much more frequently than&nbsp; $\rm A$&nbsp;.&nbsp; The following applies to the transition probabilities
 :$$p_{\rm A\hspace{0.01cm}|\hspace{0.01cm}A} \hspace{0.1cm} =  \hspace{0.1cm} 1 - p_{\rm B\hspace{0.01cm}|\hspace{0.01cm}A}= 1 - q \hspace{0.15cm} \underline {= 0.5} \hspace{0.05cm},$$
@@ Line 139: / Line 139: @@
 [[File:Inf_A_1_5f_vers2.png|right|frame|Markov diagram for subtask&nbsp; '''(6)''']]
-'''(6)'''&nbsp; With the new set of parameters&nbsp; $(p = 1/4, q = 3/4)$&nbsp; , we obtain for the symbol probabilities:
+'''(6)'''&nbsp; With the new set of parameters&nbsp; $(p = 1/4, q = 3/4)$,&nbsp; we obtain for the symbol probabilities:
 :$$ p_{\rm A} =  1/4, \ p_{\rm B} =  3/4.$$

@@ Line 36: / Line 36: @@
 *With the exception of subtask&nbsp; '''(6)'''&nbsp; &nbsp;  $p = 1/4$ &nbsp;and&nbsp; $q = 1/2$ always apply.
-*For the (ergodic) symbol probabilities of a first order Markov chain applies:
+*For the&nbsp; (ergodic)&nbsp; symbol probabilities of a first order Markov chain applies:
 :$$ p_{\rm A}  = \frac {p_{\rm A\hspace{0.01cm}|\hspace{0.01cm}B}}
 { p_{\rm A\hspace{0.01cm}|\hspace{0.01cm}B} + p_{\rm B\hspace{0.01cm}|\hspace{0.01cm}A}}  \hspace{0.05cm},  \hspace{0.3cm}
@@ Line 54: / Line 54: @@
-{What are the (unconditional) symbol probabilities?&nbsp; Let &nbsp;$p = 1/4$ &nbsp;and&nbsp; $q = 1/2$ still hold.
+{What are the&nbsp; (unconditional)&nbsp; symbol probabilities?&nbsp; Let &nbsp;$p = 1/4$ &nbsp;and&nbsp; $q = 1/2$ still hold.
 |type="{}"}
 $p_{\rm A} \ =  \  $  { 0.333 3% }
@@ Line 60: / Line 60: @@
-{Give the corresponding first-order entropy approximation.
+{Give the corresponding first order entropy approximation.
 |type="{}"}
 $H_1  \ =  \ $ { 0.918 1% } $\ \rm bit/symbol$

@@ Line 32: / Line 32: @@
 ''Hints:''
-*The task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Discrete Sources with Memory]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Discrete Sources with Memory]].
 *Reference is also made in particular to the two pages&nbsp;   [[Theory_of_Stochastic_Signals/Set_Theory_Basics#Intersection|"Intersection"]]&nbsp; and&nbsp; [[Theory_of_Stochastic_Signals/Statistical_Dependence_and_Independence#Conditional_Probability|"Conditional Probability"]].
 *With the exception of subtask&nbsp; '''(6)'''&nbsp; &nbsp;  $p = 1/4$ &nbsp;and&nbsp; $q = 1/2$ always apply.

@@ Line 33: / Line 33: @@
 ''Hints:''
 *The task belongs to the chapter&nbsp; [[Information_Theory/Discrete_Sources_with_Memory|Discrete Sources with Memory]].
-*Reference is also made in particular to the two pages&nbsp;   [[Theory_of_Stochastic_Signals/Set_Theory_Basics#Intersection|"Intersection"]]&nbsp; und&nbsp; [[Theory_of_Stochastic_Signals/Statistische_Abhängigkeit_und_Unabhängigkeit#Bedingte_Wahrscheinlichkeit|Bedingte Wahrscheinlichkeit]].
+*Reference is also made in particular to the two pages&nbsp;   [[Theory_of_Stochastic_Signals/Set_Theory_Basics#Intersection|"Intersection"]]&nbsp; and&nbsp; [[Theory_of_Stochastic_Signals/Statistical_Dependence_and_Independence#Conditional_Probability|"Conditional Probability"]].
 *With the exception of subtask&nbsp; '''(6)'''&nbsp; &nbsp;  $p = 1/4$ &nbsp;and&nbsp; $q = 1/2$ always apply.

@@ Line 18: / Line 18: @@
 {\rm log}_2\hspace{0.1cm}\frac {1}{ p_{\rm B\hspace{0.01cm}|\hspace{0.01cm}B}}
   \hspace{0.05cm}.$$
-It should be noted that in the argument of the&nbsp; "binary logarithm", the&nbsp; [[Theory_of_Stochastic_Signals/Statistical_Dependence_and_Independence#Conditional_Probability|conditional probabilities]]&nbsp; $p_{\rm A\hspace{0.01cm}|\hspace{0.01cm}A}$,&nbsp; $p_{\rm B\hspace{0.01cm}|\hspace{0.01cm}A}$, ... &nbsp; are to be used, while the&nbsp; [[Theory_of_Stochastic_Signals/Set_Theory_Basics#Intersection|joint probabilities]]&nbsp; $p_{\rm AA}$,&nbsp; $p_{\rm AB}$, ... &nbsp; are to be used for the weighting.
+It should be noted that in the argument of the&nbsp; "binary logarithm", the&nbsp; [[Theory_of_Stochastic_Signals/Statistical_Dependence_and_Independence#Conditional_Probability|conditional probabilities]]&nbsp; $p_{\rm A\hspace{0.01cm}|\hspace{0.01cm}A}$,&nbsp; $p_{\rm B\hspace{0.01cm}|\hspace{0.01cm}A}$, ... &nbsp; are to be used, while the&nbsp; [[Theory_of_Stochastic_Signals/Statistical_Dependence_and_Independence#Conditional_Probability|"Conditional Probabilities"]]&nbsp; $p_{\rm AA}$,&nbsp; $p_{\rm AB}$, ... &nbsp; are to be used for the weighting.
 Using the first order entropy approximation,
@@ Line 27: / Line 27: @@
 :$$H_k =  \frac{1}{k} \cdot \big [ H_{\rm 1} + (k-1) \cdot H_{\rm M} \big ]
   \hspace{0.05cm}.$$
@@ Line 37: / Line 33: @@
 ''Hints:''
 *The task belongs to the chapter&nbsp; [[Information_Theory/Nachrichtenquellen_mit_Gedächtnis|Sources with Memory]].
-*Reference is also made in particular to the two pages&nbsp;   [[Theory_of_Stochastic_Signals/Mengentheoretische_Grundlagen#Schnittmenge|Schnittmenge]]&nbsp; und&nbsp; [[Theory_of_Stochastic_Signals/Statistische_Abhängigkeit_und_Unabhängigkeit#Bedingte_Wahrscheinlichkeit|Bedingte Wahrscheinlichkeit]].
+*Reference is also made in particular to the two pages&nbsp;   [[Theory_of_Stochastic_Signals/Set_Theory_Basics#Intersection|"Intersection"]]&nbsp; und&nbsp; [[Theory_of_Stochastic_Signals/Statistische_Abhängigkeit_und_Unabhängigkeit#Bedingte_Wahrscheinlichkeit|Bedingte Wahrscheinlichkeit]].
 *With the exception of subtask&nbsp; '''(6)'''&nbsp; &nbsp;  $p = 1/4$ &nbsp;and&nbsp; $q = 1/2$ always apply.

@@ Line 24: / Line 24: @@
 {\rm log}_2\hspace{0.1cm}\frac {1}{ p_{\rm B}}
   \hspace{0.5cm}({\rm unit\hspace{-0.1cm}: \hspace{0.1cm}bit/symbol})\hspace{0.05cm},$$
-as well as the (actual) entropy&nbsp; $H = H_{\rm M}$&nbsp; given above, all further entropy approximations&nbsp; $(k = 2,, 3, \text{...})$&nbsp; can also be given directly for a Markov source:
+as well as the (actual) entropy&nbsp; $H = H_{\rm M}$&nbsp; given above, all further entropy approximations&nbsp; $(k = 2, 3, \text{...})$&nbsp; can also be given directly for a Markov source:
 :$$H_k =  \frac{1}{k} \cdot \big [ H_{\rm 1} + (k-1) \cdot H_{\rm M} \big ]
   \hspace{0.05cm}.$$