Aufgaben:Exercise 1.1: Entropy of the Weather - Revision history

Maintenance script: Fix interlanguage link: resolve redirect chain

2026-03-16T15:56:05Z

Fix interlanguage link: resolve redirect chain

← Older revision		Revision as of 17:56, 16 March 2026
Line 118:		Line 118:

	[[Category:Information Theory: Exercises\|^1.1 Memoryless Sources^]]		[[Category:Information Theory: Exercises\|^1.1 Memoryless Sources^]]
	[[de:Aufgaben:1.~~1_Wetterentropie~~]]		[[de:Aufgaben:Aufgabe 1.1: Wetterentropie]]

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:30:19Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:30, 16 March 2026
Line 87:		Line 87:
	{{ML-Kopf}}		{{ML-Kopf}}
	'''(1)'''  For the file  "Mixed"  the two probabilities are the same:   $p_{\rm B} = p_{\rm G} =0.5$.  This gives us for the entropy:		'''(1)'''  For the file  "Mixed"  the two probabilities are the same:   $p_{\rm B} = p_{\rm G} =0.5$.  This gives us for the entropy:
	:$$H_{\rm M} = 0.5 \cdot {\rm log}_2\hspace{0.1cm}\frac{1}{0.5} + 0.5 \cdot		:$$H_{\rm M} = 0.5 \cdot {\rm log}_2\hspace{0.1cm}\frac{1}{0.5} + 0.5 \cdot{\rm log}_2\hspace{0.1cm}\frac{1}{0.5} \hspace{0.15cm}\underline {= 1\,{\rm bit/enquiry}}\hspace{0.05cm}.$$
	{\rm log}_2\hspace{0.1cm}\frac{1}{0.5} \hspace{0.15cm}\underline {= 1\,{\rm bit/enquiry}}\hspace{0.05cm}.$$


	'''(2)'''  With  $p_{\rm B} = 0.8$  and  $p_{\rm G} =0.2$,  a smaller entropy value is obtained:		'''(2)'''  With  $p_{\rm B} = 0.8$  and  $p_{\rm G} =0.2$,  a smaller entropy value is obtained:
	:$$H_{\rm R} \hspace{-0.05cm}= \hspace{-0.05cm}0.8 \cdot {\rm log}_2\hspace{0.05cm}\frac{5}{4} \hspace{-0.05cm}+ \hspace{-0.05cm}0.2 \cdot {\rm log}_2\hspace{0.05cm}\frac{5}{1}\hspace{-0.05cm}=\hspace{-0.05cm}		:$$H_{\rm R} \hspace{-0.05cm}= \hspace{-0.05cm}0.8 \cdot {\rm log}_2\hspace{0.05cm}\frac{5}{4} \hspace{-0.05cm}+ \hspace{-0.05cm}0.2 \cdot {\rm log}_2\hspace{0.05cm}\frac{5}{1}\hspace{-0.05cm}=\hspace{-0.05cm}0.8 \cdot{\rm log}_2\hspace{0.05cm}5\hspace{-0.05cm} - \hspace{-0.05cm}0.8 \cdot {\rm log}_2\hspace{0.05cm}4 \hspace{-0.05cm}+ \hspace{-0.05cm}0.2 \cdot {\rm log}_2 \hspace{0.05cm} 5 \hspace{-0.05cm}=\hspace{-0.05cm}{\rm log}_2\hspace{0.05cm}5\hspace{-0.05cm} -\hspace{-0.05cm} 0.8 \cdot{\rm log}_2\hspace{0.1cm}4\hspace{-0.05cm} = \hspace{-0.05cm} \frac{{\rm lg} \hspace{0.1cm}5}{{\rm lg}\hspace{0.1cm}2} \hspace{-0.05cm}-\hspace{-0.05cm} 1.6 \hspace{0.15cm}\underline {= 0.722\,{\rm bit/enquiry}}\hspace{0.05cm}.$$
	0.8 \cdot{\rm log}_2\hspace{0.05cm}5\hspace{-0.05cm} - \hspace{-0.05cm}0.8 \cdot {\rm log}_2\hspace{0.05cm}4 \hspace{-0.05cm}+ \hspace{-0.05cm}0.2 \cdot {\rm log}_2 \hspace{0.05cm} 5 \hspace{-0.05cm}=\hspace{-0.05cm}
	{\rm log}_2\hspace{0.05cm}5\hspace{-0.05cm} -\hspace{-0.05cm} 0.8 \cdot
	{\rm log}_2\hspace{0.1cm}4\hspace{-0.05cm} = \hspace{-0.05cm} \frac{{\rm lg} \hspace{0.1cm}5}{{\rm lg}\hspace{0.1cm}2} \hspace{-0.05cm}-\hspace{-0.05cm} 1.6 \hspace{0.15cm}
	\underline {= 0.722\,{\rm bit/enquiry}}\hspace{0.05cm}.$$


Line 104:		Line 99:

	'''(4)'''  With  $p_{\rm B} = 1/30$  and  $p_{\rm G} =29/30$,  the information contents are as follows:		'''(4)'''  With  $p_{\rm B} = 1/30$  and  $p_{\rm G} =29/30$,  the information contents are as follows:
	:$$I_{\rm B} \hspace{0.1cm} = \hspace{0.1cm} {\rm log}_2\hspace{0.1cm}30 =		:$$I_{\rm B} \hspace{0.1cm} = \hspace{0.1cm} {\rm log}_2\hspace{0.1cm}30 =\frac{{\rm lg}\hspace{0.1cm}30}{{\rm lg}\hspace{0.1cm}2} = \frac{1.477}{0.301} \hspace{0.15cm}\underline {= 4.907\,{\rm bit/enquiry}}\hspace{0.05cm},$$
	\frac{{\rm lg}\hspace{0.1cm}30}{{\rm lg}\hspace{0.1cm}2} = \frac{1.477}{0.301} \hspace{0.15cm}		:$$I_{\rm G} \hspace{0.1cm} = \hspace{0.1cm} {\rm log}_2\hspace{0.1cm}\frac{30}{29} =\frac{{\rm lg}\hspace{0.1cm}1.034}{{\rm lg}\hspace{0.1cm}2} = \frac{1.477}{0.301} \hspace{0.15cm}\underline {= 0.049\,{\rm bit/enquiry}}\hspace{0.05cm}.$$
	\underline {= 4.907\,{\rm bit/enquiry}}\hspace{0.05cm},$$
	:$$I_{\rm G} \hspace{0.1cm} = \hspace{0.1cm} {\rm log}_2\hspace{0.1cm}\frac{30}{29} =
	\frac{{\rm lg}\hspace{0.1cm}1.034}{{\rm lg}\hspace{0.1cm}2} = \frac{1.477}{0.301} \hspace{0.15cm}
	\underline {= 0.049\,{\rm bit/enquiry}}\hspace{0.05cm}.$$


	'''(5)'''  The entropy  $H_{\rm P}$  is the average information content of the two events  $\rm B$  and  $\rm G$:		'''(5)'''  The entropy  $H_{\rm P}$  is the average information content of the two events  $\rm B$  and  $\rm G$:
	:$$H_{\rm P} = \frac{1}{30} \cdot 4.907 + \frac{29}{30} \cdot 0.049 = 0.164 + 0.047		:$$H_{\rm P} = \frac{1}{30} \cdot 4.907 + \frac{29}{30} \cdot 0.049 = 0.164 + 0.047\hspace{0.15cm}\underline {= 0.211\,{\rm bit/enquiry}}\hspace{0.05cm}.$$
	\hspace{0.15cm}
	\underline {= 0.211\,{\rm bit/enquiry}}\hspace{0.05cm}.$$
	*Although  (more precisely:  because)  event  $\rm B$  occurs less frequently than  $\rm G$, its contribution to entropy is much greater.		*Although  (more precisely:  because)  event  $\rm B$  occurs less frequently than  $\rm G$, its contribution to entropy is much greater.

Maintenance script: Add German interlanguage link

2026-03-16T12:30:10Z

Add German interlanguage link

← Older revision		Revision as of 14:30, 16 March 2026
Line 129:		Line 129:

	[[Category:Information Theory: Exercises\|^1.1 Memoryless Sources^]]		[[Category:Information Theory: Exercises\|^1.1 Memoryless Sources^]]
			[[de:Aufgaben:1.1_Wetterentropie]]

Guenter at 11:59, 10 August 2021

2021-08-10T11:59:33Z

← Older revision		Revision as of 13:59, 10 August 2021
Line 1:		Line 1:

	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Gedächtnislose Nachrichtenquellen~~		{{quiz-Header\|Buchseite=Information_Theory/Discrete_Memoryless_Sources
	}}		}}

Guenter at 11:35, 30 June 2021

2021-06-30T11:35:41Z

← Older revision		Revision as of 13:35, 30 June 2021
Line 3:		Line 3:
	}}		}}

	[[File:~~EN_Inf_A_1_1~~.png\|right\|frame\|Five different binary sources]]		[[File:EN_Inf_A_1_1_v2.png\|right\|frame\|Five different binary sources]]
	A weather station queries different regions every day and receives a message  $x$  back as a response in each case, namely		A weather station queries different regions every day and receives a message  $x$  back as a response in each case, namely

Guenter at 14:13, 16 June 2021

2021-06-16T14:13:08Z

← Older revision		Revision as of 16:13, 16 June 2021
Line 116:		Line 116:
	\hspace{0.15cm}		\hspace{0.15cm}
	\underline {= 0.211\,{\rm bit/enquiry}}\hspace{0.05cm}.$$		\underline {= 0.211\,{\rm bit/enquiry}}\hspace{0.05cm}.$$
	*Although  (more precisely:  because)  event  $\rm B$  occurs less frequently than  $\rm G$, its contribution to entropy is greater.		*Although  (more precisely:  because)  event  $\rm B$  occurs less frequently than  $\rm G$, its contribution to entropy is much greater.


	'''(6)'''  Statements <u>1 and 3</u> are correct:		'''(6)'''  Statements <u>1 and 3</u> are correct:
	*$\rm B$  and  $\rm G$  are indeed equally probable in the  "unknown"  file:   ~~the~~  $60$  symbols shown divide into  $30$ times  $\rm B$  and  $30$ times  $\rm G$.		*$\rm B$  and  $\rm G$  are indeed equally probable in the  "unknown"  file:   The  $60$  symbols shown divide into  $30$ times  $\rm B$  and  $30$ times  $\rm G$.
	*However, there are now strong statistical ties within the temporal sequence.  Long periods of good weather are usually followed by many bad days in a row.		*However, there are now strong statistical ties within the temporal sequence.  Long periods of good weather are usually followed by many bad days in a row.
	*Because of this statistical dependence within the  $\rm B/G$  sequence  $H_\text{U} = 0.722 \; \rm bit/enquiry$  is smaller than  $H_\text{M} = 1 \; \rm bit/enquiry$.		*Because of this statistical dependence within the  $\rm B/G$  sequence  $H_\text{U} = 0.722 \; \rm bit/enquiry$  is smaller than  $H_\text{M} = 1 \; \rm bit/enquiry$.

Guenter at 14:08, 16 June 2021

2021-06-16T14:08:17Z

← Older revision		Revision as of 16:08, 16 June 2021
Line 14:		Line 14:

	with the  base-2 logarithm		with the  base-2 logarithm
	:$${\rm log}_2\hspace{0.1cm}p=\frac{{\rm lg}\hspace{0.1cm}p}{{\rm lg}\hspace{0.1cm}2~~}\hspace{0.3cm} \left ( = {\rm ld}\hspace{0.1cm}p \right ) \hspace{0.05cm~~}.$$		:$${\rm log}_2\hspace{0.1cm}p=\frac{{\rm lg}\hspace{0.1cm}p}{{\rm lg}\hspace{0.1cm}2}.$$
	Here,  "lg"  denotes the logarithm to the base  $10$.  It should also be mentioned that the pseudo-unit  $\text{bit/enquiry}$  must be added in each case.		Here,  "lg"  denotes the logarithm to the base  $10$.  It should also be mentioned that the pseudo-unit  $\text{bit/enquiry}$  must be added in each case.

Line 73:		Line 73:


	{Which statements could be true for the file  "Unknown&"?		{Which statements could be true for the file  "Unknown"?
	\|type="[]"}		\|type="[]"}
	+ Events  $\rm B$  and  $\rm G$  are approximately equally probable.		+ Events  $\rm B$  and  $\rm G$  are approximately equally probable.

Guenter at 13:05, 15 June 2021

2021-06-15T13:05:10Z

← Older revision		Revision as of 15:05, 15 June 2021
Line 3:		Line 3:
	}}		}}

	[[File:~~Inf_A_1_1_vers2~~.png\|right\|frame\|Five different binary sources]]		[[File:EN_Inf_A_1_1.png\|right\|frame\|Five different binary sources]]
	A weather station queries different regions every day and receives a message  $x$  back as a response in each case, namely		A weather station queries different regions every day and receives a message  $x$  back as a response in each case, namely

Guenter at 12:17, 1 June 2021

2021-06-01T12:17:30Z

← Older revision		Revision as of 14:17, 1 June 2021
Line 39:		Line 39:
	*For the first four files it is assumed that the events  $\rm B$  and  $\rm G$  are statistically independent, a rather unrealistic assumption for weather practice.		*For the first four files it is assumed that the events  $\rm B$  and  $\rm G$  are statistically independent, a rather unrealistic assumption for weather practice.

	*The task was designed at a time when  [https://en.wikipedia.org/wiki/Greta_Thunberg Greta]  was just starting school.  We leave it to you to rename  "Paradise"  ~~tonbsp~~; "Hell".		*The task was designed at a time when  [https://en.wikipedia.org/wiki/Greta_Thunberg Greta]  was just starting school.  We leave it to you to rename  "Paradise"  to  "Hell".

Javier: Text replacement - "”" to """

2021-05-28T14:41:12Z

Text replacement - "”" to """

← Older revision		Revision as of 16:41, 28 May 2021
Line 68:		Line 68:


	{What is the entropy  (that is:  the average information content)  $H_{\rm P}$  of the file  "paradise~~”~~?  Interpret the result.		{What is the entropy  (that is:  the average information content)  $H_{\rm P}$  of the file  "paradise"?  Interpret the result.
	\|type="{}"}		\|type="{}"}
	$H_{\rm P}\ = \ $ { 0.211 3% } $\ \rm bit/enquiry$		$H_{\rm P}\ = \ $ { 0.211 3% } $\ \rm bit/enquiry$