Aufgaben:Exercise 3.3: Entropy of Ternary Quantities - Revision history

Maintenance script: Fix interlanguage link: resolve redirect chain

2026-03-16T15:57:51Z

Fix interlanguage link: resolve redirect chain

← Older revision		Revision as of 17:57, 16 March 2026
Line 18:		Line 18:
	*The lower bound  $H(X) = 0$  results when any probability  $p_\mu = 1$  and all others are zero.		*The lower bound  $H(X) = 0$  results when any probability  $p_\mu = 1$  and all others are zero.

	*The upper bound is to be derived here as in the lecture "Information Theory" by  [[Biographies_and_Bibliographies/~~Lehrstuhlinhaber_des_LNT~~#Prof._Dr._sc._techn._Gerhard_Kramer_~~.28seit_2010.29~~\|Gerhard Kramer]]  at the TU Munich:		*The upper bound is to be derived here as in the lecture "Information Theory" by  [[Biographies_and_Bibliographies/Chair_holders_of_the_LNT_since_1962#Prof._Dr._sc._techn._Gerhard_Kramer_(since_2010)\|Gerhard Kramer]]  at the TU Munich:
	[[File:P_ID2755__Inf_A_3_3_B_neu.png\|right\|frame\|Upper bound estimate for the natural logarithm]]		[[File:P_ID2755__Inf_A_3_3_B_neu.png\|right\|frame\|Upper bound estimate for the natural logarithm]]
	:* By extending the above equation by  $\|X\|$  in the numerator and denominator, using  $\log_2 \hspace{0.05cm}x= \ln(x)/\ln(2)$, we obtain:		:* By extending the above equation by  $\|X\|$  in the numerator and denominator, using  $\log_2 \hspace{0.05cm}x= \ln(x)/\ln(2)$, we obtain:
Line 34:		Line 34:
	Hints:		Hints:
	*The exercise belongs to the chapter  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen\|Some preliminary remarks on 2D random variables]].		*The exercise belongs to the chapter  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen\|Some preliminary remarks on 2D random variables]].
	*In particular, reference is made to the page  [[Information_Theory/~~Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen~~#~~Wahrscheinlichkeitsfunktion_und_Entropie~~\|Probability function and entropy]].		*In particular, reference is made to the page  [[Information_Theory/Some_Preliminary_Remarks_on_Two-Dimensional_Random_Variables#Probability_mass_function_and_entropy\|Probability function and entropy]].
	*The same constellation is assumed here as in  [[Aufgaben:Aufgabe_3.2:_Erwartungswertberechnungen\|Exercise 3.2]].		*The same constellation is assumed here as in  [[Aufgaben:Aufgabe_3.2:_Erwartungswertberechnungen\|Exercise 3.2]].

Line 115:		Line 115:

	[[Category:Information Theory: Exercises\|^3.1 General Information on 2D Random Variables^]]		[[Category:Information Theory: Exercises\|^3.1 General Information on 2D Random Variables^]]
	[[de:Aufgaben:3.~~03_Entropie_von_Ternärgrößen~~]]		[[de:Aufgaben:Aufgabe 3.3: Entropie von Ternärgrößen]]

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:48:08Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:48, 16 March 2026
Line 38:		Line 38:

	*The equation of the binary entropy function is:		*The equation of the binary entropy function is:
	:$$H_{\rm bin}(p) = p \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p} +		:$$H_{\rm bin}(p) = p \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p} +(1-p) \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{1-p} \hspace{0.05cm}.$$
	(1-p) \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{1-p} \hspace{0.05cm}.$$


Line 78:		Line 77:
	'''(1)'''  <u>Both statements are correct:</u>		'''(1)'''  <u>Both statements are correct:</u>
	*If we set  $p_3 = 0$ and formally  $p_1 = p$   ⇒    $p_2 = 1- p$, we get the binary entropy function		*If we set  $p_3 = 0$ and formally  $p_1 = p$   ⇒    $p_2 = 1- p$, we get the binary entropy function
	:$$H_{\rm bin}(p) = p \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p} +		:$$H_{\rm bin}(p) = p \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{p} +(1-p) \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{1-p} \hspace{0.05cm}.$$
	(1-p) \cdot {\rm log}_2 \hspace{0.1cm} \frac{1}{1-p} \hspace{0.05cm}.$$


	'''(2)'''  Only the <u>proposed solution 1</u> is correct:		'''(2)'''  Only the <u>proposed solution 1</u> is correct:
	*One can also put the binary entropy function into the following form because of  $\log(x) = \ln(x)/\ln(2)$ :		*One can also put the binary entropy function into the following form because of  $\log(x) = \ln(x)/\ln(2)$ :
	:$$H_{\rm bin}(p) = \frac{-1}{{\rm ln}(2)} \cdot \big [ p \cdot {\rm ln}(p) +		:$$H_{\rm bin}(p) = \frac{-1}{{\rm ln}(2)} \cdot \big [ p \cdot {\rm ln}(p) +(1-p) \cdot {\rm ln}(1-p) \big ] \hspace{0.05cm}.$$
	(1-p) \cdot {\rm ln}(1-p) \big ] \hspace{0.05cm}.$$
	*The first derivation leads to the result		*The first derivation leads to the result
	:$$\frac {{\rm d}H_{\rm bin}(p)}{{\rm d}p} = \frac{-1}{{\rm ln}(2)} \cdot \big [ {\rm ln}(p) + p \cdot \frac{1}{p} -		:$$\frac {{\rm d}H_{\rm bin}(p)}{{\rm d}p} = \frac{-1}{{\rm ln}(2)} \cdot \big [ {\rm ln}(p) + p \cdot \frac{1}{p} -{\rm ln}(1-p) - (1-p) \cdot \frac{1}{1-p} \big ] =\frac{1}{{\rm ln}(2)} \cdot \big [ {\rm ln}(1-p) - {\rm ln}(p) \big ] = {\rm log}_2 \hspace{0.1cm} \frac{1-p}{p} \hspace{0.05cm}.$$
	{\rm ln}(1-p) - (1-p) \cdot \frac{1}{1-p} \big ] =
	\frac{1}{{\rm ln}(2)} \cdot \big [ {\rm ln}(1-p) - {\rm ln}(p) \big ] = {\rm log}_2 \hspace{0.1cm} \frac{1-p}{p} \hspace{0.05cm}.$$
	*By setting this derivative to zero, we obtain the abscissa value  $p = 0.5$, which leads to the maximum of the entropy function:   $H_{\rm bin}(p =0.5) = 1$ bit <br>⇒   the proposed solution 2 is wrong.		*By setting this derivative to zero, we obtain the abscissa value  $p = 0.5$, which leads to the maximum of the entropy function:   $H_{\rm bin}(p =0.5) = 1$ bit <br>⇒   the proposed solution 2 is wrong.
	*By differentiating again, one obtains for the second derivative		*By differentiating again, one obtains for the second derivative
	:$$\frac {{\rm d}^2H_{\rm bin}(p)}{{\rm d}p^2} = \frac{1}{{\rm ln}(2)} \cdot \left		:$$\frac {{\rm d}^2H_{\rm bin}(p)}{{\rm d}p^2} = \frac{1}{{\rm ln}(2)} \cdot \left[ \frac{-1}{1-p} - \frac{1}{p} \right ] =\frac{-1}{{\rm ln}(2) \cdot p \cdot (1-p)} \hspace{0.05cm}.$$
	[ \frac{-1}{1-p} - \frac{1}{p} \right ] =
	\frac{-1}{{\rm ln}(2) \cdot p \cdot (1-p)} \hspace{0.05cm}.$$
	*This function is negative in the entire definition domain  $0 ≤ p ≤ 1$    ⇒   $H_{\rm bin}(p)$ is concave   ⇒   the proposed solution 1 is correct.		*This function is negative in the entire definition domain  $0 ≤ p ≤ 1$    ⇒   $H_{\rm bin}(p)$ is concave   ⇒   the proposed solution 1 is correct.

Line 103:		Line 96:
	* For  $p = 0$  one obtains the probability function  $P_X(X) = \big [\hspace{0.05cm}0\hspace{0.05cm}, \hspace{0.15cm} 1/2\hspace{0.05cm},\hspace{0.15cm} 1/2 \hspace{0.05cm} \big ]$   ⇒   $H(X) = 1$  bit.		* For  $p = 0$  one obtains the probability function  $P_X(X) = \big [\hspace{0.05cm}0\hspace{0.05cm}, \hspace{0.15cm} 1/2\hspace{0.05cm},\hspace{0.15cm} 1/2 \hspace{0.05cm} \big ]$   ⇒   $H(X) = 1$  bit.
	* The maximum under the condition  $p_3 = 1/2$  is obtained for  $p_1 = p_2 = 1/4$:		* The maximum under the condition  $p_3 = 1/2$  is obtained for  $p_1 = p_2 = 1/4$:
	:$$P_X(X) = \big [\hspace{0.05cm}1/4\hspace{0.05cm}, \hspace{0.05cm} 1/4\hspace{0.05cm},\hspace{0.05cm} 1/2 \hspace{0.05cm} \big ]		:$$P_X(X) = \big [\hspace{0.05cm}1/4\hspace{0.05cm}, \hspace{0.05cm} 1/4\hspace{0.05cm},\hspace{0.05cm} 1/2 \hspace{0.05cm} \big ]\hspace{0.3cm} \Rightarrow \hspace{0.3cm}{\rm Max} \ [H_{\rm B}(p)] = 1.5 \ \rm bit\hspace{0.05cm}.$$
	\hspace{0.3cm} \Rightarrow \hspace{0.3cm}
	{\rm Max} \ [H_{\rm B}(p)] = 1.5 \ \rm bit
	\hspace{0.05cm}.$$
	*In compact form,  $H_{\rm B}(p)$  with the constraint  $0 ≤ p ≤ 1/2$  can be represented as follows:		*In compact form,  $H_{\rm B}(p)$  with the constraint  $0 ≤ p ≤ 1/2$  can be represented as follows:
	:$$H_{\rm B}(p) = 1.0\,{\rm bit} + {1}/{2} \cdot H_{\rm bin}(2p)		:$$H_{\rm B}(p) = 1.0\,{\rm bit} + {1}/{2} \cdot H_{\rm bin}(2p)\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$


Line 116:		Line 105:
	:$$ {\rm Max} \ [H_{\rm G}(p)] = \log_2 (3)\ \text{bit}.$$		:$$ {\rm Max} \ [H_{\rm G}(p)] = \log_2 (3)\ \text{bit}.$$
	*In general, the entire curve in the range  $0 ≤ p ≤ 2/3$  can be expressed as follows:		*In general, the entire curve in the range  $0 ≤ p ≤ 2/3$  can be expressed as follows:
	:$$H_{\rm G}(p) = H_{\rm G}(p= 0) + {2}/{3} \cdot H_{\rm bin}(3p/2)		:$$H_{\rm G}(p) = H_{\rm G}(p= 0) + {2}/{3} \cdot H_{\rm bin}(3p/2)\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$
	*From the graph on the information page, one can also see that the following condition must be fulfilled:		*From the graph on the information page, one can also see that the following condition must be fulfilled:
	:$$H_{\rm G}(p = 0) + {2}/{3}= {\rm log}_2 \hspace{0.01cm} (3)		:$$H_{\rm G}(p = 0) + {2}/{3}= {\rm log}_2 \hspace{0.01cm} (3)\hspace{0.3cm} \Rightarrow \hspace{0.3cm}H_{\rm G}(p= 0) = 1.585 - 0.667 = 0.918 \,{\rm bit}\hspace{0.05cm}.$$
	\hspace{0.3cm} \Rightarrow \hspace{0.3cm}
	H_{\rm G}(p= 0) = 1.585 - 0.667 = 0.918 \,{\rm bit}
	\hspace{0.05cm}.$$
	*The second suggested solution is therefore wrong.  The same result can be obtained with the equation		*The second suggested solution is therefore wrong.  The same result can be obtained with the equation
	:$$H_{\rm G}(p = 0) = {1}/{3} \cdot {\rm log}_2 \hspace{0.01cm} (3)		:$$H_{\rm G}(p = 0) = {1}/{3} \cdot {\rm log}_2 \hspace{0.01cm} (3)+{2}/{3} \cdot {\rm log}_2 \hspace{0.01cm} (3/2) = {\rm log}_2 \hspace{0.01cm} (3) -2/3 \cdot{\rm log}_2 \hspace{0.01cm} (2)\hspace{0.05cm}.$$
	+{2}/{3} \cdot {\rm log}_2 \hspace{0.01cm} (3/2) = {\rm log}_2 \hspace{0.01cm} (3) -2/3 \cdot
	{\rm log}_2 \hspace{0.01cm} (2)
	\hspace{0.05cm}.$$
	{{ML-Fuß}}		{{ML-Fuß}}

Maintenance script: Add German interlanguage link

2026-03-16T12:31:46Z

Add German interlanguage link

← Older revision		Revision as of 14:31, 16 March 2026
Line 133:		Line 133:

	[[Category:Information Theory: Exercises\|^3.1 General Information on 2D Random Variables^]]		[[Category:Information Theory: Exercises\|^3.1 General Information on 2D Random Variables^]]
			[[de:Aufgaben:3.03_Entropie_von_Ternärgrößen]]

Noah at 08:13, 24 September 2021

2021-09-24T08:13:10Z

← Older revision		Revision as of 10:13, 24 September 2021
Line 1:		Line 1:

	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Einige Vorbemerkungen zu zweidimensionalen Zufallsgrößen~~		{{quiz-Header\|Buchseite=Information_Theory/Some_Preliminary_Remarks_on_Two-Dimensional_Random_Variables
	}}		}}

Noah at 20:05, 21 August 2021

2021-08-21T20:05:54Z

← Older revision		Revision as of 22:05, 21 August 2021
Line 33:		Line 33:

	Hints:		Hints:
	*The ~~task~~ belongs to the chapter  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen\|Some preliminary remarks on 2D random variables]].		*The exercise belongs to the chapter  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen\|Some preliminary remarks on 2D random variables]].
	*In particular, reference is made to the page  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Wahrscheinlichkeitsfunktion_und_Entropie\|Probability function and entropy]].		*In particular, reference is made to the page  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Wahrscheinlichkeitsfunktion_und_Entropie\|Probability function and entropy]].
	*The same constellation is assumed here as in  [[Aufgaben:Aufgabe_3.2:_Erwartungswertberechnungen\|Exercise 3.2]].		*The same constellation is assumed here as in  [[Aufgaben:Aufgabe_3.2:_Erwartungswertberechnungen\|Exercise 3.2]].

Noah at 13:26, 19 August 2021

2021-08-19T13:26:13Z

Show changes

Noah at 12:46, 19 August 2021

2021-08-19T12:46:31Z

← Older revision		Revision as of 14:46, 19 August 2021
Line 3:		Line 3:
	}}		}}

	[[File:P_ID2754__Inf_A_3_3.png\|right\|frame\|~~Vorgegebene Entropiefunktionen~~]]		[[File:P_ID2754__Inf_A_3_3.png\|right\|frame\|Given entropy functions]]
	~~Rechts sehen Sie die Entropiefunktionen~~  $H_{\rm R}(p)$,  $H_{\rm B}(p)$  ~~und~~  $H_{\rm G}(p)$, ~~wobei~~  $\rm R$  ~~für~~ "~~Rot~~" ~~steht~~,  $\rm B$  ~~für~~ "~~Blau~~" ~~und~~  $\rm G$  ~~für~~ "~~Grün~~".  ~~Die Wahrscheinlichkeitsfunktionen lauten für alle Zufallsgrößen~~:		On the right you see the entropy functions  $H_{\rm R}(p)$,  $H_{\rm B}(p)$  and  $H_{\rm G}(p)$, where  $\rm R$  stands for "red",  $\rm B$  for "blue" and  $\rm G$  for "green".  The probability functions are for all random variables:
	:$$P_X(X) = [\hspace{0.05cm}p_1\hspace{0.05cm},\ p_2\hspace{0.05cm},\ p_3\hspace{0.05cm}]\hspace{0.3cm}\hspace{0.3cm} \Rightarrow \hspace{0.3cm} \|X\| = 3\hspace{0.05cm}.$$		:$$P_X(X) = [\hspace{0.05cm}p_1\hspace{0.05cm},\ p_2\hspace{0.05cm},\ p_3\hspace{0.05cm}]\hspace{0.3cm}\hspace{0.3cm} \Rightarrow \hspace{0.3cm} \|X\| = 3\hspace{0.05cm}.$$
	~~Für den Fragebogen gilt der Zusammenhang~~  $p_1 = p$  ~~und~~  $p_2 = 1 - p_3- p$.		For the questionnaire, the relationship  $p_1 = p$  and  $p_2 = 1 - p_3- p$.

	~~Die Wahrscheinlichkeitsfunktion einer Zufallsgröße~~		The probability function of a random variable
	:$$X = \big \{\hspace{0.05cm}x_1\hspace{0.05cm}, \hspace{0.15cm} x_2\hspace{0.05cm},\hspace{0.15cm} \text{...}\hspace{0.1cm} ,\hspace{0.15cm} x_{\mu}\hspace{0.05cm}, \hspace{0.05cm}\text{...}\hspace{0.1cm} , \hspace{0.15cm} x_{M}\hspace{0.05cm}\big \}$$		:$$X = \big \{\hspace{0.05cm}x_1\hspace{0.05cm}, \hspace{0.15cm} x_2\hspace{0.05cm},\hspace{0.15cm} \text{...}\hspace{0.1cm} ,\hspace{0.15cm} x_{\mu}\hspace{0.05cm}, \hspace{0.05cm}\text{...}\hspace{0.1cm} , \hspace{0.15cm} x_{M}\hspace{0.05cm}\big \}$$
	~~mit dem Symbolumfang~~  $\|X\| = M$  ~~lautet allgemein~~:		with the symbolic range  $\|X\| = M$  is generally:
	:$$P_X(X) = [\hspace{0.05cm}p_1\hspace{0.05cm}, \hspace{0.15cm} p_2\hspace{0.05cm},\hspace{0.05cm} ...\hspace{0.1cm} ,\hspace{0.15cm} p_{\mu}\hspace{0.05cm}, \hspace{0.05cm}...\hspace{0.1cm} , \hspace{0.15cm} p_{M}\hspace{0.05cm}]\hspace{0.05cm}.$$		:$$P_X(X) = [\hspace{0.05cm}p_1\hspace{0.05cm}, \hspace{0.15cm} p_2\hspace{0.05cm},\hspace{0.05cm} ...\hspace{0.1cm} ,\hspace{0.15cm} p_{\mu}\hspace{0.05cm}, \hspace{0.05cm}...\hspace{0.1cm} , \hspace{0.15cm} p_{M}\hspace{0.05cm}]\hspace{0.05cm}.$$
	~~Die Entropie~~ (~~Unsicherheit~~) ~~dieser Zufallsgröße berechnet sich entsprechend der Gleichung~~		The entropy (uncertainty) of this random variable is calculated according to the equation
	:$$H(X) = {\rm E} \big [\log_2 \hspace{0.05cm} {1}/{P_X(X)} \big ]\hspace{0.05cm},$$		:$$H(X) = {\rm E} \big [\log_2 \hspace{0.05cm} {1}/{P_X(X)} \big ]\hspace{0.05cm},$$
	~~und liegt stets im Bereich~~  $0 \le H(X) \le \log_2 \hspace{0.05cm} \|X\|$.		and always lies in the range  $0 \le H(X) \le \log_2 \hspace{0.05cm} \|X\|$.

	*~~Die untere Schranke~~  $H(X) = 0$  ~~ergibt sich, wenn eine beliebige Wahrscheinlichkeit~~  $p_\mu = 1$  ~~ist und alle anderen Null sind~~.		*The lower bound  $H(X) = 0$  results when any probability  $p_\mu = 1$  and all others are zero.

	*~~Die obere Schranke soll hier wie~~ in ~~der Vorlesung~~ "Information Theory" ~~von~~  [[Biographies_and_Bibliographies/Lehrstuhlinhaber_des_LNT#Prof._Dr._sc._techn._Gerhard_Kramer_.28seit_2010.29\|Gerhard Kramer]]  ~~an der~~ TU ~~München hergeleitet werden~~:		*The upper bound is to be derived here as in the lecture "Information Theory" by  [[Biographies_and_Bibliographies/Lehrstuhlinhaber_des_LNT#Prof._Dr._sc._techn._Gerhard_Kramer_.28seit_2010.29\|Gerhard Kramer]]  at the TU Munich:
	[[File:P_ID2755__Inf_A_3_3_B_neu.png\|right\|frame\|~~Obere Abschätzung für den natürlichen Logarithmus~~]]		[[File:P_ID2755__Inf_A_3_3_B_neu.png\|right\|frame\|Upper bound estimate for the natural logarithm]]
	:* ~~Durch Erweiterung obiger Gleichung um~~  $\|X\|$  in ~~Zähler und Nenner erhält man unter Verwendung von~~  $\log_2 \hspace{0.05cm}x= \ln(x)/\ln(2)$:		:* By extending the above equation by  $\|X\|$  in the numerator and denominator, using  $\log_2 \hspace{0.05cm}x= \ln(x)/\ln(2)$, we obtain:
	::$$H(X) = \frac{1}{{\rm ln}(2)}\cdot {\rm E} \left [{\rm ln} \hspace{0.1cm} \frac{1}{\|X\| \cdot P_X(X)} \right ] + {\rm log}_2 \hspace{0.1cm}\|X\| \hspace{0.05cm}.$$		::$$H(X) = \frac{1}{{\rm ln}(2)}\cdot {\rm E} \left [{\rm ln} \hspace{0.1cm} \frac{1}{\|X\| \cdot P_X(X)} \right ] + {\rm log}_2 \hspace{0.1cm}\|X\| \hspace{0.05cm}.$$
	:* ~~Wie aus nebenstehender Grafik hervorgeht~~, ~~gilt die Abschätzung~~  $\ln(x) \le x-1$  ~~mit der Identität für~~  $x=1$.  ~~Somit kann geschrieben werden~~:		:* As can be seen from the graph opposite, the estimation  $\ln(x) \le x-1$  holds with the identity for  $x=1$.  Thus, it can be written:
	::$$H(X) \le \frac{1}{{\rm ln}(2)}\cdot {\rm E} \left [\frac{1}{\|X\| \cdot P_X(X)} -1 \right ] + {\rm log}_2 \hspace{0.1cm}\|X\| \hspace{0.05cm}.$$		::$$H(X) \le \frac{1}{{\rm ln}(2)}\cdot {\rm E} \left [\frac{1}{\|X\| \cdot P_X(X)} -1 \right ] + {\rm log}_2 \hspace{0.1cm}\|X\| \hspace{0.05cm}.$$
	:* In ~~der~~  [[Aufgaben:3.2_Erwartungswertberechnungen\|~~Aufgabe~~ 3.2]]  ~~wurde für den Fall  $p_\mu \ne 0$  für alle  $\mu$  der Erwartungswert~~  ${\rm E} \big [\log_2 \hspace{0.05cm} {1}/{P_X(X)} \big ] =\|X\|$  ~~berechnet~~.  ~~Damit verschwindet der erste Term und man erhält das bekannte Ergebnis~~:		:* In   [[Aufgaben:3.2_Erwartungswertberechnungen\|exercise 3.2]]  the expected value  ${\rm E} \big [\log_2 \hspace{0.05cm} {1}/{P_X(X)} \big ] =\|X\|$  was calculated for the case  $p_\mu \ne 0$  for all  $\mu$ .  Thus, the first term disappears and the known result is obtained:
	::$$H(X) \le {\rm log}_2 \hspace{0.1cm}\|X\| \hspace{0.05cm}.$$		::$$H(X) \le {\rm log}_2 \hspace{0.1cm}\|X\| \hspace{0.05cm}.$$

Line 32:		Line 32:


	~~''Hinweise~~:''		Hints:
	*Die Aufgabe gehört zum Kapitel  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen\|Einige Vorbemerkungen zu den 2D-Zufallsgrößen]].		*Die Aufgabe gehört zum Kapitel  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen\|Einige Vorbemerkungen zu den 2D-Zufallsgrößen]].
	*Insbesondere wird Bezug genommen auf die Seite  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Wahrscheinlichkeitsfunktion_und_Entropie\|Wahrscheinlichkeitsfunktion undEntropie]].		*Insbesondere wird Bezug genommen auf die Seite  [[Information_Theory/Einige_Vorbemerkungen_zu_zweidimensionalen_Zufallsgrößen#Wahrscheinlichkeitsfunktion_und_Entropie\|Wahrscheinlichkeitsfunktion undEntropie]].

Guenter at 13:32, 16 July 2021

2021-07-16T13:32:28Z

← Older revision		Revision as of 15:32, 16 July 2021
Line 132:		Line 132:


	[[Category:Information Theory: Exercises\|^3.1 ~~Allgemeines zu~~ 2D~~-Zufallsgrößen~~^]]		[[Category:Information Theory: Exercises\|^3.1 General Information on 2D Random Variables^]]

Guenter: Guenter moved page Aufgaben:Aufgabe 3.3: Entropie von Ternärgrößen to Aufgaben:Exercise 3.3: Entropy of Ternary Quantities

2021-07-16T13:20:57Z

Guenter moved page Aufgaben:Aufgabe 3.3: Entropie von Ternärgrößen to Aufgaben:Exercise 3.3: Entropy of Ternary Quantities

← Older revision	Revision as of 15:20, 16 July 2021
(No difference)

Javier: Text replacement - "”" to """

2021-05-28T14:39:04Z

Text replacement - "”" to """

← Older revision		Revision as of 16:39, 28 May 2021
Line 4:		Line 4:

	[[File:P_ID2754__Inf_A_3_3.png\|right\|frame\|Vorgegebene Entropiefunktionen]]		[[File:P_ID2754__Inf_A_3_3.png\|right\|frame\|Vorgegebene Entropiefunktionen]]
	Rechts sehen Sie die Entropiefunktionen  $H_{\rm R}(p)$,  $H_{\rm B}(p)$  und  $H_{\rm G}(p)$, wobei  $\rm R$  für "Rot~~”~~ steht,  $\rm B$  für "Blau~~”~~ und  $\rm G$  für "Grün~~”~~.  Die Wahrscheinlichkeitsfunktionen lauten für alle Zufallsgrößen:		Rechts sehen Sie die Entropiefunktionen  $H_{\rm R}(p)$,  $H_{\rm B}(p)$  und  $H_{\rm G}(p)$, wobei  $\rm R$  für "Rot" steht,  $\rm B$  für "Blau" und  $\rm G$  für "Grün".  Die Wahrscheinlichkeitsfunktionen lauten für alle Zufallsgrößen:
	:$$P_X(X) = [\hspace{0.05cm}p_1\hspace{0.05cm},\ p_2\hspace{0.05cm},\ p_3\hspace{0.05cm}]\hspace{0.3cm}\hspace{0.3cm} \Rightarrow \hspace{0.3cm} \|X\| = 3\hspace{0.05cm}.$$		:$$P_X(X) = [\hspace{0.05cm}p_1\hspace{0.05cm},\ p_2\hspace{0.05cm},\ p_3\hspace{0.05cm}]\hspace{0.3cm}\hspace{0.3cm} \Rightarrow \hspace{0.3cm} \|X\| = 3\hspace{0.05cm}.$$
	Für den Fragebogen gilt der Zusammenhang  $p_1 = p$  und  $p_2 = 1 - p_3- p$.		Für den Fragebogen gilt der Zusammenhang  $p_1 = p$  und  $p_2 = 1 - p_3- p$.
Line 18:		Line 18:
	*Die untere Schranke  $H(X) = 0$  ergibt sich, wenn eine beliebige Wahrscheinlichkeit  $p_\mu = 1$  ist und alle anderen Null sind.		*Die untere Schranke  $H(X) = 0$  ergibt sich, wenn eine beliebige Wahrscheinlichkeit  $p_\mu = 1$  ist und alle anderen Null sind.

	*Die obere Schranke soll hier wie in der Vorlesung "Information Theory~~”~~ von  [[Biographies_and_Bibliographies/Lehrstuhlinhaber_des_LNT#Prof._Dr._sc._techn._Gerhard_Kramer_.28seit_2010.29\|Gerhard Kramer]]  an der TU München hergeleitet werden:		*Die obere Schranke soll hier wie in der Vorlesung "Information Theory" von  [[Biographies_and_Bibliographies/Lehrstuhlinhaber_des_LNT#Prof._Dr._sc._techn._Gerhard_Kramer_.28seit_2010.29\|Gerhard Kramer]]  an der TU München hergeleitet werden:
	[[File:P_ID2755__Inf_A_3_3_B_neu.png\|right\|frame\|Obere Abschätzung für den natürlichen Logarithmus]]		[[File:P_ID2755__Inf_A_3_3_B_neu.png\|right\|frame\|Obere Abschätzung für den natürlichen Logarithmus]]
	:* Durch Erweiterung obiger Gleichung um  $\|X\|$  in Zähler und Nenner erhält man unter Verwendung von  $\log_2 \hspace{0.05cm}x= \ln(x)/\ln(2)$:		:* Durch Erweiterung obiger Gleichung um  $\|X\|$  in Zähler und Nenner erhält man unter Verwendung von  $\log_2 \hspace{0.05cm}x= \ln(x)/\ln(2)$: