Aufgaben:Exercise 4.2: Triangular PDF - Revision history

Maintenance script: Fix interlanguage link: resolve redirect chain

2026-03-16T15:56:36Z

Fix interlanguage link: resolve redirect chain

← Older revision		Revision as of 17:56, 16 March 2026
Line 102:		Line 102:

	[[Category:Information Theory: Exercises\|^4.1 Differential Entropy^]]		[[Category:Information Theory: Exercises\|^4.1 Differential Entropy^]]
	[[de:Aufgaben:4.~~02_Dreieckförmige_WDF~~]]		[[de:Aufgaben:Aufgabe 4.2: Dreieckförmige WDF]]

Maintenance script: Join 1 split formula blocks

2026-03-16T14:16:50Z

Join 1 split formula blocks

← Older revision		Revision as of 16:16, 16 March 2026
Line 67:		Line 67:
	:$$h_{\rm nat}(X) = \hspace{0.1cm} - \int_{0}^{1} \hspace{0.1cm} C \cdot x \cdot {\rm ln} \hspace{0.1cm} \big[ C \cdot x \big] \hspace{0.1cm}{\rm d}x =\hspace{0.1cm} - \hspace{0.1cm}\frac{1}{C} \cdot \int_{0}^{C} \hspace{0.1cm} \xi \cdot {\rm ln} \hspace{0.1cm} [ \xi ] \hspace{0.1cm}{\rm d}\xi= - \hspace{0.1cm}\frac{\xi^2}{C} \cdot\left [ \frac{{\rm ln} \hspace{0.1cm} (\xi)}{2} -\frac{1}{4}\right ]_{\xi = 0}^{\xi = C}\hspace{0.05cm}$$		:$$h_{\rm nat}(X) = \hspace{0.1cm} - \int_{0}^{1} \hspace{0.1cm} C \cdot x \cdot {\rm ln} \hspace{0.1cm} \big[ C \cdot x \big] \hspace{0.1cm}{\rm d}x =\hspace{0.1cm} - \hspace{0.1cm}\frac{1}{C} \cdot \int_{0}^{C} \hspace{0.1cm} \xi \cdot {\rm ln} \hspace{0.1cm} [ \xi ] \hspace{0.1cm}{\rm d}\xi= - \hspace{0.1cm}\frac{\xi^2}{C} \cdot\left [ \frac{{\rm ln} \hspace{0.1cm} (\xi)}{2} -\frac{1}{4}\right ]_{\xi = 0}^{\xi = C}\hspace{0.05cm}$$
	*Here the indefinite integral given in the front was used.  After inserting the limits, considering  $C=2$,  we obtain::		*Here the indefinite integral given in the front was used.  After inserting the limits, considering  $C=2$,  we obtain::
	:$$h_{\rm nat}(X) =		:$$h_{\rm nat}(X) =- C/2 \cdot\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2\big ]= - {\rm ln} \hspace{0.1cm} (2) + 1/2 =- {\rm ln} \hspace{0.1cm} (2)+ 1/2 \cdot {\rm ln} \hspace{0.1cm} ({\rm e})= {\rm ln} \hspace{0.1cm} (\sqrt{\rm e}/2)\hspace{0.05cm} = - 0.193\hspace{0.3cm} \Rightarrow\hspace{0.3cm}h(X)\hspace{0.15cm}\underline {= - 0.193\,{\rm nat}}\hspace{0.05cm}.$$
	- C/2 \cdot
	\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2
	\big ]
	= - {\rm ln} \hspace{0.1cm} (2) + 1/2 =
	- {\rm ln} \hspace{0.1cm} (2)
	+ 1/2 \cdot {\rm ln} \hspace{0.1cm} ({\rm e})
	= {\rm ln} \hspace{0.1cm} (\sqrt{\rm e}/2)\hspace{0.05cm} = - 0.193
	\hspace{0.3cm} \Rightarrow\hspace{0.3cm}
	h(X)
	\hspace{0.15cm}\underline {= - 0.193\,{\rm nat}}
	\hspace{0.05cm}.$$

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:48:33Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:48, 16 March 2026
Line 65:		Line 65:

	*Since the differential entropy is sought in  "nat",  we use the natural logarithm.  With the substitution  $\xi = C \cdot x$  we obtain:		*Since the differential entropy is sought in  "nat",  we use the natural logarithm.  With the substitution  $\xi = C \cdot x$  we obtain:
	:$$h_{\rm nat}(X) = \hspace{0.1cm} - \int_{0}^{1} \hspace{0.1cm} C \cdot x \cdot {\rm ln} \hspace{0.1cm} \big[ C \cdot x \big] \hspace{0.1cm}{\rm d}x =		:$$h_{\rm nat}(X) = \hspace{0.1cm} - \int_{0}^{1} \hspace{0.1cm} C \cdot x \cdot {\rm ln} \hspace{0.1cm} \big[ C \cdot x \big] \hspace{0.1cm}{\rm d}x =\hspace{0.1cm} - \hspace{0.1cm}\frac{1}{C} \cdot \int_{0}^{C} \hspace{0.1cm} \xi \cdot {\rm ln} \hspace{0.1cm} [ \xi ] \hspace{0.1cm}{\rm d}\xi= - \hspace{0.1cm}\frac{\xi^2}{C} \cdot\left [ \frac{{\rm ln} \hspace{0.1cm} (\xi)}{2} -\frac{1}{4}\right ]_{\xi = 0}^{\xi = C}\hspace{0.05cm}$$
	\hspace{0.1cm} - \hspace{0.1cm}\frac{1}{C} \cdot \int_{0}^{C} \hspace{0.1cm} \xi \cdot {\rm ln} \hspace{0.1cm} [ \xi ] \hspace{0.1cm}{\rm d}\xi
	= - \hspace{0.1cm}\frac{\xi^2}{C} \cdot
	\left [ \frac{{\rm ln} \hspace{0.1cm} (\xi)}{2} -
	\frac{1}{4}\right ]_{\xi = 0}^{\xi = C}
	\hspace{0.05cm}$$
	*Here the indefinite integral given in the front was used.  After inserting the limits, considering  $C=2$,  we obtain::		*Here the indefinite integral given in the front was used.  After inserting the limits, considering  $C=2$,  we obtain::
	:$$h_{\rm nat}(X) =		:$$h_{\rm nat}(X) =
Line 102:		Line 97:
	*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.		*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.
	*Now the height  $C = 1$  instead of  $C = 2$  has to be considered:		*Now the height  $C = 1$  instead of  $C = 2$  has to be considered:
	:$$I_{\rm neg} =- C/2 \cdot		:$$I_{\rm neg} =- C/2 \cdot\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2\big ] = -1/2 \cdot\big [ {\rm ln} \hspace{0.1cm} (1) - 1/2 \cdot {\rm ln} \hspace{0.1cm} ({\rm e}) \big ]= 1/4 \cdot{\rm ln} \hspace{0.1cm} ({\rm e})\hspace{0.05cm}.$$
	\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2
	\big ] = -1/2 \cdot
	\big [ {\rm ln} \hspace{0.1cm} (1) - 1/2 \cdot {\rm ln} \hspace{0.1cm} ({\rm e}) \big ]= 1/4 \cdot
	{\rm ln} \hspace{0.1cm} ({\rm e})
	\hspace{0.05cm}.$$

	*The second integrand is identical to the first except for a shift and reflection.  Moreover, the integration intervals do not overlap   ⇒   $I_{\rm pos} = I_{\rm neg}$:		*The second integrand is identical to the first except for a shift and reflection.  Moreover, the integration intervals do not overlap   ⇒   $I_{\rm pos} = I_{\rm neg}$:

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:31:08Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:31, 16 March 2026
Line 5:		Line 5:
	Two probability density functions  $\rm (PDF)$  with triangular shapes are considered.		Two probability density functions  $\rm (PDF)$  with triangular shapes are considered.
	* The random variable  $X$  is limited to the range from  $0$  to  $1$ ,  and it holds for the PDF (upper sketch):		* The random variable  $X$  is limited to the range from  $0$  to  $1$ ,  and it holds for the PDF (upper sketch):
	:$$f_X(x) = \left\{ \begin{array}{c} 2x \\ 0 \\ \end{array} \right. \begin{array}{*{20}c} {\rm{f\ddot{u}r}} \hspace{0.1cm} 0 \le x \le 1 \\ {\rm else} \\ \end{array}		:$$f_X(x) = \left\{ \begin{array}{c} 2x \\ 0 \\ \end{array} \right. \begin{array}{*{20}c} {\rm{f\ddot{u}r}} \hspace{0.1cm} 0 \le x \le 1 \\ {\rm else} \\ \end{array}\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$
	* According to the lower sketch, the random variable  $Y$  has the following PDF:		* According to the lower sketch, the random variable  $Y$  has the following PDF:
	:$$f_Y(y) = \left\{ \begin{array}{c} 1 - \|\hspace{0.03cm}y\hspace{0.03cm}\| \\ 0 \\ \end{array} \right. \begin{array}{*{20}c} {\rm{f\ddot{u}r}} \hspace{0.1cm} \|\hspace{0.03cm}y\hspace{0.03cm}\| \le 1 \\ {\rm else} \\ \end{array}		:$$f_Y(y) = \left\{ \begin{array}{c} 1 - \|\hspace{0.03cm}y\hspace{0.03cm}\| \\ 0 \\ \end{array} \right. \begin{array}{*{20}c} {\rm{f\ddot{u}r}} \hspace{0.1cm} \|\hspace{0.03cm}y\hspace{0.03cm}\| \le 1 \\ {\rm else} \\ \end{array}\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$

	For both random variables, the  [[Information_Theory/Differentielle_Entropie\|differential entropy]]  is to be determined in each case.		For both random variables, the  [[Information_Theory/Differentielle_Entropie\|differential entropy]]  is to be determined in each case.

	For example, the corresponding equation for the random variable  $X$  is:		For example, the corresponding equation for the random variable  $X$  is:
	:$$h(X) =		:$$h(X) =\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm} f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [ f_X(x) \big ] \hspace{0.1cm}{\rm d}x\hspace{0.6cm}{\rm with}\hspace{0.6cm} {\rm supp}(f_X) = \{ x\text{:} \ f_X(x) > 0 \}\hspace{0.05cm}.$$
	\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm} f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [ f_X(x) \big ] \hspace{0.1cm}{\rm d}x
	\hspace{0.6cm}{\rm with}\hspace{0.6cm} {\rm supp}(f_X) = \{ x\text{:} \ f_X(x) > 0 \}
	\hspace{0.05cm}.$$
	*If the  "natural logarithm",  the pseudo-unit  "nat"  must be added.		*If the  "natural logarithm",  the pseudo-unit  "nat"  must be added.
	*If, on the other hand, the result is asked in  "bit"  then the  "dual logarithm"   ⇒   "$\log_2$"  is to be used.		*If, on the other hand, the result is asked in  "bit"  then the  "dual logarithm"   ⇒   "$\log_2$"  is to be used.
Line 37:		Line 32:

	*Given the following indefinite integral:		*Given the following indefinite integral:
	:$$\int \xi \cdot {\rm ln} \hspace{0.1cm} (\xi)\hspace{0.1cm}{\rm d}\xi =		:$$\int \xi \cdot {\rm ln} \hspace{0.1cm} (\xi)\hspace{0.1cm}{\rm d}\xi =\xi^2 \cdot \big [1/2 \cdot {{\rm ln} \hspace{0.1cm} (\xi)} -{1}/{4}\big ] \hspace{0.05cm}.$$
	\xi^2 \cdot \big [1/2 \cdot {{\rm ln} \hspace{0.1cm} (\xi)} -
	{1}/{4}\big ] \hspace{0.05cm}.$$


Line 68:		Line 61:
	{{ML-Kopf}}		{{ML-Kopf}}
	'''(1)'''  For the probability density function, in the range  $0 \le X \le 1$ , it is agreed that:		'''(1)'''  For the probability density function, in the range  $0 \le X \le 1$ , it is agreed that:
	:$$f_X(x) = 2x = C \cdot x		:$$f_X(x) = 2x = C \cdot x\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$
	*Here we have replaced  "2"  by  $C$    ⇒   generalization in order to be able to use the following calculation again in subtask  $(3)$ .		*Here we have replaced  "2"  by  $C$    ⇒   generalization in order to be able to use the following calculation again in subtask  $(3)$ .

Line 98:		Line 90:
	[[File:P_ID2866__Inf_A_4_2c.png\|right\|frame\|To calculate  $h(Y)$]]		[[File:P_ID2866__Inf_A_4_2c.png\|right\|frame\|To calculate  $h(Y)$]]

	:$$h_{\rm bit}(X) = \frac{h_{\rm nat}(X)}{{\rm ln} \hspace{0.1cm} (2)\,{\rm nat/bit}} = - 0.279		:$$h_{\rm bit}(X) = \frac{h_{\rm nat}(X)}{{\rm ln} \hspace{0.1cm} (2)\,{\rm nat/bit}} = - 0.279\hspace{0.3cm} \Rightarrow\hspace{0.3cm}h(X)\hspace{0.15cm}\underline {= - 0.279\,{\rm bit}}\hspace{0.05cm}.$$
	\hspace{0.3cm} \Rightarrow\hspace{0.3cm}
	h(X)
	\hspace{0.15cm}\underline {= - 0.279\,{\rm bit}}
	\hspace{0.05cm}.$$
	*You can save this conversion if you directly replace  $(1)$  direct  "ln"  by  "log<sub>2</sub>"  already in the analytical result of subtask:		*You can save this conversion if you directly replace  $(1)$  direct  "ln"  by  "log<sub>2</sub>"  already in the analytical result of subtask:

	:$$h(X) = \ {\rm log}_2 \hspace{0.1cm} (\sqrt{\rm e}/2)\hspace{0.05cm}, \hspace{1.3cm}		:$$h(X) = \ {\rm log}_2 \hspace{0.1cm} (\sqrt{\rm e}/2)\hspace{0.05cm}, \hspace{1.3cm}{\rm pseudo-unit\hspace{-0.1cm}:\hspace{0.15cm} bit}\hspace{0.05cm}.$$
	{\rm pseudo-unit\hspace{-0.1cm}:\hspace{0.15cm} bit}
	\hspace{0.05cm}.$$



	'''(3)'''  We again use the natural logarithm and divide the integral into two partial integrals:		'''(3)'''  We again use the natural logarithm and divide the integral into two partial integrals:
	:$$h(Y) =		:$$h(Y) =\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}( \hspace{-0.03cm}f_Y)} \hspace{-0.35cm} f_Y(y) \cdot {\rm ln} \hspace{0.1cm} \big[ f_Y(y) \big] \hspace{0.1cm}{\rm d}y = I_{\rm neg} + I_{\rm pos}\hspace{0.05cm}.$$
	\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}
	\hspace{0.03cm}( \hspace{-0.03cm}f_Y)} \hspace{-0.35cm} f_Y(y) \cdot {\rm ln} \hspace{0.1cm} \big[ f_Y(y) \big] \hspace{0.1cm}{\rm d}y = I_{\rm neg} + I_{\rm pos}
	\hspace{0.05cm}.$$

	*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.		*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.
Line 127:		Line 110:

	*The second integrand is identical to the first except for a shift and reflection.  Moreover, the integration intervals do not overlap   ⇒   $I_{\rm pos} = I_{\rm neg}$:		*The second integrand is identical to the first except for a shift and reflection.  Moreover, the integration intervals do not overlap   ⇒   $I_{\rm pos} = I_{\rm neg}$:
	:$$h_{\rm nat}(Y) = 2 \cdot I_{\rm neg} = 1/2 \cdot		:$$h_{\rm nat}(Y) = 2 \cdot I_{\rm neg} = 1/2 \cdot{\rm ln} \hspace{0.1cm} ({\rm e}) = {\rm ln} \hspace{0.1cm} (\sqrt{\rm e}) \hspace{0.3cm}\Rightarrow\hspace{0.3cm}h_{\rm bit}(Y) = {\rm log}_2 \hspace{0.1cm} (\sqrt{\rm e})\hspace{0.3cm} \Rightarrow\hspace{0.3cm}h(Y) = {\rm log}_2 \hspace{0.1cm} (1.649)\hspace{0.15cm}\underline {= 0.721\,{\rm bit}}\hspace{0.05cm}.$$
	{\rm ln} \hspace{0.1cm} ({\rm e}) = {\rm ln} \hspace{0.1cm} (\sqrt{\rm e}) \hspace{0.3cm}
	\Rightarrow\hspace{0.3cm}h_{\rm bit}(Y) = {\rm log}_2 \hspace{0.1cm} (\sqrt{\rm e})
	\hspace{0.3cm} \Rightarrow\hspace{0.3cm}
	h(Y) = {\rm log}_2 \hspace{0.1cm} (1.649)\hspace{0.15cm}\underline {= 0.721\,{\rm bit}}\hspace{0.05cm}.$$


Line 138:		Line 117:
	:$$h(Z) = h(A \cdot Y) = h(Y) + {\rm log}_2 \hspace{0.1cm} (A) \hspace{0.05cm}.$$		:$$h(Z) = h(A \cdot Y) = h(Y) + {\rm log}_2 \hspace{0.1cm} (A) \hspace{0.05cm}.$$
	*Thus, from the requiremen  $h(Z) = 1 \ \rm bit$  and the result of subtask  $(3)$  follows:		*Thus, from the requiremen  $h(Z) = 1 \ \rm bit$  and the result of subtask  $(3)$  follows:
	:$${\rm log}_2 \hspace{0.1cm} (A) = 1\,{\rm bit} - 0.721 \,{\rm bit} = 0.279 \,{\rm bit}		:$${\rm log}_2 \hspace{0.1cm} (A) = 1\,{\rm bit} - 0.721 \,{\rm bit} = 0.279 \,{\rm bit}\hspace{0.3cm} \Rightarrow\hspace{0.3cm} A = 2^{0.279}\hspace{0.15cm}\underline{= 1.213}\hspace{0.05cm}.$$
	\hspace{0.3cm} \Rightarrow\hspace{0.3cm} A = 2^{0.279}\hspace{0.15cm}\underline
	{= 1.213}
	\hspace{0.05cm}.$$

	{{ML-Fuß}}		{{ML-Fuß}}

Maintenance script: Add German interlanguage link

2026-03-16T12:32:51Z

Add German interlanguage link

← Older revision		Revision as of 14:32, 16 March 2026
Line 147:		Line 147:

	[[Category:Information Theory: Exercises\|^4.1 Differential Entropy^]]		[[Category:Information Theory: Exercises\|^4.1 Differential Entropy^]]
			[[de:Aufgaben:4.02_Dreieckförmige_WDF]]

Javier: Text replacement - "value-continuous" to "continuous"

2021-10-11T08:27:01Z

Text replacement - "value-continuous" to "continuous"

← Older revision		Revision as of 10:27, 11 October 2021
Line 34:		Line 34:
	Hints:		Hints:
	*The task belongs to the chapter  [[Information_Theory/Differentielle_Entropie\|Differential Entropy]].		*The task belongs to the chapter  [[Information_Theory/Differentielle_Entropie\|Differential Entropy]].
	*Useful hints for solving this task and further information on ~~value-~~continuous random variables can be found in the third chapter "Continuous Random Variables" of the book  [[Theory of Stochastic Signals]].		*Useful hints for solving this task and further information on continuous random variables can be found in the third chapter "Continuous Random Variables" of the book  [[Theory of Stochastic Signals]].

	*Given the following indefinite integral:		*Given the following indefinite integral:

Noah at 10:38, 27 September 2021

2021-09-27T10:38:08Z

← Older revision		Revision as of 12:38, 27 September 2021
Line 1:		Line 1:
	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Differentielle Entropie~~		{{quiz-Header\|Buchseite=Information_Theory/Differential_Entropy
	}}		}}

Guenter at 09:47, 25 September 2021

2021-09-25T09:47:03Z

Guenter at 09:45, 25 September 2021

2021-09-25T09:45:06Z

← Older revision		Revision as of 11:45, 25 September 2021
Line 16:		Line 16:
	:$$h(X) =		:$$h(X) =
	\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm} f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [ f_X(x) \big ] \hspace{0.1cm}{\rm d}x		\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm} f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [ f_X(x) \big ] \hspace{0.1cm}{\rm d}x
	\hspace{0.6cm}{\rm ~~mit~~}\hspace{0.6cm} {\rm supp}(f_X) = \{ x\text{:} \ f_X(x) > 0 \}		\hspace{0.6cm}{\rm with}\hspace{0.6cm} {\rm supp}(f_X) = \{ x\text{:} \ f_X(x) > 0 \}
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$
	*If the  "natural logarithm",  the pseudo-unit  "nat"  must be added.		*If the  "natural logarithm",  the pseudo-unit  "nat"  must be added.
Line 70:		Line 70:
	:$$f_X(x) = 2x = C \cdot x		:$$f_X(x) = 2x = C \cdot x
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$
	*Here we have replaced "2" by  $C$    ⇒   generalization in order to be able to use the following calculation again in subtask  $(3)$ .		*Here we have replaced  "2"  by  $C$    ⇒   generalization in order to be able to use the following calculation again in subtask  $(3)$ .

	*Since the differential entropy is sought in "nat", we use the natural logarithm.  With the substitution  $\xi = C \cdot x$  we obtain:		*Since the differential entropy is sought in  "nat",  we use the natural logarithm.  With the substitution  $\xi = C \cdot x$  we obtain:
	:$$h_{\rm nat}(X) = \hspace{0.1cm} - \int_{0}^{1} \hspace{0.1cm} C \cdot x \cdot {\rm ln} \hspace{0.1cm} \big[ C \cdot x \big] \hspace{0.1cm}{\rm d}x =		:$$h_{\rm nat}(X) = \hspace{0.1cm} - \int_{0}^{1} \hspace{0.1cm} C \cdot x \cdot {\rm ln} \hspace{0.1cm} \big[ C \cdot x \big] \hspace{0.1cm}{\rm d}x =
	\hspace{0.1cm} - \hspace{0.1cm}\frac{1}{C} \cdot \int_{0}^{C} \hspace{0.1cm} \xi \cdot {\rm ln} \hspace{0.1cm} [ \xi ] \hspace{0.1cm}{\rm d}\xi		\hspace{0.1cm} - \hspace{0.1cm}\frac{1}{C} \cdot \int_{0}^{C} \hspace{0.1cm} \xi \cdot {\rm ln} \hspace{0.1cm} [ \xi ] \hspace{0.1cm}{\rm d}\xi
Line 79:		Line 79:
	\frac{1}{4}\right ]_{\xi = 0}^{\xi = C}		\frac{1}{4}\right ]_{\xi = 0}^{\xi = C}
	\hspace{0.05cm}$$		\hspace{0.05cm}$$
	*Here the indefinite integral given in the front was used.  After inserting the limits, considering  $C=2$, we obtain::		*Here the indefinite integral given in the front was used.  After inserting the limits, considering  $C=2$,  we obtain::
	:$$h_{\rm nat}(X) =		:$$h_{\rm nat}(X) =
	- C/2 \cdot		- C/2 \cdot
Line 96:		Line 96:

	'''(2)'''  In general:		'''(2)'''  In general:
			[[File:P_ID2866__Inf_A_4_2c.png\|right\|frame\|To calculate  $h(Y)$]]

	:$$h_{\rm bit}(X) = \frac{h_{\rm nat}(X)}{{\rm ln} \hspace{0.1cm} (2)\,{\rm nat/bit}} = - 0.279		:$$h_{\rm bit}(X) = \frac{h_{\rm nat}(X)}{{\rm ln} \hspace{0.1cm} (2)\,{\rm nat/bit}} = - 0.279
	\hspace{0.3cm} \Rightarrow\hspace{0.3cm}		\hspace{0.3cm} \Rightarrow\hspace{0.3cm}
Line 101:		Line 103:
	\hspace{0.15cm}\underline {= - 0.279\,{\rm bit}}		\hspace{0.15cm}\underline {= - 0.279\,{\rm bit}}
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$
	*You can save this conversion if you directly replace  $(1)$  ~~direkt~~ "ln" by "log<sub>2</sub>" already in the analytical result of subtask:		*You can save this conversion if you directly replace  $(1)$  direct  "ln"  by  "log<sub>2</sub>"  already in the analytical result of subtask:

	:$$h(X) = \ {\rm log}_2 \hspace{0.1cm} (\sqrt{\rm e}/2)\hspace{0.05cm}, \hspace{1.3cm}		:$$h(X) = \ {\rm log}_2 \hspace{0.1cm} (\sqrt{\rm e}/2)\hspace{0.05cm}, \hspace{1.3cm}
	{\rm pseudo-unit\hspace{-0.1cm}:\hspace{0.15cm} bit}		{\rm pseudo-unit\hspace{-0.1cm}:\hspace{0.15cm} bit}
Line 108:		Line 111:


	~~[[File:P_ID2866__Inf_A_4_2c.png\|right\|frame\|To calculate  $h(Y)$]]~~
	'''(3)'''  We again use the natural logarithm and divide the integral into two partial integrals:		'''(3)'''  We again use the natural logarithm and divide the integral into two partial integrals:
	:$$h(Y) =		:$$h(Y) =
Line 116:		Line 118:

	*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.		*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.
	*Now the ~~heighte~~  $C = 1$  instead of  $C = 2$ has to be considered::		*Now the height  $C = 1$  instead of  $C = 2$  has to be considered::
	:$$I_{\rm neg} =- C/2 \cdot		:$$I_{\rm neg} =- C/2 \cdot
	\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2		\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2

Guenter at 09:36, 25 September 2021

2021-09-25T09:36:21Z

← Older revision		Revision as of 11:36, 25 September 2021
Line 2:		Line 2:
	}}		}}

	[[File:P_ID2865__Inf_A_4_2.png\|right\|frame\|Two triangular ~~PDF~~]]		[[File:P_ID2865__Inf_A_4_2.png\|right\|frame\|Two triangular PDFs]]
	Two probability density functions (PDF) with a triangular ~~shape~~ are considered.		Two probability density functions  $\rm (PDF)$  with triangular shapes are considered.
	* The random variable  $X$  is limited to the range ~~of values~~ from  $0$  to  $1$ ,  and it holds for the PDF (upper sketch):		* The random variable  $X$  is limited to the range from  $0$  to  $1$ ,  and it holds for the PDF (upper sketch):
	:$$f_X(x) = \left\{ \begin{array}{c} 2x \\ 0 \\ \end{array} \right. \begin{array}{*{20}c} {\rm{f\ddot{u}r}} \hspace{0.1cm} 0 \le x \le 1 \\ {\rm else} \\ \end{array}		:$$f_X(x) = \left\{ \begin{array}{c} 2x \\ 0 \\ \end{array} \right. \begin{array}{*{20}c} {\rm{f\ddot{u}r}} \hspace{0.1cm} 0 \le x \le 1 \\ {\rm else} \\ \end{array}
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$
Line 13:		Line 13:
	For both random variables, the  [[Information_Theory/Differentielle_Entropie\|differential entropy]]  is to be determined in each case.		For both random variables, the  [[Information_Theory/Differentielle_Entropie\|differential entropy]]  is to be determined in each case.

	For example, the corresponding equation for the random variable  $X$ is:		For example, the corresponding equation for the random variable  $X$  is:
	:$$h(X) =		:$$h(X) =
	\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm} f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [ f_X(x) \big ] \hspace{0.1cm}{\rm d}x		\hspace{0.1cm} - \hspace{-0.45cm} \int\limits_{{\rm supp}\hspace{0.03cm}(\hspace{-0.03cm}f_X)} \hspace{-0.35cm} f_X(x) \cdot {\rm log} \hspace{0.1cm} \big [ f_X(x) \big ] \hspace{0.1cm}{\rm d}x
	\hspace{0.6cm}{\rm mit}\hspace{0.6cm} {\rm supp}(f_X) = \{ x\text{:} \ f_X(x) > 0 \}		\hspace{0.6cm}{\rm mit}\hspace{0.6cm} {\rm supp}(f_X) = \{ x\text{:} \ f_X(x) > 0 \}
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$
	*If the  ''natural logarithm'', the pseudo-unit  "nat"  must be added.		*If the  "natural logarithm",  the pseudo-unit  "nat"  must be added.
	*If, on the other hand, the result is asked in  "bit"  then the ~~<i>~~dual logarithm~~</i>~~   ⇒   "$\log_2$"  is to be used.		*If, on the other hand, the result is asked in  "bit"  then the  "dual logarithm"   ⇒   "$\log_2$"  is to be used.


	In the fourth subtask, the new random variable  $Z = A \cdot Y$  is considered. Here, the PDF parameter  $A$  is to be determined in such a way that the differential entropy of the new random variable  $Z$  yields exactly  $1$ bit :<br>		In the fourth subtask, the new random variable  $Z = A \cdot Y$  is considered. Here,  the PDF parameter  $A$  is to be determined in such a way that the differential entropy of the new random variable  $Z$  yields exactly  $1$  bit :<br>
	:$$h(Z) = h (A \cdot Y) = h (Y) + {\rm log}_2 \hspace{0.1cm} (A) = 1\,{\rm bit} \hspace{0.05cm}.$$		:$$h(Z) = h (A \cdot Y) = h (Y) + {\rm log}_2 \hspace{0.1cm} (A) = 1\ {\rm bit} \hspace{0.05cm}.$$


Line 34:		Line 34:
	Hints:		Hints:
	*The task belongs to the chapter  [[Information_Theory/Differentielle_Entropie\|Differential Entropy]].		*The task belongs to the chapter  [[Information_Theory/Differentielle_Entropie\|Differential Entropy]].
	*Useful hints for solving this task and further information on continuous~~-valued~~ random variables can be found in the third chapter "Continuous Random Variables" of the book  [[Theory of Stochastic Signals]].		*Useful hints for solving this task and further information on value-continuous random variables can be found in the third chapter "Continuous Random Variables" of the book  [[Theory of Stochastic Signals]].

	*Given the following indefinite integral:		*Given the following indefinite integral:

← Older revision		Revision as of 11:47, 25 September 2021
Line 118:		Line 118:

	*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.		*The first integral for the range  $-1 \le y \le 0$  is identical in form to that of subtask  $(1)$  and only shifted with respect to it, which does not affect the result.
	*Now the height  $C = 1$  instead of  $C = 2$  has to be considered::		*Now the height  $C = 1$  instead of  $C = 2$  has to be considered:
	:$$I_{\rm neg} =- C/2 \cdot		:$$I_{\rm neg} =- C/2 \cdot
	\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2		\big [ {\rm ln} \hspace{0.1cm} (C) - 1/2
Line 126:		Line 126:
	\hspace{0.05cm}.$$		\hspace{0.05cm}.$$

	*The second integrand is identical to the first except for a shift and reflection.  Moreover, the integration intervals do not ~~overla~~   ⇒   $I_{\rm pos} = I_{\rm neg}$:		*The second integrand is identical to the first except for a shift and reflection.  Moreover, the integration intervals do not overlap   ⇒   $I_{\rm pos} = I_{\rm neg}$:
	:$$h_{\rm nat}(Y) = 2 \cdot I_{\rm neg} = 1/2 \cdot		:$$h_{\rm nat}(Y) = 2 \cdot I_{\rm neg} = 1/2 \cdot
	{\rm ln} \hspace{0.1cm} ({\rm e}) = {\rm ln} \hspace{0.1cm} (\sqrt{\rm e}) \hspace{0.3cm}		{\rm ln} \hspace{0.1cm} ({\rm e}) = {\rm ln} \hspace{0.1cm} (\sqrt{\rm e}) \hspace{0.3cm}
Line 137:		Line 137:
	'''(4)'''  For the differential entropy of the random variable  $Z = A \cdot Y$  holds in general:		'''(4)'''  For the differential entropy of the random variable  $Z = A \cdot Y$  holds in general:
	:$$h(Z) = h(A \cdot Y) = h(Y) + {\rm log}_2 \hspace{0.1cm} (A) \hspace{0.05cm}.$$		:$$h(Z) = h(A \cdot Y) = h(Y) + {\rm log}_2 \hspace{0.1cm} (A) \hspace{0.05cm}.$$
	*Thus, from the requiremen  $h(Z) = 1 \ \rm bit$  and the result of ~~subta~~  $(3)$  follows:		*Thus, from the requiremen  $h(Z) = 1 \ \rm bit$  and the result of subtask  $(3)$  follows:
	:$${\rm log}_2 \hspace{0.1cm} (A) = 1\,{\rm bit} - 0.721 \,{\rm bit} = 0.279 \,{\rm bit}		:$${\rm log}_2 \hspace{0.1cm} (A) = 1\,{\rm bit} - 0.721 \,{\rm bit} = 0.279 \,{\rm bit}
	\hspace{0.3cm} \Rightarrow\hspace{0.3cm} A = 2^{0.279}\hspace{0.15cm}\underline		\hspace{0.3cm} \Rightarrow\hspace{0.3cm} A = 2^{0.279}\hspace{0.15cm}\underline