Information Theory/Entropy Coding According to Huffman - Revision history

Maintenance script: Join split $$ formula lines for correct rendering

2026-03-16T13:46:58Z

Join split $$ formula lines for correct rendering

← Older revision		Revision as of 15:46, 16 March 2026
Line 10:		Line 10:
	We now assume that the source symbols  $q_\nu$  originate from an alphabet  $\{q_μ\} = \{$ $\rm A$,  $\rm B$ ,  $\rm C$ , ...$\}$  with the symbol set size  $M$  and they are statistically independent of each other.  For example, for the symbol set size  $M = 8$:		We now assume that the source symbols  $q_\nu$  originate from an alphabet  $\{q_μ\} = \{$ $\rm A$,  $\rm B$ ,  $\rm C$ , ...$\}$  with the symbol set size  $M$  and they are statistically independent of each other.  For example, for the symbol set size  $M = 8$:

	:$$\{ \hspace{0.05cm}q_{\mu} \} = \{ \boldsymbol{\rm A} \hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm B}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm C}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm D}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm E}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm F}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm G}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm H}\hspace{0.05cm}		:$$\{ \hspace{0.05cm}q_{\mu} \} = \{ \boldsymbol{\rm A} \hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm B}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm C}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm D}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm E}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm F}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm G}\hspace{0.05cm},\hspace{0.05cm} \boldsymbol{\rm H}\hspace{0.05cm}\}\hspace{0.05cm}.$$
	\}\hspace{0.05cm}.$$

	In 1952 - i.e. shortly after Shannon's groundbreaking publications,  [https://en.wikipedia.org/wiki/David_A._Huffman $\text{David A. Huffman}$]  gave an algorithm for the construction of optimal prefix-free codes.  This  »'''Huffman Algorithm'''«  is to be given here without derivation and proof, whereby we restrict ourselves to binary codes.  That is:   For the code symbols, let  $c_ν ∈ \{$'''0''',  '''1'''$\}$ always hold.		In 1952 - i.e. shortly after Shannon's groundbreaking publications,  [https://en.wikipedia.org/wiki/David_A._Huffman $\text{David A. Huffman}$]  gave an algorithm for the construction of optimal prefix-free codes.  This  »'''Huffman Algorithm'''«  is to be given here without derivation and proof, whereby we restrict ourselves to binary codes.  That is:   For the code symbols, let  $c_ν ∈ \{$'''0''',  '''1'''$\}$ always hold.
Line 26:		Line 25:
	$\text{Example 1:}$  Without limiting the generality, we assume that the  $M = 6$  symbols  $\rm A$, ... , $\rm F$  are already ordered according to their probabilities:		$\text{Example 1:}$  Without limiting the generality, we assume that the  $M = 6$  symbols  $\rm A$, ... , $\rm F$  are already ordered according to their probabilities:

	:$$p_{\rm A} = 0.30 \hspace{0.05cm},\hspace{0.2cm}p_{\rm B} = 0.24 \hspace{0.05cm},\hspace{0.2cm}p_{\rm C} = 0.20 \hspace{0.05cm},\hspace{0.2cm}		:$$p_{\rm A} = 0.30 \hspace{0.05cm},\hspace{0.2cm}p_{\rm B} = 0.24 \hspace{0.05cm},\hspace{0.2cm}p_{\rm C} = 0.20 \hspace{0.05cm},\hspace{0.2cm}p_{\rm D} = 0.12 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = 0.10 \hspace{0.05cm},\hspace{0.2cm}p_{\rm F} = 0.04\hspace{0.05cm}.$$
	p_{\rm D} = 0.12 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = 0.10 \hspace{0.05cm},\hspace{0.2cm}p_{\rm F} = 0.04
	\hspace{0.05cm}.$$

	By pairwise combination and subsequent sorting, the following symbol combinations are obtained in five steps  (resulting probabilities in brackets):		By pairwise combination and subsequent sorting, the following symbol combinations are obtained in five steps  (resulting probabilities in brackets):
Line 50:		Line 47:
	We continue to assume the probabilities and assignments of the last example:		We continue to assume the probabilities and assignments of the last example:

	:$$p_{\rm A} = 0.30 \hspace{0.05cm},\hspace{0.2cm}p_{\rm B} = 0.24 \hspace{0.05cm},\hspace{0.2cm}p_{\rm C} = 0.20 \hspace{0.05cm},\hspace{0.2cm}		:$$p_{\rm A} = 0.30 \hspace{0.05cm},\hspace{0.2cm}p_{\rm B} = 0.24 \hspace{0.05cm},\hspace{0.2cm}p_{\rm C} = 0.20 \hspace{0.05cm},\hspace{0.2cm}p_{\rm D} = 0.12 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = 0.10 \hspace{0.05cm},\hspace{0.2cm}p_{\rm F} = 0.04\hspace{0.05cm};$$
	p_{\rm D} = 0.12 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = 0.10 \hspace{0.05cm},\hspace{0.2cm}p_{\rm F} = 0.04
	\hspace{0.05cm};$$

	:$$\boldsymbol{\rm A} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 11} \hspace{0.05cm},\hspace{0.2cm}		:$$\boldsymbol{\rm A} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 11} \hspace{0.05cm},\hspace{0.2cm}\boldsymbol{\rm B} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 01} \hspace{0.05cm},\hspace{0.2cm}\boldsymbol{\rm C} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 00} \hspace{0.05cm},\hspace{0.2cm}\boldsymbol{\rm D} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 100} \hspace{0.05cm},\hspace{0.2cm}\boldsymbol{\rm E} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 1011} \hspace{0.05cm},\hspace{0.2cm}\boldsymbol{\rm F} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 1010} \hspace{0.05cm}.$$
	\boldsymbol{\rm B} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 01} \hspace{0.05cm},\hspace{0.2cm}
	\boldsymbol{\rm C} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 00} \hspace{0.05cm},\hspace{0.2cm}
	\boldsymbol{\rm D} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 100} \hspace{0.05cm},\hspace{0.2cm}
	\boldsymbol{\rm E} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 1011} \hspace{0.05cm},\hspace{0.2cm}
	\boldsymbol{\rm F} \hspace{0.05cm} \rightarrow \hspace{0.05cm} \boldsymbol{\rm 1010} \hspace{0.05cm}.$$

	Thus, of the six source symbols, three are encoded with two bits each, one with three bits and two symbols  $(\rm E$  and  $\rm F)$  with four bits each.		Thus, of the six source symbols, three are encoded with two bits each, one with three bits and two symbols  $(\rm E$  and  $\rm F)$  with four bits each.
Line 65:		Line 55:
	The average code word length thus results in		The average code word length thus results in

	:$$L_{\rm M} = (0.30 \hspace{-0.05cm}+ \hspace{-0.05cm}0.24 \hspace{-0.05cm}+ \hspace{-0.05cm} 0.20) \cdot 2 + 0.12 \cdot 3 + (0.10 \hspace{-0.05cm}+ \hspace{-0.05cm} 0.04 ) \cdot 4 = 2.4 \,{\rm bit/source\hspace{0.15cm}symbol}		:$$L_{\rm M} = (0.30 \hspace{-0.05cm}+ \hspace{-0.05cm}0.24 \hspace{-0.05cm}+ \hspace{-0.05cm} 0.20) \cdot 2 + 0.12 \cdot 3 + (0.10 \hspace{-0.05cm}+ \hspace{-0.05cm} 0.04 ) \cdot 4 = 2.4 \,{\rm bit/source\hspace{0.15cm}symbol}\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$

	From the comparison with the source entropy  $H = 2.365 \ \rm bit/source\hspace{0.15cm}symbol$  one can see the efficiency of the Huffman coding.		From the comparison with the source entropy  $H = 2.365 \ \rm bit/source\hspace{0.15cm}symbol$  one can see the efficiency of the Huffman coding.
Line 78:		Line 67:
	$\text{Example 2:}$  If the symbol probabilities were		$\text{Example 2:}$  If the symbol probabilities were

	:$$p_{\rm A} = p_{\rm B} = p_{\rm C} = 1/4 \hspace{0.05cm},\hspace{0.2cm}		:$$p_{\rm A} = p_{\rm B} = p_{\rm C} = 1/4 \hspace{0.05cm},\hspace{0.2cm}p_{\rm D} = 1/8 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = p_{\rm F} = 1/16\hspace{0.05cm},$$
	p_{\rm D} = 1/8 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = p_{\rm F} = 1/16
	\hspace{0.05cm},$$

	then the same would apply to the entropy and to the average code word length:		then the same would apply to the entropy and to the average code word length:

	:$$H = 3 \cdot 1/4 \cdot {\rm log_2}\hspace{0.1cm}(4) + 1/8 \cdot {\rm log_2}\hspace{0.1cm}(8) + 2 \cdot 1/16 \cdot {\rm log_2}\hspace{0.1cm}(16) = 2.375 \,{\rm bit/source\hspace{0.15cm} symbol}\hspace{0.05cm},$$		:$$H = 3 \cdot 1/4 \cdot {\rm log_2}\hspace{0.1cm}(4) + 1/8 \cdot {\rm log_2}\hspace{0.1cm}(8) + 2 \cdot 1/16 \cdot {\rm log_2}\hspace{0.1cm}(16) = 2.375 \,{\rm bit/source\hspace{0.15cm} symbol}\hspace{0.05cm},$$
	:$$L_{\rm M} = 3 \cdot 1/4 \cdot 2 + 1/8 \cdot 3 + 2 \cdot 1/16 \cdot 4 = 2.375 \,{\rm bit/source\hspace{0.15cm} symbol}		:$$L_{\rm M} = 3 \cdot 1/4 \cdot 2 + 1/8 \cdot 3 + 2 \cdot 1/16 \cdot 4 = 2.375 \,{\rm bit/source\hspace{0.15cm} symbol}\hspace{0.05cm}.$$}}
	\hspace{0.05cm}.$$}}


Line 140:		Line 126:
	:$$p_{\rm D} = 0.12 \hspace{0.05cm} \Rightarrow \hspace{0.05cm} h_{\rm D} = 7/49 \approx 0.143 \hspace{0.05cm},$$		:$$p_{\rm D} = 0.12 \hspace{0.05cm} \Rightarrow \hspace{0.05cm} h_{\rm D} = 7/49 \approx 0.143 \hspace{0.05cm},$$
	:$$p_{\rm E}=0.10 \hspace{0.05cm} \Rightarrow \hspace{0.05cm} h_{\rm E} = 5/49 \approx 0.102 \hspace{0.05cm},$$		:$$p_{\rm E}=0.10 \hspace{0.05cm} \Rightarrow \hspace{0.05cm} h_{\rm E} = 5/49 \approx 0.102 \hspace{0.05cm},$$
	:$$p_{\rm F} = 0.04 \hspace{0.05cm} \Rightarrow \hspace{0.05cm} h_{\rm E} = 5/49 \approx 0.102		:$$p_{\rm F} = 0.04 \hspace{0.05cm} \Rightarrow \hspace{0.05cm} h_{\rm E} = 5/49 \approx 0.102\hspace{0.05cm}.$$
	\hspace{0.05cm}.$$

	*This results in a slightly larger entropy value:		*This results in a slightly larger entropy value:

	:$$H ({\rm regarding}\hspace{0.15cm}p_{\mu}) = 2.365 \ {\rm bit/source\hspace{0.15cm} symbol}\hspace{0.3cm}		:$$H ({\rm regarding}\hspace{0.15cm}p_{\mu}) = 2.365 \ {\rm bit/source\hspace{0.15cm} symbol}\hspace{0.3cm}\Rightarrow \hspace{0.3cm}H ({\rm regarding}\hspace{0.15cm}h_{\mu}) = 2.451 \ {\rm bit/source\hspace{0.15cm} symbol}\hspace{0.05cm}.$$
	\Rightarrow \hspace{0.3cm}
	H ({\rm regarding}\hspace{0.15cm}h_{\mu}) = 2.451 \ {\rm bit/source\hspace{0.15cm} symbol}
	\hspace{0.05cm}.$$

	If one were to form the Huffman code with these  "new probabilities"  $h_{\rm A}$, ... , $h_{\rm F}$  the following assignments would result:		If one were to form the Huffman code with these  "new probabilities"  $h_{\rm A}$, ... , $h_{\rm F}$  the following assignments would result:
Line 197:		Line 179:
	Let a second source with symbol set size  $M = 6$  be characterized by the following symbol probabilities:		Let a second source with symbol set size  $M = 6$  be characterized by the following symbol probabilities:

	:$$p_{\rm A} = 0.50 \hspace{0.05cm},\hspace{0.2cm}p_{\rm B} = 0.19 \hspace{0.05cm},\hspace{0.2cm}p_{\rm C} = 0.11 \hspace{0.05cm},\hspace{0.2cm}		:$$p_{\rm A} = 0.50 \hspace{0.05cm},\hspace{0.2cm}p_{\rm B} = 0.19 \hspace{0.05cm},\hspace{0.2cm}p_{\rm C} = 0.11 \hspace{0.05cm},\hspace{0.2cm}p_{\rm D} = 0.09 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = 0.06 \hspace{0.05cm},\hspace{0.2cm}p_{\rm F} = 0.05\hspace{0.05cm}.$$
	p_{\rm D} = 0.09 \hspace{0.05cm},\hspace{0.2cm}p_{\rm E} = 0.06 \hspace{0.05cm},\hspace{0.2cm}p_{\rm F} = 0.05
	\hspace{0.05cm}.$$

	Here the Huffman algorithm leads to the following assignment:		Here the Huffman algorithm leads to the following assignment:
Line 255:		Line 235:
	The Huffman algorithm cannot be applied directly to this source, that is, one also needs a bit for each binary source symbol without further action. But:		The Huffman algorithm cannot be applied directly to this source, that is, one also needs a bit for each binary source symbol without further action. But:
	If one combines two binary symbols at a time into a tuple of two  $(k = 2)$  corresponding to   $\rm XX$   →   $\rm A$,   $\rm XY$   →   $\rm B$,   $\rm YX$   →   $\rm C$,   $\rm YY$   →   $\rm D$  , one can apply  "Huffman"  to the resulting sequence  →   $\rm ABAACADAABCBBAC$ ...  →   with $M\hspace{-0.01cm}′ = 4$.  Because of		If one combines two binary symbols at a time into a tuple of two  $(k = 2)$  corresponding to   $\rm XX$   →   $\rm A$,   $\rm XY$   →   $\rm B$,   $\rm YX$   →   $\rm C$,   $\rm YY$   →   $\rm D$  , one can apply  "Huffman"  to the resulting sequence  →   $\rm ABAACADAABCBBAC$ ...  →   with $M\hspace{-0.01cm}′ = 4$.  Because of
	:$$p_{\rm A}= 0.8^2 = 0.64 \hspace{0.05cm}, \hspace{0.2cm}p_{\rm B}= 0.8 \cdot 0.2 = 0.16 = p_{\rm C} \hspace{0.05cm}, \hspace{0.2cm}		:$$p_{\rm A}= 0.8^2 = 0.64 \hspace{0.05cm}, \hspace{0.2cm}p_{\rm B}= 0.8 \cdot 0.2 = 0.16 = p_{\rm C} \hspace{0.05cm}, \hspace{0.2cm}p_{\rm D}= 0.2^2 = 0.04,$$
	p_{\rm D}= 0.2^2 = 0.04,$$
	we get   $\rm A$   →   '''1''',   $\rm B$   →   '''00''',   $\rm C$   →   '''011''',   $\rm D$   →   '''010'''   as well as		we get   $\rm A$   →   '''1''',   $\rm B$   →   '''00''',   $\rm C$   →   '''011''',   $\rm D$   →   '''010'''   as well as

Line 268:		Line 247:
	*For the input sequence given above, one arrives at the equivalent output sequence   $\rm AEBAGADBCC$ ...  (based on the new symbol set size $M\hspace{-0.01cm}′ = 8$) and to the following probabilities:		*For the input sequence given above, one arrives at the equivalent output sequence   $\rm AEBAGADBCC$ ...  (based on the new symbol set size $M\hspace{-0.01cm}′ = 8$) and to the following probabilities:

	:$$p_{\rm A}= 0.8^3 = 0.512 \hspace{0.05cm}, \hspace{0.5cm}p_{\rm B}= p_{\rm C}= p_{\rm E} = 0.8^2 \cdot 0.2 = 0.128\hspace{0.05cm},\hspace{0.5cm}		:$$p_{\rm A}= 0.8^3 = 0.512 \hspace{0.05cm}, \hspace{0.5cm}p_{\rm B}= p_{\rm C}= p_{\rm E} = 0.8^2 \cdot 0.2 = 0.128\hspace{0.05cm},\hspace{0.5cm}p_{\rm D}= p_{\rm F}= p_{\rm G} = 0.8 \cdot 0.2^2 = 0.032 \hspace{0.05cm}, \hspace{0.5cm}p_{\rm H}= 0.2^3 = 0.008\hspace{0.05cm}.$$
	p_{\rm D}= p_{\rm F}= p_{\rm G} = 0.8 \cdot 0.2^2 = 0.032 \hspace{0.05cm}, \hspace{0.5cm}p_{\rm H}= 0.2^3 = 0.008\hspace{0.05cm}.$$

	*The Huffman coding is thus:		*The Huffman coding is thus:

Maintenance script: Add German interlanguage link

2026-03-16T12:27:26Z

Add German interlanguage link

← Older revision		Revision as of 14:27, 16 March 2026
Line 299:		Line 299:

	{{Display}}		{{Display}}
			[[de:Informationstheorie/Entropiecodierung_nach_Huffman]]

Guenter at 17:09, 20 February 2023

2023-02-20T17:09:14Z

← Older revision		Revision as of 19:09, 20 February 2023
Line 210:		Line 210:
	\boldsymbol{\rm F} \hspace{0.05cm} \rightarrow \hspace{0.15cm}$'''1100'''$\hspace{0.05cm}.$		\boldsymbol{\rm F} \hspace{0.05cm} \rightarrow \hspace{0.15cm}$'''1100'''$\hspace{0.05cm}.$

	[[File:~~EN_Inf_T_2_3_S4c~~.png\|right\|frame\|On the error propagation of the Huffman coding ~~'''KORREKTUR: falsified'''~~]]		[[File:EN_Inf_T_2_3_S4c_v2.png\|right\|frame\|On the error propagation of the Huffman coding]]

	The source symbol sequence  $\rm ADABD$...  (upper diagram) is thus represented by the encoded sequence  '''0'100'0'111'100' '''... .  The inverted commas are only for the reader's orientation.		The source symbol sequence  $\rm ADABD$...  (upper diagram) is thus represented by the encoded sequence  '''0'100'0'111'100' '''... .  The inverted commas are only for the reader's orientation.

Guenter at 15:26, 19 February 2023

2023-02-19T15:26:49Z

← Older revision		Revision as of 17:26, 19 February 2023
Line 249:		Line 249:
	{{GraueBox\|TEXT=		{{GraueBox\|TEXT=
	$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:		$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:
	*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source~~\hspace{0.15cm} '''KORREKTUR: ?'''~~ symbol.		*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source symbol.
	*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...		*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...

Guenter at 15:24, 19 February 2023

2023-02-19T15:24:45Z

← Older revision		Revision as of 17:24, 19 February 2023
Line 214:		Line 214:
	The source symbol sequence  $\rm ADABD$...  (upper diagram) is thus represented by the encoded sequence  '''0'100'0'111'100' '''... .  The inverted commas are only for the reader's orientation.		The source symbol sequence  $\rm ADABD$...  (upper diagram) is thus represented by the encoded sequence  '''0'100'0'111'100' '''... .  The inverted commas are only for the reader's orientation.

	During transmission, the first bit is ~~distorted '''KORREKTUR:~~ falsified~~'''~~:  Instead of  '''01000111100'''...  one receives  '''11000111100'''...		During transmission, the first bit is falsified:  Instead of  '''01000111100'''...  one receives  '''11000111100'''...
	*After decoding, the first two source symbols  $\rm AD$  →  '''0100'''  become the sink symbol  $\rm F$  →  '''1100'''.		*After decoding, the first two source symbols  $\rm AD$  →  '''0100'''  become the sink symbol  $\rm F$  →  '''1100'''.
	*The further symbols are then detected correctly again, but now no longer starting at position  $ν = 3$, but at  $ν = 2$.		*The further symbols are then detected correctly again, but now no longer starting at position  $ν = 3$, but at  $ν = 2$.

Hwang at 13:21, 16 February 2023

2023-02-16T13:21:35Z

← Older revision		Revision as of 15:21, 16 February 2023
Line 249:		Line 249:
	{{GraueBox\|TEXT=		{{GraueBox\|TEXT=
	$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:		$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:
	*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source $\hspace{0.15cm}$ symbol.		*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source\hspace{0.15cm} '''KORREKTUR: ?''' symbol.
	*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...		*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...

Hwang at 13:20, 16 February 2023

2023-02-16T13:20:28Z

← Older revision		Revision as of 15:20, 16 February 2023
Line 249:		Line 249:
	{{GraueBox\|TEXT=		{{GraueBox\|TEXT=
	$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:		$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:
	*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source \hspace{0.15cm} symbol.		*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source $\hspace{0.15cm}$ symbol.
	*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...		*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...

Hwang at 13:19, 16 February 2023

2023-02-16T13:19:52Z

← Older revision		Revision as of 15:19, 16 February 2023
Line 249:		Line 249:
	{{GraueBox\|TEXT=		{{GraueBox\|TEXT=
	$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:		$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:
	*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source\hspace{0.15cm}symbol.		*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source \hspace{0.15cm} symbol.
	*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...		*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...

Hwang at 13:17, 16 February 2023

2023-02-16T13:17:56Z

← Older revision		Revision as of 15:17, 16 February 2023
Line 249:		Line 249:
	{{GraueBox\|TEXT=		{{GraueBox\|TEXT=
	$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:		$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:
	*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.&nbsp Thus the source entropy is  $H = 0.722$  bit/source\hspace{0.15cm}symbol.		*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.  Thus the source entropy is  $H = 0.722$  bit/source\hspace{0.15cm}symbol.
	*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...		*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...

Hwang at 13:16, 16 February 2023

2023-02-16T13:16:30Z

← Older revision		Revision as of 15:16, 16 February 2023
Line 248:		Line 248:

	{{GraueBox\|TEXT=		{{GraueBox\|TEXT=
	$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given::		$\text{Example 6:}$  Let a memoryless binary source $(M = 2)$ with the symbols  $\{$ $\rm X$,  $\rm Y \}$ be given:
	*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.&nbsp Thus the source entropy is  $H = 0.722$  bit/source\hspace{0.15cm}symbol.		*Let the symbol probabilities be  $p_{\rm X} = 0.8$  and  $p_{\rm Y} = 0.2$.&nbsp Thus the source entropy is  $H = 0.722$  bit/source\hspace{0.15cm}symbol.
	*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...		*We consider the symbol sequence   $\{\rm XXXYXXXXXXXXYYXXXXXYYXXYXYXXYX\ \text{...} \}$  with only a few  $\rm Y$ symbols at positions 4, 13, 14, ...