Aufgaben:Exercise 1.7: Entropy of Natural Texts - Revision history

Guenter at 12:06, 10 August 2021

2021-08-10T12:06:37Z

Javier: Text replacement - "symbol range" to "symbol set size"

2021-07-08T07:03:47Z

Text replacement - "symbol range" to "symbol set size"

Guenter at 11:47, 30 June 2021

2021-06-30T11:47:02Z

Guenter at 15:37, 22 June 2021

2021-06-22T15:37:53Z

Guenter at 15:12, 22 June 2021

2021-06-22T15:12:59Z

Guenter at 14:34, 22 June 2021

2021-06-22T14:34:53Z

Guenter at 14:18, 22 June 2021

2021-06-22T14:18:11Z

Guenter at 14:15, 22 June 2021

2021-06-22T14:15:54Z

Guenter at 11:29, 22 June 2021

2021-06-22T11:29:30Z

Guenter: Guenter moved page Aufgaben:Aufgabe 1.7: Entropie natürlicher Texte to Aufgaben:Exercise 1.7: Entropy of Natural Texts

2021-06-22T11:28:21Z

Guenter moved page Aufgabe 1.7: Entropie natürlicher Texte to Exercise 1.7: Entropy of Natural Texts

← Older revision		Revision as of 12:06, 10 August 2021
Line 1:		Line 1:
−	{{quiz-Header\|Buchseite=~~Informationstheorie~~/~~Natürliche wertdiskrete Nachrichtenquellen~~	+	{{quiz-Header\|Buchseite=Information_Theory/Natural_Discrete_Sources
	}}		}}

@@ Line 95: / Line 95: @@
 ===Solution===
 {{ML-Kopf}}
-'''(1)'''&nbsp; The symbol range in Küpfmüller's investigations was&nbsp; $\underline{M = 26}$,&nbsp; because, in contrast to Shannon, he did not initially take the space into account.
+'''(1)'''&nbsp; The symbol set size in Küpfmüller's investigations was&nbsp; $\underline{M = 26}$,&nbsp; because, in contrast to Shannon, he did not initially take the space into account.
-In the given German text of this task, the symbol range is significantly larger,
+In the given German text of this task, the symbol set size is significantly larger,
 * since the typical German characters "ä",&nbsp; "ö",&nbsp; "ü"&nbsp; and&nbsp; "ß"&nbsp; also occur,
 * there is a distinction between upper and lower case,

@@ Line 2: / Line 2: @@
 }}
-[[File:Inf_A_1_7_vers2.png|right|frame|Text with erasures or errors]]
+[[File:Inf_A_1_7_v2.png|right|frame|Text with erasures or errors]]
 In the early 1950s,&nbsp; [https://en.wikipedia.org/wiki/Claude_Shannon Claude E. Shannon]&nbsp; estimated the entropy&nbsp; $H$&nbsp; of the English language at one bit per character.&nbsp; A short time later,&nbsp; [https://en.wikipedia.org/wiki/Karl_K%C3%BCpfm%C3%BCller Karl Küpfmüller]&nbsp; arrived at an entropy value of&nbsp; $H =1.3\ \rm bit/character$, i.e. a somewhat larger value, in an empirical study of the German language.&nbsp; Interestingly, the results of Shannon and Küpfmüller are based on two completely different methods.
@@ Line 120: / Line 120: @@
-[[File:Inf_A_1_7d_vers2.png|right|frame|Original texts]]
+[[File:Inf_A_1_7d_v2.png|right|frame|Original texts]]
 Finally, the original text is given, which is only reproduced on the information page distorted by erasures or real character errors.

@@ Line 59: / Line 59: @@
 $\rm (D)$ &nbsp; &nbsp; The hit ratio depends strongly on the type of text. Values between 15% and 33% are obtained for different types of text, with an average value of 22%. However, this also means that on average 22% of all words can be determined from the context.
-$\rm (E)$ &nbsp; &nbsp;
+$\rm (E)$ &nbsp; &nbsp; In other words, the number of words in a long text can be reduced by a factor of 0.78 without significantly reducing its message content. By definition, the entropy H = 0.78 times 2.0 = 1.56 bit/character.
 }}

@@ Line 44: / Line 44: @@
 :$$r = \frac{H_0 - H}{H_0}\hspace{0.05cm}.$$
-*To solve the subtask&nbsp; '''(4)'''&nbsp; you need a good knowledge of the German language.
+*To solve the subtask&nbsp; '''(4)'''&nbsp; you need a good knowledge of the German language.&nbsp;
 ===Questions===
@@ Line 101: / Line 116: @@
 '''(4)'''&nbsp; Correct is <u>statement 2</u>:
 *Test it yourself: &nbsp; The second block of the graphic on the information page is easier to decode than the last block, because you know where there are errors.
-*If you want to keep trying: &nbsp; The exact same sequence of character errors was used for the bottom block&nbsp; '''(F)'''&nbsp; as for block&nbsp; '''(B)'''&nbsp; , that is, there are errors at characters&nbsp; $6$,&nbsp; $35$,&nbsp; $37$,&nbsp;, and so on.
+*If you want to keep trying: &nbsp; The exact same sequence of character errors was used for the bottom block&nbsp; '''(F)'''&nbsp; as for block&nbsp; '''(B)'''&nbsp; , that is, there are errors at characters&nbsp; $6$,&nbsp; $35$,&nbsp; $37$,&nbsp; and so on.

@@ Line 36: / Line 36: @@
 ''Hints:''
-*The task belongs to the chapter&nbsp; [[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen|Natural Discrete Sources]].
+*The exercise belongs to the chapter&nbsp; [[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen|Natural Discrete Sources]].
 *Bezug genommen wird insbesondere auf die beiden Seiten
 ::[[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen#Entropy_estimation_according_to_K.C3.BCpfm.C3.BCller|Entropy estimation according to Küpfmüller]],&nbsp; sowie&nbsp;
@@ Line 44: / Line 44: @@
 :$$r = \frac{H_0 - H}{H_0}\hspace{0.05cm}.$$
 ===Questions===
 <quiz display=simple>
-{What symbol range&nbsp; $M$&nbsp; did Küpfmüller assume?
+{What symbol set size&nbsp; $M$&nbsp; did Küpfmüller assume?
 |type="{}"}
 $M \ = \ $ { 26 }
@@ Line 61: / Line 64: @@
 |type="[]"}
 + Such a text file of sufficient length&nbsp; $(N \to \infty)$&nbsp; could be represented with&nbsp; $1.3 \cdot N$&nbsp; binary symbols.
-- Such a text file with&nbsp; $N= 100\hspace{0.1cm}000$&nbsp; characters could be represented with&nbsp; $130\hspace{0.1cm}000$&nbsp; binary symbols.
+- Such a text file with&nbsp; $N= 100\hspace{0.05cm}000$&nbsp; characters could be represented with&nbsp; $130\hspace{0.1cm}000$&nbsp; binary symbols.
 - A reader can still understand (or at least guess) the text even if&nbsp; $70\%$&nbsp; of the characters are erased.
 {What makes a text more difficult to understand?
-|type="[]"}
+|type="()"}
-- $20\%$&nbsp; Erasures,
+- $20\%$&nbsp; erasures,
-+ a&nbsp; $20\%$ probability of character errors.
++ $20\%$&nbsp; errors.

@@ Line 25: / Line 25: @@
 * In the upper five blocks one recognises&nbsp; "Erasures"&nbsp; with different erasure probabilities between&nbsp; $10\%$&nbsp; and&nbsp; $50\%$.
 * In the last block, "character errors" with&nbsp; $20$&ndash;error probability are inserted.
 The influence of such character errors on the readability of a text is to be compared in subtask&nbsp; '''(4)'''&nbsp; with the second block (outlined in red), for which the probability of an erasure is also&nbsp; $20\%$.
@@ Line 38: / Line 39: @@
 *Bezug genommen wird insbesondere auf die beiden Seiten
 ::[[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen#Entropy_estimation_according_to_K.C3.BCpfm.C3.BCller|Entropy estimation according to Küpfmüller]],&nbsp; sowie&nbsp;
-::[[Information_Theory/Natural_Discrete_Sources#A_further_entropy_estimation_by_K.C3.BCpfm.C3.BCller|Another entropy estimation by Küpfmüller]].
+::[[Information_Theory/Natural_Discrete_Sources#A_further_entropy_estimation_by_K.C3.BCpfm.C3.BCller|A further entropy estimation by Küpfmüller]].
 *For the&nbsp; '''relative redundancy'''&nbsp;  of a sequence with the decision content&nbsp; $H_0$&nbsp; and the entropy&nbsp; $H$&nbsp; applies

@@ Line 8: / Line 8: @@
 * Shannon assumed&nbsp; $26$ letters and the space &nbsp;&#8658;&nbsp;  $M = 27$.
-* Küpfmüller assumed only&nbsp; $M = 26$&nbsp; letters&nbsp; (i.e. without the space).
+* Küpfmüller assumed only&nbsp; $M = 26$&nbsp; letters&nbsp; (i.e. without the&nbsp; "Blank Space").
@@ Line 14: / Line 14: @@
 This task is intended to show how
-* Erasures &nbsp; &#8658; &nbsp; one knows the location of an error, resp.
+* "Erasures" &nbsp; &#8658; &nbsp; one knows the location of an error, resp.
-* Errors &nbsp; &#8658; &nbsp; it is not clear to the reader what is wrong and what is right,
+* "Errors" &nbsp; &#8658; &nbsp; it is not clear to the reader what is wrong and what is right,
-have an effect on the comprehensibility of a text.&nbsp; Our text also contains the typical German letters "ä",&nbsp; "ö",&nbsp; "ü"&nbsp; and&nbsp; "ß"&nbsp; as well as numbers and punctuation. In addition, a distinction is made between upper and lower case.
+have an effect on the comprehensibility of a text.&nbsp; Our text also contains the typical German letters "ä",&nbsp; "ö",&nbsp; "ü"&nbsp; and&nbsp; "ß"&nbsp; as well as numbers and punctuation.&nbsp; In addition, a distinction is made between upper and lower case.
-In the figure, a text dealing with Küpfmüller's approach is divided into six blocks of length between&nbsp; $N = 197$&nbsp; and&nbsp; $N = 319$&nbsp; aufgeteilt.&nbsp; Described is the check of his&nbsp; [[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen#A_further_entropy_estimation_by_K.C3.BCpfm.C3.BCller|first analysis]]&nbsp; in a completely different way, which led to the result&nbsp;  $H =1.51\ \rm bit/character$&nbsp;.
+In the figure, a text dealing with Küpfmüller's approach is divided into six blocks of length between&nbsp; $N = 197$&nbsp; and&nbsp; $N = 319$.&nbsp; Described is the check of his&nbsp; [[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen#A_further_entropy_estimation_by_K.C3.BCpfm.C3.BCller|first analysis]]&nbsp; in a completely different way, which led to the result&nbsp;  $H =1.51\ \rm bit/character$.
-* In the upper five blocks one recognises "Erasures" with different erasure probabilities between&nbsp; $10\%$&nbsp; and&nbsp; $50\%$.
+* In the upper five blocks one recognises&nbsp; "Erasures"&nbsp; with different erasure probabilities between&nbsp; $10\%$&nbsp; and&nbsp; $50\%$.
 * In the last block, "character errors" with&nbsp; $20$&ndash;error probability are inserted.
-The influence of such character errors on the readability of a text is to be compared in subtask&nbsp; '''(4)'''&nbsp; with the second block (outlined in red), for which the probability of an erasure is also&nbsp; $20\%$&nbsp;.
+The influence of such character errors on the readability of a text is to be compared in subtask&nbsp; '''(4)'''&nbsp; with the second block (outlined in red), for which the probability of an erasure is also&nbsp; $20\%$.
@@ Line 38: / Line 38: @@
 *Bezug genommen wird insbesondere auf die beiden Seiten
 ::[[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen#Entropy_estimation_according_to_K.C3.BCpfm.C3.BCller|Entropy estimation according to Küpfmüller]],&nbsp; sowie&nbsp;
-::[[Information_Theory/Natürliche_wertdiskrete_Nachrichtenquellen#A_further_entropy_estimation_by_K.C3.BCpfm.C3.BCller|Another entropy estimation by Küpfmüller]].
+::[[Information_Theory/Natural_Discrete_Sources#A_further_entropy_estimation_by_K.C3.BCpfm.C3.BCller|Another entropy estimation by Küpfmüller]].
-*For the&nbsp; ''relative redundancy''&nbsp;  of a sequence, with the decision content&nbsp; $H_0$&nbsp; and the entropy&nbsp; $H$&nbsp;, applies
+*For the&nbsp; '''relative redundancy'''&nbsp;  of a sequence with the decision content&nbsp; $H_0$&nbsp; and the entropy&nbsp; $H$&nbsp; applies
 :$$r = \frac{H_0 - H}{H_0}\hspace{0.05cm}.$$

← Older revision		Revision as of 11:29, 22 June 2021
Line 109:		Line 109:


−	[[Category:Information Theory: Exercises\|^1.3 ~~Natürliche Nachrichtenquellen~~^]]	+	[[Category:Information Theory: Exercises\|^1.3 Natural Discrete Sources^]]

← Older revision	Revision as of 11:28, 22 June 2021
(No difference)