Difference between revisions of "Information Theory"

From LNTwww
 
(30 intermediate revisions by 2 users not shown)
Line 1: Line 1:
Since the early beginnings of communications as an engineering discipline, many engineers and mathematicians have sought to find a quantitative measure of
+
===Brief summary===
*the $\rm Information$  (in general: "the knowledge of something") contained in a  $\rm message$  (here we understand  "a collection of symbols and/or states").
 
  
 +
{{BlueBox|TEXT=From the earliest beginnings of message transmission as an engineering discipline,  it has been the endeavour of many engineers and mathematicians  to find a quantitative measure for the
 +
*contained  $\rm information$  $($quite generally:  »the knowledge about something«$)$
  
The  (abstract)  information is communicated by the  (concrete)  message and can be seen as an interpretation of a message.  
+
*in a  $\rm message$  $($here we mean  »a collection of symbols and/or states»$)$.
 +
  
[https://de.wikipedia.org/wiki/Claude_Shannon Claude Elwood Shannon]  succeeded in 1948 in establishing a consistent theory of the information content of messages,  which was revolutionary in its time and created a new, still highly topical field of science:  the theory named after him  $\text{Shannon's Information Theory}$.
+
The  $($abstract$)$  information is communicated by the  $($concrete$)$  message and can be conceived as the interpretation of a message.  
  
The subject matter corresponds to a  $\text{lecture with two semester hours per week (SWS) and one additional SWS exercise}$.
+
[https://en.wikipedia.org/wiki/Claude_Shannon '''Claude Elwood Shannon''']  succeeded in 1948,  in establishing a consistent theory about the information content of messages,  which was revolutionary in its time and created a new,  still highly topical field of science:   »'''Shannon's information theory«'''  named after him.
  
Here is a table of contents based on the  $\text{four main chapters}$  with a total of  $\text{13 individual chapters}$.
+
This is what the fourth book in the  $\rm LNTwww$ series deals with,  in particular:
+
# Entropy of discrete-value sources with and without memory,  as well as natural message sources:  Definition,  meaning and computational possibilities.
+
# Source coding and data compression,  especially the   »Lempel–Ziv–Welch method«   and   »Huffman's entropy encoding«. 
 +
# Various entropies of two-dimensional discrete-value random quantities.  Mutual information and channel capacity.  Application to digital signal transmission.   
 +
# Discrete-value information theory.  Differential entropy.  AWGN channel capacity with continuous-valued as well as discrete-valued input.
 +
 
 +
 
 +
⇒   First a  »'''content overview'''«  on the basis of the  »'''four main chapters'''«  with a total of  »'''13 individual chapters'''«  and  »'''106 sections'''«:}}
 +
 
 +
 
 +
 
 +
===Content===
  
===Contents===
 
 
{{Collapsible-Kopf}}
 
{{Collapsible-Kopf}}
 
{{Collapse1| header=Entropy of Discrete Sources
 
{{Collapse1| header=Entropy of Discrete Sources
Line 37: Line 47:
 
|submenu=
 
|submenu=
 
*[[/Differential Entropy/]]
 
*[[/Differential Entropy/]]
*[[/AWGN Channel Capacity for Continuous Input/]]
+
*[[/AWGN Channel Capacity for Continuous-Valued Input/]]
*[[/AWGN Channel Capacity for Discrete Input/]]
+
*[[/AWGN Channel Capacity for Discrete-Valued Input/]]
 
}}
 
}}
 
{{Collapsible-Fuß}}
 
{{Collapsible-Fuß}}
  
In addition to these theory pages, we also offer exercises and multimedia modules that could help to clarify the teaching material:
+
===Exercises and multimedia===
  
*[https://en.lntwww.de/Category:Information_Theory:_Exercises $\text{Exercises}$]
+
{{BlaueBox|TEXT=
*[[LNTwww:Lernvideos_zu_Informationstheorie|$\text{Learning videos}$]]
+
In addition to these theory pages,  we also offer exercises and multimedia modules on this topic,  which could help to clarify the teaching material:
*[[LNTwww:HTML5-Applets_zu_Informationstheorie|$\text{Applets}$]]
+
 
<br><br>
+
$(1)$&nbsp; &nbsp; [https://en.lntwww.de/Category:Information_Theory:_Exercises $\text{Exercises}$]
$\text{More links:}$
+
 
<br><br>
+
$(2)$&nbsp; &nbsp; [[LNTwww:Learning_videos_to_"Information_Theory"|$\text{Learning videos}$]]
<br><br>
 
$\text{Other Links:}$
 
<br><br>
 
$(1)$&nbsp; &nbsp; [[LNTwww:Bibliography_to_Mobile_Communikations|$\text{Bibliography to the book}$]]
 
  
$(2)$&nbsp; &nbsp; [[LNTwww:Notes_on_the_authors_and_materials_used_in the_preparation_of_Mobile Communications|$\text{Notes on the authors and materials used in the preparation of the book}$]]  
+
$(3)$&nbsp; &nbsp; [[LNTwww:Applets_to_"Information_Theory"|$\text{Applets}$]]&nbsp;}}
<br><br>
 
  
  
 +
===Further links===
  
$(1)$&nbsp; &nbsp; [[LNTwww:Literaturempfehlung_zu_Informationstheorie|$\text{Recommended literature for the book}$]]
+
{{BlaueBox|TEXT=
 +
$(4)$&nbsp; &nbsp; [[LNTwww:Bibliography_to_"Information_Theory"|$\text{Bibliography}$]]
  
$(2)$&nbsp; &nbsp; [[LNTwww:Weitere_Hinweise_zum_Buch_Informationstheorie|$\text{General notes about the book}$]] &nbsp; (Authors,&nbsp; other participants,&nbsp; materials as a starting point for the book,&nbsp; list of sources)
+
$(5)$&nbsp; &nbsp; [[LNTwww:Imprint_for_the_book_"Information_Theory"|$\text{Impressum}$]]}}
 
<br><br>
 
<br><br>
 +
 +
  
  
 
__NOTOC__
 
__NOTOC__
 
__NOEDITSECTION__
 
__NOEDITSECTION__

Latest revision as of 17:50, 31 December 2023

Brief summary

From the earliest beginnings of message transmission as an engineering discipline,  it has been the endeavour of many engineers and mathematicians  to find a quantitative measure for the

  • contained  $\rm information$  $($quite generally:  »the knowledge about something«$)$
  • in a  $\rm message$  $($here we mean  »a collection of symbols and/or states»$)$.


The  $($abstract$)$  information is communicated by the  $($concrete$)$  message and can be conceived as the interpretation of a message.

Claude Elwood Shannon  succeeded in 1948,  in establishing a consistent theory about the information content of messages,  which was revolutionary in its time and created a new,  still highly topical field of science:  »Shannon's information theory«  named after him.

This is what the fourth book in the  $\rm LNTwww$ series deals with,  in particular:

  1. Entropy of discrete-value sources with and without memory,  as well as natural message sources:  Definition,  meaning and computational possibilities.
  2. Source coding and data compression,  especially the   »Lempel–Ziv–Welch method«   and   »Huffman's entropy encoding«.
  3. Various entropies of two-dimensional discrete-value random quantities.  Mutual information and channel capacity.  Application to digital signal transmission.
  4. Discrete-value information theory.  Differential entropy.  AWGN channel capacity with continuous-valued as well as discrete-valued input.


⇒   First a  »content overview«  on the basis of the  »four main chapters«  with a total of  »13 individual chapters«  and  »106 sections«:


Content

Exercises and multimedia

In addition to these theory pages,  we also offer exercises and multimedia modules on this topic,  which could help to clarify the teaching material:

$(1)$    $\text{Exercises}$

$(2)$    $\text{Learning videos}$

$(3)$    $\text{Applets}$ 


Further links