Information Theory: Difference between revisions

From LNTwww
← Older edit
No edit summary
Add German interlanguage link
 
(38 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Since the early beginnings of communications as an engineering discipline, many engineers and mathematicians have sought to find a quantitative measure of
===Brief summary===
*the $\rm Information$  (in general: "the knowledge of something") contained in a  $\rm message$  (here we understand "a collection of symbols and/or states").


{{BlueBox|TEXT=From the earliest beginnings of message transmission as an engineering discipline,  it has been the endeavour of many engineers and mathematicians  to find a quantitative measure for the
*contained  $\rm information$  $($quite generally:  »the knowledge about something«$)$


The (abstract) information is communicated by the (concrete) message and can be seen as an interpretation of a message.  
*in a  $\rm message$  $($here we mean  »a collection of symbols and/or states»$)$.
 
The  $($abstract$)$  information is communicated by the  $($concrete$)$  message and can be conceived as the interpretation of a message.  
 
[https://en.wikipedia.org/wiki/Claude_Shannon '''Claude Elwood Shannon''']  succeeded in 1948,  in establishing a consistent theory about the information content of messages,  which was revolutionary in its time and created a new,  still highly topical field of science:   »'''Shannon's information theory«'''  named after him.
 
This is what the fourth book in the  $\rm LNTwww$ series deals with,  in particular:
# Entropy of discrete-value sources with and without memory,  as well as natural message sources:  Definition,  meaning and computational possibilities.
# Source coding and data compression,  especially the   »Lempel–Ziv–Welch method«   and   »Huffman's entropy encoding«. 
# Various entropies of two-dimensional discrete-value random quantities.  Mutual information and channel capacity.  Application to digital signal transmission.   
# Discrete-value information theory.  Differential entropy.  AWGN channel capacity with continuous-valued as well as discrete-valued input.
 
 
⇒   First a  »'''content overview'''«  on the basis of the  »'''four main chapters'''«  with a total of  »'''13 individual chapters'''«  and  »'''106 sections'''«:}}


[https://de.wikipedia.org/wiki/Claude_Shannon Claude Elwood Shannon]  succeeded in 1948 in establishing a consistent theory of the information content of messages, which was revolutionary in its time and created a new, still highly topical field of science:  the theory named after him  $\text{Shannon's Information Theory}$.


The course material corresponds to a  $\text{lecture with two semester hours per week (SWS) and one SWS exercise}$.


Here is a table of contents based on the  $\text{four main chapters}$  with a total of  $\text{13 individual chapters}$. 
===Content===


===Contents===
{{Collapsible-Kopf}}
{{Collapsible-Kopf}}
{{Collapse1| header=Entropy of Discrete Sources
{{Collapse1| header=Entropy of Discrete Sources
Line 36: Line 47:
|submenu=
|submenu=
*[[/Differential Entropy/]]
*[[/Differential Entropy/]]
*[[/AWGN Channel Capacity for Continuous Input/]]
*[[/AWGN Channel Capacity for Continuous-Valued Input/]]
*[[/AWGN Channel Capacity for Discrete Input/]]
*[[/AWGN Channel Capacity for Discrete-Valued Input/]]
}}
}}
{{Collapsible-Fuß}}
{{Collapsible-Fuß}}


In addition to these theory pages, we also offer Exercises and multimedia modules that could help to clarify the teaching material:
===Exercises and multimedia===
 
{{BlaueBox|TEXT=
In addition to these theory pages,  we also offer exercises and multimedia modules on this topic,  which could help to clarify the teaching material:
 
$(1)$    [https://en.lntwww.lnt.ei.tum.de/Category:Information_Theory:_Exercises $\text{Exercises}$]
 
$(2)$    [[LNTwww:Learning_videos_to_"Information_Theory"|$\text{Learning videos}$]]
 
$(3)$    [[LNTwww:Applets_to_"Information_Theory"|$\text{Applets}$]] }}
 
 
===Further links===


*[https://en.lntwww.de/Category:Signal_Representation:_Exercises  $\text{Exercises}$]
{{BlaueBox|TEXT=
*[https://en.lntwww.de/Kategorie:Aufgaben_zu_Informationstheorie $\text{Exercises}$]
$(4)$    [[LNTwww:Bibliography_to_"Information_Theory"|$\text{Bibliography}$]]
*[[LNTwww:Lernvideos_zu_Informationstheorie|$\text{Learning videos}$]]
*[[LNTwww:HTML5-Applets_zu_Informationstheorie|$\text{redesigned applets}$]], based on HTML5, also executable on smartphones
*[[LNTwww:SWF-Applets_zu_Informationstheorie|$\text{former Applets}$]], based on SWF, executable only under WINDOWS with ''Adobe Flash Player''.


$(5)$    [[LNTwww:Imprint_for_the_book_"Information_Theory"|$\text{Impressum}$]]}}
<br><br>
<br><br>
$\text{More links:}$
<br><br>
$(1)$&nbsp; &nbsp; [[LNTwww:Literaturempfehlung_zu_Informationstheorie|$\text{Recommended literature for the book}$]]


$(2)$&nbsp; &nbsp; [[LNTwww:Weitere_Hinweise_zum_Buch_Informationstheorie|$\text{General notes about the book}$]] &nbsp; (Authors,&nbsp; other participants,&nbsp; materials as a starting point for the book,&nbsp; list of sources)
 
<br><br>




__NOTOC__
__NOTOC__
__NOEDITSECTION__
__NOEDITSECTION__
[[de:Informationstheorie]]

Latest revision as of 14:27, 16 March 2026

Brief summary

From the earliest beginnings of message transmission as an engineering discipline,  it has been the endeavour of many engineers and mathematicians  to find a quantitative measure for the

  • contained  $\rm information$  $($quite generally:  »the knowledge about something«$)$
  • in a  $\rm message$  $($here we mean  »a collection of symbols and/or states»$)$.


The  $($abstract$)$  information is communicated by the  $($concrete$)$  message and can be conceived as the interpretation of a message.

Claude Elwood Shannon  succeeded in 1948,  in establishing a consistent theory about the information content of messages,  which was revolutionary in its time and created a new,  still highly topical field of science:  »Shannon's information theory«  named after him.

This is what the fourth book in the  $\rm LNTwww$ series deals with,  in particular:

  1. Entropy of discrete-value sources with and without memory,  as well as natural message sources:  Definition,  meaning and computational possibilities.
  2. Source coding and data compression,  especially the   »Lempel–Ziv–Welch method«   and   »Huffman's entropy encoding«.
  3. Various entropies of two-dimensional discrete-value random quantities.  Mutual information and channel capacity.  Application to digital signal transmission.
  4. Discrete-value information theory.  Differential entropy.  AWGN channel capacity with continuous-valued as well as discrete-valued input.


⇒   First a  »content overview«  on the basis of the  »four main chapters«  with a total of  »13 individual chapters«  and  »106 sections«:


Content

Exercises and multimedia

In addition to these theory pages,  we also offer exercises and multimedia modules on this topic,  which could help to clarify the teaching material:

$(1)$    $\text{Exercises}$

$(2)$    $\text{Learning videos}$

$(3)$    $\text{Applets}$ 


Further links

$(4)$    $\text{Bibliography}$

$(5)$    $\text{Impressum}$



Retrieved from "https://en.lntwww.de/index.php?title=Information_Theory&oldid=54907"
Powered by MediaWiki