Information Theory: Difference between revisions

From LNTwww
← Older edit
No edit summary
Add German interlanguage link
 
(25 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Since the early beginnings of communications as an engineering discipline, many engineers and mathematicians have sought to find a quantitative measure of
===Brief summary===
*the $\rm Information$  (in general: "the knowledge of something") contained in a  $\rm message$  (here we understand  "a collection of symbols and/or states").


{{BlueBox|TEXT=From the earliest beginnings of message transmission as an engineering discipline,  it has been the endeavour of many engineers and mathematicians  to find a quantitative measure for the
*contained  $\rm information$  $($quite generally:  »the knowledge about something«$)$


The  (abstract)  information is communicated by the  (concrete)  message and can be seen as an interpretation of a message.  
*in a  $\rm message$  $($here we mean  »a collection of symbols and/or states»$)$.


[https://de.wikipedia.org/wiki/Claude_Shannon Claude Elwood Shannon]  succeeded in 1948 in establishing a consistent theory of the information content of messages,  which was revolutionary in its time and created a new, still highly topical field of science:  the theory named after him  $\text{Shannon's Information Theory}$.
The  $($abstract$)$  information is communicated by the  $($concrete$)$  message and can be conceived as the interpretation of a message.  


The subject matter corresponds to a  $\text{lecture with two semester hours per week (SWS) and one additional SWS exercise}$.
[https://en.wikipedia.org/wiki/Claude_Shannon '''Claude Elwood Shannon''']  succeeded in 1948,  in establishing a consistent theory about the information content of messages,  which was revolutionary in its time and created a new,  still highly topical field of science:   »'''Shannon's information theory«'''  named after him.


Here is a table of contents based on the  $\text{four main chapters}$  with a total of  $\text{13 individual chapters}$.
This is what the fourth book in the  $\rm LNTwww$ series deals with,  in particular:
# Entropy of discrete-value sources with and without memory,  as well as natural message sources:  Definition,  meaning and computational possibilities.
# Source coding and data compression,  especially the   »Lempel–Ziv–Welch method«   and   »Huffman's entropy encoding«. 
# Various entropies of two-dimensional discrete-value random quantities.  Mutual information and channel capacity.  Application to digital signal transmission.   
# Discrete-value information theory.  Differential entropy.  AWGN channel capacity with continuous-valued as well as discrete-valued input.
 
 
⇒   First a  »'''content overview'''«  on the basis of the  »'''four main chapters'''«  with a total of  »'''13 individual chapters'''«  and  »'''106 sections'''«:}}
 
 
 
===Content===


===Contents===
{{Collapsible-Kopf}}
{{Collapsible-Kopf}}
{{Collapse1| header=Entropy of Discrete Sources
{{Collapse1| header=Entropy of Discrete Sources
Line 37: Line 47:
|submenu=
|submenu=
*[[/Differential Entropy/]]
*[[/Differential Entropy/]]
*[[/AWGN Channel Capacity for Continuous Input/]]
*[[/AWGN Channel Capacity for Continuous-Valued Input/]]
*[[/AWGN Channel Capacity for Discrete Input/]]
*[[/AWGN Channel Capacity for Discrete-Valued Input/]]
}}
}}
{{Collapsible-Fuß}}
{{Collapsible-Fuß}}


In addition to these theory pages, we also offer exercises and multimedia modules that could help to clarify the teaching material:
===Exercises and multimedia===


*[https://en.lntwww.de/Category:Information_Theory:_Exercises $\text{Exercises}$]
{{BlaueBox|TEXT=
*[[LNTwww:Lernvideos_zu_Informationstheorie|$\text{Learning videos}$]]
In addition to these theory pages,  we also offer exercises and multimedia modules on this topic,  which could help to clarify the teaching material:
*[[LNTwww:HTML5-Applets_zu_Informationstheorie|$\text{Applets}$]]
 
$(1)$    [https://en.lntwww.lnt.ei.tum.de/Category:Information_Theory:_Exercises $\text{Exercises}$]
 
$(2)$    [[LNTwww:Learning_videos_to_"Information_Theory"|$\text{Learning videos}$]]
 
$(3)$    [[LNTwww:Applets_to_"Information_Theory"|$\text{Applets}$]] }}
 
 
===Further links===
 
{{BlaueBox|TEXT=
$(4)$    [[LNTwww:Bibliography_to_"Information_Theory"|$\text{Bibliography}$]]
 
$(5)$    [[LNTwww:Imprint_for_the_book_"Information_Theory"|$\text{Impressum}$]]}}
<br><br>
<br><br>
$\text{Other links:}$


$(1)$&nbsp; &nbsp; [[LNTwww:Bibliography_to_Information_Theory|$\text{Bibliography to the book}$]]


$(2)$&nbsp; &nbsp; [[LNTwww:General_Notes_to_the_Book_Information_Theory|$\text{Notes on the authors and materials used in the preparation of the book}$]]
 
<br><br>


__NOTOC__
__NOTOC__
__NOEDITSECTION__
__NOEDITSECTION__
[[de:Informationstheorie]]

Latest revision as of 14:27, 16 March 2026

Brief summary

From the earliest beginnings of message transmission as an engineering discipline,  it has been the endeavour of many engineers and mathematicians  to find a quantitative measure for the

  • contained  $\rm information$  $($quite generally:  »the knowledge about something«$)$
  • in a  $\rm message$  $($here we mean  »a collection of symbols and/or states»$)$.


The  $($abstract$)$  information is communicated by the  $($concrete$)$  message and can be conceived as the interpretation of a message.

Claude Elwood Shannon  succeeded in 1948,  in establishing a consistent theory about the information content of messages,  which was revolutionary in its time and created a new,  still highly topical field of science:  »Shannon's information theory«  named after him.

This is what the fourth book in the  $\rm LNTwww$ series deals with,  in particular:

  1. Entropy of discrete-value sources with and without memory,  as well as natural message sources:  Definition,  meaning and computational possibilities.
  2. Source coding and data compression,  especially the   »Lempel–Ziv–Welch method«   and   »Huffman's entropy encoding«.
  3. Various entropies of two-dimensional discrete-value random quantities.  Mutual information and channel capacity.  Application to digital signal transmission.
  4. Discrete-value information theory.  Differential entropy.  AWGN channel capacity with continuous-valued as well as discrete-valued input.


⇒   First a  »content overview«  on the basis of the  »four main chapters«  with a total of  »13 individual chapters«  and  »106 sections«:


Content

Exercises and multimedia

In addition to these theory pages,  we also offer exercises and multimedia modules on this topic,  which could help to clarify the teaching material:

$(1)$    $\text{Exercises}$

$(2)$    $\text{Learning videos}$

$(3)$    $\text{Applets}$ 


Further links

$(4)$    $\text{Bibliography}$

$(5)$    $\text{Impressum}$



Retrieved from "https://en.lntwww.de/index.php?title=Information_Theory&oldid=54907"
Powered by MediaWiki