Difference between revisions of "Theory of Stochastic Signals/Uniformly Distributed Random Variables"

From LNTwww
m (Text replacement - "rms value" to "standard deviation")
(5 intermediate revisions by 2 users not shown)
Line 9: Line 9:
 
[[File:P_ID45__Sto_T_3_4_S1_neu100.png |frame|PDF and CDF of uniform distribution]]
 
[[File:P_ID45__Sto_T_3_4_S1_neu100.png |frame|PDF and CDF of uniform distribution]]
 
{{BlaueBox|TEXT=   
 
{{BlaueBox|TEXT=   
$\text{Definition:}$  A random variable  $x$  is said to be  '''uniformly distributed''' if it can only take values in the range of  $x_{\rm min}$  to  $x_{\rm max}$  with equal probability.  
+
$\text{Definition:}$  A random variable  $x$  is said to be  '''uniformly distributed'''  if it can only take values in the range of  $x_{\rm min}$  to  $x_{\rm max}$  with equal probability.  
  
 
The graph shows  
 
The graph shows  
 
*on the left the probability density function  $f_{x}(x)$,  
 
*on the left the probability density function  $f_{x}(x)$,  
*right the distribution function  $F_{x}(r)$
+
*right the cumulative distribution function  $F_{x}(r)$
  
  
Line 19: Line 19:
  
  
From the graph and the definition, the following properties can be derived:  
+
From the graph and this definition,  the following properties can be derived:  
 
*The probability density function  $\rm (PDF)$  has in the range from  $x_{\rm min}$  to  $x_{\rm max}$  the constant value  $1/(x_{\rm max} - x_{\rm min})$.   
 
*The probability density function  $\rm (PDF)$  has in the range from  $x_{\rm min}$  to  $x_{\rm max}$  the constant value  $1/(x_{\rm max} - x_{\rm min})$.   
*On the range limits, only half the value - that is, the average value between the left-hand and right-hand limits - is to be set for  $f_{x}(x)$  in each case.  
+
*On the range limits,  only half the value - that is,  the average value between the left–hand and right–hand limits - is to be set for  $f_{x}(x)$  in each case.  
 
*The distribution function  $\rm (CDF)$  increases linearly from  $x_{\rm min}$  to  $x_{\rm max}$  in the range from  $0$  to  $1$ .   
 
*The distribution function  $\rm (CDF)$  increases linearly from  $x_{\rm min}$  to  $x_{\rm max}$  in the range from  $0$  to  $1$ .   
*Mean and dispersion have the following values for the uniform distribution:
+
*Mean, variance and standard deviation  (standard deviation)   of the uniform distribution have the following values:
 
:$$m_{\rm 1} = \frac{\it x_ {\rm max} \rm + \it x_{\rm min}}{2},\hspace{0.5cm}
 
:$$m_{\rm 1} = \frac{\it x_ {\rm max} \rm + \it x_{\rm min}}{2},\hspace{0.5cm}
 +
\sigma^2 = \frac{[x_{\rm max} - x_{\rm min}]^2}{12},\hspace{0.5cm}
 
\sigma = \frac{\it x_{\rm max} - \it x_{\rm min}}{2 \sqrt{3}}.$$
 
\sigma = \frac{\it x_{\rm max} - \it x_{\rm min}}{2 \sqrt{3}}.$$
 
*For symmetric PDF   ⇒   $x_{\rm min} = -x_{\rm max}$  we obtain as a special case the mean  $m_1 = 0$  and the variance  $σ^2 = x_{\rm max}^2/3.$
 
*For symmetric PDF   ⇒   $x_{\rm min} = -x_{\rm max}$  we obtain as a special case the mean  $m_1 = 0$  and the variance  $σ^2 = x_{\rm max}^2/3.$
Line 36: Line 37:
  
 
   
 
   
*On the left, statistical independence of the individual samples is assumed, that is,the random variable  $x_ν$  can take all values between  $x_{\rm min}$  and  $x_{\rm max}$  with equal probability, and independently of the past  $(x_{ν-1}, x_{ν-2}, \hspace{0.1cm}\text{...}).$  
+
*On the left,  statistical independence of the individual samples is assumed,  that is,  the random variable  $x_ν$  can take all values between  $x_{\rm min}$  and  $x_{\rm max}$  with equal probability,  and independently of the past  $(x_{ν-1}, x_{ν-2}, \hspace{0.1cm}\text{...}).$  
  
  
*For the right signal  $y(t)$  this independence of successive signal values is no longer given  Rather, this sawtooth signal represents a deterministic signal.
+
*For the right signal  $y(t)$  this independence of successive signal values is no longer given.  Rather,  this sawtooth signal represents a deterministic signal.}}
  
}}
+
==Importance of the uniform distribution for communications engineering==
 
 
==Bedeutung der Gleichverteilung für die Nachrichtentechnik==
 
 
<br>
 
<br>
Die Bedeutung gleichverteilter Zufallsgrößen für die Informations&ndash; und Kommunikationstechnik ist darauf zurückzuführen, dass diese WDF–Form aus Sicht der Informationstheorie unter der Nebenbedingung&nbsp; [[Digital_Signal_Transmission/Optimierung_der_Basisbandübertragungssysteme#Leistungs.E2.80.93_und_Spitzenwertbegrenzung|Spitzenwertbegrenzung]]&nbsp; ein Optimum darstellt:  
+
The importance of uniformly distributed random variables for information&nbsp; and communication technology is due to the fact that,&nbsp; from the point of view of information theory,&nbsp; this PDF form represents an optimum under the constraint&nbsp; [[Digital_Signal_Transmission/Optimization_of_Baseband_Transmission_Systems#Power.E2.80.93_and_peak_limiting|peak limiting]]:  
*Mit keiner anderen Verteilung als der Gleichverteilung erreicht man unter dieser Voraussetzung eine größere&nbsp; [[Information_Theory/Differentielle_Entropie#Differentielle_Entropie_einiger_spitzenwertbegrenzter_Zufallsgr.C3.B6.C3.9Fen|differentielle Entropie]].   
+
*With no distribution other than the uniform distribution does one achieve greater&nbsp; [[Information_Theory/Differential_Entropy#Differential_entropy_of_some_peak-constrained_random_variables|differential entropy]]&nbsp; under this condition.   
*Mit dieser Thematik beschäftigt sich das Kapitel&nbsp; [[Information_Theory/Differentielle_Entropie|Differentielle Entropie]]&nbsp; im Buch "Informationstheorie".  
+
*This topic is dealt with in the chapter&nbsp; [[Information_Theory/Differential_Entropy|"Differential Entropy"]]&nbsp; in the book&nbsp; "Information Theory".  
  
  
Daneben sind unter Anderem noch folgende Punkte zu nennen:
+
{{BlaueBox|TEXT=
 +
$\text{In addition,&nbsp; the following points should be mentioned,&nbsp; among others:}$
  
'''(1)''' &nbsp; Die Bedeutung der Gleichverteilung für die Simulation nachrichtentechnischer Systeme ist darauf zurückzuführen, dass man entsprechende "Pseudo–Zufallsgeneratoren" relativ einfach realisieren kann, und dass sich daraus andere Verteilungen wie zum Beispiel die&nbsp; [[Theory_of_Stochastic_Signals/Gaußverteilte_Zufallsgröße#Wahrscheinlichkeitsdichte-_und_Verteilungsfunktion|Gaußverteilung]]&nbsp; und die&nbsp; [[Theory_of_Stochastic_Signals/Exponentialverteilte_Zufallsgrößen#Einseitige_Exponentialverteilung|Exponentialverteilung]]&nbsp; leicht ableiten lassen.  
+
'''(1)''' &nbsp; The importance of the uniform distribution for the simulation of communication systems is due to the fact that one can realize corresponding&nbsp; "pseudo-random generators"&nbsp; relatively easily,&nbsp; and that other distributions,&nbsp; such as the&nbsp; [[Theory_of_Stochastic_Signals/Gaussian_Distributed_Random_Variables#Probability_density_function_.26_cumulative_density_function|Gaussian distribution]]&nbsp; and the&nbsp; [[Theory_of_Stochastic_Signals/Exponentially_Distributed_Random_Variables#One-sided_exponential_distribution|exponential distribution]]&nbsp; can be easily derived.  
  
'''(2)''' &nbsp; In der ''Bildverarbeitung & Bildcodierung'' wird oft vereinfachend mit der Gleichverteilung anstelle der tatsächlichen, meist sehr viel komplizierteren Verteilung des Originalbildes gerechnet, da der Unterschied des Informationsgehaltes zwischen ''natürlichem Bild'' und dem auf der Gleichverteilung basierenden Modell relativ gering ist.  
+
'''(2)''' &nbsp; In&nbsp; "Image Processing & Coding",&nbsp; simplifying calculations are often made using the uniform distribution instead of the actual distribution of the original image,&nbsp; which is usually much more complicated,&nbsp; since the difference in information content between a&nbsp; "natural image"&nbsp; and the model based on the uniform distribution is relatively small.  
  
'''(3)''' &nbsp; Für die Modellierung übertragungstechnischer Systeme sind gleichverteilte Zufallsgrößen dagegen die Ausnahme.&nbsp; Ein Beispiel für eine tatsächlich (nahezu) gleichverteilte Zufallsgröße ist die Phase bei kreissymmetrischen Störungen, wie sie beispielsweise bei&nbsp; ''Quadratur&ndash;Amplitudenmodulationsverfahren''&nbsp; (QAM) auftreten.  
+
'''(3)''' &nbsp; For modeling transmission systems,&nbsp; on the other hand,&nbsp; uniformly distributed random variables are the exception.&nbsp; An example of an actually&nbsp; (nearly)&nbsp; uniformly distributed random variable is the phase in the presence of circularly symmetric interference,&nbsp; such as occurs in&nbsp; "quadrature amplitude modulation techniques"&nbsp; (QAM).}}
 +
  
Das interaktive Applet&nbsp; [[Applets:WDF,_VTF_und_Momente_spezieller_Verteilungen_(Applet)|WDF, VTF und Momente spezieller Verteilungen]]&nbsp; berechnet alle Kenngrößen der Gleichverteilung für beliebige Parameter&nbsp; $x_{\rm min}$&nbsp; und&nbsp; $x_{\rm max}$.  
+
The interactive applet&nbsp; [[Applets:PDF,_CDF_and_Moments_of_Special_Distributions|"PDF, CDF and moments of special distributions"]]&nbsp; calculates all characteristics of the uniform distribution for any parameters&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$.  
  
  
==Erzeugung einer Gleichverteilung mit Pseudo&ndash;Noise&ndash;Generatoren==
+
==Generating a uniform distribution with pseudo&ndash;noise generators==
 
<br>
 
<br>
{{BlaueBox|TEXT=
+
{{BlaueBox|TEXT=
$\text{Definition}$&nbsp; Die heute verwendeten Zufallsgeneratoren sind meist&nbsp; '''pseudozufällig'''.&nbsp; Das bedeutet,  
+
$\text{Definition}$&nbsp; The random generators used today are mostly&nbsp; '''pseudo&ndash;random'''.&nbsp; This means,  
*dass die erzeugte Folge als das Ergebnis eines festen Algorithmuses eigentlich deterministisch ist,  
+
*that the sequence generated is actually deterministic as the result of a fixed algorithm,  
*für den Anwender jedoch aufgrund der großen Periodenlänge&nbsp; $P$&nbsp; als stochastisch erscheint.  
+
*but appears to the user as stochastic due to the large period length&nbsp; $P$.  
  
  
Mehr hierzu im Kapitel&nbsp; [[Theory_of_Stochastic_Signals/Erzeugung_von_diskreten_Zufallsgrößen|Erzeugung von diskreten Zufallsgrößen]]. }}  
+
More on this in the chapter&nbsp; [[Theory_of_Stochastic_Signals/Generation_of_Discrete_Random_Variables|Generation of discrete random variables]]. }}  
  
  
Für die Systemsimulation haben Pseudo&ndash;Noise&nbsp; $\rm (PN)$&ndash;Generatoren gegenüber echten Zufallsgeneratoren den entscheidenden Vorteil, dass die erzeugten Zufallsfolgen ohne Speicherung reproduzierbar sind, was
+
For system simulation,&nbsp; pseudo&ndash;noise&nbsp; $\rm (PN)$&nbsp; generators have the distinct advantage over true random generators that the generated random sequences can be reproduced without storage, which
*zum einen den Vergleich verschiedener Systemmodelle ermöglicht, und
+
*allows the comparison of different system models,&nbsp; and
*auch die Fehlersuche wesentlich erleichtert.  
+
*also makes troubleshooting much easier.  
  
  
Ein Zufallsgenerator sollte dabei folgende Kriterien erfüllen:  
+
{{BlaueBox|TEXT=
 +
$\text{A random sequence generator should meet the following criteria:}$
  
'''(1)''' &nbsp; Die Zufallsgrößen&nbsp; $x_ν$&nbsp; einer generierten Folge sollten mit sehr guter Näherung gleichverteilt sein. Bei wertdiskreter Darstellung an einem Rechner erfordert dies unter anderem eine hinreichend&nbsp; ''hohe Bitauflösung'',&nbsp; zum Beispiel mit&nbsp; $32$&nbsp; oder&nbsp; $64$&nbsp; Bit pro Abtastwert.  
+
'''(1)''' &nbsp; The random variables&nbsp; $x_ν$&nbsp; of a generated sequence should be uniformly distributed with very good approximation.&nbsp; For discrete-value representation on a computer,&nbsp; this requires,&nbsp; among other things,&nbsp; a sufficiently&nbsp; high bit resolution,&nbsp; for example, with&nbsp; $32$&nbsp; or&nbsp; $64$&nbsp; bits per sample.  
  
'''(2)''' &nbsp; Bildet man aus der sequentiellen Zufallsfolge&nbsp; $〈x_ν〉$&nbsp; jeweils nichtüberlappende Paare von Zufallsgrößen, beispielsweise&nbsp; $(x_ν, x_{ν+1})$,&nbsp; $(x_{ν+2}$,&nbsp; $x_{ν+3})$, ... , so sollten diese&nbsp; ''Tupel''&nbsp; in einer zweidimensionalen Darstellung innerhalb eines Quadrates ebenfalls gleichverteilt sein.  
+
'''(2)''' &nbsp; If one forms from the sequential random sequence&nbsp; $〈x_ν〉$&nbsp; respectively non-overlapping pairs of random variables,&nbsp; for example&nbsp; $(x_ν, x_{ν+1})$,&nbsp; $(x_{ν+2}$,&nbsp; $x_{ν+3})$, ... , then these&nbsp; "tuples"&nbsp; should also be equally distributed in a two-dimensional representation within a square.  
  
'''(3)''' &nbsp; Bildet man aus der sequentiellen Folge&nbsp; $〈x_ν〉$&nbsp; nicht überlappende&nbsp; $n$&ndash;''Tupel'' &nbsp; von Zufallsgrößen &nbsp; ⇒ &nbsp; $(x_ν$, ... , $x_{ν+n–1})$,&nbsp; $(x_{ν+n}$, ... , $x_{ν+2n–1})$&nbsp; usw., so sollten auch diese innerhalb eines&nbsp; $n$&ndash;dimensionalen Würfels möglichst die Gleichverteilung ergeben.  
+
'''(3)''' &nbsp; If one forms from the sequential series&nbsp; $〈x_ν〉$&nbsp; non-overlapping&nbsp; $n$&ndash;tuples &nbsp; of random variables &nbsp; ⇒ &nbsp; $(x_ν$, . ... , $x_{ν+n-1})$,&nbsp; $(x_{ν+n}$, ... , $x_{ν+2n-1})$&nbsp; etc., then these should also yield the uniform distribution within a&nbsp; $n$&ndash;dimensional cube,&nbsp; if possible.}}
  
  
''Anmerkung:''
+
 
*Die erste Forderung bezieht sich ausschließlich auf die&nbsp; ''Amplitudenverteilung''&nbsp; $\rm (WDF)$&nbsp; und ist im Allgemeinen leichter zu erfüllen.  
+
Note:
*Die weiteren Forderungen gewährleisten eine „ausreichende Zufälligkeit” der Folge.&nbsp; Sie betreffen die statistische Unabhängigkeit aufeinander folgender Zufallswerte.  
+
*The first requirement refers exclusively to the&nbsp; "amplitude distribution"&nbsp; $\rm (PDF)$&nbsp; and is generally easier to satisfy.  
 +
*The other requirements ensure&nbsp; "sufficient randomness"&nbsp; of the sequence.&nbsp; They concern the statistical independence of successive random values.  
  
  
 
==Multiplicative Congruental Generator==
 
==Multiplicative Congruental Generator==
 
<br>
 
<br>
$\text{Multiplicative Congruental Generator}$&nbsp; ist das bekannteste Verfahren  zur Erzeugung einer Folge&nbsp; $〈 x_\nu 〉$&nbsp; mit gleichverteilten Werten&nbsp; $ x_\nu$&nbsp; zwischen&nbsp; $0$&nbsp; und&nbsp; $1$.&nbsp; Diese Methode wird hier stichpunktartig angegeben:  
+
$\text{Multiplicative Congruental Generator}$&nbsp; is the best known method for generating a sequence&nbsp; $〈 x_\nu 〉$&nbsp; with equally distributed values&nbsp; $ x_\nu$&nbsp; between&nbsp; $0$&nbsp; and&nbsp; $1$.&nbsp; This method is given here in a bullet-point fashion:  
  
'''(1)''' &nbsp; Diese Zufallsgeneratoren basieren auf der sukzessiven Manipulation einer Integervariablen&nbsp; $k$.&nbsp; Geschieht die Zahlendarstellung im Rechner mit&nbsp; $L$&nbsp; Bit, so nimmt diese Variable bei geeigneter Behandlung des Vorzeichenbits alle Werte zwischen&nbsp; $1$&nbsp; und&nbsp; $2^{L 1}$&nbsp; jeweils genau einmal an.  
+
'''(1)''' &nbsp; These random generators are based on the successive manipulation of an integer variable&nbsp; $k$.&nbsp; If the number representation in the computer happens with&nbsp; $L$&nbsp; bit,&nbsp; this variable takes all values between&nbsp; $1$&nbsp; and&nbsp; $2^{L - 1}$&nbsp; exactly once each,&nbsp; if the sign bit is handled appropriately.  
  
'''(2)''' &nbsp; Die hieraus abgeleitete Zufallsgröße&nbsp; $x={k}/{\rm 2^{\it L - \rm 1}}$&nbsp; ist ebenfalls diskret&nbsp; $($mit Stufenzahl&nbsp; $M = 2^{L– 1})$:  
+
'''(2)''' &nbsp; The random variable derived from this&nbsp; $x={k}/{\rm 2^{\it L - \rm 1}}$&nbsp; is also discrete&nbsp; $($with level number&nbsp; $M = 2^{L- 1})$:  
:$$x={k}/{\rm 2^{\it L - \rm 1}} = k\cdot \Delta x \in   \{\Delta x, \hspace{0.05cm}2\cdot \Delta x,\hspace{0.05cm}\text{ ...}\hspace{0.05cm} , \hspace{0.05cm}1-\Delta x,\hspace{0.05cm} 1\}.$$
+
:$$x={k}/{\rm 2^{\it L - \rm 1}} = k\cdot \delta x \in \{\delta x, \hspace{0.05cm}2\cdot \delta x,\hspace{0.05cm}\text{ ...}\hspace{0.05cm} , \hspace{0.05cm}1-\delta x,\hspace{0.05cm} 1\}.$$
:Ist die Bitanzahl $L$ hinreichend groß, so ist der Abstand&nbsp; $Δx = 1/2^{L– 1}$&nbsp; zwischen zwei möglichen Werten sehr klein, und man kann&nbsp; $x$&nbsp; im Rahmen der Simulationsgenauigkeit durchaus als eine wertkontinuierliche Zufallsgröße interpretieren.  
+
:If the bit number&nbsp; $L$&nbsp; is sufficiently large,&nbsp; the distance&nbsp; $Δx = 1/2^{L- 1}$&nbsp; between two possible values is very small,&nbsp; and one may well interpret&nbsp; $x$&nbsp; as a continuous-valued random variable in the context of simulation accuracy.  
  
'''(3)''' &nbsp; Die rekursive Generierungsvorschrift eines solchen&nbsp; ''Multiplicative Congruential Generators''&nbsp; lautet:  
+
'''(3)''' &nbsp; The recursive generation rule of such&nbsp; "multiplicative congruential generators"&nbsp; is:  
 
:$$k_\nu=(a\cdot k_{\nu-1})\hspace{0.1cm} \rm mod \hspace{0.1cm} \it m.$$
 
:$$k_\nu=(a\cdot k_{\nu-1})\hspace{0.1cm} \rm mod \hspace{0.1cm} \it m.$$
  
'''(4)''' &nbsp; Die statistischen Eigenschaften der Folge hängen entscheidend von den Parametern&nbsp; $a$&nbsp; und&nbsp; $m$&nbsp; ab. Der Startwert&nbsp; $k_0$&nbsp; hat dagegen für die Statistik eine eher untergeordnete Bedeutung.  
+
'''(4)''' &nbsp; The statistical properties of the sequence depend crucially on the parameters&nbsp; $a$&nbsp; and&nbsp; $m$.&nbsp; The initial value&nbsp; $k_0$&nbsp; has a minor importance for the statistics.  
  
'''(5)''' &nbsp; Die besten Ergebnisse erzielt man mit der Basis&nbsp; $m =2\hspace{0.05cm}^l-1$, wobei&nbsp; $l$&nbsp; eine beliebige natürliche Zahl angibt. Weit verbreitet ist bei Rechnern mit 32 Bit-Architektur und einem Vorzeichenbit die Basis&nbsp; $m = 2^{31} - 1 = 2\hspace{0.08cm}147\hspace{0.08cm}483\hspace{0.08cm}647$.&nbsp; Ein entsprechender Algorithmus lautet:  
+
'''(5)''' &nbsp; The best results are obtained with the base&nbsp; $m =2\hspace{0.05cm}^l-1$,&nbsp; where&nbsp; $l$&nbsp; denotes any natural number.&nbsp; Widely used in computers with 32-bit architecture and one sign bit is the base&nbsp; $m = 2^{31} - 1 = 2\hspace{0.08cm}147\hspace{0.08cm}483\hspace{0.08cm}647$.&nbsp; A corresponding algorithm is:  
 
:$$k_\nu=(16807\cdot k_{\nu-1})\hspace{0.1cm} \rm mod\hspace{0.1cm}(2^{31}-1).$$
 
:$$k_\nu=(16807\cdot k_{\nu-1})\hspace{0.1cm} \rm mod\hspace{0.1cm}(2^{31}-1).$$
  
'''(6)''' &nbsp; Für einen solchen Generator ist nur der Startwert&nbsp; $k_0 = 0$&nbsp; nicht erlaubt.&nbsp; Für&nbsp; $k_0 \ne 0$&nbsp; beträgt die Periodendauer&nbsp; $P = 2^{31} - 2.$
+
'''(6)''' &nbsp; For such a generator, only the initial value&nbsp; $k_0 = 0$&nbsp; is not allowed.&nbsp; For&nbsp; $k_0 \ne 0$&nbsp; the period duration&nbsp; $P = 2^{31} - 2.$
 +
 
 +
[[File:EN_Sto_T_3_4_S4.png |right|frame| Multiplicative Congruental Generator&nbsp; (C program)]]
 +
{{GraueBox|TEXT=
 +
$\text{Example 2:}$&nbsp; We analyze the&nbsp; "Multiplicative Congruental Generator"&nbsp; in more detail:
 +
 
 +
 
 +
*The algorithm,&nbsp; however,&nbsp; cannot be implemented directly on a 32 bit computer,&nbsp; since the multiplication result requires up to 46 bits.
  
[[File:EN_Sto_T_3_4_S4.png |right|frame| Multiplicative Congruental Generator (C-Programm)]]
+
   
{{GraueBox|TEXT=  
+
*But it can be modified in such a way that at no time during the calculation the 32 bit&ndash;integer number range is exceeded.
$\text{Beispiel 2:}$&nbsp; Wir analysieren den oben beschriebenen&nbsp; ''Multiplicative Congruental Generator''&nbsp; genauer:
+
 
*Den Algorithmus kann man allerdings auf einem 32 Bit&ndash;Rechner nicht direkt implementieren, da das Multiplikationsergebnis bis zu 46 Bit benötigt.  
+
*Er kann aber so abgewandelt werden, dass zu keinem Zeitpunkt der Berechnung der 32 Bit&ndash;Integerzahlenbereich überschritten wird.
+
*The C program&nbsp; $\text{uniform( )}$&nbsp; thus modified is given on the right.}}
*Das so modifizierte C-Programm&nbsp; $\text{uniform( )}$&nbsp; ist rechts angegeben.}}
 
  
==Aufgaben zum Kapitel==
+
==Exercises for the chapter==
 
<br>
 
<br>
[[Aufgaben:3.5 Dreieck- und Trapezsignal|Aufgabe 3.5: Dreieck&ndash; und Trapezsignal]]
+
[[Aufgaben:Exercise_3.5:_Triangular_and_Trapezoidal_Signal|Exercise 3.5: Triangular and Trapezoidal Signal]]
  
[[Aufgaben:3.5Z Antennengebiete|Aufgabe 3.5Z: Antennengebiete]]
+
[[Aufgaben:Exercise_3.5Z:_Antenna_Areas|Exercise 3.5Z: Antenna Areas]]
  
  
 
{{Display}}
 
{{Display}}

Revision as of 16:45, 15 February 2022

General description and definition


PDF and CDF of uniform distribution

$\text{Definition:}$  A random variable  $x$  is said to be  uniformly distributed  if it can only take values in the range of  $x_{\rm min}$  to  $x_{\rm max}$  with equal probability.

The graph shows

  • on the left the probability density function  $f_{x}(x)$,
  • right the cumulative distribution function  $F_{x}(r)$


of such an equally distributed random variable  $x$.


From the graph and this definition,  the following properties can be derived:

  • The probability density function  $\rm (PDF)$  has in the range from  $x_{\rm min}$  to  $x_{\rm max}$  the constant value  $1/(x_{\rm max} - x_{\rm min})$. 
  • On the range limits,  only half the value - that is,  the average value between the left–hand and right–hand limits - is to be set for  $f_{x}(x)$  in each case.
  • The distribution function  $\rm (CDF)$  increases linearly from  $x_{\rm min}$  to  $x_{\rm max}$  in the range from  $0$  to  $1$ .
  • Mean, variance and standard deviation  (standard deviation)  of the uniform distribution have the following values:
$$m_{\rm 1} = \frac{\it x_ {\rm max} \rm + \it x_{\rm min}}{2},\hspace{0.5cm} \sigma^2 = \frac{[x_{\rm max} - x_{\rm min}]^2}{12},\hspace{0.5cm} \sigma = \frac{\it x_{\rm max} - \it x_{\rm min}}{2 \sqrt{3}}.$$
  • For symmetric PDF   ⇒   $x_{\rm min} = -x_{\rm max}$  we obtain as a special case the mean  $m_1 = 0$  and the variance  $σ^2 = x_{\rm max}^2/3.$


$\text{Example 1:}$  The graph shows two signal waveforms with uniform amplitude distribution.

Examples of uniformly distributed signals


  • On the left,  statistical independence of the individual samples is assumed,  that is,  the random variable  $x_ν$  can take all values between  $x_{\rm min}$  and  $x_{\rm max}$  with equal probability,  and independently of the past  $(x_{ν-1}, x_{ν-2}, \hspace{0.1cm}\text{...}).$


  • For the right signal  $y(t)$  this independence of successive signal values is no longer given.  Rather,  this sawtooth signal represents a deterministic signal.

Importance of the uniform distribution for communications engineering


The importance of uniformly distributed random variables for information  and communication technology is due to the fact that,  from the point of view of information theory,  this PDF form represents an optimum under the constraint  peak limiting:

  • With no distribution other than the uniform distribution does one achieve greater  differential entropy  under this condition.
  • This topic is dealt with in the chapter  "Differential Entropy"  in the book  "Information Theory".


$\text{In addition,  the following points should be mentioned,  among others:}$

(1)   The importance of the uniform distribution for the simulation of communication systems is due to the fact that one can realize corresponding  "pseudo-random generators"  relatively easily,  and that other distributions,  such as the  Gaussian distribution  and the  exponential distribution  can be easily derived.

(2)   In  "Image Processing & Coding",  simplifying calculations are often made using the uniform distribution instead of the actual distribution of the original image,  which is usually much more complicated,  since the difference in information content between a  "natural image"  and the model based on the uniform distribution is relatively small.

(3)   For modeling transmission systems,  on the other hand,  uniformly distributed random variables are the exception.  An example of an actually  (nearly)  uniformly distributed random variable is the phase in the presence of circularly symmetric interference,  such as occurs in  "quadrature amplitude modulation techniques"  (QAM).


The interactive applet  "PDF, CDF and moments of special distributions"  calculates all characteristics of the uniform distribution for any parameters  $x_{\rm min}$  and  $x_{\rm max}$.


Generating a uniform distribution with pseudo–noise generators


$\text{Definition}$  The random generators used today are mostly  pseudo–random.  This means,

  • that the sequence generated is actually deterministic as the result of a fixed algorithm,
  • but appears to the user as stochastic due to the large period length  $P$.


More on this in the chapter  Generation of discrete random variables.


For system simulation,  pseudo–noise  $\rm (PN)$  generators have the distinct advantage over true random generators that the generated random sequences can be reproduced without storage, which

  • allows the comparison of different system models,  and
  • also makes troubleshooting much easier.


$\text{A random sequence generator should meet the following criteria:}$

(1)   The random variables  $x_ν$  of a generated sequence should be uniformly distributed with very good approximation.  For discrete-value representation on a computer,  this requires,  among other things,  a sufficiently  high bit resolution,  for example, with  $32$  or  $64$  bits per sample.

(2)   If one forms from the sequential random sequence  $〈x_ν〉$  respectively non-overlapping pairs of random variables,  for example  $(x_ν, x_{ν+1})$,  $(x_{ν+2}$,  $x_{ν+3})$, ... , then these  "tuples"  should also be equally distributed in a two-dimensional representation within a square.

(3)   If one forms from the sequential series  $〈x_ν〉$  non-overlapping  $n$–tuples   of random variables   ⇒   $(x_ν$, . ... , $x_{ν+n-1})$,  $(x_{ν+n}$, ... , $x_{ν+2n-1})$  etc., then these should also yield the uniform distribution within a  $n$–dimensional cube,  if possible.


Note:

  • The first requirement refers exclusively to the  "amplitude distribution"  $\rm (PDF)$  and is generally easier to satisfy.
  • The other requirements ensure  "sufficient randomness"  of the sequence.  They concern the statistical independence of successive random values.


Multiplicative Congruental Generator


$\text{Multiplicative Congruental Generator}$  is the best known method for generating a sequence  $〈 x_\nu 〉$  with equally distributed values  $ x_\nu$  between  $0$  and  $1$.  This method is given here in a bullet-point fashion:

(1)   These random generators are based on the successive manipulation of an integer variable  $k$.  If the number representation in the computer happens with  $L$  bit,  this variable takes all values between  $1$  and  $2^{L - 1}$  exactly once each,  if the sign bit is handled appropriately.

(2)   The random variable derived from this  $x={k}/{\rm 2^{\it L - \rm 1}}$  is also discrete  $($with level number  $M = 2^{L- 1})$:

$$x={k}/{\rm 2^{\it L - \rm 1}} = k\cdot \delta x \in \{\delta x, \hspace{0.05cm}2\cdot \delta x,\hspace{0.05cm}\text{ ...}\hspace{0.05cm} , \hspace{0.05cm}1-\delta x,\hspace{0.05cm} 1\}.$$
If the bit number  $L$  is sufficiently large,  the distance  $Δx = 1/2^{L- 1}$  between two possible values is very small,  and one may well interpret  $x$  as a continuous-valued random variable in the context of simulation accuracy.

(3)   The recursive generation rule of such  "multiplicative congruential generators"  is:

$$k_\nu=(a\cdot k_{\nu-1})\hspace{0.1cm} \rm mod \hspace{0.1cm} \it m.$$

(4)   The statistical properties of the sequence depend crucially on the parameters  $a$  and  $m$.  The initial value  $k_0$  has a minor importance for the statistics.

(5)   The best results are obtained with the base  $m =2\hspace{0.05cm}^l-1$,  where  $l$  denotes any natural number.  Widely used in computers with 32-bit architecture and one sign bit is the base  $m = 2^{31} - 1 = 2\hspace{0.08cm}147\hspace{0.08cm}483\hspace{0.08cm}647$.  A corresponding algorithm is:

$$k_\nu=(16807\cdot k_{\nu-1})\hspace{0.1cm} \rm mod\hspace{0.1cm}(2^{31}-1).$$

(6)   For such a generator, only the initial value  $k_0 = 0$  is not allowed.  For  $k_0 \ne 0$  the period duration  $P = 2^{31} - 2.$

Multiplicative Congruental Generator  (C program)

$\text{Example 2:}$  We analyze the  "Multiplicative Congruental Generator"  in more detail:


  • The algorithm,  however,  cannot be implemented directly on a 32 bit computer,  since the multiplication result requires up to 46 bits.


  • But it can be modified in such a way that at no time during the calculation the 32 bit–integer number range is exceeded.


  • The C program  $\text{uniform( )}$  thus modified is given on the right.

Exercises for the chapter


Exercise 3.5: Triangular and Trapezoidal Signal

Exercise 3.5Z: Antenna Areas