Theory of Stochastic Signals/Uniformly Distributed Random Variables - Revision history

Guenter at 17:01, 20 February 2024

2024-02-20T17:01:13Z

Guenter at 16:27, 20 February 2024

2024-02-20T16:27:09Z

Guenter at 08:59, 22 December 2022

2022-12-22T08:59:25Z

Hwang at 19:19, 20 December 2022

2022-12-20T19:19:20Z

Hwang at 20:27, 13 December 2022

2022-12-13T20:27:37Z

Hwang at 20:25, 13 December 2022

2022-12-13T20:25:54Z

Hwang at 20:25, 13 December 2022

2022-12-13T20:25:17Z

Hwang at 20:23, 13 December 2022

2022-12-13T20:23:36Z

Hwang at 20:17, 13 December 2022

2022-12-13T20:17:52Z

Bene: Text replacement - "rms value" to "standard deviation"

2022-02-15T15:45:53Z

Text replacement - "rms value" to "standard deviation"

@@ Line 60: / Line 60: @@
-The interactive applet&nbsp; [[Applets:PDF,_CDF_and_Moments_of_Special_Distributions|"PDF, CDF and moments of special distributions"]]&nbsp; calculates all characteristics of the uniform distribution for any parameters&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$.
+The HTML5/JavaScript applet&nbsp; [[Applets:PDF,_CDF_and_Moments_of_Special_Distributions|&raquo;PDF, CDF and moments of special distributions&laquo;]]&nbsp; calculates the characteristics of the uniform distribution for any parameters&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$.
@@ Line 67: / Line 67: @@
 {{BlaueBox|TEXT=
 $\text{Definition}$&nbsp; The random generators used today are mostly&nbsp; &raquo;'''pseudo&ndash;random'''&laquo;.&nbsp; This means,
 *that the sequence generated is actually deterministic as the result of a fixed algorithm,
 *but appears to the user as stochastic due to the large period length&nbsp; $P$.
-More on this in the chapter&nbsp; [[Theory_of_Stochastic_Signals/Generation_of_Discrete_Random_Variables|"Generation of discrete random variables"]]. }}
+More on this in the chapter&nbsp; [[Theory_of_Stochastic_Signals/Generation_of_Discrete_Random_Variables|&raquo;Generation of discrete random variables&laquo;]]. }}
 {{BlaueBox|TEXT=
 $\text{A random sequence generator should meet the following criteria:}$
-'''(1)''' &nbsp; The random variables&nbsp; $x_ν$&nbsp; of a generated sequence should be uniformly distributed with very good approximation.&nbsp; For discrete-value representation on a computer,&nbsp; this requires,&nbsp; among other things,&nbsp; a sufficiently&nbsp; high bit resolution,&nbsp; for example, with&nbsp; $32$&nbsp; or&nbsp; $64$&nbsp; bits per sample.
+'''(1)''' &nbsp; The random variables&nbsp; $x_ν$&nbsp; of a generated sequence should be uniformly distributed with very good approximation.&nbsp; For the discrete-value representation on a computer,&nbsp; this requires,&nbsp; among other things,&nbsp; a sufficiently&nbsp; high bit resolution,&nbsp; for example, with&nbsp; $32$&nbsp; or&nbsp; $64$&nbsp; bits per sample.
-'''(2)''' &nbsp; If one forms from the sequential random sequence&nbsp; $〈x_ν〉$&nbsp; respectively non-overlapping pairs of random variables,&nbsp; for example&nbsp; $(x_ν, x_{ν+1})$,&nbsp; $(x_{ν+2}$,&nbsp; $x_{ν+3})$, ... , then these&nbsp; "tuples"&nbsp; should also be equally distributed in a two-dimensional representation within a square.
+'''(2)''' &nbsp; If one forms from the sequential random sequence&nbsp; $〈x_ν〉$&nbsp; respectively non-overlapping pairs of random variables,&nbsp; for example&nbsp; $(x_ν, x_{ν+1})$,&nbsp; $(x_{ν+2}$,&nbsp; $x_{ν+3})$, ... , then these&nbsp; &raquo;tuples&laquo;&nbsp; should also be equally distributed in a two-dimensional representation within a square.
-'''(3)''' &nbsp; If one forms from the sequential series&nbsp; $〈x_ν〉$&nbsp; non-overlapping&nbsp; $n$&ndash;tuples &nbsp; of random variables &nbsp; ⇒ &nbsp; $(x_ν$, . ... , $x_{ν+n-1})$,&nbsp; $(x_{ν+n}$, ... , $x_{ν+2n-1})$&nbsp; etc., then these should also yield the uniform distribution within a&nbsp; $n$&ndash;dimensional cube,&nbsp; if possible.}}
+'''(3)''' &nbsp; If one forms from the sequential series&nbsp; $〈x_ν〉$&nbsp; non-overlapping&nbsp; $n$&ndash;tuples &nbsp; of random variables &nbsp; ⇒ &nbsp; $(x_ν$, . ... , $x_{ν+n-1})$,&nbsp; $(x_{ν+n}$, ... , $x_{ν+2n-1})$&nbsp; and so on, <br>then these should also yield the uniform distribution within a&nbsp; $n$&ndash;dimensional cube.}}
-Note:
+$\text{Note:}$
-*The first requirement refers exclusively to the&nbsp; "amplitude distribution"&nbsp; $\rm (PDF)$&nbsp; and is generally easier to satisfy.
+*The first requirement refers exclusively to the&nbsp; &raquo;amplitude distribution&laquo;&nbsp; $\rm (PDF)$&nbsp; and is generally easier to satisfy.
-*The other requirements ensure&nbsp; "sufficient randomness"&nbsp; of the sequence.&nbsp; They concern the statistical independence of successive random values.
 ==Multiplicative Congruental Generator==
 <br>
-$\text{Multiplicative Congruental Generator}$&nbsp; is the best known method for generating a sequence&nbsp; $〈 x_\nu 〉$&nbsp; with equally distributed values&nbsp; $ x_\nu$&nbsp; between&nbsp; $0$&nbsp; and&nbsp; $1$.&nbsp; This method is given here in a bullet-point fashion:
+The&nbsp; &raquo;'''multiplicative congruental generator'''&laquo;&nbsp; is the best known method for generating a sequence&nbsp; $〈 x_\nu 〉$&nbsp; with equally distributed values&nbsp; $ x_\nu$&nbsp; between&nbsp; $0$&nbsp; and&nbsp; $1$.&nbsp; This method is given here in a bullet-point fashion:
 '''(1)''' &nbsp; These random generators are based on the successive manipulation of an integer variable&nbsp; $k$.&nbsp; If the number representation in the computer happens with&nbsp; $L$&nbsp; bit,&nbsp; this variable takes all values between&nbsp; $1$&nbsp; and&nbsp; $2^{L - 1}$&nbsp; exactly once each,&nbsp; if the sign bit is handled appropriately.
-'''(2)''' &nbsp; The random variable derived from this&nbsp; $x={k}/{\rm 2^{\it L - \rm 1}}$&nbsp; is also discrete&nbsp; $($with level number&nbsp; $M = 2^{L- 1})$:
+'''(2)''' &nbsp; The random variable&nbsp; $x={k}/{\rm 2^{\it L - \rm 1}}$&nbsp;  derived from this is also discrete-valued&nbsp; $($with level number&nbsp; $M = 2^{L- 1})$:
-:$$x={k}/{\rm 2^{\it L - \rm 1}} = k\cdot \delta x \in \{\delta x, \hspace{0.05cm}2\cdot \delta x,\hspace{0.05cm}\text{ ...}\hspace{0.05cm} , \hspace{0.05cm}1-\delta x,\hspace{0.05cm} 1\}.$$
+::$$x={k}/{\rm 2^{\it L - \rm 1}} = k\cdot \Delta x \in \{\Delta x, \hspace{0.05cm}2\cdot \Delta x,\hspace{0.05cm}\text{ ...}\hspace{0.05cm} , \hspace{0.05cm}1-\Delta x,\hspace{0.05cm} 1\}.$$
 :If the bit number&nbsp; $L$&nbsp; is sufficiently large,&nbsp; the distance&nbsp; $Δx = 1/2^{L- 1}$&nbsp; between two possible values is very small,&nbsp; and one may well interpret&nbsp; $x$&nbsp; as a continuous-valued random variable in the context of simulation accuracy.
-'''(3)''' &nbsp; The recursive generation rule of such&nbsp; "multiplicative congruential generators"&nbsp; is:
+'''(3)''' &nbsp; The recursive generation rule of such&nbsp;  &raquo;multiplicative congruential generators&laquo;&nbsp; is:
-:$$k_\nu=(a\cdot k_{\nu-1})\hspace{0.1cm} \rm mod \hspace{0.1cm} \it m.$$
+::$$k_\nu=(a\cdot k_{\nu-1})\hspace{0.1cm} \rm mod \hspace{0.1cm} \it m.$$
 '''(4)''' &nbsp; The statistical properties of the sequence depend crucially on the parameters&nbsp; $a$&nbsp; and&nbsp; $m$.&nbsp; The initial value&nbsp; $k_0$&nbsp;  has a minor importance for the statistics.
 '''(5)''' &nbsp; The best results are obtained with the base&nbsp; $m =2\hspace{0.05cm}^l-1$,&nbsp; where&nbsp; $l$&nbsp; denotes any natural number.&nbsp; Widely used in computers with 32-bit architecture and one sign bit is the base&nbsp; $m = 2^{31} - 1 = 2\hspace{0.08cm}147\hspace{0.08cm}483\hspace{0.08cm}647$.&nbsp; A corresponding algorithm is:
-:$$k_\nu=(16807\cdot k_{\nu-1})\hspace{0.1cm} \rm mod\hspace{0.1cm}(2^{31}-1).$$
+::$$k_\nu=(16807\cdot k_{\nu-1})\hspace{0.1cm} \rm mod\hspace{0.1cm}(2^{31}-1).$$
 '''(6)''' &nbsp; For such a generator, only the initial value&nbsp; $k_0 = 0$&nbsp; is not allowed.&nbsp; For&nbsp; $k_0 \ne 0$&nbsp; the period duration&nbsp; $P = 2^{31} - 2.$
 {{GraueBox|TEXT=
-$\text{Example 2:}$&nbsp; We analyze the&nbsp; "Multiplicative Congruental Generator"&nbsp; in more detail:
+[[File:EN_Sto_T_3_4_S4.png |right|frame| Multiplicative Congruental Generator&nbsp; $($C program$)$]]
-−
 *But it can be modified in such a way that at no time during the calculation the 32&ndash;bit integer number range is exceeded.
 *The C program&nbsp; $\text{uniform( )}$&nbsp;  thus modified is given on the right.}}

@@ Line 7: / Line 7: @@
 ==General description and definition==
 <br>
 {{BlaueBox|TEXT=
 $\text{Definition:}$&nbsp; A random variable&nbsp; $x$&nbsp; is said to be&nbsp; &raquo;'''uniformly distributed'''&laquo;&nbsp; if it can only take values in the range of&nbsp; $x_{\rm min}$&nbsp; to&nbsp; $x_{\rm max}$&nbsp; with equal probability.
-The graph shows
+The graph shows of such an equally distributed random variable&nbsp; $x$
-*on the left the probability density function&nbsp; $f_{x}(x)$,
+#on the left the probability density function&nbsp; $f_{x}(x)$,
-*right the cumulative distribution function&nbsp; $F_{x}(r)$
+#on the right the cumulative distribution function&nbsp; $F_{x}(r)$.
-of such an equally distributed random variable&nbsp; $x$.}}
+From the graph and this definition,&nbsp; the following properties can be derived:
-From the graph and this definition,&nbsp; the following properties can be derived:
+*The cumulative distribution function&nbsp; $\rm (CDF)$&nbsp; increases linearly from&nbsp; $x_{\rm min}$&nbsp; to&nbsp; $x_{\rm max}$&nbsp; in the range from&nbsp; $0$&nbsp; to&nbsp; $1$&nbsp;.
-*The probability density function&nbsp; $\rm (PDF)$&nbsp; has in the range from&nbsp; $x_{\rm min}$&nbsp; to&nbsp; $x_{\rm max}$&nbsp; the constant value&nbsp; $1/(x_{\rm max} - x_{\rm min})$.&nbsp;
-*On the range limits,&nbsp; only half the value - that is,&nbsp; the average value between the left&ndash;hand and right&ndash;hand limits - is to be set for&nbsp; $f_{x}(x)$&nbsp; in each case.
+*Mean, variance and standard deviation have the following values:
-*The distribution function&nbsp; $\rm (CDF)$&nbsp; increases linearly from&nbsp; $x_{\rm min}$&nbsp; to&nbsp; $x_{\rm max}$&nbsp; in the range from&nbsp; $0$&nbsp; to&nbsp; $1$&nbsp;.
+:$$m_{\rm 1} = \frac{x_ {\rm max} + x_{\rm min} }{2},\hspace{0.5cm}
 \sigma^2 = \frac{[x_{\rm max} - x_{\rm min}]^2}{12},\hspace{0.5cm}
-\sigma = \frac{\it x_{\rm max} - \it x_{\rm min}}{2 \sqrt{3}}.$$
+\sigma = \frac{x_{\rm max} - x_{\rm min} }{2 \sqrt{3} }.$$
 *For symmetric PDF &nbsp; &rArr; &nbsp; $x_{\rm min} = -x_{\rm max}$&nbsp; we obtain as a special case the mean&nbsp; $m_1 = 0$&nbsp; and the variance&nbsp; $σ^2 = x_{\rm max}^2/3.$
 {{GraueBox|TEXT=
-$\text{Example 1:}$&nbsp;
+$\text{Example 1:}$&nbsp; The graph shows two signal waveforms with uniform amplitude distribution.
-The graph shows two signal waveforms with uniform amplitude distribution.
+[[File:P_ID618__Sto_T_3_4_S2_neu100.png |right|frame|Examples of uniformly distributed signals]]
-*On the left,&nbsp; statistical independence of the individual samples is assumed,&nbsp; that is,&nbsp; the random variable&nbsp; $x_ν$&nbsp; can take all values between&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$&nbsp; with equal probability,&nbsp; and independently of the past&nbsp; $(x_{ν-1}, x_{ν-2}, \hspace{0.1cm}\text{...}).$
+$\rm (A)$&nbsp; On the left,&nbsp; statistical independence of the individual samples is assumed,&nbsp; that is,&nbsp; the random variable&nbsp; $x_ν$&nbsp; can take all values between&nbsp; $x_{\rm min}$&nbsp; and&nbsp; $x_{\rm max}$&nbsp; with equal probability,&nbsp; and independently of the past&nbsp; $(x_{ν-1}, x_{ν-2}, \hspace{0.1cm}\text{...}).$
-*For the right signal&nbsp; $y(t)$&nbsp; this independence of successive signal values is no longer given.&nbsp; Rather,&nbsp; this sawtooth signal represents a deterministic signal.}}
+$\rm (B)$&nbsp; For the right signal&nbsp; $y(t)$&nbsp; this independence of successive signal values is no longer given.&nbsp; Rather,&nbsp; this sawtooth signal represents a deterministic signal.}}
-==Importance of the uniform distribution for communications engineering==
+==Importance of the uniform distribution for Communications Engineering==
 <br>
-The importance of uniformly distributed random variables for information&nbsp; and communication technology is due to the fact that,&nbsp; from the point of view of information theory,&nbsp; this PDF form represents an optimum under the constraint&nbsp; [[Digital_Signal_Transmission/Optimization_of_Baseband_Transmission_Systems#Power_and_peak_limitation|$\text{peak limitation}$]]:
+The importance of uniformly distributed random variables for information&nbsp; and communication technology is due to the fact that,&nbsp; from the point of view of information theory,&nbsp; this PDF form represents an optimum under the constraint&nbsp; [[Digital_Signal_Transmission/Optimization_of_Baseband_Transmission_Systems#Power_and_peak_limitation|&raquo;peak limitation&laquo;]]:
-*With no distribution other than the uniform distribution one achieves greater&nbsp; [[Information_Theory/Differential_Entropy#Differential_entropy_of_some_peak-constrained_random_variables|$\text{differential entropy}$]]&nbsp; under this condition.
+*With no distribution other than the uniform distribution one achieves greater&nbsp; [[Information_Theory/Differential_Entropy#Differential_entropy_of_some_peak-constrained_random_variables|&raquo;differential entropy&laquo;]]&nbsp; under this condition.
-*This topic is dealt with in the chapter&nbsp; [[Information_Theory/Differential_Entropy|"Differential Entropy"]]&nbsp; in the book&nbsp; "Information Theory".
@@ Line 52: / Line 53: @@
 $\text{In addition,&nbsp; the following points should be mentioned,&nbsp; among others:}$
-'''(1)''' &nbsp; The importance of the uniform distribution for the simulation of communication systems is due to the fact that one can realize corresponding&nbsp; "pseudo-random generators"&nbsp; relatively easily,&nbsp; and that other distributions,&nbsp; such as the&nbsp; [[Theory_of_Stochastic_Signals/Gaussian_Distributed_Random_Variables#Probability_density_function_.E2.80.93_Cumulative_density_function|$\text{Gaussian distribution}$]]&nbsp; and the&nbsp; [[Theory_of_Stochastic_Signals/Exponentially_Distributed_Random_Variables#One-sided_exponential_distribution|$\text{exponential distribution}$]]&nbsp; can be easily derived.
+'''(1)''' &nbsp; The importance of the uniform distribution for the simulation of communication systems is due to the fact that one can realize corresponding&nbsp; &raquo;pseudo-random generators&laquo;&nbsp; relatively easily,&nbsp; and that other distributions,&nbsp; such as the&nbsp; [[Theory_of_Stochastic_Signals/Gaussian_Distributed_Random_Variables#Probability_density_function_.E2.80.93_Cumulative_density_function|&raquo;Gaussian distribution&laquo;]]&nbsp; and the&nbsp; [[Theory_of_Stochastic_Signals/Exponentially_Distributed_Random_Variables#One-sided_exponential_distribution|&raquo;exponential distribution&laquo;]]&nbsp; can be easily derived.
-'''(2)''' &nbsp; In&nbsp; "Image Processing & Coding",&nbsp; simplifying calculations are often made using the uniform distribution instead of the actual distribution of the original image,&nbsp; which is usually much more complicated,&nbsp; since the difference in information content between a&nbsp; "natural image"&nbsp; and the model based on the uniform distribution is relatively small.
+'''(2)''' &nbsp; In&nbsp; &raquo;Image Processing&laquo;,&nbsp; simplifying calculations are often made using the uniform distribution instead of the actual distribution of the original image,&nbsp; which is often much more complicated,&nbsp; since the difference in information content between a&nbsp; &raquo;natural image&laquo;&nbsp; and the model based on the uniform distribution is relatively small.
-'''(3)''' &nbsp; For modeling transmission systems,&nbsp; on the other hand,&nbsp; uniformly distributed random variables are the exception.&nbsp; An example of an actually&nbsp; (nearly)&nbsp; uniformly distributed random variable is the phase in the presence of circularly symmetric interference,&nbsp; such as occurs in&nbsp; "quadrature amplitude modulation techniques"&nbsp; (QAM).}}
+'''(3)''' &nbsp; For modeling transmission systems,&nbsp; on the other hand,&nbsp; uniformly distributed random variables are the exception.&nbsp; An example of an&nbsp; $($nearly$)$&nbsp; uniformly distributed random variable is the phase in the presence of circularly symmetric interference,&nbsp; such as occurs in&nbsp; &raquo;quadrature amplitude modulation techniques&laquo;&nbsp; $\rm (QAM)$.}}

@@ Line 23: / Line 23: @@
 *On the range limits,&nbsp; only half the value - that is,&nbsp; the average value between the left&ndash;hand and right&ndash;hand limits - is to be set for&nbsp; $f_{x}(x)$&nbsp; in each case.
 *The distribution function&nbsp; $\rm (CDF)$&nbsp; increases linearly from&nbsp; $x_{\rm min}$&nbsp; to&nbsp; $x_{\rm max}$&nbsp; in the range from&nbsp; $0$&nbsp; to&nbsp; $1$&nbsp;.
-*Mean, variance and standard deviation&nbsp; (standard deviation)&nbsp; '''KORREKTUR: 2x standard deviation?''' of the uniform distribution have the following values:
+*Mean, variance and standard deviation of the uniform distribution have the following values:
 :$$m_{\rm 1} = \frac{\it x_ {\rm max} \rm + \it x_{\rm min}}{2},\hspace{0.5cm}
 \sigma^2 = \frac{[x_{\rm max} - x_{\rm min}]^2}{12},\hspace{0.5cm}

← Older revision		Revision as of 19:19, 20 December 2022
Line 119:		Line 119:


−	*The algorithm,  however,  cannot be implemented directly on a 32 bit computer,  since the multiplication result requires up to 46 bits.	+	*The algorithm,  however,  cannot be implemented directly on a 32–bit computer,  since the multiplication result requires up to 46 bits.


−	*But it can be modified in such a way that at no time during the calculation the 32 ~~bit~~–integer number range is exceeded.	+	*But it can be modified in such a way that at no time during the calculation the 32–bit integer number range is exceeded.

@@ Line 45: / Line 45: @@
 <br>
 The importance of uniformly distributed random variables for information&nbsp; and communication technology is due to the fact that,&nbsp; from the point of view of information theory,&nbsp; this PDF form represents an optimum under the constraint&nbsp; [[Digital_Signal_Transmission/Optimization_of_Baseband_Transmission_Systems#Power_and_peak_limitation|$\text{peak limitation}$]]:
-*With no distribution other than the uniform distribution does one achieve greater&nbsp; [[Information_Theory/Differential_Entropy#Differential_entropy_of_some_peak-constrained_random_variables|$\text{differential entropy}$]]&nbsp; under this condition.
+*With no distribution other than the uniform distribution one achieves greater&nbsp; [[Information_Theory/Differential_Entropy#Differential_entropy_of_some_peak-constrained_random_variables|$\text{differential entropy}$]]&nbsp; under this condition.
 *This topic is dealt with in the chapter&nbsp; [[Information_Theory/Differential_Entropy|"Differential Entropy"]]&nbsp; in the book&nbsp; "Information Theory".