Basics of Coded Transmission
Contents
- 1 # OVERVIEW OF THE SECOND MAIN CHAPTER #
- 2 Information content – Entropy – Redundancy
- 3 Source coding – Channel coding – Line coding
- 4 System model and description variables
- 5 ACF calculation of a digital signal
- 6 PSD calculation of a digital signal
- 7 ACF and PSD for bipolar binary signals
- 8 ACF and PSD for unipolar binary signals
- 9 Exercises for the chapter
# OVERVIEW OF THE SECOND MAIN CHAPTER #
The second main chapter deals with so-called transmission coding, which is sometimes also referred to as "line coding" in literature. In this process, an adaptation of the digital transmitted signal to the characteristics of the transmission channel is achieved through the targeted addition of redundancy. In detail, the following are dealt with:
- Some basic concepts of information theory such as »information content« and »entropy«,
- the »auto-correlation function« and the »power-spectral densities« of digital signals,
- the »redundancy-free coding« which leads to a non-binary transmitted signal,
- the calculation of »symbol and bit error probability« for »multilevel systems« ,
- the so-called »4B3T codes« as an important example of »block-wise coding«, and
- the »pseudo-ternary codes«, each of which realizes symbol-wise coding.
The description is in baseband throughout and some simplifying assumptions (among others: no intersymbol interfering) are still made.
Information content – Entropy – Redundancy
We assume an M–level digital source that outputs the following source signal:
- q(t)=∑(ν)aν⋅δ(t−ν⋅T)withaν∈{a1,... ,aμ,... ,aM}.
- The source symbol sequence ⟨qν⟩ is thus mapped to the sequence ⟨aν⟩ of the dimensionless amplitude coefficients.
- Simplifying, first for the time indexing variable ν=1, ... , N is set, while the ensemble indexing variable μ can assume values between 1 and level number M.
If the ν–th sequence element is equal to aμ, its information content can be calculated with probability pνμ=Pr(aν=aμ) as follows:
- Iν=log2 (1/pνμ)=ld (1/pνμ)(unit: bit).
The logarithm to the base 2 ⇒ log2(x) is often also called ld(x) ⇒ "logarithm dualis". With the numerical evaluation the reference unit "bit" (from: "binary digit" ) is added. With the tens logarithm lg(x) and the natural logarithm ln(x) applies:
- log2(x)=lg(x)lg(2)=ln(x)ln(2).
According to this definition, which goes back to "Claude E. Shannon", the smaller the probability of occurrence of a symbol, the greater its information content.
Definition: Entropy is the "average information content" of a sequence element ("symbol").
- This important information-theoretical quantity can be determined as a time average as follows:
- H=limN→∞1N⋅N∑ν=1Iν=limN→∞1N⋅N∑ν=1log2 (1/pνμ)(unit: bit).
- Of course, the entropy can also be calculated by ensemble averaging (over the symbol set).
Note:
- If the sequence elements aν are statistically independent of each other, the probabilities pνμ=pμ are independent of ν and we obtain in this special case:
- H=M∑μ=1pμ⋅log2 (1/pμ).
- If, on the other hand, there are statistical bindings between neighboring amplitude coefficients aν, the more complicated equation according to the above definition must be used for entropy calculation.
Definitions:
- The maximum value of entropy ⇒ decision content is obtained whenever the M occurrence probabilities (of the statistically independent symbols) are all equal (pμ=1/M):
- Hmax=M∑μ=11M⋅log2(M)=log2(M)⋅M∑μ=11M=log2(M)(unit: bit).
- The relative redundancy is then the following quotient:
- r=Hmax−HHmax.
- Since 0≤H≤Hmax always holds, the relative redundancy can take values between 0 and 1 (including these limits).
From the derivation of these descriptive quantities, it is obvious that a redundancy-free (r=0) digital signal must satisfy the following properties:
- The amplitude coefficients aν are statistically independent ⇒ pνμ=Pr(aν=aμ) is identical for all ν.
- The M possible coefficients aμ occur with equal probability pμ=1/M.
Example 1: If one analyzes a German text on the basis of M=32 characters:
- a, ... , z, ä, ö, ü, ß, spaces, punctuation, no distinction between upper and lower case,
the result is the decision content Hmax=5 bit/symbol. Due to
- the different frequencies (for example, "e" occurs significantly more often than "u"), and
- statistical ties (for example "q" is followed by the letter "u" much more often than "e"),
according to "Karl Küpfmüller", the entropy of the German language is only H=1.3 bit/character. This results in a relative redundancy of r≈(5−1.3)/5=74%.
For English texts, "Claude Shannon" has given the entropy as H=1 bit/character and the relative redundancy as r≈80%.
Source coding – Channel coding – Line coding
"Coding" is the conversion of the source symbol sequence ⟨qν⟩ with symbol set size Mq into an encoder symbol sequence ⟨cν⟩ with symbol set size Mc. Usually, coding manipulates the redundancy contained in a digital signal. Often – but not always – Mq and Mc are different.
A distinction is made between different types of coding depending on the target direction:
- The task of source coding is redundancy reduction for data compression, as applied for example in image coding. By exploiting statistical ties between the individual points of an image or between the brightness values of a point at different times (in the case of moving image sequences), methods can be developed that lead to a noticeable reduction in the amount of data (measured in "bit" or "byte"), while maintaining virtually the same (subjective) image quality. A simple example of this is "differential pulse code modulation" (DPCM).
- On the other hand, with channel coding a noticeable improvement in the transmission behavior is achieved by using a redundancy specifically added at the transmitter to detect and correct transmission errors at the receiver end. Such codes, the most important of which are block codes, convolutional codes and turbo codes, are particularly important in the case of heavily disturbed channels. The greater the relative redundancy of the encoded signal, the better the correction properties of the code, albeit at a reduced user data rate.
- Line coding is used to adapt the transmitted signal to the spectral characteristics of the transmission channel and the receiving equipment by recoding the source symbols. For example, in the case of a channel with the frequency response characteristic HK(f=0)=0, over which consequently no DC signal can be transmitted, transmission coding must ensure that the encoder symbol sequence contains neither a long L sequence nor a long H sequence.
In the current book "Digital Signal Transmission" we deal exclusively with this last, transmission-technical aspect.
- "Channel Coding" has its own book dedicated to it in our learning tutorial.
- Source coding is covered in detail in the book "Information Theory" (main chapter 2).
- "Speech coding" – described in the book "Examples of Communication Systems" – is a special form of source coding.
System model and description variables
In the following we always assume the block diagram sketched on the right and the following agreements:
- Let the digital source signal q(t) be binary (Mq=2) and redundancy-free (Hq=1 bit/symbol).
- With the symbol duration Tq results for the symbol rate of the source:
- Rq=Hq/Tq=1/Tq.
- Because of Mq=2, in the following we also refer to Tq as the "bit duration" and Rq as the "bit rate".
- For the comparison of transmission systems with different coding, Tq and Rq are always assumed to be constant. Note: In later chapters we use TB=Tq and RB=Rq for this purpose.
- The encoded signal c(t) and also the transmitted signal s(t) after pulse shaping with gs(t) have the level number Mc, the symbol duration Tc and the symbol rate 1/Tc. The equivalent bit rate is
- Rc=log2(Mc)/Tc≥Rq.
- The equal sign is only valid for the "redundancy-free codes" (rc=0).
Otherwise, we obtain for the relative code redundancy:
- rc=(Rc−Rq)/Rc=1−Rq/Rc.
Notes on nomenclature:
- In the context of transmission codes, Rc always indicates in our tutorial the equivalent bit rate of the encoded signal with unit "bit/s".
- In the literature on channel coding, Rc is often used to denote the dimensionless code rate 1−rc .
- Rc=1 then indicates a redundancy-free code, while Rc=1/3 indicates a code with the relative redundancy rc=2/3.
Example 2: In the so-called "4B3T codes",
- four binary symbols (mq=4, Mq=2) are each represented by
- three ternary symbols (mc=3, Mc=3).
Because of 4⋅Tq=3⋅Tc holds:
- Rq=1/Tq,Rc=log2(3)/Tc=3/4⋅log2(3)/Tq
- ⇒rc=3/4⋅log2(3)−1≈15.9%.
Detailed information about the 4B3T codes can be found in the "chapter of the same name".
ACF calculation of a digital signal
To simplify the notation, Mc=M and Tc=T is set in the following. Thus, for the transmitted signal s(t) in the case of an unlimited-time sybol sequence with aν∈{a1, ... , aM} can be written:
- s(t)=+∞∑ν=−∞aν⋅gs(t−ν⋅T).
This signal representation includes both the source statistics (amplitude coefficients aν) and the transmission pulse shape gs(t). The diagram shows two binary bipolar transmitted signals sG(t) and sR(t) with the same amplitude coefficients aν, which thus differ only by the basic transmission pulse gs(t).
It can be seen from this figure that a digital signal is generally non-stationary:
- For the transmitted signal sG(t) with narrow Gaussian pulses, the "non-stationarity" is obvious, since, for example, at multiples of T the variance is σ2s=s20, while exactly in between σ2s≈0 holds.
- Also the signal sR(t) with NRZ rectangular pulses is non–stationary in a strict sense, because here the moments at the bit boundaries differ with respect to all other instants. For example, sR(t=±T/2)=0.
Definition:
- A random process whose moments mk(t)=mk(t+ν⋅T) repeat periodically with T is called cyclostationary.
- In this implicit definition, k and ν have integer values .
Many of the rules valid for "ergodic processes" can also be applied to "cycloergodic" (and hence to "cyclostationary") processes with only minor restrictions.
- In particular, for the "auto-correlation function" (ACF) of such random processes with sample signal s(t) holds:
- φs(τ)=E[s(t)⋅s(t+τ)].
- With the above equation of the transmitted signal, the ACF as a time average can also be written as follows:
- φs(τ)=+∞∑λ=−∞1T⋅limN→∞12N+1⋅+N∑ν=−Naν⋅aν+λ⋅∫+∞−∞gs(t)⋅gs(t+τ−λ⋅T)dt.
- Since the limit, integral and sum may be interchanged, with the substitutions
- N=TM/(2T),λ=κ−ν,t−ν⋅T→T
- for this can also be written:
- φs(τ)=limTM→∞1TM⋅∫+TM/2−TM/2+∞∑ν=−∞+∞∑κ=−∞aν⋅gs(t−ν⋅T)⋅aκ⋅gs(t+τ−κ⋅T)dt.
Now the following quantities are introduced for abbreviation:
Definitions:
- The discrete ACF of the amplitude coefficients provides statements about the linear statistical bonds of the amplitude coefficients aν and aν+λ and has no unit:
- φa(λ)=limN→∞12N+1⋅+∞∑ν=−∞aν⋅aν+λ.
- The energy ACF of the basic transmission pulse is defined similarly to the general (power) auto-correlation function. It is marked with a dot:
- φ∙gs(τ)=∫+∞−∞gs(t)⋅gs(t+τ)dt.
- ⇒ Since gs(t) is "energy-limited", the division by TM and the boundary transition can be omitted.
- For the auto-correlation function of a digital signal s(t) holds in general:
- φs(τ)=+∞∑λ=−∞1/T⋅φa(λ)⋅φ∙gs(τ−λ⋅T).
- ⇒ s(t) can be binary or multilevel, unipolar or bipolar, redundancy-free or redundant (line-coded). The pulse shape is taken into account by the energy ACF.
- Note:
- If the digital signal s(t) describes a voltage waveform,
- the energy ACF of the basic transmission pulse gs(t) has the unit V2s,
- the auto-correlation function φs(τ) of the digital signal s(t) has the unit V2 (each related to the resistor 1 Ω).
- In the strict sense of system theory, one would have to define the ACF of the amplitude coefficients as follows:
- φa,δ(τ)=+∞∑λ=−∞φa(λ)⋅δ(τ−λ⋅T).
- ⇒ Thus, the above equation would be as follows:
- φs(τ)=1/T⋅φa,δ(τ)⋆φ∙gs(τ−λ⋅T)=+∞∑λ=−∞1/T⋅φa(λ)⋅φ∙gs(τ−λ⋅T).
- ⇒ For simplicity, the discrete ACF of amplitude coefficients ⇒ φa(λ) is written without these Dirac delta functions in the following.
PSD calculation of a digital signal
The corresponding quantity to the auto-correlation function (ACF) of a random signal ⇒ φs(τ) in the frequency domain is the "power-spectral density" (PSD) ⇒ Φs(f), which is in a fixed relation with the ACF via the Fourier integral:
- φs(τ)∘−−−∙Φs(f)=∫+∞−∞φs(τ)⋅e−j2πfτdτ.
- Considering the relation between energy ACF and energy spectrum,
- φ∙gs(τ)∘−−−∙Φ∙gs(f)=|Gs(f)|2,
- and the "shifting theorem", the power-spectral density of the digital signal s(t) can be represented in the following way:
- Φs(f)=+∞∑λ=−∞1/T⋅φa(λ)⋅Φ∙gs(f)⋅e−j2πfλT=1/T⋅|Gs(f)|2⋅+∞∑λ=−∞φa(λ)⋅cos(2πfλT).
- Here it is considered that Φs(f) and |Gs(f)|2 are real-valued and at the same time φa(−λ)=φa(+λ) holds.
- If we now define the spectral power density of the amplitude coefficients to be
- Φa(f)=+∞∑λ=−∞φa(λ)⋅e−j2πfλT=φa(0)+2⋅∞∑λ=1φa(λ)⋅cos(2πfλT),
- then the following expression is obtained:
- Φs(f)=Φa(f)⋅1/T⋅|Gs(f)|2.
Conclusion: The power-spectral density Φs(f) of a digital signal s(t) can be represented as the product of two functions:
- The first term Φa(f) is dimensionless and describes the spectral shaping of the transmitted signal by the statistical constraints of the source.
- In contrast, |Gs(f)|2 takes into account the spectral shaping by the basic transmission pulse gs(t).
- The narrower gs(t) is, the broader is the energy spectrum |Gs(f)|2 and thus the larger is the bandwidth requirement.
- The energy spectrum |Gs(f)|2 has the unit V2s/Hz and the power-spectral density Φs(f) – due to the division by symbol duration T – the unit V2/Hz.
- Both specifications are again only valid for the resistor 1 Ω.
ACF and PSD for bipolar binary signals
The previous results are now illustrated by examples. Starting from binary bipolar amplitude coefficients aν∈{−1,+1}, if there are no bonds between the individual amplitude coefficients aν, we obtain:
- φa(λ)={10forforλ=0,λ≠0⇒φs(τ)=1/T⋅φ∙gs(τ).
The graph shows two signal sections each with rectangular pulses gs(t), which accordingly lead to a triangular auto-correlation function (ACF) and to a sinc2–shaped power-spectral density (PSD).
- The left pictures describe NRZ signaling ⇒ the width TS of the basic pulse is equal to the distance T of two transmitted pulses (source symbols).
- In contrast, the right pictures apply to an RZ pulse with the duty cycle TS/T=0.5.
One can see from the left representation (NRZ):
- For NRZ rectangular pulses, the transmit power (reference: 1 Ω resistor) is PS=φs(τ=0)=s20.
- The triangular ACF is limited to the range |τ|≤TS=T.
- The PSD Φs(f) as the Fourier transform of φs(τ) is sinc2–shaped with equidistant zeros at distance 1/T.
- The area under the PSD curve again gives the transmit power PS=s20.
In the case of RZ signaling (right column), the triangular ACF is smaller in height and width by a factor of TS/T=0.5, resp., compared to the left image.
Conclusion: If one compares the two power-spectral densities (lower pictures), one recognizes for TS/T=0.5 (RZ pulse) compared to TS/T=1 (NRZ pulse)
- a reduction in height by a factor of 4,
- a broadening by a factor of 2.
- ⇒ The area (power) in the right sketch is thus half as large, since in half the time s(t)=0.
ACF and PSD for unipolar binary signals
We continue to assume NRZ or RZ rectangular pulses. But let the binary amplitude coefficients now be unipolar: aν∈{0,1}. Then for the discrete ACF of the amplitude coefficients holds:
- φa(λ)={m2=0.5m21=0.25forforλ=0,λ≠0.
Assumed here are equal probability amplitude coefficients ⇒ Pr(aν=0)=Pr(aν=1)=0.5 with no statistical ties, so that both the "power" m2 and the "linear mean" m1 (DC component) are 0.5, respectively.
The graph shows a signal section, the ACF and the PSD with unipolar amplitude coefficients,
- left for rectangular NRZ pulses (TS/T=1), and
- right for RZ pulses with duty cycle TS/T=0.5.
There are the following differences compared to "bipolar signaling":
- Adding the infinite number of triangular functions at distance T (all with the same height) results in a constant DC component s20/4 for the ACF in the left graph (NRZ).
- In addition, a single triangle also with height s20/4 remains in the region |τ|≤TS, which leads to the sinc2–shaped blue curve in the power-spectral density (PSD).
- The DC component in the ACF results in a Dirac delta function at frequency f=0 with weight s20/4 in the PSD. Thus the PSD value Φs(f=0) becomes infinitely large.
From the right graph – valid for TS/T=0.5 – it can be seen that now the ACF is composed of a periodic triangular function (drawn dashed in the middle region) and additionally of a unique triangle in the region |τ|≤TS=T/2 with height s20/8.
- This unique triangle function leads to the continuous sinc2–shaped component (blue curve) of Φs(f) with the first zero at 1/TS=2/T.
- In contrast, the periodic triangular function leads to an infinite sum of Dirac delta functions with different weights at the distance 1/T (drawn in red) according to the laws of the "Fourier series".
- The weights of the Dirac delta functions are proportional to the continuous (blue) PSD component. The Dirac delta line at f=0 has the maximum weight s20/8. In contrast, the Dirac delta lines at ±2/T and multiples thereof do not exist or have the weight 0 in each case, since the continuous PSD component also has zeros here.
Note:
- Unipolar amplitude coefficients occur for example in optical transmission systems.
- In later chapters, however, we mostly restrict ourselves to bipolar signaling.
Exercises for the chapter
Exercise 2.1: ACF and PSD with Coding
Exercise 2.1Z: About the Equivalent Bitrate
Exercise 2.2: Binary Bipolar Rectangles