Channel Coding¶

Reliable communications over a noisy channel is achievable when the transmission rate remains below the channel capacity, a fundamental limit defining the maximum rate for error-free data transfer.

This reliability is facilitated by channel coding, a process that assigns messages to specific blocks of channel inputs, selectively using only a subset of all possible blocks to enhance error resilience.

Specific mappings between messages and channel input sequences have not been explored in detail here, as the focus is on theoretical bounds rather than practical implementations.

Channel capacity $C$ and channel cutoff rate $R_0$ are evaluated using random coding, a method that avoids specifying an optimal mapping.

Instead, random coding averages the error probability across all possible mappings, showing that when the transmission rate is less than $C$, the ensemble average error probability approaches zero as block length increases, due to the statistical likelihood of favorable mappings.

This implies the existence of at least one mapping where the error probability diminishes with longer blocks, providing a theoretical basis for practical code design.

Block Codes¶

Channel codes are classified into two primary types: block codes and convolutional codes, each with distinct encoding strategies.

In block codes, one of $M = 2^k$ messages, each a binary sequence of length $k$ called the information sequence, is mapped to a binary sequence of length $n$, known as the codeword, where $n \geq k$ to allow redundancy for error correction.

For example:

• A $(7,4)$ block code is a linear code where each codeword consists of $n = 7$ bits, encoding $k = 4$ information bits (the message), with $n - k = 3$ parity-check bits added for error detection and correction.

• The code maps $2^k = 2^4 = 16$ possible messages to 7-bit codewords. The total number of possible 7-bit sequences (blocks) is $2^n = 2^7 = 128$.

• The $(7,4)$ code uses only a subset of 16 out of these 128 possible blocks, chosen to maximize error resilience by ensuring a minimum Hamming distance between codewords.

Transmission occurs by sending $n$ binary symbols over the channel, often using modulation like BPSK (Binary Phase Shift Keying) to represent bits as signal phases.

Block coding schemes are memoryless, meaning the encoding of each codeword is independent of prior transmissions.

After encoding and sending a codeword, the system processes a new set of $k$ information bits, producing a codeword based solely on the current input, unaffected by previous codewords, which simplifies implementation but limits error correction across blocks.

Code Rate¶

The code rate $R_c$ of a block or convolutional code is defined as:

where $k$ is the number of information bits in the input message, and $n$ is the total number of bits in the output codeword (for block codes) or output bits per stage (for convolutional codes).

This ratio quantifies the efficiency of information transfer, representing the average number of information bits per transmitted bit in the coded sequence.

For error-correcting codes, where $n > k$, the code rate satisfies $R_c < 1$, reflecting the addition of redundancy to enhance error resilience.

When combined with a modulation scheme that maps $m$ bits to a transmitted symbol, the effective information rate becomes $m \times R_c$ bits per symbol.

For example, sing the (7,4) Hamming code:

• Code rate: $R_c = \frac{4}{7} \approx 0.571$, meaning each transmitted bit carries about 0.571 information bits.

• If modulated with BPSK ($m = 1$), information bits per symbol = $\frac{4}{7} \times 1 \approx 0.571$.

• If modulated with QPSK ($m = 2$), information bits per symbol = $\frac{4}{7} \times 2 \approx 1.14$.

• Bits per transmission (if interpreted as one codeword) = 4 information bits.

For a codeword of length $n$ transmitted via an $\color{red} N$-dimensional constellation of size $M$ (a power of 2), $L = \frac{n}{\log_2 M}$ symbols are sent per codeword, where $L$ is an integer.

Discussion¶

Recall that $N$ is the number of dimensions of the signal space in which the constellation points are defined.

The size $M$ of the constellation is the number of distinct points (symbols) in this space, and since $M$ is a power of 2, each symbol can represent $\log_2 M$ bits.

Specifically, each symbol in the constellation is a point in an $N$-dimensional Euclidean space. For example:

• $N = 1$: A one-dimensional constellation, such as Pulse Amplitude Modulation (PAM), where points lie on a line (e.g., 4-PAM with points at $\{-3, -1, 1, 3\}$, $M = 4$).

• $N = 2$: A two-dimensional constellation, such as QAM or PSK, where points are in a plane (e.g., QPSK with $M = 4$, points at $\{(1, 0), (0, 1), (-1, 0), (0, -1)\}$).

• $N > 2$: Higher-dimensional constellations, often used in advanced schemes like lattice codes or MIMO systems, where points are defined in $N$-dimensional space to increase efficiency or robustness.

• Size ($M$): The number of points in the constellation. Since $M = 2^m$ (a power of 2), each symbol carries:

  $$\log_2 M = m \text{ bits}$$

  (2)

• Codeword length ($n$): The number of bits in the coded sequence (e.g., output of a block code like the (7,4) Hamming code, where $n = 7$).

• Symbols per codeword ($L$): The number of constellation symbols needed to transmit the $n$-bit codeword. Since each symbol carries $\log_2 M$ bits, the number of symbols is:

  $$L = \frac{n}{\log_2 M}$$

  (3)

The requirement that $L$ is an integer ensures that the codeword’s bits can be exactly mapped to an integer number of symbols. For example, if $M = 4$, then $\log_2 4 = 2$, and $n$ must be a multiple of 2 for $L$ to be an integer.

Role of $N$

The parameter $N$ specifies the dimensionality of the signal space for each transmitted symbol.

However, $N$ does not directly affect the calculation of $L$, which depends only on $n$ and $M$.

Instead, $N$ defines the structure of the constellation and how bits are mapped to physical signals.

Example with a (7,4) Hamming Code.Consider a (7,4) Hamming code, where $n = 7$, and suppose we use a constellation to transmit the codeword:

• Let the constellation be $N = 2$-dimensional with $M = 8$ (e.g., 8-PSK, where points are equally spaced on a circle in the complex plane).

• Each symbol carries:

  $$\log_2 M = \log_2 8 = 3 \text{ bits}$$

  (4)

• Number of symbols per codeword:

  $$L = \frac{n}{\log_2 M} = \frac{7}{3} \approx 2.333$$

  (5)

Since $L$ is not an integer, this constellation is not suitable for transmitting a single (7,4) codeword without additional processing (e.g., combining multiple codewords or padding bits).Thus, the transmission scheme needs padding or bundling of codewords.Instead, let’s try $M = 4$ (e.g., QPSK, $N = 2$):

$$\log_2 4 = 2 \text{ bits}$$

(6)

$$L = \frac{7}{2} = 3.5$$

(7)

Again, $L$ is not an integer.To satisfy the integer constraint, we might transmit two codewords ($2 \times 7 = 14$ bits):

$$L = \frac{14}{2} = 7 \text{ symbols}$$

(8)

Here, $N = 2$ indicates that each of the 7 QPSK symbols is a 2D point, but $N$ does not enter the $L$ calculation directly.With symbol duration $T_s$, the transmission time for $k$ bits is $T = LT_s$, and the rate becomes Proakis (2007, Eq. (7.1-2)):

$$\begin{split} R &= \frac{k}{LT_s} = \frac{k}{n} \times \frac{\log_2 M}{T_s} \\ &= R_c \frac{\log_2 M}{T_s} \quad \text{bits/s} \end{split}$$

(9)

This equation ties $R_c$ to modulation complexity ($\log_2 M$) and symbol rate ($1/T_s$), showing how coding impacts overall data throughput.

Spectral Bit Rate¶

The encoded and modulated signals span a space of dimension $LN$.Per the dimensionality theorem, the minimum transmission bandwidth is:

$$W = \frac{N}{2T_s} = \frac{RN}{2R_c \log_2 M} \quad \text{bits/s}$$

(10)

The spectral bit rate, or bandwidth efficiency, is:

$$r = \frac{R}{W} = \frac{2 \log_2 M}{N} R_c$$

(11)

Here, $N$ is the signal space dimension per symbol, and $\log_2 M$ is the number of bits per symbol.Compared to an uncoded system with identical modulation, coding reduces the bit rate by $R_c$ (due to redundancy) and increases bandwidth by $\frac{1}{R_c}$, as more symbols are needed per information bit.This trade-off enhances reliability at the cost of spectral efficiency.

Energy and Transmitted Power¶

Given an average constellation energy $\mathcal{E}_{\text{avg}}$, the energy per codeword is:

$$\mathcal{E} = L \mathcal{E}_{\text{avg}} = \frac{n}{\log_2 M} \mathcal{E}_{\text{avg}}$$

(12)

This accounts for $L$ symbols per codeword, each carrying $\mathcal{E}_{\text{avg}}$.The energy per codeword component is:

$$\mathcal{E}_c = \frac{\mathcal{E}}{n} = \frac{\mathcal{E}_{\text{avg}}}{\log_2 M}$$

(13)

This distributes energy across $n$ components.The energy per information bit is:

$$\mathcal{E}_b = \frac{\mathcal{E}}{k} = \frac{\mathcal{E}_{\text{avg}}}{R_c \log_2 M}$$

(14)

Since $k < n$, $\mathcal{E}_b > \mathcal{E}_c$, reflecting redundancy’s energy cost per bit.Combining these:

$$\mathcal{E}_c = R_c \mathcal{E}_b$$

(15)

This shows $\mathcal{E}_c$ scales with $R_c$, linking code efficiency to energy allocation.

Transmitted Power¶

Transmitted power is:

$$P = \frac{\mathcal{E}}{LT_s} = \frac{\mathcal{E}_{\text{avg}}}{T_s} = R \frac{\mathcal{E}_{\text{avg}}}{R_c \log_2 M} = R \mathcal{E}_b$$

(16)

Note that since joules per second (J/s) equals watts, the units are consistent:

$$[R \mathcal{E}_b] = \frac{\text{bits}}{\text{s}} \times \frac{\text{J}}{\text{bit}} = \frac{\text{J}}{\text{s}} = \text{W}$$

(17)

Common schemes¶

• BPSK: $W = \frac{R}{R_c}, r = R_c$ (1 bit/symbol, $N=1$)

• BFSK: $W = \frac{R}{R_c}, r = R_c$ (1 bit/symbol, $N=1$, frequency-based)

• QPSK: $W = \frac{R}{2R_c}, r = 2R_c$ (2 bits/symbol, $N=2$)

These reflect how modulation affects bandwidth (halved in QPSK due to higher bits/symbol) and efficiency, with coding consistently scaling both by $R_c$.¶