Skip to content

komm.DiscreteMemorylessChannel

Discrete memoryless channel (DMC). It is defined by an input alphabet $\mathcal{X}$, an output alphabet $\mathcal{Y}$, and a transition probability matrix $p_{Y \mid X}$. Here, for simplicity, the input and output alphabets are always taken as $\mathcal{X} = \{ 0, 1, \ldots, |\mathcal{X}| - 1 \}$ and $\mathcal{Y} = \{ 0, 1, \ldots, |\mathcal{Y}| - 1 \}$, respectively. The transition probability matrix $p_{Y \mid X}$, of size $|\mathcal{X}|$-by-$|\mathcal{Y}|$, gives the conditional probability of receiving $Y = y$ given that $X = x$ is transmitted. For more details, see CT06, Ch. 7.

Attributes:

  • transition_matrix (Array2D[float])

    The channel transition probability matrix $p_{Y \mid X}$. The element in row $x \in \mathcal{X}$ and column $y \in \mathcal{Y}$ must be equal to $p_{Y \mid X}(y \mid x)$.

Input:

  • in0 (Array1D[int])

    The input sequence.

Output:

  • out0 (Array1D[int])

    The output sequence.

Examples:

>>> np.random.seed(1)
>>> dmc = komm.DiscreteMemorylessChannel([[0.9, 0.05, 0.05], [0.0, 0.5, 0.5]])
>>> x = [0, 1, 0, 1, 1, 1, 0, 0, 0, 1]
>>> y = dmc(x)
>>> y
array([0, 2, 0, 1, 1, 1, 0, 0, 0, 2])

input_cardinality property

The channel input cardinality $|\mathcal{X}|$.

output_cardinality property

The channel output cardinality $|\mathcal{Y}|$.

mutual_information

Computes the mutual information $\mathrm{I}(X ; Y)$ between the input $X$ and the output $Y$ of the channel. It is given by $$ \mathrm{I}(X ; Y) = \mathrm{H}(X) - \mathrm{H}(X \mid Y), $$ where $\mathrm{H}(X)$ is the the entropy of $X$ and $\mathrm{H}(X \mid Y)$ is the conditional entropy of $X$ given $Y$. By default, the base of the logarithm is $2$, in which case the mutual information is measured in bits. See CT06, Ch. 2.

Parameters:

  • input_pmf (Array1D[float])

    The probability mass function $p_X$ of the channel input $X$. It must be a valid pmf, that is, all of its values must be non-negative and sum up to $1$.

  • base (Optional[float | str])

    The base of the logarithm to be used. It must be a positive float or the string 'e'. The default value is 2.0.

Returns:

  • mutual_information (float)

    The mutual information $\mathrm{I}(X ; Y)$ between the input $X$ and the output $Y$.

Examples:

>>> dmc = komm.DiscreteMemorylessChannel([[0.6, 0.3, 0.1], [0.7, 0.1, 0.2], [0.5, 0.05, 0.45]])
>>> dmc.mutual_information([1/3, 1/3, 1/3])
np.float64(0.123811098798)
>>> dmc.mutual_information([1/3, 1/3, 1/3], base=3)
np.float64(0.078116106054)

capacity

Returns the channel capacity $C$. It is given by $C = \max_{p_X} \mathrm{I}(X;Y)$. This method computes the channel capacity via the Arimoto–Blahut algorithm. See CT06, Sec. 10.8.

Parameters:

  • base (Optional[float | str])

    The base of the logarithm to be used. It must be a positive float or the string 'e'. The default value is 2.0.

Returns:

  • capacity (float)

    The channel capacity $C$.

Examples:

>>> dmc = komm.DiscreteMemorylessChannel([[0.6, 0.3, 0.1], [0.7, 0.1, 0.2], [0.5, 0.05, 0.45]])
>>> dmc.capacity()
np.float64(0.1616318610)
>>> dmc.capacity(base=3)
np.float64(0.1019783502)