Skip to content

komm.DiscreteMemorylessChannel

General discrete memoryless channel (DMC). It is defined by an input alphabet $\mathcal{X}$, an output alphabet $\mathcal{Y}$, and a transition probability matrix $p_{Y \mid X}$. Here, for simplicity, the input and output alphabets are always taken as $\mathcal{X} = \{ 0, 1, \ldots, |\mathcal{X}| - 1 \}$ and $\mathcal{Y} = \{ 0, 1, \ldots, |\mathcal{Y}| - 1 \}$, respectively. The transition probability matrix $p_{Y \mid X}$, of size $|\mathcal{X}|$-by-$|\mathcal{Y}|$, gives the conditional probability of receiving $Y = y$ given that $X = x$ is transmitted. For more details, see CT06, Ch. 7.

Attributes:

  • transition_matrix (NDArray[floating]) –

    The channel transition probability matrix $p_{Y \mid X}$. The element in row $x \in \mathcal{X}$ and column $y \in \mathcal{Y}$ must be equal to $p_{Y \mid X}(y \mid x)$.

input_cardinality: int property

The channel input cardinality $|\mathcal{X}|$.

output_cardinality: int property

The channel output cardinality $|\mathcal{Y}|$.

mutual_information

Returns the mutual information $\mathrm{I}(X ; Y)$ between the input $X$ and the output $Y$ of the channel. It is given by $$ \mathrm{I}(X ; Y) = \mathrm{H}(X) - \mathrm{H}(X \mid Y), $$ where $\mathrm{H}(X)$ is the the entropy of $X$ and $\mathrm{H}(X \mid Y)$ is the conditional entropy of $X$ given $Y$. By default, the base of the logarithm is $2$, in which case the mutual information is measured in bits. See CT06, Ch. 2.

Parameters:

  • input_pmf (ArrayLike) –

    The probability mass function $p_X$ of the channel input $X$. It must be a valid pmf, that is, all of its values must be non-negative and sum up to $1$.

  • base (LogBase) –

    The base of the logarithm to be used. It must be a positive float or the string 'e'. The default value is 2.0.

Returns:

  • float –

    The mutual information $\mathrm{I}(X ; Y)$ between the input $X$ and the output $Y$.

Examples:

>>> dmc = komm.DiscreteMemorylessChannel([[0.6, 0.3, 0.1], [0.7, 0.1, 0.2], [0.5, 0.05, 0.45]])
>>> dmc.mutual_information([1/3, 1/3, 1/3]).round(6)
np.float64(0.123811)
>>> dmc.mutual_information([1/3, 1/3, 1/3], base=3).round(6)
np.float64(0.078116)
>>> dmc.mutual_information([1/3, 1/3, 1/3], base='e').round(6)
np.float64(0.085819)

capacity

Returns the channel capacity $C$. It is given by $C = \max_{p_X} \mathrm{I}(X;Y)$. This method computes the channel capacity via the Arimoto–Blahut algorithm. See CT06, Sec. 10.8.

Parameters:

  • base (LogBase) –

    The base of the logarithm to be used. It must be a positive float or the string 'e'. The default value is 2.0.

Returns:

  • float –

    The channel capacity $C$.

Examples:

>>> dmc = komm.DiscreteMemorylessChannel([[0.6, 0.3, 0.1], [0.7, 0.1, 0.2], [0.5, 0.05, 0.45]])
>>> dmc.capacity().round(6)
np.float64(0.161632)
>>> dmc.capacity(base=3).round(6)
np.float64(0.101978)
>>> dmc.capacity(base='e').round(6)
np.float64(0.112035)

__call__

Transmits the input sequence through the channel and returns the output sequence.

Parameters:

  • input (ArrayLike) –

    The input sequence.

Returns:

  • output (NDArray[integer]) –

    The output sequence.

Examples:

>>> rng = np.random.default_rng(seed=42)
>>> dmc = komm.DiscreteMemorylessChannel([[0.9, 0.05, 0.05], [0.0, 0.5, 0.5]], rng=rng)
>>> dmc([1, 1, 1, 0, 0, 0, 1, 0, 1, 0])
array([2, 1, 2, 0, 0, 2, 2, 0, 1, 0])