Directed information

This article provides insufficient context for those unfamiliar with the subject. Please help improve the article by providing more context for the reader. (September 2016) (Learn how and when to remove this message)

Directed information is an information theory measure that quantifies the information flow from the random string ${\displaystyle X^{n}=(X_{1},X_{2},\dots ,X_{n})}$ to the random string ${\displaystyle Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})}$. The term directed information was coined by James Massey and is defined as^[1]

${\displaystyle I(X^{n}\to Y^{n})\triangleq \sum _{i=1}^{n}I(X^{i};Y_{i}|Y^{i-1})}$

where ${\displaystyle I(X^{i};Y_{i}|Y^{i-1})}$ is the conditional mutual information ${\displaystyle I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})}$.

Directed information has applications to problems where causality plays an important role such as the capacity of channels with feedback,^[1][2][3][4] capacity of discrete memoryless networks,^[5] capacity of networks with in-block memory,^[6] gambling with causal side information,^[7] compression with causal side information,^[8] real-time control communication settings,^[9][10] and statistical physics.^[11]