Methods

To describe how music is modeled by Markov chains, let's first define the terminologies and notations -
A first order, discrete time Markov chain $\mathcal{C}$ is a random walk $X_t,t=1,2,3,...$, in a state space $\mathcal{S} =\{ s_1,s_2,.., s_N \}$ according to a $N \times N$ state-transition matrix $P_{i,j} = p\left(X_{t+1}=s_{j}\vert X_{t}=s_{i}\right)$, where $P_{i,j}$ denotes the element on the $i^{\mathrm{th}}$ row and $j^{\mathrm{th}}$ column, and $p(\cdot\vert\cdot)$ is the usual notation of the conditional probability distribution function.

Mathematically, it suffices to say that a Markov chain $\mathcal{C}$ is characterized by its state-transition matrix $P$, up to one-to-one mappings between homeomorphic state spaces. One can even sloppily write $\mathrm{C}=P$. However, how the transition matrix actually means depends upon how the state-space is defined. For example, if the (pentatonic) state space is defined as $\mathcal{S}= \{ \mathrm{C4,D4,E4,A4,G4}\}$, then $P_{1,5} = 0.3$ reads ``the next note is G4 thirty percents of the times when the current note is C4.''

Since higher-order Markov chains are beyond the scope of this research, in the rest of this paper, ``Markov chains'' means first-order ones unless otherwise mentioned.