Introduction to Item Streams

Heinrich Taube
School of Music, University of Illinois
taube@uiuc.edu

Foreword

This document provides an introduction to item streams, the compositional pattern generation facility in Common Music. Detailed documentation about the various patterns and stream types can be found under Item Streams in the Common Music Dictionary. Some examples of item streams can be found in items.cm. Headings in this document are marked in bold face. Normal text is printed in Time font. Dictionary words and examples are printed in Courier font. Lisp input expressions are shown in bold face. Return values are printed in Italic face.

Introduction

An item stream enumerates data according to a specified pattern. The behavior of an item stream is controlled by two characteristics, a stream type and a pattern type. The stream types are:

amplitudes returns integer or floating point amplitude values.
degrees returns integer scale positions.
intervals returns integer intervals.
items returns uninterpreted Lisp data.
notes returns symbolic note names.
numbers returns integer or floating point values.
pitches returns floating point frequencies.
rhythms returns floating point time values.
series returns integer intervals.
steps return relative integer intervals.

The pattern types are:

accumulation enumerates elements by group.
cycle enumerates elements in a loop, last returning to first.
function enumerates elements by calling user function.
graph enumerates elements by user defined transition rules.
heap enumerates elements by random permutation.
palindrome enumerates elements forwards and backwards.
random enumerates elements in random order.
rewrite enumerates elements by user specified rewrite rules.
rotation enumerates elements by user specified permutation rules.
sequence enumerates elements sequentially, sticks on last element.

Creating Item Streams

As with other composite data structures in Lisp, item streams are created with a constructor and their elements are referenced with an accessor. Each type of stream has its own constructor. These constructors are all named after the type of stream they create: amplitudes, degrees, intervals, items, notes, numbers, pitches, rhythms, series and steps:

? (items 1 2 three 4 in cycle)
#<CYCLIC-ITEM-STREAM 131352561>

? (pitches c4 d e f g in palindrome eliding end)
#<PALINDROMIC-PITCH-STREAM 131353584>

? (intervals 0 1 2 in random from 'c5 for 20)
#<RANDOM-PITCH-STREAM 132342567>

? (rhythms q e h.. 32 in accumulation tempo 90)
#<ACCUMLATING-RHYTHM-STREAM 131332661>

The constructors all share a simple pseudo-english syntax in which the name of the constructor is followed by one or more data elements followed by zero or more option value pairs:

amplitudes {amp}+ {option value}*
degrees {degree}+ {option value}*
intervals {interval}+ {option value}*
items {object}+ {option value}*
notes {note}+ {option value}*
numbers {option value}*
pitches {pitch}+ {option value}*
series {interval}+ {option value}*
steps {interval}+ {option value}*

The type of data specified to a constructor depends on the type of stream it implements. The option value pairs provide a flexible mechanism for setting non-required attributes of a stream. Options may appear in any order and need only be supplied if the specified value is different than the default value for the option. The set of allowable options depends on both the stream type and the pattern type being created. For example, the in and for options shown in the last examples may be specified in any constructor. The tempo option may be used only in conjunction with rhythms, and the eliding option is applicable only to palindrome patterns. Most of the option value pairs are beyond the scope of this document, but a few are common and apply to all streams:

in {pattern}

The in option sets the pattern type for the stream. pattern must be one of the defined pattern types: accumulation,cycle, function, graph, heap, palindrome, random, rewrite, rotation, sequence. If unspecified, in defaults to cycle:

? (items 1 2 3)
#<CYCLIC-ITEM-STREAM 131540621>

? (items 1 2 3 in rotation)
#<ROTATIONAL-ITEM-STREAM 131554061>

for {integer}

The for option sets the period length for the pattern. A period is a pattern "chunk" and will be discussed later in this document.

? (setf x (items 1 2 3))
#<CYCLIC-ITEM-STREAM 131574401>

? (read-items x)
(1 2 3) 

? (setf x (items 1 2 3 for 12))
#<CYCLIC-ITEM-STREAM 131574401>

? (read-items x)
(1 2 3 1 2 3 1 2 3 1 2 3)

named {name}

The named option provides a name for the stream. A named stream can be referenced using #@name, which is useful for building patterns out of smaller, named motives.

? (items I cant stop stuttering in random named 'rick)
#<RANDOM-ITEM-STREAM Rick>

? #@rick
#<RANDOM-ITEM-STREAM Rick>

? (read-items #@rick 12)
(I CANT STUTTERING STUTTERING CANT CANT STOP STOP I STUTTERING 
 STUTTERING CANT)

Here are a few simple examples of typical constructor syntax:

(items 1 2 3 in accumulation)
(items 1 2 3 in cycle for 12)
(items 1 2 3 in heap for 4)
(items 1 (2 weight 4) 3 in random)
(items (1 to 2) (2 to (items 1 3 in random)) (3 to 1) in graph)
(items 1 2 3 in palindrome eliding end)
(items 1 2 3 in rotation)
(items (1 to 2) (2 to 3) (3 to (1 3)) in rewrite)

See Item Streams in the Common Music Dictionary for more information.

Accessing Item Streams

Once an item stream has been created its elements may be referenced using the access function item:

? (setf x (items 1 2 in cycle))
#<CYCLIC-ITEM-STREAM 131352561>

? (item x)
1
NIL

? (item x)
2
:END-OF-PERIOD

item actually returns two values, the next element from the stream and a "state value". The state value is normally not important to the caller and is discussed in a later section of this document. The accesser read-items reads an optionally specified number of elements from a stream and returns the results in a list. If unspecified, the number or elements returned defaults to the current period length of the stream:

? (read-items x)
(1 2)

? (setf x (pitches c4 d e f g a b in random for 8))
#<RANDOM-PITCH-STREAM 131473111>

? (read-items x)
(293.66476 329.6276 440.00018 440.00018 329.6276 329.6276 
 391.99554 391.99554)

? (read-items x 3)
(440.00018 329.6276 391.99554)

Recursive Stream Definition

Perhaps the most interesting thing about item streams is the fact that elements and option values may be expressed as constant data or as item streams of data. As a general rule, any constant value in an item stream may be replaced by an item stream of values:

(notes c3 d e for 3 in cycle)

(notes c3 d e for (items 3 6 9 in heap) in cycle)

(notes (notes c3 (notes c1 c4 c6) c5 in random for 5)
       (notes d3 d4 d5 in rotation) 
       (notes e3 e4 e5 in accumulation)
       for (items 3 (items 3 6 9 in palindrome) 9 in heap)
       in cycle)

By using recursive definition, a small vocabulary of pattern types may be combined in almost limitless ways to create "hybrid patterns", i.e. patterns that share the characteristics of more than one pattern type. Another way to think about recursion is that larger patterns are defined in terms of one or more smaller subpatterns. Subpatterns may be defined to any depth. This leads to an important question: how does a superior pattern know when a subpattern has finished enumerating elements such that the superior pattern may increment to its next element? This is where the notion of "period" is important: a period is the "chunk", or length, of a subpattern that must be fully enumerated before a superior pattern (if any) may consider any other element. A special state flag :end-of-period is returned as a second value whenever a stream finishes enumerating the elements in its current period. This state value is normally not needed outside the top level pattern, and as with all multiple values in Lisp it may be simply ignored by the caller.

Expr

Item streams interpret, but never evaluate their elements. For example, items, rhythms, notes, pitches and degrees all interpret the symbol e differently, but never treat it as a Lisp variable:

? (setf e 'rick)
RICK

? (item (items e))
E

? (item (rhythms e))
.5

? (item (notes e))
E4

? (item (pitches e))
329.6276 

? (item (degrees e))
64

? e
RICK

But there are times when it is useful to evaluate an element as a Lisp expression whenever it is encountered in a pattern. The expr constructor insures that its associated form is evaluated by Lisp each time the expr element is encountered in a pattern:

? (setf x (items 1 (expr e) 3 in random))
#<RANDOM-ITEM-STREAM 132605311>

? (read-items x 12)
(RICK 1 3 RICK 1 1 RICK RICK 1 RICK 3 1)

Here is a slightly more interesting case in which a three element cyclic returns the exact pitch of e4 followed by an expr that fluctuates between e4 and f4 followed by the exact pitch of f4:

? (setf e (pitch 'e4) f (pitch 'f4))
349.2283

? (setf x (pitches e (expr (between e f)) f))
#<CYCLIC-PITCH-STREAM 137416071>

? (read-items x 12)
(329.6276 330.37244 349.2283 329.6276 342.0422 349.2283 
 329.6276 340.61166 349.2283 329.6276 340.48065 349.2283)

Multiple Items

Sometimes it is useful to generate elements in parallel, as if they were bound together by some type of invisible glue. Consider the case where random selection is used to generate both frequency and rhythmic values:

? (setf x (notes c4 d e f g in random for 12))
#<RANDOM-NOTE-STREAM 132700261>

? (setf y (rhythms q e 16 32 in random for 12))
#<RANDOM-RHYTHM-STREAM 132636171>

? (read-items x)
(E4 E4 F4 G4 D4 E4 F4 D4 C4 F4 D4 F4)

? (read-items y)
(1.0 1.0 0.125 0.25 0.25 0.25 0.25 0.5 0.25 0.25 0.5 0.5)

If we process values from these two streams in parallel we have no control over which particular note combines with which particular rhythm:

? (loop repeat 12 collect (list (item x) (item y)))
((E4 0.25) (G4 1.0) (G4 0.25) (D4 1.0) (D4 0.5) (D4 0.125)
 (G4 0.5) (G4 0.125) (G4 0.5) (C4 0.5) (E4 1.0) (D4 0.125))

But what if we wanted to randomly select c4 but always with a quarter-note rhythmic value, d4 with a cycle: quarter, eighth, and e4, f4 and g4 with either a 16th or 32nd? This would be impossible in our example because values from the note stream would never "line up" with their intended rhythms in the other stream. Multiple items solve this problem by allowing composite items to be declared. A full explanation of multiple items is outside the scope of this document, but suffice it to say that the following form defines a two element item called a "link" (the name could be anything) that associates a note element with a rhythm element:

? (defmultiple-item link (note rhythm)
  (:element-parser (lambda (n r) (list (note n) (rhythm r))))
  (:element-period note))
#<Standard-Class LINK 131632511>

Once the new link item has been defined we can manipulate links as elements of any pattern type In any given link, either or both of its elements may hold constant values or streams of values. Here is the implementation of the random stream mentioned in the preceding paragraph:

? (setf x (items (link c4 q)
               (link d4 (rhythms q e)) 
               (link (notes e f g for 1)
                     (rhythms 16 32 in random))
               in random for 12))
#<RANDOM-ITEM-STREAM 132411531>

? (read-items x)
((C4 1.0) (F4 0.125) (E4 0.25) (D4 1.0) (C4 1.0) (G4 0.125)
 (C4 1.0) (C4 1.0) (D4 0.5) (C4 1.0) (C4 1.0) (D4 1.0))

? (doitems (i x)
  (multiple-item-bind (n r) i 
    (format t "~%Note=~S Rhythm=~S" n r)))
Note=E4 Rhythm=0.125
Note=C4 Rhythm=1.0
Note=C4 Rhythm=1.0
Note=F4 Rhythm=0.25
Note=G4 Rhythm=0.25
Note=E4 Rhythm=0.125
Note=D4 Rhythm=0.5
Note=F4 Rhythm=0.125
Note=G4 Rhythm=0.25
Note=E4 Rhythm=0.125
Note=F4 Rhythm=0.25
Note=C4 Rhythm=1.0
NIL

`amplitudes`	returns integer or floating point amplitude values.
`degrees`	returns integer scale positions.
`intervals`	returns integer intervals.
`items`	returns uninterpreted Lisp data.
`notes`	returns symbolic note names.
`numbers`	returns integer or floating point values.
`pitches`	returns floating point frequencies.
`rhythms`	returns floating point time values.
`series`	returns integer intervals.
`steps`	return relative integer intervals.

`accumulation`	enumerates elements by group.
`cycle`	enumerates elements in a loop, last returning to first.
`function`	enumerates elements by calling user function.
`graph`	enumerates elements by user defined transition rules.
`heap`	enumerates elements by random permutation.
`palindrome`	enumerates elements forwards and backwards.
`random`	enumerates elements in random order.
`rewrite`	enumerates elements by user specified rewrite rules.
`rotation`	enumerates elements by user specified permutation rules.
`sequence`	enumerates elements sequentially, sticks on last element.