HW3: Interactive AudioVisual Music Sequencer
Final Deliverables
Name
Christmas Sequencer
Description
Make music and decorate Christmas trees with an interactive audiovisual step sequencer!
Screenshots
Video Demo
https://youtu.be/tdWdm5IhWcs
Instructions on Using the Sequencer
- Click "Edit" to zoom in to edit a tree
- Click "Back" to zoom out
- Press "W/A/S/D" to move selector
- Press "Up/Down" to change pitch (vertical position of the ornament)
- Press "Left/Right" to change volume (width of the ornament)
- Press "Space" to create/destroy a decoration
- Press "E" to change the instrument of the track
- Bells, Piano (High), Piano (Low), Drums
- Press "V/B" to change tempo (rate of snowfall)
- Trees default to not playing if they have no ornaments
- Number of active trees changes number of steps (0, 8, 16, 24)
- Animations
- Floating ornaments
- Falling snow
- Rotating stars when the tree is active
Production Build
https://drive.google.com/file/d/1IM4saghCw59WKJcDJ8iO6rkDR2HDQLR8/view?usp=share_link
- Please run on the macOS platform.
- Download the build file and save it to a local folder.
- Right click on the file → click “Open”.
- If you run into the “application cannot be opened” error, set the executable flag by running “chmod -R +x <app name>.app/Contents/MacOS” in the terminal, then try opening the file again.
Unity Project
https://github.com/katchen1/ChristmasSequencer
Ideas, Inspirations, and Comments
This sequencer was inspired by the festive atmosphere at the end of the year, and the fact that I enjoy decorating Christmas trees. The most difficult part was converting from a single-track sequencer (as I practiced in the Chickencer tutorial project) to a multi-track sequencer, where each track produces a different configurable sound. I also spent some time wrapping my head around how Unity and Chuck communicate with each other. Another difficult part was controlling the number of steps in the sequencer according to how many and which trees are active (have at least one ornament). The part that I enjoyed the most was creating a visually appealing and flexible interface, as well as making Christmas songs with the final product.
Acknowledgements
Milestone 2: Something Working (Wednesday, November 9, 2022)
Video link: https://youtu.be/Baqw2VIixEg
- Based on my second idea from Milestone 1, I made a lot of progress on creating the main features listed below. I used the code from the Chickencer project and modified some parts of it to include more steps, multiple tracks, and change of instruments reflected both graphically and in the audio. I also made a simple UI that allows the user to click a tree to zoom in and focus on editing an 8-beat subsequence, and click a button to zoom out to view the entire sequence.
- Implemented functions:
- Click "Edit" to zoom in to edit a tree
- Click "Back" to zoom out
- Press "W/A/S/D" to move selector
- Press "Up/Down" to change pitch (vertical position of a decoration)
- Press "Left/Right" to change volume (width of a decoration)
- Press "Space" to create/destroy a decoration
- Press "E" to change the instrument of the track
- Bells (bells)
- Piano High (green ornaments)
- Piano Low (purple ornaments)
- Drums (ribbons)
- Press "+/-" to change tempo (rate of snowfall)
- Click a star to turn the entire tree on/off (thereby changing the number of steps)
- Add animations to make it more lively
- Incorporate in-class feedback
Milestone 1 (Monday, October 31, 2022)
- Compiled a list of useful references on various types of step sequencers / designs (articles, papers, images, etc.)
- A sequencer records notes that you want to be played, then when you press a button, it plays those notes in the order you played them.
- You can change different properties (tempo, pitch, etc.) of the playback.
- Types of sequencers:
- Real-time sequencer vs. step sequencer
- Analog sequencer with knobs
- Modern built-in sequencers in the form of keyboards
- Digital audio workstations (DAW)
- For example, consider the simple case of two rows, one with seven steps and the other with eight steps. Starting together, the same juxtaposition of steps would not repeat until the 57th step.
- Sketched preliminary designs of a real-time audio-visual step sequencer!
- Three concentric circles of cats. Each circle is a different track.
- Track 1: happy meow; track 2: sad meow; track 3: angry purr.
- Use the mouse to click a cat to edit.
- Up/down keys: Height of cat maps to volume.
- Left/right keys: Width of cat maps to pitch.
- When the swimming fish gets to a cat, the cat makes its noise.
- A/D keys to decrease/increase tempo (speed of swimming fish).
- Clickable buttons to save/load sequence.
- Design 2: Christmas Tree in the Snow
- Linear. Each row of ornaments is a track, representing a different octave of piano notes. Start with 1 track (middle octave), but can add/delete tracks for higher/lower octaves. Maximum 5 tracks (middle octave + 2 higher + 2 lower).
- Use the mouse to click an ornament to edit.
- Up/down keys: Size of ornament maps to volume.
- Left/right keys: Color of ornament (red, orange, yellow, light green, dark green, light blue, dark blue, purple) maps to pitch (8 notes on the C major scale).
- Ornaments take turns glowing (playing notes) from left to right.
- A/D keys to decrease/increase tempo.
- Clickable buttons to save/load sequences.
- Design 3: Clock Orchestra
- Circular. 4 clocks, each represents a different track/instrument (violin, viola, cello, bass).
- Use the mouse to click a marker to edit.
- Up/down keys: Size of marker maps to volume.
- Left/right keys: Color of maker maps to pattern of note (single note, double note, pizzicato).
- When the second-hand passes the marker, it plays the note.
- A/D keys to decrease/increase tempo (speed of second-hand).
- Clickable buttons to save/load sequences.
