Homework #3: Interactive AudioVisual Music Sequencer

Renee Q.

November 11, 2024

Music 256A / CS 476A, Stanford University

—

Final Deliverables

Name: Celestial Sparks

Description: This sequencer is an audiovisual experience designed to bring the beauty of fireworks generated from flickering stars to musical composition. Each star triggers an ethereal blend of sound and visual effects, simulating the burst and fade of fireworks. With randomized particle shapes and colors, combined with tonal layers, the sequencer creates a unique interactive performance each time. Stars play randomized notes from a pentatonic sequence, allowing users to explore a range of ambient textures and melodies, which transform with visual effects and subtle animations.

Code

Instructions on Using the Sequencer

Load the Program: To use the sequencer, download the file and type "chuck fireworks.ck" in the terminal.

Interacting with Stars: Hover over each star to see slight color shifts and animations. Click on a star to trigger a sound and firework-like visual.

Creating a Sequence: Use multiple pads in succession to build layered sounds.

Reflection

This sequencer was inspired by the visual beauty of fireworks and the way they build tension and resolve in the sky, much like musical composition. By combining sound and visuals in an interactive sequencer, I aimed to create a system where users feel immersed in both the audible and visible aspects of the performance. Working on this project was both challenging and deeply satisfying. I enjoyed designing each interactive component, especially the star animations of light (color) intensity, scale variation, and hover effects. Fine-tuning the timing and randomization of each effect was challenging but rewarding, as it brought the sequencer to life. I also randomized the placement of the stars within a certain constrained space, which makes them more natural in the night sky. In addition, working with parametric equations for the fireworks was an interesting process as I was mentally challenged to integrate the math into the program. I also used varying color intensity to create the flickering effects, as well as the fade-out of the particles to simulate the lifespan of a firework. In terms of the sound, I programmed a layer of continuous background music to contribute to the overall ethereal atmosphere. The choice of using a pentatonic scale was also to ensure that the notes harmonize in general, creating a sense of cohesion and simplicity that makes the interactive experience more accessible and enjoyable. This harmony allows users to experiment freely without producing dissonant sounds, making each interaction feel rewarding and musically coherent, even for those with little to no musical background.

System Design

• Sequencer Core:
  • Sequencer (main file): Responsible for setting the BPM, defining the sequencer steps, initializing pads, and placing them according to window resizing events. It orchestrates the interactions between pads and melodies, and defines the scene’s layout and parameters.
  • placePadsHorizontal() and placePadsVertical(): Helper functions that arrange GPad instances on a horizontal or vertical axis within the scene.
• Pads and Interactions:
  • GPad Class: Represents each pad in the sequencer, with properties like state and color. Each pad can be in states like NONE, HOVERED, ACTIVE, and PLAYING. Pads interact with mouse events through the pollHover() and clickListener() methods.
  • Particle System: Each GPad contains particles that are spawned when a note is played. This adds visual effects to interactions, simulating a firework-like appearance.
• Instruments and Melody Generation:
  • Melody Class: Represents the musical instruments used for the sequencer. The melody uses sine wave oscillators with low-frequency oscillators (LFOs) for added modulation and an ADSR envelope to shape the notes. This class provides methods for setting frequencies and playing notes, ensuring sound production aligns with the sequencer’s scale.
• Background Music:
  • The playBackgroundMusic() function spawns a loop of low-frequency background notes with reverb for an ethereal ambiance. It selects notes randomly from the pentatonic scale, contributing to a coherent and immersive soundscape.
• Mouse Input Handling:
  • Mouse Class: Tracks and updates the mouse position in the world space, providing real-time interaction with pads for hovering and clicking actions.
• Visual Effects:
  • Particle Class: Handles individual particles in the particle system attached to each pad. Particles have properties like shape, lifespan, and color, and they decay over time while simulating flicker effects. They are controlled by randomization for added natural feel.
  • Bloom Effect: Applied globally using BloomPass to intensify and blur bright areas, enhancing the fireworks effect with a soft glow.
• Sequence Control Functions:
  • sequenceLead(): Control the sequencing logic, iterating through each pad column, triggering notes when activated, and passing the lead instrument’s array to create harmony in line with the sequencer’s timing.

—

Milestone B: Something Working

The sequencer design aims to visualize dynamic, firework-like effects, bringing an interactive and colorful display to each click and play action. Upon clicking, the pads generate a burst of randomly generated colors that emit particles, creating an initial fireworks effect before they disappear. Hovering over an activated pad relights the particles, causing them to pause in place, but as the mouse moves away, they disperse randomly and fade out after a set duration, enhancing the impression of a firework’s lifecycle. The audio component is still under development, with the intention of complementing the visual experience.

Demo Video

Code (Work in Progress)

—

Milestone A: Research + Preliminary Design

Before researching music sequencers, I primarily associated them with software like Logic Pro, known for its looping capabilities and layer adjustments. As I started doing some preliminary research, I was intrigued to discover the vast array of sequencer types and their extensive history.

Below are some useful references that explore various designs and functionalities of music sequencers:

• Basic Music Sequencer Tool: A straightforward online sequencer for creating melodies.

• https://onlinesequencer.net/ 

• Simple Online Sequencer: An easy-to-use interface for beginners to experiment with sequences.

• https://muted.io/sequencer/ 

• Interactive Minimalistic Scene and Music: Combines minimalistic visual art with sound for a unique creative experience.

• https://unseen-music.com/yume/ (I really love this one!)

• Kandinsky and Song Maker: Engage with music and art in an interactive format inspired by Wassily Kandinsky’s theories.

• https://musiclab.chromeexperiments.com/Experiments 

• Music Creation Through Text: Generate loops and music by typing text, demonstrating a novel approach to sequencing.

• https://typatone.com/ 

• Exploring Different Music Sequencers: A useful video overview of various sequencer types, highlighting their unique features and applications.

• https://youtu.be/aXTT8jUhoAg?si=TQ-E1j6YLO7Mo9xQ 

Sketch #1

Sketch #2

Sketch #3