Eleanor's Show

An audiovisualizer featuring Eleanor's little show :)

How it Works:

The little characters on the projector are mapped to the audio input's frequency spectrum. Vaguely speaking, the characters closer to the left are for lower frequencies while the characters closer to the right are for higher frequencies. The higher a character jumps, the more amplitude or magnitude there is for that certain frequency.

To see the video demo:

Screenshots:

How to Run the Program:

1. Please download the zip file and open the folder in IDE of your choice. Make sure to also download chucK-1.5.1.6 and chuGL version 2023.10.20.
2. Run the following command in terminal: chuck final_freqspectrum.ck ../soundfiles/soundfiles/audio.ck ../soundfiles/soundfiles/playmidi.ck

Files are here.

Additional Comments

I'm pretty satisfied with how the visualizer turned out. Compared to milestone 1, the idea definitely changed direction but in a way that fits the assignment's requirements better. To get rid of some of the latency and to not have so many little characters on the screen, I decided to limit the frequency spectrum to [220Hz,1000Hz], meaning I only used the FFT's [10:47] index outputs. I also adjusted the way that the characters move using 1-pole smoothing.
Resources I used include: Pages/Notability to draw, Soundly's free samples, MuseScore4, Reaper/OBS, and the many chucK and chuGL tutorials/examples + chucK read midi file example

Oct 18, 2023

Choir Frequency Spectrum

*I was using zoom screen record so there's some additional latency/chucK picking up noise from my computer fan.

Edit: Adding other variations here:

*I used voice memo on my phone to record audio & quicktime for screen record.

Comments:

For this project, I decided to go with idea #3 (as seen above) with some slight changes. I drew a few sketches of a simple character and uploaded it as a FileTexture to be applied to each box. The idea was to have each character plastered onto a box, and each box will be mapped to the spectrum's calculated positions.

For the material on the blocks, I'm leaning more towards the 2nd variation in the 2nd video.

The difficult part was to figure out how many boxes I should have in each row without cluttering the screen with boxes/covering the character. Originally I was mapping every point in the spectrum to a new box, but that was pushing the frame rate badly because I was initializing ~1024 boxes per row, with X number of rows in the window. Since the spectrum has so many points in the position array, I figured I could reduce the number of position points that are represented by the boxes and decrease the number of rows I want to be shown.

Another difficulty was getting the boxes to move along the z-axis. It took me a while to figure out, but I realized it works when I move the mesh of the box rather than the geometry itself.

I also tried to change the background of the picture to see what worked well instead of making the outline of the boxes so obvious. I tried using a transparent background as well but I'm figuring out how to get rid of the picture showing on the other faces of the cubes.

Fun: I was surprised that my sketch actually shows up well haha.

For current direction: I want to work on making the box movement more natural/smoother because you can still see the boxes jumping around. I'm also currently looking into changing up what is drawn on the boxes.

---

Milestone 1
Oct 11, 2023
Idea #1:

A 3d cube that follows the 3d color space:

By default, the cube will be white (no sound/gain under a certain x gain threshold. When there is sound, the frequencies will be mapped to RGB values (specifics TBD). Over time, the colors should fade to white. The waveform will be in the background. User should also be able to move around the camera by dragging the mouse.

---

Idea #2:

A plant which grows depending on sound input. 

We have gain g.

Sprout = no sound/under x noise threshold aka gain (g): 0 ≤ g < x
Mid-sized plant = some sound, aka gain (g): x ≤ g < y
Tree = above a certain gain threshold, aka gain (g): y > g

waveform in the background? sky?

The location of the tree in the window is mapped to a freq spectrum. 

---

Idea #3:

A cartoon character singing. 

Similar to idea #2, each character is mapped to locations across the freq spectrum/across the window.
When there's 0 sound/gain is under a certain x threshold, then we have the brain-empty character.
As time passes, the characters fade.

---

Exercise #1: Hello chuGL

---