by Yuval Adler, for Music 256A 2017, final project
A video game that would make ear training exercises more fun.
Ear training routines are often done with a kind of focus and within a stimulus vacuum that some students find difficult to commit to.
Offers an environment that engages the student in a familiar, enjoyable setting that mimics common video games, while strengthening symbolic recognition of pitched sounds' pitch names.
The Desired Outcome:
A successful design would offer an experience that would be pursued leisurely while offering significant help in ear training. The motivating educational purpose should not disrupt flow and enjoyment.
The player attempts to score points by destroying enemy units. This can be done by identifying the note names of the enemies' pitched attacks and using that note to attack back. This mechanism is the core motivator for the player to improve pitch identification.
Before starting, the scale that appears in-game, and the notes within this scale, can be altered to fit the player's stage in the learning process. Progression and scores can be tracked for each scale. (Non-standard or non-western scales should be easy to add to the game later if needed.)
In-game, notes are selected by name: "C, D, E, F, G, A, B" on the keyboard (possibly with [Shift] for sharps and [Ctrl] for flats). The fact that the letters are not organized in a clear geographical layout from low to high forces note name recognition rather than just higher/lower note comparisons. To further deter higher/lower comparison, pitches enemies produce can span four octaves. All the needed letters are within left hand reach. To attack, [Space] is pressed.
Continuous movement control of the player character is required to avoid getting hit (using the right hand on the arrow keys), thus surviving longer and possibly earning more points. This helps maintain constant engagement.
Details of the Software Created:
A full in-game loop was implemented in Unity 2D with Chunity (ChucK for Unity) as the sound engine. The meta-game aspects of scale and note selection and score and progress saving were not given an interface yet, but the underlying code supports these changes through simple parameter alterations (as well as number of concurrent enemies, projectile speeds, etc.)
Audio-visual feedback and its coherence was important to make the interaction satisfying and communicative. Matching the animations to the sound was explored through various techniques: triggering sound envelope changes from function calls in the animation key-frames, and timing animation and sound to fit by manually setting event lengths. The ability to communicate from Unity to ChucK proved a much more effective way to match visuals and audio tightly. As an example, the attack animation of the player is controlling the envelope of the sound produced, and changes to the animation sequence can be applied to the sound sequence from the animation interface alone, once the link has been set up.
Key sound design elements taken into account: the background music is pitchless to not interfere with the pitched information necessary for the core game mechanic; the enemies and player have different timbres to differentiate the source of the pitched material better.
The pixelated visuals and basic synthesized sounds generated attempt an audio-visual aesthetic that feels at home with video games possibly familiar to the student/player. These assets were custom made for the game.
The enemies are entirely alien and the main character is vaguely humanoid to keep all violence purely fantastical and abstract. The enemies crystaline shells suggest that a resonant frequency can be used to crack them. The player character's ears are prominent, drawing focus to listening.
Unity project folder with all source code and assets: HERE.
Conclusions and Future Work:
The game is fun to play for more than just a few minutes, and performance improves with practice; an urge exists to play again and outperform previous attempts - this is good!
Some modifications to controls, sprite proportions, and scene layout could help movement flow better. Successful destruction of enemy units should produce a more pronounced immediate positive feedback to the player; examples can be taken from addictive mobile/on-line games but with the appropriate caution and refinement. Testing shows early confusion regarding the attack animation as being a defensive move, and although this clears-up with play, it should be redesigned visually.