Synesthesia and Pitch Memory

Erin Baumann's Music 220c Blog

Update May 23rd 2013

It's been quite a while since I've updated, so I'll try to cover all the advances I've made in the project in the past month.

First off, I've streamlined the test program - it is now customized for each subject, flashing their selected colors with the associated note in a standard octave. I also have a control program that simply plays the 12 pitches over a white screen, with the appropriate pauses in between. Both of these programs begin with the C4 test tone I referenced in my last update.

My test program also features an onscreen button functionality that runs alongside the pitch test. When a subject presses the buttons, the program prints out the name of the button pressed, as well as the time it was pressed (relative to the start of the program) to a text file. This allows me not only to gauge the subjects' accuracy in identifying pitch, but also any hesitation different test conditions may or may not cause. Unrelated to the program, I've also added a set of instructions to be read out loud before each trial, just to ensure that subjects fully understand how to respond to the tasks that I've set.

My experimental process is as follows: subjects will complete a survey to gauge the nature of their synesthesia, as well as obtain baseline information about their musical experience. Subjects will then be asked to select from an RGB color wheel (linked in the post below) 12 colors that correspond as closely as possible to their personal color associations in a 12-note octave (if the subject is not a synesthete, they will be asked to select colors randomly). Next, subjects will complete the "control" pitch memory task, which entails identifying a series of randomly ordered tones after hearing a reference tone. This portion of the study involves no visual component. Finally, subjects will complete the "color test" pitch memory task, which is almost the same as the control task with the added element of color associations flashing onscreen.

So far, I've tested two non-synesthetes. The results have been a little surprising - both subjects performed markedly worse on the color test than on the control test! This is the opposite of what I expected; I rather thought that by asking subjects to select their own colors, I would evoke a sort of pseudo-synesthetic response. I may have to tack a follow-up survey on to my experimental process in order to gain some insight into this. My instinct at this point is that the colors are a distractor, preventing full tonal recall in the time allotted, but I suppose I'll have to examine this further. I've been putting off testing my synesthete subjects - since there are so few of them, I'm afraid of exposing them to a less-than-streamlined test program and contaminating my (already quite limited) sample. I still think that the synesthetes will perform better on the color test than the control subjects (especially given the results I've gotten so far), but I'm not sure how they will perform on the control test.

Update April 23rd 2013

Ironing out the kinks in my new project. I've mocked up a form of my trial in ChucK and Processing that flashes random colors in a computer window. Each random color flash lasts 5 seconds, after which a (presumably) neutral white screen is displayed for 10 seconds. In fully functioning form, a pitch will play along with each color flash, and the flashes will not be randomly generated, but rather will conform to each subject's pitch-color associations. I have tried to add pitches to my program, but no luck yet - a bug is keeping them from playing in the way I'd like them to.

I am also figuring out the best way to gauge the effects of color cues vs. no color cues on subjects outside of accuracy. If I could figure out a way to implement radio buttons that would return both the subject's pitch id guess and the time it took for them to select an answer, that would be ideal - I could then determine how much confusing/no cues affect their ability to complete the task assigned to them.

I'm also struggling a bit with deciding how to use reference tones. My gut instinct is that frequent reference tones will make the task so easy that results are meaningless; however, I do have perfect pitch so my view of this might be a little skewed. I may have a reference tone play once before each set of 6 pitch identification tasks. With that in mind, though, I'll have to consider how memory of the reference tone could alter my results.

Finally, I've been thinking about how to standardize my study across subjects, despite the subjectivity of the matter. It might be useful to have the non-synesthete participants choose arbitrary color associations at the beginning of their trial so they are not performing only blind pitch id tasks. This brings back the point about pseudo-synesthesia I mentioned before; adults have been observed displaying synesthesia-like behavior after significant training to change their perception. As for determining each subject's color associations, the easiest way I can think of to do this is to present them with an RGB color wheel and an octave's worth of pitches and ask them to find the color that each pitch evokes.


Update April 19th 2013

As of today, I've decided to overhaul my project and change its focus somewhat. This change is based in the subjective nature of synesthetic experiences - since each case of synesthesia is so dependent upon the individual who experiences it, I would have to test a massive number of subjects in order to determine whether or not my hypothesis (that synesthesia positively effects music learning) holds water. In addition, looking back at my experimental design, several elements could obfuscate my results, including the different instrumental timbres, the different trials each subject would be exposed to, musical experience, how each subject engages with their synesthesia, etc...

As a result, I've decided to narrow the focus of my project to several case studies. These will focus on three different musical synesthetes (myself and two others), as well as a control group of 3 non-synesthetes. I plan to directly engage the subjects' synesthetic experience by building a tonal recall program that will both play a pitch for the subject to identify and flash that subject's associated color for that pitch on the screen; the subject's accuracy at pitch identification will then be compared with a blind test, in which no visual cue will be assigned to the pitches played.

The ultimate goal of this project is still to determine whether synesthesia plays a role in music learning, on a smaller scale. I expect it will be easier for synesthetes to identify pitches when their color associations are displayed on-screen compared to when they are not. It will also be interesting to see how this affects the non-synesthete control group - this could have interesting implications for pseudo-synesthesia as evoked through training.

Update April 14th 2013

I've figured out a starting point for differentiating my tonal recall test from the skill-building exercise that I'll be using. In my survey, I'll be asking participants to carry out a very simple tone-matching exercise. A reference tone will be played once, and participants will have to choose the matching pitch from amongst four possible answers, each with a different timbre (for instance, the reference tone will be played on a piano, while the possible answers will be on four other instruments). The participant will be able to listen to the possible answer tones as many times as they wish.

The skill-building game, on the other hand, will be a little more freeform. Participants will hear a reference tone (which they will be able to play more than once) and then have to match the tone by manipulating a slider that changes the pitch of the answer tone. Once they submit their answer, the participant will be able to see if they were too low, too high, or correct (which for the purposes of this game will probably be defined as 3ish Hz).

The main problem right now with these two approaches is technical. The survey hosting site I was planning to use for my survey doesn't support audio embedding, which would make my testing phase extremely clunky! I'll need to find a new survey-hosting site or figure out some other way around this. I'll also need to figure out a way to code the skill-building phase - I've found a few tutorials on relevant functions in Java, so I might have a foothold there.


Update April 11th 2013

I'm working on creating the opening/closing survey that participants will take as part of my project. I will probably be working with people with a range of musical backgrounds (since I'll have to take what I can get in terms of finding other synesthetes), so in order to quantify musical ability I will be asking each participant for their score on the Ollen Musical Sophistication Index. This short questionnaire provides a numerical score that indicates a person's relative musical ability, with scores above 500 indicating a higher level of aptitude and scores below 500 indicating a lower level. I'm still figuring out how I'll take this into account when looking at my results, though...


Update April 8th 2013

I've narrowed down my focus as of today - I'm going to be working with the question, "Do individuals with musical synesthesia show enhanced capability for music-based learning than individuals without musical synesthesia (non-synesthetes and non-musical synesthetes, i.e. grapheme-color and the like)?" To determine this, I'm going to perform a multi-part experiment. First, I'll survey participants to determine the nature of their synesthesia, their musical background, and their tonal recall skills. Next, I'll ask them to take part in a skill-building period of a few weeks in which they play a game to hone their pitch memory. Finally, they'll take the first survey again in order to test their improvement (as well as the stability of their synesthesia, if applicable - the lack of a quantifiable test for synesthesia makes it necessary to double-check self-reporting).