Jieun Oh

Music 220C Project: Spring 2007

Methods of Measuring Perceived Song Accentuations

 

This project aims at determining the optimal software and hardware setup for accurately measuring the levels of emphasis singers perceive while singing simple tunes. The insights and results I obtain will allow me to more smoothly carry out my honors project (to be conducted throughout next school year), on exploring the interplay of perceived accentuation in spoken language and the musical melody.

 

-------------------------------------------------- WEEK 1 --------------------------------------------------

Brainstorming

- To do: test singing out-loud vs. singing in my head while tapping MIDI keyboard

-------------------------------------------------- WEEK 2 --------------------------------------------------

Initial experimentation with MIDI key board & velocity measures

- Results & reflections:

[Initial Testing with MIDI Keyboard]

- used “MIDI Monitor” to read inputs

- should I incorporate “aftertouch” as well as velocity levels?

- didn’t really like the “feel” of the keyboard

- not getting the result I want: my overt/ intentional accentuations were not clearly reflected in the velocity readings (see graph)

- To do: Try out contact microphone; check for consistency and reproducibility of data for MIDI keyboard velocity measures


-------------------------------------------------- WEEK 3 --------------------------------------------------

Try again using MIDI key board & velocity measures to test for consistency

- Results & reflections:

[Tapping a MIDI key using index finger]

[Tapping a MIDI key using middle finger]

- used “MIDI Monitor” to read inputs

- tried index finger vs. middle finger to see if there are any differences

- disappointed by lack of consistency... perhaps this is because I have both English & Korean version in my head? Should check with someone who only knows the song in English to see if the data is more consistent...

- interesting discovery that the velocity measures (the absolute number without any normalization) consistently converged in certain syllables (i.e. on "Birth" in "Happy birthday", or on "Ji" in "Dear Ji-eun.") This gives some structure to the music.

- To do: Try a contact microphone instead to see if consistency improves. Once that's set-up, test with someone who only knows the English version of "Happy Birthday" to see if that results in a more consistent data. Look for journal articles for potential data-measurement methods.


-------------------------------------------------- WEEK 4 --------------------------------------------------

Set up contact microphone and run a consistency test

- Results & reflections:

[Contact Mic Data- me]

[Contact Mic Graph- me]

- used "Velocity Sensitive Acoustic Drum Trigger" taped under a table, plugged into a quarter-inch cable

- used Audacity to record and Praat to obtain amplitude data

- improved consistency over MIDI keyboard! :)

- To do: Using the same set-up, test on people who only knows the English version to see if the apparent inconsistency is inherent in "me" (i.e. the way the emphasis is represented in an uncertain way in my own head since I know both versions...). Also, figure out a more efficient way of (automatically) retrieving data from Praat to get a table of data... (I had to manually enter them into Excel for this week...)

 

-------------------------------------------------- WEEK 5 --------------------------------------------------

Run trials with three volunteer subjects whose native language is English

- Results & reflections:

Native English Speakers Data ( + me)

[Subject 1] brook very consistent
[Subject 2] pete inconsistent
[Subject 3] yao very inconsistent
[Subject 4] jieun consistent

Summary of Native English Speakers (+ me)

Data
Chart

Observations:

- Consistency varied greatly depending on individuals

- Need a better way of comparing relative ups & downs

- Need a more efficient way to transfer data from Praat to Excel

- To do: Write code to transfer data from Praat to Excel (use C++ to read the text file from Praat and disregard points below certain threshold) and think about methods of analyzing data

 

-------------------------------------------------- WEEK 6 --------------------------------------------------

Run trials with two volunteer subjects whose native language is Korean; compare data with last week's

Also determined how to efficiently transfer useful data to easily open using excel

Finally, tried plotting first derivative of the intensity to track changes in perceived accentuations

- Results & reflections:

Native Korean Speakers Data

 
Subject 1 (DI)
Subject 2 (SY)
Data
Graph
Data
Graph
Sing in English
di_e
di_e_graph
sy_e
sy_e_graph
Sing in Korean
di_k
di_k_graph
sy_k
sy_k_graph

Summary of Native Korean Speakers (DI & SY)

Data di & sy: averages & normalized averages
Chart di & sy: averages
Normalized Chart di & sy: normalized averages

Comparison of 4 subjects who had relatively consistent data(DI, SY, Brook, Jieun)

Data di(k), sy(k), br(e), je(e): normalized averages & first derivatives
Chart di(k), sy(k), br(e), je(e): normalized averages
First Deriv. Chart di(k), sy(k), br(e), je(e): first derivatives

Observations:

- For both subjects DI and SY whose native language is Korean, data for Korean was more consistent than data for English.

- The Korean data for DI & SY was quite similar (see "Normalized Chart")

- Interesting discovery #1: Data for the four subjects seemed to converge at syllables "Birth" and "Dear", which are exactly the same places that I've noticed convergence when initially testing consistency with a MIDI Keyboard!

- Interesting discovery #2: DI sang in 7-7-8-7 (which actually sounded really natural!) as opposed to 6-6-6-6 beats per phrase for each of the four phrases of the song. :)

- To do: Figure out how feasible it is to determine inferred meter based on tapping data. This is crucial to verifying my hypothesis (involving the interplay of lyrics and musical line in meteric perception), but so far I cannot see clear trends in my data that allows me to "see" big beats...

 

-------------------------------------------------- WEEK 7 --------------------------------------------------

Summarize my work thus far... including goals, results, methods, revised hypothesis, possible future directions...

[Where I am, How I Got Here, Where to Go Next!]

- To do: Narrow down my focus based on revised hypothesis. Figure out designs of several mini-experiments I would like to conduct next school year that focuses on each of my major hypothesis, keeping in mind the overarching question: what factors result in Koreans' apparent insensibility(?) of both the anacrusis and the triple meter in the simple, popular tune Happy Birthday? Answering this question will, in turn, help me better understand the extent to which language/ lyrics affect beat perception in songs.

 

-------------------------------------------------- WEEK 8 --------------------------------------------------

Literature Review on metric perception from the perspective of cognitive developmental psychology:

Hannon, E.E., & Trehub, S.E. (2005). Metrical categories in infancy and adulthood. Psychological Science, 16, 48-55.

Hannon, E.E., & Johnson, S.P (2005). Infants use meter to categorize rhythms and melodies: Implications for musical structure learning. Cognitive Psychology.

Trehub, S.E., & Hannon, E.E. (2006). Infant music perception: Domain-general or domain-specific mechanisms? Cognition.

--> I've found these articles to be *extremely* relevant to my research (and I'm very excited!). I'll go over their content in class. :) Basically, they illustrate that (1) infants do perceive the underlying meter of a stimulus (as opposed to simple on-off patterns) and that (2) infants can perceive distortions/disruptions made in complex meters as well as in simple meters, a task that North-American adults cannot perform very well. Since these articles also cite other highly valuable sources, I will continue to spend some time reading them as a way of finalizing my four hypothesis (stated in Week 7's link above) as well as my experiment/ observational-study setup for each of the four hypothesis.

 

-------------------------------------------------- WEEK 9 --------------------------------------------------

Take another look at the data, focusing on anacrusis.

- Results & reflections:

1. Language --> Anacrusis

Pay attention to the change in emaphsis during the first three syllables (or the first two beats).

Findings: The general tendency was that the change in intensity between "PPY" AND "BIRTH" for English was somewhat positive while the change in intensity between the corresponding location "IL" and "CHUK" for Korean was definitely negative. This implies that "CHUK" (which corresponds to "BIRTH") is *not* perceived as a big beat in the Korean version. Although I definitely need to collect more data to be more certain that this is indeed the case, the data I currently have does support my initial hypothesis that Koreans do not perceive the anacrusis.

See following graphs for a better visualization (x-axis denotes change in intensity between "HA" and "PPY"; y-axis denotes change in intensity between "PPY" and "BIRTH"):

DI_KvsE

Compares DI singing the song in Korean vs. in English. In general, Korean plots lie on bottom right while English plots lie on top left. This suggests that "CHUK" is de-emphasized in Korean but not in English ("Birth")

SY_KvsE Compares SY singing the song in Korean vs. in English. The pattern described above is not as apparent here. However, we can see that the plots for the native language (Korean) is more consistent than that for the subject's second language (English)
NativeE_vs_NativeK Compares the native English speakers (Brook and Pete) singing in English with the native Korean speakers (DongIn and SookYoung) singing in Korean. Again, we can see that English plots generally lie in quadrant II and Korean plots generally lie in quadrant IV
All Combined Shows data points for all of the subjects that I've recorded. A bit messy, but interesting to look at. Perhaps I'm over-analyzing this simple data points, but it's still interesting to note that I consider myself about equally fluent in Korean and English, and my plots (in black) like in the middle-ish area...

- To do:

2. Clapping/ Learning & Culture/ Metric cues heard prior to performance --> Duple

For now, have them tap on beats (not on syllables) by using either a visual cue (i.e. blinking dots) or isochronous metronome beats to syncrhnize with. See how feasible it is to calculate perceived meter from intensity levels. Conduct this before my final presentation and summarize my findings.

 

-------------------------------------------------- WEEK 10: FINAL --------------------------------------------------

(Unsuccessfully) attempted to extract meter information based on tap information; its implications; summarize; future directions

- Results & reflections:

Four native English subjects used metronome and tapped only on the beat (as opposed to on the syllables).

[summary absolute intensity graph]

MB

data

absolute intensity graph

change in intensity graph

MJ

data

absolute intensity graph

change in intensity graph

ML

data

absolute intensity graph

change in intensity graph

MY

data

absolute intensity graph

change in intensity graph

- I could not observe/ figure out any clear up-down countour in above data that would allow me to somehow calculate the perceived meter. Also, tapping on the beat resulted in greater inconsistency than tapping on the syllables. Perhaps this resulted because subjects became more passive and less expressive with reliance on the metronome. Also, tapping on the beat forces subjects to focus on the underlying beat rather than the words / syllables they sing, which defeats the purpose of trying to measure the accentuations people perceive while singing songs with lyrics in ordinary circumstances.

- Hence, I felt that any attempt at extracting the perceived meter from the tapping data should be done on the syllable-tapping (though it's more challenging), and not on beat-tapping (which, according to the data above, did not show clear up-down patterns in change in intensity). HOWEVER... A review of the graphs from previous weeks tells me that there's no obvious, easy way for me to try to guess what meter the singer perceived while singing the song... In other words, the huge variation in emphasis contour across subjects (even with the same native language) tells me that metric perception information is not salient in the finger tap data. At this point, I'm not sure how feasible it is to try to categorize native English speaker's data as "triple" and native Korean speaker's data as "duple" based only on the intensity measurements.

- Also, a quick comparison between native English and native Korean data (see the bottom-most two links under "Week 6" along with anacrusis data from "Week 9") shows that measurable intensity differences in finger tapping occurs at the very beginning (anacrusis) and the very end ("You"/ "Da") of each phrase, which I feel is a direct result of language.

<Rough illustration of where accents occur in each language >

STRONGEST --> STRONG --> weak

HAppy BIRTHday to YOU, HAppy BIRTHday to YOU, HAppy BIRTHday dear ____, HAppy BIRTHday to YOU.

SENG-il CHU-kah HAP-ni-dah, SENG-il CHU-kah HAP-ni-dah, SA-rang Ha-neun ____, SENG-il CHU-kah HAP-ni-dah.

--> These help explain the beginning-of-phrase differences ("HAppy BIRTH" vs. "SENG-il CHU") as well as the end-of-phrase differences ("to YOU" vs. "HAP-ni-dah").

--> Hoever, at the same time, these do not in any way explain why clapping in duple along to the song occurs so frequently and naturally in the Korean version of happy birthday, since according to the accents pattern shown above, the Korean lyrics fit rather well to the 3/4 meter (only except without the pick-up). This makes me suspect that it is not so much the language, but an idiosyncratic clapping behavior/ culture/ learning that has resulted in the phenomenon of duple-clapping to the Korean version.

- Future Directions:

- (question: equipment for my mac- MOTU?)

- To do in the fall:

0. Conduct a thorough, final literature review on interplay of language and culture (of clapping) in beat perception

Part I. Anacrusis effect

Experiment: Run "finger-tapping on syllables" experiment using the set-up determined during the course of the quarter (contact microphone HW and Praat, Audacity, Excel SW setup). Test Korean-Americans of varying fluency on tapping while singing (1) Korean (2) La-la-la (3) English. Have them sing loudly.

Survey: Time permitting, it would also be interesting to try to survey English & Korean songs (children's tunes) in terms of the prevalence of anacrusis

Part II. Duple effect

Experiment: Time permitting, conduct a second set of experiments observing effects of metric cues heard prior to performance on meter perception: Shi-Jak (duple bias) vs. 1-2-3 (triple bias)

Observation: Perform observational studies (i.e. using videos on the internet) to get a better idea on cultural influences (as well as learning processes) on clapping duple patterns to the Korean Happy Birthday. (As a side note, I've also had a conversation with a friend over the weekend on how Koreans/ Asians tend to clap a lot when they laugh.)

Also: Be on the lookout for other translated tunes with potentially similar effects as "Happy Birthday"

------------------------------------------------------------------------------------

Recordings from last year:

These are recordings that I made one year ago that have inspired me to pursue this study further.

* Native English Speakers

Subject E1 audio claps on textual accentuations
Subject E2 audio claps on the first beat of each measure

* Native Korean Speakers

Group K1 audio (Eng) this is a bad verison that sadly falls apart... sorry about the giggles~
Group K2 audio (Kor) this is a bad version that sadly falls apart... sorry about the giggles~
Subject K3 audio (Eng)  
Duo K4 audio (Kor) unintentional duple bias since I cue them to start by saying "shi jak"

------------------------------------------------------------------------------------

Just for fun:

History of Korean National Anthem: The Korean national anthem was composed in the 1930s by Ahn Eak Tai. Apparently, before the music to the lyrics was written by Ahn, it was sung to numerous tunes, one of which is "Auld lang syne," a popular Scotland folk tune. Interestingly, this tune has a pick-up while the Korean National Anthem does not.

Old lyrics sung on the tune of "Auld lang syne"

The Korean National Anthem

------------------------------------------------------------------------------------

Thank You:

- Chris, Sook Young, and everyone in the class for weekly feedback and advice

- Miriam for lending me contact mic and cable for intial testing, and for helping me set-up

- Brook, Pete, Yao, John, Luke, Dong In, and Sook Young for volunteering to be my subjects

:)