Here is the final tour audio:-
In class, it was mentioned that the final concert would be held in Bing Concert Hall, and so I changed the venue of the AudioTour, and thus the overall storyboard and design.
Further, from the experiments, I came to a conclusion that the idea of a tour with a real tour guide requires that the guide is always in front of the observer, and thus, translating this into binaural terms implies that the audio always comes from directly in front of the listener. Thus, I had to find a way for people to change direction without disrupting the flow of the tour.
What I finally decided to do was to create a tour in which instead of an actual tour guide being simulated, what was being simulated was a ghost of sorts, where the voice would come and go as opposed to a continuous flow of audio from the tour guide. Further, it was less informational, and more to appreciate the acoustics of different spaces in the building, with the focus more on an ethereal experience for the listener, where they would be hearing things that weren't actually there.
This idea actually came from one of the previous versions of the tour, where test subjects heard someone walking by on their left, and would invariably immediately turn their heads in that direction.
Thus, I recorded a version where the voice was simply whispers, and there was a saxophone player sitting on a bench downstairs in Bing, and playing softly at a time when not many people were inside the concert hall, except a rehearsal going on inside the main hall.
This way, the tour could go inside the concert hall on any day and hear sound as though it were coming from the stage, but in actuality when they looked, the sound would not match up with the visuals. Sort of like ghost performers.
Further, since the saxophonist could be heard in all parts of the concert hall, as the acoustics of the spaces change through the building, a listener could orient themselves based on the sound that would be continuous in their ears. Unfortunately, what ended up happening was that as the tour went closer to the saxophonist, the audio began to clip, and due to limited recording time, a quick fix was necessary.
For the second version, I had the saxophonist play inside the concert hall, which was a thrill for him, and instead of him playing on the bench, I had a friend of mine sit on the bench and read out loud some pages from a book that he was carrying, to serve a similar purpose.
Further, from the tests that I ran on the previous version, I learned that people needed landmarks mentioned in the audio so as to have confirmation of where they are. This is a fourth type of audio cue, what I call a Confirmational Cue. For example, "We are now crossing the restrooms on your right." Alternatively, Confirmational Cues can be mixed with Directional Cues, for example, "Take a right down the stairs opposite the information desk." Both these examples are from the final tour audio.
This worked out well, and AudioTour was a success. There are still improvements that can be made, though. For example, if instead of a voice reading a book, if there were another instrument which would not clip the mics at close range, but could still be heard throughout the building, like a flute that could be played loudly, but softly when I came by with the recording equipment.
Hopefully, this will be used somewhere, and I intend to ask Professor Chafe more about whether or not he knows of anyone around campus or elsewhere where this concept can be utilised.
In class, Professor Chafe suggested that I no longer focus on the programming aspect of directionality in the original idea of using html5 for a compass and geolocation, and instead focus my efforts on perfecting the binaural tour using only audio cues.
This turned out to be far more difficult than I thought, as in the case that people turned in the wrong direction, there would be no way to get them back on track, which implies that the audio cues have to be excellently designed, and foolproof.
To test this, I have created a series of experiments that I will run in class, that allowed me to determine average times that people took to respond to different categories of audio cues.
I categorised Audio Cues into three main types: Directional Cues, Instructional Cues, and Informational Cues. The experiment includes several recordings, which run the listener through these different types of cues randomly, and include instructions to pause the file until they have completed each step, thus giving me a way to time each subject.
Using this knowledge, I will be able to design a tour that will fit for people who will respond at different speeds, keeping these variations in mind.
My next step is to design the final tour, and time it such that it fits the observations from the experiment described above.
These two weeks, I first went through and made a storyboard for the entire tour of the CCRMA building. I found my tour guide to record the voice, and I realised that there were more challenges than I originally thought.
For example, different people walk at different speeds, and so the same audio will not do for everyone. Similarly, people face different directions at different times whilst walking, but the audio should be simulated to be coming from one direction, ideally. For this to happen, the coordinates of the listener must be known, as well as the coordinates of the imaginary tour guide (sound-source). This can be figured out using the Google Maps API,as I found out from Hongchan recently.
Further, the direction in which one is facing can be solved using an html5 code that will create a compass on a website screen. The problem now arises when we get inside the building, when distances between people becomes smaller. 10m, which is the best that Google Maps can do, is too big a range inside a building, it could make the difference between two different rooms.
I have just devised a probability based method to solve this problem. Assuming a constant probability of an observer being anywhere in the range of a 10m radius, the sound can be put as though the source is in a given direction that is most likely for the person to be facing depending on where the point of the tour-guide, which is known to us, intersects with the circle of 10m radius around the listener.
I also recorded a mock-up tour using binaural mics, just to see how the tour would work out, and what it would feel like. I'll be bringing it to class tomorrow.
My next step is to start thinking about the actual code behind what I want to do, and finding the hidden challenges in that realm of things.
While I spent the first week developing ideas for what to do for the project, going through different options. I finally landed on doing using ChucK code to create pseudospeakers in the one's headphones to try to simulate a sphere of sound such that a listener can determine where the sound is coming from and follow it to a given location. The audio will be that of a tour guide guiding the listener throughout CCRMA, just as a demo for what the potential for such a tool is. I plan to demo the piece for the class at the end of the quarter such that we will follow this voice, footsteps and all, through the building.
Another idea that Professor Chafe mentioned in class on Tuesday was the "Ichabod Crane" concept where a cantaloupe or watermelon can be used to simulate the head, and binaural mics and cameras to simulate ears and eyes, respectively. These would be connected live to headphones and glasses respectively to have it simulate for an observer carrying ones own head around or displacing it to see how it throws them off.
For the first idea, I would use binaural mics to record the sound, and I will also need them for the second. My next step is to find these binaural mics and put them to good use, and to write up a script for a tour of the building.