Difference between revisions of "250a Graphics Lab"
| (11 intermediate revisions by 2 users not shown) | |||
| Line 4: | Line 4: | ||
In this lab we will explore different possibilities for computer graphics in musical interaction using Satellite CCRMA, Pd, and a custom graphics system called ''SimpleGraphics''. You need your [[MaxKit]], with Satellite CCRMA. | In this lab we will explore different possibilities for computer graphics in musical interaction using Satellite CCRMA, Pd, and a custom graphics system called ''SimpleGraphics''. You need your [[MaxKit]], with Satellite CCRMA. | ||
| + | |||
| + | == Choose A Place To Sit in the Ballroom == | ||
| + | For this lab we will be using the extra monitors in the ballroom for graphics display from the Beagleboard. | ||
| + | |||
| + | * '''It is a little bit trickier to use the ViewSonic monitors because you will need to run the following extra commands on your kit before it works (so you might prefer to sit at a SAMSUNG monitor for example because then that won't be an issue)''' | ||
| + | |||
| + | * So, '''if you are using a ViewSonic monitor''', then please run the following commands on your kit | ||
| + | |||
| + | ''cd /boot/uboot'' | ||
| + | |||
| + | ''sudo pico boot.cmd'' | ||
| + | |||
| + | * change line that starts 'setenv dvimode' to 'setenv dvimode 640x480@60' | ||
| + | |||
| + | ''sudo mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n "Ubuntu" -d ./boot.cmd ./boot.scr'' | ||
| + | |||
| + | These commands change the default HDMI video resolution to one that works with the ViewSonic monitors. The new resolution will not take effect until you reboot your kit, but you will do that in the next section anyway. | ||
| + | |||
| + | |||
| + | |||
| + | == Free Disk Space == | ||
| + | * Make sure that you have at least 300MB (or equivalently 300000kB) available on your memory card. Otherwise, since the graphics installation is relatively large, you might run out of room. | ||
| + | |||
| + | * You can check the amount of disk space using the ''df'' command, which lists space available in kB. | ||
| + | |||
| + | * If you need to remove some space, you could try deleting the source code to pd (it won't affect how pd runs) | ||
| + | |||
| + | ''sudo rm /usr/src/pd'' | ||
| + | |||
| + | ''sudo rm -Rf /usr/src/Pd-0.42.5-extended/'' | ||
== Setting Up Software == | == Setting Up Software == | ||
| Line 9: | Line 39: | ||
* Use the usual procedure (see [https://ccrma.stanford.edu/wiki/Software_Lab_250 described before]) to power up Satellite CCRMA and login as the user ''ccrma'' with the password ''temppwd''. | * Use the usual procedure (see [https://ccrma.stanford.edu/wiki/Software_Lab_250 described before]) to power up Satellite CCRMA and login as the user ''ccrma'' with the password ''temppwd''. | ||
| − | * Download the [https://ccrma.stanford.edu/ | + | * Download the [https://ccrma.stanford.edu/~eberdahl/Satellite/graphicslab2012.zip lab files] to your laptop. |
* Copy the lab files to your home directory ~ using [http://cyberduck.ch CyberDuck]. | * Copy the lab files to your home directory ~ using [http://cyberduck.ch CyberDuck]. | ||
| Line 35: | Line 65: | ||
''rm GFX.zip'' | ''rm GFX.zip'' | ||
| − | and then reboot your kit by running the following command (after this you will of course need to login again): | + | and then '''reboot your kit''' by running the following command (after this you will of course need to login again): |
''sudo reboot'' | ''sudo reboot'' | ||
| Line 64: | Line 94: | ||
== Animation == | == Animation == | ||
| − | Close simplegfx-primitives.pd, restart SimpleGraphics, and open simplegfx-animation.pd. Here you will see a more complicated patch that sends out the same command periodically with parameters that vary according to the value of a sine wave. Note that <tt>osc~</tt> is used to create the sine wave, just like for a regular audio signal, but we have to use <tt>snapshot~</tt> to convert the value into something we can stick in a SimpleGraphics message. | + | Close simplegfx-primitives.pd, restart SimpleGraphics, and open simplegfx-animation.pd. (Make sure to click "compute audio" to turn audio on for this one, as it uses a bit of audio processing.) Here you will see a more complicated patch that sends out the same command periodically with parameters that vary according to the value of a sine wave. Note that <tt>osc~</tt> is used to create the sine wave, just like for a regular audio signal, but we have to use <tt>snapshot~</tt> to convert the value into something we can stick in a SimpleGraphics message. |
Animation in computer graphics is useful for all sorts of reasons, but two stand out for use in interactive computer music. Firstly, obviously entities in the graphical space will change position and other parameters in response to human input, such as a circle that bounces around the screen when you shake an accelerometer, which we can accomplish using animation. More subtly, animation can be used to make static objects like "lively" and "natural". If in the previous example the circle's size modulates a little bit over time, it becomes more clear that this circle might be something we can interact with. | Animation in computer graphics is useful for all sorts of reasons, but two stand out for use in interactive computer music. Firstly, obviously entities in the graphical space will change position and other parameters in response to human input, such as a circle that bounces around the screen when you shake an accelerometer, which we can accomplish using animation. More subtly, animation can be used to make static objects like "lively" and "natural". If in the previous example the circle's size modulates a little bit over time, it becomes more clear that this circle might be something we can interact with. | ||
| − | Come up with your own animation using SimpleGraphics and Pd. You might try varying the color of an object with <tt>noise~</tt> -- do you want to vary the color across the full spectrum, or just add a little "crackle" to a static color? | + | Come up with your own animation using SimpleGraphics and Pd. You might try varying the color of an object with <tt>noise~</tt> -- do you want to vary the color across the full spectrum, or just add a little "crackle" to a static color? |
== Combining Sound and Graphics == | == Combining Sound and Graphics == | ||
Latest revision as of 08:34, 20 June 2014
Lab 6: Graphics
See this quarter's schedule for due dates.
In this lab we will explore different possibilities for computer graphics in musical interaction using Satellite CCRMA, Pd, and a custom graphics system called SimpleGraphics. You need your MaxKit, with Satellite CCRMA.
Contents
Choose A Place To Sit in the Ballroom
For this lab we will be using the extra monitors in the ballroom for graphics display from the Beagleboard.
- It is a little bit trickier to use the ViewSonic monitors because you will need to run the following extra commands on your kit before it works (so you might prefer to sit at a SAMSUNG monitor for example because then that won't be an issue)
- So, if you are using a ViewSonic monitor, then please run the following commands on your kit
cd /boot/uboot
sudo pico boot.cmd
- change line that starts 'setenv dvimode' to 'setenv dvimode 640x480@60'
sudo mkimage -A arm -O linux -T script -C none -a 0 -e 0 -n "Ubuntu" -d ./boot.cmd ./boot.scr
These commands change the default HDMI video resolution to one that works with the ViewSonic monitors. The new resolution will not take effect until you reboot your kit, but you will do that in the next section anyway.
Free Disk Space
- Make sure that you have at least 300MB (or equivalently 300000kB) available on your memory card. Otherwise, since the graphics installation is relatively large, you might run out of room.
- You can check the amount of disk space using the df command, which lists space available in kB.
- If you need to remove some space, you could try deleting the source code to pd (it won't affect how pd runs)
sudo rm /usr/src/pd
sudo rm -Rf /usr/src/Pd-0.42.5-extended/
Setting Up Software
- Use the usual procedure (see described before) to power up Satellite CCRMA and login as the user ccrma with the password temppwd.
- Download the lab files to your laptop.
- Copy the lab files to your home directory ~ using CyberDuck.
- Unzip the files directly on your board: change into your ~ directory and unzip the archive by running the following two commands:
cd ~
unzip graphicslab2012.zip
- Remove the archive file by typing
rm graphicslab2012.zip
- Change to the ~/graphicslab2012 directory by typing
cd ~/graphicslab2012
- Install the graphics software by running a script (this will take a while and require entering the password temppwd):
./install-SGX.sh
- If the installation was successful, then remove the graphics software archive:
rm GFX.zip
and then reboot your kit by running the following command (after this you will of course need to login again):
sudo reboot
SimpleGraphics
SimpleGraphics is a small program that can receive information about graphics primitives and then render these using OpenGLES. The result is a quick and easy visualization tool for environments that otherwise might have difficulty rendering real-time graphics, such as Pd on Satellite CCRMA.
To get started with SimpleGraphics, connect your Satellite CCRMA kit to a monitor and run the SimpleGraphics program on your kit:
cd SimpleGraphics/ ./SimpleGraphics &
At this point you should just see a blank screen, because we have not given SimpleGraphics anything to draw yet.
Open up "simplegfx-primitives.pd" in pd:
pd simplegfx-primitives.pd &
Follow the instructions in the patch.
You can see that each drawing "command" consists of
send /sg/somecommand object-name a-bunch-of-numbers
The command either creates or modifies an object that has that name, according to the parameters in the remainder of the command. For the ellipse command, these numbers correspond to
x-position y-position width height color-red color-green color-blue color-alpha
So
send /sg/rect myrect 0 0 0.1 0.1 1 0 0 1
will create a rectangle a position (0, 0) with size (0.1, 0.1), with a color of (1, 0, 0, 1), i.e. pure red.
Play around with these primitives a bit. What command will make a circle that fills the whole screen? Try combining multiple primitives to create some more complicated image, like a keyboard or a face. What happens when you change a shape's alpha to less than 1? What if you draw multiple overlapping primitives with alpha < 1?
Animation
Close simplegfx-primitives.pd, restart SimpleGraphics, and open simplegfx-animation.pd. (Make sure to click "compute audio" to turn audio on for this one, as it uses a bit of audio processing.) Here you will see a more complicated patch that sends out the same command periodically with parameters that vary according to the value of a sine wave. Note that osc~ is used to create the sine wave, just like for a regular audio signal, but we have to use snapshot~ to convert the value into something we can stick in a SimpleGraphics message.
Animation in computer graphics is useful for all sorts of reasons, but two stand out for use in interactive computer music. Firstly, obviously entities in the graphical space will change position and other parameters in response to human input, such as a circle that bounces around the screen when you shake an accelerometer, which we can accomplish using animation. More subtly, animation can be used to make static objects like "lively" and "natural". If in the previous example the circle's size modulates a little bit over time, it becomes more clear that this circle might be something we can interact with.
Come up with your own animation using SimpleGraphics and Pd. You might try varying the color of an object with noise~ -- do you want to vary the color across the full spectrum, or just add a little "crackle" to a static color?
Combining Sound and Graphics
Again restart SimpleGraphics, and now run simplegfx-sound.pd. You'll see this plays back a short audio file, mapping volume to the size of a graphical object and pitch to its color.
The issue of mapping is crucial when combining audio and visual media, as individually music and visual arts don't always have a natural correspondence.
Try changing this patch to map volume and pitch to different parameters of a different type of object. Does this make more or less sense to you than the original mapping?
Make A Musical Interaction
As the final main deliverable of the lab, create an interaction with the Satellite CCRMA kit that produces both sound and graphics in response to sensory input. Think about what you want to design before you get started, and especially consider the relationship between the generated sound and graphics.
