View source for 356-winter-2023/hw3

= Programming Etude #3: "Wekinate Your World" =
[https://ccrma.stanford.edu/courses/356/ Music and AI (Music356/CS470)] | Winter 2023 | by Ge Wang

<div style="text-align: left;">[[Image:Wekinator-wavebot.jpg|400px]]</div>

In this programming etude, you are using Wekinator to create three interactive AI utilities/toys in your everyday life! These don't have to be useful; in fact, whimsical is good! Absurd is probably good! Playful is wonderful!

=== Due Dates ===
* Final Deliverable: '''webpage due Monday (2/27, 11:59pm)'''
* In-class Presentation: '''Tuesday (2/28)'''

=== Discord Is Our Friend ===
* ''direct any questions, rumination, outputs/interesting mistakes to our class Discord''

=== Things to Think With ===
* some relevant readings (including from your Critical Response #2 assignment)
** [https://ccrma.stanford.edu/courses/356/readings/2014-power-to-the-people.pdf "Power to the People: The Role of Humans in Interactive Machine Learning"]
** [https://artful.design/hai/ "Humans in the Loop: The Design of Interactive AI Systems"]
** [https://www.cs.princeton.edu/techreports/2010/891.pdf ''Real-time Human Interaction with Supervised Learning Algorithms for Music Composition and Performance'']

=== Tools to Play With ===
* get and play with [http://www.wekinator.org/ '''Wekinator''']
* get the latest [https://ccrma.stanford.edu/courses/356/bin/chuck/ '''bleeding edge secret <code>chuck</code> build'''] (2023.01.23 or later!)
** '''macOS''' this will install both command line <code>chuck</code> and the graphical IDE miniAudicle, and replace any previous ChucK installation.
** '''Windows''' you will need to download and use the bleeding-edge command line <code>chuck</code> (for now, there is no bleeding-edge miniAudicle for Windows); can either use the default <code>cmd</code> command prompt, or might consider downloading a [https://www.puttygen.com/windows-terminal-emulators terminal emulator].
** '''Linux''' you will need to build from source, provided in the <code>linux</code> directory
** '''all platforms''' for this project, you will be using the command line version of chuck.
* NOTE: to return your chuck back to a pre-bleeding-edge state, you can always install the latest [https://chuck.stanford.edu/ official ChucK release]

=== Getting Started with Wekinator ===
* Wekinator (and its underlying idea of creating expressive mapping interactively and by examples to continually refine the model) can be used in many different forms, and with many different softwares, they include:
** '''Approach A''': using the '''Wekinator application''' in conjunction with other softwares (ChucK, Pure Data, etc.), connected via the <code>Open Sound Control (OSC)</code> communication protocol
*** the system architecture generally looks like <code>'''INPUT >--(OSC)--> Wekinator app >--(OSC)--> OUTPUT'''</code> where INPUT and OUTPUT can be different softwares, that communicate to/from Wekinator app using OSC:
*** INPUT: provides the input data (e.g., mouse locations, sensor data, gamepad/gametrak input, video frames) for training, updating, and playing with Wekinator models
*** OUTPUT: based the output parameters of Wekinator, continually renders output in real-time (e.g., sound synthesis, graphics rendering, parameters to control a game or hardware like waving a robot arm)
*** in this approach, Wekinator app's '''TRAIN mode''' iterates between adding new observations (specific input to output examples) to training the model; Wekinator app's '''RUN mode''' predicts output based on new input
** '''Approach B''': using the '''<code>Wekinator</code> object''' in <code>ChucK</code> (this allows you work entirely within ChucK without the need for network communication)
*** this approaches uses the built-in <code>Wekinator</code> object in ChAI to TRAIN (from input-output observations) and/or RUN (predict output from new input)
*** it is also possible to use <code>Wekinator.loadData()</code> to load input-output observations from an ARFF file, either saved in a Wekinator project (currentData.arff) or saved using <code>Wekinator.saveData()</code>
*** this approach can helpful in deploying a Wekinator system without using OSC; e.g., (step 1) capture observations in the Wekinator app, save the data; (step 2) load the data in the Wekinator object in ChucK, train, and run without OSC. Also maybe helpful for those wanting to deploy using WebChucK (which doesn't yet support OSC) on say a mobile device
* for this assignment, we recommend trying Approaches A and B (in that order) before deciding what to use for each of your three system designs
* Resources for learning and working with Wekinator
** check out some [http://www.wekinator.org/example-projects/ '''example projects'''] people have created using Wekinator
** for a general overview of how Wekinator works, here are [http://www.wekinator.org/instructions/ '''basic instructions''']
** get started with using [http://www.wekinator.org/walkthrough/ '''Wekinator walkthrough''']
** for a more detailed reference, check out [http://www.wekinator.org/detailed-instructions/ '''Wekinator detailed instructions''']
** here are additional resources for working with Wekinator [http://www.wekinator.org/examples/ '''examples''']
** sample ChucK code for Approach A (using OSC): [https://ccrma.stanford.edu/courses/356/code/wekinate/approach-a-osc/ '''Wekinator -> OSC -> ChucK code''']
** sample ChucK code for Approach B (using Wekinator object within ChucK): [https://ccrma.stanford.edu/courses/356/code/wekinate/approach-b-native/ '''<code>Wekinator</code> object in ChucK''']

=== Your Task ===
* observe and identify three activities or tasks you like to "wekinate" (i.e., to design a interactive AI system for—for this assignment, keep them small)
* experiment (a lot) with different ideas (possibly more than the three you end up with)!
* of your three systems, at least one should be an expressive musical instrument using Wekinator for regression
* once again, these don't have to be useful; instead, playfulness, whimsicality, and/or expressiveness are the virtues here
* deploy these in some everyday context; feel free to involve those around you (e.g., roommates! friends! enemies!) in testing out your AI systems!
** "hey I have to do this thing for a class; would you kindly try/play/listen-to this thing..."
* for each system, create a short video showing the system in action

=== A Few Bad Ideas! ===
* Using camera input and sound output, create a day / night classifier (or sonifier)—''I mean, who has time to look out the window?''
* An instrument using one or more continuous input (such as mouse, gamepad, gametrak, Kinect, wiimote, VR controller with or without the VR) controlling a multi-parameter synthesis (like one of the STK instruments like Bowed)—''try playing this for your roommate, or your cat (and film the performance and audience reception)''
* using Wekinator's dynamic time warping, train a few audio or other gestures, and create a voice command system that does meaningless things—''inject a little AI-mediated chaos into your day''
* for more ideas, check out [http://www.wekinator.org/example-projects/ these projects] (some of these are large-scale projects; for this programming etude, remember to keep the scope of your Wekinate creations small)!

=== Reflections ===
* write ~300 words of reflection on your etude. It can be about your process or the products. Tell us about your attempt to deploy them.

=== Final Deliverables ===
* create a CCRMA webpage for this etude
** the URL should live at https://ccrma.stanford.edu/~YOURUSERID/356/hw3 or https://ccrma.stanford.edu/~YOURUSERID/470/hw3
** alternately, you may use Medium or another publishing platform (but please still link to that page from your CCRMA webpage)
* your webpage is to include
** a title and description of what you made (free free to link to this wiki page)
** three video recordings corresponding to your three interactive AI systems in action
** all relevant code and data files for all three systems
** brief report on how you created these
** your 300-word reflection
** any acknowledgements (people, code, or other things that helped you through this)
* submit to Canvas '''only your webpage URL'''