The article “Humans in the Loop: The Design of Interactive AI Systems” discusses the role and importance of human engagement and interaction with AI systems. One interesting point is the idea that AI systems do not necessarily have to be perfect. In the field of machine learning and AI, an often unquestioned assumption is that AI systems must be perfect. Enormous amounts of time and effort have been spent on continuously surpassing existing state-of-the-art systems and benchmarks. Much of ML and AI research focus on qualitative metrics relating to accuracy and performance. However, in some cases, performance accuracy is not necessarily the goal.
The idea that AI systems may not be perfect is a little scary because in most situations, it is important that they are. From object detection in an autonomous vehicle to classification in medical imaging to in factual correctness in Q&A agents, accuracy is crucial and necessary. However, as ML and AI gradually bleed into the creative aspects of life, we can view the systems under a different lens and have different expectations for them. In the fields of generative art, music, and literature, it is difficult to determine what constitutes as “correct.” When giving DALL-E a prompt, there is not necessarily a right image that should be generated. This stems from both the fact that AI generative work isn’t mature enough yet to be completely sensible and that it is difficult to determine right and wrong in subjective and creative fields. By incorporating human interaction, we can reduce expectation that these AI systems be perfect and view them as general guides or baseline templates to follow and engage with. For example, Midjourney is an AI art generator that expects user intervention. For each user prompt, the system will generate four images, and then users can choose one image to create variations of and continue this cycle until they are satisfied. Alternatively, there are artists who base their work on AI-generated art.idflood is an artist that creates geometric cities and blocks through AI, but he still adjusts and fine-tunes the final result.
Generated with Midjourney using the prompt, human interacting with AI to create music in futuristic city, steampunk style, dreamy.
AI systems should not be one-sided but should involve a back-and forth between AI and humans. This engagement can also lend more confidence towards the system. A longstanding issue has been with the transparency of AI systems. Even if the system is not transparent, having some degree of control can provide more assurance on the reliability of the results. When I use an AI system like ChatGPT or Midjourney, I do not have the expectation that the results will be perfect, and I take all the results will some skepticism. However, even if the result is not perfect, I feel better about it because I often will take the results and modify or build on top of it. By having the expectation that the AI is not perfect and will require fine-tuning, I am more prepared to interact and engage with the system. I can learn to work with the system and become a part of the process. When we actively engage with it, we may develop more trust towards it. It's important to see humans-in-the-loop not just in hindsight or as something nice to have but as a natural in-built part of the system. AI is becomingly increasingly part of our everyday lives, and we must examine and determine how we want to co-exist with it.
Part 2: Interactive Activities/Tasks
- Conversational virtual agents that engage with humans in back-and-forth prompting and response for creative writing tasks.
- An AI teacher or coach using video and sound input for learning a new instrument for beginner students.
- Musical sound cues for robots to signal intent or convey emotion to coordinate with humans on collaborative tasks.
- Physical robot musicians that can both generate music and coordinate with humans to play music.
- Social robots to work with children in play or in educational spaces that can recognize and respond to human voice/gesture.
- Robots that learn from physical interaction and correction such as guiding a robot hand to handle fragile items.
- Tracking human body motion via motion sensor or camera for physical motion feedback for dance/physical performances with AI.
- Human-robot coordination in industrial environments for tasks such as lifting heavy objects together.
- AI that recognizes and understands human gaze to enhance user experience for web/AR/VR applications to identify objects of interest.
- Robots that learn individual preferences for tasks such as feeding/eating assistance for humans with mobility issues.