Monday, May 20, 2024

Stargazer: A Robot For Filming Tutorial Videos

- Advertisement -

Researchers at the University of Toronto have developed Stargazer, an interactive camera robot for captivating tutorial videos by assisting instructors and content creators.

Research led by U of T computer science PhD candidate Jiannan Li explores how an interactive camera robot can assist instructors and others in making how-to videos. Credit: Matt Hintsa
Research led by U of T computer science PhD candidate Jiannan Li explores how an interactive camera robot can assist instructors and others in making how-to videos. Credit: Matt Hintsa

Robotic filming equipment holds significant consumer market potential, emphasizing the need for robots to understand humans for effective collaboration better. The robot’s role is to assist, not replace, humans.

Researchers at the University of Toronto have created Stargazer, an interactive camera robot designed to assist university instructors and content creators produce captivating tutorial videos showcasing physical skills. Stargazer enables capturing dynamic instructional videos, overcoming static camera limitations for those without a cameraperson.

- Advertisement -

Stargazer’s robot arm has a camera and seven motors. It autonomously tracks regions of interest by moving with the subject. Camera behaviors adapt to instructor cues like body movements, gestures, and speech detected by sensors. The instructor’s voice is recorded with a wireless microphone and sent to Microsoft Azure Speech-to-Text for speech recognition. The transcribed text and a custom prompt are sent to GPT-3, a language model that determines the camera’s intended angle and framing. The researchers say these instructor camera cues guide audience attention without disrupting instruction. For instance, pointing at tools prompts Stargazer to pan while instructing viewers to observe a specific action from above, resulting in a high-angle framing for better visibility.

The team aimed to find subtle signals for the interaction vocabulary, eliminating the need for separate communication with the robot. The goal is a seamless understanding of desired camera shots, integrating naturally into the tutorial experience. Stargazer’s capabilities were tested in a study with six instructors teaching various skills for dynamic tutorial videos. The robot facilitated subject tracking, camera framing, and angle combinations, enabling video production for tasks like skateboard maintenance, interactive sculpture-making, and virtual-reality headset setup. Participants practiced and completed tutorials in two takes. The researchers found that all participants successfully created videos using only the robotic camera controls and were content with the video quality. For larger environments and diverse angles, the team plans to explore camera drones and wheeled robots. Some participants tried using objects to trigger shots, which Stargazer doesn’t recognize yet. 

Future research aims to detect subtle intentions by combining the instructor’s gaze, posture, and speech signals, progressing towards the team’s long-term goal. Stargazer is positioned as an alternative for those without professional film crews, but it currently depends on an expensive robot arm and external sensors. However, the researchers believe the Stargazer concept is not inherently restricted by costly technology.

Reference : Jiannan Li et al, Stargazer: An Interactive Camera Robot for Capturing How-To Videos Based on Subtle Instructor Cues, Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (2023). DOI: 10.1145/3544548.3580896

Nidhi Agarwal
Nidhi Agarwal
Nidhi Agarwal is a journalist at EFY. She is an Electronics and Communication Engineer with over five years of academic experience. Her expertise lies in working with development boards and IoT cloud. She enjoys writing as it enables her to share her knowledge and insights related to electronics, with like-minded techies.


Unique DIY Projects

Electronics News

Truly Innovative Tech

MOst Popular Videos

Electronics Components