Augmented Reality offers great potential to support musicians in collaborating remotely. We are building an application to enable drummers to play together in a drum circle using physical electronic drum pads and augmented reality technology. Unlike previous remote jamming applications, musicians who use this application will see AR avatars of each of the other drummers they are playing with (up to two other drummers) which they can place in their own environment via AR fiducial markers. Audio will be captured via the MIDI output from the electronic drum pads and sent to each musician in the circle, where it will be synthesized into spatial audio based on the placement of the avatars. Combining the MIDI output with hand tracking, we will enable players to communicate through body movement with their avatars, a core component of music collaboration.
See the communication techniques commonly used in a drum circle in this video.
Publications
-
Hopkins Torin, Weng Suibi Che Chuan, Vanukuru Rishi, Wenzel Emma A., Banic Amy, Gross Mark D., Do Ellen Yi-Luen (2022), AR Drum Circle: Real-Time Collaborative Drumming in AR, , Frontiers in Virtual Reality, Vol. 3, Article 847284 (11 pages), online Aug 19, 2022, URL=https://www.frontiersin.org/articles/10.3389/frvir.2022.847284, DOI=10.3389/frvir.2022.847284
-
T. Hopkins, S. C. -C. Weng, R. Vanukuru, E. Wenzel, A. Banic and E. Y. -L. Do, "How Late is Too Late? Effects of Network Latency on Audio-Visual Perception During AR Remote Musical Collaboration," 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2022, pp. 686-687, doi: 10.1109/VRW55335.2022.00194. (Christchurch, New Zealand—March12-16, 2022).
Ericsson Research Project Collaborators:
Amir Gomroki, head for 5G, North America; Héctor Caltenco, senior researcher; Per-Erik Brodin, research engineer; Ali El Essaili, senior research engineer; Chris Phillips, master researcher; Alvin Jude Hari Haran, senior researcher; Per Karlsson, director, media technology research at Ericsson and head of Ericsson Research in Silicon Valley.
Associated Researchers
