Finding Opportunity in Spatial Audio

By Yao Wang

Last spring, I was 9 months away from graduating, and the panic of post-graduation uncertainty was becoming unbearable. I was struggling to plan my career and I wanted to do something different. That was when I first got introduced to the world of virtual reality and spatial audio by my friend Jason Parks.

I spent my summer researching the ins and outs of spatial audio and decided to do a project around my research for my final Electronic Production and Design (EPD) project, the portfolio-building class EP-491. The idea is to challenge the way we enjoy and experience music. I wanted to create a completely immersive 360 audio and visual experience, where the audience would find themselves at the center of all elements, being surrounded by choir, strings, synths and imagery. Unlike traditional audio formats, spatial audio lets the listener receive a real-time binaural rendering of the audio stream in sync with their head rotation in the video. Simply put, it has the potential to make you feel like you stand in the middle of a large ensemble, and where you turn your head to, the sound from that direction will be emphasized and filtered as you naturally hear sounds. The mix will be dynamic and interactive.

The central theme of this piece is repressed and unleashed emotions. All the elements of the composition are all tailored to this theme. The lead singer sings a tempo and represents the restrained/logical self. The choir sings rubato and represents the underlying emotions and desires. As the song progresses, the exchange between the two eventually merge towards one, forming a complete and free identity.

On October 27th, we had a recording session of the choir parts with 40 students from all around the school. The recording was done using 3 ambisonic microphones (Zylia, Ambeo, TetraMic). We tried forging a 320 piece choir by asking the 40 students to shift their positions around the microphones for every overdub. We also recorded 12 close mic-ed singers to have some freedom spatializing individual mono sources. The entire process was recorded by the Berklee YouTube Production Team. The footage will be released in early 2018 as a behind-the-scenes video alongside with the formal 360 music experience also to be released on the Berklee Youtube Channel.

Throughout the course of realizing this project, many people gave me critical support and guidance. My EP-491 professor Dr. Richard Boulanger has guided me artistically and professionally, EPD chair Michael Bierylo supported me from the start and helped me gather all the resources I needed to see this project through. Jeanine Cowen, Steve Pardo and Jason Parks have generously advised and mentored me on the technicality of spatial audio.

As the size of this project spiraled into something I did not expect, I knew this would have to be a team effort. My role in this project is the project leader, composer, and executive producer. I found a most talented team of friends to work on this together: Gareth Wong and Deniz Turan as co-producers, Carlos Del Castillo as video animator, Ben Knorr as music contractor, Paden Osburn as music contractor and conductor, Jeffrey Millonig as lead engineer, Alejandro Rivero and Jane Stenvig as assistant engineers, and Sherry Li as lead vocalist and lyricist. I am truly grateful for their hard work, dedication and focus.

I don’t think that a student project like this would have been possible to realize anywhere other than Berklee; as I approach my graduation, everyday I grow more fond of my school and sad that I must leave it soon. However, I now see a brand new career path opening in front of me because of this project. I am extremely excited to deliver this piece at the end of the year and mark the perfect exit for the best few years of my life at Berklee.