By Jan Skorupa, Wojciech Raszewski, Eryk Skotarczak and Maciej Głowiak, March 2019
In February 2019 Poznań Supercomputing and Networking Center (PSNC) started experiments with ambisonic recordings and multi-channel sound projection, which are related to the Immersify project. A jazz band called Anomalia (members of which are students of the Ignacy Jan Paderewski Academy of Music in Poznań) was invited to take part in the project. The audio content recorded by the band was mixed in a spherical ambisonic space created by 24 loudspeakers in the New Media Laboratory of PSNC. In addition to the sound recordings, 8K and 360° videos were produced to show Anomalia’s performance. The audio material was eventually encoded as ambisonic sound of the third and fifth order, then decoded binaurally and finally combined with a 360° video. All of that required a proper workflow for the production and mixing of surround sound for the Immersify project.
Jan Skorupa and Wojciech Raszewski from PSNC explaining ambisonic sound (For English subtitles click icon in the bottom-right corner)
Recordings
The project began with the search for a suitable music band. Thinking about the aim of the recordings, the band we were looking for had to have a sufficiently large line-up and diverse material, which would provide an interesting projection of the spatial sound. Anomalia met our expectations as it is a jazz septet with musicians playing: drums, double bass, guitar, tenor saxophone, soprano saxophone, trombone and trumpet.
On 7 February 2019 the musicians met with researchers from the New Media Department of PSNC in order to make test recordings. For this task, a sound-adapted event room located in the PSNC was used. The place was illuminated in an atmospheric way using some historic stage lamps. We placed the members of the band in a circle, although we did not try to separate each of the instruments. We wanted to achieve the most spatial and natural recording possible. To record wind instruments we used DPA 4099 microphones. An additional condenser SE2200a microphone was used for tenor saxophone in order to obtain more accurate recording of lower frequencies. For the recording of the double bass we used line output from the amplifier – this solution, however, was not fully satisfactory because in that configuration it was impossible to record the interactions between strings. So, to achieve the desired effect, we additionally installed a DPA instrumental microphone. In the case of the electric guitar, in order to get the deepest possible sound, two different types of microphones were used: dynamic Shure SM57 and condenser Audio Technica AT 2050. Additionally, to obtain ambient sound, Ambeo VR Mic from Sennheiser was suspended from the ceiling above the Insta360 Pro camera, which made first order ambisonic recordings possible. The material from this microphone served in the further stage (mix) as the basis for multi-loudspeaker projection. The live recording was made using Midas Pro’s microphone preamps and converted to Dante through KlarkTeknik DN9650. The digital signal was then sent to a recording workstation with Reaper DAW installed.
Monitoring system
In order to mix the obtained material, a spherical ambisonic installation of 25 Genelec 8010A loudspeakers (including one subwoofer) and was set up in a separate studio. The loudspeakers were arranged in three ring planes at different heights. The first ring was placed at 30 cm from the ground, where the speakers were located at the angle of 45 degrees towards the center of the sphere; the second ring – at 165 cm, where the speakers were not tilted towards the listening point; and a third ring – at 280 cm, where the speakers were inclined at the angle of 45 degrees towards the center of the sphere. The subwoofer was positioned at the front of the speaker block.
To operate the whole system we used a Focusrite Rednet 3 interface to convert Dante output from the DAW to four ADAT links. That, however, was connected to four Behringer ADA 8200 D/A converters. Such interface combination allowed us to support 32 balanced analog signal outputs.
Mixing and production of audio files
The main problem with mixing of the recorded material in the multi-loudspeaker installation was finding proper tools. The first task we had to cope with was the selection of an appropriate DAW, which would allow us to work on such production more confortably. After many tests we decided to use Reaper – an application that let us to create tracks containing up to 64 audio channels with ease. In addition Reaper also supports export of the final product to a 64-channel WAV file. The next step was to find an appropriate external software for realization of acquired recordings in ambisonic space. For this purpose we used VST plugins, which were created in the Institute of Electronic Music and Acoustics at Kunst Uni Graz. These tools enabled a comprehensive mix and free spatialization of the recordings in an ambisonic system up to seventh order, as well as binaural representation of the effect using HRTF (Head – Realated Transfer Function).
Conclusions
The recordings were an experiment connected with the production of the 3D audio within the Immersify project. The audio content as well as created workflow significantly contributed to the creation of the cutting edge Immersify software. It is a major step forward in the accomplishment of embedding 3D sound into an application that serves as a sophisticated audio and video player. Currently further work is in progress – we are pursuing the combination of high resolution 8K video with the recorded ambisonic material.