主要内容

Binaural Audio Rendering Using Head Tracking

This example uses:

打开实时脚本

Track head orientation by fusing data received from an IMU, and then control the direction of arrival of a sound source by applying head-related transfer functions (HRTF).

在典型的虚拟现实设置中，IMU传感器连接到用户的耳机或VR耳机上，因此声音源的感知位置相对于视觉提示而言，与头部移动无关。例如，如果将声音视为来自监视器的声音，那么即使用户将头转向侧面，它也是如此。

Required Hardware

  • Arduino Uno

  • Invensense MPU-9250

Hardware Connection

First, connect the Invensense MPU-9250 to the Arduino board. For more details, seeEstimating Orientation Using Inertial Sensor Fusion and MPU-9250.

创建Sensor Object and IMU Filter

创建anarduino目的。

a = arduino;

创建Invensense MPU-9250传感器对象。

IMU= mpu9250(a);

创建and set the sample rate of the Kalman filter.

Fs = imu.SampleRate; imufilt = imufilter('SampleRate'，fs）;

Load the ARI HRTF Dataset

当声音从空间到您的耳朵的一个点传播时，您可以根据体内时间和级别的差异（ITD和ILD）进行定位。这些频率依赖性的ITD和ILD可以测量并表示为任何给定的源高程和方位角的一对脉冲响应。ARI HRTF数据集包含1550对脉冲响应，这些反应跨越360度，高度从-30到80度。您可以使用这些脉冲响应来过滤声源，从而将其视为来自传感器方向确定的位置。如果传感器连接到用户头上的设备上，那么尽管头部移动，声音被认为是从一个固定位置传来的。

First, load the HRTF dataset.

ARIDataset = load('ReferenthHrtf.mat'）；

然后，从数据集中获取相关的HRTF数据，并为我们的处理以有用的格式。

hrtfdata = double（aridataset.hrtfdata）;hrtfdata = permute（hrtfdata，[2,3,1]）;

获取关联的源位置。角度应与传感器相同的范围。将方位角从[0,360]转换为[-180,180]。

sourcePosition = ARIDataset.sourcePosition(:,[1,2]); sourcePosition(:,1) = sourcePosition(:,1) - 180;

Load Monaural Recording

Load an ambisonic recording of a helicopter. Keep only the first channel, which corresponds to an omnidirectional recording. Resample it to 48 kHz for compatibility with the HRTF data set.

[heli,originalSampleRate] = audioread('Heli_16ch_acn_sn3d.wav'）；Heli = 12*Heli（：，1）;% keep only one channelsampleRate = 48e3; heli = resample(heli,sampleRate,originalSampleRate);

Load the audio data into aSignalSource目的。Set theSampleSperframeto0.1秒。

sigsrc = dsp.SignalSource(heli,...'SamplesPerFrame',sampleRate/10,...“ signaldaction”,“循环重复”）；

Set Up the Audio Device

创建anaudioDeviceWriterwith the same sample rate as the audio signal.

DeviceWriter = AudioDeviceWriter（'SampleRate',sampleRate);

创建FIR Filters for the HRTF coefficients

创建a pair of FIR filters to perform binaural HRTF filtering.

FIR = cell(1,2); FIR{1} = dsp.FIRFilter('NumeratorSource','Input port'）；FIR{2} = dsp.FIRFilter('NumeratorSource','Input port'）；

初始化方向查看器

创建一个对象来执行实时可视化，以实现IMU传感器的方向。调用一次IMU过滤器并显示初始方向。

orientationScope = HelperOrientationViewer; data = read(imu); qimu = imufilt(data.Acceleration,data.AngularVelocity); orientationScope(qimu);

Audio Processing Loop

执行处理循环30秒。该循环执行以下步骤：

  1. 读取数据从乌兹别克斯坦伊斯兰运动传感器。

  2. Fuse IMU sensor data to estimate the orientation of the sensor. Visualize the current orientation.

  3. Convert the orientation from a quaternion representation to pitch and yaw in Euler angles.

  4. 利用interpolateHRTFto obtain a pair of HRTFs at the desired position.

  5. Read a frame of audio from the signal source.

  6. Apply the HRTFs to the mono recording and play the stereo signal. This is best experienced using headphones.

imuoverruns = 0;Audiiunderruns = 0;Audioftered = Zeros（Sigsrc.SamplesPerframe，2）;抽动尽管toc < 30％从IMU传感器中读取。[data,overrun] = read(imu);ifoverrun > 0 imuOverruns = imuOverruns + overrun;end％FUSE IMU传感器数据以估计传感器的方向。qimu = imufilt(data.Acceleration,data.AngularVelocity); orientationScope(qimu);% Convert the orientation from a quaternion representation to pitch and yaw in Euler angles.ypr = eulerd(qimu,'zyx','frame'）；yaw = ypr(end,1); pitch = ypr(end,2); desiredPosition = [yaw,pitch];％在所需位置获得一对HRTF。interpolatedIR = squeeze(interpolateHRTF(hrtfData,sourcePosition,desiredPosition));% Read audio from fileaudioIn = sigsrc();％应用HRTFAudioFiltered（：：，1）= fir {1}（Audioin，InterpoLatedir（1，:)）;% LeftaudioFiltered(:,2) = FIR{2}(audioIn, interpolatedIR(2,:));% RightaudioUnderruns = audioUnderruns + deviceWriter(squeeze(audioFiltered));end

Cleanup

发布资源，包括声音设备。

release(sigsrc) release(deviceWriter) clearIMUa