Immersive Multi-talker Microphone system
Submitted by: Ryan M. Corey, Andrew C. Singer, University of Illinois, Urbana Champaign
Common problems with hearing enhancement technology include poor performance in noisy environments or rooms with high reverberation, limitations to the number of speakers and the inability to transmit the interaural cues that are necessary to locate a sound source and to distinguish overlapping sources. Adding wireless microphones can help improve intelligibility. There are wireless microphones available commercially, mostly proprietary accessories from hearing aid manufacturers, but they only work with one talker at a time and so are not useful in group conversations. The improved system proposed by the researchers uses multiple remote microphones together with the hearing device to produce a more immersive experience.
There are basically two common types of microphones available for such a system: remote microphones worn by a talker on the collar, which is able to transmit low-noise, low reverberation speech. This type works great for understanding speech, but because it’s mono it won’t allow the distinction of multiple overlapping speakers or the ability to tell where a sound originates from. The second type is a microphone that is built into earpieces, which is capable of producing a stereo field. This has the benefit of allowing for sound localisation and distinguishing individual sources. This type of system however tends to be noisy.
As part of the ASA 2021 Tympan Design Challenge the research team set the objective to overcome these shortcomings by creating an immersive listening experience using the Tympan and hearing-aid style earpieces with front and rear microphones (issued by Tympan, production completes in summer 2022). An immersive listening experience should allow for multiple simultaneous talkers, preserve spatial cues for locating, separating and motion tracking of sound sources, and have low background noise. The key idea is to do this by combining the strong suits of the two types of microphone system: the low noise of remote mics combined with the spatial cues provided by earpiece mics.
How? For this test two talkers were fitted with wireless microphones. The sound of the talkers was transmitted to the Tympan. The Tympan simultaneously received the stereo signal through the set of earpiece microphones that were directly plugged into the Tympan microphone jack. The magic happens as follows: real-time adaptive filter algorithms running in real time on the Tympan match the magnitude and phase of signals coming from the remote microphone to the sound received by the earpiece microphones. The end result is that the earpiece output has the signal-to-noise ratio of the remote microphones but the spatial cues of the earpiece microphones.
For the test a first fully portable prototype was developed based on the Tympan and earpieces. The Android app was used to give the user control over mixer levels and to monitor the performance. Future developments foreseen by the authors include improvements of the adaptive filter, the key ingredient of this system, exploring the application of this technology in hearing aids, and including features like frequency-selective gain, feedback control and per-talker dynamic range compression.
To learn more about the immersive multi-talker microphone system please read Ryan M. Corey's blogpost about the project.
More information about the researchers' work can be found on the Augmented Listening Laboratory website.
About the TYMPAN ASA2021 Design Challenge
During the ASA (Acoustical Society of America) conference in June 2021, Tympan hosted a design challenge: What is possible with the Tympan?
10 exciting new applications were submitted and presented at the following ASA conference: Enhancements of hearing aids, spatial acoustic processing and smart earphones and much more. Stay tuned if you want to learn what is possible and to keep track of future developments.