Hearing aids today amplify everything, which can make busy social settings exhausting for users. In loud places like restaurants or parties, the devices boost all sounds indiscriminately, leaving listeners to struggle with the classic “cocktail party problem”: deciding which voice matters and putting mental effort into following it. That strain helps explain why many people stop using hearing aids when they need them most.
Researchers led by Vishal Choudhari and Nima Mesgarani at Columbia University’s Zuckerman Institute are exploring a different approach: a system that reads a listener’s brain activity and uses machine learning to determine which speaker the person is attending to. The goal is a closed-loop device that selectively amplifies the attended voice while reducing competing speech and background noise.
In a proof-of-concept study published in Nature Neuroscience, the team developed real-time auditory attention decoding (AAD) algorithms and tested them with four participants who already had intracranial electrodes implanted for clinical monitoring of epilepsy. The participants listened to recordings of two competing speakers presented from different directions, with conversation topics and background “babble” added to mimic real-world noise. The researchers’ system analyzed brain signals and adjusted the loudness of each source on the fly: the attended talker became louder and the competing talker quieter.
The system performed well in the controlled lab setup. It tracked which conversation the participant was following, whether the listener had been instructed to attend to a particular speaker or chose freely, and it could respond quickly when attention shifted from one speaker to another.
Experts praised the advance but emphasized the work is still early-stage. Volker Hohmann, a specialist in auditory signal processing, noted the impressive accuracy of attention decoding when using intracranial recordings. Bernhard Seeber pointed out that the experiment’s acoustic conditions were fixed and simplified compared with everyday life, where listeners move, turn their heads, and encounters are far more dynamic. Seeber also warned that enhancing one source makes others harder to hear, which could make it difficult for a listener to switch attention to a newly important speaker.
A major limitation is invasiveness: the current demonstrations rely on electrodes placed on the brain, which is only practical in a clinical context. For consumer technology, researchers must achieve reliable attention decoding from noninvasive sensors such as scalp electrodes or other wearable biosignals. That remains a technical challenge, especially when trying to maintain speed and accuracy in variable, noisy environments.
Despite the obstacles, the research points to promising directions. If brain signals can be decoded from comfortable, wearable sensors, smart hearing devices could dynamically follow a user’s attention instead of amplifying everything. Such devices might be integrated into earbuds or smart glasses that not only boost the desired speaker but could offer features like on-the-fly summaries, note-taking assistance, or memory support in noisy settings.
Choudhari, who conceived the research as a doctoral student under Mesgarani and is now a founding research scientist at an AI company, stresses that practical, everyday brain-controlled hearing aids are not yet here. Still, the study demonstrates that attention can be decoded in real time and used to steer audio processing, a step toward more natural, less effortful listening for people with hearing loss.
Further work will need to make the decoding robust outside the lab, reduce invasiveness, and ensure that enhancing one voice doesn’t unduly block others when listeners need to switch focus. If those hurdles can be cleared, brain-guided audio devices could offer a major improvement in how hearing aids help people participate in real-world conversations.