Starkey Hearing Research Centre
2110 Shattuck Ave #408 Berkeley CA 94704 USA
School of Medical Sciences
University of Sydney, NSW 2006 Australia
Hearables – a snap shot
Light and sound dominate our world. The consumer electronics industry has invested hugely in visual displays such as HDTV: 2D and 3D and now head mounted displays. Audio has enjoyed less investment but low energy Bluetooth and inexpensive DSP have enabled a wide range of reasonable quality, untethered audio devices that can travel with you or live in your ears. Another major area of development are the so-called smart technologies in the home, the car and wearable tech that can sense and even react to the environment. Activity and fitness trackers are the most visible of the wearable technologies, although they are yet to demonstrate lasting traction in the market. More interesting developments are at the intersections of these technologies such as AV displays for virtual and augmented reality and more recently, so-called hearables – headphones with benefits.
Other than glasses, earbuds, and to some extent headphones, are the most widely adopted wearable technology. The current incarnation of hearables combines activity tracking technologies with audio playback and/or streaming using earbuds. Although in their infancy, both commercially and technically, there is a huge amount of hype around this new technology niche. In an industry that is struggling with commoditisation, reduced margins and market saturation in many key segments, hearables provide a low barrier opportunity to deliver a new technology as there is no need to change consumer behaviour. Although listeners will continue to consume their current media with current distribution and service models, such platforms will enable new use-cases, technology convergence and services. The big question, of course, is what forms these might take. For instance, MP3 players didn’t have anywhere near as much impact on the music industry as iTunes – that is, it wasn’t the iPod but the vertically integrated services that it enabled that capitalised on the disruption produced by the MP3 player that completely transformed a monolithic and very high value industry. At this point in the evolution of the hearable however, the idea of an enabling platform is a hope rather than a strategy.
For hearables, key questions include: (1) What can/could actually be achieved technically given the price point and regulatory and other frameworks (the platform)? (2) What needs or wants might these technologies actually satisfy (the product)? (3) Who would pay for this and how would they pay for it (the monetization)? (4) How would any new business or service models be sustained in the face of competition, technical commoditization and market saturation?
Here we will look at the first of these questions. There are but a handful of “hearable” products currently on the market with a moderate number in advanced development. Made for iPhone hearing aids might arguably be the first hearables (Starkey Halo and Resound Linx2/Beltone First) with a focus on hearing enhancement for the hearing impaired. Recently released (non-hearing aid) hearables include the Bragi Dash, Doppler Labs Here and iRiver On. There are a range of other devices laying claim to the title but they are either not stereo (so called personal sound amplification devices (PSAPS), see also below) or stereo ear buds that connect to other activity monitors. There are quite a number of hearables in development as kickstarter/indegogo or other start-up projects (e.g. Nuheara, Freewavz, Elbee, Zik Sport, SkyBuds, UnitedSciences’ Aware) and the larger OEMs are also in advanced development (Samsung – Earcle, Apple – AirPods, Sony – Experia Ear, Microsoft – Clip, Google - Aura) or already in the market (Moto – Hint). Although there is much secrecy around these latter offerings it appears that they are probably not going to include any type of biosensing and are aimed more at speech interaction with proprietary speech recognition and search systems (much like Amazon’s stand-alone Echo and their Alexa agent). Back in 2014, Intel working with Valencell, released a reference platform for smart ear buds that was demonstrated at CES that integrated Valencell’s PreformTek Precision Biometrics with basic DSP capability. This system relied on power harvesting from the mobile phone headphone jack rather than Bluetooth connectivity so now might not strictly fall into the hearable category which has a strong focus on untethered/wireless ear buds.
It is probably fair to say that, like all new technologies, the birth of the hearables has not been without its complications. Some commentators are arguing that the small number of current offerings fall somewhat short of the promise (and well short of the marketing hype). Nonetheless, these early offerings do represent the first entrants into a new product category and much can be learnt from what appears to be working and what doesn’t.
While there is necessarily some overlap with the existing headphone technologies (e.g. stereo, a small form factor, the capacity to stream 2-way audio via Bluetooth) they are also set apart by the integration of other sensor technologies or specialised signal processing. Sensors currently include movement sensors (so-called 9 degrees of freedom using accelerometer, gyroscope and magnetometer), pulse oximetry for heart rate and oxygen saturation and external microphones for audio recording and/or pass through. Simple activity tracking for running, swimming and cycling and reporting using generative speech is available from the Bragi Dash, however, this is not yet well integrated into a more developed logging and analysis smartphone, PC or on-line platform. This path seems to be the direction down which many of the other hearables still in development are also taking. The target demographic is mainly sports and fitness and closely following a marketing path forged by the wrist worn activity trackers since 2009 with the launch of FitBit.
By contrast the Here by Doppler Labs boasts various signal processing features (5-band EQ, pre-set filters, bass boost, reverb and flange) and a smart phone app that can change in real time the way you listen to the world around you. Hooke’s wireless 3D audio headphones takes a slightly different approach and provides the ability to record audio using small microphones on the ear buds to “curate” your listening world. The innovation here is that the microphones are placed within the outer ear to capture the pinna filtering of the listener (the HRTFs) so that when played back over the headphones, the recordings are rendered in virtual auditory space. This is still under development but the target demographic are hip younger media consumers and creators.
Less sophisticated enhancement technologies are found in many so-called personal sound amplification products (PSAPS: e.g. Soundhawk, SoundWorld Solutions CS50+, Bean by Etymotic Research: all around USD350 and NeutronicEar’s CrystalEar - a high end USD 545-1050, plus many other sub USD100 products). These products are aimed to assist normally hearing (i.e. non-hearing impaired) listeners in a range of listening environments (such as hunting (listening for prey), bird watching, lectures with a distant speaker or distant conversations) rather than to compensate for some form of hearing impairment. This is an important distinction as the FDA classifies hearing aids as a class I or class II medical device that are required to be dispensed by suitably trained professional resulting from a referral by a licensed physician. Marketing a PSAP as an aid for the hearing impaired would require that its distribution carries the same dispensing requirements of a hearing aid. Nonetheless, many PSAP manufacturers tread a fine line and pitch their products to those who struggle in noisy environments- an FDA definition that identifies the problem as hearing impairment! A slightly more nuanced marketing approach is taken by Nuheara. Their IQ buds are closer to the Doppler Labs’ Here in that they combine Bluetooth streaming and noise cancellation with signal processing technologies not unlike those in hearing aids that aim to reduce noise interference of speech. The pitch is definitely to younger listeners and integrates the ideas of noise cancellation for mobile communication with speech enhancement in busy and complex listening environments.
Others hearables, like the recently announced Aware from UnitedSciences (and to some extent the Bragi Dash) are being developed as a platform with a range of sensor and signal processing technologies for experimentation and development of different utilities and apps. In the case of the Aware, in addition to motion sensing and pulse oximeter, EEG sensors will be integrated into the ear pieces and proposed use cases include monitoring attentional focus, stress management, monitoring sleep patterns and levels of relaxation. The ear pieces are also designed so that, like the Hooke Wireless 3D Headphone, small microphones can pass through the ambient environmental sound filtered by the listener’s HRTF and therefore enable a very higher level of spatial and situational awareness – the sort of feature important to a jogger on busy streets! What is a standout with this potential platform (there is a prototype but a commercial product is still just a Kickstarter pitch) is the inclusion of the in-ear EEG electrodes, a technology pioneered at UCL (UK), (and also the Universities of Oldenberg (Germany) and Oxford e.g.). Such electrodes can not only record brain activity but be used to pick up the electrocardiogram (ECG) from the heart, activity from the muscles of the head and neck as well as the extraocular muscles of the eyes to monitor gaze direction. While still in its technological infancy, brain computer interfaces (BCI) based on EEG offer significant promise as a control system that is responsive to the brain state of the wearer (mood, intention, state of vigilance or awareness).
On the chip side, Samsung has recently announced an advanced biosensor under development that integrates pulse oximetry, ECG, galvanic skin response and skin temperature measurement with advanced signal processing capability all into a single, low power chip. This brings us up to the bleeding edge but there are still quite a range of other technologies that could also find their way into such devices. Biosensors detecting both organic and inorganic compounds analysed from sweat, analysis of lipid permeable molecules such as alcohol, ionic composition analysis of sweat, glucose monitoring and cancer diagnostics are all being very actively researched. Some are closer to commercial realisation than others but the question with most is “when” not “if”. The ear provides a secure and reliable place for monitoring a wide range of these sorts of bio-signals. Laboratory based biochip array technologies have been available for more than a decade for a huge range of compounds ranging from DNA analysis, specific protein and antibodies as well as complex chemical compounds (e.g. ). Although, in their current incarnations, these technologies are far from providing commercial platforms in the context of a hearable, the technology does exist to monitor an enormous range of compounds that would be interesting from both a physiological and health perspective as well as for monitoring the environments of the wearers.
In returning to where we started (to paraphrase T S Eliot), we must be clear that it’s not the technology that will drive transformation in the market – it will necessarily be the problems that it solves (the product) and the way in which this can be transformed into a sustainable business. Commercial history is littered with great ideas and fantastic technology that failed because the value proposition for the potential beneficiaries was poorly articulated or the route to market was fraught. For the hearable believer there is a mine field to cross to breathe life to this vision.