Accuracy of commercially available cloud based speech to text is dramatically affected by dysarthria severity

Our paper on Automatic speech recognition in neurodegenerative disease was recently published in International Journal of Speech Technology.

Automatic speech recognition (ASR) could potentially improve communication by providing transcriptions of speech in real time. ASR is particularly useful for people with progressive disorders that lead to reduced speech intelligibility or difficulties performing motor tasks. ASR services are usually trained on healthy speech and may not be optimized for impaired speech, creating a barrier for accessing augmented assistance devices. We tested the performance of three state-of-the-art ASR platforms on two groups of people with neurodegenerative disease and healthy controls. We further examined individual differences that may explain errors in ASR services within groups, such as age and sex. Speakers were recorded while reading a standard text. Speech was elicited from individuals with multiple sclerosis, Friedreich’s ataxia, and healthy controls. Recordings were manually transcribed and compared to ASR transcriptions using Amazon Web Services, Google Cloud, and IBM Watson. Accuracy was measured as the proportion of words that were correctly classified. ASR accuracy was higher for controls than clinical groups, and higher for multiple sclerosis compared to Friedreich’s ataxia for all ASR services. Amazon Web Services and Google Cloud yielded higher accuracy than IBM Watson. ASR accuracy decreased with increased disease duration. Age and sex did not significantly affect ASR accuracy. ASR faces challenges for people with neuromuscular disorders. Until improvements are made in recognizing less intelligible speech, the true value of ASR for people requiring augmented assistance devices and alternative communication remains unrealized. We suggest potential methods to improve ASR for those with impaired speech.

Click here for details

 

Related Post

  • Posted on 17 January, 2024
    Smart devices are widely available and capable of quickly recording and uploading speech segments for health-related analysis. The switch from...
    • Posted on 25 June, 2021
      Machine learning approaches are increasingly used in health research. Applications range from the identification of disease onset, classification of disease...
      • Posted on 14 December, 2020
        Purpose To identify outcome measurement tools used to evaluate communication, voice and speech intelligibility in the mechanically ventilated ICU population....