Voice AI technology has advanced with the introduction of models capable of speaking and listening simultaneously, but challenges involving response speed, transcription accuracy, and conversational understanding continue to limit its effectiveness. Executives from PolyAI and Otter say further improvements are needed before voice assistants can reliably handle complex conversations and automated tasks.
Speaking at the HumanX conference in September 2026, PolyAI chief technology officer Shawn Wen said voice AI has yet to experience a breakthrough comparable to ChatGPT’s impact on text-based AI. Although developers have introduced full-duplex models that can listen and respond at the same time, these systems still need faster reasoning capabilities to maintain natural conversations.
Wen explained that models must retrieve information and generate answers quickly enough to avoid interrupting the flow of dialogue. Improving response speed would allow voice assistants to interact with users without lengthy pauses or unnatural conversational patterns.
Voice AI Must Build Confidence During Customer Service Calls
PolyAI develops conversational AI systems for enterprise customer service, where automated agents handle incoming calls and respond to customer requests. Wen said successful interactions depend on more than producing human-like speech.
He explained that customers need to feel confident that an AI agent understands their problems and can provide appropriate solutions. If the assistant responds effectively during the first few exchanges, callers may become more comfortable continuing the conversation without requesting a human representative.
Alex Gay, chief marketing officer at Otter, identified similar challenges in workplace meetings. He said voice AI must accurately identify speakers, understand their intentions, and combine conversation details with organizational information before it can reliably automate follow-up tasks.
Otter is also developing digital twins designed to represent individuals during meetings. Gay said these systems must reproduce emotional expression and support discussions involving debate, collaboration, and professional relationships.
Without those capabilities, he argued, AI meeting representatives would function primarily as question-and-answer chatbots rather than participating meaningfully in discussions.
Transcription Accuracy Remains a Challenge for Voice AI
Both executives identified automatic speech recognition (ASR) as an area requiring further improvement. Wen said speech recognition systems sometimes miss important keywords, causing AI assistants to lose context or misunderstand user requests.
Gay added that inaccurate transcripts can affect every subsequent task performed by an AI assistant. Incorrectly recorded conversations may lead to flawed summaries, missed instructions, or automated actions that users never intended.
He said Otter continues to improve its speech recognition models, including their ability to process different languages. Transcription accuracy is particularly important as the company develops tools that can act on information gathered during meetings.
AI Companies Address Recording Transparency
Wen and Gay also discussed the importance of informing people when they are interacting with AI or having conversations recorded. Both executives identified transparency as an important factor in establishing trust with users.
Otter is considering ways to notify meeting participants when recording takes place, even if its AI meeting assistant is not visibly present. One option involves sending notifications through a meeting’s chat function.
Wen similarly said businesses using AI-powered customer service agents should clearly inform callers that they are speaking with an automated system.
Featured image credits: Magnific.com
For more stories like it, click the +Follow button at the top of this page to follow us.