VoIP Outperforms PSTN in Audio Quality
SAN MATEO, Calif.,-- New insights into the performance of leading VoIP providers was released by Keynote Systems, an Internet and mobile test and measurement services company. Keynote's third VoIP competitive intelligence study revealed that overall VoIP quality has improved across the board since Keynote's last study in December 2005 and that the leading VoIP providers have actually surpassed PSTN (traditional phone service) in overall audio quality, but still lag behind PSTN in audio delay.
Twelve leading VoIP providers are part of the benchmark study that includes AT&T, Comcast, Lingo, Packet8, Skype, SunRocket, TimeWarner Cable, TrueVoice, Verizon, Vonage, Vonics and Windows Live Messenger.
To benchmark and rank the quality of consumer VoIP services, Keynote measured the relative performance of the leading VoIP providers in the New York and San Francisco markets, including digital cable, adapter-based VoIP (hard phone) and PC-based software (soft phone) services, as well as the performance of leading VoIP providers against PSTN service in those cities. Keynote then rated the leading VoIP service providers on critical performance factors that influence the end-user experience using Keynote Voice Perspective, which is Keynote's VoIP quality test and measurement product.
Based on the results of the survey, which was conducted over a one month period from Aug. 1-Aug. 31, 2006, Keynote found that overall reliability among the various competing VoIP providers had improved across the board and that the leading digital cable providers had in fact outperformed PSTN in overall reliability. Overall reliability is a computed index score based on performance measurements in three performance factors: service availability, average number of dial attempts and dropped calls.
Leading digital cable VoIP providers were also found to deliver better audio quality than the competition, with the leading cable providers achieving excellent audio responsiveness (a measure of audio delay) and audio clarity (measured by Mean Opinion Score, or MOS), two key contributors to overall audio quality. However, there was still room for improvement among the rest of the pack, with 10 of 12 VoIP service providers studied achieving less than a 4.0 MOS, which is considered to be "toll quality," that is, comparable to the audio quality of a toll call over PSTN.
Despite the shortcomings of the lower-ranked service providers, the overall average MOS of the VoIP providers studied continues to improve over time, with the overall average MOS of 3.58 reaching levels comparable with GSM mobile phone quality. In Keynote's December 2005 study the overall average MOS among VoIP providers was 3.55.
The study also examined the relative performance variations of the various VoIP service providers (as well as against the PSTN benchmark) during peak versus non-peak hours in terms of audio delay and Mean Opinion Score. It had been thought that cable modem subscribers would suffer overall service degradation during peak hours (8:00 PM-1:00 AM EDT), however the study revealed that while cable modem subscribers did experience greater instances of audio delay during peak hours, audio clarity (as measured by Mean Opinion Score) was not affected by the increased traffic associated with peak hours. DSL connections, on the other hand, were found to deliver more consistent peak versus non-peak audio delay performance, but were less consistent as measured by MOS.
Although the top performers in the consumer VoIP services market have improved the quality of call audio, calls placed on VoIP phones continue to exhibit considerably more audio delay than calls placed on traditional PSTN phones. This audio delay can cause callers to talk over each other, leading to conversational disruption and missed information, which can create frustration among users, especially in a business setting.