The listening test is the most important part of the evaluation of the acoustic echo canceller. It is the only time to evaluate the performance of the state machine, which is the most important factor for audio quality. The listening test environment should be taken into consideration if different echo cancellers are not all tested in the same location.

A panel of several people should be chosen to evaluate the echo canceller. If possible, the same people should evaluate all of the echo cancellers under consideration during a short period of time. These people should listen for the common problems listed in Table 3, as well as for the overall audio quality.

The most important part of the evaluation is on the opposite end of the echo canceller (the far or remote end). This is where the echo would be heard in the first place and most of the echo canceller’s problems become evident. If the echo canceller is sold as part of a complete system (including microphones and speakers), some evaluation also should be done on the near end to ensure all of the audio components are of good quality.

On the opposite end of the echo canceller, either a handset or another echo canceller of the same type should be used. A listening test should not be performed with a half-duplex speakerphone or a different echo canceller on the other end. Otherwise, it would not be clear which end had problems.

Ideally, the operating environment for testing should be similar for all of the echo cancellers, since room acoustics have such a large impact. If this cannot be arranged, at least consider the operating environment differences in each case in the final decision. If possible, listen to the room acoustics with the echo canceller disabled so the effects of the different rooms can be compared.  

Seven Things to Listen For

• Residual Echo.

 If there is excessive residual echo, the sound may have a hollow, distant quality or there may even be distinct audible echoes. This is especially noticeable during the receive mode, when there is no near speech to mask the echo. If this is due to a short tail length, the residual echo may sound delayed.

• Loss of Convergence.

 When the echo canceller loses convergence, the result is an audible residual echo that could be louder than an echo with no echo canceller at all. This is generally caused when the state machine mistakes a double-talk situation for a receive state. If this happens, the echo canceller begins to adapt to the near talker’s speech as well as the echo, and goes out of convergence.

• Howling.

 Pitched squealing noises may occur when both parties have hands-free systems with open speakers and mics. This is caused by either a lack of howling rejection, or howling rejection that is not working properly.

• Attenuated Speech During Double-talk.

Noticeable changes in volume levels may occur during double-talk. This is caused when the state machine mistakes double-talk for a receive state, and applies switch loss (attenuates the near signal to reduce the residual echo level).

• Half-duplex Behavior.

This is basically an extreme case of attenuated speech during double-talk. If one of the sides is attenuated so much as to become inaudible, then it would be impossible for them to interrupt the other party.

• Clipped or Noisy Speech During Double-talk.

 Very harsh and annoying distortion can be added to the speech signal when nonlinear processing (usually center clipping) is applied during double-talk. The speech may be distorted beyond recognition. This occurs when the state machine reports a receive state during double-talk.

• Audible State Transitions. Audible changes in background noise level, clicks, or changes in overall volume levels may be noticeable during state transitions. This may even occur between words or short pauses in speech. This is caused by a state machine that switches between states too abruptly, or too often.