Your Call May Be Recorded for Automatic Quality-Control

A call center quality control process typically relies on human labor to evaluate sample conversations according to a quality monitoring (QM) questionnaire. Due to the effort involved, the sample of calls evaluated is often very small and likely to miss problematic calls. This paper presents an automatic call quality monitoring (QM) system for contact centers, which applies natural language processing (NLP) and machine learning techniques. Specifically, the system aims at categorizing contact center calls into good calls which meet or exceed a company's quality expectation and bad calls which are below the expectation.

In this work, we first transcribe a call using an automatic speech recognition (ASR) system, and extract features from the call transcript using various text mining techniques. The features include timing features, lexical features and structural features that indicate various aspects of call quality. We then apply maximum entropy classification to decide if a question in a company's QM questionnaire is satisfied or not resulting in as many maximum entropy classifiers as the number of questions in the QM questionnaire. The system produces a score for each question depending on the classification result. All scores are then combined to generate a quality score for the call. If the total quality score is above a predetermined threshold, the call is regarded as a good call.

We have conducted experiments with 387 customer calls to an automotive company. The system was trained using 310 calls with associated manual monitoring results and tested on the remaining 77 calls. 70% calls in the training data were rated as good calls by human monitors. The experimental result shows 72.7% classi¯cation accuracy, which is very promising given the fact that the system was trained with a very small and highly biased data set.

By: Youngja Park

Published in: RC24574 in 2008


This Research Report is available. This report has been submitted for publication outside of IBM and will probably be copyrighted if accepted for publication. It has been issued as a Research Report for early dissemination of its contents. In view of the transfer of copyright to the outside publisher, its distribution outside of IBM prior to publication should be limited to peer communications and specific requests. After outside publication, requests should be filled only by reprints or legally obtained copies of the article (e.g., payment of royalties). I have read and understand this notice and am a member of the scientific community outside or inside of IBM seeking a single copy only.


Questions about this service can be mailed to .