Fuzziness Reduction and Training Set Generation for Text Classification in IT Services Domain

Extracting domain-specific concepts from a large set of documents containing unstructured text to enable concept-based search has become a critical step for fulfilling the core information management requirements in enterprises today. Given the demanding computational requirements of performing text analysis for concept extraction, and the small fraction of documents that typically have useful concepts worth extracting (less than 1% in our system), a key challenge to be addressed is how to perform an intelligent selection of documents for further detailed analysis without compromising recall.

In this paper, we present a methodology to semiautomatically generate training sets for a classifier and an algorithm to reduce the ambiguity inherent in our data, for using a Support Vector Machine (SVM) classifier to identify candidate documents for concept extraction in the IT Services domain. In such proprietary domains, due to privacy and security concerns, very few training datasets are publicly available to train a classifier. Furthermore, documents in the domain are normally noisy and fuzzy (contain terms not discriminating across categories). Experiments show that our method has improved the SVM performance, achieving an accuracy of 92%, a recall of 99.2% and an F-measure of 92.6%.