In this white paper, we described the implementation of a multiclass text classification model for classifying support services case logs into a predetermined set of issue categories. We discussed important design decisions behind deriving classification taxonomy, the methodology adopted for training-data curation, and model selection strategy. Key experiments carried out including different modeling techniques and feature representation techniques along with their results have also been presented. The deployment architecture of the model has been elaborated in detail. Model architecture includes the batch prediction architecture as well as the near real time prediction architecture with the details on different pipelines involved and model service architecture. It also includes the implementation of model feedback loop leveraging in-house developed prediction validation tool known as Elixir.
The Intelligent Case Classification (ICC) solution helps to understand the intent behind customer-initiated contacts and enables profiling of issues. ICC has already been proven to bring substantial business benefits. It has helped surface issue patterns related to repeat customer contacts and repeat dispatches of hardware components. It has been instrumental in identification of substantial repeat contacts reduction opportunities leading to large cost reductions. The results mentioned above are achieved by identifying technical issue patterns resulting from ineffective usage of tech support tools. Furthermore, ICC is also effective in identifying trending issue patterns. Standardization of issue descriptions has enabled interoperability between data analysis tools and reporting. Compared to a manual process, with the help of ICC, we have been able to scale the solution to all the regions for three languages with consistent results. ICC predictions are also being used as an input to other ML/AI systems.
Future work includes exploration of large, pretrained multilingual models for expansion to additional languages. Experimentation with multilingual feature representation techniques, such as multilingual embeddings, is already underway to streamline models. The aim is to train a single model that can accept case logs of a given set of languages as the input and predict the T1-T2-T3 issue category as the output. Furthermore, unsupervised methodologies are being explored to address the limitations imposed by static nature of labels.
Other work focuses on creating a fully automated model retraining pipeline with modularized components for data ingestion, data preprocessing, feature engineering, model training with hyperparameter tuning, model inference, and model monitoring.
