Your Browser is Out of Date

Nytro.ai uses technology that works best in other browsers.
For a full experience use one of the browsers below
Intelligent Case Classification
Introduction Methodology
Methodology
Data set curation
Standardized issue taxonomy
Supervised raw training dataset
Exploratory data analysis
Model experimentation
Architecture Conclusions and Future Work

Supervised raw training dataset

Supervised raw training dataset

  • Having defined issue taxonomy and a comprehensive set of labels, the next step builds an ML training dataset. 

    As mentioned in the Introduction, Dell supports agents use CRM to record customer issues, queries, requests, and other important diagnostic/troubleshooting notes, which are known as “Case Logs.” Case logs are free text entered by agents that contain information about customer-facing issues, diagnostic steps, and resolution recommendations. Therefore, we decided to use these logs for training our ICC model. The logs are saved in a database along with other key information related to products and troubleshooting activities. 

    To build the training dataset, two approaches were used:

    • The first approach is a CRM system was augmented to include a new user interface (UI) field for technical issue classification (alongside the case log field). Next, this feature was rolled out to a selected set of support agents, where they were asked to annotate each case log with appropriate standardized label as a part of their support activity workflow. Label search functionality was implemented to facilitate selection of an existing label; however, support agents were also allowed to enter their own label. 
    • The other approach involved the in-house development of a dedicated data annotation tool with active learning capability known as Elixir. The latter was deployed internally and used by 10 to 15 subject matter experts, who annotated a substantial number of (>40k) textual case logs within a brief period. The functionality and implementation of the tool is described in Elixir – A Novel Data Labelling and Prediction Validation Tool.

    The results are multiclass datasets for consumer issues with approximately 24 K records in English, Mandarin, and Portuguese collected during 2019 to 2020. Each record primarily consists of two textual fields, which make the source: log title, log description and the target into one or more standardized T1-T2-T3 labels. In addition, a primary key was also stored for traceability – linking to each diagnostic/communication activity.