Home > AI Solutions > Artificial Intelligence > White Papers > Rethinking Hierarchical Text Classification: Insights from Multi-Agent Experiments with Small Language Models > Business challenge
In customer support, particularly in the technical services division of a large company such as Dell Technologies, managing and analyzing large amounts of customer interaction data is a major challenge. Dell Support alone generates millions of text records weekly through various channels. This information, which is logged in textual format through complex Customer Relationship Management (CRM) systems, forms the backbone of Customer Support operations. The sheer volume of this data, encompassing customer-originated call logs, voice transcripts, chat logs, email messages, and telemetry data related to technical issues, presents a significant challenge. Manually analyzing these case notes to extract insights such as frequently occurring issues is not just tedious and time-consuming, but also prone to inconsistencies. This challenge has driven the development of diverse Artificial Intelligence (AI) solutions aimed at optimizing the Customer Support process.
These AI solutions range from predicting resolutions for support agents based on symptoms such as Next Best Action within Dell, to powering chatbots for customers and automating email composition. The same data is leveraged for AI-aided root cause analysis and part dispatch optimization. At the core of these applications is a fundamental Natural Language Understanding (NLU) task of precisely understanding and representing customer issues.
To address this challenge, the Dell Services Operations Applied Science and Engineering (SOAS) team focused on classifying technical support case notes into a three-tier label taxonomy. The taxonomy includes the following high-level categories:
The goal is to select the most appropriate and detailed label hierarchy, preferably at Tier 2 or Tier 3, that accurately describes the symptoms extracted from the support case note.
It is impossible to overstate the importance of a standardized, hierarchical taxonomy for issue description. Such a taxonomy enables different AI solutions to interoperate seamlessly, avoiding contradictory understandings of technical issues. For instance, the taxonomy can prevent scenarios where one system identifies the core problem as a “power issue” while another identifies it as a “boot issue.” This standardization improves the consistency across AI solutions that are built on the same dataset and enhances confidence in system identification when multiple AI solutions converge on the same symptom. Consider a scenario where an AI tool predicts possible resolutions to a technical issue based on a chat conversation. The tool might first identify the core symptom as [boot]-[no boot]-[message: inaccessible boot device]. If another AI solution that leverages telemetry data identifies the same problem, confidence in the system’s identification ability increases significantly. Further, if the AI analysis of the customer's online journey before they contact Dell Support confirms the same symptom, the analysis provides a strong indication of what the issue is, allowing for faster and more precise resolution or diagnostic processes.
This hierarchical approach can potentially improve the performance of existing systems. In this hierarchy, issue-resolution mappings can be stored in databases containing issue descriptions, allowing for faster candidate resolution identification. Data can be sharded based on the issue hierarchy for optimal lookup. For example, if a customer reports a boot issue, the system can narrow its search to the [boot] database, significantly speeding up the resolution process.
A hierarchy enables better diagnosis. If a customer mentions only that the "computer is not booting," it is possible to initially classify the core issue into an overall level such as [boot], but not enough information is available at this stage for accurate diagnosis. However, looking at the rest of the hierarchy components (level 2 - [boot]-[no boot], [boot]-[slow boot]) enables us to ask the customer whether they have a no boot issue or a slow boot issue. This facilitates navigation to the next hierarchy level to better identify the technical issue and provide the best resolution.
The team’s previous work in this domain, the Intelligent Case Classification (ICC) system, demonstrated the potential of Natural Language Processing (NLP) and Machine Learning (ML) in automating the classification of case logs into a hierarchical issue taxonomy. ICC facilitated the identification and analysis of the intents and root causes behind customer contacts, enabling issue profiling and the implementation of issue prevention strategies. Building on these foundations, the team’s current research explores a new approach involving leveraging small language models (SLM) with less than 10 billion parameters in multi-agent frameworks. These frameworks assign specific roles to the models and orchestrate them within structured workflows known as agentic workflows to collaboratively tackle complex tasks. The key research question is this: Can agentic frameworks and multi-agent systems using SLMs with 10 billion parameters or fewer effectively solve complex hierarchical text classification issues in technical support contexts?
A key innovation in the new approach is the ability of the multi-agent system to propose, create, or introduce new labels when existing labels are insufficient. This feature ensures that the classification system remains dynamic and adaptive to emerging issues, while maintaining consistency with the existing taxonomy. The work significantly extends previous approaches by addressing the limitations of ICC, which relied on separate multi-class text classification models for each hierarchy level. While functional, these earlier models had difficulty with multi-label scenarios and adjusting classification granularity based on available information.