Home > AI Solutions > Artificial Intelligence > White Papers > Rethinking Hierarchical Text Classification: Insights from Multi-Agent Experiments with Small Language Models > Conclusions and future work
This paper explored the application of multi-agent systems and SLMs in addressing the complex challenge of hierarchical text classification in technical support contexts. Through a series of experiments and developing prototypes, the Dell SOAS team investigated various agentic workflows, seeking to overcome limitations of traditional classification methods and explore new possibilities for adaptive and intelligent case classification systems. The team’s research has revealed that, despite the potential of these technologies, the outcomes were not entirely positive. Significant challenges arose related to latency and consistency issues in complex business use cases. The sub-optimal results observed in the experiments are attributable to multiple factors, primarily the lack of domain and business knowledge in SLMs and their limited reasoning and decision-making capabilities when handling complex classification tasks. Nonetheless, this study has yielded valuable insights into the practical application of multi-agent systems and SLMs in real-world support scenarios. Based on detailed experimental workflows, this paper proposes a multi-step agent-based architecture to tackle this complex classification challenge, providing a foundation for future research and development in this area.
Building on the foundations laid by this research, future work should focus on several key areas. Primarily, efforts should be directed towards implementing and evaluating the proposed architecture, with particular emphasis on refining the new label creation process, including the development of robust human-in-the-loop validation mechanisms. Concurrently, research should explore the representativeness of the current taxonomy, assessing the sufficiency of the three-tier hierarchy and investigating possibilities for dynamic extension while maximizing existing taxonomy reuse.
To address the observed limitations, future work should also explore fine-tuning SLMs with domain-specific data and experimenting with cutting-edge models and hardware acceleration techniques such as on-the-chip transformers to improve performance and speed. As the field evolves, incorporating advanced agent prompting and design patterns could further enhance the effectiveness of multi-agent systems in this context. Also, evaluating the multilingual performance of these models in agentic workflows will be crucial for global technical support operations. An important avenue for future research is exploring the generalizability of our approach beyond the current use case, investigating its applicability to other hierarchical text classification challenges across various domains. This exploration could uncover new insights and potential adaptations that might be necessary for broader implementation. Finally, future research should focus on scaling the proposed solutions and assessing their performance in real-world environments, considering factors such as response time, accuracy, and adaptability to evolving issues. These efforts will collectively contribute to advancing the application of multi-agent systems in complex NLP tasks beyond technical support classification.