Advanced Development of a Customer Assistance Chatbot using Multimodal Large Language Models to Enhance User Experience

The quality of online customer assistance plays an essential role in shaping the overall user experience across various industries. As customer bases grow, the volume of queries and usage issues also increases, necessitating ef-ficient and cost-effective solutions to manage these demands. The rise of Large Language Models (LLMs) has made automating chatbot solutions es-sential for delivering effective services, such as technical assistance, trouble-shooting, and personalised recommendations. With the abilities of Retrieval-Augmented Generation (RAG) architectures, applying LLMs to domain-specific tasks requiring specialised knowledge has become increasingly im-portant. In this paper, we experiment with different approaches to develop a multimodal chatbot capable of responding to user queries in various formats, including text, images, tables, and audio, across multiple languages. To ad-dress the limitations of LLMs in providing accurate answers related to private knowledge sectors, as well as the constraints of traditional RAG models in retrieving information from general knowledge on the web that is not includ-ed in the private knowledge base, to tackle this this study presents a multi-agent approach. The chatbot leverages private domain knowledge bases while maintaining the ability to access web-based resources for out-of-domain queries using a multi-agent paradigm. Our study evaluates the impact of different vector stores, embedding models, and chunking parameters on model performance. To refine our results, the evaluation was conducted in two stages, narrowing the focus to the optimal model parameters that achieved higher accuracy. Performance was assessed using RAGAS and hit image rate evaluation methods, ensuring robust measurement of retrieval ac-curacy. This study demonstrates the applicability of the proposed approach to diverse datasets across various domains, effectively addressing business-specific queries. To produce a smarter, and more capable customised AI chatbot assistant that enhances the overall user experience.