Evaluation of The Effectiveness of Artificial Intelligence Chatbot in Improving User Experience in Academic Libraries

Abstract

Academic libraries are increasingly adopting artificial intelligence (AI) technologies to enhance user experience
and streamline access to information. This study evaluates the effectiveness of an AI-powered chatbot prototype
in improving user interaction with academic library services. A mixed-methods approach was employed,
integrating quantitative data from structured Likert-scale questionnaires with qualitative insights from semi-
structured interviews. The study involved 120 respondents, including students, faculty, librarians, and ICT staff.
Quantitative analysis using SPSS revealed high user satisfaction, with strong ratings for chatbot accuracy, user-
friendliness, response time, and ease of access to academic materials. Regression analysis indicated that available
resources and existing technological infrastructure were significant, though modest, predictors of chatbot
effectiveness (R² = 0.082, p < 0.05). Qualitative findings highlighted the chatbot’s role in providing faster
responses, improving resource accessibility, and enhancing overall user experience, particularly for open and
distance e-learning (ODEL) students. However, participants suggested enhancements in personalization and
advanced natural language processing to deliver more tailored interactions. Overall, the findings demonstrate
the potential of AI chatbots to significantly improve academic library services and underscore the need for
continuous refinement and scalability to meet evolving user needs.