Publication
A Chatbot for Tourism in Porto: Building a Chatbot to support daily Tourists’ activities in Porto
| Summary: | Porto’s tourism has doubled in number of visitors from 2009 to 2019 and keeps increasing, making the development of data-efficient solutions an advantage to manage these people’s inflows. As a result, the present study focuses on creating a tailored chatbot that supports Porto tourists’ daily activities in order to ease their engagement with the city. To reach this, the present research performs several experiments to achieve a high-quality Retrieval Augmented Generation chatbot configuration, focusing on chunk-size, retriever method, TopK results, re-ranking and generative model. The chatbot was developed with Langchain and the final architecture is composed by a Hybrid Search Retriever with re-ranking and Llama3.1- 8B as generative model. The information retrieval component comprises a Dense Vector and a BM25 retriever, along with gte-large-en-v1.5 embedding model, and Top-K equals to 50. On top of it, query re-ranking is applied throughout ms-marco-MiniLM-L-6-v2 cross-encoder which retrieves the Top-8 results out of the 50. This solution is backed with 7.382 documents sourced from VisitPorto, TimeOut, Civitalis, Booking, Tripadvisor, Visit-Portugal, and Wikipedia that were chunked to a maximum of 3.000 characters per document. The results show that the retriever achieves a Hit Rate of 89% and by incorporating it with Llama3.1-8B, the RAG model achieves a Bertscore and Semantic Answer Similarity of 0.82 and 0.64, respectively, when testing its generation quality. Lastly, the present study also demonstrates that RAG models always outperform baseline models. |
|---|---|
| Main Authors: | Dona, Ricardo Francisco Montenegro |
| Subject: | Chatbot RAG Tourism Porto Llama Langchain Hybrid Search Retriever SDG 8 - Decent work and economic growth |
| Year: | 2025 |
| Country: | Portugal |
| Document type: | master thesis |
| Access type: | open access |
| Associated institution: | Universidade Nova de Lisboa |
| Language: | English |
| Origin: | Repositório Institucional da UNL |
| Summary: | Porto’s tourism has doubled in number of visitors from 2009 to 2019 and keeps increasing, making the development of data-efficient solutions an advantage to manage these people’s inflows. As a result, the present study focuses on creating a tailored chatbot that supports Porto tourists’ daily activities in order to ease their engagement with the city. To reach this, the present research performs several experiments to achieve a high-quality Retrieval Augmented Generation chatbot configuration, focusing on chunk-size, retriever method, TopK results, re-ranking and generative model. The chatbot was developed with Langchain and the final architecture is composed by a Hybrid Search Retriever with re-ranking and Llama3.1- 8B as generative model. The information retrieval component comprises a Dense Vector and a BM25 retriever, along with gte-large-en-v1.5 embedding model, and Top-K equals to 50. On top of it, query re-ranking is applied throughout ms-marco-MiniLM-L-6-v2 cross-encoder which retrieves the Top-8 results out of the 50. This solution is backed with 7.382 documents sourced from VisitPorto, TimeOut, Civitalis, Booking, Tripadvisor, Visit-Portugal, and Wikipedia that were chunked to a maximum of 3.000 characters per document. The results show that the retriever achieves a Hit Rate of 89% and by incorporating it with Llama3.1-8B, the RAG model achieves a Bertscore and Semantic Answer Similarity of 0.82 and 0.64, respectively, when testing its generation quality. Lastly, the present study also demonstrates that RAG models always outperform baseline models. |
|---|
Funded activities
No funded projects found.