Publicação
Integration of large language models in the lifelog application MEMORIA
| Resumo: | Lifelogging is the practice of recording one’s life events with the help of various different tools and has been gaining popularity in recent years. This practice generates a large amount of data, or lifelogs, which can be used to extract information about the lifelogger’s life. To organize and make sense of this data, lifelog systems were created that aim to extract textual information from lifelogs, which allows retrieval of such information. Large Language Models are language models that have been trained on large amounts of data and have gained imense popularity in the last years due to their ability to generate human-like text and reasoning abilities. This work presents the integration of Large Language Models in MEMORIA, a lifelog system, presenting different ways to use these models to improve the quality of the extracted information and user experience. To do this, four main functionalities were developed, the generation of event descriptions, the generation of day summaries, a chat system and user query processing. Following the integration, a user study was conducted to evaluate the impact of the integration on the user experience and the quality of the extracted information, which revealed that both metrics improved with the integration of Large Language Models. |
|---|---|
| Autores principais: | Kaluza, Bernardo Francis Martins |
| Assunto: | Image annotation Image retrieval Large language models Lifelogging Machine learning Retrieval augmented generation |
| Ano: | 2024 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade de Aveiro |
| Idioma: | inglês |
| Origem: | RIA - Repositório Institucional da Universidade de Aveiro |
| Resumo: | Lifelogging is the practice of recording one’s life events with the help of various different tools and has been gaining popularity in recent years. This practice generates a large amount of data, or lifelogs, which can be used to extract information about the lifelogger’s life. To organize and make sense of this data, lifelog systems were created that aim to extract textual information from lifelogs, which allows retrieval of such information. Large Language Models are language models that have been trained on large amounts of data and have gained imense popularity in the last years due to their ability to generate human-like text and reasoning abilities. This work presents the integration of Large Language Models in MEMORIA, a lifelog system, presenting different ways to use these models to improve the quality of the extracted information and user experience. To do this, four main functionalities were developed, the generation of event descriptions, the generation of day summaries, a chat system and user query processing. Following the integration, a user study was conducted to evaluate the impact of the integration on the user experience and the quality of the extracted information, which revealed that both metrics improved with the integration of Large Language Models. |
|---|
Atividades financiadas
Não foram encontrados projetos financiados.