Publicação
Topological Expressiveness of Neural Networks: Topology of Learning
| Resumo: | Given a neural network, how many di erent problems can it solve? This important and open question in deep learning is usually referred to as the problem of the expressive power of a neural network. Previous research has tackled this issue through statistical and geometrical methods. This work proposes a new method based on a topological perspective. Topology is the eld of mathematics aimed at describing spaces and functions through robust characterizing features. Topological Data Analysis is the young eld developed to extract topological insight from data. We rst show that topological features of the decision boundary describe the closest notion of the intrinsic complexity of a classi cation problem. These topological features divide classi cation problems into several equivalence classes. Linear-separability is an example of such a class. We establish the topological expressive power of a network architecture as the number of di erent topological classes it is able to express. Being a novel work in a young research eld, most of the thesis is devoted to developing this perspective and creating the tools required. The main objective of this thesis is to tackle neural network’s understanding in general and architecture selection in particular, through a novel approach. Our results show that topological expressiveness has a complex correlation with many features in a neural network’s architecture depending weakly on the total number of parameters. Some of our results recapitulate previous research on geometrical properties, while others are unique to this novel topological point of view, sometimes challenging previous research. |
|---|---|
| Autores principais: | Leitão, António Maria Lage de Sousa |
| Assunto: | Topological Data Analysis Neural Networks Machine Learning Decision Boundary Persistent Homology Voronoi Diagram Análise Topológica de Dados Redes Neuronais Aprendizagem Autónoma |
| Ano: | 2021 |
| País: | Portugal |
| Tipo de documento: | dissertação de mestrado |
| Tipo de acesso: | acesso aberto |
| Instituição associada: | Universidade Nova de Lisboa |
| Idioma: | inglês |
| Origem: | Repositório Institucional da UNL |
| Resumo: | Given a neural network, how many di erent problems can it solve? This important and open question in deep learning is usually referred to as the problem of the expressive power of a neural network. Previous research has tackled this issue through statistical and geometrical methods. This work proposes a new method based on a topological perspective. Topology is the eld of mathematics aimed at describing spaces and functions through robust characterizing features. Topological Data Analysis is the young eld developed to extract topological insight from data. We rst show that topological features of the decision boundary describe the closest notion of the intrinsic complexity of a classi cation problem. These topological features divide classi cation problems into several equivalence classes. Linear-separability is an example of such a class. We establish the topological expressive power of a network architecture as the number of di erent topological classes it is able to express. Being a novel work in a young research eld, most of the thesis is devoted to developing this perspective and creating the tools required. The main objective of this thesis is to tackle neural network’s understanding in general and architecture selection in particular, through a novel approach. Our results show that topological expressiveness has a complex correlation with many features in a neural network’s architecture depending weakly on the total number of parameters. Some of our results recapitulate previous research on geometrical properties, while others are unique to this novel topological point of view, sometimes challenging previous research. |
|---|
Atividades financiadas
Não foram encontrados projetos financiados.