Abstract
Our recently introduced Neocortex-Inspired Locally Recurrent Neural Network is a machine learning system that is able to learn feature extraction functions from sequential data in an unsupervised way. While it was designed with the main purpose of feature learning, its structure and desired functioning is highly inspired by models of the feedforward circuits in the neocortex. In this work, we study the behavior of our system when it takes shifting images as input, and we compare it with known behavior of the primary visual cortex. The results show that some of the best-known emerging properties in the primary visual cortex, such as the emergence of simple and complex cells as well as orientation maps, also occur in our system, indicating that also their behaviors can be considered analogous. This validates our system as a potential model of the primary visual cortex that may contribute to further understanding of its functioning. In addition, considering that most areas in the neocortex show similarities in terms of structure and operation, future studies of our system over inputs other than images may also bring new insights about other neocortical areas.
Introduction: Unsupervised feature learning refers to the problem of learning useful feature extraction functions from unlabeled data. Despite the great success of deep learning networks in this task in recent years, both for static and for sequential data, these systems can in general still not compete with the high performance of our brain at learning to extract useful representations from its sensory input.
Methods: We propose the Neocortex-Inspired Locally Recurrent Neural Network: a new neural network for unsupervised feature learning in sequential data that brings ideas from the structure and function of the neocortex to the well-established fields of machine learning and neural networks. By mimicking connection patterns in the feedforward circuits of the neocortex, our system tries to generalize some of the ideas behind the success of convolutional neural networks to types of data other than images.
Results: To evaluate the performance of our system at extracting useful features, we have trained different classifiers using those and other learnt features as input and we have compared the obtained accuracies. Our system has shown to outperform other shallow feature learning systems in this task, both in terms of the accuracies achieved and in terms of how fast the classification task is learnt.
Conclusion: The results obtained confirm our system as a state-of-the-art shallow feature learning system for sequential data, and suggest that extending it to or integrating it into deep architectures may lead to new successful networks that are competent at dealing with complex sequential tasks.