The HTM Spatial Pooler – a neocortical algorithm for online sparse distributed coding

Abstract

Each region in the cortex receives input through millions of axons from sensory organs and from other cortical regions. It remains a mystery how cortical neurons learn to form specific connections from this large number of unlabeled inputs in order to support further computations. Hierarchical temporal memory (HTM) provides a theoretical framework for understanding the computational principles in the neocortex. In this paper we describe an important component of HTM, the HTM spatial pooler that models how neurons learn feedforward connections. The spatial pooler converts arbitrary binary input patterns into sparse distributed representations (SDRs) using competitive Hebbian learning rules and homeostasis excitability control mechanisms. Through a series of simulations, we demonstrate the key computational properties of HTM spatial pooler, including preserving semantic similarity among inputs, fast adaptation to changing statistics of the inputs, improved noise robustness over learning, efficient use of all cells and flexibility in the event of cell death or loss of input afferents. To quantify these properties, we developed a set of metrics that can be directly measured from the spatial pooler outputs. These metrics can be used as complementary performance indicators for any sparse coding algorithm. We discuss the relationship with neuroscience and previous studies of sparse coding and competitive learning. The HTM spatial pooler represents a neurally inspired algorithm for learning SDRs from noisy data streams online.