Abstract:
In this article we present a new modelling framework for structured concepts using a category-theoretic generalisation of conceptual spaces, and show how the conceptual representations can be learned automatically from data, using two very different instantiations: one classical and one quantum. A contribution of the work is a thorough category-theoretic formalisation of our framework. We claim that the use of category theory, and in particular the use of string diagrams to describe quantum processes, helps elucidate some of the most important features of our approach. We build upon Gardenfors’ classical framework of conceptual spaces, in which cognition is modelled geometrically through the use of convex spaces, which in turn factorise in terms of simpler spaces called domains. We show how concepts from the domains of shape, colour, size and position can be learned from images of simple shapes, where concepts are represented as Gaussians in the classical implementation, and quantum effects in the quantum one. In the classical case we develop a new model which is inspired by the Beta-VAE model of concepts, but is designed to be more closely connected with language, so that the names of concepts form part of the graphical model. In the quantum case, concepts are learned by a hybrid classical-quantum network trained to perform concept classification, where the classical image processing is carried out by a convolutional neural network and the quantum representations are produced by a parameterised quantum circuit. Finally, we consider the question of whether our quantum models of concepts can be considered conceptual spaces in the Gardenfors sense.
Из вступления:
In this article we present a new modelling framework for concepts based on the mathematical formalism used in quantum theory, and demonstrate how the conceptual representations can be learned automatically from data, using both classical and quantum-inspired models. A contribution of the work is a thorough category-theoretic formalisation of our framework, following Bolt et al. (2019) and Tull (2021). Formalisation of conceptual models is not new (Ganter & Obiedkov, 2016), but we claim that the use of category theory (Fong, 2019), and in particular the use of string diagrams to describe quantum processes (Coecke & Kissinger, 2017), helps elucidate some of the most important features of our approach to concept modelling. This aspect of our work also fits with the recent push to introduce category theory into machine learning and AI more broadly. The motivation is to make deep learning less ad-hoc and less driven by heuristics, by viewing deep learning models through the compositional lens of category theory (Shiebler et al., 2021).
Murphy (2002, p.1) describes concepts as “the glue that holds our mental world together”. But how should concepts be modelled and represented mathematically? There are many modelling frameworks in the literature, including the classical theory (Margolis & Laurence, 2022), the prototype theory (Rosch, 1973), and the theory theory (Gopnik & Meltzoff, 1997). Here we build upon G¨ardenfors’ framework of conceptual spaces (G¨ardenfors, 2004, 2014), in which cognition is modelled geometrically through the use of convex spaces, which in turn factorise in terms of simpler spaces called domains.
Our category-theoretic formalisation of conceptual spaces allows flexibility in how the framework is instantiated and then implemented, with the particular instantiation determined by the choice of category. First we show how the framework can be instantiated and implemented classically, by using the formalisation of “fuzzy” conceptual spaces from Tull (2021), and developing a probabilistic model based on Variational Autoencoders (VAEs) (Rezende et al., 2014; Kingma & Welling, 2014). Having “fuzzy” probabilistic representations not only extends G¨ardenfors’ framework in a useful way, it also provides a natural mechanism for dealing with the vagueness inherent in the human conceptual system, and allows us to draw on the toolkit from machine learning to provide effective learning mechanisms. Our new model—which we call the Conceptual VAE—is an extension of the β-VAE from Higgins et al. (2017), with the concepts having explicit labels and represented as multivariate Gaussians in a factored conceptual space.