6 Introduction (VAEs), learn latent representations to describe the probability distribution of observed data and allow for density estimation of unseen data points. This makes them well-suited for the task of probabilistic anomaly detection, or outlier detection (Pimentel et al., 2014). By training a generative model on what is considered normal data, the model should be able to detect outliers, i.e. anomalous or abnormal datapoints, as they should get assigned a lower probability density. Thus, we pose the following research question: Q: How can the density estimation of probabilistic generative models—in particular Variational Autoencoders (VAEs)—help with anomaly detection; i.e. detecting out-of-distribution datapoints? 1.2.2 Quantifying and Learning Linear Symmetry-Based Disentanglement (LSBD) As motivated above, we want representations in Z to capture the underlying mechanisms of the real worldW. Regular VAEs try to achieve this by learning how to generate data inXfromZ, but this still imposes little structure onZ. The real world can often be described in terms of independent factors of variation. E.g. images of animals can be described with factors such as fur colour, ear shape, and pose. Such factors are often calledgenerative factors, as knowing their values allows to generate data observations. Generative factors are implicitly present inW, and it would be beneficial to find representations inZ that model these factors as well. This is the goal of disentanglement, or learning disentangled representations (Bengio et al., 2012). The idea is that each factor should be represented in a separate subspace of Z, such that changes in one generative factor only lead to changes in one subspace. Several approaches have been proposed to learn and quantify disentangled representations (Locatello et al., 2018), we refer to these collectively as traditional disentanglement. However, it is difficult to turn these ideas into a formal definition that can be used to design disentanglement models. It is not obvious how to formally describe what a factor is. To address this, Higgins et al. (2018) propose the formal definition of Linear Symmetry-Based Disentanglement (LSBD), arguing that symmetries of the real worldWare what cause variability in the data. The real-world mechanisms that we want to disentangle in Z are then symmetry transformations that change one aspect of the real world but leave all others invariant. The mathematical language of group theory is used to capture this in a formal definition.
RkJQdWJsaXNoZXIy MjY0ODMw