1.2 Research Questions 5 Since X contains observations that are essentially generated from world descriptors in W, and we want Z to share desirable properties with W, a reasonable objective is that we should also be able to generate data observations fromZ. In that case, we know that Z contains a decent description of the data that could be much lower-dimensional thanX. This is an example of generative modelling (Tomczak, 2022), where the goal is to learn to describe the data Xitself by learning how to generate it, as opposed to discriminative modelling, where the goal is to learn how to assign labels inY to data X. In probabilistic terms, generative models learn a distributionp(X), or in the presence of labels a joint distributionp(X,Y). This allows a model to include information that is present inXbut not inY. Discriminative models on the other hand learn a conditional distributionp(Y|X), which is only described over Y. In particular, a generative model with a latent space Z is called a latent variablemodel. Here, p(X) is modelled indirectly through a latent space prior p(Z) and a conditional distributionp(X|Z) such that the joint distribution becomes p(X,Z) = p(Z)p(X|Z). Such a model can generate data in X by sampling from a simple prior distribution over Z and then from the learned conditional distributionp(X|Z). Using neural networks to model the parameters of these distributions, we can formulate a Variational Autoencoder (VAE) (Kingma and Welling, 2013; Rezende et al., 2014). In a VAE, the parameters of the conditional distributionp(X|Z) are modelled by a decoder network. Furthermore, the parameters of an approximate posterior distribution q(Z|X) are modelled by a encoder network, to perform inference of latent variables given input data. The prior p(Z) is often simple and fixed. As motivated before, ideally we want Z to share desirable properties withW, or in other words, we want the mechanisms of the real worldWto apply to Zas well. This should help us with building more reliable models that can do well on downstream tasks. Motivated by this, we present the following research topics and questions. 1.2 Research Questions 1.2.1 Anomaly Detection with Probabilistic Generative Models A key motivation for learning representations is that they are useful for downstream tasks. Probabilistic generative models, such as Variational Autoencoders
RkJQdWJsaXNoZXIy MjY0ODMw