4 Introduction discriminative models, since Y carries less information thanW. Therefore, it is desirable to have a modelling paradigm that considers more aspects of the real worldW. Figure 1.3: Directly predicting labels in Y fromdata Xessentially sidesteps the world descriptors W, yet fromWit’s typically trivial to predict Y. A common approach to consider these real-world aspects is to model an additional representation space Z (Bengio et al., 2012), in such a way that Z shares desirable properties with W, as illustrated in Figure 1.4. This space Z is typically much lower-dimensional than the data space X, and can ultimately function as a step in betweenXand Y, i.e. it should be easier and more reliable to predict targets inY fromZ than directly fromX. But Z can already be useful on its own, e.g. for other still unknown tasks, often referred to as downstream tasks. Values in Z represent unobserved variables, also called latent variables, so we often refer to Z as the latent space. A key question is which properties from Wwe want to model inZ, and how to do so. Figure 1.4: Latent variable modelling: the lower-dimensional latent space Z describes high-dimensional data in X. Ideally, Z should be similar to W, i.e. share similar properties or factors. Given some targets inY, it should then be much easier and reliable to predict Y fromZ. But simply modelling Zcan already be useful on its own, to provide insight into what the data describes.
RkJQdWJsaXNoZXIy MjY0ODMw