2.2 Disentanglement 15 With the chosen Gaussian distributions for the prior and approximate posterior, the KL divergence in Equation (2.2) can be computed analytically and becomes KL(qϕ(z|x)||p(z))= 1 2 KX k=1 (m2 k +s 2 k −logs 2 k −1), (2.14) where Kis the dimensionality of the latent space, µenc(x) = (m1, . . . ,mK) T, and σenc(x) = (s1, . . . ,sK) T. Gradients with respect to µenc and σenc can be computed exactly for this expression, so it can directly be used for gradient-based optimisation. 2.2 Disentanglement In Section 1.2.2, we motivated the need for learning disentangled representations (Bengio et al., 2012), where the idea is that data contains underlying generative factors that we wish to model in separate dimensions or subspaces of our latent representation. Liu et al. (2022) provide an overview of key concepts and methods in the field of disentanglement, in particular for applications in the imaging domain. Here, we briefly summarise a number of methods that have been proposed to learn disentangled representations, based on extending the VAE framework with various loss components to encourage disentanglement in the latent space. We also summarise some proposed metrics for quantifying the level of disentanglement in representations. Furthermore, we briefly discuss some limitations of current disentanglement approaches, including the lack of a formal agreed-upon definition for disentanglement. Unsupervised disentanglement methods The following unsupervised methods all extend the VAE loss function with some regulariser to encourage disentanglement. In particular, they assume that generative factors are one-dimensional, and should thus be modelled in single independent dimensions in the latent space. β−VAE (Higgins et al., 2017) adds a weight parameter β > 1 to the KL divergence term in the VAE loss, thereby constraining the capacity of the VAE bottleneck. This forces the posterior (encoder) distribution to better match the prior, which is typically a factorised unit Gaussian, thus this should lead to more
RkJQdWJsaXNoZXIy MjY0ODMw