4.5 LSBD-VAE: Learning LSBD Representations 69 • An embedded sub-manifold ZG ⊆ Z such that the action of G on Z restricted to ZG is regular, andZG is invariant under the action. Only ZG will then be used as the co-domain for the encoding function, h: X→ZG. We demonstrate fulfilling the assumptions above for the common case of disentangling two cyclic factors: • The group structure is G=SO(2) ×SO(2). • For the latent space we use Z =R2 ⊕R2. • For the group representation ρ =ρ1 ⊕ρ2 we use rotation matrices in R2 for ρ1 andρ2. • We can then use 1-spheres S1 ={z ∈ R2 : ∥z∥ = 1} for the embedded sub-manifold: ZG =S1 ×S1. In this case, the action of GonZ restricted to ZG is indeedregular, andZG is invariant under the action. Requiring the group structureGto be known is a relatively strong assumption, which limits the practical applicability of our method. However, a group structure can often be given as expert knowledge, like the presence of cyclic factors such as rotation, or in situations where transformations between observed data can easily be acquired such as in reinforcement learning. Moreover, explicitly stating these assumptions provides a clear pathway for future work, by attempting to relax each of these assumptions individually. 4.5.2 Unsupervised Learning on a Latent Manifold with∆VAE To learn encodings only on the latent manifoldZG, we use a Diffusion Variational Autoencoder (∆VAE) (Pérez Rey et al., 2019). ∆VAEs can use any closed Riemannian manifold embedded in a Euclidean space as a latent space (or latent manifold), provided that a certainprojection function from the Euclidean embedding space into the latent manifold is known and the scalar curvature of the manifold is available. The ∆VAE uses a parametric family of posterior approximates obtained from a diffusion process (i.e. Brownian motion) over the latent manifold. To estimate the intractable terms of the negative ELBO, the reparameterization trick is implemented via a random walk. The prior distribution is uniform over the Riemannian manifold, providing the additional benefit of an uninformative prior (Davidson et al., 2018). In the case of S1 as a latent (sub-)manifold, we consider R2 as the Euclidean embedding space, and the projection function7 Π : R2 →S1 normalises points 7This projection function is not defined for z =0, but this value does not occur in practice.
RkJQdWJsaXNoZXIy MjY0ODMw