68 Quantifying and Learning Linear Symmetry-Based Disentanglement (LSBD) space Ω⊆Z, e.g. Ω= [−10, 10] for finding the optimal ω ∈Ωthat minimises the dispersion. This leads to the following expression: D(k) LSBD =min ω∈Ω 1 |G| X (g1,..,gK)∈G ∥ρk,ω(g−1 k ) · z′(g1, ..,gK) −Mk,ω∥ 2. (4.23) EachD(k) LSBD measures the degree of equivariance of the projected encodings for the k-th subgroup corresponding to the best fitting group representation. The upper bound to the metric is finally obtained by averaging across all subgroups: DLSBD ≤ 1 K KX k=1 D(k) LSBD. (4.24) Our practical implementation of DLSBD is for SO(2) subgroups, however the procedure can in principle be extended to other subgroups as well. A practical implementation of the metric requires (i) identifying the subspaces invariant to a subgroup and (ii) identifying a parametric representation of the subgroup that can be fitted to the subspace data representations. In cases where the exact form of the subgroup is unknown, an option is to use the method by Pfau et al. (2020) to factorise the sub-manifolds associated with different generative factors. 4.5 LSBD-VAE: Learning LSBD Representations In this section we present LSBD-VAE, a semi-supervised VAE-based method to learn LSBD representations. The main idea is to train an unsupervised Variational Autoencoder (VAE) (Kingma and Welling, 2013; Rezende et al., 2014) with a suitable latent space topology, and use our DLSBD metric as an additional loss term for batches of transformation-labelled data. 4.5.1 Assumptions LSBD-VAE requires some knowledge about the group structure Gthat is to be disentangled. Concretely, we assume that the following elements are given: • The group Gand its decompositionG=G1 ×. . . ×GK. • A decomposable latent space Z =Z1 ⊕. . . ⊕ZK. • A suitable linearly disentangled group representationρ : G→GL(Z).
RkJQdWJsaXNoZXIy MjY0ODMw