80 Quantifying and Learning Linear Symmetry-Based Disentanglement (LSBD) 4.7 Results: Evaluating LSBD withDLSBD We first highlight three key observations from our experimental results. In particular, we differentiate between the traditional disentanglement methods (VAE, β-VAE, CC-VAE, FACTOR, DIP-I, DIP-II) and metrics (BETA, FACTOR, SAP, DCI, MIG, MOD); and the methods (∆VAE, QUESSARD, LSBD-VAE) and metric (DLSBD) that focus specifically on LSBD. We then provide the full quantitative results (Section 4.7.4), as well as some further qualitative results (Section 4.7.5). VAE -VAE cc-VAE FactorVAE DIP-VAE-I DIP-VAE-II AdaGVAE AdaMLVAE Quessard LSBD-VAE/0 LSBD-VAE/256 LSBD-VAE/512 LSBD-VAE/768 LSBD-VAE/1024 LSBD-VAE/1280 LSBD-VAE/1536 LSBD-VAE/1792 LSBD-VAE/full LSBD-VAE/paths 0.0 0.25 0.5 0.75 1.0 1.25 1.5 1.75 2.0 DLSBD dataset Arrow Airplane Square (a) Datasets withSO(2) ×SO(2) symmetries. VAE -VAE cc-VAE FactorVAE DIP-VAE-I DIP-VAE-II AdaGVAE AdaMLVAE Quessard LSBD-VAE/0 LSBD-VAE/full 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 DLSBD dataset COIL-100 ModelNet40 (b) Datasets with SO(2) and nonsymmetric variation. Figure 4.6: DLSBD scores for all methods on all datasets. 4.7.1 Standard Disentanglement Methods Don’t Learn LSBD Representations Figure 4.6 summarises the DLSBD scores (lower is better) for all methods on all datasets. Bars show the mean scores over 10 runs for each method, the vertical lines represent standard deviations. LSBD-VAE/Lindicates our method trainedonLlabelled pairs (LSBD-VAE/0 corresponds to the unsupervised∆VAE), LSBD-VAE/full indicates our method where all images are involved in exactly one labelled pair. and LSBD-VAE/paths indicates our method trained with paths of consecutive observations. Note that LSBD-VAE obtained very good scores (near 0) on the Arrow and Square datasets, hence the missing bars. None of the traditional disentanglement methods achieve goodDLSBDscores, even if they score well on the traditional disentanglement metrics. This implies that LSBD isn’t achieved by traditional methods. Moreover, from the full results in Section 4.7.4 we see that the traditional methods on these datasets do not
RkJQdWJsaXNoZXIy MjY0ODMw