52 Quantifying and Learning Linear Symmetry-Based Disentanglement (LSBD) based methods (in particular our own LSBD-VAE) canlearn LSBD representations needing only limited supervision on transformations, and (3) various desirable properties expressed by existing disentanglement metrics are also achieved by LSBD representations. 4.1 Introduction Learning low-dimensional representations that disentangle the underlying factors of variation in data is considered an important step towards interpretable machine learning with good generalisation (Bengio et al., 2012). To address the fact that there is no consensus on what disentanglement entails and how to formalise it, Higgins et al. (2018) propose a formal definition for Linear Symmetry-Based Disentanglement, or LSBD, arguing that underlying real-world symmetries give exploitable structure to data. We will discuss the exact definition in Section 4.2. LSBD emphasises that the variability in data observations is often due to some real-world transformations, and that good data representations should reflect these transformations. A typical setting is that of an agent interacting with its environment. An action of the agent will transform some aspect of the environment and its observation thereof, but keeps all other aspects invariant. It is often easy and cheap to register the actions that an agent performs and how they transform the observed environment, which can provide useful information for learning disentangled representations. However, there is currently no general metric to quantify LSBD. Such a metric is crucial to properly evaluate methods aiming to learn LSBD representations and to relate LSBD to previous definitions of disentanglement (which we refer to as traditional disentanglement). Although previous works have evaluated LSBD by measuring performance on downstream tasks (Caselles-Dupré et al., 2019) or by measuring specific traits related to LSBD (Painter et al., 2020; Quessard et al., 2020), none of these evaluation methods directly quantify LSBD according to its formal definition. We propose DLSBD, a well-formalised and generally applicable metric that quantifies the level of LSBD in learned data representations (Section 4.4). We show an intuitive justification of this metric, as well as its theoretical derivation. We also provide a practical implementation to computeDLSBDfor commonSO(2) symmetry groups. Furthermore, we show that our metric formulation can be used to derive a semi-supervised method to learn LSBD representations, which we call LSBD-VAE (Section 4.5). To make LSBD-VAE more widely applicable, we also demonstrate how to disentangle symmetric properties from other non-symmetric
RkJQdWJsaXNoZXIy MjY0ODMw