5.1 Introduction 107 Disentanglement (SBD) (Higgins et al., 2018) as discussed in Chapter 4, which provides a formal language (using group theory) to reason about the structure of underlying factors. This allows to model this structure separate from the distribution of the data. SBD focuses on disentangling symmetry transformations that act on the data, reasoning that these transformations induce the underlying factors of variation. The idea is to learn representations that are equivariant to such transformations. By focusing on these transformations during training, we hope to learn a model that can generalise well to unseen factor combinations, since such combinations are still the result of applying transformations that were seen during training. Figure 5.1 shows a simple illustrative example of why this focus on transformations should help with OOD generalisation. Rotate 180° Rotate 180° Red-to-yellow Red-to-yellow Observations Generalisation Unobserved (OOD) Figure 5.1: Illustrative example of the misalignment between underlying factors and observed distributions. Left: suppose we observe arrows with different orientations and colours, but most orientations are only observed in red arrows, whereas other colours are only observed in one orientation. Right: if we can identify the concepts of orientation and colour, we can generalise to new unobserved examples such as a yellow arrow pointing down. A model based on identifying underlying transformations (e.g. rotations and colour changes) could generalise to such an OOD example, but a model that only learns to represent the data distribution would not. In this chapter, we evaluate how well disentangled representations generalise to unseen factor combinations, with a particular focus on Linear SBD (LSBD) representations, which provide a fixed formulation of how transformations affect representations. We investigate how much coverage of factor combinations is needed for current methods to generalise well to unseen combinations, exposing the limits of these methods. We confirm previous findings that disentanglement
RkJQdWJsaXNoZXIy MjY0ODMw