5.3 Experimental Setup 111 Each factor can attain 64 values, yielding 4096 data points in each dataset. Both Square and Arrow are also used in Chapter 4. dSprites (Matthey et al., 2017) has factors shape, scale, orientation, x-position, and y-position, with 3, 6, 40, 32, and 32 values each, for a total of 737,280 data points. The shape factor is categorical, whereas the other factors describe continuous properties, of which orientation is cyclic. 3D Shapes (Burgess and Kim, 2018) has factors floor hue, wall hue, object hue, scale, shape, andorientation, with 10, 10, 10, 8, 4, and 15 values each, for a total of 480,000 data points. The shape factor is categorical, whereas the other factors describe continuous properties, of which all hue values are cyclic. 5.3.2 OOD Splits: Left-out Factor Combinations For Arrow and Square we define different OOD splits by leaving out images where both factors are within a certain range. If we represent the factors as values f1 and f2 on a scale from 0 to 1, we split off images with factor values f1 < r and f2 < r simultaneously (see Figure 5.3). We do this for various values of r, namely 0.125, 0.25, 0.375, 0.5, 0.625, 0.750, and 0.875, leading to 7 different OOD splits. Note that the ratio of the number of data points we split off from the full dataset grows quadratically as r2, i.e. each OOD split contains roughly 1.5%, 6%, 14%, 25%, 39%, 56%, and 77% of the full dataset, respectively. r=0.625 OOD Training f1 f2 Figure 5.3: Visualisation of OOD splits for datasets with 2 factors.
RkJQdWJsaXNoZXIy MjY0ODMw