102 Quantifying and Learning Linear Symmetry-Based Disentanglement (LSBD) LSBD-VAE-TL scores worse than a simple encoder with triplet loss, although its scores on most metrics are still decent. Again we believe that the LSBD-VAE hasn’t learned to disentangle orientation from shape well enough, and rotational symmetries in the 3D objects contradict the assumption of LSBD-VAE that the underlying group action is regular. Table 4.12: Evaluation measures for the retrieval-by-culture challenge. This table shows all the evaluation metrics for all the submitted runs. Highest values are in bold. Methods NN FT ST mAP nDCG TL (ours) 0.8254 0.8356 0.8229 0.8291 0.8989 AE-TL (ours) 0.7619 0.6481 0.7661 0.6510 0.8542 VAE-TL (ours) 0.8360 0.8477 0.8503 0.8450 0.9112 ∆VAE-TL (ours) 0.7989 0.8092 0.8034 0.8031 0.8827 LSBD-VAE-TL (ours) 0.7937 0.5960 0.7639 0.6210 0.8543 DNE (run1) 0.7831 0.7466 0.7767 0.7457 0.8799 DNE (run2) 0.7566 0.7074 0.7673 0.7190 0.8715 DRF-L (run1) 0.7302 0.6302 0.7831 0.6729 0.8648 DRF-L (run2) 0.7249 0.5516 0.7670 0.5713 0.8383 MVCNN (run1) 0.1852 0.8088 0.7989 0.7410 0.8532 MVCNN (run2) 0.2063 0.8292 0.8130 0.7626 0.8675 MVCNN (run3) 0.1534 0.7932 0.7925 0.7882 0.8748 MVCNN (run4) 0.1799 0.8130 0.8077 0.8081 0.8889 MVCNN (run5) 0.7566 0.6354 0.7783 0.6772 0.8686 MeshNN (run1) 0.7249 0.7418 0.7354 0.7415 0.8353 NPC (run1) 0.7037 0.5489 0.7539 0.5736 0.8267 NPC (run2) 0.6772 0.5523 0.7565 0.5776 0.8273 RVN(run1) 0.8254 0.8497 0.8536 0.8484 0.9043 RVN(run2) 0.8360 0.8738 0.8783 0.8698 0.9186 RVN(run3) 0.8307 0.8080 0.8257 0.8207 0.9138 RVN(run4) 0.8201 0.8252 0.8420 0.8320 0.9176 SE3D (run1) 0.6772 0.6029 0.7605 0.6091 0.8374 SE3D (run2) 0.7619 0.6916 0.7656 0.7133 0.8663 SE3D (run3) 0.7778 0.6957 0.7612 0.7181 0.8663 SE3D (run4) 0.7090 0.7444 0.7727 0.7487 0.8611 SE3D (run5) 0.7037 0.7180 0.7582 0.7321 0.8542 Overall, we conclude that LSBD-VAE can be combined with a triplet loss for retrieval tasks, but that the results are not (yet) state of the art, and it is difficult to train a well-disentangled model on real-world datasets such as in this challenge. Nevertheless, we believe it is an interesting direction of future research, and improvements in the training of models such as LSBD-VAE in real-world settings can also lead to improved performance in downstream tasks
RkJQdWJsaXNoZXIy MjY0ODMw