3.3 Anomaly Detection with Generative Models 29 • high-dimensional data; e.g. images of 224×224=50, 176 pixels, • few data points, • very high imbalance w.r.t. the target variable; there are only few defective products and many good products. We approach these challenges by: • defining a model for likelihood estimation, • training this model on majority class (non-anomalous) data, • finding a suitable threshold for the likelihood produced by the model; the threshold determines the decision of anomaly detection, • localising the anomalous parts of the image for visualisation. For the likelihood estimation, we aim to discover the distributionpdata of normal data, by defining a parametric distribution pθ and estimating the optimal parameters θ to approximate pdata. We then define anomalies as points in lower-likelihood regions of pθ. 3.3.1 Likelihood Estimation with VAEs Variational Autoencoders (Kingma and Welling, 2013; Rezende et al., 2014) (VAEs) are latent variable models where the parameters ϕ of an approximate posterior distributionqϕ(z|x) and the parameters θ of a generative model pθ(x|z) are modelled by neural networks. Herexrepresents data points, andzrepresents latent variables that are assumed to follow some prior distribution p(z). A detailed explanation is given in Section 2.1, but we repeat the key concepts here for completeness. VAEs are trained by maximising the evidence lower bound (ELBO), a lower bound to the marginal log likelihood logpθ(x) of the data. This ELBO can be expressed as: ELBO(ϕ,θ; x)=Eqϕ(z|x)[logpθ(x|z)] −KL(qϕ(z|x)||p(z)). (3.1) Here, KL(·||·) represents the KL divergence between two probability distributions. Equation (3.1) features two terms; the first can be interpreted as a pixel-level reconstruction error, whereas the KL divergence in the second term acts as a
RkJQdWJsaXNoZXIy MjY0ODMw