Iterative versus amortized inference solutions to the constellation problem

Abstract

Making sense of the visual inputs is an essential part of human intelligence. While processing in the human visual cortex has been observed to have recurrent nature, machine vision systems with one feedforward pass from input into prediction have dominated computer vision benchmarks. This discrepancy may be explained through lack of challenging datasets where gradual refinement of solution would be necessary to lead to correct solution. Such a dataset, where local information about the encoded objects has been erased, was recently proposed. The current thesis represents the first attempt to solve this novel dataset. We propose to use generative models DCGAN and VAE with optimization algorithm CMA-ME to refine the solutions as iterative inference, and use generative models Pix2pix and CycleGAN as feedforward or amortized inference. Through solving the problem posed in the novel computer vision dataset, we show the prevalence of iterative refinement of hypotheses over the single-prediction paradigm, encouraging further research in the field of iterative inference.