A decision-theoretic formulation of visual saliency, first proposed for top-down processing (object recognition), is extended to encompass the problem of bottom-up saliency. Under this formulation, optimality is defined in the minimum probability of error sense, under a constraint of computational parsimony. The saliency of the visual features at a given location of the visual field is defined as the power of those features to discriminate between the stimulus at the location and a null hypothesis. For bottom-up saliency, this is the set of visual features that surround the location under consideration. Discrimination is defined in an information-theoretic sense, and the optimal saliency detector derived for the class of stimuli that comply with various known statistical properties of natural images. The optimal detector is shown to replicate the fundamental properties of the psychophysics of saliency. This includes pop-out, inability to detect feature conjunctions, saliency asymmetries with respect to feature presence vs. absence, compliance with Weber's law, and decreasing saliency with background heterogeneity. Finally, it is shown that the optimal detector has a one to one mapping to the standard architecture of primary visual cortex (V1), and can be applied to the solution of generic inference problems. In particular, for the broad class of stimuli studied, it performs the three fundamental operations of statistical inference: assessment of probabilities, implementation of Bayes decision rule, and feature selection. |