Image Sampling

Eric Dubois

This web site summarizes and provides links to selected work on image and video sampling. I have been promoting the use of lattices to describe sampling structures since the early 1980s, starting with sampling structures for NTSC signals.

• J.-Y. Ouellet and E. Dubois, "Sampling and reconstruction of NTSC video signals at twice the color subcarrier frequency," IEEE Trans. on Communications, vol. COM-29, pp. 1823-1832, Dec. 1981. doi: 10.1109/TCOM.1981.1094952 [IEEE Xplore Citation]. This paper studied different spatiotemporal sampling structures for the NTSC video signal, concentrating on a horizontal sampling frequency of twice the color subcarrier frequency. Although this sampling frequency is significantly 'sub-Nyquist' when viewed in one dimension, when viewed in three dimensions (two spatial and one time) with appropriate sampling structures, the multidimensional Nyquist criterion is approximately satisfied. The so-called 'field-quincunx' or QT pattern was found to give the best results.

• E. Dubois, "The sampling and reconstruction of time-varying imagery with application in video systems (invited)," Proc. IEEE, vol. 73, pp. 502-522, April 1985. doi: 10.1109/PROC.1985.13182 [IEEE Xplore citation] (Also appeared in Visual Communications Systems, A.N. Netravali and B. Prasada, ed., IEEE Press, 1989; Selected Papers on Visual Communication: Technology and Applications ,T. Russell Hsing and Andrew G. Tescher, ed., SPIE Press, 1990; Digital Video: Concepts and Applications Across Industries, T.S. Rzeszewski, ed., IEEE Press, 1995. ) [PDF: 899kB (with correction)]. The paper is still regularly cited.

• E. Dubois, "Video sampling and interpolation," in The Essential Guide to Video Processing, (A. Bovik, ed.), ch. 2, Academic Press, 2009. [Elsevier Catalog]. Previous versions of this chapter appeared in Handbook of Image and Video Processing, A. Bovik, ed, first and second edition.

• H.A. Aly and E. Dubois, "Design of optimal camera apertures adapted to display devices over arbitrary sampling lattices," IEEE Signal Processing Letters, vol. 11, pp. 443-445, April 2004. doi: 10.1109/LSP.2004.824062 [IEEE Xplore citation]. This short note extends a standard result for one-dimensional sampling, on optimizing the camera aperture for best reconstruction with a given display aperture.

• H.A. Aly and E. Dubois, "Specification of the observation model for regularized image up-sampling," IEEE Trans. Image Process., vol. 14, pp. 567-576, May 2005. doi: 10.1109/TIP.2005.846019 [IEEE Xplore citation]. A key element of regularized upsampling is the observation model relating the observed lower resolution (LR) image to the desired higher resolution (HR) up-sampled image, used in the data-fidelity term of the regularization cost function. This paper presents an algorithm to determine this observation model based on a model of the physical acquisition process for the LR image, and the ideal acquisition process for the desired HR image, both from the same underlying continuous-space image.

• H.A. Aly and E. Dubois, "Image up-sampling using total-variation regularization with a new observation model," IEEE Trans. Image Process., vol. 14, pp. 1647-1659, Oct. 2005. doi: 10.1109/TIP.2005.851684 [IEEE Xplore citation]. This paper presents a new formulation of the regularized image up-sampling problem that incorporates models of the image acquisition and display processes. This approach leads to a new data fidelity term that has been coupled with a total-variation regularizer to yield our objective function. This objective function is minimized using a level-sets motion that is based on the level-set method, with two types of motion that interact simultaneously. A new choice of these motions leads to a stable solution scheme that has a unique minimum.