Applied & Computational Harmonic Analysis: Comments, Handouts, and References Page (Spring 2004)

• Please fill out the following Course Questionnaire and return it to me in person or via email.
• My Frequently Asked Questions on Wavelets is available here. This is an extremely easy and intuitive piece but may be very useful for further study.
• Check Useful Links page

• H. Dym & H. McKean: Fourier Series and Integrals , Academic Press, 1972.  Chap. 2.
• G. B. Folland: Fourier Analysis and Its Applications , Brooks/Cole, 1992. Chap. 7.
• M. A. Pinsky: Introduction to Fourier Analysis and Wavelets , Brooks/Cole, 2002. Chap. 2.

For quantization, which will not be discussed in this course due to the time constraint, see:

R. M. Gray and D. L. Neuhoff: "Quantization," IEEE Trans. Inform. Theory , vol. 44, no.6, pp.2325-2383, 1998.

K. Sayood: Introduction to Data Compression, 2nd Ed., Morgan Kaufmann Publ., 2000. In particular, Chapters 8 & 9.

Basics:

• Dym & McKean, Chap. 2.
• Folland: Fourier Analysis, Chap. 7.
• Pinsky: Chap.2.
• Course handout on the Fourier Inversion Theorem and the L2 Theory

Details of L2 theory:

• G. B. Folland: Real Analysis, 2nd Ed., Wiley Interscience, 1999.  Chap. 8.
• Pinsky: Chap.2.
• E. M. Stein & G. L. Weiss: Introduction to Fourier Analysis on Euclidean Spaces, Princeton Univ. Press, 1970.  Chap. 1.

Survey on the uncertainty principle (advanced):

• G. B. Folland & A. Sitaram: "The uncertainty principle: A mathematical survey,"  Journal of Fourier Analysis and Applications , vol.3, no.3, pp.207-238, 1999.

Sampling Theorems:
• R. N. Bracewell: The Fourier Transform and Its Applications , 2nd Ed., Revised, McGraw-Hill, 1987. Chap. 10.
• W. L. Briggs & V. E. Henson: The DFT: An Owner's Manual for the Discrete Fourier Transform, SIAM, 1995. Sec. 3.4, Chap. 6.
  
• Pinsky: Chap.4.

For more details on Sampling Theorems and Non-Uniform Sampling Schemes, see:
• H. J. Landau: "Sampling, data transmission, and the Nyquist rate," Proc. IEEE, vol.55, no.10, pp.1701-1706, 1967.
• A. Aldroubi and K. Gröchenig, "Nonuniform sampling and reconstruction in shift-invariant spaces," SIAM Review, vol.43, no.4, pp.585-620, 2001.
  

For the historical articles on the sampling theorems, see:
• E. T. Whittaker: "On the functions which are represented by the expansions of the interpolation-theory," Proc. Royal Soc. Edinburgh , Sec. A, vol.35, pp.181-194, 1915.
• C. E. Shannon: "Communication in the presence of noise," Proc. IRE, vol.37, pp.10-21, 1949.

Generalized Functions (Distributions):

• Folland: Fourier Analysis, Chap. 9.
• Folland: Real Analysis, Chap. 9.
• Stein & Weiss, Chap. 1.
• Course handout on the Generalized Functions

Basic Fourier Series Theory:

• Dym & McKean, Chap. 1.
• Folland: Fourier Analysis, Chap. 2.
• Pinsky: Chap. 1.
• Course handout on the Fourier Series

Smoothness Class and Decay Rate of the Fourier Coefficients:
• Folland: Fourier Analysis, Sec. 2.3.
• Pinsky: Sec. 1.2.3.

Specific reference made in the class:
• M. Taibleson: "Fourier coefficients of functions of bounded variation," Proc. Amer. Math. Soc., vol.18, pp.766, 1967.

Discrete Fourier Transform:
• Course handout on the Discrete Fourier Transform
• Briggs & Henson: Chap. 2.
• Folland: Fourier Analysis, Sec. 7.6.

For advanced topics on DFT, check out the following articles and book:

• J. H. McClellan ad T. W. Parks: "Eigenvalue and eigenvector decomposition of the discrete Fourier transform," IEEE Trans. Audio and Electroacoustics, vol.AU-20, no.1, pp.66-74, 1972 (with comments, vol.AU-21, pp.65, 1973).
• L. Auslander and R. Tolimieri: "Is computing with the finite Fourier transform pure or applied mathematics?" Bull. AMS, vol.1, no.6, pp.847-897, 1979.
• A. Terras: Fourier Analysis on Finite Groups and Applications, London Mathematical Society Student Texts vol.43, Cambridge Univ.Press, 1999.

Discrete Fourier Transform II: In addition to the previous references, I would recommend:
• Cleve Moler: Numerical Computing with MATLAB, SIAM, 2004, Chap.8 Fourier Analysis.

Relationship between Fourier Series, DFT, and Trigonometric Interpolation:
• C. Lanczos: Discourse on Fourier Series, Hafner Publishing Co., New York, 1966. Sec 17.
• I. P. Natanson: "On the convergence of trigonometrical interpolation at equi-distant knots," Annals of Math., vol.45, pp.457-471, 1944.
• J. P. Boyd: Chebyshev and Fourier Spectram Methods, 2nd Ed., Dover, 2001. Chapter 4, in particular, Sec.4.5.

There are many references on FFT, but the following are particularly useful:

• J. W. Cooley and J. W. Tukey: "An algorithm for the machine calculation of complex Fourier series," Math. Comput., vol.19, pp.297-301, 1965.
• Briggs & Henson: Chap. 10.
• C. F. Van Loan: Computational Frameworks for the Fast Fourier Transform, SIAM, 1992.
  
• W. T. Cochran et al.: "What is the Fast Fourier Transform?", Proc. IEEE, vol.55, no.10, pp.1664-1674, 1967.

For advanced and thought-provoking aspects of FFT, read:

• A. Edelman, P. McCorquodale, and S. Toledo: "The future fast Fourier transform?" SIAM J. Sci. Comput., vol.20, no.3, pp.1094-1114, 1999.

The history of the FFT is another very interesting subject.  See e.g.,
• M. T. Heideman, D. H. Johnson, and C. S. Burrus: "Gauss and the history of the fast Fourier transform," Arch. Hist. Exact Sciences , vol.34, pp.265-277, 1985.
  

The following is also an excellent review paper on FFT/DFT and its application to complex analysis:
• P. Henrici:  Fast Fourier methods in computational complex analysis, SIAM Review, vol.21, no.4, pp.481-527, 1979.

Perhaps, the best FFT public software package is:
• FFTW, which also includes higher dimesional FFT as well as DCT/DST.

Here are my handwritten slides of the FFT algorithm shown in the lecture.

• N. Ahmed, T. Natarayan, and K. R. Rao: "Discrete cosine transform," IEEE Trans. Comput., vol.COM-23, pp.90-93, 1974.
• K. R. Rao and P. Yip: Discrete Cosine Transform: Algorithms, Advantages, and Applications, Academic Press, 1990.
• G. Strang: "The discrete cosine transform," SIAM Review, vol.41, no.1, pp.135-147, 1999.
• M. V. Wickerhauser: Adapted Wavelet Analysis from Theory to Software, A K Peters, Ltd., 1994.  Chap. 3.

For the Sturm-Liouville Theory I referred to in today's lecture, the following are nice references:
• Folland: Fourier Analysis: Sec.3.5, 3.6, 7.4.
• Dym & McKean: Sec. 1.7, 1.9.
• R. Courant & D. Hilbert: Methods of Mathematical Physics, Vol. I, First English Edition, John Wiley & Sons, 1953.  Republished as Wiley Classics Library in 1989.  See Chap. V in particular.

Discrete version (aka Principal Component Analysis [PCA]):

• K. Fukunaga: Introduction to Statistical Pattern Recognition, 2nd Edition, Academic Press, 1990. Chap. 9  & Appendix A.
• K. V. Madia, J. T. Kent, and J. M. Bibby: Multivariate Analysis , Academic Press, 1979. Chap. 8.
• S. Watanabe: "Karhunen-Loève expansion and factor analysis: Theoretical remarks and applications," Trans. 4th Prague Conf. Inform. Theory, Statist. Decision Functions, Random Processes ,  Publishing House of the Czechoslovak Academy of Sciences, Prague, pp.635-660, 1965.

Continuous version:

• U. Grenander: Stochastic processes and statistical inference, Arkiv för Matematik, vol.1, pp.195-277, 1950.
• W. B. Davenport and W. L. Root: An Introduction to the Theory of Random Signals and Noise, McGraw Hill, 1958, republished by IEEE Press, 1987. Chap. 6.
• W. D. Ray and R. M. Driver: "Further decomposition of the Karhunen-Loève series representation of a stationary random process," IEEE Trans. Inform. Theory, vol.IT-16, no.6, pp.663-668, 1970.

See also:

• F. Riesz and B. Sz.-Nagy: Functional Analysis, Frederic Ungar, 1950, republished by Dover, 1990.  Chap. VI.
• R. Courant & D. Hilbert: Methods of Mathematical Physics, Vol. I, First English Edition, John Wiley & Sons, 1953.  Republished as Wiley Classics Library in 1989.  Chap. 3.

More about the Rogues's Gallery Problem:

• M. Kirby and L. Sirovich, "Application of Karhunen-Loeve procedure for the characterization of human faces," IEEE Trans. Pattern Anal. Machine Intell., vol.12, no.1, pp.103-108, 1990.
• N. Saito: "Image approximation and modeling via least statistically dependent bases", Pattern Recognition, vol.34, no.9, pp.1765-1784, 2001.

Relationship with SVD:

• N. Saito: "Image approximation and modeling via least statistically dependent bases", Pattern Recognition, vol.34, no.9, pp.1765-1784, 2001.

Relationship with DCT:

• N. Ahmed, T. Natarajan, and K. R. Rao, "Discrete cosine transform," IEEE Trans. Comput., vol.COM-23, pp.90-93, 1974.
• R. J. Clarke: "Relation between the Karhunen Loève and cosine transforms," IEE Proc., vol.128, Part F, no.6, pp.359-360, 1981.

On Toeplitz matrices:

• U. Grenander and G. Szegö: Toeplitz Forms and Their Applications , 2nd Ed., AMS-Chelsea, 1984.
• R. M. Gray: Toeplitz and circulant matrices: A review, Technical Report, Information Systems Laboratory, Department of Electrical Engineering, Stanford University, 2002.
• H. Widom: "Toeplitz matrices," in Studies in Real and Complex Analysis (I. I. Hirschman, Jr. ed.), MAA Studies in Mathematics, 1965.

Ramp Process:

• Y. Meyer: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations, University Lecture Series, vol.22, AMS, 2001, Sec.1.8-1.10.

• S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed., Academic Press, 1999.  Chap. 4.
• I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.  Chap. 2.
• I. Daubechies: "The wavelet-transform, time-frequency localization and signal analysis," IEEE Trans. Inform. Theory, vol.36, pp.961-1005, 1990.

Some historical papers:

• D. Gabor: "Theory of communication," J. IEE (London), vol.93, pp.429-457, 1946.
• J. Ville: "Théorie et applications de la notion de signal analytique," Cables et Transmissions, 2ème A, no.1, pp.61-74, 1948.

• S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed., Academic Press, 1999.  Chap. 4 & 5.
• I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.  Chap. 3, 4.
• I. Daubechies: "The wavelet-transform, time-frequency localization and signal analysis," IEEE Trans. Inform. Theory, vol.36, pp.961-1005, 1990.

• S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed., Academic Press, 1999.  Chap. 5.
• I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.  Chap. 3, 4.
• J.-P. Kahan and P.-G. Lemarié-Rieusset: Fourier Series and Wavelets, Studies in the Development of Modern Mathematcis, Vol. 3, Gordon and Breach Publishers, 1995. Chap. 1 of the Wavelet portion.
• Course handout on the Balian-Low theorem.

See also the following original papers:

• R. Balian: "Un principe d'incertitude fort en théorie du signal ou en mécanique quantique," C. R. Acad. Sci. Paris , vol.292, pp.1357-1362, 1981.
• M. J. Bastiaans: "Gabor's signal expansion and degrees of freedom of a signal," Proc. IEEE, vol.68, pp.538-539, 1980.
• G. Battle: "Heisenberg proof of the Balian-Low theorem," Lett. Math. Phys., vol.15, pp.175-177, 1988.
• I. Daubechies: "The wavelet-transform, time-frequency localization and signal analysis," IEEE Trans. Inform. Theory, vol.36, pp.961-1005, 1990.
• I. Daubechies and A. J. E. M. Janssen: "Two theorems on lattice expansions," IEEE Trans. Inform. Theory, vol.39, pp.3-6, 1993.

• S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed., Academic Press, 1999.  Chap. 4, 5.
• I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.  Chap. 3, 4.
• M. Holschneider: Wavelets: An Analysis Tool, Clarendon Press, Oxford, 1995.  Chap. 1.
  

Analytic Signals:

• A. Papoulis: Signal Analysis, McGraw-Hill, 1977. Sec.4-2.

• S. Mallat: A Wavelet Tour of Signal Processing, 2nd Ed., Academic Press, 1999.  Chap. 7.
• I. Daubechies: Ten Lectures on Wavelets, SIAM, 1992.  Chap. 5.
• J.-P. Kahan and P.-G. Lemarié-Rieusset: Fourier Series and Wavelets, Studies in the Development of Modern Mathematcis, Vol. 3, Gordon and Breach Publishers, 1995. Chap. 3 of the Wavelet portion.