The literature listed below is by far not exhaustive, but it represents good starting points for further study.

Lecture 1:
E. Candes and M. Wakin. An introduction to compressive sampling. IEEE Signal Process. Magazine, 25(2):21-30, 2008.

D. Donoho. Compressed sensing. IEEE Trans. Inform. Theory 52, no.4, 1289--1306, 2006.

R. Baraniuk. Compressive sensing. IEEE Signal Process. Magazine, 24(4):118-121, 2007.

Lecture 2:
S.S. Chen, D.L. Donoho, and M.A. Saunders. Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20:33-61, 1998.

Lecture 3:
D.L. Donoho and X. Huo. Uncertainty principles and ideal atomic decomposition, IEEE Trans. Inform. Theory, 47:2845-2862, 1999.

O. Christensen. An Introduction to Frames and Riesz Bases. Applied and Numerical Harmonic Analysis. Birkauser, Boston, 2003.

Lecture 4:
R. Gribonval and M. Nielsen. Sparse representations in unions of bases. IEEE Trans. Inform. Theory, 49(12):3320-3325, 2003.

T. Strohmer and R. W. Heath. Grassmannian frames with applications to coding and communication. Appl. Comput. Harmon. Anal., 14(3):257-275, 2003.

Lecture 5:
E. Candes, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Comm. Pure Appl. Math., 59(8):1207-1223, 2006.

E. Candes and T. Tao. Decoding by linear programming. IEEE Trans. Inform. Theory, 51(12):4203-4215, 2005.

D. Donoho and M. Elad. Optimally Sparse Representation from Overcomplete Dictionaries via l_1 norm minimization. Proc. Natl. Acad. Sci. USA, 100, no.5, 2197-2202, 2003.

Lecture 6:
E. Candes. The restricted isometry property and its implications for compressed sensing. Compte Rendus de l'Academie des Sciences, Paris, Serie I, 346 589--592.

Lecture 7:
J. Fuchs. On Sparse Representations in Arbitrary Redundant Bases. IEEE Transactions on Information Theory, vol. 50, no. 6, 2004.

J. Tropp. Recovery of short, complex linear combinations via l_1 minimization. IEEE Trans. Info. Theory, vol. 51, num. 4, pp. 1568-1570, 2005.

Lecture 8:
Two online introductions to probability theory
C.M.Grinstead and J.L.Snell, Introduction to Probability

Probability Theory course notes by Roman Vershynin

Lecture 9:
R. Baraniuk, M. Davenport, R. DeVore, and M. Wakin. A Simple Proof of the Restricted Isometry Property for Random Matrices. Constructive Approximation, vol. 28, no. 3, pp. 253-263, 2008.

Lecture 10:
H. Rauhut. Compressive sensing and structured random matrices. In Theoretical Foundations and Numerical Methods for Sparse Recovery, Radon Series Comp. Appl. Math. deGruyter, to appear. Version of April 5, 2010.

Lecture 11:
S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press.

S.S. Chen, D.L. Donoho, and M.A. Saunders. Atomic decomposition by basis pursuit, SIAM J. Sci. Comput., 20:33-61, 1998.

Lecture 12:
J. Tropp. Greed is good: Algorithmic results for sparse approximation. IEEE Trans. Info. Theory, vol. 50, num. 10, pp. 2231-2242, Oct. 2004.

D. Needell, J. A. Tropp, and R. Vershynin. Greedy signal recovery review In Proc. 42nd Asilomar Conf. Signals, Systems and Computers, Pacific Grove, Oct. 2008.

W. Dai and O. Milenkovic. Subspace Pursuit for Compressive Sensing Signal Reconstruction.

Several Matlab codes can be found here

Lecture 13:
E. J. Candès and B. Recht. Exact matrix completion via convex optimization. Found. of Comput. Math., 9 717--772.

Raghunandan H. Keshavan, Sewoong Oh and Andrea Montanari. Matrix Completion from a Few Entries. http://arxiv.org/abs/0901.3150, 2009.

Lecture 14:
E. J. Candes, X. Li, Y. Ma, and J. Wright. Robust Principal Component Analysis? Preprint.

Benjamin Recht, Maryam Fazel, and Pablo A. Parrilo. Guaranteed Minimum Rank Solutions to Linear Matrix Equations via Nuclear Norm Minimization. To appear in SIAM Review.

Lecture 15:
M. Herman and T. Strohmer. High Resolution Radar via Compressed Sensing. IEEE Trans. Signal Processing, vol.57(6): 2275-2284, 2009.

Lecture 16:
D. Donoho. Neighborly Polytopes and Sparse Solutions of Underdetermined Linear Equations. 2005.

D. Donoho and J. Tanner. Exponential Bounds Implying Construction of Compressed Sensing Matrices, Error-Correcting Codes and Neighborly Polytopes by Random Sampling. 2008.

R. G. Baraniuk, V. Cevher, M. Duarte, and C. Hegde. Model-based Compressive Sensing. To appear in IEEE Transactions on Information Theory, 2010.