CS599: Convex and Combinatorial Optimization (Fall 2013)
Basic information
- Lecture time: Tuesdays and Thursdays 2 pm - 3:20 pm
- Lecture place: KAP 158
- Instructor: Shaddin Dughmi
- Email: shaddin@usc.edu
- Office: SAL 234
- Office Hours: Tuesdays 3:30pm - 4:30pm
- TA: Yu Cheng
- Email: chen533@usc.edu
- Office Hours: Friday 4pm-5pm in SAL 219
- Course Homepage: www-bcf.usc.edu/~shaddin/cs599fa13
Announcements
- Dec 12 : Homework 3 solutions have been posted.
- Nov 24 : Homework 2 solutions have been posted.
- Nov 24: Homework 3 has been updated with 2 more problems.
- Nov 13: The first two problems of homework 3 have been posted. As mentioned in class, homework 3 will be released piecemeal over the last few weeks of class.
- Oct 19: Details for the class project have been posted here.
- Oct 15: Homework 2 is out. It is due on Tuesday 11/5.
- Oct9 : Homework 1 solutions have been posted.
- Sep30: Yu's office hours have changed. See above.
- Sep9 : Homework 1 is out. It is due on Thursday 9/26.
- Sep5: Yu's office hours have been set. See above.
- Sep5 : We have a new, bigger room! KAP 158.
Schedule by Week
- Week 1: Introduction to Optimization. Linear Programming and Duality.
- Week 2: Wrapping up LP duality. Convex Sets.
- Week 3: Convex Functions
- Week 4: Geometric Duality, Wrapping up Convex Functions
- Weeks 5: Convex optimization Problems
- Weeks 6-7: Duality of Convex Optimization Problems
- Weeks 7-9: Combinatorial problems as linear and convex programs
- Weeks 10 -12: Algorithms: Simplex method, ellipsoid method and its consequences.
- Slides: Lecture 18, Lecture 19, Lectures 20 and 21.
- Korte Vygen Chapter 3 (Simplex Algorithm) and Chapter 4 (Ellipsoid Algorithm).
- Additional Reading: Luenberger and Ye Chapter 3 and Vince Conitzer's lecture notes for an algebraic treatment of the simplex method
- Highly recommended: Ryan O'Donnell's lecture notes on the Ellipsoid method: here and here.
- Ben Tal and Nemirovski's lecture notes on the ellipsoid method and polynomial sovability of convex programming (Chapter 8). These lecture notes describe the most useful and general polynomial-solvability guarantee for convex programming which I could find, and Shaddin is puzzled as to why this isn't the standard "solvability statement" presented in optimization textbooks.
- For equivalence of separation and optimization, I recommend the impeccably-written breakthrough paper by Grotschel, Lovasz and Schrijver. You may have to use the USC library proxy to get access, or find it elsewhere online.
- Week 12-13: Matroid theory. Optimization over matroids and matroid intersections.
- Week 14-15: Submodular functions and optimization.
- Slides: Lecture 24, 25, 26.
- Lecture notes by Jan Vondrak (lectures 16-19)
- A survey article by your instructor on continuous extensions of submodular functions and their algorithmic applications.
- The original paper by Laszlo Lovasz on submodular functions and convexity. You may have to use the USC library proxy to get access, or find it elsewhere online.
- The Calinescu et al paper on maximizing a monotone submodular functions subject to a matroid constraint.
- The recent paper by Buchbinder et al on unconstrained maximization of non-monotone submodular functions.
- Slides from a recent tutorial by Jan Vondrak, with an overview of many of the "state of the art" results.
- For those interested in applications of submodularity to machine learning, see the materials and references on this page maintained by Andreas Krause and Carlos Guestrin.
Course Description
Over the past half century or so, computer science and mathematical optimization have witnessed the development and maturity of two different paradigms for algorithm design.
The first approach, most familiar to computer scientists, is combinatorial in nature. The tools of discrete mathematics are used to understand the structure of the problem, and algorithms effectively exploit this structure to search over a large yet finite set of possible solutions. The second approach, standard in much of the operations research and mathematical optimization communities, primarily employs the tools of continuous mathematics, high dimensional geometry, and convex analysis. Problems are posed as a search over a set of points in high-dimensional Euclidean space, which can can be performed efficiently when the search space and objective function are convex.
Whereas many optimization problems are best modeled either as a discrete or convex optimization problem, researchers have increasingly discovered that many problems are best tackled by a combination of combinatorial and continuous techniques. The ability to seemlessly transition between the two views has become an important skill to every researcher working in algorithm design and analysis. This course intends to instill this skill by presenting a unified treatment of both approaches, focusing on algorithm design tasks that employ techniques from both. The intended audience for this course are PhD students, Masters students, and advanced undergraduates interested in research questions in algorithm design, mathematical optimization, or related disciplines.
Prerequisites
The main prerequisites for this class are mathematical maturity, a grounding in linear algebra, as well as exposure to algorithm design and analysis at the beginning graduate level (CS570 or 670). Students without such background can seek permission of the instructor.
Requirements and Grading
Homework assignments will count for 75% of the grade. There will be 3-4 assignments, roughly 3 weeks apart each. The homeworks will be proof-based, and are intended to be very challenging. Collaboration and discussion among students is allowed, even encouraged, though students must write up their solutions independently.
The remaining 25% of the grade will be allocated to a final project. Students will have to choose a related research topic, read several papers in that area, and write a survey of the area.
Late Homework Policy: Students will be allowed one late homework, at most two days from the due date. No additional late homework will be accepted.
References
We will refer to two main texts: Convex Optimization by Boyd and Vandenberghe, available free online, and Combinatorial Optimization by Korte and Vygen, available online through USC libraries. Additional references include Combinatorial Optimization by Schrijver, Linear and Nonlinear Programming by Luenberger and Ye, available online through usc libraries, as well as lecture notes from related courses elsewhere.
Additionally, we might refer to research papers throughout the course, which will be linked on the course homepage.