EE596A - Submodularity Functions, Optimization, and Application to Machine Learning - Fall Quarter, 2012

Last updated: $Id: index.html 786 2013-05-14 19:15:08Z bilmes $

This page is located at A version of this class was taught previously, and its contents and slides can be found at


Prof. Jeff A. Bilmes --- Email me
Office: 418 EE/CS Bldg., +1 206 221 5236
Office hours: TBD, EEB-418


Unfortunately, no TA for this class


Homework IT-I

Homework must be done and submitted electronically via the following link here.
  1. Homework 1 Due Thursday, Nov 1st, 11:45pm via our dropbox

Lecture Slides

Lecture slides will be made available as they are being prepared --- they will probably appear here right before a given lecture, and they will be in PDF format (original source is latex). Note, that these slides are corrected after the lecture (and might also include some additional discussion we had during lecture). If you find bugs/typos in these slides, please email me. The slides are available as "presentation slides" and also in (mostly the same content) 2-up form for printing.
Lec. # Slides Print Lecture Date Last Updated Contents
1 pdf pdf 9/26/12 9/28/12 Introduction, Motivation, Applications, and the ubiquity of submodularity/supermodularity.
2 pdf pdf 9/28/12 10/9/12 The goal of this lecture is to develop good intuition about submodularity and supermodularity.
3 pdf pdf 10/3/12 10/9/12 More examples and intuition on submodularity, including the many equivalent ways to define a submodular function. Start independence systems and matroids (including matroid properties and examples).
4 pdf pdf 10/5/12 10/11/12 Today, we'll be discussing mostly matroid properties and examples (partition, laminar, transversals, etc.). Matroids are extremely powerful combinatorial objects that have many applications.
5 pdf pdf 10/10/12 10/11/12 More matroid properties, transversals, Hall's theorem, Rado's theorem, and a generalization, and how they relate to submodular function minimization. Greedy algorithm.
6 pdf pdf 10/12/12 10/16/2012 More on matroids, dual matroids, how the greedy algorithm defines matroids, and matroid polyhedra.
7 pdf pdf 10/17/12 10/19/12 Matroid Polytopes, and their generalization to polymatroids, side-by-side comparison of matroids and polymatroids.
8 pdf pdf 10/19/12 10/19/12 Review of polymatroid properties, the equivalence between the independence polytope of a polymatroid and the polytope associated with polymatroid (normalized, monotone non-decreasing, submodular) function, most violated inequality, review of augmenting paths in bipartite matchines
9 pdf pdf 10/24/12 11/6/12 Polymatroids and the greedy algorithm, primal/dual theorm for greedy in polymatroids.
10 pdf pdf 11/2/12 11/2/12 Tools for submodular polyhedra: most violated inequality, matroid closure vs. polymatroidal saturation (sat), matroid fundamental circuit vs. polymatroidal dependence (dep), visual examples.
11 pdf pdf 11/7/12 11/20/12 definition of supp, supp and sat and tight sets, many visual examples of polymatroids, dependence, asymmetric determinism, separation, important properties on the base polytope. tightness of supp, and supp+e with e in sat.
12 pdf pdf 11/9/12 11/11/12 Definition and properties of exchange capacity. The min-norm point, and proof (using sat/dep) that computation of the min-norm point in the base solves submodular function minimization. Relation between min-norm point and all minimizers of a submodular function. The Lovasz extension and the Choquet integral.
13 pdf pdf 11/14/12 11/16/12 More on the Lovasz extension and the Choquet integral, with emphasis on intuition.
13.5 pdf pdf --- --- Extra lecture slides on Birkhoff lattices, partially ordered sets, modular and submodular lattices, and (in passing) why is submodularity called submodularity.
14 pdf pdf 11/16/12 11/21/12 Still more discussion of the Lovasz extension, a number of examples. Start the discussion on the partial order of polymatroidal extreme points.
15 pdf pdf 11/21/12 11/21/12 On final project. All technical material continue with lecture14 slides for now. Review the partial order section.
16 pdf pdf 11/28/12 11/29/12 Preprocessing for submodular function minimization. alternating sequences in the case of matching in bipartite graphs, and maximum independent set in two matroids (i.e., matroid intersection). Special cases of SFM.
17 pdf pdf 11/21/12 11/21/12 Special cases of SFM (Edmond's matroid partition, and Cunninghams rank minus x), and brief discussion of general SFM (other than min-norm point alg). This is the last day of SFM, next time we'll start SF Maximization.
18 pdf pdf 12/11/12 12/11/12 Symmetric submodular functions, Symmetric submodular functions and generalized independence/separators, Queyranne's O(n^3) algorithm for symmetric submodular function minimization, pendant pairs, max k-cover problem, set cover problem, and generalization to submodular max and submodular cover, maximizing a non-negative monotone nondecreasing submodular (i.e., polymatroid), proof of 1-1/e (and a more general) bound for the greedy algorithm, Minoux's accelerated greedy algorithm.
19 pdf pdf 12/13/12 12/13/12 submodular maximization under more general constraints (knapsack and matroid constraints), application: generalizd bipartite matching and string-string alignment in machine translation, application: submodular welfare, properties of local maxima of submodular functions (and lattice restriction), 1/3 algorithm via local search for unconstrained non-monotone submodular max, tight 1/2-approximate linear time randomized algorithm for unconstrained non-monotone submodular max, summary of algorithms, curvature of submodular funtions and curvature based bounds, multilinear extension some of its uses.
Lec. # Slides Print Lecture Date Last Updated Contents

Actually Presented Lecture Slides, Fall, 2012

Lecture slides that were presented in class, along with all of the bugs and typos, my ink corrections of (perhaps some) of the bugs and typos, and any other little notes/discussions/drawings I drew on the slides during class. The above slides often contain more material than these as any discussions during class were added to the above after class. On the other hand, there might be a few hand-drawn figures in the below that I have not yet added to the above. Note: not all PDF readers can see the annotations in these slides (e.g., at least the Safari embedded reader on the iphone/ipad doesn't see them) --- the annotations were done with Adobe acrobat.
  1. Lecture 1 from 9/26/12.
  2. Lecture 2 from 9/28/12.
  3. Lecture 3 from 10/03/12.
  4. Lecture 4 from 10/05/12.
  5. Lecture 5 from 10/10/12.
  6. Lecture 6 from 10/12/12.
  7. Lecture 7 from 10/17/12.
  8. Lecture 8 from 10/19/12.
  9. Lecture 9 from 10/24/12.
  10. Lecture 10 from 11/2/12.
  11. Lecture 11 from 11/7/12.
  12. Lecture 12 from 11/9/12.
  13. Lecture 13 from 11/14/12.
  14. Lecture 14 from 11/16/12.
  15. Lecture 14/15 from 11/21/12.
  16. Lecture 16 from 11/28/12.
  17. Lecture 17 from 11/30/12.
  18. Lecture 18 from 12/11/12.

Discussion Board

You can post questions, discussion topics, or general information at this link.

Relevant Books

There are many books available that discuss some the material that we are covering in this course. Some good books are listed below, but see the end of the lecture slides for books/papers that are relevant to each specific lecture.

Important Dates/Exceptions

Alternative Contact

If you must, you can send me anonymous email, but first let me know you want to send me anonymous email so I can set it up.