# EE596A - Submodularity Functions, Optimization, and Application to Machine Learning - Fall Quarter, 2012

Last updated: $Id: index.html 786 2013-05-14 19:15:08Z bilmes $

This page is located at http://j.ee.washington.edu/~bilmes/classes/ee596a_fall_2012/. A version of this class was taught previously, and its contents and slides can be found at http://ssli.ee.washington.edu/~bilmes/ee595a_spring_2011/. The class was offered again in Spring 2014, and youtube videos of the lectures are available.

# Instructor:

**Prof. Jeff A. Bilmes**--- Email me

Office: 418 EE/CS Bldg., +1 206 221 5236

Office hours: TBD, EEB-418

### TA:

**Unfortunately, no TA for this class**

# Announcements

- (Tue Oct 23th, 2012) Homework 1 is out and due next Thursday, see below.
- (Wed Sep 26th, 2012) Welcome to the class.
- (Wed Sep 26th, 2012) For a class overview, see lecture 1
- (Wed Sep 26th, 2012) Take a look at UW's academic calendar for a list of dates, holidays, etc.

# Homework IT-I

Homework must be done and submitted electronically via the following link here.- Homework 1 Due Thursday, Nov 1st, 11:45pm via our dropbox

# Lecture Slides

Lecture slides will be made available as they are being prepared --- they will probably appear here right before a given lecture, and they will be in PDF format (original source is latex). Note, that these slides are corrected after the lecture (and might also include some additional discussion we had during lecture). If you find bugs/typos in these slides, please email me. The slides are available as "presentation slides" and also in (mostly the same content) 2-up form for printing.Lec. # | Slides | Lecture Date | Last Updated | Contents | |

1 | 9/26/12 | 9/28/12 | Introduction, Motivation, Applications, and the ubiquity of submodularity/supermodularity. | ||

2 | 9/28/12 | 10/9/12 | The goal of this lecture is to develop good intuition about submodularity and supermodularity. | ||

3 | 10/3/12 | 10/9/12 | More examples and intuition on submodularity, including the many equivalent ways to define a submodular function. Start independence systems and matroids (including matroid properties and examples). | ||

4 | 10/5/12 | 10/11/12 | Today, we'll be discussing mostly matroid properties and examples (partition, laminar, transversals, etc.). Matroids are extremely powerful combinatorial objects that have many applications. | ||

5 | 10/10/12 | 10/11/12 | More matroid properties, transversals, Hall's theorem, Rado's theorem, and a generalization, and how they relate to submodular function minimization. Greedy algorithm. | ||

6 | 10/12/12 | 10/16/2012 | More on matroids, dual matroids, how the greedy algorithm defines matroids, and matroid polyhedra. | ||

7 | 10/17/12 | 10/19/12 | Matroid Polytopes, and their generalization to polymatroids, side-by-side comparison of matroids and polymatroids. | ||

8 | 10/19/12 | 10/19/12 | Review of polymatroid properties, the equivalence between the independence polytope of a polymatroid and the polytope associated with polymatroid (normalized, monotone non-decreasing, submodular) function, most violated inequality, review of augmenting paths in bipartite matchines | ||

9 | 10/24/12 | 11/6/12 | Polymatroids and the greedy algorithm, primal/dual theorm for greedy in polymatroids. | ||

10 | 11/2/12 | 11/2/12 | Tools for submodular polyhedra: most violated inequality, matroid closure vs. polymatroidal saturation (sat), matroid fundamental circuit vs. polymatroidal dependence (dep), visual examples. | ||

11 | 11/7/12 | 11/20/12 | definition of supp, supp and sat and tight sets, many visual examples of polymatroids, dependence, asymmetric determinism, separation, important properties on the base polytope. tightness of supp, and supp+e with e in sat. | ||

12 | 11/9/12 | 11/11/12 | Definition and properties of exchange capacity. The min-norm point, and proof (using sat/dep) that computation of the min-norm point in the base solves submodular function minimization. Relation between min-norm point and all minimizers of a submodular function. The Lovasz extension and the Choquet integral. | ||

13 | 11/14/12 | 11/16/12 | More on the Lovasz extension and the Choquet integral, with emphasis on intuition. | ||

13.5 | --- | --- | Extra lecture slides on Birkhoff lattices, partially ordered sets, modular and submodular lattices, and (in passing) why is submodularity called submodularity. | ||

14 | 11/16/12 | 11/21/12 | Still more discussion of the Lovasz extension, a number of examples. Start the discussion on the partial order of polymatroidal extreme points. | ||

15 | 11/21/12 | 11/21/12 | On final project. All technical material continue with lecture14 slides for now. Review the partial order section. | ||

16 | 11/28/12 | 11/29/12 | Preprocessing for submodular function minimization. alternating sequences in the case of matching in bipartite graphs, and maximum independent set in two matroids (i.e., matroid intersection). Special cases of SFM. | ||

17 | 11/21/12 | 11/21/12 | Special cases of SFM (Edmond's matroid partition, and Cunninghams rank minus x), and brief discussion of general SFM (other than min-norm point alg). This is the last day of SFM, next time we'll start SF Maximization. | ||

18 | 12/11/12 | 12/11/12 | Symmetric submodular functions, Symmetric submodular functions and generalized independence/separators, Queyranne's O(n^3) algorithm for symmetric submodular function minimization, pendant pairs, max k-cover problem, set cover problem, and generalization to submodular max and submodular cover, maximizing a non-negative monotone nondecreasing submodular (i.e., polymatroid), proof of 1-1/e (and a more general) bound for the greedy algorithm, Minoux's accelerated greedy algorithm. | ||

19 | 12/13/12 | 12/13/12 | submodular maximization under more general constraints (knapsack and matroid constraints), application: generalizd bipartite matching and string-string alignment in machine translation, application: submodular welfare, properties of local maxima of submodular functions (and lattice restriction), 1/3 algorithm via local search for unconstrained non-monotone submodular max, tight 1/2-approximate linear time randomized algorithm for unconstrained non-monotone submodular max, summary of algorithms, curvature of submodular funtions and curvature based bounds, multilinear extension some of its uses. | ||

Lec. # | Slides | Lecture Date | Last Updated | Contents |

# Actually Presented Lecture Slides, Fall, 2012

Lecture slides that were presented in class, along with all of the bugs and typos, my ink corrections of (perhaps some) of the bugs and typos, and any other little notes/discussions/drawings I drew on the slides during class. The above slides often contain more material than these as any discussions during class were added to the above after class. On the other hand, there might be a few hand-drawn figures in the below that I have not yet added to the above. Note: not all PDF readers can see the annotations in these slides (e.g., at least the Safari embedded reader on the iphone/ipad doesn't see them) --- the annotations were done with Adobe acrobat.- Lecture 1 from 9/26/12.
- Lecture 2 from 9/28/12.
- Lecture 3 from 10/03/12.
- Lecture 4 from 10/05/12.
- Lecture 5 from 10/10/12.
- Lecture 6 from 10/12/12.
- Lecture 7 from 10/17/12.
- Lecture 8 from 10/19/12.
- Lecture 9 from 10/24/12.
- Lecture 10 from 11/2/12.
- Lecture 11 from 11/7/12.
- Lecture 12 from 11/9/12.
- Lecture 13 from 11/14/12.
- Lecture 14 from 11/16/12.
- Lecture 14/15 from 11/21/12.
- Lecture 16 from 11/28/12.
- Lecture 17 from 11/30/12.
- Lecture 18 from 12/11/12.

# Discussion Board

You can post questions, discussion topics, or general information at this link.# Relevant Books

There are many books available that discuss some the material that we are covering in this course. Some good books are listed below, but see the end of the lecture slides for books/papers that are relevant to each specific lecture.- Fujishige, "Submodular Functions and Optimization", 2005
- Narayanan, "Submodular Functions and Elecrical Networks", 1997
- Welsh, "Matroid Theory", 1975.
- Oxley, "Matroid Theory", 1992 (and 2011).
- Lawler, "Combinatorial Optimization: Networks and Matroids", 1976.
- Schrijver, "Combinatorial Optimization", 2003
- Gruenbaum, "Convex Polytopes, 2nd Ed", 2003.
- See lecture 1 slides for our text and relevant other texts.

# Important Dates/Exceptions

- Dec 5-7th, NIPS

# Alternative Contact

If you must, you can send me anonymous email, but first let me know you want to send me anonymous email so I can set it up.