# EE596B - Submodular Functions, Optimization, and Applications to Machine Learning, Spring Quarter, 2014

Last updated: $Id: index.html 1371 2014-06-11 02:16:37Z bilmes $

This page is located at http://j.ee.washington.edu/~bilmes/classes/ee596b_spring_2014/.

A version of this class was taught previously, and its contents and slides can be found at http://j.ee.washington.edu/~bilmes/classes/ee596a_fall_2012/

# Instructor:

**Prof. Jeff A. Bilmes**--- Email me

Office: 418 EE/CS Bldg., +1 206 221 5236

Office hours: TBD, EEB-418 (+ online)

### TA

**Rishabh Iyer (rkiyer@u.washington.edu)**

Office: EEB-417

Office hours: TBD

### Time/Location

Class is held: Mon/Wed 3:00-4:50, Mueller Hall, Room 154.# Announcements

- Sticky: Almost all of the announcements, homework solutions, and discussion for our class are going to be posted on canvas, and can be found at this link. However, do see the youtube videos below and you can see everything.
- Homework 1 is posted on our assignments page. It is due, Wednesday night 4/23 at 11:45pm electronically.

# Submodular Functions, Optimization, and Applications to Machine Learning

Description: This class will be a thorough introduction to submodular functions.

This class will be a thorough introduction to submodular functions. Applications of submodularity are vast, and include areas in in computer vision, constraint satisfaction, game theory, social networks, economics, information theory, structured convex norms, natural language processing, sensor networks, graphical models and probabilistic inference, and other areas of machine learning. Submodularity is a good model for cooperation, complexity, and attractiveness as well as for diversity, coverage, and information.

In this class, we will learn about a variety of properties of submodularity and supermodularity. Motivated by applications, we'll cover submodularity's definitions, its properties, the many operations that preserve submodularity, variants and extensions of submodularity, and certain special submodular functions, and computational properties. The goal of this section will be to develop a deep intuitive understanding of both submodular and supermodular functions.

Other topics we will overview include: the theory of matroids and lattices, polyhedral properties of submodular functions, subdifferentials and superdifferentials, the Lovasz extension (i.e., the Choquet integral) of submodular functions and convex and concave extensions in general.

As for submodular optimization, we'll discuss submodular maximization algorithms in the unconstrained and constrained (i.e., knapsack, matroid, combinatorial, etc.) cases, the ever important greedy algorithm and its uses. For submodular minimization, we'll give a history of submodular minimization, including both numerical and combinatorial algorithms, computational properties of these algorithms, and descriptions of both known results and currently open problems in this area (as well as discuss both unconstrained and constrained cases).

Other problems we'll discuss include submodular cover problems, submodular flow problems, the principle partition of a submodular function and its variants. We'll see, for example, how submodularity can be used to solve non-submodular problems, for example difference of submodular programs, and submodular relaxation strategies.

# Homework

Homework must be done and submitted electronically via the following link https://canvas.uw.edu/courses/895956/assignments.# Lecture Slides

Lecture slides will be made available as they are being prepared --- they will probably appear soon before a given lecture, and they will be in PDF format (original source is latex). Note, that these slides are corrected after the lecture (and might also include some additional discussion we had during lecture). If you find bugs/typos in these slides, please email me. The slides are available as "presentation slides" and also in (mostly the same content) 2-up form for printing. After lecture, the marked up slides will appear under the "presented slides" column (and might include typo fixes, ink corrections, and other little notes/discussions/drawings that occured during class).Lec. # | Slides | 2-Up Slides | Lecture Date | Last Updated | Contents | Presented Slides | Video |

1 | 3/30/2014 | 3/30/2014 | Logistics, Motivation, Applications (video delayed, will post as soon as I can) | ||||

2 | 4/2/2014 | 4/2/2014 | Applications, Basic Definitions, Properties, Matrix Rank, Venn Visualization of Submodularity, | ||||

3 | 4/7/2014 | 4/7/2014 | More examples and properties (e.g., closure properties), and examples, proofs of equivalent definitions, spanning trees | ||||

4 | 4/9/2014 | 4/9/2014 | proofs of equivalent definitions, independence, start matroids | ||||

5 | 4/14/2014 | 4/14/2014 | Matroids, basic definitions and examples | ||||

6 | 4/16/2014 | 4/16/2014 | More on matroids, System of Distinct Reps, Transversals, Transversal Matroid, Matroid and representation | ||||

7 | 4/21/2014 | 4/21/2014 | Dual Matroids, other matroid properties, Combinatorial Geometries | ||||

8 | 4/23/2014 | 4/23/2014 | Combinatorial Geometries, matroids and greedy, Polyhedra, Matroid Polytopes, | ||||

9 | 4/28/2014 | 4/28/2014 | From matroid polytopes to polymatroids. | ||||

10 | 5/5/2014 | 5/5/2014 | Polymatroids and Submodularity | ||||

11 | 5/7/2014 | 7/7/2014 | More properties of polymatroids, SFM special cases | ||||

12 | 5/12/2014 | 5/12/2014 | polymatroid properties, extreme points polymatroids, sat, dep. | ||||

13 | 5/14/2014 | 5/14/2014 | sat, dep, exchange capacity, examples | ||||

14 | 5/18/2014 | 5/18/2014 | Lattice theory: partially ordered sets; lattices; distributive, modular, submodular, and boolean lattices; ideals and join irreducibles. (note, no video for this lecture). | ||||

15 | 5/19/2014 | 5/19/2014 | Supp, Base polytope, polymatroids and entropic Venn diagrams, exchange capacity, many more examples of polymatroids | ||||

16 | 5/21/2014 | 5/21/2014 | proof that minimum norm point yields min of submodular function, and the lattice of minimizers of a submodular function, Lovasz extension | ||||

17 | 5/28/2014 | 5/28/2014 | Lovasz extension, Choquet Integration, more properties/examples of Lovasz extension, convex minimization and SFM. | ||||

18 | 6/2/2014 | 6/2/2014 | Lovasz extension examples and structured convex norms, The Min-Norm Point Algorithm detailed. | ||||

19 | 6/4/2014 | 6/4/2014 | symmetric submodular function minimization, maximizing monotone submodular function w. card constraints. | ||||

20 | 6/12/2014 | 6/12/2014 | maximizing monotone submodular function w. other constraints, non-monotone maximization. | ||||

Lec. # | Slides | 2-Up Slides | Lecture Date | Last Updated | Contents | Presented Slides | Video |

# Discussion Board

You can post questions, discussion topics, or general information at this link.

# Relevant Books

There are many books available that discuss some the material that we are covering in this course. Some good books are listed below, but see the end of the lecture slides for books/papers that are relevant to each specific lecture. There are many books available that discuss some the material that we are covering in this course. See the end of the lecture slides for books and papers that are relevant to that specific lecture, and see lecture1.pdf for a list of general texts that are relevant to this class. Other books are below, but note that for some of the material, the best reference is the slides themselves.- Fujishige, "Submodular Functions and Optimization", 2005
- Narayanan, "Submodular Functions and Elecrical Networks", 1997
- Welsh, "Matroid Theory", 1975.
- Oxley, "Matroid Theory", 1992 (and 2011).
- Lawler, "Combinatorial Optimization: Networks and Matroids", 1976.
- Schrijver, "Combinatorial Optimization", 2003
- Gruenbaum, "Convex Polytopes, 2nd Ed", 2003.
- See lecture 1 slides for most relevant texts.

# Important Dates/Exceptions (also see either this or this academic calendar).

- Monday, 5/26, no class (holiday, Memorial day).
- Monday, 6/9, final presentations in class.

# Alternative Contact

If you must, you can send me or the TA anonymous email