EE512A - Advanced Inference in Graphical Models - Fall Quarter 2011

Last updated: $Id: index.html,v 1.20 2011-10-15 02:01:54 bilmes Exp $

This page is located at http://j.ee.washington.edu/~bilmes/classes/ee512a_fall_2011/

Instructor:

Prof. Jeff A. Bilmes --- Email me
Office: 418 EE/CS Bldg., +1 206 221 5236
Office hours: Wednesdays, 12:30-1:30, EEB-418

Announcements

Information

Description: This course will cover certain aspects of advanced inference techniques in graphical models.

We will briefly review exact probabilistic inference (which essentially boils down to inference on trees, and includes methods such as junction trees, exact belief propagation and message passing in its various forms, optimal graph triangulations, the elimination family of algorithms, the generalized distributed law (GDL), and its relation to dynamic programming). We'll also cover NP hardness results for inference, NP hardness results for finding the best form, and the inapproximability of inference in graphical models (which seems bad).

Next we'll move to the optimistic aspect of the course, where we will cover two general classes of approximate inference methods.

The first general class of techniques is that of exponential models and variational methods. We will follow the 2008 book by Wainwright and Jordan pretty closely and which you can get from the link above if you are at an educational institution (although I urge you to purchase a copy as it is not too expensive). Of particular interest will be the polyhedral approaches (e.g., the marginal polytope), and the linear programming relaxation methods mentioned in this book.

The second is a class of inference algorithms (more precisely, algorithms for finding the most probable configuration of a set of random variables, or called MPE (most probable explanation) or Viterbi inference) that have become popular in the compute vision community. While we will not dwell on the computer vision applications, we will most abstract the algorithms for use in general graphical models and we might draw from vision examples to show results. These include algorithms that are used when the tree-width of the model very high and that, even in the high tree-width case, can sometimes provide exact MPE solutions in low-order polynomial time. This includes many of the graph cut methods, including higher order approximations, and methods and techniques when the variables are not binary (including alpha-beta swaps, alpha expansions, fusion moves, and other recent more sophisticated and energy aware "move making" algorithms to that can improved efficiency, such as various forms of "expansion" moves). We will also discuss what to do with global potential functions (i.e., ones where a factor might involve many variables).

We will be covering papers of the form that you can find in a new text on inference in Markov random fields for computer vision although, again, we we will talk about the methods more generically (the methods indeed are much more widely applicable than just to problems in computer vision, yet there has been much innovation in graphical model inference in the computer vision community recently. We will also draw from papers that have recently appeared (the links will be posted here on this web page as the class proceeds).

Course Format: Two two-hour lectures per week (WF 10:30-12:20 Mech. Engr (MEB)-242 per week.

Prerequisites: ideally knowledge in probability, statistics, convex optimization, and some combinatorial optimization although these will be reviewed as necessary. It would be useful to have some basic knowledge of graphical models as we'll review the basics fairly rapidly. The course is open to students in all UW departments. If you are in doubt about taking this course, please talk to me in class or office hours.

Texts: see above.

Grades and Assignments: Grades will be based on a combination of a final project (35%), homeworks (35%), and the take home midterm exam (30%).

There will be between 2-5 homeworks during the quarter. Some of the homeworks will include project proposals, project proposal revisions, and project progress reports.

Final project: The final project will consist of a 4-page paper (conference style) and a final project presentation. The project must involve using or dealing mathematically with graphical models in some way or another. Please contact me and/or stop by office hours early in the quarter to discuss project ideas.

Homework

Homework must be done and submitted electronically via the following link here.
  1. Homework 1 Due next electronically Friday, Oct 28th, 11:45pm. Solutions are here
  2. Homework 2 Due next electronically Wednesday, Nov 16th, 10:00am. Solutions are here

Lecture Slides

Lecture slides will be made available as they are being prepared --- they will probably appear here right before a given lecture, and they will be in PDF format (original source is latex). Note, that these slides are corrected after the lecture (and might also include some additional discussion we had during lecture). If you find bugs/typos in these slides, please email me. The slides are available as "slides" and also in 2-up form for printing.
Lec. # Slides Print Lecture Date Last Updated Contents
1 pdf pdf 9/28/11 10/3/11 Introduction, Intuition, about, graphical models as families.
2 pdf pdf 10/5/11 10/5/11 inference on trees
3 pdf pdf 10/7/11 10/7/11 trees, elimination on non-trees
4 pdf pdf 10/12/11 10/12/11 non-trees, triangulated graphs
5 pdf pdf 10/14/11 10/24/11 non-trees as trees, triangulated, decomposable
6 pdf pdf 10/19/11 10/24/11 decomposable, junction tree
7 pdf pdf 10/21/11 10/21/11 junction tree, intersection graphs, JT inference
8 pdf pdf 10/26/11 10/26/11 JT, message passing, overview conditioning
9 pdf pdf 10/28/11 10/28/11 time-space tradeoff, conditioning, approximation, LBP
10 pdf pdf 11/2/11 11/2/11 LBP, exp. moels
11 pdf pdf 11/4/11 11/4/11 exp. models, marg poly
12 pdf pdf 11/9/11 11/9/11 marg. poly, Bethe approx
13 pdf pdf 11/16/11 11/22/11 Bethe approx, loop series
14 pdf pdf 11/18/11 11/18/11 loop series, Kikuchi
15 pdf pdf 11/23/11 11/23/11 EP
16 pdf pdf 11/30/11 11/30/11 mean field
17 pdf pdf 12/2/11 12/6/11 convexified, tree reweighted
18 pdf pdf 12/7/11 12/7/11 tree reweighted, variational MPE
19 pdf pdf 12/9/11 12/9/11 MPE, LP relaxation
Lec. # Slides Print Lecture Date Last Updated Contents

Actually Presented Lecture Slides, Fall, 2011

Lecture slides that were presented in class, along with all of the bugs and typos, my ink corrections of (perhaps some) of the bugs and typos, and any other little notes/discussions/drawings I drew on the slides during class. The above slides often contain more material than these as any discussions during class were added to the above after class. On the other hand, there might be a few hand-drawn figures in the below that I have not yet added to the above. Note: not all PDF readers can see the annotations in these slides (e.g., at least the Safari embedded reader on the iphone/ipad doesn't see them) --- the annotations were done with Adobe acrobat.
  1. Lecture 1 from 9/28/11.
  2. Lecture 2 from 10/5/11.
  3. Lecture 3 from 10/7/11.
  4. Lecture 4 from 10/12/11.
  5. Lecture 5 from 10/14/11.
  6. Lecture 6 from 10/19/11.
  7. Lecture 7 from 10/21/11.
  8. Lecture 8 from 10/26/11.
  9. Lecture 9 from 10/28/11.
  10. Lecture 10 from 11/2/11.
  11. Lecture 11 from 11/4/11.
  12. Lecture 12 from 11/9/11.
  13. Lecture 13 from 11/16/11.
  14. Lecture 14 from 11/18/11.
  15. Lecture 15 from 11/23/11.
  16. Lecture 16 from 11/30/11.
  17. Lecture 17 from 12/2/11.
  18. Lecture 18 from 12/7/11.
  19. Lecture 19 from 12/9/11.

Discussion Board

You can post questions, discussion topics, or general information at this link.

Relevant Books

There are many books available that discuss some the material that we are covering in this course. Some good books are listed below, but see the end of the lecture slides for books/papers that are relevant to each specific lecture. The first two books we will be covering closely this term. Other very good books that might be useful to you are included below.

Important Dates/Exceptions

Alternative Contact

If you must, you can send me anonymous email

Go to my home page Email me