## Table of Contents

This course is a broad introduction to computer vision. Topics include camera models, multi-view geometry, reconstruction, some low-level image processing, and high-level vision tasks like image classification and object detection.

Here is a rough outline of topics and the number of lectures spent on each:

- Image formation / projective geometry / lighting
*(3 lectures)* - Practical linear algebra
*(1 lecture)* - Image processing / descriptors
*(3 lectures)* - Image warping
*(2 lectures)* - Linear models + optimization
*(2 lectures)* - Neural networks
*(3 lectures)* - Applications of neural networks
*(3 lectures)* - Motion and flow
*(1 lectures)* - Single-view geometry
*(2 lectures)* - Multi-view geometry
*(3 lectures)* - Applications
*(2 lectures)*

Lectures will be delivered via Zoom, and recordings will be posted after each lecture. Lecture attendance is encouraged but not required.

**You are free to come to either live lecture**. While the registrar has set the course up as two sections, you are free to come to whichever lecture and work with whoever for homeworks and projects.

**Only enrolled students may join the live lectures.** The Zoom meeting ID and passcode are posted on Canvas; please don’t share them with other students. If you are on the waitlist or otherwise interested to follow along with the course, you can watch the recorded videos that we post after each lecture.

Zoom Etiquette:

- Leave your camera and mic off unless you are speaking
- You will be muted by default
- To ask a question, you can either:
- Type your question into the Zoom chat; we will actively monitor the chat and respond to questions
- Use the “raise hand” functionality in Zoom; then we will unmute you to ask your questino

Office hours are your time to ask questions one-on-one with course staff, and get clarification on concepts from the course. We encourage going to GSI office hours for implementation questions about the homework and faculty office hours for conceptual questions.

All office hours will be held virtually (tbd).

Some students may have had some prior exposure to computer vision, machine learning, or image processing, but none of these are required. We will assume you have a basic level of expertise in programming, computer science, mathematics (and specifically linear algebra).

Concretely, we will assume that you are familar with the following topics; they will not be reviewed in class:

**Programming**including algorithms and data structures at the level of EECS 281.**Python**: All course assignments will involve programming in Python. You should either have prior experience with Python, or be willing to quickly learn a new language. One homework will help teach and reinforce some of the concepts of numerical programming in python.

It will also be extremely helpful for you to have background in the following topics. We will provide some refreshers of the necessary concepts as they arise, but we may not go through a comprehensive treatment of these topics:

**Array Manipulation**: Homework assignments will involve manipulating multidimensional arrays using numpy and PyTorch. Some prior exposure to either of these frameworks will be useful, but if you’ve never used them before then the first homework assignment will help get you up to speed.**Linear Algebra**: In addition to basic matrix and vector operations, it will be good to know least squares, Eigen- and singular-value decompositions.**Calculus**: You should be comfortable with the chain rule, and taking gradients and partial derivatives of vector-valued functions.

Much of modern computer vision involves applying ideas from linear algebra to real-world data, and doing so often requires doing calculus with vector-valued functions. If you are unfamiliar with linear algebra or calculus, you should consider taking both: without these tools, you are likely to struggle with the course. Don’t worry if you have seen these topics before but are rusty; we will provide some math refreshers of the necessary topics. These refreshers will be insufficient as a first introduction to these topics, but should serve to remind you of concepts that you’ve forgotten or that weren’t covered in your previous courses.

There are no required textbooks. However the following optional books may be useful, and we will provide suggested reading from these books to accompany some lectures:

*Computer Vision: Algorithms and Applications*by Richard Szeliski. Available for free online.*Computer Vision: A Modern Approach (Second Edition)*by David Forsyth and Jean Ponce.*Elements of Statistical Learning*by Trevor Hastie, Robert Tibshirani, and Jerome Friedman. Available for free online (Warning: Direct PDF link).*Multiple View Geometry in Computer Vision (Second Edition)*by Richard Hartley and Andrew Zisserman. Available for free online through the UM Library (Login required).

Your grade will be based on:

**Homework Assignments (76%)**: There will be seven programming assignments over the semester. Homeworks 1-5 are each worth 12%; the first assignment (HW0) is shorter than the rest and is worth 6%; the last one is easier than the rest because you will be working on the project, and is worth 10%.**Course Project (24%)**: You will work together in groups of 3-5 students to produce a substantial project over the second half of the semester.

We will use the following base scheme to convert raw scores in the course into letter grades:

- A+: >= 98%
- A: >= 94%
- A-: >= 90%
- B+: >= 88%
- B: >= 84%
- B-: >= 80%
- C+: >= 78%
- C: >= 74%
- C-: >= 70%
- D+: >= 68%
- D: >= 64%
- D-: >= 60%
- F: < 60%

Depending on the distribution of raw scores at the end of the semester, we may apply a curve; however **the final curve will be no stricter than the base scheme**. For example, if you receive a raw score of 85% in the course, you will be guaranteed a letter grade of at least B; however you may receive a higher letter grade depending on the overall distribution of raw scores.

The primary way to communicate with the course staff is through Piazza. If you have questions about course concepts, need help with homework, or have questions about course logistics, you should **post on Piazza instead of emailing course staff directly**. Since Piazza is a shared discussion forum, asking and answering questions there can benefit other students as well.

However **do not post solutions to homework assignments on Piazza**. If you have questions about a particular piece of code you have written for an assignment, then you should make a private post on Piazza.

We will also use Piazza to make announcments about the course, such as the release of new homework assignments, details about the course project, or other logistical issues.

If you need to discuss a sensitive matter that you would prefer not be shared with the entire course staff then you can email the instructor directly. However the vast majority of questions about the course should be asked on Piazza.

#### Formatting and submission

We will not accept handwritten homework in any shape or form. You do not need to typeset your assignments in LaTeX or otherwise make them pretty. You can make a reasonable pdf in Google Docs, Microsoft Word, Open Office, or LaTex. If you’d like to try LaTex, try Overleaf.

Type it up in any format and submit a pdf to gradescope, marking the document up for where the questions are on gradescope. Stuff that doesn’t follow this simple formatting requirement will get a *0 that you can fix by resubmitting it in the right format.*

#### Collaboration / Honor Code

This is a class with a lot more freedom and self-teaching compared to intro-sequence courses. You are **strongly encouraged to work on the homework in teams** (and you have to for the project anyway). The role of the teaching staff is to provide you guidance and background knowledge, but the learning comes from you engaging with the material. It’s often useful to have peers along this journey.

**With Students**: You should never know the specific implementation details of anyone else’s homework or see their code. Working in teams and dispensing general advice about output or strategies is great and highly encouraged (e.g., “oh if the image is overly dark where it overlaps, you’re doing averaging wrong”). But pair-programming is forbidden, as is sitting next to someone and debugging their code. We believe that the overwhelming majority of UM students are ethical and honorable. However, to change incentives for those who are less honorable, we will run data mining software periodically on the submitted homework.**Consulting Outside Material**: In this class, you’re going to get some fairly vague specifications, potentially at the level of turn that 40-word description on a slide into a functioning program. You can, and should, turn to other documentation (the textbooks I suggest, other professors’ lecture notes, other professors’ slides, documentation from various libraries) What you may not do is read a bunch of actual code (pseudocode is fine however). If you come across code, close the window, but don’t worry about it.**Things you should never worry about**: (some adapted from here): reading documentation of publicly available libraries; clarifying ambiguities and mistakes in assignments, slides, handouts, or textbooks, or documentation; discussing the general material; helping with specific class-independent stuff like cryptic numpy errors; discussing the assignments to better understand what’s expected and general solution strategies; discussing the starter code; discussing general strategies for writing and debugging code.**If you’re panicking and think copying will solve your issues**: Don’t panic. We’ve tried to structure the incentives so that doing the right thing is rewarded. Submit it late (that’s why we have late days and a sliding late-penalty). Submit it half-done (that’s why we have partial credit). In the absolute worst case, don’t submit it, and take a 0. In a year or two, you will not be worried about doing poorly on any one particular homework.

#### Late Policy

Stuff happens, so there’s a generous late policy to account for the unexpected (e.g., food poisoning, a surprisingly difficult homework in this or another class, an interview, an alien abduction, etc, etc.).

**Penalties**: Late homework will be penalized at a rate of 1% per hour, rounding to the nearest hour, ties away from zero. For example, if the homework is due Monday at 11:59:59pm, homework submitted until Tuesday 12:30am will be accepted with no penalty. After 12:30, there is a 1% penalty. If you submit it Wednesday 6:30am, there is a 30% penalty. Penalties are applied additively: for instance, if you got a 90 and you were 20 hours late, it is now a 70.**Late Hours**: Because unexpected things happen, you will have 72 late hours over the semester that you can use on homework. This will be done automatically using times recorded in gradescope. I will not keep track of lateness for you.**Course Project**: The project proposal is subject to the same late policy as the homework assignments. The 72 free late hours are shared between the homework assignments and the proposal.**We will not accept late work for the final project report and showcase video**.

Given this lenient policy, please **do not ask course staff to waive late deductions or give extra late days**, except for truly exceptional circumstances that merit it.

#### Regrade Policy

In a large class, it is likely that we are going to make a few mistakes. We are committed to making mistakes right. You are free to submit regrade requests, especially if you believe there is a serious error (although not all regrade requests will be approved).

**Regrade Deadline**: Submit your regrade request within one week of the grades being released. This regrade deadline is firm. We will announce when grades are released. It is your responsibility to check it.**Small Judgment Calls**: Please do not quibble about small ambiguous decisions. Regrade requests for things that are off by ≤ 1 points on any particular problem, or which in total change the homework by ≤ 3 points will not be considered. If you are worried this will change your grade due to rounding, send an email to course staff titled detailing your request and, at the end of the semester, if the points could change your grade, then we’ll handle it then.**Administrative Fs**: If you submit handwritten homework or don’t mark the pages in gradescope, you will get a 0. These are subject to the same deadlines as other regrades. Typeset your homework or mark the pages in gradescope and we’ll happily grade them.

This is an opportunity to explore a topic in depth and should involve substantial work. This can be in implementation (e.g., implementing an existing algorithm), applications (e.g., applying computer vision to an existing problem), or research (e.g., trying something new in computer vision). Your project should amount to about one homeworks’ worth of work per person.

If you cannot find a partner, there will be a piazza discussion for finding project partners. All written work should be in CVPR format. Overall, please remember that we do not see your hard work, we only see the products you deliver.

Concretely, the course project has the following deliverables:

**Proposal (1 pages – 2%)**: The proposal should aim to explain what the problem is, why it’s feasible to solve in the given timeline, and how you plan to achieve it. **While this counts for few points, this is important to get right.** In particular, your project proposal should explain:

- What specific problem are you trying to solve? Are you appling computer vision, trying to extend computer vision, or implementing something? All are reasonable options. But please make it clear which you are trying to accomplish.
- How are you going to solve the problem? You don’t need to have everything already planned out, but you should have a clear idea. What parts are likely to work, and how can you still get work done if your plan fails.
- What do you need that you don’t currently have access to? This could be data, knowledge of certain machine learning techniques, or particular code.
- What do you expect as an outcome and how will you identify and quantify success?

**Final Project Report (4 pages, 14%)**: Please write up your project in the style of a CVPR paper (see samples here). Please remember to claim credit for everything you did and be clear what part of the project is yours and what part comes from others (e.g., did the data come from somewhere else? the initial implementation that you improved? a 3rd party library?). Your final report should explain:

- The problem that you are trying to solve. Describe it concretely and explain an intuition about why it should be solvable.
- Your approach. Please include diagrams – math is useful once the high-level idea is established, but diagrams will help us more quickly get up to speed.
- Your results. Please explain not only the results but also why you ran the experiments you did. What do the experiments actually test? Please include both quantitative results (i.e., measuring numerically what’s happening) as well as qualitative results (i.e., describing in words and images what’s happening).
- Details about your implementation. If you are building on the code of others, please indicate what is yours and what was others’ work.

**Virtual Showcase (8%)** Instead of a poster presentation, you will participate in a virtual poster session. Most of this will be you a ~4 minute video of your group members presenting your project. This doesn’t need to be fancy or overly produced; talking over a set of PowerPoint slides is perfectly acceptable.

The exact format and structure of the presentation can vary depending on the nature of your project. But you should clearly cover the following:

- What is the problem you were solving?
- Why is this problem important?
- How have people tried to solve this problem before? Give a brief summary of 1-3 key pieces of related work.
- How do you approach the problem? What is your key insight?
- What are your main experimental results? You don’t have to go into all details, but you should mention your main experiments.
- What challenges remain? If you were to keep working on this project, what would you do in the future?

**Doing well in class**: You are highly encouraged to read the course document on doing well: computer vision is a relatively difficult subject and requires simultaneous mastery of linear algebra, programming, and converting relatively vague specs into fairly specific code. If you approach the assignments from the wrong angle, they will be incredibly burdensome.**Classroom etiquette**: Above all, please avoid being disruptive in class, for both the instructor and for your classmates.**Accommodations**: If you think that you need an accommodation for a learning or other disability, please let me know. We will work with the Office of Service for Students with Disabilities to make proper accommodations.**Counseling Center**: Staff at the Counseling Center are trained to help you deal with a wide range of issues, such as how to deal with exam-related stress and other academic and nonacademic issues. Services are free and confidential and do not impact student records.**Academic Integrity**: All students in the class are: (a) presumed to be decent and honorable; (b) bound by the College of Engineering Honor Code; and (c) expected to read, understand, and follow the honor code. Information about this can be found here. You are highly encouraged to discuss the course materials, assignments, and projects. You should read the discussion in the homework policy of what constitutes acceptable collaboration.

### Schedule

Lecture | Date | Topic | Materials | Assignments |
---|---|---|---|---|

Lec. 1 | Mon, Aug. 30 | Introduction About the course | ps1 out (filtering) | |

Lec. 2 | Wed, Sep. 1 | Linear filtering Convolution | ||

Sec. 1 | Linear algebra | |||

Mon, Sep. 6 | No class - Labor Day | |||

Lec. 3 | Wed, Sep. 8 | More filtering Template matching | ||

Sec. 2 | Fourier Transform | |||

Lec. 4 | Mon, Sep. 13 | Frequency Image bases | ||

Lec. 5 | Wed, Sep. 15 | Image pyramids Resampling | ps1 due ps2 out (frequency) | |

Sec. 3 | Pyramids and frequency | |||

Lec. 6 | Mon, Sep. 20 | Machine learning Nearest neighbor | ||

Lec. 7 | Wed, Sep. 22 | Linear classifiers Logistic regression | ps2 due ps3 out (intro to ML) | |

Sec. 4 | Machine learning tutorial | |||

Lec. 8 | Mon, Sep. 27 | Neural networks Nonlinearities | ||

Lec. 9 | Wed, Sep. 29 | Optimization Computation graphs | ps3 due ps4 out (backprop) | |

Sec. 5 | Backpropagation tutorial | |||

Lec. 10 | Mon, Oct. 4 | Convolutional networks Convolution layers | ||

Lec. 11 | Wed, Oct. 6 | Scene understanding Scene recognition | ps4 due ps5 out (scene recognition) | |

Sec. 6 | PyTorch tutorial | |||

Lec. 12 | Mon, Oct. 11 | Object detection Sliding window | ||

Lec. 13 | Wed, Oct. 13 | Image synthesis Texture synthesis | ps5 due ps6 out (image synthesis) | |

Sec. 7 | Office hours + GANs | |||

Mon, Oct. 18 | No class - Fall Break | |||

Lec. 14 | Wed, Oct. 20 | Temporal models 3D convolutions | proposal info out | |

Sec. 8 | ||||

Lec. 15 | Mon, Oct. 25 | Project office hours | ||

Lec. 16 | Wed, Oct. 27 | Representation learning Transfer learning | ps6 due ps7 out (object detection) | |

Sec. 9 | ||||

Lec. 17 | Mon, Nov. 1 | Multimodal models Neural nets for other signals | ||

Lec. 18 | Wed, Nov. 3 | Motion Optical flow | ps7 due ps8 out (representation learning) | |

Sec. 10 | Project office hours | |||

Lec. 19 | Mon, Nov. 8 | Multi-view geometry Homogeneous coordinates | ||

Lec. 20 | Wed, Nov. 10 | Fitting geometric models Feature matching | ps8 due ps9 out (panorama stitching) | |

Sec. 10 | Project office hours | |||

Mon, Nov. 15 | No class | |||

Lec. 21 | Wed, Nov. 17 | Depth estimation Binocular stereo | ps9 due ps10 out (3D reconstruction) | |

Sec. 11 | Geometry tutorial | |||

Lec. 22 | Mon, Nov. 22 | Light, shading, and color Shape from shading | ||

Wed, Nov. 24 | No class - Thanksgiving | |||

Project office hours | ||||

Lec. 23 | Mon, Nov. 29 | Language Captioning | ||

Lec. 24 | Wed, Dec. 1 | Modern network architectures Transformers | ps10 due | |

Sec. 12 | ||||

Lec. 25 | Mon, Dec. 6 | Embodied vision Learning from demonstrations | ||

Lec. 26 | Wed, Dec. 8 | Bias and disinformation Datasets |

### Staff & Office Hours

**Office Hours**

Day | Time | Name | Location |
---|---|---|---|

Monday | 11:00am - 12:00pm | Xin Hu | Varies week to week. |

4:30pm - 5:00pm | Andrew Owens | EECS 3312 | |

Tuesday | 9:00am - 10:00am | Xin Hu | Zoom |

3:00pm - 4:00pm | Tony Pan | EECS 3312 | |

Wednesday | 11:00am - 12:00pm | Nikhil Devraj | EECS 3312 |

10:00am - 11:00am | Tony Pan | Zoom | |

Thursday | 11:30am - 12:30pm | Nikhil Devraj | Zoom |

Friday | 4:00pm - 5:00pm | Andrew Owens | Zoom |

Office hours will be offered in person and over Zoom, using the same link as lecture.

To keep track of office hours times, you may find it helpful to subscribe to the class calendar.

### Course information

EECS 442 is an advanced undergraduate-level computer vision class. Class topics include low-level vision, object recognition, motion, 3D reconstruction, basic signal processing, and deep learning. We'll also touch on very recent advances, including image synthesis, self-supervised learning, and embodied perception.

**Lectures:**

Lectures will take place Monday and Wednesday, 3:00 - 4:30pm. Attendance will not be required, but it is *highly encouraged*. There are multiple ways to participate:

- In person in Dow 1013. Due to space limits, this is only available to students who have registered for the in-person version of the class.
- Live-streamed on Zoom. Please see here for the link.
*Please do not share it*, so that we can avoid Zoom bombing. - We'll post lecture recordings online here.

**Discussion section:**

This class has three discussion sections. One of these sections will be simultaneously streamed over Zoom, using the same Zoom link as lecture. We'll post video recordings of this section, as well.

Time | Place |
---|---|

Thu 4:00-5:00pm | 1005 DOW |

Fri 10:30-11:30am | 1200 EECS |

Fri 12:30-1:30pm | G906 COOL + Zoom |

**Prerequisites:**

- This course puts a strong emphasis on mathematical methods. We'll cover a wide range of techniques in a short amount of time. Background in linear algebra is required. For a refresher, please see here. This material should mostly look familiar to you.
- This class will require a significant amount of programming. All programming will be completed in Python, using numerical libraries such as numpy, scipy, and PyTorch. The problem sets will be completed using Jupyter notebooks, generally using Google Colab. In some assignments, we'll give you starter code; in others, we'll ask you to write a large amount of code from scratch.

**Q&A:** This course has a Piazza forum, where you can ask public questions. If you cannot make your post public (e.g., due to revealing problem set solutions), please mark your post *private*, or come to office hours. Please note, however, that the course staff cannot provide help debugging code, and there is no guarantee that they'll be able to answer all questions — especially last-minute questions about the homework. We also greadly appreciate it when you respond to questions from other students! If you have an important question that you would prefer to discuss over email, you may email the course staff ([email protected]), or you can contact the instructor by email directly.

**Homework:** There will be homework assignments *approximately every week*. All programming assignments are to be completed in Python, using the starter code that we provide. Assignments will always be due at midnight (11:59pm) on the due date. The assignments will all be worth approximately equal amounts. Written problems will usually be submitted to Gradescope. You may be asked to annotate your pdf (e.g. by selecting your solution to each problem).

**Final project:** In lieu of a final exam, we'll have final project. This project will be completed in small groups during the last weeks of the class. The direction for this project is open-ended: you can either choose from a list of project ideas that we distribute, or you can propose a topic of your own. A short project proposal will be due approximately halfway through the course. During the final exam period, you'll turn in a final report and give a short presentation. You may use an ongoing research work for your final project, as long it meets the requirements.

**Textbook:** There are no required textbooks to purchase. As an experiment, we'll be using a new *draft* version of the online book:

The following textbooks may also be useful as references:

- Goodfellow, Bengio, Courville.
*Deep Learning*. (available for free online) - Forsyth and Ponce.
*Computer Vision: A Modern Approach*. - Hartley and Zisserman.
*Multiple View Geometry in Computer Vision*.

**Acknowledgements:** This course draws heavily from MIT's 6.869: Advances in Computer Vision by Antonio Torralba, William Freeman, and Phillip Isola. It also includes lecture slides from other researchers, including Svetlana Lazebnik, Alexei Efros, David Fouhey, and Noah Snavely (please see acknowledgments in the lecture slides).

**Late policy:** You'll have **7 late days** to use over the course of the semester. Each time you use one, you may submit a homework assignment one day late without penalty. You are allowed to use multiple late days on a single assignment. For example, you can use all of your days at once to turn in one assignment a week late. You do *not* need to notify us when you use a late day; we'll deduct it automatically. If you run out of late days and still submit late, your assignment will be penalized at a rate of 30% per day. If you edit your assignment after the deadline, this will count as a late submission, and we'll use the revision time as the date of submission (after a short grace period of a few minutes). We will not provide additional late time, except under exceptional circumstances, and for these we'll require documentation (e.g., a doctor's note). Please note that the late days are provided to help you deal with minor setbacks, such as routine illness or injury, paper deadlines, interviews, and computer problems; these do not generally qualify for an additional extension.

Please note that, due to the number of late days available, there will be a long (2+ week) lag between submission and grading — we'll need to wait for the late submissions to arrive before we can finish.

**Regrade requests:** If you think there was a grading error, you'll have **9 days** to submit a regrade request, using Gradescope. This will be a strict deadline, even for significant mistakes such as missing grades, so please look over your graded assignments!

**Support:** The counseling and psychological services center (CAPS) provides support for a variety of issues, including mental health and stress.

**Grading:**

- Grades will be computed as follows, with all homeworks equally weighted:
Homework 70% Final project 30% - We'll use these approximate grade thresholds:

These areA+ Curved A 92% A- 90% B+ 88% B 82% B- 80% C+ 78% C 72% C- 70% *lower bounds*on letter score grades. For example, if you get an 81%, you will be guaranteed to get a B-. We may gently curve the class*up*, in a way that would only*improve*your letter grade: e.g. after the curve, an 81%*might*round up to a B, but it would not round down to a C+.

**Academic integrity:** While you are encouraged to discuss homework assignments with other students, *your programming work must be completed individually*. You must also write up your solution on your own. You may not search for solutions online, or to use existing implementations of the algorithms in the assignments. Please see the Michigan engineering honor code for more information.

## EECS 442: Computer Vision (Fall 2019)

**Instructor**: David Fouhey (fouhey)

**GSI**: Mohamed El Banani (mbanani)

**IAs**: Zhengyuan Dong (dongzy), Shengyu Feng (shengyuf), Qichen Fu (fuqichen)

**Lecture**: Monday/Wednesday 5:30PM - 7:00PM, 1571 GGBL

**Discussions**:

Thursday 4:00PM - 5:00PM, 1018 DOW

Friday 10:30AM - 11:30AM, 1200 EECS

Friday 12:30PM - 1:30PM, 1012 FXB

**Office Hours**:

Tuesday 10:30-Noon, Fouhey, BBB 3777

Tuesday 3:00-4:30PM, Dong, BBB 4901

Wednesday 9:00AM - 10:30AM, Fu, BBB Learning Center

Wednesday 3:00PM - 4:30PM, El Banani, BBB 4901

Thursday 1:30PM - 3:00PM, Feng, BBB Learning Center

### Waitlist

I will make every effort to ensure that as many people as possible are moved off of the waitlist and into the course. **I will not re-order the waitlist or ensure that particular people get moved off the waitlist – this leads to me making arbitrary decisions based on whether people emailed me or not.**

### Overview

This is an introduction to computer vision. Topics include: camera models, multi-view geometry, reconstruction, some low-level image processing, and high-level vision problems like object and scene recognition.

This course will be taught assuming

computer science knowledge at the level of EECS 281 (data structures) and corresponding programming ability;

the ability to program in Python, or if not, the ability to learn to program in a new language quickly.

It would be **extremely** helpful to know the following topics (i.e., we will provide a brief refresher when introducing it, but will not explain the details):

*Programming*: Some knowledge of numpy would be useful.*Math*: Computer vision is basically linear algebra + data. You will need knowledge of basic linear algebra and calculus.**If you are unfamiliar with linear algebra or basic calculus, please consider taking both: without these tools, you will likely be lost for large parts of the course.**If you are rusty, do not worry but do make an effort to refresh your memory of both at the start of the course.

In particular, in addition to knowing basic matrix operations, it would be good to know least-squares, Eigen- and singular-value decompositions, some notion of basic continuous optimization, partial derivatives, the chain rule.

I will include two math refesher lectures. They are, however, meant to remind people of the linear algebra they forgot and to plug gaps that just aren't covered. However, this is a refresher course, not a first course. It is also not meant to be the**only**refresher if you are rusty with linear algebra.

### Syllabus and Schedule

Here is: a tentative syllabus and a tentative schedule.

What's my grade? Here is a calculator for doing the math. The late penalties can be applied in any order since it's linear, so the calculator just does it greedily (but not using the points to rescue e.g., an assignment that's a 0).

### Credit for materials

I am extremely grateful to the many researchers who have made their slides and course materials available. Please feel to re-use any of my materials while crediting appropriately and making sure original attributions to these generous researchers is preserved.

## EECS 442: Computer Vision (Winter 2019)

**Instructor**: David Fouhey (fouhey)

**GSI**: Richard Higgins (relh)

**IAs**: Linyi Jin (jinlinyi), Shengyi Qian (syqian), Yi Wen (wennyi)

**Lecture**: Tuesday/Thursday 10:30AM – 12:00 Noon, 1571 GGBL

**Discussions**:

Monday 12:30PM - 1:30PM, 2246 SRB (previously 1200 EECS)

Wednesday 5PM – 6PM, 1571 GGBL

**Office Hours**:

Tuesday 3:00-4:30pm, BBB Learning Center

Thursday 2:30-4:00pm, BBB Learning Center

Friday 10:30-12:00 Noon, BBB 3777

### Overview

This is an introduction to computer vision. Topics include: camera models, multi-view geometry, reconstruction, some low-level image processing, and high-level vision problems like object and scene recognition.

This course will be taught assuming

computer science knowledge at the level of EECS 281 (data structures) and corresponding programming ability;

the ability to program in Python, or if not, the ability to learn to program in a new language quickly.

It would be **extremely** helpful to know the following topics (i.e., we will provide a brief refresher when introducing it, but will not explain the details):

*Programming*: Some knowledge of numpy would be useful.*Math*: Knowledge of basic linear algebra, calculus. Learning and geometry depend on both.**If you are unfamiliar with linear algebra or basic calculus, please consider taking both: without these tools, you will likely be lost for large parts of the course.**If you are rusty, do not worry but do make an effort to refresh your memory of both at the start of the course. In particular, in addition to knowing basic matrix operations, it would be good to know least-squares, Eigen- and singular-value decompositions, some notion of basic continuous optimization, partial derivatives, the chain rule.

### Syllabus and Schedule

Here is: a tentative syllabus and a tentative schedule.

What's my homework grade?

*After a little thinking, we'll have the late days count as fractional as well, since this makes the calculation more clear and hopefully reduces any confusion*

Here is a calculator for doing the math. The late penalties can be applied in any order since it's linear, so the calculator just does it greedily.

### Waitlist

I will make every effort to ensure that as many people as possible are moved off of the waitlist and into the course. I cannot re-order the waitlist or ensure that particular people get moved off the waitlist.

### Credit for materials

I am extremely grateful to the many researchers who have made their slides and course materials available. Please feel to re-use any of my materials while crediting appropriately and making sure original attributions to these generous researchers is preserved.

## 442 umich eecs

S is Computer Vision: Algorithms and Applications by Richard Szeliski, which can be found here.

H&Z is Multiple View Geometry in Computer Vision by Richard Hartley and Andrew Zisserman, which is available from the UM Library (Login required).

ESL is Elements of Statistical Learning by Hastie, Tibshirani, and Friedman, which can be found here

All slides for this course were adapted from the Fall 2019 iteration of this course, taught by David Fouhey.

January 9

**Introduction**

Computer vision overview

Course logistics [slides (pdf)]

[slides (pptx)] Lecture 2Tuesday

January 14

**Cameras I**

Pinhole camera

Homogenous coordinates

Intrinsic / Extrinsic matrices [slides (pdf)]

[slides (pptx)]

[Reading: S 2.1]

[Reading: H&Z 2, 6] Lecture 3Thursday

January 16

**Cameras II**

Thin lens model

Depth of field [slides (pdf)]

[slides (pptx)] Lecture 4Tuesday

January 21

**Light and Shading**

Human vision

Color vision

Color spaces

Diffuse and Specular Reflection [slides (pdf)]

[slides (pptx)]

[Reading: S2.2, S2.3] Lecture 5Thursday

January 23

**Math Review I**

Floating point numbers

Vectors, Matrices [slides (pdf)]

[slides (pptx)] Lecture 6Tuesday

January 28

**Math Review II**

Matrices, Vectorization

Linear Algebra [slides (pdf)]

[slides (pptx)] HW0 DueWednesday

January 29

**Homework 0 Due**[HW0] Lecture 7Thursday

January 30

**Math III + Image Filtering**

Eigenvalues, Eigenvectors

Least Squares

Singular Values

Derivatives

Linear Filters [slides (pdf)]

[slides (pptx)]

[Reading: S3.2] Lecture 8Tuesday

February 4

**Image Filtering II**

Blur and sharpening filters

Separable filters

Image Gradients [slides (pdf)]

[slides (pptx)]

[Reading: S3.2] HW1 DueWednesday

February 5

**Homework 1 Due**[HW1] Lecture 9Thursday

February 6

**Edge and Corner Detection**

Edge detection

Gaussian Derivative filter

Harris Corner detector [slides (pdf)]

[slides (pptx)]

[Reading: S4.1] Lecture 10Tuesday

February 11

**Image Descriptors**

Scale-space pyramid

Laplacian blob detectors

SIFT descriptors [slides (pdf)]

[slides (pptx)]

[Reading: S4.1] Lecture 11Thursday

February 13

**Transformations I**

Linear Regression

Total Linear Regression

RANSAC

Hough Transform [slides (pdf)]

[slides (pptx)]

[Reading: S2.1, S6] Lecture 12Tuesday

February 18

**Transformations II**

Affine, Projective transforms

Fitting transforms [slides (pdf)]

[slides (pptx)]

[Reading: S2.1, S6] HW2 DueWednesday

February 19

**Homework 2 Due**[HW2] Lecture 13Thursday

February 20

**Intro to Machine Learning**

Image warping / blending

Supervised vs Unsupervised learning

Train / Test splits

Linear Regression

Regularization [slides (pdf)]

[slides (pptx)] Lecture 14Tuesday

February 25

**Linear Models**

Cross-Validation

K-Nearest Neighbors

SVM loss

Cross-Entropy loss [slides (pdf)]

[slides (pptx)]

[CS231n Linear Classification] Lecture 15Thursday

February 27

**Optimization**

Stochastic Gradient Descent

SGD + Momentum [slides (pdf)]

[slides (pptx)]

[CS231n Optimization] Tuesday

March 3

**Spring Break**HW3 DueWednesday

March 4

**Homework 3 Due**[HW3] Thursday

March 5

**Spring Break**Lecture 16Tuesday

March 10

**Neural Networks**

Overfitting / Underfitting

Bias / Variance tradeoff

Fully-connected neural networks

Biological neurons [slides (pdf)]

[slides (pptx)]

[CS231n Neural Networks] Wednesday

March 11

**Project Proposal Due**Thursday

March 12

**No Class**Lecture 17Tuesday

March 17

**Backpropagation**

Computational Graphs

Backpropagation

Matrix multiplication example [video (from EECS 498/598)]

[slides (from EECS 498/598)]

[231n Backpropagation]

[Backprop for Matrix Multiply]

[Olah on Backprop]

[Nielsen on Backprop] Lecture 18Thursday

March 19

**Convolutional Networks**

Convolution

Pooling

Batch Normalization [video (from EECS 498/598)]

[slides (from EECS 498/598)]

[231n ConvNets]

[Goodfellow, Chapter 9] Lecture 19Tuesday

March 24

**CNN Architectures**

AlexNet, VGG, ResNet

Size vs Accuracy

Neural Architecture Search [video (from EECS 498/598)]

[slides (from EECS 498/598)]

[AlexNet paper]

[VGG paper]

[GoogLeNet paper]

[ResNet paper] Lecture 20Thursday

March 26

**Training Neural Networks I**

Activation Functions

Data preprocessing

Weight initialization

Data Augmentation

Regularization [video (from EECS 498/598)]

[slides (from EECS 498/598)]

[231n Training I] Lecture 21Tuesday

March 31

**Training Neural Networks II**

Learning rate schedules

Hyperparameter optimization

Model ensembles

Transfer learning

Large-batch training [video (EECS 498/598)]

[slides (from EECS 498/598)]

[231n Training II]

[Karpathy "Recipe for Training"] HW4 DueWednesday

April 1

**Homework 4 Due**[HW4] Lecture 22Thursday

April 2

**Object Detection**

Single-Stage detectors

Two-Stage detectors [video (from EECS 498/598)]

[slides (from EECS 498/598)] Lecture 23Tuesday

April 7

**Image Segmentation**

Semantic segmentation

Intance segmentation

Keypoint estimation [video (from EECS 498/598)]

[slides (from EECS 498/598)] Wednesday

April 8

**Progress Report Due**Lecture 24Thursday

April 9

**3D Shape Prediction**

3D shape representations

Depth estimation

3D shape prediction

Voxels, Pointclouds, SDFs, Meshes [video (from EECS 498/598)]

[slides (from EECS 498/598)] Lecture 25Tuesday

April 14

**Video Models**

Video classification

Early / Late fusion

3D CNNs

Two-stream networks [video (from EECS 498/598)]

[slides (from EECS 498/598)] HW5 DueWednesday

April 15

**Homework 5 Due**[HW5] Wednesday

April 22

**Final Report Due**

I hear a sob. -It hurts very much, I know my girl. forgive me the fool. I don't want to hurt. sorry.

### You will also be interested:

- America healthways education
- Xfactor best auditions
- War robot news
- Orbi satellite ethernet
- Pocket sheer curtains
- Honda classic bikes
- Minecraft tattoo simple
- Nails in davie
- Dark matter cammo
- Remove definition synonyms
- Maine coon newborn

I will love you even more after that. And in the evening Lyubov and I had the best sex in the world. Thanks to me, thanks to my wife and special thanks to our friend Andrey. Even now, many years later, we live in a happy marriage.

**248**249 250