Assignment 5: Feature Detection and Matching

Introduction

In this homework assignment, we will focus on putting to practice the code we have seen in lecture for feature detection and matching. This assignment will have two parts. The first part of the assignment will involve us capturing five different images, and the second part will involve us trying to do feature detection between the template and the remaining four images. We would like to stress that we want you to capture these images with your camera, do not take them from the web.

Before we get started, download the assignment files under Resources > Assignments > Assignment 5 that contains the files you will be working with.

Once you have extracted the file above, you are ready to get started.

In the zipped file you can see the sample images which are provided for you in the images/source/sample directory. Every time you run the code, it will look inside each folder under images/source/, and attempt to find a folder that has images with filenames that contain '_1' and '_2'. Once it finds a folder that contains two images with those characters in the name, it will apply a feature detection and matching procedure to them, and save the output to images/output/.

As with previous assignments, running assignment5_test.py directly will apply a unit test to your code and print out helpful feedback. It will now also output the image result if your code is working. You can use this to debug your functions.

Part 1: Taking five images with the same subject.

First, find a subject. A stuffed animal, or small object (things with more texture will work better) that you wish to use. Take a picture of the object, and crop it so it is just the object.

  1. original image : This will be your template. If you were 'finding Waldo', this is your picture of just Waldo (usually best to crop it to remove some background noise). In this occassion, I decided to play Finding Buzz, so this is my example template:
  2. sample: This will be a sample image with your subject. Make sure your subject is standing very similar to how your template was taken. This will be your proof of concept before we begin to make changes to the image. This is my example sample image:
  3. lighting: Take a picture of a scene with your subject, but make sure the lighting has changed so your object is visually different (it can be a bit darker or brighter, or if you have colored lights play with that). This example shows a Buzz in a darker setting.
  4. rotation: Take a picture of a scene with your subject, but this time rotate the subject. Put it on its side, or in a silly way that makes the object be rotated. This example shows a Buzz lying upside down. Note: You may not just rotate one of your other images computationally, we want you to capture a new image with your object rotated.
  5. scale: Take a picture of a scene with your subject, but this time drastically change the size of your object. It can be a closeup, or at a distance (granted if you go too far and your object is too small SIFT will probably not work). This example shows a closeup of Buzz.

Note: You'll probably take multiple pictures for each of the above categories to find the image that gives you the best results for feature detection, continue reading the entire assignment before you start this part.

Take a look at the images/source/ folder. I have included the above images for testing, each in their own folder. You should replace them with your own images since you will need to output your results for the PDF.

Part 2: Feature Detection and Matching

findMatchesBetweenImages

For this assignment you will complete one function. We strongly suggest that you look at the lectures for feature detection and matching in order to tackle the code as the implementation is fairly straightforward to what we have done in lecture.

The code documentation in assignment5.py further elaborates on what you have to do.

We also include some code that is commented out to quickly output your result. You may run this or simply run the code that is available as part of assignment5_test.py which we heavily recommend you use to test your function.

Note: We do not expect your feature detection to work perfectly, it should have some matches that line up correctly (if they all do, awesome!) and some matches that are erroneous.

The Writeup

This is what we want you to do for the PDF.

  1. Demonstrate your five input images (the template, and the sample, lighting, scale, rotation images)
  2. Explain how you implemented your function using OpenCV's libraries.

Then, for each of the four images (sample, lighting, scale, rotation), explain the following:

What to turn in: