---

Deadline

Mar 21, 2022 at 11:59:59 PM (Individual Submissions).

What you need to do?

In your last assignment, you detected an orange barrel. Congratulations! However, most real robots need to detect something more sophisticated. Let us go with the example of a self-driving car. Such a car needs to detect traffic barrels to avoid road closures. Particularly, the traffic barrel is an orange colored cylinder (same color as the one from previous assignments) but has two horizontal white stripes around on body (as shown in Fig. 1). To make the problem more realistic, there are also orange barrels in the video from the previous assignment. Your task is to only detect the traffic barrel. Note that, in the real-world, there can be more than one traffic barrel or orange barrel in the same image/video frame.

Step 1: Download Data

The data is given in the form of a .zip file that contains an input color video called Vid.mp4 along with the masks video called Masks.mp4 where the barrel exists just like the last assignment. The videos are in .mp4 format. The data can be downloaded from here. Feel free to convert the video to frames for debugging. Read and visualize the video to understand how the data looks.

Step 2: Make your own traffic barrel detector using Template Matching

To detect a traffic barrel we need to match patterns, for example, "a vertical rectangular orange blob with three horizontal white stripes". This can get really complicated very quickly as the patterns can be quite complex. Luckily, experts in computer vision remarked that we can use a concept called template matching. Here, we create a template or an image of what we want to detect. We then use this template and slide over the input image (as in 2D convolution) to compare the template and patch of input image under the template image to "see" if the object we desire is present. Follow the following steps to complete this assignment:

• Prepare a template as a grayscale image, you can be creative on how you obtain this. Simple example is to crop the barrel manually or use a mask given and save the image. Remember, you need ONLY one template image in grayscale.
• You work with grayscale images in this assignment as OpenCV's template matching function works only on grayscale images. Read the video (you need to perform this barrel detection on each frame of the video by looping over, just like in ASN3), convert the current video frame to grayscale.
• Follow OpenCV's Template Matching tutorial to use the template matching funcions to build your own barrel detector.
• Overlay the bounding box of any color on the input video to visualize your detection results. You will need to save this video and submit it.
• Experiment with different template matching methods, talk about what works the best and why? in your document (You need to write code to experiment here.)
• Think and write about how one can use color in a template matching method (You do not need to any write code for this part.)

A sample output for a single frame with bounding box oyerlaid is shown in Fig. 1 and bounding box overlaid on video is shown in Fig. 2. You are free to use any third-party or built-in code for this assignment with the appropriate citation.


Fig. 1: Sample input RGB frame from the video with bounding box overlaid.


Fig. 2: "Reasonable" RGB video that shows input with bounding box overlaid.

Using third-party code

You are allowed to use any code available on the internet with an appropriate citation. However, you are not allowed to submit any other classmate's work as your own. Remember collabotation amongst classmates is encouraged but plagiarism is strictly prohibited.

What you need to submit?

A .zip file that has a document (feel free to use Word or Google Docs or any other software for this) converted to PDF with the following answers and required codes as mentioned below:

• Code (as .py python files and inside the document as a screenshot or copy pasted text).
• Comments on your experiment with different template matching methods, talk about what works the best and why? (You need to write code to experiment here.)
• Think and write about how one can use color in a template matching method (You do not need to any write code for this part.)
• Output RGB Video with bounding box overlaid for detection as shown in Fig. 2 in .mp4 format named as Output.mp4.
• Challenges you faced in solving this assignment along with lessons learned.
• Positive or Negative feedback about this assignment if you have any.

IMPORTANT NOTE: The submissions are made through ELMS with the name ASN4_DirID.pdf. Here, DirID is your directory ID, i.e., the first part of your terpmail email address. For e.g., if your terpmail email address is ABCD@terpmail.umd.edu, then your DirID is ABCD. Keep your submissions professional, grammatically correct without spelling mistakes. Do not use slangs and chat shorthands on your submissions. You'll get 25% grade penalty for not following the submission guidelines.