---

Deadline

September 28, 2021 at 11:59:59 PM. Only one submission per group.

What you need to do?

Since most of the projects you pitched in ASN1 involves some form of robot perception: interpretting data from a robot's sensors, in this assignment you'll work on data from a color camera. The reason for the choice of this sensor is because of its cost-effectiveness and the ubiquitous nature of the sensors. Moreover, one can get a lot of things done with images/video from a color camera.

Your task is to detect and draw oriented bounding boxes (the length axis of the bounding box is parallel to the vertical part of the barrel) around the orange cylindrical traffic barrel which acts as our makeshift replacement for a disaster survivor. Note that, this barrel has two horizontal white stripes around on body. There are other orange cylinders (let's call them non-survivors and they don't have white bars) in the environment that should not be detected. More than one survivor can be present at a time and all of them need to be detected: Remember, leave no survivors behind.

Step 0: Prep your System

Install OpenCV (version >= 3.3) and any other additional packages you'll need to complete this project. Feel free to use numpy and scipy among other packages. The project has to be completed in Python (version >= 3.6). It is recommended to use a machine with a Linux installation, specifically Ubuntu 20.04 either as a dual-boot or a virtual machine like Virtual Box. Alternatively, windows users can use Windows Subsystem for Linux. For MacOS users, I would highly recommend using Virtual Box. Alternatively, check out this thread on Github or this tutorial or run it in your browser. Note that I have not tested either of the docker methods on Mac, but the Virtual Box method works great.

Step 1: Download Data and Understand basic concepts

The data is given in the form of a RGB color video in .mp4 format and can be downloaded from here and is shown below for preview. Feel free to convert the video to frames for debugging. Read and visualize the video to understand how the data looks. A written and extended version of today's class (21 Sept 2021) can be found here. You don't have to read everything from this, treat this as a refresher to today's class with a few advanced concepts.


Video in which barrel has to be detected.

Step 2: Make your own survivor detector

Come up with any method to detect survivors given you have no constraint on computing time or memory requirement. Feel free to use something as basic as color segmentation to pattern matching or deep learning. A sample output for somne simple frames are shown in Fig. 1 and for complex frames are shown in Fig. 2.


Fig. 1: Sample output on frames from the video for easy samples.


Fig. 2: Sample output on frames from the video for a hard sample.

Step 3: Make your results better (Optional for Extra credit of upto 25%)

Now that you have detected survivors in every frame of the video, you can be creative and fuse readings from multiple frames by filtering them.

Using third-party code

You are allowed to use any code available on the internet with an appropriate citation. However, you are not allowed to submit any other classmate's work as your own. Remember collabotation amongst classmates is encouraged but plagiarism is strictly prohibited.

What you need to submit?

A report in double column IEEE format (maximum of 6 pages) in LaTeX per group along with a .mp4 video all zipped together into one .zip file. Your report should be of this quality.

Your report should include the following:

• Names of the team members and the team
• Detailed description of your approach
• Failure cases or shortcomings of your approach
• Sample image outputs (bounding boxes overlaid)
• Appropriate references for any methods or code used (you can use .bib files to make your life easier)
• Distribution of work amongst team-members

You can use any LaTeX tool you like, I would recommend Overleaf as it is free and you can use it to collaborate online. If you are new to LaTeX, you can learn from this awesome tutorial from Overleaf. Alternatively, if you want to work locally, you can use any of these editors. The LaTeX template can be found here. The submission should be made through ELMS with the name ASN2.

Your .mp4 video is the oriented bounding box(es) output overlaid on the original video.