0.2 Approaches

Approach i: template matching only

In the template matching algorithm, a sign template is correlated with a larger image that may or may not include the sign. We used the MATLAB function normxcorr2 to correlate the template and image.  Though template matching works perfectly when a smaller reference image is directly cropped from the larger image, in practice we would not have the cropped image.  Thus, our first task was to choose a general reference image.  After finding a good reference image, our second task was to write a program that would find the correlation for many image:reference ratios which would then allow us to estimate the size of the sign within the larger image. This is necessary because we don't know the size of the stop sign in the larger image and if the template is the wrong size there will not be a high correlation even if the template is in the image.

We implemented the template matching algorithm on stop signs first.  The first template tried for stop signs was stop signs with a black background (fig. 1).  This template yielded results of ~75% stop signs detected.  We saw improved results, ~85% signs detected, if instead of using a template with the entire sign, we used a template that was a rectangular red box with the word “stop” (fig. 2).  This is because the area around the edge of the stop sign in the image was significantly different than the template with a purely black background leading to lower correlation values.  

When a full speed limit sign with a single speed was used for the general template, we lost accuracy because the difference in the speed limits themselves (i.e. 55 vs. 40) reduced the correlation.  The top half of a speed limit sign (fig. 3) proved to be a much better template and we were able to detect a majority of speed limit signs (~85%).  We were also able to successfully implement our code on do not enter and one way signs.

If a template is a different size than the sign within the larger image, the sign will not be detected.  To solve this problem we calculated the correlation of the template and image for many different template/image ratios. We scaled down the image as oppose to scaling up the template to cut down on computation time. When a sign is present and when the template and image have the correct ratio, there is a spike in the correlation values (fig. 4). When no sign that matches the template is present, there is not spike in correlation (fig. 5).  By looking at the scaling of highest correlation we are able to estimate the size of the sign and by looking at the point of highest correlation for this ratio we are able to determine the location of the sign.

Approach ii: edge detection only

In the Edge Detection algorithm, regions of the image are found and then put through another algorithm that determines if they are a part of the sign.  We used the MATLAB function edge to find the boundaries of the objects in the image (fig. 6) and then used the MATLAB function regionprops to find regions within the image.