0.2 Approaches  (Page 2/2)

We first tried to find the border of the sign.  The outside edge of the sign is a region most of the time, so this was a possibility.  However, there is not a good way of determining which region was the sign.  A first guess was to choose the biggest region, however this did not always work as the sign was not always the biggest object in the picture. We then noticed that the O of stop showed up as two regions, the inner ring and the outer ring.  Since these two regions have approximately the same center (fig. 7), it is possible to find regions that have centers close together and then analyze them to see if they are the sign in question.  For example, if the point is on a stop sign, then the point, the point a bit above it, and the point a bit below it will all be red.  If the point is on a do not enter sign, then the point will be white and the points above and below will be red.  

The points above and below were chosen by using the height of the region found.  By choosing points the height of the region above and below the identified center, the points are on the outside of the letter.  This tells the background of the sign itself.  

This method of finding signs is not the most accurate.  It can be fooled by other signs. For example, a sign for a business that had an O on a red background would be considered a stop sign by this algorithm.  This could be helped by checking more points.  Points immediately around the center should also be red and some points in between the center and the currently checked outer points should be white.  This would improve the accuracy, but it could still be fooled. This algorithm also works best for signs with color.  Signs with white backgrounds such as speed limit signs are hard to find because values that are defined as ‘white’ cover a large range of pale colors and therefore many things not signs are identified as signs.  

Despite these drawbacks, this code is advantageous because it does not require large convolutions like the first method. This means that the method is much faster. To find the sign of the correct size with the first approach, it takes about 30 seconds. To find the sign including the size with the second approach, it takes about .5-1 second.

Approach iii: template matching and edge detection

For our final algorithm we used both template matching and edge detection in an attempt to combine the positive aspects of the two approaches.  The combined algorithm first utilizes edge detection to isolate regions within the image, throwing out all regions of a size below a certain threshold in order not to waste time with extraneous convolutions.  Once the significant regions, usually numbering between twenty and one hundred, have been identified, each is re-sized to a standard small template size and compared to each image in the template library.  Regions with high correlations with a template image are assumed to be the part of the sign that template was created from.

This combined algorithm performed quite well, although it was not error-free.  As was the case with pure template matching, attempting to match pictures of entire signs was prone to error because the background of any given sign can vary, occasionally leading to quite low correlations.  However, we discovered that using this algorithm to match only specific elements of a sign, such as the “S”, “T”, “O”, and “P” in stop (fig. 8), was much more reliable, as the conditions for such sub-sign features are much more consistent than for the overall sign.  In addition, searching for individual sign features and not just the sign itself means that partially obscured signs that would be beyond our first template matching algorithm would present no problems for this modified one. Our template library was expanded so as to contain a number of sub-sign features for both stop signs and speed limit signs, as both of these signs contain changing characteristics (background for the stop sign, speed limit for the speed limit sign) that can thwart conventional template matching.  Again, we found that matching individual letters yielded a high success rate (fig. 9).  Our basic algorithm was also fairly successful in identifying one way signs and do not enter signs, even though no sub-feature library was created for these signs and only the sign itself was used as a template.

This algorithm showed significant success in combining the positive aspects of both tablet matching and edge detection.  In comparison to pure template matching, this method performs correlation calculations on much smaller images and size matching is a non-issue because all regions are re-sized to match the templates.  In comparison to pure edge detection, this method is less prone to false positives and can be much more easily expanded to detect new signs simply by adding to the template library.  Overall, the combination of template matching and edge detection yields a more robust, flexible, and reliable method of sign detection than either algorithm used alone.