Файл: Embedded Robotics (Thomas Braunl, 2 ed, 2006).pdf

17 Real-Time Image Processing

At this level, we are not concerned about continuity of the matching pixels in a column. There may be two or more separate sections of matching pixels, which may be due to either occlusions or reflections on the same object – or there might be two different objects of the same color. A more detailed analysis of the resulting histogram could distinguish between these cases.

Program 17.5: RGB to hue conversion

1int RGBtoHue(BYTE r, BYTE g, BYTE b)

2/* return hue value for RGB color */

3#define NO_HUE -1

4{ int hue, delta, max, min;

8delta = max - min;

9hue =0; /* init hue*/

11if (2*delta <= max) hue = NO_HUE; /* gray, no color */

14else if (g==max) hue = 126 +42*(b-r)/delta; /* 3*42 */

15else if (b==max) hue = 210 +42*(r-g)/delta; /* 5*42 */

16}

17return hue; /* now: hue is in range [0..252] */

18}

Program 17.5 shows the conversion of an RGB image to an image (hue, saturation, value), following [Hearn, Baker 1997]. We drop the saturation and value components, since we only need the hue for detecting a colored object like a ball. However, they are used to detect invalid hues (NO_HUE) in case of a too low saturation (r, g, and b having similar or identical values for grayscales), because in these cases arbitrary hue values can occur.

Input image

with sample column marked

Histogram

0 0 0 0 5 21 32 18 3 0 1 0 2 0 0 0 0 0 0 with counts of matching pixels per column

Column with maximum number of matches

Figure 17.8: Color detection example

252

Color Object Detection

The next step is to generate a histogram over all x-positions (over all columns) of the image, as shown in Figure 17.8. We need two nested loops going over every single pixel and incrementing the histogram array in the corresponding position. The specified threshold limits the allowed deviation from the desired object color hue. Program 17.6 shows the implementation.

Program 17.6: Histogram generation

3/* generate histogram over all columns */

4{ int x,y;

5for (x=0;x<imagecolumns;x++)

6{ histogram[x] = 0;

7for (y=0;y<imagerows;y++)

8if (hue_img[y][x] != NO_HUE &&

9(abs(hue_img[y][x] - obj_hue) < thres ||

10253 - abs(hue_img[y][x] - obj_hue) < thres)

12}

13}

Finally, we need to find the maximum position in the generated histogram. This again is a very simple operation in a single loop, running over all positions of the histogram. The function returns both the maximum position and the maximum value, so the calling program can determine whether a sufficient number of matching pixels has been found. Program 17.7 shows the implementation.

Program 17.7: Object localization

1void FindMax(line histogram, int *pos, int *val)

2/* return maximum position and value of histogram */

3int x;

4 { *pos = -1; *val = 0; /* init */

5for (x=0; x<imagecolumns; x++)

6if (histogram[x] > *val)

7{ *val = histogram[x]; *pos = x; }

8}

Programs 17.6 and 17.7 can be combined for a more efficient implementation with only a single loop and reduced execution time. This also eliminates the need for explicitly storing the histogram, since we are only interested in the maximum value. Program 17.8 shows the optimized version of the complete algorithm.

For demonstration purposes, the program draws a line in each image column representing the number of matching pixels, thereby optically visualizing the histogram. This method works equally well on the simulator as on the real

253

17 Real-Time Image Processing

Program 17.8: Optimized color search

3/* find x position of color object, return pos and value*/

4{ int x,y, count, h, distance;

5 *pos = -1; *val = 0; /* init */

6for (x=0;x<imagecolumns;x++)

7{ count = 0;

8for (y=0;y<imagerows;y++)

9{ h = RGBtoHue(img[y][x][0],img[y][x][1],

11if (h != NO_HUE)

12{ distance = abs((int)h-obj_hue); /* hue dist. */

13if (distance > 126) distance = 253-distance;

14if (distance < thres) count++;

15}

16}

17if (count > *val) { *val = count; *pos = x; }

18LCDLine(x,53, x, 53-count, 2); /* visualization only*/

19}

20}

Figure 17.9: Color detection on EyeSim simulator

254

Color Object Detection

robot. In Figure 17.9 the environment window with a colored ball and the console window with displayed image and histogram can be seen.

Program 17.9: Color search main program

3int main()

4{ colimage c;

5int hue, pos, val;

7LCDPrintf("Teach Color\n");

8LCDMenu("TEA","","","");

9CAMInit(NORMAL);

10while (KEYRead() != KEY1)

11{ CAMGetColFrame(&c,0);

12LCDPutColorGraphic(&c);

13hue = RGBtoHue(c[Y][X][0], c[Y][X][1], c[Y][X][2]);

14LCDSetPos(1,0);

15LCDPrintf("R%3d G%3d B%3d\n",

21LCDClear();

22LCDPrintf("Detect Color\n");

23LCDMenu("","","","END");

24while (KEYRead() != KEY4)

25{ CAMGetColFrame(&c,0);

26LCDPutColorGraphic(&c);

27ColSearch(c, hue, 10, &pos, &val); /* search image */

28LCDSetPos(1,0);

29LCDPrintf("h%3d p%2d v%2d\n", hue, pos, val);

30 LCDLine (pos, 0, pos, 53, 2); /* vertical line */

31}

32return 0;

33}

The main program for the color search is shown in Program 17.9. In its first phase, the camera image is constantly displayed together with the RGB value and hue value of the middle position. The user can record the hue value of an object to be searched. In the second phase, the color search routine is called with every image displayed. This will display the color detection histogram and also locate the object’s x-position.

This algorithm only determines the x-position of a colored object. It could easily be extended to do the same histogram analysis over all lines (instead of over all columns) as well and thereby produce the full [x, y] coordinates of an object. To make object detection more robust, we could further extend this

255

17 Real-Time Image Processing

algorithm by asserting that a detected object has more than a certain minimum number of similar pixels per line or per column. By returning a start and finish value for the line diagram and the column diagram, we will get [x1, y1] as the object’s start coordinates and [x2, y2] as the object’s finish coordinates. This rectangular area can be transformed into object center and object size.

17.7 Image Segmentation

Detecting a single object that differs significantly either in shape or in color from the background is relatively easy. A more ambitious application is segmenting an image into disjoint regions. One way of doing this, for example in a grayscale image, is to use connectivity and edge information (see Section 17.3, [Bräunl 2001], and [Bräunl 2006] for an interactive system). The algorithm shown here, however, uses color information for faster segmentation results [Leclercq, Bräunl 2001].

This color segmentation approach transforms all images from RGB to rgb (normalized RGB) as a pre-processing step. Then, a color class lookup table is constructed that translates each rgb value to a “color class”, where different color classes ideally represent different objects. This table is a three-dimen- sional array with (rgb) as indices. Each entry is a reference number for a certain “color class”.

17.7.1 Static Color Class Allocation

In a similar way, we can distinguish orange and yellow, by a combination of thresholds on the red and green component:

If (rgb) were coded as 8bit values, the table would comprise (28)3 entries, which comes to 16MB when using 1 byte per entry. This is too much memory

256

Image Segmentation

for a small embedded system, and also too high a resolution for this color segmentation task. Therefore, we only use the five most significant bits of each color component, which comes to a more manageable size of (25)3 = 32KB.

In order to determine the correct threshold values, we start with an image of the blue goal. We keep changing the blue threshold until the recognized rectangle in the image matches the right projected goal dimensions. The thresholds for red and green are determined in a similar manner, trying different settings until the best distinction is found (for example the orange ball should not be classified as the yellow goal and vice versa). With all thresholds determined, the corresponding color class (for example 1 for ball, 2 or 3 for goals) is calculated and entered for each rgb position in the color table. If none of the criteria is fulfilled, then the particular rgb value belongs to none of the color classes and 0 is entered in the table. In case that more than one criterion is fulfilled, then the color classes have not been properly defined and there is an overlap between them.

17.7.2 Dynamic Color Class Allocation

17.7.3 Object Localization

Having completed the color class table, segmenting an image seems simple. All we have to do is look up the color class for each pixel’s rgb value. This gives us a situation as sketched in Figure 17.10. Although to a human observer, coherent color areas and therefore objects are easy to detect, it is not trivial to extract this information from the 2D segmented output image.

257