Файл: Embedded Robotics (Thomas Braunl, 2 ed, 2006).pdf

2 Sensors

Figure 2.13: EyeCam camera module

ideal solution for slower controllers. However, the standard camera chip provides its own clock signal and sends the full image data as a stream with some frame-start signal. This means the controller CPU has to be fast enough to keep up with the data stream.

The parameters that can be set in software vary between sensor chips. Most common are the setting of frame rate, image start in (x,y), image size in (x,y), brightness, contrast, color intensity, and auto-brightness.

The simplest camera interface to a CPU is shown in Figure 2.14. The camera clock is linked to a CPU interrupt, while the parallel camera data output is connected directly to the data bus. Every single image byte from the camera will cause an interrupt at the CPU, which will then enable the camera output and read one image data byte from the data bus.

Figure 2.14: Camera interface

Every interrupt creates considerable overhead, since system registers have to be saved and restored on the stack. Starting and returning from an interrupt takes about 10 times the execution time of a normal command, depending on the microcontroller used. Therefore, creating one interrupt per image byte is not the best possible solution. It would be better to buffer a number of bytes and then use an interrupt much less frequently to do a bulk data transfer of image data. Figure 2.15 shows this approach using a FIFO buffer for intermediate storing of image data. The advantage of a FIFO buffer is that it supports unsynchronized read and write in parallel. So while the camera is writing data

32

Digital Camera

to the FIFO buffer, the CPU can read data out, with the remaining buffer contents staying undisturbed.The camera output is linked to the FIFO input, with the camera’s pixel clock triggering the FIFO write line. From the CPU side, the FIFO data output is connected to the system’s data bus, with the chip select triggering the FIFO read line. The FIFO provides three additional status lines:

•Empty flag

•Full flag

•Half full flag

These digital outputs of the FIFO can be used to control the bulk reading of data from the FIFO. Since there is a continuous data stream going into the FIFO, the most important of these lines in our application is the half full flag, which we connected to a CPU interrupt line. Whenever the FIFO is half full, we initiate a bulk read operation of 50% of the FIFO’s contents. Assuming the CPU responds quickly enough, the full flag should never be activated, since this would indicate an imminent loss of image data.

Figure 2.15: Camera interface with FIFO buffer

2.9.2 Camera Sensor Data

We have to distinguish between grayscale and color cameras, although, as we will see, there is only a minor difference between the two. The simplest available sensor chips provide a grayscale image of 120 lines by 160 columns with 1 byte per pixel (for example VLSI Vision VV5301 in grayscale or VV6301 in color). A value of zero represents a black pixel, a value of 255 is a white pixel, everything in between is a shade of gray. Figure 2.16 illustrates such an image. The camera transmits the image data in row-major order, usually after a certain frame-start sequence.

Creating a color camera sensor chip from a grayscale camera sensor chip is very simple. All it needs is a layer of paint over the pixel mask. The standard Bayer pattern technique for pixels arranged in a grid is the Bayer pattern (Figure 2.17). Pixels in odd rows (1, 3, 5, etc.) are colored alternately in green and red, while pixels in even rows (2, 4, 6, etc.) are colored alternately in blue and green.

33

2 Sensors

Figure 2.16: Grayscale image

With this colored filter over the pixel array, each pixel only records the intensity of a certain color component. For example, a pixel with a red filter will only record the red intensity at its position. At first glance, this requires 4 bytes per color pixel: green and red from one line, and blue and green (again) from the line below. This would result effectively in a 60u80 color image with an additional, redundant green byte per pixel.

However, there is one thing that is easily overlooked. The four components red, green1, blue, and green2 are not sampled at the same position. For example, the blue sensor pixel is below and to the right of the red pixel. So by treating the four components as one pixel, we have already applied some sort of filtering and lost information.

Figure 2.17: Color image

Demosaicing A technique called “demosaicing” can be used to restore the image in full 120u160 resolution and in full color. This technique basically recalculates the three color component values (R, G, B) for each pixel position, for example by averaging the four closest component neighbors of the same color. Figure 2.18 shows the three times four pixels used for demosaicing the red, green, and blue components of the pixel at position [3,2] (assuming the image starts in the top left corner with [0,0]).

34

Digital Camera

Figure 2.18: Demosaic of single pixel position

Averaging, however, is only the simplest method of image value restoration and does not produce the best results. A number of articles have researched better algorithms for demosaicing [Kimmel 1999], [Muresan, Parks 2002].

2.9.3 Camera Driver

There are three commonly used capture modes available for receiving data from a digital camera:

•Read mode: The application requests a frame from the driver and blocks CPU execution. The driver waits for the next complete frame from the camera and captures it. Once a frame has been completely read in, the data is passed to the application and the application continues. In this mode, the driver will first have to wait for the new frame to start. This means that the application will be blocked for up to two frames, one to find the start of a new frame and one to read the current frame.

•Continuous capture mode: In this mode, the driver continuously captures a frame from the camera and stores it in one of two buffers. A pointer to the last buffer read in is passed to the application when the application requests a frame.

•Synchronous continuous capture mode: In this mode, the driver is working in the background. It receives every frame from the camera and stores it in a buffer. When a frame has been completely read in, a trap signal/software interrupt is sent to the application. The application’s signal handler then processes the data. The processing time of the interrupt handler is limited by the acquisition time for one camera image.

Most of these modes may be extended through the use of additional buffers. For example, in the synchronous capture mode, a driver may fill more than a single buffer. Most high-end capture programs running on workstations use the synchronous capture mode when recording video. This of course makes

35

2 Sensors

sense, since for recording video, all frames (or as many frames as possible) lead to the best result.

The question is which of these capture modes is best suited for mobile robotics applications on slower microprocessors. There is a significant overhead for the M68332 when reading in a frame from the camera via the parallel port. The camera reads in every byte via the parallel port. Given the low resolution color camera sensor chip VLSI Vision VV6301, 54% of the CPU usage is used to read in a frame, most of which will not actually be used in the application.

Another problem is that the shown image is already outdated (one frame old), which can affect the results. For example, when panning the camera quickly, it may be required to insert delays in the code to wait for the capture driver to catch up to the robot motion.

Therefore, the “read” interface is considered the most suitable one for mobile robotics applications. It provides the least amount of overhead at the cost of a small delay in the processing. This delay can often be eliminated by requesting a frame just before the motion command ends.

2.9.4 Camera RoBIOS Interface

All interaction between camera and CPU occurs in the background through external interrupts from the sensor or via periodic timer interrupts. This makes the camera user interface very simple. The routines listed in Program 2.1 all apply to a number of different cameras and different interfaces (i.e. with or without hardware buffers), for which drivers have been written for the EyeBot.

Program 2.1: Camera interface routines

typedef BYTE image [imagerows][imagecolumns]; typedef BYTE colimage[imagerows][imagecolumns][3];

The only mode supported for current EyeCam camera models is NORMAL, while older QuickCam cameras also support zoom modes. CAMInit returns the

36

Digital Camera

code number of the camera found or an error code if not successful (see Appendix B.5.4).

The standard image size for grayscale and color images is 62 rows by 82 columns. For grayscale, each pixel uses 1 byte, with values from 0 (black) over 128 (medium-gray) to 255 (white). For color, each pixel comprises 3 bytes in the order red, green, blue. For example, medium green is represented by (0, 128, 0), fully red is (255, 0, 0), bright yellow is (200, 200, 0), black is (0, 0, 0), white is (255, 255, 255).

The standard camera read functions return images of size 62u82 (including a 1-pixel-wide white border) for all camera models, irrespective of their internal resolution:

This originated from the original camera sensor chips (QuickCam and EyeCam C1) supplying 60u80 pixels. A single-pixel-wide border around the image had been added to simplify coding of image operators without having to check image boundaries.

Function CAMGetColFrame has a second parameter that allows immediate conversion into a grayscale image in-place. The following call allows grayscale image processing using a color camera:

image buffer; CAMGetColFrame((colimage*)&buffer, 1);

Newer cameras like EyeCam C2, however, have up to full VGA resolution. In order to be able to use the full image resolution, three additional camera interface functions have been added for reading images at the camera sensor’s resolution (i.e. returning different image sizes for different camera models, see Appendix B.5.4). The functions are:

•CAMGetFrameMono (read one grayscale image)

•CAMGetFrameColor (read one color image in RGB 3byte format)

•CAMGetFrameBayer (read one color image in Bayer 4byte format)

Since the data volume is considerably larger for these functions, they may require considerably more transmission time than the CAMGetFrame/CAMGetColFrame functions.

Different camera models support different parameter settings and return different camera control values. For this reason, the semantics of the camera routines CAMSet and CAMGet is not unique among different cameras. For the camera model EyeCam C2, only the first parameter of CAMSet is used, allowing the specification of the camera speed (see Appendix B.5.4):

FPS60, FPS30, FPS15, FPS7_5, FPS3_75, FPS1_875

For cameras EyeCam C2, routine CAMGet returns the current frame rate in frames per second (fps), the full supported image width, and image height (see Appendix B.5.4 for details). Function CAMMode can be used for switching the camera’s auto-brightness mode on or off, if supported by the camera model used (see Appendix B.5.4).

37

2 Sensors

Program 2.2: Camera application program

5int main()

6{ camera=CAMInit(NORMAL);

7LCDMenu("","","","End");

8while (KEYRead()!=KEY4)

9{ CAMGetFrame (&grayimg);

10LCDPutGraphic(&grayimg);

11}

12return 0;

13}

2.10References

BARSHAN, B., AYRULU, B., UTETE, S. Neural network-based target differentiation using sonar for robotics applications, IEEE Transactions on Robotics and Automation, vol. 16, no. 4, August 2000, pp. 435-442 (8)

BRÄUNL, T. Parallel Image Processing, Springer-Verlag, Berlin Heidelberg, 2001

DINSMORE, Data Sheet Dinsmore Analog Sensor No. 1525, Dinsmore Instrument Co., http://dinsmoregroup.com/dico, 1999

EVERETT, H.R. Sensors for Mobile Robots, AK Peters, Wellesley MA, 1995

38