Multiple Objects Tracking Via Collaborative Background Subtraction Computer Science Essay Example

is displayed when there is no moving object, with the number 28.

Figure 6.2: Wind causes the curtain to move 28

Figure 6.3 depicts the movement of a stand fan in motion. The frames progress downwards and from left to right, totaling 31 frames.

Charge-couple Device (CCD) refers to a type of technology

FPS stands for Frames per Second.

GUI, which stands for Graphical User Interface, refers to the visual elements and interactions that allow users to interact with software applications.

ID Identification Number

USB stands for Universal Serial Bus.

VGA stands for Video Graphics Array.

The text discusses the concept of an object tracking system, a real-time vision system that can autonomously perform surveillance tasks (Nguyen, K. et al., 2002). This system can detect moving objects on the street, such as vehicles or pedestrians, without human involvement. It also gathers data for Intelligent Transportation Systems by providing information about the number of vehicles in a specific area (R. Reulke et al., 2002). Additionally, this tracking system handles environmental changes like shadows from nearby buildings or slow-moving vehicles. The primary objective of this project is to establish a practical and functional object tracking system by connecting a vision system to a computer for object detection.

The object tracking system is able to independently distinguish between a static background and moving objects. Once detected, it can display and track these moving objects. This functionality allows for effective monitoring of busy streets with heavy traffic flow. Furthermore, if these areas are equipped with an Intelligent Traffic System, the object tracking system can assist in collecting data and reducing waiting times for vehicles at traffic lights.

Since the year 2000, various algorithms for fast response time or accurate

object detection have been released, including background subtraction, mean shift, Kalman filter, Markov Chain Monte Carlo, Kernel Density, and others.

The Object Tracking System is composed of two key systems: the vision system and the software system for detecting and tracking moving objects. The vision system is responsible for capturing and exporting the video stream, which is then transmitted to the tracking system. Users can monitor and receive notifications when a moving object is detected through the tracking system. The project aims to create an object tracking system that can accurately identify and track vehicles on the street. However, challenges may arise in detecting fast-moving objects, low light conditions, or shadows caused by buildings. Therefore, it is crucial for the detection algorithm to efficiently process each frame from the vision system while effectively handling shadowy surroundings and ensuring a prompt response time from the tracking system.

The effectiveness of current real-time object tracking systems is often limited by slow response times. To address this issue, it is crucial to develop a faster computation algorithm. By incorporating background subtraction into the initial detection phase, the computation time can be reduced, leading to faster real-time object detection. Additionally, a more precise detection and tracking algorithm will be utilized to enhance tracking accuracy. This anticipated algorithm is projected to improve both the speed and accuracy of moving object tracking.

The aim of this project is to use a vision system that operates in real-time to detect multiple moving objects. To accomplish this, the following sub-goals must be achieved.

The objective is to analyze and determine quantifiable variables for tracking the motion of an object.

Background subtraction is essential for achieving real-time detection purposes.

The

goal is to enhance the current algorithm for continuous tracking.

In order to assess and enhance the effectiveness of a tracking system that utilizes background subtraction, it is necessary.

The aim of this project is to develop a system that can track and detect moving objects. This system will comprise a vision system and an image processing system, enabling it to continually identify and track moving objects.

The MATLAB control m-file is the central component of the object tracking system. It detects and tracks moving vehicles in the video provided by the vision system, displaying the vehicle tracking system in a GUI window.

The vision system will serve as a supplier for providing a tracking system that captures video in a desired area. This system should be compact for easy setup and removal.

This report is composed of seven well-divided and planned chapters. Each chapter focuses on the discussion of the vision system and object tracking system.

The second chapter provided an examination of the current methods available for object tracking and detection.

In Chapter 3, the workflow for this tracking system is explained. It covers the essential parameters needed for the system to function, as well as input and output prediction. Furthermore, it introduces the idea of constructing this tracking system using readily available vision systems in the market.

In Chapter 4, the process of setting up both the hardware and software for the tracking system implementation is explained. This involves ensuring that the vision system can provide the required video for the tracking system and optimizing MATLAB's ability to process the provided video by arranging memory allocation appropriately.

This chapter explores the use of background subtraction algorithms in the project, specifically focusing

on frame difference. A new M-file will be created to include the necessary functions for establishing the tracking system. The tracking system should be able to run utilizing both the hardware and software setup discussed in the previous chapter, along with this M-file.

Chapter 6 showcases the image output and results obtained while running this tracking system. Firstly, it demonstrates successful background subtraction. Secondly, it illustrates the distortion in the surroundings, such as the shadow of an object.

Chapter 7 provides a summary and conclusion of the report, explaining the project's limitations and future work.

This chapter focuses on reviewing existing methods for detecting and tracking objects. We will also study algorithms that are suitable for detection and tracking. The student will review several algorithms.

In their research, Al-amri et al. (2010) found that the median filter is commonly used to reduce small noise in images. They observed that small noise in grayscale images often has a different value compared to its neighboring pixels. By replacing the gray value of the noisy pixel with the median value of its neighbors, this technique effectively removes the noise.

By referring to Figure 2.1, it is evident that the surrounding pixel values are 115, 119, 120, 123, 124, 125, 126, 127, and 150. To remove the noise, we can replace the central pixel with the median value of these pixels which is calculated as 124.

To achieve a more precise median value, it is necessary to increase the number of neighboring values involved in the calculation. However, this approach becomes increasingly complex when applied to larger images. Additionally, there is a relatively high computational cost and time requirement due to the need to sort

all the values in the neighborhood.

Edge detection is a well-known technique introduced by Canny (Neoh, H.S et al., 2005). Tracking an object using this method involves a few steps.

Remove small noise by applying a smoothing filter to the image.

Two gradient images are produced using one of the gradient operators. These images are created in both the vertical and horizontal directions, using the previous image as a base.

The result shown is Gx (m, n) and Gy(m, n), where m and n denote pixel coordinates.

Utilize the two aforementioned images to generate images that display both edge size and orientation.

The magnitude of the edge, M (m,n), is

Edge direction,

Threshold the image M by setting pixels to zero if their edge magnitude value is below a predefined threshold.

When the non-zero pixels in MT (m, n) undergo a non-maxima operation, their edge breadth decreases. If these pixels have values that are equal to or lower than their neighbors in the given direction, they will be changed to zero.

The authenticity of edges is determined using two thresholds, T1 and T2, where T1 is smaller than T2. Edges with a magnitude less than T1 are eliminated, while those greater than T2 are considered authentic edges. Additionally, edge pixels connected to an edge within the range of magnitudes between T1 and T2 are also recognized as edges.

The Hough Transform technique can detect objects represented in a Hough parameter space such as polynomials, straight lines, circles, etc. Peaks in the Hough parameter space describe the object space using parameters like length from origin to line and orientation for a line segment representation.

The CamShift technique tracks objects based on their color using center

and size information in an image. The tracking process involves setting the search window size, initializing its location, and computing centroid location within the search window based on moments 0 and 1. This process continues until the search window moves less than a preset distance threshold from its current position. It should be noted that this technique requires identical colored objects for reliable use and may not work well with complex colors.

The Kalman Filter algorithm acts as a state estimation technique utilizing a feedback control mechanism. It predicts the process state and receives feedback from measurements.The Kalman filter equation is divided into two groups: the time update equation and the measurement update equations. The time update equation predicts the current state and error covariance, providing a prediction for the next time step. The measurement update equations incorporate a new measurement into their prior prediction, resulting in an improved estimation compared to other methods. However, this filter is not suitable for detecting fast-moving objects like vehicles on a highway due to drastic changes in speed and acceleration between consecutive frames. It also requires more computation time.

On the other hand, Markov Chain Monte Carlo (MCMC) is an algorithm that samples from probability distributions by constructing a Markov Chain with the desired distribution as its equilibrium. This process involves three stages: Model Construction, Bayesian formulation, and detection of vehicles using MCMC.

Background subtraction is another technique that involves comparing two images captured in the same location. The first image represents an empty background with no moving objects, while the second image contains a moving object. Subtracting these two images produces an image displaying only the moving object without any

background. Image processing interprets this resultant image as an array format using matrix coordinates (x, y) to represent each pixel. Equation 4.1 defines the intensity at position (x, y) as I(x,y).Equation 4.2 is used to calculate brightness during image subtraction, taking into account contributions from foreground objects (lc), as well as background objects (Ib and Is). By utilizing foreground objects, position adjustment between the two images can be easily achieved. Subtracting the first image from the second one containing the foreground object (Equation 4.2) allows us to obtain the desired foreground object.

In Chapter 3 titled "Conceptual Design," a method for detecting and tracking objects is discussed. A vision system captures video within a specified area and sends it to MATLAB for processing. The data is then processed by MATLAB, which performs tracking actions. Figure 3.1 shows how a webcam is connected to a computer via USB in order to capture data that MATLAB retrieves and processes.

If any moving objects are detected by the vision system, they are displayed in a GUI window. The selection of MATLAB as the platform for object detection and tracking was based on its powerful toolbox that can synchronize with a webcam and create a program for detecting and tracking vehicles. Furthermore, MATLAB has the capability to generate a GUI window, which is crucial for the tracking system.

To detect and track moving objects, frame differencing will be used to subtract the background and obtain a mask of the object's movementIn order to improve accuracy, we will utilize a more precise algorithm for tracking objects. This process involves receiving video frames from the camera, preprocessing them, saving the current frame as the

background image, subtracting subsequent frames from the background image and storing it in memory. We then update the current frame as the new background and continuously display and track the moving object. Figure 3.2 illustrates this system's workflow for object tracking in a flowchart.

In chapter 3.3 of our discussion, we explain how we developed a draft and prototype of our tracking system based on previously discussed conceptual designs. The next chapter delves into pre-processing techniques and provides a detailed explanation of how to connect a webcam with MATLAB while preparing configurations.

CHAPTER 4 Hardware and Software Setup provides an overview of preparing hardware tools or instruments, as well as software programs and programming languages before establishing a simulation. This setup ensures seamless connection between hardware and software components in order to achieve specific goals. Engineers have the option to either use available tools or design their own tools according to their requirements. Likewise, engineers can efficiently utilize various software programs by designing suitable workflows that meet their expectations.One instance of this is the creation of a surveillance system, where mathematical modeling can be utilized to examine and track objects captured by a camera. In this chapter, we focus on the necessary hardware and software setup needed for designing a street monitoring system. This includes connecting a webcam to MATLAB, which allows for real-time video recording from the webcam. The setup also involves M-file coding that contains algorithms for extracting the static background from moving objects like vehicles or pedestrians. The outcome of the background subtraction process is displayed as moving images through a graphical user interface (GUI). Various tools and software were used to ensure success

in this project. Specifically, it incorporates a Logitech Quick Cam Pro 4000 webcam that connects to a computer via USB 2.0. This webcam has the ability to capture static images or record real-time videos. For this project, MATLAB R2009a was employed with the Image Acquisition Toolbox and Image Processing Toolbox playing essential roles in its implementation. The Image Acquisition Toolbox enables real-time recording from a webcam and sends it to MATLAB, while the Image Processing Toolbox processes continuous frames captured by the webcam, stored in MATLAB, and displays moving objects through background subtraction.The Logitech Quick Cam Pro 4000 has a video capture resolution of 640 x 480 and snapshot image resolution of 1280 x 960.
The Logitech Quick Cam Pro 4000 webcam includes a built-in microphone and can achieve up to 30 frames per second with its advanced VGA CCD sensor. During the student's experiments, different video input formats (YCbCr, grayscale, and RGB) were tested on this vision system. The system only supports three color spaces, so three experiments were conducted to determine the most suitable one. Each experiment involved testing different light intensities (low, normal, and high) on a battery object. In the low light intensity scenario, the captured image should be sufficiently dark. The normal light intensity test took place indoors with medium lighting conditions while avoiding strong light sources like sunlight or spotlights. In the last case, images were taken with the camera pointed at a strong light source such as a torchlight. These experiments used a webcam connected to MATLAB and executing command codes.

Table 4.1 provides an overview of the results from these three experiments. Experiment 1 utilized YCbCr as the video

input format and presented outcomes in figure form. To conduct this test, the webcam was connected to MATLAB and executed the following code:
``` vid = videoinput('winvideo',1);set(vid,'ReturnedColorSpace','YCbCr');preview(vid) ```

Figure 4.2(a) clearly shows that due to low light intensity surrounding the object, the captured image appears significantly dark.
Figure 4.2(b) shows that the object can be perceived vividly with human eyes. Despite a white spot caused by strong light in Figure 4.2(c), the object remains visible. These images belong to the Figure 4.2 series, which were captured for YCbCr return color space. Specifically, (a) represents low light intensity, (b) represents normal light intensity, and (c) represents high light intensity. This experiment demonstrates the potential of using YCbCr return color space in this project as it retains color properties but undergoes slight changes in color during high light intensity situations.

For Experiment 2, video input is converted into grayscale format and depicted in a figure. To conduct this experiment, it is necessary to remove the previous video object from MATLAB workspace and execute the following code: ``` vid = videoinput('winvideo',1);set(vid,'ReturnedColorSpace','YCbCr');preview(vid) ```

Figures 4.3(b) and 4.3(c) show the object's color attribute appearing as black and white respectively. Figure 4.3(c) does not exhibit overexposure, but it is still difficult to perceive the object in Figure 4.3(a). The series of images in Figure 4.3 showcase grayscale return color space, with (a) representing low light intensity, (b) representing normal light intensity, and (c) illustrating high light intensityThe current return color space is not considered due to its limited color property, which would restrict potential enhancements for algorithms requiring color. In Experiment 3, RGB is used as the video input format and the results are presented

as a figure. The default return color space in MATLAB is RGB. To capture images in a dark setting using this webcam, the previous built-in video object is deleted and a new video input with direct previewing is created without specifying a return color space. The code used for this purpose is:

``` vid = videoinput('winvideo',1);preview(vid)```

In Figure 4.4(a), capturing an image in a dark environment is not possible; however, Figure 4.4(b) accurately represents each object's colors with detailed accuracy. This specific return color space avoids overexposure issues seen in Figure 4.4(c). The images captured using grayscale return color space are displayed as Figures 4.4(a), 4.4(b), and 4.5(c). The experiment demonstrates that the selected return color space is the most suitable among the three options for this project as it retains color properties and handles overexposure.

Table 4.1 provides a summary of the conducted experiments, outlining the properties of each color space (YCbCr, Grayscale, and RGB) regarding their ability to detect objects in low light intensity, available return color options, and susceptibility to overexposureTable 4.1 shows that RGB is the most suitable return color space because grayscale loses its color characteristics, converting figures into black and white, making it difficult to recognize objects based on unique colors. YCbCr is an encoding method for RGB information, so using RGB preserves the original characteristics. Additionally, there are other possible uses for RGB since it offers more flexibility. The default return color space in the MATLAB toolbox is already set to RGB; therefore, there is no need to specify it when importing video input using MATLAB coding.

To obtain input from a webcam initially, create an object named "obj" using the MATLAB

command: obj = videoinput('winvideo',1), where 1 represents the camera input ID number. Upon successful execution of this command, "obj" will be stored as an object in the MATLAB workspace.

To continuously acquire data from a video input object in MATLAB, use the command triggerconfig(obj,'manual'). At the beginning of the MATLAB M-file, set the properties of the video input object by associating it with a figure in the MATLAB GUI using and their contents: set(obj, 'Tag', appTitle, 'FramesAcquiredFcnCount', 1,...'TimealrFcn', @locFrameCallback, 'TimerPeriod', 0.01). If this figure already exists, it will be used; otherwise, a new one will be created.

An empty array called appdata.background will be used to store application data for the video input object.The array's dimensions and values are not set. The HTML text below demonstrates how to use the obj.UserData parameter and imaqmotion function in MATLAB. To avoid errors, it is important to compile the imaqmotion function together.

To execute this function, create a video input object and run it using its name followed by brackets containing the name of the video input object. To stop any existing video input objects from running, include a stop instruction in the M-file using stop(obj). This ensures that only one desired new video input object is used for monitoring process.

If importing frames from webcam takes too long, MATLAB may display a warning about excessive frame requests. Use the command "warning off imaq:peekdata:tooManyFramesRequested" to disable this warning.

During execution, if an unexpected error occurs in MATLAB, it will be caught and a warning message will be displayed indicating the occurrence of an error. This allows graceful termination of function execution.

In this chapter, students demonstrate how to connect MATLAB with a webcam

for importing real-time recordings. The video input object is declared and stored in the MATLAB workspace for later use in subtracting objects from a static background. This step is crucial for the central aspect of the project.

These steps mentioned above aim to enable users to execute multiple instructions with one simple instruction stored in a MATLAB M-file.In the upcoming chapter, we will demonstrate the process of comparing two consecutive frames to detect and locate objects that differ from the previous frame. These objects will be displayed in a MATLAB GUI. In Chapter 5, titled "Background Subtraction Using Frame Difference," we provide an overview of how our project aims to detect moving objects within a vision system. To achieve this, we need to develop a monitoring system that can differentiate between moving objects and a stationary background. Writing an algorithm using programming languages like C, Open CV, or MATLAB can assist in accomplishing this task. This chapter specifically focuses on implementing background subtraction through frame difference as part of our project. Frame difference entails comparing the current frame with the previous frame to identify any pixels representing moving objects not present in the previous frame. We selected this method for its simplicity and ability to reduce processing time for imported frames from the vision system. To implement this technique, we store the background frame as an array with constant elements containing pixel informationThe text describes various functionalities and abilities that will be incorporated into an M-file. These include updating image display, GUI window, creating and initializing a figure and a bar display. Section 5.2 of the text focuses on initializing and creating a background image when

importing a new video input object. The process involves comparing the background image to each subsequent frame. To initialize the background image, the video input object is imported along with 'UserData' into this function. The appdata is obtained using the get() function on vid with 'UserData' as an argument and assigned to a variable called background. The first frame is stored in memory by using MATLAB's snapshot operation to create the background image after importing data from the video input object.If there is no existing background, it will be set to getsnapshot(vid). To update the background image for each subsequent frame, we capture and send that frame along with the latest data to the required function localUpdateFig(vid, appdata.figureHandles, frame, background). This updates the local figure with these inputs. The updated background becomes set as the frame for our video (vid) and stored in appdata. In a previous section, we discussed a method called frame differencing within context of localUpdateFig function. This method subtracts the current frame from our background and displays result in GUI window. If no moving object is detected, MATLAB's GUI will show a blank screen. The subtraction of two images calculates their absolute difference. Additionally,a bar acts as a marker to display the disparity between the current frames and the background imageThe imabsdiff(frame, background) function is used to execute frame differencing by subtracting each pixel of the two images and obtaining the absolute difference.An empty background image serves as a reference point (background) as shown in Figure 5.1(a).The subsequent frame (frame) captures a moving object depicted in Figure 5.1(b). The absolute difference between these two images can be observed in Figure 5.1(c)

when they are subtracted.