Track objects with Camshift using OpenCV

In the realm of computer vision, object tracking plays a pivotal role in various applications such as surveillance, automotive safety systems, and augmented reality. One robust and widely used algorithm for object tracking is CamShift (Continuously Adaptive Mean Shift) in conjunction with the OpenCV library. In this article, we will delve into the intricacies of CamShift and explore how it can be employed to effectively track objects in real-time.

Understanding CamShift Algorithm

The CamShift (Continuously Adaptive Mean Shift) algorithm is a powerful computer vision technique used for object tracking in videos or image sequences. It builds upon the Mean Shift algorithm and adds the ability to adaptively adjust the size and orientation of the tracking window, making it suitable for objects that can undergo changes in scale and rotation.

The beauty of CamShift lies in its continuous adaptation. After each iteration, the algorithm adjusts the size and orientation of the tracking window based on the mean shift vector. This adaptive process ensures that the algorithm can handle changes in the object's scale and rotation, making it particularly useful for scenarios where the target object undergoes such transformations.

Working Structure of CamShift:

The CamShift algorithm is based on color information and operates in the color space of the images or frames. It uses a histogram of color values within a specified region of interest (ROI) to track the object of interest. Here's how the CamShift algorithm works:

  1. Initialization: The algorithm starts by selecting an initial region of interest (ROI) around the object to be tracked in the first frame. This ROI serves as the tracking window.
  2. Histogram Calculation: The color histogram of the selected ROI is computed in a color space such as HSV (Hue, Saturation, Value). The histogram represents the distribution of colors within the ROI.
  3. Back Projection: In each subsequent frame, the histogram of the ROI is back-projected onto the entire frame. This back-projection creates a probability map that highlights regions with colors similar to those in the ROI histogram.
  4. Mean Shift Iteration: The mean shift algorithm is applied to the back-projected probability map. Mean shift calculates the centroid of the probability distribution and shifts the tracking window to this new centroid.

Requirements:

OpenCV Installation: OpenCV is a popular computer vision library that provides the tools necessary for image and video processing, including object tracking with CamShift. You need to install OpenCV to use its functions and capabilities. Install OpenCV using the following command:

  1. pip install opencv-python  

Integrated Development Environment (IDE): You'll need an environment where you can write, run, and debug your Python code. There are several options available, such as Visual Studio Code, PyCharm, Jupyter Notebook, and more. Choose the one you're most comfortable with.

Importing of Libraries:

  1. import cv2  
  2. import numpy as np  

Implementation of Code:

  1. import numpy as np  
  2. import cv2 as cv  
  3.   
  4. # Read the input video  
  5. cap = cv.VideoCapture('sample.mp4')  
  6.   
  7. # Take first frame of the video  
  8. ret, frame = cap.read()  
  9.   
  10. # Setup initial region of tracker  
  11. x, y, width, height = 400440150150  
  12. track_window = (x, y, width, height)  
  13.   
  14. # Set up the Region of Interest for tracking  
  15. roi = frame[y:y + height, x:x + width]  
  16.   
  17. # Convert ROI from BGR to HSV format  
  18. hsv_roi = cv.cvtColor(roi, cv.COLOR_BGR2HSV)  
  19.   
  20. # Perform masking operation  
  21. mask = cv.inRange(hsv_roi, np.array((0., 60., 32.)), np.array((180., 255., 255)))  
  22.   
  23. roi_hist = cv.calcHist([hsv_roi], [0], mask, [180], [0180])  
  24. cv.normalize(roi_hist, roi_hist, 0255, cv.NORM_MINMAX)  
  25.   
  26. # Setup the termination criteria, either 15 iterations or move by at least 2 points  
  27. term_crit = (cv.TERM_CRITERIA_EPS | cv.TERM_CRITERIA_COUNT, 152)  
  28.   
  29. while(1):  
  30.     ret, frame = cap.read()  
  31.       
  32.     # Resize the video frames  
  33.     frame = cv.resize(frame, (720720), fx=0, fy=0, interpolation=cv.INTER_CUBIC)  
  34.     cv.imshow('Original', frame)  
  35.   
  36.     # Perform thresholding on the video frames  
  37.     ret1, frame1 = cv.threshold(frame, 180155, cv.THRESH_TOZERO_INV)  
  38.   
  39.     # Convert from BGR to HSV format  
  40.     hsv = cv.cvtColor(frame1, cv.COLOR_BGR2HSV)  
  41.   
  42.     dst = cv.calcBackProject([hsv], [0], roi_hist, [0180], 1)  
  43.   
  44.     # Apply CamShift to get the new location  
  45.     ret2, track_window = cv.CamShift(dst, track_window, term_crit)  
  46.   
  47.     # Draw it on the image  
  48.     pts = cv.boxPoints(ret2)  
  49.     pts = np.int0(pts)  
  50.   
  51.     # Draw Tracking window on the video frame  
  52.     Result = cv.polylines(frame, [pts], True, (0255255), 2)  
  53.     cv.imshow('Camshift', Result)  
  54.   
  55.     # Set ESC key as the exit button  
  56.     k = cv.waitKey(30) & 0xff  
  57.     if k == 27:  
  58.         break  
  59.   
  60. # Release the cap object  
  61. cap.release()  
  62.   
  63. # Close all opened windows  
  64. cv.destroyAllWindows()  

Output:

Track objects with Camshift using OpenCV
Track objects with Camshift using OpenCV
Next Topic#