https://www.pyimagesearch.com/2016/01/18/multiple-cameras-with-the-raspberry-pi-and-opencv/

Multiple cameras with the Raspberry Pi and OpenCV

by Adrian Rosebrock on January 18, 2016 in Raspberry Pi, Tutorials

0

62

I’ll keep the introduction to today’s post short, since I think the title of this post and GIF animation above speak for themselves.

Inside this post, I’ll demonstrate how to attach multiple cameras to your Raspberry Pi…and access all of them using a single Python script.

Regardless if your setup includes:

• Multiple USB webcams.
• Or the Raspberry Pi camera module + additional USB cameras…

…the code detailed in this post will allow you to access all of your video streams — and perform motion detection on each of them!

Best of all, our implementation of multiple camera access with the Raspberry Pi and OpenCV is capable of running in real-time (or near real-time, depending on the number of cameras you have attached), making it perfect for creating your own multi-camera home surveillance system.

Keep reading to learn more.

Looking for the source code to this post?
Jump right to the downloads section.

Multiple cameras with the Raspberry Pi and OpenCV

When building a Raspberry Pi setup to leverage multiple cameras, you have two options:

• Simply use multiple USB web cams.
• Or use one Raspberry Pi camera module and at least one USB web camera.

The Raspberry Pi board has only one camera port, so you will not be able to use multipleRaspberry Pi camera boards (unless you want to perform some extensive hacks to your Pi). So in order to attach multiple cameras to your Pi, you’ll need to leverage at least one (if not more) USB cameras.

That said, in order to build my own multi-camera Raspberry Pi setup, I ended up using:

1. A Raspberry Pi camera module + camera housing (optional). We can interface with the camera using the picamera  Python package or (preferably) the threaded VideoStream  class defined in a previous blog post.
2. A Logitech C920 webcam that is plug-and-play compatible with the Raspberry Pi. We can access this camera using either the cv2.VideoCapture  function built-in to OpenCV or the VideoStream  class from this lesson.

You can see an example of my setup below:

Figure 1: My multiple camera Raspberry Pi setup.

Here we can see my Raspberry Pi 2, along with the Raspberry Pi camera module (sitting on top of the Pi 2) and my Logitech C920 webcam.

The Raspberry Pi camera module is pointing towards my apartment door to monitor anyone that is entering and leaving, while the USB webcam is pointed towards the kitchen, observing any activity that may be going on:

Figure 2: The Raspberry Pi camera module and USB camera are both hooked up to my Raspberry Pi, but are monitoring different areas of the room.

Ignore the electrical tape and cardboard on the USB camera — this was from a previous experiment which should (hopefully) be published on the PyImageSearch blog soon.

Finally, you can see an example of both video feeds displayed to my Raspberry Pi in the image below:

Figure 3: An example screenshot of monitoring both video feeds from the multiple camera Raspberry Pi setup.

In the remainder of this blog post, we’ll define a simple motion detection class that can detect if a person/object is moving in the field of view of a given camera. We’ll then write a Python driver script that instantiates our two video streams and performs motion detection in both of them.

As we’ll see, by using the threaded video stream capture classes (where one thread per camera is dedicated to perform I/O operations, allowing the main program thread to continue unblocked), we can easily get our motion detectors for multiple cameras to run in real-time on the Raspberry Pi 2.

Let’s go ahead and get started by defining the simple motion detector class.

Defining our simple motion detector

In this section, we’ll build a simple Python class that can be used to detect motion in a field of view of a given camera.

For efficiency, this class will assume there is only one object moving in the camera view at a time — in future blog posts, we’ll look at more advanced motion detection and background subtraction methods to track multiple objects.

In fact, we have already (partially) reviewed this motion detection method in our previous lesson, home surveillance and motion detection with the Raspberry Pi, Python, OpenCV, and Dropbox — we are now formalizing this implementation into a reusable class rather than just inline code.

Let’s get started by opening a new file, naming it basicmotiondetector.py , and adding in the following code:

Multiple cameras with the Raspberry Pi and OpenCV

Python

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

# import the necessary packages import imutils import cv2   class BasicMotionDetector: def __init__(self, accumWeight=0.5, deltaThresh=5, minArea=5000): # determine the OpenCV version, followed by storing the # the frame accumulation weight, the fixed threshold for # the delta image, and finally the minimum area required # for "motion" to be reported self.isv2 = imutils.is_cv2() self.accumWeight = accumWeight self.deltaThresh = deltaThresh self.minArea = minArea   # initialize the average image for motion detection self.avg = None

Line 6 defines the constructor to our BasicMotionDetector  class. The constructor accepts three optional keyword arguments, which include:

• accumWeight : The floating point value used for the taking the weighted average between the current frame and the previous set of frames. A larger accumWeight  will result in the background model having less “memory” and quickly “forgetting” what previous frames looked like. Using a high value of accumWeight  is useful if you except lots of motion in a short amount of time. Conversely, smaller values of accumWeight  give more weight to the background model than the current frame, allowing you to detect larger changes in the foreground. We’ll use a default value of 0.5 in this example, just keep in mind that this is a tunable parameter that you should consider working with.
• deltaThresh : After computing the difference between the current frame and the background model, we’ll need to apply thresholding to find regions in a frame that contain motion — this deltaThresh  value is used for the thresholding. Smaller values ofdeltaThresh  will detect more motion, while larger values will detect less motion.
• minArea : After applying thresholding, we’ll be left with a binary image that we extract contours from. In order to handle noise and ignore small regions of motion, we can use the minArea  parameter. Any region with > minArea  is labeled as “motion”; otherwise, it is ignored.

Finally, Line 17 initializes avg , which is simply the running, weighted average of the previous frames the BasicMotionDetector  has seen.

Let’s move on to our update  method:

Multiple cameras with the Raspberry Pi and OpenCV

Python

19 20 21 22 23 24 25 26 27 28 29 30 31 32 33

def update(self, image): # initialize the list of locations containing motion locs = []   # if the average image is None, initialize it if self.avg is None: self.avg = image.astype("float") return locs   # otherwise, accumulate the weighted average between # the current frame and the previous frames, then compute # the pixel-wise differences between the current frame # and running average cv2.accumulateWeighted(image, self.avg, self.accumWeight) frameDelta = cv2.absdiff(image, cv2.convertScaleAbs(self.avg)

The update  function requires a single parameter — the image we want to detect motion in.

Line 21 initializes locs , the list of contours that correspond to motion locations in the image. However, if the avg  has not been initialized (Lines 24-26), we set avg  to the current frame and return from the method.

Otherwise, the avg  has already been initialized so we accumulate the running, weighted average between the previous frames and the current frames, using the accumWeight  value supplied to the constructor (Line 32). Taking the absolute value difference between the current frame and the running average yields regions of the image that contain motion — we call this our delta image.

However, in order to actually detect regions in our delta image that contain motion, we first need to apply thresholding and contour detection:

Multiple cameras with the Raspberry Pi and OpenCV

Python

35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55

# threshold the delta image and apply a series of dilations # to help fill in holes thresh = cv2.threshold(frameDelta, self.deltaThresh, 255, cv2.THRESH_BINARY)[1] thresh = cv2.dilate(thresh, None, iterations=2)   # find contours in the thresholded image, taking care to # use the appropriate version of OpenCV cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE) cnts = cnts[0] if self.isv2 else cnts[1]   # loop over the contours for c in cnts: # only add the contour to the locations list if it # exceeds the minimum area if cv2.contourArea(c) > self.minArea: locs.append(c) # return the set of locations return locs

Calling cv2.threshold  using the supplied value of deltaThresh  allows us to binarize the delta image, which we then find contours in (Lines 37-45).

Note: Take special care when examining Lines 43-45. As we know, the cv2.findContours  method return signature changed between OpenCV 2.4 and 3. This codeblock allows us to use cv2.findContours  in both OpenCV 2.4 and 3 without having to change a line of code (or worry about versioning issues).

Finally, Lines 48-52 loop over the detected contours, check to see if their area is greater than the supplied minArea , and if so, updates the locs  list.

The list of contours containing motion are then returned to calling method on Line 55.

Note: Again, for a more detailed review of the motion detection algorithm, please see the home surveillance tutorial.

Accessing multiple cameras on the Raspberry Pi

Now that our BasicMotionDetector  class has been defined, we are now ready to create themulti_cam_motion.py  driver script to access the multiple cameras with the Raspberry Pi — and apply motion detection to each of the video streams.

Let’s go ahead and get started defining our driver script:

Multiple cameras with the Raspberry Pi and OpenCV

Python

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

# import the necessary packages from __future__ import print_function from pyimagesearch.basicmotiondetector import BasicMotionDetector from imutils.video import VideoStream import numpy as np import datetime import imutils import time import cv2   # initialize the video streams and allow them to warmup print("[INFO] starting cameras...") webcam = VideoStream(src=0).start() picam = VideoStream(usePiCamera=True).start() time.sleep(2.0)   # initialize the two motion detectors, along with the total # number of frames read camMotion = BasicMotionDetector() piMotion = BasicMotionDetector() total = 0

We start off on Lines 2-9 by importing our required Python packages. Notice how we have placed the BasicMotionDetector  class inside the pyimagesearch  module for organizational purposes. We also import VideoStream , our threaded video stream class that is capable of accessing both the Raspberry Pi camera module and built-in/USB web cameras.

The VideoStream  class is part of the imutils package, so if you do not already have it installed, just execute the following command:

Multiple cameras with the Raspberry Pi and OpenCV

Shell

1	$ pip install imutils

Line 13 initializes our USB webcam VideoStream  class while Line 14 initializes our Raspberry Pi camera module VideoStream  class (by specifying usePiCamera=True ).

In the case that you do not want to use the Raspberry Pi camera module and instead want to leverage two USB cameras, simply changes Lines 13 and 14 to:

Multiple cameras with the Raspberry Pi and OpenCV

Python

1 2	webcam1 = VideoStream(src=0).start() webcam2 = VideoStream(src=1).start()

Where the src  parameter controls the index of the camera on your machine. Also note that you’ll have to replace webcam  and picam  with webcam1  and webcam2 , respectively throughout the rest of this script as well.

Finally, Lines 19 and 20 instantiate two BasicMotionDetector ‘s, one for the USB camera and a second for the Raspberry Pi camera module.

We are now ready to perform motion detection in both video feeds:

Multiple cameras with the Raspberry Pi and OpenCV

Python

23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45

# loop over frames from the video streams while True: # initialize the list of frames that have been processed frames = []   # loop over the frames and their respective motion detectors for (stream, motion) in zip((webcam, picam), (camMotion, piMotion)): # read the next frame from the video stream and resize # it to have a maximum width of 400 pixels frame = stream.read() frame = imutils.resize(frame, width=400)   # convert the frame to grayscale, blur it slightly, update # the motion detector gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) gray = cv2.GaussianBlur(gray, (21, 21), 0) locs = motion.update(gray)   # we should allow the motion detector to "run" for a bit # and accumulate a set of frames to form a nice average if total < 32: frames.append(frame) continue

On Line 24 we start an infinite loop that is used to constantly poll frames from our (two) camera sensors. We initialize a list of such frames  on Line 26.

Then, Line 29 defines a for  loop that loops over each of the video stream and motion detectors, respectively. We use the stream  to read a frame  from our camera sensor and then resize the frame to have a fixed width of 400 pixels.

Further pre-processing is performed on Lines 37 and 38 by converting the frame to grayscale and applying a Gaussian smoothing operation to reduce high frequency noise. Finally, the processed frame is passed to our motion  detector where the actual motion detection is performed (Line 39).

However, it’s important to let our motion detector “run” for a bit so that it can obtain an accurate running average of what our background “looks like”. We’ll allow 32 frames to be used in the average background computation before applying any motion detection (Lines 43-45).

After we have allowed 32 frames to be passed into our BasicMotionDetector’s, we can check to see if any motion was detected:

Multiple cameras with the Raspberry Pi and OpenCV

Python

47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66

# otherwise, check to see if motion was detected if len(locs) > 0: # initialize the minimum and maximum (x, y)-coordinates, # respectively (minX, minY) = (np.inf, np.inf) (maxX, maxY) = (-np.inf, -np.inf)   # loop over the locations of motion and accumulate the # minimum and maximum locations of the bounding boxes for l in locs: (x, y, w, h) = cv2.boundingRect(l) (minX, maxX) = (min(minX, x), max(maxX, x + w)) (minY, maxY) = (min(minY, y), max(maxY, y + h))   # draw the bounding box cv2.rectangle(frame, (minX, minY), (maxX, maxY), (0, 0, 255), 3) # update the frames list frames.append(frame)

Line 48 checks to see if motion was detected in the frame  of the current video stream .

Provided that motion was detected, we initialize the minimum and maximum (x, y)-coordinates associated with the contours (i.e., locs ). We then loop over the contours individually and use them to determine the smallest bounding box that encompasses allcontours (Lines 51-59).

The bounding box is then drawn surrounding the motion region on Lines 62 and 63, followed by our list of frames  updated on Line 66.

Again, the code detailed in this blog post assumes that there is only one object/person moving at a time in the given frame, hence this approach will obtain the desired result. However, if there are multiple moving objects, then we’ll need to use more advanced background subtraction and tracking methods — future blog posts on PyImageSearch will cover how to perform multi-object tracking.

The last step is to display our frames  to our screen:

Multiple cameras with the Raspberry Pi and OpenCV

Python

68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92

# increment the total number of frames read and grab the # current timestamp total += 1 timestamp = datetime.datetime.now() ts = timestamp.strftime("%A %d %B %Y %I:%M:%S%p")   # loop over the frames a second time for (frame, name) in zip(frames, ("Webcam", "Picamera")): # draw the timestamp on the frame and display it cv2.putText(frame, ts, (10, frame.shape[0] - 10), cv2.FONT_HERSHEY_SIMPLEX, 0.35, (0, 0, 255), 1) cv2.imshow(name, frame)   # check to see if a key was pressed key = cv2.waitKey(1) & 0xFF   # if the `q` key was pressed, break from the loop if key == ord("q"): break   # do a bit of cleanup print("[INFO] cleaning up...") cv2.destroyAllWindows() webcam.stop() picam.stop()

Liens 70-72 increments the total  number of frames processed, followed by grabbing and formatting the current timestamp.

We then loop over each of the frames  we have processed for motion on Line 75 and display them to our screen.

Finally, Lines 82-86 check to see if the q  key is pressed, indicating that we should break from the frame reading loop. Lines 89-92 then perform a bit of cleanup.

Motion detection on the Raspberry Pi with multiple cameras

To see our multiple camera motion detector run on the Raspberry Pi, just execute the following command:

Multiple cameras with the Raspberry Pi and OpenCV

Shell

1	$ python multi_cam_motion.py

I have included a series of “highlight frames” in the following GIF that demonstrate our multi-camera motion detector in action:

Figure 4: An example of applying motion detection to multiple cameras using the Raspberry Pi, OpenCV, and Python.

Notice how I start in the kitchen, open a cabinet, reach for a mug, and head to the sink to fill the mug up with water — this series of actions and motion are detected on the first camera.

Finally, I head to the trash can to throw out a paper towel before exiting the frame view of the second camera.

A full video demo of multiple camera access using the Raspberry Pi can be seen below:

Summary

In this blog post, we learned how to access multiple cameras using the Raspberry Pi 2, OpenCV, and Python.

When accessing multiple cameras on the Raspberry Pi, you have two choices when constructing your setup:

1. Either use multiple USB webcams.
2. Or using a single Raspberry Pi camera module and at least one USB webcam.

Since the Raspberry Pi board has only one camera input, you cannot leverage multiple Pi camera boards — atleast without extensive hacks to your Pi.

In order to provide an interesting implementation of multiple camera access with the Raspberry Pi, we created a simple motion detection class that can be used to detect motion in the frame views of each camera connected to the Pi.

While basic, this motion detector demonstrated that multiple camera access is capable of being executed in real-time on the Raspberry Pi — especially with the help of our threadedPiVideoStream  and VideoStream  classes implemented in blog posts a few weeks ago.

If you are interested in learning more about using the Raspberry Pi for computer vision, along with other tips, tricks, and hacks related to OpenCV, be sure to signup for the PyImageSearch Newsletter using the form at the bottom of this post.

See you next week!

Downloads:

If you would like to download the code and images used in this post, please enter your email address in the form below. Not only will you get a .zip of the code, I’ll also send you a FREE 11-page Resource Guide on Computer Vision and Image Search Engines, including exclusive techniques that I don’t post on this blog! Sound good? If so, enter your email address and I’ll send you the code immediately!

Email address:

Resource Guide (it’s totally free).

Enter your email address below to get my free 11-page Image Search Engine Resource Guide PDF. Uncover exclusive techniques that I don't publish on this blog and start building image search engines of your own!

 motion, motion detection, motion tracking, multiprocessing, picamera, raspberry pi, segmentation, threading, tracking, usb camera, video processing, video stream, webcam

 OpenCV panorama stitching

Real-time panorama and image stitching with OpenCV