Custom Image processing¶
If you merely wish to display a single camera stream and do not want to process the images, do NOT use this code. Instead, see one of the following sections about automatic streaming:
The first step you need to do is create a file – let’s call it
vision.py must contain some function to be called, let’s call it
and at the minimum it needs to do the following operations:
- Create a CameraServer instance
- Start capturing from USB
- Get a cvSink object that images can be retrieved from
- Loop and capture images
Here’s a full example:
# Import the camera server from cscore import CameraServer # Import OpenCV and NumPy import cv2 import numpy as np def main(): cs = CameraServer.getInstance() cs.enableLogging() # Capture from the first USB Camera on the system camera = cs.startAutomaticCapture() camera.setResolution(320, 240) # Get a CvSink. This will capture images from the camera cvSink = cs.getVideo() # (optional) Setup a CvSource. This will send images back to the Dashboard outputStream = cs.putVideo("Name", 320, 240) # Allocating new images is very expensive, always try to preallocate img = np.zeros(shape=(240, 320, 3), dtype=np.uint8) while True: # Tell the CvSink to grab a frame from the camera and put it # in the source image. If there is an error notify the output. time, img = cvSink.grabFrame(img) if time == 0: # Send the output the error. outputStream.notifyError(cvSink.getError()); # skip the rest of the current iteration continue # # Insert your image processing logic here! # # (optional) send some image back to the dashboard outputStream.putFrame(img)
This code will work both on a RoboRIO and on other platforms. The exact mechanism to run it differs depending on whether you’re on a RoboRIO or a coprocessor:
cscore easily supports multiple cameras! Here’s a really simple
file that will get you started streaming two cameras to the FRC Dashboard
from cscore import CameraServer def main(): cs = CameraServer.getInstance() cs.enableLogging() usb1 = cs.startAutomaticCapture(dev=0) usb2 = cs.startAutomaticCapture(dev=1) cs.waitForever()
One thing to be careful of: if you get USB Bandwidth errors, then you probably need to do one of the following:
- Reduce framerate (FPS). The default is 30, but you can get by with 10 or even as low as 5 FPS.
- Lower image resolution: you’d be surprised how much you can do with a 160x120 image!
Sometimes the first and second camera swap!?¶
When using multiple USB cameras, Linux will sometimes order the cameras unpredictably – so camera 1 will become camera 0. Sometimes.
The way to deal with this is to tell cscore to use a specific camera by its path
on the file system. First, identify the cameras
dev paths by using SSH to
access the robot and execute
find /dev/v4l. You should see output similar
/dev/v4l /dev/v4l/by-path /dev/v4l/by-path/pci-0000:00:1a.0-usb-0:1.4:1.0-video-index0 /dev/v4l/by-path/pci-0000:00:1d.0-usb-0:1.4:1.2-video-index0 /dev/v4l/by-id ...
What you need to do is figure out what paths belong to which camera, and then when you start the camera server, pass it a name and a path via:
usb1 = cs.startAutomaticCapture(name="cam1", path='/dev/v4l/by-id/some-path-here') usb2 = cs.startAutomaticCapture(name="cam2", path='/dev/v4l/by-id/some-other-path-here')
Generally speaking, if your cameras have unique IDs associated with them (you can tell because the by-id path has a random string of characters in it), then using by-id paths are the best, as they’ll always be the same regardness which port the camera is plugged into.
However, if your camera does NOT have unique IDs associated with them, then you should use the by-path versions instead. These device paths are unique to each USB port plugged in. They should be fairly deterministic, but sometimes with USB hubs they have been known to change.
The Microsoft Lifecam cameras commonly used in FRC don’t have unique
IDs associated with them, so you’ll want to use the
versions of the links if you are using two Lifecams.