Give Your Raspberry Pi Smart Vision

Give Your Raspberry Pi Smart Vision

What is Google Cloud Vision API? Check out the video by Google here:

Use Google Cloud Vision on the Raspberry Pi to take a picture with the Raspberry Pi Cameraand classify it with the Google Cloud Vision API.   First, we’ll walk you through setting up the Google Cloud Platform.

Next, we will use the Raspberry Pi Camera to take a picture of an object, and then use the Raspberry Pi to upload the picture taken to Google Cloud.

Finally, we can analyze the picture in the cloud, and the cloud will either try to label the picture (tell us in text what’s going on in the picture), or try to find logos in the picture (company logos that are in the picture) or analyze faces in the picture (tell us everything from emotions on the face to where the face is on the picture).  Skynet here we come!

These directions are meant to simplify getting started with Cloud Vision on your Raspberry Pi.  They are derived from directions found here but modified to work with the Raspberry Pi, and simplify set up.  We will use Google Cloud Vision on the Raspberry Pi to detect images with these three python examples.

Tools

• Raspberry Pi
• Raspbian for Robots SD Card
• Power Supply
• Raspberry Pi Camera
• Google Cloud Account

The Google Cloud Account is free for 60 days.  After 60 days, you will be charged based on usage.  You can create an account here with either your Google or Gmail login. Google offers the first 60 days for free.

These directions assume you’re using Raspbian for Robots on your SD Card.  You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here.

In this picture we’ve asked Google Cloud Vision to analyze the GoPiGo on the left, with the returned text response on the right.  It returns “labels” (the things it sees in the picture) such as “vehicle”, “wheel”, “toy”, “product”, and “model car”.  Pretty good job Google!

Google Cloud Set Up

The first step is to set up a Google Cloud Account.  You can create an account here. You can create an account here with either your Google or Gmail login. Google offers the first 60 days for free.

After you’ve created your account, you can set up a project and enable billing.  First, click here to go to the Google Cloud Projects Page.

Create a new project.

In our example, we named it “vision1”

You will need to enable billing for your account (you won’t be charged).  You can do this by clicking here.  Go to the Cloud Platform Console here.  Select the project “vision1” and click the hamburger in the upper left hand corner of the page.

Select Billing on the left hand side.  From here, you can enable billing on your Google Cloud account.

Finally, we’ll enable the API.  Click here to enable the API.  Select the “vision1” account we created.  Click “Continue”.  You should get a message that API is enabled.

Optional: Test the Account

Now that the account is setup, why not upload an image from your computer and test it here?  You can follow this tutorial to try out a few images and see what Google thinks they are!

Use our picture above.  Is it an animal? A reptile?  Google will tell you!

Getting a JSON Key

Now we want to get a JSON key to put on our Raspberry Pi.  This JSON key will handle all the authentication to use our Google Cloud Account.  Instead of a password, we can use the file to authenticate our account on the Raspberry Pi.

Caution: You should be careful where you store this key.  Be sure to change the password on your Raspberry Pi before putting the JSON key on it, leaving your JSON key unprotected could expose you to something malicious!  In the command line type:

sudo passwd

And follow the prompts to change the password on the Raspberry Pi.

Head back to the Console in Google Cloud.  Find the box titled “Use Google APIs” and click “Enable and manage APIs”

Click “Credentials” and Create Credentials.  Credentials is on the left hand side, with a picture of a key next to it.

Select “Create an Service Account Key”. See more about what we’re doing here.

Under “Service Account” Select “New Service Account”.  We’ll give it a name, “vision”

Finally, create a role.  We’ll give it full access, so select “Project” and “Owner” to give the Pi Full access to all resources.

A popup should come up telling you you have created a new key, and an automatic download of the JSON key should begin.  Keep track of this file!

That should be it, you should have a service account key!

You should now use an FTP program (such as FileZilla) or Samba (see our tutorial here!) to move the JSON file over to your Raspberry Pi.  Place the JSON file in the home directory/home/pi

Set Up the Hardware

The hardware is pretty simple!  We’ll be using a Raspberry Pi Camera with the Raspberry Pi, here are some directions on setting up the camera.

These Raspberry Pi Camera setup directions assume you’re using Raspbian for Robots on your SD Card.  You can buy an SD Card with Raspbian for Robots on it, or you can download it for free and install it using our directions here.  If you’re not using Raspbian for Robots, you will need to take additional steps to enable and setup your camera; here are some directions on setting up the camera.

Prepare the Raspberry Pi

In this project, we’re assuming you’re using our image, Raspbian for Robots.  If you’re not, you can download it here!

Next, we’ll upgrade Pip.  Pip is a package manager for python language installations.  Note, you should have Pip installed (it comes installed on Raspbian for Robots); if you don’t upgrade, you will get an error on installation!

 sudo pip install --upgrade pip
 sudo apt-get install libjpeg8-dev

Next, install Google API Python Client.   Again in the command line, run:

sudo pip install --upgrade google-api-python-client

Next, install Python Imaging Library.  Again in the command line, run:

sudo pip install --upgrade Pillow

Install Python Picamera:

sudo apt-get install python-picamera

Turn on Super User.  In the command line, type the command “su”:

su

You’ll be prompted for your password; this is the password you used to login to your Raspberry Pi.

In the home directory, we will make the JSON file available to any application we’re running.  Run the command:

export GOOGLE_APPLICATION_CREDENTIALS=filename.json

Be sure to substitute your JSON filename in this command with the name of the file you have on your Raspberry Pi.

Optional: Make Credentials Permanent

You can also make the Google Application Credentials permanent.  You can make the JSON credentials from Google accessible to any python program running on your Raspberry Pi.

First, open the bashrc file:

sudo nano ~/.bashrc

Scroll to the very bottom of the file, and add the line:

GOOGLE_APPLICATION_CREDENTIALS="/home/pi/vision123.json"

Be sure to replace the variable “vision123” with the name of your file.

Now, load the credential by typing in the command line:

source ~/.bashrc

Finally, confirm that the credentials are set:

echo $GOOGLE_APPLICATION_CREDENTIALS.

This should return the file path and name:  ‘/home/pi/vision123.json”

Get the Example Code

Get the example code: we have three python examples on Github here.

sudo git clone https://github.com/DexterInd/GoogleVisionTutorials

Detect a Logo

First, we’ll try to get the Raspberry Pi to detect the Raspberry Pi Logo.  We’ll use the big Raspberry Pi label on the Raspberry Pi Box.  Set the Pi box about 1 ft from the camera.  We’ll be propping up the camera with a Raspberry Pi Robot, the GoPiGo.  It holds the camera in place with the acrylic body, and we will use it later for some fun projects!

You might want to take a test picture to make sure the label is visible.  In the command line, run

raspistill -o cam.jpg

If the picture looks ok, we’ll move along!

We’ll go into super user mode on the Pi.

su

You may need to runsudo su.  We’ll make our JSON credentials available.  In the command line, type:

export GOOGLE_APPLICATION_CREDENTIALS=filename.json

And now, in the example directory, run:

python camera-vision-logo.py

You should get “Raspberry Pi” back!

Exciting stuff!  Now you can go around the house looking for more logos!

Label Pictures

One of the cool things you can do with Google Vision is get the cloud to figure out what’s in the picture you take.  Let’s take a picture of a GoPiGo Wheel and see what the cloud thinks it is.

Set up the camera and run the sample program.  Again, we’ll be holding the camera in place with the GoPiGo robot kit.

With the camera set up, we’ll take a test picture.  Looks like the wheel is in there.

In this sample program we’ll return the entire JSON response.  We get a few interesting results from the picture:  “yellow” (with an 80% score), “wheel”, “product”, “tire”, “toy” and “automotive tire”.

Facial Detection With the Raspberry Pi

Here’s the scary part!  Let’s detect a face and see if Google can detect some emotions!  With Google Cloud Vision you can detect a range of emotions, headwear, some head coordinates, and other fun data.  For this part, I made a special selfie-Pi using the GrovePi Case.

I ran the program in the command line:

python camera-vision-face.py

Here’s the picture I took:

“Work is so much fun!”

Clearly I’m trying to smile a little for the camera!  The response Google gives for facial analysis is pretty long, the JSON is very detailed, so I’m pulling the highlights below:

According to Google Vision, I’m not likely to be Angry (VERY_UNLIKELY)

I’m not likely to be wearing headwear,  however, it is possible that I’ve got joy (joyLikelihood is “POSSIBLE”).

Finally, I’m not likely to be sorrowful, and I’m not likely to be surprised.  These are just some of the emotions that Google Cloud Vision will scan faces for and return information on.

Conclusion

Google Cloud Vision is a really fun service to just play around with!  Google Cloud Vision on the Raspberry Pi is perfect for projects with the Raspberry Pi and Raspberry Pi Camera, and adds in-context vision for the Raspberry Pi.

Above is a picture of “Dex” our company mascot, analyzed by Google for labels.  You can see the computer correctly analyzes it as “electronics” and “technology”!

Troubleshooting

A few issues we’ve run into that might help you!

Optimal image size:  The optimal image size for face recognition with Google Vision is 1600 x 1200 – you can set this using the python lines

camera = picamera.PiCamera()
camera.resolution = (1600, 200)

Clock and Date Issues:  For proper authentication with the Google Cloud services, your clock must be correct (or darn near correct within about 12 hours).  If it is not, you will see the following error:

raise HttpAccessTokenRefreshError(error_msg, status=resp.status)
oauth2client.client.HttpAccessTokenRefreshError: invalid_grant: Invalid JWT: Token must be a short-lived token and in a reasonable timeframe

If the python example throws this error, you need to update your clock.  This can be tricky: it should be automatically updated if you boot the Pi while plugged into an ethernet network.  However, if you are using wifi, or your network isn’t working properly, you can set the Raspberry Pi time manually with the command:

date -u 110316262016

This is using the format “MMDDhhmmCCYY” where M is the month, D is the day, h is the hour, m is the minute, C is the century (probably 20), and Y is the year (in 2016, this is 16).

Questions?

Ask on our forums!

Learn More!

If you liked this tutorial, consider buying the SD Card with Raspbian for Robots here, Raspberry Pi Camera here and the GoPiGo Starter kit here.