Welcome to the authentication guide for the Parsons project! It has three parts:
An introduction to basic concepts such as APIs, HTTP requests, and types of authentication
Instructions for how to authenticate Parsons connectors, including walkthroughs for two examples: Twilio and Google Sheets
A brief overview of how Parsons works “under the hood” to make your authentication experience as easy as possible
This guide, like most of our guides, was written for people from a campaign/movement background. It does not assume an extensive technical background.
If you want to show thanks for this free guide, please give feedback. Feel free to add questions and suggestions to this guide by leaving comments.
Introduction to Basic Concepts
APIs and Requests
Parsons works by moving data from one place to another. Whenever you’re working with a third party service, you’ll need to prove who you are so you can have access to your data, to read it or to change it. That’s true when you’re browsing online and need to login to a website, and that’s true when you’re interacting with data through an API like you do with Parsons.
API stands for “Application Programming Interface”, and it’s a way for you to interact with an application—aka a platform or website—programmatically, that is, through code.
An API typically consists of a set of endpoints which do different things. Endpoints are just URLs. For an example, if you look at the endpoints listed on this page, you can see how the endpoints look like partial URLs.
When we use an endpoint, we’re sending an HTTP request to the API. Again, this is the same thing that happens when you go to a page in your browser. When you go to a page in your browser, your browser says to that website, “Hey, I’m sending an HTTP request to get this page. Can I have it?” And the website says yes and sends it. With an API we’re doing the same thing, except instead of using the browser manually, you’re writing a piece of code that sends the request directly to that endpoint.
When your browser makes a request to a server, the response is usually formatted with HTML. This HTML displays a webpage for human eyes.
When a script makes a request to a server’s API, the response is usually formatted as JSON, or some other structured data format. Here’s a random example of an endpoint that gives us cat facts formatted in JSON.
Sidebar: this type of request is a GET request. It’s one of 9 types of HTTP requests. You don’t really need to know about request methods, but if you’re interested, GET is the default. It allows you to read data but not change it. The other common method is POST, which allows you to send new data or update existing data. This happens on browsers when you fill out a webform, and again, you can make a POST request with your API.
Some APIs are public, which means anyone can access them without having to identify themselves. But most APIs are private, which means you have to authenticate or prove you are who you say you are, so they know whether to go along with your request.
Different Ways to Authenticate
There are a bunch of different ways to authenticate. For any given Parsons connector, you’re going to have to go to the docs for that connector. Instructions for how to connect should be at the very top in a little blue box, for example:
We try to keep these instructions accurate and up to date, but sometimes we miss things, so if those docs aren’t working for you please reach out so we can fix them.
One common way to authenticate to a platform is using a username and password. This is how ActBlue does things.
Another common option is to use something called an API key. An API key is a unique string that’s associated with your account.
API keys are nice because they can be associated with different scopes. So you can make an API key and say, “okay, requests with this key only get fulfilled if they’re read only” or “requests with this key can only access data from this specific sub-project”. Also, if API keys get lost or compromised they’re a lot easier to replace than a username and password. Mobilize and Action Network use API keys.
Sometimes connectors use more unique or unusual methods. The Google Sheets connector which we’ll be walking through together has a pretty elaborate process. You have to follow their instructions to create a custom credentials file and then send that to them with each request.
Zoom has another complicated system called Javascript Web Tokens or JWTs. This is a pretty advanced topic, so we’re not going to go into it, but please do reach out to us for help if you end up wanting to authenticate to Zoom sometime.
In addition to having different methods to authenticate, some platforms limit who can access their API. Several Parsons connectors require you to email a contact to get authentication info or require you to have a paid account.
With Parsons, we try to handle as much custom stuff for you as we can. But there’s a limit to how much we can do, because only you have access to your authentication information. We can standardize using it, but you have to be the one to get it.
Walkthroughs
Setup
Before we walk through the authentication process for our example connectors, we need to do a bit of setup. We’re going to open up a command line and create a virtual environment we can install Parsons into.
This guide goes into depth about how to get setup. Here’s the tl:dr:
Start by checking if Python is installed by typing “python –version” into your command line. If it’s not, install Python.
Next, we’ll install a virtual environment and activate it:
Windows
Install a virtual environment tool with the following steps:
Find the Scripts directory for your Python installation, such as C:\Users\<User>\AppData\Local\Programs\Python\Python37\Scripts\. Add the Scripts directory to your $PATH.
You can then create and activate your virtual environment:
mkvirtualenv parsons
workon parsons
Mac/Linux
If you’re using Mac/Linux but Python version is below 3.4:
pip install virtualenv
virtualenv $path/$name
source $path/$name/bin/activate
If you’ve got Python 3.4+ on Mac/Linux, you’re in luck, a tool called “venv” comes pre-installed! You can skip right to using it:
python -m venv $path/$name
source $path/$name/bin/activate
Final step: Install Parsons
Once your virtual environment is activated, finish setup by installing Parsons: “pip install parsons”
Twilio Authentication
If we go to the Twilio docs, we see the following instructions: “Twilio requires an account SID and an authorization token, which can be found in the Admin Console”
You’ll need to start by making a Twilio account, which should be free. This will require providing an email and phone number. Once you’ve done that, the main console page should show your SID and Auth Token.
That’s the account and SID you’ll use for real workflows. But right now we’re doing a training. So we’re going to go find a special testing SID and auth token. You can find those by:
Going to “Account” in the top righthand corner
Clicking API keys and tokens
Looking in the right-hand column where it says “test credentials”
We’re going to save this information by setting it as an environmental variable. Remember when we started our virtual environment before? We’re going to save a variable to that environment. We do this by going to the command line and typing:
Windows
set TWILIO_ACCOUNT_SID=”XXXXX”
set TWILIO_AUTH_TOKEN=”XXXXX”
Linux/Mac
export TWILIO_ACCOUNT_SID=”XXXXX”
export TWILIO_AUTH_TOKEN=”XXXXX”
You can check that this worked by echoing/printing the value back to the command line:
Mac/Linux: printenv TWILIO_ACCOUNT_SID
Windows: echo $TWILIO_ACCOUNT_ID
Google Sheets Authentication
Setting up the Google Sheets connector takes several steps.
Go to the Google Developer Console and create a new project. (Or, in your real work, sometimes you’ll select an existing project.) Make sure the project is selected.
Click “APIs & Services” and then “Enabled APIs and Services”.
Look for the button near the top that says “Enable APIs and Services”.
Select the Drive API from among the (many!) API options and select “enable”.
Do the same thing for the Google Sheets API.
Now that you’ve enabled the API, you can get the credentials that allow you to interact with it.
Click the blue “create credentials” button in the righthand corner. (If you can’t find it, you should be able to get there via the “credentials” option in the left sidebar.)
Make the following selections:
You’re accessing “application data” not “user data”
You’re not using Cloud Engine and the other apps.
Click next, and fill out the account name.
Skip the two optional elements, giving the account access to the project, and giving users access to the account. In a real workflow, you might use this, but we don’t need it for our test training.
Select “done”.
We’ve generated our credentials. Now we just need to save them locally.
Go back to the credentials tab. (See left sidebar)
Select the only account under Service Accounts.
Select “keys” and then “add key” and “create new key”. Pick the key type JSON.
A file with a name like “project-name-12345678.json” should be automatically downloaded. Move that somewhere you can remember it, like maybe in the folder where you put other Parsons code.
We’re now going to add a path to that credential as an environment variable, using the same syntax as before:
Run either of these commands in your terminal, depending on your system type:
set GOOGLE_DRIVE_CREDENTIALS="C:\Home\Projects\" # Windows
(Note: if you’re not sure exactly what the path is, make sure you are in the same folder as the file and type pwd in Linux/Mac or dir in Windows.)
This should be all! However you may run into access issues still. If the above steps don’t work for you and you are getting access denied errors, try this to manually give access:
Look inside the json file we downloaded for a client email. It will look something like [email protected].
Create a folder and make a copy of this spreadsheet into it. You can create a new spreadsheet and copy and paste the data, or use the “copy” function - it doesn’t matter.
Finally, share the folder with the client email. Make sure to give them editor to permissions.
And we’re done! Phew.
Running the Script
I’ve created a sample script that uses our test Google Sheets account and our test Twilio account to pretend-send text messages to the list of people from the Google Sheet. If you’re following along, you should be able to as well.
Get the spreadsheet ID from the URl of the new copied sheet and put it in the spreadsheet_id variable in the script. You’ll also want to put your phone number in the phone number column in the format: 3334445555. Because we’re using test credentials, no actual texts will be sent to your phone number, but we need to use a valid phone number here.
The script should run without errors and print out the log statements for getting the five people from the google sheet, and sending the five texts. If you have any issues, please ask in the Slack!
Parsons Magic Behind the Scenes
This part of the training is a deeper dive into how Parsons work. You don’t need to know it in order to use Parsons! This is more to satisfy your curiosity, help teach you some Python concepts, and make it easier for you to continue to Parsons or debug Parsons code if you want/need to in the future.
Instantiating a class
You may have written some scripts that were just a bunch of code all in a sequence. That’s a totally acceptable and great way to write short scripts, but for complex problems, developers usually group code into segments. The two most common ways to group code are called functions and classes.
A function takes some input and returns some output. Classes are a bit more complex. They can hold state, and they can even have other classes or functions linked to them. Functions that exist on a class are called methods.
When you create a class, you’re creating a sort of broad, abstract version of that thing. For instance, our ActBlue connector class is an abstract version of any possible ActBlue connection. If you want to add your authentication information and make calls to your ActBlue account, you need to create an instance of that class.
You do this by calling the class using parentheses, like this:
twilio = Twilio()
sheets = GoogleSheets()
Some classes require you to provide information when instantiating a class. Others do not.
How a class is instantiated is defined by the __init__ method. See, for example, the ActBlue connector’s __init__ method. This method showcases two things that most Parsons connectors do: check the environment for variables using check_env, and sending authentication info to the third-party server using a client.
Getting variables from the environment using check_env
Parsons connectors all use a helper function called check_env to look in the environment for the information. It looks like this:
When using this helper function, we pass it env, the environmental variable name, and field, the value for that name which may already have a value, or may be empty. We also indicate whether the variable in question is optional or whether it must have a value.
The substance of this function is between lines 10 and 17. On Line 10, we check to see if there’s already a value for the field (our environmental variable name). If there is, we skip everything and just return the field.
check_env uses a Python library called os, which stands for operating system. The os.environ object here gets all the variables in the current environment and puts them in a Python dictionary.
We attempt to look up our variable in the dictionary using os.environ[env]. If we don’t find it, that throws an exception.
Throwing an exception
Throwing an exception basically means saying, “Wait, stop there’s been an error!” In this case, when we look for a key in a dictionary and don’t find it, we throw a KeyError.
Sometimes you don’t actually want to stop your whole script when there’s an error. That’s why we use Try-Except statements. You can see here that the try-except statement has two parts. We try to do something, in this case look up a given key in the os.environ dictionary, and if that causes an error, we do whatever’s in the except block.
Side note: if we get a different error besides a key error, any other kind of error, we’ll still crash the script. The Try-Except statement is only handling the KeyError.
What’s happening in the except block is interesting. Basically we’re checking if the variable we’re looking for is optional. If it is optional, then nothing happens in this block. We continue on without errors. But if it’s required - aka not optional - then we throw the error anyway, and provide a more specific error message.
Sending the authentication info
How Parsons passes authentication info to the third party platforms depends on the specific connector.
Some connectors use a client library. This is a common development practice and many connectors that are maintained by bigger companies use it. For example, you can see with the AirTable connector, we import the client library. We give the authentication info to the client library in the init method, and the client library handles everything.
Google Sheets also uses a client but it’s a bit more complicated. We can see in the init method for that connector that the client requires quite a bit more work.
Movement orgs typically don’t have clients. For instance, ActBlue doesn’t. If we look in the connector’s init method, we see something else: a helper class called APIConnector.
If we go to the APIConnector class we see it’s fairly complicated. Let me just highlight a few things:
Note that in the init method, the class takes a URI. This is the base URL for all the endpoints. So for ActBlue, it’s on line 10: ACTBLUE_API_ENDPOINT = “https://secure.actblue.com/api/v1”
Note that init also takes a variable called auth. That’s always going to be the authentication info for the connector, whether that’s a username and password, an API key, or something else.
Most of the other methods on the APIConnector are variations on “request”. That includes get_request, post_request, delete_request, etc. Remember the 9 HTTP methods I mentioned before? These are a few of those 9.
There’s a couple other methods that do some basic validation on the data we get back, and handle pagination. (Pagination is when your request is for so much data that the API you’re talking to wants to give it to you one page at a time.)
Conclusion
And that’s it - all you need to know (and maybe more than you need to know!) - about authentication in Parsons!
If you think this guide misses anything important, please let us know.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
