A lot of progressive data work involves moving data into and out of third-party tools. While Parsons aims to make that easier, it’s still pretty hard. One skill that’s really helpful is getting comfortable reading documentation, and in particular, API documentation. So: this is a primer on how to read API docs.
The first part is an introduction to some basic concepts like what an API is and how requests work. If you already know all that, you can skip it.
The second part takes a look at the documentation for a few specific APIs. I focus mostly on Hustle, but briefly discuss Action Network.
API Basic Concepts
API stands for “Application Programming Interface”. It’s a way for you to interact with an application (aka a platform or website) programmatically - that is, through code.
An API typically consists of a set of endpoints which do different things. Endpoints are just URLs (though they often have security features that prevent access for unauthorized users). For an example, if you look at the endpoints listed on this page, you can see how the endpoints look like partial URLs.
When we use an endpoint, we’re sending an HTTP request to the API. Again, this is the same thing that happens when you go to a page in your browser. When you go to a page in your browser, your browser says to that website, “Hey, I’m sending an HTTP request to get this page. Can I have it?” And the website says sure and sends it. With an API we’re doing the same thing, except instead of using the browser to display a website or data, you’re writing a piece of code that sends the request directly to that endpoint and gets data back.
There are nine different kinds of HTTP requests. The most commonly used are GET and POST, and to a lesser extent PUT and DELETE. GET is the default. It allows you to read data from the URL, but not change it. POST allows you to send new data or update existing data. Sometimes people do all their updating and deleting via POST, and sometimes they use the separate PUT (update) and DELETE methods to do it.
There are three main parts of an HTTP request: the request line (kind of like a summary or title), the headers, and the body. The headers contain metadata about the request - for instance, one common header says, essentially, “Please return data in JSON format”.
The body is optional. POST requests will put data they’re sending in the message body, but GET requests don’t have a body. This makes it harder to keep the data they’re sending private, since they have to be sent through query parameters stored in the URL itself, eg. the text after the question mark in www.example.com?data=dontsendprivatedatathisway
This is why most people use POST (or PUT) for sending data.
When you send an HTTP Request, the server should issue some kind of response. This response should include a status code. Common status codes include 200 (“ok”) and 404 (“not found”).
Generally speaking, an error code in the 200s means your request was successful, a code in the 400s means you did something wrong, and a code in the 500s means something’s wrong with the server behind the API. (Although sometimes the server is wrong about who is wrong! As someone who’s written server side code, I’ve definitely given people a 400 error even when it was a bug on my end. Whoops!)
If you want to learn more about status codes, Wikipedia has a full list, and the HTTP Status Cats are a really funny and memorable way to learn some of the codes. You definitely don’t need to learn all of them, though!
One last note before we move on to our example. Many APIs return data in JSON format. There are many online tools, such as this one, that can help you validate or “beautify” your JSON (aka make them easier to read). There is also a very good library in Python to convert data from JSON to Python dictionary data structures, which Parsons uses under the hood when creating Parsons tables from API responses.
Example: Hustle
We’ll be looking at the Hustle API, which follows a lot of industry best practices for API documentation. The bulk of the documentation is in Hustle’s API Reference, which we’ll get to in a moment. But first…
Authentication
The Hustle docs have a section on authentication. The authentication details are precisely the sort of thing that Parsons tries to take care of so users don’t have to. But this is an educational guide, so let’s take a moment to read and explain:
The docs here are showing us an example of how to authenticate to their API using curl. Curl is a popular command line tool that’s most often used on Mac and Linux, though it can be installed on Windows as well. You can use it to make HTTP requests from the command line, and it is common to see examples in API docs using curl.
Some API documentation goes beyond this, and provides examples in other languages. For example, Twilio, the programmable text and call service Hustle is based on, provides examples in Python, PHP, Java, and several other languages as well as curl. (They also maintain “helper” or “client” libraries to make it even easier for programmers to use their APIs. Parsons relies on the Python client library for our Twilio connector. These are really nice, but unfortunately most movement orgs don’t have the resources to maintain tools like this. 😢)
The curl example screenshotted above uses a lot of jargon which we won’t dive into. Just note the client_id and client_secret variables, which are the only authentication details surfaced to the user in the Parsons docs:
Behind the scenes, Parsons is following the instructions in the Hustle API Docs. First it sends a message with the client_id and the client_secret to Hustle, and gets an access token back. It then sends that access token in the header of all future requests. Also, as part of each future request, it checks if the access token has expired and generates a new one if necessary.
If you are trying to use the Hustle API outside of Parsons, you will have to handle all of this logic yourself. That’s why we recommend using Parsons! ;)
The Hustle API Docs also have a section on pagination. Again, this is something that Parsons handles for you, but it’s worth discussing briefly.
When you ask for a lot of data, many APIs like Hustle will only give you one subset of it at a time. This process is called pagination, since API users can request data in distinct “pages”. As explained by the Hustle docs, the API gives you three pieces of information: the total number of pages (in this example, 2), whether there’s another page after the page you just requested (“true”), and what the unique cursor indicating “where you are” in the sequence of pages is (“WzEsMV0”):
That cursor is then used for the next request, to get the next set of data:
A hundred records here, a hundred records there, it all adds up to a lot of data!
API Reference
Let’s move on to the API Reference, which is the heart of the documentation. It lists all functions available to you, and tells you four basic things:
What url or “endpoint” that function is available at
What HTTP method (e.g. GET or POST) you should use to access it
What parameters/inputs you can give it, and what influence those parameters have on the data that’s sent back
What responses you might get, and how they’re structured
It is best practice for API documentation to include each of these things, although not all do, alas.
Endpoints
The “endpoint” is what gets added to the “base url” to specify the unique path to the function.
And the getTags function, for instance, is available as: /organizations/{organizationId}/tags, where you fill in the organization ID of a given Hustle account where {organizationId} is listed. So if you’re organization had an ID of 1234, the function would be at /organizations/1234/tags.
When you make an HTTP request, you always have to use one of the nine basic methods described above.
The Hustle docs show what HTTP request method to use for each endpoint in the sidebar on the left:
That information is also shown in the details on the right:
Parameters/Inputs
Parameters are data you pass to an endpoint to narrow down or change what you want back from the API. Sometimes parameters are required, and sometimes they’re optional.
Hustle’s docs list the parameters for each endpoint. They separate them into three groups: path parameters, query parameters, and parameters passed through the message body. (Note that they also indicate if a given parameter is required.)
Path parameters are parameters that are part of the URL “path” to the endpoint itself:
Path parameters are almost always required, but not all API docs list them with their parameters. Some consider them to just be part of the endpoint.
Query parameters are parameters passed in as part of a query string. A query string is part of a URL that a server can parse and is the way you send data with GET requests. To make a GET request to the getTags endpoint screenshotted above, you might GET the endpoint:
There is a third way to pass in parameters: through the message body. This is how you pass in parameters for POST and PUT requests.
createGroup is a Hustle function which creates a new group. It is reached via a POST method, and data is passed in through both path parameters and through the body of the request. There are three parameters being passed in through the body: name, countryCode, and location.
For each of these, Hustle tells us what type of data it’s expecting for each parameter. For instance, name should be a text string. CountryCode is also a string, but it can only take one of three enumerated (pre-defined) values: “US”, “CA”, or “PR”, hence the prompt of “Enum:”.
The data type of location is an object. “Wait,” you might ask, “what kind of object?” We can click to learn more about the structure of the location object:
This is helpful, but you still might be confused about exactly the format of the data you need to send. Luckily, Hustle provides a “request sample” to show you what your request might look like:
I find request samples to be a huge time saver, and I love when API docs have them. Great job, Hustle!
Responses/Outputs
The final piece of information that API Documentation should give you is what kind of responses to expect. It should answer two main questions: when we make a successful request, how is the response formatted? And: what kind of errors might we receive if our request isn’t successful?
In addition to request samples, Hustle also provides response samples:
Responses are usually (but not always) in JSON format, and the response sample shown above displays the structure of the JSON object being returned, including the pagination information described previously.
You may have noticed in the screenshot that the response sample is listed in the green 200 tab. That’s because 200 is the general status code for “everything worked ok!” We might also get 400, 401, and 404 errors, which correspond to “Bad request”, “Unauthorized” and “Not found”. There’s also a “default” response for all other errors. Each returns a small JSON dictionary with a message field, which should contain a (ideally useful) error message.
Conclusion
Not every API is as nicely documented as Hustle’s. It’s hard for movement orgs to find the resources to build and maintain such nice docs. But hopefully this provides you with a mental model to help navigate a variety of API docs.
For example, the Action Network API docs don’t look much like Hustle’s, but they contain a lot of the same information. Endpoints are grouped together by the underlying record type, with request samples and response samples in the “scenarios” at the bottom of each page. The parameters for GET and POST requests are combined, which I personally find a little confusing, but they also provide a lot more descriptive detail for each parameter than Hustle does:
The more you read different API docs, the easier it will become to pick out the relevant information, regardless of how docs are designed.
I’ve also generally found that most orgs are both responsive and friendly if you email them a question like, “What is the format of this parameter?” or “I’m not getting the response I expected from this endpoint.” Action Network, in particular, has provided some of the best user support I’ve ever experienced!
I’ll wrap up this blog post here. If you have any questions, leave a comment! You can do so by highlighting the text and clicking the “comment” option. (You will need to sign up to our host service, Pubpub, in order to comment. Once you’ve signed up, email us or ping us on Slack to be added to the site!) You can also send feedback or questions via email or Slack.
Creative Commons Attribution 4.0 International License (CC-BY 4.0)
