API scribblings

2017-07-24

Before I leave Hypothesis (😢) I wanted to get a couple of thoughts out of my head about the evolution of the software we’ve been building together, specifically about the API.

Why are we talking about the API specifically?

Good question! I want to emphasise that this isn’t about the API because the API is the most important part of Hypothesis. It’s not.

Instead, it’s about the API because talking about the API forces me to be specific and concrete in my suggestions, which in turn hopefully means that poor ideas will become apparent more quickly. Some of the suggestions below have impact that stretches far above the API, but the API is still a useful layer at which to discuss them. If the low-level domain model of Hypothesis doesn’t make sense, it’s not fair to expect a regular user to form an implicit mental model of it.

Ok, so what problem are we trying to solve here?

Honestly, it’s kind of a jumble of different but related things. The changes I propose are supposed to:

Let’s pick one. Why is using the API right now hard for developers?

Two main reasons2. The first is that our API doesn’t support OAuth, which makes it hard for people to build tools against our API that other people can then use. This is in the process of being fixed, so I won’t say more about it.

The second reason is that two of the most important actions in the API – creating annotations and finding annotations – are, relatively speaking, quite complicated and quite poorly documented. A surprising amount of functionality is hidden behind each of these endpoints, in sometimes non-obvious ways.

Like what?

Like the fact that when you create an annotation using the API, you include:

There’s nothing fundamentally wrong with this stuff, it’s just not necessarily needed. There is a way to keep group and permissions information entirely out of the body of the annotation.

So what are you proposing?

First and foremost, that we make groups a first-class object exposed by the API. And second, that most (but not necessarily all) access to annotations is mediated through groups.

To get concrete, for the first part I’m imagining stuff like:

# Get a list of the user's groups
GET /api/profile/groups

# Get a list of all groups in creation order
GET /api/groups

# Get information about a specific group
GET /api/groups/:groupid

# Create a group
POST /api/groups

# Change group settings (for group admins)
PUT /api/groups/:groupid

# Join a group (for groups with membership)
PUT /api/groups/:groupid/members/:username

# Leave a group (for groups with membership)
DELETE /api/groups/:groupid/members/:username

And, for the second:

# List all annotations in a group
GET /api/groups/:groupid/annotations

# List annotations in a group for a specific page
GET /api/groups/:groupid/annotations?uri=...

# Create a new annotation in a group
POST /api/groups/:groupid/annotations

And so on, and so on.

I haven’t thought carefully about whether we’d want to go all in and have all RESTful access to annotations under this namespace, so I’m not quite sure if

# Fetch information about a specific annotation
GET /api/groups/:groupid/annotations/:annotationid

is the logical extension of these ideas.

Cool, so we can access groups through the API. So what?

I think there is a pretty long list of benefits that flow from doing things this way. I’m not going to try and enumerate all of them, but here are a few:

But what about private annotations?

Right. Most of the improvements I mention above assume that we’ve made another change at the same time, which is to turn “Only Me” into a group rather than a status that an annotation can have within a group.

We could quite easily give each user their own virtual “Only Me” group, or simply leave it up to them to create a group which only they can see.

As I’ve written about this elsewhere I won’t expand on this much more here.

I’m sold! So why wouldn’t we do this?

On the face of it, there aren’t too many downsides to this way of doing things, at least as far as I can see. That said, it’s potentially quite a big change to the way the API works, and could result in substantial additional complexity if the old way of doing things has to work forever, too.

I’d suggest doing a serious evaluation of whether this could be a clean break with the existing API, with an accelerated deprecation plan for the old one. We’re lucky, in that the number of people making serious use of our current API probably numbers in the low dozens at the moment, so now might be the right time to do something like that.


  1. Discussing why we might want to do this is probably beyond the scope of this document. Suffice to say I think it’s critically important to ensuring the continued quality of content on Hypothesis.

  2. For more insight on what it’s like to be a smart person trying to use our API, have a look at Sean R’s issue on this subject.