Before I leave Hypothesis (😢) I wanted to get a couple of thoughts out of my head about the evolution of the software we’ve been building together, specifically about the API.
Good question! I want to emphasise that this isn’t about the API because the API is the most important part of Hypothesis. It’s not.
Instead, it’s about the API because talking about the API forces me to be specific and concrete in my suggestions, which in turn hopefully means that poor ideas will become apparent more quickly. Some of the suggestions below have impact that stretches far above the API, but the API is still a useful layer at which to discuss them. If the low-level domain model of Hypothesis doesn’t make sense, it’s not fair to expect a regular user to form an implicit mental model of it.
Honestly, it’s kind of a jumble of different but related things. The changes I propose are supposed to:
Two main reasons2. The first is that our API doesn’t support OAuth, which makes it hard for people to build tools against our API that other people can then use. This is in the process of being fixed, so I won’t say more about it.
The second reason is that two of the most important actions in the API – creating annotations and finding annotations – are, relatively speaking, quite complicated and quite poorly documented. A surprising amount of functionality is hidden behind each of these endpoints, in sometimes non-obvious ways.
Like the fact that when you create an annotation using the API, you include:
documentfield. This is entirely optional, but given that this could be entirely independent of the action of creating an annotation it’s weird that it’s in here.
groupfield. But there’s no (easy-to-use) API that allows a user to navigate the available groups. Representing this as a “field” on the annotation is also misleading. It’s a fundamental decision about where the annotation goes which can’t be changed later.
There’s nothing fundamentally wrong with this stuff, it’s just not necessarily needed. There is a way to keep group and permissions information entirely out of the body of the annotation.
First and foremost, that we make groups a first-class object exposed by the API. And second, that most (but not necessarily all) access to annotations is mediated through groups.
To get concrete, for the first part I’m imagining stuff like:
# Get a list of the user's groups GET /api/profile/groups # Get a list of all groups in creation order GET /api/groups # Get information about a specific group GET /api/groups/:groupid # Create a group POST /api/groups # Change group settings (for group admins) PUT /api/groups/:groupid # Join a group (for groups with membership) PUT /api/groups/:groupid/members/:username # Leave a group (for groups with membership) DELETE /api/groups/:groupid/members/:username
And, for the second:
# List all annotations in a group GET /api/groups/:groupid/annotations # List annotations in a group for a specific page GET /api/groups/:groupid/annotations?uri=... # Create a new annotation in a group POST /api/groups/:groupid/annotations
And so on, and so on.
I haven’t thought carefully about whether we’d want to go all in and have all RESTful access to annotations under this namespace, so I’m not quite sure if
# Fetch information about a specific annotation GET /api/groups/:groupid/annotations/:annotationid
is the logical extension of these ideas.
I think there is a pretty long list of benefits that flow from doing things this way. I’m not going to try and enumerate all of them, but here are a few:
Creating annotations via the API doesn’t require specifying any permissions or group information in the POST payload: it’s all implied by the URL. As a result, creating annotations is programmatically simpler. The JSON representation of an annotation is shorter and clearer.
Fetching annotations for a particular group and page is a dedicated URL, rather than an abuse of a “search” endpoint. This potentially simplifies client implementation, and is likely to make caching these lookups on the server much easier. (It provides a very clear point at which to invalidate the cache: when a new annotation is written to the group.)
A group is a natural analogue for the W3C Annotation Protocol’s Annotation Containers, and could be implemented in a way that conforms to that specification.
Groups as Annotation Containers extends naturally to “virtual groups” such as the “Only Me” group (see below) as well as groups populated by feeds from external sources. That is: not all groups have to contain annotations made by humans.
It seems likely that implementing annotation lookup and creation in this way would also result in some significant simplification of the code which does access control checking for annotations/groups.
Right. Most of the improvements I mention above assume that we’ve made another change at the same time, which is to turn “Only Me” into a group rather than a status that an annotation can have within a group.
We could quite easily give each user their own virtual “Only Me” group, or simply leave it up to them to create a group which only they can see.
As I’ve written about this elsewhere I won’t expand on this much more here.
On the face of it, there aren’t too many downsides to this way of doing things, at least as far as I can see. That said, it’s potentially quite a big change to the way the API works, and could result in substantial additional complexity if the old way of doing things has to work forever, too.
I’d suggest doing a serious evaluation of whether this could be a clean break with the existing API, with an accelerated deprecation plan for the old one. We’re lucky, in that the number of people making serious use of our current API probably numbers in the low dozens at the moment, so now might be the right time to do something like that.
Discussing why we might want to do this is probably beyond the scope of this document. Suffice to say I think it’s critically important to ensuring the continued quality of content on Hypothesis.↩
For more insight on what it’s like to be a smart person trying to use our API, have a look at Sean R’s issue on this subject.↩