Best practices for good Restful API design

Ironically, I am going to start this with a line that a good API design is not easy to achieve. People often don’t give it much of a thought before starting designing the APIs and a bit later in the lifecycle they realize that they have to write too much of a repeated code or it’s getting hard to maintain or cope with the consumer application’s needs.

It is very important that even before you write a single line of code for you API, you think through end to end what is the purpose of your API and who will be consuming it. Remember, unlike website, your API consumers will be developers only who are generally impatient and always prefer to integrate quickly and easily. So before crafting your API, you need to consider below aspects of your API carefully.

Data design & structure

SOAP or JSON? – Now-a-days at least this decision has been simplified as everyone is happy with consuming JSON.

Secondly, you need to understand what data you will be exposing with your API and accordingly design your API’s endpoints and methods. This is very important as this describes how the endpoints of your API will look like and if they logically doesn’t make sense, people will end up using wrong ones making mistakes.

And finally, who are the consumers of your API. Is it generic or business specific and accordingly you can decide further aspects of your API.

Vocabulary

What HTTP verbs to use and where. Everyone sort of knows HTTP GET & POST but there are 5 other verbs available as well which you can use to enhance the experience of your API. Also, even for security perspective, it is important to use right verb for some actions. Let’s see what verb is made for what:-

1. GET:  Get is equivalent to SELECT to fetch some information from the server.
2. POST: Post is used to Create a resource on the server.
3. PUT: Put is equivalent to Update when the whole resource is passed as a request.
4. PATCH: Patch is again for update but only the information changed is sent as a request.
5. DELETE: Delete is used to remove a resource from the server.
6. HEAD: It returns metadata about the resource for example, when was the last time it was modified.
7. OPTIONS: Returns what you can do with the resource for example, is it read only or read write etc.

Now, browsers tend to behave differently for all above verbs. For example, HTTP GET requests can be cached by the browsers. So make sure you are not using this verb for something which has to be unique like returning a unique GUID or next available counter etc.

POST are used for secure communications and hence browsers can warn you doing second time POSTs. HEAD & OPTIONS are not that used but HEAD is more like GET without a response body and can be cached by the browser.

Security

APIs are purely for information sharing and hence need to be secured with top level of security whatever you can implement.  I have seen API security evolved since I wrote my first web service from transport level to Identity aware now-a-days.

Your API communication should always be on SSL/TLS to make sure network level security. Then you need to add some sort of security to your endpoints as well just to save it from DOS attacks and depending upon the type of data your service returns, you can include the identity verification logic to your endpoint i.e. person A querying the data can only query his/her own data.

Response Status codes

It is important that your API shouldn’t throw errors and break consumer’s code. The best way to do this is handle everything in your code and return HTTP status codes. Don’t invent your own codes but use the HTTP codes assigned for these purposes as browsers already know how to behave on those. See below some known HTTP status codes you can use:-

1. 200 OK – Used commonly against GET requests where data/resource requested from the server has been found & returned.
2. 201 CREATED – Used against POST/PUT/PATCH where the request has been fulfilled and resulted in a new resource being created
3. 202 ACCEPTED - The request has been accepted for processing, but the processing has not been completed.
4. 204 No Content - The server successfully processed the request, but is not returning any content.
5. 400 Bad Request - The server cannot or will not process the request due to something that is perceived to be a client error (e.g., malformed request syntax, invalid request message framing, or deceptive request routing)
6. 401 Unauthorized – Authentication failed
7. 404 Not Found - The requested resource could not be found on the server
8. 500 Internal Server error - A generic error message, given when an unexpected condition was encountered and no more specific message is suitable.

Courtesy Wiki for above list. These are just few of the HTTP status codes I listed above, you can use any of them as per your requirements.

Versioning

No matter how much thought you put in designing your API endpoints & behavior, they will change. APIs evolve and you will be required to change the contract to add new features. Remember API contract is sort of agreement between client and server and changing that will break client applications and that is the reason you need to version your endpoints.

Hence keep a good versioning strategy in place and instead of changing existing contracts, release a new version. This will keep the existing client applications running and give new client the functionality they want. Another benefit of doing this is you can mark the older methods as deprecated and promote usage of new version of your API.

Analytics

This is not really a functional requirement but can be really handy in taking major decisions around your API. Always have some sort of analytics in place on your API endpoints and methods.

Documentation

I know this is not a favorite topic of developers but one of the crucial factors for your API consumers. Remember, if you end up integrating with third party API in your application, what would be your first question. Is there any document available for this?

So keep in mind below things to include in your API documentations:-

1. Endpoint/methods names and version available in the API.
2. Sample request/response for methods
3. Response status codes and what they mean in your API
4. Error messages associated with your endpoint methods this can include both validation errors & business error messages

One more thing you can add to your API if you have time for this is a experimental console where developers can play with your API. It is not difficult if you use some out of the box tools for this. One I know of and is very good is Swagger.

This is the list I have and obviously you are not limited to this and can go any far in making your API self explainable.

The last thing I want to discuss is “Hypermedia API”. It’s been a while since these API are known but not really that popular yet but I definitely think this is the future of APIs.

Hypermedia APIs

Hypermedia APIs are designed to overcome the obvious limitation with any (REST or SOAP) APIs currently exists. In order to consume any API today, you need to know the URL of the endpoint before hand. Not only this, this URL is part of contract you share with client meaning changing the URL breaks the contract and the client application.

It is not a complicated concept to understand. Hypermedia is based on the concept of how users interact with any website on web browser. You navigate to the root url of the website, then you are presented with links to go further whatever you wish to do on the site and so on and so on until you reach to the page where you find what information you are looking for. Hypermedia APIs mimics the same behavior in API world.

In Hypermedia APIs, the URLs are not hardcoded but discovered at runtime. This may doesn’t sound like a revolutionary idea but it gives API developers the freedom of changing and enhancing the API without breaking client apps. This idea can be used for scaling your applications or transparently test your API without affecting anything else.

Consider a small example (taken from WIKI) below:-

GET request to fetch an Account resource, requesting details in an XML representation

Here is the response:

Note the response contains 4 possible follow-up links - to make a deposit, a withdrawal, a transfer or to close the account.

Sometime later the account information is retrieved again, but now the account is overdrawn:

Now only one link is available: to deposit more money. In its current state, the other links are not available. Hence the term Engine of Application State. What actions are possible vary as the state of the resource varies.

A client does not need to understand every media type and communication mechanism offered by the server. The ability to understand new media types can be acquired at run-time through "code-on-demand" provided to the client by the server.

You can also implement an intelligent security around your APIs using Hypermedia showing next actions only to authorized audience.

This is just a small example around what Hypermedia API can do. You can search lot more detailed architectural and functional benefits of Hypermedia API.

Hope you would have found this information helpful.

IIS Policy Agent does more than Injecting Headers

I haven’t touch base around Policy agent in my last blog while introducing ForgeRock components but Policy Agent is kind a part of OpenAM itself.

Policy agent is a traffic inspector which intercepts each incoming request to your web server and check whether user is allowed to access the URL or not. If the URL is configured as a secured URL then Policy agent intercepts that request and sends the user to OpenAM login page. User is then authenticated here and then sent back to the original request he/she was trying to access. Below picture sort of depicts what I am trying to explain here. (Image downloaded from Google)

Policy Agent is also capable of injecting headers in the request as well for authenticated users. The headers can contain attributes which application can use to setup user session. Policy agents come in different flavors depending upon what type of application and web server you have. You can have IIS Policy agent, Apache or J2EE.

Theoretically, you should be able to use any above your application as all Policy agent has to do is to intercept traffic and send users to login page when required but recently I learned something interesting, IIS Policy agent over .Net application does not only intercepts the traffic and inject headers, it does also set the security context of the user in the application after which if you check Request.IsAuthenticated attribute in your app, it will return as true the same doesn’t happen with Apache Policy agent.

So let’s just check two scenarios below.

First, I have an Asp.Net application written protected by IIS Policy agent on my local IIS. The app has no custom code written in it but it just prints the header values & values of some security context variables.

See below what it prints post login from OpenAM. The value in Green is the OpenAM Cookie. The values in Blue are my headers but the values in Red are surprisingly all True.  IIS Policy agent post authentication sets the identity in WindowsPrincipal & GenericPrincipal after which if you do Request.IsAuthenticated, it returns true.

In second scenario, we have the same application code hosted on a separate website in IIS which is now protected by Apache policy agent also running locally on my machine. When I try to access this site, Agent running on Apache send me to OpenAM login page where post authentication, I again lands on my application but notice the difference between the values in RED.

    Apache Policy agent is not setting the user identity in WindowsPrincipal or GenericPrincipal due to which, If I do Request.IsAuthenticated, it returns false even after successful login after OpenAM. That is something you don’t want in .Net applications after you log in.

    Ideally you shouldn’t be using Apache Policy agent for IIS websites as the security should be closed to your application but we only had to do because, OpenAM IIS policy agent is not capable of securing multiple websites on IIS and 4.0 they released which can do this has it’s own problems.

    Anyways, I still need to understand how IIS Policy agent sets this security context for the application and how can we mimic this behavior with Apache. Will share this as well when I will figure it out.

A new topic in life - ForgeRock

It's been a while since I have written anything on my blog and the reason has been ForgeRock. In IT life, every two years you are introduced to a new game for which you have to learn the rules and play or else you will be sitting in the pavilion :)

Hence been a ForgeRock monkey from last 6 months hopping over the branches of OpenAM, OpenIDM & OpenDJ. I am not going to share my plan & feelings around this game but overall I quite liked it and hence I am writing a quick intro about ForgeRock technology in this blog.

ForgeRock

ForgeRock is a company providing Identity & access management solution via it’s multiple products called OpenAM, OpenIDM, OpenDJ, OpenIG & OpenUMA. Originally part of Sun but when taken over by Oracle, ForgeRock became a separate entity in itself. You can read more about this on wiki here.

For technical people, it is important to mention that it is a Java based product. Although not pure Java required most of the times but all customizations happens in server side JavaScript. And when we talk about Java, the remaining components automcatically switches from IIS to Apache & Tomcat. Windows to Linux and so on.

You can imagine how exciting my life would have been in last six months being a purely Microsoft guy and getting into all this. Anyways has been a good experience and all these bloody things are no longer a black box for me as well so let’s get into it.

OpenAM

OpenAM in ForgeRock’s family is responsible for access management like Authentication, SSO, adaptive risk, federation and all. It is highly scalable, modular & customizable product. You can read about OpenAM here.

OpenIDM

OpenIDM is Identity management. This one gives you out of the box functionalities around various use cases of identity management like user provisioning to backend systems, user self service, work flows around different processes & since now-a-days everything is in cloud, it gives you connectors for various SAAS products like Google, SalesForce & Office 365. You can read more about it here.

OpenDJ

And as we need some sort of database or directory in all applications. We have OpenDJ for this in ForgeRock stack. OpenDJ provide directory services with high performance, scalability & availability. You can read about this here.

OpenIG

This is a new member in ForgeRock stack and basically can act as a identity gateway for you legacy applications, APIs providing lots of out of the box functionalities like password capture and replay, API security etc. Read more about this here.

OpenUMA

Again this is a new addition in ForgeRock stack which is User managed Access. Quite powerful when I saw the demo of this but we aren’t using it so far. Read more about this here.

These are all the products which ForgeRock has to offer for identity and access management. So far using it, a good thing I can say about it is “it is highly customizable”. A bad thing I have to say about it is “it is highly customizable”. The problem is, everything is customizable and you have to configure it which makes you feel like you have been forced to sit in the cockpit of Boeing 747 without a manual.

But still, a good product if used in right environment & infrastructure by right people. I will share few of the use cases of what you can do with all these OpenAM, IDM & DJ in my blogs soon along with Microsoft stuff. Yes, I am not going to leave Microsoft Azure & whatever I used to do :)