RESTful Web Services

Author: Leonard Richardson, Sam Ruby
3.9
All Stack Overflow 37
This Year Stack Overflow 2
This Month Stack Overflow 2

RESTful Web Services

3.9

Review Date:

Comments

by anonymous   2018-03-19

If "validation failure" means that there is some client error in the request, then use HTTP 400 (Bad Request). For instance if the URI is supposed to have an ISO-8601 date and you find that it's in the wrong format or refers to February 31st, then you would return an HTTP 400. Ditto if you expect well-formed XML in an entity body and it fails to parse.

(1/2016): Over the last five years WebDAV's more specific HTTP 422 (Unprocessable Entity) has become a very reasonable alternative to HTTP 400. See for instance its use in JSON API. But do note that HTTP 422 has not made it into HTTP 1.1, RFC-7231.

Richardson and Ruby's RESTful Web Services contains a very helpful appendix on when to use the various HTTP response codes. They say:

400 (“Bad Request”)
Importance: High.
This is the generic client-side error status, used when no other 4xx error code is appropriate. It’s commonly used when the client submits a representation along with a PUT or POST request, and the representation is in the right format, but it doesn’t make any sense. (p. 381)

and:

401 (“Unauthorized”)
Importance: High.
The client tried to operate on a protected resource without providing the proper authentication credentials. It may have provided the wrong credentials, or none at all. The credentials may be a username and password, an API key, or an authentication token—whatever the service in question is expecting. It’s common for a client to make a request for a URI and accept a 401 just so it knows what kind of credentials to send and in what format. [...]

by anonymous   2017-08-20

I'm in the same situation. I'm going to develop an Android app with a REST backend (PHP + MySQL). A few things I'm taking into account:

And I would like use Python instead of PHP, but there are less sharing hosting services that support GCI for Python.

by anonymous   2017-08-20

Why aim for a RESTful design?

The RESTful principles bring the features that make web sites easy (for a random human user to "surf" them) to the web services API design, so they are easy for a programmer to use. REST isn't good because it's REST, it's good because it's good. And it is good mostly because it is simple.

The simplicity of plain HTTP (without SOAP envelopes and single-URI overloaded POST services), what some may call "lack of features", is actually its greatest strength. Right off the bat, HTTP asks you to have addressability and statelessness: the two basic design decisions that keep HTTP scalable up to today's mega-sites (and mega-services).

But REST is not the silver bulltet: Sometimes an RPC-style ("Remote Procedure Call" - such as SOAP) may be appropriate, and sometimes other needs take precedence over the virtues of the Web. This is fine. What we don't really like is needless complexity. Too often a programmer or a company brings in RPC-style Services for a job that plain old HTTP could handle just fine. The effect is that HTTP is reduced to a transport protocol for an enormous XML payload that explains what's "really" going on (not the URI or the HTTP method give a clue about it). The resulting service is far too complex, impossible to debug, and won't work unless your clients have the exact setup as the developer intended.

Same way a Java/C# code can be not object-oriented, just using HTTP does not make a design RESTful. One may be caught up in the rush of thinking about his services in terms of actions and remote methods that should be called. No wonder this will mostly end up in a RPC-Style service (or a REST-RPC-hybrid). The first step is to think different. A RESTful design can be achieved in many ways, one way (the simplest, some might say) is to think of your application in terms of resources, not actions:

  • Instead of thinking in terms of actions ("do a search for places on the map"),
  • Think in terms of the results of that action ("the list of places on the map matching a search criteria").

I'll go for examples below. (Other key aspect of REST is the use of HATEOAS - I don't brush it here, but I talk about it quickly at another post.)

About the first design

Let's take a look a the proposed design:

ACTION http://api.animals.com/v1/dogs/1/

First off, we should not consider creating a new HTTP verb (ACTION). Generally speaking, this is undesirable for several reasons:

  • (1) Given only the service URI, how will a "random" programmer know the ACTION verb exists?
  • (2) if the programmer knows it exists, how will he know its semantics? What does that verb mean?
  • (3) what properties (safety, idempotence) should one expect that verb to have?
  • (4) what if the programmer has a very simple client that only handles standard HTTP verbs?
  • (5) ...

Now let's consider using POST (I'll discuss why below, just take my word for it now):

POST /v1/dogs/1/ HTTP/1.1
Host: api.animals.com

{"action":"bark"}

This could be OK... but only if:

  • {"action":"bark"} was a document; and
  • /v1/dogs/1/ was a "document processor" (factory-like) URI. A "document processor" is a URI that you'd just "throw things at" and "forget" about them - the processor may redirect you to a newly created resource after the "throwing". E.g. the URI for posting messages at a message broker service, which, after the posting would redirect you to a URI that shows the status of the message's processing.

I don't know much about your system, but I'd already bet both aren't true:

  • {"action":"bark"} is not a document, it actually is the method you are trying to ninja-sneak into the service; and
  • the /v1/dogs/1/ URI represents a "dog" resource (probably the dog with id==1) and not a document processor.

So all we know now is that the design above is not so RESTful, but what is that exactly? What is so bad about it? Basically, it is bad because that is complex URI with complex meanings. You can't infer anything from it. How would a programmer know a dog have a bark action that can be secretly infused with a POST into it?

Designing your question's API calls

So let's cut to the chase and try to design those barks RESTfully by thinking in terms of resources. Allow me to quote the Restful Web Services book:

A POST request is an attempt to create a new resource from an existing one. The existing resource may be the parent of the new one in a data-structure sense, the way the root of a tree is the parent of all its leaf nodes. Or the existing resource may be a special "factory" resource whose only purpose is to generate other resources. The representation sent along with a POST request describes the initial state of the new resource. As with PUT, a POST request doesn’t need to include a representation at all.

Following the description above we can see that bark can be modeled as a subresource of a dog (since a bark is contained within a dog, that is, a bark is "barked" by a dog).

From that reasoning we already got:

  • The method is POST
  • The resource is /barks, subresource of dog: /v1/dogs/1/barks, representing a bark "factory". That URI is unique for each dog (since it is under /v1/dogs/{id}).

Now each case of your list has a specific behavior.

1. bark just sends an email to dog.email and records nothing.

Firstly, is barking (sending an email) a synchronous or an asynchronous task? Secondly the bark request requires any document (the email, maybe) or is it empty?

1.1 bark sends an email to dog.email and records nothing (as a synchronous task)

This case is simple. A call to the barks factory resource yields a bark (an email sent) right away and the response (if OK or not) is given right away:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(entity-body is empty - or, if you require a **document**, place it here)

200 OK

As it records (changes) nothing, 200 OK is enough. It shows that everything went as expected.

1.2 bark sends an email to dog.email and records nothing (as an asynchronous task)

In this case, the client must have a way to track the bark task. The bark task then should be a resource with it's own URI.:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

202 Accepted
Location: http://api.animals.com/v1/dogs/1/bark/a65h44

This way, each bark is traceable. The client can then issue a GET to the bark URI to know it's current state. Maybe even use a DELETE to cancel it.

2. bark sends an email to dog.email and the increments dog.barkCount by 1

This one can be trickier, if you want to let the client know the dog resource gets changed:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

303 See Other
Location: http://api.animals.com/v1/dogs/1

In this case, the location header's intent is to let the client know he should take a look at dog. From the HTTP RFC about 303:

This method exists primarily to allow the output of a POST-activated script to redirect the user agent to a selected resource.

If the task is asynchronous, a bark subresource is needed just like the 1.2 situation and the 303 should be returned at a GET .../bark/Y when the task is complete.

3. bark creates a new "bark" record with bark.timestamp recording when the bark occured. It also increments dog.barkCount by 1.

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

201 Created
Location: http://api.animals.com/v1/dogs/1/bark/a65h44

In here, the bark is a created due to the request, so the status 201 Created is applied.

If the creation is asynchronous, a 202 Accepted is required (as the HTTP RFC says) instead.

The timestamp saved is a part of bark resource and can be retrieved with a GET to it. The updated dog can be "documented" in that GET dogs/X/bark/Y as well.

4. bark runs a system command to pull the latest version of the dog code down from Github. It then sends a text message to dog.owner telling them that the new dog code is in production.

The wording of this one is complicated, but it pretty much is a simple asynchronous task:

POST /v1/dogs/1/barks HTTP/1.1
Host: api.animals.com
Authorization: Basic mAUhhuE08u724bh249a2xaP=

(document body, if needed)

202 Accepted
Location: http://api.animals.com/v1/dogs/1/bark/a65h44

The client then would issue GETs to /v1/dogs/1/bark/a65h44 to know the current state (if the code was pulled, it the e-mail was sent to the owner and such). Whenever the dog changes, a 303 is appliable.

Wrapping up

Quoting Roy Fielding:

The only thing REST requires of methods is that they be uniformly defined for all resources (i.e., so that intermediaries don’t have to know the resource type in order to understand the meaning of the request).

In the above examples, POST is uniformly designed. It will make the dog "bark". That is not safe (meaning bark has effects on the resources) nor idempotent (each request yields a new bark), which fits the POST verb well.

A programmer would know: a POST to barks yields a bark. The response status codes (also with entity-body and headers when necessary) do the job of explaining what changed and how the client can and should proceed.

Note: The primary sources used were: "Restful Web Services" book, the HTTP RFC and Roy Fielding's blog.


Update:

The original question asked about the design of a URI like:

ACTION http://api.animals.com/v1/dogs/1/?action=bark

Below is the explanation of why it is not a good choice:

How clients tell the server WHAT TO DO with the data is the method information.

  • RESTful web services convey method information in the HTTP method.
  • Typical RPC-Style and SOAP services keep theirs in the entity-body and HTTP header.

WHICH PART of the data [the client wants the server] to operate on is the scoping information.

  • RESTful services use the URI. SOAP/RPC-Style services once again use the entity-body and HTTP headers.

As an example, take Google's URI http://www.google.com/search?q=DOG. There, the method information is GET and the scoping information is /search?q=DOG.

Long story short:

  • In RESTful architectures, the method information goes into the HTTP method.
  • In Resource-Oriented Architectures, the scoping information goes into the URI.

And the rule of thumb:

If the HTTP method doesn’t match the method information, the service isn’t RESTful. If the scoping information isn’t in the URI, the service isn’t resource-oriented.

You can put the "bark" "action" in the URL (or in the entity-body) and use POST. No problem there, it works, and may be the simplest way to do it, but this isn't RESTful.

To keep your service really RESTful, you may have to take a step back and think about what you really want to do here (what effects will it have on the resources).

I can't talk about your specific business needs, but let me give you an example: Consider a RESTful ordering service where orders are at URIs like example.com/order/123.

Now say we want to cancel an order, how are we gonna do it? One may be tempted to think that is a "cancellation" "action" and design it as POST example.com/order/123?do=cancel.

That is not RESTful, as we talked above. Instead, we might PUT a new representation of the order with a canceled element sent to true:

PUT /order/123 HTTP/1.1
Content-Type: application/xml

<order id="123">
    <customer id="89987">...</customer>
    <canceled>true</canceled>
    ...
</order>

And that's it. If the order can't be canceled, a specific status code can be returned. (A subresource design, like POST /order/123/canceled with the entity-body true may, for simplicity, also be available.)

In your specific scenario, you may try something similar. That way, while a dog is barking, for example, a GET at /v1/dogs/1/ could include that information (e.g. <barking>true</barking>). Or... if that's too complicated, loosen up your RESTful requirement and stick with POST.

Update:

I don't want to make the answer too big, but it takes a while to get the hang of exposing an algorithm (an action) as a set of resources. Instead of thinking in terms of actions ("do a search for places on the map"), one needs to think in terms of the results of that action ("the list of places on the map matching a search criteria").

You may find yourself coming back to this step if you find that your design doesn't fit HTTP's uniform interface.

Query variables are scoping information, but do not denote new resources (/post?lang=en is clearly the same resource as /post?lang=jp, just a different representation). Rather, they are used to convey client state (like ?page=10, so that state is not kept in the server; ?lang=en is also an example here) or input parameters to algorithmic resources (/search?q=dogs, /dogs?code=1). Again, not distinct resources.

HTTP verbs' (methods) properties:

Another clear point that shows ?action=something in the URI is not RESTful, are the properties of HTTP verbs:

  • GET and HEAD are safe (and idempotent);
  • PUT and DELETE are idempotent only;
  • POST is neither.

Safety: A GET or HEAD request is a request to read some data, not a request to change any server state. The client can make a GET or HEAD request 10 times and it's the same as making it once, or never making it at all.

Idempotence: An idempotent operation in one that has the same effect whether you apply it once or more than once (in math, multiplying by zero is idempotent). If you DELETE a resource once, deleting again will have the same effect (the resource is GONE already).

POST is neither safe nor idempotent. Making two identical POST requests to a 'factory' resource will probably result in two subordinate resources containing the same information. With overloaded (method in URI or entity-body) POST, all bets are off.

Both these properties were important to the success of the HTTP protocol (over unreliable networks!): how many times have you updated (GET) the page without waiting until it is fully loaded?

Creating an action and placing it in the URL clearly breaks the HTTP methods' contract. Once again, the technology allows you, you can do it, but that is not RESTful design.

by anonymous   2017-08-20

From your question I understand that the web service's interface looks something like this:

/product/
/product/{ProductId}
/product/{ProductName}
/product/category/{CategoryId}

Since you are asking if you should store CategoryName, I assume that it is unique (same as ProductName).
I also assume that the web service handles cases where products or categories are renamed transparently (i.e. by providing a redirect or any other means which allow you to detect this and handle it accordingly). If it doesn't, do not consider storing names as references to products or categories - always use IDs.

I would provide the same answer to your questions #1 and #2. Even though uniqueness of ProductName and CategoryName will technically allow you to store them in your application as unique identifiers of products and categories, I would opt for storing their IDs instead. The main decision point would be your storage medium. Since you are using a database, and the web service allows you to access objects by unique numerical IDs, database normalization rules should apply - hence you should store IDs.

The above however assumes that you are using a relational database - if you are using a NoSQL database, I assume that storing names instead of IDs would be a viable option as well (at least as far as I can tell with my current understanding of NoSQL solutions, unfortunately I don't have any practical experience with any of them yet).

Regarding question #3 - I would stick with the naming conventions that you already use in your database. There are many different conventions for naming tables and columns out there, so I really doubt that there are any standardized conventions on how to name columns referencing web service objects. I would name them according to your existing naming conventions and in a way that purpose of the columns is clear to everybody who is using the system. Note that if there is a chance that you will be using other web services in the future, you should consider keeping the name of the service in the column name rather than using a generic ws prefix - e.g. AmazonProductId or AmazonCategoryId.

I'll try to point out a few items from my experience, but I would not label them as best practices - just topics to think about.

In my experience, I found it useful to treat data from web services in the same fashion as the data from a database - at least from an application's perspective, where your storage layer would be abstracted from application logic. By this I mean that you would should think about and prepare for similar scenarios regardless if your storage medium is a database or a web service. Same as databases, web services can go down, both can have their data or integrity corrupt, both will require you to sanitize or otherwise process data on input.

Caching of data should be an item which is high on your list - apart from the obvious performance reasons, it can allow you to deal with outages of the web service (to an extend limited by which data you cache).
An example would be that your application displays a list products most frequently purchased products in your application. If your application stores only IDs of products, you will have to do one or more requests to the web service in order to retrieve the names of all products which you need to display in the list. If you cache product names locally or in your database, you will achieve better performance, conserve your resources and you will also have a failsafe scenario in case that the web service goes down.

Referential integrity is one other important aspect to think about when working with web services. As the web service is completely separate from your database, you do not have the option to create foreign keys as you would do in a database-only solution. This means that data changes in the web service (i.e. product updates or deletions) can break the integrity of data in your database.

Regarding references, these depend mostly on the type of web service that you are about to use (you didn't specify which service you will be using). If the service is based on REST principles, I can recommend Restful Web Services by Leonard Richardson and Sam Ruby. Even though it isn't focused on application/service integration as such, it's a great introduction into REST.

by anonymous   2017-08-20

Your second example of "request parameters" is not correct because "get" is included as part of the path. GET is the request type, it should not be part of the path.

There are 4 main types of requests:

 GET
 PUT
 POST
 DELETE

GET requests should always be able to be completed without any information in the request body. Additionally, GET requests should be "safe", meaning that no significant data is modified by the request.

Besides the caching concern mentioned above, parameters in the URL path would tend to be required and/or expected because they are also part of your routing, whereas parameters passed in the query string are more variable and don't affect which part of your application the request is routed to. Although could potentially also pass a variable length set of parameters through the url:

GET somedomain.com/states/Virginia,California,Mississippi/

A good book to read as a primer on this topic is "Restful Web Services". Though I will warn you to be prepared to skim over some redundant information.

by anonymous   2017-08-20

You are on the right track. You essentially need a server side application stack that will connect to your mysql database, read out recordsets, and send back HTTP responses that include the JSON payload.

You can do this by simply writing a PHP method that returns the JSON formatted response (this would be fairly easy to get up and running). However if you are looking for a more robust/longterm solution I would invest the time in developing a RESTful web service. RESTful services provide a common interface to access data intensive applications.

As for how to do this, you can check out Phil Sturgeon's blogpost here that describes how to setup a RESTful service in the popular PHP framework, CodeIgniter.

Also if you are not attached to PHP, I would recommend Ruby on Rails as it is designed out of the box to create RESTful web services. DHH, one of the creators of the Ruby on Rails framework, helped write the book on RESTful web services.